2023-10-06T11:01:41.606 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T11:01:41.606 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390 --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --verbose -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/orig.config.yaml 2023-10-06T11:01:41.652 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:01:41.735 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7412390' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T11:01:41.736 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T11:01:41.736 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T11:01:41.737 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T11:01:41.738 INFO:teuthology.task.internal:Checking packages... 2023-10-06T11:01:41.761 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T11:01:41.761 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T11:01:41.761 INFO:teuthology.packaging:ref: None 2023-10-06T11:01:41.761 INFO:teuthology.packaging:tag: None 2023-10-06T11:01:41.761 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:01:41.762 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:01:41.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:01:41.995 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:01:41.995 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T11:01:42.002 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T11:01:42.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T11:01:42.026 INFO:teuthology.task.internal:Saving configuration 2023-10-06T11:01:42.046 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T11:01:42.053 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:01:42.075 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:55:45.328367', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T11:01:42.099 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:55:45.329843', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-06T11:01:42.100 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T11:01:42.106 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T11:01:42.106 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T11:01:42.107 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T11:01:42.182 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f69b28cf040>, signals=[15]) 2023-10-06T11:01:42.183 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T11:01:42.189 INFO:teuthology.task.internal:Opening connections... 2023-10-06T11:01:42.189 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:01:42.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:01:42.240 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:01:42.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:01:42.287 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T11:01:42.294 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2023-10-06T11:01:42.325 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2023-10-06T11:01:42.325 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="20.04" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T11:01:42.372 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=focal 2023-10-06T11:01:42.373 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=focal 2023-10-06T11:01:42.373 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-10-06T11:01:42.392 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-10-06T11:01:42.399 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-10-06T11:01:42.399 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-10-06T11:01:42.447 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Ubuntu" 2023-10-06T11:01:42.447 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-06T11:01:42.447 INFO:teuthology.orchestra.run.smithi132.stdout:ID=ubuntu 2023-10-06T11:01:42.447 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE=debian 2023-10-06T11:01:42.447 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="20.04" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_CODENAME=focal 2023-10-06T11:01:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:UBUNTU_CODENAME=focal 2023-10-06T11:01:42.448 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-10-06T11:01:42.467 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T11:01:42.477 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T11:01:42.482 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T11:01:42.482 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:01:42.485 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:01:42.490 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T11:01:42.496 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T11:01:42.497 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:01:42.499 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:01:42.515 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T11:01:42.569 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T11:01:42.585 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T11:01:42.585 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-06T11:01:42.586 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-06T11:01:42.586 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:01:42.586 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:01:42.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:01:42.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-10-06T11:01:42.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:01:42.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-10-06T11:01:42.753 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:01:42.753 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T11:01:42.753 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-10-06T11:01:42.758 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:01:42.758 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T11:01:42.758 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2023-10-06T11:01:42.760 INFO:teuthology.orchestra.run.smithi046.stdout:5.4.0-124-generic 2023-10-06T11:01:42.760 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:01:42.760 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T11:01:42.761 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-10-06T11:01:42.761 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-06T11:01:42.764 INFO:teuthology.orchestra.run.smithi132.stdout:5.4.0-124-generic 2023-10-06T11:01:42.765 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:01:42.765 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T11:01:42.765 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-10-06T11:01:42.765 DEBUG:teuthology.orchestra.run.smithi132:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-10-06T11:01:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-10-06T11:01:42.820 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-10-06T11:01:42.893 INFO:teuthology.orchestra.run.smithi046.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-06T11:01:42.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-06T11:01:42.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-06T11:01:42.940 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-06T11:01:42.946 INFO:teuthology.orchestra.run.smithi132.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-10-06T11:01:42.948 DEBUG:teuthology.orchestra.run.smithi132:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-10-06T11:01:42.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-10-06T11:01:43.015 DEBUG:teuthology.orchestra.run.smithi132:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-10-06T11:02:45.482 INFO:teuthology.orchestra.run.smithi132.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-06T11:02:45.483 INFO:teuthology.orchestra.run.smithi132.stderr: * [new branch] main -> origin/main 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190312 -> 20190312 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190416 -> 20190416 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190514 -> 20190514 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190618 -> 20190618 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190717 -> 20190717 2023-10-06T11:02:45.539 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190815 -> 20190815 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20190923 -> 20190923 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20191022 -> 20191022 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20191215 -> 20191215 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200122 -> 20200122 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200316 -> 20200316 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200421 -> 20200421 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200519 -> 20200519 2023-10-06T11:02:45.542 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200619 -> 20200619 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200721 -> 20200721 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200817 -> 20200817 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20200918 -> 20200918 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20201022 -> 20201022 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20201118 -> 20201118 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20201218 -> 20201218 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210208 -> 20210208 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210315 -> 20210315 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210511 -> 20210511 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210716 -> 20210716 2023-10-06T11:02:45.543 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210818 -> 20210818 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20210919 -> 20210919 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20211027 -> 20211027 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20211216 -> 20211216 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220209 -> 20220209 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220310 -> 20220310 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220411 -> 20220411 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220509 -> 20220509 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220610 -> 20220610 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220708 -> 20220708 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220815 -> 20220815 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20220913 -> 20220913 2023-10-06T11:02:45.546 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20221012 -> 20221012 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20221109 -> 20221109 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20221214 -> 20221214 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230117 -> 20230117 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230210 -> 20230210 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230310 -> 20230310 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230404 -> 20230404 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230515 -> 20230515 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230625 -> 20230625 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230804 -> 20230804 2023-10-06T11:02:45.547 INFO:teuthology.orchestra.run.smithi132.stderr: * [new tag] 20230919 -> 20230919 2023-10-06T11:02:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-10-06T11:02:46.957 INFO:teuthology.orchestra.run.smithi046.stderr: * [new branch] main -> origin/main 2023-10-06T11:02:47.012 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190312 -> 20190312 2023-10-06T11:02:47.012 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190416 -> 20190416 2023-10-06T11:02:47.013 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190514 -> 20190514 2023-10-06T11:02:47.013 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190618 -> 20190618 2023-10-06T11:02:47.013 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190717 -> 20190717 2023-10-06T11:02:47.013 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190815 -> 20190815 2023-10-06T11:02:47.013 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190923 -> 20190923 2023-10-06T11:02:47.016 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191022 -> 20191022 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191215 -> 20191215 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200122 -> 20200122 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200316 -> 20200316 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200421 -> 20200421 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200519 -> 20200519 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200619 -> 20200619 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200721 -> 20200721 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200817 -> 20200817 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200918 -> 20200918 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201022 -> 20201022 2023-10-06T11:02:47.017 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201118 -> 20201118 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201218 -> 20201218 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210208 -> 20210208 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210315 -> 20210315 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210511 -> 20210511 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210716 -> 20210716 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210818 -> 20210818 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210919 -> 20210919 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211027 -> 20211027 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211216 -> 20211216 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220209 -> 20220209 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220310 -> 20220310 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220411 -> 20220411 2023-10-06T11:02:47.018 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220509 -> 20220509 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220610 -> 20220610 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220708 -> 20220708 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220815 -> 20220815 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220913 -> 20220913 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221012 -> 20221012 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221109 -> 20221109 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221214 -> 20221214 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230117 -> 20230117 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230210 -> 20230210 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230310 -> 20230310 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230404 -> 20230404 2023-10-06T11:02:47.019 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230515 -> 20230515 2023-10-06T11:02:47.020 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230625 -> 20230625 2023-10-06T11:02:47.020 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230804 -> 20230804 2023-10-06T11:02:47.020 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230919 -> 20230919 2023-10-06T11:02:49.206 INFO:teuthology.orchestra.run.smithi132.stderr:Updating files: 51% (1630/3158) Updating files: 52% (1643/3158) Updating files: 53% (1674/3158) Updating files: 54% (1706/3158) Updating files: 55% (1737/3158) Updating files: 56% (1769/3158) Updating files: 57% (1801/3158) Updating files: 58% (1832/3158) Updating files: 59% (1864/3158) Updating files: 60% (1895/3158) Updating files: 61% (1927/3158) Updating files: 62% (1958/3158) Updating files: 63% (1990/3158) Updating files: 64% (2022/3158) Updating files: 65% (2053/3158) Updating files: 66% (2085/3158) Updating files: 66% (2104/3158) Updating files: 67% (2116/3158) Updating files: 68% (2148/3158) Updating files: 69% (2180/3158) Updating files: 70% (2211/3158) Updating files: 71% (2243/3158) Updating files: 72% (2274/3158) Updating files: 73% (2306/3158) Updating files: 74% (2337/3158) Updating files: 75% (2369/3158) Updating files: 76% (2401/3158) Updating files: 77% (2432/3158) Updating files: 78% (2464/3158) Updating files: 79% (2495/3158) Updating files: 80% (2527/3158) Updating files: 81% (2558/3158) Updating files: 81% (2587/3158) Updating files: 82% (2590/3158) Updating files: 83% (2622/3158) Updating files: 84% (2653/3158) Updating files: 85% (2685/3158) Updating files: 86% (2716/3158) Updating files: 87% (2748/3158) Updating files: 88% (2780/3158) Updating files: 89% (2811/3158) Updating files: 90% (2843/3158) Updating files: 91% (2874/3158) Updating files: 92% (2906/3158) Updating files: 93% (2937/3158) Updating files: 94% (2969/3158) Updating files: 95% (3001/3158) Updating files: 96% (3032/3158) Updating files: 97% (3064/3158) Updating files: 98% (3095/3158) Updating files: 99% (3127/3158) Updating files: 100% (3158/3158) Updating files: 100% (3158/3158), done. 2023-10-06T11:02:49.210 INFO:teuthology.orchestra.run.smithi132.stdout:HEAD is now at 5105ff4b Merge branch 'mlimonci/upstream-packaging' into 'main' 2023-10-06T11:02:49.226 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T11:02:49.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:02:49.450 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:02:49.450 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ --input-file=- 2023-10-06T11:02:49.452 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi132.front.sepia.ceph.com to complete... 2023-10-06T11:02:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:Updating files: 49% (1566/3158) Updating files: 50% (1579/3158) Updating files: 51% (1611/3158) Updating files: 52% (1643/3158) Updating files: 53% (1674/3158) Updating files: 54% (1706/3158) Updating files: 55% (1737/3158) Updating files: 56% (1769/3158) Updating files: 57% (1801/3158) Updating files: 58% (1832/3158) Updating files: 59% (1864/3158) Updating files: 60% (1895/3158) Updating files: 61% (1927/3158) Updating files: 62% (1958/3158) Updating files: 63% (1990/3158) Updating files: 64% (2022/3158) Updating files: 64% (2050/3158) Updating files: 65% (2053/3158) Updating files: 66% (2085/3158) Updating files: 67% (2116/3158) Updating files: 68% (2148/3158) Updating files: 69% (2180/3158) Updating files: 70% (2211/3158) Updating files: 71% (2243/3158) Updating files: 72% (2274/3158) Updating files: 73% (2306/3158) Updating files: 74% (2337/3158) Updating files: 75% (2369/3158) Updating files: 76% (2401/3158) Updating files: 77% (2432/3158) Updating files: 78% (2464/3158) Updating files: 79% (2495/3158) Updating files: 80% (2527/3158) Updating files: 81% (2558/3158) Updating files: 81% (2573/3158) Updating files: 82% (2590/3158) Updating files: 83% (2622/3158) Updating files: 84% (2653/3158) Updating files: 85% (2685/3158) Updating files: 86% (2716/3158) Updating files: 87% (2748/3158) Updating files: 88% (2780/3158) Updating files: 89% (2811/3158) Updating files: 90% (2843/3158) Updating files: 91% (2874/3158) Updating files: 92% (2906/3158) Updating files: 93% (2937/3158) Updating files: 94% (2969/3158) Updating files: 95% (3001/3158) Updating files: 96% (3032/3158) Updating files: 97% (3064/3158) Updating files: 98% (3095/3158) Updating files: 99% (3127/3158) Updating files: 100% (3158/3158) Updating files: 100% (3158/3158), done. 2023-10-06T11:02:50.864 INFO:teuthology.orchestra.run.smithi046.stdout:HEAD is now at 5105ff4b Merge branch 'mlimonci/upstream-packaging' into 'main' 2023-10-06T11:02:50.881 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T11:02:50.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T11:02:51.099 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:02:51.099 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/linux-image.deb && echo linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/ --input-file=- 2023-10-06T11:02:51.101 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2023-10-06T11:02:51.136 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06 11:02:51 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb [38832758/38832758] -> "/tmp/linux-image.deb" [1] 2023-10-06T11:02:51.136 INFO:teuthology.orchestra.run.smithi132.stderr:FINISHED --2023-10-06 11:02:51-- 2023-10-06T11:02:51.136 INFO:teuthology.orchestra.run.smithi132.stderr:Total wall clock time: 1.7s 2023-10-06T11:02:51.136 INFO:teuthology.orchestra.run.smithi132.stderr:Downloaded: 1 files, 37M in 1.5s (25.3 MB/s) 2023-10-06T11:02:51.138 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T11:02:51.138 DEBUG:teuthology.orchestra.run.smithi132:> sudo dpkg -i /tmp/linux-image.deb 2023-10-06T11:02:51.411 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package linux-image-6.5.0-rc7-gfef64f93da2b. 2023-10-06T11:02:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06 11:02:52 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/ubuntu/focal/flavors/default/pool/main/l/linux-6.5.0-rc7-gfef64f93da2b/linux-image-6.5.0-rc7-gfef64f93da2b_6.5.0-rc7-gfef64f93da2b-1_amd64.deb [38832758/38832758] -> "/tmp/linux-image.deb" [1] 2023-10-06T11:02:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2023-10-06 11:02:52-- 2023-10-06T11:02:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 1.6s 2023-10-06T11:02:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 37M in 1.4s (25.6 MB/s) 2023-10-06T11:02:52.777 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T11:02:52.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo dpkg -i /tmp/linux-image.deb 2023-10-06T11:02:53.092 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-6.5.0-rc7-gfef64f93da2b. 2023-10-06T11:02:53.143 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-10-06T11:02:53.148 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-06T11:02:53.297 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-06T11:02:54.776 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-10-06T11:02:54.781 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-10-06T11:02:54.907 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-06T11:02:56.288 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-06T11:02:56.393 INFO:teuthology.orchestra.run.smithi132.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:02:57.785 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-6.5.0-rc7-gfef64f93da2b (6.5.0-rc7-gfef64f93da2b-1) ... 2023-10-06T11:02:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:update-initramfs: Generating /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:21.942 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:03:21.951 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:03:21.985 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2023-10-06T11:03:22.330 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:22.365 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:22.726 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:03:22.739 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:03:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:03:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:03:23.847 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2023-10-06T11:03:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:24.289 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:03:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:03:25.387 INFO:teuthology.orchestra.run.smithi132.stderr:done 2023-10-06T11:03:25.567 DEBUG:teuthology.orchestra.run.smithi132:> dpkg-deb -c /tmp/linux-image.deb 2023-10-06T11:03:25.577 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./ 2023-10-06T11:03:25.577 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./boot/ 2023-10-06T11:03:25.577 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 4382168 2023-09-11 02:20 ./boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.593 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 168454 2023-09-11 02:20 ./boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.593 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11294080 2023-09-11 02:20 ./boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postinst.d/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postrm.d/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/preinst.d/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/prerm.d/ 2023-10-06T11:03:25.615 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 490385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 93113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T11:03:25.616 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 106729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T11:03:25.617 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 152241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T11:03:25.618 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 83889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 58585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T11:03:25.619 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/ 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/ 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpuid.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/msr.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/ 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 348201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T11:03:25.620 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 882105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T11:03:25.621 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 2277369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/af_alg.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_hash.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_skcipher.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T11:03:25.637 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/ 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/ 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T11:03:25.638 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T11:03:25.641 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cryptd.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_simd.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T11:03:25.642 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lrw.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T11:03:25.643 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 149193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xts.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ 2023-10-06T11:03:25.644 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/ 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/einj.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T11:03:25.645 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 88625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 87841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T11:03:25.646 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_legacy.ko 2023-10-06T11:03:25.647 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T11:03:25.648 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_radisys.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rz1000.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 83225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T11:03:25.649 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ 2023-10-06T11:03:25.650 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/ 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 124457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/ 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/ 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 113705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/ 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 968433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 173201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/ 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 94297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 115345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T11:03:25.651 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/ 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 110361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 98225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 259401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/ 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 98129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/ 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/applicom.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/ 2023-10-06T11:03:25.652 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 150665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 139057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T11:03:25.653 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/ 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 84121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/nvram.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/ 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-06T11:03:25.654 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/ 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/ 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-palmas.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-si5351.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-twl6040.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-wm831x.ko 2023-10-06T11:03:25.655 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 234473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-06T11:03:25.656 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 408489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T11:03:25.703 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-06T11:03:25.703 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T11:03:25.703 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/ 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/ 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/ 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 122641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T11:03:25.704 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T11:03:25.705 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/ 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/ 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-06T11:03:25.706 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/ 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/ 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 77849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-generic.ko 2023-10-06T11:03:25.707 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch.ko 2023-10-06T11:03:25.708 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/ 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-a4tech.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-apple.ko 2023-10-06T11:03:25.709 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-belkin.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cherry.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-chicony.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cp2112.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cypress.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T11:03:25.710 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ezkey.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-generic.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kensington.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T11:03:25.711 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 101009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-microsoft.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-monterey.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T11:03:25.712 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ntrig.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 72937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T11:03:25.713 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T11:03:25.714 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T11:03:25.715 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 143585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 267689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/ 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 125721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 217401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/ 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T11:03:25.716 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 301449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 97977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T11:03:25.717 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 74433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T11:03:25.718 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 58617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 99601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T11:03:25.719 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc2103.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 94457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hih6130.ko 2023-10-06T11:03:25.720 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 129217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/jc42.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T11:03:25.721 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 72833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 108841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T11:03:25.722 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T11:03:25.723 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 116697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 82337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T11:03:25.724 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T11:03:25.725 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T11:03:25.726 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smm665.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T11:03:25.727 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 101457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 119505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-06T11:03:25.728 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/ 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/ 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/ 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T11:03:25.729 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T11:03:25.730 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-06T11:03:25.731 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/ 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-06T11:03:25.732 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 273641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 876913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 324145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 97961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 279345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 492569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T11:03:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 419193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 739705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 296649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 153681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 169017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-06T11:03:25.734 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 263177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 111993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 129081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 129553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/evbug.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ff-memless.ko 2023-10-06T11:03:25.735 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/ 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-06T11:03:25.736 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-06T11:03:25.737 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/matrix-keymap.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/ 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T11:03:25.738 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 123681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 335849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/psmouse.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/ 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 109849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ 2023-10-06T11:03:25.739 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ct82c710.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/parkbd.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/pcips2.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ps2mult.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serport.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/ 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/ 2023-10-06T11:03:25.740 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/ 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-adp5520.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-bd2802.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da903x.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da9052.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-gpio.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3533.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm355x.ko 2023-10-06T11:03:25.741 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3642.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp8788.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-max8997.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-mc13783.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca9532.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca955x.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca963x.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-tca6507.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-06T11:03:25.742 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm8350.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/ 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/ 2023-10-06T11:03:25.743 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 655473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bufio.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 155993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 107049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-era.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-mirror.ko 2023-10-06T11:03:25.744 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-region-hash.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 109593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-snapshot.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 181585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-zero.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T11:03:25.745 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/ 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 199937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 112609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 160553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 396153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/ 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/ 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 178113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T11:03:25.746 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 123353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 79017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm800.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm805.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm80x.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 123721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/bcm590xx.ko 2023-10-06T11:03:25.747 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/dln2.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/kempld-core.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-core.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lp3943.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/max8907.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-06T11:03:25.748 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/menf21bmc.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/retu-mfd.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/si476x-core.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps65010.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps6507x.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T11:03:25.749 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wm8994.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/ 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/ 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-06T11:03:25.750 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/core.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/ 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ds1682.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/dummy-irq.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/ 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/echo.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/ 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T11:03:25.751 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/ 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 152137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hmc6352.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 72601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ics932s401.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/ 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T11:03:25.752 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/ 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 125209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 309857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/phantom.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/ 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 74905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T11:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 183857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 487865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 106009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-06T11:03:25.754 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 452681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/ 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 130489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T11:03:25.755 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 99113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/ 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/ 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/ 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/ 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/ 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T11:03:25.756 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 87345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 97321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 104609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 90257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/ 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T11:03:25.757 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 176881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1422209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 128377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 132289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 312145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 323553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-06T11:03:25.758 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 193433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 347641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1044145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 166113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 204329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T11:03:25.759 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/ 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 141161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/ 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-06T11:03:25.760 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 281329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 90465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 295713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 584249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T11:03:25.761 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-06T11:03:25.762 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 246321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T11:03:25.762 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-06T11:03:25.762 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 924017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 302577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 514889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 93593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 734297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-06T11:03:25.816 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 151841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 86921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/ 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 102385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 122681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/ 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 727433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 280513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-06T11:03:25.817 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1097537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 95665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/ 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 108809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T11:03:25.818 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/ 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 150905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/ 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 136785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/ 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/ 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 227113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 87081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T11:03:25.819 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 671393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 83921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 189817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 104241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/ 2023-10-06T11:03:25.820 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 952929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/ 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/ 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/ 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/ 2023-10-06T11:03:25.821 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 481681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/ 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 103321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 166121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/ 2023-10-06T11:03:25.822 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/ 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 86465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/ 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 83681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/ 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/ 2023-10-06T11:03:25.823 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/ 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/defxx.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/ 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 216977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 77401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/ 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 235593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/ 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ 2023-10-06T11:03:25.824 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 83993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/ 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T11:03:25.825 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/ 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T11:03:25.826 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 75713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 97265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/ 2023-10-06T11:03:25.827 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/plip.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sb1000.ko 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/ 2023-10-06T11:03:25.828 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 64937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/ 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 106025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 88257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T11:03:25.829 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 120497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 293441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 143889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/ 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 264585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 102393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T11:03:25.830 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/ 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 133817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 84337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/ 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/ 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pci-stub.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/ 2023-10-06T11:03:25.831 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 145785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 112913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/ 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/ 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/ 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/ 2023-10-06T11:03:25.832 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/ 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/ 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/ 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ 2023-10-06T11:03:25.833 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 106873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T11:03:25.834 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/ 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T11:03:25.835 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/ 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-06T11:03:25.836 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-06T11:03:25.837 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8925_power.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/test_power.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/ 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T11:03:25.838 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/ 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/ 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/ 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-06T11:03:25.839 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T11:03:25.840 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-06T11:03:25.841 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T11:03:25.842 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T11:03:25.843 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-06T11:03:25.845 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-06T11:03:25.845 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 77617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 135169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/ 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 272353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T11:03:25.846 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/ 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 356265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 295801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/ 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 189321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/ 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 106569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/ 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 246897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T11:03:25.847 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1172185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 237169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 115393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 308889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 76193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-06T11:03:25.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 101449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 146785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 75529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/ 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 212401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/ 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 158193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fdomain.ko 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/ 2023-10-06T11:03:25.849 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 262729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 201793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/imm.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 266521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 101881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/ 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 317249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T11:03:25.850 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/ 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 314961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 145569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/ 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 217209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/ 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 2192585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/ 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 310729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T11:03:25.851 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 84697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/ 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 659513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/ 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 119929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/ 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/ 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 364257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T11:03:25.852 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 123489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ppa.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/ 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1860193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 85313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/ 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 558865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 238801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T11:03:25.853 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 173921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 278865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 97705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 115849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 162121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T11:03:25.854 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 129417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/ 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/ 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 739089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/ 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 931721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T11:03:25.855 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 122545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/ 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 75641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/ 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/ 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T11:03:25.856 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ 2023-10-06T11:03:25.857 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/moxa.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 50793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/mxser.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 107545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/ 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/ 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-06T11:03:25.858 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 72193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/ 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 103017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/rp2.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/ 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 536809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ 2023-10-06T11:03:25.859 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/ 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_mf624.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_netx.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T11:03:25.860 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/ 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/ 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 143785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/ 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 85457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T11:03:25.861 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ulpi.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/ 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/ 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 445217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/ 2023-10-06T11:03:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/ 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 92377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 122073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-06T11:03:25.863 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 85841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 58433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/ 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-06T11:03:25.864 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 62153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 161321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/ 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-06T11:03:25.865 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 157665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 76033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 75057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-06T11:03:25.866 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 156057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 87337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/ 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T11:03:25.867 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cytherm.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ehset.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T11:03:25.868 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/lvstest.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 64841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbtest.ko 2023-10-06T11:03:25.869 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/ 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/usbmon.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/ 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 301521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/ 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-06T11:03:25.870 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 70209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/f81232.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 175145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T11:03:25.871 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 137105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 124673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 107417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T11:03:25.872 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 78113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mxuport.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 190233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T11:03:25.873 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 51297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 125505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usbserial.ko 2023-10-06T11:03:25.874 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T11:03:25.875 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 180017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/ 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 105401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T11:03:25.876 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/ 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/ 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 153697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 75137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/ 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 113889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 58137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/ 2023-10-06T11:03:25.877 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/ 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/bd6107.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-06T11:03:25.878 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T11:03:25.879 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/ 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/ 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T11:03:25.880 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 102657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-06T11:03:25.881 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T11:03:25.882 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T11:03:25.883 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T11:03:25.884 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T11:03:25.885 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/ 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/ 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 176417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-scsiback.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/ 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/ 2023-10-06T11:03:25.886 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 165625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/adfs.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 162617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 931937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/kafs.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/autofs4.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 64481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/ 2023-10-06T11:03:25.887 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/bfs.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 3474249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 305745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1316257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 108073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/configfs.ko 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/ 2023-10-06T11:03:25.888 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 29273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 633825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/efs.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1500513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/freevxfs.ko 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/ 2023-10-06T11:03:25.889 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 226505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/ 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/ 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1226385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/ 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 141241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/ 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 221553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/ 2023-10-06T11:03:25.890 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 193577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/hpfs.ko 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/ 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 74393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/ 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 421097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/ 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 279041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/ 2023-10-06T11:03:25.926 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 87833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/ 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 124697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/ 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/ 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/ 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/ 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 117705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1053889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T11:03:25.927 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv2.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 119801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 2041809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/ 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/ 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1504929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/ 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 591401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/ 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T11:03:25.928 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_ascii.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T11:03:25.929 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T11:03:25.930 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 94137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 142673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 165745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 118121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T11:03:25.931 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T11:03:25.932 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 219361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ntfs.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 244177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 440785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 2589873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T11:03:25.933 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/ 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 52569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/omfs.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/ 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 336889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/ 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/qnx4.ko 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/ 2023-10-06T11:03:25.934 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 46649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/qnx6.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/ 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_tree.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v1.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v2.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/ 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 530697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/ 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/ 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/ 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 2661209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T11:03:25.935 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/ 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/ 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 134313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/ 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/ 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 248049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 160857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T11:03:25.936 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/ 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 3998129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/ 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/ 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 74865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/locktorture.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/ 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/test_udelay.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 58473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc7.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T11:03:25.937 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/ 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/interval_tree_test.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/ 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/ 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T11:03:25.938 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/memory-notifier-error-inject.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/notifier-error-inject.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/percpu_test.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/pm-notifier-error-inject.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/ 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 143433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/rbtree_test.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/ 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-kstrtox.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-string_helpers.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 4760825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_bpf.ko 2023-10-06T11:03:25.939 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_firmware.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_module.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_user_copy.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/ 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/xz_dec_test.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/ 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/ 2023-10-06T11:03:25.940 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 6985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/ 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T11:03:25.941 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/ 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 81017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/ 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 198465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/ 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 652553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T11:03:25.942 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/ 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 63825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 772505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T11:03:25.943 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T11:03:25.944 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 55937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/ 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1046921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/ 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 124793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/ 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 194321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 56945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T11:03:25.945 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 59457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/ 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 390297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/ 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 100257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/hsr.ko 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ 2023-10-06T11:03:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 45169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 254601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 60593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T11:03:25.947 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T11:03:25.948 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-06T11:03:25.949 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T11:03:25.950 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 41577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T11:03:25.951 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 80793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_gre.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 92817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/mip6.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 42209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T11:03:25.952 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T11:03:25.953 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T11:03:25.954 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 73161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/ 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 82929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/ 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 125873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 25865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T11:03:25.955 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 53153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/ 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/lapb.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/ 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 267257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc2.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/ 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 291041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/ 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ 2023-10-06T11:03:25.956 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 119649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 102033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 103121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 104593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 105129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 114393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 111161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T11:03:25.957 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 114673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 114329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 113217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 116217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 412889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T11:03:25.958 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 408665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T11:03:25.959 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 118145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 108393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 66641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T11:03:25.960 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 88513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 37129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 95353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 44025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 609257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T11:03:25.961 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 39905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 49673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 40721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T11:03:25.962 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_osf.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T11:03:25.963 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_synproxy.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tproxy.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 89329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/x_tables.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 18169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T11:03:25.964 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T11:03:25.965 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T11:03:25.966 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T11:03:25.967 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T11:03:25.968 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 8553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T11:03:25.969 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/ 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/ 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 12425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/ 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 379545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T11:03:25.970 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/ 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/ 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 109961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/phonet.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 54761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/pn_pep.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/ 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 252889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 165081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 67241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/ 2023-10-06T11:03:25.971 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 820137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/rxrpc.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/ 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 34057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 43817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T11:03:25.972 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_cgroup.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 38489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T11:03:25.973 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 21649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 33609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq_codel.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 48409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 16065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T11:03:25.974 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 35601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 32953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 26881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 31425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T11:03:25.975 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 28897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 27729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/ 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 930369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 19033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/ 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/ 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 335569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 61833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 1514673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/ 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 707401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/ 2023-10-06T11:03:25.976 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 862385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/ 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/ 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 84841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 68033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 95985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 10705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 17521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/ 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 125097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/x25.ko 2023-10-06T11:03:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/ 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 22657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_algo.ko 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 20705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 102353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_user.ko 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/ 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/ 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 15513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 638591 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 624324 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 7656 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 23361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-06T11:03:25.978 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 9821 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 69921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 146449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 232695 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 210 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 72754 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 879 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 252880 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 307898 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/ 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/ 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/ 2023-10-06T11:03:25.979 INFO:teuthology.orchestra.run.smithi132.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:03:25.980 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 172 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/changelog.Debian.gz 2023-10-06T11:03:25.980 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r-- root/root 691 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/copyright 2023-10-06T11:03:25.980 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.981 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.981 DEBUG:teuthology.orchestra.run.smithi132:> egrep '(submenu|menuentry.*6.5.0-rc7-gfef64f93da2b).*{' /boot/grub/grub.cfg 2023-10-06T11:03:25.987 INFO:teuthology.orchestra.run.smithi132.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:25.987 INFO:teuthology.orchestra.run.smithi132.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:25.988 INFO:teuthology.orchestra.run.smithi132.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:25.988 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-06T11:03:25.988 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:25.988 DEBUG:teuthology.orchestra.run.smithi132:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-06T11:03:25.990 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi132.front.sepia.ceph.com to complete... 2023-10-06T11:03:26.666 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:03:26.676 INFO:teuthology.orchestra.run.smithi132.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:03:26.684 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2023-10-06T11:03:26.910 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:26.958 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.269 INFO:teuthology.orchestra.run.smithi132.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:03:27.281 INFO:teuthology.orchestra.run.smithi132.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:03:27.409 INFO:teuthology.orchestra.run.smithi046.stderr:done 2023-10-06T11:03:27.553 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-deb -c /tmp/linux-image.deb 2023-10-06T11:03:27.563 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./ 2023-10-06T11:03:27.563 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./boot/ 2023-10-06T11:03:27.563 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4382168 2023-09-11 02:20 ./boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.578 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 168454 2023-09-11 02:20 ./boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.579 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11294080 2023-09-11 02:20 ./boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postinst.d/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/postrm.d/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/preinst.d/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./etc/kernel/prerm.d/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/ 2023-10-06T11:03:27.600 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:03:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/ 2023-10-06T11:03:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/ 2023-10-06T11:03:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/ 2023-10-06T11:03:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ 2023-10-06T11:03:27.601 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 490385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/aesni-intel.ko 2023-10-06T11:03:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T11:03:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T11:03:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T11:03:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T11:03:27.604 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/ 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/ 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/ 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpuid.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/msr.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/ 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 348201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T11:03:27.605 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 882105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T11:03:27.606 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2277369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/af_alg.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_hash.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/algif_skcipher.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/ 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-10-06T11:03:27.619 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/ 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T11:03:27.620 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cryptd.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_simd.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T11:03:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lrw.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T11:03:27.622 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 149193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T11:03:27.623 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xts.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_extlog.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/ 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/apei/einj.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T11:03:27.624 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T11:03:27.625 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T11:03:27.626 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_legacy.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T11:03:27.627 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_radisys.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rz1000.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T11:03:27.628 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T11:03:27.629 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/ 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/ 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/ 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/ 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 968433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T11:03:27.630 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 173201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/ 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/ 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T11:03:27.631 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 259401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T11:03:27.676 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/ 2023-10-06T11:03:27.676 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/ 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/applicom.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/ 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T11:03:27.677 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/ 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/nvram.ko 2023-10-06T11:03:27.678 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T11:03:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T11:03:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/ 2023-10-06T11:03:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T11:03:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/ 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_class.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_core.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/ 2023-10-06T11:03:27.682 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-palmas.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-si5351.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-twl6040.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/clk/clk-wm831x.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/ 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 234473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T11:03:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 408489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ 2023-10-06T11:03:27.684 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/ 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T11:03:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/ 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-gpio.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max14577.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max77693.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-max8997.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-palmas.ko 2023-10-06T11:03:27.686 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-rt8973a.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/extcon/extcon-sm5502.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/ 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/dmi-sysfs.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/ 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/efi/efi-pstore.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/ 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-adp5520.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-amd8111.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-arizona.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9052.ko 2023-10-06T11:03:27.687 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-da9055.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-dln2.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-f7188x.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-generic.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-kempld.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-lp3943.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max7300.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max730x.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-max732x.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pca953x.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-pcf857x.ko 2023-10-06T11:03:27.688 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-rdc321x.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-sch311x.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-tps65912.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl4030.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-twl6040.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-vx855.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm831x.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8350.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-wm8994.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/ 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-a4tech.ko 2023-10-06T11:03:27.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-apple.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-belkin.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cherry.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-chicony.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cp2112.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-cypress.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T11:03:27.690 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ezkey.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-generic.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T11:03:27.691 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kensington.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lg-g15.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-magicmouse.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-microsoft.ko 2023-10-06T11:03:27.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-monterey.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ntrig.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T11:03:27.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T11:03:27.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T11:03:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 267689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/ 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbhid.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbkbd.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/usbhid/usbmouse.ko 2023-10-06T11:03:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 217401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/ 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 301449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T11:03:27.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T11:03:27.698 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9052-hwmon.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/da9055-hwmon.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T11:03:27.699 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc2103.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T11:03:27.700 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hih6130.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/jc42.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T11:03:27.701 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T11:03:27.702 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T11:03:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mc13783-adc.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T11:03:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T11:03:27.705 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T11:03:27.706 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smm665.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T11:03:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T11:03:27.708 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/ 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/ 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/ 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T11:03:27.709 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T11:03:27.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T11:03:27.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/ 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-10-06T11:03:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 273641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 876913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 324145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 279345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ 2023-10-06T11:03:27.713 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 492569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 419193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 739705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 296649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ 2023-10-06T11:03:27.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 263177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 111993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/evbug.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/ff-memless.ko 2023-10-06T11:03:27.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/ 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-10-06T11:03:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lkkbd.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8323.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/lm8333.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/newtonkbd.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt1070.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/qt2160.ko 2023-10-06T11:03:27.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/stowaway.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/sunkbd.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/xtkbd.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/matrix-keymap.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/ 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T11:03:27.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/gpio_mouse.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 335849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/psmouse.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/ 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ct82c710.ko 2023-10-06T11:03:27.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/parkbd.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/pcips2.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/ps2mult.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serport.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/ 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/ 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/ 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-88pm860x.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-adp5520.ko 2023-10-06T11:03:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-bd2802.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da903x.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-da9052.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-gpio.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3533.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm355x.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3642.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp8788.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-max8997.ko 2023-10-06T11:03:27.721 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-mc13783.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-menf21bmc.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca9532.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca955x.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-pca963x.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-tca6507.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm831x-status.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-wm8350.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T11:03:27.722 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/ 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/ 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 655473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bufio.ko 2023-10-06T11:03:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 155993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-era.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-mirror.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-region-hash.ko 2023-10-06T11:03:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-snapshot.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 181585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-zero.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/ 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 199937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T11:03:27.725 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 160553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 396153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/ 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/ 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 178113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T11:03:27.726 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm800.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm805.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/88pm80x.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona-i2c.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/arizona.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/bcm590xx.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/cros_ec_dev.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/dln2.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/janz-cmodio.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/kempld-core.ko 2023-10-06T11:03:27.727 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-core.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lp3943.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/max8907.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-core.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/menf21bmc.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-adc.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633-gpio.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/pcf50633.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-10-06T11:03:27.728 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/retu-mfd.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/si476x-core.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps65010.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/tps6507x.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wm8994.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-10-06T11:03:27.729 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ad525x_dpot.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/ 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/ 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/c2port/core.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/ 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ds1682.ko 2023-10-06T11:03:27.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/dummy-irq.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/ 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/echo/echo.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/ 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/ 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hmc6352.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T11:03:27.731 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ics932s401.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/ 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/ 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 309857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T11:03:27.732 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/phantom.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/ 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/ti-st/st_drv.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/ 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 183857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/ 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 487865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T11:03:27.733 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/ 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/ 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/ 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 452681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/ 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T11:03:27.734 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 130489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/ 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/ 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/ 2023-10-06T11:03:27.735 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/ 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/ 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/ 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/ 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T11:03:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/ 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/ 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-10-06T11:03:27.737 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1422209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T11:03:27.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 132289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 312145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 323553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-10-06T11:03:27.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 347641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1044145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 166113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 204329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/ 2023-10-06T11:03:27.794 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/ 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/ 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ec_bhf.ko 2023-10-06T11:03:27.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 281329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 295713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/ 2023-10-06T11:03:27.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 584249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 246321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 924017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 302577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 514889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 734297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T11:03:27.797 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/ 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/ 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 727433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 280513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T11:03:27.798 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1097537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T11:03:27.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/ 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/ 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/ 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/ 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 227113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T11:03:27.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 671393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/ 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/ 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 189817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/ 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-10-06T11:03:27.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/ 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 952929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/ 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/ 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/ 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T11:03:27.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/ 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 481681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/ 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 166121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T11:03:27.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/ 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/ 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/ 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/ 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/ 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T11:03:27.804 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/ 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/defxx.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/ 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 216977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/fddi/skfp/skfp.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/ 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 235593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/ 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T11:03:27.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/ 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-gpio.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T11:03:27.806 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/ 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T11:03:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/ 2023-10-06T11:03:27.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/plip/plip.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sb1000.ko 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/ 2023-10-06T11:03:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/ 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/ 2023-10-06T11:03:27.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 293441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 264585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/ 2023-10-06T11:03:27.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/ 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/ 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pci-stub.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/ 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T11:03:27.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 145785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_core.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/ 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/ 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/ 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/ 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/ 2023-10-06T11:03:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/ 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/ 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T11:03:27.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/ibm_rtl.ko 2023-10-06T11:03:27.815 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/ 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T11:03:27.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/ 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_battery.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/88pm860x_charger.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq2415x_charger.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24190_charger.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/bq24735-charger.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9030_battery.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/da9052-battery.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2760_battery.ko 2023-10-06T11:03:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2780_battery.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2781_battery.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/ds2782_battery.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/gpio-charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/isp1704_charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/lp8727_charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max14577_charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17040_battery.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max17042_battery.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8903_charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/max8925_power.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/pcf50633-charger.ko 2023-10-06T11:03:27.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/sbs-battery.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/test_power.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/tps65090-charger.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_backup.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm831x_power.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/power/supply/wm8350_power.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/ 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/ 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/ 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T11:03:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/ 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm80x.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-88pm860x.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9052.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9055.ko 2023-10-06T11:03:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-da9063.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T11:03:27.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-lp8788.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t86.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8907.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8925.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8997.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max8998.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-10-06T11:03:27.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-palmas.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf50633.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rc5t583.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T11:03:27.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-s35390a.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps6586x.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-tps65910.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm831x.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-wm8350.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T11:03:27.824 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/ 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 272353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/ 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 295801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/ 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 189321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T11:03:27.825 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/ 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/ 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 246897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/ 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1172185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/ 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 237169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/ 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T11:03:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/ 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 308889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/ 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/ 2023-10-06T11:03:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 212401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/ 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fdomain.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/ 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 262729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 201793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/imm.ko 2023-10-06T11:03:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 266521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/ 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 317249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/ 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 314961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 145569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/ 2023-10-06T11:03:27.829 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 217209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/ 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2192585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/ 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 310729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/ 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 659513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/ 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T11:03:27.830 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/ 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/ 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 364257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ppa.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/ 2023-10-06T11:03:27.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1860193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/ 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 558865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qlogicfas408.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 238801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 173921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 278865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T11:03:27.832 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/ 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 162121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/xen-scsifront.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/ 2023-10-06T11:03:27.833 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/ 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 739089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/ 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 931721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/ 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/ 2023-10-06T11:03:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/ 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/ 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T11:03:27.835 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/moxa.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/mxser.ko 2023-10-06T11:03:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/ 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/ 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/altera_uart.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/ 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T11:03:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/rp2.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/sc16is7xx.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/ 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 536809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/ 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T11:03:27.838 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_dmem_genirq.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_mf624.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_netx.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/ 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/ 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/c67x00/c67x00.ko 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ 2023-10-06T11:03:27.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/ 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/common/ulpi.ko 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/ 2023-10-06T11:03:27.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/ 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 445217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/dwc3/dwc3.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/ 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/ 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_ether.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/u_serial.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-10-06T11:03:27.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/ 2023-10-06T11:03:27.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-10-06T11:03:27.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15201 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 161321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/libcomposite.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/ 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/ 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2272.ko 2023-10-06T11:03:27.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/net2280.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 156057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/bcma-hcd.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/isp116x-hcd.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T11:03:27.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811_cs.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/ssb-hcd.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/ 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/cytherm.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ehset.ko 2023-10-06T11:03:27.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/lvstest.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/ 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T11:03:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbtest.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/ 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/mon/usbmon.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/ 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 301521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/musb/musb_hdrc.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/ 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-10-06T11:03:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-isp1301.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-tahvo.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/f81232.ko 2023-10-06T11:03:27.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 175145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T11:03:27.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/metro-usb.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mxuport.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 190233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T11:03:27.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usbserial.ko 2023-10-06T11:03:27.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/wishbone-serial.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T11:03:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 180017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/ 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T11:03:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/ 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/ 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/ 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/ 2023-10-06T11:03:27.855 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/ 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/aat2870_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp5520_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8860_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/adp8870_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/as3711_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/bd6107.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da903x_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/da9052_bl.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/gpio_backlight.ko 2023-10-06T11:03:27.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/kb3886_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3533_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lm3639_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lv5207lp.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/max8925_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pandora_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/wm831x_bl.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/ 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T11:03:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/matrox_w1.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/w1-gpio.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/ 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T11:03:27.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/acquirewdt.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19977 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/advantechwdt.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/cpu5wdt.ko 2023-10-06T11:03:27.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9052_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9055_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/da9063_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/dw_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/eurotechwdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T11:03:27.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/kempld_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/mena21_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pc87413_wdt.ko 2023-10-06T11:03:27.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/retu_wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sc1200wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/twl4030_wdt.ko 2023-10-06T11:03:27.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wafer5823wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm831x_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wm8350_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/ 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T11:03:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176417 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-scsiback.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/adfs/adfs.ko 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/ 2023-10-06T11:03:27.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 162617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/ 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 931937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/afs/kafs.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/ 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/autofs/autofs4.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/ 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/ 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/bfs/bfs.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/ 2023-10-06T11:03:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3474249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/ 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 305745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1316257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/ 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/ 2023-10-06T11:03:27.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/configfs/configfs.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 633825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/efs/efs.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1500513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/ 2023-10-06T11:03:27.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/freevxfs/freevxfs.ko 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/ 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 226505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/ 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/ 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1226385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/ 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/ 2023-10-06T11:03:27.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 221553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hpfs/hpfs.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 421097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 279041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/ 2023-10-06T11:03:27.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/ 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/ 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/ 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/ 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1053889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv2.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2041809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T11:03:27.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/ 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/ 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1504929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/ 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 591401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/ 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T11:03:27.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13481 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_ascii.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T11:03:27.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T11:03:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T11:03:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 219361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ntfs/ntfs.ko 2023-10-06T11:03:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 244177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 440785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2589873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T11:03:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/omfs/omfs.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 336889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx4/qnx4.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/qnx6/qnx6.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/ 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_tree.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v1.ko 2023-10-06T11:03:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/quota/quota_v2.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 530697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2661209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/ 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T11:03:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 248049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 160857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3998129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/locking/locktorture.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/ 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/time/test_udelay.ko 2023-10-06T11:03:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc7.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/ 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/interval_tree_test.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T11:03:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/ 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/ 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/memory-notifier-error-inject.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/notifier-error-inject.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/percpu_test.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/pm-notifier-error-inject.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/ 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/rbtree_test.ko 2023-10-06T11:03:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/ 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-kstrtox.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test-string_helpers.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4760825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_bpf.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_firmware.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_module.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/test_user_copy.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/ 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/xz/xz_dec_test.ko 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/ 2023-10-06T11:03:27.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/ 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/ 2023-10-06T11:03:27.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/ 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/ 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 198465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T11:03:27.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/ 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 652553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/ 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 772505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T11:03:27.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7585 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T11:03:27.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/ 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1046921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/ 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T11:03:27.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/ 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 194321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/ 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 390297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/ 2023-10-06T11:03:27.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/hsr/hsr.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 254601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48529 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T11:03:27.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61281 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ip_tables.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-10-06T11:03:27.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T11:03:27.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20561 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T11:03:27.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T11:03:27.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_gre.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19017 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/mip6.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T11:03:27.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8913 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10937 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T11:03:27.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9009 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T11:03:27.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24193 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23305 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/ 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/ 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35049 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53153 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T11:03:27.935 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/lapb/lapb.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23225 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 267257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc2.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 291041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T11:03:27.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103121 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 114393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 111161 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 114673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 114329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T11:03:27.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116217 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 412889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10681 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T11:03:27.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7697 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10993 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 408665 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10177 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T11:03:27.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10113 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9593 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T11:03:27.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28929 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 609257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39905 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22713 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T11:03:27.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40721 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11345 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13745 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14185 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T11:03:27.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_osf.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12737 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20393 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_synproxy.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tproxy.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/x_tables.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T11:03:27.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18169 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10497 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38857 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13873 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T11:03:27.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13289 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19689 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15577 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11209 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T11:03:27.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10465 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8801 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T11:03:27.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7865 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11785 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16041 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8001 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8641 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T11:03:27.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8297 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7633 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8257 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13329 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T11:03:27.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16089 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9505 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9473 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8809 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8553 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15777 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpudp.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11129 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9105 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/ 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13969 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/ 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T11:03:27.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/ 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 379545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10321 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/ 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/ 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109961 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/phonet.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54761 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/phonet/pn_pep.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/ 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 252889 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T11:03:27.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165081 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67241 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/ 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 820137 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rxrpc/rxrpc.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/ 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23537 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43817 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T11:03:27.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20313 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36057 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26145 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33945 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19233 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_cgroup.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31377 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31769 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38489 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7441 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T11:03:27.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33337 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10625 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7249 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21649 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33753 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33609 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq_codel.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35433 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48409 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22073 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T11:03:27.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30273 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10793 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26065 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35601 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27545 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10265 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47849 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32953 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26881 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31425 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T11:03:27.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28897 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/ 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 930369 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/ 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/ 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 335569 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61833 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1514673 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/ 2023-10-06T11:03:27.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 707401 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/ 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10025 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 862385 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/ 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15729 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/ 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84841 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68033 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95985 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17521 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T11:03:27.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/ 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125097 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/x25/x25.ko 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/ 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22657 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_algo.ko 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20705 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102353 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_user.ko 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/ 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/ 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15513 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 638591 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 624324 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7656 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23361 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-06T11:03:27.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9821 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69921 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146449 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 232695 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 210 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72754 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 879 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 252880 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 307898 2023-09-11 02:20 ./lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/ 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/ 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/ 2023-10-06T11:03:27.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/ 2023-10-06T11:03:27.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/changelog.Debian.gz 2023-10-06T11:03:27.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 691 2023-09-11 02:20 ./usr/share/doc/linux-image-6.5.0-rc7-gfef64f93da2b/copyright 2023-10-06T11:03:27.959 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.959 INFO:teuthology.task.kernel:searching for kernel 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.960 DEBUG:teuthology.orchestra.run.smithi046:> egrep '(submenu|menuentry.*6.5.0-rc7-gfef64f93da2b).*{' /boot/grub/grub.cfg 2023-10-06T11:03:27.968 INFO:teuthology.orchestra.run.smithi046.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:27.968 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.5.0-rc7-gfef64f93da2b-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-10-06T11:03:27.969 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-10-06T11:03:27.969 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:27.969 DEBUG:teuthology.orchestra.run.smithi046:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-10-06T11:03:27.971 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi046.front.sepia.ceph.com to complete... 2023-10-06T11:03:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2023-10-06T11:03:28.744 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-06T11:03:28.796 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2023-10-06T11:03:28.886 INFO:teuthology.orchestra.run.smithi132.stderr:done 2023-10-06T11:03:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:29.084 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.5.0-rc7-gfef64f93da2b 2023-10-06T11:03:29.532 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-06T11:03:29.544 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-06T11:03:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:done 2023-10-06T11:04:00.029 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T11:04:00.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:04:00.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:04:02.328 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T11:04:02.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:04:02.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:04:18.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-10-06T11:04:20.859 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-06T11:04:27.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:04:27.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:04:29.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:04:29.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:04:32.922 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-10-06T11:04:44.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:04:44.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:05:27.606 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T11:05:39.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:05:39.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:05:39.870 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-10-06T11:05:40.587 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:05:40.587 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T11:05:40.588 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2023-10-06T11:05:40.652 INFO:teuthology.orchestra.run.smithi132.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T11:05:40.653 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:05:40.653 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:05:40.653 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T11:05:41.654 DEBUG:teuthology.parallel:result is None 2023-10-06T11:05:44.937 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T11:05:59.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:05:59.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:06:00.178 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-10-06T11:06:00.900 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:06:00.900 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T11:06:00.901 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-10-06T11:06:00.957 INFO:teuthology.orchestra.run.smithi046.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T11:06:00.958 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T11:06:00.958 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:06:00.958 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-06T11:06:01.959 DEBUG:teuthology.parallel:result is None 2023-10-06T11:06:01.959 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T11:06:01.966 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T11:06:01.967 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:06:01.974 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:06:01.994 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T11:06:02.002 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T11:06:02.013 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T11:06:02.013 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:06:02.040 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:06:02.071 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T11:06:02.079 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T11:06:02.080 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:06:02.116 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:06:02.215 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:06:02.240 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:06:02.242 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:06:02.262 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:06:02.265 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T11:06:02.273 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T11:06:02.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:06:02.280 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:06:02.315 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T11:06:02.326 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T11:06:02.326 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:06:02.361 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:06:02.381 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:06:02.437 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:06:02.493 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:06:02.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:06:02.583 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:06:02.600 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:06:02.665 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:06:02.665 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:06:02.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2023-10-06T11:06:02.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2023-10-06T11:06:02.914 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T11:06:02.922 INFO:teuthology.task.internal:Starting timer... 2023-10-06T11:06:02.922 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T11:06:02.947 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T11:06:02.962 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2023-10-06T11:06:02.962 DEBUG:teuthology.task.selinux:Excluding smithi132: OS 'ubuntu' does not support SELinux 2023-10-06T11:06:02.963 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T11:06:02.963 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-06T11:06:02.963 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T11:06:02.963 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T11:06:02.973 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T11:06:02.973 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T11:06:02.990 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T11:06:02.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T11:13:48.648 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2023-10-06T11:13:48.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:13:48.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:13:48.758 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-10-06T11:13:48.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-10-06T11:13:49.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:13:49.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:13:49.095 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-10-06T11:13:49.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-06T11:13:49.336 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T11:13:49.347 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T11:13:49.347 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:13:49.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:13:49.355 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:13:49.356 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:13:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T11:13:49.426 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Command line: ntpd -gq 2023-10-06T11:13:49.428 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: proto: precision = 0.082 usec (-23) 2023-10-06T11:13:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:13:49.429 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:13:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen normally on 3 ens1f0 172.21.15.46:123 2023-10-06T11:13:49.430 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen normally on 4 lo [::1]:123 2023-10-06T11:13:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2023-10-06T11:13:49.431 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:49 ntpd[22597]: Listening on routing socket on fd #22 for interface updates 2023-10-06T11:13:49.461 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-06T11:13:49.461 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Command line: ntpd -gq 2023-10-06T11:13:49.462 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: proto: precision = 0.043 usec (-24) 2023-10-06T11:13:49.462 INFO:teuthology.orchestra.run.smithi132.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:13:49.462 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-06T11:13:49.462 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:13:49.463 INFO:teuthology.orchestra.run.smithi132.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen and drop on 0 v6wildcard [::]:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen normally on 2 lo 127.0.0.1:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen normally on 3 enp3s0f1 172.21.15.132:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen normally on 4 lo [::1]:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:773f%5]:123 2023-10-06T11:13:49.464 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:49 ntpd[22695]: Listening on routing socket on fd #22 for interface updates 2023-10-06T11:13:50.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:50.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:50.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:50.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:50.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:50.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:50 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:50.461 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:50 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:50.461 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:50 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:51.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:51 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:51.464 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:51 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:52.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:52.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:52.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:52.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:52.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:52.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:52 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:53.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:53.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:53 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:53.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:53.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:53 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:54.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:54.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:54.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:54 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:54.461 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.461 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:54.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:54.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:54.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:54 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:55.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.427 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:55.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:55 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:55.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:55.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:55 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:56.428 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:56.429 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:56.429 INFO:teuthology.orchestra.run.smithi046.stdout: 6 Oct 11:13:56 ntpd[22597]: ntpd: time slew +0.001398 s 2023-10-06T11:13:56.429 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew +0.001398s 2023-10-06T11:13:56.430 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T11:13:56.430 INFO:teuthology.orchestra.run.smithi046.stderr: 6 Oct 11:13:56 ntpd[22597]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T11:13:56.461 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:56.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:56.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:56.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:56 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:56.572 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:13:56.573 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-10-06T11:13:56.573 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:56.573 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:56.573 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:56.573 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: can't open /var/log/ntpstats/rawstats.20231006: Permission denied 2023-10-06T11:13:57.462 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-06T11:13:57.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: can't open /var/log/ntpstats/peerstats.20231006: Permission denied 2023-10-06T11:13:57.463 INFO:teuthology.orchestra.run.smithi132.stdout: 6 Oct 11:13:57 ntpd[22695]: ntpd: time slew -0.000982 s 2023-10-06T11:13:57.463 INFO:teuthology.orchestra.run.smithi132.stdout:ntpd: time slew -0.000982s 2023-10-06T11:13:57.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-06T11:13:57.463 INFO:teuthology.orchestra.run.smithi132.stderr: 6 Oct 11:13:57 ntpd[22695]: can't open /var/log/ntpstats/loopstats.20231006: Permission denied 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================== 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:57.594 INFO:teuthology.orchestra.run.smithi132.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-06T11:13:57.595 INFO:teuthology.run_tasks:Running task install... 2023-10-06T11:13:57.609 DEBUG:teuthology.task.install:project ceph 2023-10-06T11:13:57.609 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T11:13:57.609 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T11:13:57.609 INFO:teuthology.task.install:Using flavor: default 2023-10-06T11:13:57.615 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T11:13:57.615 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T11:13:57.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2023-10-06T11:13:57.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-key list | grep Ceph 2023-10-06T11:13:57.848 INFO:teuthology.orchestra.run.smithi132.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T11:13:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-06T11:13:57.985 INFO:teuthology.orchestra.run.smithi132.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T11:13:57.985 INFO:teuthology.orchestra.run.smithi132.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T11:13:57.987 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T11:13:57.987 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T11:13:57.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:13:58.006 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-06T11:13:58.007 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2023-10-06T11:13:58.008 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-06T11:13:58.009 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-06T11:13:58.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:13:58.155 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default/ 2023-10-06T11:13:58.155 INFO:teuthology.task.install.deb:Package version is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:13:58.238 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default/ 2023-10-06T11:13:58.239 INFO:teuthology.task.install.deb:Package version is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:13:58.343 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:13:58.343 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T11:13:58.345 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:13:58.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-06T11:13:58.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2023-10-06T11:13:58.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo apt-get update 2023-10-06T11:13:58.650 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:13:58.676 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:13:58.676 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:13:58.676 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-06T11:13:58.689 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:13:58.717 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal InRelease 2023-10-06T11:13:58.738 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T11:13:58.752 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-06T11:13:58.753 INFO:teuthology.orchestra.run.smithi132.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal InRelease 2023-10-06T11:13:58.763 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T11:13:58.786 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main i386 Packages [3,391 B] 2023-10-06T11:13:58.790 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-06T11:13:58.828 INFO:teuthology.orchestra.run.smithi132.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal Release.gpg 2023-10-06T11:13:58.831 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T11:13:58.834 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-06T11:13:58.868 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main i386 Packages [3,391 B] 2023-10-06T11:13:58.917 INFO:teuthology.orchestra.run.smithi132.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-06T11:13:58.930 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-06T11:14:00.091 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 28.4 kB in 1s (20.6 kB/s) 2023-10-06T11:14:00.134 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.4 kB in 1s (20.1 kB/s) 2023-10-06T11:14:01.326 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2023-10-06T11:14:01.365 DEBUG:teuthology.orchestra.run.smithi132:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1274-g68e53a4a-1focal cephadm=17.2.6-1274-g68e53a4a-1focal ceph-mds=17.2.6-1274-g68e53a4a-1focal ceph-mgr=17.2.6-1274-g68e53a4a-1focal ceph-common=17.2.6-1274-g68e53a4a-1focal ceph-fuse=17.2.6-1274-g68e53a4a-1focal ceph-test=17.2.6-1274-g68e53a4a-1focal ceph-volume=17.2.6-1274-g68e53a4a-1focal radosgw=17.2.6-1274-g68e53a4a-1focal python3-rados=17.2.6-1274-g68e53a4a-1focal python3-rgw=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal python3-rbd=17.2.6-1274-g68e53a4a-1focal libcephfs2=17.2.6-1274-g68e53a4a-1focal libcephfs-dev=17.2.6-1274-g68e53a4a-1focal librados2=17.2.6-1274-g68e53a4a-1focal librbd1=17.2.6-1274-g68e53a4a-1focal rbd-fuse=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal cephfs-shell=17.2.6-1274-g68e53a4a-1focal cephfs-top=17.2.6-1274-g68e53a4a-1focal cephfs-mirror=17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:14:01.425 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-06T11:14:01.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1274-g68e53a4a-1focal cephadm=17.2.6-1274-g68e53a4a-1focal ceph-mds=17.2.6-1274-g68e53a4a-1focal ceph-mgr=17.2.6-1274-g68e53a4a-1focal ceph-common=17.2.6-1274-g68e53a4a-1focal ceph-fuse=17.2.6-1274-g68e53a4a-1focal ceph-test=17.2.6-1274-g68e53a4a-1focal ceph-volume=17.2.6-1274-g68e53a4a-1focal radosgw=17.2.6-1274-g68e53a4a-1focal python3-rados=17.2.6-1274-g68e53a4a-1focal python3-rgw=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal python3-rbd=17.2.6-1274-g68e53a4a-1focal libcephfs2=17.2.6-1274-g68e53a4a-1focal libcephfs-dev=17.2.6-1274-g68e53a4a-1focal librados2=17.2.6-1274-g68e53a4a-1focal librbd1=17.2.6-1274-g68e53a4a-1focal rbd-fuse=17.2.6-1274-g68e53a4a-1focal python3-cephfs=17.2.6-1274-g68e53a4a-1focal cephfs-shell=17.2.6-1274-g68e53a4a-1focal cephfs-top=17.2.6-1274-g68e53a4a-1focal cephfs-mirror=17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:14:01.484 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2023-10-06T11:14:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-06T11:14:01.686 INFO:teuthology.orchestra.run.smithi132.stdout:Building dependency tree... 2023-10-06T11:14:01.686 INFO:teuthology.orchestra.run.smithi132.stdout:Reading state information... 2023-10-06T11:14:01.773 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-10-06T11:14:01.774 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-10-06T11:14:01.874 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:14:01.875 INFO:teuthology.orchestra.run.smithi132.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout:The following additional packages will be installed: 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T11:14:01.876 INFO:teuthology.orchestra.run.smithi132.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:14:01.877 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:14:01.878 INFO:teuthology.orchestra.run.smithi132.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout:Suggested packages: 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T11:14:01.879 INFO:teuthology.orchestra.run.smithi132.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T11:14:01.880 INFO:teuthology.orchestra.run.smithi132.stdout:Recommended packages: 2023-10-06T11:14:01.880 INFO:teuthology.orchestra.run.smithi132.stdout: btrfs-tools 2023-10-06T11:14:01.954 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:14:01.955 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:14:01.955 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:14:01.956 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-10-06T11:14:01.956 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-06T11:14:01.957 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:14:01.958 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-06T11:14:01.959 INFO:teuthology.orchestra.run.smithi046.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-06T11:14:01.960 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2023-10-06T11:14:01.960 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2023-10-06T11:14:02.080 INFO:teuthology.orchestra.run.smithi132.stdout:The following NEW packages will be installed: 2023-10-06T11:14:02.080 INFO:teuthology.orchestra.run.smithi132.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T11:14:02.080 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T11:14:02.080 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T11:14:02.081 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T11:14:02.081 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T11:14:02.081 INFO:teuthology.orchestra.run.smithi132.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T11:14:02.081 INFO:teuthology.orchestra.run.smithi132.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T11:14:02.083 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T11:14:02.083 INFO:teuthology.orchestra.run.smithi132.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T11:14:02.083 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:14:02.083 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T11:14:02.084 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T11:14:02.084 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T11:14:02.084 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:14:02.084 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:14:02.084 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:14:02.085 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:14:02.086 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:14:02.086 INFO:teuthology.orchestra.run.smithi132.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T11:14:02.086 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages will be upgraded: 2023-10-06T11:14:02.086 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 librbd1 2023-10-06T11:14:02.163 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-10-06T11:14:02.163 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-06T11:14:02.164 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-06T11:14:02.165 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-06T11:14:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2023-10-06T11:14:02.167 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 librbd1 2023-10-06T11:14:02.254 INFO:teuthology.orchestra.run.smithi046.stdout:2 upgraded, 114 newly installed, 0 to remove and 256 not upgraded. 2023-10-06T11:14:02.255 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 210 MB of archives. 2023-10-06T11:14:02.255 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T11:14:02.255 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T11:14:02.262 INFO:teuthology.orchestra.run.smithi132.stdout:2 upgraded, 114 newly installed, 0 to remove and 256 not upgraded. 2023-10-06T11:14:02.262 INFO:teuthology.orchestra.run.smithi132.stdout:Need to get 210 MB of archives. 2023-10-06T11:14:02.262 INFO:teuthology.orchestra.run.smithi132.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-06T11:14:02.262 INFO:teuthology.orchestra.run.smithi132.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1274-g68e53a4a-1focal [3,501 kB] 2023-10-06T11:14:02.274 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T11:14:02.282 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T11:14:02.299 INFO:teuthology.orchestra.run.smithi132.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-06T11:14:02.308 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T11:14:02.336 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T11:14:02.338 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T11:14:02.347 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1274-g68e53a4a-1focal [3,501 kB] 2023-10-06T11:14:02.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T11:14:02.351 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T11:14:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T11:14:02.369 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T11:14:02.374 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T11:14:02.384 INFO:teuthology.orchestra.run.smithi132.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-06T11:14:02.405 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T11:14:02.408 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T11:14:02.416 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T11:14:02.419 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T11:14:02.419 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-06T11:14:02.420 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T11:14:02.422 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T11:14:02.422 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T11:14:02.423 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T11:14:02.426 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T11:14:02.426 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T11:14:02.440 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T11:14:02.441 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T11:14:02.442 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T11:14:02.445 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T11:14:02.456 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T11:14:02.457 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T11:14:02.458 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T11:14:02.459 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T11:14:02.461 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T11:14:02.463 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T11:14:02.473 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T11:14:02.480 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T11:14:02.480 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T11:14:02.481 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T11:14:02.491 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T11:14:02.493 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T11:14:02.495 INFO:teuthology.orchestra.run.smithi132.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1274-g68e53a4a-1focal [3,551 kB] 2023-10-06T11:14:02.497 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T11:14:02.499 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T11:14:02.500 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T11:14:02.501 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T11:14:02.508 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T11:14:02.510 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T11:14:02.511 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T11:14:02.512 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T11:14:02.544 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-06T11:14:02.615 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T11:14:02.616 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1274-g68e53a4a-1focal [760 kB] 2023-10-06T11:14:02.637 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T11:14:02.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T11:14:02.639 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1274-g68e53a4a-1focal [413 kB] 2023-10-06T11:14:02.655 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T11:14:02.655 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T11:14:02.656 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T11:14:02.656 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T11:14:02.657 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T11:14:02.657 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1274-g68e53a4a-1focal [22.5 kB] 2023-10-06T11:14:02.658 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T11:14:02.658 INFO:teuthology.orchestra.run.smithi132.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1274-g68e53a4a-1focal [200 kB] 2023-10-06T11:14:02.658 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T11:14:02.659 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T11:14:02.664 INFO:teuthology.orchestra.run.smithi132.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1274-g68e53a4a-1focal [47.1 kB] 2023-10-06T11:14:02.666 INFO:teuthology.orchestra.run.smithi132.stdout:Get:12 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1274-g68e53a4a-1focal [386 kB] 2023-10-06T11:14:02.671 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T11:14:02.677 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T11:14:02.678 INFO:teuthology.orchestra.run.smithi132.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1274-g68e53a4a-1focal [5,567 kB] 2023-10-06T11:14:02.682 INFO:teuthology.orchestra.run.smithi132.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-06T11:14:02.683 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T11:14:02.686 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T11:14:02.689 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T11:14:02.690 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T11:14:02.691 INFO:teuthology.orchestra.run.smithi132.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-06T11:14:02.695 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T11:14:02.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T11:14:02.698 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T11:14:02.719 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T11:14:02.722 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T11:14:02.736 INFO:teuthology.orchestra.run.smithi132.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-06T11:14:02.747 INFO:teuthology.orchestra.run.smithi132.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-06T11:14:02.774 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T11:14:02.776 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T11:14:02.776 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T11:14:02.776 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T11:14:02.777 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T11:14:02.778 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T11:14:02.793 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T11:14:02.793 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T11:14:02.794 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T11:14:02.794 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T11:14:02.808 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T11:14:02.811 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T11:14:02.813 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T11:14:02.817 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T11:14:02.818 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T11:14:02.830 INFO:teuthology.orchestra.run.smithi132.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-06T11:14:02.835 INFO:teuthology.orchestra.run.smithi132.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-06T11:14:02.858 INFO:teuthology.orchestra.run.smithi132.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-06T11:14:02.882 INFO:teuthology.orchestra.run.smithi132.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1274-g68e53a4a-1focal [116 kB] 2023-10-06T11:14:02.885 INFO:teuthology.orchestra.run.smithi132.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1274-g68e53a4a-1focal [479 kB] 2023-10-06T11:14:02.899 INFO:teuthology.orchestra.run.smithi132.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1274-g68e53a4a-1focal [27.7 MB] 2023-10-06T11:14:03.152 INFO:teuthology.orchestra.run.smithi132.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-06T11:14:03.585 INFO:teuthology.orchestra.run.smithi132.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-06T11:14:03.594 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1274-g68e53a4a-1focal [3,551 kB] 2023-10-06T11:14:03.768 INFO:teuthology.orchestra.run.smithi132.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-06T11:14:03.798 INFO:teuthology.orchestra.run.smithi132.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-06T11:14:03.804 INFO:teuthology.orchestra.run.smithi132.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-06T11:14:03.829 INFO:teuthology.orchestra.run.smithi132.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-06T11:14:03.834 INFO:teuthology.orchestra.run.smithi132.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-06T11:14:03.836 INFO:teuthology.orchestra.run.smithi132.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-06T11:14:03.849 INFO:teuthology.orchestra.run.smithi132.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-06T11:14:03.852 INFO:teuthology.orchestra.run.smithi132.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-06T11:14:03.853 INFO:teuthology.orchestra.run.smithi132.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-06T11:14:03.929 INFO:teuthology.orchestra.run.smithi132.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1274-g68e53a4a-1focal [6,449 kB] 2023-10-06T11:14:04.018 INFO:teuthology.orchestra.run.smithi132.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-06T11:14:04.146 INFO:teuthology.orchestra.run.smithi132.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1274-g68e53a4a-1focal [221 kB] 2023-10-06T11:14:04.150 INFO:teuthology.orchestra.run.smithi132.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1274-g68e53a4a-1focal [152 kB] 2023-10-06T11:14:04.154 INFO:teuthology.orchestra.run.smithi132.stdout:Get:39 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1274-g68e53a4a-1focal [1,590 kB] 2023-10-06T11:14:04.184 INFO:teuthology.orchestra.run.smithi132.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-06T11:14:04.214 INFO:teuthology.orchestra.run.smithi132.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1274-g68e53a4a-1focal [7,846 kB] 2023-10-06T11:14:04.326 INFO:teuthology.orchestra.run.smithi132.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-06T11:14:04.343 INFO:teuthology.orchestra.run.smithi132.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-06T11:14:04.355 INFO:teuthology.orchestra.run.smithi132.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-06T11:14:04.432 INFO:teuthology.orchestra.run.smithi132.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-06T11:14:04.453 INFO:teuthology.orchestra.run.smithi132.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-06T11:14:04.493 INFO:teuthology.orchestra.run.smithi132.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-06T11:14:04.509 INFO:teuthology.orchestra.run.smithi132.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1274-g68e53a4a-1focal [28.7 MB] 2023-10-06T11:14:04.524 INFO:teuthology.orchestra.run.smithi132.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-06T11:14:04.530 INFO:teuthology.orchestra.run.smithi132.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-06T11:14:04.623 INFO:teuthology.orchestra.run.smithi132.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-06T11:14:04.765 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1274-g68e53a4a-1focal [760 kB] 2023-10-06T11:14:04.787 INFO:teuthology.orchestra.run.smithi132.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-06T11:14:04.973 INFO:teuthology.orchestra.run.smithi132.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-06T11:14:04.973 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1274-g68e53a4a-1focal [413 kB] 2023-10-06T11:14:05.023 INFO:teuthology.orchestra.run.smithi132.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-06T11:14:05.111 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1274-g68e53a4a-1focal [22.5 kB] 2023-10-06T11:14:05.154 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1274-g68e53a4a-1focal [200 kB] 2023-10-06T11:14:05.155 INFO:teuthology.orchestra.run.smithi132.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-06T11:14:05.181 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1274-g68e53a4a-1focal [47.1 kB] 2023-10-06T11:14:05.246 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1274-g68e53a4a-1focal [386 kB] 2023-10-06T11:14:05.260 INFO:teuthology.orchestra.run.smithi132.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-06T11:14:05.290 INFO:teuthology.orchestra.run.smithi132.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-06T11:14:05.293 INFO:teuthology.orchestra.run.smithi132.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-06T11:14:05.317 INFO:teuthology.orchestra.run.smithi132.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-06T11:14:05.323 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1274-g68e53a4a-1focal [5,567 kB] 2023-10-06T11:14:05.338 INFO:teuthology.orchestra.run.smithi132.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-06T11:14:05.355 INFO:teuthology.orchestra.run.smithi132.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-06T11:14:05.358 INFO:teuthology.orchestra.run.smithi132.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-06T11:14:05.526 INFO:teuthology.orchestra.run.smithi132.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-06T11:14:05.565 INFO:teuthology.orchestra.run.smithi132.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1274-g68e53a4a-1focal [3,928 B] 2023-10-06T11:14:05.566 INFO:teuthology.orchestra.run.smithi132.stdout:Get:65 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1274-g68e53a4a-1focal [870 kB] 2023-10-06T11:14:05.579 INFO:teuthology.orchestra.run.smithi132.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1274-g68e53a4a-1focal [2,406 kB] 2023-10-06T11:14:05.667 INFO:teuthology.orchestra.run.smithi132.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1274-g68e53a4a-1focal [79.1 kB] 2023-10-06T11:14:05.668 INFO:teuthology.orchestra.run.smithi132.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1274-g68e53a4a-1focal [117 kB] 2023-10-06T11:14:05.672 INFO:teuthology.orchestra.run.smithi132.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1274-g68e53a4a-1focal [1,760 kB] 2023-10-06T11:14:05.738 INFO:teuthology.orchestra.run.smithi132.stdout:Get:70 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1274-g68e53a4a-1focal [7,886 kB] 2023-10-06T11:14:06.031 INFO:teuthology.orchestra.run.smithi132.stdout:Get:71 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1274-g68e53a4a-1focal [4,040 B] 2023-10-06T11:14:06.032 INFO:teuthology.orchestra.run.smithi132.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1274-g68e53a4a-1focal [56.3 MB] 2023-10-06T11:14:06.501 INFO:teuthology.orchestra.run.smithi132.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-06T11:14:06.527 INFO:teuthology.orchestra.run.smithi132.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-06T11:14:06.529 INFO:teuthology.orchestra.run.smithi132.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-06T11:14:06.600 INFO:teuthology.orchestra.run.smithi132.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-06T11:14:06.600 INFO:teuthology.orchestra.run.smithi132.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-06T11:14:06.601 INFO:teuthology.orchestra.run.smithi132.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-06T11:14:06.602 INFO:teuthology.orchestra.run.smithi132.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-06T11:14:06.603 INFO:teuthology.orchestra.run.smithi132.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-06T11:14:06.603 INFO:teuthology.orchestra.run.smithi132.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-06T11:14:06.604 INFO:teuthology.orchestra.run.smithi132.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-06T11:14:06.770 INFO:teuthology.orchestra.run.smithi132.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-06T11:14:06.939 INFO:teuthology.orchestra.run.smithi132.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-06T11:14:07.021 INFO:teuthology.orchestra.run.smithi132.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-06T11:14:07.119 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1274-g68e53a4a-1focal [116 kB] 2023-10-06T11:14:07.203 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1274-g68e53a4a-1focal [479 kB] 2023-10-06T11:14:07.256 INFO:teuthology.orchestra.run.smithi132.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-06T11:14:07.300 INFO:teuthology.orchestra.run.smithi132.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-06T11:14:07.329 INFO:teuthology.orchestra.run.smithi132.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-06T11:14:07.339 INFO:teuthology.orchestra.run.smithi132.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-06T11:14:07.347 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1274-g68e53a4a-1focal [27.7 MB] 2023-10-06T11:14:07.389 INFO:teuthology.orchestra.run.smithi132.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-06T11:14:07.417 INFO:teuthology.orchestra.run.smithi132.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-06T11:14:07.419 INFO:teuthology.orchestra.run.smithi132.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-06T11:14:07.540 INFO:teuthology.orchestra.run.smithi132.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-06T11:14:07.713 INFO:teuthology.orchestra.run.smithi132.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-06T11:14:08.112 INFO:teuthology.orchestra.run.smithi132.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1274-g68e53a4a-1focal [229 kB] 2023-10-06T11:14:08.115 INFO:teuthology.orchestra.run.smithi132.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1274-g68e53a4a-1focal [239 kB] 2023-10-06T11:14:08.117 INFO:teuthology.orchestra.run.smithi132.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1274-g68e53a4a-1focal [22.0 kB] 2023-10-06T11:14:08.117 INFO:teuthology.orchestra.run.smithi132.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1274-g68e53a4a-1focal [12.8 kB] 2023-10-06T11:14:08.118 INFO:teuthology.orchestra.run.smithi132.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1274-g68e53a4a-1focal [21.9 kB] 2023-10-06T11:14:08.119 INFO:teuthology.orchestra.run.smithi132.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1274-g68e53a4a-1focal [17.0 MB] 2023-10-06T11:14:08.543 INFO:teuthology.orchestra.run.smithi132.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-06T11:14:08.546 INFO:teuthology.orchestra.run.smithi132.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-06T11:14:08.547 INFO:teuthology.orchestra.run.smithi132.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-06T11:14:08.549 INFO:teuthology.orchestra.run.smithi132.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-06T11:14:08.550 INFO:teuthology.orchestra.run.smithi132.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-06T11:14:08.553 INFO:teuthology.orchestra.run.smithi132.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-06T11:14:08.568 INFO:teuthology.orchestra.run.smithi132.stdout:Get:107 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-06T11:14:08.568 INFO:teuthology.orchestra.run.smithi132.stdout:Get:108 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-06T11:14:08.569 INFO:teuthology.orchestra.run.smithi132.stdout:Get:109 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-06T11:14:08.624 INFO:teuthology.orchestra.run.smithi132.stdout:Get:110 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-06T11:14:08.766 INFO:teuthology.orchestra.run.smithi132.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1274-g68e53a4a-1focal [79.4 kB] 2023-10-06T11:14:08.793 INFO:teuthology.orchestra.run.smithi132.stdout:Get:112 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-06T11:14:09.079 INFO:teuthology.orchestra.run.smithi132.stdout:Get:113 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-06T11:14:09.099 INFO:teuthology.orchestra.run.smithi132.stdout:Get:114 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-06T11:14:09.251 INFO:teuthology.orchestra.run.smithi132.stdout:Get:115 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-06T11:14:09.269 INFO:teuthology.orchestra.run.smithi132.stdout:Get:116 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-06T11:14:10.373 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 210 MB in 7s (28.4 MB/s) 2023-10-06T11:14:10.532 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T11:14:10.588 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1274-g68e53a4a-1focal [6,449 kB] 2023-10-06T11:14:10.599 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107762 files and directories currently installed.) 2023-10-06T11:14:10.603 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T11:14:10.645 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T11:14:11.015 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T11:14:11.031 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1274-g68e53a4a-1focal [221 kB] 2023-10-06T11:14:11.033 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T11:14:11.045 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1274-g68e53a4a-1focal [152 kB] 2023-10-06T11:14:11.052 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1274-g68e53a4a-1focal [1,590 kB] 2023-10-06T11:14:11.073 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:14:11.169 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1274-g68e53a4a-1focal [7,846 kB] 2023-10-06T11:14:11.434 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T11:14:11.453 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T11:14:11.493 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:14:11.688 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1274-g68e53a4a-1focal [28.7 MB] 2023-10-06T11:14:11.888 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T11:14:11.906 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T11:14:12.021 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:14:12.418 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../004-librbd1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:12.542 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librbd1 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:14:13.306 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../005-librados2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:13.393 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1274-g68e53a4a-1focal [3,928 B] 2023-10-06T11:14:13.394 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1274-g68e53a4a-1focal [870 kB] 2023-10-06T11:14:13.429 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librados2 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:14:13.436 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1274-g68e53a4a-1focal [2,406 kB] 2023-10-06T11:14:13.565 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1274-g68e53a4a-1focal [79.1 kB] 2023-10-06T11:14:13.568 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1274-g68e53a4a-1focal [117 kB] 2023-10-06T11:14:13.570 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1274-g68e53a4a-1focal [1,760 kB] 2023-10-06T11:14:13.668 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1274-g68e53a4a-1focal [7,886 kB] 2023-10-06T11:14:14.085 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1274-g68e53a4a-1focal [4,040 B] 2023-10-06T11:14:14.085 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1274-g68e53a4a-1focal [56.3 MB] 2023-10-06T11:14:14.150 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T11:14:14.169 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:14.475 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:14.870 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rados. 2023-10-06T11:14:14.889 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:14.928 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:15.347 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T11:14:15.366 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:15.406 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:15.709 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T11:14:15.727 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:15.767 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:16.170 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T11:14:16.189 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:16.228 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:16.625 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T11:14:16.642 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T11:14:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1274-g68e53a4a-1focal [229 kB] 2023-10-06T11:14:16.683 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1274-g68e53a4a-1focal [239 kB] 2023-10-06T11:14:16.686 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1274-g68e53a4a-1focal [22.0 kB] 2023-10-06T11:14:16.686 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1274-g68e53a4a-1focal [12.8 kB] 2023-10-06T11:14:16.687 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1274-g68e53a4a-1focal [21.9 kB] 2023-10-06T11:14:16.687 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1274-g68e53a4a-1focal [17.0 MB] 2023-10-06T11:14:16.698 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T11:14:16.976 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T11:14:16.994 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:17.034 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:17.317 INFO:teuthology.orchestra.run.smithi046.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1274-g68e53a4a-1focal [79.4 kB] 2023-10-06T11:14:17.453 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T11:14:17.472 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T11:14:17.536 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:14:17.956 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T11:14:17.974 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T11:14:18.014 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:14:18.272 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 210 MB in 15s (13.9 MB/s) 2023-10-06T11:14:18.401 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T11:14:18.420 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T11:14:18.459 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:14:18.460 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-06T11:14:18.533 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107762 files and directories currently installed.) 2023-10-06T11:14:18.536 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-06T11:14:18.614 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-06T11:14:18.895 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T11:14:18.914 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T11:14:18.953 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:14:18.959 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-06T11:14:18.978 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-06T11:14:19.042 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:14:19.348 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T11:14:19.367 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T11:14:19.406 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:14:19.439 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2023-10-06T11:14:19.452 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-06T11:14:19.503 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:14:19.943 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T11:14:19.955 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:19.981 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-06T11:14:19.994 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-06T11:14:20.015 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:14:20.447 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-librbd1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:20.568 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:14:20.580 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T11:14:20.592 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:20.655 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:21.074 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T11:14:21.092 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:21.141 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:21.412 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-librados2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:21.569 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T11:14:21.573 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (17.2.6-1274-g68e53a4a-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-06T11:14:21.582 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T11:14:21.619 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:14:22.005 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package librgw2. 2023-10-06T11:14:22.017 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../022-librgw2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:22.055 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:22.369 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2023-10-06T11:14:22.388 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:22.427 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:22.796 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2023-10-06T11:14:22.815 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:22.854 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:22.908 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T11:14:22.926 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:22.974 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:23.274 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-06T11:14:23.293 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:23.348 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:23.352 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liboath0. 2023-10-06T11:14:23.364 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T11:14:23.427 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T11:14:23.644 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2023-10-06T11:14:23.661 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:23.702 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:23.801 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T11:14:23.818 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:23.863 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:24.088 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-06T11:14:24.107 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:24.146 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:24.384 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-common. 2023-10-06T11:14:24.402 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:24.441 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:24.524 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2023-10-06T11:14:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-06T11:14:24.582 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-06T11:14:24.877 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2023-10-06T11:14:24.896 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:24.934 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:25.346 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-06T11:14:25.365 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-06T11:14:25.404 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:14:25.799 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-06T11:14:25.818 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-06T11:14:25.857 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:14:26.660 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-06T11:14:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-06T11:14:26.726 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:14:27.130 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-base. 2023-10-06T11:14:27.149 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:27.163 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-06T11:14:27.181 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-06T11:14:27.211 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:27.220 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:14:27.616 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-06T11:14:27.635 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-06T11:14:27.674 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:14:28.211 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-06T11:14:28.230 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:28.268 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T11:14:28.287 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T11:14:28.287 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:28.326 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:14:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T11:14:28.790 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T11:14:28.813 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-06T11:14:28.829 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T11:14:28.830 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:28.872 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:29.166 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T11:14:29.185 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:29.233 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:29.283 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-06T11:14:29.302 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-06T11:14:29.341 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:14:29.695 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T11:14:29.708 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T11:14:29.771 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:14:29.795 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-06T11:14:29.814 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-06T11:14:29.853 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:14:30.190 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T11:14:30.209 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T11:14:30.248 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:14:30.256 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2023-10-06T11:14:30.275 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-librgw2_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:30.314 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:30.660 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T11:14:30.679 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T11:14:30.719 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T11:14:31.097 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T11:14:31.115 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T11:14:31.150 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2023-10-06T11:14:31.154 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:14:31.168 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:31.208 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:31.542 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T11:14:31.561 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T11:14:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T11:14:31.961 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T11:14:31.979 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T11:14:32.044 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:14:32.270 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0. 2023-10-06T11:14:32.288 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-06T11:14:32.352 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-06T11:14:32.388 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-mako. 2023-10-06T11:14:32.407 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T11:14:32.471 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:14:32.680 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2023-10-06T11:14:32.700 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:32.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:32.858 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T11:14:32.871 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T11:14:32.899 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:14:33.253 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T11:14:33.272 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T11:14:33.283 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2023-10-06T11:14:33.302 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:33.311 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:14:33.341 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:33.681 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-webob. 2023-10-06T11:14:33.699 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T11:14:33.738 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T11:14:34.125 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T11:14:34.144 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T11:14:34.183 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:14:34.503 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T11:14:34.521 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T11:14:34.561 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T11:14:35.623 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T11:14:35.641 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T11:14:35.687 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:14:36.084 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T11:14:36.104 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T11:14:36.139 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2023-10-06T11:14:36.142 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T11:14:36.156 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:36.217 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:36.512 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-paste. 2023-10-06T11:14:36.530 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T11:14:36.570 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:14:36.950 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T11:14:36.967 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T11:14:37.031 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:14:37.243 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cryptography. 2023-10-06T11:14:37.261 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-06T11:14:37.301 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:14:37.376 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T11:14:37.395 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T11:14:37.442 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:14:37.755 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2023-10-06T11:14:37.774 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-06T11:14:37.804 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T11:14:37.822 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T11:14:37.837 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-06T11:14:37.873 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T11:14:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-06T11:14:38.176 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:38.240 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:38.248 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T11:14:38.267 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T11:14:38.306 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:14:38.720 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-06T11:14:38.737 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T11:14:38.739 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-06T11:14:38.755 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T11:14:38.778 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:14:38.794 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:14:39.114 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-06T11:14:39.133 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-06T11:14:39.172 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:14:39.190 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T11:14:39.210 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:39.247 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:39.568 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2023-10-06T11:14:39.587 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-06T11:14:39.625 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-06T11:14:39.668 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T11:14:39.687 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:39.750 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:39.987 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jwt. 2023-10-06T11:14:40.006 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-06T11:14:40.037 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:14:40.262 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T11:14:40.281 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:40.320 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:40.390 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2023-10-06T11:14:40.409 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-06T11:14:40.448 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-06T11:14:40.817 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-06T11:14:40.836 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-06T11:14:40.875 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:14:41.204 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2023-10-06T11:14:41.222 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-06T11:14:41.261 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:14:41.457 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T11:14:41.474 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:41.515 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:41.623 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-06T11:14:41.642 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-06T11:14:41.681 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:14:42.051 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-06T11:14:42.069 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-06T11:14:42.109 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:14:42.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2023-10-06T11:14:42.474 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-06T11:14:42.511 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-06T11:14:42.881 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-06T11:14:42.901 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-06T11:14:42.939 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:14:43.300 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bs4. 2023-10-06T11:14:43.320 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-06T11:14:43.358 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-06T11:14:43.804 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2023-10-06T11:14:43.823 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-06T11:14:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:14:43.977 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph. 2023-10-06T11:14:43.995 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../055-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:44.035 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:44.256 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2023-10-06T11:14:44.275 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-06T11:14:44.314 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-06T11:14:44.421 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T11:14:44.440 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:44.487 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:44.676 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2023-10-06T11:14:44.694 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-06T11:14:44.733 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:14:44.957 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T11:14:44.974 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:45.015 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:45.129 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-06T11:14:45.148 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-06T11:14:45.187 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:14:45.565 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-06T11:14:45.585 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-06T11:14:45.620 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package cephadm. 2023-10-06T11:14:45.622 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:14:45.640 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../058-cephadm_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:45.932 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:46.009 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2023-10-06T11:14:46.029 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-06T11:14:46.067 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-06T11:14:46.323 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T11:14:46.341 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T11:14:46.414 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T11:14:46.470 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2023-10-06T11:14:46.490 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-06T11:14:46.552 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:14:46.810 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T11:14:46.828 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:46.867 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:47.000 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-06T11:14:47.018 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-06T11:14:47.057 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:14:47.254 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T11:14:47.273 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T11:14:47.312 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T11:14:47.461 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-06T11:14:47.479 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:47.519 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:47.606 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T11:14:47.625 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T11:14:47.656 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:14:47.930 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2023-10-06T11:14:47.950 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:47.959 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T11:14:47.978 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T11:14:47.988 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:48.016 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:14:48.378 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-routes. 2023-10-06T11:14:48.396 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T11:14:48.436 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T11:14:48.583 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2023-10-06T11:14:48.603 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:48.666 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:48.797 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T11:14:48.816 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:48.857 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:49.452 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T11:14:49.470 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T11:14:49.517 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:14:49.803 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2023-10-06T11:14:49.822 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:49.861 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:49.862 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T11:14:49.882 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T11:14:49.911 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:14:50.298 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T11:14:50.318 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T11:14:50.372 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T11:14:51.840 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T11:14:51.858 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T11:14:51.898 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:14:52.398 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2023-10-06T11:14:52.418 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-ceph_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:52.455 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:52.485 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T11:14:52.504 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T11:14:52.584 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T11:14:52.833 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2023-10-06T11:14:52.853 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:52.891 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:53.005 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T11:14:53.024 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T11:14:53.071 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:14:53.520 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2023-10-06T11:14:53.539 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:53.577 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:53.888 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T11:14:53.907 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:53.946 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:54.181 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2023-10-06T11:14:54.200 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-cephadm_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:14:54.258 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:54.617 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2023-10-06T11:14:54.636 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-06T11:14:54.708 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-06T11:14:55.061 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-06T11:14:55.079 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:55.144 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:55.205 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T11:14:55.225 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T11:14:55.288 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T11:14:55.547 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-06T11:14:55.566 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-06T11:14:55.605 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-06T11:14:55.808 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T11:14:55.827 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T11:14:55.866 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:14:55.882 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-underscore. 2023-10-06T11:14:55.902 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-06T11:14:55.973 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:14:56.253 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T11:14:56.265 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T11:14:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-06T11:14:56.287 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-06T11:14:56.310 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:14:56.351 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:14:56.688 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T11:14:56.695 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2023-10-06T11:14:56.701 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T11:14:56.714 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-06T11:14:56.746 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T11:14:56.778 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-06T11:14:57.125 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T11:14:57.138 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T11:14:57.149 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-06T11:14:57.167 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:14:57.183 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T11:14:57.439 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:14:57.569 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T11:14:57.582 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T11:14:57.627 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:14:57.973 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T11:14:57.991 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T11:14:58.034 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-decorator. 2023-10-06T11:14:58.047 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-06T11:14:58.055 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:14:58.092 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:14:58.445 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-06T11:14:58.458 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-06T11:14:58.476 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T11:14:58.494 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T11:14:58.495 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:14:58.534 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:14:58.864 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-scipy. 2023-10-06T11:14:58.877 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-06T11:14:58.914 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-06T11:14:59.018 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T11:14:59.038 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T11:14:59.076 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:14:59.471 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T11:14:59.490 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T11:14:59.528 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:15:00.243 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T11:15:00.262 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:00.300 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:00.383 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-06T11:15:00.403 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-06T11:15:00.441 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:00.712 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T11:15:00.732 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T11:15:00.803 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:15:01.036 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2023-10-06T11:15:01.056 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-06T11:15:01.102 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-06T11:15:01.249 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T11:15:01.269 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:15:01.307 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:01.532 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2023-10-06T11:15:01.551 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-06T11:15:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:01.635 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package jq. 2023-10-06T11:15:01.654 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:15:01.693 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:02.005 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package socat. 2023-10-06T11:15:02.024 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T11:15:02.062 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T11:15:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-06T11:15:02.434 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:02.466 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T11:15:02.473 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:02.486 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T11:15:02.524 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:15:02.962 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-test. 2023-10-06T11:15:02.982 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:03.028 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:03.759 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2023-10-06T11:15:03.778 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-06T11:15:03.847 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-06T11:15:04.193 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-06T11:15:04.212 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-06T11:15:04.276 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:15:04.629 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-06T11:15:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-06T11:15:04.703 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:15:05.099 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2023-10-06T11:15:05.118 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-06T11:15:05.156 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-06T11:15:05.535 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2023-10-06T11:15:05.555 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-06T11:15:05.593 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-06T11:15:06.004 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-blinker. 2023-10-06T11:15:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-06T11:15:06.062 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:15:06.440 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-06T11:15:06.460 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-06T11:15:06.498 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:15:06.911 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-06T11:15:06.931 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-06T11:15:06.968 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:15:07.436 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2023-10-06T11:15:07.455 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-06T11:15:07.494 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:15:07.691 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T11:15:07.709 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:07.748 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:07.898 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-06T11:15:07.918 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-06T11:15:07.956 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:15:08.246 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T11:15:08.265 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:08.303 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:08.635 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-06T11:15:08.649 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:08.693 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:08.698 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T11:15:08.718 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T11:15:08.756 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T11:15:09.072 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-06T11:15:09.092 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-06T11:15:09.092 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T11:15:09.111 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:09.129 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:15:09.150 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:09.453 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T11:15:09.473 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:09.519 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:09.550 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-06T11:15:09.570 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:15:09.616 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:09.830 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T11:15:09.850 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:15:09.888 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:09.953 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2023-10-06T11:15:09.973 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:15:10.011 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:10.339 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2023-10-06T11:15:10.359 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-06T11:15:10.396 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-06T11:15:10.443 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T11:15:10.463 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:10.501 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:10.750 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2023-10-06T11:15:10.770 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-06T11:15:10.808 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:15:10.997 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T11:15:11.015 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T11:15:11.087 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:15:11.238 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2023-10-06T11:15:11.255 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:11.295 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:11.493 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T11:15:11.512 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:15:11.550 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:13.562 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T11:15:13.582 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T11:15:13.619 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T11:15:13.989 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T11:15:14.007 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T11:15:14.047 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:15:14.413 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-attr. 2023-10-06T11:15:14.433 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T11:15:14.470 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T11:15:14.857 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T11:15:14.876 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T11:15:14.915 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:15:15.285 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T11:15:15.306 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T11:15:15.376 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T11:15:15.774 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T11:15:15.793 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T11:15:15.831 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:15:16.109 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2023-10-06T11:15:16.129 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:16.167 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:16.327 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T11:15:16.347 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T11:15:16.384 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T11:15:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-06T11:15:16.699 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:16.738 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:17.021 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T11:15:17.041 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T11:15:17.078 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T11:15:17.117 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2023-10-06T11:15:17.135 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-06T11:15:17.175 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-06T11:15:17.461 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2023-10-06T11:15:17.465 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T11:15:17.480 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:17.487 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T11:15:17.535 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:17.540 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T11:15:17.813 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2023-10-06T11:15:17.834 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1274-g68e53a4a-1focal_all.deb ... 2023-10-06T11:15:17.871 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:17.919 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T11:15:17.939 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T11:15:17.976 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:15:18.191 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-06T11:15:18.211 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:15:18.248 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:18.379 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-py. 2023-10-06T11:15:18.398 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T11:15:18.437 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:15:18.824 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T11:15:18.843 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T11:15:18.879 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-06T11:15:18.881 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:15:18.899 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:18.937 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:19.252 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T11:15:19.271 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T11:15:19.309 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T11:15:19.382 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2023-10-06T11:15:19.402 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-06T11:15:19.475 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:15:19.705 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T11:15:19.719 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T11:15:19.762 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:15:19.861 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-06T11:15:19.875 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-06T11:15:19.919 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:20.100 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T11:15:20.113 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T11:15:20.157 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:15:20.982 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package radosgw. 2023-10-06T11:15:21.002 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../114-radosgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:21.072 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:22.162 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-06T11:15:22.182 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-06T11:15:22.244 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-06T11:15:22.656 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-06T11:15:22.676 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-06T11:15:22.714 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:15:22.793 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T11:15:22.813 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:22.850 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:23.146 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-attr. 2023-10-06T11:15:23.160 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-06T11:15:23.204 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-06T11:15:23.249 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T11:15:23.599 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webencodings. 2023-10-06T11:15:23.619 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-06T11:15:23.649 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:15:23.651 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:15:23.810 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:15:23.936 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:15:24.010 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-html5lib. 2023-10-06T11:15:24.032 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-06T11:15:24.076 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-06T11:15:24.171 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:24.455 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-06T11:15:24.475 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-06T11:15:24.523 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:15:24.930 INFO:teuthology.orchestra.run.smithi132.stdout:Adding system user cephadm....done 2023-10-06T11:15:25.045 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-packaging. 2023-10-06T11:15:25.065 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-06T11:15:25.121 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:25.139 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-06T11:15:25.247 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:15:25.473 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2023-10-06T11:15:25.492 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-06T11:15:25.522 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-06T11:15:25.699 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T11:15:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2023-10-06T11:15:25.912 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-06T11:15:25.969 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-06T11:15:26.067 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:26.193 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T11:15:26.304 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2023-10-06T11:15:26.319 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:15:26.324 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-06T11:15:26.361 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:15:26.712 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T11:15:26.748 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2023-10-06T11:15:26.768 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-06T11:15:26.847 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:15:27.248 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:15:27.259 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-06T11:15:27.279 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-06T11:15:27.317 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:15:27.615 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T11:15:27.704 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2023-10-06T11:15:27.724 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-06T11:15:27.769 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-06T11:15:28.000 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T11:15:28.376 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:15:28.540 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2023-10-06T11:15:28.560 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-06T11:15:28.598 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:15:28.751 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:15:28.951 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-06T11:15:28.972 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-06T11:15:29.017 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:15:29.145 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:29.521 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:29.646 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T11:15:29.882 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2023-10-06T11:15:29.903 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-radosgw_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:29.931 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:29.998 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:15:30.357 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:15:30.708 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:15:30.834 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:15:31.294 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T11:15:31.611 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2023-10-06T11:15:31.631 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1274-g68e53a4a-1focal_amd64.deb ... 2023-10-06T11:15:31.653 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:15:31.670 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:31.996 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:15:32.049 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-06T11:15:32.397 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:15:32.569 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-06T11:15:32.720 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-06T11:15:32.748 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T11:15:32.846 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-06T11:15:33.072 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:33.166 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T11:15:33.517 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T11:15:33.832 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2023-10-06T11:15:33.877 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:15:34.002 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:15:34.047 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:34.145 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:15:34.173 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-06T11:15:34.521 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:15:34.600 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-06T11:15:34.953 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-06T11:15:34.998 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:15:35.077 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-06T11:15:35.245 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-06T11:15:35.358 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:15:35.655 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-06T11:15:35.784 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:15:36.149 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-06T11:15:36.219 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T11:15:36.345 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:36.508 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-06T11:15:36.771 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:36.884 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-06T11:15:36.981 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T11:15:37.235 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-06T11:15:37.407 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:15:37.533 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:15:37.603 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-06T11:15:37.709 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T11:15:37.988 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:38.061 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:15:38.339 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:38.445 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:38.465 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-06T11:15:38.571 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:15:38.697 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:15:38.849 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-06T11:15:39.165 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:15:39.225 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-06T11:15:39.558 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T11:15:39.576 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-06T11:15:39.719 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-06T11:15:39.963 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:15:40.093 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:15:40.220 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-06T11:15:40.436 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T11:15:40.621 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-06T11:15:40.804 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:15:40.956 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-06T11:15:41.173 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:15:41.399 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-06T11:15:41.576 INFO:teuthology.orchestra.run.smithi132.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T11:15:41.774 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-06T11:15:41.933 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:15:42.076 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:15:42.192 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-06T11:15:42.452 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:15:42.577 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-06T11:15:42.878 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:15:42.986 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-06T11:15:43.004 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:15:43.129 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-06T11:15:43.163 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T11:15:43.263 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-06T11:15:43.564 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:15:43.639 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-06T11:15:43.933 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librados2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:44.059 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T11:15:44.132 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-06T11:15:44.518 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-06T11:15:44.960 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-06T11:15:45.404 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-06T11:15:45.538 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:45.801 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:45.926 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:45.956 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:46.052 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T11:15:46.140 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-06T11:15:46.529 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T11:15:46.575 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-06T11:15:46.693 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-06T11:15:46.838 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-06T11:15:46.913 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:15:47.203 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-06T11:15:47.473 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:47.599 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T11:15:47.630 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:47.780 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-06T11:15:47.906 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-06T11:15:48.009 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:48.135 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:15:48.316 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-06T11:15:48.547 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:48.659 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-06T11:15:48.707 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:15:49.027 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-06T11:15:49.108 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:49.162 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-06T11:15:49.504 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-06T11:15:49.864 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-06T11:15:50.240 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-06T11:15:50.321 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librbd1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:50.447 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:15:50.607 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T11:15:50.627 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-06T11:15:50.734 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:50.860 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T11:15:50.967 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-06T11:15:51.118 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-06T11:15:51.253 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:15:51.404 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T11:15:51.511 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-06T11:15:51.849 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:51.946 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-06T11:15:51.975 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:52.072 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-06T11:15:52.198 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-06T11:15:52.224 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:15:52.224 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:15:52.607 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-06T11:15:52.983 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:53.118 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:53.142 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-06T11:15:53.269 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:15:53.604 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T11:15:54.072 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:15:54.508 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:54.635 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:54.777 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:54.936 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:54.993 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:55.062 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:55.119 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:55.221 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T11:15:55.270 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-06T11:15:55.606 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T11:15:55.738 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-06T11:15:55.991 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:56.106 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-06T11:15:56.117 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:56.242 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:56.402 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:15:56.466 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-06T11:15:56.600 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-06T11:15:57.009 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:57.135 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-06T11:15:57.571 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:57.590 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:57.733 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-06T11:15:58.134 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-06T11:15:58.197 INFO:teuthology.orchestra.run.smithi132.stdout:Adding group ceph....done 2023-10-06T11:15:58.680 INFO:teuthology.orchestra.run.smithi132.stdout:Adding system user ceph....done 2023-10-06T11:15:58.924 INFO:teuthology.orchestra.run.smithi132.stdout:Setting system user ceph properties....done 2023-10-06T11:15:58.962 INFO:teuthology.orchestra.run.smithi132.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T11:15:59.156 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T11:15:59.422 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:15:59.548 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-06T11:15:59.674 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-06T11:15:59.685 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T11:15:59.850 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-06T11:15:59.976 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-06T11:16:00.346 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-06T11:16:00.463 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-06T11:16:00.675 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:00.927 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:16:00.927 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:16:00.958 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:01.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:01.378 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:16:01.378 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-06T11:16:01.776 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:01.927 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:02.240 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:16:02.241 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:16:02.252 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:02.387 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-06T11:16:02.746 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-06T11:16:03.140 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-06T11:16:03.478 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:03.617 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:03.743 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:03.869 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:03.965 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T11:16:04.287 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:04.438 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:04.564 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-06T11:16:04.855 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:04.915 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-06T11:16:05.189 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:16:05.190 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:16:05.283 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:05.409 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:05.578 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-06T11:16:06.124 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:06.394 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:16:06.395 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:16:06.973 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:07.233 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:07.579 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2023-10-06T11:16:08.029 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2023-10-06T11:16:08.091 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:16:08.091 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:16:08.267 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2023-10-06T11:16:08.302 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-06T11:16:08.509 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-06T11:16:08.935 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:09.056 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-06T11:16:09.061 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:09.221 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:09.480 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:16:09.481 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:16:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:10.234 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:16:10.234 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-06T11:16:10.372 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:10.523 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:10.649 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:10.825 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:10.952 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:16:11.085 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:11.145 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:16:11.210 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:11.530 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:16:11.531 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-06T11:16:11.622 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:16:12.813 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:13.323 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-06T11:16:14.348 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:14.626 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:16:14.627 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-06T11:16:15.433 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:15.716 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:16:15.716 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-06T11:16:16.576 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:16.677 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T11:16:17.409 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:16:17.409 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-06T11:16:17.447 INFO:teuthology.orchestra.run.smithi132.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:16:17.453 DEBUG:teuthology.orchestra.run.smithi132:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T11:16:17.578 INFO:teuthology.orchestra.run.smithi132.stdout:Reading package lists... 2023-10-06T11:16:17.790 INFO:teuthology.orchestra.run.smithi132.stdout:Building dependency tree... 2023-10-06T11:16:17.791 INFO:teuthology.orchestra.run.smithi132.stdout:Reading state information... 2023-10-06T11:16:17.992 INFO:teuthology.orchestra.run.smithi132.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T11:16:17.993 INFO:teuthology.orchestra.run.smithi132.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T11:16:17.993 INFO:teuthology.orchestra.run.smithi132.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T11:16:17.993 INFO:teuthology.orchestra.run.smithi132.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T11:16:17.993 INFO:teuthology.orchestra.run.smithi132.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T11:16:17.993 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:16:17.994 INFO:teuthology.orchestra.run.smithi132.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:16:17.994 INFO:teuthology.orchestra.run.smithi132.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:16:17.994 INFO:teuthology.orchestra.run.smithi132.stdout:The following additional packages will be installed: 2023-10-06T11:16:17.994 INFO:teuthology.orchestra.run.smithi132.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T11:16:17.994 INFO:teuthology.orchestra.run.smithi132.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T11:16:17.995 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T11:16:17.995 INFO:teuthology.orchestra.run.smithi132.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T11:16:17.996 INFO:teuthology.orchestra.run.smithi132.stdout:Suggested packages: 2023-10-06T11:16:17.996 INFO:teuthology.orchestra.run.smithi132.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T11:16:17.996 INFO:teuthology.orchestra.run.smithi132.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T11:16:18.081 INFO:teuthology.orchestra.run.smithi132.stdout:The following NEW packages will be installed: 2023-10-06T11:16:18.081 INFO:teuthology.orchestra.run.smithi132.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T11:16:18.082 INFO:teuthology.orchestra.run.smithi132.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T11:16:18.082 INFO:teuthology.orchestra.run.smithi132.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T11:16:18.082 INFO:teuthology.orchestra.run.smithi132.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T11:16:18.083 INFO:teuthology.orchestra.run.smithi132.stdout:The following packages will be upgraded: 2023-10-06T11:16:18.084 INFO:teuthology.orchestra.run.smithi132.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T11:16:18.261 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:18.278 INFO:teuthology.orchestra.run.smithi132.stdout:6 upgraded, 23 newly installed, 0 to remove and 250 not upgraded. 2023-10-06T11:16:18.278 INFO:teuthology.orchestra.run.smithi132.stdout:Need to get 10.4 MB of archives. 2023-10-06T11:16:18.278 INFO:teuthology.orchestra.run.smithi132.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T11:16:18.278 INFO:teuthology.orchestra.run.smithi132.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T11:16:18.387 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:18.596 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:18.861 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:16:18.861 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-06T11:16:18.900 INFO:teuthology.orchestra.run.smithi132.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T11:16:18.902 INFO:teuthology.orchestra.run.smithi132.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T11:16:18.904 INFO:teuthology.orchestra.run.smithi132.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T11:16:18.917 INFO:teuthology.orchestra.run.smithi132.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T11:16:18.918 INFO:teuthology.orchestra.run.smithi132.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T11:16:19.017 INFO:teuthology.orchestra.run.smithi132.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T11:16:19.019 INFO:teuthology.orchestra.run.smithi132.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T11:16:19.026 INFO:teuthology.orchestra.run.smithi132.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T11:16:19.031 INFO:teuthology.orchestra.run.smithi132.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T11:16:19.031 INFO:teuthology.orchestra.run.smithi132.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T11:16:19.199 INFO:teuthology.orchestra.run.smithi132.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T11:16:19.471 INFO:teuthology.orchestra.run.smithi132.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T11:16:19.600 INFO:teuthology.orchestra.run.smithi132.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T11:16:19.606 INFO:teuthology.orchestra.run.smithi132.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T11:16:19.701 INFO:teuthology.orchestra.run.smithi132.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T11:16:19.705 INFO:teuthology.orchestra.run.smithi132.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T11:16:19.748 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:19.788 INFO:teuthology.orchestra.run.smithi132.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T11:16:19.862 INFO:teuthology.orchestra.run.smithi132.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T11:16:19.862 INFO:teuthology.orchestra.run.smithi132.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T11:16:19.899 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:19.952 INFO:teuthology.orchestra.run.smithi132.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T11:16:19.953 INFO:teuthology.orchestra.run.smithi132.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T11:16:20.025 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:20.147 INFO:teuthology.orchestra.run.smithi132.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T11:16:20.193 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (17.2.6-1274-g68e53a4a-1focal) ... 2023-10-06T11:16:20.328 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:16:20.488 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:16:20.786 INFO:teuthology.orchestra.run.smithi132.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T11:16:20.901 INFO:teuthology.orchestra.run.smithi132.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T11:16:20.915 INFO:teuthology.orchestra.run.smithi132.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T11:16:20.917 INFO:teuthology.orchestra.run.smithi132.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T11:16:20.918 INFO:teuthology.orchestra.run.smithi132.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T11:16:20.920 INFO:teuthology.orchestra.run.smithi132.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T11:16:21.114 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:16:21.345 INFO:teuthology.orchestra.run.smithi132.stdout:Preconfiguring packages ... 2023-10-06T11:16:21.524 INFO:teuthology.orchestra.run.smithi132.stdout:Fetched 10.4 MB in 3s (3,688 kB/s) 2023-10-06T11:16:21.879 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:21.883 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:22.085 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:23.767 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:25.782 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:25.786 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:25.946 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:26.301 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-06T11:16:26.414 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:26.594 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:26.986 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:16:27.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-06T11:16:27.262 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-10-06T11:16:27.469 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-10-06T11:16:27.470 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-10-06T11:16:27.543 INFO:teuthology.orchestra.run.smithi132.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:27.546 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:27.679 INFO:teuthology.orchestra.run.smithi046.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-06T11:16:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-06T11:16:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-06T11:16:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-06T11:16:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-06T11:16:27.680 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-10-06T11:16:27.681 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-06T11:16:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-06T11:16:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-10-06T11:16:27.681 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-06T11:16:27.681 INFO:teuthology.orchestra.run.smithi046.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-06T11:16:27.682 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-06T11:16:27.682 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch-data wpasupplicant 2023-10-06T11:16:27.683 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-10-06T11:16:27.683 INFO:teuthology.orchestra.run.smithi046.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-06T11:16:27.683 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-06T11:16:27.686 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:16:27.768 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-10-06T11:16:27.769 INFO:teuthology.orchestra.run.smithi046.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-06T11:16:27.769 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-06T11:16:27.769 INFO:teuthology.orchestra.run.smithi046.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-06T11:16:27.770 INFO:teuthology.orchestra.run.smithi046.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-06T11:16:27.771 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2023-10-06T11:16:27.772 INFO:teuthology.orchestra.run.smithi046.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-06T11:16:27.991 INFO:teuthology.orchestra.run.smithi046.stdout:6 upgraded, 23 newly installed, 0 to remove and 250 not upgraded. 2023-10-06T11:16:27.991 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 10.4 MB of archives. 2023-10-06T11:16:27.992 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-06T11:16:27.992 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-06T11:16:28.237 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:28.616 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-06T11:16:28.618 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-06T11:16:28.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-06T11:16:28.632 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-06T11:16:28.634 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-06T11:16:28.652 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:16:28.734 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-06T11:16:28.735 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-06T11:16:28.743 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-06T11:16:28.749 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-06T11:16:28.749 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-06T11:16:28.917 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-06T11:16:29.184 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:16:29.189 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-06T11:16:29.319 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-06T11:16:29.323 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T11:16:29.324 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-06T11:16:29.419 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-06T11:16:29.423 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-06T11:16:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-06T11:16:29.580 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-06T11:16:29.581 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-06T11:16:29.671 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-06T11:16:29.671 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-06T11:16:29.865 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-06T11:16:29.902 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T11:16:29.923 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T11:16:29.961 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:16:30.347 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package dump. 2023-10-06T11:16:30.367 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T11:16:30.405 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T11:16:30.505 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-06T11:16:30.618 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-06T11:16:30.634 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-06T11:16:30.635 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-06T11:16:30.636 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-06T11:16:30.639 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-06T11:16:30.779 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package indent. 2023-10-06T11:16:30.795 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:16:30.836 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:16:31.073 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2023-10-06T11:16:31.204 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T11:16:31.224 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T11:16:31.257 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 10.4 MB in 3s (3,651 kB/s) 2023-10-06T11:16:31.262 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:16:31.588 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:31.592 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:31.658 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T11:16:31.679 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T11:16:31.757 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:16:31.777 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:32.044 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T11:16:32.064 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:32.135 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:32.530 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T11:16:32.551 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:32.588 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:32.925 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T11:16:32.946 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:32.983 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:33.386 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T11:16:33.406 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:33.444 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:33.472 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:33.881 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T11:16:33.901 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T11:16:33.939 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:16:34.342 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T11:16:34.362 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:16:34.400 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:34.870 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T11:16:34.891 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:34.928 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:35.298 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T11:16:35.318 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:35.356 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:35.477 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:35.480 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:35.635 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:35.684 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T11:16:35.705 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:16:35.742 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:36.137 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-06T11:16:36.308 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-06T11:16:36.338 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T11:16:36.359 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T11:16:36.419 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:16:36.712 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:36.875 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package modemmanager. 2023-10-06T11:16:36.895 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:36.954 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:37.208 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118691 files and directories currently installed.) 2023-10-06T11:16:37.211 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:37.359 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:16:37.689 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T11:16:37.704 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T11:16:37.747 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:16:37.985 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:38.114 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-06T11:16:38.386 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package network-manager. 2023-10-06T11:16:38.400 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:16:38.493 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:38.706 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:16:38.844 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-06T11:16:39.319 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package ppp. 2023-10-06T11:16:39.396 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T11:16:39.414 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-06T11:16:39.614 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-06T11:16:39.614 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:16:39.810 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:16:40.120 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2023-10-06T11:16:40.134 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-06T11:16:40.177 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b46-6) ... 2023-10-06T11:16:40.225 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T11:16:40.245 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T11:16:40.282 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T11:16:40.548 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2023-10-06T11:16:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-06T11:16:40.606 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:16:40.728 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T11:16:40.747 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T11:16:40.821 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T11:16:41.000 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-06T11:16:41.021 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-06T11:16:41.058 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:16:41.131 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T11:16:41.151 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T11:16:41.190 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T11:16:41.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-06T11:16:41.484 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-06T11:16:41.553 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:16:41.626 INFO:teuthology.orchestra.run.smithi132.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T11:16:41.646 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T11:16:41.706 INFO:teuthology.orchestra.run.smithi132.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:16:41.848 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-06T11:16:41.869 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-06T11:16:41.906 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:42.085 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:42.285 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-06T11:16:42.305 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:42.363 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:42.448 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:16:42.575 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:16:42.678 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-06T11:16:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:42.726 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:16:42.736 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:43.131 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-06T11:16:43.149 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:43.157 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:16:43.157 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:16:43.214 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:43.642 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-06T11:16:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-06T11:16:43.725 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:16:44.046 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:44.120 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-06T11:16:44.141 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:16:44.186 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:44.192 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:16:44.323 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:44.466 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T11:16:44.639 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-06T11:16:44.642 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:44.660 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:44.697 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:44.768 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:16:45.083 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-06T11:16:45.104 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:45.141 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:45.444 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-06T11:16:45.464 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-06T11:16:45.502 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:46.053 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T11:16:46.081 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-06T11:16:46.102 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-06T11:16:46.174 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:16:46.516 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T11:16:46.592 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2023-10-06T11:16:46.613 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-06T11:16:46.622 INFO:teuthology.orchestra.run.smithi132.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T11:16:46.674 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:46.685 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:46.815 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:16:46.987 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:16:47.113 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:16:47.323 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2023-10-06T11:16:47.344 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-06T11:16:47.348 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:16:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:16:47.474 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:47.600 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:47.752 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T11:16:47.911 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:48.036 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2023-10-06T11:16:48.056 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-06T11:16:48.071 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:48.145 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:48.943 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2023-10-06T11:16:48.961 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-06T11:16:49.265 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:16:49.265 INFO:teuthology.orchestra.run.smithi132.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T11:16:49.265 INFO:teuthology.orchestra.run.smithi132.stdout:which means they are currently configured by ifupdown: 2023-10-06T11:16:49.265 INFO:teuthology.orchestra.run.smithi132.stdout:- enp3s0f1 2023-10-06T11:16:49.266 INFO:teuthology.orchestra.run.smithi132.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T11:16:49.266 INFO:teuthology.orchestra.run.smithi132.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T11:16:49.266 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:16:49.468 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:16:49.494 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T11:16:49.880 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2023-10-06T11:16:49.894 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-06T11:16:49.939 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-06T11:16:50.013 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T11:16:50.342 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-06T11:16:50.356 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-06T11:16:50.437 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-06T11:16:50.778 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-06T11:16:50.792 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-06T11:16:50.836 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-06T11:16:50.965 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T11:16:51.248 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-06T11:16:51.262 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-06T11:16:51.329 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:16:51.725 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:51.941 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:52.067 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T11:16:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-06T11:16:52.219 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:52.296 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-06T11:16:52.379 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:52.447 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-06T11:16:52.504 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:16:52.634 INFO:teuthology.orchestra.run.smithi132.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:52.880 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:16:52.881 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-06T11:16:52.971 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:16:52.971 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:16:53.800 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-06T11:16:53.926 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-06T11:16:54.085 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-06T11:16:54.211 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-06T11:16:54.254 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T11:16:54.337 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:54.451 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T11:16:54.496 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-06T11:16:54.585 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:16:54.778 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:16:55.254 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:16:55.750 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-06T11:16:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b46-6) ... 2023-10-06T11:16:56.321 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-06T11:16:56.404 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:16:56.555 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-06T11:16:56.673 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-06T11:16:56.799 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-06T11:16:57.000 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-06T11:16:57.126 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:57.252 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:16:57.403 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-06T11:16:57.580 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:16:57.706 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-06T11:16:58.901 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:16:58.901 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-06T11:16:58.902 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2023-10-06T11:16:58.902 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2023-10-06T11:16:58.902 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-06T11:16:58.902 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2023-10-06T11:16:58.902 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:16:59.146 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-06T11:16:59.674 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-06T11:17:00.630 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-06T11:17:01.584 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-06T11:17:01.753 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-06T11:17:01.899 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T11:17:01.913 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-06T11:17:02.016 INFO:teuthology.orchestra.run.smithi132.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T11:17:02.039 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:17:02.205 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-06T11:17:02.341 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-06T11:17:02.673 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:17:02.673 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-06T11:17:03.315 INFO:teuthology.orchestra.run.smithi132.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:17:03.321 DEBUG:teuthology.parallel:result is None 2023-10-06T11:17:03.633 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-06T11:17:03.751 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-06T11:17:03.910 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-06T11:17:04.070 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-06T11:17:04.571 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-06T11:17:11.433 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-06T11:17:11.548 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-06T11:17:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-06T11:17:12.779 DEBUG:teuthology.parallel:result is None 2023-10-06T11:17:12.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:17:12.997 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2023-10-06T11:17:13.032 INFO:teuthology.orchestra.run.smithi046.stdout:17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:17:13.033 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:17:13.033 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274-g68e53a4a-1focal is installed. 2023-10-06T11:17:13.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:17:13.176 DEBUG:teuthology.orchestra.run.smithi132:> dpkg-query -W -f '${Version}' ceph 2023-10-06T11:17:13.211 INFO:teuthology.orchestra.run.smithi132.stdout:17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:17:13.211 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274-g68e53a4a-1focal 2023-10-06T11:17:13.211 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274-g68e53a4a-1focal is installed. 2023-10-06T11:17:13.211 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T11:17:13.211 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:13.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:17:13.247 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:13.247 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:17:13.285 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T11:17:13.285 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:13.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:17:13.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:17:13.399 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:13.399 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:17:13.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:17:13.512 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T11:17:13.513 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:13.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:17:13.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:17:13.623 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:13.624 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:17:13.656 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:17:13.732 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T11:17:13.733 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:13.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:17:13.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:17:13.842 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:13.843 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:17:13.876 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:17:13.952 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T11:17:14.017 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T11:17:14.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2023-10-06T11:17:14.024 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2023-10-06T11:17:14.055 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T11:17:14.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:17:14.101 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:17:14.133 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T11:17:14.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:17:14.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:17:14.219 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T11:17:14.219 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T11:17:14.219 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T11:17:14.220 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:14.220 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:17:14.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:17:14.398 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:14.398 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:17:14.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:17:14.545 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T11:17:14.545 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T11:17:14.545 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412390', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T11:17:14.545 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:17:14.553 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:17:14.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:17:14.613 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:17:14.649 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:14.650 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:17:14.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:17:14.704 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2023-10-06T11:17:14.765 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-06 11:11:55.170863451 +0000 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-06 11:11:54.678874796 +0000 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-06 11:11:54.678874796 +0000 2023-10-06T11:17:14.766 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-06T11:17:14.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:17:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-06T11:17:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-06T11:17:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000594025 s, 862 kB/s 2023-10-06T11:17:14.840 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:17:14.909 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2023-10-06T11:17:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:17:14.969 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 506 Links: 1 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-06 11:11:55.978844816 +0000 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-06 11:11:55.498855886 +0000 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-06 11:11:55.498855886 +0000 2023-10-06T11:17:14.970 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-06T11:17:14.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:17:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-06T11:17:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-06T11:17:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000594039 s, 862 kB/s 2023-10-06T11:17:15.043 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:17:15.110 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2023-10-06T11:17:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:17:15.173 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:15.173 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 512 Links: 1 2023-10-06T11:17:15.173 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:15.174 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-06 11:11:56.686828489 +0000 2023-10-06T11:17:15.174 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-06 11:11:56.238838821 +0000 2023-10-06T11:17:15.174 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-06 11:11:56.238838821 +0000 2023-10-06T11:17:15.174 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-06T11:17:15.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:17:15.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-06T11:17:15.246 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-06T11:17:15.246 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000506216 s, 1.0 MB/s 2023-10-06T11:17:15.248 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:17:15.313 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 516 Links: 1 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-10-06 11:11:57.486810041 +0000 2023-10-06T11:17:15.373 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-10-06 11:11:56.990821479 +0000 2023-10-06T11:17:15.374 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-10-06 11:11:56.990821479 +0000 2023-10-06T11:17:15.374 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-10-06T11:17:15.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:17:15.442 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-10-06T11:17:15.443 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-10-06T11:17:15.443 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000337303 s, 1.5 MB/s 2023-10-06T11:17:15.445 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:17:15.511 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:15.511 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:15.511 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:17:15.529 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:17:15.530 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2023-10-06T11:17:15.590 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:17:15.590 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:15.590 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-10-06T11:17:15.591 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:15.591 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-06 11:11:51.854440876 +0000 2023-10-06T11:17:15.591 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-06 11:11:51.402460786 +0000 2023-10-06T11:17:15.591 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-06 11:11:51.402460786 +0000 2023-10-06T11:17:15.591 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-06T11:17:15.591 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:17:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-06T11:17:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-06T11:17:15.662 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000662997 s, 772 kB/s 2023-10-06T11:17:15.664 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:17:15.736 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2023-10-06T11:17:15.798 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 504 Links: 1 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-06 11:11:52.614408010 +0000 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-06 11:11:52.158427641 +0000 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-06 11:11:52.158427641 +0000 2023-10-06T11:17:15.799 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-06T11:17:15.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:17:15.870 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-06T11:17:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-06T11:17:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000665616 s, 769 kB/s 2023-10-06T11:17:15.873 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:17:15.937 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2023-10-06T11:17:16.002 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 510 Links: 1 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-06 11:11:53.406374318 +0000 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-06 11:11:52.934394234 +0000 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-06 11:11:52.934394234 +0000 2023-10-06T11:17:16.003 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-06T11:17:16.004 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:17:16.077 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-06T11:17:16.078 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-06T11:17:16.078 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000504497 s, 1.0 MB/s 2023-10-06T11:17:16.080 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:17:16.149 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2023-10-06T11:17:16.209 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:17:16.209 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:17:16.209 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 515 Links: 1 2023-10-06T11:17:16.209 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:17:16.210 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-06 11:11:54.182341687 +0000 2023-10-06T11:17:16.210 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-06 11:11:53.726360862 +0000 2023-10-06T11:17:16.210 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-06 11:11:53.726360862 +0000 2023-10-06T11:17:16.210 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-06T11:17:16.210 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:17:16.282 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-06T11:17:16.282 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-06T11:17:16.282 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00063809 s, 802 kB/s 2023-10-06T11:17:16.284 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:17:16.351 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:16.352 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:17:16.352 INFO:tasks.ceph:Generating config... 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] pid file = 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T11:17:16.354 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T11:17:16.354 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T11:17:16.354 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T11:17:16.354 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T11:17:16.355 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T11:17:16.356 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T11:17:16.356 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T11:17:16.356 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T11:17:16.356 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T11:17:16.356 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T11:17:16.356 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T11:17:16.357 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T11:17:16.357 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T11:17:16.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T11:17:16.420 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T11:17:16.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T11:17:16.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:17:16.522 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]')] 2023-10-06T11:17:16.522 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T11:17:16.523 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:16.523 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:17:16.583 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:16.689 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:16.689 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 36b5f543-5934-4e63-898c-253575113a16 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = octopus 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 36b5f543-5934-4e63-898c-253575113a16 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2023-10-06T11:17:16.687101+0000 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:created 2023-10-06T11:17:16.687101+0000 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 15 (octopus) 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.b 2023-10-06T11:17:16.690 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2023-10-06T11:17:16.691 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T11:17:16.692 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:17:16.709 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 36b5f543-5934-4e63-898c-253575113a16... 2023-10-06T11:17:16.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:17:16.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:17:16.833 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2023-10-06T11:17:16.834 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:17:16.835 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2023-10-06T11:17:16.836 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2023-10-06T11:17:16.837 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 36b5f543-5934-4e63-898c-253575113a16 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.838 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.839 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2023-10-06T11:17:16.840 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2023-10-06T11:17:16.841 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2023-10-06T11:17:16.842 INFO:teuthology.orchestra.run.smithi046.stdout: debug finisher = 20 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2023-10-06T11:17:16.843 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.844 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2023-10-06T11:17:16.845 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-10-06T11:17:16.846 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2023-10-06T11:17:16.847 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.848 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2023-10-06T11:17:16.849 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2023-10-06T11:17:16.850 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.46,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = 36b5f543-5934-4e63-898c-253575113a16 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.851 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2023-10-06T11:17:16.852 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.853 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2023-10-06T11:17:16.854 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:17:16.855 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug finisher = 20 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2023-10-06T11:17:16.856 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:17:16.857 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:17:16.858 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-10-06T11:17:16.859 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2023-10-06T11:17:16.860 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2023-10-06T11:17:16.862 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T11:17:16.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T11:17:16.939 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T11:17:16.939 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:16.940 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T11:17:16.962 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:16.962 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T11:17:17.023 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:17:17.023 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:17.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:17:17.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:17:17.131 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:17.131 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:17.191 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:17:17.191 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:17.191 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:17:17.191 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:17:17.253 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:17.254 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:17.313 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T11:17:17.314 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T11:17:17.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:17:17.404 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:17:17.411 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:17:17.505 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:17:17.512 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T11:17:17.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:17:17.606 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:17:17.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T11:17:17.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:17:17.780 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:17:17.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T11:17:17.819 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:17:17.912 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:17:17.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T11:17:17.951 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:17:18.083 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:17:18.090 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T11:17:18.122 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T11:17:18.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T11:17:18.189 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T11:17:18.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T11:17:18.286 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T11:17:18.315 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T11:17:18.383 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T11:17:18.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T11:17:18.477 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T11:17:18.505 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T11:17:18.505 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:17:18.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T11:17:18.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:18.540 INFO:tasks.ceph:role: osd.0 2023-10-06T11:17:18.541 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:17:18.541 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:18.657 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:18.658 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:18.658 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:18.662 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-06T11:17:18.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T11:17:18.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T11:17:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:18.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:18.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T11:17:18.860 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:18.860 INFO:tasks.ceph:role: osd.1 2023-10-06T11:17:18.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:17:18.861 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:17:18.976 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2023-10-06T11:17:18.976 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:18.977 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:18.977 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:18.977 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:18.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-06T11:17:18.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T11:17:19.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T11:17:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:19.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:19.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T11:17:19.170 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:19.170 INFO:tasks.ceph:role: osd.2 2023-10-06T11:17:19.170 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:17:19.171 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:17:19.283 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:19.283 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:19.283 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:19.284 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:19.288 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-06T11:17:19.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T11:17:19.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T11:17:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:19.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:19.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T11:17:19.481 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:19.481 INFO:tasks.ceph:role: osd.3 2023-10-06T11:17:19.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:17:19.481 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:17:19.592 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:19.592 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 2023-10-06T11:17:19.592 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:19.593 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:19.593 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:19.593 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:19.593 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:19.593 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:19.597 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-10-06T11:17:19.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T11:17:19.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T11:17:19.711 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:19.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:19.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.859+0000 7fe43acfa3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T11:17:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.859+0000 7fe43acfa3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T11:17:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.859+0000 7fe43acfa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:17:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.859+0000 7fe43acfa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:17:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.859+0000 7fe43acfa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:17:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:19.863+0000 7fe43acfa3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T11:17:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:20.435+0000 7fe43acfa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:21.223+0000 7fe43acfa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:22.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:22.019+0000 7fe43acfa3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:22.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T11:17:22.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.063+0000 7ffa4d8eb3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T11:17:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.063+0000 7ffa4d8eb3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T11:17:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.063+0000 7ffa4d8eb3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:17:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.063+0000 7ffa4d8eb3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:17:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.063+0000 7ffa4d8eb3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:17:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.067+0000 7ffa4d8eb3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T11:17:23.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:23.635+0000 7ffa4d8eb3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:24.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:24.431+0000 7ffa4d8eb3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:25.211+0000 7ffa4d8eb3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:26.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T11:17:26.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.275+0000 7fba531183c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T11:17:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.275+0000 7fba531183c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T11:17:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.279+0000 7fba531183c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:17:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.279+0000 7fba531183c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:17:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.279+0000 7fba531183c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:17:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.279+0000 7fba531183c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T11:17:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:26.859+0000 7fba531183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:27.639+0000 7fba531183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:28.431+0000 7fba531183c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:29.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T11:17:29.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:29.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.471+0000 7f33a5cdd3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T11:17:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.475+0000 7f33a5cdd3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T11:17:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.475+0000 7f33a5cdd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:17:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.475+0000 7f33a5cdd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:17:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.475+0000 7f33a5cdd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:17:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:29.475+0000 7f33a5cdd3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T11:17:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:30.063+0000 7f33a5cdd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:30.851+0000 7f33a5cdd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:31.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:31.643+0000 7f33a5cdd3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:32.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T11:17:32.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T11:17:32.618 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:32.618 INFO:tasks.ceph:role: osd.4 2023-10-06T11:17:32.618 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:17:32.618 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:17:33.215 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:33.215 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:33.215 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:33.216 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:33.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-10-06T11:17:33.219 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T11:17:33.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T11:17:33.366 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:33.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:33.367 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T11:17:33.445 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:33.445 INFO:tasks.ceph:role: osd.5 2023-10-06T11:17:33.445 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:17:33.445 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:17:33.975 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:33.975 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:33.975 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:33.975 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:33.976 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:33.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-10-06T11:17:33.981 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T11:17:34.055 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T11:17:34.080 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:34.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:34.081 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T11:17:34.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:34.163 INFO:tasks.ceph:role: osd.6 2023-10-06T11:17:34.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:17:34.163 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:17:34.711 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:34.712 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:34.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-10-06T11:17:34.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T11:17:34.815 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T11:17:34.834 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:34.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:34.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T11:17:34.916 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:17:34.916 INFO:tasks.ceph:role: osd.7 2023-10-06T11:17:34.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:17:34.917 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:17:35.497 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:17:35.498 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:17:35.502 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2023-10-06T11:17:35.502 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T11:17:35.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T11:17:35.631 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: /sbin/restorecon: command not found 2023-10-06T11:17:35.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:17:35.632 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:35.786 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.781+0000 7f9e51b6c3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T11:17:35.789 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.785+0000 7f9e51b6c3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T11:17:35.789 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.785+0000 7f9e51b6c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:17:35.789 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.785+0000 7f9e51b6c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:17:35.789 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.785+0000 7f9e51b6c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:17:35.791 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:35.785+0000 7f9e51b6c3c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T11:17:36.364 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:36.361+0000 7f9e51b6c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:37.154 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:37.149+0000 7f9e51b6c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:37.950 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:37.945+0000 7f9e51b6c3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:38.847 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T11:17:38.880 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:38.994 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.989+0000 7f517548f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T11:17:38.995 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.989+0000 7f517548f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T11:17:38.996 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.989+0000 7f517548f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:17:38.996 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.989+0000 7f517548f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:17:38.996 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.993+0000 7f517548f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:17:38.998 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:38.993+0000 7f517548f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T11:17:39.568 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:39.564+0000 7f517548f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:40.358 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:40.352+0000 7f517548f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:41.150 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:41.144+0000 7f517548f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:42.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T11:17:42.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:42.184 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T11:17:42.185 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T11:17:42.185 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:17:42.185 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:17:42.185 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:17:42.187 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.180+0000 7fea364d93c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T11:17:42.760 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:42.756+0000 7fea364d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:43.570 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:43.564+0000 7fea364d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:44.356 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:44.348+0000 7fea364d93c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:44.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:17:44.554 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:17:45.272 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T11:17:45.304 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:45.418 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.412+0000 7fbf9336e3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T11:17:45.419 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.412+0000 7fbf9336e3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T11:17:45.419 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.412+0000 7fbf9336e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:17:45.419 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.412+0000 7fbf9336e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:17:45.419 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.412+0000 7fbf9336e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:17:45.422 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.416+0000 7fbf9336e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T11:17:45.996 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:45.992+0000 7fbf9336e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:46.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:46.776+0000 7fbf9336e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:47.573 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:47.568+0000 7fbf9336e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:17:48.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T11:17:48.498 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T11:17:48.498 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T11:17:48.533 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T11:17:48.613 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T11:17:48.687 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T11:17:48.764 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T11:17:48.837 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T11:17:48.912 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:48.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T11:17:48.988 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:48.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T11:17:49.024 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.024 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T11:17:49.099 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T11:17:49.177 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T11:17:49.256 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T11:17:49.336 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.336 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T11:17:49.412 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.412 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T11:17:49.491 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:49.492 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T11:17:49.508 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:49.508 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T11:17:49.567 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.567 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T11:17:49.585 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:49.585 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T11:17:49.645 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T11:17:49.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:17:49.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw== 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ== 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw== 2023-10-06T11:17:49.695 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g== 2023-10-06T11:17:49.696 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg== 2023-10-06T11:17:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2023-10-06T11:17:49.698 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ== 2023-10-06T11:17:49.698 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2023-10-06T11:17:49.698 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ== 2023-10-06T11:17:49.698 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2023-10-06T11:17:49.698 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw== 2023-10-06T11:17:49.711 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2023-10-06T11:17:49.711 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw== 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ== 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw== 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ== 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw== 2023-10-06T11:17:49.712 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw== 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2023-10-06T11:17:49.713 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ== 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag== 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ== 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg== 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout:[client.2] 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ== 2023-10-06T11:17:49.714 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2023-10-06T11:17:49.715 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ== 2023-10-06T11:17:49.715 INFO:teuthology.orchestra.run.smithi132.stdout:[client.3] 2023-10-06T11:17:49.715 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw== 2023-10-06T11:17:49.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:17:49.753 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:17:49.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:49.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:49.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:49.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:49.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:49.998 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.075 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.149 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.223 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:17:50.303 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:17:50.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:50.381 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:50.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:50.462 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:17:50.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.545 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.631 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.711 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.790 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:17:50.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:50.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:50.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:50.954 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:51.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:51.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:51.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:51.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:17:51.186 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T11:17:51.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T11:17:51.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:17:51.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T11:17:51.390 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T11:17:51.426 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:17:51.557 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T11:17:51.591 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T11:17:51.671 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:17:51.801 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T11:17:51.835 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:51.842 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:17:51.893 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T11:17:51.894 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T11:17:51.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T11:17:51.909 INFO:tasks.ceph.mon.a:Started 2023-10-06T11:17:51.909 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T11:17:51.909 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T11:17:51.942 INFO:tasks.ceph.mon.b:Started 2023-10-06T11:17:51.942 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T11:17:51.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T11:17:51.946 INFO:tasks.ceph.mon.c:Started 2023-10-06T11:17:51.946 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T11:17:51.947 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T11:17:51.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T11:17:51.954 INFO:tasks.ceph.mgr.y:Started 2023-10-06T11:17:51.954 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T11:17:51.954 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T11:17:51.960 INFO:tasks.ceph.mgr.x:Started 2023-10-06T11:17:51.960 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T11:17:51.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T11:17:52.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.095+0000 7f1d12923700 1 Processor -- start 2023-10-06T11:17:52.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 -- start start 2023-10-06T11:17:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c0c10a0 con 0x7f1d0c159240 2023-10-06T11:17:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c0ca240 con 0x7f1d0c0c72d0 2023-10-06T11:17:52.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12923700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c0ca520 con 0x7f1d0c158e70 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55170/0 (socket says 172.21.15.46:55170) 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 learned_addr learned my addr 172.21.15.46:0/2000865878 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T11:17:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:17:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.099+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:17:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T11:17:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:17:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:17:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T11:17:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:52.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:17:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T11:17:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:17:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:53.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:17:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 msgr2=0x7f1d0c0a9840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.132:3300/0 conn(0x7f1d0c158e70 0x7f1d0c0a9840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 -- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 msgr2=0x7f1d0c0c7a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 --2- 172.21.15.46:0/2000865878 >> v2:172.21.15.46:3300/0 conn(0x7f1d0c0c72d0 0x7f1d0c0c7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 -- 172.21.15.46:0/2000865878 >> v1:172.21.15.46:6789/0 conn(0x7f1d0c159240 legacy=0x7f1d0c0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 -- 172.21.15.46:0/2000865878 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c0c0a50 con 0x7f1cf0006190 2023-10-06T11:17:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d037fe700 1 -- 172.21.15.46:0/2000865878 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0ca240 con 0x7f1cf0003700 2023-10-06T11:17:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.099+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:17:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.303+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.303+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.303+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:17:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.707+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.707+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:55.707+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:17:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:56.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:56.511+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:17:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:56.511+0000 7f1d11921700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:17:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:56.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:17:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:56.511+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:17:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.303+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070223740 0 0) 0x7f1d0c0c0a50 con 0x7f1cf0006190 2023-10-06T11:17:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.303+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf8003570 con 0x7f1cf0006190 2023-10-06T11:17:57.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428240640 0 0) 0x7f1cf8003570 con 0x7f1cf0006190 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 msgr2=0x7f1cf0005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf0003700 0x7f1cf0005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf000af80 con 0x7f1cf0006190 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f1cfc0014f0 con 0x7f1cf0006190 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 >> v1:172.21.15.132:6789/0 conn(0x7f1cf0006190 legacy=0x7f1cf0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d12122700 1 -- 172.21.15.46:0/2000865878 reap_dead start 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 0x7f1cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf80052e0 0x7f1cf800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0ca240 con 0x7f1cf8004810 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf800c250 con 0x7f1cf8003a60 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf800c410 con 0x7f1cf80052e0 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 0x7f1cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d12122700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf80052e0 0x7f1cf800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11921700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf80052e0 msgr2=0x7f1cf800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf80052e0 0x7f1cf800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 msgr2=0x7f1cf8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.311+0000 7f1d11120700 1 -- 172.21.15.46:0/2000865878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8003570 con 0x7f1cf8004810 2023-10-06T11:17:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d11120700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 0x7f1cf8004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c2a0 tx=0x7f1cfc0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb3a45319075b74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00cbd0 con 0x7f1cf8004810 2023-10-06T11:17:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc002e90 con 0x7f1cf8004810 2023-10-06T11:17:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d1091f700 1 -- 172.21.15.46:0/2000865878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc012400 con 0x7f1cf8004810 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 -- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 msgr2=0x7f1cf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 0x7f1cf8004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c2a0 tx=0x7f1cfc0078c0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 -- 172.21.15.46:0/2000865878 shutdown_connections 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf80052e0 0x7f1cf800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf8004810 0x7f1cf8004c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 --2- 172.21.15.46:0/2000865878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.315+0000 7f1d12923700 1 -- 172.21.15.46:0/2000865878 >> 172.21.15.46:0/2000865878 conn(0x7f1d0c0cbc10 msgr2=0x7f1d0c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 -- 172.21.15.46:0/2000865878 shutdown_connections 2023-10-06T11:17:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 -- 172.21.15.46:0/2000865878 wait complete. 2023-10-06T11:17:57.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 Processor -- start 2023-10-06T11:17:57.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 -- start start 2023-10-06T11:17:57.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1d0c16d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf8004810 0x7f1d0c152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.319+0000 7f1d12923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0afa50 con 0x7f1cf80052e0 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0b0040 con 0x7f1cf8003a60 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0c2fe0 con 0x7f1cf8004810 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43102/0 (socket says 172.21.15.46:43102) 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 -- 172.21.15.46:0/174285948 learned_addr learned my addr 172.21.15.46:0/174285948 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d11120700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf8004810 0x7f1d0c152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d11921700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1d0c16d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 -- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf8004810 msgr2=0x7f1d0c152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf8004810 0x7f1d0c152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 -- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 msgr2=0x7f1d0c16d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:57.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1d0c16d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 -- 172.21.15.46:0/174285948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c16d800 con 0x7f1cf80052e0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12122700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1d040159a0 tx=0x7f1d0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d040138a0 con 0x7f1cf80052e0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0400f040 con 0x7f1cf80052e0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04013c70 con 0x7f1cf80052e0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c0aee40 con 0x7f1cf80052e0 2023-10-06T11:17:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c0af400 con 0x7f1cf80052e0 2023-10-06T11:17:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1d04018660 con 0x7f1cf80052e0 2023-10-06T11:17:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.323+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1d040217a0 con 0x7f1cf80052e0 2023-10-06T11:17:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.327+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c064590 con 0x7f1cf80052e0 2023-10-06T11:17:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.331+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1d0401c070 con 0x7f1cf80052e0 2023-10-06T11:17:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.407+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1d0c0c0d30 con 0x7f1cf80052e0 2023-10-06T11:17:57.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d01ffb700 1 -- 172.21.15.46:0/174285948 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1d04013dd0 con 0x7f1cf80052e0 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 msgr2=0x7f1d0c157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1d040159a0 tx=0x7f1d0400b040 comp rx=0 tx=0).stop 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 shutdown_connections 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cf80052e0 0x7f1d0c157570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cf8004810 0x7f1d0c152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 --2- 172.21.15.46:0/174285948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf8003a60 0x7f1d0c16d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 >> 172.21.15.46:0/174285948 conn(0x7f1d0c0cbc10 msgr2=0x7f1d0c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:57.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.415+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 shutdown_connections 2023-10-06T11:17:57.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:17:57.419+0000 7f1d12923700 1 -- 172.21.15.46:0/174285948 wait complete. 2023-10-06T11:17:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2023-10-06T11:17:57.440 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T11:17:57.440 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:57.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T11:17:57.475 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:57.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T11:17:57.534 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:57.527+0000 7f4bf4310000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:17:57.553 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:57.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T11:17:57.566 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:57.560+0000 7f96a6943000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:17:57.631 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-10-06T11:17:57.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T11:17:57.707 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:57.707 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T11:17:57.741 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:57.741 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T11:17:57.815 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:57.815 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T11:17:57.891 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-06T11:17:57.892 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T11:17:57.967 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new ba6dfc44-0d94-4db5-bc84-34c7777dc7e9 0 2023-10-06T11:17:58.160 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 Processor -- start 2023-10-06T11:17:58.160 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 -- start start 2023-10-06T11:17:58.162 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc4780b2360 0x7fc4780b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.162 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780af6f0 con 0x7fc478159550 2023-10-06T11:17:58.162 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4780b43e0 con 0x7fc4780aee00 2023-10-06T11:17:58.162 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47edc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4780b04b0 con 0x7fc4780b2360 2023-10-06T11:17:58.162 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc477fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc4780aee00 0x7fc478158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58470/0 (socket says 172.21.15.132:58470) 2023-10-06T11:17:58.163 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc477fff700 1 -- 172.21.15.132:0/2833551450 learned_addr learned my addr 172.21.15.132:0/2833551450 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-10-06T11:17:58.163 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47cb61700 1 --2- 172.21.15.132:0/2833551450 >> v2:172.21.15.132:3300/0 conn(0x7fc4780b2360 0x7fc4780b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618940661 0 0) 0x7fc4780b43e0 con 0x7fc4780aee00 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003570 con 0x7fc4780aee00 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862665949 0 0) 0x7fc4780af6f0 con 0x7fc478159550 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47cb61700 1 -- 172.21.15.132:0/2833551450 >> v1:172.21.15.132:6789/0 conn(0x7fc4780aee00 legacy=0x7fc478158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47cb61700 1 -- 172.21.15.132:0/2833551450 >> v1:172.21.15.46:6789/0 conn(0x7fc478159550 legacy=0x7fc4780b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.164 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.156+0000 7fc47cb61700 1 -- 172.21.15.132:0/2833551450 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780b0790 con 0x7fc4780b2360 2023-10-06T11:17:58.165 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47cb61700 1 --2- 172.21.15.132:0/2833551450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc4780b2730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009860 tx=0x7fc46c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b1423d982568ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.165 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464001470 con 0x7fc4780b2360 2023-10-06T11:17:58.165 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c011420 con 0x7fc4780b2360 2023-10-06T11:17:58.165 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc4777fe700 1 -- 172.21.15.132:0/2833551450 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c0117c0 con 0x7fc4780b2360 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 -- 172.21.15.132:0/2833551450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 msgr2=0x7fc4780b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2833551450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc4780b2730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009860 tx=0x7fc46c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 -- 172.21.15.132:0/2833551450 shutdown_connections 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2833551450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc4780b2730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 -- 172.21.15.132:0/2833551450 >> 172.21.15.132:0/2833551450 conn(0x7fc47806e780 msgr2=0x7fc47806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 -- 172.21.15.132:0/2833551450 shutdown_connections 2023-10-06T11:17:58.166 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.160+0000 7fc47edc5700 1 -- 172.21.15.132:0/2833551450 wait complete. 2023-10-06T11:17:58.168 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 Processor -- start 2023-10-06T11:17:58.168 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 -- start start 2023-10-06T11:17:58.170 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 0x7fc478157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.170 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.170 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 0x7fc47815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.170 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47815a7c0 con 0x7fc4780aee00 2023-10-06T11:17:58.170 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc478159bb0 con 0x7fc4780b2360 2023-10-06T11:17:58.173 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47edc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc478159e90 con 0x7fc478159550 2023-10-06T11:17:58.173 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.173 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47d362700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 0x7fc47815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.173 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60528/0 (socket says 172.21.15.132:60528) 2023-10-06T11:17:58.173 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 -- 172.21.15.132:0/2146138794 learned_addr learned my addr 172.21.15.132:0/2146138794 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47cb61700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 0x7fc478157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 -- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 msgr2=0x7fc47815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 0x7fc47815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 -- 172.21.15.132:0/2146138794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 msgr2=0x7fc478157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 0x7fc478157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 -- 172.21.15.132:0/2146138794 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c009420 con 0x7fc4780b2360 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47d362700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 0x7fc47815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc47cb61700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 0x7fc478157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:17:58.175 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc477fff700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4640015a0 tx=0x7fc464001b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.164+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464007e90 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464007360 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47815a170 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464007700 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47815ac50 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc464004cc0 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc46401d790 con 0x7fc4780b2360 2023-10-06T11:17:58.176 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.168+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47815a5f0 con 0x7fc4780b2360 2023-10-06T11:17:58.178 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.172+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc464018070 con 0x7fc4780b2360 2023-10-06T11:17:58.246 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:58.243+0000 7f4bf4310000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:17:58.313 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.308+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba6dfc44-0d94-4db5-bc84-34c7777dc7e9", "id": 0} v 0) v1 -- 0x7fc478088840 con 0x7fc4780b2360 2023-10-06T11:17:58.318 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc474ff9700 1 -- 172.21.15.132:0/2146138794 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba6dfc44-0d94-4db5-bc84-34c7777dc7e9", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc46400ad10 con 0x7fc4780b2360 2023-10-06T11:17:58.318 INFO:teuthology.orchestra.run.smithi132.stdout:0 2023-10-06T11:17:58.319 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 msgr2=0x7fc47815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.319 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4640015a0 tx=0x7fc464001b50 comp rx=0 tx=0).stop 2023-10-06T11:17:58.319 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 shutdown_connections 2023-10-06T11:17:58.319 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc478159550 0x7fc47815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.320 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4780b2360 0x7fc47815d8a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.320 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 --2- 172.21.15.132:0/2146138794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4780aee00 0x7fc478157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.320 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 >> 172.21.15.132:0/2146138794 conn(0x7fc47806e780 msgr2=0x7fc478157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.320 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 shutdown_connections 2023-10-06T11:17:58.320 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.312+0000 7fc47edc5700 1 -- 172.21.15.132:0/2146138794 wait complete. 2023-10-06T11:17:58.322 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:58.319+0000 7f4bf4310000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:17:58.335 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:58.328+0000 7f96a6943000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:17:58.336 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new fe46237c-5610-4359-a899-959b16e568ae 1 2023-10-06T11:17:58.414 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:58.408+0000 7f96a6943000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:17:58.462 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:58.455+0000 7f4bf4310000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:17:58.476 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.472+0000 7fba0b1c5700 1 Processor -- start 2023-10-06T11:17:58.477 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.472+0000 7fba0b1c5700 1 -- start start 2023-10-06T11:17:58.480 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba0b1c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 0x7fba04067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba0b1c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba04067590 0x7fba040aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba0b1c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba040cbff0 con 0x7fba0415c600 2023-10-06T11:17:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba0b1c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040a9840 con 0x7fba040c8160 2023-10-06T11:17:58.481 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba0b1c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040aa0e0 con 0x7fba04067590 2023-10-06T11:17:58.483 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba04067590 0x7fba040aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.483 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba08f61700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fba0415c600 0x7fba040cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58476/0 (socket says 172.21.15.132:58476) 2023-10-06T11:17:58.483 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba08f61700 1 -- 172.21.15.132:0/2561795973 learned_addr learned my addr 172.21.15.132:0/2561795973 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-10-06T11:17:58.483 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba03fff700 1 --2- 172.21.15.132:0/2561795973 >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 0x7fba04067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.484 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba037fe700 1 -- 172.21.15.132:0/2561795973 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090021858 0 0) 0x7fb9f8001110 con 0x7fba0415c600 2023-10-06T11:17:58.484 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba037fe700 1 -- 172.21.15.132:0/2561795973 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec003570 con 0x7fba0415c600 2023-10-06T11:17:58.484 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 -- 172.21.15.132:0/2561795973 >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 msgr2=0x7fba04067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:58.484 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 --2- 172.21.15.132:0/2561795973 >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 0x7fba04067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.484 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 -- 172.21.15.132:0/2561795973 >> v1:172.21.15.132:6789/0 conn(0x7fba0415c600 legacy=0x7fba040cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 -- 172.21.15.132:0/2561795973 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040679c0 con 0x7fba04067590 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba03fff700 1 --2- 172.21.15.132:0/2561795973 >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 0x7fba04067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba09762700 1 --2- 172.21.15.132:0/2561795973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040aa570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba040c74b0 tx=0x7fb9fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d775c53f8c10257 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba037fe700 1 -- 172.21.15.132:0/2561795973 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00e070 con 0x7fba04067590 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba037fe700 1 -- 172.21.15.132:0/2561795973 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00bab0 con 0x7fba04067590 2023-10-06T11:17:58.485 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.476+0000 7fba037fe700 1 -- 172.21.15.132:0/2561795973 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00be70 con 0x7fba04067590 2023-10-06T11:17:58.487 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- 172.21.15.132:0/2561795973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 msgr2=0x7fba040aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/2561795973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040aa570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba040c74b0 tx=0x7fb9fc009780 comp rx=0 tx=0).stop 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- 172.21.15.132:0/2561795973 shutdown_connections 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/2561795973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040aa570 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/2561795973 >> v2:172.21.15.46:3300/0 conn(0x7fba040c8160 0x7fba04067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- 172.21.15.132:0/2561795973 >> 172.21.15.132:0/2561795973 conn(0x7fba040ccef0 msgr2=0x7fba040cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.488 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- 172.21.15.132:0/2561795973 shutdown_connections 2023-10-06T11:17:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- 172.21.15.132:0/2561795973 wait complete. 2023-10-06T11:17:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 Processor -- start 2023-10-06T11:17:58.489 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.480+0000 7fba0b1c5700 1 -- start start 2023-10-06T11:17:58.491 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba040c8160 0x7fba0414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba0415c600 0x7fba040b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c43b0 con 0x7fba040c8160 2023-10-06T11:17:58.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0414e060 con 0x7fba04067590 2023-10-06T11:17:58.492 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040b16d0 con 0x7fba0415c600 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60546/0 (socket says 172.21.15.132:60546) 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 -- 172.21.15.132:0/3337581199 learned_addr learned my addr 172.21.15.132:0/3337581199 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba09762700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba0415c600 0x7fba040b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba03fff700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba040c8160 0x7fba0414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.493 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 -- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba0415c600 msgr2=0x7fba040b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba0415c600 0x7fba040b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 -- 172.21.15.132:0/3337581199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba040c8160 msgr2=0x7fba0414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba040c8160 0x7fba0414db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 -- 172.21.15.132:0/3337581199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4009420 con 0x7fba04067590 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba08f61700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c590 tx=0x7fb9f8007c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8005b30 con 0x7fba04067590 2023-10-06T11:17:58.494 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8005cb0 con 0x7fba04067590 2023-10-06T11:17:58.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8007320 con 0x7fba04067590 2023-10-06T11:17:58.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc009a20 con 0x7fba04067590 2023-10-06T11:17:58.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb9f8012680 con 0x7fba04067590 2023-10-06T11:17:58.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba040b13f0 con 0x7fba04067590 2023-10-06T11:17:58.495 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.484+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fb9f8005e20 con 0x7fba04067590 2023-10-06T11:17:58.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.488+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0406d240 con 0x7fba04067590 2023-10-06T11:17:58.496 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.492+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb9f8017070 con 0x7fba04067590 2023-10-06T11:17:58.555 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:58.548+0000 7f96a6943000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:17:58.570 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.564+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe46237c-5610-4359-a899-959b16e568ae", "id": 1} v 0) v1 -- 0x7fba0404b5c0 con 0x7fba04067590 2023-10-06T11:17:58.578 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.572+0000 7fba00ff9700 1 -- 172.21.15.132:0/3337581199 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe46237c-5610-4359-a899-959b16e568ae", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba040cbff0 con 0x7fba04067590 2023-10-06T11:17:58.579 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 msgr2=0x7fba040b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c590 tx=0x7fb9f8007c70 comp rx=0 tx=0).stop 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 shutdown_connections 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba0415c600 0x7fba040b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba040c8160 0x7fba0414db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.582 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 --2- 172.21.15.132:0/3337581199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba04067590 0x7fba040b0b00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 >> 172.21.15.132:0/3337581199 conn(0x7fba040ccef0 msgr2=0x7fba040c32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 shutdown_connections 2023-10-06T11:17:58.583 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.576+0000 7fba0b1c5700 1 -- 172.21.15.132:0/3337581199 wait complete. 2023-10-06T11:17:58.600 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 9362b61f-06e8-402e-9afa-754a1cc5f1c4 2 2023-10-06T11:17:58.741 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 Processor -- start 2023-10-06T11:17:58.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 -- start start 2023-10-06T11:17:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f840a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f84159240 0x7f7f840c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f840c72d0 0x7f7f840c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f840c10a0 con 0x7f7f840c72d0 2023-10-06T11:17:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f840ca240 con 0x7f7f84159240 2023-10-06T11:17:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8ac8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f840ca520 con 0x7f7f84158e70 2023-10-06T11:17:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8948b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f84159240 0x7f7f840c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f89c8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f840a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8948b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f84159240 0x7f7f840c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60548/0 (socket says 172.21.15.132:60548) 2023-10-06T11:17:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.736+0000 7f7f8948b700 1 -- 172.21.15.132:0/3424430466 learned_addr learned my addr 172.21.15.132:0/3424430466 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8a48d700 1 --2- 172.21.15.132:0/3424430466 >> v2:172.21.15.46:3300/0 conn(0x7f7f840c72d0 0x7f7f840c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 -- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 msgr2=0x7f7f840a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f840a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 -- 172.21.15.132:0/3424430466 >> v2:172.21.15.46:3300/0 conn(0x7f7f840c72d0 msgr2=0x7f7f840c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 --2- 172.21.15.132:0/3424430466 >> v2:172.21.15.46:3300/0 conn(0x7f7f840c72d0 0x7f7f840c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 -- 172.21.15.132:0/3424430466 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f840c0a50 con 0x7f7f84159240 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f89c8c700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f840a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8948b700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84159240 0x7f7f840c99f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406c400 tx=0x7f7f74009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6794ec9b265a14a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f88c8a700 1 -- 172.21.15.132:0/3424430466 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400b640 con 0x7f7f84159240 2023-10-06T11:17:58.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f88c8a700 1 -- 172.21.15.132:0/3424430466 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400b7a0 con 0x7f7f84159240 2023-10-06T11:17:58.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f88c8a700 1 -- 172.21.15.132:0/3424430466 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400bb60 con 0x7f7f84159240 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84159240 msgr2=0x7f7f840c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84159240 0x7f7f840c99f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7f8406c400 tx=0x7f7f74009250 comp rx=0 tx=0).stop 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/3424430466 shutdown_connections 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f84159240 0x7f7f840c99f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/3424430466 >> v2:172.21.15.46:3300/0 conn(0x7f7f840c72d0 0x7f7f840c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/3424430466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f840a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/3424430466 >> 172.21.15.132:0/3424430466 conn(0x7f7f840cbc10 msgr2=0x7f7f840cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/3424430466 shutdown_connections 2023-10-06T11:17:58.748 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.740+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/3424430466 wait complete. 2023-10-06T11:17:58.749 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 Processor -- start 2023-10-06T11:17:58.749 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 -- start start 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f84156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 0x7f7f84169720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f84157f40 con 0x7f7f84159240 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f84153130 con 0x7f7f840c72d0 2023-10-06T11:17:58.753 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.744+0000 7f7f8ac8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8415a4e0 con 0x7f7f84158e70 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60558/0 (socket says 172.21.15.132:60558) 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 -- 172.21.15.132:0/38532872 learned_addr learned my addr 172.21.15.132:0/38532872 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8948b700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f84156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8a48d700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 0x7f7f84169720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 -- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 msgr2=0x7f7f84156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f84156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 -- 172.21.15.132:0/38532872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 msgr2=0x7f7f84169720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.754 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 0x7f7f84169720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 -- 172.21.15.132:0/38532872 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f80009420 con 0x7f7f840c72d0 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8a48d700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 0x7f7f84169720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8948b700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f84156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f89c8c700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ec80 tx=0x7f7f800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000c8f0 con 0x7f7f840c72d0 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8000ca50 con 0x7f7f840c72d0 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f740094f0 con 0x7f7f840c72d0 2023-10-06T11:17:58.755 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000cdd0 con 0x7f7f840c72d0 2023-10-06T11:17:58.756 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7f80013900 con 0x7f7f840c72d0 2023-10-06T11:17:58.756 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f84157610 con 0x7f7f840c72d0 2023-10-06T11:17:58.756 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7f80013d40 con 0x7f7f840c72d0 2023-10-06T11:17:58.756 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.748+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f84064590 con 0x7f7f840c72d0 2023-10-06T11:17:58.756 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:58.751+0000 7f4bf4310000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:17:58.759 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.752+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7f8003b6d0 con 0x7f7f840c72d0 2023-10-06T11:17:58.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:58.827+0000 7f4bf4310000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:17:58.834 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.828+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9362b61f-06e8-402e-9afa-754a1cc5f1c4", "id": 2} v 0) v1 -- 0x7f7f840c0d30 con 0x7f7f840c72d0 2023-10-06T11:17:58.843 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.836+0000 7f7f79ffb700 1 -- 172.21.15.132:0/38532872 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9362b61f-06e8-402e-9afa-754a1cc5f1c4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f8002a070 con 0x7f7f840c72d0 2023-10-06T11:17:58.843 INFO:teuthology.orchestra.run.smithi132.stdout:2 2023-10-06T11:17:58.845 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f96a6943000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:17:58.845 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 msgr2=0x7f7f841567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000ec80 tx=0x7f7f800091a0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 shutdown_connections 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f84159240 0x7f7f84169720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f84158e70 0x7f7f84156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 --2- 172.21.15.132:0/38532872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f840c72d0 0x7f7f841567c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 >> 172.21.15.132:0/38532872 conn(0x7f7f840cbc10 msgr2=0x7f7f840c1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 shutdown_connections 2023-10-06T11:17:58.846 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:58.840+0000 7f7f8ac8e700 1 -- 172.21.15.132:0/38532872 wait complete. 2023-10-06T11:17:58.862 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 7d6b6c9d-807d-4f23-be87-3e08f3536612 3 2023-10-06T11:17:58.916 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:58.908+0000 7f96a6943000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:17:59.007 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.000+0000 7f995c585700 1 Processor -- start 2023-10-06T11:17:59.007 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.000+0000 7f995c585700 1 -- start start 2023-10-06T11:17:59.008 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995c585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.009 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995c585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f995406ed10 0x7f99540c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.009 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995c585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540c1bf0 con 0x7f995406e8a0 2023-10-06T11:17:59.009 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995c585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99540abbe0 con 0x7f995406ed10 2023-10-06T11:17:59.009 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995c585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995406cf80 con 0x7f99540c6470 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:34548/0 (socket says 172.21.15.132:34548) 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 -- 172.21.15.132:0/3978221177 learned_addr learned my addr 172.21.15.132:0/3978221177 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995ab22700 1 --2- 172.21.15.132:0/3978221177 >> v2:172.21.15.46:3300/0 conn(0x7f995406ed10 0x7f99540c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 -- 172.21.15.132:0/3978221177 >> v2:172.21.15.46:3300/0 conn(0x7f995406ed10 msgr2=0x7f99540c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 --2- 172.21.15.132:0/3978221177 >> v2:172.21.15.46:3300/0 conn(0x7f995406ed10 0x7f99540c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 -- 172.21.15.132:0/3978221177 >> v1:172.21.15.46:6789/0 conn(0x7f995406e8a0 legacy=0x7f99540c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:59.010 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 -- 172.21.15.132:0/3978221177 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995406d260 con 0x7f99540c6470 2023-10-06T11:17:59.011 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995a321700 1 --2- 172.21.15.132:0/3978221177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99500099b0 tx=0x7f9950009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebfd623360828197 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.011 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995931f700 1 -- 172.21.15.132:0/3978221177 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000b820 con 0x7f99540c6470 2023-10-06T11:17:59.011 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995931f700 1 -- 172.21.15.132:0/3978221177 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000b980 con 0x7f99540c6470 2023-10-06T11:17:59.011 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.004+0000 7f995931f700 1 -- 172.21.15.132:0/3978221177 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000bd20 con 0x7f99540c6470 2023-10-06T11:17:59.012 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- 172.21.15.132:0/3978221177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 msgr2=0x7f99540ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.012 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- 172.21.15.132:0/3978221177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99500099b0 tx=0x7f9950009250 comp rx=0 tx=0).stop 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- 172.21.15.132:0/3978221177 shutdown_connections 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- 172.21.15.132:0/3978221177 >> v2:172.21.15.46:3300/0 conn(0x7f995406ed10 0x7f99540c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- 172.21.15.132:0/3978221177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99540c6470 0x7f99540ab6a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- 172.21.15.132:0/3978221177 >> 172.21.15.132:0/3978221177 conn(0x7f99540cbe30 msgr2=0x7f99540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- 172.21.15.132:0/3978221177 shutdown_connections 2023-10-06T11:17:59.013 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- 172.21.15.132:0/3978221177 wait complete. 2023-10-06T11:17:59.014 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 Processor -- start 2023-10-06T11:17:59.014 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 -- start start 2023-10-06T11:17:59.016 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.016 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 0x7f99540ac400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.008+0000 7f995c585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99540c6470 0x7f99540ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995415b2f0 con 0x7f99540c6470 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ca220 con 0x7f995406e8a0 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995415a6e0 con 0x7f995406ed10 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60572/0 (socket says 172.21.15.132:60572) 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 -- 172.21.15.132:0/1929580134 learned_addr learned my addr 172.21.15.132:0/1929580134 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f9959b20700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 0x7f99540ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995ab22700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99540c6470 0x7f99540ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 -- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 msgr2=0x7f99540ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.017 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 0x7f99540ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.018 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 -- 172.21.15.132:0/1929580134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99540c6470 msgr2=0x7f99540ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.018 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99540c6470 0x7f99540ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.018 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 -- 172.21.15.132:0/1929580134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500094f0 con 0x7f995406e8a0 2023-10-06T11:17:59.018 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f9959b20700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 0x7f99540ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995a321700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9950009290 tx=0x7f9950016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950011ca0 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950020070 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995001b6a0 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995415a9c0 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995415af80 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9950022030 con 0x7f995406e8a0 2023-10-06T11:17:59.019 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f995001bc90 con 0x7f995406e8a0 2023-10-06T11:17:59.020 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.012+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99540660f0 con 0x7f995406e8a0 2023-10-06T11:17:59.022 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.016+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f99500456d0 con 0x7f995406e8a0 2023-10-06T11:17:59.058 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:59.051+0000 7f4bf4310000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:17:59.101 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.096+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7d6b6c9d-807d-4f23-be87-3e08f3536612", "id": 3} v 0) v1 -- 0x7f99540660f0 con 0x7f995406e8a0 2023-10-06T11:17:59.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.104+0000 7f99467fc700 1 -- 172.21.15.132:0/1929580134 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d6b6c9d-807d-4f23-be87-3e08f3536612", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9950034070 con 0x7f995406e8a0 2023-10-06T11:17:59.111 INFO:teuthology.orchestra.run.smithi132.stdout:3 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 msgr2=0x7f99540cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9950009290 tx=0x7f9950016000 comp rx=0 tx=0).stop 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 shutdown_connections 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99540c6470 0x7f99540ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f995406ed10 0x7f99540ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 --2- 172.21.15.132:0/1929580134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f995406e8a0 0x7f99540cb260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 >> 172.21.15.132:0/1929580134 conn(0x7f99540cbe30 msgr2=0x7f99540aa150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 shutdown_connections 2023-10-06T11:17:59.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.108+0000 7f995c585700 1 -- 172.21.15.132:0/1929580134 wait complete. 2023-10-06T11:17:59.128 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new d3bb5b7f-c6c9-4975-99ad-8585b870fe34 4 2023-10-06T11:17:59.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:59.139+0000 7f4bf4310000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:17:59.155 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:59.148+0000 7f96a6943000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:17:59.234 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:59.228+0000 7f96a6943000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:17:59.235 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:59.231+0000 7f4bf4310000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:17:59.278 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.272+0000 7fddf9345700 1 Processor -- start 2023-10-06T11:17:59.278 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.272+0000 7fddf9345700 1 -- start start 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 0x7fddf40b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40ccc10 con 0x7fddf41571b0 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40cc000 con 0x7fddf4156de0 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40cc2e0 con 0x7fddf40b3400 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf2ffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fddf4156de0 0x7fddf40cb7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58478/0 (socket says 172.21.15.132:58478) 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf2ffd700 1 -- 172.21.15.132:0/3347704743 learned_addr learned my addr 172.21.15.132:0/3347704743 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-10-06T11:17:59.280 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf37fe700 1 --2- 172.21.15.132:0/3347704743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 0x7fddf40b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.281 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf37fe700 1 -- 172.21.15.132:0/3347704743 >> v1:172.21.15.132:6789/0 conn(0x7fddf4156de0 legacy=0x7fddf40cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.281 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf37fe700 1 -- 172.21.15.132:0/3347704743 >> v1:172.21.15.46:6789/0 conn(0x7fddf41571b0 legacy=0x7fddf40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf37fe700 1 -- 172.21.15.132:0/3347704743 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40cc5c0 con 0x7fddf40b3400 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf27fc700 1 -- 172.21.15.132:0/3347704743 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142169880 0 0) 0x7fddf40cc000 con 0x7fddf4156de0 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf37fe700 1 --2- 172.21.15.132:0/3347704743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 0x7fddf40b1cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fddec009e90 tx=0x7fddec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0712fdc4c6f7f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf27fc700 1 -- 172.21.15.132:0/3347704743 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00e070 con 0x7fddf40b3400 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf27fc700 1 -- 172.21.15.132:0/3347704743 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00bc30 con 0x7fddf40b3400 2023-10-06T11:17:59.282 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf27fc700 1 -- 172.21.15.132:0/3347704743 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec012430 con 0x7fddf40b3400 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- 172.21.15.132:0/3347704743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 msgr2=0x7fddf40b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 --2- 172.21.15.132:0/3347704743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 0x7fddf40b1cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fddec009e90 tx=0x7fddec009780 comp rx=0 tx=0).stop 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- 172.21.15.132:0/3347704743 shutdown_connections 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 --2- 172.21.15.132:0/3347704743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf40b3400 0x7fddf40b1cb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- 172.21.15.132:0/3347704743 >> 172.21.15.132:0/3347704743 conn(0x7fddf406af10 msgr2=0x7fddf406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- 172.21.15.132:0/3347704743 shutdown_connections 2023-10-06T11:17:59.283 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.276+0000 7fddf9345700 1 -- 172.21.15.132:0/3347704743 wait complete. 2023-10-06T11:17:59.284 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 Processor -- start 2023-10-06T11:17:59.284 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- start start 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf4156de0 0x7fddf415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf41571b0 0x7fddf415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf41533e0 con 0x7fddf41571b0 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf406a170 con 0x7fddf40b3400 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf41527d0 con 0x7fddf4156de0 2023-10-06T11:17:59.286 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.287 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddebfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf4156de0 0x7fddf415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.287 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60576/0 (socket says 172.21.15.132:60576) 2023-10-06T11:17:59.287 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 -- 172.21.15.132:0/4087763754 learned_addr learned my addr 172.21.15.132:0/4087763754 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.287 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf37fe700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf41571b0 0x7fddf415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 -- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf4156de0 msgr2=0x7fddf415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf4156de0 0x7fddf415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 -- 172.21.15.132:0/4087763754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf41571b0 msgr2=0x7fddf415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.288 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf41571b0 0x7fddf415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 -- 172.21.15.132:0/4087763754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009a20 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf2ffd700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c810 tx=0x7fdde4007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4005c00 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4005d60 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4007320 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf4152ab0 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.280+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf4153070 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.284+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fdde401c6c0 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.284+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fdde4007750 con 0x7fddf40b3400 2023-10-06T11:17:59.289 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.284+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf40b17d0 con 0x7fddf40b3400 2023-10-06T11:17:59.291 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.284+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fddf40b17d0 con 0x7fddf40b3400 2023-10-06T11:17:59.320 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:59.316+0000 7f96a6943000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:17:59.370 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.364+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d3bb5b7f-c6c9-4975-99ad-8585b870fe34", "id": 4} v 0) v1 -- 0x7fddf4156960 con 0x7fddf40b3400 2023-10-06T11:17:59.379 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.372+0000 7fddeaffd700 1 -- 172.21.15.132:0/4087763754 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d3bb5b7f-c6c9-4975-99ad-8585b870fe34", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdde4017070 con 0x7fddf40b3400 2023-10-06T11:17:59.379 INFO:teuthology.orchestra.run.smithi132.stdout:4 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 msgr2=0x7fddf413c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c810 tx=0x7fdde4007d90 comp rx=0 tx=0).stop 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 shutdown_connections 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf41571b0 0x7fddf415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddf4156de0 0x7fddf415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 --2- 172.21.15.132:0/4087763754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddf40b3400 0x7fddf413c5b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 >> 172.21.15.132:0/4087763754 conn(0x7fddf406af10 msgr2=0x7fddf40aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 shutdown_connections 2023-10-06T11:17:59.381 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.376+0000 7fddf9345700 1 -- 172.21.15.132:0/4087763754 wait complete. 2023-10-06T11:17:59.397 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 6cb5c16b-a678-4717-a443-5bdb45be8e0a 5 2023-10-06T11:17:59.591 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.584+0000 7f8df948e700 1 Processor -- start 2023-10-06T11:17:59.591 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.584+0000 7f8df948e700 1 -- start start 2023-10-06T11:17:59.594 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df948e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 0x7f8df4069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.594 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df948e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40ce6e0 con 0x7f8df4158aa0 2023-10-06T11:17:59.594 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df948e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40cddb0 con 0x7f8df4159850 2023-10-06T11:17:59.594 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df948e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df415a5c0 con 0x7f8df4069270 2023-10-06T11:17:59.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df8c8d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8df4159850 0x7f8df4159ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58494/0 (socket says 172.21.15.132:58494) 2023-10-06T11:17:59.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df8c8d700 1 -- 172.21.15.132:0/2200361278 learned_addr learned my addr 172.21.15.132:0/2200361278 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-10-06T11:17:59.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df3fff700 1 --2- 172.21.15.132:0/2200361278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 0x7f8df4069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df2ffd700 1 -- 172.21.15.132:0/2200361278 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870877294 0 0) 0x7f8df40cddb0 con 0x7f8df4159850 2023-10-06T11:17:59.595 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df2ffd700 1 -- 172.21.15.132:0/2200361278 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc003570 con 0x7f8df4159850 2023-10-06T11:17:59.596 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df3fff700 1 -- 172.21.15.132:0/2200361278 >> v1:172.21.15.132:6789/0 conn(0x7f8df4159850 legacy=0x7f8df4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.596 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df3fff700 1 -- 172.21.15.132:0/2200361278 >> v1:172.21.15.46:6789/0 conn(0x7f8df4158aa0 legacy=0x7f8df4158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.596 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.588+0000 7f8df3fff700 1 -- 172.21.15.132:0/2200361278 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40ce090 con 0x7f8df4069270 2023-10-06T11:17:59.596 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df3fff700 1 --2- 172.21.15.132:0/2200361278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 0x7f8df4069640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009e90 tx=0x7f8de4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59cf42dd85220f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.597 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df2ffd700 1 -- 172.21.15.132:0/2200361278 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400e070 con 0x7f8df4069270 2023-10-06T11:17:59.597 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df2ffd700 1 -- 172.21.15.132:0/2200361278 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de400bc30 con 0x7f8df4069270 2023-10-06T11:17:59.597 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df2ffd700 1 -- 172.21.15.132:0/2200361278 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4012430 con 0x7f8df4069270 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 -- 172.21.15.132:0/2200361278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 msgr2=0x7f8df4069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 --2- 172.21.15.132:0/2200361278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 0x7f8df4069640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009e90 tx=0x7f8de4009780 comp rx=0 tx=0).stop 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 -- 172.21.15.132:0/2200361278 shutdown_connections 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 --2- 172.21.15.132:0/2200361278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4069270 0x7f8df4069640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 -- 172.21.15.132:0/2200361278 >> 172.21.15.132:0/2200361278 conn(0x7f8df4068390 msgr2=0x7f8df40d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 -- 172.21.15.132:0/2200361278 shutdown_connections 2023-10-06T11:17:59.598 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.592+0000 7f8df948e700 1 -- 172.21.15.132:0/2200361278 wait complete. 2023-10-06T11:17:59.600 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 Processor -- start 2023-10-06T11:17:59.600 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 -- start start 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df4069270 0x7f8df40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df4158aa0 0x7f8df416ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40c76a0 con 0x7f8df4069270 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40b11a0 con 0x7f8df4158aa0 2023-10-06T11:17:59.602 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df948e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40b1480 con 0x7f8df4159850 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df4158aa0 0x7f8df416ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:34584/0 (socket says 172.21.15.132:34584) 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 -- 172.21.15.132:0/304618654 learned_addr learned my addr 172.21.15.132:0/304618654 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df3fff700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df4069270 0x7f8df40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 -- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df4158aa0 msgr2=0x7f8df416ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.604 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df4158aa0 0x7f8df416ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 -- 172.21.15.132:0/304618654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df4069270 msgr2=0x7f8df40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df4069270 0x7f8df40b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 -- 172.21.15.132:0/304618654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4009a20 con 0x7f8df4159850 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.596+0000 7f8df8c8d700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0010e0 tx=0x7f8dec007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec005c00 con 0x7f8df4159850 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec005d60 con 0x7f8df4159850 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df40b1760 con 0x7f8df4159850 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0072e0 con 0x7f8df4159850 2023-10-06T11:17:59.605 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df41554d0 con 0x7f8df4159850 2023-10-06T11:17:59.606 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8dec01c6c0 con 0x7f8df4159850 2023-10-06T11:17:59.606 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f8dec0076d0 con 0x7f8df4159850 2023-10-06T11:17:59.606 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.600+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df4064590 con 0x7f8df4159850 2023-10-06T11:17:59.610 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.604+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8dec021060 con 0x7f8df4159850 2023-10-06T11:17:59.689 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.684+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cb5c16b-a678-4717-a443-5bdb45be8e0a", "id": 5} v 0) v1 -- 0x7f8df4088380 con 0x7f8df4159850 2023-10-06T11:17:59.699 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.692+0000 7f8ddbfff700 1 -- 172.21.15.132:0/304618654 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cb5c16b-a678-4717-a443-5bdb45be8e0a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8dec017070 con 0x7f8df4159850 2023-10-06T11:17:59.699 INFO:teuthology.orchestra.run.smithi132.stdout:5 2023-10-06T11:17:59.700 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 msgr2=0x7f8df416d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0010e0 tx=0x7f8dec007dd0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 shutdown_connections 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8df4159850 0x7f8df416d1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df4158aa0 0x7f8df416ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 --2- 172.21.15.132:0/304618654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df4069270 0x7f8df40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 >> 172.21.15.132:0/304618654 conn(0x7f8df4068390 msgr2=0x7f8df40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 shutdown_connections 2023-10-06T11:17:59.701 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.696+0000 7f8df948e700 1 -- 172.21.15.132:0/304618654 wait complete. 2023-10-06T11:17:59.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new a60ee872-6821-4158-876c-68a1353a6c5f 6 2023-10-06T11:17:59.727 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:59.723+0000 7f4bf4310000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:17:59.805 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:59.800+0000 7f96a6943000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:17:59.853 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.848+0000 7fd54846e700 1 Processor -- start 2023-10-06T11:17:59.853 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.848+0000 7fd54846e700 1 -- start start 2023-10-06T11:17:59.860 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.861 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5400ca430 0x7fd5400c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.861 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.861 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5400c1630 con 0x7fd5400ca430 2023-10-06T11:17:59.861 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5400c0d00 con 0x7fd5400c7d80 2023-10-06T11:17:59.861 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c0fe0 con 0x7fd5400a9840 2023-10-06T11:17:59.862 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54620a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.862 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.862 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:34586/0 (socket says 172.21.15.132:34586) 2023-10-06T11:17:59.862 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54620a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:60590/0 (socket says 172.21.15.132:60590) 2023-10-06T11:17:59.862 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 -- 172.21.15.132:0/682821105 learned_addr learned my addr 172.21.15.132:0/682821105 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd545a09700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.46:3300/0 conn(0x7fd5400ca430 0x7fd5400c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 -- 172.21.15.132:0/682821105 >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 msgr2=0x7fd5400c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 -- 172.21.15.132:0/682821105 >> v2:172.21.15.46:3300/0 conn(0x7fd5400ca430 msgr2=0x7fd5400c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.46:3300/0 conn(0x7fd5400ca430 0x7fd5400c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.863 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 -- 172.21.15.132:0/682821105 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400c12c0 con 0x7fd5400a9840 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54620a700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd546a0b700 1 --2- 172.21.15.132:0/682821105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd54006c640 tx=0x7fd538009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f3de3e0b3816a55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd545208700 1 -- 172.21.15.132:0/682821105 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800b640 con 0x7fd5400a9840 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd545208700 1 -- 172.21.15.132:0/682821105 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800b7a0 con 0x7fd5400a9840 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd545208700 1 -- 172.21.15.132:0/682821105 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800bb60 con 0x7fd5400a9840 2023-10-06T11:17:59.864 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 -- 172.21.15.132:0/682821105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 msgr2=0x7fd5400c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.852+0000 7fd54846e700 1 --2- 172.21.15.132:0/682821105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd54006c640 tx=0x7fd538009250 comp rx=0 tx=0).stop 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- 172.21.15.132:0/682821105 shutdown_connections 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- 172.21.15.132:0/682821105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5400c1ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.46:3300/0 conn(0x7fd5400ca430 0x7fd5400c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- 172.21.15.132:0/682821105 >> v2:172.21.15.132:3300/0 conn(0x7fd5400c7d80 0x7fd5400c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.865 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- 172.21.15.132:0/682821105 >> 172.21.15.132:0/682821105 conn(0x7fd5400cbe30 msgr2=0x7fd5400cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.866 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- 172.21.15.132:0/682821105 shutdown_connections 2023-10-06T11:17:59.866 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- 172.21.15.132:0/682821105 wait complete. 2023-10-06T11:17:59.870 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 Processor -- start 2023-10-06T11:17:59.870 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- start start 2023-10-06T11:17:59.870 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.870 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 0x7fd54015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 0x7fd5400ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540152b20 con 0x7fd5400c7d80 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400aec70 con 0x7fd5400ca430 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54846e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400aef50 con 0x7fd5400a9840 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54620a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.871 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd546a0b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 0x7fd5400ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.872 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.856+0000 7fd54620a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:34588/0 (socket says 172.21.15.132:34588) 2023-10-06T11:17:59.872 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd54620a700 1 -- 172.21.15.132:0/1146504951 learned_addr learned my addr 172.21.15.132:0/1146504951 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:17:59.872 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd545a09700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 0x7fd54015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:17:59.872 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 -- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 msgr2=0x7fd5401525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.872 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 -- 172.21.15.132:0/1146504951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 msgr2=0x7fd54015cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 0x7fd54015cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 -- 172.21.15.132:0/1146504951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c009a20 con 0x7fd5400ca430 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd545a09700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 0x7fd54015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd54620a700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:17:59.873 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd546a0b700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 0x7fd5400ac0b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5400cba50 tx=0x7fd538017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800bb20 con 0x7fd5400ca430 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538011420 con 0x7fd5400ca430 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538011800 con 0x7fd5400ca430 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5380094f0 con 0x7fd5400ca430 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5400af510 con 0x7fd5400ca430 2023-10-06T11:17:59.874 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd53802e400 con 0x7fd5400ca430 2023-10-06T11:17:59.875 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fd538006d40 con 0x7fd5400ca430 2023-10-06T11:17:59.875 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.860+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5400660f0 con 0x7fd5400ca430 2023-10-06T11:17:59.875 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.864+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd538015020 con 0x7fd5400ca430 2023-10-06T11:17:59.899 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:17:59.895+0000 7f4bf4310000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:17:59.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.936+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a60ee872-6821-4158-876c-68a1353a6c5f", "id": 6} v 0) v1 -- 0x7fd54004b5c0 con 0x7fd5400ca430 2023-10-06T11:17:59.952 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd5367fc700 1 -- 172.21.15.132:0/1146504951 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a60ee872-6821-4158-876c-68a1353a6c5f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd538011e30 con 0x7fd5400ca430 2023-10-06T11:17:59.953 INFO:teuthology.orchestra.run.smithi132.stdout:6 2023-10-06T11:17:59.954 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 msgr2=0x7fd5400ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:17:59.954 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 0x7fd5400ac0b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5400cba50 tx=0x7fd538017000 comp rx=0 tx=0).stop 2023-10-06T11:17:59.954 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 shutdown_connections 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5400ca430 0x7fd5400ac0b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5400c7d80 0x7fd54015cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 --2- 172.21.15.132:0/1146504951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5400a9840 0x7fd5401525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 >> 172.21.15.132:0/1146504951 conn(0x7fd5400cbe30 msgr2=0x7fd54006fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 shutdown_connections 2023-10-06T11:17:59.955 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:17:59.948+0000 7fd54846e700 1 -- 172.21.15.132:0/1146504951 wait complete. 2023-10-06T11:17:59.966 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:17:59.960+0000 7f96a6943000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:17:59.971 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 19b3420f-679c-4173-baa5-e6418959fd88 7 2023-10-06T11:18:00.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.104+0000 7f317328e700 1 Processor -- start 2023-10-06T11:18:00.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.104+0000 7f317328e700 1 -- start start 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317328e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f316c0adbe0 0x7f316c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317328e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0afd20 con 0x7f316c0d0d60 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317328e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0b0000 con 0x7f316c0b0dc0 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317328e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c159740 con 0x7f316c0adbe0 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f316c0b0dc0 0x7f316c0b11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:58504/0 (socket says 172.21.15.132:58504) 2023-10-06T11:18:00.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 -- 172.21.15.132:0/2090567309 learned_addr learned my addr 172.21.15.132:0/2090567309 (peer_addr_for_me v1:172.21.15.132:0/0) 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317228c700 1 --2- 172.21.15.132:0/2090567309 >> v2:172.21.15.132:3300/0 conn(0x7f316c0adbe0 0x7f316c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684654494 0 0) 0x7f316c0b0000 con 0x7f316c0b0dc0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3158003570 con 0x7f316c0b0dc0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2343634025 0 0) 0x7f3158003570 con 0x7f316c0b0dc0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 >> v2:172.21.15.132:3300/0 conn(0x7f316c0adbe0 msgr2=0x7f316c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 --2- 172.21.15.132:0/2090567309 >> v2:172.21.15.132:3300/0 conn(0x7f316c0adbe0 0x7f316c0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 >> v1:172.21.15.46:6789/0 conn(0x7f316c0d0d60 legacy=0x7f316c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0b02e0 con 0x7f316c0b0dc0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f31640017f0 con 0x7f316c0b0dc0 2023-10-06T11:18:00.116 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 >> v1:172.21.15.132:6789/0 conn(0x7f316c0b0dc0 legacy=0x7f316c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3158003a60 0x7f3158003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158004810 0x7f3158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 0x7f315800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315800c0a0 con 0x7f3158004810 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315800c210 con 0x7f31580051c0 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315800c3d0 con 0x7f3158003a60 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 0x7f315800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f317228c700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3158003a60 0x7f3158003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3171a8b700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158004810 0x7f3158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 -- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3158003a60 msgr2=0x7f3158003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3158003a60 0x7f3158003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.117 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 -- 172.21.15.132:0/2090567309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158004810 msgr2=0x7f3158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158004810 0x7f3158004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 -- 172.21.15.132:0/2090567309 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158003570 con 0x7f31580051c0 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 -- 172.21.15.132:0/2090567309 reap_dead start 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.108+0000 7f3172a8d700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 0x7f315800ba60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f316400a930 tx=0x7f31640021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46af117e048fd786 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640070d0 con 0x7f31580051c0 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164007230 con 0x7f31580051c0 2023-10-06T11:18:00.118 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317128a700 1 -- 172.21.15.132:0/2090567309 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640075d0 con 0x7f31580051c0 2023-10-06T11:18:00.119 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 msgr2=0x7f315800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 0x7f315800ba60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f316400a930 tx=0x7f31640021d0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- 172.21.15.132:0/2090567309 shutdown_connections 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 --2- 172.21.15.132:0/2090567309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31580051c0 0x7f315800ba60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- 172.21.15.132:0/2090567309 >> 172.21.15.132:0/2090567309 conn(0x7f316c06ef60 msgr2=0x7f316c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- 172.21.15.132:0/2090567309 shutdown_connections 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- 172.21.15.132:0/2090567309 wait complete. 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 Processor -- start 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- start start 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 0x7f316c143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.120 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f316c15bc20 0x7f316c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c15a950 con 0x7f316c14d920 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c15b790 con 0x7f316c15bc20 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317328e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c14e130 con 0x7f31580051c0 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317228c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317228c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:34604/0 (socket says 172.21.15.132:34604) 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.112+0000 7f317228c700 1 -- 172.21.15.132:0/3384730471 learned_addr learned my addr 172.21.15.132:0/3384730471 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-06T11:18:00.121 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f3172a8d700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f316c15bc20 0x7f316c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.122 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f3171a8b700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 0x7f316c143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.122 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 -- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f316c15bc20 msgr2=0x7f316c068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.122 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f316c15bc20 0x7f316c068b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 -- 172.21.15.132:0/3384730471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 msgr2=0x7f316c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 0x7f316c143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 -- 172.21.15.132:0/3384730471 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c15c0a0 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f3171a8b700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 0x7f316c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317228c700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3168011a90 tx=0x7f31680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800fa10 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316800fb70 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c15ab80 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168014440 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c15b140 con 0x7f31580051c0 2023-10-06T11:18:00.123 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3168014af0 con 0x7f31580051c0 2023-10-06T11:18:00.124 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f3168018030 con 0x7f31580051c0 2023-10-06T11:18:00.124 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.116+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c064590 con 0x7f31580051c0 2023-10-06T11:18:00.126 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.120+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3168022060 con 0x7f31580051c0 2023-10-06T11:18:00.204 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.200+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "19b3420f-679c-4173-baa5-e6418959fd88", "id": 7} v 0) v1 -- 0x7f316c0675d0 con 0x7f31580051c0 2023-10-06T11:18:00.213 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f31627fc700 1 -- 172.21.15.132:0/3384730471 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19b3420f-679c-4173-baa5-e6418959fd88", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f316802b0b0 con 0x7f31580051c0 2023-10-06T11:18:00.214 INFO:teuthology.orchestra.run.smithi132.stdout:7 2023-10-06T11:18:00.215 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 msgr2=0x7f316c1523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.215 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3168011a90 tx=0x7f31680091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 shutdown_connections 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f316c15bc20 0x7f316c068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f316c14d920 0x7f316c143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 --2- 172.21.15.132:0/3384730471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31580051c0 0x7f316c1523b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 >> 172.21.15.132:0/3384730471 conn(0x7f316c06ef60 msgr2=0x7f316c0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 shutdown_connections 2023-10-06T11:18:00.217 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-06T11:18:00.208+0000 7f317328e700 1 -- 172.21.15.132:0/3384730471 wait complete. 2023-10-06T11:18:00.234 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T11:18:00.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T11:18:00.242 INFO:tasks.ceph.osd.0:Started 2023-10-06T11:18:00.242 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T11:18:00.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T11:18:00.247 INFO:tasks.ceph.osd.1:Started 2023-10-06T11:18:00.247 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T11:18:00.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T11:18:00.252 INFO:tasks.ceph.osd.2:Started 2023-10-06T11:18:00.252 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T11:18:00.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T11:18:00.258 INFO:tasks.ceph.osd.3:Started 2023-10-06T11:18:00.259 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T11:18:00.259 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T11:18:00.266 INFO:tasks.ceph.osd.4:Started 2023-10-06T11:18:00.266 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T11:18:00.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T11:18:00.270 INFO:tasks.ceph.osd.5:Started 2023-10-06T11:18:00.271 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T11:18:00.271 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T11:18:00.275 INFO:tasks.ceph.osd.6:Started 2023-10-06T11:18:00.275 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T11:18:00.275 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T11:18:00.281 INFO:tasks.ceph.osd.7:Started 2023-10-06T11:18:00.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:00.296 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:00.291+0000 7f4bf4310000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:18:00.375 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:00.368+0000 7f96a6943000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:18:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.451+0000 7f7ba5585700 1 Processor -- start 2023-10-06T11:18:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.451+0000 7f7ba5585700 1 -- start start 2023-10-06T11:18:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7ba5585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ba00d0aa0 0x7f7ba0159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7ba5585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba00b5830 con 0x7f7ba00b5410 2023-10-06T11:18:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7ba5585700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba00b4f80 con 0x7f7ba0159050 2023-10-06T11:18:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7ba5585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00b40c0 con 0x7f7ba00d0aa0 2023-10-06T11:18:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ba00b5410 0x7f7ba00b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38340/0 (socket says 172.21.15.46:38340) 2023-10-06T11:18:00.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9f7fe700 1 -- 172.21.15.46:0/1759391807 learned_addr learned my addr 172.21.15.46:0/1759391807 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9e7fc700 1 --2- 172.21.15.46:0/1759391807 >> v2:172.21.15.46:3300/0 conn(0x7f7ba00d0aa0 0x7f7ba0159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817718130 0 0) 0x7f7ba00b5830 con 0x7f7ba00b5410 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b88003570 con 0x7f7ba00b5410 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951458845 0 0) 0x7f7ba00b4f80 con 0x7f7ba0159050 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba00b5830 con 0x7f7ba0159050 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 484966885 0 0) 0x7f7b88003570 con 0x7f7ba00b5410 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 >> v2:172.21.15.46:3300/0 conn(0x7f7ba00d0aa0 msgr2=0x7f7ba0159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 --2- 172.21.15.46:0/1759391807 >> v2:172.21.15.46:3300/0 conn(0x7f7ba00d0aa0 0x7f7ba0159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 >> v1:172.21.15.132:6789/0 conn(0x7f7ba0159050 legacy=0x7f7ba00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00b4370 con 0x7f7ba00b5410 2023-10-06T11:18:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f7b980017f0 con 0x7f7ba00b5410 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 >> v1:172.21.15.46:6789/0 conn(0x7f7ba00b5410 legacy=0x7f7ba00b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 0x7f7b88003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b88004810 0x7f7b88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b880051c0 0x7f7b8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00b40c0 con 0x7f7b88003bf0 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8800c0b0 con 0x7f7b88004810 2023-10-06T11:18:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8800c270 con 0x7f7b880051c0 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 0x7f7b88003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 -- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b880051c0 msgr2=0x7f7b8800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b880051c0 0x7f7b8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 -- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b88004810 msgr2=0x7f7b88004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b88004810 0x7f7b88004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9f7fe700 1 -- 172.21.15.46:0/1759391807 reap_dead start 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 -- 172.21.15.46:0/1759391807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b88003570 con 0x7f7b88003bf0 2023-10-06T11:18:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9effd700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 0x7f7b88003fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7b90001a10 tx=0x7f7b90007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2068d884851b0858 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:00.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000c5d0 con 0x7f7b88003bf0 2023-10-06T11:18:00.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000c730 con 0x7f7b88003bf0 2023-10-06T11:18:00.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.455+0000 7f7b9dffb700 1 -- 172.21.15.46:0/1759391807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000cad0 con 0x7f7b88003bf0 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 -- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 msgr2=0x7f7b88003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 0x7f7b88003fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7b90001a10 tx=0x7f7b90007d30 comp rx=0 tx=0).stop 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 -- 172.21.15.46:0/1759391807 shutdown_connections 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 --2- 172.21.15.46:0/1759391807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b88003bf0 0x7f7b88003fe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 -- 172.21.15.46:0/1759391807 >> 172.21.15.46:0/1759391807 conn(0x7f7ba006e780 msgr2=0x7f7ba006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 -- 172.21.15.46:0/1759391807 shutdown_connections 2023-10-06T11:18:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.459+0000 7f7ba5585700 1 -- 172.21.15.46:0/1759391807 wait complete. 2023-10-06T11:18:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 Processor -- start 2023-10-06T11:18:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- start start 2023-10-06T11:18:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b88003bf0 0x7f7ba0159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba015bb70 0x7f7ba015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00c3dc0 con 0x7f7ba0157060 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01549b0 con 0x7f7ba015bb70 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba015c520 con 0x7f7b88003bf0 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43138/0 (socket says 172.21.15.46:43138) 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 -- 172.21.15.46:0/739574013 learned_addr learned my addr 172.21.15.46:0/739574013 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 -- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b88003bf0 msgr2=0x7f7ba0159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b88003bf0 0x7f7ba0159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 -- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba015bb70 msgr2=0x7f7ba015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba015bb70 0x7f7ba015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 -- 172.21.15.46:0/739574013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0157930 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b9e7fc700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9d0 tx=0x7f7b940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400c7f0 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9400c970 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400cd30 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba0157c10 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7b94013900 con 0x7f7ba0157060 2023-10-06T11:18:00.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.463+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01581d0 con 0x7f7ba0157060 2023-10-06T11:18:00.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.467+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f7b94013d40 con 0x7f7ba0157060 2023-10-06T11:18:00.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.467+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0088820 con 0x7f7ba0157060 2023-10-06T11:18:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.471+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7b94021060 con 0x7f7ba0157060 2023-10-06T11:18:00.535 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:00.531+0000 7f4bf4310000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:18:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ba00b4650 con 0x7f7ba0157060 2023-10-06T11:18:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7b86ffd700 1 -- 172.21.15.46:0/739574013 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f7b94017070 con 0x7f7ba0157060 2023-10-06T11:18:00.540 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:00.540 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:00.203519+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 msgr2=0x7f7ba015b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400e9d0 tx=0x7f7b940091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 shutdown_connections 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ba015bb70 0x7f7ba015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0157060 0x7f7ba015b630 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 --2- 172.21.15.46:0/739574013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b88003bf0 0x7f7ba0159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 >> 172.21.15.46:0/739574013 conn(0x7f7ba006e780 msgr2=0x7f7ba00cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 shutdown_connections 2023-10-06T11:18:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:00.535+0000 7f7ba5585700 1 -- 172.21.15.46:0/739574013 wait complete. 2023-10-06T11:18:00.559 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T11:18:00.560 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T11:18:00.604 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:00.596+0000 7f96a6943000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:18:00.631 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:00.627+0000 7f4bf4310000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:18:00.700 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:00.692+0000 7f96a6943000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:18:00.708 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:00.703+0000 7f4bf4310000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:18:00.778 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:00.772+0000 7f96a6943000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:18:00.789 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:00.783+0000 7f4bf4310000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:18:00.863 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:00.856+0000 7f96a6943000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:18:00.921 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-10-06T11:18:00.915+0000 7f24eaba13c0 -1 Falling back to public interface 2023-10-06T11:18:00.922 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-10-06T11:18:00.915+0000 7ff4634133c0 -1 Falling back to public interface 2023-10-06T11:18:00.924 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-10-06T11:18:00.919+0000 7f2cb96f13c0 -1 Falling back to public interface 2023-10-06T11:18:00.926 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-10-06T11:18:00.920+0000 7efc089df3c0 -1 Falling back to public interface 2023-10-06T11:18:00.935 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-10-06T11:18:00.928+0000 7f2def6e73c0 -1 Falling back to public interface 2023-10-06T11:18:00.935 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-10-06T11:18:00.928+0000 7fa231f003c0 -1 Falling back to public interface 2023-10-06T11:18:00.936 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-10-06T11:18:00.932+0000 7fdc6f31d3c0 -1 Falling back to public interface 2023-10-06T11:18:00.939 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-10-06T11:18:00.935+0000 7f21cc3ba3c0 -1 Falling back to public interface 2023-10-06T11:18:01.109 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:01.103+0000 7f4bf4310000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:18:01.190 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:01.187+0000 7f4bf4310000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:18:01.225 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:01.220+0000 7f96a6943000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:18:01.278 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:01.275+0000 7f4bf4310000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:18:01.308 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:01.304+0000 7f96a6943000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:18:01.366 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:01.406 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:01.400+0000 7f96a6943000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:18:01.486 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:01.483+0000 7f4bf4310000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:18:01.510 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.503+0000 7fb58f4a3700 1 Processor -- start 2023-10-06T11:18:01.510 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.503+0000 7fb58f4a3700 1 -- start start 2023-10-06T11:18:01.513 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58f4a3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:01.513 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58f4a3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb5880c6c90 0x7fb588159080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:01.513 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58f4a3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58806d600 con 0x7fb58806e700 2023-10-06T11:18:01.513 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58f4a3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880c8260 con 0x7fb5880c6c90 2023-10-06T11:18:01.513 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58f4a3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58806ec80 con 0x7fb588069d00 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58e4a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58e4a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:35966/0 (socket says 172.21.15.46:35966) 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1996288967 learned_addr learned my addr 172.21.15.46:0/1996288967 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58eca2700 1 --2- 172.21.15.46:0/1996288967 >> v2:172.21.15.132:3300/0 conn(0x7fb5880c6c90 0x7fb588159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1996288967 >> v2:172.21.15.132:3300/0 conn(0x7fb5880c6c90 msgr2=0x7fb588159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:01.514 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.507+0000 7fb58e4a1700 1 --2- 172.21.15.46:0/1996288967 >> v2:172.21.15.132:3300/0 conn(0x7fb5880c6c90 0x7fb588159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.515 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1996288967 >> v1:172.21.15.46:6789/0 conn(0x7fb58806e700 legacy=0x7fb58806da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:01.515 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1996288967 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58806cfb0 con 0x7fb588069d00 2023-10-06T11:18:01.515 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58e4a1700 1 --2- 172.21.15.46:0/1996288967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb5840099b0 tx=0x7fb584009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34a934dfcf243a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:01.516 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58d49f700 1 -- 172.21.15.46:0/1996288967 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584009ea0 con 0x7fb588069d00 2023-10-06T11:18:01.516 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58d49f700 1 -- 172.21.15.46:0/1996288967 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58400b860 con 0x7fb588069d00 2023-10-06T11:18:01.516 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58d49f700 1 -- 172.21.15.46:0/1996288967 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58400bc00 con 0x7fb588069d00 2023-10-06T11:18:01.516 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1996288967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 msgr2=0x7fb58806a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:01.517 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1996288967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb5840099b0 tx=0x7fb584009250 comp rx=0 tx=0).stop 2023-10-06T11:18:01.517 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1996288967 shutdown_connections 2023-10-06T11:18:01.518 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1996288967 >> v2:172.21.15.132:3300/0 conn(0x7fb5880c6c90 0x7fb588159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.518 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1996288967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb58806a0d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.518 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1996288967 >> 172.21.15.46:0/1996288967 conn(0x7fb588067050 msgr2=0x7fb588067450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:01.518 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1996288967 shutdown_connections 2023-10-06T11:18:01.518 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.511+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1996288967 wait complete. 2023-10-06T11:18:01.520 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 Processor -- start 2023-10-06T11:18:01.520 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 -- start start 2023-10-06T11:18:01.522 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:01.522 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 0x7fb5880c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:01.522 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5880c6c90 0x7fb5880c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:01.522 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb588156850 con 0x7fb58806e700 2023-10-06T11:18:01.522 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58816dfc0 con 0x7fb5880c6c90 2023-10-06T11:18:01.523 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.515+0000 7fb58f4a3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58816e2a0 con 0x7fb588069d00 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58dca0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 0x7fb5880c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:35976/0 (socket says 172.21.15.46:35976) 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1497463269 learned_addr learned my addr 172.21.15.46:0/1497463269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58eca2700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5880c6c90 0x7fb5880c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5880c6c90 msgr2=0x7fb5880c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5880c6c90 0x7fb5880c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.525 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1497463269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 msgr2=0x7fb5880c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 0x7fb5880c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 -- 172.21.15.46:0/1497463269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840094f0 con 0x7fb588069d00 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58dca0700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 0x7fb5880c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58e4a1700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb5880c83a0 tx=0x7fb584017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584000d10 con 0x7fb588069d00 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58400bbe0 con 0x7fb588069d00 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58816e580 con 0x7fb588069d00 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584011670 con 0x7fb588069d00 2023-10-06T11:18:01.526 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5880c47c0 con 0x7fb588069d00 2023-10-06T11:18:01.527 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb584011aa0 con 0x7fb588069d00 2023-10-06T11:18:01.527 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fb584023ad0 con 0x7fb588069d00 2023-10-06T11:18:01.527 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.519+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880c2aa0 con 0x7fb588069d00 2023-10-06T11:18:01.530 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.527+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb584028070 con 0x7fb588069d00 2023-10-06T11:18:01.581 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:01.575+0000 7f4bf4310000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:18:01.599 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.595+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb58806d290 con 0x7fb588069d00 2023-10-06T11:18:01.600 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.595+0000 7fb57e7fc700 1 -- 172.21.15.46:0/1497463269 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fb584015020 con 0x7fb588069d00 2023-10-06T11:18:01.601 INFO:teuthology.misc.health.smithi046.stdout: 2023-10-06T11:18:01.601 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:00.203519+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:01.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 msgr2=0x7fb5880c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:01.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb5880c83a0 tx=0x7fb584017000 comp rx=0 tx=0).stop 2023-10-06T11:18:01.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 shutdown_connections 2023-10-06T11:18:01.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5880c6c90 0x7fb5880c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb58806e700 0x7fb5880c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 --2- 172.21.15.46:0/1497463269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb588069d00 0x7fb5880c4280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:01.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 >> 172.21.15.46:0/1497463269 conn(0x7fb588067050 msgr2=0x7fb58814f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:01.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 shutdown_connections 2023-10-06T11:18:01.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:01.599+0000 7fb58f4a3700 1 -- 172.21.15.46:0/1497463269 wait complete. 2023-10-06T11:18:01.619 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T11:18:01.629 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:01.624+0000 7f96a6943000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:18:01.722 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:01.716+0000 7f96a6943000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:18:02.459 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:02.455+0000 7f4bf4310000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:18:02.606 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:18:02.600+0000 7f96a6943000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:18:03.498 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:03.491+0000 7f4bdd13e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T11:18:03.498 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:18:03.491+0000 7f4bdd13e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T11:18:04.860 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-10-06T11:18:04.856+0000 7f2def6e73c0 -1 osd.5 0 log_to_monitors true 2023-10-06T11:18:04.878 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-10-06T11:18:04.872+0000 7efc089df3c0 -1 osd.4 0 log_to_monitors true 2023-10-06T11:18:05.118 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-10-06T11:18:05.111+0000 7f24eaba13c0 -1 osd.2 0 log_to_monitors true 2023-10-06T11:18:05.151 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-10-06T11:18:05.144+0000 7fdc6f31d3c0 -1 osd.6 0 log_to_monitors true 2023-10-06T11:18:05.166 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-10-06T11:18:05.163+0000 7f21cc3ba3c0 -1 osd.0 0 log_to_monitors true 2023-10-06T11:18:05.183 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-10-06T11:18:05.179+0000 7ff4634133c0 -1 osd.3 0 log_to_monitors true 2023-10-06T11:18:05.218 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-10-06T11:18:05.215+0000 7f2cb96f13c0 -1 osd.1 0 log_to_monitors true 2023-10-06T11:18:05.240 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-10-06T11:18:05.236+0000 7fa231f003c0 -1 osd.7 0 log_to_monitors true 2023-10-06T11:18:08.426 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:08.492 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-10-06T11:18:08.486+0000 7f24e6343700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T11:18:08.503 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-10-06T11:18:08.498+0000 7f24df0e7700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:08.547 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-10-06T11:18:08.542+0000 7f21c7b5a700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T11:18:08.557 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-10-06T11:18:08.550+0000 7f21c08fe700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:08.599 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-10-06T11:18:08.594+0000 7ff45ebb3700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T11:18:08.600 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.594+0000 7ff7913cc700 1 Processor -- start 2023-10-06T11:18:08.601 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.594+0000 7ff7913cc700 1 -- start start 2023-10-06T11:18:08.602 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff7913cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 0x7ff78c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.602 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff7913cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 0x7ff78c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff790bcb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff78c0b1e50 0x7ff78c0b22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55780/0 (socket says 172.21.15.46:55780) 2023-10-06T11:18:08.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff790bcb700 1 -- 172.21.15.46:0/2634394729 learned_addr learned my addr 172.21.15.46:0/2634394729 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:08.603 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/2634394729 >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 0x7ff78c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c15dc50 con 0x7ff78c0b1e50 2023-10-06T11:18:08.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78bfff700 1 --2- 172.21.15.46:0/2634394729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 0x7ff78c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0cb7a0 con 0x7ff78c15d400 2023-10-06T11:18:08.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cae70 con 0x7ff78c1590a0 2023-10-06T11:18:08.604 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78bfff700 1 -- 172.21.15.46:0/2634394729 >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 msgr2=0x7ff78c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:08.605 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78bfff700 1 --2- 172.21.15.46:0/2634394729 >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 0x7ff78c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.605 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78bfff700 1 -- 172.21.15.46:0/2634394729 >> v1:172.21.15.46:6789/0 conn(0x7ff78c0b1e50 legacy=0x7ff78c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.605 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78bfff700 1 -- 172.21.15.46:0/2634394729 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0cb150 con 0x7ff78c1590a0 2023-10-06T11:18:08.605 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.598+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/2634394729 >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 0x7ff78c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:08.606 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff457957700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:08.607 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff78bfff700 1 --2- 172.21.15.46:0/2634394729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 0x7ff78c0cd1c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098e0 tx=0x7ff77c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e0d9feda8ef24d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:08.608 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-10-06T11:18:08.602+0000 7f2cb5df4700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T11:18:08.608 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff78affd700 1 -- 172.21.15.46:0/2634394729 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00b640 con 0x7ff78c1590a0 2023-10-06T11:18:08.609 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff78affd700 1 -- 172.21.15.46:0/2634394729 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00b7a0 con 0x7ff78c1590a0 2023-10-06T11:18:08.609 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff78affd700 1 -- 172.21.15.46:0/2634394729 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00bb40 con 0x7ff78c1590a0 2023-10-06T11:18:08.609 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 msgr2=0x7ff78c0cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.609 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 --2- 172.21.15.46:0/2634394729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 0x7ff78c0cd1c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098e0 tx=0x7ff77c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.610 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 shutdown_connections 2023-10-06T11:18:08.610 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 --2- 172.21.15.46:0/2634394729 >> v2:172.21.15.132:3300/0 conn(0x7ff78c15d400 0x7ff78c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.610 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 --2- 172.21.15.46:0/2634394729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c1590a0 0x7ff78c0cd1c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.610 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 >> 172.21.15.46:0/2634394729 conn(0x7ff78c06e780 msgr2=0x7ff78c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:08.610 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.602+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 shutdown_connections 2023-10-06T11:18:08.611 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.606+0000 7ff7913cc700 1 -- 172.21.15.46:0/2634394729 wait complete. 2023-10-06T11:18:08.613 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.606+0000 7ff7913cc700 1 Processor -- start 2023-10-06T11:18:08.614 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.606+0000 7ff7913cc700 1 -- start start 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 0x7ff78c147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c15d400 0x7ff78c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c169c70 con 0x7ff78c0b1e50 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c169ee0 con 0x7ff78c1590a0 2023-10-06T11:18:08.616 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c16a1c0 con 0x7ff78c15d400 2023-10-06T11:18:08.617 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.617 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33314/0 (socket says 172.21.15.46:33314) 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78bfff700 1 -- 172.21.15.46:0/942507004 learned_addr learned my addr 172.21.15.46:0/942507004 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 0x7ff78c147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff790bcb700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c15d400 0x7ff78c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 -- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c15d400 msgr2=0x7ff78c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c15d400 0x7ff78c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 -- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 msgr2=0x7ff78c169730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 -- 172.21.15.46:0/942507004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff780009420 con 0x7ff78c1590a0 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78bfff700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:08.618 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff78b7fe700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 0x7ff78c147480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff78000ea00 tx=0x7ff7800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:08.619 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780009660 con 0x7ff78c1590a0 2023-10-06T11:18:08.619 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000c850 con 0x7ff78c1590a0 2023-10-06T11:18:08.619 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c009420 con 0x7ff78c1590a0 2023-10-06T11:18:08.619 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.610+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000cbf0 con 0x7ff78c1590a0 2023-10-06T11:18:08.620 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.614+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c16ad80 con 0x7ff78c1590a0 2023-10-06T11:18:08.620 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-10-06T11:18:08.614+0000 7f2cad384700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:08.620 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.614+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c16abe0 con 0x7ff78c1590a0 2023-10-06T11:18:08.622 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.618+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7ff78000cd50 con 0x7ff78c1590a0 2023-10-06T11:18:08.626 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.618+0000 7ff773fff700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 0x7ff778062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:08.626 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.618+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7ff78007ba90 con 0x7ff78c1590a0 2023-10-06T11:18:08.626 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.618+0000 7ff78bfff700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 0x7ff778062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:08.626 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.618+0000 7ff78bfff700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 0x7ff778062970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff77c003080 tx=0x7ff77c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:08.631 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.626+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff78004ced0 con 0x7ff78c1590a0 2023-10-06T11:18:08.802 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.794+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff78c16abe0 con 0x7ff78c1590a0 2023-10-06T11:18:08.803 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff773fff700 1 -- 172.21.15.46:0/942507004 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7ff78c16abe0 con 0x7ff78c1590a0 2023-10-06T11:18:08.804 INFO:teuthology.misc.health.smithi046.stdout: 2023-10-06T11:18:08.804 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":12,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:06.508943+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:08.805 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 msgr2=0x7ff778062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 0x7ff778062970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff77c003080 tx=0x7ff77c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 msgr2=0x7ff78c147480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 0x7ff78c147480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff78000ea00 tx=0x7ff7800091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 shutdown_connections 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7780604c0 0x7ff778062970 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff78c15d400 0x7ff78c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff78c1590a0 0x7ff78c147480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 --2- 172.21.15.46:0/942507004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff78c0b1e50 0x7ff78c169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:08.806 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.798+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 >> 172.21.15.46:0/942507004 conn(0x7ff78c06e780 msgr2=0x7ff78c0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:08.807 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.802+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 shutdown_connections 2023-10-06T11:18:08.807 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:08.802+0000 7ff7913cc700 1 -- 172.21.15.46:0/942507004 wait complete. 2023-10-06T11:18:08.822 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T11:18:08.963 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-10-06T11:18:08.956+0000 7fa22ceb5700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T11:18:08.973 INFO:tasks.ceph.osd.7.smithi132.stderr:2023-10-06T11:18:08.968+0000 7fa225c59700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:09.012 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-10-06T11:18:09.008+0000 7fdc6a2c2700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T11:18:09.022 INFO:tasks.ceph.osd.6.smithi132.stderr:2023-10-06T11:18:09.016+0000 7fdc63066700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:09.026 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-10-06T11:18:09.020+0000 7efc059a4700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T11:18:09.032 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-10-06T11:18:09.028+0000 7f2dea65c700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T11:18:09.033 INFO:tasks.ceph.osd.4.smithi132.stderr:2023-10-06T11:18:09.028+0000 7efbfcf23700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:09.043 INFO:tasks.ceph.osd.5.smithi132.stderr:2023-10-06T11:18:09.036+0000 7f2de3400700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:18:14.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:18:14.652 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:18:15.628 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:15.781 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.774+0000 7feed759e700 1 Processor -- start 2023-10-06T11:18:15.781 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.774+0000 7feed759e700 1 -- start start 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feed8140690 0x7feed80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed815b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feed80c8610 0x7feed80c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed806d050 con 0x7feed8140690 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80c8f90 con 0x7feed80c8610 2023-10-06T11:18:15.784 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed815b990 con 0x7feed806cab0 2023-10-06T11:18:15.785 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feed8140690 0x7feed80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.785 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feed8140690 0x7feed80aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39450/0 (socket says 172.21.15.46:39450) 2023-10-06T11:18:15.785 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 -- 172.21.15.46:0/2604936243 learned_addr learned my addr 172.21.15.46:0/2604936243 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:15.785 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed6d9d700 1 --2- 172.21.15.46:0/2604936243 >> v2:172.21.15.132:3300/0 conn(0x7feed80c8610 0x7feed80c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.785 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feecdd9b700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.786 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 -- 172.21.15.46:0/2604936243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 msgr2=0x7feed815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:15.786 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed815b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.786 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 -- 172.21.15.46:0/2604936243 >> v2:172.21.15.132:3300/0 conn(0x7feed80c8610 msgr2=0x7feed80c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:15.786 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 --2- 172.21.15.46:0/2604936243 >> v2:172.21.15.132:3300/0 conn(0x7feed80c8610 0x7feed80c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.786 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.778+0000 7feed659c700 1 -- 172.21.15.46:0/2604936243 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed806e740 con 0x7feed8140690 2023-10-06T11:18:15.787 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed659c700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80aeb10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feed806afa0 tx=0x7feec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2fd738b4f9d5e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:15.787 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed5d9b700 1 -- 172.21.15.46:0/2604936243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800b640 con 0x7feed8140690 2023-10-06T11:18:15.787 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed5d9b700 1 -- 172.21.15.46:0/2604936243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800b7a0 con 0x7feed8140690 2023-10-06T11:18:15.787 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed5d9b700 1 -- 172.21.15.46:0/2604936243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec800bb60 con 0x7feed8140690 2023-10-06T11:18:15.788 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 -- 172.21.15.46:0/2604936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 msgr2=0x7feed80aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:15.788 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80aeb10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feed806afa0 tx=0x7feec8009250 comp rx=0 tx=0).stop 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 -- 172.21.15.46:0/2604936243 shutdown_connections 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 --2- 172.21.15.46:0/2604936243 >> v2:172.21.15.132:3300/0 conn(0x7feed80c8610 0x7feed80c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 --2- 172.21.15.46:0/2604936243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80aeb10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 -- 172.21.15.46:0/2604936243 >> 172.21.15.46:0/2604936243 conn(0x7feed8067050 msgr2=0x7feed8067450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 -- 172.21.15.46:0/2604936243 shutdown_connections 2023-10-06T11:18:15.789 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.782+0000 7feed759e700 1 -- 172.21.15.46:0/2604936243 wait complete. 2023-10-06T11:18:15.791 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 Processor -- start 2023-10-06T11:18:15.791 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 -- start start 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed80caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 0x7feed80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c9d60 con 0x7feed8140690 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80ca040 con 0x7feed80c8610 2023-10-06T11:18:15.793 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80ca320 con 0x7feed806cab0 2023-10-06T11:18:15.794 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.794 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39454/0 (socket says 172.21.15.46:39454) 2023-10-06T11:18:15.794 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 -- 172.21.15.46:0/1408539759 learned_addr learned my addr 172.21.15.46:0/1408539759 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feecdd9b700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 0x7feed80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed659c700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed80caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 -- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 msgr2=0x7feed80caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed80caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 -- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 msgr2=0x7feed80b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 0x7feed80b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed6d9d700 1 -- 172.21.15.46:0/1408539759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80094f0 con 0x7feed8140690 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feecdd9b700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 0x7feed80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:15.795 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.786+0000 7feed659c700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed80caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feed6d9d700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e90 tx=0x7feed000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000cc20 con 0x7feed8140690 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed000f040 con 0x7feed8140690 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0014480 con 0x7feed8140690 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed80ca600 con 0x7feed8140690 2023-10-06T11:18:15.796 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed80b0d10 con 0x7feed8140690 2023-10-06T11:18:15.797 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.790+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed80c60a0 con 0x7feed8140690 2023-10-06T11:18:15.798 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.794+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7feed0014830 con 0x7feed8140690 2023-10-06T11:18:15.800 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.794+0000 7feeceffd700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 0x7feeb8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:15.800 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.794+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7feed007cfa0 con 0x7feed8140690 2023-10-06T11:18:15.800 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.794+0000 7feed659c700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 0x7feeb8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:15.801 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.794+0000 7feed659c700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 0x7feeb8062a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feed806afa0 tx=0x7feec8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:15.808 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.802+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feed80c60a0 con 0x7feed8140690 2023-10-06T11:18:15.979 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.974+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feed80c60a0 con 0x7feed8140690 2023-10-06T11:18:15.980 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.974+0000 7feeceffd700 1 -- 172.21.15.46:0/1408539759 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7feed80c60a0 con 0x7feed8140690 2023-10-06T11:18:15.981 INFO:teuthology.misc.health.smithi046.stdout: 2023-10-06T11:18:15.981 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":16,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:13.044840+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 msgr2=0x7feeb8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 0x7feeb8062a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feed806afa0 tx=0x7feec8017000 comp rx=0 tx=0).stop 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 msgr2=0x7feed80c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e90 tx=0x7feed000b040 comp rx=0 tx=0).stop 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 shutdown_connections 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feeb8060550 0x7feeb8062a00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feed8140690 0x7feed80c45a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feed80c8610 0x7feed80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.984 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 --2- 172.21.15.46:0/1408539759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feed806cab0 0x7feed80caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:15.985 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 >> 172.21.15.46:0/1408539759 conn(0x7feed8067050 msgr2=0x7feed80c7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:15.985 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 shutdown_connections 2023-10-06T11:18:15.985 INFO:teuthology.misc.health.smithi046.stderr:2023-10-06T11:18:15.978+0000 7feed759e700 1 -- 172.21.15.46:0/1408539759 wait complete. 2023-10-06T11:18:16.003 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T11:18:16.004 INFO:tasks.ceph:Creating RBD pool 2023-10-06T11:18:16.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T11:18:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.154+0000 7fca0ee4c700 1 Processor -- start 2023-10-06T11:18:16.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 -- start start 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 0x7fca08067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca08159340 0x7fca08067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca080ac990 0x7fca080ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080ab550 con 0x7fca08158f70 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080aa980 con 0x7fca08159340 2023-10-06T11:18:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.158+0000 7fca0ee4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080aac20 con 0x7fca080ac990 2023-10-06T11:18:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0de4a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 0x7fca08067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca08159340 0x7fca08067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0e64b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca080ac990 0x7fca080ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca08159340 0x7fca08067590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:39564/0 (socket says 172.21.15.46:39564) 2023-10-06T11:18:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 -- 172.21.15.46:0/2795388319 learned_addr learned my addr 172.21.15.46:0/2795388319 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 -- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca080ac990 msgr2=0x7fca080ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 --2- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca080ac990 0x7fca080ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 -- 172.21.15.46:0/2795388319 >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 msgr2=0x7fca08067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 --2- 172.21.15.46:0/2795388319 >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 0x7fca08067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 -- 172.21.15.46:0/2795388319 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080aaf00 con 0x7fca08159340 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0de4a700 1 --2- 172.21.15.46:0/2795388319 >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 0x7fca08067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0d649700 1 --2- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca08159340 0x7fca08067590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca080b2e20 tx=0x7fc9f8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673ab22ae024f1ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0ce48700 1 -- 172.21.15.46:0/2795388319 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800b640 con 0x7fca08159340 2023-10-06T11:18:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0ce48700 1 -- 172.21.15.46:0/2795388319 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f800b7a0 con 0x7fca08159340 2023-10-06T11:18:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.162+0000 7fca0ce48700 1 -- 172.21.15.46:0/2795388319 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800bb60 con 0x7fca08159340 2023-10-06T11:18:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca08159340 msgr2=0x7fca08067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca08159340 0x7fca08067590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca080b2e20 tx=0x7fc9f8009250 comp rx=0 tx=0).stop 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2795388319 shutdown_connections 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca08159340 0x7fca08067590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2795388319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca080ac990 0x7fca080ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2795388319 >> v2:172.21.15.46:3300/0 conn(0x7fca08158f70 0x7fca08067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2795388319 >> 172.21.15.46:0/2795388319 conn(0x7fca080ccd50 msgr2=0x7fca080cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2795388319 shutdown_connections 2023-10-06T11:18:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2795388319 wait complete. 2023-10-06T11:18:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 Processor -- start 2023-10-06T11:18:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 -- start start 2023-10-06T11:18:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.166+0000 7fca0ee4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca080ac990 0x7fca081426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca08159340 0x7fca0815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0815c060 con 0x7fca08158f70 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0815b450 con 0x7fca080ac990 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0815b730 con 0x7fca08159340 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39482/0 (socket says 172.21.15.46:39482) 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 -- 172.21.15.46:0/2419288037 learned_addr learned my addr 172.21.15.46:0/2419288037 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0de4a700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca080ac990 0x7fca081426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0e64b700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca08159340 0x7fca0815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 -- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca08159340 msgr2=0x7fca0815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca08159340 0x7fca0815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 -- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca080ac990 msgr2=0x7fca081426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca080ac990 0x7fca081426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 -- 172.21.15.46:0/2419288037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009420 con 0x7fca08158f70 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0d649700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca080aaac0 tx=0x7fc9f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800b910 con 0x7fca08158f70 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f800ba70 con 0x7fca08158f70 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f801a660 con 0x7fca08158f70 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80094f0 con 0x7fca08158f70 2023-10-06T11:18:16.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.170+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0815bcf0 con 0x7fca08158f70 2023-10-06T11:18:16.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.174+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0815c210 con 0x7fca08158f70 2023-10-06T11:18:16.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.174+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc9f801aa50 con 0x7fca08158f70 2023-10-06T11:18:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.174+0000 7fc9fdffb700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 0x7fc9ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.174+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc9f807e000 con 0x7fca08158f70 2023-10-06T11:18:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.174+0000 7fca0de4a700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 0x7fc9ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.178+0000 7fca0de4a700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 0x7fc9ec062a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca040093f0 tx=0x7fca04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.182+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fca0815c210 con 0x7fca08158f70 2023-10-06T11:18:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.354+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fca08064590 con 0x7fca08158f70 2023-10-06T11:18:16.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.670+0000 7fc9fdffb700 1 -- 172.21.15.46:0/2419288037 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc9f804e920 con 0x7fca08158f70 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 msgr2=0x7fc9ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 0x7fc9ec062a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca040093f0 tx=0x7fca04008040 comp rx=0 tx=0).stop 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 msgr2=0x7fca0815aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca080aaac0 tx=0x7fc9f8017000 comp rx=0 tx=0).stop 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 shutdown_connections 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9ec060550 0x7fc9ec062a00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca08159340 0x7fca0815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca08158f70 0x7fca0815aae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 --2- 172.21.15.46:0/2419288037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca080ac990 0x7fca081426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 >> 172.21.15.46:0/2419288037 conn(0x7fca080ccd50 msgr2=0x7fca080cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 shutdown_connections 2023-10-06T11:18:16.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.678+0000 7fca0ee4c700 1 -- 172.21.15.46:0/2419288037 wait complete. 2023-10-06T11:18:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'rbd' created 2023-10-06T11:18:16.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T11:18:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.858+0000 7f887f48e700 1 Processor -- start 2023-10-06T11:18:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.858+0000 7f887f48e700 1 -- start start 2023-10-06T11:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887f48e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887f48e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f88780cc500 0x7f887815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887f48e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88780cf360 con 0x7f88781595c0 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887f48e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887815dd60 con 0x7f88780cc500 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887f48e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88780cea30 con 0x7f88781591f0 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887e48c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.862+0000 7f887ec8d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f88780cc500 0x7f887815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:32996/0 (socket says 172.21.15.46:32996) 2023-10-06T11:18:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 -- 172.21.15.46:0/1468152572 learned_addr learned my addr 172.21.15.46:0/1468152572 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 -- 172.21.15.46:0/1468152572 >> v2:172.21.15.132:3300/0 conn(0x7f88780cc500 msgr2=0x7f887815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 --2- 172.21.15.46:0/1468152572 >> v2:172.21.15.132:3300/0 conn(0x7f88780cc500 0x7f887815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 -- 172.21.15.46:0/1468152572 >> v1:172.21.15.132:6789/0 conn(0x7f88781595c0 legacy=0x7f88780af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 -- 172.21.15.46:0/1468152572 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88780ced10 con 0x7f88781591f0 2023-10-06T11:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887e48c700 1 --2- 172.21.15.46:0/1468152572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f88740099b0 tx=0x7f8874009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a75bfa847389ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887d48a700 1 -- 172.21.15.46:0/1468152572 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874009ea0 con 0x7f88781591f0 2023-10-06T11:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887d48a700 1 -- 172.21.15.46:0/1468152572 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400b8f0 con 0x7f88781591f0 2023-10-06T11:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887d48a700 1 -- 172.21.15.46:0/1468152572 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400bc90 con 0x7f88781591f0 2023-10-06T11:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 -- 172.21.15.46:0/1468152572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 msgr2=0x7f88780cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 --2- 172.21.15.46:0/1468152572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f88740099b0 tx=0x7f8874009250 comp rx=0 tx=0).stop 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 -- 172.21.15.46:0/1468152572 shutdown_connections 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 --2- 172.21.15.46:0/1468152572 >> v2:172.21.15.132:3300/0 conn(0x7f88780cc500 0x7f887815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 --2- 172.21.15.46:0/1468152572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88781591f0 0x7f88780cf7f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 -- 172.21.15.46:0/1468152572 >> 172.21.15.46:0/1468152572 conn(0x7f887806ef60 msgr2=0x7f88780d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 -- 172.21.15.46:0/1468152572 shutdown_connections 2023-10-06T11:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.866+0000 7f887f48e700 1 -- 172.21.15.46:0/1468152572 wait complete. 2023-10-06T11:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.870+0000 7f887f48e700 1 Processor -- start 2023-10-06T11:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.870+0000 7f887f48e700 1 -- start start 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 0x7f887816cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88780b08e0 0x7f8878152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8878156210 con 0x7f88781591f0 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8878155e70 con 0x7f88780cc500 2023-10-06T11:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887816d670 con 0x7f88780b08e0 2023-10-06T11:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39490/0 (socket says 172.21.15.46:39490) 2023-10-06T11:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 -- 172.21.15.46:0/3623679893 learned_addr learned my addr 172.21.15.46:0/3623679893 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887ec8d700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88780b08e0 0x7f8878152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887e48c700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 0x7f887816cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 -- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88780b08e0 msgr2=0x7f8878152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88780b08e0 0x7f8878152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 -- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 msgr2=0x7f887816cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 0x7f887816cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 -- 172.21.15.46:0/3623679893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740094f0 con 0x7f88781591f0 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887e48c700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 0x7f887816cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887dc8b700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f886800c590 tx=0x7f886800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011590 con 0x7f88781591f0 2023-10-06T11:18:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868011710 con 0x7f88781591f0 2023-10-06T11:18:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868011ad0 con 0x7f88781591f0 2023-10-06T11:18:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887816d950 con 0x7f88781591f0 2023-10-06T11:18:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.874+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887816d2b0 con 0x7f88781591f0 2023-10-06T11:18:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.878+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8878156350 con 0x7f88781591f0 2023-10-06T11:18:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.882+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f886801b3b0 con 0x7f88781591f0 2023-10-06T11:18:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.882+0000 7f886e7fc700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 0x7f885c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.882+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f886807b9d0 con 0x7f88781591f0 2023-10-06T11:18:16.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.882+0000 7f887e48c700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 0x7f885c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.882+0000 7f887e48c700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 0x7f885c062a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88740099b0 tx=0x7f8874017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:16.890+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8878156350 con 0x7f88781591f0 2023-10-06T11:18:17.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.062+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8878156350 con 0x7f88781591f0 2023-10-06T11:18:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.678+0000 7f886e7fc700 1 -- 172.21.15.46:0/3623679893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8878156350 con 0x7f88781591f0 2023-10-06T11:18:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.682+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 msgr2=0x7f885c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.682+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 0x7f885c062a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88740099b0 tx=0x7f8874017000 comp rx=0 tx=0).stop 2023-10-06T11:18:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.682+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 msgr2=0x7f887809d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.682+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f886800c590 tx=0x7f886800c8a0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 shutdown_connections 2023-10-06T11:18:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f885c060550 0x7f885c062a00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88780b08e0 0x7f8878152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88781591f0 0x7f887809d430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 --2- 172.21.15.46:0/3623679893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88780cc500 0x7f887816cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 >> 172.21.15.46:0/3623679893 conn(0x7f887806ef60 msgr2=0x7f88780cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 shutdown_connections 2023-10-06T11:18:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.686+0000 7f887f48e700 1 -- 172.21.15.46:0/3623679893 wait complete. 2023-10-06T11:18:17.693 INFO:teuthology.orchestra.run.smithi046.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T11:18:17.710 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T11:18:17.710 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:18:17.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:18:17.717 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:18:17.718 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:18:17.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:18:17.723 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:18:17.723 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:18:17.723 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:18:17.728 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:18:17.728 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:18:17.728 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:18:17.733 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:18:17.734 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T11:18:17.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:17.822 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:18:17.824 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:18:17.829 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:18:17.835 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:18:17.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.890+0000 7f978ca2d700 1 Processor -- start 2023-10-06T11:18:17.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.890+0000 7f978ca2d700 1 -- start start 2023-10-06T11:18:17.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f97880b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97880d07b0 0x7f97880b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97880b1130 0x7f97880ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97880b15b0 con 0x7f97880d07b0 2023-10-06T11:18:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97880ae340 con 0x7f97880b1130 2023-10-06T11:18:17.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978ca2d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880b09a0 con 0x7f9788159290 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97880d07b0 0x7f97880b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97880d07b0 0x7f97880b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39536/0 (socket says 172.21.15.46:39536) 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 -- 172.21.15.46:0/81164129 learned_addr learned my addr 172.21.15.46:0/81164129 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f9786d9d700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f97880b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978759e700 1 --2- 172.21.15.46:0/81164129 >> v2:172.21.15.132:3300/0 conn(0x7f97880b1130 0x7f97880ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 -- 172.21.15.46:0/81164129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 msgr2=0x7f97880b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f97880b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 -- 172.21.15.46:0/81164129 >> v2:172.21.15.132:3300/0 conn(0x7f97880b1130 msgr2=0x7f97880ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 --2- 172.21.15.46:0/81164129 >> v2:172.21.15.132:3300/0 conn(0x7f97880b1130 0x7f97880ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 -- 172.21.15.46:0/81164129 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9788159660 con 0x7f97880d07b0 2023-10-06T11:18:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.894+0000 7f978659c700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f97880b0bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97880cffd0 tx=0x7f977c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f980013ca3f4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f9785d9b700 1 -- 172.21.15.46:0/81164129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00b6d0 con 0x7f97880d07b0 2023-10-06T11:18:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f9785d9b700 1 -- 172.21.15.46:0/81164129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c00b830 con 0x7f97880d07b0 2023-10-06T11:18:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f9785d9b700 1 -- 172.21.15.46:0/81164129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00bbf0 con 0x7f97880d07b0 2023-10-06T11:18:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- 172.21.15.46:0/81164129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 msgr2=0x7f97880b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f97880b0bf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97880cffd0 tx=0x7f977c009250 comp rx=0 tx=0).stop 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- 172.21.15.46:0/81164129 shutdown_connections 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 --2- 172.21.15.46:0/81164129 >> v2:172.21.15.132:3300/0 conn(0x7f97880b1130 0x7f97880ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f97880b0bf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 --2- 172.21.15.46:0/81164129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f97880b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- 172.21.15.46:0/81164129 >> 172.21.15.46:0/81164129 conn(0x7f978806ef60 msgr2=0x7f97880d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- 172.21.15.46:0/81164129 shutdown_connections 2023-10-06T11:18:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- 172.21.15.46:0/81164129 wait complete. 2023-10-06T11:18:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 Processor -- start 2023-10-06T11:18:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.898+0000 7f978ca2d700 1 -- start start 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 0x7f978814db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f978815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97880cf6b0 con 0x7f97880d07b0 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978814e040 con 0x7f97880b1130 2023-10-06T11:18:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97880ceaa0 con 0x7f9788159290 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39546/0 (socket says 172.21.15.46:39546) 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 -- 172.21.15.46:0/2717103110 learned_addr learned my addr 172.21.15.46:0/2717103110 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f9786d9d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 0x7f978814db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978759e700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f978815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 -- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 msgr2=0x7f978815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:17.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f978815dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 -- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 msgr2=0x7f978814db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 0x7f978814db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 -- 172.21.15.46:0/2717103110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0094f0 con 0x7f97880d07b0 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f9786d9d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 0x7f978814db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978659c700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f977c0094c0 tx=0x7f977c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00b9a0 con 0x7f97880d07b0 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c00bb00 con 0x7f97880d07b0 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c026660 con 0x7f97880d07b0 2023-10-06T11:18:17.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97880ced80 con 0x7f97880d07b0 2023-10-06T11:18:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.902+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97880cf340 con 0x7f97880d07b0 2023-10-06T11:18:17.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.906+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97880cf820 con 0x7f97880d07b0 2023-10-06T11:18:17.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.906+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f977c023030 con 0x7f97880d07b0 2023-10-06T11:18:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.910+0000 7f976effd700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 0x7f9774062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.910+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f977c07de90 con 0x7f97880d07b0 2023-10-06T11:18:17.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.910+0000 7f9786d9d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 0x7f9774062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.910+0000 7f9786d9d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 0x7f9774062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9778009d80 tx=0x7f9778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:17.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:17.914+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f977c04e650 con 0x7f97880d07b0 2023-10-06T11:18:18.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.086+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9788064590 con 0x7f97880d07b0 2023-10-06T11:18:18.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.090+0000 7f976effd700 1 -- 172.21.15.46:0/2717103110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f977c051c70 con 0x7f97880d07b0 2023-10-06T11:18:18.097 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:18.097 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:17.671891+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 msgr2=0x7f9774062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 0x7f9774062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9778009d80 tx=0x7f9778008040 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 msgr2=0x7f978814fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f977c0094c0 tx=0x7f977c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 shutdown_connections 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9774060550 0x7f9774062a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9788159290 0x7f978815dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97880d07b0 0x7f978814fcd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 --2- 172.21.15.46:0/2717103110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97880b1130 0x7f978814db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 >> 172.21.15.46:0/2717103110 conn(0x7f978806ef60 msgr2=0x7f97880acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:18.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 shutdown_connections 2023-10-06T11:18:18.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.094+0000 7f978ca2d700 1 -- 172.21.15.46:0/2717103110 wait complete. 2023-10-06T11:18:18.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:18:18.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.274+0000 7f8ce944c700 1 Processor -- start 2023-10-06T11:18:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.274+0000 7f8ce944c700 1 -- start start 2023-10-06T11:18:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce944c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4159240 0x7f8ce40a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce944c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8ce40c72d0 0x7f8ce40c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce944c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce40c10a0 con 0x7f8ce4158e70 2023-10-06T11:18:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce944c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce40ca240 con 0x7f8ce40c72d0 2023-10-06T11:18:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce944c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce40ca520 con 0x7f8ce4159240 2023-10-06T11:18:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce3fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ce4158e70 0x7f8ce40c99f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45984/0 (socket says 172.21.15.46:45984) 2023-10-06T11:18:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce3fff700 1 -- 172.21.15.46:0/3191709320 learned_addr learned my addr 172.21.15.46:0/3191709320 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 --2- 172.21.15.46:0/3191709320 >> v2:172.21.15.132:3300/0 conn(0x7f8ce40c72d0 0x7f8ce40c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8cdb7fe700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4159240 0x7f8ce40a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce37fe700 1 -- 172.21.15.46:0/3191709320 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811830325 0 0) 0x7f8ce40c10a0 con 0x7f8ce4158e70 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce37fe700 1 -- 172.21.15.46:0/3191709320 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd0003570 con 0x7f8ce4158e70 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 -- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4159240 msgr2=0x7f8ce40a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4159240 0x7f8ce40a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 -- 172.21.15.46:0/3191709320 >> v1:172.21.15.46:6789/0 conn(0x7f8ce4158e70 legacy=0x7f8ce40c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 -- 172.21.15.46:0/3191709320 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce40c0a50 con 0x7f8ce40c72d0 2023-10-06T11:18:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.278+0000 7f8ce8c4b700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40c7a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009e60 tx=0x7f8cdc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e5cf698592ae12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce37fe700 1 -- 172.21.15.46:0/3191709320 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00e070 con 0x7f8ce40c72d0 2023-10-06T11:18:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce37fe700 1 -- 172.21.15.46:0/3191709320 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00bb60 con 0x7f8ce40c72d0 2023-10-06T11:18:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce37fe700 1 -- 172.21.15.46:0/3191709320 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc012430 con 0x7f8ce40c72d0 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 -- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40c72d0 msgr2=0x7f8ce40c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40c7a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009e60 tx=0x7f8cdc009780 comp rx=0 tx=0).stop 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 -- 172.21.15.46:0/3191709320 shutdown_connections 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40c7a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 --2- 172.21.15.46:0/3191709320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4159240 0x7f8ce40a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 -- 172.21.15.46:0/3191709320 >> 172.21.15.46:0/3191709320 conn(0x7f8ce40cbc10 msgr2=0x7f8ce40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 -- 172.21.15.46:0/3191709320 shutdown_connections 2023-10-06T11:18:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 -- 172.21.15.46:0/3191709320 wait complete. 2023-10-06T11:18:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.282+0000 7f8ce944c700 1 Processor -- start 2023-10-06T11:18:18.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 -- start start 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4158e70 0x7f8ce4169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 0x7f8ce415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4169e50 con 0x7f8ce40c72d0 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce415c130 con 0x7f8ce4159240 2023-10-06T11:18:18.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4068380 con 0x7f8ce4158e70 2023-10-06T11:18:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39552/0 (socket says 172.21.15.46:39552) 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 -- 172.21.15.46:0/1770974890 learned_addr learned my addr 172.21.15.46:0/1770974890 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce8c4b700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 0x7f8ce415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8cdb7fe700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4158e70 0x7f8ce4169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 -- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4158e70 msgr2=0x7f8ce4169910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4158e70 0x7f8ce4169910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 -- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 msgr2=0x7f8ce415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 0x7f8ce415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 -- 172.21.15.46:0/1770974890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc009a20 con 0x7f8ce40c72d0 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce8c4b700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 0x7f8ce415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce3fff700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40011e0 tx=0x7f8cd4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4005b30 con 0x7f8ce40c72d0 2023-10-06T11:18:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4005c90 con 0x7f8ce40c72d0 2023-10-06T11:18:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce40af070 con 0x7f8ce40c72d0 2023-10-06T11:18:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.286+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40072e0 con 0x7f8ce40c72d0 2023-10-06T11:18:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.290+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce4068900 con 0x7f8ce40c72d0 2023-10-06T11:18:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.290+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce4064590 con 0x7f8ce40c72d0 2023-10-06T11:18:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.290+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8cd4007650 con 0x7f8ce40c72d0 2023-10-06T11:18:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.294+0000 7f8ce0ff9700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 0x7f8cc4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.294+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8cd407ba60 con 0x7f8ce40c72d0 2023-10-06T11:18:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.294+0000 7f8cdb7fe700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 0x7f8cc4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.294+0000 7f8cdb7fe700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 0x7f8cc4062a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ce406e570 tx=0x7f8ccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.302+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8cd4049be0 con 0x7f8ce40c72d0 2023-10-06T11:18:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.478+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ce40c0d30 con 0x7f8ce40c72d0 2023-10-06T11:18:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.478+0000 7f8ce0ff9700 1 -- 172.21.15.46:0/1770974890 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cd40170a0 con 0x7f8ce40c72d0 2023-10-06T11:18:18.484 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 msgr2=0x7f8cc4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 0x7f8cc4062a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ce406e570 tx=0x7f8ccc008040 comp rx=0 tx=0).stop 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 msgr2=0x7f8ce40ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40011e0 tx=0x7f8cd4007df0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 shutdown_connections 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8cc4060580 0x7f8cc4062a30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce4159240 0x7f8ce415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce4158e70 0x7f8ce4169910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 --2- 172.21.15.46:0/1770974890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c72d0 0x7f8ce40ae9b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 >> 172.21.15.46:0/1770974890 conn(0x7f8ce40cbc10 msgr2=0x7f8ce41544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 shutdown_connections 2023-10-06T11:18:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.482+0000 7f8ce944c700 1 -- 172.21.15.46:0/1770974890 wait complete. 2023-10-06T11:18:18.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.666+0000 7f26b15cc700 1 Processor -- start 2023-10-06T11:18:18.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 -- start start 2023-10-06T11:18:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 0x7f26ac0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26ac0c7f20 0x7f26ac15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0c6cf0 con 0x7f26ac154880 2023-10-06T11:18:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0c2a30 con 0x7f26ac0c7f20 2023-10-06T11:18:18.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b15cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0c83e0 con 0x7f26ac154c50 2023-10-06T11:18:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26b0dcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26ac0c7f20 0x7f26ac15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26abfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26ac154880 0x7f26ac06c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45998/0 (socket says 172.21.15.46:45998) 2023-10-06T11:18:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26ab7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 0x7f26ac0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.670+0000 7f26abfff700 1 -- 172.21.15.46:0/3403763509 learned_addr learned my addr 172.21.15.46:0/3403763509 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b0dcb700 1 -- 172.21.15.46:0/3403763509 >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 msgr2=0x7f26ac0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b0dcb700 1 --2- 172.21.15.46:0/3403763509 >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 0x7f26ac0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b0dcb700 1 -- 172.21.15.46:0/3403763509 >> v1:172.21.15.46:6789/0 conn(0x7f26ac154880 legacy=0x7f26ac06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26aaffd700 1 -- 172.21.15.46:0/3403763509 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529552875 0 0) 0x7f26ac0c6cf0 con 0x7f26ac154880 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b0dcb700 1 -- 172.21.15.46:0/3403763509 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac06e5d0 con 0x7f26ac0c7f20 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26ab7fe700 1 --2- 172.21.15.46:0/3403763509 >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 0x7f26ac0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:18.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b0dcb700 1 --2- 172.21.15.46:0/3403763509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac0c7f20 0x7f26ac15aa30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26ac15bb40 tx=0x7f26a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c70b203c3ceafd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26aaffd700 1 -- 172.21.15.46:0/3403763509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400e070 con 0x7f26ac0c7f20 2023-10-06T11:18:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26aaffd700 1 -- 172.21.15.46:0/3403763509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400bb60 con 0x7f26ac0c7f20 2023-10-06T11:18:18.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26aaffd700 1 -- 172.21.15.46:0/3403763509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012430 con 0x7f26ac0c7f20 2023-10-06T11:18:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 -- 172.21.15.46:0/3403763509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac0c7f20 msgr2=0x7f26ac15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 --2- 172.21.15.46:0/3403763509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac0c7f20 0x7f26ac15aa30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26ac15bb40 tx=0x7f26a4009780 comp rx=0 tx=0).stop 2023-10-06T11:18:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 -- 172.21.15.46:0/3403763509 shutdown_connections 2023-10-06T11:18:18.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 --2- 172.21.15.46:0/3403763509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac0c7f20 0x7f26ac15aa30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 --2- 172.21.15.46:0/3403763509 >> v2:172.21.15.132:3300/0 conn(0x7f26ac154c50 0x7f26ac0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 -- 172.21.15.46:0/3403763509 >> 172.21.15.46:0/3403763509 conn(0x7f26ac0cbc10 msgr2=0x7f26ac0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 -- 172.21.15.46:0/3403763509 shutdown_connections 2023-10-06T11:18:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.674+0000 7f26b15cc700 1 -- 172.21.15.46:0/3403763509 wait complete. 2023-10-06T11:18:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 Processor -- start 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 -- start start 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0c7f20 0x7f26ac152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac154c50 0x7f26ac1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c3800 con 0x7f26ac154880 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c3ae0 con 0x7f26ac0c7f20 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26b15cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0c3dc0 con 0x7f26ac154c50 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26ab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26ab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39570/0 (socket says 172.21.15.46:39570) 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26ab7fe700 1 -- 172.21.15.46:0/1870931978 learned_addr learned my addr 172.21.15.46:0/1870931978 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.678+0000 7f26abfff700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0c7f20 0x7f26ac152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26b0dcb700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac154c50 0x7f26ac1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 -- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac154c50 msgr2=0x7f26ac1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac154c50 0x7f26ac1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 -- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0c7f20 msgr2=0x7f26ac152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0c7f20 0x7f26ac152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 -- 172.21.15.46:0/1870931978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009420 con 0x7f26ac154880 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26ab7fe700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26a000ec80 tx=0x7f26a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000c9d0 con 0x7f26ac154880 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000cb30 con 0x7f26ac154880 2023-10-06T11:18:18.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a4009a20 con 0x7f26ac154880 2023-10-06T11:18:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0013470 con 0x7f26ac154880 2023-10-06T11:18:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac0c4380 con 0x7f26ac154880 2023-10-06T11:18:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.682+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac064590 con 0x7f26ac154880 2023-10-06T11:18:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.686+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f26a0013820 con 0x7f26ac154880 2023-10-06T11:18:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.686+0000 7f2693fff700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 0x7f2698062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.686+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f26a007c6a0 con 0x7f26ac154880 2023-10-06T11:18:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.686+0000 7f26abfff700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 0x7f2698062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.686+0000 7f26abfff700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 0x7f2698062a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f269c0011e0 tx=0x7f269c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.694+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f26a004ce80 con 0x7f26ac154880 2023-10-06T11:18:18.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.862+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26ac06e8b0 con 0x7f26ac154880 2023-10-06T11:18:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.862+0000 7f2693fff700 1 -- 172.21.15.46:0/1870931978 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26a00504a0 con 0x7f26ac154880 2023-10-06T11:18:18.870 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 msgr2=0x7f2698062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 0x7f2698062a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f269c0011e0 tx=0x7f269c009040 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 msgr2=0x7f26ac15af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26a000ec80 tx=0x7f26a00091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 shutdown_connections 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2698060550 0x7f2698062a00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac154c50 0x7f26ac1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac154880 0x7f26ac15af60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 --2- 172.21.15.46:0/1870931978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0c7f20 0x7f26ac152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:18.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 >> 172.21.15.46:0/1870931978 conn(0x7f26ac0cbc10 msgr2=0x7f26ac0ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:18.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 shutdown_connections 2023-10-06T11:18:18.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:18.866+0000 7f26b15cc700 1 -- 172.21.15.46:0/1870931978 wait complete. 2023-10-06T11:18:18.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:19.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.042+0000 7ff321616700 1 Processor -- start 2023-10-06T11:18:19.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 -- start start 2023-10-06T11:18:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff31c067760 0x7ff31c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff31c1591a0 0x7ff31c069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c15d6c0 con 0x7ff31c158dd0 2023-10-06T11:18:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c0699c0 con 0x7ff31c067760 2023-10-06T11:18:19.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff321616700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c15db50 con 0x7ff31c1591a0 2023-10-06T11:18:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff31c067760 0x7ff31c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff31c067760 0x7ff31c068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39582/0 (socket says 172.21.15.46:39582) 2023-10-06T11:18:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff320e15700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff31c1591a0 0x7ff31c069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 -- 172.21.15.46:0/1790197777 learned_addr learned my addr 172.21.15.46:0/1790197777 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 -- 172.21.15.46:0/1790197777 >> v2:172.21.15.132:3300/0 conn(0x7ff31c1591a0 msgr2=0x7ff31c069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 --2- 172.21.15.46:0/1790197777 >> v2:172.21.15.132:3300/0 conn(0x7ff31c1591a0 0x7ff31c069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 -- 172.21.15.46:0/1790197777 >> v1:172.21.15.132:6789/0 conn(0x7ff31c158dd0 legacy=0x7ff31c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 -- 172.21.15.46:0/1790197777 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c15cd90 con 0x7ff31c067760 2023-10-06T11:18:19.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.046+0000 7ff3137fe700 1 --2- 172.21.15.46:0/1790197777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c067760 0x7ff31c068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c2b50 tx=0x7ff304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c87634486f12b67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff31b7fe700 1 -- 172.21.15.46:0/1790197777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304009ea0 con 0x7ff31c067760 2023-10-06T11:18:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff31b7fe700 1 -- 172.21.15.46:0/1790197777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30400b820 con 0x7ff31c067760 2023-10-06T11:18:19.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff31b7fe700 1 -- 172.21.15.46:0/1790197777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400bbc0 con 0x7ff31c067760 2023-10-06T11:18:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 -- 172.21.15.46:0/1790197777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c067760 msgr2=0x7ff31c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 --2- 172.21.15.46:0/1790197777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c067760 0x7ff31c068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c2b50 tx=0x7ff304009250 comp rx=0 tx=0).stop 2023-10-06T11:18:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 -- 172.21.15.46:0/1790197777 shutdown_connections 2023-10-06T11:18:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 --2- 172.21.15.46:0/1790197777 >> v2:172.21.15.132:3300/0 conn(0x7ff31c1591a0 0x7ff31c069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 --2- 172.21.15.46:0/1790197777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c067760 0x7ff31c068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 -- 172.21.15.46:0/1790197777 >> 172.21.15.46:0/1790197777 conn(0x7ff31c062ef0 msgr2=0x7ff31c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 -- 172.21.15.46:0/1790197777 shutdown_connections 2023-10-06T11:18:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.050+0000 7ff321616700 1 -- 172.21.15.46:0/1790197777 wait complete. 2023-10-06T11:18:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.054+0000 7ff321616700 1 Processor -- start 2023-10-06T11:18:19.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.054+0000 7ff321616700 1 -- start start 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff31c067760 0x7ff31c0c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff31c158dd0 0x7ff31c16e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0ba910 con 0x7ff31c1591a0 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c16ea10 con 0x7ff31c067760 2023-10-06T11:18:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0c00e0 con 0x7ff31c158dd0 2023-10-06T11:18:19.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39596/0 (socket says 172.21.15.46:39596) 2023-10-06T11:18:19.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 -- 172.21.15.46:0/3851898450 learned_addr learned my addr 172.21.15.46:0/3851898450 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:19.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff31bfff700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff31c067760 0x7ff31c0c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff3137fe700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff31c158dd0 0x7ff31c16e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 -- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff31c158dd0 msgr2=0x7ff31c16e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff31c158dd0 0x7ff31c16e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 -- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff31c067760 msgr2=0x7ff31c0c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff31c067760 0x7ff31c0c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 -- 172.21.15.46:0/3851898450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040094f0 con 0x7ff31c1591a0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff320e15700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff314009df0 tx=0x7ff31400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400caa0 con 0x7ff31c1591a0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c0bf1f0 con 0x7ff31c1591a0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400f040 con 0x7ff31c1591a0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400ce70 con 0x7ff31c1591a0 2023-10-06T11:18:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.058+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c0bf7b0 con 0x7ff31c1591a0 2023-10-06T11:18:19.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.062+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c0c0270 con 0x7ff31c1591a0 2023-10-06T11:18:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.062+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff314014690 con 0x7ff31c1591a0 2023-10-06T11:18:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.062+0000 7ff318ff9700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 0x7ff2fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.062+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff31407d130 con 0x7ff31c1591a0 2023-10-06T11:18:19.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.062+0000 7ff31bfff700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 0x7ff2fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.066+0000 7ff31bfff700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 0x7ff2fc062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff30c003a80 tx=0x7ff30c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.070+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff3140513a0 con 0x7ff31c1591a0 2023-10-06T11:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.278+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff31c15d070 con 0x7ff31c1591a0 2023-10-06T11:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.282+0000 7ff318ff9700 1 -- 172.21.15.46:0/3851898450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7ff314018de0 con 0x7ff31c1591a0 2023-10-06T11:18:19.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:19.287 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 msgr2=0x7ff2fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 0x7ff2fc062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff30c003a80 tx=0x7ff30c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 msgr2=0x7ff31c14df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff314009df0 tx=0x7ff31400b040 comp rx=0 tx=0).stop 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 shutdown_connections 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2fc060550 0x7ff2fc062a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff31c1591a0 0x7ff31c14df50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff31c158dd0 0x7ff31c16e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 --2- 172.21.15.46:0/3851898450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff31c067760 0x7ff31c0c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 >> 172.21.15.46:0/3851898450 conn(0x7ff31c062ef0 msgr2=0x7ff31c06dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 shutdown_connections 2023-10-06T11:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.286+0000 7ff321616700 1 -- 172.21.15.46:0/3851898450 wait complete. 2023-10-06T11:18:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2023-10-06T11:18:19.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:19.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.470+0000 7fe73f2ae700 1 Processor -- start 2023-10-06T11:18:19.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.474+0000 7fe73f2ae700 1 -- start start 2023-10-06T11:18:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.474+0000 7fe73f2ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 0x7fe73806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.474+0000 7fe73f2ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe73806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7380ad6c0 0x7fe7380adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73806d460 con 0x7fe7380ad6c0 2023-10-06T11:18:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7380c7000 con 0x7fe73815ab60 2023-10-06T11:18:19.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380c72c0 con 0x7fe73815af30 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7380ad6c0 0x7fe7380adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7380ad6c0 0x7fe7380adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39608/0 (socket says 172.21.15.46:39608) 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 -- 172.21.15.46:0/3471311317 learned_addr learned my addr 172.21.15.46:0/3471311317 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d04a700 1 --2- 172.21.15.46:0/3471311317 >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 0x7fe73806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73c849700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe73806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 -- 172.21.15.46:0/3471311317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 msgr2=0x7fe73806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe73806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 -- 172.21.15.46:0/3471311317 >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 msgr2=0x7fe73806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 --2- 172.21.15.46:0/3471311317 >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 0x7fe73806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 -- 172.21.15.46:0/3471311317 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380c75a0 con 0x7fe7380ad6c0 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d04a700 1 --2- 172.21.15.46:0/3471311317 >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 0x7fe73806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73d84b700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7380ad6c0 0x7fe7380adb20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe738063be0 tx=0x7fe730009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8650ab305ac92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe72ffff700 1 -- 172.21.15.46:0/3471311317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73000e070 con 0x7fe7380ad6c0 2023-10-06T11:18:19.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe72ffff700 1 -- 172.21.15.46:0/3471311317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73000b970 con 0x7fe7380ad6c0 2023-10-06T11:18:19.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe72ffff700 1 -- 172.21.15.46:0/3471311317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300092e0 con 0x7fe7380ad6c0 2023-10-06T11:18:19.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- 172.21.15.46:0/3471311317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7380ad6c0 msgr2=0x7fe7380adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7380ad6c0 0x7fe7380adb20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe738063be0 tx=0x7fe730009840 comp rx=0 tx=0).stop 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- 172.21.15.46:0/3471311317 shutdown_connections 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7380ad6c0 0x7fe7380adb20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/3471311317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe73806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/3471311317 >> v2:172.21.15.132:3300/0 conn(0x7fe73815ab60 0x7fe73806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.478+0000 7fe73f2ae700 1 -- 172.21.15.46:0/3471311317 >> 172.21.15.46:0/3471311317 conn(0x7fe7380cbe30 msgr2=0x7fe7380cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.482+0000 7fe73f2ae700 1 -- 172.21.15.46:0/3471311317 shutdown_connections 2023-10-06T11:18:19.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.482+0000 7fe73f2ae700 1 -- 172.21.15.46:0/3471311317 wait complete. 2023-10-06T11:18:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.482+0000 7fe73f2ae700 1 Processor -- start 2023-10-06T11:18:19.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.482+0000 7fe73f2ae700 1 -- start start 2023-10-06T11:18:19.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7380ad6c0 0x7fe738153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe7380ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380b1830 con 0x7fe73815ab60 2023-10-06T11:18:19.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738156550 con 0x7fe7380ad6c0 2023-10-06T11:18:19.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe738156830 con 0x7fe73815af30 2023-10-06T11:18:19.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39616/0 (socket says 172.21.15.46:39616) 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 -- 172.21.15.46:0/1738155186 learned_addr learned my addr 172.21.15.46:0/1738155186 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73d04a700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7380ad6c0 0x7fe738153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73d84b700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe7380ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 -- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 msgr2=0x7fe7380ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe7380ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 -- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7380ad6c0 msgr2=0x7fe738153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7380ad6c0 0x7fe738153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 -- 172.21.15.46:0/1738155186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340093c0 con 0x7fe73815ab60 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73c849700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe72800ec80 tx=0x7fe7280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800c850 con 0x7fe73815ab60 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72800c9b0 con 0x7fe73815ab60 2023-10-06T11:18:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800cd50 con 0x7fe73815ab60 2023-10-06T11:18:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe730009ae0 con 0x7fe73815ab60 2023-10-06T11:18:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.486+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe738156df0 con 0x7fe73815ab60 2023-10-06T11:18:19.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.490+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe738063f90 con 0x7fe73815ab60 2023-10-06T11:18:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.490+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe728013680 con 0x7fe73815ab60 2023-10-06T11:18:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.494+0000 7fe72d7fa700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 0x7fe71c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.494+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe72807c880 con 0x7fe73815ab60 2023-10-06T11:18:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.494+0000 7fe73d04a700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 0x7fe71c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.494+0000 7fe73d04a700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 0x7fe71c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe734009e90 tx=0x7fe734009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.498+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe72804cfc0 con 0x7fe73815ab60 2023-10-06T11:18:19.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.670+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe738068bf0 con 0x7fe73815ab60 2023-10-06T11:18:19.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.670+0000 7fe72d7fa700 1 -- 172.21.15.46:0/1738155186 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13647 (secure 0 0 0) 0x7fe7280505e0 con 0x7fe73815ab60 2023-10-06T11:18:19.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:19.678 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:18.687361+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 msgr2=0x7fe71c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 0x7fe71c062a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe734009e90 tx=0x7fe734009120 comp rx=0 tx=0).stop 2023-10-06T11:18:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 msgr2=0x7fe7380b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe72800ec80 tx=0x7fe7280091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 shutdown_connections 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe71c060550 0x7fe71c062a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe73815af30 0x7fe7380ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73815ab60 0x7fe7380b12f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 --2- 172.21.15.46:0/1738155186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7380ad6c0 0x7fe738153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 >> 172.21.15.46:0/1738155186 conn(0x7fe7380cbe30 msgr2=0x7fe7380c1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 shutdown_connections 2023-10-06T11:18:19.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.674+0000 7fe73f2ae700 1 -- 172.21.15.46:0/1738155186 wait complete. 2023-10-06T11:18:19.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:18:19.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:19.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.858+0000 7f89322cd700 1 Processor -- start 2023-10-06T11:18:19.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.858+0000 7f89322cd700 1 -- start start 2023-10-06T11:18:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f89322cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f892c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f89322cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0cd220 con 0x7f892c157330 2023-10-06T11:18:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f89322cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0cc610 con 0x7f892c0b00f0 2023-10-06T11:18:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f89322cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0cc8f0 con 0x7f892c156f60 2023-10-06T11:18:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f8923fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f892c157330 0x7f892c0af9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46006/0 (socket says 172.21.15.46:46006) 2023-10-06T11:18:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f8923fff700 1 -- 172.21.15.46:0/3803161643 learned_addr learned my addr 172.21.15.46:0/3803161643 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892bfff700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f892c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218319360 0 0) 0x7f892c0cd220 con 0x7f892c157330 2023-10-06T11:18:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8918003570 con 0x7f892c157330 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327539780 0 0) 0x7f8918003570 con 0x7f892c157330 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 msgr2=0x7f892c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f892c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 >> v1:172.21.15.132:6789/0 conn(0x7f892c0b00f0 legacy=0x7f892c0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0ccbd0 con 0x7f892c157330 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f8914001470 con 0x7f892c157330 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 >> v1:172.21.15.46:6789/0 conn(0x7f892c157330 legacy=0x7f892c0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f8918003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004010 0x7f8918004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8918004d50 0x7f89180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0cc8f0 con 0x7f8918004d50 2023-10-06T11:18:19.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8918010000 con 0x7f8918004010 2023-10-06T11:18:19.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89180101c0 con 0x7f892c156f60 2023-10-06T11:18:19.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f893086a700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8918004d50 0x7f89180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f8923fff700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004010 0x7f8918004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.862+0000 7f892bfff700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f8918003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 -- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 msgr2=0x7f8918003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c156f60 0x7f8918003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 -- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004010 msgr2=0x7f8918004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004010 0x7f8918004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 -- 172.21.15.46:0/3803161643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8918003570 con 0x7f8918004d50 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8918004d50 0x7f89180051c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8924001f90 tx=0x7f8924001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89561e98448ef85a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924001950 con 0x7f8918004d50 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400c8b0 con 0x7f8918004d50 2023-10-06T11:18:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f892b7fe700 1 -- 172.21.15.46:0/3803161643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400cc70 con 0x7f8918004d50 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 -- 172.21.15.46:0/3803161643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8918004d50 msgr2=0x7f89180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 --2- 172.21.15.46:0/3803161643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8918004d50 0x7f89180051c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8924001f90 tx=0x7f8924001640 comp rx=0 tx=0).stop 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f893086a700 1 -- 172.21.15.46:0/3803161643 reap_dead start 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 -- 172.21.15.46:0/3803161643 shutdown_connections 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 -- 172.21.15.46:0/3803161643 >> 172.21.15.46:0/3803161643 conn(0x7f892c06ae60 msgr2=0x7f892c06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 -- 172.21.15.46:0/3803161643 shutdown_connections 2023-10-06T11:18:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.866+0000 7f89322cd700 1 -- 172.21.15.46:0/3803161643 wait complete. 2023-10-06T11:18:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 Processor -- start 2023-10-06T11:18:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 -- start start 2023-10-06T11:18:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004d50 0x7f892c14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c1660f0 0x7f892c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c155d30 con 0x7f892c165c80 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c14f170 con 0x7f8918004d50 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f89322cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c165240 con 0x7f892c1660f0 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39632/0 (socket says 172.21.15.46:39632) 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 -- 172.21.15.46:0/2807079800 learned_addr learned my addr 172.21.15.46:0/2807079800 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f892bfff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004d50 0x7f892c14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f893086a700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c1660f0 0x7f892c1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 -- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c1660f0 msgr2=0x7f892c1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c1660f0 0x7f892c1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 -- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004d50 msgr2=0x7f892c14ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004d50 0x7f892c14ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 -- 172.21.15.46:0/2807079800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c1657f0 con 0x7f892c165c80 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.870+0000 7f8923fff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89140015a0 tx=0x7f8914001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914018030 con 0x7f892c165c80 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891400f630 con 0x7f892c165c80 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c14f620 con 0x7f892c165c80 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400f9f0 con 0x7f892c165c80 2023-10-06T11:18:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c155a50 con 0x7f892c165c80 2023-10-06T11:18:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c165980 con 0x7f892c165c80 2023-10-06T11:18:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.874+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f891401b070 con 0x7f892c165c80 2023-10-06T11:18:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.878+0000 7f8928ff9700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 0x7f890c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.878+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8914019070 con 0x7f892c165c80 2023-10-06T11:18:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.878+0000 7f892bfff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 0x7f890c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.878+0000 7f892bfff700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 0x7f890c062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f891c006f00 tx=0x7f891c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:19.886+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f891404cbb0 con 0x7f892c165c80 2023-10-06T11:18:20.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.050+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f892c0cceb0 con 0x7f892c165c80 2023-10-06T11:18:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.050+0000 7f8928ff9700 1 -- 172.21.15.46:0/2807079800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89140501d0 con 0x7f892c165c80 2023-10-06T11:18:20.057 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 msgr2=0x7f890c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 0x7f890c062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f891c006f00 tx=0x7f891c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 msgr2=0x7f892c154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89140015a0 tx=0x7f8914001e50 comp rx=0 tx=0).stop 2023-10-06T11:18:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 shutdown_connections 2023-10-06T11:18:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f890c060550 0x7f890c062a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f892c1660f0 0x7f892c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f892c165c80 0x7f892c154b90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 --2- 172.21.15.46:0/2807079800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8918004d50 0x7f892c14ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 >> 172.21.15.46:0/2807079800 conn(0x7f892c06ae60 msgr2=0x7f892c0ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 shutdown_connections 2023-10-06T11:18:20.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.054+0000 7f89322cd700 1 -- 172.21.15.46:0/2807079800 wait complete. 2023-10-06T11:18:20.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:20.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.242+0000 7fb741145700 1 Processor -- start 2023-10-06T11:18:20.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.242+0000 7fb741145700 1 -- start start 2023-10-06T11:18:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb741145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb741145700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb73c0b1710 0x7fb73c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb741145700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c15a5c0 con 0x7fb73c068160 2023-10-06T11:18:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb741145700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c1597e0 con 0x7fb73c0b1710 2023-10-06T11:18:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb741145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0b1b20 con 0x7fb73c069c10 2023-10-06T11:18:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb73c068160 0x7fb73c0b1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46014/0 (socket says 172.21.15.46:46014) 2023-10-06T11:18:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73a59c700 1 -- 172.21.15.46:0/894302520 learned_addr learned my addr 172.21.15.46:0/894302520 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/894302520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73b59e700 1 --2- 172.21.15.46:0/894302520 >> v2:172.21.15.132:3300/0 conn(0x7fb73c0b1710 0x7fb73c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb739d9b700 1 -- 172.21.15.46:0/894302520 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884253906 0 0) 0x7fb73c15a5c0 con 0x7fb73c068160 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb739d9b700 1 -- 172.21.15.46:0/894302520 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb724003570 con 0x7fb73c068160 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 -- 172.21.15.46:0/894302520 >> v2:172.21.15.132:3300/0 conn(0x7fb73c0b1710 msgr2=0x7fb73c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/894302520 >> v2:172.21.15.132:3300/0 conn(0x7fb73c0b1710 0x7fb73c0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 -- 172.21.15.46:0/894302520 >> v1:172.21.15.46:6789/0 conn(0x7fb73c068160 legacy=0x7fb73c0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 -- 172.21.15.46:0/894302520 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c1599f0 con 0x7fb73c069c10 2023-10-06T11:18:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.246+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/894302520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c0af9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0098e0 tx=0x7fb72c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25730c58b2ce3e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb739d9b700 1 -- 172.21.15.46:0/894302520 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00b640 con 0x7fb73c069c10 2023-10-06T11:18:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb739d9b700 1 -- 172.21.15.46:0/894302520 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c00b7a0 con 0x7fb73c069c10 2023-10-06T11:18:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb739d9b700 1 -- 172.21.15.46:0/894302520 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00bb40 con 0x7fb73c069c10 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 -- 172.21.15.46:0/894302520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 msgr2=0x7fb73c0af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 --2- 172.21.15.46:0/894302520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c0af9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0098e0 tx=0x7fb72c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 -- 172.21.15.46:0/894302520 shutdown_connections 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 --2- 172.21.15.46:0/894302520 >> v2:172.21.15.132:3300/0 conn(0x7fb73c0b1710 0x7fb73c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 --2- 172.21.15.46:0/894302520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c0af9e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 -- 172.21.15.46:0/894302520 >> 172.21.15.46:0/894302520 conn(0x7fb73c06ae60 msgr2=0x7fb73c06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 -- 172.21.15.46:0/894302520 shutdown_connections 2023-10-06T11:18:20.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.250+0000 7fb741145700 1 -- 172.21.15.46:0/894302520 wait complete. 2023-10-06T11:18:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 Processor -- start 2023-10-06T11:18:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 -- start start 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb73c0b1710 0x7fb73c16a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c13e2a0 con 0x7fb73c068160 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c154380 con 0x7fb73c0b1710 2023-10-06T11:18:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb741145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c13dce0 con 0x7fb73c069c10 2023-10-06T11:18:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39640/0 (socket says 172.21.15.46:39640) 2023-10-06T11:18:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 -- 172.21.15.46:0/2166434828 learned_addr learned my addr 172.21.15.46:0/2166434828 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73a59c700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73b59e700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb73c0b1710 0x7fb73c16a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 -- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 msgr2=0x7fb73c169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c169bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 -- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb73c0b1710 msgr2=0x7fb73c16a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb73c0b1710 0x7fb73c16a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.254+0000 7fb73ad9d700 1 -- 172.21.15.46:0/2166434828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009420 con 0x7fb73c068160 2023-10-06T11:18:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb73a59c700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb73ad9d700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00a010 tx=0x7fb72c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c01bc10 con 0x7fb73c068160 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c01bd70 con 0x7fb73c068160 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c13dfc0 con 0x7fb73c068160 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c025670 con 0x7fb73c068160 2023-10-06T11:18:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c13e850 con 0x7fb73c068160 2023-10-06T11:18:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.258+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c153e70 con 0x7fb73c068160 2023-10-06T11:18:20.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.262+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb72c023070 con 0x7fb73c068160 2023-10-06T11:18:20.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.262+0000 7fb722ffd700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 0x7fb728062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.262+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb72c07db60 con 0x7fb73c068160 2023-10-06T11:18:20.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.262+0000 7fb73a59c700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 0x7fb728062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.266+0000 7fb73a59c700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 0x7fb728062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0ce950 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.270+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb73c153e70 con 0x7fb73c068160 2023-10-06T11:18:20.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.442+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb73c153e70 con 0x7fb73c068160 2023-10-06T11:18:20.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.442+0000 7fb722ffd700 1 -- 172.21.15.46:0/2166434828 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb73c153e70 con 0x7fb73c068160 2023-10-06T11:18:20.448 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:20.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 msgr2=0x7fb728062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 0x7fb728062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0ce950 tx=0x7fb730009040 comp rx=0 tx=0).stop 2023-10-06T11:18:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 msgr2=0x7fb73c13d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00a010 tx=0x7fb72c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 shutdown_connections 2023-10-06T11:18:20.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb728060550 0x7fb728062a00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb73c0b1710 0x7fb73c16a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb73c069c10 0x7fb73c169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 --2- 172.21.15.46:0/2166434828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb73c068160 0x7fb73c13d1f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 >> 172.21.15.46:0/2166434828 conn(0x7fb73c06ae60 msgr2=0x7fb73c15aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 shutdown_connections 2023-10-06T11:18:20.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.446+0000 7fb741145700 1 -- 172.21.15.46:0/2166434828 wait complete. 2023-10-06T11:18:20.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:18:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.626+0000 7fcee9545700 1 Processor -- start 2023-10-06T11:18:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.626+0000 7fcee9545700 1 -- start start 2023-10-06T11:18:20.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcee40c1ec0 0x7fcee40ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee406d460 con 0x7fcee406e8a0 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee40c7000 con 0x7fcee415ab60 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee406ed40 con 0x7fcee40c1ec0 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee37fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcee40c1ec0 0x7fcee40ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcedbfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcee406e8a0 0x7fcee406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46020/0 (socket says 172.21.15.46:46020) 2023-10-06T11:18:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcedbfff700 1 -- 172.21.15.46:0/3751519599 learned_addr learned my addr 172.21.15.46:0/3751519599 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee27fc700 1 -- 172.21.15.46:0/3751519599 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678373907 0 0) 0x7fcee406d460 con 0x7fcee406e8a0 2023-10-06T11:18:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee37fe700 1 -- 172.21.15.46:0/3751519599 >> v1:172.21.15.132:6789/0 conn(0x7fcee415ab60 legacy=0x7fcee415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee37fe700 1 -- 172.21.15.46:0/3751519599 >> v1:172.21.15.46:6789/0 conn(0x7fcee406e8a0 legacy=0x7fcee406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee37fe700 1 -- 172.21.15.46:0/3751519599 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40c72c0 con 0x7fcee40c1ec0 2023-10-06T11:18:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee37fe700 1 --2- 172.21.15.46:0/3751519599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee40c1ec0 0x7fcee40ad6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009e60 tx=0x7fcedc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a811d1ebb39a4cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee27fc700 1 -- 172.21.15.46:0/3751519599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00e070 con 0x7fcee40c1ec0 2023-10-06T11:18:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee27fc700 1 -- 172.21.15.46:0/3751519599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc00bc30 con 0x7fcee40c1ec0 2023-10-06T11:18:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee27fc700 1 -- 172.21.15.46:0/3751519599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc012430 con 0x7fcee40c1ec0 2023-10-06T11:18:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- 172.21.15.46:0/3751519599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee40c1ec0 msgr2=0x7fcee40ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 --2- 172.21.15.46:0/3751519599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee40c1ec0 0x7fcee40ad6c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009e60 tx=0x7fcedc009780 comp rx=0 tx=0).stop 2023-10-06T11:18:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- 172.21.15.46:0/3751519599 shutdown_connections 2023-10-06T11:18:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 --2- 172.21.15.46:0/3751519599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee40c1ec0 0x7fcee40ad6c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.630+0000 7fcee9545700 1 -- 172.21.15.46:0/3751519599 >> 172.21.15.46:0/3751519599 conn(0x7fcee40cbe30 msgr2=0x7fcee40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- 172.21.15.46:0/3751519599 shutdown_connections 2023-10-06T11:18:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- 172.21.15.46:0/3751519599 wait complete. 2023-10-06T11:18:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 Processor -- start 2023-10-06T11:18:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- start start 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee40c1ec0 0x7fcee40ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee415ab60 0x7fcee416c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40af260 con 0x7fcee406e8a0 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40c9250 con 0x7fcee415ab60 2023-10-06T11:18:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee9545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee40c9e10 con 0x7fcee40c1ec0 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.634+0000 7fcee2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39654/0 (socket says 172.21.15.46:39654) 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 -- 172.21.15.46:0/2124499173 learned_addr learned my addr 172.21.15.46:0/2124499173 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcedbfff700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee40c1ec0 0x7fcee40ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee37fe700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee415ab60 0x7fcee416c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 -- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee40c1ec0 msgr2=0x7fcee40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee40c1ec0 0x7fcee40ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 -- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee415ab60 msgr2=0x7fcee416c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee415ab60 0x7fcee416c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 -- 172.21.15.46:0/2124499173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc009a20 con 0x7fcee406e8a0 2023-10-06T11:18:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee2ffd700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fced400c5a0 tx=0x7fced4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4011590 con 0x7fcee406e8a0 2023-10-06T11:18:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced40116f0 con 0x7fcee406e8a0 2023-10-06T11:18:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4011ab0 con 0x7fcee406e8a0 2023-10-06T11:18:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee40c8920 con 0x7fcee406e8a0 2023-10-06T11:18:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee40c8ee0 con 0x7fcee406e8a0 2023-10-06T11:18:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.638+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee4063f90 con 0x7fcee406e8a0 2023-10-06T11:18:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.642+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fced401b3b0 con 0x7fcee406e8a0 2023-10-06T11:18:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.642+0000 7fcedaffd700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 0x7fcec4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.642+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fced407ba90 con 0x7fcee406e8a0 2023-10-06T11:18:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.642+0000 7fcedbfff700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 0x7fcec4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.646+0000 7fcedbfff700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 0x7fcec4062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcee415ca50 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.650+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fced404c280 con 0x7fcee406e8a0 2023-10-06T11:18:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.826+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcee404b5c0 con 0x7fcee406e8a0 2023-10-06T11:18:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.826+0000 7fcedaffd700 1 -- 172.21.15.46:0/2124499173 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7fced4017020 con 0x7fcee406e8a0 2023-10-06T11:18:20.833 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:20.833 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:18:20.834 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:18:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 msgr2=0x7fcec4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 0x7fcec4062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcee415ca50 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2023-10-06T11:18:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 msgr2=0x7fcee40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fced400c5a0 tx=0x7fced4001070 comp rx=0 tx=0).stop 2023-10-06T11:18:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 shutdown_connections 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcec4060550 0x7fcec4062a00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee415ab60 0x7fcee416c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee40c1ec0 0x7fcee40ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 --2- 172.21.15.46:0/2124499173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcee406e8a0 0x7fcee40aebd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 >> 172.21.15.46:0/2124499173 conn(0x7fcee40cbe30 msgr2=0x7fcee4159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 shutdown_connections 2023-10-06T11:18:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:20.830+0000 7fcee9545700 1 -- 172.21.15.46:0/2124499173 wait complete. 2023-10-06T11:18:20.855 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:18:20.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:18:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.002+0000 7f3b6fe2e700 1 Processor -- start 2023-10-06T11:18:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.002+0000 7f3b6fe2e700 1 -- start start 2023-10-06T11:18:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6fe2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b681592d0 0x7f3b680acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6fe2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b680c6950 0x7f3b680ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6fe2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b680c5db0 con 0x7f3b681596a0 2023-10-06T11:18:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6fe2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b680c5ab0 con 0x7f3b680c6950 2023-10-06T11:18:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6fe2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b680ad2f0 con 0x7f3b681592d0 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6d3c9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3b681596a0 0x7f3b680c6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46024/0 (socket says 172.21.15.46:46024) 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6d3c9700 1 -- 172.21.15.46:0/1894875361 learned_addr learned my addr 172.21.15.46:0/1894875361 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.006+0000 7f3b6e3cb700 1 --2- 172.21.15.46:0/1894875361 >> v2:172.21.15.46:3300/0 conn(0x7f3b680c6950 0x7f3b680ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6dbca700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b681592d0 0x7f3b680acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6cbc8700 1 -- 172.21.15.46:0/1894875361 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438024821 0 0) 0x7f3b680c5db0 con 0x7f3b681596a0 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6cbc8700 1 -- 172.21.15.46:0/1894875361 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b54003570 con 0x7f3b681596a0 2023-10-06T11:18:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6e3cb700 1 -- 172.21.15.46:0/1894875361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b681592d0 msgr2=0x7f3b680acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6e3cb700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b681592d0 0x7f3b680acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6e3cb700 1 -- 172.21.15.46:0/1894875361 >> v1:172.21.15.46:6789/0 conn(0x7f3b681596a0 legacy=0x7f3b680c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6e3cb700 1 -- 172.21.15.46:0/1894875361 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b680d0aa0 con 0x7f3b680c6950 2023-10-06T11:18:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6e3cb700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b680c6950 0x7f3b680ce040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009e60 tx=0x7f3b60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89529634f4efdda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6cbc8700 1 -- 172.21.15.46:0/1894875361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000e070 con 0x7f3b680c6950 2023-10-06T11:18:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6cbc8700 1 -- 172.21.15.46:0/1894875361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6000bab0 con 0x7f3b680c6950 2023-10-06T11:18:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6cbc8700 1 -- 172.21.15.46:0/1894875361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000be50 con 0x7f3b680c6950 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/1894875361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b680c6950 msgr2=0x7f3b680ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b680c6950 0x7f3b680ce040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009e60 tx=0x7f3b60009780 comp rx=0 tx=0).stop 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/1894875361 shutdown_connections 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b680c6950 0x7f3b680ce040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/1894875361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b681592d0 0x7f3b680acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/1894875361 >> 172.21.15.46:0/1894875361 conn(0x7f3b6806ef60 msgr2=0x7f3b680d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/1894875361 shutdown_connections 2023-10-06T11:18:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.010+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/1894875361 wait complete. 2023-10-06T11:18:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 Processor -- start 2023-10-06T11:18:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 -- start start 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b680c6950 0x7f3b68099f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b681596a0 0x7f3b6816ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b680ca790 con 0x7f3b681592d0 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b680b3fd0 con 0x7f3b681596a0 2023-10-06T11:18:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6fe2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6816dca0 con 0x7f3b680c6950 2023-10-06T11:18:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39666/0 (socket says 172.21.15.46:39666) 2023-10-06T11:18:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 -- 172.21.15.46:0/2960515899 learned_addr learned my addr 172.21.15.46:0/2960515899 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6dbca700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b680c6950 0x7f3b68099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6e3cb700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b681596a0 0x7f3b6816ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 -- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b680c6950 msgr2=0x7f3b68099f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b680c6950 0x7f3b68099f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 -- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b681596a0 msgr2=0x7f3b6816ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b681596a0 0x7f3b6816ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 -- 172.21.15.46:0/2960515899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b60009a20 con 0x7f3b681592d0 2023-10-06T11:18:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.014+0000 7f3b6d3c9700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c790 tx=0x7f3b58007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580072e0 con 0x7f3b681592d0 2023-10-06T11:18:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b58007440 con 0x7f3b681592d0 2023-10-06T11:18:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b680b4af0 con 0x7f3b681592d0 2023-10-06T11:18:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580057f0 con 0x7f3b681592d0 2023-10-06T11:18:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b680b44a0 con 0x7f3b681592d0 2023-10-06T11:18:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b68148280 con 0x7f3b681592d0 2023-10-06T11:18:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.018+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3b58005be0 con 0x7f3b681592d0 2023-10-06T11:18:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.022+0000 7f3b5dffb700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 0x7f3b4c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.022+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3b5807b890 con 0x7f3b681592d0 2023-10-06T11:18:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.022+0000 7f3b6dbca700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 0x7f3b4c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.022+0000 7f3b6dbca700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 0x7f3b4c062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009d00 tx=0x7f3b64009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.026+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3b58049be0 con 0x7f3b681592d0 2023-10-06T11:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.198+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3b68069e00 con 0x7f3b681592d0 2023-10-06T11:18:21.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.738+0000 7f3b5dffb700 1 -- 172.21.15.46:0/2960515899 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b580170b0 con 0x7f3b681592d0 2023-10-06T11:18:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 msgr2=0x7f3b4c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 0x7f3b4c062a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009d00 tx=0x7f3b64009120 comp rx=0 tx=0).stop 2023-10-06T11:18:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 msgr2=0x7f3b68147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c790 tx=0x7f3b58007c80 comp rx=0 tx=0).stop 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 shutdown_connections 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b4c060550 0x7f3b4c062a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b681596a0 0x7f3b6816ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b681592d0 0x7f3b68147d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 --2- 172.21.15.46:0/2960515899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b680c6950 0x7f3b68099f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 >> 172.21.15.46:0/2960515899 conn(0x7f3b6806ef60 msgr2=0x7f3b680cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 shutdown_connections 2023-10-06T11:18:21.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.746+0000 7f3b6fe2e700 1 -- 172.21.15.46:0/2960515899 wait complete. 2023-10-06T11:18:21.762 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:18:21.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:18:21.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.942+0000 7f9ef6f63700 1 Processor -- start 2023-10-06T11:18:21.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.942+0000 7f9ef6f63700 1 -- start start 2023-10-06T11:18:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.942+0000 7f9ef6f63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9ef00d0750 0x7f9ef00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.942+0000 7f9ef6f63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6f63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef00cf050 con 0x7f9ef00cb020 2023-10-06T11:18:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6f63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00cf8f0 con 0x7f9ef00d0750 2023-10-06T11:18:21.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6f63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00ce440 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:33190/0 (socket says 172.21.15.46:33190) 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 -- 172.21.15.46:0/4111478820 learned_addr learned my addr 172.21.15.46:0/4111478820 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef5760700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9ef00d0750 0x7f9ef00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 -- 172.21.15.46:0/4111478820 >> v2:172.21.15.132:3300/0 conn(0x7f9ef00d0750 msgr2=0x7f9ef00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 --2- 172.21.15.46:0/4111478820 >> v2:172.21.15.132:3300/0 conn(0x7f9ef00d0750 0x7f9ef00cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 -- 172.21.15.46:0/4111478820 >> v1:172.21.15.132:6789/0 conn(0x7f9ef00cb020 legacy=0x7f9ef00cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 -- 172.21.15.46:0/4111478820 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00ce720 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef6762700 1 --2- 172.21.15.46:0/4111478820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009f20 tx=0x7f9ee8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af687fa32d880f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef4f5f700 1 -- 172.21.15.46:0/4111478820 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800e070 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef4f5f700 1 -- 172.21.15.46:0/4111478820 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee800b9d0 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.946+0000 7f9ef4f5f700 1 -- 172.21.15.46:0/4111478820 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80092e0 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- 172.21.15.46:0/4111478820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 msgr2=0x7f9ef0158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/4111478820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009f20 tx=0x7f9ee8009840 comp rx=0 tx=0).stop 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- 172.21.15.46:0/4111478820 shutdown_connections 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/4111478820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00cf4e0 0x7f9ef0158bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/4111478820 >> v2:172.21.15.132:3300/0 conn(0x7f9ef00d0750 0x7f9ef00cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- 172.21.15.46:0/4111478820 >> 172.21.15.46:0/4111478820 conn(0x7f9ef00683d0 msgr2=0x7f9ef00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- 172.21.15.46:0/4111478820 shutdown_connections 2023-10-06T11:18:21.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- 172.21.15.46:0/4111478820 wait complete. 2023-10-06T11:18:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 Processor -- start 2023-10-06T11:18:21.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.950+0000 7f9ef6f63700 1 -- start start 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 0x7f9ef014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 0x7f9ef00b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c8500 con 0x7f9ef00cb020 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef014dd00 con 0x7f9ef00cf4e0 2023-10-06T11:18:21.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c7ca0 con 0x7f9ef00d0750 2023-10-06T11:18:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39678/0 (socket says 172.21.15.46:39678) 2023-10-06T11:18:21.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 -- 172.21.15.46:0/2469767677 learned_addr learned my addr 172.21.15.46:0/2469767677 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5760700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 0x7f9ef014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6762700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 0x7f9ef00b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 -- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 msgr2=0x7f9ef00b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 0x7f9ef00b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 -- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 msgr2=0x7f9ef014d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 0x7f9ef014d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 -- 172.21.15.46:0/2469767677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8009ae0 con 0x7f9ef00cb020 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5760700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 0x7f9ef014d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6762700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 0x7f9ef00b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef5f61700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c590 tx=0x7f9eec007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011590 con 0x7f9ef00cb020 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec011710 con 0x7f9ef00cb020 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011ad0 con 0x7f9ef00cb020 2023-10-06T11:18:21.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef014e690 con 0x7f9ef00cb020 2023-10-06T11:18:21.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.954+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef00c8220 con 0x7f9ef00cb020 2023-10-06T11:18:21.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.958+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0064590 con 0x7f9ef00cb020 2023-10-06T11:18:21.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.958+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9eec01b3b0 con 0x7f9ef00cb020 2023-10-06T11:18:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.962+0000 7f9ee5ffb700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 0x7f9ed4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:21.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.962+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f9eec07bca0 con 0x7f9ef00cb020 2023-10-06T11:18:21.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.962+0000 7f9ef5760700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 0x7f9ed4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:21.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.962+0000 7f9ef5760700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 0x7f9ed4062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0156620 tx=0x7f9ee0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:21.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:21.966+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9eec04c270 con 0x7f9ef00cb020 2023-10-06T11:18:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.142+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ef00cea00 con 0x7f9ef00cb020 2023-10-06T11:18:22.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.750+0000 7f9ee5ffb700 1 -- 172.21.15.46:0/2469767677 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9eec017020 con 0x7f9ef00cb020 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 msgr2=0x7f9ed4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 0x7f9ed4062a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0156620 tx=0x7f9ee0008040 comp rx=0 tx=0).stop 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 msgr2=0x7f9ef014e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c590 tx=0x7f9eec007f30 comp rx=0 tx=0).stop 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 shutdown_connections 2023-10-06T11:18:22.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9ed4060550 0x7f9ed4062a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ef00d0750 0x7f9ef00b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ef00cf4e0 0x7f9ef014d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 --2- 172.21.15.46:0/2469767677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef00cb020 0x7f9ef014e000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.778+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 >> 172.21.15.46:0/2469767677 conn(0x7f9ef00683d0 msgr2=0x7f9ef00b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:22.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.782+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 shutdown_connections 2023-10-06T11:18:22.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.782+0000 7f9ef6f63700 1 -- 172.21.15.46:0/2469767677 wait complete. 2023-10-06T11:18:22.790 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:18:22.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:18:22.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.966+0000 7f1456c43700 1 Processor -- start 2023-10-06T11:18:22.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.966+0000 7f1456c43700 1 -- start start 2023-10-06T11:18:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500adbe0 0x7f14500d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14500d0d60 0x7f14500b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14500afd20 con 0x7f14500b3630 2023-10-06T11:18:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1450159740 con 0x7f14500d0d60 2023-10-06T11:18:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14500b0000 con 0x7f14500adbe0 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14500d0d60 0x7f14500b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14500d0d60 0x7f14500b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39682/0 (socket says 172.21.15.46:39682) 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 -- 172.21.15.46:0/3405078914 learned_addr learned my addr 172.21.15.46:0/3405078914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455c41700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500adbe0 0x7f14500d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 -- 172.21.15.46:0/3405078914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500adbe0 msgr2=0x7f14500d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:22.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500adbe0 0x7f14500d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 -- 172.21.15.46:0/3405078914 >> v1:172.21.15.46:6789/0 conn(0x7f14500b3630 legacy=0x7f14500b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1454c3f700 1 -- 172.21.15.46:0/3405078914 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878894206 0 0) 0x7f14500afd20 con 0x7f14500b3630 2023-10-06T11:18:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 -- 172.21.15.46:0/3405078914 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14500b02e0 con 0x7f14500d0d60 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1455440700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14500b30f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14400099b0 tx=0x7f1440009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415acfa7dc84068d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1454c3f700 1 -- 172.21.15.46:0/3405078914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000b860 con 0x7f14500d0d60 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1454c3f700 1 -- 172.21.15.46:0/3405078914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144000b9c0 con 0x7f14500d0d60 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1454c3f700 1 -- 172.21.15.46:0/3405078914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000bd60 con 0x7f14500d0d60 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- 172.21.15.46:0/3405078914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 msgr2=0x7f14500b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14500b30f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14400099b0 tx=0x7f1440009250 comp rx=0 tx=0).stop 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- 172.21.15.46:0/3405078914 shutdown_connections 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14500b30f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 --2- 172.21.15.46:0/3405078914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500adbe0 0x7f14500d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- 172.21.15.46:0/3405078914 >> 172.21.15.46:0/3405078914 conn(0x7f145006ef60 msgr2=0x7f14500d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- 172.21.15.46:0/3405078914 shutdown_connections 2023-10-06T11:18:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.970+0000 7f1456c43700 1 -- 172.21.15.46:0/3405078914 wait complete. 2023-10-06T11:18:22.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 Processor -- start 2023-10-06T11:18:22.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 -- start start 2023-10-06T11:18:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14500adbe0 0x7f145015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500b3630 0x7f145015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450155b60 con 0x7f14500d0d60 2023-10-06T11:18:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145015c810 con 0x7f14500adbe0 2023-10-06T11:18:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456c43700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14501553a0 con 0x7f14500b3630 2023-10-06T11:18:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.974+0000 7f1456442700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1455c41700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14500adbe0 0x7f145015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39688/0 (socket says 172.21.15.46:39688) 2023-10-06T11:18:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 -- 172.21.15.46:0/3914430366 learned_addr learned my addr 172.21.15.46:0/3914430366 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1455440700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500b3630 0x7f145015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 -- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500b3630 msgr2=0x7f145015c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500b3630 0x7f145015c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 -- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14500adbe0 msgr2=0x7f145015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14500adbe0 0x7f145015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 -- 172.21.15.46:0/3914430366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400094f0 con 0x7f14500d0d60 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456442700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1448001560 tx=0x7f14480016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448012630 con 0x7f14500d0d60 2023-10-06T11:18:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448012790 con 0x7f14500d0d60 2023-10-06T11:18:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448012b50 con 0x7f14500d0d60 2023-10-06T11:18:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145015a1b0 con 0x7f14500d0d60 2023-10-06T11:18:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.978+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145015a7d0 con 0x7f14500d0d60 2023-10-06T11:18:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.982+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1450064590 con 0x7f14500d0d60 2023-10-06T11:18:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.982+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1448003030 con 0x7f14500d0d60 2023-10-06T11:18:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.982+0000 7f1445ffb700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 0x7f1434062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.982+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f144807c730 con 0x7f14500d0d60 2023-10-06T11:18:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.986+0000 7f1455c41700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 0x7f1434062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.986+0000 7f1455c41700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 0x7f1434062a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f144c006fd0 tx=0x7f144c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:23.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:22.990+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f144804cc20 con 0x7f14500d0d60 2023-10-06T11:18:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:23.202+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f14500af510 con 0x7f14500d0d60 2023-10-06T11:18:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.058+0000 7f1445ffb700 1 -- 172.21.15.46:0/3914430366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1448050240 con 0x7f14500d0d60 2023-10-06T11:18:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 msgr2=0x7f1434062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 0x7f1434062a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f144c006fd0 tx=0x7f144c008040 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 msgr2=0x7f14501504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1448001560 tx=0x7f14480016c0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 shutdown_connections 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1434060580 0x7f1434062a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14500d0d60 0x7f14501504a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14500b3630 0x7f145015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 --2- 172.21.15.46:0/3914430366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14500adbe0 0x7f145015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 >> 172.21.15.46:0/3914430366 conn(0x7f145006ef60 msgr2=0x7f14500ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 shutdown_connections 2023-10-06T11:18:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.066+0000 7f1456c43700 1 -- 172.21.15.46:0/3914430366 wait complete. 2023-10-06T11:18:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T11:18:24.095 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.246+0000 7f9b9f59e700 1 Processor -- start 2023-10-06T11:18:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.246+0000 7f9b9f59e700 1 -- start start 2023-10-06T11:18:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 0x7f9ba0067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9f59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9ba01591a0 0x7f9ba0067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba015d6c0 con 0x7f9ba0068bf0 2023-10-06T11:18:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00699c0 con 0x7f9ba0158dd0 2023-10-06T11:18:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9f59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba015db50 con 0x7f9ba01591a0 2023-10-06T11:18:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9e59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 0x7f9ba0067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9e59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 0x7f9ba0067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52300/0 (socket says 172.21.15.46:52300) 2023-10-06T11:18:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9e59c700 1 -- 172.21.15.46:0/2936128496 learned_addr learned my addr 172.21.15.46:0/2936128496 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/2936128496 >> v2:172.21.15.132:3300/0 conn(0x7f9ba01591a0 0x7f9ba0067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 -- 172.21.15.46:0/2936128496 >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 msgr2=0x7f9ba0067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/2936128496 >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 0x7f9ba0067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 -- 172.21.15.46:0/2936128496 >> v1:172.21.15.132:6789/0 conn(0x7f9ba0068bf0 legacy=0x7f9ba0069050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 -- 172.21.15.46:0/2936128496 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0069760 con 0x7f9ba01591a0 2023-10-06T11:18:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/2936128496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba01591a0 0x7f9ba0067590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ba006a410 tx=0x7f9b880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0af1cc4ae59a0b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9dd9b700 1 -- 172.21.15.46:0/2936128496 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88011630 con 0x7f9ba01591a0 2023-10-06T11:18:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9dd9b700 1 -- 172.21.15.46:0/2936128496 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88011790 con 0x7f9ba01591a0 2023-10-06T11:18:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.250+0000 7f9b9dd9b700 1 -- 172.21.15.46:0/2936128496 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88011b30 con 0x7f9ba01591a0 2023-10-06T11:18:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 -- 172.21.15.46:0/2936128496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba01591a0 msgr2=0x7f9ba0067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/2936128496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba01591a0 0x7f9ba0067590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ba006a410 tx=0x7f9b880091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 -- 172.21.15.46:0/2936128496 shutdown_connections 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/2936128496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba01591a0 0x7f9ba0067590 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/2936128496 >> v2:172.21.15.46:3300/0 conn(0x7f9ba0158dd0 0x7f9ba0067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 -- 172.21.15.46:0/2936128496 >> 172.21.15.46:0/2936128496 conn(0x7f9ba0062ef0 msgr2=0x7f9ba00cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 -- 172.21.15.46:0/2936128496 shutdown_connections 2023-10-06T11:18:24.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.254+0000 7f9b9f59e700 1 -- 172.21.15.46:0/2936128496 wait complete. 2023-10-06T11:18:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 Processor -- start 2023-10-06T11:18:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 -- start start 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 0x7f9ba00c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ba0158dd0 0x7f9ba00c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 0x7f9ba00aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c4410 con 0x7f9ba01591a0 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c3800 con 0x7f9ba0068bf0 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c3ae0 con 0x7f9ba0158dd0 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 0x7f9ba00aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 0x7f9ba00aaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52304/0 (socket says 172.21.15.46:52304) 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 0x7f9ba00c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9ed9d700 1 -- 172.21.15.46:0/3241752814 learned_addr learned my addr 172.21.15.46:0/3241752814 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ba0158dd0 0x7f9ba00c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 -- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ba0158dd0 msgr2=0x7f9ba00c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ba0158dd0 0x7f9ba00c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 -- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 msgr2=0x7f9ba00aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 0x7f9ba00aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 -- 172.21.15.46:0/3241752814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88009420 con 0x7f9ba0068bf0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.258+0000 7f9b9e59c700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 0x7f9ba00c48a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006d40 tx=0x7f9b90009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90013680 con 0x7f9ba0068bf0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900137e0 con 0x7f9ba0068bf0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90013b60 con 0x7f9ba0068bf0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba00c3dc0 con 0x7f9ba0068bf0 2023-10-06T11:18:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba00c4de0 con 0x7f9ba0068bf0 2023-10-06T11:18:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0064590 con 0x7f9ba0068bf0 2023-10-06T11:18:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.262+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9b90005a00 con 0x7f9ba0068bf0 2023-10-06T11:18:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.266+0000 7f9b96ffd700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 0x7f9b80062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.266+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9b9007cd20 con 0x7f9ba0068bf0 2023-10-06T11:18:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.266+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 0x7f9b80062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.266+0000 7f9b95d9b700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 0x7f9b80062ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b880091e0 tx=0x7f9b88017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.274+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9b9004d210 con 0x7f9ba0068bf0 2023-10-06T11:18:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.450+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ba015cd90 con 0x7f9ba0068bf0 2023-10-06T11:18:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b96ffd700 1 -- 172.21.15.46:0/3241752814 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f9b90050830 con 0x7f9ba0068bf0 2023-10-06T11:18:24.459 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:24.459 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:23.205555+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 msgr2=0x7f9b80062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 0x7f9b80062ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b880091e0 tx=0x7f9b88017000 comp rx=0 tx=0).stop 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 msgr2=0x7f9ba00c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 0x7f9ba00c48a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b90006d40 tx=0x7f9b90009250 comp rx=0 tx=0).stop 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 shutdown_connections 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b80060620 0x7f9b80062ad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ba01591a0 0x7f9ba00aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ba0158dd0 0x7f9ba00c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 --2- 172.21.15.46:0/3241752814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ba0068bf0 0x7f9ba00c48a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 >> 172.21.15.46:0/3241752814 conn(0x7f9ba0062ef0 msgr2=0x7f9ba0158050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 shutdown_connections 2023-10-06T11:18:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.454+0000 7f9b9f59e700 1 -- 172.21.15.46:0/3241752814 wait complete. 2023-10-06T11:18:24.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:24.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 Processor -- start 2023-10-06T11:18:24.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 -- start start 2023-10-06T11:18:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b3c0b10b0 0x7f9b3c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c1588a0 con 0x7f9b3c0abd20 2023-10-06T11:18:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0acd20 con 0x7f9b3c157790 2023-10-06T11:18:24.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c157c90 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b3c0b10b0 0x7f9b3c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b3c0b10b0 0x7f9b3c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52320/0 (socket says 172.21.15.46:52320) 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 -- 172.21.15.46:0/1199419355 learned_addr learned my addr 172.21.15.46:0/1199419355 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 -- 172.21.15.46:0/1199419355 >> v1:172.21.15.132:6789/0 conn(0x7f9b3c157790 legacy=0x7f9b3c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 -- 172.21.15.46:0/1199419355 >> v1:172.21.15.46:6789/0 conn(0x7f9b3c0abd20 legacy=0x7f9b3c0ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.642+0000 7f9b3ad9d700 1 -- 172.21.15.46:0/1199419355 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c157f70 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3ad9d700 1 --2- 172.21.15.46:0/1199419355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c158d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009f20 tx=0x7f9b34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125aba133b72189e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3959a700 1 -- 172.21.15.46:0/1199419355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400e070 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3959a700 1 -- 172.21.15.46:0/1199419355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3400ba40 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3959a700 1 -- 172.21.15.46:0/1199419355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340092e0 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 -- 172.21.15.46:0/1199419355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 msgr2=0x7f9b3c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/1199419355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c158d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009f20 tx=0x7f9b34009840 comp rx=0 tx=0).stop 2023-10-06T11:18:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 -- 172.21.15.46:0/1199419355 shutdown_connections 2023-10-06T11:18:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/1199419355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c158d30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 -- 172.21.15.46:0/1199419355 >> 172.21.15.46:0/1199419355 conn(0x7f9b3c0ccd50 msgr2=0x7f9b3c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 -- 172.21.15.46:0/1199419355 shutdown_connections 2023-10-06T11:18:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.646+0000 7f9b3b59e700 1 -- 172.21.15.46:0/1199419355 wait complete. 2023-10-06T11:18:24.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 Processor -- start 2023-10-06T11:18:24.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 -- start start 2023-10-06T11:18:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b3c0abd20 0x7f9b3c0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b3c157790 0x7f9b3c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0c8a90 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0c2a50 con 0x7f9b3c0abd20 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c1698d0 con 0x7f9b3c157790 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52332/0 (socket says 172.21.15.46:52332) 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 -- 172.21.15.46:0/4112255032 learned_addr learned my addr 172.21.15.46:0/4112255032 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3a59c700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b3c0abd20 0x7f9b3c0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b3ad9d700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b3c157790 0x7f9b3c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 -- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b3c157790 msgr2=0x7f9b3c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b3c157790 0x7f9b3c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 -- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b3c0abd20 msgr2=0x7f9b3c0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b3c0abd20 0x7f9b3c0cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.650+0000 7f9b39d9b700 1 -- 172.21.15.46:0/4112255032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b34009ae0 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b39d9b700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000b5e0 tx=0x7f9b30009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000f510 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3000f670 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000fa30 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c0cb200 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c0cb840 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.654+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c064590 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.658+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9b3001a400 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.658+0000 7f9b2a7fc700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 0x7f9b1c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.658+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9b3007c3b0 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.658+0000 7f9b3a59c700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 0x7f9b1c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.662+0000 7f9b3a59c700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 0x7f9b1c062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c003a80 tx=0x7f9b2c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:24.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.666+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9b3004c820 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.842+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b3c158250 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.842+0000 7f9b2a7fc700 1 -- 172.21.15.46:0/4112255032 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f9b3004fe40 con 0x7f9b3c0b10b0 2023-10-06T11:18:24.851 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:24.851 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 msgr2=0x7f9b1c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 0x7f9b1c062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c003a80 tx=0x7f9b2c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 msgr2=0x7f9b3c0c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000b5e0 tx=0x7f9b30009120 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 shutdown_connections 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b1c060550 0x7f9b1c062a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b3c157790 0x7f9b3c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b3c0b10b0 0x7f9b3c0c2510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 --2- 172.21.15.46:0/4112255032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b3c0abd20 0x7f9b3c0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 >> 172.21.15.46:0/4112255032 conn(0x7f9b3c0ccd50 msgr2=0x7f9b3c1558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 shutdown_connections 2023-10-06T11:18:24.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:24.846+0000 7f9b3b59e700 1 -- 172.21.15.46:0/4112255032 wait complete. 2023-10-06T11:18:24.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:18:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.034+0000 7f856902e700 1 Processor -- start 2023-10-06T11:18:25.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.034+0000 7f856902e700 1 -- start start 2023-10-06T11:18:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856902e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f856406e8a0 0x7f85640c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856902e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f856406ed10 0x7f85640c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856902e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640c1bf0 con 0x7f85640c6470 2023-10-06T11:18:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856902e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640c72c0 con 0x7f856406e8a0 2023-10-06T11:18:25.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856902e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856406cf80 con 0x7f856406ed10 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f856406e8a0 0x7f85640c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f856406e8a0 0x7f85640c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52338/0 (socket says 172.21.15.46:52338) 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 -- 172.21.15.46:0/2393206795 learned_addr learned my addr 172.21.15.46:0/2393206795 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856359e700 1 --2- 172.21.15.46:0/2393206795 >> v2:172.21.15.132:3300/0 conn(0x7f856406ed10 0x7f85640c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 -- 172.21.15.46:0/2393206795 >> v2:172.21.15.132:3300/0 conn(0x7f856406ed10 msgr2=0x7f85640c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:25.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 --2- 172.21.15.46:0/2393206795 >> v2:172.21.15.132:3300/0 conn(0x7f856406ed10 0x7f85640c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 -- 172.21.15.46:0/2393206795 >> v1:172.21.15.132:6789/0 conn(0x7f85640c6470 legacy=0x7f85640ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 -- 172.21.15.46:0/2393206795 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856406d260 con 0x7f856406e8a0 2023-10-06T11:18:25.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f856259c700 1 --2- 172.21.15.46:0/2393206795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f85640c6840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f856406c800 tx=0x7f8558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d3580a932f0d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f8561d9b700 1 -- 172.21.15.46:0/2393206795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558009e70 con 0x7f856406e8a0 2023-10-06T11:18:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f8561d9b700 1 -- 172.21.15.46:0/2393206795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855800b860 con 0x7f856406e8a0 2023-10-06T11:18:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.038+0000 7f8561d9b700 1 -- 172.21.15.46:0/2393206795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855800bc00 con 0x7f856406e8a0 2023-10-06T11:18:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 -- 172.21.15.46:0/2393206795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 msgr2=0x7f85640c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 --2- 172.21.15.46:0/2393206795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f85640c6840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f856406c800 tx=0x7f8558009250 comp rx=0 tx=0).stop 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 -- 172.21.15.46:0/2393206795 shutdown_connections 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 --2- 172.21.15.46:0/2393206795 >> v2:172.21.15.132:3300/0 conn(0x7f856406ed10 0x7f85640c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 --2- 172.21.15.46:0/2393206795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f85640c6840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 -- 172.21.15.46:0/2393206795 >> 172.21.15.46:0/2393206795 conn(0x7f85640cbe30 msgr2=0x7f85640cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 -- 172.21.15.46:0/2393206795 shutdown_connections 2023-10-06T11:18:25.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.042+0000 7f856902e700 1 -- 172.21.15.46:0/2393206795 wait complete. 2023-10-06T11:18:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 Processor -- start 2023-10-06T11:18:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 -- start start 2023-10-06T11:18:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856406ed10 0x7f856415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85640c6470 0x7f856413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640cba70 con 0x7f856406e8a0 2023-10-06T11:18:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856416db20 con 0x7f856406ed10 2023-10-06T11:18:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856902e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856416d840 con 0x7f85640c6470 2023-10-06T11:18:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52340/0 (socket says 172.21.15.46:52340) 2023-10-06T11:18:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 -- 172.21.15.46:0/3992740366 learned_addr learned my addr 172.21.15.46:0/3992740366 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856259c700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856406ed10 0x7f856415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f856359e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85640c6470 0x7f856413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 -- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85640c6470 msgr2=0x7f856413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85640c6470 0x7f856413d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 -- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856406ed10 msgr2=0x7f856415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856406ed10 0x7f856415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.046+0000 7f8562d9d700 1 -- 172.21.15.46:0/3992740366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580094f0 con 0x7f856406e8a0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f8562d9d700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f855400c590 tx=0x7f855400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554011590 con 0x7f856406e8a0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85540116f0 con 0x7f856406e8a0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554011ab0 con 0x7f856406e8a0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8564169740 con 0x7f856406e8a0 2023-10-06T11:18:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8564169d00 con 0x7f856406e8a0 2023-10-06T11:18:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.050+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8564169f20 con 0x7f856406e8a0 2023-10-06T11:18:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.054+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f855401b3b0 con 0x7f856406e8a0 2023-10-06T11:18:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.054+0000 7f854affd700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 0x7f8550062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.054+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f855407bfb0 con 0x7f856406e8a0 2023-10-06T11:18:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.054+0000 7f856259c700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 0x7f8550062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.054+0000 7f856259c700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 0x7f8550062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f856406c800 tx=0x7f8558017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.062+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f855404c3a0 con 0x7f856406e8a0 2023-10-06T11:18:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:25.278+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f85640660f0 con 0x7f856406e8a0 2023-10-06T11:18:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.078+0000 7f854affd700 1 -- 172.21.15.46:0/3992740366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8554017070 con 0x7f856406e8a0 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 msgr2=0x7f8550062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 0x7f8550062a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f856406c800 tx=0x7f8558017000 comp rx=0 tx=0).stop 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 msgr2=0x7f856416a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f855400c590 tx=0x7f855400c950 comp rx=0 tx=0).stop 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 shutdown_connections 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8550060550 0x7f8550062a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85640c6470 0x7f856413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f856406ed10 0x7f856415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 --2- 172.21.15.46:0/3992740366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f856406e8a0 0x7f856416a220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.082+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 >> 172.21.15.46:0/3992740366 conn(0x7f85640cbe30 msgr2=0x7f85640c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.086+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 shutdown_connections 2023-10-06T11:18:26.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.086+0000 7f856902e700 1 -- 172.21.15.46:0/3992740366 wait complete. 2023-10-06T11:18:26.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:18:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 Processor -- start 2023-10-06T11:18:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 -- start start 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fef68158aa0 0x7fef68158f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef68159850 0x7fef68159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6815a5c0 con 0x7fef68159850 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680ce6e0 con 0x7fef68158aa0 2023-10-06T11:18:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.274+0000 7fef6de0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680cddb0 con 0x7fef68069270 2023-10-06T11:18:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6d60b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef68159850 0x7fef68159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42006/0 (socket says 172.21.15.46:42006) 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 -- 172.21.15.46:0/227357622 learned_addr learned my addr 172.21.15.46:0/227357622 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef67fff700 1 --2- 172.21.15.46:0/227357622 >> v2:172.21.15.132:3300/0 conn(0x7fef68158aa0 0x7fef68158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 -- 172.21.15.46:0/227357622 >> v2:172.21.15.132:3300/0 conn(0x7fef68158aa0 msgr2=0x7fef68158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/227357622 >> v2:172.21.15.132:3300/0 conn(0x7fef68158aa0 0x7fef68158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 -- 172.21.15.46:0/227357622 >> v2:172.21.15.46:3300/0 conn(0x7fef68159850 msgr2=0x7fef68159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/227357622 >> v2:172.21.15.46:3300/0 conn(0x7fef68159850 0x7fef68159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 -- 172.21.15.46:0/227357622 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680ce090 con 0x7fef68069270 2023-10-06T11:18:26.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/227357622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef68088850 tx=0x7fef5c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f076b84c4586c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:26.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef677fe700 1 -- 172.21.15.46:0/227357622 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00b640 con 0x7fef68069270 2023-10-06T11:18:26.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef677fe700 1 -- 172.21.15.46:0/227357622 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c00b7a0 con 0x7fef68069270 2023-10-06T11:18:26.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef677fe700 1 -- 172.21.15.46:0/227357622 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00bb60 con 0x7fef68069270 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 -- 172.21.15.46:0/227357622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 msgr2=0x7fef68069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 --2- 172.21.15.46:0/227357622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef68088850 tx=0x7fef5c009250 comp rx=0 tx=0).stop 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 -- 172.21.15.46:0/227357622 shutdown_connections 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 --2- 172.21.15.46:0/227357622 >> v2:172.21.15.132:3300/0 conn(0x7fef68158aa0 0x7fef68158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 --2- 172.21.15.46:0/227357622 >> v2:172.21.15.46:3300/0 conn(0x7fef68159850 0x7fef68159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 --2- 172.21.15.46:0/227357622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68069270 0x7fef68069640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 -- 172.21.15.46:0/227357622 >> 172.21.15.46:0/227357622 conn(0x7fef68068390 msgr2=0x7fef680d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 -- 172.21.15.46:0/227357622 shutdown_connections 2023-10-06T11:18:26.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.278+0000 7fef6de0c700 1 -- 172.21.15.46:0/227357622 wait complete. 2023-10-06T11:18:26.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 Processor -- start 2023-10-06T11:18:26.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 -- start start 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 0x7fef68143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 0x7fef680b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680afd50 con 0x7fef68159850 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680b0bb0 con 0x7fef68069270 2023-10-06T11:18:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.282+0000 7fef6de0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680affa0 con 0x7fef68158aa0 2023-10-06T11:18:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef67fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 0x7fef680b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52362/0 (socket says 172.21.15.46:52362) 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 -- 172.21.15.46:0/3535471192 learned_addr learned my addr 172.21.15.46:0/3535471192 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 0x7fef68143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 -- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 msgr2=0x7fef680b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 0x7fef680b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 -- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 msgr2=0x7fef68143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 0x7fef68143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 -- 172.21.15.46:0/3535471192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c0094f0 con 0x7fef68159850 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef67fff700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 0x7fef680b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 0x7fef68143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6d60b700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef60009e20 tx=0x7fef6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000caa0 con 0x7fef68159850 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000f040 con 0x7fef68159850 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000ce70 con 0x7fef68159850 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef680b0280 con 0x7fef68159850 2023-10-06T11:18:26.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef680b0840 con 0x7fef68159850 2023-10-06T11:18:26.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.286+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef680b0d40 con 0x7fef68159850 2023-10-06T11:18:26.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.290+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fef60014690 con 0x7fef68159850 2023-10-06T11:18:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.290+0000 7fef64ff9700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 0x7fef4c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:26.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.290+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fef6007d190 con 0x7fef68159850 2023-10-06T11:18:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.290+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 0x7fef4c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.294+0000 7fef6ce0a700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 0x7fef4c062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef68088850 tx=0x7fef5c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.298+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fef60051010 con 0x7fef68159850 2023-10-06T11:18:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:26.506+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fef68064590 con 0x7fef68159850 2023-10-06T11:18:27.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.090+0000 7fef64ff9700 1 -- 172.21.15.46:0/3535471192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fef60050ba0 con 0x7fef68159850 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 msgr2=0x7fef4c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 0x7fef4c062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef68088850 tx=0x7fef5c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 msgr2=0x7fef680b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef60009e20 tx=0x7fef6000b040 comp rx=0 tx=0).stop 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 shutdown_connections 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef4c060550 0x7fef4c062a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef68159850 0x7fef680b1040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef68158aa0 0x7fef680b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 --2- 172.21.15.46:0/3535471192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef68069270 0x7fef68143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 >> 172.21.15.46:0/3535471192 conn(0x7fef68068390 msgr2=0x7fef680cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 shutdown_connections 2023-10-06T11:18:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.094+0000 7fef6de0c700 1 -- 172.21.15.46:0/3535471192 wait complete. 2023-10-06T11:18:27.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:27.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 Processor -- start 2023-10-06T11:18:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 -- start start 2023-10-06T11:18:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff4dc15c770 0x7ff4dc0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0cc150 con 0x7ff4dc0cec00 2023-10-06T11:18:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0cb280 con 0x7ff4dc0ccb50 2023-10-06T11:18:27.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e306e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0cfbb0 con 0x7ff4dc15c770 2023-10-06T11:18:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e160b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff4dc0cec00 0x7ff4dc162e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52196/0 (socket says 172.21.15.46:52196) 2023-10-06T11:18:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.286+0000 7ff4e160b700 1 -- 172.21.15.46:0/1144815320 learned_addr learned my addr 172.21.15.46:0/1144815320 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e0e0a700 1 --2- 172.21.15.46:0/1144815320 >> v2:172.21.15.132:3300/0 conn(0x7ff4dc15c770 0x7ff4dc0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329857943 0 0) 0x7ff4dc0cb280 con 0x7ff4dc0ccb50 2023-10-06T11:18:27.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4cc003570 con 0x7ff4dc0ccb50 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913483661 0 0) 0x7ff4dc0cc150 con 0x7ff4dc0cec00 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc0cb280 con 0x7ff4dc0cec00 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e0e0a700 1 -- 172.21.15.46:0/1144815320 >> v1:172.21.15.132:6789/0 conn(0x7ff4dc0ccb50 legacy=0x7ff4dc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e0e0a700 1 -- 172.21.15.46:0/1144815320 >> v1:172.21.15.46:6789/0 conn(0x7ff4dc0cec00 legacy=0x7ff4dc162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e0e0a700 1 -- 172.21.15.46:0/1144815320 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0cb540 con 0x7ff4dc15c770 2023-10-06T11:18:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e0e0a700 1 --2- 172.21.15.46:0/1144815320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc0cad40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098d0 tx=0x7ff4d0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d681e95a7c0adb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0005c10 con 0x7ff4dc15c770 2023-10-06T11:18:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0005d70 con 0x7ff4dc15c770 2023-10-06T11:18:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4db7fe700 1 -- 172.21.15.46:0/1144815320 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0006260 con 0x7ff4dc15c770 2023-10-06T11:18:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e306e700 1 -- 172.21.15.46:0/1144815320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 msgr2=0x7ff4dc0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.290+0000 7ff4e306e700 1 --2- 172.21.15.46:0/1144815320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc0cad40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098d0 tx=0x7ff4d0009120 comp rx=0 tx=0).stop 2023-10-06T11:18:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 -- 172.21.15.46:0/1144815320 shutdown_connections 2023-10-06T11:18:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 --2- 172.21.15.46:0/1144815320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc0cad40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 -- 172.21.15.46:0/1144815320 >> 172.21.15.46:0/1144815320 conn(0x7ff4dc099480 msgr2=0x7ff4dc06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 -- 172.21.15.46:0/1144815320 shutdown_connections 2023-10-06T11:18:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 -- 172.21.15.46:0/1144815320 wait complete. 2023-10-06T11:18:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 Processor -- start 2023-10-06T11:18:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 -- start start 2023-10-06T11:18:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4dc0ccb50 0x7ff4dc152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.294+0000 7ff4e306e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc16f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0b54f0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0b3210 con 0x7ff4dc15c770 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc16e7e0 con 0x7ff4dc0ccb50 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52376/0 (socket says 172.21.15.46:52376) 2023-10-06T11:18:27.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 -- 172.21.15.46:0/280549504 learned_addr learned my addr 172.21.15.46:0/280549504 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e0e0a700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4dc0ccb50 0x7ff4dc152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e160b700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc16f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 -- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4dc0ccb50 msgr2=0x7ff4dc152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4dc0ccb50 0x7ff4dc152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 -- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 msgr2=0x7ff4dc16f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc16f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 -- 172.21.15.46:0/280549504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d00093c0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4dbfff700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800c790 tx=0x7ff4c8007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8011590 con 0x7ff4dc0cec00 2023-10-06T11:18:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c8011710 con 0x7ff4dc0cec00 2023-10-06T11:18:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc16ea80 con 0x7ff4dc0cec00 2023-10-06T11:18:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8011ad0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.298+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc16f040 con 0x7ff4dc0cec00 2023-10-06T11:18:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.302+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc0b2fe0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.302+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff4c8012420 con 0x7ff4dc0cec00 2023-10-06T11:18:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.306+0000 7ff4d8ff9700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 0x7ff4c0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.306+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff4c807c3b0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.306+0000 7ff4e0e0a700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 0x7ff4c0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.306+0000 7ff4e0e0a700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 0x7ff4c0062a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009160 tx=0x7ff4d0016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.310+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff4dc0b2fe0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.518+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4dc0b2fe0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.518+0000 7ff4d8ff9700 1 -- 172.21.15.46:0/280549504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7ff4dc0b2fe0 con 0x7ff4dc0cec00 2023-10-06T11:18:27.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:27.525 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:18:27.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 msgr2=0x7ff4c0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 0x7ff4c0062a30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009160 tx=0x7ff4d0016000 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 msgr2=0x7ff4dc16e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800c790 tx=0x7ff4c8007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 shutdown_connections 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4c0060580 0x7ff4c0062a30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4dc15c770 0x7ff4dc16f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4dc0cec00 0x7ff4dc16e2a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 --2- 172.21.15.46:0/280549504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4dc0ccb50 0x7ff4dc152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 >> 172.21.15.46:0/280549504 conn(0x7ff4dc099480 msgr2=0x7ff4dc068730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:27.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 shutdown_connections 2023-10-06T11:18:27.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.522+0000 7ff4e306e700 1 -- 172.21.15.46:0/280549504 wait complete. 2023-10-06T11:18:27.530 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:18:27.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.702+0000 7fc53b523700 1 Processor -- start 2023-10-06T11:18:27.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.702+0000 7fc53b523700 1 -- start start 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 0x7fc5340b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5340d07b0 0x7fc5340b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5340b1130 0x7fc5340ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340b15b0 con 0x7fc5340d07b0 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5340ae340 con 0x7fc5340b1130 2023-10-06T11:18:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53b523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340b09a0 con 0x7fc534159290 2023-10-06T11:18:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5340d07b0 0x7fc5340b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5340d07b0 0x7fc5340b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52380/0 (socket says 172.21.15.46:52380) 2023-10-06T11:18:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 -- 172.21.15.46:0/3058163873 learned_addr learned my addr 172.21.15.46:0/3058163873 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53a521700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 0x7fc5340b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53ad22700 1 --2- 172.21.15.46:0/3058163873 >> v2:172.21.15.132:3300/0 conn(0x7fc5340b1130 0x7fc5340ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 -- 172.21.15.46:0/3058163873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 msgr2=0x7fc5340b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 0x7fc5340b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 -- 172.21.15.46:0/3058163873 >> v2:172.21.15.132:3300/0 conn(0x7fc5340b1130 msgr2=0x7fc5340ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 --2- 172.21.15.46:0/3058163873 >> v2:172.21.15.132:3300/0 conn(0x7fc5340b1130 0x7fc5340ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 -- 172.21.15.46:0/3058163873 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc534159660 con 0x7fc5340d07b0 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53a521700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 0x7fc5340b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:27.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc539d20700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5340d07b0 0x7fc5340b0bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5340cffd0 tx=0x7fc524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fdfe2a827ce385 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53951f700 1 -- 172.21.15.46:0/3058163873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400b8f0 con 0x7fc5340d07b0 2023-10-06T11:18:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53951f700 1 -- 172.21.15.46:0/3058163873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400ba50 con 0x7fc5340d07b0 2023-10-06T11:18:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.706+0000 7fc53951f700 1 -- 172.21.15.46:0/3058163873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400bdf0 con 0x7fc5340d07b0 2023-10-06T11:18:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 -- 172.21.15.46:0/3058163873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5340d07b0 msgr2=0x7fc5340b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5340d07b0 0x7fc5340b0bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5340cffd0 tx=0x7fc524009250 comp rx=0 tx=0).stop 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 -- 172.21.15.46:0/3058163873 shutdown_connections 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 --2- 172.21.15.46:0/3058163873 >> v2:172.21.15.132:3300/0 conn(0x7fc5340b1130 0x7fc5340ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5340d07b0 0x7fc5340b0bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 --2- 172.21.15.46:0/3058163873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc534159290 0x7fc5340b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 -- 172.21.15.46:0/3058163873 >> 172.21.15.46:0/3058163873 conn(0x7fc53406ef60 msgr2=0x7fc5340d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 -- 172.21.15.46:0/3058163873 shutdown_connections 2023-10-06T11:18:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.710+0000 7fc53b523700 1 -- 172.21.15.46:0/3058163873 wait complete. 2023-10-06T11:18:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 Processor -- start 2023-10-06T11:18:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- start start 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5340b1130 0x7fc53416d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 0x7fc5340ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340cf350 con 0x7fc534159290 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340cf630 con 0x7fc5340d07b0 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340cf910 con 0x7fc5340b1130 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52392/0 (socket says 172.21.15.46:52392) 2023-10-06T11:18:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 -- 172.21.15.46:0/3352124021 learned_addr learned my addr 172.21.15.46:0/3352124021 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53a521700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5340b1130 0x7fc53416d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc539d20700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 0x7fc5340ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 -- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5340b1130 msgr2=0x7fc53416d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5340b1130 0x7fc53416d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 -- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 msgr2=0x7fc5340ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 0x7fc5340ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 -- 172.21.15.46:0/3352124021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240094f0 con 0x7fc534159290 2023-10-06T11:18:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc539d20700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 0x7fc5340ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53ad22700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc52c007f20 tx=0x7fc52c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00eca0 con 0x7fc534159290 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c00ee20 con 0x7fc534159290 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5340cfbf0 con 0x7fc534159290 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c014670 con 0x7fc534159290 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.714+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5340cd870 con 0x7fc534159290 2023-10-06T11:18:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.718+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5340d0db0 con 0x7fc534159290 2023-10-06T11:18:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.718+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc52c0149e0 con 0x7fc534159290 2023-10-06T11:18:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.722+0000 7fc52a7fc700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 0x7fc518062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.722+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc52c07d0a0 con 0x7fc534159290 2023-10-06T11:18:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.722+0000 7fc53a521700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 0x7fc518062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.722+0000 7fc53a521700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 0x7fc518062a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc530006f00 tx=0x7fc530008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.730+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc5340d0db0 con 0x7fc534159290 2023-10-06T11:18:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.906+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5340d0db0 con 0x7fc534159290 2023-10-06T11:18:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.906+0000 7fc52a7fc700 1 -- 172.21.15.46:0/3352124021 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fc5340d0db0 con 0x7fc534159290 2023-10-06T11:18:27.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:27.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 msgr2=0x7fc518062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 0x7fc518062a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc530006f00 tx=0x7fc530008040 comp rx=0 tx=0).stop 2023-10-06T11:18:27.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 msgr2=0x7fc53414c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc52c007f20 tx=0x7fc52c009860 comp rx=0 tx=0).stop 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 shutdown_connections 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc518060580 0x7fc518062a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc534159290 0x7fc53414c460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5340d07b0 0x7fc5340ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 --2- 172.21.15.46:0/3352124021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5340b1130 0x7fc53416d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 >> 172.21.15.46:0/3352124021 conn(0x7fc53406ef60 msgr2=0x7fc534068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 shutdown_connections 2023-10-06T11:18:27.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:27.910+0000 7fc53b523700 1 -- 172.21.15.46:0/3352124021 wait complete. 2023-10-06T11:18:27.934 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:18:27.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:18:28.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.086+0000 7f7d2359e700 1 Processor -- start 2023-10-06T11:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.086+0000 7f7d2359e700 1 -- start start 2023-10-06T11:18:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d240cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d240cb4f0 0x7f7d240c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2359e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240ccfa0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2359e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240cd970 con 0x7f7d240cb4f0 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240cc390 con 0x7f7d240c3160 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d240cb4f0 0x7f7d240c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d2259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d240cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d240cb4f0 0x7f7d240c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52398/0 (socket says 172.21.15.46:52398) 2023-10-06T11:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 -- 172.21.15.46:0/2203591981 learned_addr learned my addr 172.21.15.46:0/2203591981 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 -- 172.21.15.46:0/2203591981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 msgr2=0x7f7d240cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 --2- 172.21.15.46:0/2203591981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d240cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 -- 172.21.15.46:0/2203591981 >> v1:172.21.15.46:6789/0 conn(0x7f7d240cb0a0 legacy=0x7f7d240c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.090+0000 7f7d22d9d700 1 -- 172.21.15.46:0/2203591981 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240cc670 con 0x7f7d240cb4f0 2023-10-06T11:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d22d9d700 1 --2- 172.21.15.46:0/2203591981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb4f0 0x7f7d240c84c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009f20 tx=0x7f7d1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20723327d15b77d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2159a700 1 -- 172.21.15.46:0/2203591981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00e070 con 0x7f7d240cb4f0 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2159a700 1 -- 172.21.15.46:0/2203591981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00b9d0 con 0x7f7d240cb4f0 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2159a700 1 -- 172.21.15.46:0/2203591981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0092e0 con 0x7f7d240cb4f0 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 -- 172.21.15.46:0/2203591981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb4f0 msgr2=0x7f7d240c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2203591981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb4f0 0x7f7d240c84c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009f20 tx=0x7f7d1c009840 comp rx=0 tx=0).stop 2023-10-06T11:18:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 -- 172.21.15.46:0/2203591981 shutdown_connections 2023-10-06T11:18:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2203591981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb4f0 0x7f7d240c84c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2203591981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d240cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 -- 172.21.15.46:0/2203591981 >> 172.21.15.46:0/2203591981 conn(0x7f7d240970c0 msgr2=0x7f7d24067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 -- 172.21.15.46:0/2203591981 shutdown_connections 2023-10-06T11:18:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.094+0000 7f7d2359e700 1 -- 172.21.15.46:0/2203591981 wait complete. 2023-10-06T11:18:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 Processor -- start 2023-10-06T11:18:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 -- start start 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d2415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d240cb4f0 0x7f7d24145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415e1d0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415e4b0 con 0x7f7d240cb4f0 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2415e790 con 0x7f7d240c3160 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52414/0 (socket says 172.21.15.46:52414) 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 -- 172.21.15.46:0/2697807700 learned_addr learned my addr 172.21.15.46:0/2697807700 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d2259c700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d2415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d22d9d700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d240cb4f0 0x7f7d24145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 -- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 msgr2=0x7f7d2415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d2415d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 -- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d240cb4f0 msgr2=0x7f7d24145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d240cb4f0 0x7f7d24145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 -- 172.21.15.46:0/2697807700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009ae0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d21d9b700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c590 tx=0x7f7d1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.098+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011590 con 0x7f7d240cb0a0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d180116f0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2415ea70 con 0x7f7d240cb0a0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011ab0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d24068de0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24069000 con 0x7f7d240cb0a0 2023-10-06T11:18:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7d1801b3b0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d127fc700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 0x7f7d04062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.102+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7d1807bfe0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.106+0000 7f7d2259c700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 0x7f7d04062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.106+0000 7f7d2259c700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 0x7f7d04062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009c90 tx=0x7f7d14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.110+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d1804c3f0 con 0x7f7d240cb0a0 2023-10-06T11:18:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.278+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d2415dfc0 con 0x7f7d04060550 2023-10-06T11:18:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.286+0000 7f7d127fc700 1 -- 172.21.15.46:0/2697807700 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286463 (secure 0 0 0) 0x7f7d2415dfc0 con 0x7f7d04060550 2023-10-06T11:18:28.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 msgr2=0x7f7d04062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 0x7f7d04062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009c90 tx=0x7f7d14008040 comp rx=0 tx=0).stop 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 msgr2=0x7f7d240688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c590 tx=0x7f7d1800c950 comp rx=0 tx=0).stop 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 shutdown_connections 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d04060550 0x7f7d04062a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d240cb4f0 0x7f7d24145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d240cb0a0 0x7f7d240688a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.294+0000 7f7d2359e700 1 --2- 172.21.15.46:0/2697807700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d240c3160 0x7f7d2415d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.298+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 >> 172.21.15.46:0/2697807700 conn(0x7f7d240970c0 msgr2=0x7f7d24153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.298+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 shutdown_connections 2023-10-06T11:18:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.298+0000 7f7d2359e700 1 -- 172.21.15.46:0/2697807700 wait complete. 2023-10-06T11:18:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:18:28.325 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T11:18:26.484340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397684,"kb_used_data":2996,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577036,"statfs":{"total":773094113280,"available":770638884864,"internally_reserved":0,"allocated":3067904,"data_stored":1839032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.745099"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797220+0000","last_change":"2023-10-06T11:18:23.797220+0000","last_active":"2023-10-06T11:18:23.797220+0000","last_peered":"2023-10-06T11:18:23.797220+0000","last_clean":"2023-10-06T11:18:23.797220+0000","last_became_active":"2023-10-06T11:18:23.796644+0000","last_became_peered":"2023-10-06T11:18:23.796644+0000","last_unstale":"2023-10-06T11:18:23.797220+0000","last_undegraded":"2023-10-06T11:18:23.797220+0000","last_fullsized":"2023-10-06T11:18:23.797220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:18.835072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798280+0000","last_change":"2023-10-06T11:18:23.798280+0000","last_active":"2023-10-06T11:18:23.798280+0000","last_peered":"2023-10-06T11:18:23.798280+0000","last_clean":"2023-10-06T11:18:23.798280+0000","last_became_active":"2023-10-06T11:18:23.797912+0000","last_became_peered":"2023-10-06T11:18:23.797912+0000","last_unstale":"2023-10-06T11:18:23.798280+0000","last_undegraded":"2023-10-06T11:18:23.798280+0000","last_fullsized":"2023-10-06T11:18:23.798280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797573+0000","last_change":"2023-10-06T11:18:23.797573+0000","last_active":"2023-10-06T11:18:23.797573+0000","last_peered":"2023-10-06T11:18:23.797573+0000","last_clean":"2023-10-06T11:18:23.797573+0000","last_became_active":"2023-10-06T11:18:23.797165+0000","last_became_peered":"2023-10-06T11:18:23.797165+0000","last_unstale":"2023-10-06T11:18:23.797573+0000","last_undegraded":"2023-10-06T11:18:23.797573+0000","last_fullsized":"2023-10-06T11:18:23.797573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795447+0000","last_change":"2023-10-06T11:18:23.795447+0000","last_active":"2023-10-06T11:18:23.795447+0000","last_peered":"2023-10-06T11:18:23.795447+0000","last_clean":"2023-10-06T11:18:23.795447+0000","last_became_active":"2023-10-06T11:18:23.794305+0000","last_became_peered":"2023-10-06T11:18:23.794305+0000","last_unstale":"2023-10-06T11:18:23.795447+0000","last_undegraded":"2023-10-06T11:18:23.795447+0000","last_fullsized":"2023-10-06T11:18:23.795447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:35.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796286+0000","last_change":"2023-10-06T11:18:23.796286+0000","last_active":"2023-10-06T11:18:23.796286+0000","last_peered":"2023-10-06T11:18:23.796286+0000","last_clean":"2023-10-06T11:18:23.796286+0000","last_became_active":"2023-10-06T11:18:23.795746+0000","last_became_peered":"2023-10-06T11:18:23.795746+0000","last_unstale":"2023-10-06T11:18:23.796286+0000","last_undegraded":"2023-10-06T11:18:23.796286+0000","last_fullsized":"2023-10-06T11:18:23.796286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793146+0000","last_change":"2023-10-06T11:18:23.793146+0000","last_active":"2023-10-06T11:18:23.793146+0000","last_peered":"2023-10-06T11:18:23.793146+0000","last_clean":"2023-10-06T11:18:23.793146+0000","last_became_active":"2023-10-06T11:18:23.792852+0000","last_became_peered":"2023-10-06T11:18:23.792852+0000","last_unstale":"2023-10-06T11:18:23.793146+0000","last_undegraded":"2023-10-06T11:18:23.793146+0000","last_fullsized":"2023-10-06T11:18:23.793146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:31.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809263+0000","last_change":"2023-10-06T11:18:23.809263+0000","last_active":"2023-10-06T11:18:23.809263+0000","last_peered":"2023-10-06T11:18:23.809263+0000","last_clean":"2023-10-06T11:18:23.809263+0000","last_became_active":"2023-10-06T11:18:23.808943+0000","last_became_peered":"2023-10-06T11:18:23.808943+0000","last_unstale":"2023-10-06T11:18:23.809263+0000","last_undegraded":"2023-10-06T11:18:23.809263+0000","last_fullsized":"2023-10-06T11:18:23.809263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788701+0000","last_change":"2023-10-06T11:18:23.788701+0000","last_active":"2023-10-06T11:18:23.788701+0000","last_peered":"2023-10-06T11:18:23.788701+0000","last_clean":"2023-10-06T11:18:23.788701+0000","last_became_active":"2023-10-06T11:18:23.788343+0000","last_became_peered":"2023-10-06T11:18:23.788343+0000","last_unstale":"2023-10-06T11:18:23.788701+0000","last_undegraded":"2023-10-06T11:18:23.788701+0000","last_fullsized":"2023-10-06T11:18:23.788701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802504+0000","last_change":"2023-10-06T11:18:23.802504+0000","last_active":"2023-10-06T11:18:23.802504+0000","last_peered":"2023-10-06T11:18:23.802504+0000","last_clean":"2023-10-06T11:18:23.802504+0000","last_became_active":"2023-10-06T11:18:23.801692+0000","last_became_peered":"2023-10-06T11:18:23.801692+0000","last_unstale":"2023-10-06T11:18:23.802504+0000","last_undegraded":"2023-10-06T11:18:23.802504+0000","last_fullsized":"2023-10-06T11:18:23.802504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795845+0000","last_change":"2023-10-06T11:18:23.795845+0000","last_active":"2023-10-06T11:18:23.795845+0000","last_peered":"2023-10-06T11:18:23.795845+0000","last_clean":"2023-10-06T11:18:23.795845+0000","last_became_active":"2023-10-06T11:18:23.795351+0000","last_became_peered":"2023-10-06T11:18:23.795351+0000","last_unstale":"2023-10-06T11:18:23.795845+0000","last_undegraded":"2023-10-06T11:18:23.795845+0000","last_fullsized":"2023-10-06T11:18:23.795845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.805599+0000","last_change":"2023-10-06T11:18:23.805599+0000","last_active":"2023-10-06T11:18:23.805599+0000","last_peered":"2023-10-06T11:18:23.805599+0000","last_clean":"2023-10-06T11:18:23.805599+0000","last_became_active":"2023-10-06T11:18:23.805371+0000","last_became_peered":"2023-10-06T11:18:23.805371+0000","last_unstale":"2023-10-06T11:18:23.805599+0000","last_undegraded":"2023-10-06T11:18:23.805599+0000","last_fullsized":"2023-10-06T11:18:23.805599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:25.984507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799204+0000","last_change":"2023-10-06T11:18:23.799204+0000","last_active":"2023-10-06T11:18:23.799204+0000","last_peered":"2023-10-06T11:18:23.799204+0000","last_clean":"2023-10-06T11:18:23.799204+0000","last_became_active":"2023-10-06T11:18:23.798864+0000","last_became_peered":"2023-10-06T11:18:23.798864+0000","last_unstale":"2023-10-06T11:18:23.799204+0000","last_undegraded":"2023-10-06T11:18:23.799204+0000","last_fullsized":"2023-10-06T11:18:23.799204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795240+0000","last_change":"2023-10-06T11:18:23.795240+0000","last_active":"2023-10-06T11:18:23.795240+0000","last_peered":"2023-10-06T11:18:23.795240+0000","last_clean":"2023-10-06T11:18:23.795240+0000","last_became_active":"2023-10-06T11:18:23.794759+0000","last_became_peered":"2023-10-06T11:18:23.794759+0000","last_unstale":"2023-10-06T11:18:23.795240+0000","last_undegraded":"2023-10-06T11:18:23.795240+0000","last_fullsized":"2023-10-06T11:18:23.795240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:08.424249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789935+0000","last_change":"2023-10-06T11:18:23.789935+0000","last_active":"2023-10-06T11:18:23.789935+0000","last_peered":"2023-10-06T11:18:23.789935+0000","last_clean":"2023-10-06T11:18:23.789935+0000","last_became_active":"2023-10-06T11:18:23.789663+0000","last_became_peered":"2023-10-06T11:18:23.789663+0000","last_unstale":"2023-10-06T11:18:23.789935+0000","last_undegraded":"2023-10-06T11:18:23.789935+0000","last_fullsized":"2023-10-06T11:18:23.789935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809338+0000","last_change":"2023-10-06T11:18:23.809338+0000","last_active":"2023-10-06T11:18:23.809338+0000","last_peered":"2023-10-06T11:18:23.809338+0000","last_clean":"2023-10-06T11:18:23.809338+0000","last_became_active":"2023-10-06T11:18:23.808949+0000","last_became_peered":"2023-10-06T11:18:23.808949+0000","last_unstale":"2023-10-06T11:18:23.809338+0000","last_undegraded":"2023-10-06T11:18:23.809338+0000","last_fullsized":"2023-10-06T11:18:23.809338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794359+0000","last_change":"2023-10-06T11:18:23.794359+0000","last_active":"2023-10-06T11:18:23.794359+0000","last_peered":"2023-10-06T11:18:23.794359+0000","last_clean":"2023-10-06T11:18:23.794359+0000","last_became_active":"2023-10-06T11:18:23.794050+0000","last_became_peered":"2023-10-06T11:18:23.794050+0000","last_unstale":"2023-10-06T11:18:23.794359+0000","last_undegraded":"2023-10-06T11:18:23.794359+0000","last_fullsized":"2023-10-06T11:18:23.794359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802662+0000","last_change":"2023-10-06T11:18:23.802662+0000","last_active":"2023-10-06T11:18:23.802662+0000","last_peered":"2023-10-06T11:18:23.802662+0000","last_clean":"2023-10-06T11:18:23.802662+0000","last_became_active":"2023-10-06T11:18:23.802233+0000","last_became_peered":"2023-10-06T11:18:23.802233+0000","last_unstale":"2023-10-06T11:18:23.802662+0000","last_undegraded":"2023-10-06T11:18:23.802662+0000","last_fullsized":"2023-10-06T11:18:23.802662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795210+0000","last_change":"2023-10-06T11:18:23.795210+0000","last_active":"2023-10-06T11:18:23.795210+0000","last_peered":"2023-10-06T11:18:23.795210+0000","last_clean":"2023-10-06T11:18:23.795210+0000","last_became_active":"2023-10-06T11:18:23.794877+0000","last_became_peered":"2023-10-06T11:18:23.794877+0000","last_unstale":"2023-10-06T11:18:23.795210+0000","last_undegraded":"2023-10-06T11:18:23.795210+0000","last_fullsized":"2023-10-06T11:18:23.795210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:16.625365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789054+0000","last_change":"2023-10-06T11:18:23.789054+0000","last_active":"2023-10-06T11:18:23.789054+0000","last_peered":"2023-10-06T11:18:23.789054+0000","last_clean":"2023-10-06T11:18:23.789054+0000","last_became_active":"2023-10-06T11:18:23.788629+0000","last_became_peered":"2023-10-06T11:18:23.788629+0000","last_unstale":"2023-10-06T11:18:23.789054+0000","last_undegraded":"2023-10-06T11:18:23.789054+0000","last_fullsized":"2023-10-06T11:18:23.789054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:37.399272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792433+0000","last_change":"2023-10-06T11:18:23.792433+0000","last_active":"2023-10-06T11:18:23.792433+0000","last_peered":"2023-10-06T11:18:23.792433+0000","last_clean":"2023-10-06T11:18:23.792433+0000","last_became_active":"2023-10-06T11:18:23.792138+0000","last_became_peered":"2023-10-06T11:18:23.792138+0000","last_unstale":"2023-10-06T11:18:23.792433+0000","last_undegraded":"2023-10-06T11:18:23.792433+0000","last_fullsized":"2023-10-06T11:18:23.792433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797785+0000","last_change":"2023-10-06T11:18:23.797785+0000","last_active":"2023-10-06T11:18:23.797785+0000","last_peered":"2023-10-06T11:18:23.797785+0000","last_clean":"2023-10-06T11:18:23.797785+0000","last_became_active":"2023-10-06T11:18:23.797415+0000","last_became_peered":"2023-10-06T11:18:23.797415+0000","last_unstale":"2023-10-06T11:18:23.797785+0000","last_undegraded":"2023-10-06T11:18:23.797785+0000","last_fullsized":"2023-10-06T11:18:23.797785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808011+0000","last_change":"2023-10-06T11:18:23.808011+0000","last_active":"2023-10-06T11:18:23.808011+0000","last_peered":"2023-10-06T11:18:23.808011+0000","last_clean":"2023-10-06T11:18:23.808011+0000","last_became_active":"2023-10-06T11:18:23.807627+0000","last_became_peered":"2023-10-06T11:18:23.807627+0000","last_unstale":"2023-10-06T11:18:23.808011+0000","last_undegraded":"2023-10-06T11:18:23.808011+0000","last_fullsized":"2023-10-06T11:18:23.808011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794460+0000","last_change":"2023-10-06T11:18:23.794460+0000","last_active":"2023-10-06T11:18:23.794460+0000","last_peered":"2023-10-06T11:18:23.794460+0000","last_clean":"2023-10-06T11:18:23.794460+0000","last_became_active":"2023-10-06T11:18:23.794145+0000","last_became_peered":"2023-10-06T11:18:23.794145+0000","last_unstale":"2023-10-06T11:18:23.794460+0000","last_undegraded":"2023-10-06T11:18:23.794460+0000","last_fullsized":"2023-10-06T11:18:23.794460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800752+0000","last_change":"2023-10-06T11:18:23.800752+0000","last_active":"2023-10-06T11:18:23.800752+0000","last_peered":"2023-10-06T11:18:23.800752+0000","last_clean":"2023-10-06T11:18:23.800752+0000","last_became_active":"2023-10-06T11:18:23.798826+0000","last_became_peered":"2023-10-06T11:18:23.798826+0000","last_unstale":"2023-10-06T11:18:23.800752+0000","last_undegraded":"2023-10-06T11:18:23.800752+0000","last_fullsized":"2023-10-06T11:18:23.800752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801213+0000","last_change":"2023-10-06T11:18:23.801213+0000","last_active":"2023-10-06T11:18:23.801213+0000","last_peered":"2023-10-06T11:18:23.801213+0000","last_clean":"2023-10-06T11:18:23.801213+0000","last_became_active":"2023-10-06T11:18:23.800123+0000","last_became_peered":"2023-10-06T11:18:23.800123+0000","last_unstale":"2023-10-06T11:18:23.801213+0000","last_undegraded":"2023-10-06T11:18:23.801213+0000","last_fullsized":"2023-10-06T11:18:23.801213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795738+0000","last_change":"2023-10-06T11:18:23.795738+0000","last_active":"2023-10-06T11:18:23.795738+0000","last_peered":"2023-10-06T11:18:23.795738+0000","last_clean":"2023-10-06T11:18:23.795738+0000","last_became_active":"2023-10-06T11:18:23.795423+0000","last_became_peered":"2023-10-06T11:18:23.795423+0000","last_unstale":"2023-10-06T11:18:23.795738+0000","last_undegraded":"2023-10-06T11:18:23.795738+0000","last_fullsized":"2023-10-06T11:18:23.795738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794728+0000","last_change":"2023-10-06T11:18:23.794728+0000","last_active":"2023-10-06T11:18:23.794728+0000","last_peered":"2023-10-06T11:18:23.794728+0000","last_clean":"2023-10-06T11:18:23.794728+0000","last_became_active":"2023-10-06T11:18:23.794324+0000","last_became_peered":"2023-10-06T11:18:23.794324+0000","last_unstale":"2023-10-06T11:18:23.794728+0000","last_undegraded":"2023-10-06T11:18:23.794728+0000","last_fullsized":"2023-10-06T11:18:23.794728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809631+0000","last_change":"2023-10-06T11:18:23.809631+0000","last_active":"2023-10-06T11:18:23.809631+0000","last_peered":"2023-10-06T11:18:23.809631+0000","last_clean":"2023-10-06T11:18:23.809631+0000","last_became_active":"2023-10-06T11:18:23.806649+0000","last_became_peered":"2023-10-06T11:18:23.806649+0000","last_unstale":"2023-10-06T11:18:23.809631+0000","last_undegraded":"2023-10-06T11:18:23.809631+0000","last_fullsized":"2023-10-06T11:18:23.809631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800427+0000","last_change":"2023-10-06T11:18:23.800427+0000","last_active":"2023-10-06T11:18:23.800427+0000","last_peered":"2023-10-06T11:18:23.800427+0000","last_clean":"2023-10-06T11:18:23.800427+0000","last_became_active":"2023-10-06T11:18:23.799956+0000","last_became_peered":"2023-10-06T11:18:23.799956+0000","last_unstale":"2023-10-06T11:18:23.800427+0000","last_undegraded":"2023-10-06T11:18:23.800427+0000","last_fullsized":"2023-10-06T11:18:23.800427+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793233+0000","last_change":"2023-10-06T11:18:23.793233+0000","last_active":"2023-10-06T11:18:23.793233+0000","last_peered":"2023-10-06T11:18:23.793233+0000","last_clean":"2023-10-06T11:18:23.793233+0000","last_became_active":"2023-10-06T11:18:23.792917+0000","last_became_peered":"2023-10-06T11:18:23.792917+0000","last_unstale":"2023-10-06T11:18:23.793233+0000","last_undegraded":"2023-10-06T11:18:23.793233+0000","last_fullsized":"2023-10-06T11:18:23.793233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:01:00.993266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811339+0000","last_change":"2023-10-06T11:18:23.811339+0000","last_active":"2023-10-06T11:18:23.811339+0000","last_peered":"2023-10-06T11:18:23.811339+0000","last_clean":"2023-10-06T11:18:23.811339+0000","last_became_active":"2023-10-06T11:18:23.810982+0000","last_became_peered":"2023-10-06T11:18:23.810982+0000","last_unstale":"2023-10-06T11:18:23.811339+0000","last_undegraded":"2023-10-06T11:18:23.811339+0000","last_fullsized":"2023-10-06T11:18:23.811339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796079+0000","last_change":"2023-10-06T11:18:23.796079+0000","last_active":"2023-10-06T11:18:23.796079+0000","last_peered":"2023-10-06T11:18:23.796079+0000","last_clean":"2023-10-06T11:18:23.796079+0000","last_became_active":"2023-10-06T11:18:23.795767+0000","last_became_peered":"2023-10-06T11:18:23.795767+0000","last_unstale":"2023-10-06T11:18:23.796079+0000","last_undegraded":"2023-10-06T11:18:23.796079+0000","last_fullsized":"2023-10-06T11:18:23.796079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:24.987705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811468+0000","last_change":"2023-10-06T11:18:23.811468+0000","last_active":"2023-10-06T11:18:23.811468+0000","last_peered":"2023-10-06T11:18:23.811468+0000","last_clean":"2023-10-06T11:18:23.811468+0000","last_became_active":"2023-10-06T11:18:23.811047+0000","last_became_peered":"2023-10-06T11:18:23.811047+0000","last_unstale":"2023-10-06T11:18:23.811468+0000","last_undegraded":"2023-10-06T11:18:23.811468+0000","last_fullsized":"2023-10-06T11:18:23.811468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080611+0000","last_change":"2023-10-06T11:18:22.785631+0000","last_active":"2023-10-06T11:18:24.080611+0000","last_peered":"2023-10-06T11:18:24.080611+0000","last_clean":"2023-10-06T11:18:24.080611+0000","last_became_active":"2023-10-06T11:18:22.785163+0000","last_became_peered":"2023-10-06T11:18:22.785163+0000","last_unstale":"2023-10-06T11:18:24.080611+0000","last_undegraded":"2023-10-06T11:18:24.080611+0000","last_fullsized":"2023-10-06T11:18:24.080611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796325+0000","last_change":"2023-10-06T11:18:23.796325+0000","last_active":"2023-10-06T11:18:23.796325+0000","last_peered":"2023-10-06T11:18:23.796325+0000","last_clean":"2023-10-06T11:18:23.796325+0000","last_became_active":"2023-10-06T11:18:23.795981+0000","last_became_peered":"2023-10-06T11:18:23.795981+0000","last_unstale":"2023-10-06T11:18:23.796325+0000","last_undegraded":"2023-10-06T11:18:23.796325+0000","last_fullsized":"2023-10-06T11:18:23.796325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.775454+0000","last_change":"2023-10-06T11:18:22.771501+0000","last_active":"2023-10-06T11:18:22.775454+0000","last_peered":"2023-10-06T11:18:22.775454+0000","last_clean":"2023-10-06T11:18:22.775454+0000","last_became_active":"2023-10-06T11:18:22.771165+0000","last_became_peered":"2023-10-06T11:18:22.771165+0000","last_unstale":"2023-10-06T11:18:22.775454+0000","last_undegraded":"2023-10-06T11:18:22.775454+0000","last_fullsized":"2023-10-06T11:18:22.775454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801731+0000","last_change":"2023-10-06T11:18:23.801731+0000","last_active":"2023-10-06T11:18:23.801731+0000","last_peered":"2023-10-06T11:18:23.801731+0000","last_clean":"2023-10-06T11:18:23.801731+0000","last_became_active":"2023-10-06T11:18:23.801224+0000","last_became_peered":"2023-10-06T11:18:23.801224+0000","last_unstale":"2023-10-06T11:18:23.801731+0000","last_undegraded":"2023-10-06T11:18:23.801731+0000","last_fullsized":"2023-10-06T11:18:23.801731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786601+0000","last_change":"2023-10-06T11:18:22.783842+0000","last_active":"2023-10-06T11:18:22.786601+0000","last_peered":"2023-10-06T11:18:22.786601+0000","last_clean":"2023-10-06T11:18:22.786601+0000","last_became_active":"2023-10-06T11:18:22.783033+0000","last_became_peered":"2023-10-06T11:18:22.783033+0000","last_unstale":"2023-10-06T11:18:22.786601+0000","last_undegraded":"2023-10-06T11:18:22.786601+0000","last_fullsized":"2023-10-06T11:18:22.786601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793261+0000","last_change":"2023-10-06T11:18:23.793261+0000","last_active":"2023-10-06T11:18:23.793261+0000","last_peered":"2023-10-06T11:18:23.793261+0000","last_clean":"2023-10-06T11:18:23.793261+0000","last_became_active":"2023-10-06T11:18:23.792951+0000","last_became_peered":"2023-10-06T11:18:23.792951+0000","last_unstale":"2023-10-06T11:18:23.793261+0000","last_undegraded":"2023-10-06T11:18:23.793261+0000","last_fullsized":"2023-10-06T11:18:23.793261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.082435+0000","last_change":"2023-10-06T11:18:22.773438+0000","last_active":"2023-10-06T11:18:24.082435+0000","last_peered":"2023-10-06T11:18:24.082435+0000","last_clean":"2023-10-06T11:18:24.082435+0000","last_became_active":"2023-10-06T11:18:22.773047+0000","last_became_peered":"2023-10-06T11:18:22.773047+0000","last_unstale":"2023-10-06T11:18:24.082435+0000","last_undegraded":"2023-10-06T11:18:24.082435+0000","last_fullsized":"2023-10-06T11:18:24.082435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788147+0000","last_change":"2023-10-06T11:18:22.783414+0000","last_active":"2023-10-06T11:18:23.788147+0000","last_peered":"2023-10-06T11:18:23.788147+0000","last_clean":"2023-10-06T11:18:23.788147+0000","last_became_active":"2023-10-06T11:18:22.783068+0000","last_became_peered":"2023-10-06T11:18:22.783068+0000","last_unstale":"2023-10-06T11:18:23.788147+0000","last_undegraded":"2023-10-06T11:18:23.788147+0000","last_fullsized":"2023-10-06T11:18:23.788147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792041+0000","last_change":"2023-10-06T11:18:23.792041+0000","last_active":"2023-10-06T11:18:23.792041+0000","last_peered":"2023-10-06T11:18:23.792041+0000","last_clean":"2023-10-06T11:18:23.792041+0000","last_became_active":"2023-10-06T11:18:23.791770+0000","last_became_peered":"2023-10-06T11:18:23.791770+0000","last_unstale":"2023-10-06T11:18:23.792041+0000","last_undegraded":"2023-10-06T11:18:23.792041+0000","last_fullsized":"2023-10-06T11:18:23.792041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080529+0000","last_change":"2023-10-06T11:18:22.783471+0000","last_active":"2023-10-06T11:18:24.080529+0000","last_peered":"2023-10-06T11:18:24.080529+0000","last_clean":"2023-10-06T11:18:24.080529+0000","last_became_active":"2023-10-06T11:18:22.783150+0000","last_became_peered":"2023-10-06T11:18:22.783150+0000","last_unstale":"2023-10-06T11:18:24.080529+0000","last_undegraded":"2023-10-06T11:18:24.080529+0000","last_fullsized":"2023-10-06T11:18:24.080529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:57.801692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797336+0000","last_change":"2023-10-06T11:18:23.797336+0000","last_active":"2023-10-06T11:18:23.797336+0000","last_peered":"2023-10-06T11:18:23.797336+0000","last_clean":"2023-10-06T11:18:23.797336+0000","last_became_active":"2023-10-06T11:18:23.797002+0000","last_became_peered":"2023-10-06T11:18:23.797002+0000","last_unstale":"2023-10-06T11:18:23.797336+0000","last_undegraded":"2023-10-06T11:18:23.797336+0000","last_fullsized":"2023-10-06T11:18:23.797336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:58.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786077+0000","last_change":"2023-10-06T11:18:22.777073+0000","last_active":"2023-10-06T11:18:23.786077+0000","last_peered":"2023-10-06T11:18:23.786077+0000","last_clean":"2023-10-06T11:18:23.786077+0000","last_became_active":"2023-10-06T11:18:22.776784+0000","last_became_peered":"2023-10-06T11:18:22.776784+0000","last_unstale":"2023-10-06T11:18:23.786077+0000","last_undegraded":"2023-10-06T11:18:23.786077+0000","last_fullsized":"2023-10-06T11:18:23.786077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.807687+0000","last_change":"2023-10-06T11:18:23.807687+0000","last_active":"2023-10-06T11:18:23.807687+0000","last_peered":"2023-10-06T11:18:23.807687+0000","last_clean":"2023-10-06T11:18:23.807687+0000","last_became_active":"2023-10-06T11:18:23.807468+0000","last_became_peered":"2023-10-06T11:18:23.807468+0000","last_unstale":"2023-10-06T11:18:23.807687+0000","last_undegraded":"2023-10-06T11:18:23.807687+0000","last_fullsized":"2023-10-06T11:18:23.807687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:11.773581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.775336+0000","last_change":"2023-10-06T11:18:22.770019+0000","last_active":"2023-10-06T11:18:22.775336+0000","last_peered":"2023-10-06T11:18:22.775336+0000","last_clean":"2023-10-06T11:18:22.775336+0000","last_became_active":"2023-10-06T11:18:22.769681+0000","last_became_peered":"2023-10-06T11:18:22.769681+0000","last_unstale":"2023-10-06T11:18:22.775336+0000","last_undegraded":"2023-10-06T11:18:22.775336+0000","last_fullsized":"2023-10-06T11:18:22.775336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804489+0000","last_change":"2023-10-06T11:18:23.804489+0000","last_active":"2023-10-06T11:18:23.804489+0000","last_peered":"2023-10-06T11:18:23.804489+0000","last_clean":"2023-10-06T11:18:23.804489+0000","last_became_active":"2023-10-06T11:18:23.804154+0000","last_became_peered":"2023-10-06T11:18:23.804154+0000","last_unstale":"2023-10-06T11:18:23.804489+0000","last_undegraded":"2023-10-06T11:18:23.804489+0000","last_fullsized":"2023-10-06T11:18:23.804489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.078639+0000","last_change":"2023-10-06T11:18:22.783172+0000","last_active":"2023-10-06T11:18:24.078639+0000","last_peered":"2023-10-06T11:18:24.078639+0000","last_clean":"2023-10-06T11:18:24.078639+0000","last_became_active":"2023-10-06T11:18:22.782848+0000","last_became_peered":"2023-10-06T11:18:22.782848+0000","last_unstale":"2023-10-06T11:18:24.078639+0000","last_undegraded":"2023-10-06T11:18:24.078639+0000","last_fullsized":"2023-10-06T11:18:24.078639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799401+0000","last_change":"2023-10-06T11:18:23.799401+0000","last_active":"2023-10-06T11:18:23.799401+0000","last_peered":"2023-10-06T11:18:23.799401+0000","last_clean":"2023-10-06T11:18:23.799401+0000","last_became_active":"2023-10-06T11:18:23.799110+0000","last_became_peered":"2023-10-06T11:18:23.799110+0000","last_unstale":"2023-10-06T11:18:23.799401+0000","last_undegraded":"2023-10-06T11:18:23.799401+0000","last_fullsized":"2023-10-06T11:18:23.799401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.789863+0000","last_change":"2023-10-06T11:18:22.788211+0000","last_active":"2023-10-06T11:18:22.789863+0000","last_peered":"2023-10-06T11:18:22.789863+0000","last_clean":"2023-10-06T11:18:22.789863+0000","last_became_active":"2023-10-06T11:18:22.787835+0000","last_became_peered":"2023-10-06T11:18:22.787835+0000","last_unstale":"2023-10-06T11:18:22.789863+0000","last_undegraded":"2023-10-06T11:18:22.789863+0000","last_fullsized":"2023-10-06T11:18:22.789863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794251+0000","last_change":"2023-10-06T11:18:23.794251+0000","last_active":"2023-10-06T11:18:23.794251+0000","last_peered":"2023-10-06T11:18:23.794251+0000","last_clean":"2023-10-06T11:18:23.794251+0000","last_became_active":"2023-10-06T11:18:23.793840+0000","last_became_peered":"2023-10-06T11:18:23.793840+0000","last_unstale":"2023-10-06T11:18:23.794251+0000","last_undegraded":"2023-10-06T11:18:23.794251+0000","last_fullsized":"2023-10-06T11:18:23.794251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:26.606422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.788730+0000","last_change":"2023-10-06T11:18:22.786197+0000","last_active":"2023-10-06T11:18:22.788730+0000","last_peered":"2023-10-06T11:18:22.788730+0000","last_clean":"2023-10-06T11:18:22.788730+0000","last_became_active":"2023-10-06T11:18:22.785533+0000","last_became_peered":"2023-10-06T11:18:22.785533+0000","last_unstale":"2023-10-06T11:18:22.788730+0000","last_undegraded":"2023-10-06T11:18:22.788730+0000","last_fullsized":"2023-10-06T11:18:22.788730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:46.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792349+0000","last_change":"2023-10-06T11:18:23.792349+0000","last_active":"2023-10-06T11:18:23.792349+0000","last_peered":"2023-10-06T11:18:23.792349+0000","last_clean":"2023-10-06T11:18:23.792349+0000","last_became_active":"2023-10-06T11:18:23.792058+0000","last_became_peered":"2023-10-06T11:18:23.792058+0000","last_unstale":"2023-10-06T11:18:23.792349+0000","last_undegraded":"2023-10-06T11:18:23.792349+0000","last_fullsized":"2023-10-06T11:18:23.792349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.782703+0000","last_change":"2023-10-06T11:18:22.778447+0000","last_active":"2023-10-06T11:18:22.782703+0000","last_peered":"2023-10-06T11:18:22.782703+0000","last_clean":"2023-10-06T11:18:22.782703+0000","last_became_active":"2023-10-06T11:18:22.778005+0000","last_became_peered":"2023-10-06T11:18:22.778005+0000","last_unstale":"2023-10-06T11:18:22.782703+0000","last_undegraded":"2023-10-06T11:18:22.782703+0000","last_fullsized":"2023-10-06T11:18:22.782703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.805117+0000","last_change":"2023-10-06T11:18:23.805117+0000","last_active":"2023-10-06T11:18:23.805117+0000","last_peered":"2023-10-06T11:18:23.805117+0000","last_clean":"2023-10-06T11:18:23.805117+0000","last_became_active":"2023-10-06T11:18:23.804771+0000","last_became_peered":"2023-10-06T11:18:23.804771+0000","last_unstale":"2023-10-06T11:18:23.805117+0000","last_undegraded":"2023-10-06T11:18:23.805117+0000","last_fullsized":"2023-10-06T11:18:23.805117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.766166+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:18:22.766166+0000","last_peered":"2023-10-06T11:18:22.766166+0000","last_clean":"2023-10-06T11:18:22.766166+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:18:22.766166+0000","last_undegraded":"2023-10-06T11:18:22.766166+0000","last_fullsized":"2023-10-06T11:18:22.766166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.764968+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:18:22.764968+0000","last_peered":"2023-10-06T11:18:22.764968+0000","last_clean":"2023-10-06T11:18:22.764968+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:18:22.764968+0000","last_undegraded":"2023-10-06T11:18:22.764968+0000","last_fullsized":"2023-10-06T11:18:22.764968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.779389+0000","last_change":"2023-10-06T11:18:22.774869+0000","last_active":"2023-10-06T11:18:22.779389+0000","last_peered":"2023-10-06T11:18:22.779389+0000","last_clean":"2023-10-06T11:18:22.779389+0000","last_became_active":"2023-10-06T11:18:22.773053+0000","last_became_peered":"2023-10-06T11:18:22.773053+0000","last_unstale":"2023-10-06T11:18:22.779389+0000","last_undegraded":"2023-10-06T11:18:22.779389+0000","last_fullsized":"2023-10-06T11:18:22.779389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798289+0000","last_change":"2023-10-06T11:18:23.798289+0000","last_active":"2023-10-06T11:18:23.798289+0000","last_peered":"2023-10-06T11:18:23.798289+0000","last_clean":"2023-10-06T11:18:23.798289+0000","last_became_active":"2023-10-06T11:18:23.797843+0000","last_became_peered":"2023-10-06T11:18:23.797843+0000","last_unstale":"2023-10-06T11:18:23.798289+0000","last_undegraded":"2023-10-06T11:18:23.798289+0000","last_fullsized":"2023-10-06T11:18:23.798289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.766066+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:18:22.766066+0000","last_peered":"2023-10-06T11:18:22.766066+0000","last_clean":"2023-10-06T11:18:22.766066+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:18:22.766066+0000","last_undegraded":"2023-10-06T11:18:22.766066+0000","last_fullsized":"2023-10-06T11:18:22.766066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786478+0000","last_change":"2023-10-06T11:18:22.784973+0000","last_active":"2023-10-06T11:18:22.786478+0000","last_peered":"2023-10-06T11:18:22.786478+0000","last_clean":"2023-10-06T11:18:22.786478+0000","last_became_active":"2023-10-06T11:18:22.784647+0000","last_became_peered":"2023-10-06T11:18:22.784647+0000","last_unstale":"2023-10-06T11:18:22.786478+0000","last_undegraded":"2023-10-06T11:18:22.786478+0000","last_fullsized":"2023-10-06T11:18:22.786478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808610+0000","last_change":"2023-10-06T11:18:23.808610+0000","last_active":"2023-10-06T11:18:23.808610+0000","last_peered":"2023-10-06T11:18:23.808610+0000","last_clean":"2023-10-06T11:18:23.808610+0000","last_became_active":"2023-10-06T11:18:23.808298+0000","last_became_peered":"2023-10-06T11:18:23.808298+0000","last_unstale":"2023-10-06T11:18:23.808610+0000","last_undegraded":"2023-10-06T11:18:23.808610+0000","last_fullsized":"2023-10-06T11:18:23.808610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.774027+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:18:22.774027+0000","last_peered":"2023-10-06T11:18:22.774027+0000","last_clean":"2023-10-06T11:18:22.774027+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:18:22.774027+0000","last_undegraded":"2023-10-06T11:18:22.774027+0000","last_fullsized":"2023-10-06T11:18:22.774027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786502+0000","last_change":"2023-10-06T11:18:22.782337+0000","last_active":"2023-10-06T11:18:22.786502+0000","last_peered":"2023-10-06T11:18:22.786502+0000","last_clean":"2023-10-06T11:18:22.786502+0000","last_became_active":"2023-10-06T11:18:22.780763+0000","last_became_peered":"2023-10-06T11:18:22.780763+0000","last_unstale":"2023-10-06T11:18:22.786502+0000","last_undegraded":"2023-10-06T11:18:22.786502+0000","last_fullsized":"2023-10-06T11:18:22.786502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798843+0000","last_change":"2023-10-06T11:18:23.798843+0000","last_active":"2023-10-06T11:18:23.798843+0000","last_peered":"2023-10-06T11:18:23.798843+0000","last_clean":"2023-10-06T11:18:23.798843+0000","last_became_active":"2023-10-06T11:18:23.798591+0000","last_became_peered":"2023-10-06T11:18:23.798591+0000","last_unstale":"2023-10-06T11:18:23.798843+0000","last_undegraded":"2023-10-06T11:18:23.798843+0000","last_fullsized":"2023-10-06T11:18:23.798843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.765052+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:18:22.765052+0000","last_peered":"2023-10-06T11:18:22.765052+0000","last_clean":"2023-10-06T11:18:22.765052+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:18:22.765052+0000","last_undegraded":"2023-10-06T11:18:22.765052+0000","last_fullsized":"2023-10-06T11:18:22.765052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.787357+0000","last_change":"2023-10-06T11:18:22.781890+0000","last_active":"2023-10-06T11:18:22.787357+0000","last_peered":"2023-10-06T11:18:22.787357+0000","last_clean":"2023-10-06T11:18:22.787357+0000","last_became_active":"2023-10-06T11:18:22.781589+0000","last_became_peered":"2023-10-06T11:18:22.781589+0000","last_unstale":"2023-10-06T11:18:22.787357+0000","last_undegraded":"2023-10-06T11:18:22.787357+0000","last_fullsized":"2023-10-06T11:18:22.787357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803070+0000","last_change":"2023-10-06T11:18:23.803070+0000","last_active":"2023-10-06T11:18:23.803070+0000","last_peered":"2023-10-06T11:18:23.803070+0000","last_clean":"2023-10-06T11:18:23.803070+0000","last_became_active":"2023-10-06T11:18:23.799605+0000","last_became_peered":"2023-10-06T11:18:23.799605+0000","last_unstale":"2023-10-06T11:18:23.803070+0000","last_undegraded":"2023-10-06T11:18:23.803070+0000","last_fullsized":"2023-10-06T11:18:23.803070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.776258+0000","last_change":"2023-10-06T11:18:22.770766+0000","last_active":"2023-10-06T11:18:22.776258+0000","last_peered":"2023-10-06T11:18:22.776258+0000","last_clean":"2023-10-06T11:18:22.776258+0000","last_became_active":"2023-10-06T11:18:22.770433+0000","last_became_peered":"2023-10-06T11:18:22.770433+0000","last_unstale":"2023-10-06T11:18:22.776258+0000","last_undegraded":"2023-10-06T11:18:22.776258+0000","last_fullsized":"2023-10-06T11:18:22.776258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.750418+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:18:22.750418+0000","last_peered":"2023-10-06T11:18:22.750418+0000","last_clean":"2023-10-06T11:18:22.750418+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:18:22.750418+0000","last_undegraded":"2023-10-06T11:18:22.750418+0000","last_fullsized":"2023-10-06T11:18:22.750418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804833+0000","last_change":"2023-10-06T11:18:23.804833+0000","last_active":"2023-10-06T11:18:23.804833+0000","last_peered":"2023-10-06T11:18:23.804833+0000","last_clean":"2023-10-06T11:18:23.804833+0000","last_became_active":"2023-10-06T11:18:23.804036+0000","last_became_peered":"2023-10-06T11:18:23.804036+0000","last_unstale":"2023-10-06T11:18:23.804833+0000","last_undegraded":"2023-10-06T11:18:23.804833+0000","last_fullsized":"2023-10-06T11:18:23.804833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786609+0000","last_change":"2023-10-06T11:18:22.783968+0000","last_active":"2023-10-06T11:18:22.786609+0000","last_peered":"2023-10-06T11:18:22.786609+0000","last_clean":"2023-10-06T11:18:22.786609+0000","last_became_active":"2023-10-06T11:18:22.783402+0000","last_became_peered":"2023-10-06T11:18:22.783402+0000","last_unstale":"2023-10-06T11:18:22.786609+0000","last_undegraded":"2023-10-06T11:18:22.786609+0000","last_fullsized":"2023-10-06T11:18:22.786609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.767007+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:18:22.767007+0000","last_peered":"2023-10-06T11:18:22.767007+0000","last_clean":"2023-10-06T11:18:22.767007+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:18:22.767007+0000","last_undegraded":"2023-10-06T11:18:22.767007+0000","last_fullsized":"2023-10-06T11:18:22.767007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792986+0000","last_change":"2023-10-06T11:18:23.792986+0000","last_active":"2023-10-06T11:18:23.792986+0000","last_peered":"2023-10-06T11:18:23.792986+0000","last_clean":"2023-10-06T11:18:23.792986+0000","last_became_active":"2023-10-06T11:18:23.792504+0000","last_became_peered":"2023-10-06T11:18:23.792504+0000","last_unstale":"2023-10-06T11:18:23.792986+0000","last_undegraded":"2023-10-06T11:18:23.792986+0000","last_fullsized":"2023-10-06T11:18:23.792986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079144+0000","last_change":"2023-10-06T11:18:22.773444+0000","last_active":"2023-10-06T11:18:24.079144+0000","last_peered":"2023-10-06T11:18:24.079144+0000","last_clean":"2023-10-06T11:18:24.079144+0000","last_became_active":"2023-10-06T11:18:22.773056+0000","last_became_peered":"2023-10-06T11:18:22.773056+0000","last_unstale":"2023-10-06T11:18:24.079144+0000","last_undegraded":"2023-10-06T11:18:24.079144+0000","last_fullsized":"2023-10-06T11:18:24.079144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.750442+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:18:22.750442+0000","last_peered":"2023-10-06T11:18:22.750442+0000","last_clean":"2023-10-06T11:18:22.750442+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:18:22.750442+0000","last_undegraded":"2023-10-06T11:18:22.750442+0000","last_fullsized":"2023-10-06T11:18:22.750442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799064+0000","last_change":"2023-10-06T11:18:23.799064+0000","last_active":"2023-10-06T11:18:23.799064+0000","last_peered":"2023-10-06T11:18:23.799064+0000","last_clean":"2023-10-06T11:18:23.799064+0000","last_became_active":"2023-10-06T11:18:23.798759+0000","last_became_peered":"2023-10-06T11:18:23.798759+0000","last_unstale":"2023-10-06T11:18:23.799064+0000","last_undegraded":"2023-10-06T11:18:23.799064+0000","last_fullsized":"2023-10-06T11:18:23.799064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.785242+0000","last_change":"2023-10-06T11:18:22.781854+0000","last_active":"2023-10-06T11:18:22.785242+0000","last_peered":"2023-10-06T11:18:22.785242+0000","last_clean":"2023-10-06T11:18:22.785242+0000","last_became_active":"2023-10-06T11:18:22.781438+0000","last_became_peered":"2023-10-06T11:18:22.781438+0000","last_unstale":"2023-10-06T11:18:22.785242+0000","last_undegraded":"2023-10-06T11:18:22.785242+0000","last_fullsized":"2023-10-06T11:18:22.785242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.765678+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:18:22.765678+0000","last_peered":"2023-10-06T11:18:22.765678+0000","last_clean":"2023-10-06T11:18:22.765678+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:18:22.765678+0000","last_undegraded":"2023-10-06T11:18:22.765678+0000","last_fullsized":"2023-10-06T11:18:22.765678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786622+0000","last_change":"2023-10-06T11:18:23.786622+0000","last_active":"2023-10-06T11:18:23.786622+0000","last_peered":"2023-10-06T11:18:23.786622+0000","last_clean":"2023-10-06T11:18:23.786622+0000","last_became_active":"2023-10-06T11:18:23.786299+0000","last_became_peered":"2023-10-06T11:18:23.786299+0000","last_unstale":"2023-10-06T11:18:23.786622+0000","last_undegraded":"2023-10-06T11:18:23.786622+0000","last_fullsized":"2023-10-06T11:18:23.786622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792771+0000","last_change":"2023-10-06T11:18:23.792771+0000","last_active":"2023-10-06T11:18:23.792771+0000","last_peered":"2023-10-06T11:18:23.792771+0000","last_clean":"2023-10-06T11:18:23.792771+0000","last_became_active":"2023-10-06T11:18:23.787870+0000","last_became_peered":"2023-10-06T11:18:23.787870+0000","last_unstale":"2023-10-06T11:18:23.792771+0000","last_undegraded":"2023-10-06T11:18:23.792771+0000","last_fullsized":"2023-10-06T11:18:23.792771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081899+0000","last_change":"2023-10-06T11:18:22.781178+0000","last_active":"2023-10-06T11:18:24.081899+0000","last_peered":"2023-10-06T11:18:24.081899+0000","last_clean":"2023-10-06T11:18:24.081899+0000","last_became_active":"2023-10-06T11:18:22.778268+0000","last_became_peered":"2023-10-06T11:18:22.778268+0000","last_unstale":"2023-10-06T11:18:24.081899+0000","last_undegraded":"2023-10-06T11:18:24.081899+0000","last_fullsized":"2023-10-06T11:18:24.081899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800123+0000","last_change":"2023-10-06T11:18:23.800123+0000","last_active":"2023-10-06T11:18:23.800123+0000","last_peered":"2023-10-06T11:18:23.800123+0000","last_clean":"2023-10-06T11:18:23.800123+0000","last_became_active":"2023-10-06T11:18:23.799794+0000","last_became_peered":"2023-10-06T11:18:23.799794+0000","last_unstale":"2023-10-06T11:18:23.800123+0000","last_undegraded":"2023-10-06T11:18:23.800123+0000","last_fullsized":"2023-10-06T11:18:23.800123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.078042+0000","last_change":"2023-10-06T11:18:22.782320+0000","last_active":"2023-10-06T11:18:24.078042+0000","last_peered":"2023-10-06T11:18:24.078042+0000","last_clean":"2023-10-06T11:18:24.078042+0000","last_became_active":"2023-10-06T11:18:22.782025+0000","last_became_peered":"2023-10-06T11:18:22.782025+0000","last_unstale":"2023-10-06T11:18:24.078042+0000","last_undegraded":"2023-10-06T11:18:24.078042+0000","last_fullsized":"2023-10-06T11:18:24.078042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.776433+0000","last_change":"2023-10-06T11:18:22.773270+0000","last_active":"2023-10-06T11:18:22.776433+0000","last_peered":"2023-10-06T11:18:22.776433+0000","last_clean":"2023-10-06T11:18:22.776433+0000","last_became_active":"2023-10-06T11:18:22.772211+0000","last_became_peered":"2023-10-06T11:18:22.772211+0000","last_unstale":"2023-10-06T11:18:22.776433+0000","last_undegraded":"2023-10-06T11:18:22.776433+0000","last_fullsized":"2023-10-06T11:18:22.776433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794332+0000","last_change":"2023-10-06T11:18:23.794332+0000","last_active":"2023-10-06T11:18:23.794332+0000","last_peered":"2023-10-06T11:18:23.794332+0000","last_clean":"2023-10-06T11:18:23.794332+0000","last_became_active":"2023-10-06T11:18:23.793993+0000","last_became_peered":"2023-10-06T11:18:23.793993+0000","last_unstale":"2023-10-06T11:18:23.794332+0000","last_undegraded":"2023-10-06T11:18:23.794332+0000","last_fullsized":"2023-10-06T11:18:23.794332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.780979+0000","last_change":"2023-10-06T11:18:22.774828+0000","last_active":"2023-10-06T11:18:22.780979+0000","last_peered":"2023-10-06T11:18:22.780979+0000","last_clean":"2023-10-06T11:18:22.780979+0000","last_became_active":"2023-10-06T11:18:22.774440+0000","last_became_peered":"2023-10-06T11:18:22.774440+0000","last_unstale":"2023-10-06T11:18:22.780979+0000","last_undegraded":"2023-10-06T11:18:22.780979+0000","last_fullsized":"2023-10-06T11:18:22.780979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804188+0000","last_change":"2023-10-06T11:18:23.804188+0000","last_active":"2023-10-06T11:18:23.804188+0000","last_peered":"2023-10-06T11:18:23.804188+0000","last_clean":"2023-10-06T11:18:23.804188+0000","last_became_active":"2023-10-06T11:18:23.803863+0000","last_became_peered":"2023-10-06T11:18:23.803863+0000","last_unstale":"2023-10-06T11:18:23.804188+0000","last_undegraded":"2023-10-06T11:18:23.804188+0000","last_fullsized":"2023-10-06T11:18:23.804188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789620+0000","last_change":"2023-10-06T11:18:23.789620+0000","last_active":"2023-10-06T11:18:23.789620+0000","last_peered":"2023-10-06T11:18:23.789620+0000","last_clean":"2023-10-06T11:18:23.789620+0000","last_became_active":"2023-10-06T11:18:23.789349+0000","last_became_peered":"2023-10-06T11:18:23.789349+0000","last_unstale":"2023-10-06T11:18:23.789620+0000","last_undegraded":"2023-10-06T11:18:23.789620+0000","last_fullsized":"2023-10-06T11:18:23.789620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.782986+0000","last_change":"2023-10-06T11:18:22.781828+0000","last_active":"2023-10-06T11:18:22.782986+0000","last_peered":"2023-10-06T11:18:22.782986+0000","last_clean":"2023-10-06T11:18:22.782986+0000","last_became_active":"2023-10-06T11:18:22.781529+0000","last_became_peered":"2023-10-06T11:18:22.781529+0000","last_unstale":"2023-10-06T11:18:22.782986+0000","last_undegraded":"2023-10-06T11:18:22.782986+0000","last_fullsized":"2023-10-06T11:18:22.782986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792292+0000","last_change":"2023-10-06T11:18:23.792292+0000","last_active":"2023-10-06T11:18:23.792292+0000","last_peered":"2023-10-06T11:18:23.792292+0000","last_clean":"2023-10-06T11:18:23.792292+0000","last_became_active":"2023-10-06T11:18:23.791847+0000","last_became_peered":"2023-10-06T11:18:23.791847+0000","last_unstale":"2023-10-06T11:18:23.792292+0000","last_undegraded":"2023-10-06T11:18:23.792292+0000","last_fullsized":"2023-10-06T11:18:23.792292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.784824+0000","last_change":"2023-10-06T11:18:22.774677+0000","last_active":"2023-10-06T11:18:22.784824+0000","last_peered":"2023-10-06T11:18:22.784824+0000","last_clean":"2023-10-06T11:18:22.784824+0000","last_became_active":"2023-10-06T11:18:22.772746+0000","last_became_peered":"2023-10-06T11:18:22.772746+0000","last_unstale":"2023-10-06T11:18:22.784824+0000","last_undegraded":"2023-10-06T11:18:22.784824+0000","last_fullsized":"2023-10-06T11:18:22.784824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806392+0000","last_change":"2023-10-06T11:18:23.806392+0000","last_active":"2023-10-06T11:18:23.806392+0000","last_peered":"2023-10-06T11:18:23.806392+0000","last_clean":"2023-10-06T11:18:23.806392+0000","last_became_active":"2023-10-06T11:18:23.806086+0000","last_became_peered":"2023-10-06T11:18:23.806086+0000","last_unstale":"2023-10-06T11:18:23.806392+0000","last_undegraded":"2023-10-06T11:18:23.806392+0000","last_fullsized":"2023-10-06T11:18:23.806392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780727+0000","last_change":"2023-10-06T11:18:22.786752+0000","last_active":"2023-10-06T11:18:23.780727+0000","last_peered":"2023-10-06T11:18:23.780727+0000","last_clean":"2023-10-06T11:18:23.780727+0000","last_became_active":"2023-10-06T11:18:22.784966+0000","last_became_peered":"2023-10-06T11:18:22.784966+0000","last_unstale":"2023-10-06T11:18:23.780727+0000","last_undegraded":"2023-10-06T11:18:23.780727+0000","last_fullsized":"2023-10-06T11:18:23.780727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811011+0000","last_change":"2023-10-06T11:18:23.811011+0000","last_active":"2023-10-06T11:18:23.811011+0000","last_peered":"2023-10-06T11:18:23.811011+0000","last_clean":"2023-10-06T11:18:23.811011+0000","last_became_active":"2023-10-06T11:18:23.810563+0000","last_became_peered":"2023-10-06T11:18:23.810563+0000","last_unstale":"2023-10-06T11:18:23.811011+0000","last_undegraded":"2023-10-06T11:18:23.811011+0000","last_fullsized":"2023-10-06T11:18:23.811011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:06.048708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787200+0000","last_change":"2023-10-06T11:18:22.787138+0000","last_active":"2023-10-06T11:18:23.787200+0000","last_peered":"2023-10-06T11:18:23.787200+0000","last_clean":"2023-10-06T11:18:23.787200+0000","last_became_active":"2023-10-06T11:18:22.786799+0000","last_became_peered":"2023-10-06T11:18:22.786799+0000","last_unstale":"2023-10-06T11:18:23.787200+0000","last_undegraded":"2023-10-06T11:18:23.787200+0000","last_fullsized":"2023-10-06T11:18:23.787200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797231+0000","last_change":"2023-10-06T11:18:23.797231+0000","last_active":"2023-10-06T11:18:23.797231+0000","last_peered":"2023-10-06T11:18:23.797231+0000","last_clean":"2023-10-06T11:18:23.797231+0000","last_became_active":"2023-10-06T11:18:23.796942+0000","last_became_peered":"2023-10-06T11:18:23.796942+0000","last_unstale":"2023-10-06T11:18:23.797231+0000","last_undegraded":"2023-10-06T11:18:23.797231+0000","last_fullsized":"2023-10-06T11:18:23.797231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079862+0000","last_change":"2023-10-06T11:18:22.784448+0000","last_active":"2023-10-06T11:18:24.079862+0000","last_peered":"2023-10-06T11:18:24.079862+0000","last_clean":"2023-10-06T11:18:24.079862+0000","last_became_active":"2023-10-06T11:18:22.784131+0000","last_became_peered":"2023-10-06T11:18:22.784131+0000","last_unstale":"2023-10-06T11:18:24.079862+0000","last_undegraded":"2023-10-06T11:18:24.079862+0000","last_fullsized":"2023-10-06T11:18:24.079862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796243+0000","last_change":"2023-10-06T11:18:23.796243+0000","last_active":"2023-10-06T11:18:23.796243+0000","last_peered":"2023-10-06T11:18:23.796243+0000","last_clean":"2023-10-06T11:18:23.796243+0000","last_became_active":"2023-10-06T11:18:23.795924+0000","last_became_peered":"2023-10-06T11:18:23.795924+0000","last_unstale":"2023-10-06T11:18:23.796243+0000","last_undegraded":"2023-10-06T11:18:23.796243+0000","last_fullsized":"2023-10-06T11:18:23.796243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080409+0000","last_change":"2023-10-06T11:18:22.789060+0000","last_active":"2023-10-06T11:18:24.080409+0000","last_peered":"2023-10-06T11:18:24.080409+0000","last_clean":"2023-10-06T11:18:24.080409+0000","last_became_active":"2023-10-06T11:18:22.788376+0000","last_became_peered":"2023-10-06T11:18:22.788376+0000","last_unstale":"2023-10-06T11:18:24.080409+0000","last_undegraded":"2023-10-06T11:18:24.080409+0000","last_fullsized":"2023-10-06T11:18:24.080409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802973+0000","last_change":"2023-10-06T11:18:23.802973+0000","last_active":"2023-10-06T11:18:23.802973+0000","last_peered":"2023-10-06T11:18:23.802973+0000","last_clean":"2023-10-06T11:18:23.802973+0000","last_became_active":"2023-10-06T11:18:23.802597+0000","last_became_peered":"2023-10-06T11:18:23.802597+0000","last_unstale":"2023-10-06T11:18:23.802973+0000","last_undegraded":"2023-10-06T11:18:23.802973+0000","last_fullsized":"2023-10-06T11:18:23.802973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780039+0000","last_change":"2023-10-06T11:18:22.770917+0000","last_active":"2023-10-06T11:18:23.780039+0000","last_peered":"2023-10-06T11:18:23.780039+0000","last_clean":"2023-10-06T11:18:23.780039+0000","last_became_active":"2023-10-06T11:18:22.770587+0000","last_became_peered":"2023-10-06T11:18:22.770587+0000","last_unstale":"2023-10-06T11:18:23.780039+0000","last_undegraded":"2023-10-06T11:18:23.780039+0000","last_fullsized":"2023-10-06T11:18:23.780039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797236+0000","last_change":"2023-10-06T11:18:23.797236+0000","last_active":"2023-10-06T11:18:23.797236+0000","last_peered":"2023-10-06T11:18:23.797236+0000","last_clean":"2023-10-06T11:18:23.797236+0000","last_became_active":"2023-10-06T11:18:23.796889+0000","last_became_peered":"2023-10-06T11:18:23.796889+0000","last_unstale":"2023-10-06T11:18:23.797236+0000","last_undegraded":"2023-10-06T11:18:23.797236+0000","last_fullsized":"2023-10-06T11:18:23.797236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.077534+0000","last_change":"2023-10-06T11:18:22.778280+0000","last_active":"2023-10-06T11:18:24.077534+0000","last_peered":"2023-10-06T11:18:24.077534+0000","last_clean":"2023-10-06T11:18:24.077534+0000","last_became_active":"2023-10-06T11:18:22.776627+0000","last_became_peered":"2023-10-06T11:18:22.776627+0000","last_unstale":"2023-10-06T11:18:24.077534+0000","last_undegraded":"2023-10-06T11:18:24.077534+0000","last_fullsized":"2023-10-06T11:18:24.077534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":84270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301612,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070228,"statfs":{"total":96636764160,"available":96327913472,"internally_reserved":0,"allocated":233472,"data_stored":82768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6870000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302192,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069648,"statfs":{"total":96636764160,"available":96327319552,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:18:28.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:18:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.518+0000 7f245aa4c700 1 Processor -- start 2023-10-06T11:18:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 -- start start 2023-10-06T11:18:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24540aaf30 0x7f2454067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454067590 0x7f2454068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540679f0 con 0x7f2454156e20 2023-10-06T11:18:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24540acd60 con 0x7f2454067590 2023-10-06T11:18:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245aa4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24540ad040 con 0x7f24540aaf30 2023-10-06T11:18:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454067590 0x7f2454068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454067590 0x7f2454068a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52418/0 (socket says 172.21.15.46:52418) 2023-10-06T11:18:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f2459a4a700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f2454156e20 0x7f245406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46230/0 (socket says 172.21.15.46:46230) 2023-10-06T11:18:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 -- 172.21.15.46:0/1708023941 learned_addr learned my addr 172.21.15.46:0/1708023941 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f2459249700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24540aaf30 0x7f2454067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f2458a48700 1 -- 172.21.15.46:0/1708023941 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604995274 0 0) 0x7f2450001490 con 0x7f2454156e20 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f2458a48700 1 -- 172.21.15.46:0/1708023941 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440003570 con 0x7f2454156e20 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 -- 172.21.15.46:0/1708023941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24540aaf30 msgr2=0x7f2454067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24540aaf30 0x7f2454067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 -- 172.21.15.46:0/1708023941 >> v1:172.21.15.132:6789/0 conn(0x7f2454156e20 legacy=0x7f245406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.522+0000 7f245a24b700 1 -- 172.21.15.46:0/1708023941 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24540ad320 con 0x7f2454067590 2023-10-06T11:18:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245a24b700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454067590 0x7f2454068a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f244c0099b0 tx=0x7f244c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2abbba366c2b03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f2458a48700 1 -- 172.21.15.46:0/1708023941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00b680 con 0x7f2454067590 2023-10-06T11:18:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f2458a48700 1 -- 172.21.15.46:0/1708023941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c00b7e0 con 0x7f2454067590 2023-10-06T11:18:28.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f2458a48700 1 -- 172.21.15.46:0/1708023941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00bb80 con 0x7f2454067590 2023-10-06T11:18:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 -- 172.21.15.46:0/1708023941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454067590 msgr2=0x7f2454068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454067590 0x7f2454068a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f244c0099b0 tx=0x7f244c009250 comp rx=0 tx=0).stop 2023-10-06T11:18:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 -- 172.21.15.46:0/1708023941 shutdown_connections 2023-10-06T11:18:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454067590 0x7f2454068a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 --2- 172.21.15.46:0/1708023941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24540aaf30 0x7f2454067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 -- 172.21.15.46:0/1708023941 >> 172.21.15.46:0/1708023941 conn(0x7f2454062ef0 msgr2=0x7f24540cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 -- 172.21.15.46:0/1708023941 shutdown_connections 2023-10-06T11:18:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.526+0000 7f245aa4c700 1 -- 172.21.15.46:0/1708023941 wait complete. 2023-10-06T11:18:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 Processor -- start 2023-10-06T11:18:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 -- start start 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 0x7f2454069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2454156e20 0x7f245415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245416cd10 con 0x7f24540aaf30 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245416d890 con 0x7f2454067590 2023-10-06T11:18:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245aa4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245416cf60 con 0x7f2454156e20 2023-10-06T11:18:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459249700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459249700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52434/0 (socket says 172.21.15.46:52434) 2023-10-06T11:18:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459249700 1 -- 172.21.15.46:0/2428534435 learned_addr learned my addr 172.21.15.46:0/2428534435 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459a4a700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 0x7f2454069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f245a24b700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2454156e20 0x7f245415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459249700 1 -- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2454156e20 msgr2=0x7f245415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.530+0000 7f2459249700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2454156e20 0x7f245415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2459249700 1 -- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 msgr2=0x7f2454069470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2459249700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 0x7f2454069470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2459249700 1 -- 172.21.15.46:0/2428534435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0094f0 con 0x7f24540aaf30 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2459a4a700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 0x7f2454069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2459249700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f244400ec80 tx=0x7f24440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400c9d0 con 0x7f24540aaf30 2023-10-06T11:18:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244400cb30 con 0x7f24540aaf30 2023-10-06T11:18:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444013470 con 0x7f24540aaf30 2023-10-06T11:18:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245416d240 con 0x7f24540aaf30 2023-10-06T11:18:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540699b0 con 0x7f24540aaf30 2023-10-06T11:18:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.534+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245406d700 con 0x7f24540aaf30 2023-10-06T11:18:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.538+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2444013860 con 0x7f24540aaf30 2023-10-06T11:18:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.538+0000 7f2449ffb700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 0x7f2438062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.538+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f244407cd60 con 0x7f24540aaf30 2023-10-06T11:18:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.538+0000 7f2459a4a700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 0x7f2438062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.538+0000 7f2459a4a700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 0x7f2438062a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24540c5f30 tx=0x7f2450009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.542+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f245406d700 con 0x7f24540aaf30 2023-10-06T11:18:28.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.710+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2454063fe0 con 0x7f2438060550 2023-10-06T11:18:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.718+0000 7f2449ffb700 1 -- 172.21.15.46:0/2428534435 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286463 (secure 0 0 0) 0x7f2454063fe0 con 0x7f2438060550 2023-10-06T11:18:28.729 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:28.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 msgr2=0x7f2438062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 0x7f2438062a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24540c5f30 tx=0x7f2450009040 comp rx=0 tx=0).stop 2023-10-06T11:18:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 msgr2=0x7f245415b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f244400ec80 tx=0x7f24440091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 shutdown_connections 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2438060550 0x7f2438062a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2454156e20 0x7f245415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540aaf30 0x7f245415b520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 --2- 172.21.15.46:0/2428534435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2454067590 0x7f2454069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 >> 172.21.15.46:0/2428534435 conn(0x7f2454062ef0 msgr2=0x7f24540c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 shutdown_connections 2023-10-06T11:18:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.726+0000 7f245aa4c700 1 -- 172.21.15.46:0/2428534435 wait complete. 2023-10-06T11:18:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:18:28.754 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T11:18:28.484821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397684,"kb_used_data":2996,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577036,"statfs":{"total":773094113280,"available":770638884864,"internally_reserved":0,"allocated":3067904,"data_stored":1839032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001828"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797220+0000","last_change":"2023-10-06T11:18:23.797220+0000","last_active":"2023-10-06T11:18:23.797220+0000","last_peered":"2023-10-06T11:18:23.797220+0000","last_clean":"2023-10-06T11:18:23.797220+0000","last_became_active":"2023-10-06T11:18:23.796644+0000","last_became_peered":"2023-10-06T11:18:23.796644+0000","last_unstale":"2023-10-06T11:18:23.797220+0000","last_undegraded":"2023-10-06T11:18:23.797220+0000","last_fullsized":"2023-10-06T11:18:23.797220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:18.835072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798280+0000","last_change":"2023-10-06T11:18:23.798280+0000","last_active":"2023-10-06T11:18:23.798280+0000","last_peered":"2023-10-06T11:18:23.798280+0000","last_clean":"2023-10-06T11:18:23.798280+0000","last_became_active":"2023-10-06T11:18:23.797912+0000","last_became_peered":"2023-10-06T11:18:23.797912+0000","last_unstale":"2023-10-06T11:18:23.798280+0000","last_undegraded":"2023-10-06T11:18:23.798280+0000","last_fullsized":"2023-10-06T11:18:23.798280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797573+0000","last_change":"2023-10-06T11:18:23.797573+0000","last_active":"2023-10-06T11:18:23.797573+0000","last_peered":"2023-10-06T11:18:23.797573+0000","last_clean":"2023-10-06T11:18:23.797573+0000","last_became_active":"2023-10-06T11:18:23.797165+0000","last_became_peered":"2023-10-06T11:18:23.797165+0000","last_unstale":"2023-10-06T11:18:23.797573+0000","last_undegraded":"2023-10-06T11:18:23.797573+0000","last_fullsized":"2023-10-06T11:18:23.797573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795447+0000","last_change":"2023-10-06T11:18:23.795447+0000","last_active":"2023-10-06T11:18:23.795447+0000","last_peered":"2023-10-06T11:18:23.795447+0000","last_clean":"2023-10-06T11:18:23.795447+0000","last_became_active":"2023-10-06T11:18:23.794305+0000","last_became_peered":"2023-10-06T11:18:23.794305+0000","last_unstale":"2023-10-06T11:18:23.795447+0000","last_undegraded":"2023-10-06T11:18:23.795447+0000","last_fullsized":"2023-10-06T11:18:23.795447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:35.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796286+0000","last_change":"2023-10-06T11:18:23.796286+0000","last_active":"2023-10-06T11:18:23.796286+0000","last_peered":"2023-10-06T11:18:23.796286+0000","last_clean":"2023-10-06T11:18:23.796286+0000","last_became_active":"2023-10-06T11:18:23.795746+0000","last_became_peered":"2023-10-06T11:18:23.795746+0000","last_unstale":"2023-10-06T11:18:23.796286+0000","last_undegraded":"2023-10-06T11:18:23.796286+0000","last_fullsized":"2023-10-06T11:18:23.796286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793146+0000","last_change":"2023-10-06T11:18:23.793146+0000","last_active":"2023-10-06T11:18:23.793146+0000","last_peered":"2023-10-06T11:18:23.793146+0000","last_clean":"2023-10-06T11:18:23.793146+0000","last_became_active":"2023-10-06T11:18:23.792852+0000","last_became_peered":"2023-10-06T11:18:23.792852+0000","last_unstale":"2023-10-06T11:18:23.793146+0000","last_undegraded":"2023-10-06T11:18:23.793146+0000","last_fullsized":"2023-10-06T11:18:23.793146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:31.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809263+0000","last_change":"2023-10-06T11:18:23.809263+0000","last_active":"2023-10-06T11:18:23.809263+0000","last_peered":"2023-10-06T11:18:23.809263+0000","last_clean":"2023-10-06T11:18:23.809263+0000","last_became_active":"2023-10-06T11:18:23.808943+0000","last_became_peered":"2023-10-06T11:18:23.808943+0000","last_unstale":"2023-10-06T11:18:23.809263+0000","last_undegraded":"2023-10-06T11:18:23.809263+0000","last_fullsized":"2023-10-06T11:18:23.809263+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788701+0000","last_change":"2023-10-06T11:18:23.788701+0000","last_active":"2023-10-06T11:18:23.788701+0000","last_peered":"2023-10-06T11:18:23.788701+0000","last_clean":"2023-10-06T11:18:23.788701+0000","last_became_active":"2023-10-06T11:18:23.788343+0000","last_became_peered":"2023-10-06T11:18:23.788343+0000","last_unstale":"2023-10-06T11:18:23.788701+0000","last_undegraded":"2023-10-06T11:18:23.788701+0000","last_fullsized":"2023-10-06T11:18:23.788701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802504+0000","last_change":"2023-10-06T11:18:23.802504+0000","last_active":"2023-10-06T11:18:23.802504+0000","last_peered":"2023-10-06T11:18:23.802504+0000","last_clean":"2023-10-06T11:18:23.802504+0000","last_became_active":"2023-10-06T11:18:23.801692+0000","last_became_peered":"2023-10-06T11:18:23.801692+0000","last_unstale":"2023-10-06T11:18:23.802504+0000","last_undegraded":"2023-10-06T11:18:23.802504+0000","last_fullsized":"2023-10-06T11:18:23.802504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795845+0000","last_change":"2023-10-06T11:18:23.795845+0000","last_active":"2023-10-06T11:18:23.795845+0000","last_peered":"2023-10-06T11:18:23.795845+0000","last_clean":"2023-10-06T11:18:23.795845+0000","last_became_active":"2023-10-06T11:18:23.795351+0000","last_became_peered":"2023-10-06T11:18:23.795351+0000","last_unstale":"2023-10-06T11:18:23.795845+0000","last_undegraded":"2023-10-06T11:18:23.795845+0000","last_fullsized":"2023-10-06T11:18:23.795845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.805599+0000","last_change":"2023-10-06T11:18:23.805599+0000","last_active":"2023-10-06T11:18:23.805599+0000","last_peered":"2023-10-06T11:18:23.805599+0000","last_clean":"2023-10-06T11:18:23.805599+0000","last_became_active":"2023-10-06T11:18:23.805371+0000","last_became_peered":"2023-10-06T11:18:23.805371+0000","last_unstale":"2023-10-06T11:18:23.805599+0000","last_undegraded":"2023-10-06T11:18:23.805599+0000","last_fullsized":"2023-10-06T11:18:23.805599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:25.984507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799204+0000","last_change":"2023-10-06T11:18:23.799204+0000","last_active":"2023-10-06T11:18:23.799204+0000","last_peered":"2023-10-06T11:18:23.799204+0000","last_clean":"2023-10-06T11:18:23.799204+0000","last_became_active":"2023-10-06T11:18:23.798864+0000","last_became_peered":"2023-10-06T11:18:23.798864+0000","last_unstale":"2023-10-06T11:18:23.799204+0000","last_undegraded":"2023-10-06T11:18:23.799204+0000","last_fullsized":"2023-10-06T11:18:23.799204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795240+0000","last_change":"2023-10-06T11:18:23.795240+0000","last_active":"2023-10-06T11:18:23.795240+0000","last_peered":"2023-10-06T11:18:23.795240+0000","last_clean":"2023-10-06T11:18:23.795240+0000","last_became_active":"2023-10-06T11:18:23.794759+0000","last_became_peered":"2023-10-06T11:18:23.794759+0000","last_unstale":"2023-10-06T11:18:23.795240+0000","last_undegraded":"2023-10-06T11:18:23.795240+0000","last_fullsized":"2023-10-06T11:18:23.795240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:08.424249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789935+0000","last_change":"2023-10-06T11:18:23.789935+0000","last_active":"2023-10-06T11:18:23.789935+0000","last_peered":"2023-10-06T11:18:23.789935+0000","last_clean":"2023-10-06T11:18:23.789935+0000","last_became_active":"2023-10-06T11:18:23.789663+0000","last_became_peered":"2023-10-06T11:18:23.789663+0000","last_unstale":"2023-10-06T11:18:23.789935+0000","last_undegraded":"2023-10-06T11:18:23.789935+0000","last_fullsized":"2023-10-06T11:18:23.789935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809338+0000","last_change":"2023-10-06T11:18:23.809338+0000","last_active":"2023-10-06T11:18:23.809338+0000","last_peered":"2023-10-06T11:18:23.809338+0000","last_clean":"2023-10-06T11:18:23.809338+0000","last_became_active":"2023-10-06T11:18:23.808949+0000","last_became_peered":"2023-10-06T11:18:23.808949+0000","last_unstale":"2023-10-06T11:18:23.809338+0000","last_undegraded":"2023-10-06T11:18:23.809338+0000","last_fullsized":"2023-10-06T11:18:23.809338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794359+0000","last_change":"2023-10-06T11:18:23.794359+0000","last_active":"2023-10-06T11:18:23.794359+0000","last_peered":"2023-10-06T11:18:23.794359+0000","last_clean":"2023-10-06T11:18:23.794359+0000","last_became_active":"2023-10-06T11:18:23.794050+0000","last_became_peered":"2023-10-06T11:18:23.794050+0000","last_unstale":"2023-10-06T11:18:23.794359+0000","last_undegraded":"2023-10-06T11:18:23.794359+0000","last_fullsized":"2023-10-06T11:18:23.794359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802662+0000","last_change":"2023-10-06T11:18:23.802662+0000","last_active":"2023-10-06T11:18:23.802662+0000","last_peered":"2023-10-06T11:18:23.802662+0000","last_clean":"2023-10-06T11:18:23.802662+0000","last_became_active":"2023-10-06T11:18:23.802233+0000","last_became_peered":"2023-10-06T11:18:23.802233+0000","last_unstale":"2023-10-06T11:18:23.802662+0000","last_undegraded":"2023-10-06T11:18:23.802662+0000","last_fullsized":"2023-10-06T11:18:23.802662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795210+0000","last_change":"2023-10-06T11:18:23.795210+0000","last_active":"2023-10-06T11:18:23.795210+0000","last_peered":"2023-10-06T11:18:23.795210+0000","last_clean":"2023-10-06T11:18:23.795210+0000","last_became_active":"2023-10-06T11:18:23.794877+0000","last_became_peered":"2023-10-06T11:18:23.794877+0000","last_unstale":"2023-10-06T11:18:23.795210+0000","last_undegraded":"2023-10-06T11:18:23.795210+0000","last_fullsized":"2023-10-06T11:18:23.795210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:16.625365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789054+0000","last_change":"2023-10-06T11:18:23.789054+0000","last_active":"2023-10-06T11:18:23.789054+0000","last_peered":"2023-10-06T11:18:23.789054+0000","last_clean":"2023-10-06T11:18:23.789054+0000","last_became_active":"2023-10-06T11:18:23.788629+0000","last_became_peered":"2023-10-06T11:18:23.788629+0000","last_unstale":"2023-10-06T11:18:23.789054+0000","last_undegraded":"2023-10-06T11:18:23.789054+0000","last_fullsized":"2023-10-06T11:18:23.789054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:37.399272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792433+0000","last_change":"2023-10-06T11:18:23.792433+0000","last_active":"2023-10-06T11:18:23.792433+0000","last_peered":"2023-10-06T11:18:23.792433+0000","last_clean":"2023-10-06T11:18:23.792433+0000","last_became_active":"2023-10-06T11:18:23.792138+0000","last_became_peered":"2023-10-06T11:18:23.792138+0000","last_unstale":"2023-10-06T11:18:23.792433+0000","last_undegraded":"2023-10-06T11:18:23.792433+0000","last_fullsized":"2023-10-06T11:18:23.792433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797785+0000","last_change":"2023-10-06T11:18:23.797785+0000","last_active":"2023-10-06T11:18:23.797785+0000","last_peered":"2023-10-06T11:18:23.797785+0000","last_clean":"2023-10-06T11:18:23.797785+0000","last_became_active":"2023-10-06T11:18:23.797415+0000","last_became_peered":"2023-10-06T11:18:23.797415+0000","last_unstale":"2023-10-06T11:18:23.797785+0000","last_undegraded":"2023-10-06T11:18:23.797785+0000","last_fullsized":"2023-10-06T11:18:23.797785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808011+0000","last_change":"2023-10-06T11:18:23.808011+0000","last_active":"2023-10-06T11:18:23.808011+0000","last_peered":"2023-10-06T11:18:23.808011+0000","last_clean":"2023-10-06T11:18:23.808011+0000","last_became_active":"2023-10-06T11:18:23.807627+0000","last_became_peered":"2023-10-06T11:18:23.807627+0000","last_unstale":"2023-10-06T11:18:23.808011+0000","last_undegraded":"2023-10-06T11:18:23.808011+0000","last_fullsized":"2023-10-06T11:18:23.808011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794460+0000","last_change":"2023-10-06T11:18:23.794460+0000","last_active":"2023-10-06T11:18:23.794460+0000","last_peered":"2023-10-06T11:18:23.794460+0000","last_clean":"2023-10-06T11:18:23.794460+0000","last_became_active":"2023-10-06T11:18:23.794145+0000","last_became_peered":"2023-10-06T11:18:23.794145+0000","last_unstale":"2023-10-06T11:18:23.794460+0000","last_undegraded":"2023-10-06T11:18:23.794460+0000","last_fullsized":"2023-10-06T11:18:23.794460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800752+0000","last_change":"2023-10-06T11:18:23.800752+0000","last_active":"2023-10-06T11:18:23.800752+0000","last_peered":"2023-10-06T11:18:23.800752+0000","last_clean":"2023-10-06T11:18:23.800752+0000","last_became_active":"2023-10-06T11:18:23.798826+0000","last_became_peered":"2023-10-06T11:18:23.798826+0000","last_unstale":"2023-10-06T11:18:23.800752+0000","last_undegraded":"2023-10-06T11:18:23.800752+0000","last_fullsized":"2023-10-06T11:18:23.800752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801213+0000","last_change":"2023-10-06T11:18:23.801213+0000","last_active":"2023-10-06T11:18:23.801213+0000","last_peered":"2023-10-06T11:18:23.801213+0000","last_clean":"2023-10-06T11:18:23.801213+0000","last_became_active":"2023-10-06T11:18:23.800123+0000","last_became_peered":"2023-10-06T11:18:23.800123+0000","last_unstale":"2023-10-06T11:18:23.801213+0000","last_undegraded":"2023-10-06T11:18:23.801213+0000","last_fullsized":"2023-10-06T11:18:23.801213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.795738+0000","last_change":"2023-10-06T11:18:23.795738+0000","last_active":"2023-10-06T11:18:23.795738+0000","last_peered":"2023-10-06T11:18:23.795738+0000","last_clean":"2023-10-06T11:18:23.795738+0000","last_became_active":"2023-10-06T11:18:23.795423+0000","last_became_peered":"2023-10-06T11:18:23.795423+0000","last_unstale":"2023-10-06T11:18:23.795738+0000","last_undegraded":"2023-10-06T11:18:23.795738+0000","last_fullsized":"2023-10-06T11:18:23.795738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794728+0000","last_change":"2023-10-06T11:18:23.794728+0000","last_active":"2023-10-06T11:18:23.794728+0000","last_peered":"2023-10-06T11:18:23.794728+0000","last_clean":"2023-10-06T11:18:23.794728+0000","last_became_active":"2023-10-06T11:18:23.794324+0000","last_became_peered":"2023-10-06T11:18:23.794324+0000","last_unstale":"2023-10-06T11:18:23.794728+0000","last_undegraded":"2023-10-06T11:18:23.794728+0000","last_fullsized":"2023-10-06T11:18:23.794728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809631+0000","last_change":"2023-10-06T11:18:23.809631+0000","last_active":"2023-10-06T11:18:23.809631+0000","last_peered":"2023-10-06T11:18:23.809631+0000","last_clean":"2023-10-06T11:18:23.809631+0000","last_became_active":"2023-10-06T11:18:23.806649+0000","last_became_peered":"2023-10-06T11:18:23.806649+0000","last_unstale":"2023-10-06T11:18:23.809631+0000","last_undegraded":"2023-10-06T11:18:23.809631+0000","last_fullsized":"2023-10-06T11:18:23.809631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800427+0000","last_change":"2023-10-06T11:18:23.800427+0000","last_active":"2023-10-06T11:18:23.800427+0000","last_peered":"2023-10-06T11:18:23.800427+0000","last_clean":"2023-10-06T11:18:23.800427+0000","last_became_active":"2023-10-06T11:18:23.799956+0000","last_became_peered":"2023-10-06T11:18:23.799956+0000","last_unstale":"2023-10-06T11:18:23.800427+0000","last_undegraded":"2023-10-06T11:18:23.800427+0000","last_fullsized":"2023-10-06T11:18:23.800427+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793233+0000","last_change":"2023-10-06T11:18:23.793233+0000","last_active":"2023-10-06T11:18:23.793233+0000","last_peered":"2023-10-06T11:18:23.793233+0000","last_clean":"2023-10-06T11:18:23.793233+0000","last_became_active":"2023-10-06T11:18:23.792917+0000","last_became_peered":"2023-10-06T11:18:23.792917+0000","last_unstale":"2023-10-06T11:18:23.793233+0000","last_undegraded":"2023-10-06T11:18:23.793233+0000","last_fullsized":"2023-10-06T11:18:23.793233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:01:00.993266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811339+0000","last_change":"2023-10-06T11:18:23.811339+0000","last_active":"2023-10-06T11:18:23.811339+0000","last_peered":"2023-10-06T11:18:23.811339+0000","last_clean":"2023-10-06T11:18:23.811339+0000","last_became_active":"2023-10-06T11:18:23.810982+0000","last_became_peered":"2023-10-06T11:18:23.810982+0000","last_unstale":"2023-10-06T11:18:23.811339+0000","last_undegraded":"2023-10-06T11:18:23.811339+0000","last_fullsized":"2023-10-06T11:18:23.811339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796079+0000","last_change":"2023-10-06T11:18:23.796079+0000","last_active":"2023-10-06T11:18:23.796079+0000","last_peered":"2023-10-06T11:18:23.796079+0000","last_clean":"2023-10-06T11:18:23.796079+0000","last_became_active":"2023-10-06T11:18:23.795767+0000","last_became_peered":"2023-10-06T11:18:23.795767+0000","last_unstale":"2023-10-06T11:18:23.796079+0000","last_undegraded":"2023-10-06T11:18:23.796079+0000","last_fullsized":"2023-10-06T11:18:23.796079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:24.987705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811468+0000","last_change":"2023-10-06T11:18:23.811468+0000","last_active":"2023-10-06T11:18:23.811468+0000","last_peered":"2023-10-06T11:18:23.811468+0000","last_clean":"2023-10-06T11:18:23.811468+0000","last_became_active":"2023-10-06T11:18:23.811047+0000","last_became_peered":"2023-10-06T11:18:23.811047+0000","last_unstale":"2023-10-06T11:18:23.811468+0000","last_undegraded":"2023-10-06T11:18:23.811468+0000","last_fullsized":"2023-10-06T11:18:23.811468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080611+0000","last_change":"2023-10-06T11:18:22.785631+0000","last_active":"2023-10-06T11:18:24.080611+0000","last_peered":"2023-10-06T11:18:24.080611+0000","last_clean":"2023-10-06T11:18:24.080611+0000","last_became_active":"2023-10-06T11:18:22.785163+0000","last_became_peered":"2023-10-06T11:18:22.785163+0000","last_unstale":"2023-10-06T11:18:24.080611+0000","last_undegraded":"2023-10-06T11:18:24.080611+0000","last_fullsized":"2023-10-06T11:18:24.080611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796325+0000","last_change":"2023-10-06T11:18:23.796325+0000","last_active":"2023-10-06T11:18:23.796325+0000","last_peered":"2023-10-06T11:18:23.796325+0000","last_clean":"2023-10-06T11:18:23.796325+0000","last_became_active":"2023-10-06T11:18:23.795981+0000","last_became_peered":"2023-10-06T11:18:23.795981+0000","last_unstale":"2023-10-06T11:18:23.796325+0000","last_undegraded":"2023-10-06T11:18:23.796325+0000","last_fullsized":"2023-10-06T11:18:23.796325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.775454+0000","last_change":"2023-10-06T11:18:22.771501+0000","last_active":"2023-10-06T11:18:22.775454+0000","last_peered":"2023-10-06T11:18:22.775454+0000","last_clean":"2023-10-06T11:18:22.775454+0000","last_became_active":"2023-10-06T11:18:22.771165+0000","last_became_peered":"2023-10-06T11:18:22.771165+0000","last_unstale":"2023-10-06T11:18:22.775454+0000","last_undegraded":"2023-10-06T11:18:22.775454+0000","last_fullsized":"2023-10-06T11:18:22.775454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801731+0000","last_change":"2023-10-06T11:18:23.801731+0000","last_active":"2023-10-06T11:18:23.801731+0000","last_peered":"2023-10-06T11:18:23.801731+0000","last_clean":"2023-10-06T11:18:23.801731+0000","last_became_active":"2023-10-06T11:18:23.801224+0000","last_became_peered":"2023-10-06T11:18:23.801224+0000","last_unstale":"2023-10-06T11:18:23.801731+0000","last_undegraded":"2023-10-06T11:18:23.801731+0000","last_fullsized":"2023-10-06T11:18:23.801731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786601+0000","last_change":"2023-10-06T11:18:22.783842+0000","last_active":"2023-10-06T11:18:22.786601+0000","last_peered":"2023-10-06T11:18:22.786601+0000","last_clean":"2023-10-06T11:18:22.786601+0000","last_became_active":"2023-10-06T11:18:22.783033+0000","last_became_peered":"2023-10-06T11:18:22.783033+0000","last_unstale":"2023-10-06T11:18:22.786601+0000","last_undegraded":"2023-10-06T11:18:22.786601+0000","last_fullsized":"2023-10-06T11:18:22.786601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.793261+0000","last_change":"2023-10-06T11:18:23.793261+0000","last_active":"2023-10-06T11:18:23.793261+0000","last_peered":"2023-10-06T11:18:23.793261+0000","last_clean":"2023-10-06T11:18:23.793261+0000","last_became_active":"2023-10-06T11:18:23.792951+0000","last_became_peered":"2023-10-06T11:18:23.792951+0000","last_unstale":"2023-10-06T11:18:23.793261+0000","last_undegraded":"2023-10-06T11:18:23.793261+0000","last_fullsized":"2023-10-06T11:18:23.793261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.082435+0000","last_change":"2023-10-06T11:18:22.773438+0000","last_active":"2023-10-06T11:18:24.082435+0000","last_peered":"2023-10-06T11:18:24.082435+0000","last_clean":"2023-10-06T11:18:24.082435+0000","last_became_active":"2023-10-06T11:18:22.773047+0000","last_became_peered":"2023-10-06T11:18:22.773047+0000","last_unstale":"2023-10-06T11:18:24.082435+0000","last_undegraded":"2023-10-06T11:18:24.082435+0000","last_fullsized":"2023-10-06T11:18:24.082435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788147+0000","last_change":"2023-10-06T11:18:22.783414+0000","last_active":"2023-10-06T11:18:23.788147+0000","last_peered":"2023-10-06T11:18:23.788147+0000","last_clean":"2023-10-06T11:18:23.788147+0000","last_became_active":"2023-10-06T11:18:22.783068+0000","last_became_peered":"2023-10-06T11:18:22.783068+0000","last_unstale":"2023-10-06T11:18:23.788147+0000","last_undegraded":"2023-10-06T11:18:23.788147+0000","last_fullsized":"2023-10-06T11:18:23.788147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792041+0000","last_change":"2023-10-06T11:18:23.792041+0000","last_active":"2023-10-06T11:18:23.792041+0000","last_peered":"2023-10-06T11:18:23.792041+0000","last_clean":"2023-10-06T11:18:23.792041+0000","last_became_active":"2023-10-06T11:18:23.791770+0000","last_became_peered":"2023-10-06T11:18:23.791770+0000","last_unstale":"2023-10-06T11:18:23.792041+0000","last_undegraded":"2023-10-06T11:18:23.792041+0000","last_fullsized":"2023-10-06T11:18:23.792041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080529+0000","last_change":"2023-10-06T11:18:22.783471+0000","last_active":"2023-10-06T11:18:24.080529+0000","last_peered":"2023-10-06T11:18:24.080529+0000","last_clean":"2023-10-06T11:18:24.080529+0000","last_became_active":"2023-10-06T11:18:22.783150+0000","last_became_peered":"2023-10-06T11:18:22.783150+0000","last_unstale":"2023-10-06T11:18:24.080529+0000","last_undegraded":"2023-10-06T11:18:24.080529+0000","last_fullsized":"2023-10-06T11:18:24.080529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:57.801692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797336+0000","last_change":"2023-10-06T11:18:23.797336+0000","last_active":"2023-10-06T11:18:23.797336+0000","last_peered":"2023-10-06T11:18:23.797336+0000","last_clean":"2023-10-06T11:18:23.797336+0000","last_became_active":"2023-10-06T11:18:23.797002+0000","last_became_peered":"2023-10-06T11:18:23.797002+0000","last_unstale":"2023-10-06T11:18:23.797336+0000","last_undegraded":"2023-10-06T11:18:23.797336+0000","last_fullsized":"2023-10-06T11:18:23.797336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:58.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786077+0000","last_change":"2023-10-06T11:18:22.777073+0000","last_active":"2023-10-06T11:18:23.786077+0000","last_peered":"2023-10-06T11:18:23.786077+0000","last_clean":"2023-10-06T11:18:23.786077+0000","last_became_active":"2023-10-06T11:18:22.776784+0000","last_became_peered":"2023-10-06T11:18:22.776784+0000","last_unstale":"2023-10-06T11:18:23.786077+0000","last_undegraded":"2023-10-06T11:18:23.786077+0000","last_fullsized":"2023-10-06T11:18:23.786077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.807687+0000","last_change":"2023-10-06T11:18:23.807687+0000","last_active":"2023-10-06T11:18:23.807687+0000","last_peered":"2023-10-06T11:18:23.807687+0000","last_clean":"2023-10-06T11:18:23.807687+0000","last_became_active":"2023-10-06T11:18:23.807468+0000","last_became_peered":"2023-10-06T11:18:23.807468+0000","last_unstale":"2023-10-06T11:18:23.807687+0000","last_undegraded":"2023-10-06T11:18:23.807687+0000","last_fullsized":"2023-10-06T11:18:23.807687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:11.773581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.775336+0000","last_change":"2023-10-06T11:18:22.770019+0000","last_active":"2023-10-06T11:18:22.775336+0000","last_peered":"2023-10-06T11:18:22.775336+0000","last_clean":"2023-10-06T11:18:22.775336+0000","last_became_active":"2023-10-06T11:18:22.769681+0000","last_became_peered":"2023-10-06T11:18:22.769681+0000","last_unstale":"2023-10-06T11:18:22.775336+0000","last_undegraded":"2023-10-06T11:18:22.775336+0000","last_fullsized":"2023-10-06T11:18:22.775336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804489+0000","last_change":"2023-10-06T11:18:23.804489+0000","last_active":"2023-10-06T11:18:23.804489+0000","last_peered":"2023-10-06T11:18:23.804489+0000","last_clean":"2023-10-06T11:18:23.804489+0000","last_became_active":"2023-10-06T11:18:23.804154+0000","last_became_peered":"2023-10-06T11:18:23.804154+0000","last_unstale":"2023-10-06T11:18:23.804489+0000","last_undegraded":"2023-10-06T11:18:23.804489+0000","last_fullsized":"2023-10-06T11:18:23.804489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.078639+0000","last_change":"2023-10-06T11:18:22.783172+0000","last_active":"2023-10-06T11:18:24.078639+0000","last_peered":"2023-10-06T11:18:24.078639+0000","last_clean":"2023-10-06T11:18:24.078639+0000","last_became_active":"2023-10-06T11:18:22.782848+0000","last_became_peered":"2023-10-06T11:18:22.782848+0000","last_unstale":"2023-10-06T11:18:24.078639+0000","last_undegraded":"2023-10-06T11:18:24.078639+0000","last_fullsized":"2023-10-06T11:18:24.078639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799401+0000","last_change":"2023-10-06T11:18:23.799401+0000","last_active":"2023-10-06T11:18:23.799401+0000","last_peered":"2023-10-06T11:18:23.799401+0000","last_clean":"2023-10-06T11:18:23.799401+0000","last_became_active":"2023-10-06T11:18:23.799110+0000","last_became_peered":"2023-10-06T11:18:23.799110+0000","last_unstale":"2023-10-06T11:18:23.799401+0000","last_undegraded":"2023-10-06T11:18:23.799401+0000","last_fullsized":"2023-10-06T11:18:23.799401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.789863+0000","last_change":"2023-10-06T11:18:22.788211+0000","last_active":"2023-10-06T11:18:22.789863+0000","last_peered":"2023-10-06T11:18:22.789863+0000","last_clean":"2023-10-06T11:18:22.789863+0000","last_became_active":"2023-10-06T11:18:22.787835+0000","last_became_peered":"2023-10-06T11:18:22.787835+0000","last_unstale":"2023-10-06T11:18:22.789863+0000","last_undegraded":"2023-10-06T11:18:22.789863+0000","last_fullsized":"2023-10-06T11:18:22.789863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794251+0000","last_change":"2023-10-06T11:18:23.794251+0000","last_active":"2023-10-06T11:18:23.794251+0000","last_peered":"2023-10-06T11:18:23.794251+0000","last_clean":"2023-10-06T11:18:23.794251+0000","last_became_active":"2023-10-06T11:18:23.793840+0000","last_became_peered":"2023-10-06T11:18:23.793840+0000","last_unstale":"2023-10-06T11:18:23.794251+0000","last_undegraded":"2023-10-06T11:18:23.794251+0000","last_fullsized":"2023-10-06T11:18:23.794251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:26.606422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.788730+0000","last_change":"2023-10-06T11:18:22.786197+0000","last_active":"2023-10-06T11:18:22.788730+0000","last_peered":"2023-10-06T11:18:22.788730+0000","last_clean":"2023-10-06T11:18:22.788730+0000","last_became_active":"2023-10-06T11:18:22.785533+0000","last_became_peered":"2023-10-06T11:18:22.785533+0000","last_unstale":"2023-10-06T11:18:22.788730+0000","last_undegraded":"2023-10-06T11:18:22.788730+0000","last_fullsized":"2023-10-06T11:18:22.788730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:46.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792349+0000","last_change":"2023-10-06T11:18:23.792349+0000","last_active":"2023-10-06T11:18:23.792349+0000","last_peered":"2023-10-06T11:18:23.792349+0000","last_clean":"2023-10-06T11:18:23.792349+0000","last_became_active":"2023-10-06T11:18:23.792058+0000","last_became_peered":"2023-10-06T11:18:23.792058+0000","last_unstale":"2023-10-06T11:18:23.792349+0000","last_undegraded":"2023-10-06T11:18:23.792349+0000","last_fullsized":"2023-10-06T11:18:23.792349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.782703+0000","last_change":"2023-10-06T11:18:22.778447+0000","last_active":"2023-10-06T11:18:22.782703+0000","last_peered":"2023-10-06T11:18:22.782703+0000","last_clean":"2023-10-06T11:18:22.782703+0000","last_became_active":"2023-10-06T11:18:22.778005+0000","last_became_peered":"2023-10-06T11:18:22.778005+0000","last_unstale":"2023-10-06T11:18:22.782703+0000","last_undegraded":"2023-10-06T11:18:22.782703+0000","last_fullsized":"2023-10-06T11:18:22.782703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.805117+0000","last_change":"2023-10-06T11:18:23.805117+0000","last_active":"2023-10-06T11:18:23.805117+0000","last_peered":"2023-10-06T11:18:23.805117+0000","last_clean":"2023-10-06T11:18:23.805117+0000","last_became_active":"2023-10-06T11:18:23.804771+0000","last_became_peered":"2023-10-06T11:18:23.804771+0000","last_unstale":"2023-10-06T11:18:23.805117+0000","last_undegraded":"2023-10-06T11:18:23.805117+0000","last_fullsized":"2023-10-06T11:18:23.805117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.766166+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:18:22.766166+0000","last_peered":"2023-10-06T11:18:22.766166+0000","last_clean":"2023-10-06T11:18:22.766166+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:18:22.766166+0000","last_undegraded":"2023-10-06T11:18:22.766166+0000","last_fullsized":"2023-10-06T11:18:22.766166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.764968+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:18:22.764968+0000","last_peered":"2023-10-06T11:18:22.764968+0000","last_clean":"2023-10-06T11:18:22.764968+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:18:22.764968+0000","last_undegraded":"2023-10-06T11:18:22.764968+0000","last_fullsized":"2023-10-06T11:18:22.764968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.779389+0000","last_change":"2023-10-06T11:18:22.774869+0000","last_active":"2023-10-06T11:18:22.779389+0000","last_peered":"2023-10-06T11:18:22.779389+0000","last_clean":"2023-10-06T11:18:22.779389+0000","last_became_active":"2023-10-06T11:18:22.773053+0000","last_became_peered":"2023-10-06T11:18:22.773053+0000","last_unstale":"2023-10-06T11:18:22.779389+0000","last_undegraded":"2023-10-06T11:18:22.779389+0000","last_fullsized":"2023-10-06T11:18:22.779389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798289+0000","last_change":"2023-10-06T11:18:23.798289+0000","last_active":"2023-10-06T11:18:23.798289+0000","last_peered":"2023-10-06T11:18:23.798289+0000","last_clean":"2023-10-06T11:18:23.798289+0000","last_became_active":"2023-10-06T11:18:23.797843+0000","last_became_peered":"2023-10-06T11:18:23.797843+0000","last_unstale":"2023-10-06T11:18:23.798289+0000","last_undegraded":"2023-10-06T11:18:23.798289+0000","last_fullsized":"2023-10-06T11:18:23.798289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.766066+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:18:22.766066+0000","last_peered":"2023-10-06T11:18:22.766066+0000","last_clean":"2023-10-06T11:18:22.766066+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:18:22.766066+0000","last_undegraded":"2023-10-06T11:18:22.766066+0000","last_fullsized":"2023-10-06T11:18:22.766066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786478+0000","last_change":"2023-10-06T11:18:22.784973+0000","last_active":"2023-10-06T11:18:22.786478+0000","last_peered":"2023-10-06T11:18:22.786478+0000","last_clean":"2023-10-06T11:18:22.786478+0000","last_became_active":"2023-10-06T11:18:22.784647+0000","last_became_peered":"2023-10-06T11:18:22.784647+0000","last_unstale":"2023-10-06T11:18:22.786478+0000","last_undegraded":"2023-10-06T11:18:22.786478+0000","last_fullsized":"2023-10-06T11:18:22.786478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808610+0000","last_change":"2023-10-06T11:18:23.808610+0000","last_active":"2023-10-06T11:18:23.808610+0000","last_peered":"2023-10-06T11:18:23.808610+0000","last_clean":"2023-10-06T11:18:23.808610+0000","last_became_active":"2023-10-06T11:18:23.808298+0000","last_became_peered":"2023-10-06T11:18:23.808298+0000","last_unstale":"2023-10-06T11:18:23.808610+0000","last_undegraded":"2023-10-06T11:18:23.808610+0000","last_fullsized":"2023-10-06T11:18:23.808610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.774027+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:18:22.774027+0000","last_peered":"2023-10-06T11:18:22.774027+0000","last_clean":"2023-10-06T11:18:22.774027+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:18:22.774027+0000","last_undegraded":"2023-10-06T11:18:22.774027+0000","last_fullsized":"2023-10-06T11:18:22.774027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786502+0000","last_change":"2023-10-06T11:18:22.782337+0000","last_active":"2023-10-06T11:18:22.786502+0000","last_peered":"2023-10-06T11:18:22.786502+0000","last_clean":"2023-10-06T11:18:22.786502+0000","last_became_active":"2023-10-06T11:18:22.780763+0000","last_became_peered":"2023-10-06T11:18:22.780763+0000","last_unstale":"2023-10-06T11:18:22.786502+0000","last_undegraded":"2023-10-06T11:18:22.786502+0000","last_fullsized":"2023-10-06T11:18:22.786502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798843+0000","last_change":"2023-10-06T11:18:23.798843+0000","last_active":"2023-10-06T11:18:23.798843+0000","last_peered":"2023-10-06T11:18:23.798843+0000","last_clean":"2023-10-06T11:18:23.798843+0000","last_became_active":"2023-10-06T11:18:23.798591+0000","last_became_peered":"2023-10-06T11:18:23.798591+0000","last_unstale":"2023-10-06T11:18:23.798843+0000","last_undegraded":"2023-10-06T11:18:23.798843+0000","last_fullsized":"2023-10-06T11:18:23.798843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.765052+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:18:22.765052+0000","last_peered":"2023-10-06T11:18:22.765052+0000","last_clean":"2023-10-06T11:18:22.765052+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:18:22.765052+0000","last_undegraded":"2023-10-06T11:18:22.765052+0000","last_fullsized":"2023-10-06T11:18:22.765052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.787357+0000","last_change":"2023-10-06T11:18:22.781890+0000","last_active":"2023-10-06T11:18:22.787357+0000","last_peered":"2023-10-06T11:18:22.787357+0000","last_clean":"2023-10-06T11:18:22.787357+0000","last_became_active":"2023-10-06T11:18:22.781589+0000","last_became_peered":"2023-10-06T11:18:22.781589+0000","last_unstale":"2023-10-06T11:18:22.787357+0000","last_undegraded":"2023-10-06T11:18:22.787357+0000","last_fullsized":"2023-10-06T11:18:22.787357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803070+0000","last_change":"2023-10-06T11:18:23.803070+0000","last_active":"2023-10-06T11:18:23.803070+0000","last_peered":"2023-10-06T11:18:23.803070+0000","last_clean":"2023-10-06T11:18:23.803070+0000","last_became_active":"2023-10-06T11:18:23.799605+0000","last_became_peered":"2023-10-06T11:18:23.799605+0000","last_unstale":"2023-10-06T11:18:23.803070+0000","last_undegraded":"2023-10-06T11:18:23.803070+0000","last_fullsized":"2023-10-06T11:18:23.803070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.776258+0000","last_change":"2023-10-06T11:18:22.770766+0000","last_active":"2023-10-06T11:18:22.776258+0000","last_peered":"2023-10-06T11:18:22.776258+0000","last_clean":"2023-10-06T11:18:22.776258+0000","last_became_active":"2023-10-06T11:18:22.770433+0000","last_became_peered":"2023-10-06T11:18:22.770433+0000","last_unstale":"2023-10-06T11:18:22.776258+0000","last_undegraded":"2023-10-06T11:18:22.776258+0000","last_fullsized":"2023-10-06T11:18:22.776258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.750418+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:18:22.750418+0000","last_peered":"2023-10-06T11:18:22.750418+0000","last_clean":"2023-10-06T11:18:22.750418+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:18:22.750418+0000","last_undegraded":"2023-10-06T11:18:22.750418+0000","last_fullsized":"2023-10-06T11:18:22.750418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804833+0000","last_change":"2023-10-06T11:18:23.804833+0000","last_active":"2023-10-06T11:18:23.804833+0000","last_peered":"2023-10-06T11:18:23.804833+0000","last_clean":"2023-10-06T11:18:23.804833+0000","last_became_active":"2023-10-06T11:18:23.804036+0000","last_became_peered":"2023-10-06T11:18:23.804036+0000","last_unstale":"2023-10-06T11:18:23.804833+0000","last_undegraded":"2023-10-06T11:18:23.804833+0000","last_fullsized":"2023-10-06T11:18:23.804833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.786609+0000","last_change":"2023-10-06T11:18:22.783968+0000","last_active":"2023-10-06T11:18:22.786609+0000","last_peered":"2023-10-06T11:18:22.786609+0000","last_clean":"2023-10-06T11:18:22.786609+0000","last_became_active":"2023-10-06T11:18:22.783402+0000","last_became_peered":"2023-10-06T11:18:22.783402+0000","last_unstale":"2023-10-06T11:18:22.786609+0000","last_undegraded":"2023-10-06T11:18:22.786609+0000","last_fullsized":"2023-10-06T11:18:22.786609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.767007+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:18:22.767007+0000","last_peered":"2023-10-06T11:18:22.767007+0000","last_clean":"2023-10-06T11:18:22.767007+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:18:22.767007+0000","last_undegraded":"2023-10-06T11:18:22.767007+0000","last_fullsized":"2023-10-06T11:18:22.767007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792986+0000","last_change":"2023-10-06T11:18:23.792986+0000","last_active":"2023-10-06T11:18:23.792986+0000","last_peered":"2023-10-06T11:18:23.792986+0000","last_clean":"2023-10-06T11:18:23.792986+0000","last_became_active":"2023-10-06T11:18:23.792504+0000","last_became_peered":"2023-10-06T11:18:23.792504+0000","last_unstale":"2023-10-06T11:18:23.792986+0000","last_undegraded":"2023-10-06T11:18:23.792986+0000","last_fullsized":"2023-10-06T11:18:23.792986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079144+0000","last_change":"2023-10-06T11:18:22.773444+0000","last_active":"2023-10-06T11:18:24.079144+0000","last_peered":"2023-10-06T11:18:24.079144+0000","last_clean":"2023-10-06T11:18:24.079144+0000","last_became_active":"2023-10-06T11:18:22.773056+0000","last_became_peered":"2023-10-06T11:18:22.773056+0000","last_unstale":"2023-10-06T11:18:24.079144+0000","last_undegraded":"2023-10-06T11:18:24.079144+0000","last_fullsized":"2023-10-06T11:18:24.079144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.750442+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:18:22.750442+0000","last_peered":"2023-10-06T11:18:22.750442+0000","last_clean":"2023-10-06T11:18:22.750442+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:18:22.750442+0000","last_undegraded":"2023-10-06T11:18:22.750442+0000","last_fullsized":"2023-10-06T11:18:22.750442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799064+0000","last_change":"2023-10-06T11:18:23.799064+0000","last_active":"2023-10-06T11:18:23.799064+0000","last_peered":"2023-10-06T11:18:23.799064+0000","last_clean":"2023-10-06T11:18:23.799064+0000","last_became_active":"2023-10-06T11:18:23.798759+0000","last_became_peered":"2023-10-06T11:18:23.798759+0000","last_unstale":"2023-10-06T11:18:23.799064+0000","last_undegraded":"2023-10-06T11:18:23.799064+0000","last_fullsized":"2023-10-06T11:18:23.799064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.785242+0000","last_change":"2023-10-06T11:18:22.781854+0000","last_active":"2023-10-06T11:18:22.785242+0000","last_peered":"2023-10-06T11:18:22.785242+0000","last_clean":"2023-10-06T11:18:22.785242+0000","last_became_active":"2023-10-06T11:18:22.781438+0000","last_became_peered":"2023-10-06T11:18:22.781438+0000","last_unstale":"2023-10-06T11:18:22.785242+0000","last_undegraded":"2023-10-06T11:18:22.785242+0000","last_fullsized":"2023-10-06T11:18:22.785242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.765678+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:18:22.765678+0000","last_peered":"2023-10-06T11:18:22.765678+0000","last_clean":"2023-10-06T11:18:22.765678+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:18:22.765678+0000","last_undegraded":"2023-10-06T11:18:22.765678+0000","last_fullsized":"2023-10-06T11:18:22.765678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786622+0000","last_change":"2023-10-06T11:18:23.786622+0000","last_active":"2023-10-06T11:18:23.786622+0000","last_peered":"2023-10-06T11:18:23.786622+0000","last_clean":"2023-10-06T11:18:23.786622+0000","last_became_active":"2023-10-06T11:18:23.786299+0000","last_became_peered":"2023-10-06T11:18:23.786299+0000","last_unstale":"2023-10-06T11:18:23.786622+0000","last_undegraded":"2023-10-06T11:18:23.786622+0000","last_fullsized":"2023-10-06T11:18:23.786622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792771+0000","last_change":"2023-10-06T11:18:23.792771+0000","last_active":"2023-10-06T11:18:23.792771+0000","last_peered":"2023-10-06T11:18:23.792771+0000","last_clean":"2023-10-06T11:18:23.792771+0000","last_became_active":"2023-10-06T11:18:23.787870+0000","last_became_peered":"2023-10-06T11:18:23.787870+0000","last_unstale":"2023-10-06T11:18:23.792771+0000","last_undegraded":"2023-10-06T11:18:23.792771+0000","last_fullsized":"2023-10-06T11:18:23.792771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081899+0000","last_change":"2023-10-06T11:18:22.781178+0000","last_active":"2023-10-06T11:18:24.081899+0000","last_peered":"2023-10-06T11:18:24.081899+0000","last_clean":"2023-10-06T11:18:24.081899+0000","last_became_active":"2023-10-06T11:18:22.778268+0000","last_became_peered":"2023-10-06T11:18:22.778268+0000","last_unstale":"2023-10-06T11:18:24.081899+0000","last_undegraded":"2023-10-06T11:18:24.081899+0000","last_fullsized":"2023-10-06T11:18:24.081899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800123+0000","last_change":"2023-10-06T11:18:23.800123+0000","last_active":"2023-10-06T11:18:23.800123+0000","last_peered":"2023-10-06T11:18:23.800123+0000","last_clean":"2023-10-06T11:18:23.800123+0000","last_became_active":"2023-10-06T11:18:23.799794+0000","last_became_peered":"2023-10-06T11:18:23.799794+0000","last_unstale":"2023-10-06T11:18:23.800123+0000","last_undegraded":"2023-10-06T11:18:23.800123+0000","last_fullsized":"2023-10-06T11:18:23.800123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.078042+0000","last_change":"2023-10-06T11:18:22.782320+0000","last_active":"2023-10-06T11:18:24.078042+0000","last_peered":"2023-10-06T11:18:24.078042+0000","last_clean":"2023-10-06T11:18:24.078042+0000","last_became_active":"2023-10-06T11:18:22.782025+0000","last_became_peered":"2023-10-06T11:18:22.782025+0000","last_unstale":"2023-10-06T11:18:24.078042+0000","last_undegraded":"2023-10-06T11:18:24.078042+0000","last_fullsized":"2023-10-06T11:18:24.078042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.776433+0000","last_change":"2023-10-06T11:18:22.773270+0000","last_active":"2023-10-06T11:18:22.776433+0000","last_peered":"2023-10-06T11:18:22.776433+0000","last_clean":"2023-10-06T11:18:22.776433+0000","last_became_active":"2023-10-06T11:18:22.772211+0000","last_became_peered":"2023-10-06T11:18:22.772211+0000","last_unstale":"2023-10-06T11:18:22.776433+0000","last_undegraded":"2023-10-06T11:18:22.776433+0000","last_fullsized":"2023-10-06T11:18:22.776433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794332+0000","last_change":"2023-10-06T11:18:23.794332+0000","last_active":"2023-10-06T11:18:23.794332+0000","last_peered":"2023-10-06T11:18:23.794332+0000","last_clean":"2023-10-06T11:18:23.794332+0000","last_became_active":"2023-10-06T11:18:23.793993+0000","last_became_peered":"2023-10-06T11:18:23.793993+0000","last_unstale":"2023-10-06T11:18:23.794332+0000","last_undegraded":"2023-10-06T11:18:23.794332+0000","last_fullsized":"2023-10-06T11:18:23.794332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.780979+0000","last_change":"2023-10-06T11:18:22.774828+0000","last_active":"2023-10-06T11:18:22.780979+0000","last_peered":"2023-10-06T11:18:22.780979+0000","last_clean":"2023-10-06T11:18:22.780979+0000","last_became_active":"2023-10-06T11:18:22.774440+0000","last_became_peered":"2023-10-06T11:18:22.774440+0000","last_unstale":"2023-10-06T11:18:22.780979+0000","last_undegraded":"2023-10-06T11:18:22.780979+0000","last_fullsized":"2023-10-06T11:18:22.780979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804188+0000","last_change":"2023-10-06T11:18:23.804188+0000","last_active":"2023-10-06T11:18:23.804188+0000","last_peered":"2023-10-06T11:18:23.804188+0000","last_clean":"2023-10-06T11:18:23.804188+0000","last_became_active":"2023-10-06T11:18:23.803863+0000","last_became_peered":"2023-10-06T11:18:23.803863+0000","last_unstale":"2023-10-06T11:18:23.804188+0000","last_undegraded":"2023-10-06T11:18:23.804188+0000","last_fullsized":"2023-10-06T11:18:23.804188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789620+0000","last_change":"2023-10-06T11:18:23.789620+0000","last_active":"2023-10-06T11:18:23.789620+0000","last_peered":"2023-10-06T11:18:23.789620+0000","last_clean":"2023-10-06T11:18:23.789620+0000","last_became_active":"2023-10-06T11:18:23.789349+0000","last_became_peered":"2023-10-06T11:18:23.789349+0000","last_unstale":"2023-10-06T11:18:23.789620+0000","last_undegraded":"2023-10-06T11:18:23.789620+0000","last_fullsized":"2023-10-06T11:18:23.789620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.782986+0000","last_change":"2023-10-06T11:18:22.781828+0000","last_active":"2023-10-06T11:18:22.782986+0000","last_peered":"2023-10-06T11:18:22.782986+0000","last_clean":"2023-10-06T11:18:22.782986+0000","last_became_active":"2023-10-06T11:18:22.781529+0000","last_became_peered":"2023-10-06T11:18:22.781529+0000","last_unstale":"2023-10-06T11:18:22.782986+0000","last_undegraded":"2023-10-06T11:18:22.782986+0000","last_fullsized":"2023-10-06T11:18:22.782986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792292+0000","last_change":"2023-10-06T11:18:23.792292+0000","last_active":"2023-10-06T11:18:23.792292+0000","last_peered":"2023-10-06T11:18:23.792292+0000","last_clean":"2023-10-06T11:18:23.792292+0000","last_became_active":"2023-10-06T11:18:23.791847+0000","last_became_peered":"2023-10-06T11:18:23.791847+0000","last_unstale":"2023-10-06T11:18:23.792292+0000","last_undegraded":"2023-10-06T11:18:23.792292+0000","last_fullsized":"2023-10-06T11:18:23.792292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:18:22.784824+0000","last_change":"2023-10-06T11:18:22.774677+0000","last_active":"2023-10-06T11:18:22.784824+0000","last_peered":"2023-10-06T11:18:22.784824+0000","last_clean":"2023-10-06T11:18:22.784824+0000","last_became_active":"2023-10-06T11:18:22.772746+0000","last_became_peered":"2023-10-06T11:18:22.772746+0000","last_unstale":"2023-10-06T11:18:22.784824+0000","last_undegraded":"2023-10-06T11:18:22.784824+0000","last_fullsized":"2023-10-06T11:18:22.784824+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806392+0000","last_change":"2023-10-06T11:18:23.806392+0000","last_active":"2023-10-06T11:18:23.806392+0000","last_peered":"2023-10-06T11:18:23.806392+0000","last_clean":"2023-10-06T11:18:23.806392+0000","last_became_active":"2023-10-06T11:18:23.806086+0000","last_became_peered":"2023-10-06T11:18:23.806086+0000","last_unstale":"2023-10-06T11:18:23.806392+0000","last_undegraded":"2023-10-06T11:18:23.806392+0000","last_fullsized":"2023-10-06T11:18:23.806392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780727+0000","last_change":"2023-10-06T11:18:22.786752+0000","last_active":"2023-10-06T11:18:23.780727+0000","last_peered":"2023-10-06T11:18:23.780727+0000","last_clean":"2023-10-06T11:18:23.780727+0000","last_became_active":"2023-10-06T11:18:22.784966+0000","last_became_peered":"2023-10-06T11:18:22.784966+0000","last_unstale":"2023-10-06T11:18:23.780727+0000","last_undegraded":"2023-10-06T11:18:23.780727+0000","last_fullsized":"2023-10-06T11:18:23.780727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811011+0000","last_change":"2023-10-06T11:18:23.811011+0000","last_active":"2023-10-06T11:18:23.811011+0000","last_peered":"2023-10-06T11:18:23.811011+0000","last_clean":"2023-10-06T11:18:23.811011+0000","last_became_active":"2023-10-06T11:18:23.810563+0000","last_became_peered":"2023-10-06T11:18:23.810563+0000","last_unstale":"2023-10-06T11:18:23.811011+0000","last_undegraded":"2023-10-06T11:18:23.811011+0000","last_fullsized":"2023-10-06T11:18:23.811011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:06.048708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787200+0000","last_change":"2023-10-06T11:18:22.787138+0000","last_active":"2023-10-06T11:18:23.787200+0000","last_peered":"2023-10-06T11:18:23.787200+0000","last_clean":"2023-10-06T11:18:23.787200+0000","last_became_active":"2023-10-06T11:18:22.786799+0000","last_became_peered":"2023-10-06T11:18:22.786799+0000","last_unstale":"2023-10-06T11:18:23.787200+0000","last_undegraded":"2023-10-06T11:18:23.787200+0000","last_fullsized":"2023-10-06T11:18:23.787200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797231+0000","last_change":"2023-10-06T11:18:23.797231+0000","last_active":"2023-10-06T11:18:23.797231+0000","last_peered":"2023-10-06T11:18:23.797231+0000","last_clean":"2023-10-06T11:18:23.797231+0000","last_became_active":"2023-10-06T11:18:23.796942+0000","last_became_peered":"2023-10-06T11:18:23.796942+0000","last_unstale":"2023-10-06T11:18:23.797231+0000","last_undegraded":"2023-10-06T11:18:23.797231+0000","last_fullsized":"2023-10-06T11:18:23.797231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079862+0000","last_change":"2023-10-06T11:18:22.784448+0000","last_active":"2023-10-06T11:18:24.079862+0000","last_peered":"2023-10-06T11:18:24.079862+0000","last_clean":"2023-10-06T11:18:24.079862+0000","last_became_active":"2023-10-06T11:18:22.784131+0000","last_became_peered":"2023-10-06T11:18:22.784131+0000","last_unstale":"2023-10-06T11:18:24.079862+0000","last_undegraded":"2023-10-06T11:18:24.079862+0000","last_fullsized":"2023-10-06T11:18:24.079862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.796243+0000","last_change":"2023-10-06T11:18:23.796243+0000","last_active":"2023-10-06T11:18:23.796243+0000","last_peered":"2023-10-06T11:18:23.796243+0000","last_clean":"2023-10-06T11:18:23.796243+0000","last_became_active":"2023-10-06T11:18:23.795924+0000","last_became_peered":"2023-10-06T11:18:23.795924+0000","last_unstale":"2023-10-06T11:18:23.796243+0000","last_undegraded":"2023-10-06T11:18:23.796243+0000","last_fullsized":"2023-10-06T11:18:23.796243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080409+0000","last_change":"2023-10-06T11:18:22.789060+0000","last_active":"2023-10-06T11:18:24.080409+0000","last_peered":"2023-10-06T11:18:24.080409+0000","last_clean":"2023-10-06T11:18:24.080409+0000","last_became_active":"2023-10-06T11:18:22.788376+0000","last_became_peered":"2023-10-06T11:18:22.788376+0000","last_unstale":"2023-10-06T11:18:24.080409+0000","last_undegraded":"2023-10-06T11:18:24.080409+0000","last_fullsized":"2023-10-06T11:18:24.080409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802973+0000","last_change":"2023-10-06T11:18:23.802973+0000","last_active":"2023-10-06T11:18:23.802973+0000","last_peered":"2023-10-06T11:18:23.802973+0000","last_clean":"2023-10-06T11:18:23.802973+0000","last_became_active":"2023-10-06T11:18:23.802597+0000","last_became_peered":"2023-10-06T11:18:23.802597+0000","last_unstale":"2023-10-06T11:18:23.802973+0000","last_undegraded":"2023-10-06T11:18:23.802973+0000","last_fullsized":"2023-10-06T11:18:23.802973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780039+0000","last_change":"2023-10-06T11:18:22.770917+0000","last_active":"2023-10-06T11:18:23.780039+0000","last_peered":"2023-10-06T11:18:23.780039+0000","last_clean":"2023-10-06T11:18:23.780039+0000","last_became_active":"2023-10-06T11:18:22.770587+0000","last_became_peered":"2023-10-06T11:18:22.770587+0000","last_unstale":"2023-10-06T11:18:23.780039+0000","last_undegraded":"2023-10-06T11:18:23.780039+0000","last_fullsized":"2023-10-06T11:18:23.780039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797236+0000","last_change":"2023-10-06T11:18:23.797236+0000","last_active":"2023-10-06T11:18:23.797236+0000","last_peered":"2023-10-06T11:18:23.797236+0000","last_clean":"2023-10-06T11:18:23.797236+0000","last_became_active":"2023-10-06T11:18:23.796889+0000","last_became_peered":"2023-10-06T11:18:23.796889+0000","last_unstale":"2023-10-06T11:18:23.797236+0000","last_undegraded":"2023-10-06T11:18:23.797236+0000","last_fullsized":"2023-10-06T11:18:23.797236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.077534+0000","last_change":"2023-10-06T11:18:22.778280+0000","last_active":"2023-10-06T11:18:24.077534+0000","last_peered":"2023-10-06T11:18:24.077534+0000","last_clean":"2023-10-06T11:18:24.077534+0000","last_became_active":"2023-10-06T11:18:22.776627+0000","last_became_peered":"2023-10-06T11:18:22.776627+0000","last_unstale":"2023-10-06T11:18:24.077534+0000","last_undegraded":"2023-10-06T11:18:24.077534+0000","last_fullsized":"2023-10-06T11:18:24.077534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.669}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.625}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":84270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301612,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070228,"statfs":{"total":96636764160,"available":96327913472,"internally_reserved":0,"allocated":233472,"data_stored":82768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.655}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6870000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302192,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069648,"statfs":{"total":96636764160,"available":96327319552,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:18:28.761 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:18:28.761 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T11:18:28.761 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T11:18:28.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T11:18:28.762 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T11:18:28.861 INFO:tasks.ceph.mds.d.smithi132.stderr:2023-10-06T11:18:28.855+0000 7fb76e1a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:18:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 Processor -- start 2023-10-06T11:18:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 -- start start 2023-10-06T11:18:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 0x7f3ff4159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40b4f80 con 0x7f3ff40b5410 2023-10-06T11:18:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40b40c0 con 0x7f3ff4159050 2023-10-06T11:18:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.954+0000 7f3ffbef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40b4370 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ffa48d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ff40b5410 0x7f3ff40b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52210/0 (socket says 172.21.15.46:52210) 2023-10-06T11:18:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ffa48d700 1 -- 172.21.15.46:0/3078344652 learned_addr learned my addr 172.21.15.46:0/3078344652 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff948b700 1 --2- 172.21.15.46:0/3078344652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 0x7f3ff4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408363935 0 0) 0x7f3ff40b4f80 con 0x7f3ff40b5410 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe0003570 con 0x7f3ff40b5410 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176848890 0 0) 0x7f3ff40b40c0 con 0x7f3ff4159050 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff40b4f80 con 0x7f3ff4159050 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff948b700 1 -- 172.21.15.46:0/3078344652 >> v1:172.21.15.132:6789/0 conn(0x7f3ff4159050 legacy=0x7f3ff40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff948b700 1 -- 172.21.15.46:0/3078344652 >> v1:172.21.15.46:6789/0 conn(0x7f3ff40b5410 legacy=0x7f3ff40b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff948b700 1 -- 172.21.15.46:0/3078344652 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40b4650 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff948b700 1 --2- 172.21.15.46:0/3078344652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 0x7f3ff4159420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009910 tx=0x7f3fe40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163f7b256a5e21d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011630 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4011790 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.958+0000 7f3ff8c8a700 1 -- 172.21.15.46:0/3078344652 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011b30 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- 172.21.15.46:0/3078344652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 msgr2=0x7f3ff4159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/3078344652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 0x7f3ff4159420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009910 tx=0x7f3fe40091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- 172.21.15.46:0/3078344652 shutdown_connections 2023-10-06T11:18:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/3078344652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff40d0aa0 0x7f3ff4159420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- 172.21.15.46:0/3078344652 >> 172.21.15.46:0/3078344652 conn(0x7f3ff406e780 msgr2=0x7f3ff406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- 172.21.15.46:0/3078344652 shutdown_connections 2023-10-06T11:18:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- 172.21.15.46:0/3078344652 wait complete. 2023-10-06T11:18:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 Processor -- start 2023-10-06T11:18:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.962+0000 7f3ffbef0700 1 -- start start 2023-10-06T11:18:28.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ff40d0aa0 0x7f3ff415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff4159050 0x7f3ff4156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff415c900 con 0x7f3ff40b5410 2023-10-06T11:18:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff415aad0 con 0x7f3ff40d0aa0 2023-10-06T11:18:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff415adb0 con 0x7f3ff4159050 2023-10-06T11:18:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52440/0 (socket says 172.21.15.46:52440) 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 -- 172.21.15.46:0/1147945050 learned_addr learned my addr 172.21.15.46:0/1147945050 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffa48d700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff4159050 0x7f3ff4156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff948b700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ff40d0aa0 0x7f3ff415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 -- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff4159050 msgr2=0x7f3ff4156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff4159050 0x7f3ff4156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 -- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ff40d0aa0 msgr2=0x7f3ff415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ff40d0aa0 0x7f3ff415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 -- 172.21.15.46:0/1147945050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4009420 con 0x7f3ff40b5410 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ff9c8c700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c790 tx=0x7f3ff0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010510 con 0x7f3ff40b5410 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0010690 con 0x7f3ff40b5410 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010a50 con 0x7f3ff40b5410 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.966+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff415b090 con 0x7f3ff40b5410 2023-10-06T11:18:28.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.970+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff40c56c0 con 0x7f3ff40b5410 2023-10-06T11:18:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.970+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff40c5520 con 0x7f3ff40b5410 2023-10-06T11:18:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.970+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3ff001b3b0 con 0x7f3ff40b5410 2023-10-06T11:18:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.974+0000 7f3fe9ffb700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 0x7f3fd8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.974+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ff007c190 con 0x7f3ff40b5410 2023-10-06T11:18:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.974+0000 7f3ff948b700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 0x7f3fd8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.974+0000 7f3ff948b700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 0x7f3fd8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009860 tx=0x7f3fe4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:28.982+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3ff40c5520 con 0x7f3ff40b5410 2023-10-06T11:18:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.186+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3ff40c5520 con 0x7f3ff40b5410 2023-10-06T11:18:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.194+0000 7f3fe9ffb700 1 -- 172.21.15.46:0/1147945050 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154130 (secure 0 0 0) 0x7f3ff40c5520 con 0x7f3ff40b5410 2023-10-06T11:18:29.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 msgr2=0x7f3fd8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 0x7f3fd8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009860 tx=0x7f3fe4017000 comp rx=0 tx=0).stop 2023-10-06T11:18:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 msgr2=0x7f3ff415a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c790 tx=0x7f3ff0007e10 comp rx=0 tx=0).stop 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 shutdown_connections 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3fd8060550 0x7f3fd8062a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ff4159050 0x7f3ff4156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ff40d0aa0 0x7f3ff415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 --2- 172.21.15.46:0/1147945050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ff40b5410 0x7f3ff415a4a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 >> 172.21.15.46:0/1147945050 conn(0x7f3ff406e780 msgr2=0x7f3ff40cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 shutdown_connections 2023-10-06T11:18:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.202+0000 7f3ffbef0700 1 -- 172.21.15.46:0/1147945050 wait complete. 2023-10-06T11:18:29.229 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":32974},{"type":"v1","addr":"172.21.15.46:6833","nonce":32974}]},"active_addr":"172.21.15.46:6833/32974","active_change":"2023-10-06T11:18:02.461227+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":568326314}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3406707913}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2277270196}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3506103189}]}]} 2023-10-06T11:18:29.235 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T11:18:29.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T11:18:29.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.390+0000 7fbb8ba78700 1 Processor -- start 2023-10-06T11:18:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.390+0000 7fbb8ba78700 1 -- start start 2023-10-06T11:18:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8ba78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbb840c7b50 0x7fbb840c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8ba78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8ba78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb840c8500 con 0x7fbb84156c80 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8ba78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840c6470 con 0x7fbb840c7b50 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8ba78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840c7760 con 0x7fbb840c0bd0 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb89013700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbb840c7b50 0x7fbb840c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb89814700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbb84156c80 0x7fbb840ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46254/0 (socket says 172.21.15.46:46254) 2023-10-06T11:18:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42116/0 (socket says 172.21.15.46:42116) 2023-10-06T11:18:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb89814700 1 -- 172.21.15.46:0/2421925761 learned_addr learned my addr 172.21.15.46:0/2421925761 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 -- 172.21.15.46:0/2421925761 >> v2:172.21.15.132:3300/0 conn(0x7fbb840c7b50 msgr2=0x7fbb840c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 --2- 172.21.15.46:0/2421925761 >> v2:172.21.15.132:3300/0 conn(0x7fbb840c7b50 0x7fbb840c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 -- 172.21.15.46:0/2421925761 >> v1:172.21.15.132:6789/0 conn(0x7fbb84156c80 legacy=0x7fbb840ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 -- 172.21.15.46:0/2421925761 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840c66c0 con 0x7fbb840c0bd0 2023-10-06T11:18:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb88812700 1 -- 172.21.15.46:0/2421925761 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286671746 0 0) 0x7fbb840c8500 con 0x7fbb84156c80 2023-10-06T11:18:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb8a015700 1 --2- 172.21.15.46:0/2421925761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009e60 tx=0x7fbb7c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da42ae2f367e4db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb88812700 1 -- 172.21.15.46:0/2421925761 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00e070 con 0x7fbb840c0bd0 2023-10-06T11:18:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb88812700 1 -- 172.21.15.46:0/2421925761 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00bb60 con 0x7fbb840c0bd0 2023-10-06T11:18:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.394+0000 7fbb88812700 1 -- 172.21.15.46:0/2421925761 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c012430 con 0x7fbb840c0bd0 2023-10-06T11:18:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- 172.21.15.46:0/2421925761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 msgr2=0x7fbb840c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/2421925761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009e60 tx=0x7fbb7c009780 comp rx=0 tx=0).stop 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- 172.21.15.46:0/2421925761 shutdown_connections 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/2421925761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c0bd0 0x7fbb840c1050 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/2421925761 >> v2:172.21.15.132:3300/0 conn(0x7fbb840c7b50 0x7fbb840c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- 172.21.15.46:0/2421925761 >> 172.21.15.46:0/2421925761 conn(0x7fbb840cbe30 msgr2=0x7fbb840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- 172.21.15.46:0/2421925761 shutdown_connections 2023-10-06T11:18:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- 172.21.15.46:0/2421925761 wait complete. 2023-10-06T11:18:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 Processor -- start 2023-10-06T11:18:29.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.398+0000 7fbb8ba78700 1 -- start start 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 0x7fbb840c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 0x7fbb841509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8414fc80 con 0x7fbb840c0bd0 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b0ad0 con 0x7fbb84156c80 2023-10-06T11:18:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840b01a0 con 0x7fbb840c7b50 2023-10-06T11:18:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89814700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89814700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52446/0 (socket says 172.21.15.46:52446) 2023-10-06T11:18:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89814700 1 -- 172.21.15.46:0/3090710244 learned_addr learned my addr 172.21.15.46:0/3090710244 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 0x7fbb841509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89013700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 0x7fbb840c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 -- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 msgr2=0x7fbb840c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 0x7fbb840c3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 -- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 msgr2=0x7fbb8414f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 -- 172.21.15.46:0/3090710244 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009a20 con 0x7fbb84156c80 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89013700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 0x7fbb840c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb89814700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8a015700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 0x7fbb841509c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c016000 tx=0x7fbb7c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c012ea0 con 0x7fbb84156c80 2023-10-06T11:18:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c027470 con 0x7fbb84156c80 2023-10-06T11:18:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb840b0480 con 0x7fbb84156c80 2023-10-06T11:18:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.402+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c027830 con 0x7fbb84156c80 2023-10-06T11:18:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.406+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb840c4300 con 0x7fbb84156c80 2023-10-06T11:18:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.406+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb840b0c40 con 0x7fbb84156c80 2023-10-06T11:18:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.406+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbb7c024070 con 0x7fbb84156c80 2023-10-06T11:18:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.410+0000 7fbb79ffb700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 0x7fbb68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.410+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb7c07f510 con 0x7fbb84156c80 2023-10-06T11:18:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.410+0000 7fbb89814700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 0x7fbb68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.410+0000 7fbb89814700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 0x7fbb68062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb80001230 tx=0x7fbb80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.418+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb7c04f980 con 0x7fbb84156c80 2023-10-06T11:18:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.590+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb84063f90 con 0x7fbb84156c80 2023-10-06T11:18:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.594+0000 7fbb79ffb700 1 -- 172.21.15.46:0/3090710244 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fbb7c052fa0 con 0x7fbb84156c80 2023-10-06T11:18:29.601 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:29.601 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 msgr2=0x7fbb68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 0x7fbb68062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb80001230 tx=0x7fbb80009040 comp rx=0 tx=0).stop 2023-10-06T11:18:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 msgr2=0x7fbb841509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 0x7fbb841509c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c016000 tx=0x7fbb7c019000 comp rx=0 tx=0).stop 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 shutdown_connections 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb68060550 0x7fbb68062a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb84156c80 0x7fbb841509c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb840c7b50 0x7fbb840c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 --2- 172.21.15.46:0/3090710244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb840c0bd0 0x7fbb8414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 >> 172.21.15.46:0/3090710244 conn(0x7fbb840cbe30 msgr2=0x7fbb840699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 shutdown_connections 2023-10-06T11:18:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.598+0000 7fbb8ba78700 1 -- 172.21.15.46:0/3090710244 wait complete. 2023-10-06T11:18:29.626 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T11:18:29.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 Processor -- start 2023-10-06T11:18:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 -- start start 2023-10-06T11:18:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1eec0aaf30 0x7f1eec067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0679f0 con 0x7f1eec156e20 2023-10-06T11:18:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec0acd60 con 0x7f1eec0aaf30 2023-10-06T11:18:29.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1ef082d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0ad040 con 0x7f1eec067590 2023-10-06T11:18:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1eeb59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1eea59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1eec0aaf30 0x7f1eec067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.778+0000 7f1eeb59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec068a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42138/0 (socket says 172.21.15.46:42138) 2023-10-06T11:18:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1eec0aaf30 0x7f1eec067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:32980/0 (socket says 172.21.15.46:32980) 2023-10-06T11:18:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eeb59e700 1 -- 172.21.15.46:0/1926985925 learned_addr learned my addr 172.21.15.46:0/1926985925 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eead9d700 1 --1- 172.21.15.46:0/1926985925 >> v1:172.21.15.132:6789/0 conn(0x7f1eec156e20 0x7f1eec06d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46256/0 (socket says 172.21.15.46:46256) 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 -- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 msgr2=0x7f1eec068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 --2- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 -- 172.21.15.46:0/1926985925 >> v1:172.21.15.132:6789/0 conn(0x7f1eec156e20 legacy=0x7f1eec06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 -- 172.21.15.46:0/1926985925 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0ad320 con 0x7f1eec0aaf30 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1eea59c700 1 --2- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec0aaf30 0x7f1eec067050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00098e0 tx=0x7f1ee00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae14d0ae78d05db2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ee9d9b700 1 -- 172.21.15.46:0/1926985925 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000b640 con 0x7f1eec0aaf30 2023-10-06T11:18:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ee9d9b700 1 -- 172.21.15.46:0/1926985925 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000b7a0 con 0x7f1eec0aaf30 2023-10-06T11:18:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ee9d9b700 1 -- 172.21.15.46:0/1926985925 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000bb40 con 0x7f1eec0aaf30 2023-10-06T11:18:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 -- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec0aaf30 msgr2=0x7f1eec067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 --2- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec0aaf30 0x7f1eec067050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00098e0 tx=0x7f1ee00091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 -- 172.21.15.46:0/1926985925 shutdown_connections 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 --2- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 --2- 172.21.15.46:0/1926985925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec0aaf30 0x7f1eec067050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 -- 172.21.15.46:0/1926985925 >> 172.21.15.46:0/1926985925 conn(0x7f1eec062ef0 msgr2=0x7f1eec0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 -- 172.21.15.46:0/1926985925 shutdown_connections 2023-10-06T11:18:29.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.782+0000 7f1ef082d700 1 -- 172.21.15.46:0/1926985925 wait complete. 2023-10-06T11:18:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.786+0000 7f1ef082d700 1 Processor -- start 2023-10-06T11:18:29.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.786+0000 7f1ef082d700 1 -- start start 2023-10-06T11:18:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec0aaf30 0x7f1eec0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 0x7f1eec0a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec16c9a0 con 0x7f1eec0aaf30 2023-10-06T11:18:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c6530 con 0x7f1eec156e20 2023-10-06T11:18:29.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c6810 con 0x7f1eec067590 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec0aaf30 0x7f1eec0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42146/0 (socket says 172.21.15.46:42146) 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 -- 172.21.15.46:0/2005629256 learned_addr learned my addr 172.21.15.46:0/2005629256 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eeb59e700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 0x7f1eec0a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 -- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 msgr2=0x7f1eec0a9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 0x7f1eec0a9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 -- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec0aaf30 msgr2=0x7f1eec0a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec0aaf30 0x7f1eec0a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 -- 172.21.15.46:0/2005629256 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0009420 con 0x7f1eec067590 2023-10-06T11:18:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eeb59e700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 0x7f1eec0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1eead9d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001eb0 tx=0x7f1edc007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00ce80 con 0x7f1eec067590 2023-10-06T11:18:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc007360 con 0x7f1eec067590 2023-10-06T11:18:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0076e0 con 0x7f1eec067590 2023-10-06T11:18:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec0c6af0 con 0x7f1eec067590 2023-10-06T11:18:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.790+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec0c70b0 con 0x7f1eec067590 2023-10-06T11:18:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.794+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec064590 con 0x7f1eec067590 2023-10-06T11:18:29.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.794+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1edc004a40 con 0x7f1eec067590 2023-10-06T11:18:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.794+0000 7f1ed2ffd700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 0x7f1ed8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.794+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1edc07c330 con 0x7f1eec067590 2023-10-06T11:18:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.798+0000 7f1eea59c700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 0x7f1ed8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.798+0000 7f1eea59c700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 0x7f1ed8062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00093f0 tx=0x7f1ee0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:29.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.806+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1edc04c7a0 con 0x7f1eec067590 2023-10-06T11:18:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.978+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eec1696b0 con 0x7f1eec067590 2023-10-06T11:18:29.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.982+0000 7f1ed2ffd700 1 -- 172.21.15.46:0/2005629256 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f1edc04fdc0 con 0x7f1eec067590 2023-10-06T11:18:29.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:29.988 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 msgr2=0x7f1ed8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 0x7f1ed8062a00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00093f0 tx=0x7f1ee0017000 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 msgr2=0x7f1eec16c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1edc001eb0 tx=0x7f1edc007f40 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 shutdown_connections 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ed8060550 0x7f1ed8062a00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eec156e20 0x7f1eec0a9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec0aaf30 0x7f1eec0a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 --2- 172.21.15.46:0/2005629256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eec067590 0x7f1eec16c460 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 >> 172.21.15.46:0/2005629256 conn(0x7f1eec062ef0 msgr2=0x7f1eec155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 shutdown_connections 2023-10-06T11:18:29.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:29.986+0000 7f1ef082d700 1 -- 172.21.15.46:0/2005629256 wait complete. 2023-10-06T11:18:30.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T11:18:30.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T11:18:30.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T11:18:30.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T11:18:30.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T11:18:30.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T11:18:30.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T11:18:30.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T11:18:30.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 Processor -- start 2023-10-06T11:18:30.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 -- start start 2023-10-06T11:18:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1e4152a40 0x7fb1e406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40686f0 con 0x7fb1e4152ee0 2023-10-06T11:18:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40c6de0 con 0x7fb1e40c8d60 2023-10-06T11:18:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1eb86e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e415c170 con 0x7fb1e4152a40 2023-10-06T11:18:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1e960a700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb1e40c8d60 0x7fb1e40c6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46262/0 (socket says 172.21.15.46:46262) 2023-10-06T11:18:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1e960a700 1 -- 172.21.15.46:0/1169674435 learned_addr learned my addr 172.21.15.46:0/1169674435 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.230+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> v2:172.21.15.46:3300/0 conn(0x7fb1e4152a40 0x7fb1e406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563005785 0 0) 0x7fb1e40c6de0 con 0x7fb1e40c8d60 2023-10-06T11:18:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003570 con 0x7fb1e40c8d60 2023-10-06T11:18:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642722827 0 0) 0x7fb1e40686f0 con 0x7fb1e4152ee0 2023-10-06T11:18:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e40c6de0 con 0x7fb1e4152ee0 2023-10-06T11:18:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 107569465 0 0) 0x7fb1d0003570 con 0x7fb1e40c8d60 2023-10-06T11:18:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 >> v2:172.21.15.46:3300/0 conn(0x7fb1e4152a40 msgr2=0x7fb1e406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 --2- 172.21.15.46:0/1169674435 >> v2:172.21.15.46:3300/0 conn(0x7fb1e4152a40 0x7fb1e406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 >> v1:172.21.15.46:6789/0 conn(0x7fb1e4152ee0 legacy=0x7fb1e4068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40c70c0 con 0x7fb1e40c8d60 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fb1dc0017f0 con 0x7fb1e40c8d60 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> v2:172.21.15.46:3300/0 conn(0x7fb1e4152a40 0x7fb1e406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 >> v1:172.21.15.132:6789/0 conn(0x7fb1e40c8d60 legacy=0x7fb1e40c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1d0003bf0 0x7fb1d0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 0x7fb1d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d00051c0 0x7fb1d0014270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e415c170 con 0x7fb1d0004810 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00148c0 con 0x7fb1d0003bf0 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0014a80 con 0x7fb1d00051c0 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 0x7fb1d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e960a700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1d0003bf0 0x7fb1d0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 -- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d00051c0 msgr2=0x7fb1d0014270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:18:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d00051c0 0x7fb1d0014270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 -- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1d0003bf0 msgr2=0x7fb1d0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1d0003bf0 0x7fb1d0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e8e09700 1 -- 172.21.15.46:0/1169674435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0003570 con 0x7fb1d0004810 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.234+0000 7fb1e9e0b700 1 -- 172.21.15.46:0/1169674435 reap_dead start 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 0x7fb1d0004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009df0 tx=0x7fb1e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc863fc27188349 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000e920 con 0x7fb1d0004810 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e000ea80 con 0x7fb1d0004810 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1dbfff700 1 -- 172.21.15.46:0/1169674435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000ee20 con 0x7fb1d0004810 2023-10-06T11:18:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7f04f40ae700 1 Processor -- start 2023-10-06T11:18:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 -- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 msgr2=0x7fb1d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 0x7fb1d0004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009df0 tx=0x7fb1e00091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 -- 172.21.15.46:0/1169674435 shutdown_connections 2023-10-06T11:18:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 --2- 172.21.15.46:0/1169674435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d0004810 0x7fb1d0004c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 -- 172.21.15.46:0/1169674435 >> 172.21.15.46:0/1169674435 conn(0x7fb1e40cc6b0 msgr2=0x7fb1e40c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7f04f40ae700 1 -- start start 2023-10-06T11:18:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7f4a5d8ae700 1 Processor -- start 2023-10-06T11:18:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 -- 172.21.15.46:0/1169674435 shutdown_connections 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7fb1eb86e700 1 -- 172.21.15.46:0/1169674435 wait complete. 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.238+0000 7f4a5d8ae700 1 -- start start 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f40ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04ec156cb0 0x7f04ec0685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f40ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 0x7f04ec0a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f40ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec0a9a80 con 0x7f04ec068b00 2023-10-06T11:18:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f40ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec0699d0 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f40ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec068f50 con 0x7f04ec156cb0 2023-10-06T11:18:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 Processor -- start 2023-10-06T11:18:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f264b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 0x7f04ec0a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a5d8ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4a580ad920 0x7f4a5806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a5d8ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a580adcf0 0x7f4a580c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a5d8ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5806c8d0 con 0x7f4a580c6180 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a5d8ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5806d1c0 con 0x7f4a580adcf0 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a5d8ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580c65c0 con 0x7f4a580ad920 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a580adcf0 0x7f4a580c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a580adcf0 0x7f4a580c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52476/0 (socket says 172.21.15.46:52476) 2023-10-06T11:18:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 -- 172.21.15.46:0/1801351309 learned_addr learned my addr 172.21.15.46:0/1801351309 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 -- start start 2023-10-06T11:18:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f264b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 0x7f04ec0a9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52468/0 (socket says 172.21.15.46:52468) 2023-10-06T11:18:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f264b700 1 -- 172.21.15.46:0/2709654913 learned_addr learned my addr 172.21.15.46:0/2709654913 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 --2- 172.21.15.46:0/2709654913 >> v2:172.21.15.132:3300/0 conn(0x7f04ec156cb0 0x7f04ec0685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a56ffd700 1 --2- 172.21.15.46:0/1801351309 >> v2:172.21.15.132:3300/0 conn(0x7f4a580ad920 0x7f4a5806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 -- 172.21.15.46:0/2709654913 >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 msgr2=0x7f04ec0a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 --2- 172.21.15.46:0/2709654913 >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 0x7f04ec0a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 -- 172.21.15.46:0/2709654913 >> v1:172.21.15.46:6789/0 conn(0x7f04ec068b00 legacy=0x7f04ec0c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 -- 172.21.15.46:0/2709654913 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec15a6b0 con 0x7f04ec156cb0 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 -- 172.21.15.46:0/1801351309 >> v2:172.21.15.132:3300/0 conn(0x7f4a580ad920 msgr2=0x7f4a5806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 --2- 172.21.15.46:0/1801351309 >> v2:172.21.15.132:3300/0 conn(0x7f4a580ad920 0x7f4a5806e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 -- 172.21.15.46:0/1801351309 >> v1:172.21.15.132:6789/0 conn(0x7f4a580c6180 legacy=0x7f4a58161130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f4a4e7fc700 1 -- 172.21.15.46:0/1801351309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5806eb90 con 0x7f4a580adcf0 2023-10-06T11:18:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f1fbb59e700 1 Processor -- start 2023-10-06T11:18:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f1e4a700 1 --2- 172.21.15.46:0/2709654913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0685c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04ec064fc0 tx=0x7f04e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de005e1e5c61247 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d0004810 0x7fb1e40aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 0x7fb1e414cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e4154200 0x7fb1e414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e414d630 con 0x7fb1e40af100 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4154700 con 0x7fb1e4154200 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1eb86e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41549e0 con 0x7fb1d0004810 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1e8e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 0x7fb1e414cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1e8e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 0x7fb1e414cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52488/0 (socket says 172.21.15.46:52488) 2023-10-06T11:18:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1e8e09700 1 -- 172.21.15.46:0/2120400910 learned_addr learned my addr 172.21.15.46:0/2120400910 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7fb1e960a700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d0004810 0x7fb1e40aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e9e0b700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e4154200 0x7fb1e414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 Processor -- start 2023-10-06T11:18:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a4e7fc700 1 --2- 172.21.15.46:0/1801351309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580adcf0 0x7f4a580c9130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806a230 tx=0x7f4a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3056ba185a4d0977 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 -- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d0004810 msgr2=0x7fb1e40aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1d0004810 0x7fb1e40aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 -- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e4154200 msgr2=0x7fb1e414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e4154200 0x7fb1e414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e4154cc0 con 0x7fb1e40af100 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f0e48700 1 -- 172.21.15.46:0/2709654913 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8011630 con 0x7f04ec156cb0 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f0e48700 1 -- 172.21.15.46:0/2709654913 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e8011790 con 0x7f04ec156cb0 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f04f0e48700 1 -- 172.21.15.46:0/2709654913 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8011b30 con 0x7f04ec156cb0 2023-10-06T11:18:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 -- 172.21.15.46:0/2709654913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 msgr2=0x7f04ec0685c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 --2- 172.21.15.46:0/2709654913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0685c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04ec064fc0 tx=0x7f04e80091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1e8e09700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 0x7fb1e414cbb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0014890 tx=0x7fb1e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 -- start start 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.242+0000 7f1fbb59e700 1 -- start start 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbb59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1fbc1527e0 0x7f1fbc152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbb59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 0x7f1fbc067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbb59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc06d470 con 0x7f1fbc0c7ec0 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbb59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc158ec0 con 0x7f1fbc159350 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbb59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc158b90 con 0x7f1fbc1527e0 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 -- 172.21.15.46:0/2709654913 shutdown_connections 2023-10-06T11:18:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 --2- 172.21.15.46:0/2709654913 >> v2:172.21.15.46:3300/0 conn(0x7f04ec0c6aa0 0x7f04ec0a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 --2- 172.21.15.46:0/2709654913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0685c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 -- 172.21.15.46:0/2709654913 >> 172.21.15.46:0/2709654913 conn(0x7f04ec0cc6b0 msgr2=0x7f04ec0c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a567fc700 1 -- 172.21.15.46:0/1801351309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40009e70 con 0x7f4a580adcf0 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0022070 con 0x7fb1e40af100 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1eb86e700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e4154fa0 con 0x7fb1e40af100 2023-10-06T11:18:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1eb86e700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40af5b0 con 0x7fb1e40af100 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a567fc700 1 -- 172.21.15.46:0/1801351309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000b860 con 0x7f4a580adcf0 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a567fc700 1 -- 172.21.15.46:0/1801351309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000bc00 con 0x7f4a580adcf0 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fb9d9b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1fbc0c7ec0 0x7f1fbc155590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46274/0 (socket says 172.21.15.46:46274) 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fb9d9b700 1 -- 172.21.15.46:0/3672562841 learned_addr learned my addr 172.21.15.46:0/3672562841 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fba59c700 1 --2- 172.21.15.46:0/3672562841 >> v2:172.21.15.132:3300/0 conn(0x7f1fbc1527e0 0x7f1fbc152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/3672562841 >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 0x7f1fbc067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f9a2cfc3700 1 Processor -- start 2023-10-06T11:18:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdd34034200 0x7fdd340de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd34053c60 con 0x7fdd34052320 2023-10-06T11:18:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/1801351309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580adcf0 msgr2=0x7f4a580c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 --2- 172.21.15.46:0/1801351309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580adcf0 0x7f4a580c9130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806a230 tx=0x7f4a40009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e0013b30 con 0x7fb1e40af100 2023-10-06T11:18:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000e920 con 0x7fb1e40af100 2023-10-06T11:18:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd34053f00 con 0x7fdd340ddf00 2023-10-06T11:18:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd40b63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd340541e0 con 0x7fdd34034200 2023-10-06T11:18:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/1801351309 shutdown_connections 2023-10-06T11:18:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 --2- 172.21.15.46:0/1801351309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580adcf0 0x7f4a580c9130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 --2- 172.21.15.46:0/1801351309 >> v2:172.21.15.132:3300/0 conn(0x7f4a580ad920 0x7f4a5806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/1801351309 >> 172.21.15.46:0/1801351309 conn(0x7f4a580cc6b0 msgr2=0x7f4a580c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 -- 172.21.15.46:0/2709654913 shutdown_connections 2023-10-06T11:18:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f1fba59c700 1 -- 172.21.15.46:0/3672562841 >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 msgr2=0x7f1fbc067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f9a2cfc3700 1 -- start start 2023-10-06T11:18:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 Processor -- start 2023-10-06T11:18:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fb1eb86e700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1c4000fd0 con 0x7fb1e40af100 2023-10-06T11:18:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd3affd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdd340ddf00 0x7fdd34033af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46286/0 (socket says 172.21.15.46:46286) 2023-10-06T11:18:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7fdd3affd700 1 -- 172.21.15.46:0/1439653395 learned_addr learned my addr 172.21.15.46:0/1439653395 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/1801351309 shutdown_connections 2023-10-06T11:18:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 -- start start 2023-10-06T11:18:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/1439653395 >> v2:172.21.15.132:3300/0 conn(0x7fdd34034200 0x7fdd340de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.246+0000 7f04f40ae700 1 -- 172.21.15.46:0/2709654913 wait complete. 2023-10-06T11:18:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3a7fc700 1 -- 172.21.15.46:0/1439653395 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250421110 0 0) 0x7fdd34053f00 con 0x7fdd340ddf00 2023-10-06T11:18:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3bfff700 1 -- 172.21.15.46:0/1439653395 >> v1:172.21.15.132:6789/0 conn(0x7fdd340ddf00 legacy=0x7fdd34033af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3bfff700 1 -- 172.21.15.46:0/1439653395 >> v1:172.21.15.46:6789/0 conn(0x7fdd34052320 legacy=0x7fdd3404fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3bfff700 1 -- 172.21.15.46:0/1439653395 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340544c0 con 0x7fdd34034200 2023-10-06T11:18:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fba59c700 1 --2- 172.21.15.46:0/3672562841 >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 0x7f1fbc067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fba59c700 1 -- 172.21.15.46:0/3672562841 >> v1:172.21.15.132:6789/0 conn(0x7f1fbc0c7ec0 legacy=0x7f1fbc155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fba59c700 1 -- 172.21.15.46:0/3672562841 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc155dc0 con 0x7f1fbc1527e0 2023-10-06T11:18:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/3672562841 >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 0x7f1fbc067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/1439653395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34034200 0x7fdd340de350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd300098e0 tx=0x7fdd300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fddcc7f55c30fc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fba59c700 1 --2- 172.21.15.46:0/3672562841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc152bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc068ee0 tx=0x7f1fb400a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de8d6fa2b526f5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3a7fc700 1 -- 172.21.15.46:0/1439653395 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30009e80 con 0x7fdd34034200 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3a7fc700 1 -- 172.21.15.46:0/1439653395 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3000b860 con 0x7fdd34034200 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd3a7fc700 1 -- 172.21.15.46:0/1439653395 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3000bc00 con 0x7fdd34034200 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fb959a700 1 -- 172.21.15.46:0/3672562841 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400b050 con 0x7f1fbc1527e0 2023-10-06T11:18:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fb959a700 1 -- 172.21.15.46:0/3672562841 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb400b1b0 con 0x7f1fbc1527e0 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f1fb959a700 1 -- 172.21.15.46:0/3672562841 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4018680 con 0x7f1fbc1527e0 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f04f40ae700 1 Processor -- start 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a2cfc3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a28067760 0x7f9a280c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a2cfc3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a280bf1a0 0x7f9a280c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a2cfc3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a280c8480 con 0x7f9a280bedd0 2023-10-06T11:18:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a2cfc3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a28069660 con 0x7f9a28067760 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a2cfc3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a28068ac0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/1801351309 wait complete. 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f4a5d8ae700 1 Processor -- start 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c080e66b0 0x7f9c080ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c08054140 con 0x7f9c080ea280 2023-10-06T11:18:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c08053320 con 0x7f9c080e66b0 2023-10-06T11:18:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9c16d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08050520 con 0x7f9c080500a0 2023-10-06T11:18:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd40b63700 1 -- 172.21.15.46:0/1439653395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34034200 msgr2=0x7fdd340de350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd40b63700 1 --2- 172.21.15.46:0/1439653395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34034200 0x7fdd340de350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd300098e0 tx=0x7fdd300091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a27fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a280bf1a0 0x7f9a280c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd40b63700 1 -- 172.21.15.46:0/1439653395 shutdown_connections 2023-10-06T11:18:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd40b63700 1 --2- 172.21.15.46:0/1439653395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34034200 0x7fdd340de350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7fdd40b63700 1 -- 172.21.15.46:0/1439653395 >> 172.21.15.46:0/1439653395 conn(0x7fdd34051810 msgr2=0x7fdd34051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f4a5d8ae700 1 -- start start 2023-10-06T11:18:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a27fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a280bf1a0 0x7f9a280c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33068/0 (socket says 172.21.15.46:33068) 2023-10-06T11:18:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f9a27fff700 1 -- 172.21.15.46:0/923498804 learned_addr learned my addr 172.21.15.46:0/923498804 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f49a7fff700 1 Processor -- start 2023-10-06T11:18:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 -- start start 2023-10-06T11:18:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a27fff700 1 -- 172.21.15.46:0/923498804 >> v2:172.21.15.46:3300/0 conn(0x7f9a28067760 msgr2=0x7f9a280c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a27fff700 1 --2- 172.21.15.46:0/923498804 >> v2:172.21.15.46:3300/0 conn(0x7f9a28067760 0x7f9a280c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c0ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c080e66b0 0x7f9c080ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42188/0 (socket says 172.21.15.46:42188) 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 -- 172.21.15.46:0/514104931 learned_addr learned my addr 172.21.15.46:0/514104931 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.250+0000 7f04f40ae700 1 -- start start 2023-10-06T11:18:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04ec068b00 0x7f04ec156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a27fff700 1 -- 172.21.15.46:0/923498804 >> v1:172.21.15.132:6789/0 conn(0x7f9a280bedd0 legacy=0x7f9a28067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a27fff700 1 -- 172.21.15.46:0/923498804 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28068d30 con 0x7f9a280bf1a0 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fdd40b63700 1 -- 172.21.15.46:0/1439653395 shutdown_connections 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 -- 172.21.15.46:0/514104931 >> v2:172.21.15.46:3300/0 conn(0x7f9c080e66b0 msgr2=0x7f9c080ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 --2- 172.21.15.46:0/514104931 >> v2:172.21.15.46:3300/0 conn(0x7f9c080e66b0 0x7f9c080ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 -- 172.21.15.46:0/514104931 >> v1:172.21.15.46:6789/0 conn(0x7f9c080ea280 legacy=0x7f9c080e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 -- 172.21.15.46:0/514104931 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c08053530 con 0x7f9c080500a0 2023-10-06T11:18:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a27fff700 1 --2- 172.21.15.46:0/923498804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bf1a0 0x7f9a280c7f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a28068060 tx=0x7f9a20009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f09d072be8a54a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 0x7f04ec0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c152e2700 1 --2- 172.21.15.46:0/514104931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9c040099b0 tx=0x7f9c04009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3a40656bcbe4ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 0x7f4a581760f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a267fc700 1 -- 172.21.15.46:0/923498804 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000b860 con 0x7f9a280bf1a0 2023-10-06T11:18:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a267fc700 1 -- 172.21.15.46:0/923498804 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2000b9c0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a267fc700 1 -- 172.21.15.46:0/923498804 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000bd60 con 0x7f9a280bf1a0 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f1e4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04ec068b00 0x7f04ec156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f1e4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04ec068b00 0x7f04ec156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52514/0 (socket says 172.21.15.46:52514) 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c0f7fe700 1 -- 172.21.15.46:0/514104931 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400b860 con 0x7f9c080500a0 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c0f7fe700 1 -- 172.21.15.46:0/514104931 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0400b9c0 con 0x7f9c080500a0 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9c0f7fe700 1 -- 172.21.15.46:0/514104931 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400bd60 con 0x7f9c080500a0 2023-10-06T11:18:30.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f1649700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 0x7f04ec0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f1649700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 0x7f04ec0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42194/0 (socket says 172.21.15.46:42194) 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb1e0026070 con 0x7fb1e40af100 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1c80606a0 0x7fb1c8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb1e007dfe0 con 0x7fb1e40af100 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7fb1c8068f50 0x7fb1c806b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1c806ba10 con 0x7fb1c8068f50 2023-10-06T11:18:30.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e0080580 con 0x7fb1e40af100 2023-10-06T11:18:30.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f1e4a700 1 -- 172.21.15.46:0/165751037 learned_addr learned my addr 172.21.15.46:0/165751037 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec15ccf0 con 0x7f04ec068b00 2023-10-06T11:18:30.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 0x7f4a581760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 0x7f4a581760f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52522/0 (socket says 172.21.15.46:52522) 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a56ffd700 1 -- 172.21.15.46:0/2006077442 learned_addr learned my addr 172.21.15.46:0/2006077442 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a580adcf0 0x7f4a580c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0ac550 con 0x7f04ec156cb0 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a580c6180 0x7f4a580c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1e960a700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1c80606a0 0x7fb1c8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1e9e0b700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7fb1c8068f50 0x7fb1c806b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f40ae700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0ac830 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/923498804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bf1a0 msgr2=0x7f9a280c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f9a2cfc3700 1 --2- 172.21.15.46:0/923498804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bf1a0 0x7f9a280c7f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a28068060 tx=0x7f9a20009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580c3090 con 0x7f4a580ad920 2023-10-06T11:18:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a581770d0 con 0x7f4a580c6180 2023-10-06T11:18:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58176b10 con 0x7f4a580adcf0 2023-10-06T11:18:30.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f04f264b700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49a80b1300 0x7f49a80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a80ae340 con 0x7f49a80d07b0 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a80b09a0 con 0x7f49a80b1300 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f49a7fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8159660 con 0x7f49a8159290 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f1fbb59e700 1 -- 172.21.15.46:0/3672562841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc1527e0 msgr2=0x7f1fbc152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f1fbb59e700 1 --2- 172.21.15.46:0/3672562841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc152bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc068ee0 tx=0x7f1fb400a310 comp rx=0 tx=0).stop 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 -- 172.21.15.46:0/3672562841 shutdown_connections 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 --2- 172.21.15.46:0/3672562841 >> v2:172.21.15.46:3300/0 conn(0x7f1fbc159350 0x7f1fbc067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 --2- 172.21.15.46:0/3672562841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc152bb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 -- 172.21.15.46:0/3672562841 >> 172.21.15.46:0/3672562841 conn(0x7f1fbc0cbbb0 msgr2=0x7f1fbc0cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fb1e9e0b700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7fb1c8068f50 0x7fb1c806b360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42218/0 (socket says 172.21.15.46:42218) 2023-10-06T11:18:30.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 -- 172.21.15.46:0/3158644190 learned_addr learned my addr 172.21.15.46:0/3158644190 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a577fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a580c6180 0x7f4a580c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7f4a4e7fc700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a580adcf0 0x7f4a580c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 -- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 msgr2=0x7f04ec0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec156cb0 0x7f04ec0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 -- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04ec068b00 msgr2=0x7f04ec156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04ec068b00 0x7f04ec156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e8009420 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== osd.0 v2:172.21.15.46:6824/33078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb1c806ba10 con 0x7fb1c8068f50 2023-10-06T11:18:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.254+0000 7fdd40b63700 1 -- 172.21.15.46:0/1439653395 wait complete. 2023-10-06T11:18:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 -- 172.21.15.46:0/3672562841 shutdown_connections 2023-10-06T11:18:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/923498804 shutdown_connections 2023-10-06T11:18:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 -- 172.21.15.46:0/514104931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 msgr2=0x7f9c08052de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 --2- 172.21.15.46:0/514104931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9c040099b0 tx=0x7f9c04009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 -- 172.21.15.46:0/514104931 shutdown_connections 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 --2- 172.21.15.46:0/514104931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080500a0 0x7f9c08052de0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 --2- 172.21.15.46:0/514104931 >> v2:172.21.15.46:3300/0 conn(0x7f9c080e66b0 0x7f9c080ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 -- 172.21.15.46:0/514104931 >> 172.21.15.46:0/514104931 conn(0x7f9c08054c90 msgr2=0x7f9c08055090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7fb1e960a700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1c80606a0 0x7fb1c8062b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0013d0 tx=0x7fb1dc006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7fdd40b63700 1 Processor -- start 2023-10-06T11:18:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7fdd40b63700 1 -- start start 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 -- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a580adcf0 msgr2=0x7f4a580c1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a580adcf0 0x7f4a580c1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 -- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a580c6180 msgr2=0x7f4a580c1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a580c6180 0x7f4a580c1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400094f0 con 0x7f4a580ad920 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a577fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a580c6180 0x7f4a580c1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a56ffd700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 0x7f4a581760f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009ef0 tx=0x7f4a4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800cc20 con 0x7f4a580ad920 2023-10-06T11:18:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4800f040 con 0x7f4a580ad920 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48014480 con 0x7f4a580ad920 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a58176df0 con 0x7f4a580ad920 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a58177300 con 0x7f4a580ad920 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 -- 172.21.15.46:0/3672562841 wait complete. 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 Processor -- start 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f1fbb59e700 1 -- start start 2023-10-06T11:18:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9a2cfc3700 1 --2- 172.21.15.46:0/923498804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bf1a0 0x7f9a280c7f40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9a2cfc3700 1 --2- 172.21.15.46:0/923498804 >> v2:172.21.15.46:3300/0 conn(0x7f9a28067760 0x7f9a280c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/923498804 >> 172.21.15.46:0/923498804 conn(0x7f9a28062ef0 msgr2=0x7f9a280cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/923498804 shutdown_connections 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/923498804 wait complete. 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 -- 172.21.15.46:0/514104931 shutdown_connections 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f9c16d45700 1 -- 172.21.15.46:0/514104931 wait complete. 2023-10-06T11:18:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 -- 172.21.15.46:0/3158644190 >> v2:172.21.15.46:3300/0 conn(0x7f49a80b1300 msgr2=0x7f49a80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 --2- 172.21.15.46:0/3158644190 >> v2:172.21.15.46:3300/0 conn(0x7f49a80b1300 0x7f49a80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 -- 172.21.15.46:0/3158644190 >> v1:172.21.15.46:6789/0 conn(0x7f49a80d07b0 legacy=0x7f49a80b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 -- 172.21.15.46:0/3158644190 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80b3450 con 0x7f49a8159290 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a6ffd700 1 --2- 172.21.15.46:0/3158644190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49980099b0 tx=0x7f4998009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa48631c43c7890 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a5ffb700 1 -- 172.21.15.46:0/3158644190 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800b8b0 con 0x7f49a8159290 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a5ffb700 1 -- 172.21.15.46:0/3158644190 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499800ba10 con 0x7f49a8159290 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a5ffb700 1 -- 172.21.15.46:0/3158644190 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800bdb0 con 0x7f49a8159290 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a7fff700 1 -- 172.21.15.46:0/3158644190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 msgr2=0x7f49a80b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3158644190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49980099b0 tx=0x7f4998009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.258+0000 7f04f1649700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 0x7f04ec0ad730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c490 tx=0x7f04e0007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd34034200 0x7fdd340d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34052320 0x7fdd340d2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd340ddf00 0x7fdd34035ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd34034200 0x7fdd340d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd34034200 0x7fdd340d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42220/0 (socket says 172.21.15.46:42220) 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3b7fe700 1 -- 172.21.15.46:0/4009218392 learned_addr learned my addr 172.21.15.46:0/4009218392 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34035a30 con 0x7fdd340ddf00 2023-10-06T11:18:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34034e20 con 0x7fdd34052320 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34052320 0x7fdd340d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34035100 con 0x7fdd34034200 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd340ddf00 0x7fdd34035ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 -- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd34034200 msgr2=0x7fdd340d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd34034200 0x7fdd340d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 -- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd340ddf00 msgr2=0x7fdd34035ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd340ddf00 0x7fdd34035ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30009420 con 0x7fdd34052320 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd340ddf00 0x7fdd34035ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd3affd700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34052320 0x7fdd340d2740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c04ccb0 tx=0x7fdd3c066e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c06f2b0 con 0x7fdd34052320 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c06f430 con 0x7fdd34052320 2023-10-06T11:18:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd340353e0 con 0x7fdd34052320 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c08f320 con 0x7fdd34052320 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd340d1a80 con 0x7fdd34052320 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a58064590 con 0x7f4a580ad920 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 0x7f1fbc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc0c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 0x7f1fbc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 0x7f1fbc1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33094/0 (socket says 172.21.15.46:33094) 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fb9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc0c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fb9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc0c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52546/0 (socket says 172.21.15.46:52546) 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fba59c700 1 -- 172.21.15.46:0/1913598484 learned_addr learned my addr 172.21.15.46:0/1913598484 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc159350 0x7f1fbc0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0c4990 con 0x7f1fbc1527e0 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0c4c70 con 0x7f1fbc0c7ec0 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbb59e700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0c4f50 con 0x7f1fbc159350 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc159350 0x7f1fbc0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f9c16d45700 1 Processor -- start 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 -- start start 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 -- 172.21.15.46:0/3158644190 shutdown_connections 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3158644190 >> v2:172.21.15.46:3300/0 conn(0x7f49a80b1300 0x7f49a80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3158644190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a8159290 0x7f49a80b2f10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 -- 172.21.15.46:0/3158644190 >> 172.21.15.46:0/3158644190 conn(0x7f49a806ef60 msgr2=0x7f49a80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 -- 172.21.15.46:0/3158644190 shutdown_connections 2023-10-06T11:18:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f49a7fff700 1 -- 172.21.15.46:0/3158644190 wait complete. 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 -- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 msgr2=0x7f1fbc1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 0x7f1fbc1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 -- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc1527e0 msgr2=0x7f1fbc0c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc1527e0 0x7f1fbc0c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb400b850 con 0x7f1fbc159350 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fba59c700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0c7ec0 0x7f1fbc1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7fdd257fa700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd14000fd0 con 0x7fdd34052320 2023-10-06T11:18:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.262+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4a480147f0 con 0x7f4a580ad920 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4f7fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4a380606a0 0x7f4a38062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a4807da30 con 0x7f4a580ad920 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4f7fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f4a38068f50 0x7f4a3806b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a3806ba10 con 0x7f4a38068f50 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4807dd60 con 0x7f4a580ad920 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1fbad9d700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc159350 0x7f1fbc0c42f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c880 tx=0x7f1fb0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0011600 con 0x7f1fbc159350 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb0011760 con 0x7f1fbc159350 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0011b20 con 0x7f1fbc159350 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0011590 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e0011710 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a4e7fc700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4a380606a0 0x7f4a38062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f4a577fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f4a38068f50 0x7f4a3806b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c080500a0 0x7f9c080f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080e66b0 0x7f9c080e9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c080ea280 0x7f9c08035e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08034b70 con 0x7f9c080500a0 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080359d0 con 0x7f9c080ea280 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c16d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080f78a0 con 0x7f9c080e66b0 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c080ea280 0x7f9c08035e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c0ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080e66b0 0x7f9c080e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c0ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080e66b0 0x7f9c080e9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42244/0 (socket says 172.21.15.46:42244) 2023-10-06T11:18:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c0ffff700 1 -- 172.21.15.46:0/1114758079 learned_addr learned my addr 172.21.15.46:0/1114758079 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f49a7fff700 1 Processor -- start 2023-10-06T11:18:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdd3c062d50 con 0x7fdd34052320 2023-10-06T11:18:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9a2cfc3700 1 Processor -- start 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 -- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080e66b0 msgr2=0x7f9c080e9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c080e66b0 0x7f9c080e9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 -- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c080500a0 msgr2=0x7f9c080f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c080500a0 0x7f9c080f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9c152e2700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040094f0 con 0x7f9c080ea280 2023-10-06T11:18:30.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f49a7fff700 1 -- start start 2023-10-06T11:18:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f9a2cfc3700 1 -- start start 2023-10-06T11:18:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd20060600 0x7fdd20062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9c152e2700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c080ea280 0x7f9c08035e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800d660 tx=0x7f9c04017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd3c0f3210 con 0x7fdd34052320 2023-10-06T11:18:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f04f40ae700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec0acb10 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f04f40ae700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec0ac160 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.266+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0011ad0 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04c8000fd0 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fdd20068eb0 0x7fdd2006b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd2006b970 con 0x7fdd20068eb0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd3b7fe700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd20060600 0x7fdd20062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd3c0f57b0 con 0x7fdd34052320 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fdd20068eb0 0x7fdd2006b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd3b7fe700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd20060600 0x7fdd20062ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007fd0 tx=0x7fdd28007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd3bfff700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fdd20068eb0 0x7fdd2006b2c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f4a577fe700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f4a38068f50 0x7f4a3806b360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f4a4e7fc700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4a380606a0 0x7f4a38062b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806a230 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== osd.3 v2:172.21.15.46:6806/33080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4a3806ba10 con 0x7f4a38068f50 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a28067760 0x7f9a280b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bedd0 0x7f9a280be310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a280bf1a0 0x7f9a2816e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bedd0 0x7f9a280be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bedd0 0x7f9a280be310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33104/0 (socket says 172.21.15.46:33104) 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a277fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a28067760 0x7f9a280b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a277fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a28067760 0x7f9a280b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42252/0 (socket says 172.21.15.46:42252) 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a26ffd700 1 -- 172.21.15.46:0/2928993552 learned_addr learned my addr 172.21.15.46:0/2928993552 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280b0e40 con 0x7f9a280bf1a0 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280bb6f0 con 0x7f9a280bedd0 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400b860 con 0x7f9c080ea280 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0401a630 con 0x7f9c080ea280 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0401a9d0 con 0x7f9c080ea280 2023-10-06T11:18:30.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280baae0 con 0x7f9a28067760 2023-10-06T11:18:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f1fbb59e700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc0c5230 con 0x7f1fbc159350 2023-10-06T11:18:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a280bf1a0 0x7f9a2816e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== osd.2 v2:172.21.15.46:6800/33077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdd2006b970 con 0x7fdd20068eb0 2023-10-06T11:18:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f1fbb59e700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc0b1700 con 0x7f1fbc159350 2023-10-06T11:18:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f04e001b3b0 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fbc064590 con 0x7f1fbc159350 2023-10-06T11:18:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 -- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a28067760 msgr2=0x7f9a280b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a28067760 0x7f9a280b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 -- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bedd0 msgr2=0x7f9a280be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a280bedd0 0x7f9a280be310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.270+0000 7f9a27fff700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200094f0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9a27fff700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a280bf1a0 0x7f9a2816e060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a280cca40 tx=0x7f9a20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000b860 con 0x7f9a280bf1a0 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2001f070 con 0x7f9a280bf1a0 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2001a870 con 0x7f9a280bf1a0 2023-10-06T11:18:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a280badc0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08034dc0 con 0x7f9c080ea280 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08035380 con 0x7f9c080ea280 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a80b1300 0x7f49a816a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a80d07b0 0x7f49a809acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a8159290 0x7f49a809b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a816a6f0 con 0x7f49a8159290 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a809ca90 con 0x7f49a80b1300 2023-10-06T11:18:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f49a7fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a809cd70 con 0x7f49a80d07b0 2023-10-06T11:18:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1fb00193b0 con 0x7f1fbc159350 2023-10-06T11:18:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9a2001ac60 con 0x7f9a280bf1a0 2023-10-06T11:18:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1f9c060620 0x7f1f9c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.274+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c08005390 con 0x7f9c080ea280 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c040253f0 con 0x7f9c080ea280 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0cff9700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9bfc0606a0 0x7f9bfc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a80b1300 0x7f49a816a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a80b1300 0x7f49a816a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33116/0 (socket says 172.21.15.46:33116) 2023-10-06T11:18:30.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a6ffd700 1 -- 172.21.15.46:0/3902312551 learned_addr learned my addr 172.21.15.46:0/3902312551 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1fb007c760 con 0x7f1fbc159350 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1fba59c700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1f9c060620 0x7f1f9c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7f1f9c068ed0 0x7f1f9c06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f9c06b990 con 0x7f1f9c068ed0 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb007ecc0 con 0x7f1fbc159350 2023-10-06T11:18:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a8159290 0x7f49a809b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f04ddffb700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04d0060620 0x7f04d0062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f04e007bce0 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1fb9d9b700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7f1f9c068ed0 0x7f1f9c06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c0407e450 con 0x7f9c080ea280 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0cff9700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f9bfc068ed0 0x7f9bfc06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bfc06b990 con 0x7f9bfc068ed0 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 -- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a80d07b0 msgr2=0x7f49a809acc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a80d07b0 0x7f49a809acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 -- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a80b1300 msgr2=0x7f49a816a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a80b1300 0x7f49a816a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f49a77fe700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980094f0 con 0x7f49a8159290 2023-10-06T11:18:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f04f1e4a700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04d0060620 0x7f04d0062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c14ae1700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9bfc0606a0 0x7f9bfc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f9c0ffff700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f9bfc068ed0 0x7f9bfc06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.278+0000 7f1fba59c700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1f9c060620 0x7f1f9c062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400bfd0 tx=0x7f1fb401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a1f7fe700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a0c060620 0x7f9a0c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04ddffb700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f04d0068d60 0x7f04d006b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04d006b820 con 0x7f04d0068d60 2023-10-06T11:18:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04e0081070 con 0x7f04ec0c6aa0 2023-10-06T11:18:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f49a77fe700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a8159290 0x7f49a809b200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04f1e4a700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04d0060620 0x7f04d0062ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04e8003080 tx=0x7f04e8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a277fe700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a0c060620 0x7f9a0c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04f264b700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f04d0068d60 0x7f04d006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9c14ae1700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9bfc0606a0 0x7f9bfc062b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c10060ba0 tx=0x7f9c10064cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9c0ffff700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f9bfc068ed0 0x7f9bfc06b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f1fb9d9b700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7f1f9c068ed0 0x7f1f9c06b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c0404e8c0 con 0x7f9c080ea280 2023-10-06T11:18:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000c7e0 con 0x7f49a8159290 2023-10-06T11:18:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000c960 con 0x7f49a8159290 2023-10-06T11:18:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000cd20 con 0x7f49a8159290 2023-10-06T11:18:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a277fe700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a0c060620 0x7f9a0c062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a18003a80 tx=0x7f9a18007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f04f264b700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f04d0068d60 0x7f04d006b170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a2cfc3700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a280bb380 con 0x7f9a280bf1a0 2023-10-06T11:18:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a809d050 con 0x7f49a8159290 2023-10-06T11:18:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a809d610 con 0x7f49a8159290 2023-10-06T11:18:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== osd.6 v2:172.21.15.132:6811/33424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9bfc06b990 con 0x7f9bfc068ed0 2023-10-06T11:18:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a04000fd0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9a20021070 con 0x7f9a280bf1a0 2023-10-06T11:18:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f498cff9700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49a8064590 con 0x7f49a8159290 2023-10-06T11:18:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.282+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== osd.4 v2:172.21.15.132:6800/33423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1f9c06b990 con 0x7f1f9c068ed0 2023-10-06T11:18:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== osd.1 v2:172.21.15.46:6810/33081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f04d006b820 con 0x7f04d0068d60 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a1f7fe700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f9a0c068dd0 0x7f9a0c06b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a0c06b890 con 0x7f9a0c068dd0 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a2004e4b0 con 0x7f9a280bf1a0 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a26ffd700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f9a0c068dd0 0x7f9a0c06b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a26ffd700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f9a0c068dd0 0x7f9a0c06b1e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.286+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== osd.5 v2:172.21.15.132:6809/33426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9a0c06b890 con 0x7f9a0c068dd0 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f49a0014620 con 0x7f49a8159290 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49940606a0 0x7f4994062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f49a007c850 con 0x7f49a8159290 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f4994068f50 0x7f499406b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f499406ba10 con 0x7f4994068f50 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49a007ed90 con 0x7f49a8159290 2023-10-06T11:18:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7f49a6ffd700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49940606a0 0x7f4994062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.290+0000 7fb1eb86e700 1 -- 172.21.15.46:0/2120400910 --> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1c4002d30 con 0x7fb1c8068f50 2023-10-06T11:18:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7f49a6ffd700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49940606a0 0x7f4994062b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4998003080 tx=0x7f4998016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1d97fa700 1 -- 172.21.15.46:0/2120400910 <== osd.0 v2:172.21.15.46:6824/33078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb1c4002d30 con 0x7fb1c8068f50 2023-10-06T11:18:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7fb1c8068f50 msgr2=0x7fb1c806b360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7fb1c8068f50 0x7fb1c806b360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7f49a67fc700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f4994068f50 0x7f499406b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1c80606a0 msgr2=0x7fb1c8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1c80606a0 0x7fb1c8062b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0013d0 tx=0x7fb1dc006040 comp rx=0 tx=0).stop 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 msgr2=0x7fb1e414cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 --2- 172.21.15.46:0/2120400910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e40af100 0x7fb1e414cbb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0014890 tx=0x7fb1e001a000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1e9e0b700 1 -- 172.21.15.46:0/2120400910 reap_dead start 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 shutdown_connections 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 >> 172.21.15.46:0/2120400910 conn(0x7fb1e40cc6b0 msgr2=0x7fb1e41581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 shutdown_connections 2023-10-06T11:18:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7f49a67fc700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f4994068f50 0x7f499406b360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7fb1ceffd700 1 -- 172.21.15.46:0/2120400910 wait complete. 2023-10-06T11:18:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.294+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== osd.7 v2:172.21.15.132:6808/33425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f499406ba10 con 0x7f4994068f50 2023-10-06T11:18:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.302+0000 7fdd257fa700 1 -- 172.21.15.46:0/4009218392 --> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd14002d30 con 0x7fdd20068eb0 2023-10-06T11:18:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.302+0000 7f4a5d8ae700 1 -- 172.21.15.46:0/2006077442 --> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a58063fe0 con 0x7f4a38068f50 2023-10-06T11:18:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4f7fe700 1 -- 172.21.15.46:0/2006077442 <== osd.3 v2:172.21.15.46:6806/33080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a58063fe0 con 0x7f4a38068f50 2023-10-06T11:18:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd277fe700 1 -- 172.21.15.46:0/4009218392 <== osd.2 v2:172.21.15.46:6800/33077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdd14002d30 con 0x7fdd20068eb0 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fdd20068eb0 msgr2=0x7fdd2006b2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f4a38068f50 msgr2=0x7f4a3806b360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f4a38068f50 0x7f4a3806b360 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4a380606a0 msgr2=0x7f4a38062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4a380606a0 0x7f4a38062b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a5806a230 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 msgr2=0x7f4a581760f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 --2- 172.21.15.46:0/2006077442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a580ad920 0x7f4a581760f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009ef0 tx=0x7f4a4800b040 comp rx=0 tx=0).stop 2023-10-06T11:18:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a577fe700 1 -- 172.21.15.46:0/2006077442 reap_dead start 2023-10-06T11:18:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fdd20068eb0 0x7fdd2006b2c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 shutdown_connections 2023-10-06T11:18:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 >> 172.21.15.46:0/2006077442 conn(0x7f4a580cc6b0 msgr2=0x7f4a580cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 shutdown_connections 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd20060600 msgr2=0x7fdd20062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd20060600 0x7fdd20062ab0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007fd0 tx=0x7fdd28007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34052320 msgr2=0x7fdd340d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 --2- 172.21.15.46:0/4009218392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34052320 0x7fdd340d2740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c04ccb0 tx=0x7fdd3c066e50 comp rx=0 tx=0).stop 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd3bfff700 1 -- 172.21.15.46:0/4009218392 reap_dead start 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 shutdown_connections 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 >> 172.21.15.46:0/4009218392 conn(0x7fdd34051810 msgr2=0x7fdd34051be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 shutdown_connections 2023-10-06T11:18:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7f4a4cff9700 1 -- 172.21.15.46:0/2006077442 wait complete. 2023-10-06T11:18:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.306+0000 7fdd40b63700 1 -- 172.21.15.46:0/4009218392 wait complete. 2023-10-06T11:18:30.338 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-10-06T11:18:30.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:18:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 --> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1fbc0c5090 con 0x7f1f9c068ed0 2023-10-06T11:18:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 --> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c08053810 con 0x7f9bfc068ed0 2023-10-06T11:18:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f1faa7fc700 1 -- 172.21.15.46:0/1913598484 <== osd.4 v2:172.21.15.132:6800/33423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1fbc0c5090 con 0x7f1f9c068ed0 2023-10-06T11:18:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f9c0cff9700 1 -- 172.21.15.46:0/1114758079 <== osd.6 v2:172.21.15.132:6811/33424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c08053810 con 0x7f9bfc068ed0 2023-10-06T11:18:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7f1f9c068ed0 msgr2=0x7f1f9c06b2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f1f9bfff700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7f1f9c068ed0 0x7f1f9c06b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.314+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f9bfc068ed0 msgr2=0x7f9bfc06b2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f9bfc068ed0 0x7f9bfc06b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1f9c060620 msgr2=0x7f1f9c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1f9c060620 0x7f1f9c062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400bfd0 tx=0x7f1fb401a000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9bfc0606a0 msgr2=0x7f9bfc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9bfc0606a0 0x7f9bfc062b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c10060ba0 tx=0x7f9c10064cd0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c080ea280 msgr2=0x7f9c08035e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc159350 msgr2=0x7f1fbc0c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 --2- 172.21.15.46:0/1913598484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc159350 0x7f1fbc0c42f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c880 tx=0x7f1fb0008040 comp rx=0 tx=0).stop 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1fbad9d700 1 -- 172.21.15.46:0/1913598484 reap_dead start 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 --2- 172.21.15.46:0/1114758079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c080ea280 0x7f9c08035e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800d660 tx=0x7f9c04017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c152e2700 1 -- 172.21.15.46:0/1114758079 reap_dead start 2023-10-06T11:18:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 shutdown_connections 2023-10-06T11:18:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 >> 172.21.15.46:0/1913598484 conn(0x7f1fbc0cbbb0 msgr2=0x7f1fbc0c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 shutdown_connections 2023-10-06T11:18:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 >> 172.21.15.46:0/1114758079 conn(0x7f9c08054c90 msgr2=0x7f9c08055060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 shutdown_connections 2023-10-06T11:18:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9c16d45700 1 -- 172.21.15.46:0/1114758079 wait complete. 2023-10-06T11:18:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 shutdown_connections 2023-10-06T11:18:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f1f9bfff700 1 -- 172.21.15.46:0/1913598484 wait complete. 2023-10-06T11:18:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 --> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a04002d30 con 0x7f9a0c068dd0 2023-10-06T11:18:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1f7fe700 1 -- 172.21.15.46:0/2928993552 <== osd.5 v2:172.21.15.132:6809/33426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a04002d30 con 0x7f9a0c068dd0 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f9a0c068dd0 msgr2=0x7f9a0c06b1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f9a0c068dd0 0x7f9a0c06b1e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a0c060620 msgr2=0x7f9a0c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a0c060620 0x7f9a0c062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a18003a80 tx=0x7f9a18007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a280bf1a0 msgr2=0x7f9a2816e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 --2- 172.21.15.46:0/2928993552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a280bf1a0 0x7f9a2816e060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9a280cca40 tx=0x7f9a20017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a27fff700 1 -- 172.21.15.46:0/2928993552 reap_dead start 2023-10-06T11:18:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 --> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04c8002d30 con 0x7f04d0068d60 2023-10-06T11:18:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 shutdown_connections 2023-10-06T11:18:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.318+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 >> 172.21.15.46:0/2928993552 conn(0x7f9a28062ef0 msgr2=0x7f9a280c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 shutdown_connections 2023-10-06T11:18:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f9a1d7fa700 1 -- 172.21.15.46:0/2928993552 wait complete. 2023-10-06T11:18:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04ddffb700 1 -- 172.21.15.46:0/165751037 <== osd.1 v2:172.21.15.46:6810/33081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04c8002d30 con 0x7f04d0068d60 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f04d0068d60 msgr2=0x7f04d006b170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f04d0068d60 0x7f04d006b170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04d0060620 msgr2=0x7f04d0062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04d0060620 0x7f04d0062ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04e8003080 tx=0x7f04e8017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 msgr2=0x7f04ec0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 --2- 172.21.15.46:0/165751037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0c6aa0 0x7f04ec0ad730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c490 tx=0x7f04e0007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04f264b700 1 -- 172.21.15.46:0/165751037 reap_dead start 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 shutdown_connections 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 >> 172.21.15.46:0/165751037 conn(0x7f04ec0cc6b0 msgr2=0x7f04ec15b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 shutdown_connections 2023-10-06T11:18:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f04cf7fe700 1 -- 172.21.15.46:0/165751037 wait complete. 2023-10-06T11:18:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.322+0000 7f498cff9700 1 -- 172.21.15.46:0/3902312551 --> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49a809ab50 con 0x7f4994068f50 2023-10-06T11:18:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.326+0000 7f498effd700 1 -- 172.21.15.46:0/3902312551 <== osd.7 v2:172.21.15.132:6808/33425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49a809ab50 con 0x7f4994068f50 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.326+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f4994068f50 msgr2=0x7f499406b360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.326+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f4994068f50 0x7f499406b360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49940606a0 msgr2=0x7f4994062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49940606a0 0x7f4994062b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4998003080 tx=0x7f4998016000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a8159290 msgr2=0x7f49a809b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 --2- 172.21.15.46:0/3902312551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49a8159290 0x7f49a809b200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a0009780 comp rx=0 tx=0).stop 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a77fe700 1 -- 172.21.15.46:0/3902312551 reap_dead start 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 shutdown_connections 2023-10-06T11:18:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 >> 172.21.15.46:0/3902312551 conn(0x7f49a806ef60 msgr2=0x7f49a8140a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 shutdown_connections 2023-10-06T11:18:30.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.330+0000 7f49a7fff700 1 -- 172.21.15.46:0/3902312551 wait complete. 2023-10-06T11:18:30.351 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:18:30.352 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-10-06T11:18:30.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:18:30.353 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-10-06T11:18:30.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:18:30.354 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:18:30.355 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:18:30.356 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:18:30.357 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:18:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.550+0000 7fae68d45700 1 Processor -- start 2023-10-06T11:18:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.550+0000 7fae68d45700 1 -- start start 2023-10-06T11:18:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae68d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae68d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c0eaad0 con 0x7fae5c0e66b0 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae68d45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c054140 con 0x7fae5c0ea280 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae68d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0505e0 con 0x7fae5c0ea650 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae62d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae61d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fae5c0e66b0 0x7fae5c034910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52256/0 (socket says 172.21.15.46:52256) 2023-10-06T11:18:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae61d9b700 1 -- 172.21.15.46:0/2307243618 learned_addr learned my addr 172.21.15.46:0/2307243618 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98925820 0 0) 0x7fae5c0eaad0 con 0x7fae5c0e66b0 2023-10-06T11:18:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae4c003570 con 0x7fae5c0e66b0 2023-10-06T11:18:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191690876 0 0) 0x7fae5c054140 con 0x7fae5c0ea280 2023-10-06T11:18:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae62d9d700 1 -- 172.21.15.46:0/2307243618 >> v1:172.21.15.132:6789/0 conn(0x7fae5c0ea280 legacy=0x7fae5c0e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae62d9d700 1 -- 172.21.15.46:0/2307243618 >> v1:172.21.15.46:6789/0 conn(0x7fae5c0e66b0 legacy=0x7fae5c034910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae62d9d700 1 -- 172.21.15.46:0/2307243618 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c053530 con 0x7fae5c0ea650 2023-10-06T11:18:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae62d9d700 1 --2- 172.21.15.46:0/2307243618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0500a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae6406fbe0 tx=0x7fae64065a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10cd28ffd7492ce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6404be60 con 0x7fae5c0ea650 2023-10-06T11:18:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64063f90 con 0x7fae5c0ea650 2023-10-06T11:18:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.554+0000 7fae6159a700 1 -- 172.21.15.46:0/2307243618 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64064120 con 0x7fae5c0ea650 2023-10-06T11:18:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- 172.21.15.46:0/2307243618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 msgr2=0x7fae5c0500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 --2- 172.21.15.46:0/2307243618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0500a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae6406fbe0 tx=0x7fae64065a20 comp rx=0 tx=0).stop 2023-10-06T11:18:30.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- 172.21.15.46:0/2307243618 shutdown_connections 2023-10-06T11:18:30.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 --2- 172.21.15.46:0/2307243618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0500a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- 172.21.15.46:0/2307243618 >> 172.21.15.46:0/2307243618 conn(0x7fae5c054c90 msgr2=0x7fae5c055090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 Processor -- start 2023-10-06T11:18:30.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 -- start start 2023-10-06T11:18:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- 172.21.15.46:0/2307243618 shutdown_connections 2023-10-06T11:18:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- 172.21.15.46:0/2307243618 wait complete. 2023-10-06T11:18:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 Processor -- start 2023-10-06T11:18:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7fae68d45700 1 -- start start 2023-10-06T11:18:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ec8104060 0x7f7ec8006940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec80537f0 con 0x7f7ec804fc90 2023-10-06T11:18:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec80076d0 con 0x7f7ec8101520 2023-10-06T11:18:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed6e78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec80531a0 con 0x7f7ec8104060 2023-10-06T11:18:30.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae5c0e66b0 0x7fae5c0385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0faac0 con 0x7fae5c0ea280 2023-10-06T11:18:30.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c038af0 con 0x7fae5c0e66b0 2023-10-06T11:18:30.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0f9f40 con 0x7fae5c0ea650 2023-10-06T11:18:30.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52582/0 (socket says 172.21.15.46:52582) 2023-10-06T11:18:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae61d9b700 1 -- 172.21.15.46:0/2205990414 learned_addr learned my addr 172.21.15.46:0/2205990414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae6259c700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae5c0e66b0 0x7fae5c0385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed4c14700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ec804fc90 0x7f7ec8103950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52258/0 (socket says 172.21.15.46:52258) 2023-10-06T11:18:30.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.558+0000 7f7ed4c14700 1 -- 172.21.15.46:0/723987071 learned_addr learned my addr 172.21.15.46:0/723987071 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959690142 0 0) 0x7f7ec80537f0 con 0x7f7ec804fc90 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec0003570 con 0x7f7ec804fc90 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850630641 0 0) 0x7f7ec80076d0 con 0x7f7ec8101520 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec80537f0 con 0x7f7ec8101520 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950193434 0 0) 0x7f7ec0003570 con 0x7f7ec804fc90 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 >> v2:172.21.15.132:3300/0 conn(0x7f7ec8104060 msgr2=0x7f7ec8006940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 --2- 172.21.15.46:0/723987071 >> v2:172.21.15.132:3300/0 conn(0x7f7ec8104060 0x7f7ec8006940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 >> v1:172.21.15.132:6789/0 conn(0x7f7ec8101520 legacy=0x7f7ec8006e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8053480 con 0x7f7ec804fc90 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f7ec4001440 con 0x7f7ec804fc90 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 >> v1:172.21.15.46:6789/0 conn(0x7f7ec804fc90 legacy=0x7f7ec8103950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0003bf0 0x7f7ec0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec0004810 0x7f7ec0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 0x7f7ec000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80531a0 con 0x7f7ec0004810 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000c160 con 0x7f7ec0003bf0 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000c320 con 0x7f7ec0005230 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 0x7f7ec000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed4c14700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0003bf0 0x7f7ec0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecffff700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec0004810 0x7f7ec0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 -- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae5c0e66b0 msgr2=0x7fae5c0385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae5c0e66b0 0x7fae5c0385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 -- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 msgr2=0x7fae5c04de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 -- 172.21.15.46:0/2205990414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64062b60 con 0x7fae5c0ea650 2023-10-06T11:18:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae61d9b700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae62d9d700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0dcf10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae64060ba0 tx=0x7fae6406b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 -- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0003bf0 msgr2=0x7f7ec0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0003bf0 0x7f7ec0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 -- 172.21.15.46:0/723987071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec0004810 msgr2=0x7f7ec0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec0004810 0x7f7ec0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 -- 172.21.15.46:0/723987071 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8053480 con 0x7f7ec0005230 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 -- 172.21.15.46:0/723987071 reap_dead start 2023-10-06T11:18:30.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ecffff700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec0004810 0x7f7ec0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64099070 con 0x7fae5c0ea650 2023-10-06T11:18:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64094e40 con 0x7fae5c0ea650 2023-10-06T11:18:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64094290 con 0x7fae5c0ea650 2023-10-06T11:18:30.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7fae68d45700 1 -- 172.21.15.46:0/2205990414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c0fa220 con 0x7fae5c0ea650 2023-10-06T11:18:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.562+0000 7f7ed5415700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 0x7f7ec000ba60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00656c0 tx=0x7f7ed0065a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9939b3ec51c27094 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0063f90 con 0x7f7ec0005230 2023-10-06T11:18:30.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed00640f0 con 0x7f7ec0005230 2023-10-06T11:18:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ecf7fe700 1 -- 172.21.15.46:0/723987071 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00642a0 con 0x7f7ec0005230 2023-10-06T11:18:30.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7fae68d45700 1 -- 172.21.15.46:0/2205990414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c0fa7e0 con 0x7fae5c0ea650 2023-10-06T11:18:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 -- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 msgr2=0x7f7ec000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 0x7f7ec000ba60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00656c0 tx=0x7f7ed0065a20 comp rx=0 tx=0).stop 2023-10-06T11:18:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 -- 172.21.15.46:0/723987071 shutdown_connections 2023-10-06T11:18:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/723987071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0005230 0x7f7ec000ba60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 -- 172.21.15.46:0/723987071 >> 172.21.15.46:0/723987071 conn(0x7f7ec8051700 msgr2=0x7f7ec8051b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 -- 172.21.15.46:0/723987071 shutdown_connections 2023-10-06T11:18:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 -- 172.21.15.46:0/723987071 wait complete. 2023-10-06T11:18:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fae64062d50 con 0x7fae5c0ea650 2023-10-06T11:18:30.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.566+0000 7f7ed6e78700 1 Processor -- start 2023-10-06T11:18:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 -- start start 2023-10-06T11:18:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7fae527fc700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 0x7fae44062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae640f4a90 con 0x7fae5c0ea650 2023-10-06T11:18:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7fae68d45700 1 -- 172.21.15.46:0/2205990414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c0349d0 con 0x7fae5c0ea650 2023-10-06T11:18:30.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 Processor -- start 2023-10-06T11:18:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7fae6259c700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 0x7fae44062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 0x7f7ec80fb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 -- start start 2023-10-06T11:18:30.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 0x7f7ec8038210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7fae6259c700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 0x7fae44062b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae54003a90 tx=0x7fae54007da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52604/0 (socket says 172.21.15.46:52604) 2023-10-06T11:18:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed4c14700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 0x7f7ec80fb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed4c14700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 0x7f7ec80fb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33150/0 (socket says 172.21.15.46:33150) 2023-10-06T11:18:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f122b59e700 1 Processor -- start 2023-10-06T11:18:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ecffff700 1 -- 172.21.15.46:0/3155767780 learned_addr learned my addr 172.21.15.46:0/3155767780 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80356e0 con 0x7f7ec80fbee0 2023-10-06T11:18:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 0x7f283c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 0x7f283c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80fd910 con 0x7f7ec0005230 2023-10-06T11:18:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec8036370 con 0x7f7ec803a620 2023-10-06T11:18:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f7ed5415700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 0x7f7ec8038210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283c0ae340 con 0x7f283c0b1130 2023-10-06T11:18:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283c0b15b0 con 0x7f283c159290 2023-10-06T11:18:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f283b7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 0x7f283c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f283b7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 0x7f283c0b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33166/0 (socket says 172.21.15.46:33166) 2023-10-06T11:18:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f283b7fe700 1 -- 172.21.15.46:0/197367699 learned_addr learned my addr 172.21.15.46:0/197367699 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c0b09a0 con 0x7f283c0d07b0 2023-10-06T11:18:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 --2- 172.21.15.46:0/197367699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 0x7f283c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.570+0000 7f122b59e700 1 -- start start 2023-10-06T11:18:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 -- 172.21.15.46:0/197367699 >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 msgr2=0x7f283c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 --2- 172.21.15.46:0/197367699 >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 0x7f283c0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 -- 172.21.15.46:0/197367699 >> v1:172.21.15.46:6789/0 conn(0x7f283c0b1130 legacy=0x7f283c0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 -- 172.21.15.46:0/197367699 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c159660 con 0x7f283c0d07b0 2023-10-06T11:18:30.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 -- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 msgr2=0x7f7ec80fb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 0x7f7ec80fb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 -- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 msgr2=0x7f7ec803a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8036650 con 0x7f7ec803a620 2023-10-06T11:18:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ecffff700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f7ed5415700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 0x7f7ec8038210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ed004cd10 tx=0x7f7ed006a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f2833fff700 1 --2- 172.21.15.46:0/197367699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 0x7f283c0b0bf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098e0 tx=0x7f282c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727fcd337ef46914 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f122c0adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0b2fd0 con 0x7f122c15d480 2023-10-06T11:18:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0b17b0 con 0x7f122c0b4450 2023-10-06T11:18:30.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0b0ba0 con 0x7f122c0b5af0 2023-10-06T11:18:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f122c15d480 0x7f122c0b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52280/0 (socket says 172.21.15.46:52280) 2023-10-06T11:18:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122a59c700 1 -- 172.21.15.46:0/2195597095 learned_addr learned my addr 172.21.15.46:0/2195597095 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367222835 0 0) 0x7f122c0b2fd0 con 0x7f122c15d480 2023-10-06T11:18:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1214003570 con 0x7f122c15d480 2023-10-06T11:18:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7faeb9923700 1 Processor -- start 2023-10-06T11:18:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066056731 0 0) 0x7f122c0b17b0 con 0x7f122c0b4450 2023-10-06T11:18:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f122c0b2fd0 con 0x7f122c0b4450 2023-10-06T11:18:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4209390967 0 0) 0x7f1214003570 con 0x7f122c15d480 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 msgr2=0x7f122c0adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f1229d9b700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f122c0adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f122c0adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 >> v1:172.21.15.132:6789/0 conn(0x7f122c0b4450 legacy=0x7f122c0ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0b0e80 con 0x7f122c15d480 2023-10-06T11:18:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f121c0013f0 con 0x7f122c15d480 2023-10-06T11:18:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7faeb9923700 1 -- start start 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 >> v1:172.21.15.46:6789/0 conn(0x7f122c15d480 legacy=0x7f122c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f1214003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140041d0 0x7f1214004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1214004f50 0x7f121400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c0b0ba0 con 0x7f12140041d0 2023-10-06T11:18:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1214005600 con 0x7f1214004f50 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121400bfa0 con 0x7f122c0b5af0 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.574+0000 7f122a59c700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f1214003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f1229d9b700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140041d0 0x7f1214004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122ad9d700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1214004f50 0x7f121400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 -- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1214004f50 msgr2=0x7f121400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1214004f50 0x7f121400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 -- 172.21.15.46:0/2195597095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140041d0 msgr2=0x7f1214004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140041d0 0x7f1214004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 -- 172.21.15.46:0/2195597095 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214003570 con 0x7f122c0b5af0 2023-10-06T11:18:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f1229d9b700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12140041d0 0x7f1214004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122a59c700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f1214003c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f121c00c160 tx=0x7f121c001cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8f8e768365fd693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f8c089c4700 1 Processor -- start 2023-10-06T11:18:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb9923700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb4159290 0x7faeb40b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f8c089c4700 1 -- start start 2023-10-06T11:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7fe07c86d700 1 Processor -- start 2023-10-06T11:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7fe07c86d700 1 -- start start 2023-10-06T11:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0063f90 con 0x7f7ec803a620 2023-10-06T11:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0083e90 con 0x7f7ec803a620 2023-10-06T11:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0080690 con 0x7f7ec803a620 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb8921700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb4159290 0x7faeb40b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb8921700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faeb4159290 0x7faeb40b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52626/0 (socket says 172.21.15.46:52626) 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb8921700 1 -- 172.21.15.46:0/819962806 learned_addr learned my addr 172.21.15.46:0/819962806 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb9923700 1 --2- 172.21.15.46:0/819962806 >> v2:172.21.15.132:3300/0 conn(0x7faeb40d07b0 0x7faeb40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec8036930 con 0x7f7ec803a620 2023-10-06T11:18:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec8036ef0 con 0x7f7ec803a620 2023-10-06T11:18:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00ca50 con 0x7f122c0b5af0 2023-10-06T11:18:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c00cbb0 con 0x7f122c0b5af0 2023-10-06T11:18:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122959a700 1 -- 172.21.15.46:0/2195597095 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c005920 con 0x7f122c0b5af0 2023-10-06T11:18:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fae5c0349d0 con 0x7fae5c0ea650 2023-10-06T11:18:30.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb40ae340 con 0x7faeb40b1130 2023-10-06T11:18:30.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40b15b0 con 0x7faeb4159290 2023-10-06T11:18:30.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7faeb3fff700 1 --2- 172.21.15.46:0/819962806 >> v2:172.21.15.132:3300/0 conn(0x7faeb40d07b0 0x7faeb40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122b59e700 1 -- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 msgr2=0x7f1214003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.578+0000 7f122b59e700 1 --2- 172.21.15.46:0/2195597095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c0b5af0 0x7f1214003c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f121c00c160 tx=0x7f121c001cc0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f122ad9d700 1 -- 172.21.15.46:0/2195597095 reap_dead start 2023-10-06T11:18:30.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40b09a0 con 0x7faeb40d07b0 2023-10-06T11:18:30.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f122b59e700 1 -- 172.21.15.46:0/2195597095 shutdown_connections 2023-10-06T11:18:30.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f122b59e700 1 -- 172.21.15.46:0/2195597095 >> 172.21.15.46:0/2195597095 conn(0x7f122c06ef60 msgr2=0x7f122c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07c86d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0780af000 0x7fe0780cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07c86d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe0780cb4c0 0x7fe0780cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07c86d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0780cf100 con 0x7fe0780af3d0 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07c86d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780ce4f0 con 0x7fe0780af000 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07c86d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780ce7d0 con 0x7fe0780cb4c0 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb37fe700 1 -- 172.21.15.46:0/819962806 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049998131 0 0) 0x7faeb40ae340 con 0x7faeb40b1130 2023-10-06T11:18:30.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb37fe700 1 -- 172.21.15.46:0/819962806 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae98003570 con 0x7faeb40b1130 2023-10-06T11:18:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0780af000 0x7fe0780cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0780af000 0x7fe0780cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52640/0 (socket says 172.21.15.46:52640) 2023-10-06T11:18:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 -- 172.21.15.46:0/1516735957 learned_addr learned my addr 172.21.15.46:0/1516735957 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe07659c700 1 --2- 172.21.15.46:0/1516735957 >> v2:172.21.15.132:3300/0 conn(0x7fe0780cb4c0 0x7fe0780cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 -- 172.21.15.46:0/1516735957 >> v2:172.21.15.132:3300/0 conn(0x7fe0780cb4c0 msgr2=0x7fe0780cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 --2- 172.21.15.46:0/1516735957 >> v2:172.21.15.132:3300/0 conn(0x7fe0780cb4c0 0x7fe0780cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f283affd700 1 -- 172.21.15.46:0/197367699 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00b6d0 con 0x7f283c0d07b0 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f283affd700 1 -- 172.21.15.46:0/197367699 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c00b830 con 0x7f283c0d07b0 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f283affd700 1 -- 172.21.15.46:0/197367699 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00bbd0 con 0x7f283c0d07b0 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7ed0062d50 con 0x7f7ec803a620 2023-10-06T11:18:30.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80fda80 con 0x7f7ec803a620 2023-10-06T11:18:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 -- 172.21.15.46:0/1516735957 >> v1:172.21.15.132:6789/0 conn(0x7fe0780af3d0 legacy=0x7fe0780690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 -- 172.21.15.46:0/1516735957 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780ceab0 con 0x7fe0780af000 2023-10-06T11:18:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f122b59e700 1 -- 172.21.15.46:0/2195597095 shutdown_connections 2023-10-06T11:18:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb8921700 1 -- 172.21.15.46:0/819962806 >> v2:172.21.15.132:3300/0 conn(0x7faeb40d07b0 msgr2=0x7faeb40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb8921700 1 --2- 172.21.15.46:0/819962806 >> v2:172.21.15.132:3300/0 conn(0x7faeb40d07b0 0x7faeb40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb8921700 1 -- 172.21.15.46:0/819962806 >> v1:172.21.15.46:6789/0 conn(0x7faeb40b1130 legacy=0x7faeb40ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7faeb8921700 1 -- 172.21.15.46:0/819962806 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4159660 con 0x7faeb4159290 2023-10-06T11:18:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 msgr2=0x7f283c0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 --2- 172.21.15.46:0/197367699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 0x7f283c0b0bf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098e0 tx=0x7f282c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7fe076d9d700 1 --2- 172.21.15.46:0/1516735957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780af000 0x7fe0780cf590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe078087fd0 tx=0x7fe068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bdc54fdcd6712b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c089c4700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8c040d07b0 0x7f8c040b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c089c4700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040ae340 con 0x7f8c040b1130 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 shutdown_connections 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 --2- 172.21.15.46:0/197367699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c0d07b0 0x7f283c0b0bf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 --2- 172.21.15.46:0/197367699 >> v2:172.21.15.132:3300/0 conn(0x7f283c159290 0x7f283c0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 >> 172.21.15.46:0/197367699 conn(0x7f283c06ef60 msgr2=0x7f283c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c089c4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040b09a0 con 0x7f8c04159290 2023-10-06T11:18:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 shutdown_connections 2023-10-06T11:18:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c089c4700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040b15b0 con 0x7f8c040d07b0 2023-10-06T11:18:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c02d9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8c04159290 0x7f8c040b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46326/0 (socket says 172.21.15.46:46326) 2023-10-06T11:18:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c02d9d700 1 -- 172.21.15.46:0/740126266 learned_addr learned my addr 172.21.15.46:0/740126266 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c0259c700 1 --2- 172.21.15.46:0/740126266 >> v2:172.21.15.46:3300/0 conn(0x7f8c040d07b0 0x7f8c040b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c0259c700 1 -- 172.21.15.46:0/740126266 >> v1:172.21.15.132:6789/0 conn(0x7f8c04159290 legacy=0x7f8c040b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f8c0259c700 1 -- 172.21.15.46:0/740126266 >> v1:172.21.15.46:6789/0 conn(0x7f8c040b1130 legacy=0x7f8c040ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe075d9b700 1 -- 172.21.15.46:0/1516735957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068009e70 con 0x7fe0780af000 2023-10-06T11:18:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe075d9b700 1 -- 172.21.15.46:0/1516735957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06800b860 con 0x7fe0780af000 2023-10-06T11:18:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe075d9b700 1 -- 172.21.15.46:0/1516735957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800bc00 con 0x7fe0780af000 2023-10-06T11:18:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c0259c700 1 -- 172.21.15.46:0/740126266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04159660 con 0x7f8c040d07b0 2023-10-06T11:18:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb8921700 1 --2- 172.21.15.46:0/819962806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb4159290 0x7faeb40b2f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faeb4064240 tx=0x7faeac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf876816e7a4c2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f7eccff9700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 0x7f7eb4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ed00e3250 con 0x7f7ec803a620 2023-10-06T11:18:30.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f7ed4c14700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 0x7f7eb4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f122b59e700 1 -- 172.21.15.46:0/2195597095 wait complete. 2023-10-06T11:18:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 Processor -- start 2023-10-06T11:18:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c0259c700 1 --2- 172.21.15.46:0/740126266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c040d07b0 0x7f8c040b0bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ffd0 tx=0x7f8bf800f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23cef43340ffb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 -- start start 2023-10-06T11:18:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb37fe700 1 -- 172.21.15.46:0/819962806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00e070 con 0x7faeb4159290 2023-10-06T11:18:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb37fe700 1 -- 172.21.15.46:0/819962806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac00b9c0 con 0x7faeb4159290 2023-10-06T11:18:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb37fe700 1 -- 172.21.15.46:0/819962806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0092e0 con 0x7faeb4159290 2023-10-06T11:18:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.582+0000 7f2840f23700 1 -- 172.21.15.46:0/197367699 wait complete. 2023-10-06T11:18:30.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f2840f23700 1 Processor -- start 2023-10-06T11:18:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f7ed4c14700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 0x7f7eb4062b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004890 tx=0x7f7ec40131e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c01d9b700 1 -- 172.21.15.46:0/740126266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8014ee0 con 0x7f8c040d07b0 2023-10-06T11:18:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 -- 172.21.15.46:0/1516735957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780af000 msgr2=0x7fe0780cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 --2- 172.21.15.46:0/1516735957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780af000 0x7fe0780cf590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe078087fd0 tx=0x7fe068009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 -- 172.21.15.46:0/1516735957 shutdown_connections 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 --2- 172.21.15.46:0/1516735957 >> v2:172.21.15.132:3300/0 conn(0x7fe0780cb4c0 0x7fe0780cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 --2- 172.21.15.46:0/1516735957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780af000 0x7fe0780cf590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7fe07c86d700 1 -- 172.21.15.46:0/1516735957 >> 172.21.15.46:0/1516735957 conn(0x7fe0780683b0 msgr2=0x7fe0780d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c01d9b700 1 -- 172.21.15.46:0/740126266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8015630 con 0x7f8c040d07b0 2023-10-06T11:18:30.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f2840f23700 1 -- start start 2023-10-06T11:18:30.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c01d9b700 1 -- 172.21.15.46:0/740126266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80159d0 con 0x7f8c040d07b0 2023-10-06T11:18:30.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb4159290 msgr2=0x7faeb40b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06a530 0x7f122c14ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 --2- 172.21.15.46:0/819962806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb4159290 0x7faeb40b2f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faeb4064240 tx=0x7faeac009840 comp rx=0 tx=0).stop 2023-10-06T11:18:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c15fbd0 0x7f122c179080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 shutdown_connections 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06d280 con 0x7f122c06a530 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06c670 con 0x7f122c0b5af0 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06c950 con 0x7f122c15fbd0 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 --2- 172.21.15.46:0/819962806 >> v2:172.21.15.132:3300/0 conn(0x7faeb40d07b0 0x7faeb40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 --2- 172.21.15.46:0/819962806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb4159290 0x7faeb40b2f10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 >> 172.21.15.46:0/819962806 conn(0x7faeb406ef60 msgr2=0x7faeb40d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33198/0 (socket says 172.21.15.46:33198) 2023-10-06T11:18:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122a59c700 1 -- 172.21.15.46:0/94219021 learned_addr learned my addr 172.21.15.46:0/94219021 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f1229d9b700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06a530 0x7f122c14ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 -- 172.21.15.46:0/740126266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c040d07b0 msgr2=0x7f8c040b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 --2- 172.21.15.46:0/740126266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c040d07b0 0x7f8c040b0bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ffd0 tx=0x7f8bf800f9c0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 -- 172.21.15.46:0/740126266 shutdown_connections 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 --2- 172.21.15.46:0/740126266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c040d07b0 0x7f8c040b0bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 -- 172.21.15.46:0/740126266 >> 172.21.15.46:0/740126266 conn(0x7f8c0406ef60 msgr2=0x7f8c040d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f8c089c4700 1 -- 172.21.15.46:0/740126266 shutdown_connections 2023-10-06T11:18:30.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.586+0000 7f122ad9d700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c15fbd0 0x7f122c179080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 shutdown_connections 2023-10-06T11:18:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7fe07c86d700 1 -- 172.21.15.46:0/1516735957 shutdown_connections 2023-10-06T11:18:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7faeb9923700 1 -- 172.21.15.46:0/819962806 wait complete. 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 0x7f283c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f283b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52666/0 (socket says 172.21.15.46:52666) 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f283b7fe700 1 -- 172.21.15.46:0/1131662416 learned_addr learned my addr 172.21.15.46:0/1131662416 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7fe07c86d700 1 -- 172.21.15.46:0/1516735957 wait complete. 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 -- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c15fbd0 msgr2=0x7f122c179080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c15fbd0 0x7f122c179080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 -- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06a530 msgr2=0x7f122c14ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06a530 0x7f122c14ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 -- 172.21.15.46:0/94219021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c06cc30 con 0x7f122c0b5af0 2023-10-06T11:18:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122a59c700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f121c011000 tx=0x7f121c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c159290 0x7f283c154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c13ff60 con 0x7f283c0b1130 2023-10-06T11:18:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0070d0 con 0x7f122c0b5af0 2023-10-06T11:18:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c007250 con 0x7f122c0b5af0 2023-10-06T11:18:30.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c06cf10 con 0x7f122c0b5af0 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 0x7f283c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c15e390 con 0x7f283c0d07b0 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2840f23700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c15ddd0 con 0x7f283c159290 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c007610 con 0x7f122c0b5af0 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c06c420 con 0x7f122c0b5af0 2023-10-06T11:18:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7faeb9923700 1 Processor -- start 2023-10-06T11:18:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7fe07c86d700 1 Processor -- start 2023-10-06T11:18:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c064590 con 0x7f122c0b5af0 2023-10-06T11:18:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f283bfff700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c159290 0x7f283c154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 -- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c159290 msgr2=0x7f283c154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c159290 0x7f283c154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 -- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 msgr2=0x7f283c13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f2833fff700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009420 con 0x7f283c0d07b0 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7faeb9923700 1 -- start start 2023-10-06T11:18:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7ec80fda80 con 0x7f7ec803a620 2023-10-06T11:18:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f283b7fe700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f2833fff700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 0x7f283c15d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f282c016000 tx=0x7f282c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 -- start start 2023-10-06T11:18:30.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 Processor -- start 2023-10-06T11:18:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 -- start start 2023-10-06T11:18:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f8c089c4700 1 -- 172.21.15.46:0/740126266 wait complete. 2023-10-06T11:18:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f121c019030 con 0x7f122c0b5af0 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f8c089c4700 1 Processor -- start 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.590+0000 7f8c089c4700 1 -- start start 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f8c089c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c040b1130 0x7f8c0406ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb8921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb8921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52678/0 (socket says 172.21.15.46:52678) 2023-10-06T11:18:30.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb8921700 1 -- 172.21.15.46:0/843326140 learned_addr learned my addr 172.21.15.46:0/843326140 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f8c089c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cb4c0 0x7fe07815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 0x7faeb4150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 0x7fe078154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 0x7fa7880c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 0x7faeb413d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb414b660 con 0x7faeb40b1130 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb414aa50 con 0x7faeb40d07b0 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb414ad30 con 0x7faeb4159290 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe076d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe076d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33214/0 (socket says 172.21.15.46:33214) 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe076d9d700 1 -- 172.21.15.46:0/2591929227 learned_addr learned my addr 172.21.15.46:0/2591929227 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815c640 con 0x7fe0780cb4c0 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815a920 con 0x7fe0780af3d0 2023-10-06T11:18:30.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f121a7fc700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 0x7f120c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f8c0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33224/0 (socket says 172.21.15.46:33224) 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7880ca7c0 0x7fa7880cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880c90b0 con 0x7fa7880b02a0 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880cb180 con 0x7fa7880ca7c0 2023-10-06T11:18:30.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa78cc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880cc980 con 0x7fa788158100 2023-10-06T11:18:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07816d940 con 0x7fe0780c57c0 2023-10-06T11:18:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f121c07e510 con 0x7f122c0b5af0 2023-10-06T11:18:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa7877fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa7880b02a0 0x7fa7880cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46330/0 (socket says 172.21.15.46:46330) 2023-10-06T11:18:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa7877fe700 1 -- 172.21.15.46:0/2343687838 learned_addr learned my addr 172.21.15.46:0/2343687838 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:30.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7fa786ffd700 1 --2- 172.21.15.46:0/2343687838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 0x7fa7880c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.594+0000 7faeb9122700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 0x7faeb413d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb3fff700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 0x7faeb4150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 -- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 msgr2=0x7faeb4150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 0x7faeb4150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 -- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 msgr2=0x7faeb414a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac009ae0 con 0x7faeb4159290 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb3fff700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 0x7faeb4150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:30.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb8921700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 0x7fe078154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07659c700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cb4c0 0x7fe07815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 -- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 msgr2=0x7fe07815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 -- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cb4c0 msgr2=0x7fe07815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cb4c0 0x7fe07815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0680094f0 con 0x7fe0780c57c0 2023-10-06T11:18:30.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe076d9d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f1229d9b700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 0x7f120c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c089c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c04159290 0x7f8c0406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c089c4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c041587f0 con 0x7f8c04159290 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c089c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04158a20 con 0x7f8c040d07b0 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c089c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0406c990 con 0x7f8c040b1130 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 -- 172.21.15.46:0/4145355427 learned_addr learned my addr 172.21.15.46:0/4145355427 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 -- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c040b1130 msgr2=0x7f8c0406ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c040b1130 0x7f8c0406ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 -- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c04159290 msgr2=0x7f8c0406d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c04159290 0x7f8c0406d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 -- 172.21.15.46:0/4145355427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf800fc60 con 0x7f8c040d07b0 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa786ffd700 1 -- 172.21.15.46:0/2343687838 >> v2:172.21.15.46:3300/0 conn(0x7fa7880ca7c0 msgr2=0x7fa7880cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa786ffd700 1 --2- 172.21.15.46:0/2343687838 >> v2:172.21.15.46:3300/0 conn(0x7fa7880ca7c0 0x7fa7880cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa786ffd700 1 -- 172.21.15.46:0/2343687838 >> v1:172.21.15.132:6789/0 conn(0x7fa7880b02a0 legacy=0x7fa7880cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa786ffd700 1 -- 172.21.15.46:0/2343687838 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880ccc60 con 0x7fa788158100 2023-10-06T11:18:30.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa786ffd700 1 --2- 172.21.15.46:0/2343687838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 0x7fa7880c2c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0098e0 tx=0x7fa77c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd26ebd157efd265 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9122700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 0x7faeb413d940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea400c490 tx=0x7faea4007e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07759e700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 0x7fe078154930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c810 tx=0x7fe070001660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f1229d9b700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 0x7f120c062a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f122c160850 tx=0x7f1220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa7867fc700 1 -- 172.21.15.46:0/2343687838 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c009e80 con 0x7fa788158100 2023-10-06T11:18:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7f8c0259c700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800fa00 tx=0x7f8bf801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa7867fc700 1 -- 172.21.15.46:0/2343687838 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00b860 con 0x7fa788158100 2023-10-06T11:18:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa7867fc700 1 -- 172.21.15.46:0/2343687838 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00bc00 con 0x7fa788158100 2023-10-06T11:18:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070012630 con 0x7fe0780c57c0 2023-10-06T11:18:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400ce90 con 0x7faeb4159290 2023-10-06T11:18:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4011590 con 0x7faeb4159290 2023-10-06T11:18:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4011950 con 0x7faeb4159290 2023-10-06T11:18:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe070012790 con 0x7fe0780c57c0 2023-10-06T11:18:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07816ddd0 con 0x7fe0780c57c0 2023-10-06T11:18:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb414b010 con 0x7faeb4159290 2023-10-06T11:18:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb413de80 con 0x7faeb4159290 2023-10-06T11:18:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 -- 172.21.15.46:0/2343687838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 msgr2=0x7fa7880c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 --2- 172.21.15.46:0/2343687838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 0x7fa7880c2c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0098e0 tx=0x7fa77c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070012b50 con 0x7fe0780c57c0 2023-10-06T11:18:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 -- 172.21.15.46:0/2343687838 shutdown_connections 2023-10-06T11:18:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07816d010 con 0x7fe0780c57c0 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 --2- 172.21.15.46:0/2343687838 >> v2:172.21.15.46:3300/0 conn(0x7fa7880ca7c0 0x7fa7880cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 --2- 172.21.15.46:0/2343687838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa788158100 0x7fa7880c2c20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.598+0000 7fa78cc8e700 1 -- 172.21.15.46:0/2343687838 >> 172.21.15.46:0/2343687838 conn(0x7fa7880970c0 msgr2=0x7fa788067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fa78cc8e700 1 -- 172.21.15.46:0/2343687838 shutdown_connections 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fa78cc8e700 1 -- 172.21.15.46:0/2343687838 wait complete. 2023-10-06T11:18:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb414b810 con 0x7faeb4159290 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c01be00 con 0x7f283c0d07b0 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c011420 con 0x7f283c0d07b0 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c011820 con 0x7f283c0d07b0 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07816dab0 con 0x7fe0780c57c0 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801d920 con 0x7f8c040d07b0 2023-10-06T11:18:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf801da80 con 0x7f8c040d07b0 2023-10-06T11:18:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801de00 con 0x7f8c040d07b0 2023-10-06T11:18:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faea4011d40 con 0x7faeb4159290 2023-10-06T11:18:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f8c089c4700 1 -- 172.21.15.46:0/4145355427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0406cc70 con 0x7f8c040d07b0 2023-10-06T11:18:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f8c089c4700 1 -- 172.21.15.46:0/4145355427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0406c640 con 0x7f8c040d07b0 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7faeb0ff9700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 0x7fae9c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7faeb8921700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 0x7fae9c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faea407bb90 con 0x7faeb4159290 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f2840f23700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c15e0b0 con 0x7f283c0d07b0 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe070003030 con 0x7fe0780c57c0 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fe05effd700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 0x7fe064062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe07007c150 con 0x7fe0780c57c0 2023-10-06T11:18:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.602+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f121c04e900 con 0x7f122c0b5af0 2023-10-06T11:18:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7fe076d9d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 0x7fe064062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f2840f23700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c0cdef0 con 0x7f283c0d07b0 2023-10-06T11:18:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7faeb8921700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 0x7fae9c062ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ab0 tx=0x7faeac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7fe076d9d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 0x7fe064062ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe068009fd0 tx=0x7fe068017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28140052b0 con 0x7f283c0d07b0 2023-10-06T11:18:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f282c011c50 con 0x7f283c0d07b0 2023-10-06T11:18:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8bf8027620 con 0x7f8c040d07b0 2023-10-06T11:18:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7f8bf2ffd700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 0x7f8be8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7fa78cc8e700 1 Processor -- start 2023-10-06T11:18:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.606+0000 7fa78cc8e700 1 -- start start 2023-10-06T11:18:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f8c02d9d700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 0x7f8be8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe07004c5c0 con 0x7fe0780c57c0 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f28337fe700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 0x7f281c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f282c07e220 con 0x7f283c0d07b0 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7880b02a0 0x7fa788097b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 0x7fa7880c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa788147b50 con 0x7fa788158100 2023-10-06T11:18:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa788098d20 con 0x7fa7880b02a0 2023-10-06T11:18:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa78cc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78806a170 con 0x7fa7880ca7c0 2023-10-06T11:18:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f283b7fe700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 0x7f281c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8bf80873d0 con 0x7f8c040d07b0 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa787fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa787fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52702/0 (socket says 172.21.15.46:52702) 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 0x7fa7880c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa787fff700 1 -- 172.21.15.46:0/908254393 learned_addr learned my addr 172.21.15.46:0/908254393 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa7877fe700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7880b02a0 0x7fa788097b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c04064590 con 0x7f8c040d07b0 2023-10-06T11:18:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 -- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7880b02a0 msgr2=0x7fa788097b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7880b02a0 0x7fa788097b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 -- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 msgr2=0x7fa788147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 -- 172.21.15.46:0/908254393 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009420 con 0x7fa7880ca7c0 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa787fff700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f8c02d9d700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 0x7f8be8062b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4007fd0 tx=0x7f8bf4007b00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7fa786ffd700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 0x7fa7880c74c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa77c016000 tx=0x7fa77c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.610+0000 7f283b7fe700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 0x7f281c062ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2824011540 tx=0x7f2824009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00ba50 con 0x7fa7880ca7c0 2023-10-06T11:18:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00bbf0 con 0x7fa7880ca7c0 2023-10-06T11:18:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c025670 con 0x7fa7880ca7c0 2023-10-06T11:18:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7fa78cc8e700 1 -- 172.21.15.46:0/908254393 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7880c7a00 con 0x7fa7880ca7c0 2023-10-06T11:18:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faea4049be0 con 0x7faeb4159290 2023-10-06T11:18:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.614+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f282c04e820 con 0x7f283c0d07b0 2023-10-06T11:18:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.618+0000 7fa78cc8e700 1 -- 172.21.15.46:0/908254393 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7880983f0 con 0x7fa7880ca7c0 2023-10-06T11:18:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.618+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7600052b0 con 0x7fa7880ca7c0 2023-10-06T11:18:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.618+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa77c023070 con 0x7fa7880ca7c0 2023-10-06T11:18:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.622+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8bf80579f0 con 0x7f8c040d07b0 2023-10-06T11:18:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.622+0000 7fa76f7fe700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 0x7fa774062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.622+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa77c07df50 con 0x7fa7880ca7c0 2023-10-06T11:18:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.622+0000 7fa7877fe700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 0x7fa774062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.626+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa77c04e470 con 0x7fa7880ca7c0 2023-10-06T11:18:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.630+0000 7fa7877fe700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 0x7fa774062ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa778007700 tx=0x7fa778007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.898+0000 7fae68d45700 1 -- 172.21.15.46:0/2205990414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae5c0349d0 con 0x7fae5c0ea650 2023-10-06T11:18:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.898+0000 7fae527fc700 1 -- 172.21.15.46:0/2205990414 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae5c0349d0 con 0x7fae5c0ea650 2023-10-06T11:18:30.906 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-10-06T11:18:30.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 msgr2=0x7fae44062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 0x7fae44062b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae54003a90 tx=0x7fae54007da0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 msgr2=0x7fae5c0dcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0dcf10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae64060ba0 tx=0x7fae6406b000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 shutdown_connections 2023-10-06T11:18:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae440606a0 0x7fae44062b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae5c0ea650 0x7fae5c0dcf10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae5c0ea280 0x7fae5c04de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 --2- 172.21.15.46:0/2205990414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae5c0e66b0 0x7fae5c0385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 >> 172.21.15.46:0/2205990414 conn(0x7fae5c054c90 msgr2=0x7fae5c004a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 shutdown_connections 2023-10-06T11:18:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.906+0000 7fae43fff700 1 -- 172.21.15.46:0/2205990414 wait complete. 2023-10-06T11:18:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.926+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7ec80fda80 con 0x7f7ec803a620 2023-10-06T11:18:30.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.926+0000 7f7eccff9700 1 -- 172.21.15.46:0/3155767780 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ec80fda80 con 0x7f7ec803a620 2023-10-06T11:18:30.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-10-06T11:18:30.935 INFO:teuthology.orchestra.run.smithi046.stdout:55834574852 2023-10-06T11:18:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 msgr2=0x7f7eb4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 0x7f7eb4062b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004890 tx=0x7f7ec40131e0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 msgr2=0x7f7ec8038210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 0x7f7ec8038210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ed004cd10 tx=0x7f7ed006a000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 shutdown_connections 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7eb40606a0 0x7f7eb4062b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec803a620 0x7f7ec8038210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ec80fbee0 0x7f7ec803a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.930+0000 7f7ed6e78700 1 --2- 172.21.15.46:0/3155767780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec0005230 0x7f7ec80fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.934+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 >> 172.21.15.46:0/3155767780 conn(0x7f7ec8051700 msgr2=0x7f7ec8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.934+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 shutdown_connections 2023-10-06T11:18:30.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.934+0000 7f7ed6e78700 1 -- 172.21.15.46:0/3155767780 wait complete. 2023-10-06T11:18:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.934+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe0780ced90 con 0x7fe0780c57c0 2023-10-06T11:18:30.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe05effd700 1 -- 172.21.15.46:0/2591929227 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe07004fbe0 con 0x7fe0780c57c0 2023-10-06T11:18:30.943 INFO:teuthology.orchestra.run.smithi046.stdout:55834574852 2023-10-06T11:18:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 msgr2=0x7fe064062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 0x7fe064062ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe068009fd0 tx=0x7fe068017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 msgr2=0x7fe078154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 0x7fe078154930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c810 tx=0x7fe070001660 comp rx=0 tx=0).stop 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 shutdown_connections 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.938+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe064060620 0x7fe064062ad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0780c57c0 0x7fe078154930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cb4c0 0x7fe07815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 --2- 172.21.15.46:0/2591929227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0780af3d0 0x7fe07815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 >> 172.21.15.46:0/2591929227 conn(0x7fe0780683b0 msgr2=0x7fe0780690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 shutdown_connections 2023-10-06T11:18:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.942+0000 7fe07c86d700 1 -- 172.21.15.46:0/2591929227 wait complete. 2023-10-06T11:18:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.946+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f122c063830 con 0x7f122c0b5af0 2023-10-06T11:18:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.950+0000 7f121a7fc700 1 -- 172.21.15.46:0/94219021 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f121c051f20 con 0x7f122c0b5af0 2023-10-06T11:18:30.954 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.950+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 msgr2=0x7f120c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.950+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 0x7f120c062a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f122c160850 tx=0x7f1220008040 comp rx=0 tx=0).stop 2023-10-06T11:18:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.950+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 msgr2=0x7f122c06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.950+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f121c011000 tx=0x7f121c015000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 shutdown_connections 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f120c060580 0x7f120c062a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f122c15fbd0 0x7f122c179080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06a530 0x7f122c14ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 --2- 172.21.15.46:0/94219021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f122c0b5af0 0x7f122c06bd40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 >> 172.21.15.46:0/94219021 conn(0x7f122c06ef60 msgr2=0x7f122c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 shutdown_connections 2023-10-06T11:18:30.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f122b59e700 1 -- 172.21.15.46:0/94219021 wait complete. 2023-10-06T11:18:30.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8c04064590 con 0x7f8c040d07b0 2023-10-06T11:18:30.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7f8bf2ffd700 1 -- 172.21.15.46:0/4145355427 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8bf805b010 con 0x7f8c040d07b0 2023-10-06T11:18:30.962 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-10-06T11:18:30.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.954+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faeb414ae70 con 0x7faeb4159290 2023-10-06T11:18:30.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.2 2023-10-06T11:18:30.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb0ff9700 1 -- 172.21.15.46:0/843326140 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faeb414ae70 con 0x7faeb4159290 2023-10-06T11:18:30.966 INFO:teuthology.orchestra.run.smithi046.stdout:55834574852 2023-10-06T11:18:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 msgr2=0x7f8be8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 0x7f8be8062b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4007fd0 tx=0x7f8bf4007b00 comp rx=0 tx=0).stop 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 msgr2=0x7f8c041582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800fa00 tx=0x7f8bf801e000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 shutdown_connections 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be8060670 0x7f8be8062b20 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4007fd0 tx=0x7f8bf4007b00 comp rx=0 tx=0).stop 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c04159290 0x7f8c0406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c040d07b0 0x7f8c041582b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 --2- 172.21.15.46:0/4145355427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c040b1130 0x7f8c0406ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 >> 172.21.15.46:0/4145355427 conn(0x7f8c0406ef60 msgr2=0x7f8c04152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2814005560 con 0x7f283c0d07b0 2023-10-06T11:18:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 shutdown_connections 2023-10-06T11:18:30.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f8bf0ff9700 1 -- 172.21.15.46:0/4145355427 wait complete. 2023-10-06T11:18:30.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2023-10-06T11:18:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7f28337fe700 1 -- 172.21.15.46:0/1131662416 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f282c00f2e0 con 0x7f283c0d07b0 2023-10-06T11:18:30.971 INFO:teuthology.orchestra.run.smithi046.stdout:55834574852 2023-10-06T11:18:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 msgr2=0x7fae9c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 0x7fae9c062ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ab0 tx=0x7faeac017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 msgr2=0x7faeb413d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 0x7faeb413d940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea400c490 tx=0x7faea4007e80 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 shutdown_connections 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fae9c060620 0x7fae9c062ad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faeb4159290 0x7faeb413d940 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faeb40d07b0 0x7faeb4150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 --2- 172.21.15.46:0/843326140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faeb40b1130 0x7faeb414a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.958+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 >> 172.21.15.46:0/843326140 conn(0x7faeb406ef60 msgr2=0x7faeb4067c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 shutdown_connections 2023-10-06T11:18:30.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7faeb9923700 1 -- 172.21.15.46:0/843326140 wait complete. 2023-10-06T11:18:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 msgr2=0x7f281c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 0x7f281c062ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2824011540 tx=0x7f2824009250 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 msgr2=0x7f283c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 0x7f283c15d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f282c016000 tx=0x7f282c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 shutdown_connections 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f281c060620 0x7f281c062ad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f283c159290 0x7f283c154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f283c0d07b0 0x7f283c15d480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 --2- 172.21.15.46:0/1131662416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f283c0b1130 0x7f283c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 >> 172.21.15.46:0/1131662416 conn(0x7f283c06ef60 msgr2=0x7f283c0ad620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 shutdown_connections 2023-10-06T11:18:30.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.962+0000 7f28317fa700 1 -- 172.21.15.46:0/1131662416 wait complete. 2023-10-06T11:18:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.966+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa760005560 con 0x7fa7880ca7c0 2023-10-06T11:18:30.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76f7fe700 1 -- 172.21.15.46:0/908254393 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa77c02a090 con 0x7fa7880ca7c0 2023-10-06T11:18:30.975 INFO:teuthology.orchestra.run.smithi046.stdout:55834574852 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 msgr2=0x7fa774062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 0x7fa774062ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa778007700 tx=0x7fa778007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 msgr2=0x7fa7880c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 0x7fa7880c74c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa77c016000 tx=0x7fa77c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 shutdown_connections 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa774060620 0x7fa774062ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa788158100 0x7fa788147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7880ca7c0 0x7fa7880c74c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 --2- 172.21.15.46:0/908254393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7880b02a0 0x7fa788097b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 >> 172.21.15.46:0/908254393 conn(0x7fa7880970c0 msgr2=0x7fa7880c2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.970+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 shutdown_connections 2023-10-06T11:18:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:30.974+0000 7fa76d7fa700 1 -- 172.21.15.46:0/908254393 wait complete. 2023-10-06T11:18:30.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.3 2023-10-06T11:18:30.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.6 2023-10-06T11:18:30.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-10-06T11:18:30.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2023-10-06T11:18:31.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2023-10-06T11:18:31.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:18:31.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:18:31.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:18:31.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:18:31.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:18:31.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:18:31.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:18:32.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:18:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.126+0000 7f747caf0700 1 Processor -- start 2023-10-06T11:18:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.126+0000 7f747caf0700 1 -- start start 2023-10-06T11:18:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f747caf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 0x7f747806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f747caf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74780c6830 0x7f747806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f747caf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74780686f0 con 0x7f74780c8d60 2023-10-06T11:18:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f747caf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74780c7680 con 0x7f74780c6830 2023-10-06T11:18:32.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f747caf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74780c6de0 con 0x7f74780c6390 2023-10-06T11:18:32.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74780c6830 0x7f747806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74780c6830 0x7f747806ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52710/0 (socket says 172.21.15.46:52710) 2023-10-06T11:18:32.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7475d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 0x7f747806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 -- 172.21.15.46:0/4273782439 learned_addr learned my addr 172.21.15.46:0/4273782439 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 -- 172.21.15.46:0/4273782439 >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 msgr2=0x7f747806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 --2- 172.21.15.46:0/4273782439 >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 0x7f747806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 -- 172.21.15.46:0/4273782439 >> v1:172.21.15.132:6789/0 conn(0x7f74780c8d60 legacy=0x7f7478152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7476d9d700 1 -- 172.21.15.46:0/4273782439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780c70c0 con 0x7f74780c6830 2023-10-06T11:18:32.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.130+0000 7f7475d9b700 1 --2- 172.21.15.46:0/4273782439 >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 0x7f747806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:32.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f7476d9d700 1 --2- 172.21.15.46:0/4273782439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6830 0x7f747806ce10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7478068110 tx=0x7f7468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe4e73cdaa7723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747559a700 1 -- 172.21.15.46:0/4273782439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468009e70 con 0x7f74780c6830 2023-10-06T11:18:32.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747559a700 1 -- 172.21.15.46:0/4273782439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746800b8f0 con 0x7f74780c6830 2023-10-06T11:18:32.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747559a700 1 -- 172.21.15.46:0/4273782439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800bc90 con 0x7f74780c6830 2023-10-06T11:18:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 -- 172.21.15.46:0/4273782439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6830 msgr2=0x7f747806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 --2- 172.21.15.46:0/4273782439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6830 0x7f747806ce10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7478068110 tx=0x7f7468009250 comp rx=0 tx=0).stop 2023-10-06T11:18:32.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 -- 172.21.15.46:0/4273782439 shutdown_connections 2023-10-06T11:18:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 --2- 172.21.15.46:0/4273782439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6830 0x7f747806ce10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 --2- 172.21.15.46:0/4273782439 >> v2:172.21.15.132:3300/0 conn(0x7f74780c6390 0x7f747806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 -- 172.21.15.46:0/4273782439 >> 172.21.15.46:0/4273782439 conn(0x7f74780cc6b0 msgr2=0x7f74780c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 -- 172.21.15.46:0/4273782439 shutdown_connections 2023-10-06T11:18:32.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.134+0000 7f747caf0700 1 -- 172.21.15.46:0/4273782439 wait complete. 2023-10-06T11:18:32.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.138+0000 7f747caf0700 1 Processor -- start 2023-10-06T11:18:32.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.138+0000 7f747caf0700 1 -- start start 2023-10-06T11:18:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.138+0000 7f747caf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74780c6830 0x7f74780ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74780c8d60 0x7f7478145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74780afea0 con 0x7f74780c6390 2023-10-06T11:18:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74780af290 con 0x7f74780c6830 2023-10-06T11:18:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74780af570 con 0x7f74780c8d60 2023-10-06T11:18:32.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52714/0 (socket says 172.21.15.46:52714) 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 -- 172.21.15.46:0/1695757570 learned_addr learned my addr 172.21.15.46:0/1695757570 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f7475d9b700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74780c6830 0x7f74780ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f7476d9d700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74780c8d60 0x7f7478145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 -- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74780c8d60 msgr2=0x7f7478145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74780c8d60 0x7f7478145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 -- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74780c6830 msgr2=0x7f74780ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74780c6830 0x7f74780ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 -- 172.21.15.46:0/1695757570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470009a20 con 0x7f74780c6390 2023-10-06T11:18:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747659c700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c4e0 tx=0x7f746c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c011590 con 0x7f74780c6390 2023-10-06T11:18:32.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c011710 con 0x7f74780c6390 2023-10-06T11:18:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c011ad0 con 0x7f74780c6390 2023-10-06T11:18:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 -- 172.21.15.46:0/1695757570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74680094f0 con 0x7f74780c6390 2023-10-06T11:18:32.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.142+0000 7f747caf0700 1 -- 172.21.15.46:0/1695757570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74780afb30 con 0x7f74780c6390 2023-10-06T11:18:32.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.146+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f746c01b3b0 con 0x7f74780c6390 2023-10-06T11:18:32.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.146+0000 7f74667fc700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 0x7f7458062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.146+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f746c07be60 con 0x7f74780c6390 2023-10-06T11:18:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.150+0000 7f7475d9b700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 0x7f7458062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.150+0000 7f7475d9b700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 0x7f7458062b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74700099f0 tx=0x7f747000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.150+0000 7f747caf0700 1 -- 172.21.15.46:0/1695757570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c0052b0 con 0x7f74780c6390 2023-10-06T11:18:32.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.158+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f746c04c2d0 con 0x7f74780c6390 2023-10-06T11:18:32.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 Processor -- start 2023-10-06T11:18:32.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 -- start start 2023-10-06T11:18:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 0x7f5da8067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da815cbe0 con 0x7f5da8067590 2023-10-06T11:18:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da815da40 con 0x7f5da8159100 2023-10-06T11:18:32.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dadae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da81594d0 con 0x7f5da815a9c0 2023-10-06T11:18:32.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dad2e2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5da8067590 0x7f5da8159800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52308/0 (socket says 172.21.15.46:52308) 2023-10-06T11:18:32.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7f5dad2e2700 1 -- 172.21.15.46:0/3441884754 learned_addr learned my addr 172.21.15.46:0/3441884754 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7ff1b744c700 1 Processor -- start 2023-10-06T11:18:32.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da7fff700 1 --2- 172.21.15.46:0/3441884754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 0x7f5da8067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.182+0000 7ff1b744c700 1 -- start start 2023-10-06T11:18:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1b00beed0 0x7ff1b00aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b006c890 0x7ff1b0160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 0x7ff1b00674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da7fff700 1 -- 172.21.15.46:0/3441884754 >> v1:172.21.15.132:6789/0 conn(0x7f5da8159100 legacy=0x7f5da80c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da7fff700 1 -- 172.21.15.46:0/3441884754 >> v1:172.21.15.46:6789/0 conn(0x7f5da8067590 legacy=0x7f5da8159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da7fff700 1 -- 172.21.15.46:0/3441884754 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da815ce30 con 0x7f5da815a9c0 2023-10-06T11:18:32.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b0067a10 con 0x7ff1b00beed0 2023-10-06T11:18:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b00c8340 con 0x7ff1b0067050 2023-10-06T11:18:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b744c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c7a10 con 0x7ff1b006c890 2023-10-06T11:18:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1b00beed0 0x7ff1b00aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1b00beed0 0x7ff1b00aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52722/0 (socket says 172.21.15.46:52722) 2023-10-06T11:18:32.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 -- 172.21.15.46:0/335309257 learned_addr learned my addr 172.21.15.46:0/335309257 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b5c49700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b006c890 0x7ff1b0160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b6c4b700 1 --2- 172.21.15.46:0/335309257 >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 0x7ff1b00674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 -- 172.21.15.46:0/335309257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b006c890 msgr2=0x7ff1b0160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b006c890 0x7ff1b0160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 -- 172.21.15.46:0/335309257 >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 msgr2=0x7ff1b00674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 --2- 172.21.15.46:0/335309257 >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 0x7ff1b00674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 -- 172.21.15.46:0/335309257 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b00c7cf0 con 0x7ff1b00beed0 2023-10-06T11:18:32.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b6c4b700 1 --2- 172.21.15.46:0/335309257 >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 0x7ff1b00674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:32.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da7fff700 1 --2- 172.21.15.46:0/3441884754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 0x7f5da8067050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009890 tx=0x7f5d980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10faea844e89eadf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b644a700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b00beed0 0x7ff1b00aa7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ab410 tx=0x7ff1a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c1f958ecaebf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b5448700 1 -- 172.21.15.46:0/335309257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000b860 con 0x7ff1b00beed0 2023-10-06T11:18:32.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7ff1b5448700 1 -- 172.21.15.46:0/335309257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000b9c0 con 0x7ff1b00beed0 2023-10-06T11:18:32.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da77fe700 1 -- 172.21.15.46:0/3441884754 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98011630 con 0x7f5da815a9c0 2023-10-06T11:18:32.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da77fe700 1 -- 172.21.15.46:0/3441884754 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98011790 con 0x7f5da815a9c0 2023-10-06T11:18:32.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.186+0000 7f5da77fe700 1 -- 172.21.15.46:0/3441884754 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98011b50 con 0x7f5da815a9c0 2023-10-06T11:18:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 -- 172.21.15.46:0/3441884754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 msgr2=0x7f5da8067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3441884754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 0x7f5da8067050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009890 tx=0x7f5d980091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 -- 172.21.15.46:0/335309257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b00beed0 msgr2=0x7ff1b00aa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b00beed0 0x7ff1b00aa7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ab410 tx=0x7ff1a0009250 comp rx=0 tx=0).stop 2023-10-06T11:18:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 -- 172.21.15.46:0/335309257 shutdown_connections 2023-10-06T11:18:32.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 --2- 172.21.15.46:0/335309257 >> v2:172.21.15.132:3300/0 conn(0x7ff1b0067050 0x7ff1b00674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b006c890 0x7ff1b0160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 --2- 172.21.15.46:0/335309257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b00beed0 0x7ff1b00aa7c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 -- 172.21.15.46:0/335309257 >> 172.21.15.46:0/335309257 conn(0x7ff1b00cbbb0 msgr2=0x7ff1b00cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 -- 172.21.15.46:0/335309257 shutdown_connections 2023-10-06T11:18:32.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 -- 172.21.15.46:0/3441884754 shutdown_connections 2023-10-06T11:18:32.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3441884754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da815a9c0 0x7f5da8067050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 -- 172.21.15.46:0/3441884754 >> 172.21.15.46:0/3441884754 conn(0x7f5da80cbbb0 msgr2=0x7f5da80cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 -- 172.21.15.46:0/3441884754 shutdown_connections 2023-10-06T11:18:32.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 -- 172.21.15.46:0/335309257 wait complete. 2023-10-06T11:18:32.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7ff1b744c700 1 Processor -- start 2023-10-06T11:18:32.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.190+0000 7f5dadae3700 1 -- 172.21.15.46:0/3441884754 wait complete. 2023-10-06T11:18:32.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b744c700 1 -- start start 2023-10-06T11:18:32.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7f5dadae3700 1 Processor -- start 2023-10-06T11:18:32.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b744c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 0x7ff1b015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7f5dadae3700 1 -- start start 2023-10-06T11:18:32.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b744c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b744c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b00beed0 0x7ff1b00abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b644a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 0x7ff1b015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b644a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 0x7ff1b015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42448/0 (socket says 172.21.15.46:42448) 2023-10-06T11:18:32.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b5c49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b5c49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52732/0 (socket says 172.21.15.46:52732) 2023-10-06T11:18:32.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b644a700 1 -- 172.21.15.46:0/2142866199 learned_addr learned my addr 172.21.15.46:0/2142866199 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.194+0000 7ff1b6c4b700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b00beed0 0x7ff1b00abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da8159100 0x7f5da80ba700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5da806c890 0x7f5da806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da806d270 con 0x7f5da8159100 2023-10-06T11:18:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da806e240 con 0x7f5da806c890 2023-10-06T11:18:32.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da816d790 con 0x7f5da8067590 2023-10-06T11:18:32.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c0b40 con 0x7ff1b006c890 2023-10-06T11:18:32.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c2d40 con 0x7ff1b00beed0 2023-10-06T11:18:32.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c0510 con 0x7ff1b0067050 2023-10-06T11:18:32.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dad2e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5da806c890 0x7f5da806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42450/0 (socket says 172.21.15.46:42450) 2023-10-06T11:18:32.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 -- 172.21.15.46:0/3755035588 learned_addr learned my addr 172.21.15.46:0/3755035588 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5da7fff700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da8159100 0x7f5da80ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 -- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 msgr2=0x7ff1b015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 0x7ff1b015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 -- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b00beed0 msgr2=0x7ff1b00abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b00beed0 0x7ff1b00abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00094f0 con 0x7ff1b006c890 2023-10-06T11:18:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 -- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5da806c890 msgr2=0x7f5da806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5da806c890 0x7f5da806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 -- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da8159100 msgr2=0x7f5da80ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da8159100 0x7f5da80ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 -- 172.21.15.46:0/3755035588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98009420 con 0x7f5da8067590 2023-10-06T11:18:32.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b5c49700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8007f60 tx=0x7ff1a8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dacae1700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c840 tx=0x7f5d9c007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800eb90 con 0x7ff1b006c890 2023-10-06T11:18:32.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c006820 con 0x7f5da8067590 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c006980 con 0x7f5da8067590 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c006d00 con 0x7f5da8067590 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800ecf0 con 0x7ff1b006c890 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b00afc40 con 0x7ff1b006c890 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8014650 con 0x7ff1b006c890 2023-10-06T11:18:32.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b00c07d0 con 0x7ff1b006c890 2023-10-06T11:18:32.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da816da50 con 0x7f5da8067590 2023-10-06T11:18:32.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da816e010 con 0x7f5da8067590 2023-10-06T11:18:32.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 Processor -- start 2023-10-06T11:18:32.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b00c0cf0 con 0x7ff1b006c890 2023-10-06T11:18:32.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da816e8c0 con 0x7f5da8067590 2023-10-06T11:18:32.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f2da5b85700 1 Processor -- start 2023-10-06T11:18:32.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.198+0000 7f2da5b85700 1 -- start start 2023-10-06T11:18:32.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5d9c035ba0 con 0x7f5da8067590 2023-10-06T11:18:32.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 -- start start 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480aef50 0x7fe9480af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9480cfa60 0x7fe9480caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480cf5d0 con 0x7fe9480cb400 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9480af860 con 0x7fe9480cfa60 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94f545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480ce9c0 con 0x7fe9480aef50 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.202+0000 7fe94cae0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9480cfa60 0x7fe9480caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94dae2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9480cb400 0x7fe948069090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52316/0 (socket says 172.21.15.46:52316) 2023-10-06T11:18:32.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94dae2700 1 -- 172.21.15.46:0/456570354 learned_addr learned my addr 172.21.15.46:0/456570354 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7f5da4ff9700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 0x7f5d94062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d9c07c890 con 0x7f5da8067590 2023-10-06T11:18:32.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7f5da7fff700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 0x7f5d94062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94cae0700 1 -- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480aef50 msgr2=0x7fe9480af320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94cae0700 1 --2- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480aef50 0x7fe9480af320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94cae0700 1 -- 172.21.15.46:0/456570354 >> v1:172.21.15.46:6789/0 conn(0x7fe9480cb400 legacy=0x7fe948069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94cae0700 1 -- 172.21.15.46:0/456570354 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480ceca0 con 0x7fe9480cfa60 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94cae0700 1 --2- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480cfa60 0x7fe9480caec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9440098d0 tx=0x7fe944009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf74d6ea21c2f0dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7f5da7fff700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 0x7f5d94062a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009860 tx=0x7f5d98016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe93ffff700 1 -- 172.21.15.46:0/456570354 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944009e70 con 0x7fe9480cfa60 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe93ffff700 1 -- 172.21.15.46:0/456570354 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944005c10 con 0x7fe9480cfa60 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe93ffff700 1 -- 172.21.15.46:0/456570354 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400b650 con 0x7fe9480cfa60 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 -- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480cfa60 msgr2=0x7fe9480caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 --2- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480cfa60 0x7fe9480caec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9440098d0 tx=0x7fe944009120 comp rx=0 tx=0).stop 2023-10-06T11:18:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 -- 172.21.15.46:0/456570354 shutdown_connections 2023-10-06T11:18:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 --2- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480cfa60 0x7fe9480caec0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff1a8014a80 con 0x7ff1b006c890 2023-10-06T11:18:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 --2- 172.21.15.46:0/456570354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480aef50 0x7fe9480af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.206+0000 7fe94f545700 1 -- 172.21.15.46:0/456570354 >> 172.21.15.46:0/456570354 conn(0x7fe94806f200 msgr2=0x7fe9480d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7fe94f545700 1 -- 172.21.15.46:0/456570354 shutdown_connections 2023-10-06T11:18:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da5b85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2da015d180 0x7f2da015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da5b85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da00c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da5b85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00c75a0 con 0x7f2da00c7a30 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da5b85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00c6990 con 0x7f2da015d180 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da5b85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00c6c70 con 0x7f2da006d4c0 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9effd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2da00c7a30 0x7f2da006cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52318/0 (socket says 172.21.15.46:52318) 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9effd700 1 -- 172.21.15.46:0/1044772359 learned_addr learned my addr 172.21.15.46:0/1044772359 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/1044772359 >> v2:172.21.15.132:3300/0 conn(0x7f2da015d180 0x7f2da015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2da4923700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da00c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7ff1a67fc700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 0x7ff194062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1a807d7b0 con 0x7ff1b006c890 2023-10-06T11:18:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7fe94f545700 1 -- 172.21.15.46:0/456570354 wait complete. 2023-10-06T11:18:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 msgr2=0x7f2da00c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da00c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7ff1b644a700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 0x7ff194062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/1044772359 >> v1:172.21.15.46:6789/0 conn(0x7f2da00c7a30 legacy=0x7f2da006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/1044772359 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00c6f50 con 0x7f2da015d180 2023-10-06T11:18:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da015d180 0x7f2da015d550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0098e0 tx=0x7f2d8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afa112a031233ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7ff1b644a700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 0x7ff194062b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0003080 tx=0x7ff1a0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9e7fc700 1 -- 172.21.15.46:0/1044772359 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00b640 con 0x7f2da015d180 2023-10-06T11:18:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9e7fc700 1 -- 172.21.15.46:0/1044772359 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c00b7a0 con 0x7f2da015d180 2023-10-06T11:18:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7f2d9e7fc700 1 -- 172.21.15.46:0/1044772359 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00bb40 con 0x7f2da015d180 2023-10-06T11:18:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff1a804dca0 con 0x7ff1b006c890 2023-10-06T11:18:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.210+0000 7fe94f545700 1 Processor -- start 2023-10-06T11:18:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 -- start start 2023-10-06T11:18:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da015d180 msgr2=0x7f2da015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da015d180 0x7f2da015d550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0098e0 tx=0x7f2d8c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- 172.21.15.46:0/1044772359 shutdown_connections 2023-10-06T11:18:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da00c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 --2- 172.21.15.46:0/1044772359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da015d180 0x7f2da015d550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- 172.21.15.46:0/1044772359 >> 172.21.15.46:0/1044772359 conn(0x7f2da00cbcc0 msgr2=0x7f2da00cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d9c050790 con 0x7f5da8067590 2023-10-06T11:18:32.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- 172.21.15.46:0/1044772359 shutdown_connections 2023-10-06T11:18:32.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- 172.21.15.46:0/1044772359 wait complete. 2023-10-06T11:18:32.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480aef50 0x7fe948148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 0x7fe9480b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42492/0 (socket says 172.21.15.46:42492) 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94dae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 0x7fe9480b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94dae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 0x7fe9480b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52758/0 (socket says 172.21.15.46:52758) 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 -- 172.21.15.46:0/4139617762 learned_addr learned my addr 172.21.15.46:0/4139617762 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480c8630 con 0x7fe9480cfa60 2023-10-06T11:18:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480c7a20 con 0x7fe9480aef50 2023-10-06T11:18:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480c7d00 con 0x7fe9480cb400 2023-10-06T11:18:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 Processor -- start 2023-10-06T11:18:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 -- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480aef50 msgr2=0x7fe948148c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480aef50 0x7fe948148c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 -- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 msgr2=0x7fe9480b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 0x7fe9480b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7fe94cae0700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440093c0 con 0x7fe9480cb400 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe94cae0700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe944009fd0 tx=0x7fe944016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94401dc10 con 0x7fe9480cb400 2023-10-06T11:18:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.214+0000 7f2da5b85700 1 -- start start 2023-10-06T11:18:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00c7a30 0x7f2da00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da015d180 0x7f2da00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da015a400 con 0x7f2da015d180 2023-10-06T11:18:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00c3990 con 0x7f2da00c7a30 2023-10-06T11:18:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42502/0 (socket says 172.21.15.46:42502) 2023-10-06T11:18:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/782082232 learned_addr learned my addr 172.21.15.46:0/782082232 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9effd700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00c7a30 0x7f2da00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0159e10 con 0x7f2da006d4c0 2023-10-06T11:18:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2da4923700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da015d180 0x7f2da00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9480c7fe0 con 0x7fe9480cb400 2023-10-06T11:18:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9480c7660 con 0x7fe9480cb400 2023-10-06T11:18:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94401dd70 con 0x7fe9480cb400 2023-10-06T11:18:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944026660 con 0x7fe9480cb400 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00c7a30 msgr2=0x7f2da00aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00c7a30 0x7f2da00aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da015d180 msgr2=0x7f2da00aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da015d180 0x7f2da00aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.218+0000 7f2d9f7fe700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c009420 con 0x7f2da006d4c0 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2d9f7fe700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0093f0 tx=0x7f2d8c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c01bc40 con 0x7f2da006d4c0 2023-10-06T11:18:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c01bde0 con 0x7f2da006d4c0 2023-10-06T11:18:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00c3080 con 0x7f2da006d4c0 2023-10-06T11:18:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00c3620 con 0x7f2da006d4c0 2023-10-06T11:18:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c011670 con 0x7f2da006d4c0 2023-10-06T11:18:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948064590 con 0x7fe9480cb400 2023-10-06T11:18:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.222+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe94401a420 con 0x7fe9480cb400 2023-10-06T11:18:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7fe93d7fa700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 0x7fe92c062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe94407e2c0 con 0x7fe9480cb400 2023-10-06T11:18:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7fe94d2e1700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 0x7fe92c062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2d8c011aa0 con 0x7f2da006d4c0 2023-10-06T11:18:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7f2d8b7fe700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 0x7f2d90062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d8c07e020 con 0x7f2da006d4c0 2023-10-06T11:18:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7fe94d2e1700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 0x7fe92c062b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe938009740 tx=0x7fe938006c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7f2d9effd700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 0x7f2d90062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.226+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d800052b0 con 0x7f2da006d4c0 2023-10-06T11:18:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.230+0000 7f2d9effd700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 0x7f2d90062ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2da015a880 tx=0x7f2d94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.234+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe94404e7b0 con 0x7fe9480cb400 2023-10-06T11:18:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.238+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2d8c04e490 con 0x7f2da006d4c0 2023-10-06T11:18:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.262+0000 7fdd2a0e3700 1 Processor -- start 2023-10-06T11:18:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.262+0000 7fdd2a0e3700 1 -- start start 2023-10-06T11:18:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 0x7fdd24067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdd240bf1a0 0x7fdd24067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd240c7a00 0x7fdd240c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd24069660 con 0x7fdd240c7a00 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd24068ac0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd2a0e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd24068d30 con 0x7fdd240bedd0 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd240c7a00 0x7fdd240c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd240c7a00 0x7fdd240c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52766/0 (socket says 172.21.15.46:52766) 2023-10-06T11:18:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 -- 172.21.15.46:0/2537074416 learned_addr learned my addr 172.21.15.46:0/2537074416 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd290e1700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 0x7fdd24067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd288e0700 1 --2- 172.21.15.46:0/2537074416 >> v2:172.21.15.132:3300/0 conn(0x7fdd240bf1a0 0x7fdd24067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 -- 172.21.15.46:0/2537074416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 msgr2=0x7fdd24067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 0x7fdd24067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 -- 172.21.15.46:0/2537074416 >> v2:172.21.15.132:3300/0 conn(0x7fdd240bf1a0 msgr2=0x7fdd24067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 --2- 172.21.15.46:0/2537074416 >> v2:172.21.15.132:3300/0 conn(0x7fdd240bf1a0 0x7fdd24067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 -- 172.21.15.46:0/2537074416 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24069010 con 0x7fdd240c7a00 2023-10-06T11:18:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd290e1700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 0x7fdd24067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 Processor -- start 2023-10-06T11:18:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 -- start start 2023-10-06T11:18:32.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fdd298e2700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240c7a00 0x7fdd240c7ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd24063720 tx=0x7fdd14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c2c198cb60a8e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed6c1578b0 0x7fed6c161a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c069b80 0x7fed6c15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c161fd0 con 0x7fed6c157c80 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0cbcf0 con 0x7fed6c1578b0 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed708ad700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15da80 con 0x7fed6c069b80 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed6b59e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed6c157c80 0x7fed6c068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52330/0 (socket says 172.21.15.46:52330) 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed6b59e700 1 -- 172.21.15.46:0/496542096 learned_addr learned my addr 172.21.15.46:0/496542096 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed6a59c700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c069b80 0x7fed6c15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.266+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/496542096 >> v2:172.21.15.46:3300/0 conn(0x7fed6c1578b0 0x7fed6c161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd1bfff700 1 -- 172.21.15.46:0/2537074416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400b860 con 0x7fdd240c7a00 2023-10-06T11:18:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed6ad9d700 1 -- 172.21.15.46:0/496542096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c069b80 msgr2=0x7fed6c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c069b80 0x7fed6c15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed6ad9d700 1 -- 172.21.15.46:0/496542096 >> v1:172.21.15.46:6789/0 conn(0x7fed6c157c80 legacy=0x7fed6c068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed6ad9d700 1 -- 172.21.15.46:0/496542096 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c069ff0 con 0x7fed6c1578b0 2023-10-06T11:18:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd1bfff700 1 -- 172.21.15.46:0/2537074416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1400b9c0 con 0x7fdd240c7a00 2023-10-06T11:18:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd1bfff700 1 -- 172.21.15.46:0/2537074416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400bd60 con 0x7fdd240c7a00 2023-10-06T11:18:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c1578b0 0x7fed6c161a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed60009910 tx=0x7fed600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb1b1271707e1dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed69d9b700 1 -- 172.21.15.46:0/496542096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60011630 con 0x7fed6c1578b0 2023-10-06T11:18:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed69d9b700 1 -- 172.21.15.46:0/496542096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed60011790 con 0x7fed6c1578b0 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed69d9b700 1 -- 172.21.15.46:0/496542096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60011b30 con 0x7fed6c1578b0 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/2537074416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240c7a00 msgr2=0x7fdd240c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240c7a00 0x7fdd240c7ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd24063720 tx=0x7fdd14009250 comp rx=0 tx=0).stop 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/2537074416 shutdown_connections 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/2537074416 >> v2:172.21.15.132:3300/0 conn(0x7fdd240bf1a0 0x7fdd24067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240c7a00 0x7fdd240c7ea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/2537074416 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240bedd0 0x7fdd24067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/2537074416 >> 172.21.15.46:0/2537074416 conn(0x7fdd24062ef0 msgr2=0x7fdd240cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 -- 172.21.15.46:0/496542096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c1578b0 msgr2=0x7fed6c161a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c1578b0 0x7fed6c161a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed60009910 tx=0x7fed600091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/2537074416 shutdown_connections 2023-10-06T11:18:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/2537074416 wait complete. 2023-10-06T11:18:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 -- 172.21.15.46:0/496542096 shutdown_connections 2023-10-06T11:18:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c069b80 0x7fed6c15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 --2- 172.21.15.46:0/496542096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c1578b0 0x7fed6c161a90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 -- 172.21.15.46:0/496542096 >> 172.21.15.46:0/496542096 conn(0x7fed6c06ad30 msgr2=0x7fed6c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 -- 172.21.15.46:0/496542096 shutdown_connections 2023-10-06T11:18:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.270+0000 7fed708ad700 1 -- 172.21.15.46:0/496542096 wait complete. 2023-10-06T11:18:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fdd2a0e3700 1 Processor -- start 2023-10-06T11:18:32.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 Processor -- start 2023-10-06T11:18:32.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 -- start start 2023-10-06T11:18:32.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fdd2a0e3700 1 -- start start 2023-10-06T11:18:32.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fdd2a0e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 0x7fdd2415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 0x7fed6c0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fdd2a0e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 0x7fed6c0c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b3ac0 con 0x7fed6c157c80 2023-10-06T11:18:32.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b1a90 con 0x7fed6c069b80 2023-10-06T11:18:32.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fed708ad700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c16df90 con 0x7fed6c1578b0 2023-10-06T11:18:32.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.274+0000 7fdd2a0e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240c7a00 0x7fdd2415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd2a0e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2415b330 con 0x7fdd240bf1a0 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd290e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 0x7fdd2415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd290e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 0x7fdd2415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33344/0 (socket says 172.21.15.46:33344) 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52788/0 (socket says 172.21.15.46:52788) 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd290e1700 1 -- 172.21.15.46:0/1003286965 learned_addr learned my addr 172.21.15.46:0/1003286965 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240aea90 con 0x7fdd240bedd0 2023-10-06T11:18:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 0x7fed6c0c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42532/0 (socket says 172.21.15.46:42532) 2023-10-06T11:18:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 -- 172.21.15.46:0/1451423638 learned_addr learned my addr 172.21.15.46:0/1451423638 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 0x7fed6c0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2415ad50 con 0x7fdd240c7a00 2023-10-06T11:18:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd298e2700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240c7a00 0x7fdd2415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 -- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 msgr2=0x7fed6c0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 0x7fed6c0b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 -- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 msgr2=0x7fed6c0c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 0x7fed6c0c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 -- 172.21.15.46:0/1451423638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60009420 con 0x7fed6c1578b0 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6b59e700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 0x7fed6c0c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 0x7fed6c0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:32.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed6a59c700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e930 tx=0x7fed5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 -- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240c7a00 msgr2=0x7fdd2415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240c7a00 0x7fdd2415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 -- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 msgr2=0x7fdd2415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 0x7fdd2415f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140094f0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00eed0 con 0x7fed6c1578b0 2023-10-06T11:18:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd288e0700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000d860 tx=0x7fdd20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0b0be0 con 0x7fed6c1578b0 2023-10-06T11:18:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0b1160 con 0x7fed6c1578b0 2023-10-06T11:18:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200125f0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2415e570 con 0x7fdd240bf1a0 2023-10-06T11:18:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2415eb30 con 0x7fdd240bf1a0 2023-10-06T11:18:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.278+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20012750 con 0x7fdd240bf1a0 2023-10-06T11:18:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00c850 con 0x7fed6c1578b0 2023-10-06T11:18:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20012b10 con 0x7fdd240bf1a0 2023-10-06T11:18:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00cbf0 con 0x7fed6c1578b0 2023-10-06T11:18:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040052b0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4c0052b0 con 0x7fed6c1578b0 2023-10-06T11:18:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7f91c46cd700 1 Processor -- start 2023-10-06T11:18:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7f91c46cd700 1 -- start start 2023-10-06T11:18:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.282+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fed5c013470 con 0x7fed6c1578b0 2023-10-06T11:18:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7fed52ffd700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 0x7fed58062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed5c07cea0 con 0x7fed6c1578b0 2023-10-06T11:18:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c46cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c46cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc06c8d0 con 0x7f91bc161840 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c46cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc06eb90 con 0x7f91bc0ad920 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c46cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc06d1c0 con 0x7f91bc0adcf0 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c1c68700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f91bc161840 0x7f91bc15a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52334/0 (socket says 172.21.15.46:52334) 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c1c68700 1 -- 172.21.15.46:0/1717723540 learned_addr learned my addr 172.21.15.46:0/1717723540 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 0x7fed58062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c2c6a700 1 --2- 172.21.15.46:0/1717723540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdd2001e460 con 0x7fdd240bf1a0 2023-10-06T11:18:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.286+0000 7f91c1467700 1 -- 172.21.15.46:0/1717723540 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256535494 0 0) 0x7f91bc06eb90 con 0x7f91bc0ad920 2023-10-06T11:18:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7f91c2c6a700 1 -- 172.21.15.46:0/1717723540 >> v1:172.21.15.132:6789/0 conn(0x7f91bc0ad920 legacy=0x7f91bc161130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7f91c2c6a700 1 -- 172.21.15.46:0/1717723540 >> v1:172.21.15.46:6789/0 conn(0x7f91bc161840 legacy=0x7f91bc15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7f91c2c6a700 1 -- 172.21.15.46:0/1717723540 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc06cb70 con 0x7f91bc0adcf0 2023-10-06T11:18:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7f91c2c6a700 1 --2- 172.21.15.46:0/1717723540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc06e5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91bc06d300 tx=0x7f91b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7c2b800c2758391 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fed6ad9d700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 0x7fed58062ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed600030c0 tx=0x7fed60016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fdd197fa700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 0x7fdd08062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdd2007cd50 con 0x7fdd240bf1a0 2023-10-06T11:18:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fdd290e1700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 0x7fdd08062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.290+0000 7fdd290e1700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 0x7fdd08062ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd240bff20 tx=0x7fdd1c007870 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdd2004d1c0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fed5c04d310 con 0x7fed6c1578b0 2023-10-06T11:18:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7f91c1467700 1 -- 172.21.15.46:0/1717723540 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4016070 con 0x7f91bc0adcf0 2023-10-06T11:18:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7f91c1467700 1 -- 172.21.15.46:0/1717723540 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b400bc30 con 0x7f91bc0adcf0 2023-10-06T11:18:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7f91c1467700 1 -- 172.21.15.46:0/1717723540 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400e070 con 0x7f91bc0adcf0 2023-10-06T11:18:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7f91c46cd700 1 -- 172.21.15.46:0/1717723540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 msgr2=0x7f91bc06e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.294+0000 7f91c46cd700 1 --2- 172.21.15.46:0/1717723540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc06e5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91bc06d300 tx=0x7f91b4009780 comp rx=0 tx=0).stop 2023-10-06T11:18:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.298+0000 7f91c46cd700 1 -- 172.21.15.46:0/1717723540 shutdown_connections 2023-10-06T11:18:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.298+0000 7f91c46cd700 1 --2- 172.21.15.46:0/1717723540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc06e5b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.298+0000 7f91c46cd700 1 -- 172.21.15.46:0/1717723540 >> 172.21.15.46:0/1717723540 conn(0x7f91bc0cc6b0 msgr2=0x7f91bc0c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.302+0000 7f91c46cd700 1 -- 172.21.15.46:0/1717723540 shutdown_connections 2023-10-06T11:18:32.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.302+0000 7f91c46cd700 1 -- 172.21.15.46:0/1717723540 wait complete. 2023-10-06T11:18:32.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.302+0000 7f91c46cd700 1 Processor -- start 2023-10-06T11:18:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.302+0000 7f91c46cd700 1 -- start start 2023-10-06T11:18:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 0x7f91bc0bf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42560/0 (socket says 172.21.15.46:42560) 2023-10-06T11:18:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c2469700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 0x7f91bc0bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c2469700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 0x7f91bc0bf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33358/0 (socket says 172.21.15.46:33358) 2023-10-06T11:18:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c2c6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c2c6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52804/0 (socket says 172.21.15.46:52804) 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc16bfa0 con 0x7f91bc161840 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c2c90 con 0x7f91bc0ad920 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c46cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c2f70 con 0x7f91bc0adcf0 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 -- 172.21.15.46:0/2822397157 learned_addr learned my addr 172.21.15.46:0/2822397157 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 -- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 msgr2=0x7f91bc0bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 0x7f91bc0bf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 -- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 msgr2=0x7f91bc0c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 -- 172.21.15.46:0/2822397157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4009a20 con 0x7f91bc0adcf0 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c2c6a700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.306+0000 7f91c1c68700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91b800c590 tx=0x7f91b8001020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.310+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8005b30 con 0x7f91bc0adcf0 2023-10-06T11:18:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.310+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc0c3250 con 0x7f91bc0adcf0 2023-10-06T11:18:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.310+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc0bfb60 con 0x7f91bc0adcf0 2023-10-06T11:18:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.310+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8005cb0 con 0x7f91bc0adcf0 2023-10-06T11:18:32.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.310+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8007320 con 0x7f91bc0adcf0 2023-10-06T11:18:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.314+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f91b801b3b0 con 0x7f91bc0adcf0 2023-10-06T11:18:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.314+0000 7f91b27fc700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 0x7f91a0062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.314+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91b807bcc0 con 0x7f91bc0adcf0 2023-10-06T11:18:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.314+0000 7f91c2469700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 0x7f91a0062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.314+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a40052b0 con 0x7f91bc0adcf0 2023-10-06T11:18:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.318+0000 7f91c2469700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 0x7f91a0062ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0011e0 tx=0x7f91ac007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.326+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f91b804c130 con 0x7f91bc0adcf0 2023-10-06T11:18:32.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.486+0000 7f747caf0700 1 -- 172.21.15.46:0/1695757570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f745c005560 con 0x7f74780c6390 2023-10-06T11:18:32.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.486+0000 7f74667fc700 1 -- 172.21.15.46:0/1695757570 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f746c017030 con 0x7f74780c6390 2023-10-06T11:18:32.491 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-10-06T11:18:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 msgr2=0x7f7458062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 0x7f7458062b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f74700099f0 tx=0x7f747000b040 comp rx=0 tx=0).stop 2023-10-06T11:18:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 msgr2=0x7f74781472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c4e0 tx=0x7f746c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 shutdown_connections 2023-10-06T11:18:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74580606d0 0x7f7458062b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74780c8d60 0x7f7478145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74780c6830 0x7f74780ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 --2- 172.21.15.46:0/1695757570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74780c6390 0x7f74781472a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 >> 172.21.15.46:0/1695757570 conn(0x7f74780cc6b0 msgr2=0x7f74780c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 shutdown_connections 2023-10-06T11:18:32.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.491+0000 7f7457fff700 1 -- 172.21.15.46:0/1695757570 wait complete. 2023-10-06T11:18:32.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-10-06T11:18:32.519 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.527+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff1b0064590 con 0x7ff1b006c890 2023-10-06T11:18:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.527+0000 7ff1a67fc700 1 -- 172.21.15.46:0/2142866199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1a80099a0 con 0x7ff1b006c890 2023-10-06T11:18:32.532 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 msgr2=0x7ff194062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 0x7ff194062b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0003080 tx=0x7ff1a0017000 comp rx=0 tx=0).stop 2023-10-06T11:18:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 msgr2=0x7ff1b00b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8007f60 tx=0x7ff1a8009880 comp rx=0 tx=0).stop 2023-10-06T11:18:32.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 shutdown_connections 2023-10-06T11:18:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1940606a0 0x7ff194062b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1b00beed0 0x7ff1b00abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.531+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b006c890 0x7ff1b00b0c60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.535+0000 7ff1b744c700 1 --2- 172.21.15.46:0/2142866199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1b0067050 0x7ff1b015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.535+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 >> 172.21.15.46:0/2142866199 conn(0x7ff1b00cbbb0 msgr2=0x7ff1b015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.535+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 shutdown_connections 2023-10-06T11:18:32.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.535+0000 7ff1b744c700 1 -- 172.21.15.46:0/2142866199 wait complete. 2023-10-06T11:18:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.543+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5da8064590 con 0x7f5da8067590 2023-10-06T11:18:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.547+0000 7f5da4ff9700 1 -- 172.21.15.46:0/3755035588 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d9c050320 con 0x7f5da8067590 2023-10-06T11:18:32.550 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-10-06T11:18:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 msgr2=0x7f5d94062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 0x7f5d94062a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009860 tx=0x7f5d98016000 comp rx=0 tx=0).stop 2023-10-06T11:18:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 msgr2=0x7f5da806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00c840 tx=0x7f5d9c007f70 comp rx=0 tx=0).stop 2023-10-06T11:18:32.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 shutdown_connections 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d940605d0 0x7f5d94062a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5da806c890 0x7f5da806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da8159100 0x7f5da80ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 --2- 172.21.15.46:0/3755035588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5da8067590 0x7f5da806dd00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 >> 172.21.15.46:0/3755035588 conn(0x7f5da80cbbb0 msgr2=0x7f5da80c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 shutdown_connections 2023-10-06T11:18:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.551+0000 7f5dadae3700 1 -- 172.21.15.46:0/3755035588 wait complete. 2023-10-06T11:18:32.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-06T11:18:32.561 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.567+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe9480cef80 con 0x7fe9480cb400 2023-10-06T11:18:32.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.567+0000 7fe93d7fa700 1 -- 172.21.15.46:0/4139617762 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe94402b030 con 0x7fe9480cb400 2023-10-06T11:18:32.571 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:32.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 msgr2=0x7fe92c062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 0x7fe92c062b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe938009740 tx=0x7fe938006c60 comp rx=0 tx=0).stop 2023-10-06T11:18:32.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 msgr2=0x7fe9480c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe944009fd0 tx=0x7fe944016000 comp rx=0 tx=0).stop 2023-10-06T11:18:32.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2d80005560 con 0x7f2da006d4c0 2023-10-06T11:18:32.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 shutdown_connections 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe92c0606d0 0x7fe92c062b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9480cfa60 0x7fe9480b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9480cb400 0x7fe9480c7120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 --2- 172.21.15.46:0/4139617762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9480aef50 0x7fe948148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 >> 172.21.15.46:0/4139617762 conn(0x7fe94806f200 msgr2=0x7fe94815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 shutdown_connections 2023-10-06T11:18:32.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7fe94f545700 1 -- 172.21.15.46:0/4139617762 wait complete. 2023-10-06T11:18:32.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.571+0000 7f2d8b7fe700 1 -- 172.21.15.46:0/782082232 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d8c015020 con 0x7f2da006d4c0 2023-10-06T11:18:32.577 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:32.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-06T11:18:32.578 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 msgr2=0x7f2d90062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 0x7f2d90062ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2da015a880 tx=0x7f2d94009040 comp rx=0 tx=0).stop 2023-10-06T11:18:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 msgr2=0x7f2da01598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0093f0 tx=0x7f2d8c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fed4c005560 con 0x7fed6c1578b0 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 shutdown_connections 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2d900607f0 0x7f2d90062ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2da015d180 0x7f2da00aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00c7a30 0x7f2da00aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 --2- 172.21.15.46:0/782082232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2da006d4c0 0x7f2da01598d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 >> 172.21.15.46:0/782082232 conn(0x7f2da00cbcc0 msgr2=0x7f2da006cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 shutdown_connections 2023-10-06T11:18:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.575+0000 7f2da5b85700 1 -- 172.21.15.46:0/782082232 wait complete. 2023-10-06T11:18:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed52ffd700 1 -- 172.21.15.46:0/1451423638 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed5c050930 con 0x7fed6c1578b0 2023-10-06T11:18:32.583 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 msgr2=0x7fed58062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 0x7fed58062ad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed600030c0 tx=0x7fed60016000 comp rx=0 tx=0).stop 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 msgr2=0x7fed6c16da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00e930 tx=0x7fed5c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 shutdown_connections 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed58060620 0x7fed58062ad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c157c80 0x7fed6c0c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed6c1578b0 0x7fed6c16da50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 --2- 172.21.15.46:0/1451423638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c069b80 0x7fed6c0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 >> 172.21.15.46:0/1451423638 conn(0x7fed6c06ad30 msgr2=0x7fed6c154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 shutdown_connections 2023-10-06T11:18:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.579+0000 7fed708ad700 1 -- 172.21.15.46:0/1451423638 wait complete. 2023-10-06T11:18:32.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-06T11:18:32.598 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-06T11:18:32.599 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-06T11:18:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.599+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdd04005560 con 0x7fdd240bf1a0 2023-10-06T11:18:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.599+0000 7fdd197fa700 1 -- 172.21.15.46:0/1003286965 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd200507e0 con 0x7fdd240bf1a0 2023-10-06T11:18:32.604 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.604 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-10-06T11:18:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 msgr2=0x7fdd08062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 0x7fdd08062ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd240bff20 tx=0x7fdd1c007870 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 msgr2=0x7fdd240b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000d860 tx=0x7fdd20008040 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 shutdown_connections 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdd08060620 0x7fdd08062ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd240c7a00 0x7fdd2415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd240bf1a0 0x7fdd240b13c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 --2- 172.21.15.46:0/1003286965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd240bedd0 0x7fdd2415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 >> 172.21.15.46:0/1003286965 conn(0x7fdd24062ef0 msgr2=0x7fdd240c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 shutdown_connections 2023-10-06T11:18:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.603+0000 7fdd2a0e3700 1 -- 172.21.15.46:0/1003286965 wait complete. 2023-10-06T11:18:32.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.615+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f91a4005560 con 0x7f91bc0adcf0 2023-10-06T11:18:32.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91b27fc700 1 -- 172.21.15.46:0/2822397157 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91b8017090 con 0x7f91bc0adcf0 2023-10-06T11:18:32.622 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-10-06T11:18:32.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-06T11:18:32.625 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 msgr2=0x7f91a0062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 0x7f91a0062ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0011e0 tx=0x7f91ac007ce0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 msgr2=0x7f91bc0c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91b800c590 tx=0x7f91b8001020 comp rx=0 tx=0).stop 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 shutdown_connections 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f91a0060620 0x7f91a0062ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91bc161840 0x7f91bc0c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91bc0adcf0 0x7f91bc0c1940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 --2- 172.21.15.46:0/2822397157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91bc0ad920 0x7f91bc0bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 >> 172.21.15.46:0/2822397157 conn(0x7f91bc0cc6b0 msgr2=0x7f91bc160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 shutdown_connections 2023-10-06T11:18:32.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.619+0000 7f91c46cd700 1 -- 172.21.15.46:0/2822397157 wait complete. 2023-10-06T11:18:32.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-06T11:18:32.641 DEBUG:teuthology.parallel:result is None 2023-10-06T11:18:32.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T11:18:32.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:18:32.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.831+0000 7fce8604c700 1 Processor -- start 2023-10-06T11:18:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.831+0000 7fce8604c700 1 -- start start 2023-10-06T11:18:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8604c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 0x7fce800b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8604c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800ae340 con 0x7fce800b1130 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8604c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800b09a0 con 0x7fce80159290 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8604c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800b15b0 con 0x7fce800d07b0 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8584b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce800b1130 0x7fce800ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52336/0 (socket says 172.21.15.46:52336) 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce8584b700 1 -- 172.21.15.46:0/3517667037 learned_addr learned my addr 172.21.15.46:0/3517667037 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.835+0000 7fce84849700 1 --2- 172.21.15.46:0/3517667037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 0x7fce800b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404164058 0 0) 0x7fce800ae340 con 0x7fce800b1130 2023-10-06T11:18:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6c003570 con 0x7fce800b1130 2023-10-06T11:18:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce84849700 1 -- 172.21.15.46:0/3517667037 >> v1:172.21.15.132:6789/0 conn(0x7fce80159290 legacy=0x7fce800b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592860792 0 0) 0x7fce800b09a0 con 0x7fce80159290 2023-10-06T11:18:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce84849700 1 -- 172.21.15.46:0/3517667037 >> v1:172.21.15.46:6789/0 conn(0x7fce800b1130 legacy=0x7fce800ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce84849700 1 -- 172.21.15.46:0/3517667037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80159660 con 0x7fce800d07b0 2023-10-06T11:18:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce84849700 1 --2- 172.21.15.46:0/3517667037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 0x7fce800b0bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce70009910 tx=0x7fce700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5509eadaa836dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7000b710 con 0x7fce800d07b0 2023-10-06T11:18:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7000b870 con 0x7fce800d07b0 2023-10-06T11:18:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce77fff700 1 -- 172.21.15.46:0/3517667037 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7000bc30 con 0x7fce800d07b0 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 -- 172.21.15.46:0/3517667037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 msgr2=0x7fce800b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 --2- 172.21.15.46:0/3517667037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 0x7fce800b0bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce70009910 tx=0x7fce700091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 -- 172.21.15.46:0/3517667037 shutdown_connections 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 --2- 172.21.15.46:0/3517667037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800d07b0 0x7fce800b0bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 -- 172.21.15.46:0/3517667037 >> 172.21.15.46:0/3517667037 conn(0x7fce8006ef60 msgr2=0x7fce800d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 -- 172.21.15.46:0/3517667037 shutdown_connections 2023-10-06T11:18:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.839+0000 7fce8604c700 1 -- 172.21.15.46:0/3517667037 wait complete. 2023-10-06T11:18:32.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 Processor -- start 2023-10-06T11:18:32.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 -- start start 2023-10-06T11:18:32.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800b1130 0x7fce801439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce80159290 0x7fce8015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009bc60 con 0x7fce800d07b0 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009c7e0 con 0x7fce80159290 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8604c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009beb0 con 0x7fce800b1130 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52812/0 (socket says 172.21.15.46:52812) 2023-10-06T11:18:32.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 -- 172.21.15.46:0/3340570431 learned_addr learned my addr 172.21.15.46:0/3340570431 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8584b700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce80159290 0x7fce8015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce8504a700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800b1130 0x7fce801439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 -- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800b1130 msgr2=0x7fce801439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800b1130 0x7fce801439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 -- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce80159290 msgr2=0x7fce8015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce80159290 0x7fce8015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 -- 172.21.15.46:0/3340570431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70009420 con 0x7fce800d07b0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.843+0000 7fce84849700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c010 tx=0x7fce70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7000bbf0 con 0x7fce800d07b0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70027450 con 0x7fce800d07b0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700277f0 con 0x7fce800d07b0 2023-10-06T11:18:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8009c190 con 0x7fce800d07b0 2023-10-06T11:18:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce80143f10 con 0x7fce800d07b0 2023-10-06T11:18:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8009c950 con 0x7fce800d07b0 2023-10-06T11:18:32.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.847+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fce7001a430 con 0x7fce800d07b0 2023-10-06T11:18:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.851+0000 7fce757fa700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 0x7fce64062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.851+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce7007e8f0 con 0x7fce800d07b0 2023-10-06T11:18:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.851+0000 7fce8504a700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 0x7fce64062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.851+0000 7fce8504a700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 0x7fce64062a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce7c003a40 tx=0x7fce7c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:32.855+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce7004ece0 con 0x7fce800d07b0 2023-10-06T11:18:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.015+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce80064590 con 0x7fce64060580 2023-10-06T11:18:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.023+0000 7fce757fa700 1 -- 172.21.15.46:0/3340570431 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286405 (secure 0 0 0) 0x7fce80064590 con 0x7fce64060580 2023-10-06T11:18:33.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 msgr2=0x7fce64062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 0x7fce64062a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce7c003a40 tx=0x7fce7c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 msgr2=0x7fce8015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c010 tx=0x7fce70017000 comp rx=0 tx=0).stop 2023-10-06T11:18:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 shutdown_connections 2023-10-06T11:18:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce64060580 0x7fce64062a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce80159290 0x7fce8015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce800d07b0 0x7fce8015a1f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 --2- 172.21.15.46:0/3340570431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce800b1130 0x7fce801439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 >> 172.21.15.46:0/3340570431 conn(0x7fce8006ef60 msgr2=0x7fce80154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 shutdown_connections 2023-10-06T11:18:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.031+0000 7fce8604c700 1 -- 172.21.15.46:0/3340570431 wait complete. 2023-10-06T11:18:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:18:33.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T11:18:32.485972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397800,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576920,"statfs":{"total":773094113280,"available":770638766080,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.716808"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801947+0000","last_change":"2023-10-06T11:18:23.797220+0000","last_active":"2023-10-06T11:18:23.801947+0000","last_peered":"2023-10-06T11:18:23.801947+0000","last_clean":"2023-10-06T11:18:23.801947+0000","last_became_active":"2023-10-06T11:18:23.796644+0000","last_became_peered":"2023-10-06T11:18:23.796644+0000","last_unstale":"2023-10-06T11:18:23.801947+0000","last_undegraded":"2023-10-06T11:18:23.801947+0000","last_fullsized":"2023-10-06T11:18:23.801947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:18.835072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801783+0000","last_change":"2023-10-06T11:18:23.798280+0000","last_active":"2023-10-06T11:18:23.801783+0000","last_peered":"2023-10-06T11:18:23.801783+0000","last_clean":"2023-10-06T11:18:23.801783+0000","last_became_active":"2023-10-06T11:18:23.797912+0000","last_became_peered":"2023-10-06T11:18:23.797912+0000","last_unstale":"2023-10-06T11:18:23.801783+0000","last_undegraded":"2023-10-06T11:18:23.801783+0000","last_fullsized":"2023-10-06T11:18:23.801783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804604+0000","last_change":"2023-10-06T11:18:23.797573+0000","last_active":"2023-10-06T11:18:23.804604+0000","last_peered":"2023-10-06T11:18:23.804604+0000","last_clean":"2023-10-06T11:18:23.804604+0000","last_became_active":"2023-10-06T11:18:23.797165+0000","last_became_peered":"2023-10-06T11:18:23.797165+0000","last_unstale":"2023-10-06T11:18:23.804604+0000","last_undegraded":"2023-10-06T11:18:23.804604+0000","last_fullsized":"2023-10-06T11:18:23.804604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800705+0000","last_change":"2023-10-06T11:18:23.795447+0000","last_active":"2023-10-06T11:18:23.800705+0000","last_peered":"2023-10-06T11:18:23.800705+0000","last_clean":"2023-10-06T11:18:23.800705+0000","last_became_active":"2023-10-06T11:18:23.794305+0000","last_became_peered":"2023-10-06T11:18:23.794305+0000","last_unstale":"2023-10-06T11:18:23.800705+0000","last_undegraded":"2023-10-06T11:18:23.800705+0000","last_fullsized":"2023-10-06T11:18:23.800705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:35.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808978+0000","last_change":"2023-10-06T11:18:23.796286+0000","last_active":"2023-10-06T11:18:23.808978+0000","last_peered":"2023-10-06T11:18:23.808978+0000","last_clean":"2023-10-06T11:18:23.808978+0000","last_became_active":"2023-10-06T11:18:23.795746+0000","last_became_peered":"2023-10-06T11:18:23.795746+0000","last_unstale":"2023-10-06T11:18:23.808978+0000","last_undegraded":"2023-10-06T11:18:23.808978+0000","last_fullsized":"2023-10-06T11:18:23.808978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802203+0000","last_change":"2023-10-06T11:18:23.793146+0000","last_active":"2023-10-06T11:18:23.802203+0000","last_peered":"2023-10-06T11:18:23.802203+0000","last_clean":"2023-10-06T11:18:23.802203+0000","last_became_active":"2023-10-06T11:18:23.792852+0000","last_became_peered":"2023-10-06T11:18:23.792852+0000","last_unstale":"2023-10-06T11:18:23.802203+0000","last_undegraded":"2023-10-06T11:18:23.802203+0000","last_fullsized":"2023-10-06T11:18:23.802203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:31.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810657+0000","last_change":"2023-10-06T11:18:23.809263+0000","last_active":"2023-10-06T11:18:23.810657+0000","last_peered":"2023-10-06T11:18:23.810657+0000","last_clean":"2023-10-06T11:18:23.810657+0000","last_became_active":"2023-10-06T11:18:23.808943+0000","last_became_peered":"2023-10-06T11:18:23.808943+0000","last_unstale":"2023-10-06T11:18:23.810657+0000","last_undegraded":"2023-10-06T11:18:23.810657+0000","last_fullsized":"2023-10-06T11:18:23.810657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789862+0000","last_change":"2023-10-06T11:18:23.788701+0000","last_active":"2023-10-06T11:18:23.789862+0000","last_peered":"2023-10-06T11:18:23.789862+0000","last_clean":"2023-10-06T11:18:23.789862+0000","last_became_active":"2023-10-06T11:18:23.788343+0000","last_became_peered":"2023-10-06T11:18:23.788343+0000","last_unstale":"2023-10-06T11:18:23.789862+0000","last_undegraded":"2023-10-06T11:18:23.789862+0000","last_fullsized":"2023-10-06T11:18:23.789862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809400+0000","last_change":"2023-10-06T11:18:23.802504+0000","last_active":"2023-10-06T11:18:23.809400+0000","last_peered":"2023-10-06T11:18:23.809400+0000","last_clean":"2023-10-06T11:18:23.809400+0000","last_became_active":"2023-10-06T11:18:23.801692+0000","last_became_peered":"2023-10-06T11:18:23.801692+0000","last_unstale":"2023-10-06T11:18:23.809400+0000","last_undegraded":"2023-10-06T11:18:23.809400+0000","last_fullsized":"2023-10-06T11:18:23.809400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800609+0000","last_change":"2023-10-06T11:18:23.795845+0000","last_active":"2023-10-06T11:18:23.800609+0000","last_peered":"2023-10-06T11:18:23.800609+0000","last_clean":"2023-10-06T11:18:23.800609+0000","last_became_active":"2023-10-06T11:18:23.795351+0000","last_became_peered":"2023-10-06T11:18:23.795351+0000","last_unstale":"2023-10-06T11:18:23.800609+0000","last_undegraded":"2023-10-06T11:18:23.800609+0000","last_fullsized":"2023-10-06T11:18:23.800609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810148+0000","last_change":"2023-10-06T11:18:23.805599+0000","last_active":"2023-10-06T11:18:23.810148+0000","last_peered":"2023-10-06T11:18:23.810148+0000","last_clean":"2023-10-06T11:18:23.810148+0000","last_became_active":"2023-10-06T11:18:23.805371+0000","last_became_peered":"2023-10-06T11:18:23.805371+0000","last_unstale":"2023-10-06T11:18:23.810148+0000","last_undegraded":"2023-10-06T11:18:23.810148+0000","last_fullsized":"2023-10-06T11:18:23.810148+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:25.984507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802009+0000","last_change":"2023-10-06T11:18:23.799204+0000","last_active":"2023-10-06T11:18:23.802009+0000","last_peered":"2023-10-06T11:18:23.802009+0000","last_clean":"2023-10-06T11:18:23.802009+0000","last_became_active":"2023-10-06T11:18:23.798864+0000","last_became_peered":"2023-10-06T11:18:23.798864+0000","last_unstale":"2023-10-06T11:18:23.802009+0000","last_undegraded":"2023-10-06T11:18:23.802009+0000","last_fullsized":"2023-10-06T11:18:23.802009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799186+0000","last_change":"2023-10-06T11:18:23.795240+0000","last_active":"2023-10-06T11:18:23.799186+0000","last_peered":"2023-10-06T11:18:23.799186+0000","last_clean":"2023-10-06T11:18:23.799186+0000","last_became_active":"2023-10-06T11:18:23.794759+0000","last_became_peered":"2023-10-06T11:18:23.794759+0000","last_unstale":"2023-10-06T11:18:23.799186+0000","last_undegraded":"2023-10-06T11:18:23.799186+0000","last_fullsized":"2023-10-06T11:18:23.799186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:08.424249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.791279+0000","last_change":"2023-10-06T11:18:23.789935+0000","last_active":"2023-10-06T11:18:23.791279+0000","last_peered":"2023-10-06T11:18:23.791279+0000","last_clean":"2023-10-06T11:18:23.791279+0000","last_became_active":"2023-10-06T11:18:23.789663+0000","last_became_peered":"2023-10-06T11:18:23.789663+0000","last_unstale":"2023-10-06T11:18:23.791279+0000","last_undegraded":"2023-10-06T11:18:23.791279+0000","last_fullsized":"2023-10-06T11:18:23.791279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810670+0000","last_change":"2023-10-06T11:18:23.809338+0000","last_active":"2023-10-06T11:18:23.810670+0000","last_peered":"2023-10-06T11:18:23.810670+0000","last_clean":"2023-10-06T11:18:23.810670+0000","last_became_active":"2023-10-06T11:18:23.808949+0000","last_became_peered":"2023-10-06T11:18:23.808949+0000","last_unstale":"2023-10-06T11:18:23.810670+0000","last_undegraded":"2023-10-06T11:18:23.810670+0000","last_fullsized":"2023-10-06T11:18:23.810670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802664+0000","last_change":"2023-10-06T11:18:23.794359+0000","last_active":"2023-10-06T11:18:23.802664+0000","last_peered":"2023-10-06T11:18:23.802664+0000","last_clean":"2023-10-06T11:18:23.802664+0000","last_became_active":"2023-10-06T11:18:23.794050+0000","last_became_peered":"2023-10-06T11:18:23.794050+0000","last_unstale":"2023-10-06T11:18:23.802664+0000","last_undegraded":"2023-10-06T11:18:23.802664+0000","last_fullsized":"2023-10-06T11:18:23.802664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809419+0000","last_change":"2023-10-06T11:18:23.802662+0000","last_active":"2023-10-06T11:18:23.809419+0000","last_peered":"2023-10-06T11:18:23.809419+0000","last_clean":"2023-10-06T11:18:23.809419+0000","last_became_active":"2023-10-06T11:18:23.802233+0000","last_became_peered":"2023-10-06T11:18:23.802233+0000","last_unstale":"2023-10-06T11:18:23.809419+0000","last_undegraded":"2023-10-06T11:18:23.809419+0000","last_fullsized":"2023-10-06T11:18:23.809419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802467+0000","last_change":"2023-10-06T11:18:23.795210+0000","last_active":"2023-10-06T11:18:23.802467+0000","last_peered":"2023-10-06T11:18:23.802467+0000","last_clean":"2023-10-06T11:18:23.802467+0000","last_became_active":"2023-10-06T11:18:23.794877+0000","last_became_peered":"2023-10-06T11:18:23.794877+0000","last_unstale":"2023-10-06T11:18:23.802467+0000","last_undegraded":"2023-10-06T11:18:23.802467+0000","last_fullsized":"2023-10-06T11:18:23.802467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:16.625365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797049+0000","last_change":"2023-10-06T11:18:23.789054+0000","last_active":"2023-10-06T11:18:23.797049+0000","last_peered":"2023-10-06T11:18:23.797049+0000","last_clean":"2023-10-06T11:18:23.797049+0000","last_became_active":"2023-10-06T11:18:23.788629+0000","last_became_peered":"2023-10-06T11:18:23.788629+0000","last_unstale":"2023-10-06T11:18:23.797049+0000","last_undegraded":"2023-10-06T11:18:23.797049+0000","last_fullsized":"2023-10-06T11:18:23.797049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:37.399272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802330+0000","last_change":"2023-10-06T11:18:23.792433+0000","last_active":"2023-10-06T11:18:23.802330+0000","last_peered":"2023-10-06T11:18:23.802330+0000","last_clean":"2023-10-06T11:18:23.802330+0000","last_became_active":"2023-10-06T11:18:23.792138+0000","last_became_peered":"2023-10-06T11:18:23.792138+0000","last_unstale":"2023-10-06T11:18:23.802330+0000","last_undegraded":"2023-10-06T11:18:23.802330+0000","last_fullsized":"2023-10-06T11:18:23.802330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801919+0000","last_change":"2023-10-06T11:18:23.797785+0000","last_active":"2023-10-06T11:18:23.801919+0000","last_peered":"2023-10-06T11:18:23.801919+0000","last_clean":"2023-10-06T11:18:23.801919+0000","last_became_active":"2023-10-06T11:18:23.797415+0000","last_became_peered":"2023-10-06T11:18:23.797415+0000","last_unstale":"2023-10-06T11:18:23.801919+0000","last_undegraded":"2023-10-06T11:18:23.801919+0000","last_fullsized":"2023-10-06T11:18:23.801919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810255+0000","last_change":"2023-10-06T11:18:23.808011+0000","last_active":"2023-10-06T11:18:23.810255+0000","last_peered":"2023-10-06T11:18:23.810255+0000","last_clean":"2023-10-06T11:18:23.810255+0000","last_became_active":"2023-10-06T11:18:23.807627+0000","last_became_peered":"2023-10-06T11:18:23.807627+0000","last_unstale":"2023-10-06T11:18:23.810255+0000","last_undegraded":"2023-10-06T11:18:23.810255+0000","last_fullsized":"2023-10-06T11:18:23.810255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797697+0000","last_change":"2023-10-06T11:18:23.794460+0000","last_active":"2023-10-06T11:18:23.797697+0000","last_peered":"2023-10-06T11:18:23.797697+0000","last_clean":"2023-10-06T11:18:23.797697+0000","last_became_active":"2023-10-06T11:18:23.794145+0000","last_became_peered":"2023-10-06T11:18:23.794145+0000","last_unstale":"2023-10-06T11:18:23.797697+0000","last_undegraded":"2023-10-06T11:18:23.797697+0000","last_fullsized":"2023-10-06T11:18:23.797697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804691+0000","last_change":"2023-10-06T11:18:23.800752+0000","last_active":"2023-10-06T11:18:23.804691+0000","last_peered":"2023-10-06T11:18:23.804691+0000","last_clean":"2023-10-06T11:18:23.804691+0000","last_became_active":"2023-10-06T11:18:23.798826+0000","last_became_peered":"2023-10-06T11:18:23.798826+0000","last_unstale":"2023-10-06T11:18:23.804691+0000","last_undegraded":"2023-10-06T11:18:23.804691+0000","last_fullsized":"2023-10-06T11:18:23.804691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804606+0000","last_change":"2023-10-06T11:18:23.801213+0000","last_active":"2023-10-06T11:18:23.804606+0000","last_peered":"2023-10-06T11:18:23.804606+0000","last_clean":"2023-10-06T11:18:23.804606+0000","last_became_active":"2023-10-06T11:18:23.800123+0000","last_became_peered":"2023-10-06T11:18:23.800123+0000","last_unstale":"2023-10-06T11:18:23.804606+0000","last_undegraded":"2023-10-06T11:18:23.804606+0000","last_fullsized":"2023-10-06T11:18:23.804606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803343+0000","last_change":"2023-10-06T11:18:23.795738+0000","last_active":"2023-10-06T11:18:23.803343+0000","last_peered":"2023-10-06T11:18:23.803343+0000","last_clean":"2023-10-06T11:18:23.803343+0000","last_became_active":"2023-10-06T11:18:23.795423+0000","last_became_peered":"2023-10-06T11:18:23.795423+0000","last_unstale":"2023-10-06T11:18:23.803343+0000","last_undegraded":"2023-10-06T11:18:23.803343+0000","last_fullsized":"2023-10-06T11:18:23.803343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800576+0000","last_change":"2023-10-06T11:18:23.794728+0000","last_active":"2023-10-06T11:18:23.800576+0000","last_peered":"2023-10-06T11:18:23.800576+0000","last_clean":"2023-10-06T11:18:23.800576+0000","last_became_active":"2023-10-06T11:18:23.794324+0000","last_became_peered":"2023-10-06T11:18:23.794324+0000","last_unstale":"2023-10-06T11:18:23.800576+0000","last_undegraded":"2023-10-06T11:18:23.800576+0000","last_fullsized":"2023-10-06T11:18:23.800576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811964+0000","last_change":"2023-10-06T11:18:23.809631+0000","last_active":"2023-10-06T11:18:23.811964+0000","last_peered":"2023-10-06T11:18:23.811964+0000","last_clean":"2023-10-06T11:18:23.811964+0000","last_became_active":"2023-10-06T11:18:23.806649+0000","last_became_peered":"2023-10-06T11:18:23.806649+0000","last_unstale":"2023-10-06T11:18:23.811964+0000","last_undegraded":"2023-10-06T11:18:23.811964+0000","last_fullsized":"2023-10-06T11:18:23.811964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803531+0000","last_change":"2023-10-06T11:18:23.800427+0000","last_active":"2023-10-06T11:18:23.803531+0000","last_peered":"2023-10-06T11:18:23.803531+0000","last_clean":"2023-10-06T11:18:23.803531+0000","last_became_active":"2023-10-06T11:18:23.799956+0000","last_became_peered":"2023-10-06T11:18:23.799956+0000","last_unstale":"2023-10-06T11:18:23.803531+0000","last_undegraded":"2023-10-06T11:18:23.803531+0000","last_fullsized":"2023-10-06T11:18:23.803531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802221+0000","last_change":"2023-10-06T11:18:23.793233+0000","last_active":"2023-10-06T11:18:23.802221+0000","last_peered":"2023-10-06T11:18:23.802221+0000","last_clean":"2023-10-06T11:18:23.802221+0000","last_became_active":"2023-10-06T11:18:23.792917+0000","last_became_peered":"2023-10-06T11:18:23.792917+0000","last_unstale":"2023-10-06T11:18:23.802221+0000","last_undegraded":"2023-10-06T11:18:23.802221+0000","last_fullsized":"2023-10-06T11:18:23.802221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:01:00.993266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812686+0000","last_change":"2023-10-06T11:18:23.811339+0000","last_active":"2023-10-06T11:18:23.812686+0000","last_peered":"2023-10-06T11:18:23.812686+0000","last_clean":"2023-10-06T11:18:23.812686+0000","last_became_active":"2023-10-06T11:18:23.810982+0000","last_became_peered":"2023-10-06T11:18:23.810982+0000","last_unstale":"2023-10-06T11:18:23.812686+0000","last_undegraded":"2023-10-06T11:18:23.812686+0000","last_fullsized":"2023-10-06T11:18:23.812686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800733+0000","last_change":"2023-10-06T11:18:23.796079+0000","last_active":"2023-10-06T11:18:23.800733+0000","last_peered":"2023-10-06T11:18:23.800733+0000","last_clean":"2023-10-06T11:18:23.800733+0000","last_became_active":"2023-10-06T11:18:23.795767+0000","last_became_peered":"2023-10-06T11:18:23.795767+0000","last_unstale":"2023-10-06T11:18:23.800733+0000","last_undegraded":"2023-10-06T11:18:23.800733+0000","last_fullsized":"2023-10-06T11:18:23.800733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:24.987705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812681+0000","last_change":"2023-10-06T11:18:23.811468+0000","last_active":"2023-10-06T11:18:23.812681+0000","last_peered":"2023-10-06T11:18:23.812681+0000","last_clean":"2023-10-06T11:18:23.812681+0000","last_became_active":"2023-10-06T11:18:23.811047+0000","last_became_peered":"2023-10-06T11:18:23.811047+0000","last_unstale":"2023-10-06T11:18:23.812681+0000","last_undegraded":"2023-10-06T11:18:23.812681+0000","last_fullsized":"2023-10-06T11:18:23.812681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080733+0000","last_change":"2023-10-06T11:18:22.785631+0000","last_active":"2023-10-06T11:18:24.080733+0000","last_peered":"2023-10-06T11:18:24.080733+0000","last_clean":"2023-10-06T11:18:24.080733+0000","last_became_active":"2023-10-06T11:18:22.785163+0000","last_became_peered":"2023-10-06T11:18:22.785163+0000","last_unstale":"2023-10-06T11:18:24.080733+0000","last_undegraded":"2023-10-06T11:18:24.080733+0000","last_fullsized":"2023-10-06T11:18:24.080733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801774+0000","last_change":"2023-10-06T11:18:23.796325+0000","last_active":"2023-10-06T11:18:23.801774+0000","last_peered":"2023-10-06T11:18:23.801774+0000","last_clean":"2023-10-06T11:18:23.801774+0000","last_became_active":"2023-10-06T11:18:23.795981+0000","last_became_peered":"2023-10-06T11:18:23.795981+0000","last_unstale":"2023-10-06T11:18:23.801774+0000","last_undegraded":"2023-10-06T11:18:23.801774+0000","last_fullsized":"2023-10-06T11:18:23.801774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785579+0000","last_change":"2023-10-06T11:18:22.771501+0000","last_active":"2023-10-06T11:18:23.785579+0000","last_peered":"2023-10-06T11:18:23.785579+0000","last_clean":"2023-10-06T11:18:23.785579+0000","last_became_active":"2023-10-06T11:18:22.771165+0000","last_became_peered":"2023-10-06T11:18:22.771165+0000","last_unstale":"2023-10-06T11:18:23.785579+0000","last_undegraded":"2023-10-06T11:18:23.785579+0000","last_fullsized":"2023-10-06T11:18:23.785579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809479+0000","last_change":"2023-10-06T11:18:23.801731+0000","last_active":"2023-10-06T11:18:23.809479+0000","last_peered":"2023-10-06T11:18:23.809479+0000","last_clean":"2023-10-06T11:18:23.809479+0000","last_became_active":"2023-10-06T11:18:23.801224+0000","last_became_peered":"2023-10-06T11:18:23.801224+0000","last_unstale":"2023-10-06T11:18:23.809479+0000","last_undegraded":"2023-10-06T11:18:23.809479+0000","last_fullsized":"2023-10-06T11:18:23.809479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787033+0000","last_change":"2023-10-06T11:18:22.783842+0000","last_active":"2023-10-06T11:18:23.787033+0000","last_peered":"2023-10-06T11:18:23.787033+0000","last_clean":"2023-10-06T11:18:23.787033+0000","last_became_active":"2023-10-06T11:18:22.783033+0000","last_became_peered":"2023-10-06T11:18:22.783033+0000","last_unstale":"2023-10-06T11:18:23.787033+0000","last_undegraded":"2023-10-06T11:18:23.787033+0000","last_fullsized":"2023-10-06T11:18:23.787033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797362+0000","last_change":"2023-10-06T11:18:23.793261+0000","last_active":"2023-10-06T11:18:23.797362+0000","last_peered":"2023-10-06T11:18:23.797362+0000","last_clean":"2023-10-06T11:18:23.797362+0000","last_became_active":"2023-10-06T11:18:23.792951+0000","last_became_peered":"2023-10-06T11:18:23.792951+0000","last_unstale":"2023-10-06T11:18:23.797362+0000","last_undegraded":"2023-10-06T11:18:23.797362+0000","last_fullsized":"2023-10-06T11:18:23.797362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.082723+0000","last_change":"2023-10-06T11:18:22.773438+0000","last_active":"2023-10-06T11:18:24.082723+0000","last_peered":"2023-10-06T11:18:24.082723+0000","last_clean":"2023-10-06T11:18:24.082723+0000","last_became_active":"2023-10-06T11:18:22.773047+0000","last_became_peered":"2023-10-06T11:18:22.773047+0000","last_unstale":"2023-10-06T11:18:24.082723+0000","last_undegraded":"2023-10-06T11:18:24.082723+0000","last_fullsized":"2023-10-06T11:18:24.082723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080613+0000","last_change":"2023-10-06T11:18:22.783414+0000","last_active":"2023-10-06T11:18:24.080613+0000","last_peered":"2023-10-06T11:18:24.080613+0000","last_clean":"2023-10-06T11:18:24.080613+0000","last_became_active":"2023-10-06T11:18:22.783068+0000","last_became_peered":"2023-10-06T11:18:22.783068+0000","last_unstale":"2023-10-06T11:18:24.080613+0000","last_undegraded":"2023-10-06T11:18:24.080613+0000","last_fullsized":"2023-10-06T11:18:24.080613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794917+0000","last_change":"2023-10-06T11:18:23.792041+0000","last_active":"2023-10-06T11:18:23.794917+0000","last_peered":"2023-10-06T11:18:23.794917+0000","last_clean":"2023-10-06T11:18:23.794917+0000","last_became_active":"2023-10-06T11:18:23.791770+0000","last_became_peered":"2023-10-06T11:18:23.791770+0000","last_unstale":"2023-10-06T11:18:23.794917+0000","last_undegraded":"2023-10-06T11:18:23.794917+0000","last_fullsized":"2023-10-06T11:18:23.794917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081788+0000","last_change":"2023-10-06T11:18:22.783471+0000","last_active":"2023-10-06T11:18:24.081788+0000","last_peered":"2023-10-06T11:18:24.081788+0000","last_clean":"2023-10-06T11:18:24.081788+0000","last_became_active":"2023-10-06T11:18:22.783150+0000","last_became_peered":"2023-10-06T11:18:22.783150+0000","last_unstale":"2023-10-06T11:18:24.081788+0000","last_undegraded":"2023-10-06T11:18:24.081788+0000","last_fullsized":"2023-10-06T11:18:24.081788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:57.801692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802567+0000","last_change":"2023-10-06T11:18:23.797336+0000","last_active":"2023-10-06T11:18:23.802567+0000","last_peered":"2023-10-06T11:18:23.802567+0000","last_clean":"2023-10-06T11:18:23.802567+0000","last_became_active":"2023-10-06T11:18:23.797002+0000","last_became_peered":"2023-10-06T11:18:23.797002+0000","last_unstale":"2023-10-06T11:18:23.802567+0000","last_undegraded":"2023-10-06T11:18:23.802567+0000","last_fullsized":"2023-10-06T11:18:23.802567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:58.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081680+0000","last_change":"2023-10-06T11:18:22.777073+0000","last_active":"2023-10-06T11:18:24.081680+0000","last_peered":"2023-10-06T11:18:24.081680+0000","last_clean":"2023-10-06T11:18:24.081680+0000","last_became_active":"2023-10-06T11:18:22.776784+0000","last_became_peered":"2023-10-06T11:18:22.776784+0000","last_unstale":"2023-10-06T11:18:24.081680+0000","last_undegraded":"2023-10-06T11:18:24.081680+0000","last_fullsized":"2023-10-06T11:18:24.081680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810078+0000","last_change":"2023-10-06T11:18:23.807687+0000","last_active":"2023-10-06T11:18:23.810078+0000","last_peered":"2023-10-06T11:18:23.810078+0000","last_clean":"2023-10-06T11:18:23.810078+0000","last_became_active":"2023-10-06T11:18:23.807468+0000","last_became_peered":"2023-10-06T11:18:23.807468+0000","last_unstale":"2023-10-06T11:18:23.810078+0000","last_undegraded":"2023-10-06T11:18:23.810078+0000","last_fullsized":"2023-10-06T11:18:23.810078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:11.773581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777670+0000","last_change":"2023-10-06T11:18:22.770019+0000","last_active":"2023-10-06T11:18:23.777670+0000","last_peered":"2023-10-06T11:18:23.777670+0000","last_clean":"2023-10-06T11:18:23.777670+0000","last_became_active":"2023-10-06T11:18:22.769681+0000","last_became_peered":"2023-10-06T11:18:22.769681+0000","last_unstale":"2023-10-06T11:18:23.777670+0000","last_undegraded":"2023-10-06T11:18:23.777670+0000","last_fullsized":"2023-10-06T11:18:23.777670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809159+0000","last_change":"2023-10-06T11:18:23.804489+0000","last_active":"2023-10-06T11:18:23.809159+0000","last_peered":"2023-10-06T11:18:23.809159+0000","last_clean":"2023-10-06T11:18:23.809159+0000","last_became_active":"2023-10-06T11:18:23.804154+0000","last_became_peered":"2023-10-06T11:18:23.804154+0000","last_unstale":"2023-10-06T11:18:23.809159+0000","last_undegraded":"2023-10-06T11:18:23.809159+0000","last_fullsized":"2023-10-06T11:18:23.809159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079612+0000","last_change":"2023-10-06T11:18:22.783172+0000","last_active":"2023-10-06T11:18:24.079612+0000","last_peered":"2023-10-06T11:18:24.079612+0000","last_clean":"2023-10-06T11:18:24.079612+0000","last_became_active":"2023-10-06T11:18:22.782848+0000","last_became_peered":"2023-10-06T11:18:22.782848+0000","last_unstale":"2023-10-06T11:18:24.079612+0000","last_undegraded":"2023-10-06T11:18:24.079612+0000","last_fullsized":"2023-10-06T11:18:24.079612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804625+0000","last_change":"2023-10-06T11:18:23.799401+0000","last_active":"2023-10-06T11:18:23.804625+0000","last_peered":"2023-10-06T11:18:23.804625+0000","last_clean":"2023-10-06T11:18:23.804625+0000","last_became_active":"2023-10-06T11:18:23.799110+0000","last_became_peered":"2023-10-06T11:18:23.799110+0000","last_unstale":"2023-10-06T11:18:23.804625+0000","last_undegraded":"2023-10-06T11:18:23.804625+0000","last_fullsized":"2023-10-06T11:18:23.804625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787968+0000","last_change":"2023-10-06T11:18:22.788211+0000","last_active":"2023-10-06T11:18:23.787968+0000","last_peered":"2023-10-06T11:18:23.787968+0000","last_clean":"2023-10-06T11:18:23.787968+0000","last_became_active":"2023-10-06T11:18:22.787835+0000","last_became_peered":"2023-10-06T11:18:22.787835+0000","last_unstale":"2023-10-06T11:18:23.787968+0000","last_undegraded":"2023-10-06T11:18:23.787968+0000","last_fullsized":"2023-10-06T11:18:23.787968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802350+0000","last_change":"2023-10-06T11:18:23.794251+0000","last_active":"2023-10-06T11:18:23.802350+0000","last_peered":"2023-10-06T11:18:23.802350+0000","last_clean":"2023-10-06T11:18:23.802350+0000","last_became_active":"2023-10-06T11:18:23.793840+0000","last_became_peered":"2023-10-06T11:18:23.793840+0000","last_unstale":"2023-10-06T11:18:23.802350+0000","last_undegraded":"2023-10-06T11:18:23.802350+0000","last_fullsized":"2023-10-06T11:18:23.802350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:26.606422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780768+0000","last_change":"2023-10-06T11:18:22.786197+0000","last_active":"2023-10-06T11:18:23.780768+0000","last_peered":"2023-10-06T11:18:23.780768+0000","last_clean":"2023-10-06T11:18:23.780768+0000","last_became_active":"2023-10-06T11:18:22.785533+0000","last_became_peered":"2023-10-06T11:18:22.785533+0000","last_unstale":"2023-10-06T11:18:23.780768+0000","last_undegraded":"2023-10-06T11:18:23.780768+0000","last_fullsized":"2023-10-06T11:18:23.780768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:46.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802002+0000","last_change":"2023-10-06T11:18:23.792349+0000","last_active":"2023-10-06T11:18:23.802002+0000","last_peered":"2023-10-06T11:18:23.802002+0000","last_clean":"2023-10-06T11:18:23.802002+0000","last_became_active":"2023-10-06T11:18:23.792058+0000","last_became_peered":"2023-10-06T11:18:23.792058+0000","last_unstale":"2023-10-06T11:18:23.802002+0000","last_undegraded":"2023-10-06T11:18:23.802002+0000","last_fullsized":"2023-10-06T11:18:23.802002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781709+0000","last_change":"2023-10-06T11:18:22.778447+0000","last_active":"2023-10-06T11:18:23.781709+0000","last_peered":"2023-10-06T11:18:23.781709+0000","last_clean":"2023-10-06T11:18:23.781709+0000","last_became_active":"2023-10-06T11:18:22.778005+0000","last_became_peered":"2023-10-06T11:18:22.778005+0000","last_unstale":"2023-10-06T11:18:23.781709+0000","last_undegraded":"2023-10-06T11:18:23.781709+0000","last_fullsized":"2023-10-06T11:18:23.781709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808325+0000","last_change":"2023-10-06T11:18:23.805117+0000","last_active":"2023-10-06T11:18:23.808325+0000","last_peered":"2023-10-06T11:18:23.808325+0000","last_clean":"2023-10-06T11:18:23.808325+0000","last_became_active":"2023-10-06T11:18:23.804771+0000","last_became_peered":"2023-10-06T11:18:23.804771+0000","last_unstale":"2023-10-06T11:18:23.808325+0000","last_undegraded":"2023-10-06T11:18:23.808325+0000","last_fullsized":"2023-10-06T11:18:23.808325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785628+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:18:23.785628+0000","last_peered":"2023-10-06T11:18:23.785628+0000","last_clean":"2023-10-06T11:18:23.785628+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:18:23.785628+0000","last_undegraded":"2023-10-06T11:18:23.785628+0000","last_fullsized":"2023-10-06T11:18:23.785628+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777308+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:18:23.777308+0000","last_peered":"2023-10-06T11:18:23.777308+0000","last_clean":"2023-10-06T11:18:23.777308+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:18:23.777308+0000","last_undegraded":"2023-10-06T11:18:23.777308+0000","last_fullsized":"2023-10-06T11:18:23.777308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.783667+0000","last_change":"2023-10-06T11:18:22.774869+0000","last_active":"2023-10-06T11:18:23.783667+0000","last_peered":"2023-10-06T11:18:23.783667+0000","last_clean":"2023-10-06T11:18:23.783667+0000","last_became_active":"2023-10-06T11:18:22.773053+0000","last_became_peered":"2023-10-06T11:18:22.773053+0000","last_unstale":"2023-10-06T11:18:23.783667+0000","last_undegraded":"2023-10-06T11:18:23.783667+0000","last_fullsized":"2023-10-06T11:18:23.783667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800870+0000","last_change":"2023-10-06T11:18:23.798289+0000","last_active":"2023-10-06T11:18:23.800870+0000","last_peered":"2023-10-06T11:18:23.800870+0000","last_clean":"2023-10-06T11:18:23.800870+0000","last_became_active":"2023-10-06T11:18:23.797843+0000","last_became_peered":"2023-10-06T11:18:23.797843+0000","last_unstale":"2023-10-06T11:18:23.800870+0000","last_undegraded":"2023-10-06T11:18:23.800870+0000","last_fullsized":"2023-10-06T11:18:23.800870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785539+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:18:23.785539+0000","last_peered":"2023-10-06T11:18:23.785539+0000","last_clean":"2023-10-06T11:18:23.785539+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:18:23.785539+0000","last_undegraded":"2023-10-06T11:18:23.785539+0000","last_fullsized":"2023-10-06T11:18:23.785539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.779941+0000","last_change":"2023-10-06T11:18:22.784973+0000","last_active":"2023-10-06T11:18:23.779941+0000","last_peered":"2023-10-06T11:18:23.779941+0000","last_clean":"2023-10-06T11:18:23.779941+0000","last_became_active":"2023-10-06T11:18:22.784647+0000","last_became_peered":"2023-10-06T11:18:22.784647+0000","last_unstale":"2023-10-06T11:18:23.779941+0000","last_undegraded":"2023-10-06T11:18:23.779941+0000","last_fullsized":"2023-10-06T11:18:23.779941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810291+0000","last_change":"2023-10-06T11:18:23.808610+0000","last_active":"2023-10-06T11:18:23.810291+0000","last_peered":"2023-10-06T11:18:23.810291+0000","last_clean":"2023-10-06T11:18:23.810291+0000","last_became_active":"2023-10-06T11:18:23.808298+0000","last_became_peered":"2023-10-06T11:18:23.808298+0000","last_unstale":"2023-10-06T11:18:23.810291+0000","last_undegraded":"2023-10-06T11:18:23.810291+0000","last_fullsized":"2023-10-06T11:18:23.810291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.773939+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:18:23.773939+0000","last_peered":"2023-10-06T11:18:23.773939+0000","last_clean":"2023-10-06T11:18:23.773939+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:18:23.773939+0000","last_undegraded":"2023-10-06T11:18:23.773939+0000","last_fullsized":"2023-10-06T11:18:23.773939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798493+0000","last_change":"2023-10-06T11:18:22.782337+0000","last_active":"2023-10-06T11:18:23.798493+0000","last_peered":"2023-10-06T11:18:23.798493+0000","last_clean":"2023-10-06T11:18:23.798493+0000","last_became_active":"2023-10-06T11:18:22.780763+0000","last_became_peered":"2023-10-06T11:18:22.780763+0000","last_unstale":"2023-10-06T11:18:23.798493+0000","last_undegraded":"2023-10-06T11:18:23.798493+0000","last_fullsized":"2023-10-06T11:18:23.798493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804588+0000","last_change":"2023-10-06T11:18:23.798843+0000","last_active":"2023-10-06T11:18:23.804588+0000","last_peered":"2023-10-06T11:18:23.804588+0000","last_clean":"2023-10-06T11:18:23.804588+0000","last_became_active":"2023-10-06T11:18:23.798591+0000","last_became_peered":"2023-10-06T11:18:23.798591+0000","last_unstale":"2023-10-06T11:18:23.804588+0000","last_undegraded":"2023-10-06T11:18:23.804588+0000","last_fullsized":"2023-10-06T11:18:23.804588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777790+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:18:23.777790+0000","last_peered":"2023-10-06T11:18:23.777790+0000","last_clean":"2023-10-06T11:18:23.777790+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:18:23.777790+0000","last_undegraded":"2023-10-06T11:18:23.777790+0000","last_fullsized":"2023-10-06T11:18:23.777790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781811+0000","last_change":"2023-10-06T11:18:22.781890+0000","last_active":"2023-10-06T11:18:23.781811+0000","last_peered":"2023-10-06T11:18:23.781811+0000","last_clean":"2023-10-06T11:18:23.781811+0000","last_became_active":"2023-10-06T11:18:22.781589+0000","last_became_peered":"2023-10-06T11:18:22.781589+0000","last_unstale":"2023-10-06T11:18:23.781811+0000","last_undegraded":"2023-10-06T11:18:23.781811+0000","last_fullsized":"2023-10-06T11:18:23.781811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806464+0000","last_change":"2023-10-06T11:18:23.803070+0000","last_active":"2023-10-06T11:18:23.806464+0000","last_peered":"2023-10-06T11:18:23.806464+0000","last_clean":"2023-10-06T11:18:23.806464+0000","last_became_active":"2023-10-06T11:18:23.799605+0000","last_became_peered":"2023-10-06T11:18:23.799605+0000","last_unstale":"2023-10-06T11:18:23.806464+0000","last_undegraded":"2023-10-06T11:18:23.806464+0000","last_fullsized":"2023-10-06T11:18:23.806464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777975+0000","last_change":"2023-10-06T11:18:22.770766+0000","last_active":"2023-10-06T11:18:23.777975+0000","last_peered":"2023-10-06T11:18:23.777975+0000","last_clean":"2023-10-06T11:18:23.777975+0000","last_became_active":"2023-10-06T11:18:22.770433+0000","last_became_peered":"2023-10-06T11:18:22.770433+0000","last_unstale":"2023-10-06T11:18:23.777975+0000","last_undegraded":"2023-10-06T11:18:23.777975+0000","last_fullsized":"2023-10-06T11:18:23.777975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804918+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:18:23.804918+0000","last_peered":"2023-10-06T11:18:23.804918+0000","last_clean":"2023-10-06T11:18:23.804918+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:18:23.804918+0000","last_undegraded":"2023-10-06T11:18:23.804918+0000","last_fullsized":"2023-10-06T11:18:23.804918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806290+0000","last_change":"2023-10-06T11:18:23.804833+0000","last_active":"2023-10-06T11:18:23.806290+0000","last_peered":"2023-10-06T11:18:23.806290+0000","last_clean":"2023-10-06T11:18:23.806290+0000","last_became_active":"2023-10-06T11:18:23.804036+0000","last_became_peered":"2023-10-06T11:18:23.804036+0000","last_unstale":"2023-10-06T11:18:23.806290+0000","last_undegraded":"2023-10-06T11:18:23.806290+0000","last_fullsized":"2023-10-06T11:18:23.806290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804802+0000","last_change":"2023-10-06T11:18:22.783968+0000","last_active":"2023-10-06T11:18:23.804802+0000","last_peered":"2023-10-06T11:18:23.804802+0000","last_clean":"2023-10-06T11:18:23.804802+0000","last_became_active":"2023-10-06T11:18:22.783402+0000","last_became_peered":"2023-10-06T11:18:22.783402+0000","last_unstale":"2023-10-06T11:18:23.804802+0000","last_undegraded":"2023-10-06T11:18:23.804802+0000","last_fullsized":"2023-10-06T11:18:23.804802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786168+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:18:23.786168+0000","last_peered":"2023-10-06T11:18:23.786168+0000","last_clean":"2023-10-06T11:18:23.786168+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:18:23.786168+0000","last_undegraded":"2023-10-06T11:18:23.786168+0000","last_fullsized":"2023-10-06T11:18:23.786168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808954+0000","last_change":"2023-10-06T11:18:23.792986+0000","last_active":"2023-10-06T11:18:23.808954+0000","last_peered":"2023-10-06T11:18:23.808954+0000","last_clean":"2023-10-06T11:18:23.808954+0000","last_became_active":"2023-10-06T11:18:23.792504+0000","last_became_peered":"2023-10-06T11:18:23.792504+0000","last_unstale":"2023-10-06T11:18:23.808954+0000","last_undegraded":"2023-10-06T11:18:23.808954+0000","last_fullsized":"2023-10-06T11:18:23.808954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079305+0000","last_change":"2023-10-06T11:18:22.773444+0000","last_active":"2023-10-06T11:18:24.079305+0000","last_peered":"2023-10-06T11:18:24.079305+0000","last_clean":"2023-10-06T11:18:24.079305+0000","last_became_active":"2023-10-06T11:18:22.773056+0000","last_became_peered":"2023-10-06T11:18:22.773056+0000","last_unstale":"2023-10-06T11:18:24.079305+0000","last_undegraded":"2023-10-06T11:18:24.079305+0000","last_fullsized":"2023-10-06T11:18:24.079305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804439+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:18:23.804439+0000","last_peered":"2023-10-06T11:18:23.804439+0000","last_clean":"2023-10-06T11:18:23.804439+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:18:23.804439+0000","last_undegraded":"2023-10-06T11:18:23.804439+0000","last_fullsized":"2023-10-06T11:18:23.804439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801518+0000","last_change":"2023-10-06T11:18:23.799064+0000","last_active":"2023-10-06T11:18:23.801518+0000","last_peered":"2023-10-06T11:18:23.801518+0000","last_clean":"2023-10-06T11:18:23.801518+0000","last_became_active":"2023-10-06T11:18:23.798759+0000","last_became_peered":"2023-10-06T11:18:23.798759+0000","last_unstale":"2023-10-06T11:18:23.801518+0000","last_undegraded":"2023-10-06T11:18:23.801518+0000","last_fullsized":"2023-10-06T11:18:23.801518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T11:18:25.180698+0000","last_change":"2023-10-06T11:18:22.781854+0000","last_active":"2023-10-06T11:18:25.180698+0000","last_peered":"2023-10-06T11:18:25.180698+0000","last_clean":"2023-10-06T11:18:25.180698+0000","last_became_active":"2023-10-06T11:18:22.781438+0000","last_became_peered":"2023-10-06T11:18:22.781438+0000","last_unstale":"2023-10-06T11:18:25.180698+0000","last_undegraded":"2023-10-06T11:18:25.180698+0000","last_fullsized":"2023-10-06T11:18:25.180698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788055+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:18:23.788055+0000","last_peered":"2023-10-06T11:18:23.788055+0000","last_clean":"2023-10-06T11:18:23.788055+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:18:23.788055+0000","last_undegraded":"2023-10-06T11:18:23.788055+0000","last_fullsized":"2023-10-06T11:18:23.788055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801967+0000","last_change":"2023-10-06T11:18:23.786622+0000","last_active":"2023-10-06T11:18:23.801967+0000","last_peered":"2023-10-06T11:18:23.801967+0000","last_clean":"2023-10-06T11:18:23.801967+0000","last_became_active":"2023-10-06T11:18:23.786299+0000","last_became_peered":"2023-10-06T11:18:23.786299+0000","last_unstale":"2023-10-06T11:18:23.801967+0000","last_undegraded":"2023-10-06T11:18:23.801967+0000","last_fullsized":"2023-10-06T11:18:23.801967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802096+0000","last_change":"2023-10-06T11:18:23.792771+0000","last_active":"2023-10-06T11:18:23.802096+0000","last_peered":"2023-10-06T11:18:23.802096+0000","last_clean":"2023-10-06T11:18:23.802096+0000","last_became_active":"2023-10-06T11:18:23.787870+0000","last_became_peered":"2023-10-06T11:18:23.787870+0000","last_unstale":"2023-10-06T11:18:23.802096+0000","last_undegraded":"2023-10-06T11:18:23.802096+0000","last_fullsized":"2023-10-06T11:18:23.802096+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.084229+0000","last_change":"2023-10-06T11:18:22.781178+0000","last_active":"2023-10-06T11:18:24.084229+0000","last_peered":"2023-10-06T11:18:24.084229+0000","last_clean":"2023-10-06T11:18:24.084229+0000","last_became_active":"2023-10-06T11:18:22.778268+0000","last_became_peered":"2023-10-06T11:18:22.778268+0000","last_unstale":"2023-10-06T11:18:24.084229+0000","last_undegraded":"2023-10-06T11:18:24.084229+0000","last_fullsized":"2023-10-06T11:18:24.084229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804992+0000","last_change":"2023-10-06T11:18:23.800123+0000","last_active":"2023-10-06T11:18:23.804992+0000","last_peered":"2023-10-06T11:18:23.804992+0000","last_clean":"2023-10-06T11:18:23.804992+0000","last_became_active":"2023-10-06T11:18:23.799794+0000","last_became_peered":"2023-10-06T11:18:23.799794+0000","last_unstale":"2023-10-06T11:18:23.804992+0000","last_undegraded":"2023-10-06T11:18:23.804992+0000","last_fullsized":"2023-10-06T11:18:23.804992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081932+0000","last_change":"2023-10-06T11:18:22.782320+0000","last_active":"2023-10-06T11:18:24.081932+0000","last_peered":"2023-10-06T11:18:24.081932+0000","last_clean":"2023-10-06T11:18:24.081932+0000","last_became_active":"2023-10-06T11:18:22.782025+0000","last_became_peered":"2023-10-06T11:18:22.782025+0000","last_unstale":"2023-10-06T11:18:24.081932+0000","last_undegraded":"2023-10-06T11:18:24.081932+0000","last_fullsized":"2023-10-06T11:18:24.081932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777975+0000","last_change":"2023-10-06T11:18:22.773270+0000","last_active":"2023-10-06T11:18:23.777975+0000","last_peered":"2023-10-06T11:18:23.777975+0000","last_clean":"2023-10-06T11:18:23.777975+0000","last_became_active":"2023-10-06T11:18:22.772211+0000","last_became_peered":"2023-10-06T11:18:22.772211+0000","last_unstale":"2023-10-06T11:18:23.777975+0000","last_undegraded":"2023-10-06T11:18:23.777975+0000","last_fullsized":"2023-10-06T11:18:23.777975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797678+0000","last_change":"2023-10-06T11:18:23.794332+0000","last_active":"2023-10-06T11:18:23.797678+0000","last_peered":"2023-10-06T11:18:23.797678+0000","last_clean":"2023-10-06T11:18:23.797678+0000","last_became_active":"2023-10-06T11:18:23.793993+0000","last_became_peered":"2023-10-06T11:18:23.793993+0000","last_unstale":"2023-10-06T11:18:23.797678+0000","last_undegraded":"2023-10-06T11:18:23.797678+0000","last_fullsized":"2023-10-06T11:18:23.797678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.783823+0000","last_change":"2023-10-06T11:18:22.774828+0000","last_active":"2023-10-06T11:18:23.783823+0000","last_peered":"2023-10-06T11:18:23.783823+0000","last_clean":"2023-10-06T11:18:23.783823+0000","last_became_active":"2023-10-06T11:18:22.774440+0000","last_became_peered":"2023-10-06T11:18:22.774440+0000","last_unstale":"2023-10-06T11:18:23.783823+0000","last_undegraded":"2023-10-06T11:18:23.783823+0000","last_fullsized":"2023-10-06T11:18:23.783823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.807787+0000","last_change":"2023-10-06T11:18:23.804188+0000","last_active":"2023-10-06T11:18:23.807787+0000","last_peered":"2023-10-06T11:18:23.807787+0000","last_clean":"2023-10-06T11:18:23.807787+0000","last_became_active":"2023-10-06T11:18:23.803863+0000","last_became_peered":"2023-10-06T11:18:23.803863+0000","last_unstale":"2023-10-06T11:18:23.807787+0000","last_undegraded":"2023-10-06T11:18:23.807787+0000","last_fullsized":"2023-10-06T11:18:23.807787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792190+0000","last_change":"2023-10-06T11:18:23.789620+0000","last_active":"2023-10-06T11:18:23.792190+0000","last_peered":"2023-10-06T11:18:23.792190+0000","last_clean":"2023-10-06T11:18:23.792190+0000","last_became_active":"2023-10-06T11:18:23.789349+0000","last_became_peered":"2023-10-06T11:18:23.789349+0000","last_unstale":"2023-10-06T11:18:23.792190+0000","last_undegraded":"2023-10-06T11:18:23.792190+0000","last_fullsized":"2023-10-06T11:18:23.792190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781628+0000","last_change":"2023-10-06T11:18:22.781828+0000","last_active":"2023-10-06T11:18:23.781628+0000","last_peered":"2023-10-06T11:18:23.781628+0000","last_clean":"2023-10-06T11:18:23.781628+0000","last_became_active":"2023-10-06T11:18:22.781529+0000","last_became_peered":"2023-10-06T11:18:22.781529+0000","last_unstale":"2023-10-06T11:18:23.781628+0000","last_undegraded":"2023-10-06T11:18:23.781628+0000","last_fullsized":"2023-10-06T11:18:23.781628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808495+0000","last_change":"2023-10-06T11:18:23.792292+0000","last_active":"2023-10-06T11:18:23.808495+0000","last_peered":"2023-10-06T11:18:23.808495+0000","last_clean":"2023-10-06T11:18:23.808495+0000","last_became_active":"2023-10-06T11:18:23.791847+0000","last_became_peered":"2023-10-06T11:18:23.791847+0000","last_unstale":"2023-10-06T11:18:23.808495+0000","last_undegraded":"2023-10-06T11:18:23.808495+0000","last_fullsized":"2023-10-06T11:18:23.808495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803596+0000","last_change":"2023-10-06T11:18:22.774677+0000","last_active":"2023-10-06T11:18:23.803596+0000","last_peered":"2023-10-06T11:18:23.803596+0000","last_clean":"2023-10-06T11:18:23.803596+0000","last_became_active":"2023-10-06T11:18:22.772746+0000","last_became_peered":"2023-10-06T11:18:22.772746+0000","last_unstale":"2023-10-06T11:18:23.803596+0000","last_undegraded":"2023-10-06T11:18:23.803596+0000","last_fullsized":"2023-10-06T11:18:23.803596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809429+0000","last_change":"2023-10-06T11:18:23.806392+0000","last_active":"2023-10-06T11:18:23.809429+0000","last_peered":"2023-10-06T11:18:23.809429+0000","last_clean":"2023-10-06T11:18:23.809429+0000","last_became_active":"2023-10-06T11:18:23.806086+0000","last_became_peered":"2023-10-06T11:18:23.806086+0000","last_unstale":"2023-10-06T11:18:23.809429+0000","last_undegraded":"2023-10-06T11:18:23.809429+0000","last_fullsized":"2023-10-06T11:18:23.809429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.074160+0000","last_change":"2023-10-06T11:18:22.786752+0000","last_active":"2023-10-06T11:18:24.074160+0000","last_peered":"2023-10-06T11:18:24.074160+0000","last_clean":"2023-10-06T11:18:24.074160+0000","last_became_active":"2023-10-06T11:18:22.784966+0000","last_became_peered":"2023-10-06T11:18:22.784966+0000","last_unstale":"2023-10-06T11:18:24.074160+0000","last_undegraded":"2023-10-06T11:18:24.074160+0000","last_fullsized":"2023-10-06T11:18:24.074160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812296+0000","last_change":"2023-10-06T11:18:23.811011+0000","last_active":"2023-10-06T11:18:23.812296+0000","last_peered":"2023-10-06T11:18:23.812296+0000","last_clean":"2023-10-06T11:18:23.812296+0000","last_became_active":"2023-10-06T11:18:23.810563+0000","last_became_peered":"2023-10-06T11:18:23.810563+0000","last_unstale":"2023-10-06T11:18:23.812296+0000","last_undegraded":"2023-10-06T11:18:23.812296+0000","last_fullsized":"2023-10-06T11:18:23.812296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:06.048708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.073707+0000","last_change":"2023-10-06T11:18:22.787138+0000","last_active":"2023-10-06T11:18:24.073707+0000","last_peered":"2023-10-06T11:18:24.073707+0000","last_clean":"2023-10-06T11:18:24.073707+0000","last_became_active":"2023-10-06T11:18:22.786799+0000","last_became_peered":"2023-10-06T11:18:22.786799+0000","last_unstale":"2023-10-06T11:18:24.073707+0000","last_undegraded":"2023-10-06T11:18:24.073707+0000","last_fullsized":"2023-10-06T11:18:24.073707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802632+0000","last_change":"2023-10-06T11:18:23.797231+0000","last_active":"2023-10-06T11:18:23.802632+0000","last_peered":"2023-10-06T11:18:23.802632+0000","last_clean":"2023-10-06T11:18:23.802632+0000","last_became_active":"2023-10-06T11:18:23.796942+0000","last_became_peered":"2023-10-06T11:18:23.796942+0000","last_unstale":"2023-10-06T11:18:23.802632+0000","last_undegraded":"2023-10-06T11:18:23.802632+0000","last_fullsized":"2023-10-06T11:18:23.802632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080861+0000","last_change":"2023-10-06T11:18:22.784448+0000","last_active":"2023-10-06T11:18:24.080861+0000","last_peered":"2023-10-06T11:18:24.080861+0000","last_clean":"2023-10-06T11:18:24.080861+0000","last_became_active":"2023-10-06T11:18:22.784131+0000","last_became_peered":"2023-10-06T11:18:22.784131+0000","last_unstale":"2023-10-06T11:18:24.080861+0000","last_undegraded":"2023-10-06T11:18:24.080861+0000","last_fullsized":"2023-10-06T11:18:24.080861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802495+0000","last_change":"2023-10-06T11:18:23.796243+0000","last_active":"2023-10-06T11:18:23.802495+0000","last_peered":"2023-10-06T11:18:23.802495+0000","last_clean":"2023-10-06T11:18:23.802495+0000","last_became_active":"2023-10-06T11:18:23.795924+0000","last_became_peered":"2023-10-06T11:18:23.795924+0000","last_unstale":"2023-10-06T11:18:23.802495+0000","last_undegraded":"2023-10-06T11:18:23.802495+0000","last_fullsized":"2023-10-06T11:18:23.802495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080525+0000","last_change":"2023-10-06T11:18:22.789060+0000","last_active":"2023-10-06T11:18:24.080525+0000","last_peered":"2023-10-06T11:18:24.080525+0000","last_clean":"2023-10-06T11:18:24.080525+0000","last_became_active":"2023-10-06T11:18:22.788376+0000","last_became_peered":"2023-10-06T11:18:22.788376+0000","last_unstale":"2023-10-06T11:18:24.080525+0000","last_undegraded":"2023-10-06T11:18:24.080525+0000","last_fullsized":"2023-10-06T11:18:24.080525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804659+0000","last_change":"2023-10-06T11:18:23.802973+0000","last_active":"2023-10-06T11:18:23.804659+0000","last_peered":"2023-10-06T11:18:23.804659+0000","last_clean":"2023-10-06T11:18:23.804659+0000","last_became_active":"2023-10-06T11:18:23.802597+0000","last_became_peered":"2023-10-06T11:18:23.802597+0000","last_unstale":"2023-10-06T11:18:23.804659+0000","last_undegraded":"2023-10-06T11:18:23.804659+0000","last_fullsized":"2023-10-06T11:18:23.804659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079378+0000","last_change":"2023-10-06T11:18:22.770917+0000","last_active":"2023-10-06T11:18:24.079378+0000","last_peered":"2023-10-06T11:18:24.079378+0000","last_clean":"2023-10-06T11:18:24.079378+0000","last_became_active":"2023-10-06T11:18:22.770587+0000","last_became_peered":"2023-10-06T11:18:22.770587+0000","last_unstale":"2023-10-06T11:18:24.079378+0000","last_undegraded":"2023-10-06T11:18:24.079378+0000","last_fullsized":"2023-10-06T11:18:24.079378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802685+0000","last_change":"2023-10-06T11:18:23.797236+0000","last_active":"2023-10-06T11:18:23.802685+0000","last_peered":"2023-10-06T11:18:23.802685+0000","last_clean":"2023-10-06T11:18:23.802685+0000","last_became_active":"2023-10-06T11:18:23.796889+0000","last_became_peered":"2023-10-06T11:18:23.796889+0000","last_unstale":"2023-10-06T11:18:23.802685+0000","last_undegraded":"2023-10-06T11:18:23.802685+0000","last_fullsized":"2023-10-06T11:18:23.802685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079742+0000","last_change":"2023-10-06T11:18:22.778280+0000","last_active":"2023-10-06T11:18:24.079742+0000","last_peered":"2023-10-06T11:18:24.079742+0000","last_clean":"2023-10-06T11:18:24.079742+0000","last_became_active":"2023-10-06T11:18:22.776627+0000","last_became_peered":"2023-10-06T11:18:22.776627+0000","last_unstale":"2023-10-06T11:18:24.079742+0000","last_undegraded":"2023-10-06T11:18:24.079742+0000","last_fullsized":"2023-10-06T11:18:24.079742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.905}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301624,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070216,"statfs":{"total":96636764160,"available":96327901184,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302204,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069636,"statfs":{"total":96636764160,"available":96327307264,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:18:33.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:18:33.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.247+0000 7fa15ebc5700 1 Processor -- start 2023-10-06T11:18:33.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.247+0000 7fa15ebc5700 1 -- start start 2023-10-06T11:18:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15ebc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 0x7fa15806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15ebc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15806d460 con 0x7fa1580ad6c0 2023-10-06T11:18:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15ebc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580c7000 con 0x7fa15815ab60 2023-10-06T11:18:33.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15ebc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15806ed10 con 0x7fa15806e8a0 2023-10-06T11:18:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15d162700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa1580ad6c0 0x7fa1580adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43244/0 (socket says 172.21.15.46:43244) 2023-10-06T11:18:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa15d162700 1 -- 172.21.15.46:0/514595744 learned_addr learned my addr 172.21.15.46:0/514595744 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:33.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa157fff700 1 --2- 172.21.15.46:0/514595744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 0x7fa15806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350298804 0 0) 0x7fa1580c7000 con 0x7fa15815ab60 2023-10-06T11:18:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa148003570 con 0x7fa15815ab60 2023-10-06T11:18:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125605349 0 0) 0x7fa15806d460 con 0x7fa1580ad6c0 2023-10-06T11:18:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1580c7000 con 0x7fa1580ad6c0 2023-10-06T11:18:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa157fff700 1 -- 172.21.15.46:0/514595744 >> v1:172.21.15.132:6789/0 conn(0x7fa15815ab60 legacy=0x7fa15815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa157fff700 1 -- 172.21.15.46:0/514595744 >> v1:172.21.15.46:6789/0 conn(0x7fa1580ad6c0 legacy=0x7fa1580adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.251+0000 7fa157fff700 1 -- 172.21.15.46:0/514595744 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580c72c0 con 0x7fa15806e8a0 2023-10-06T11:18:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa157fff700 1 --2- 172.21.15.46:0/514595744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 0x7fa15806cc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa144009910 tx=0x7fa1440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f311176281ebf56f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144011630 con 0x7fa15806e8a0 2023-10-06T11:18:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144011790 con 0x7fa15806e8a0 2023-10-06T11:18:33.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa1577fe700 1 -- 172.21.15.46:0/514595744 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144011b30 con 0x7fa15806e8a0 2023-10-06T11:18:33.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 -- 172.21.15.46:0/514595744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 msgr2=0x7fa15806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/514595744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 0x7fa15806cc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa144009910 tx=0x7fa1440091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 -- 172.21.15.46:0/514595744 shutdown_connections 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/514595744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa15806e8a0 0x7fa15806cc30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 -- 172.21.15.46:0/514595744 >> 172.21.15.46:0/514595744 conn(0x7fa1580cbe30 msgr2=0x7fa1580cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 -- 172.21.15.46:0/514595744 shutdown_connections 2023-10-06T11:18:33.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.255+0000 7fa15ebc5700 1 -- 172.21.15.46:0/514595744 wait complete. 2023-10-06T11:18:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 Processor -- start 2023-10-06T11:18:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 -- start start 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1580ad6c0 0x7fa158157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 0x7fa15814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580b0f40 con 0x7fa15806e8a0 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ca8f0 con 0x7fa15815ab60 2023-10-06T11:18:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15ebc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581583c0 con 0x7fa1580ad6c0 2023-10-06T11:18:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15c961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15c961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44276/0 (socket says 172.21.15.46:44276) 2023-10-06T11:18:33.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15c961700 1 -- 172.21.15.46:0/2983412318 learned_addr learned my addr 172.21.15.46:0/2983412318 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa157fff700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1580ad6c0 0x7fa158157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.259+0000 7fa15d162700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 0x7fa15814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 -- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1580ad6c0 msgr2=0x7fa158157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1580ad6c0 0x7fa158157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 -- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 msgr2=0x7fa15814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 0x7fa15814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 -- 172.21.15.46:0/2983412318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144009420 con 0x7fa15806e8a0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15d162700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 0x7fa15814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15c961700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00cd00 tx=0x7fa14c002bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00ce60 con 0x7fa15806e8a0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00ab70 con 0x7fa15806e8a0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1580c9a00 con 0x7fa15806e8a0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c002030 con 0x7fa15806e8a0 2023-10-06T11:18:33.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1580c9fc0 con 0x7fa15806e8a0 2023-10-06T11:18:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.263+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158063f90 con 0x7fa15806e8a0 2023-10-06T11:18:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.267+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa14c001520 con 0x7fa15806e8a0 2023-10-06T11:18:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.267+0000 7fa154ff9700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 0x7fa13c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.267+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa14c07ce10 con 0x7fa15806e8a0 2023-10-06T11:18:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.267+0000 7fa157fff700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 0x7fa13c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.267+0000 7fa157fff700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 0x7fa13c062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1440093f0 tx=0x7fa144017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.275+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa14c080030 con 0x7fa15806e8a0 2023-10-06T11:18:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.439+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa15804b5c0 con 0x7fa13c060550 2023-10-06T11:18:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.447+0000 7fa154ff9700 1 -- 172.21.15.46:0/2983412318 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286405 (secure 0 0 0) 0x7fa15804b5c0 con 0x7fa13c060550 2023-10-06T11:18:33.458 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 msgr2=0x7fa13c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 0x7fa13c062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1440093f0 tx=0x7fa144017000 comp rx=0 tx=0).stop 2023-10-06T11:18:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 msgr2=0x7fa15814dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00cd00 tx=0x7fa14c002bb0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 shutdown_connections 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa13c060550 0x7fa13c062a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa15815ab60 0x7fa15814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1580ad6c0 0x7fa158157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 --2- 172.21.15.46:0/2983412318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa15806e8a0 0x7fa15814dd20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 >> 172.21.15.46:0/2983412318 conn(0x7fa1580cbe30 msgr2=0x7fa158069970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 shutdown_connections 2023-10-06T11:18:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.455+0000 7fa15ebc5700 1 -- 172.21.15.46:0/2983412318 wait complete. 2023-10-06T11:18:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:18:33.479 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T11:18:32.485972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397800,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576920,"statfs":{"total":773094113280,"available":770638766080,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.716808"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801947+0000","last_change":"2023-10-06T11:18:23.797220+0000","last_active":"2023-10-06T11:18:23.801947+0000","last_peered":"2023-10-06T11:18:23.801947+0000","last_clean":"2023-10-06T11:18:23.801947+0000","last_became_active":"2023-10-06T11:18:23.796644+0000","last_became_peered":"2023-10-06T11:18:23.796644+0000","last_unstale":"2023-10-06T11:18:23.801947+0000","last_undegraded":"2023-10-06T11:18:23.801947+0000","last_fullsized":"2023-10-06T11:18:23.801947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:18.835072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801783+0000","last_change":"2023-10-06T11:18:23.798280+0000","last_active":"2023-10-06T11:18:23.801783+0000","last_peered":"2023-10-06T11:18:23.801783+0000","last_clean":"2023-10-06T11:18:23.801783+0000","last_became_active":"2023-10-06T11:18:23.797912+0000","last_became_peered":"2023-10-06T11:18:23.797912+0000","last_unstale":"2023-10-06T11:18:23.801783+0000","last_undegraded":"2023-10-06T11:18:23.801783+0000","last_fullsized":"2023-10-06T11:18:23.801783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804604+0000","last_change":"2023-10-06T11:18:23.797573+0000","last_active":"2023-10-06T11:18:23.804604+0000","last_peered":"2023-10-06T11:18:23.804604+0000","last_clean":"2023-10-06T11:18:23.804604+0000","last_became_active":"2023-10-06T11:18:23.797165+0000","last_became_peered":"2023-10-06T11:18:23.797165+0000","last_unstale":"2023-10-06T11:18:23.804604+0000","last_undegraded":"2023-10-06T11:18:23.804604+0000","last_fullsized":"2023-10-06T11:18:23.804604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800705+0000","last_change":"2023-10-06T11:18:23.795447+0000","last_active":"2023-10-06T11:18:23.800705+0000","last_peered":"2023-10-06T11:18:23.800705+0000","last_clean":"2023-10-06T11:18:23.800705+0000","last_became_active":"2023-10-06T11:18:23.794305+0000","last_became_peered":"2023-10-06T11:18:23.794305+0000","last_unstale":"2023-10-06T11:18:23.800705+0000","last_undegraded":"2023-10-06T11:18:23.800705+0000","last_fullsized":"2023-10-06T11:18:23.800705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:35.500496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808978+0000","last_change":"2023-10-06T11:18:23.796286+0000","last_active":"2023-10-06T11:18:23.808978+0000","last_peered":"2023-10-06T11:18:23.808978+0000","last_clean":"2023-10-06T11:18:23.808978+0000","last_became_active":"2023-10-06T11:18:23.795746+0000","last_became_peered":"2023-10-06T11:18:23.795746+0000","last_unstale":"2023-10-06T11:18:23.808978+0000","last_undegraded":"2023-10-06T11:18:23.808978+0000","last_fullsized":"2023-10-06T11:18:23.808978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802203+0000","last_change":"2023-10-06T11:18:23.793146+0000","last_active":"2023-10-06T11:18:23.802203+0000","last_peered":"2023-10-06T11:18:23.802203+0000","last_clean":"2023-10-06T11:18:23.802203+0000","last_became_active":"2023-10-06T11:18:23.792852+0000","last_became_peered":"2023-10-06T11:18:23.792852+0000","last_unstale":"2023-10-06T11:18:23.802203+0000","last_undegraded":"2023-10-06T11:18:23.802203+0000","last_fullsized":"2023-10-06T11:18:23.802203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:31.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810657+0000","last_change":"2023-10-06T11:18:23.809263+0000","last_active":"2023-10-06T11:18:23.810657+0000","last_peered":"2023-10-06T11:18:23.810657+0000","last_clean":"2023-10-06T11:18:23.810657+0000","last_became_active":"2023-10-06T11:18:23.808943+0000","last_became_peered":"2023-10-06T11:18:23.808943+0000","last_unstale":"2023-10-06T11:18:23.810657+0000","last_undegraded":"2023-10-06T11:18:23.810657+0000","last_fullsized":"2023-10-06T11:18:23.810657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.789862+0000","last_change":"2023-10-06T11:18:23.788701+0000","last_active":"2023-10-06T11:18:23.789862+0000","last_peered":"2023-10-06T11:18:23.789862+0000","last_clean":"2023-10-06T11:18:23.789862+0000","last_became_active":"2023-10-06T11:18:23.788343+0000","last_became_peered":"2023-10-06T11:18:23.788343+0000","last_unstale":"2023-10-06T11:18:23.789862+0000","last_undegraded":"2023-10-06T11:18:23.789862+0000","last_fullsized":"2023-10-06T11:18:23.789862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809400+0000","last_change":"2023-10-06T11:18:23.802504+0000","last_active":"2023-10-06T11:18:23.809400+0000","last_peered":"2023-10-06T11:18:23.809400+0000","last_clean":"2023-10-06T11:18:23.809400+0000","last_became_active":"2023-10-06T11:18:23.801692+0000","last_became_peered":"2023-10-06T11:18:23.801692+0000","last_unstale":"2023-10-06T11:18:23.809400+0000","last_undegraded":"2023-10-06T11:18:23.809400+0000","last_fullsized":"2023-10-06T11:18:23.809400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800609+0000","last_change":"2023-10-06T11:18:23.795845+0000","last_active":"2023-10-06T11:18:23.800609+0000","last_peered":"2023-10-06T11:18:23.800609+0000","last_clean":"2023-10-06T11:18:23.800609+0000","last_became_active":"2023-10-06T11:18:23.795351+0000","last_became_peered":"2023-10-06T11:18:23.795351+0000","last_unstale":"2023-10-06T11:18:23.800609+0000","last_undegraded":"2023-10-06T11:18:23.800609+0000","last_fullsized":"2023-10-06T11:18:23.800609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810148+0000","last_change":"2023-10-06T11:18:23.805599+0000","last_active":"2023-10-06T11:18:23.810148+0000","last_peered":"2023-10-06T11:18:23.810148+0000","last_clean":"2023-10-06T11:18:23.810148+0000","last_became_active":"2023-10-06T11:18:23.805371+0000","last_became_peered":"2023-10-06T11:18:23.805371+0000","last_unstale":"2023-10-06T11:18:23.810148+0000","last_undegraded":"2023-10-06T11:18:23.810148+0000","last_fullsized":"2023-10-06T11:18:23.810148+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:25.984507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802009+0000","last_change":"2023-10-06T11:18:23.799204+0000","last_active":"2023-10-06T11:18:23.802009+0000","last_peered":"2023-10-06T11:18:23.802009+0000","last_clean":"2023-10-06T11:18:23.802009+0000","last_became_active":"2023-10-06T11:18:23.798864+0000","last_became_peered":"2023-10-06T11:18:23.798864+0000","last_unstale":"2023-10-06T11:18:23.802009+0000","last_undegraded":"2023-10-06T11:18:23.802009+0000","last_fullsized":"2023-10-06T11:18:23.802009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.799186+0000","last_change":"2023-10-06T11:18:23.795240+0000","last_active":"2023-10-06T11:18:23.799186+0000","last_peered":"2023-10-06T11:18:23.799186+0000","last_clean":"2023-10-06T11:18:23.799186+0000","last_became_active":"2023-10-06T11:18:23.794759+0000","last_became_peered":"2023-10-06T11:18:23.794759+0000","last_unstale":"2023-10-06T11:18:23.799186+0000","last_undegraded":"2023-10-06T11:18:23.799186+0000","last_fullsized":"2023-10-06T11:18:23.799186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:08.424249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.791279+0000","last_change":"2023-10-06T11:18:23.789935+0000","last_active":"2023-10-06T11:18:23.791279+0000","last_peered":"2023-10-06T11:18:23.791279+0000","last_clean":"2023-10-06T11:18:23.791279+0000","last_became_active":"2023-10-06T11:18:23.789663+0000","last_became_peered":"2023-10-06T11:18:23.789663+0000","last_unstale":"2023-10-06T11:18:23.791279+0000","last_undegraded":"2023-10-06T11:18:23.791279+0000","last_fullsized":"2023-10-06T11:18:23.791279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810670+0000","last_change":"2023-10-06T11:18:23.809338+0000","last_active":"2023-10-06T11:18:23.810670+0000","last_peered":"2023-10-06T11:18:23.810670+0000","last_clean":"2023-10-06T11:18:23.810670+0000","last_became_active":"2023-10-06T11:18:23.808949+0000","last_became_peered":"2023-10-06T11:18:23.808949+0000","last_unstale":"2023-10-06T11:18:23.810670+0000","last_undegraded":"2023-10-06T11:18:23.810670+0000","last_fullsized":"2023-10-06T11:18:23.810670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802664+0000","last_change":"2023-10-06T11:18:23.794359+0000","last_active":"2023-10-06T11:18:23.802664+0000","last_peered":"2023-10-06T11:18:23.802664+0000","last_clean":"2023-10-06T11:18:23.802664+0000","last_became_active":"2023-10-06T11:18:23.794050+0000","last_became_peered":"2023-10-06T11:18:23.794050+0000","last_unstale":"2023-10-06T11:18:23.802664+0000","last_undegraded":"2023-10-06T11:18:23.802664+0000","last_fullsized":"2023-10-06T11:18:23.802664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809419+0000","last_change":"2023-10-06T11:18:23.802662+0000","last_active":"2023-10-06T11:18:23.809419+0000","last_peered":"2023-10-06T11:18:23.809419+0000","last_clean":"2023-10-06T11:18:23.809419+0000","last_became_active":"2023-10-06T11:18:23.802233+0000","last_became_peered":"2023-10-06T11:18:23.802233+0000","last_unstale":"2023-10-06T11:18:23.809419+0000","last_undegraded":"2023-10-06T11:18:23.809419+0000","last_fullsized":"2023-10-06T11:18:23.809419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802467+0000","last_change":"2023-10-06T11:18:23.795210+0000","last_active":"2023-10-06T11:18:23.802467+0000","last_peered":"2023-10-06T11:18:23.802467+0000","last_clean":"2023-10-06T11:18:23.802467+0000","last_became_active":"2023-10-06T11:18:23.794877+0000","last_became_peered":"2023-10-06T11:18:23.794877+0000","last_unstale":"2023-10-06T11:18:23.802467+0000","last_undegraded":"2023-10-06T11:18:23.802467+0000","last_fullsized":"2023-10-06T11:18:23.802467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:16.625365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797049+0000","last_change":"2023-10-06T11:18:23.789054+0000","last_active":"2023-10-06T11:18:23.797049+0000","last_peered":"2023-10-06T11:18:23.797049+0000","last_clean":"2023-10-06T11:18:23.797049+0000","last_became_active":"2023-10-06T11:18:23.788629+0000","last_became_peered":"2023-10-06T11:18:23.788629+0000","last_unstale":"2023-10-06T11:18:23.797049+0000","last_undegraded":"2023-10-06T11:18:23.797049+0000","last_fullsized":"2023-10-06T11:18:23.797049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:37.399272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802330+0000","last_change":"2023-10-06T11:18:23.792433+0000","last_active":"2023-10-06T11:18:23.802330+0000","last_peered":"2023-10-06T11:18:23.802330+0000","last_clean":"2023-10-06T11:18:23.802330+0000","last_became_active":"2023-10-06T11:18:23.792138+0000","last_became_peered":"2023-10-06T11:18:23.792138+0000","last_unstale":"2023-10-06T11:18:23.802330+0000","last_undegraded":"2023-10-06T11:18:23.802330+0000","last_fullsized":"2023-10-06T11:18:23.802330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801919+0000","last_change":"2023-10-06T11:18:23.797785+0000","last_active":"2023-10-06T11:18:23.801919+0000","last_peered":"2023-10-06T11:18:23.801919+0000","last_clean":"2023-10-06T11:18:23.801919+0000","last_became_active":"2023-10-06T11:18:23.797415+0000","last_became_peered":"2023-10-06T11:18:23.797415+0000","last_unstale":"2023-10-06T11:18:23.801919+0000","last_undegraded":"2023-10-06T11:18:23.801919+0000","last_fullsized":"2023-10-06T11:18:23.801919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810255+0000","last_change":"2023-10-06T11:18:23.808011+0000","last_active":"2023-10-06T11:18:23.810255+0000","last_peered":"2023-10-06T11:18:23.810255+0000","last_clean":"2023-10-06T11:18:23.810255+0000","last_became_active":"2023-10-06T11:18:23.807627+0000","last_became_peered":"2023-10-06T11:18:23.807627+0000","last_unstale":"2023-10-06T11:18:23.810255+0000","last_undegraded":"2023-10-06T11:18:23.810255+0000","last_fullsized":"2023-10-06T11:18:23.810255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797697+0000","last_change":"2023-10-06T11:18:23.794460+0000","last_active":"2023-10-06T11:18:23.797697+0000","last_peered":"2023-10-06T11:18:23.797697+0000","last_clean":"2023-10-06T11:18:23.797697+0000","last_became_active":"2023-10-06T11:18:23.794145+0000","last_became_peered":"2023-10-06T11:18:23.794145+0000","last_unstale":"2023-10-06T11:18:23.797697+0000","last_undegraded":"2023-10-06T11:18:23.797697+0000","last_fullsized":"2023-10-06T11:18:23.797697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804691+0000","last_change":"2023-10-06T11:18:23.800752+0000","last_active":"2023-10-06T11:18:23.804691+0000","last_peered":"2023-10-06T11:18:23.804691+0000","last_clean":"2023-10-06T11:18:23.804691+0000","last_became_active":"2023-10-06T11:18:23.798826+0000","last_became_peered":"2023-10-06T11:18:23.798826+0000","last_unstale":"2023-10-06T11:18:23.804691+0000","last_undegraded":"2023-10-06T11:18:23.804691+0000","last_fullsized":"2023-10-06T11:18:23.804691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804606+0000","last_change":"2023-10-06T11:18:23.801213+0000","last_active":"2023-10-06T11:18:23.804606+0000","last_peered":"2023-10-06T11:18:23.804606+0000","last_clean":"2023-10-06T11:18:23.804606+0000","last_became_active":"2023-10-06T11:18:23.800123+0000","last_became_peered":"2023-10-06T11:18:23.800123+0000","last_unstale":"2023-10-06T11:18:23.804606+0000","last_undegraded":"2023-10-06T11:18:23.804606+0000","last_fullsized":"2023-10-06T11:18:23.804606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803343+0000","last_change":"2023-10-06T11:18:23.795738+0000","last_active":"2023-10-06T11:18:23.803343+0000","last_peered":"2023-10-06T11:18:23.803343+0000","last_clean":"2023-10-06T11:18:23.803343+0000","last_became_active":"2023-10-06T11:18:23.795423+0000","last_became_peered":"2023-10-06T11:18:23.795423+0000","last_unstale":"2023-10-06T11:18:23.803343+0000","last_undegraded":"2023-10-06T11:18:23.803343+0000","last_fullsized":"2023-10-06T11:18:23.803343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800576+0000","last_change":"2023-10-06T11:18:23.794728+0000","last_active":"2023-10-06T11:18:23.800576+0000","last_peered":"2023-10-06T11:18:23.800576+0000","last_clean":"2023-10-06T11:18:23.800576+0000","last_became_active":"2023-10-06T11:18:23.794324+0000","last_became_peered":"2023-10-06T11:18:23.794324+0000","last_unstale":"2023-10-06T11:18:23.800576+0000","last_undegraded":"2023-10-06T11:18:23.800576+0000","last_fullsized":"2023-10-06T11:18:23.800576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.811964+0000","last_change":"2023-10-06T11:18:23.809631+0000","last_active":"2023-10-06T11:18:23.811964+0000","last_peered":"2023-10-06T11:18:23.811964+0000","last_clean":"2023-10-06T11:18:23.811964+0000","last_became_active":"2023-10-06T11:18:23.806649+0000","last_became_peered":"2023-10-06T11:18:23.806649+0000","last_unstale":"2023-10-06T11:18:23.811964+0000","last_undegraded":"2023-10-06T11:18:23.811964+0000","last_fullsized":"2023-10-06T11:18:23.811964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:25.521938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803531+0000","last_change":"2023-10-06T11:18:23.800427+0000","last_active":"2023-10-06T11:18:23.803531+0000","last_peered":"2023-10-06T11:18:23.803531+0000","last_clean":"2023-10-06T11:18:23.803531+0000","last_became_active":"2023-10-06T11:18:23.799956+0000","last_became_peered":"2023-10-06T11:18:23.799956+0000","last_unstale":"2023-10-06T11:18:23.803531+0000","last_undegraded":"2023-10-06T11:18:23.803531+0000","last_fullsized":"2023-10-06T11:18:23.803531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802221+0000","last_change":"2023-10-06T11:18:23.793233+0000","last_active":"2023-10-06T11:18:23.802221+0000","last_peered":"2023-10-06T11:18:23.802221+0000","last_clean":"2023-10-06T11:18:23.802221+0000","last_became_active":"2023-10-06T11:18:23.792917+0000","last_became_peered":"2023-10-06T11:18:23.792917+0000","last_unstale":"2023-10-06T11:18:23.802221+0000","last_undegraded":"2023-10-06T11:18:23.802221+0000","last_fullsized":"2023-10-06T11:18:23.802221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:01:00.993266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812686+0000","last_change":"2023-10-06T11:18:23.811339+0000","last_active":"2023-10-06T11:18:23.812686+0000","last_peered":"2023-10-06T11:18:23.812686+0000","last_clean":"2023-10-06T11:18:23.812686+0000","last_became_active":"2023-10-06T11:18:23.810982+0000","last_became_peered":"2023-10-06T11:18:23.810982+0000","last_unstale":"2023-10-06T11:18:23.812686+0000","last_undegraded":"2023-10-06T11:18:23.812686+0000","last_fullsized":"2023-10-06T11:18:23.812686+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:53.104908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800733+0000","last_change":"2023-10-06T11:18:23.796079+0000","last_active":"2023-10-06T11:18:23.800733+0000","last_peered":"2023-10-06T11:18:23.800733+0000","last_clean":"2023-10-06T11:18:23.800733+0000","last_became_active":"2023-10-06T11:18:23.795767+0000","last_became_peered":"2023-10-06T11:18:23.795767+0000","last_unstale":"2023-10-06T11:18:23.800733+0000","last_undegraded":"2023-10-06T11:18:23.800733+0000","last_fullsized":"2023-10-06T11:18:23.800733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:24.987705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812681+0000","last_change":"2023-10-06T11:18:23.811468+0000","last_active":"2023-10-06T11:18:23.812681+0000","last_peered":"2023-10-06T11:18:23.812681+0000","last_clean":"2023-10-06T11:18:23.812681+0000","last_became_active":"2023-10-06T11:18:23.811047+0000","last_became_peered":"2023-10-06T11:18:23.811047+0000","last_unstale":"2023-10-06T11:18:23.812681+0000","last_undegraded":"2023-10-06T11:18:23.812681+0000","last_fullsized":"2023-10-06T11:18:23.812681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080733+0000","last_change":"2023-10-06T11:18:22.785631+0000","last_active":"2023-10-06T11:18:24.080733+0000","last_peered":"2023-10-06T11:18:24.080733+0000","last_clean":"2023-10-06T11:18:24.080733+0000","last_became_active":"2023-10-06T11:18:22.785163+0000","last_became_peered":"2023-10-06T11:18:22.785163+0000","last_unstale":"2023-10-06T11:18:24.080733+0000","last_undegraded":"2023-10-06T11:18:24.080733+0000","last_fullsized":"2023-10-06T11:18:24.080733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801774+0000","last_change":"2023-10-06T11:18:23.796325+0000","last_active":"2023-10-06T11:18:23.801774+0000","last_peered":"2023-10-06T11:18:23.801774+0000","last_clean":"2023-10-06T11:18:23.801774+0000","last_became_active":"2023-10-06T11:18:23.795981+0000","last_became_peered":"2023-10-06T11:18:23.795981+0000","last_unstale":"2023-10-06T11:18:23.801774+0000","last_undegraded":"2023-10-06T11:18:23.801774+0000","last_fullsized":"2023-10-06T11:18:23.801774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785579+0000","last_change":"2023-10-06T11:18:22.771501+0000","last_active":"2023-10-06T11:18:23.785579+0000","last_peered":"2023-10-06T11:18:23.785579+0000","last_clean":"2023-10-06T11:18:23.785579+0000","last_became_active":"2023-10-06T11:18:22.771165+0000","last_became_peered":"2023-10-06T11:18:22.771165+0000","last_unstale":"2023-10-06T11:18:23.785579+0000","last_undegraded":"2023-10-06T11:18:23.785579+0000","last_fullsized":"2023-10-06T11:18:23.785579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809479+0000","last_change":"2023-10-06T11:18:23.801731+0000","last_active":"2023-10-06T11:18:23.809479+0000","last_peered":"2023-10-06T11:18:23.809479+0000","last_clean":"2023-10-06T11:18:23.809479+0000","last_became_active":"2023-10-06T11:18:23.801224+0000","last_became_peered":"2023-10-06T11:18:23.801224+0000","last_unstale":"2023-10-06T11:18:23.809479+0000","last_undegraded":"2023-10-06T11:18:23.809479+0000","last_fullsized":"2023-10-06T11:18:23.809479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787033+0000","last_change":"2023-10-06T11:18:22.783842+0000","last_active":"2023-10-06T11:18:23.787033+0000","last_peered":"2023-10-06T11:18:23.787033+0000","last_clean":"2023-10-06T11:18:23.787033+0000","last_became_active":"2023-10-06T11:18:22.783033+0000","last_became_peered":"2023-10-06T11:18:22.783033+0000","last_unstale":"2023-10-06T11:18:23.787033+0000","last_undegraded":"2023-10-06T11:18:23.787033+0000","last_fullsized":"2023-10-06T11:18:23.787033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797362+0000","last_change":"2023-10-06T11:18:23.793261+0000","last_active":"2023-10-06T11:18:23.797362+0000","last_peered":"2023-10-06T11:18:23.797362+0000","last_clean":"2023-10-06T11:18:23.797362+0000","last_became_active":"2023-10-06T11:18:23.792951+0000","last_became_peered":"2023-10-06T11:18:23.792951+0000","last_unstale":"2023-10-06T11:18:23.797362+0000","last_undegraded":"2023-10-06T11:18:23.797362+0000","last_fullsized":"2023-10-06T11:18:23.797362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.082723+0000","last_change":"2023-10-06T11:18:22.773438+0000","last_active":"2023-10-06T11:18:24.082723+0000","last_peered":"2023-10-06T11:18:24.082723+0000","last_clean":"2023-10-06T11:18:24.082723+0000","last_became_active":"2023-10-06T11:18:22.773047+0000","last_became_peered":"2023-10-06T11:18:22.773047+0000","last_unstale":"2023-10-06T11:18:24.082723+0000","last_undegraded":"2023-10-06T11:18:24.082723+0000","last_fullsized":"2023-10-06T11:18:24.082723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080613+0000","last_change":"2023-10-06T11:18:22.783414+0000","last_active":"2023-10-06T11:18:24.080613+0000","last_peered":"2023-10-06T11:18:24.080613+0000","last_clean":"2023-10-06T11:18:24.080613+0000","last_became_active":"2023-10-06T11:18:22.783068+0000","last_became_peered":"2023-10-06T11:18:22.783068+0000","last_unstale":"2023-10-06T11:18:24.080613+0000","last_undegraded":"2023-10-06T11:18:24.080613+0000","last_fullsized":"2023-10-06T11:18:24.080613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.794917+0000","last_change":"2023-10-06T11:18:23.792041+0000","last_active":"2023-10-06T11:18:23.794917+0000","last_peered":"2023-10-06T11:18:23.794917+0000","last_clean":"2023-10-06T11:18:23.794917+0000","last_became_active":"2023-10-06T11:18:23.791770+0000","last_became_peered":"2023-10-06T11:18:23.791770+0000","last_unstale":"2023-10-06T11:18:23.794917+0000","last_undegraded":"2023-10-06T11:18:23.794917+0000","last_fullsized":"2023-10-06T11:18:23.794917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081788+0000","last_change":"2023-10-06T11:18:22.783471+0000","last_active":"2023-10-06T11:18:24.081788+0000","last_peered":"2023-10-06T11:18:24.081788+0000","last_clean":"2023-10-06T11:18:24.081788+0000","last_became_active":"2023-10-06T11:18:22.783150+0000","last_became_peered":"2023-10-06T11:18:22.783150+0000","last_unstale":"2023-10-06T11:18:24.081788+0000","last_undegraded":"2023-10-06T11:18:24.081788+0000","last_fullsized":"2023-10-06T11:18:24.081788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:57.801692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802567+0000","last_change":"2023-10-06T11:18:23.797336+0000","last_active":"2023-10-06T11:18:23.802567+0000","last_peered":"2023-10-06T11:18:23.802567+0000","last_clean":"2023-10-06T11:18:23.802567+0000","last_became_active":"2023-10-06T11:18:23.797002+0000","last_became_peered":"2023-10-06T11:18:23.797002+0000","last_unstale":"2023-10-06T11:18:23.802567+0000","last_undegraded":"2023-10-06T11:18:23.802567+0000","last_fullsized":"2023-10-06T11:18:23.802567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:58.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081680+0000","last_change":"2023-10-06T11:18:22.777073+0000","last_active":"2023-10-06T11:18:24.081680+0000","last_peered":"2023-10-06T11:18:24.081680+0000","last_clean":"2023-10-06T11:18:24.081680+0000","last_became_active":"2023-10-06T11:18:22.776784+0000","last_became_peered":"2023-10-06T11:18:22.776784+0000","last_unstale":"2023-10-06T11:18:24.081680+0000","last_undegraded":"2023-10-06T11:18:24.081680+0000","last_fullsized":"2023-10-06T11:18:24.081680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810078+0000","last_change":"2023-10-06T11:18:23.807687+0000","last_active":"2023-10-06T11:18:23.810078+0000","last_peered":"2023-10-06T11:18:23.810078+0000","last_clean":"2023-10-06T11:18:23.810078+0000","last_became_active":"2023-10-06T11:18:23.807468+0000","last_became_peered":"2023-10-06T11:18:23.807468+0000","last_unstale":"2023-10-06T11:18:23.810078+0000","last_undegraded":"2023-10-06T11:18:23.810078+0000","last_fullsized":"2023-10-06T11:18:23.810078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:11.773581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777670+0000","last_change":"2023-10-06T11:18:22.770019+0000","last_active":"2023-10-06T11:18:23.777670+0000","last_peered":"2023-10-06T11:18:23.777670+0000","last_clean":"2023-10-06T11:18:23.777670+0000","last_became_active":"2023-10-06T11:18:22.769681+0000","last_became_peered":"2023-10-06T11:18:22.769681+0000","last_unstale":"2023-10-06T11:18:23.777670+0000","last_undegraded":"2023-10-06T11:18:23.777670+0000","last_fullsized":"2023-10-06T11:18:23.777670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:34.485710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809159+0000","last_change":"2023-10-06T11:18:23.804489+0000","last_active":"2023-10-06T11:18:23.809159+0000","last_peered":"2023-10-06T11:18:23.809159+0000","last_clean":"2023-10-06T11:18:23.809159+0000","last_became_active":"2023-10-06T11:18:23.804154+0000","last_became_peered":"2023-10-06T11:18:23.804154+0000","last_unstale":"2023-10-06T11:18:23.809159+0000","last_undegraded":"2023-10-06T11:18:23.809159+0000","last_fullsized":"2023-10-06T11:18:23.809159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079612+0000","last_change":"2023-10-06T11:18:22.783172+0000","last_active":"2023-10-06T11:18:24.079612+0000","last_peered":"2023-10-06T11:18:24.079612+0000","last_clean":"2023-10-06T11:18:24.079612+0000","last_became_active":"2023-10-06T11:18:22.782848+0000","last_became_peered":"2023-10-06T11:18:22.782848+0000","last_unstale":"2023-10-06T11:18:24.079612+0000","last_undegraded":"2023-10-06T11:18:24.079612+0000","last_fullsized":"2023-10-06T11:18:24.079612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804625+0000","last_change":"2023-10-06T11:18:23.799401+0000","last_active":"2023-10-06T11:18:23.804625+0000","last_peered":"2023-10-06T11:18:23.804625+0000","last_clean":"2023-10-06T11:18:23.804625+0000","last_became_active":"2023-10-06T11:18:23.799110+0000","last_became_peered":"2023-10-06T11:18:23.799110+0000","last_unstale":"2023-10-06T11:18:23.804625+0000","last_undegraded":"2023-10-06T11:18:23.804625+0000","last_fullsized":"2023-10-06T11:18:23.804625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.787968+0000","last_change":"2023-10-06T11:18:22.788211+0000","last_active":"2023-10-06T11:18:23.787968+0000","last_peered":"2023-10-06T11:18:23.787968+0000","last_clean":"2023-10-06T11:18:23.787968+0000","last_became_active":"2023-10-06T11:18:22.787835+0000","last_became_peered":"2023-10-06T11:18:22.787835+0000","last_unstale":"2023-10-06T11:18:23.787968+0000","last_undegraded":"2023-10-06T11:18:23.787968+0000","last_fullsized":"2023-10-06T11:18:23.787968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802350+0000","last_change":"2023-10-06T11:18:23.794251+0000","last_active":"2023-10-06T11:18:23.802350+0000","last_peered":"2023-10-06T11:18:23.802350+0000","last_clean":"2023-10-06T11:18:23.802350+0000","last_became_active":"2023-10-06T11:18:23.793840+0000","last_became_peered":"2023-10-06T11:18:23.793840+0000","last_unstale":"2023-10-06T11:18:23.802350+0000","last_undegraded":"2023-10-06T11:18:23.802350+0000","last_fullsized":"2023-10-06T11:18:23.802350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:26.606422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.780768+0000","last_change":"2023-10-06T11:18:22.786197+0000","last_active":"2023-10-06T11:18:23.780768+0000","last_peered":"2023-10-06T11:18:23.780768+0000","last_clean":"2023-10-06T11:18:23.780768+0000","last_became_active":"2023-10-06T11:18:22.785533+0000","last_became_peered":"2023-10-06T11:18:22.785533+0000","last_unstale":"2023-10-06T11:18:23.780768+0000","last_undegraded":"2023-10-06T11:18:23.780768+0000","last_fullsized":"2023-10-06T11:18:23.780768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:46.908854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802002+0000","last_change":"2023-10-06T11:18:23.792349+0000","last_active":"2023-10-06T11:18:23.802002+0000","last_peered":"2023-10-06T11:18:23.802002+0000","last_clean":"2023-10-06T11:18:23.802002+0000","last_became_active":"2023-10-06T11:18:23.792058+0000","last_became_peered":"2023-10-06T11:18:23.792058+0000","last_unstale":"2023-10-06T11:18:23.802002+0000","last_undegraded":"2023-10-06T11:18:23.802002+0000","last_fullsized":"2023-10-06T11:18:23.802002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:30.680013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781709+0000","last_change":"2023-10-06T11:18:22.778447+0000","last_active":"2023-10-06T11:18:23.781709+0000","last_peered":"2023-10-06T11:18:23.781709+0000","last_clean":"2023-10-06T11:18:23.781709+0000","last_became_active":"2023-10-06T11:18:22.778005+0000","last_became_peered":"2023-10-06T11:18:22.778005+0000","last_unstale":"2023-10-06T11:18:23.781709+0000","last_undegraded":"2023-10-06T11:18:23.781709+0000","last_fullsized":"2023-10-06T11:18:23.781709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808325+0000","last_change":"2023-10-06T11:18:23.805117+0000","last_active":"2023-10-06T11:18:23.808325+0000","last_peered":"2023-10-06T11:18:23.808325+0000","last_clean":"2023-10-06T11:18:23.808325+0000","last_became_active":"2023-10-06T11:18:23.804771+0000","last_became_peered":"2023-10-06T11:18:23.804771+0000","last_unstale":"2023-10-06T11:18:23.808325+0000","last_undegraded":"2023-10-06T11:18:23.808325+0000","last_fullsized":"2023-10-06T11:18:23.808325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.661054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785628+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:18:23.785628+0000","last_peered":"2023-10-06T11:18:23.785628+0000","last_clean":"2023-10-06T11:18:23.785628+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:18:23.785628+0000","last_undegraded":"2023-10-06T11:18:23.785628+0000","last_fullsized":"2023-10-06T11:18:23.785628+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777308+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:18:23.777308+0000","last_peered":"2023-10-06T11:18:23.777308+0000","last_clean":"2023-10-06T11:18:23.777308+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:18:23.777308+0000","last_undegraded":"2023-10-06T11:18:23.777308+0000","last_fullsized":"2023-10-06T11:18:23.777308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.783667+0000","last_change":"2023-10-06T11:18:22.774869+0000","last_active":"2023-10-06T11:18:23.783667+0000","last_peered":"2023-10-06T11:18:23.783667+0000","last_clean":"2023-10-06T11:18:23.783667+0000","last_became_active":"2023-10-06T11:18:22.773053+0000","last_became_peered":"2023-10-06T11:18:22.773053+0000","last_unstale":"2023-10-06T11:18:23.783667+0000","last_undegraded":"2023-10-06T11:18:23.783667+0000","last_fullsized":"2023-10-06T11:18:23.783667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.800870+0000","last_change":"2023-10-06T11:18:23.798289+0000","last_active":"2023-10-06T11:18:23.800870+0000","last_peered":"2023-10-06T11:18:23.800870+0000","last_clean":"2023-10-06T11:18:23.800870+0000","last_became_active":"2023-10-06T11:18:23.797843+0000","last_became_peered":"2023-10-06T11:18:23.797843+0000","last_unstale":"2023-10-06T11:18:23.800870+0000","last_undegraded":"2023-10-06T11:18:23.800870+0000","last_fullsized":"2023-10-06T11:18:23.800870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:58.816478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.785539+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:18:23.785539+0000","last_peered":"2023-10-06T11:18:23.785539+0000","last_clean":"2023-10-06T11:18:23.785539+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:18:23.785539+0000","last_undegraded":"2023-10-06T11:18:23.785539+0000","last_fullsized":"2023-10-06T11:18:23.785539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.779941+0000","last_change":"2023-10-06T11:18:22.784973+0000","last_active":"2023-10-06T11:18:23.779941+0000","last_peered":"2023-10-06T11:18:23.779941+0000","last_clean":"2023-10-06T11:18:23.779941+0000","last_became_active":"2023-10-06T11:18:22.784647+0000","last_became_peered":"2023-10-06T11:18:22.784647+0000","last_unstale":"2023-10-06T11:18:23.779941+0000","last_undegraded":"2023-10-06T11:18:23.779941+0000","last_fullsized":"2023-10-06T11:18:23.779941+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.810291+0000","last_change":"2023-10-06T11:18:23.808610+0000","last_active":"2023-10-06T11:18:23.810291+0000","last_peered":"2023-10-06T11:18:23.810291+0000","last_clean":"2023-10-06T11:18:23.810291+0000","last_became_active":"2023-10-06T11:18:23.808298+0000","last_became_peered":"2023-10-06T11:18:23.808298+0000","last_unstale":"2023-10-06T11:18:23.810291+0000","last_undegraded":"2023-10-06T11:18:23.810291+0000","last_fullsized":"2023-10-06T11:18:23.810291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.773939+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:18:23.773939+0000","last_peered":"2023-10-06T11:18:23.773939+0000","last_clean":"2023-10-06T11:18:23.773939+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:18:23.773939+0000","last_undegraded":"2023-10-06T11:18:23.773939+0000","last_fullsized":"2023-10-06T11:18:23.773939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.798493+0000","last_change":"2023-10-06T11:18:22.782337+0000","last_active":"2023-10-06T11:18:23.798493+0000","last_peered":"2023-10-06T11:18:23.798493+0000","last_clean":"2023-10-06T11:18:23.798493+0000","last_became_active":"2023-10-06T11:18:22.780763+0000","last_became_peered":"2023-10-06T11:18:22.780763+0000","last_unstale":"2023-10-06T11:18:23.798493+0000","last_undegraded":"2023-10-06T11:18:23.798493+0000","last_fullsized":"2023-10-06T11:18:23.798493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804588+0000","last_change":"2023-10-06T11:18:23.798843+0000","last_active":"2023-10-06T11:18:23.804588+0000","last_peered":"2023-10-06T11:18:23.804588+0000","last_clean":"2023-10-06T11:18:23.804588+0000","last_became_active":"2023-10-06T11:18:23.798591+0000","last_became_peered":"2023-10-06T11:18:23.798591+0000","last_unstale":"2023-10-06T11:18:23.804588+0000","last_undegraded":"2023-10-06T11:18:23.804588+0000","last_fullsized":"2023-10-06T11:18:23.804588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:09.880019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777790+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:18:23.777790+0000","last_peered":"2023-10-06T11:18:23.777790+0000","last_clean":"2023-10-06T11:18:23.777790+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:18:23.777790+0000","last_undegraded":"2023-10-06T11:18:23.777790+0000","last_fullsized":"2023-10-06T11:18:23.777790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781811+0000","last_change":"2023-10-06T11:18:22.781890+0000","last_active":"2023-10-06T11:18:23.781811+0000","last_peered":"2023-10-06T11:18:23.781811+0000","last_clean":"2023-10-06T11:18:23.781811+0000","last_became_active":"2023-10-06T11:18:22.781589+0000","last_became_peered":"2023-10-06T11:18:22.781589+0000","last_unstale":"2023-10-06T11:18:23.781811+0000","last_undegraded":"2023-10-06T11:18:23.781811+0000","last_fullsized":"2023-10-06T11:18:23.781811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:46.714907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806464+0000","last_change":"2023-10-06T11:18:23.803070+0000","last_active":"2023-10-06T11:18:23.806464+0000","last_peered":"2023-10-06T11:18:23.806464+0000","last_clean":"2023-10-06T11:18:23.806464+0000","last_became_active":"2023-10-06T11:18:23.799605+0000","last_became_peered":"2023-10-06T11:18:23.799605+0000","last_unstale":"2023-10-06T11:18:23.806464+0000","last_undegraded":"2023-10-06T11:18:23.806464+0000","last_fullsized":"2023-10-06T11:18:23.806464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777975+0000","last_change":"2023-10-06T11:18:22.770766+0000","last_active":"2023-10-06T11:18:23.777975+0000","last_peered":"2023-10-06T11:18:23.777975+0000","last_clean":"2023-10-06T11:18:23.777975+0000","last_became_active":"2023-10-06T11:18:22.770433+0000","last_became_peered":"2023-10-06T11:18:22.770433+0000","last_unstale":"2023-10-06T11:18:23.777975+0000","last_undegraded":"2023-10-06T11:18:23.777975+0000","last_fullsized":"2023-10-06T11:18:23.777975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:50:23.972918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804918+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:18:23.804918+0000","last_peered":"2023-10-06T11:18:23.804918+0000","last_clean":"2023-10-06T11:18:23.804918+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:18:23.804918+0000","last_undegraded":"2023-10-06T11:18:23.804918+0000","last_fullsized":"2023-10-06T11:18:23.804918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.806290+0000","last_change":"2023-10-06T11:18:23.804833+0000","last_active":"2023-10-06T11:18:23.806290+0000","last_peered":"2023-10-06T11:18:23.806290+0000","last_clean":"2023-10-06T11:18:23.806290+0000","last_became_active":"2023-10-06T11:18:23.804036+0000","last_became_peered":"2023-10-06T11:18:23.804036+0000","last_unstale":"2023-10-06T11:18:23.806290+0000","last_undegraded":"2023-10-06T11:18:23.806290+0000","last_fullsized":"2023-10-06T11:18:23.806290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:51.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804802+0000","last_change":"2023-10-06T11:18:22.783968+0000","last_active":"2023-10-06T11:18:23.804802+0000","last_peered":"2023-10-06T11:18:23.804802+0000","last_clean":"2023-10-06T11:18:23.804802+0000","last_became_active":"2023-10-06T11:18:22.783402+0000","last_became_peered":"2023-10-06T11:18:22.783402+0000","last_unstale":"2023-10-06T11:18:23.804802+0000","last_undegraded":"2023-10-06T11:18:23.804802+0000","last_fullsized":"2023-10-06T11:18:23.804802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.786168+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:18:23.786168+0000","last_peered":"2023-10-06T11:18:23.786168+0000","last_clean":"2023-10-06T11:18:23.786168+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:18:23.786168+0000","last_undegraded":"2023-10-06T11:18:23.786168+0000","last_fullsized":"2023-10-06T11:18:23.786168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808954+0000","last_change":"2023-10-06T11:18:23.792986+0000","last_active":"2023-10-06T11:18:23.808954+0000","last_peered":"2023-10-06T11:18:23.808954+0000","last_clean":"2023-10-06T11:18:23.808954+0000","last_became_active":"2023-10-06T11:18:23.792504+0000","last_became_peered":"2023-10-06T11:18:23.792504+0000","last_unstale":"2023-10-06T11:18:23.808954+0000","last_undegraded":"2023-10-06T11:18:23.808954+0000","last_fullsized":"2023-10-06T11:18:23.808954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:29.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079305+0000","last_change":"2023-10-06T11:18:22.773444+0000","last_active":"2023-10-06T11:18:24.079305+0000","last_peered":"2023-10-06T11:18:24.079305+0000","last_clean":"2023-10-06T11:18:24.079305+0000","last_became_active":"2023-10-06T11:18:22.773056+0000","last_became_peered":"2023-10-06T11:18:22.773056+0000","last_unstale":"2023-10-06T11:18:24.079305+0000","last_undegraded":"2023-10-06T11:18:24.079305+0000","last_fullsized":"2023-10-06T11:18:24.079305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804439+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:18:23.804439+0000","last_peered":"2023-10-06T11:18:23.804439+0000","last_clean":"2023-10-06T11:18:23.804439+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:18:23.804439+0000","last_undegraded":"2023-10-06T11:18:23.804439+0000","last_fullsized":"2023-10-06T11:18:23.804439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801518+0000","last_change":"2023-10-06T11:18:23.799064+0000","last_active":"2023-10-06T11:18:23.801518+0000","last_peered":"2023-10-06T11:18:23.801518+0000","last_clean":"2023-10-06T11:18:23.801518+0000","last_became_active":"2023-10-06T11:18:23.798759+0000","last_became_peered":"2023-10-06T11:18:23.798759+0000","last_unstale":"2023-10-06T11:18:23.801518+0000","last_undegraded":"2023-10-06T11:18:23.801518+0000","last_fullsized":"2023-10-06T11:18:23.801518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T11:18:25.180698+0000","last_change":"2023-10-06T11:18:22.781854+0000","last_active":"2023-10-06T11:18:25.180698+0000","last_peered":"2023-10-06T11:18:25.180698+0000","last_clean":"2023-10-06T11:18:25.180698+0000","last_became_active":"2023-10-06T11:18:22.781438+0000","last_became_peered":"2023-10-06T11:18:22.781438+0000","last_unstale":"2023-10-06T11:18:25.180698+0000","last_undegraded":"2023-10-06T11:18:25.180698+0000","last_fullsized":"2023-10-06T11:18:25.180698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:36.384486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.788055+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:18:23.788055+0000","last_peered":"2023-10-06T11:18:23.788055+0000","last_clean":"2023-10-06T11:18:23.788055+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:18:23.788055+0000","last_undegraded":"2023-10-06T11:18:23.788055+0000","last_fullsized":"2023-10-06T11:18:23.788055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.801967+0000","last_change":"2023-10-06T11:18:23.786622+0000","last_active":"2023-10-06T11:18:23.801967+0000","last_peered":"2023-10-06T11:18:23.801967+0000","last_clean":"2023-10-06T11:18:23.801967+0000","last_became_active":"2023-10-06T11:18:23.786299+0000","last_became_peered":"2023-10-06T11:18:23.786299+0000","last_unstale":"2023-10-06T11:18:23.801967+0000","last_undegraded":"2023-10-06T11:18:23.801967+0000","last_fullsized":"2023-10-06T11:18:23.801967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802096+0000","last_change":"2023-10-06T11:18:23.792771+0000","last_active":"2023-10-06T11:18:23.802096+0000","last_peered":"2023-10-06T11:18:23.802096+0000","last_clean":"2023-10-06T11:18:23.802096+0000","last_became_active":"2023-10-06T11:18:23.787870+0000","last_became_peered":"2023-10-06T11:18:23.787870+0000","last_unstale":"2023-10-06T11:18:23.802096+0000","last_undegraded":"2023-10-06T11:18:23.802096+0000","last_fullsized":"2023-10-06T11:18:23.802096+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.084229+0000","last_change":"2023-10-06T11:18:22.781178+0000","last_active":"2023-10-06T11:18:24.084229+0000","last_peered":"2023-10-06T11:18:24.084229+0000","last_clean":"2023-10-06T11:18:24.084229+0000","last_became_active":"2023-10-06T11:18:22.778268+0000","last_became_peered":"2023-10-06T11:18:22.778268+0000","last_unstale":"2023-10-06T11:18:24.084229+0000","last_undegraded":"2023-10-06T11:18:24.084229+0000","last_fullsized":"2023-10-06T11:18:24.084229+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:07.669018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804992+0000","last_change":"2023-10-06T11:18:23.800123+0000","last_active":"2023-10-06T11:18:23.804992+0000","last_peered":"2023-10-06T11:18:23.804992+0000","last_clean":"2023-10-06T11:18:23.804992+0000","last_became_active":"2023-10-06T11:18:23.799794+0000","last_became_peered":"2023-10-06T11:18:23.799794+0000","last_unstale":"2023-10-06T11:18:23.804992+0000","last_undegraded":"2023-10-06T11:18:23.804992+0000","last_fullsized":"2023-10-06T11:18:23.804992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:41.795949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.081932+0000","last_change":"2023-10-06T11:18:22.782320+0000","last_active":"2023-10-06T11:18:24.081932+0000","last_peered":"2023-10-06T11:18:24.081932+0000","last_clean":"2023-10-06T11:18:24.081932+0000","last_became_active":"2023-10-06T11:18:22.782025+0000","last_became_peered":"2023-10-06T11:18:22.782025+0000","last_unstale":"2023-10-06T11:18:24.081932+0000","last_undegraded":"2023-10-06T11:18:24.081932+0000","last_fullsized":"2023-10-06T11:18:24.081932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.777975+0000","last_change":"2023-10-06T11:18:22.773270+0000","last_active":"2023-10-06T11:18:23.777975+0000","last_peered":"2023-10-06T11:18:23.777975+0000","last_clean":"2023-10-06T11:18:23.777975+0000","last_became_active":"2023-10-06T11:18:22.772211+0000","last_became_peered":"2023-10-06T11:18:22.772211+0000","last_unstale":"2023-10-06T11:18:23.777975+0000","last_undegraded":"2023-10-06T11:18:23.777975+0000","last_fullsized":"2023-10-06T11:18:23.777975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.758795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.797678+0000","last_change":"2023-10-06T11:18:23.794332+0000","last_active":"2023-10-06T11:18:23.797678+0000","last_peered":"2023-10-06T11:18:23.797678+0000","last_clean":"2023-10-06T11:18:23.797678+0000","last_became_active":"2023-10-06T11:18:23.793993+0000","last_became_peered":"2023-10-06T11:18:23.793993+0000","last_unstale":"2023-10-06T11:18:23.797678+0000","last_undegraded":"2023-10-06T11:18:23.797678+0000","last_fullsized":"2023-10-06T11:18:23.797678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:13.984054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.783823+0000","last_change":"2023-10-06T11:18:22.774828+0000","last_active":"2023-10-06T11:18:23.783823+0000","last_peered":"2023-10-06T11:18:23.783823+0000","last_clean":"2023-10-06T11:18:23.783823+0000","last_became_active":"2023-10-06T11:18:22.774440+0000","last_became_peered":"2023-10-06T11:18:22.774440+0000","last_unstale":"2023-10-06T11:18:23.783823+0000","last_undegraded":"2023-10-06T11:18:23.783823+0000","last_fullsized":"2023-10-06T11:18:23.783823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:54.030770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.807787+0000","last_change":"2023-10-06T11:18:23.804188+0000","last_active":"2023-10-06T11:18:23.807787+0000","last_peered":"2023-10-06T11:18:23.807787+0000","last_clean":"2023-10-06T11:18:23.807787+0000","last_became_active":"2023-10-06T11:18:23.803863+0000","last_became_peered":"2023-10-06T11:18:23.803863+0000","last_unstale":"2023-10-06T11:18:23.807787+0000","last_undegraded":"2023-10-06T11:18:23.807787+0000","last_fullsized":"2023-10-06T11:18:23.807787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:10.418946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.792190+0000","last_change":"2023-10-06T11:18:23.789620+0000","last_active":"2023-10-06T11:18:23.792190+0000","last_peered":"2023-10-06T11:18:23.792190+0000","last_clean":"2023-10-06T11:18:23.792190+0000","last_became_active":"2023-10-06T11:18:23.789349+0000","last_became_peered":"2023-10-06T11:18:23.789349+0000","last_unstale":"2023-10-06T11:18:23.792190+0000","last_undegraded":"2023-10-06T11:18:23.792190+0000","last_fullsized":"2023-10-06T11:18:23.792190+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:26.747191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.781628+0000","last_change":"2023-10-06T11:18:22.781828+0000","last_active":"2023-10-06T11:18:23.781628+0000","last_peered":"2023-10-06T11:18:23.781628+0000","last_clean":"2023-10-06T11:18:23.781628+0000","last_became_active":"2023-10-06T11:18:22.781529+0000","last_became_peered":"2023-10-06T11:18:22.781529+0000","last_unstale":"2023-10-06T11:18:23.781628+0000","last_undegraded":"2023-10-06T11:18:23.781628+0000","last_fullsized":"2023-10-06T11:18:23.781628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.808495+0000","last_change":"2023-10-06T11:18:23.792292+0000","last_active":"2023-10-06T11:18:23.808495+0000","last_peered":"2023-10-06T11:18:23.808495+0000","last_clean":"2023-10-06T11:18:23.808495+0000","last_became_active":"2023-10-06T11:18:23.791847+0000","last_became_peered":"2023-10-06T11:18:23.791847+0000","last_unstale":"2023-10-06T11:18:23.808495+0000","last_undegraded":"2023-10-06T11:18:23.808495+0000","last_fullsized":"2023-10-06T11:18:23.808495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:20.672764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.803596+0000","last_change":"2023-10-06T11:18:22.774677+0000","last_active":"2023-10-06T11:18:23.803596+0000","last_peered":"2023-10-06T11:18:23.803596+0000","last_clean":"2023-10-06T11:18:23.803596+0000","last_became_active":"2023-10-06T11:18:22.772746+0000","last_became_peered":"2023-10-06T11:18:22.772746+0000","last_unstale":"2023-10-06T11:18:23.803596+0000","last_undegraded":"2023-10-06T11:18:23.803596+0000","last_fullsized":"2023-10-06T11:18:23.803596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.809429+0000","last_change":"2023-10-06T11:18:23.806392+0000","last_active":"2023-10-06T11:18:23.809429+0000","last_peered":"2023-10-06T11:18:23.809429+0000","last_clean":"2023-10-06T11:18:23.809429+0000","last_became_active":"2023-10-06T11:18:23.806086+0000","last_became_peered":"2023-10-06T11:18:23.806086+0000","last_unstale":"2023-10-06T11:18:23.809429+0000","last_undegraded":"2023-10-06T11:18:23.809429+0000","last_fullsized":"2023-10-06T11:18:23.809429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:32:33.405197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.074160+0000","last_change":"2023-10-06T11:18:22.786752+0000","last_active":"2023-10-06T11:18:24.074160+0000","last_peered":"2023-10-06T11:18:24.074160+0000","last_clean":"2023-10-06T11:18:24.074160+0000","last_became_active":"2023-10-06T11:18:22.784966+0000","last_became_peered":"2023-10-06T11:18:22.784966+0000","last_unstale":"2023-10-06T11:18:24.074160+0000","last_undegraded":"2023-10-06T11:18:24.074160+0000","last_fullsized":"2023-10-06T11:18:24.074160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:15:14.924890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.812296+0000","last_change":"2023-10-06T11:18:23.811011+0000","last_active":"2023-10-06T11:18:23.812296+0000","last_peered":"2023-10-06T11:18:23.812296+0000","last_clean":"2023-10-06T11:18:23.812296+0000","last_became_active":"2023-10-06T11:18:23.810563+0000","last_became_peered":"2023-10-06T11:18:23.810563+0000","last_unstale":"2023-10-06T11:18:23.812296+0000","last_undegraded":"2023-10-06T11:18:23.812296+0000","last_fullsized":"2023-10-06T11:18:23.812296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:06.048708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.073707+0000","last_change":"2023-10-06T11:18:22.787138+0000","last_active":"2023-10-06T11:18:24.073707+0000","last_peered":"2023-10-06T11:18:24.073707+0000","last_clean":"2023-10-06T11:18:24.073707+0000","last_became_active":"2023-10-06T11:18:22.786799+0000","last_became_peered":"2023-10-06T11:18:22.786799+0000","last_unstale":"2023-10-06T11:18:24.073707+0000","last_undegraded":"2023-10-06T11:18:24.073707+0000","last_fullsized":"2023-10-06T11:18:24.073707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:17.820286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802632+0000","last_change":"2023-10-06T11:18:23.797231+0000","last_active":"2023-10-06T11:18:23.802632+0000","last_peered":"2023-10-06T11:18:23.802632+0000","last_clean":"2023-10-06T11:18:23.802632+0000","last_became_active":"2023-10-06T11:18:23.796942+0000","last_became_peered":"2023-10-06T11:18:23.796942+0000","last_unstale":"2023-10-06T11:18:23.802632+0000","last_undegraded":"2023-10-06T11:18:23.802632+0000","last_fullsized":"2023-10-06T11:18:23.802632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080861+0000","last_change":"2023-10-06T11:18:22.784448+0000","last_active":"2023-10-06T11:18:24.080861+0000","last_peered":"2023-10-06T11:18:24.080861+0000","last_clean":"2023-10-06T11:18:24.080861+0000","last_became_active":"2023-10-06T11:18:22.784131+0000","last_became_peered":"2023-10-06T11:18:22.784131+0000","last_unstale":"2023-10-06T11:18:24.080861+0000","last_undegraded":"2023-10-06T11:18:24.080861+0000","last_fullsized":"2023-10-06T11:18:24.080861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802495+0000","last_change":"2023-10-06T11:18:23.796243+0000","last_active":"2023-10-06T11:18:23.802495+0000","last_peered":"2023-10-06T11:18:23.802495+0000","last_clean":"2023-10-06T11:18:23.802495+0000","last_became_active":"2023-10-06T11:18:23.795924+0000","last_became_peered":"2023-10-06T11:18:23.795924+0000","last_unstale":"2023-10-06T11:18:23.802495+0000","last_undegraded":"2023-10-06T11:18:23.802495+0000","last_fullsized":"2023-10-06T11:18:23.802495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:13:35.808098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.080525+0000","last_change":"2023-10-06T11:18:22.789060+0000","last_active":"2023-10-06T11:18:24.080525+0000","last_peered":"2023-10-06T11:18:24.080525+0000","last_clean":"2023-10-06T11:18:24.080525+0000","last_became_active":"2023-10-06T11:18:22.788376+0000","last_became_peered":"2023-10-06T11:18:22.788376+0000","last_unstale":"2023-10-06T11:18:24.080525+0000","last_undegraded":"2023-10-06T11:18:24.080525+0000","last_fullsized":"2023-10-06T11:18:24.080525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:25.732405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.804659+0000","last_change":"2023-10-06T11:18:23.802973+0000","last_active":"2023-10-06T11:18:23.804659+0000","last_peered":"2023-10-06T11:18:23.804659+0000","last_clean":"2023-10-06T11:18:23.804659+0000","last_became_active":"2023-10-06T11:18:23.802597+0000","last_became_peered":"2023-10-06T11:18:23.802597+0000","last_unstale":"2023-10-06T11:18:23.804659+0000","last_undegraded":"2023-10-06T11:18:23.804659+0000","last_fullsized":"2023-10-06T11:18:23.804659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:42.551739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079378+0000","last_change":"2023-10-06T11:18:22.770917+0000","last_active":"2023-10-06T11:18:24.079378+0000","last_peered":"2023-10-06T11:18:24.079378+0000","last_clean":"2023-10-06T11:18:24.079378+0000","last_became_active":"2023-10-06T11:18:22.770587+0000","last_became_peered":"2023-10-06T11:18:22.770587+0000","last_unstale":"2023-10-06T11:18:24.079378+0000","last_undegraded":"2023-10-06T11:18:24.079378+0000","last_fullsized":"2023-10-06T11:18:24.079378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:23.802685+0000","last_change":"2023-10-06T11:18:23.797236+0000","last_active":"2023-10-06T11:18:23.802685+0000","last_peered":"2023-10-06T11:18:23.802685+0000","last_clean":"2023-10-06T11:18:23.802685+0000","last_became_active":"2023-10-06T11:18:23.796889+0000","last_became_peered":"2023-10-06T11:18:23.796889+0000","last_unstale":"2023-10-06T11:18:23.802685+0000","last_undegraded":"2023-10-06T11:18:23.802685+0000","last_fullsized":"2023-10-06T11:18:23.802685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:22.738452+0000","last_clean_scrub_stamp":"2023-10-06T11:18:22.738452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:56.579251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T11:18:24.079742+0000","last_change":"2023-10-06T11:18:22.778280+0000","last_active":"2023-10-06T11:18:24.079742+0000","last_peered":"2023-10-06T11:18:24.079742+0000","last_clean":"2023-10-06T11:18:24.079742+0000","last_became_active":"2023-10-06T11:18:22.776627+0000","last_became_peered":"2023-10-06T11:18:22.776627+0000","last_unstale":"2023-10-06T11:18:24.079742+0000","last_undegraded":"2023-10-06T11:18:24.079742+0000","last_fullsized":"2023-10-06T11:18:24.079742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:21.723666+0000","last_clean_scrub_stamp":"2023-10-06T11:18:21.723666+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:07.409463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.905}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.859}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.895}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301624,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070216,"statfs":{"total":96636764160,"available":96327901184,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5289999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302204,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069636,"statfs":{"total":96636764160,"available":96327307264,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:18:33.486 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T11:18:33.486 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T11:18:33.486 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T11:18:33.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T11:18:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 Processor -- start 2023-10-06T11:18:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 -- start start 2023-10-06T11:18:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9ec0c6430 0x7fb9ec0aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0c74b0 con 0x7fb9ec0ab0a0 2023-10-06T11:18:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0c68a0 con 0x7fb9ec142760 2023-10-06T11:18:33.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9f1545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c6b80 con 0x7fb9ec0c6430 2023-10-06T11:18:33.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9e27fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9ec0c6430 0x7fb9ec0aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9eb7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb9ec0ab0a0 0x7fb9ec0c9b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43256/0 (socket says 172.21.15.46:43256) 2023-10-06T11:18:33.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.671+0000 7fb9eb7fe700 1 -- 172.21.15.46:0/4007535426 learned_addr learned my addr 172.21.15.46:0/4007535426 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224921654 0 0) 0x7fb9ec0c74b0 con 0x7fb9ec0ab0a0 2023-10-06T11:18:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9cc003570 con 0x7fb9ec0ab0a0 2023-10-06T11:18:33.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691594475 0 0) 0x7fb9ec0c68a0 con 0x7fb9ec142760 2023-10-06T11:18:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec0c74b0 con 0x7fb9ec142760 2023-10-06T11:18:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9e27fc700 1 -- 172.21.15.46:0/4007535426 >> v1:172.21.15.132:6789/0 conn(0x7fb9ec142760 legacy=0x7fb9ec0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9e27fc700 1 -- 172.21.15.46:0/4007535426 >> v1:172.21.15.46:6789/0 conn(0x7fb9ec0ab0a0 legacy=0x7fb9ec0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9e27fc700 1 -- 172.21.15.46:0/4007535426 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0c6e60 con 0x7fb9ec0c6430 2023-10-06T11:18:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9e27fc700 1 --2- 172.21.15.46:0/4007535426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec0aab60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40098e0 tx=0x7fb9d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951737717a177fc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011630 con 0x7fb9ec0c6430 2023-10-06T11:18:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4011790 con 0x7fb9ec0c6430 2023-10-06T11:18:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9ea7fc700 1 -- 172.21.15.46:0/4007535426 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4011b30 con 0x7fb9ec0c6430 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 -- 172.21.15.46:0/4007535426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec0c6430 msgr2=0x7fb9ec0aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 --2- 172.21.15.46:0/4007535426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec0aab60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40098e0 tx=0x7fb9d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 -- 172.21.15.46:0/4007535426 shutdown_connections 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 --2- 172.21.15.46:0/4007535426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec0aab60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 -- 172.21.15.46:0/4007535426 >> 172.21.15.46:0/4007535426 conn(0x7fb9ec0cbd80 msgr2=0x7fb9ec0cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 -- 172.21.15.46:0/4007535426 shutdown_connections 2023-10-06T11:18:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.675+0000 7fb9f1545700 1 -- 172.21.15.46:0/4007535426 wait complete. 2023-10-06T11:18:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.679+0000 7fb9f1545700 1 Processor -- start 2023-10-06T11:18:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.679+0000 7fb9f1545700 1 -- start start 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9ec0ab0a0 0x7fb9ec0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 0x7fb9ec15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15cd70 con 0x7fb9ec0c6430 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15c160 con 0x7fb9ec142760 2023-10-06T11:18:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec15c440 con 0x7fb9ec0ab0a0 2023-10-06T11:18:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44284/0 (socket says 172.21.15.46:44284) 2023-10-06T11:18:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 -- 172.21.15.46:0/3085203662 learned_addr learned my addr 172.21.15.46:0/3085203662 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9eaffd700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9ec0ab0a0 0x7fb9ec0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9eb7fe700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 0x7fb9ec15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 -- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9ec0ab0a0 msgr2=0x7fb9ec0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9ec0ab0a0 0x7fb9ec0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 -- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 msgr2=0x7fb9ec15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 0x7fb9ec15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 -- 172.21.15.46:0/3085203662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009420 con 0x7fb9ec0c6430 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9eb7fe700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 0x7fb9ec15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e27fc700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40093f0 tx=0x7fb9d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d401f070 con 0x7fb9ec0c6430 2023-10-06T11:18:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4000bc0 con 0x7fb9ec0c6430 2023-10-06T11:18:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4025420 con 0x7fb9ec0c6430 2023-10-06T11:18:33.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec15c720 con 0x7fb9ec0c6430 2023-10-06T11:18:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.683+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec0ae770 con 0x7fb9ec0c6430 2023-10-06T11:18:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.687+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec15dd70 con 0x7fb9ec0c6430 2023-10-06T11:18:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.687+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb9d4023070 con 0x7fb9ec0c6430 2023-10-06T11:18:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.691+0000 7fb9e37fe700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 0x7fb9d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.691+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb9d407db50 con 0x7fb9ec0c6430 2023-10-06T11:18:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.691+0000 7fb9eaffd700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 0x7fb9d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:33.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.691+0000 7fb9eaffd700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 0x7fb9d0062a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc003a40 tx=0x7fb9dc007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:33.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.695+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb9d404df40 con 0x7fb9ec0c6430 2023-10-06T11:18:33.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.899+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb9ec06b820 con 0x7fb9ec0c6430 2023-10-06T11:18:33.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9e37fe700 1 -- 172.21.15.46:0/3085203662 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb9d4051560 con 0x7fb9ec0c6430 2023-10-06T11:18:33.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:33.906 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T11:18:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 msgr2=0x7fb9d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 0x7fb9d0062a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc003a40 tx=0x7fb9dc007d40 comp rx=0 tx=0).stop 2023-10-06T11:18:33.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 msgr2=0x7fb9ec068d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40093f0 tx=0x7fb9d4017000 comp rx=0 tx=0).stop 2023-10-06T11:18:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 shutdown_connections 2023-10-06T11:18:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb9d0060550 0x7fb9d0062a00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9ec142760 0x7fb9ec15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9ec0c6430 0x7fb9ec068d50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 --2- 172.21.15.46:0/3085203662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9ec0ab0a0 0x7fb9ec0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 >> 172.21.15.46:0/3085203662 conn(0x7fb9ec0cbd80 msgr2=0x7fb9ec0c0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 shutdown_connections 2023-10-06T11:18:33.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:33.903+0000 7fb9f1545700 1 -- 172.21.15.46:0/3085203662 wait complete. 2023-10-06T11:18:33.928 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T11:18:33.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.079+0000 7fc2982f0700 1 Processor -- start 2023-10-06T11:18:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 -- start start 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 0x7fc2900c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc2900c9100 0x7fc29006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2900c7fc0 con 0x7fc290156160 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2900cba20 con 0x7fc2900c9100 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc2982f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900c9590 con 0x7fc290155d90 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc29588b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc290156160 0x7fc2900c89f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43260/0 (socket says 172.21.15.46:43260) 2023-10-06T11:18:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc29588b700 1 -- 172.21.15.46:0/3782950360 learned_addr learned my addr 172.21.15.46:0/3782950360 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc29608c700 1 --2- 172.21.15.46:0/3782950360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 0x7fc2900c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc29688d700 1 --2- 172.21.15.46:0/3782950360 >> v2:172.21.15.132:3300/0 conn(0x7fc2900c9100 0x7fc29006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.083+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646342609 0 0) 0x7fc2900c7fc0 con 0x7fc290156160 2023-10-06T11:18:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc27c003570 con 0x7fc290156160 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29608c700 1 -- 172.21.15.46:0/3782950360 >> v2:172.21.15.132:3300/0 conn(0x7fc2900c9100 msgr2=0x7fc29006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51157515 0 0) 0x7fc27c003570 con 0x7fc290156160 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29608c700 1 --2- 172.21.15.46:0/3782950360 >> v2:172.21.15.132:3300/0 conn(0x7fc2900c9100 0x7fc29006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29608c700 1 -- 172.21.15.46:0/3782950360 >> v1:172.21.15.46:6789/0 conn(0x7fc290156160 legacy=0x7fc2900c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29608c700 1 -- 172.21.15.46:0/3782950360 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2900c7690 con 0x7fc290155d90 2023-10-06T11:18:34.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29608c700 1 --2- 172.21.15.46:0/3782950360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 0x7fc2900c8450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009910 tx=0x7fc28c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f0ba9fd5199b88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280001470 con 0x7fc290155d90 2023-10-06T11:18:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c011420 con 0x7fc290155d90 2023-10-06T11:18:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc29508a700 1 -- 172.21.15.46:0/3782950360 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0117c0 con 0x7fc290155d90 2023-10-06T11:18:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 -- 172.21.15.46:0/3782950360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 msgr2=0x7fc2900c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 --2- 172.21.15.46:0/3782950360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 0x7fc2900c8450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009910 tx=0x7fc28c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 -- 172.21.15.46:0/3782950360 shutdown_connections 2023-10-06T11:18:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 --2- 172.21.15.46:0/3782950360 >> v2:172.21.15.132:3300/0 conn(0x7fc2900c9100 0x7fc29006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 --2- 172.21.15.46:0/3782950360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc290155d90 0x7fc2900c8450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 -- 172.21.15.46:0/3782950360 >> 172.21.15.46:0/3782950360 conn(0x7fc2900cced0 msgr2=0x7fc2900cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 -- 172.21.15.46:0/3782950360 shutdown_connections 2023-10-06T11:18:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.087+0000 7fc2982f0700 1 -- 172.21.15.46:0/3782950360 wait complete. 2023-10-06T11:18:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 Processor -- start 2023-10-06T11:18:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 -- start start 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2900c9100 0x7fc2900b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2900a9cc0 0x7fc2900b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc290169e40 con 0x7fc290155d90 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900aa1c0 con 0x7fc2900c9100 2023-10-06T11:18:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc2982f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc290169510 con 0x7fc2900a9cc0 2023-10-06T11:18:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc29588b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc29588b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44296/0 (socket says 172.21.15.46:44296) 2023-10-06T11:18:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc29588b700 1 -- 172.21.15.46:0/2848378686 learned_addr learned my addr 172.21.15.46:0/2848378686 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.091+0000 7fc29688d700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2900a9cc0 0x7fc2900b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29608c700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2900c9100 0x7fc2900b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 -- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2900a9cc0 msgr2=0x7fc2900b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2900a9cc0 0x7fc2900b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 -- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2900c9100 msgr2=0x7fc2900b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2900c9100 0x7fc2900b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 -- 172.21.15.46:0/2848378686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009420 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc29588b700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c840 tx=0x7fc280001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280007360 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2800074e0 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280004830 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2901697f0 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29016a2d0 con 0x7fc290155d90 2023-10-06T11:18:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.095+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29006d220 con 0x7fc290155d90 2023-10-06T11:18:34.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.099+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc280004c00 con 0x7fc290155d90 2023-10-06T11:18:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.099+0000 7fc2867fc700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 0x7fc274062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.099+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc28007c080 con 0x7fc290155d90 2023-10-06T11:18:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.099+0000 7fc29608c700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 0x7fc274062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.099+0000 7fc29608c700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 0x7fc274062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0098e0 tx=0x7fc28c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.107+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc28004c4f0 con 0x7fc290155d90 2023-10-06T11:18:34.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.295+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc29015ee10 con 0x7fc290155d90 2023-10-06T11:18:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.295+0000 7fc2867fc700 1 -- 172.21.15.46:0/2848378686 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fc280018020 con 0x7fc290155d90 2023-10-06T11:18:34.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:34.301 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:34.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 msgr2=0x7fc274062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 0x7fc274062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0098e0 tx=0x7fc28c016000 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 msgr2=0x7fc2901505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc28000c840 tx=0x7fc280001a10 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 shutdown_connections 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc274060550 0x7fc274062a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2900a9cc0 0x7fc2900b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc290155d90 0x7fc2901505b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 --2- 172.21.15.46:0/2848378686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2900c9100 0x7fc2900b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 >> 172.21.15.46:0/2848378686 conn(0x7fc2900cced0 msgr2=0x7fc290151760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.299+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 shutdown_connections 2023-10-06T11:18:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.303+0000 7fc2982f0700 1 -- 172.21.15.46:0/2848378686 wait complete. 2023-10-06T11:18:34.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:34.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.487+0000 7f43262e3700 1 Processor -- start 2023-10-06T11:18:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 -- start start 2023-10-06T11:18:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c7d40 0x7f432006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f432006cf30 0x7f43200c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200c1d90 con 0x7f43200c97e0 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43200c8190 con 0x7f432006cf30 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43262e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200c1460 con 0x7f43200c7d40 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43252e1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f43200c97e0 0x7f43200c6250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43272/0 (socket says 172.21.15.46:43272) 2023-10-06T11:18:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f43252e1700 1 -- 172.21.15.46:0/2196646994 learned_addr learned my addr 172.21.15.46:0/2196646994 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4325ae2700 1 --2- 172.21.15.46:0/2196646994 >> v2:172.21.15.46:3300/0 conn(0x7f432006cf30 0x7f43200c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4324ae0700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c7d40 0x7f432006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4317fff700 1 -- 172.21.15.46:0/2196646994 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261676442 0 0) 0x7f43200c1d90 con 0x7f43200c97e0 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4317fff700 1 -- 172.21.15.46:0/2196646994 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c003570 con 0x7f43200c97e0 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4325ae2700 1 -- 172.21.15.46:0/2196646994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c7d40 msgr2=0x7f432006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4325ae2700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c7d40 0x7f432006c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4325ae2700 1 -- 172.21.15.46:0/2196646994 >> v1:172.21.15.46:6789/0 conn(0x7f43200c97e0 legacy=0x7f43200c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.491+0000 7f4325ae2700 1 -- 172.21.15.46:0/2196646994 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200c1740 con 0x7f432006cf30 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f4325ae2700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f43200c2220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4318009e60 tx=0x7f4318009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1af7dd60a3553e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f4317fff700 1 -- 172.21.15.46:0/2196646994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431800e070 con 0x7f432006cf30 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f4317fff700 1 -- 172.21.15.46:0/2196646994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431800bab0 con 0x7f432006cf30 2023-10-06T11:18:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f4317fff700 1 -- 172.21.15.46:0/2196646994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431800be70 con 0x7f432006cf30 2023-10-06T11:18:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 -- 172.21.15.46:0/2196646994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 msgr2=0x7f43200c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f43200c2220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4318009e60 tx=0x7f4318009780 comp rx=0 tx=0).stop 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 -- 172.21.15.46:0/2196646994 shutdown_connections 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f43200c2220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 --2- 172.21.15.46:0/2196646994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c7d40 0x7f432006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 -- 172.21.15.46:0/2196646994 >> 172.21.15.46:0/2196646994 conn(0x7f43200cbc10 msgr2=0x7f43200cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 -- 172.21.15.46:0/2196646994 shutdown_connections 2023-10-06T11:18:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.495+0000 7f43262e3700 1 -- 172.21.15.46:0/2196646994 wait complete. 2023-10-06T11:18:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 Processor -- start 2023-10-06T11:18:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 -- start start 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43200c7d40 0x7f432015eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c97e0 0x7f43200caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320160530 con 0x7f432006cf30 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432015f920 con 0x7f43200c7d40 2023-10-06T11:18:34.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43262e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432015fc00 con 0x7f43200c97e0 2023-10-06T11:18:34.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44326/0 (socket says 172.21.15.46:44326) 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 -- 172.21.15.46:0/2349493141 learned_addr learned my addr 172.21.15.46:0/2349493141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f4324ae0700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43200c7d40 0x7f432015eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f4325ae2700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c97e0 0x7f43200caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 -- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c97e0 msgr2=0x7f43200caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c97e0 0x7f43200caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 -- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43200c7d40 msgr2=0x7f432015eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43200c7d40 0x7f432015eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 -- 172.21.15.46:0/2349493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4318009a20 con 0x7f432006cf30 2023-10-06T11:18:34.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.499+0000 7f43252e1700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c7c0 tx=0x7f431c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c005b30 con 0x7f432006cf30 2023-10-06T11:18:34.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c005c90 con 0x7f432006cf30 2023-10-06T11:18:34.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432015fee0 con 0x7f432006cf30 2023-10-06T11:18:34.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c007340 con 0x7f432006cf30 2023-10-06T11:18:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432015da20 con 0x7f432006cf30 2023-10-06T11:18:34.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4320160360 con 0x7f432006cf30 2023-10-06T11:18:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.503+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f431c01b400 con 0x7f432006cf30 2023-10-06T11:18:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.507+0000 7f43157fa700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 0x7f4304062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.507+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f431c07c410 con 0x7f432006cf30 2023-10-06T11:18:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.507+0000 7f4324ae0700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 0x7f4304062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.507+0000 7f4324ae0700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 0x7f4304062ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43200ae550 tx=0x7f4310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.511+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f431c04c900 con 0x7f432006cf30 2023-10-06T11:18:34.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.683+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4320064590 con 0x7f432006cf30 2023-10-06T11:18:34.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.683+0000 7f43157fa700 1 -- 172.21.15.46:0/2349493141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f431c04ff20 con 0x7f432006cf30 2023-10-06T11:18:34.687 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.683+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 msgr2=0x7f4304062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.683+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 0x7f4304062ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43200ae550 tx=0x7f4310008040 comp rx=0 tx=0).stop 2023-10-06T11:18:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 msgr2=0x7f432014ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c7c0 tx=0x7f431c007e00 comp rx=0 tx=0).stop 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 shutdown_connections 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4304060620 0x7f4304062ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43200c97e0 0x7f43200caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43200c7d40 0x7f432015eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 --2- 172.21.15.46:0/2349493141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f432006cf30 0x7f432014ffe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 >> 172.21.15.46:0/2349493141 conn(0x7f43200cbc10 msgr2=0x7f43200c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 shutdown_connections 2023-10-06T11:18:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.687+0000 7f43262e3700 1 -- 172.21.15.46:0/2349493141 wait complete. 2023-10-06T11:18:34.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:34.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.875+0000 7fb5a3985700 1 Processor -- start 2023-10-06T11:18:34.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.875+0000 7fb5a3985700 1 -- start start 2023-10-06T11:18:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a3985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb59c069410 0x7fb59c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a3985700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c0ce470 con 0x7fb59c0697e0 2023-10-06T11:18:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a3985700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c15d470 con 0x7fb59c159420 2023-10-06T11:18:34.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a3985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c15d750 con 0x7fb59c069410 2023-10-06T11:18:34.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a0f20700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb59c0697e0 0x7fb59c158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43276/0 (socket says 172.21.15.46:43276) 2023-10-06T11:18:34.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a0f20700 1 -- 172.21.15.46:0/3944366216 learned_addr learned my addr 172.21.15.46:0/3944366216 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:34.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a1721700 1 --2- 172.21.15.46:0/3944366216 >> v2:172.21.15.132:3300/0 conn(0x7fb59c069410 0x7fb59c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861570189 0 0) 0x7fb59c15d470 con 0x7fb59c159420 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb588003570 con 0x7fb59c159420 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225796577 0 0) 0x7fb59c0ce470 con 0x7fb59c0697e0 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59c15d470 con 0x7fb59c0697e0 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a1721700 1 -- 172.21.15.46:0/3944366216 >> v1:172.21.15.132:6789/0 conn(0x7fb59c159420 legacy=0x7fb59c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a1721700 1 -- 172.21.15.46:0/3944366216 >> v1:172.21.15.46:6789/0 conn(0x7fb59c0697e0 legacy=0x7fb59c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a1721700 1 -- 172.21.15.46:0/3944366216 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c15da30 con 0x7fb59c069410 2023-10-06T11:18:34.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb5a1721700 1 --2- 172.21.15.46:0/3944366216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c15ce90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5980098e0 tx=0x7fb5980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58dc37c3e032cc67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.879+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598011630 con 0x7fb59c069410 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598011790 con 0x7fb59c069410 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb593fff700 1 -- 172.21.15.46:0/3944366216 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598011b30 con 0x7fb59c069410 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- 172.21.15.46:0/3944366216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 msgr2=0x7fb59c15ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 --2- 172.21.15.46:0/3944366216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c15ce90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5980098e0 tx=0x7fb5980091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- 172.21.15.46:0/3944366216 shutdown_connections 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 --2- 172.21.15.46:0/3944366216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c15ce90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- 172.21.15.46:0/3944366216 >> 172.21.15.46:0/3944366216 conn(0x7fb59c0d0210 msgr2=0x7fb59c0d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:34.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- 172.21.15.46:0/3944366216 shutdown_connections 2023-10-06T11:18:34.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- 172.21.15.46:0/3944366216 wait complete. 2023-10-06T11:18:34.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 Processor -- start 2023-10-06T11:18:34.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- start start 2023-10-06T11:18:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb59c159420 0x7fb59c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0afeb0 con 0x7fb59c0697e0 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0b0100 con 0x7fb59c069410 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.883+0000 7fb5a3985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0b03e0 con 0x7fb59c159420 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44340/0 (socket says 172.21.15.46:44340) 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 -- 172.21.15.46:0/4247880814 learned_addr learned my addr 172.21.15.46:0/4247880814 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a1721700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 -- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb59c159420 msgr2=0x7fb59c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb59c159420 0x7fb59c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 -- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 msgr2=0x7fb59c14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c14cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 -- 172.21.15.46:0/4247880814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb598009420 con 0x7fb59c0697e0 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a1721700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a0f20700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c7c0 tx=0x7fb58c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005b30 con 0x7fb59c0697e0 2023-10-06T11:18:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c005c90 con 0x7fb59c0697e0 2023-10-06T11:18:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c0b06c0 con 0x7fb59c0697e0 2023-10-06T11:18:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c007340 con 0x7fb59c0697e0 2023-10-06T11:18:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c0b0ec0 con 0x7fb59c0697e0 2023-10-06T11:18:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.887+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0b0ba0 con 0x7fb59c0697e0 2023-10-06T11:18:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.891+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb58c01b400 con 0x7fb59c0697e0 2023-10-06T11:18:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.891+0000 7fb5917fa700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 0x7fb580062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.891+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb58c07c510 con 0x7fb59c0697e0 2023-10-06T11:18:34.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.891+0000 7fb5a1721700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 0x7fb580062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:34.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.891+0000 7fb5a1721700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 0x7fb580062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5980098e0 tx=0x7fb598017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:34.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:34.895+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb58c04c980 con 0x7fb59c0697e0 2023-10-06T11:18:35.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.063+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb59c0caae0 con 0x7fb59c0697e0 2023-10-06T11:18:35.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.063+0000 7fb5917fa700 1 -- 172.21.15.46:0/4247880814 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb58c04ffa0 con 0x7fb59c0697e0 2023-10-06T11:18:35.070 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 msgr2=0x7fb580062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 0x7fb580062a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5980098e0 tx=0x7fb598017000 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 msgr2=0x7fb59c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00c7c0 tx=0x7fb58c007e00 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 shutdown_connections 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb580060550 0x7fb580062a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb59c159420 0x7fb59c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb59c0697e0 0x7fb59c15a350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 --2- 172.21.15.46:0/4247880814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb59c069410 0x7fb59c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 >> 172.21.15.46:0/4247880814 conn(0x7fb59c0d0210 msgr2=0x7fb59c15c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 shutdown_connections 2023-10-06T11:18:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.067+0000 7fb5a3985700 1 -- 172.21.15.46:0/4247880814 wait complete. 2023-10-06T11:18:35.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.259+0000 7f52c566e700 1 Processor -- start 2023-10-06T11:18:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.263+0000 7f52c566e700 1 -- start start 2023-10-06T11:18:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52c0159420 0x7f52c015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52c00697e0 0x7f52c015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00ce470 con 0x7f52c0069410 2023-10-06T11:18:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c015a890 con 0x7f52c00697e0 2023-10-06T11:18:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c0069c40 con 0x7f52c0159420 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52c0159420 0x7f52c015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52beffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f52c0069410 0x7f52c0158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40376/0 (socket says 172.21.15.46:40376) 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52c0159420 0x7f52c015ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:34720/0 (socket says 172.21.15.46:34720) 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 -- 172.21.15.46:0/1219996381 learned_addr learned my addr 172.21.15.46:0/1219996381 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52bf7fe700 1 --2- 172.21.15.46:0/1219996381 >> v2:172.21.15.46:3300/0 conn(0x7f52c00697e0 0x7f52c015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 -- 172.21.15.46:0/1219996381 >> v2:172.21.15.46:3300/0 conn(0x7f52c00697e0 msgr2=0x7f52c015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:35.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 --2- 172.21.15.46:0/1219996381 >> v2:172.21.15.46:3300/0 conn(0x7f52c00697e0 0x7f52c015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 -- 172.21.15.46:0/1219996381 >> v1:172.21.15.132:6789/0 conn(0x7f52c0069410 legacy=0x7f52c0158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 -- 172.21.15.46:0/1219996381 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c015d470 con 0x7f52c0159420 2023-10-06T11:18:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52be7fc700 1 --2- 172.21.15.46:0/1219996381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c015ce90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52c00b2d20 tx=0x7f52b4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f27846c1a487ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52bdffb700 1 -- 172.21.15.46:0/1219996381 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4009e70 con 0x7f52c0159420 2023-10-06T11:18:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52bdffb700 1 -- 172.21.15.46:0/1219996381 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b400b860 con 0x7f52c0159420 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52bdffb700 1 -- 172.21.15.46:0/1219996381 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400bc00 con 0x7f52c0159420 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 -- 172.21.15.46:0/1219996381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 msgr2=0x7f52c015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.267+0000 7f52c566e700 1 --2- 172.21.15.46:0/1219996381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c015ce90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52c00b2d20 tx=0x7f52b4009250 comp rx=0 tx=0).stop 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- 172.21.15.46:0/1219996381 shutdown_connections 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 --2- 172.21.15.46:0/1219996381 >> v2:172.21.15.46:3300/0 conn(0x7f52c00697e0 0x7f52c015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 --2- 172.21.15.46:0/1219996381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c015ce90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- 172.21.15.46:0/1219996381 >> 172.21.15.46:0/1219996381 conn(0x7f52c00d0210 msgr2=0x7f52c00d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- 172.21.15.46:0/1219996381 shutdown_connections 2023-10-06T11:18:35.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- 172.21.15.46:0/1219996381 wait complete. 2023-10-06T11:18:35.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 Processor -- start 2023-10-06T11:18:35.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- start start 2023-10-06T11:18:35.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 0x7f52c0143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c00b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00b3a30 con 0x7f52c00697e0 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c5ca0 con 0x7f52c0159420 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52c566e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c5f80 con 0x7f52c0069410 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52be7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52be7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44352/0 (socket says 172.21.15.46:44352) 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.271+0000 7f52be7fc700 1 -- 172.21.15.46:0/2069755493 learned_addr learned my addr 172.21.15.46:0/2069755493 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52bf7fe700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c00b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52beffd700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 0x7f52c0143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 -- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 msgr2=0x7f52c0143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 0x7f52c0143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 -- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 msgr2=0x7f52c00b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c00b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 -- 172.21.15.46:0/2069755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40094f0 con 0x7f52c00697e0 2023-10-06T11:18:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52beffd700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 0x7f52c0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:35.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52be7fc700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f52c0069d80 tx=0x7f52b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400bbe0 con 0x7f52c00697e0 2023-10-06T11:18:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4011420 con 0x7f52c00697e0 2023-10-06T11:18:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40117a0 con 0x7f52c00697e0 2023-10-06T11:18:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c00c6260 con 0x7f52c00697e0 2023-10-06T11:18:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00c6820 con 0x7f52c00697e0 2023-10-06T11:18:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.275+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0088870 con 0x7f52c00697e0 2023-10-06T11:18:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.279+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52b4011b90 con 0x7f52c00697e0 2023-10-06T11:18:35.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.279+0000 7f52a6ffd700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 0x7f52ac062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.279+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52b407e150 con 0x7f52c00697e0 2023-10-06T11:18:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.279+0000 7f52beffd700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 0x7f52ac062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.283+0000 7f52beffd700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 0x7f52ac062a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52b0007fd0 tx=0x7f52b0007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.287+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52b404e540 con 0x7f52c00697e0 2023-10-06T11:18:35.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.455+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52c015d750 con 0x7f52c00697e0 2023-10-06T11:18:35.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.455+0000 7f52a6ffd700 1 -- 172.21.15.46:0/2069755493 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52b404e540 con 0x7f52c00697e0 2023-10-06T11:18:35.461 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 msgr2=0x7f52ac062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 0x7f52ac062a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52b0007fd0 tx=0x7f52b0007e30 comp rx=0 tx=0).stop 2023-10-06T11:18:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 msgr2=0x7f52c00b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f52c0069d80 tx=0x7f52b4017000 comp rx=0 tx=0).stop 2023-10-06T11:18:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 shutdown_connections 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52ac060580 0x7f52ac062a30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c0159420 0x7f52c00b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52c00697e0 0x7f52c00b2fb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 --2- 172.21.15.46:0/2069755493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0069410 0x7f52c0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 >> 172.21.15.46:0/2069755493 conn(0x7f52c00d0210 msgr2=0x7f52c015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 shutdown_connections 2023-10-06T11:18:35.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.459+0000 7f52c566e700 1 -- 172.21.15.46:0/2069755493 wait complete. 2023-10-06T11:18:35.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.651+0000 7f2780c2e700 1 Processor -- start 2023-10-06T11:18:35.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.651+0000 7f2780c2e700 1 -- start start 2023-10-06T11:18:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f2780c2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f277c0b26d0 0x7f277c159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f2780c2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277c0b4f80 con 0x7f277c159050 2023-10-06T11:18:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f2780c2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277c0b40c0 con 0x7f277c0d0aa0 2023-10-06T11:18:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f2780c2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277c0b4370 con 0x7f277c0b26d0 2023-10-06T11:18:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f277c159050 0x7f277c0b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43278/0 (socket says 172.21.15.46:43278) 2023-10-06T11:18:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277a59c700 1 -- 172.21.15.46:0/2155545618 learned_addr learned my addr 172.21.15.46:0/2155545618 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:35.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277ad9d700 1 --2- 172.21.15.46:0/2155545618 >> v2:172.21.15.132:3300/0 conn(0x7f277c0b26d0 0x7f277c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171123207 0 0) 0x7f277c0b4f80 con 0x7f277c159050 2023-10-06T11:18:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2764003570 con 0x7f277c159050 2023-10-06T11:18:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525601996 0 0) 0x7f277c0b40c0 con 0x7f277c0d0aa0 2023-10-06T11:18:35.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f277c0b4f80 con 0x7f277c0d0aa0 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232189580 0 0) 0x7f2764003570 con 0x7f277c159050 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 >> v2:172.21.15.132:3300/0 conn(0x7f277c0b26d0 msgr2=0x7f277c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 --2- 172.21.15.46:0/2155545618 >> v2:172.21.15.132:3300/0 conn(0x7f277c0b26d0 0x7f277c159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 >> v1:172.21.15.132:6789/0 conn(0x7f277c0d0aa0 legacy=0x7f277c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0b4650 con 0x7f277c159050 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f276c001470 con 0x7f277c159050 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 >> v1:172.21.15.46:6789/0 conn(0x7f277c159050 legacy=0x7f277c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764003bf0 0x7f2764003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 0x7f2764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27640051c0 0x7f276400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c0b4370 con 0x7f2764004810 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276400c0b0 con 0x7f2764003bf0 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.655+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276400c270 con 0x7f27640051c0 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 0x7f2764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277a59c700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764003bf0 0x7f2764003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277ad9d700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27640051c0 0x7f276400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 -- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27640051c0 msgr2=0x7f276400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27640051c0 0x7f276400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 -- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764003bf0 msgr2=0x7f2764003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764003bf0 0x7f2764003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 -- 172.21.15.46:0/2155545618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764003570 con 0x7f2764004810 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277ad9d700 1 -- 172.21.15.46:0/2155545618 reap_dead start 2023-10-06T11:18:35.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2779d9b700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 0x7f2764004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2770001a10 tx=0x7f2770007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374126e6cfbe2fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000c590 con 0x7f2764004810 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277000c6f0 con 0x7f2764004810 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f277959a700 1 -- 172.21.15.46:0/2155545618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000ca90 con 0x7f2764004810 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 -- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 msgr2=0x7f2764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 0x7f2764004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2770001a10 tx=0x7f2770007d30 comp rx=0 tx=0).stop 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 -- 172.21.15.46:0/2155545618 shutdown_connections 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 --2- 172.21.15.46:0/2155545618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2764004810 0x7f2764004c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 -- 172.21.15.46:0/2155545618 >> 172.21.15.46:0/2155545618 conn(0x7f277c06e780 msgr2=0x7f277c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.659+0000 7f2780c2e700 1 -- 172.21.15.46:0/2155545618 shutdown_connections 2023-10-06T11:18:35.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 -- 172.21.15.46:0/2155545618 wait complete. 2023-10-06T11:18:35.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 Processor -- start 2023-10-06T11:18:35.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 -- start start 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764004810 0x7f277c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f277c0cf6b0 0x7f277c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c0cf220 con 0x7f277c0cfb00 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c0b0570 con 0x7f2764004810 2023-10-06T11:18:35.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f2780c2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c0c4760 con 0x7f277c0cf6b0 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44382/0 (socket says 172.21.15.46:44382) 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f277ad9d700 1 -- 172.21.15.46:0/1945072065 learned_addr learned my addr 172.21.15.46:0/1945072065 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.663+0000 7f277a59c700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764004810 0x7f277c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f2779d9b700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f277c0cf6b0 0x7f277c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 -- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f277c0cf6b0 msgr2=0x7f277c0c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f277c0cf6b0 0x7f277c0c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 -- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764004810 msgr2=0x7f277c0b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764004810 0x7f277c0b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 -- 172.21.15.46:0/1945072065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0ce610 con 0x7f277c0cfb00 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f277ad9d700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f277c159990 tx=0x7f277400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277400e040 con 0x7f277c0cfb00 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277400f040 con 0x7f277c0cfb00 2023-10-06T11:18:35.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277400fe40 con 0x7f277c0cfb00 2023-10-06T11:18:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c0ce8f0 con 0x7f277c0cfb00 2023-10-06T11:18:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c0ceeb0 con 0x7f277c0cfb00 2023-10-06T11:18:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.667+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c0cf390 con 0x7f277c0cfb00 2023-10-06T11:18:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.671+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2774015640 con 0x7f277c0cfb00 2023-10-06T11:18:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.671+0000 7f276a7fc700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 0x7f275c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.671+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f277407d410 con 0x7f277c0cfb00 2023-10-06T11:18:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.671+0000 7f277a59c700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 0x7f275c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:35.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.671+0000 7f277a59c700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 0x7f275c062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f277c066710 tx=0x7f276c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:35.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.679+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f277404d800 con 0x7f277c0cfb00 2023-10-06T11:18:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.851+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f277c04b5c0 con 0x7f277c0cfb00 2023-10-06T11:18:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.851+0000 7f276a7fc700 1 -- 172.21.15.46:0/1945072065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2774050e20 con 0x7f277c0cfb00 2023-10-06T11:18:35.856 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 msgr2=0x7f275c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 0x7f275c062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f277c066710 tx=0x7f276c009040 comp rx=0 tx=0).stop 2023-10-06T11:18:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 msgr2=0x7f277c0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f277c159990 tx=0x7f277400b040 comp rx=0 tx=0).stop 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 shutdown_connections 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f275c060550 0x7f275c062a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c0cfb00 0x7f277c0c71e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f277c0cf6b0 0x7f277c0c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 --2- 172.21.15.46:0/1945072065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764004810 0x7f277c0b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 >> 172.21.15.46:0/1945072065 conn(0x7f277c06e780 msgr2=0x7f277c1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 shutdown_connections 2023-10-06T11:18:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:35.855+0000 7f2780c2e700 1 -- 172.21.15.46:0/1945072065 wait complete. 2023-10-06T11:18:35.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.035+0000 7f446c545700 1 Processor -- start 2023-10-06T11:18:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 -- start start 2023-10-06T11:18:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f446415ab60 0x7f446406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44640ad6c0 0x7f44640adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446406d460 con 0x7f446415af30 2023-10-06T11:18:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640c7000 con 0x7f44640ad6c0 2023-10-06T11:18:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446c545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640c72c0 con 0x7f446415ab60 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.039+0000 7f446aae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44640ad6c0 0x7f44640adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44640ad6c0 0x7f44640adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44394/0 (socket says 172.21.15.46:44394) 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 -- 172.21.15.46:0/3009186402 learned_addr learned my addr 172.21.15.46:0/3009186402 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446a2e1700 1 --2- 172.21.15.46:0/3009186402 >> v2:172.21.15.132:3300/0 conn(0x7f446415ab60 0x7f446406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 -- 172.21.15.46:0/3009186402 >> v2:172.21.15.132:3300/0 conn(0x7f446415ab60 msgr2=0x7f446406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 --2- 172.21.15.46:0/3009186402 >> v2:172.21.15.132:3300/0 conn(0x7f446415ab60 0x7f446406e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 -- 172.21.15.46:0/3009186402 >> v1:172.21.15.132:6789/0 conn(0x7f446415af30 legacy=0x7f446406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 -- 172.21.15.46:0/3009186402 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44640c75a0 con 0x7f44640ad6c0 2023-10-06T11:18:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446aae2700 1 --2- 172.21.15.46:0/3009186402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640ad6c0 0x7f44640adb20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4464063be0 tx=0x7f445c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81287a6b8c5ab59a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f44692df700 1 -- 172.21.15.46:0/3009186402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00e070 con 0x7f44640ad6c0 2023-10-06T11:18:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f44692df700 1 -- 172.21.15.46:0/3009186402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c00ba20 con 0x7f44640ad6c0 2023-10-06T11:18:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f44692df700 1 -- 172.21.15.46:0/3009186402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0092e0 con 0x7f44640ad6c0 2023-10-06T11:18:36.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 -- 172.21.15.46:0/3009186402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640ad6c0 msgr2=0x7f44640adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 --2- 172.21.15.46:0/3009186402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640ad6c0 0x7f44640adb20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4464063be0 tx=0x7f445c009840 comp rx=0 tx=0).stop 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 -- 172.21.15.46:0/3009186402 shutdown_connections 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 --2- 172.21.15.46:0/3009186402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640ad6c0 0x7f44640adb20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 --2- 172.21.15.46:0/3009186402 >> v2:172.21.15.132:3300/0 conn(0x7f446415ab60 0x7f446406e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 -- 172.21.15.46:0/3009186402 >> 172.21.15.46:0/3009186402 conn(0x7f44640cbe30 msgr2=0x7f44640cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 -- 172.21.15.46:0/3009186402 shutdown_connections 2023-10-06T11:18:36.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.043+0000 7f446c545700 1 -- 172.21.15.46:0/3009186402 wait complete. 2023-10-06T11:18:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 Processor -- start 2023-10-06T11:18:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 -- start start 2023-10-06T11:18:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44640ad6c0 0x7f44640c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f446415ab60 0x7f44640c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640c6620 con 0x7f446415af30 2023-10-06T11:18:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640c9f00 con 0x7f446415ab60 2023-10-06T11:18:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446c545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4464150a70 con 0x7f44640ad6c0 2023-10-06T11:18:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446aae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446aae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44398/0 (socket says 172.21.15.46:44398) 2023-10-06T11:18:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446aae2700 1 -- 172.21.15.46:0/3550871903 learned_addr learned my addr 172.21.15.46:0/3550871903 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446a2e1700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44640ad6c0 0x7f44640c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f4469ae0700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f446415ab60 0x7f44640c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446aae2700 1 -- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44640ad6c0 msgr2=0x7f44640c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.047+0000 7f446aae2700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44640ad6c0 0x7f44640c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446aae2700 1 -- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f446415ab60 msgr2=0x7f44640c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446aae2700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f446415ab60 0x7f44640c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446aae2700 1 -- 172.21.15.46:0/3550871903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c009ae0 con 0x7f446415af30 2023-10-06T11:18:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446aae2700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f445c009880 tx=0x7f445c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c008be0 con 0x7f446415af30 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c008d40 con 0x7f446415af30 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44640c53a0 con 0x7f446415af30 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0256b0 con 0x7f446415af30 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44640c9b90 con 0x7f446415af30 2023-10-06T11:18:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.051+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44640c9710 con 0x7f446415af30 2023-10-06T11:18:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.055+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f445c003040 con 0x7f446415af30 2023-10-06T11:18:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.055+0000 7f445a7fc700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 0x7f4448062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.055+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f445c07dfb0 con 0x7f446415af30 2023-10-06T11:18:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.055+0000 7f446a2e1700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 0x7f4448062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.055+0000 7f446a2e1700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 0x7f4448062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4460009d10 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.059+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f44640c9710 con 0x7f446415af30 2023-10-06T11:18:36.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.267+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44640c9710 con 0x7f446415af30 2023-10-06T11:18:36.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.271+0000 7f445a7fc700 1 -- 172.21.15.46:0/3550871903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f44640c9710 con 0x7f446415af30 2023-10-06T11:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:36.275 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:18:36.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 msgr2=0x7f4448062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 0x7f4448062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4460009d10 tx=0x7f4460008040 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 msgr2=0x7f4464150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f445c009880 tx=0x7f445c018000 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 shutdown_connections 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4448060550 0x7f4448062a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f446415af30 0x7f4464150530 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f446415ab60 0x7f44640c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 --2- 172.21.15.46:0/3550871903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44640ad6c0 0x7f44640c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 >> 172.21.15.46:0/3550871903 conn(0x7f44640cbe30 msgr2=0x7f4464157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 shutdown_connections 2023-10-06T11:18:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.275+0000 7f446c545700 1 -- 172.21.15.46:0/3550871903 wait complete. 2023-10-06T11:18:36.283 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:18:36.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 Processor -- start 2023-10-06T11:18:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 -- start start 2023-10-06T11:18:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f843c06e8a0 0x7f843c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f843c0ad6c0 0x7f843c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c06d460 con 0x7f843c15ab60 2023-10-06T11:18:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c06ed40 con 0x7f843c0ad6c0 2023-10-06T11:18:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f8441af0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0c7000 con 0x7f843c06e8a0 2023-10-06T11:18:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f843bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f843c0ad6c0 0x7f843c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f843b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f843c15ab60 0x7f843c15af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43292/0 (socket says 172.21.15.46:43292) 2023-10-06T11:18:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f843b7fe700 1 -- 172.21.15.46:0/3429779867 learned_addr learned my addr 172.21.15.46:0/3429779867 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.459+0000 7f843affd700 1 --2- 172.21.15.46:0/3429779867 >> v2:172.21.15.132:3300/0 conn(0x7f843c06e8a0 0x7f843c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843bfff700 1 -- 172.21.15.46:0/3429779867 >> v2:172.21.15.132:3300/0 conn(0x7f843c06e8a0 msgr2=0x7f843c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843bfff700 1 --2- 172.21.15.46:0/3429779867 >> v2:172.21.15.132:3300/0 conn(0x7f843c06e8a0 0x7f843c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843bfff700 1 -- 172.21.15.46:0/3429779867 >> v1:172.21.15.46:6789/0 conn(0x7f843c15ab60 legacy=0x7f843c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843bfff700 1 -- 172.21.15.46:0/3429779867 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0c72c0 con 0x7f843c0ad6c0 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843bfff700 1 --2- 172.21.15.46:0/3429779867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0ad6c0 0x7f843c0adaf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f843c06f0b0 tx=0x7f8434009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69fda5076ef04e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843a7fc700 1 -- 172.21.15.46:0/3429779867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400e070 con 0x7f843c0ad6c0 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843a7fc700 1 -- 172.21.15.46:0/3429779867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400bb60 con 0x7f843c0ad6c0 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f843a7fc700 1 -- 172.21.15.46:0/3429779867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434012430 con 0x7f843c0ad6c0 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 -- 172.21.15.46:0/3429779867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0ad6c0 msgr2=0x7f843c0adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 --2- 172.21.15.46:0/3429779867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0ad6c0 0x7f843c0adaf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f843c06f0b0 tx=0x7f8434009780 comp rx=0 tx=0).stop 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 -- 172.21.15.46:0/3429779867 shutdown_connections 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 --2- 172.21.15.46:0/3429779867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c0ad6c0 0x7f843c0adaf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 --2- 172.21.15.46:0/3429779867 >> v2:172.21.15.132:3300/0 conn(0x7f843c06e8a0 0x7f843c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 -- 172.21.15.46:0/3429779867 >> 172.21.15.46:0/3429779867 conn(0x7f843c0cbe30 msgr2=0x7f843c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 -- 172.21.15.46:0/3429779867 shutdown_connections 2023-10-06T11:18:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.463+0000 7f8441af0700 1 -- 172.21.15.46:0/3429779867 wait complete. 2023-10-06T11:18:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 Processor -- start 2023-10-06T11:18:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 -- start start 2023-10-06T11:18:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 0x7f843c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f843c0ad6c0 0x7f843c0c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0afb50 con 0x7f843c15ab60 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0afe30 con 0x7f843c06e8a0 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f8441af0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b0110 con 0x7f843c0ad6c0 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 0x7f843c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843affd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f843c0ad6c0 0x7f843c0c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44402/0 (socket says 172.21.15.46:44402) 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 -- 172.21.15.46:0/762373459 learned_addr learned my addr 172.21.15.46:0/762373459 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843b7fe700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 0x7f843c0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:34786/0 (socket says 172.21.15.46:34786) 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 -- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f843c0ad6c0 msgr2=0x7f843c0c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f843c0ad6c0 0x7f843c0c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 -- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 msgr2=0x7f843c0c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 0x7f843c0c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 -- 172.21.15.46:0/762373459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434009a20 con 0x7f843c15ab60 2023-10-06T11:18:36.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.467+0000 7f843bfff700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8434009fd0 tx=0x7f8434019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434027470 con 0x7f843c15ab60 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340275f0 con 0x7f843c15ab60 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c0b03f0 con 0x7f843c15ab60 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340279b0 con 0x7f843c15ab60 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c0af790 con 0x7f843c15ab60 2023-10-06T11:18:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c063f90 con 0x7f843c15ab60 2023-10-06T11:18:36.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.471+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8434024070 con 0x7f843c15ab60 2023-10-06T11:18:36.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.475+0000 7f84237fe700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 0x7f8428062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.475+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f843407f240 con 0x7f843c15ab60 2023-10-06T11:18:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.475+0000 7f843b7fe700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 0x7f8428062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.475+0000 7f843b7fe700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 0x7f8428062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f842c007fd0 tx=0x7f842c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.479+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f843404f630 con 0x7f843c15ab60 2023-10-06T11:18:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.651+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f843c04b5c0 con 0x7f843c15ab60 2023-10-06T11:18:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.651+0000 7f84237fe700 1 -- 172.21.15.46:0/762373459 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f8434052c50 con 0x7f843c15ab60 2023-10-06T11:18:36.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:36.657 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 msgr2=0x7f8428062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 0x7f8428062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f842c007fd0 tx=0x7f842c009040 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 msgr2=0x7f843c0c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8434009fd0 tx=0x7f8434019000 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 shutdown_connections 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8428060550 0x7f8428062a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f843c15ab60 0x7f843c0c2ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f843c0ad6c0 0x7f843c0c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 --2- 172.21.15.46:0/762373459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f843c06e8a0 0x7f843c0c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 >> 172.21.15.46:0/762373459 conn(0x7f843c0cbe30 msgr2=0x7f843c0ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 shutdown_connections 2023-10-06T11:18:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.655+0000 7f8441af0700 1 -- 172.21.15.46:0/762373459 wait complete. 2023-10-06T11:18:36.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:36.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:36.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.835+0000 7f71890ae700 1 Processor -- start 2023-10-06T11:18:36.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 -- start start 2023-10-06T11:18:36.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7184068160 0x7f71840b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71840b00f0 0x7f71840b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718415a5c0 con 0x7f7184069c10 2023-10-06T11:18:36.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71841597e0 con 0x7f71840b00f0 2023-10-06T11:18:36.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f71890ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71841599f0 con 0x7f7184068160 2023-10-06T11:18:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f718359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f71840b00f0 0x7f71840b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7184068160 0x7f71840b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f7182d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7184069c10 0x7f71840af9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43294/0 (socket says 172.21.15.46:43294) 2023-10-06T11:18:36.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f7182d9d700 1 -- 172.21.15.46:0/3455298738 learned_addr learned my addr 172.21.15.46:0/3455298738 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 -- 172.21.15.46:0/3455298738 >> v2:172.21.15.46:3300/0 conn(0x7f71840b00f0 msgr2=0x7f71840b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 --2- 172.21.15.46:0/3455298738 >> v2:172.21.15.46:3300/0 conn(0x7f71840b00f0 0x7f71840b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 -- 172.21.15.46:0/3455298738 >> v1:172.21.15.46:6789/0 conn(0x7f7184069c10 legacy=0x7f71840af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 -- 172.21.15.46:0/3455298738 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184159c90 con 0x7f7184068160 2023-10-06T11:18:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.839+0000 7f717bfff700 1 --2- 172.21.15.46:0/3455298738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f71840b1000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71840b0b20 tx=0x7f716c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c96c23808f77d39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f718259c700 1 -- 172.21.15.46:0/3455298738 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c011630 con 0x7f7184068160 2023-10-06T11:18:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f718259c700 1 -- 172.21.15.46:0/3455298738 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c011790 con 0x7f7184068160 2023-10-06T11:18:36.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f718259c700 1 -- 172.21.15.46:0/3455298738 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c011b30 con 0x7f7184068160 2023-10-06T11:18:36.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- 172.21.15.46:0/3455298738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 msgr2=0x7f71840b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 --2- 172.21.15.46:0/3455298738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f71840b1000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71840b0b20 tx=0x7f716c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- 172.21.15.46:0/3455298738 shutdown_connections 2023-10-06T11:18:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 --2- 172.21.15.46:0/3455298738 >> v2:172.21.15.46:3300/0 conn(0x7f71840b00f0 0x7f71840b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 --2- 172.21.15.46:0/3455298738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f71840b1000 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- 172.21.15.46:0/3455298738 >> 172.21.15.46:0/3455298738 conn(0x7f718406ae60 msgr2=0x7f718406b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:36.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- 172.21.15.46:0/3455298738 shutdown_connections 2023-10-06T11:18:36.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- 172.21.15.46:0/3455298738 wait complete. 2023-10-06T11:18:36.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 Processor -- start 2023-10-06T11:18:36.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.843+0000 7f71890ae700 1 -- start start 2023-10-06T11:18:36.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f7184153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 0x7f71841548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840af120 con 0x7f7184069c10 2023-10-06T11:18:36.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840ae510 con 0x7f7184068160 2023-10-06T11:18:36.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71840ae7f0 con 0x7f71840b00f0 2023-10-06T11:18:36.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44432/0 (socket says 172.21.15.46:44432) 2023-10-06T11:18:36.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 -- 172.21.15.46:0/3640781633 learned_addr learned my addr 172.21.15.46:0/3640781633 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:36.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f718359e700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 0x7f71841548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f7182d9d700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f7184153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 -- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 msgr2=0x7f71841548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 0x7f71841548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 -- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 msgr2=0x7f7184153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:36.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f7184153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 -- 172.21.15.46:0/3640781633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c009420 con 0x7f7184069c10 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f718359e700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 0x7f71841548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f7182d9d700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f7184153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717bfff700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f716c013000 tx=0x7f716c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c011b10 con 0x7f7184069c10 2023-10-06T11:18:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0253f0 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71840aead0 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c025790 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7184153aa0 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.847+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718406da40 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.851+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f716c025b80 con 0x7f7184069c10 2023-10-06T11:18:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.851+0000 7f717affd700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 0x7f7164062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.851+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f716c07e2b0 con 0x7f7184069c10 2023-10-06T11:18:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.851+0000 7f7182d9d700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 0x7f7164062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.855+0000 7f7182d9d700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 0x7f7164062ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f718413f9e0 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:36.859+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f716c04e6a0 con 0x7f7184069c10 2023-10-06T11:18:37.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.031+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7184159f70 con 0x7f7184069c10 2023-10-06T11:18:37.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.031+0000 7f717affd700 1 -- 172.21.15.46:0/3640781633 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f716c051cc0 con 0x7f7184069c10 2023-10-06T11:18:37.036 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.031+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 msgr2=0x7f7164062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.031+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 0x7f7164062ab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f718413f9e0 tx=0x7f7174009040 comp rx=0 tx=0).stop 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 msgr2=0x7f71841543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f716c013000 tx=0x7f716c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 shutdown_connections 2023-10-06T11:18:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7164060600 0x7f7164062ab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840b00f0 0x7f71841548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7184069c10 0x7f71841543b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 --2- 172.21.15.46:0/3640781633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184068160 0x7f7184153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 >> 172.21.15.46:0/3640781633 conn(0x7f718406ae60 msgr2=0x7f718415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 shutdown_connections 2023-10-06T11:18:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.035+0000 7f71890ae700 1 -- 172.21.15.46:0/3640781633 wait complete. 2023-10-06T11:18:37.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:37.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.219+0000 7f26ee016700 1 Processor -- start 2023-10-06T11:18:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.219+0000 7f26ee016700 1 -- start start 2023-10-06T11:18:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80adbe0 0x7f26e80d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26e80d0d60 0x7f26e80b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 0x7f26e80b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e8159740 con 0x7f26e80d0d60 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80afd20 con 0x7f26e80b3630 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ee016700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b0000 con 0x7f26e80adbe0 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26e80d0d60 0x7f26e80b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26e80d0d60 0x7f26e80b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44438/0 (socket says 172.21.15.46:44438) 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 -- 172.21.15.46:0/1875157069 learned_addr learned my addr 172.21.15.46:0/1875157069 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ed014700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80adbe0 0x7f26e80d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ed815700 1 --2- 172.21.15.46:0/1875157069 >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 0x7f26e80b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 -- 172.21.15.46:0/1875157069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80adbe0 msgr2=0x7f26e80d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80adbe0 0x7f26e80d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 -- 172.21.15.46:0/1875157069 >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 msgr2=0x7f26e80b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 --2- 172.21.15.46:0/1875157069 >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 0x7f26e80b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 -- 172.21.15.46:0/1875157069 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80b02e0 con 0x7f26e80d0d60 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ed815700 1 --2- 172.21.15.46:0/1875157069 >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 0x7f26e80b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.223+0000 7f26ec813700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80d0d60 0x7f26e80b30f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26e80885a0 tx=0x7f26d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c83f5ed2524e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26dffff700 1 -- 172.21.15.46:0/1875157069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8009e70 con 0x7f26e80d0d60 2023-10-06T11:18:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26dffff700 1 -- 172.21.15.46:0/1875157069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800b860 con 0x7f26e80d0d60 2023-10-06T11:18:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26dffff700 1 -- 172.21.15.46:0/1875157069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800bc00 con 0x7f26e80d0d60 2023-10-06T11:18:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 -- 172.21.15.46:0/1875157069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80d0d60 msgr2=0x7f26e80b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80d0d60 0x7f26e80b30f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26e80885a0 tx=0x7f26d8009250 comp rx=0 tx=0).stop 2023-10-06T11:18:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 -- 172.21.15.46:0/1875157069 shutdown_connections 2023-10-06T11:18:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 --2- 172.21.15.46:0/1875157069 >> v2:172.21.15.132:3300/0 conn(0x7f26e80b3630 0x7f26e80b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80d0d60 0x7f26e80b30f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 --2- 172.21.15.46:0/1875157069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80adbe0 0x7f26e80d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 -- 172.21.15.46:0/1875157069 >> 172.21.15.46:0/1875157069 conn(0x7f26e806ef60 msgr2=0x7f26e80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 -- 172.21.15.46:0/1875157069 shutdown_connections 2023-10-06T11:18:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.227+0000 7f26ee016700 1 -- 172.21.15.46:0/1875157069 wait complete. 2023-10-06T11:18:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 Processor -- start 2023-10-06T11:18:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 -- start start 2023-10-06T11:18:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e80b3630 0x7f26e8169d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 0x7f26e8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e809c750 con 0x7f26e80adbe0 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e815a0e0 con 0x7f26e80b3630 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ee016700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e815a3c0 con 0x7f26e80d0d60 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44444/0 (socket says 172.21.15.46:44444) 2023-10-06T11:18:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 -- 172.21.15.46:0/2096761393 learned_addr learned my addr 172.21.15.46:0/2096761393 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed815700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 0x7f26e8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ec813700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e80b3630 0x7f26e8169d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 -- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 msgr2=0x7f26e8158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 0x7f26e8158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 -- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e80b3630 msgr2=0x7f26e8169d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e80b3630 0x7f26e8169d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed014700 1 -- 172.21.15.46:0/2096761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80094f0 con 0x7f26e80adbe0 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.231+0000 7f26ed815700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 0x7f26e8158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26ed014700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e930 tx=0x7f26e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400eed0 con 0x7f26e80adbe0 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e400c850 con 0x7f26e80adbe0 2023-10-06T11:18:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e400cbf0 con 0x7f26e80adbe0 2023-10-06T11:18:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e815a6a0 con 0x7f26e80adbe0 2023-10-06T11:18:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e8169840 con 0x7f26e80adbe0 2023-10-06T11:18:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.235+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e8064590 con 0x7f26e80adbe0 2023-10-06T11:18:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.239+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f26e4013470 con 0x7f26e80adbe0 2023-10-06T11:18:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.239+0000 7f26dd7fa700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 0x7f26cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.239+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26e407cc20 con 0x7f26e80adbe0 2023-10-06T11:18:37.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.239+0000 7f26ec813700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 0x7f26cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.239+0000 7f26ec813700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 0x7f26cc062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26e80885a0 tx=0x7f26d8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.243+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f26e404d010 con 0x7f26e80adbe0 2023-10-06T11:18:37.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.415+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26e80675d0 con 0x7f26e80adbe0 2023-10-06T11:18:37.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.415+0000 7f26dd7fa700 1 -- 172.21.15.46:0/2096761393 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f26e4050630 con 0x7f26e80adbe0 2023-10-06T11:18:37.422 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:37.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 msgr2=0x7f26cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 0x7f26cc062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26e80885a0 tx=0x7f26d8017000 comp rx=0 tx=0).stop 2023-10-06T11:18:37.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 msgr2=0x7f26e8159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e930 tx=0x7f26e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 shutdown_connections 2023-10-06T11:18:37.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26cc060550 0x7f26cc062a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e80d0d60 0x7f26e8158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e80b3630 0x7f26e8169d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 --2- 172.21.15.46:0/2096761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e80adbe0 0x7f26e8159890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 >> 172.21.15.46:0/2096761393 conn(0x7f26e806ef60 msgr2=0x7f26e80b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:37.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 shutdown_connections 2023-10-06T11:18:37.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.419+0000 7f26ee016700 1 -- 172.21.15.46:0/2096761393 wait complete. 2023-10-06T11:18:37.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:37.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.607+0000 7f7650a8e700 1 Processor -- start 2023-10-06T11:18:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.607+0000 7f7650a8e700 1 -- start start 2023-10-06T11:18:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7650a8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0adbe0 0x7f764c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7650a8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f764c0d0d60 0x7f764c0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7650a8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0afd20 con 0x7f764c0b3630 2023-10-06T11:18:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7650a8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c159740 con 0x7f764c0d0d60 2023-10-06T11:18:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7650a8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0b0000 con 0x7f764c0adbe0 2023-10-06T11:18:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f764c0d0d60 0x7f764c0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0adbe0 0x7f764c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f764c0d0d60 0x7f764c0b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:34818/0 (socket says 172.21.15.46:34818) 2023-10-06T11:18:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 -- 172.21.15.46:0/445738138 learned_addr learned my addr 172.21.15.46:0/445738138 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 -- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0adbe0 msgr2=0x7f764c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 --2- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0adbe0 0x7f764c0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 -- 172.21.15.46:0/445738138 >> v1:172.21.15.132:6789/0 conn(0x7f764c0b3630 legacy=0x7f764c0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f7643fff700 1 -- 172.21.15.46:0/445738138 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0b02e0 con 0x7f764c0d0d60 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.611+0000 7f764affd700 1 -- 172.21.15.46:0/445738138 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215798055 0 0) 0x7f764c0afd20 con 0x7f764c0b3630 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7643fff700 1 --2- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c0b30f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76340099b0 tx=0x7f7634009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f112d6e2536bce64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f764affd700 1 -- 172.21.15.46:0/445738138 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634009ea0 con 0x7f764c0d0d60 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f764affd700 1 -- 172.21.15.46:0/445738138 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763400b860 con 0x7f764c0d0d60 2023-10-06T11:18:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f764affd700 1 -- 172.21.15.46:0/445738138 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400bc00 con 0x7f764c0d0d60 2023-10-06T11:18:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 -- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 msgr2=0x7f764c0b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 --2- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c0b30f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76340099b0 tx=0x7f7634009250 comp rx=0 tx=0).stop 2023-10-06T11:18:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 -- 172.21.15.46:0/445738138 shutdown_connections 2023-10-06T11:18:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 --2- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c0b30f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 --2- 172.21.15.46:0/445738138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0adbe0 0x7f764c0d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 -- 172.21.15.46:0/445738138 >> 172.21.15.46:0/445738138 conn(0x7f764c06ef60 msgr2=0x7f764c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 -- 172.21.15.46:0/445738138 shutdown_connections 2023-10-06T11:18:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.615+0000 7f7650a8e700 1 -- 172.21.15.46:0/445738138 wait complete. 2023-10-06T11:18:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.619+0000 7f7650a8e700 1 Processor -- start 2023-10-06T11:18:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.619+0000 7f7650a8e700 1 -- start start 2023-10-06T11:18:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0b3630 0x7f764c099d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c09ce70 con 0x7f764c0adbe0 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c09d150 con 0x7f764c0d0d60 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c09d430 con 0x7f764c0b3630 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44460/0 (socket says 172.21.15.46:44460) 2023-10-06T11:18:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 -- 172.21.15.46:0/3912824660 learned_addr learned my addr 172.21.15.46:0/3912824660 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764bfff700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7643fff700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0b3630 0x7f764c099d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 -- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0b3630 msgr2=0x7f764c099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0b3630 0x7f764c099d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 -- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 msgr2=0x7f764c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 -- 172.21.15.46:0/3912824660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340094f0 con 0x7f764c0adbe0 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764bfff700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f764b7fe700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e930 tx=0x7f763c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00eed0 con 0x7f764c0adbe0 2023-10-06T11:18:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00c850 con 0x7f764c0adbe0 2023-10-06T11:18:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00cbf0 con 0x7f764c0adbe0 2023-10-06T11:18:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.623+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c09d710 con 0x7f764c0adbe0 2023-10-06T11:18:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.627+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c09a280 con 0x7f764c0adbe0 2023-10-06T11:18:37.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.627+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c064590 con 0x7f764c0adbe0 2023-10-06T11:18:37.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.627+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f763c013470 con 0x7f764c0adbe0 2023-10-06T11:18:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.631+0000 7f76437fe700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 0x7f762c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.631+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f763c07cbc0 con 0x7f764c0adbe0 2023-10-06T11:18:37.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.631+0000 7f7643fff700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 0x7f762c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:37.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.631+0000 7f7643fff700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 0x7f762c062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76340099b0 tx=0x7f7634017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.639+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f763c04cfb0 con 0x7f764c0adbe0 2023-10-06T11:18:37.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.803+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f764c0675d0 con 0x7f764c0adbe0 2023-10-06T11:18:37.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.807+0000 7f76437fe700 1 -- 172.21.15.46:0/3912824660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f763c0505d0 con 0x7f764c0adbe0 2023-10-06T11:18:37.811 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.807+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 msgr2=0x7f762c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.807+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 0x7f762c062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f76340099b0 tx=0x7f7634017000 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.807+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 msgr2=0x7f764c09c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.807+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e930 tx=0x7f763c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 shutdown_connections 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f762c060550 0x7f762c062a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f764c0d0d60 0x7f764c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f764c0b3630 0x7f764c099d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 --2- 172.21.15.46:0/3912824660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f764c0adbe0 0x7f764c09c890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 >> 172.21.15.46:0/3912824660 conn(0x7f764c06ef60 msgr2=0x7f764c0aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 shutdown_connections 2023-10-06T11:18:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.811+0000 7f7650a8e700 1 -- 172.21.15.46:0/3912824660 wait complete. 2023-10-06T11:18:37.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.991+0000 7fa5b78cd700 1 Processor -- start 2023-10-06T11:18:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.991+0000 7fa5b78cd700 1 -- start start 2023-10-06T11:18:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b78cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b78cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5b0067050 0x7fa5b00674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b78cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b006e250 con 0x7fa5b00af560 2023-10-06T11:18:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b78cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b006dcd0 con 0x7fa5b0067050 2023-10-06T11:18:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b78cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0067a10 con 0x7fa5b00cc6e0 2023-10-06T11:18:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5b0067050 0x7fa5b00674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5b0067050 0x7fa5b00674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44470/0 (socket says 172.21.15.46:44470) 2023-10-06T11:18:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b4e68700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa5b00af560 0x7fa5b00ab910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40416/0 (socket says 172.21.15.46:40416) 2023-10-06T11:18:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 -- 172.21.15.46:0/661641116 learned_addr learned my addr 172.21.15.46:0/661641116 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5669700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 -- 172.21.15.46:0/661641116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 msgr2=0x7fa5b006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 -- 172.21.15.46:0/661641116 >> v1:172.21.15.132:6789/0 conn(0x7fa5b00af560 legacy=0x7fa5b00ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.995+0000 7fa5b5e6a700 1 -- 172.21.15.46:0/661641116 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00ac1f0 con 0x7fa5b0067050 2023-10-06T11:18:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b5e6a700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b0067050 0x7fa5b00674d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009f20 tx=0x7fa5a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2a6c87cfcdb74d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5a7fff700 1 -- 172.21.15.46:0/661641116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800e070 con 0x7fa5b0067050 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5a7fff700 1 -- 172.21.15.46:0/661641116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800b8d0 con 0x7fa5b0067050 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5a7fff700 1 -- 172.21.15.46:0/661641116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80092e0 con 0x7fa5b0067050 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 -- 172.21.15.46:0/661641116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b0067050 msgr2=0x7fa5b00674d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b0067050 0x7fa5b00674d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009f20 tx=0x7fa5a8009840 comp rx=0 tx=0).stop 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 -- 172.21.15.46:0/661641116 shutdown_connections 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b0067050 0x7fa5b00674d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/661641116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 -- 172.21.15.46:0/661641116 >> 172.21.15.46:0/661641116 conn(0x7fa5b0068c30 msgr2=0x7fa5b00aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 -- 172.21.15.46:0/661641116 shutdown_connections 2023-10-06T11:18:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:37.999+0000 7fa5b78cd700 1 -- 172.21.15.46:0/661641116 wait complete. 2023-10-06T11:18:38.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 Processor -- start 2023-10-06T11:18:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 -- start start 2023-10-06T11:18:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5b0067050 0x7fa5b015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b00c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015a460 con 0x7fa5b00af560 2023-10-06T11:18:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015a740 con 0x7fa5b0067050 2023-10-06T11:18:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b78cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015df90 con 0x7fa5b00cc6e0 2023-10-06T11:18:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.003+0000 7fa5b4e68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44474/0 (socket says 172.21.15.46:44474) 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 -- 172.21.15.46:0/2311998364 learned_addr learned my addr 172.21.15.46:0/2311998364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b5669700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5b0067050 0x7fa5b015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b5e6a700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b00c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 -- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 msgr2=0x7fa5b00c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b00c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 -- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5b0067050 msgr2=0x7fa5b015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5b0067050 0x7fa5b015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 -- 172.21.15.46:0/2311998364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009ae0 con 0x7fa5b00af560 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b4e68700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c840 tx=0x7fa5a0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0005a70 con 0x7fa5b00af560 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0005bf0 con 0x7fa5b00af560 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00070d0 con 0x7fa5b00af560 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b015e270 con 0x7fa5b00af560 2023-10-06T11:18:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.007+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b015dc50 con 0x7fa5b00af560 2023-10-06T11:18:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b015e770 con 0x7fa5b00af560 2023-10-06T11:18:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa5a0007440 con 0x7fa5b00af560 2023-10-06T11:18:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5a57fa700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 0x7fa594062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa5a007bf40 con 0x7fa5b00af560 2023-10-06T11:18:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5b5669700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 0x7fa594062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.011+0000 7fa5b5669700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 0x7fa594062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009db0 tx=0x7fa5ac0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.019+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa5a004c330 con 0x7fa5b00af560 2023-10-06T11:18:38.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.187+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5b0069550 con 0x7fa5b00af560 2023-10-06T11:18:38.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.191+0000 7fa5a57fa700 1 -- 172.21.15.46:0/2311998364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5a00170a0 con 0x7fa5b00af560 2023-10-06T11:18:38.194 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 msgr2=0x7fa594062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 0x7fa594062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009db0 tx=0x7fa5ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 msgr2=0x7fa5b0155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c840 tx=0x7fa5a0007f30 comp rx=0 tx=0).stop 2023-10-06T11:18:38.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 shutdown_connections 2023-10-06T11:18:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa594060550 0x7fa594062a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5b00cc6e0 0x7fa5b00c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5b00af560 0x7fa5b0155320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 --2- 172.21.15.46:0/2311998364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5b0067050 0x7fa5b015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 >> 172.21.15.46:0/2311998364 conn(0x7fa5b0068c30 msgr2=0x7fa5b00ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 shutdown_connections 2023-10-06T11:18:38.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.195+0000 7fa5b78cd700 1 -- 172.21.15.46:0/2311998364 wait complete. 2023-10-06T11:18:38.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 Processor -- start 2023-10-06T11:18:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 -- start start 2023-10-06T11:18:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d0c156c80 0x7f5d0c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d0c0c7b50 0x7f5d0c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0c8500 con 0x7f5d0c15cde0 2023-10-06T11:18:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c6470 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d1202e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c7760 con 0x7f5d0c156c80 2023-10-06T11:18:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d0affd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d0c0c7b50 0x7f5d0c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d0bfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5d0c15cde0 0x7f5d0c15d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43304/0 (socket says 172.21.15.46:43304) 2023-10-06T11:18:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.379+0000 7f5d0bfff700 1 -- 172.21.15.46:0/318094250 learned_addr learned my addr 172.21.15.46:0/318094250 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:38.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0b7fe700 1 --2- 172.21.15.46:0/318094250 >> v2:172.21.15.132:3300/0 conn(0x7f5d0c156c80 0x7f5d0c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0affd700 1 -- 172.21.15.46:0/318094250 >> v2:172.21.15.132:3300/0 conn(0x7f5d0c156c80 msgr2=0x7f5d0c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0a7fc700 1 -- 172.21.15.46:0/318094250 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149376058 0 0) 0x7f5d0c0c8500 con 0x7f5d0c15cde0 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0affd700 1 --2- 172.21.15.46:0/318094250 >> v2:172.21.15.132:3300/0 conn(0x7f5d0c156c80 0x7f5d0c0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0affd700 1 -- 172.21.15.46:0/318094250 >> v1:172.21.15.46:6789/0 conn(0x7f5d0c15cde0 legacy=0x7f5d0c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0affd700 1 -- 172.21.15.46:0/318094250 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0ca710 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0affd700 1 --2- 172.21.15.46:0/318094250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c7fc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c06c640 tx=0x7f5d000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9349d151597722a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0a7fc700 1 -- 172.21.15.46:0/318094250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011630 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0a7fc700 1 -- 172.21.15.46:0/318094250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00011790 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d0a7fc700 1 -- 172.21.15.46:0/318094250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011b30 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 -- 172.21.15.46:0/318094250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c0c7b50 msgr2=0x7f5d0c0c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 --2- 172.21.15.46:0/318094250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c7fc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c06c640 tx=0x7f5d000091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 -- 172.21.15.46:0/318094250 shutdown_connections 2023-10-06T11:18:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 --2- 172.21.15.46:0/318094250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c7fc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 --2- 172.21.15.46:0/318094250 >> v2:172.21.15.132:3300/0 conn(0x7f5d0c156c80 0x7f5d0c0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 -- 172.21.15.46:0/318094250 >> 172.21.15.46:0/318094250 conn(0x7f5d0c0cbe30 msgr2=0x7f5d0c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 -- 172.21.15.46:0/318094250 shutdown_connections 2023-10-06T11:18:38.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.383+0000 7f5d1202e700 1 -- 172.21.15.46:0/318094250 wait complete. 2023-10-06T11:18:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.387+0000 7f5d1202e700 1 Processor -- start 2023-10-06T11:18:38.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.387+0000 7f5d1202e700 1 -- start start 2023-10-06T11:18:38.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 0x7f5d0c150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c06cbf0 con 0x7f5d0c15cde0 2023-10-06T11:18:38.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c06da50 con 0x7f5d0c156c80 2023-10-06T11:18:38.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0c3420 con 0x7f5d0c0c7b50 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 0x7f5d0c150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 0x7f5d0c150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:34852/0 (socket says 172.21.15.46:34852) 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44478/0 (socket says 172.21.15.46:44478) 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0affd700 1 -- 172.21.15.46:0/3658419274 learned_addr learned my addr 172.21.15.46:0/3658419274 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:38.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:35536/0 (socket says 172.21.15.46:35536) 2023-10-06T11:18:38.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 -- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 msgr2=0x7f5d0c0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 -- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 msgr2=0x7f5d0c150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 0x7f5d0c150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 -- 172.21.15.46:0/3658419274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00009420 con 0x7f5d0c15cde0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d0bfff700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400cc40 tx=0x7f5d040019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400a8c0 con 0x7f5d0c15cde0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400aa40 con 0x7f5d0c15cde0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c06ce40 con 0x7f5d0c15cde0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400ae00 con 0x7f5d0c15cde0 2023-10-06T11:18:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.391+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c06d400 con 0x7f5d0c15cde0 2023-10-06T11:18:38.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.395+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c063f90 con 0x7f5d0c15cde0 2023-10-06T11:18:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.395+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5d04007300 con 0x7f5d0c15cde0 2023-10-06T11:18:38.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.399+0000 7f5cf37fe700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.399+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d0407ccc0 con 0x7f5d0c15cde0 2023-10-06T11:18:38.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.399+0000 7f5d0b7fe700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.399+0000 7f5d0b7fe700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 0x7f5cf8062a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009da0 tx=0x7f5cfc009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.403+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d04083080 con 0x7f5d0c15cde0 2023-10-06T11:18:38.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.615+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d0c0c1e90 con 0x7f5d0c15cde0 2023-10-06T11:18:38.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.615+0000 7f5cf37fe700 1 -- 172.21.15.46:0/3658419274 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f5d0404d130 con 0x7f5d0c15cde0 2023-10-06T11:18:38.620 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:38.620 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 msgr2=0x7f5cf8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 0x7f5cf8062a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009da0 tx=0x7f5cfc009120 comp rx=0 tx=0).stop 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 msgr2=0x7f5d0c14ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400cc40 tx=0x7f5d040019a0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 shutdown_connections 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf8060580 0x7f5cf8062a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d0c15cde0 0x7f5d0c14ff70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d0c156c80 0x7f5d0c150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 --2- 172.21.15.46:0/3658419274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d0c0c7b50 0x7f5d0c0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 >> 172.21.15.46:0/3658419274 conn(0x7f5d0c0cbe30 msgr2=0x7f5d0c15d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:38.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 shutdown_connections 2023-10-06T11:18:38.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.619+0000 7f5d1202e700 1 -- 172.21.15.46:0/3658419274 wait complete. 2023-10-06T11:18:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:18:38.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.791+0000 7fc60e08e700 1 Processor -- start 2023-10-06T11:18:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 -- start start 2023-10-06T11:18:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 0x7fc60806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc608067050 0x7fc608067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080c74d0 con 0x7fc6080c6150 2023-10-06T11:18:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080679d0 con 0x7fc608155c70 2023-10-06T11:18:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60e08e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080c68c0 con 0x7fc608067050 2023-10-06T11:18:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60d08c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 0x7fc60806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60d08c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 0x7fc60806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44492/0 (socket says 172.21.15.46:44492) 2023-10-06T11:18:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60d08c700 1 -- 172.21.15.46:0/2559761080 learned_addr learned my addr 172.21.15.46:0/2559761080 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.795+0000 7fc60d88d700 1 --2- 172.21.15.46:0/2559761080 >> v2:172.21.15.132:3300/0 conn(0x7fc608067050 0x7fc608067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60d88d700 1 -- 172.21.15.46:0/2559761080 >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 msgr2=0x7fc60806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60d88d700 1 --2- 172.21.15.46:0/2559761080 >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 0x7fc60806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60d88d700 1 -- 172.21.15.46:0/2559761080 >> v1:172.21.15.132:6789/0 conn(0x7fc6080c6150 legacy=0x7fc6080c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60d88d700 1 -- 172.21.15.46:0/2559761080 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080c6ba0 con 0x7fc608067050 2023-10-06T11:18:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60d88d700 1 --2- 172.21.15.46:0/2559761080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc608067490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc60806c920 tx=0x7fc6000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2bfa8d43f33dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc5fffff700 1 -- 172.21.15.46:0/2559761080 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011630 con 0x7fc608067050 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc5fffff700 1 -- 172.21.15.46:0/2559761080 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600011790 con 0x7fc608067050 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc5fffff700 1 -- 172.21.15.46:0/2559761080 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600011b30 con 0x7fc608067050 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 -- 172.21.15.46:0/2559761080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 msgr2=0x7fc608067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2559761080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc608067490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc60806c920 tx=0x7fc6000091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 -- 172.21.15.46:0/2559761080 shutdown_connections 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2559761080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc608067490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2559761080 >> v2:172.21.15.46:3300/0 conn(0x7fc608155c70 0x7fc60806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 -- 172.21.15.46:0/2559761080 >> 172.21.15.46:0/2559761080 conn(0x7fc608062ef0 msgr2=0x7fc6080cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 -- 172.21.15.46:0/2559761080 shutdown_connections 2023-10-06T11:18:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.799+0000 7fc60e08e700 1 -- 172.21.15.46:0/2559761080 wait complete. 2023-10-06T11:18:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 Processor -- start 2023-10-06T11:18:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 -- start start 2023-10-06T11:18:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc6080c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 0x7fc608169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 0x7fc6080af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60816ab70 con 0x7fc6080c6150 2023-10-06T11:18:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60816b210 con 0x7fc608067050 2023-10-06T11:18:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60e08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60816a5b0 con 0x7fc608155c70 2023-10-06T11:18:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60c88b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 0x7fc608169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60d88d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 0x7fc6080af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60d08c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc6080c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60d88d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 0x7fc6080af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:35550/0 (socket says 172.21.15.46:35550) 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.803+0000 7fc60d88d700 1 -- 172.21.15.46:0/2280787178 learned_addr learned my addr 172.21.15.46:0/2280787178 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 -- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 msgr2=0x7fc6080af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 0x7fc6080af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 -- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 msgr2=0x7fc6080c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc6080c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 -- 172.21.15.46:0/2280787178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc600009420 con 0x7fc6080c6150 2023-10-06T11:18:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60c88b700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 0x7fc608169b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c590 tx=0x7fc5f8007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8011590 con 0x7fc6080c6150 2023-10-06T11:18:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8011710 con 0x7fc6080c6150 2023-10-06T11:18:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8011ad0 con 0x7fc6080c6150 2023-10-06T11:18:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60816a890 con 0x7fc6080c6150 2023-10-06T11:18:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6081547c0 con 0x7fc6080c6150 2023-10-06T11:18:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.807+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60816ace0 con 0x7fc6080c6150 2023-10-06T11:18:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.811+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc5f801c440 con 0x7fc6080c6150 2023-10-06T11:18:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.811+0000 7fc5fd7fa700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.811+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc5f807bd20 con 0x7fc6080c6150 2023-10-06T11:18:38.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.811+0000 7fc60d08c700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.811+0000 7fc60d08c700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 0x7fc5ec062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc604009fd0 tx=0x7fc60400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.815+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc5f804c110 con 0x7fc6080c6150 2023-10-06T11:18:38.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.991+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc608064590 con 0x7fc6080c6150 2023-10-06T11:18:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.991+0000 7fc5fd7fa700 1 -- 172.21.15.46:0/2280787178 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fc5f8017020 con 0x7fc6080c6150 2023-10-06T11:18:38.997 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:38.997 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 msgr2=0x7fc5ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 0x7fc5ec062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc604009fd0 tx=0x7fc60400b040 comp rx=0 tx=0).stop 2023-10-06T11:18:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 msgr2=0x7fc608169b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 0x7fc608169b80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c590 tx=0x7fc5f8007ec0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 shutdown_connections 2023-10-06T11:18:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5ec060550 0x7fc5ec062a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc608155c70 0x7fc6080af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080c6150 0x7fc608169b80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 --2- 172.21.15.46:0/2280787178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc608067050 0x7fc6080c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 >> 172.21.15.46:0/2280787178 conn(0x7fc608062ef0 msgr2=0x7fc608152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 shutdown_connections 2023-10-06T11:18:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:38.995+0000 7fc60e08e700 1 -- 172.21.15.46:0/2280787178 wait complete. 2023-10-06T11:18:39.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 Processor -- start 2023-10-06T11:18:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 -- start start 2023-10-06T11:18:39.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bcc159380 0x7f7bcc0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0ae980 con 0x7f7bcc15da30 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0ac470 con 0x7f7bcc0ac890 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.175+0000 7f7bd1545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc069b70 con 0x7f7bcc159380 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bca7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7bcc15da30 0x7f7bcc0af970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43314/0 (socket says 172.21.15.46:43314) 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bca7fc700 1 -- 172.21.15.46:0/3689952303 learned_addr learned my addr 172.21.15.46:0/3689952303 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:39.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bcaffd700 1 --2- 172.21.15.46:0/3689952303 >> v2:172.21.15.46:3300/0 conn(0x7f7bcc159380 0x7f7bcc0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126610824 0 0) 0x7f7bcc0ae980 con 0x7f7bcc15da30 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb4003570 con 0x7f7bcc15da30 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596034568 0 0) 0x7f7bcc0ac470 con 0x7f7bcc0ac890 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bcc0ae980 con 0x7f7bcc0ac890 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bcaffd700 1 -- 172.21.15.46:0/3689952303 >> v1:172.21.15.132:6789/0 conn(0x7f7bcc0ac890 legacy=0x7f7bcc0acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bcaffd700 1 -- 172.21.15.46:0/3689952303 >> v1:172.21.15.46:6789/0 conn(0x7f7bcc15da30 legacy=0x7f7bcc0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bcaffd700 1 -- 172.21.15.46:0/3689952303 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0af4e0 con 0x7f7bcc159380 2023-10-06T11:18:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bcaffd700 1 --2- 172.21.15.46:0/3689952303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc159380 0x7f7bcc0abc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0098d0 tx=0x7f7bbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4e5d2b7c54e3df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc011630 con 0x7f7bcc159380 2023-10-06T11:18:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc011790 con 0x7f7bcc159380 2023-10-06T11:18:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bc9ffb700 1 -- 172.21.15.46:0/3689952303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc011b30 con 0x7f7bcc159380 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 -- 172.21.15.46:0/3689952303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc159380 msgr2=0x7f7bcc0abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 --2- 172.21.15.46:0/3689952303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc159380 0x7f7bcc0abc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0098d0 tx=0x7f7bbc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 -- 172.21.15.46:0/3689952303 shutdown_connections 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 --2- 172.21.15.46:0/3689952303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc159380 0x7f7bcc0abc20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 -- 172.21.15.46:0/3689952303 >> 172.21.15.46:0/3689952303 conn(0x7f7bcc097190 msgr2=0x7f7bcc067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:39.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 -- 172.21.15.46:0/3689952303 shutdown_connections 2023-10-06T11:18:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.179+0000 7f7bd1545700 1 -- 172.21.15.46:0/3689952303 wait complete. 2023-10-06T11:18:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 Processor -- start 2023-10-06T11:18:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 -- start start 2023-10-06T11:18:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bcc0ac890 0x7f7bcc156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 0x7f7bcc15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b0080 con 0x7f7bcc15da30 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc16b0f0 con 0x7f7bcc159380 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bd1545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc15ae70 con 0x7f7bcc0ac890 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bcb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bcb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44536/0 (socket says 172.21.15.46:44536) 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bcb7fe700 1 -- 172.21.15.46:0/2908716853 learned_addr learned my addr 172.21.15.46:0/2908716853 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bcaffd700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bcc0ac890 0x7f7bcc156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.183+0000 7f7bca7fc700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 0x7f7bcc15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 -- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bcc0ac890 msgr2=0x7f7bcc156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bcc0ac890 0x7f7bcc156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 -- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 msgr2=0x7f7bcc15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 0x7f7bcc15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 -- 172.21.15.46:0/2908716853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0093c0 con 0x7f7bcc15da30 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bca7fc700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 0x7f7bcc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bcb7fe700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c910 tx=0x7f7bc40027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4012630 con 0x7f7bcc15da30 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc40127b0 con 0x7f7bcc15da30 2023-10-06T11:18:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc16ab30 con 0x7f7bcc15da30 2023-10-06T11:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4012b70 con 0x7f7bcc15da30 2023-10-06T11:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc15ab00 con 0x7f7bcc15da30 2023-10-06T11:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.187+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc064590 con 0x7f7bcc15da30 2023-10-06T11:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.191+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bc401b400 con 0x7f7bcc15da30 2023-10-06T11:18:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.191+0000 7f7bb2ffd700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 0x7f7bb8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.191+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7bc407cda0 con 0x7f7bcc15da30 2023-10-06T11:18:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.191+0000 7f7bcaffd700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 0x7f7bb8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.191+0000 7f7bcaffd700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 0x7f7bb8062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0091e0 tx=0x7f7bbc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.199+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7bc404d190 con 0x7f7bcc15da30 2023-10-06T11:18:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.403+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bcc0aebb0 con 0x7f7bcc15da30 2023-10-06T11:18:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.403+0000 7f7bb2ffd700 1 -- 172.21.15.46:0/2908716853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f7bc4016090 con 0x7f7bcc15da30 2023-10-06T11:18:39.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:39.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:18:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 msgr2=0x7f7bb8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 0x7f7bb8062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0091e0 tx=0x7f7bbc017000 comp rx=0 tx=0).stop 2023-10-06T11:18:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 msgr2=0x7f7bcc16a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400c910 tx=0x7f7bc40027a0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 shutdown_connections 2023-10-06T11:18:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7bb8060550 0x7f7bb8062a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc15da30 0x7f7bcc16a0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bcc159380 0x7f7bcc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 --2- 172.21.15.46:0/2908716853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bcc0ac890 0x7f7bcc156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 >> 172.21.15.46:0/2908716853 conn(0x7f7bcc097190 msgr2=0x7f7bcc06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 shutdown_connections 2023-10-06T11:18:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.407+0000 7f7bd1545700 1 -- 172.21.15.46:0/2908716853 wait complete. 2023-10-06T11:18:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:18:39.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:18:23.206297+0000', 'modified': '2023-10-06T11:18:27.079467+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6835/1894883507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 1894883507}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 1894883507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:18:39.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:18:39.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T11:18:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.999+0000 7fbffbbc5700 1 Processor -- start 2023-10-06T11:18:40.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:39.999+0000 7fbffbbc5700 1 -- start start 2023-10-06T11:18:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffbbc5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbff406cc30 0x7fbff406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffbbc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40c18f0 con 0x7fbff4161840 2023-10-06T11:18:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffbbc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40cb120 con 0x7fbff4161470 2023-10-06T11:18:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffbbc5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40c0ce0 con 0x7fbff406cc30 2023-10-06T11:18:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbff406cc30 0x7fbff406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbff406cc30 0x7fbff406d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44548/0 (socket says 172.21.15.46:44548) 2023-10-06T11:18:40.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 -- 172.21.15.46:0/1992672593 learned_addr learned my addr 172.21.15.46:0/1992672593 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 -- 172.21.15.46:0/1992672593 >> v1:172.21.15.132:6789/0 conn(0x7fbff4161470 legacy=0x7fbff40c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 -- 172.21.15.46:0/1992672593 >> v1:172.21.15.46:6789/0 conn(0x7fbff4161840 legacy=0x7fbff40caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 -- 172.21.15.46:0/1992672593 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40c0fc0 con 0x7fbff406cc30 2023-10-06T11:18:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbffa162700 1 --2- 172.21.15.46:0/1992672593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff406cc30 0x7fbff406d0b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009f20 tx=0x7fbfec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ade39b0bbc613b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbff895f700 1 -- 172.21.15.46:0/1992672593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00e070 con 0x7fbff406cc30 2023-10-06T11:18:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbff895f700 1 -- 172.21.15.46:0/1992672593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00bb10 con 0x7fbff406cc30 2023-10-06T11:18:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.003+0000 7fbff895f700 1 -- 172.21.15.46:0/1992672593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0092e0 con 0x7fbff406cc30 2023-10-06T11:18:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- 172.21.15.46:0/1992672593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff406cc30 msgr2=0x7fbff406d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/1992672593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff406cc30 0x7fbff406d0b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009f20 tx=0x7fbfec009840 comp rx=0 tx=0).stop 2023-10-06T11:18:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- 172.21.15.46:0/1992672593 shutdown_connections 2023-10-06T11:18:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/1992672593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff406cc30 0x7fbff406d0b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- 172.21.15.46:0/1992672593 >> 172.21.15.46:0/1992672593 conn(0x7fbff40cbe30 msgr2=0x7fbff40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- 172.21.15.46:0/1992672593 shutdown_connections 2023-10-06T11:18:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- 172.21.15.46:0/1992672593 wait complete. 2023-10-06T11:18:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 Processor -- start 2023-10-06T11:18:40.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.007+0000 7fbffbbc5700 1 -- start start 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff406cc30 0x7fbff414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff4161840 0x7fbff40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40afdf0 con 0x7fbff4161470 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40c5920 con 0x7fbff406cc30 2023-10-06T11:18:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40c5c00 con 0x7fbff4161840 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44558/0 (socket says 172.21.15.46:44558) 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 -- 172.21.15.46:0/3593042242 learned_addr learned my addr 172.21.15.46:0/3593042242 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffa162700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff4161840 0x7fbff40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9961700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff406cc30 0x7fbff414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 -- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff4161840 msgr2=0x7fbff40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff4161840 0x7fbff40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 -- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff406cc30 msgr2=0x7fbff414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff406cc30 0x7fbff414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 -- 172.21.15.46:0/3593042242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec009ae0 con 0x7fbff4161470 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbff9160700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c4e0 tx=0x7fbfe400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbfe9ffb700 1 -- 172.21.15.46:0/3593042242 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011590 con 0x7fbff4161470 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbfe9ffb700 1 -- 172.21.15.46:0/3593042242 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4011710 con 0x7fbff4161470 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff40c5ee0 con 0x7fbff4161470 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbfe9ffb700 1 -- 172.21.15.46:0/3593042242 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011ad0 con 0x7fbff4161470 2023-10-06T11:18:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.011+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff415d690 con 0x7fbff4161470 2023-10-06T11:18:40.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbfe9ffb700 1 -- 172.21.15.46:0/3593042242 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbfe401b3b0 con 0x7fbff4161470 2023-10-06T11:18:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbfe9ffb700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 0x7fbfd8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbfe9ffb700 1 -- 172.21.15.46:0/3593042242 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbfe407bd30 con 0x7fbff4161470 2023-10-06T11:18:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbff9961700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 0x7fbfd8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbffde95740 1 Processor -- start 2023-10-06T11:18:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbff9961700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 0x7fbfd8062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007fd0 tx=0x7fbff0007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.015+0000 7fbffde95740 1 -- start start 2023-10-06T11:18:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x244a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x2378f30 con 0x2218790 2023-10-06T11:18:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2374ed0 con 0x21cd710 2023-10-06T11:18:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x233b0b0 con 0x21cdae0 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44560/0 (socket says 172.21.15.46:44560) 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 -- 172.21.15.46:0/469101066 learned_addr learned my addr 172.21.15.46:0/469101066 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbff9160700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbff9961700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x244a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 -- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 msgr2=0x244a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x244a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 -- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 msgr2=0x244ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 -- 172.21.15.46:0/469101066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x22199e0 con 0x2218790 2023-10-06T11:18:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbff9961700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x244a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffa162700 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbff406c150 tx=0x7fbfec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6670939f3ac63e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbfdeffd700 1 -- 172.21.15.46:0/469101066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0092e0 con 0x2218790 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbfdeffd700 1 -- 172.21.15.46:0/469101066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00bc80 con 0x2218790 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbfdeffd700 1 -- 172.21.15.46:0/469101066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007820 con 0x2218790 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- 172.21.15.46:0/469101066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 msgr2=0x244b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbff406c150 tx=0x7fbfec018000 comp rx=0 tx=0).stop 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- 172.21.15.46:0/469101066 shutdown_connections 2023-10-06T11:18:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x2218790 0x244b0b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 --2- 172.21.15.46:0/469101066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x244a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.019+0000 7fbffde95740 1 -- 172.21.15.46:0/469101066 >> 172.21.15.46:0/469101066 conn(0x21cdeb0 msgr2=0x2407060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- 172.21.15.46:0/469101066 shutdown_connections 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- 172.21.15.46:0/469101066 wait complete. 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- start start 2023-10-06T11:18:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 Processor -- start 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x2404c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x233ad50 con 0x21cdae0 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2313970 con 0x21cd710 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.023+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x22b1430 con 0x2218790 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 10 client.0 _ll_register_callbacks cb 0x2317b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:18:40.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44572/0 (socket says 172.21.15.46:44572) 2023-10-06T11:18:40.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 -- 172.21.15.46:0/1221342057 learned_addr learned my addr 172.21.15.46:0/1221342057 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9961700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffa162700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x2404c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 -- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 msgr2=0x2404c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x2404c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 -- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 msgr2=0x244ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 -- 172.21.15.46:0/1221342057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40c0fc0 con 0x21cdae0 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffa162700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x2404c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:40.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9160700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400a040 tx=0x7fbfe4052040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb198afb078460d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011e80 con 0x21cdae0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbfe40550b0 con 0x21cdae0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2313cc0 con 0x21cdae0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe404e180 con 0x21cdae0 2023-10-06T11:18:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 10 client.4677 fetch_fsmap learned FSMap version 8 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x2343370 con 0x21cdae0 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4078070 con 0x21cdae0 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbfe404f2d0 con 0x21cdae0 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 20 client.4677 populate_metadata read hostname 'smithi046' 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2406170 0x2459de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 4 client.4677 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 --> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x23173e0 con 0x2406170 2023-10-06T11:18:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9961700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2406170 0x2459de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbff9961700 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2406170 0x2459de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc2d4b9212d2ed3 server_cookie=623e756edc272a75 in_seq=0 out_seq=0 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.027+0000 7fbfd3fff700 10 client.4677 ms_handle_connect on v2:172.21.15.132:6834/1894883507 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbfd3fff700 1 -- 172.21.15.46:0/1221342057 <== mds.0 v2:172.21.15.132:6834/1894883507 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x23173e0 con 0x2406170 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbfd3fff700 10 client.4677 handle_command_reply: tid=0 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 client.4677 shutdown 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 msgr2=0x244f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400a040 tx=0x7fbfe4052040 comp rx=0 tx=0).stop 2023-10-06T11:18:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 shutdown_connections 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x2404c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x244f180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2406170 0x2459de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 --2- 172.21.15.46:0/1221342057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x244ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbfd3fff700 0 client.4677 ms_handle_reset on v2:172.21.15.132:6834/1894883507 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 >> 172.21.15.46:0/1221342057 conn(0x21cdeb0 msgr2=0x242e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 shutdown_connections 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 1 -- 172.21.15.46:0/1221342057 wait complete. 2023-10-06T11:18:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.031+0000 7fbffde95740 20 client.4677 trim_cache size 0 max 16384 2023-10-06T11:18:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 Processor -- start 2023-10-06T11:18:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 -- start start 2023-10-06T11:18:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x2406540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 0x2406a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x23785d0 con 0x21cd710 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2349610 con 0x2218790 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x234a550 con 0x21cdae0 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44576/0 (socket says 172.21.15.46:44576) 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 -- 172.21.15.46:0/4152753236 learned_addr learned my addr 172.21.15.46:0/4152753236 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9160700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x2406540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffa162700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 0x2406a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 -- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 msgr2=0x2406540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x2406540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 -- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 msgr2=0x2406a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 0x2406a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 -- 172.21.15.46:0/4152753236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff406e8b0 con 0x21cd710 2023-10-06T11:18:40.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbffa162700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 0x2406a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.051+0000 7fbff9961700 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbff0002600 tx=0x7fbff0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b20b59294680cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbfdf7fe700 1 -- 172.21.15.46:0/4152753236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0002e20 con 0x21cd710 2023-10-06T11:18:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbfdf7fe700 1 -- 172.21.15.46:0/4152753236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0027a70 con 0x21cd710 2023-10-06T11:18:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbfdf7fe700 1 -- 172.21.15.46:0/4152753236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0027df0 con 0x21cd710 2023-10-06T11:18:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- 172.21.15.46:0/4152753236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 msgr2=0x249cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbff0002600 tx=0x7fbff0014000 comp rx=0 tx=0).stop 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- 172.21.15.46:0/4152753236 shutdown_connections 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x2218790 0x2406a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x21cdae0 0x2406540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 --2- 172.21.15.46:0/4152753236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cd710 0x249cbf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- 172.21.15.46:0/4152753236 >> 172.21.15.46:0/4152753236 conn(0x21cdeb0 msgr2=0x242eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- 172.21.15.46:0/4152753236 shutdown_connections 2023-10-06T11:18:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- 172.21.15.46:0/4152753236 wait complete. 2023-10-06T11:18:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.055+0000 7fbffde95740 1 -- start start 2023-10-06T11:18:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 Processor -- start 2023-10-06T11:18:40.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x24a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x24280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x2379ef0 con 0x21cdae0 2023-10-06T11:18:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x234a010 con 0x21cd710 2023-10-06T11:18:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x2323c90 con 0x2218790 2023-10-06T11:18:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffde95740 10 client.0 _ll_register_callbacks cb 0x2340110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:18:40.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbff9160700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44586/0 (socket says 172.21.15.46:44586) 2023-10-06T11:18:40.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbff9160700 1 -- 172.21.15.46:0/3950090619 learned_addr learned my addr 172.21.15.46:0/3950090619 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbff9961700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x24a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.059+0000 7fbffa162700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x24280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 -- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 msgr2=0x24280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x24280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 -- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 msgr2=0x24a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x24a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 -- 172.21.15.46:0/3950090619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff0019040 con 0x21cdae0 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9160700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4055010 tx=0x7fbfe401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b47466f3ba67ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe404f920 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfec00bc80 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbfe40550b0 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfe404f2d0 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 10 client.4683 fetch_fsmap learned FSMap version 8 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x2329d80 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00bc80 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbfe4011c30 con 0x21cdae0 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 10 client.4683 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T11:18:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T11:18:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 20 client.4683 populate_metadata read hostname 'smithi046' 2023-10-06T11:18:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2428a50 0x2428e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 4 client.4683 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-10-06T11:18:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 --> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2349410 con 0x2428a50 2023-10-06T11:18:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9961700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2428a50 0x2428e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbff9961700 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2428a50 0x2428e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20dda8b56d680854 server_cookie=dbdc4acdc7f87b8f in_seq=0 out_seq=0 2023-10-06T11:18:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.063+0000 7fbfd3fff700 10 client.4683 ms_handle_connect on v2:172.21.15.132:6834/1894883507 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbfd3fff700 1 -- 172.21.15.46:0/3950090619 <== mds.0 v2:172.21.15.132:6834/1894883507 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x2349410 con 0x2428a50 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbfd3fff700 10 client.4683 handle_command_reply: tid=0 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 client.4683 shutdown 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 msgr2=0x24a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4055010 tx=0x7fbfe401e000 comp rx=0 tx=0).stop 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 shutdown_connections 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x2218790 0x24280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x21cdae0 0x24a4d30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:6834/1894883507,v1:172.21.15.132:6835/1894883507] conn(0x2428a50 0x2428e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 --2- 172.21.15.46:0/3950090619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x21cd710 0x24a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbfd3fff700 0 client.4683 ms_handle_reset on v2:172.21.15.132:6834/1894883507 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 >> 172.21.15.46:0/3950090619 conn(0x21cdeb0 msgr2=0x2406540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 shutdown_connections 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 1 -- 172.21.15.46:0/3950090619 wait complete. 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffde95740 20 client.4683 trim_cache size 0 max 16384 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:18:40.075 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4306, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 16.008531875999999, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 8, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 23, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 23, 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 22.229572710999999 2023-10-06T11:18:40.076 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:18:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 msgr2=0x7fbfd8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 0x7fbfd8062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbff0007fd0 tx=0x7fbff0007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 msgr2=0x7fbff40c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c4e0 tx=0x7fbfe400c7f0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 shutdown_connections 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfd8060550 0x7fbfd8062a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff4161840 0x7fbff40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbff4161470 0x7fbff40c4df0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 --2- 172.21.15.46:0/3593042242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff406cc30 0x7fbff414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 >> 172.21.15.46:0/3593042242 conn(0x7fbff40cbe30 msgr2=0x7fbff40ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 shutdown_connections 2023-10-06T11:18:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.067+0000 7fbffbbc5700 1 -- 172.21.15.46:0/3593042242 wait complete. 2023-10-06T11:18:40.095 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T11:18:40.117 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T11:18:40.118 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T11:18:40.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.118 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:18:40.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-10-06T11:18:40.221 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2023-10-06T11:18:40.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:18:40.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.224 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:18:40.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-10-06T11:18:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2023-10-06T11:18:40.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:18:40.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.379 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T11:18:40.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.399 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-10-06T11:18:40.486 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2023-10-06T11:18:40.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:18:40.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.490 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T11:18:40.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:40.551 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-10-06T11:18:40.634 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2023-10-06T11:18:40.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:18:40.637 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-06T11:18:40.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:18:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 Processor -- start 2023-10-06T11:18:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 -- start start 2023-10-06T11:18:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161840 0x7fbd440caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd440caf50 0x7fbd4406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440c18f0 con 0x7fbd44161470 2023-10-06T11:18:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd440c0ce0 con 0x7fbd440caf50 2023-10-06T11:18:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.799+0000 7fbd492a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440c0fc0 con 0x7fbd44161840 2023-10-06T11:18:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd440caf50 0x7fbd4406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd42ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd44161470 0x7fbd440c1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43332/0 (socket says 172.21.15.46:43332) 2023-10-06T11:18:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd42ffd700 1 -- 172.21.15.46:0/478096346 learned_addr learned my addr 172.21.15.46:0/478096346 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd3bfff700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161840 0x7fbd440caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 -- 172.21.15.46:0/478096346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161840 msgr2=0x7fbd440caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161840 0x7fbd440caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 -- 172.21.15.46:0/478096346 >> v1:172.21.15.46:6789/0 conn(0x7fbd44161470 legacy=0x7fbd440c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 -- 172.21.15.46:0/478096346 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440c12a0 con 0x7fbd440caf50 2023-10-06T11:18:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd427fc700 1 -- 172.21.15.46:0/478096346 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988949478 0 0) 0x7fbd440c18f0 con 0x7fbd44161470 2023-10-06T11:18:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd437fe700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd440caf50 0x7fbd4406cc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009e60 tx=0x7fbd3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168adf691477a820 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd427fc700 1 -- 172.21.15.46:0/478096346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c00e070 con 0x7fbd440caf50 2023-10-06T11:18:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd427fc700 1 -- 172.21.15.46:0/478096346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c00bbf0 con 0x7fbd440caf50 2023-10-06T11:18:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd427fc700 1 -- 172.21.15.46:0/478096346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c012430 con 0x7fbd440caf50 2023-10-06T11:18:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 -- 172.21.15.46:0/478096346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd440caf50 msgr2=0x7fbd4406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd440caf50 0x7fbd4406cc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009e60 tx=0x7fbd3c009780 comp rx=0 tx=0).stop 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 -- 172.21.15.46:0/478096346 shutdown_connections 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd440caf50 0x7fbd4406cc30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 --2- 172.21.15.46:0/478096346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161840 0x7fbd440caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 -- 172.21.15.46:0/478096346 >> 172.21.15.46:0/478096346 conn(0x7fbd440cbe30 msgr2=0x7fbd440cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 -- 172.21.15.46:0/478096346 shutdown_connections 2023-10-06T11:18:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.803+0000 7fbd492a5700 1 -- 172.21.15.46:0/478096346 wait complete. 2023-10-06T11:18:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 Processor -- start 2023-10-06T11:18:40.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 -- start start 2023-10-06T11:18:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 0x7fbd440aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161470 0x7fbd4414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4415e9b0 con 0x7fbd44161840 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4416d730 con 0x7fbd440caf50 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd492a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4416da10 con 0x7fbd44161470 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44602/0 (socket says 172.21.15.46:44602) 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.807+0000 7fbd437fe700 1 -- 172.21.15.46:0/7612538 learned_addr learned my addr 172.21.15.46:0/7612538 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd3bfff700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161470 0x7fbd4414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd42ffd700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 0x7fbd440aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 -- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161470 msgr2=0x7fbd4414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161470 0x7fbd4414f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 -- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 msgr2=0x7fbd440aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 0x7fbd440aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 -- 172.21.15.46:0/7612538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c009a20 con 0x7fbd44161840 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd42ffd700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 0x7fbd440aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd437fe700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b6c0 tx=0x7fbd3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c012e40 con 0x7fbd44161840 2023-10-06T11:18:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c01d630 con 0x7fbd44161840 2023-10-06T11:18:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c01d9f0 con 0x7fbd44161840 2023-10-06T11:18:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4416dcf0 con 0x7fbd44161840 2023-10-06T11:18:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4414f5a0 con 0x7fbd44161840 2023-10-06T11:18:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.811+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4416e1f0 con 0x7fbd44161840 2023-10-06T11:18:40.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.815+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbd3c024070 con 0x7fbd44161840 2023-10-06T11:18:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.815+0000 7fbd3affd700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 0x7fbd24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.815+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbd3c07f1b0 con 0x7fbd44161840 2023-10-06T11:18:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.815+0000 7fbd42ffd700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 0x7fbd24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.819+0000 7fbd42ffd700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 0x7fbd24062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd34004c50 tx=0x7fbd34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:40.823+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbd3c04f5a0 con 0x7fbd44161840 2023-10-06T11:18:41.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.035+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd44063f90 con 0x7fbd44161840 2023-10-06T11:18:41.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.039+0000 7fbd3affd700 1 -- 172.21.15.46:0/7612538 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd3c052bc0 con 0x7fbd44161840 2023-10-06T11:18:41.042 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:18:41.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 msgr2=0x7fbd24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 0x7fbd24062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd34004c50 tx=0x7fbd34009040 comp rx=0 tx=0).stop 2023-10-06T11:18:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 msgr2=0x7fbd440ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b6c0 tx=0x7fbd3c019000 comp rx=0 tx=0).stop 2023-10-06T11:18:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 shutdown_connections 2023-10-06T11:18:41.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbd24060550 0x7fbd24062a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd44161840 0x7fbd440ca250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44161470 0x7fbd4414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 --2- 172.21.15.46:0/7612538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440caf50 0x7fbd440aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 >> 172.21.15.46:0/7612538 conn(0x7fbd440cbe30 msgr2=0x7fbd4406a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 shutdown_connections 2023-10-06T11:18:41.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:41.043+0000 7fbd492a5700 1 -- 172.21.15.46:0/7612538 wait complete. 2023-10-06T11:18:41.067 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:18:41.067 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:18:41.067 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:18:41.068 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:18:41.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.068 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:18:41.090 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:41.090 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:41.090 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:41.091 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:41.091 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:41.091 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:41.092 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:41.092 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:18:41.092 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:41.093 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:41.093 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:18:41.093 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:18:41.093 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:41.094 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:18:41.094 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:18:41.094 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42409sec preferred_lft 42409sec 2023-10-06T11:18:41.095 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:18:41.095 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:41.095 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:41.096 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:41.096 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:18:41.096 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:18:41.096 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:41.097 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:18:41.097 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:41.097 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:18:41.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.098 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:41.098 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:41.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:18:41.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2023-10-06T11:18:41.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2023-10-06T11:18:41.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:18:41.099 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:41.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.249 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:18:41.286 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:18:41.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.289 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:18:41.385 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:18:41.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:18:41.386 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:41.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.523 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:18:41.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.540 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2023-10-06T11:18:41.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.605 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:41.605 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:41.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:18:41.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T11:18:41.605 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:41.723 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T11:18:41.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T11:18:41.724 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:41.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:41.902 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:41.902 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:41.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:18:41.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:18:41.902 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:41.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:41.978 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:18:41.996 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:18:41.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:41.996 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:18:42.054 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:18:42.054 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:18:42.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:18:42.183 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:18:42.183 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:18:42.183 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:42.319 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:18:42.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:18:42.319 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:42.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:42.319 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:18:42.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:18:42.345 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:18:42.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:42.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:18:42.381 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-06T11:18:42.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:18:42.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 Processor -- start 2023-10-06T11:18:42.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 -- start start 2023-10-06T11:18:42.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f206015c770 0x7f20600cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 0x7f20600ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600cc150 con 0x7f20600cea30 2023-10-06T11:18:42.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600cfbb0 con 0x7f206015c770 2023-10-06T11:18:42.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.599+0000 7f2067bc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600cdbf0 con 0x7f20600ccb50 2023-10-06T11:18:42.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2066162700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f20600cea30 0x7f20600c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43348/0 (socket says 172.21.15.46:43348) 2023-10-06T11:18:42.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2066162700 1 -- 172.21.15.46:0/3316276901 learned_addr learned my addr 172.21.15.46:0/3316276901 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:42.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 --2- 172.21.15.46:0/3316276901 >> v2:172.21.15.46:3300/0 conn(0x7f206015c770 0x7f20600cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065160700 1 --2- 172.21.15.46:0/3316276901 >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 0x7f20600ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990780052 0 0) 0x7f20600cc150 con 0x7f20600cea30 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204c003570 con 0x7f20600cea30 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 -- 172.21.15.46:0/3316276901 >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 msgr2=0x7f20600ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 --2- 172.21.15.46:0/3316276901 >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 0x7f20600ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350262429 0 0) 0x7f204c003570 con 0x7f20600cea30 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 -- 172.21.15.46:0/3316276901 >> v1:172.21.15.46:6789/0 conn(0x7f20600cea30 legacy=0x7f20600c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 -- 172.21.15.46:0/3316276901 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600cee50 con 0x7f206015c770 2023-10-06T11:18:42.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065160700 1 --2- 172.21.15.46:0/3316276901 >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 0x7f20600ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:42.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f2065961700 1 --2- 172.21.15.46:0/3316276901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600cad40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20600ca810 tx=0x7f205c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9361c70a2320099c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:42.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580017f0 con 0x7f206015c770 2023-10-06T11:18:42.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c011420 con 0x7f206015c770 2023-10-06T11:18:42.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.603+0000 7f206495f700 1 -- 172.21.15.46:0/3316276901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c0117c0 con 0x7f206015c770 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- 172.21.15.46:0/3316276901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 msgr2=0x7f20600cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 --2- 172.21.15.46:0/3316276901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600cad40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20600ca810 tx=0x7f205c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- 172.21.15.46:0/3316276901 shutdown_connections 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 --2- 172.21.15.46:0/3316276901 >> v2:172.21.15.132:3300/0 conn(0x7f20600ccb50 0x7f20600ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 --2- 172.21.15.46:0/3316276901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600cad40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- 172.21.15.46:0/3316276901 >> 172.21.15.46:0/3316276901 conn(0x7f2060099480 msgr2=0x7f206006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- 172.21.15.46:0/3316276901 shutdown_connections 2023-10-06T11:18:42.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- 172.21.15.46:0/3316276901 wait complete. 2023-10-06T11:18:42.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 Processor -- start 2023-10-06T11:18:42.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.607+0000 7f2067bc5700 1 -- start start 2023-10-06T11:18:42.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20600ccb50 0x7f2060158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20600cea30 0x7f20600c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206009b3a0 con 0x7f206015c770 2023-10-06T11:18:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060159150 con 0x7f20600cea30 2023-10-06T11:18:42.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2067bc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c81d0 con 0x7f20600ccb50 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44626/0 (socket says 172.21.15.46:44626) 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 -- 172.21.15.46:0/431187761 learned_addr learned my addr 172.21.15.46:0/431187761 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2065160700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20600cea30 0x7f20600c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2065961700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20600ccb50 0x7f2060158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 -- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20600ccb50 msgr2=0x7f2060158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20600ccb50 0x7f2060158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 -- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20600cea30 msgr2=0x7f20600c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20600cea30 0x7f20600c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 -- 172.21.15.46:0/431187761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050009420 con 0x7f206015c770 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.611+0000 7f2066162700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2058001560 tx=0x7f2058001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058004dc0 con 0x7f206015c770 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800a730 con 0x7f206015c770 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c0093c0 con 0x7f206015c770 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800aaf0 con 0x7f206015c770 2023-10-06T11:18:42.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20600c8790 con 0x7f206015c770 2023-10-06T11:18:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.615+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206009b170 con 0x7f206015c770 2023-10-06T11:18:42.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.619+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2058007150 con 0x7f206015c770 2023-10-06T11:18:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.619+0000 7f2055ffb700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 0x7f2044062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.619+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f205807c370 con 0x7f206015c770 2023-10-06T11:18:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.619+0000 7f2065961700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 0x7f2044062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.623+0000 7f2065961700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 0x7f2044062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f205c0030c0 tx=0x7f205c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.627+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f206009b170 con 0x7f206015c770 2023-10-06T11:18:42.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.831+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f206009b170 con 0x7f206015c770 2023-10-06T11:18:42.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2055ffb700 1 -- 172.21.15.46:0/431187761 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f206009b170 con 0x7f206015c770 2023-10-06T11:18:42.838 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:18:42.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 msgr2=0x7f2044062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 0x7f2044062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f205c0030c0 tx=0x7f205c016000 comp rx=0 tx=0).stop 2023-10-06T11:18:42.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 msgr2=0x7f20600c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:42.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2058001560 tx=0x7f2058001640 comp rx=0 tx=0).stop 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 shutdown_connections 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2044060550 0x7f2044062a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206015c770 0x7f20600c5520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20600cea30 0x7f20600c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 --2- 172.21.15.46:0/431187761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20600ccb50 0x7f2060158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 >> 172.21.15.46:0/431187761 conn(0x7f2060099480 msgr2=0x7f2060068af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 shutdown_connections 2023-10-06T11:18:42.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:42.835+0000 7f2067bc5700 1 -- 172.21.15.46:0/431187761 wait complete. 2023-10-06T11:18:42.863 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:18:42.864 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:18:42.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:42.865 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:18:42.883 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:42.883 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:42.883 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:42.883 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.883 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:18:42.884 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42407sec preferred_lft 42407sec 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:18:42.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link tentative 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link tentative 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:42.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:42.886 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:18:42.963 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:18:42.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:42.966 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:18:43.033 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:18:43.033 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:18:43.034 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:18:43.034 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:18:43.034 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:18:43.034 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:18:43.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:18:43.036 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:43.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.177 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:18:43.198 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:18:43.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.198 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2023-10-06T11:18:43.260 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T11:18:43.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.261 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:43.262 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:43.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:18:43.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T11:18:43.262 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:43.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:18:43.422 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.423 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:18:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether d6:fd:d9:a7:ad:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:18:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:18:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::d4fd:d9ff:fea7:ad08/64 scope link tentative 2023-10-06T11:18:43.424 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:18:43.510 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether d6:fd:d9:a7:ad:08 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:18:43.511 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:18:43.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.512 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::d4fd:d9ff:fea7:ad08/64 scope link tentative 2023-10-06T11:18:43.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:43.514 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T11:18:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T11:18:43.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T11:18:43.516 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:43.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:43.732 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:18:43.732 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:18:43.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2023-10-06T11:18:43.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:18:43.732 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:18:43.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:43.808 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T11:18:43.824 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:18:43.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:43.824 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:18:43.882 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:18:43.882 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:18:43.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T11:18:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T11:18:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T11:18:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T11:18:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T11:18:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:18:44.046 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:18:44.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:18:44.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:18:44.083 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-06T11:18:44.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:18:44.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.283+0000 7f0476985700 1 Processor -- start 2023-10-06T11:18:44.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.283+0000 7f0476985700 1 -- start start 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 0x7f04700ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04700cea30 0x7f04700c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700cfbb0 con 0x7f04700cea30 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700cdbf0 con 0x7f047015c770 2023-10-06T11:18:44.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0476985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04700cc150 con 0x7f04700ccb50 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0474f22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04700cea30 0x7f04700c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f046ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f04677fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 0x7f04700ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f046ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:35360/0 (socket says 172.21.15.46:35360) 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f046ffff700 1 -- 172.21.15.46:0/3988670411 learned_addr learned my addr 172.21.15.46:0/3988670411 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:44.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.287+0000 7f0474f22700 1 --2- 172.21.15.46:0/3988670411 >> v2:172.21.15.46:3300/0 conn(0x7f04700cea30 0x7f04700c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44634/0 (socket says 172.21.15.46:44634) 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 -- 172.21.15.46:0/3988670411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 msgr2=0x7f04700ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 0x7f04700ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 -- 172.21.15.46:0/3988670411 >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 msgr2=0x7f04700cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 --2- 172.21.15.46:0/3988670411 >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 -- 172.21.15.46:0/3988670411 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700cee70 con 0x7f04700cea30 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f04677fe700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 0x7f04700ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f046ffff700 1 --2- 172.21.15.46:0/3988670411 >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0474f22700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700cea30 0x7f04700c4fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04700cf000 tx=0x7f04680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3082662abffa3e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:44.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f046f7fe700 1 -- 172.21.15.46:0/3988670411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468011630 con 0x7f04700cea30 2023-10-06T11:18:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f046f7fe700 1 -- 172.21.15.46:0/3988670411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0468011790 con 0x7f04700cea30 2023-10-06T11:18:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f046f7fe700 1 -- 172.21.15.46:0/3988670411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468011b50 con 0x7f04700cea30 2023-10-06T11:18:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 -- 172.21.15.46:0/3988670411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700cea30 msgr2=0x7f04700c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700cea30 0x7f04700c4fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f04700cf000 tx=0x7f04680091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 -- 172.21.15.46:0/3988670411 shutdown_connections 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700cea30 0x7f04700c4fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 --2- 172.21.15.46:0/3988670411 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700ccb50 0x7f04700ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 --2- 172.21.15.46:0/3988670411 >> v2:172.21.15.132:3300/0 conn(0x7f047015c770 0x7f04700cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 -- 172.21.15.46:0/3988670411 >> 172.21.15.46:0/3988670411 conn(0x7f0470099480 msgr2=0x7f047006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 -- 172.21.15.46:0/3988670411 shutdown_connections 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.291+0000 7f0476985700 1 -- 172.21.15.46:0/3988670411 wait complete. 2023-10-06T11:18:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 Processor -- start 2023-10-06T11:18:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- start start 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700cea30 0x7f04700c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f047015c770 0x7f047014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04700ca600 con 0x7f04700ccb50 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470169390 con 0x7f047015c770 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047015abc0 con 0x7f04700cea30 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44648/0 (socket says 172.21.15.46:44648) 2023-10-06T11:18:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 -- 172.21.15.46:0/3649370958 learned_addr learned my addr 172.21.15.46:0/3649370958 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0474f22700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f047015c770 0x7f047014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f04677fe700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700cea30 0x7f04700c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 -- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700cea30 msgr2=0x7f04700c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700cea30 0x7f04700c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 -- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f047015c770 msgr2=0x7f047014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f047015c770 0x7f047014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 -- 172.21.15.46:0/3649370958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458009420 con 0x7f04700ccb50 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046ffff700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f046000c0f0 tx=0x7f046000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046000cb80 con 0x7f04700ccb50 2023-10-06T11:18:44.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046000f040 con 0x7f04700ccb50 2023-10-06T11:18:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460014480 con 0x7f04700ccb50 2023-10-06T11:18:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460009ae0 con 0x7f04700ccb50 2023-10-06T11:18:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.295+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468009420 con 0x7f04700ccb50 2023-10-06T11:18:44.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.299+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04700660f0 con 0x7f04700ccb50 2023-10-06T11:18:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.299+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0460014830 con 0x7f04700ccb50 2023-10-06T11:18:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.303+0000 7f046cff9700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 0x7f0450062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.303+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f046007d2e0 con 0x7f04700ccb50 2023-10-06T11:18:44.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.303+0000 7f04677fe700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 0x7f0450062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:44.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.303+0000 7f04677fe700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 0x7f0450062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04580093f0 tx=0x7f0458008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:44.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.307+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f046004d6d0 con 0x7f04700ccb50 2023-10-06T11:18:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.515+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04700cb2b0 con 0x7f04700ccb50 2023-10-06T11:18:44.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.515+0000 7f046cff9700 1 -- 172.21.15.46:0/3649370958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0460050cf0 con 0x7f04700ccb50 2023-10-06T11:18:44.519 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 msgr2=0x7f0450062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 0x7f0450062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04580093f0 tx=0x7f0458008040 comp rx=0 tx=0).stop 2023-10-06T11:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 msgr2=0x7f0470157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f046000c0f0 tx=0x7f046000b040 comp rx=0 tx=0).stop 2023-10-06T11:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 shutdown_connections 2023-10-06T11:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0450060550 0x7f0450062a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f047015c770 0x7f047014ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04700cea30 0x7f04700c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 --2- 172.21.15.46:0/3649370958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700ccb50 0x7f0470157e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 >> 172.21.15.46:0/3649370958 conn(0x7f0470099480 msgr2=0x7f04700cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 shutdown_connections 2023-10-06T11:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:44.519+0000 7f0476985700 1 -- 172.21.15.46:0/3649370958 wait complete. 2023-10-06T11:18:44.544 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:18:44.544 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:18:44.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2023-10-06T11:18:44.544 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T11:18:44.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T11:18:44.545 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:18:44.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:18:44.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:18:44.545 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:18:44.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.545 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2023-10-06T11:18:44.563 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:44.563 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:44.563 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:44.563 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:44.563 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:44.564 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 02:42:75:5b:f2:6f brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:18:44.565 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:44.566 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:18:44.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.566 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:44.566 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:44.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:18:44.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2023-10-06T11:18:44.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2023-10-06T11:18:44.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:18:44.567 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:44.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.719 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:18:44.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:18:44.748 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:18:44.754 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-10-06T11:18:44.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.761 DEBUG:teuthology.orchestra.run.smithi132:> route 2023-10-06T11:18:44.837 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2023-10-06T11:18:44.838 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:18:44.838 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-06T11:18:44.838 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T11:18:44.839 INFO:teuthology.orchestra.run.smithi132.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:18:44.839 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-06T11:18:44.839 INFO:teuthology.orchestra.run.smithi132.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-06T11:18:44.839 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:18:44.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.840 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:44.840 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:44.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T11:18:44.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T11:18:44.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T11:18:44.841 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:44.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.971 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T11:18:44.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:44.988 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2023-10-06T11:18:45.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:45.049 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:45.049 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:45.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:18:45.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T11:18:45.049 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:45.171 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T11:18:45.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:45.171 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:45.171 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:45.171 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T11:18:45.171 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:18:45.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T11:18:45.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T11:18:45.172 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T11:18:45.172 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:45.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:45.347 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:45.347 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:45.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2023-10-06T11:18:45.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:18:45.348 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:45.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:45.420 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T11:18:45.434 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:18:45.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:45.435 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:18:45.491 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:18:45.491 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:18:45.492 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T11:18:45.612 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T11:18:45.612 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T11:18:45.612 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:45.747 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T11:18:45.747 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T11:18:45.747 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:45.747 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:45.748 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:45.765 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:18:45.766 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:18:45.770 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:18:45.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:45.770 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:18:45.801 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-06T11:18:45.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:18:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 Processor -- start 2023-10-06T11:18:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 -- start start 2023-10-06T11:18:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f553015ab60 0x7f553006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f553015af30 0x7f553006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553006d460 con 0x7f55300ad6c0 2023-10-06T11:18:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55300c7000 con 0x7f553015af30 2023-10-06T11:18:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5535078700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55300c72c0 con 0x7f553015ab60 2023-10-06T11:18:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5527fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f553015af30 0x7f553006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5527fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f553015af30 0x7f553006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44658/0 (socket says 172.21.15.46:44658) 2023-10-06T11:18:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f5527fff700 1 -- 172.21.15.46:0/3442220663 learned_addr learned my addr 172.21.15.46:0/3442220663 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.971+0000 7f552ed9d700 1 --2- 172.21.15.46:0/3442220663 >> v2:172.21.15.132:3300/0 conn(0x7f553015ab60 0x7f553006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f552e59c700 1 -- 172.21.15.46:0/3442220663 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578149007 0 0) 0x7f553006d460 con 0x7f55300ad6c0 2023-10-06T11:18:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5527fff700 1 -- 172.21.15.46:0/3442220663 >> v2:172.21.15.132:3300/0 conn(0x7f553015ab60 msgr2=0x7f553006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5527fff700 1 --2- 172.21.15.46:0/3442220663 >> v2:172.21.15.132:3300/0 conn(0x7f553015ab60 0x7f553006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:45.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5527fff700 1 -- 172.21.15.46:0/3442220663 >> v1:172.21.15.132:6789/0 conn(0x7f55300ad6c0 legacy=0x7f55300adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5527fff700 1 -- 172.21.15.46:0/3442220663 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55300c75a0 con 0x7f553015af30 2023-10-06T11:18:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5527fff700 1 --2- 172.21.15.46:0/3442220663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f553006cc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5530063be0 tx=0x7f5518009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a783cce2dfc54a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f552e59c700 1 -- 172.21.15.46:0/3442220663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518009e70 con 0x7f553015af30 2023-10-06T11:18:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f552e59c700 1 -- 172.21.15.46:0/3442220663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551800b860 con 0x7f553015af30 2023-10-06T11:18:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f552e59c700 1 -- 172.21.15.46:0/3442220663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800bc00 con 0x7f553015af30 2023-10-06T11:18:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 -- 172.21.15.46:0/3442220663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 msgr2=0x7f553006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 --2- 172.21.15.46:0/3442220663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f553006cc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5530063be0 tx=0x7f5518009250 comp rx=0 tx=0).stop 2023-10-06T11:18:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 -- 172.21.15.46:0/3442220663 shutdown_connections 2023-10-06T11:18:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 --2- 172.21.15.46:0/3442220663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f553006cc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 --2- 172.21.15.46:0/3442220663 >> v2:172.21.15.132:3300/0 conn(0x7f553015ab60 0x7f553006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 -- 172.21.15.46:0/3442220663 >> 172.21.15.46:0/3442220663 conn(0x7f55300cbe30 msgr2=0x7f55300cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 -- 172.21.15.46:0/3442220663 shutdown_connections 2023-10-06T11:18:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.975+0000 7f5535078700 1 -- 172.21.15.46:0/3442220663 wait complete. 2023-10-06T11:18:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 Processor -- start 2023-10-06T11:18:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 -- start start 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55300ad6c0 0x7f5530150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 0x7f55300af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530150fe0 con 0x7f553015af30 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300c97c0 con 0x7f553015ab60 2023-10-06T11:18:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f5535078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55300b0e30 con 0x7f55300ad6c0 2023-10-06T11:18:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.979+0000 7f552f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44666/0 (socket says 172.21.15.46:44666) 2023-10-06T11:18:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 -- 172.21.15.46:0/1189794813 learned_addr learned my addr 172.21.15.46:0/1189794813 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552ed9d700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55300ad6c0 0x7f5530150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5527fff700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 0x7f55300af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 -- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55300ad6c0 msgr2=0x7f5530150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55300ad6c0 0x7f5530150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 -- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 msgr2=0x7f55300af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 0x7f55300af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 -- 172.21.15.46:0/1189794813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55180094f0 con 0x7f553015af30 2023-10-06T11:18:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5527fff700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 0x7f55300af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f552f59e700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f552800c890 tx=0x7f5528001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528017070 con 0x7f553015af30 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5528006ba0 con 0x7f553015af30 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528012630 con 0x7f553015af30 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55300c58b0 con 0x7f553015af30 2023-10-06T11:18:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.983+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55300aed20 con 0x7f553015af30 2023-10-06T11:18:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.987+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55300c9950 con 0x7f553015af30 2023-10-06T11:18:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.987+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5528012a60 con 0x7f553015af30 2023-10-06T11:18:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.987+0000 7f5526ffd700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 0x7f5510062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.987+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f552807c630 con 0x7f553015af30 2023-10-06T11:18:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.987+0000 7f552ed9d700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 0x7f5510062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.991+0000 7f552ed9d700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 0x7f5510062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f553016c550 tx=0x7f5520009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:45.991+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f552804cac0 con 0x7f553015af30 2023-10-06T11:18:46.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5530063f90 con 0x7f553015af30 2023-10-06T11:18:46.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5526ffd700 1 -- 172.21.15.46:0/1189794813 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f55280500e0 con 0x7f553015af30 2023-10-06T11:18:46.203 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 msgr2=0x7f5510062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 0x7f5510062a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f553016c550 tx=0x7f5520009120 comp rx=0 tx=0).stop 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 msgr2=0x7f55300af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.199+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f552800c890 tx=0x7f5528001670 comp rx=0 tx=0).stop 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 shutdown_connections 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5510060550 0x7f5510062a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f553015af30 0x7f55300af6d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f553015ab60 0x7f55300af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 --2- 172.21.15.46:0/1189794813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55300ad6c0 0x7f5530150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 >> 172.21.15.46:0/1189794813 conn(0x7f55300cbe30 msgr2=0x7f55300ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 shutdown_connections 2023-10-06T11:18:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:46.203+0000 7f5535078700 1 -- 172.21.15.46:0/1189794813 wait complete. 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:18:46.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:18:46.226 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:18:46.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.226 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:46.243 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:18:46.244 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42375sec preferred_lft 42375sec 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 02:42:75:5b:f2:6f brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether f6:be:49:37:67:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:18:46.245 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::cef:1cff:fe46:24fc/64 scope link tentative 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether f6:be:49:37:67:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::f4be:49ff:fe37:67ed/64 scope link tentative 2023-10-06T11:18:46.246 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.247 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:18:46.326 INFO:teuthology.orchestra.run.smithi132.stdout:1 2023-10-06T11:18:46.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.330 DEBUG:teuthology.orchestra.run.smithi132:> route 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:18:46.395 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-06T11:18:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-06T11:18:46.396 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:18:46.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.396 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:46.396 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:46.396 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T11:18:46.397 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T11:18:46.397 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T11:18:46.397 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:46.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.534 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T11:18:46.548 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:18:46.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.550 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2023-10-06T11:18:46.607 INFO:teuthology.orchestra.run.smithi132.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T11:18:46.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.608 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:46.608 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:46.608 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:18:46.608 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T11:18:46.608 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:46.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.730 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:18:46.765 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:46.765 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether da:11:64:a5:99:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::d811:64ff:fea5:99d1/64 scope link 2023-10-06T11:18:46.766 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.768 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:18:46.853 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:18:46.853 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:18:46.853 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:18:46.853 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.853 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether da:11:64:a5:99:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::d811:64ff:fea5:99d1/64 scope link 2023-10-06T11:18:46.854 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:18:46.856 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T11:18:46.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T11:18:46.857 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:47.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:47.050 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2023-10-06T11:18:47.050 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2023-10-06T11:18:47.050 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.1 up 2023-10-06T11:18:47.051 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:18:47.051 DEBUG:teuthology.orchestra.run.smithi132:> ') 2023-10-06T11:18:47.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:47.117 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T11:18:47.131 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:18:47.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:18:47.132 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:18:47.190 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:18:47.190 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:18:47.191 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T11:18:47.301 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T11:18:47.302 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T11:18:47.302 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:47.326 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T11:18:47.326 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T11:18:47.326 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:47.326 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:47.326 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:18:47.340 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:18:47.346 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:18:47.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:47.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:18:47.375 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T11:18:47.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:47.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.551+0000 7f67e7d85700 1 Processor -- start 2023-10-06T11:18:47.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.551+0000 7f67e7d85700 1 -- start start 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e7d85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67e0161840 0x7f67e00caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e7d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e7d85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00c18f0 con 0x7f67e0161470 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e7d85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00c0ce0 con 0x7f67e0161840 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e7d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00c0fc0 con 0x7f67e00caf50 2023-10-06T11:18:47.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e5320700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67e0161840 0x7f67e00caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e5b21700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f67e0161470 0x7f67e00c1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:41830/0 (socket says 172.21.15.46:41830) 2023-10-06T11:18:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e5b21700 1 -- 172.21.15.46:0/1437011434 learned_addr learned my addr 172.21.15.46:0/1437011434 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.555+0000 7f67e6322700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e5320700 1 -- 172.21.15.46:0/1437011434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 msgr2=0x7f67e006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e5320700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e5320700 1 -- 172.21.15.46:0/1437011434 >> v1:172.21.15.132:6789/0 conn(0x7f67e0161470 legacy=0x7f67e00c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e5320700 1 -- 172.21.15.46:0/1437011434 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00c12a0 con 0x7f67e0161840 2023-10-06T11:18:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e4b1f700 1 -- 172.21.15.46:0/1437011434 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775893994 0 0) 0x7f67e00c18f0 con 0x7f67e0161470 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e5320700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161840 0x7f67e00caa10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae14ac2cea0f8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e4b1f700 1 -- 172.21.15.46:0/1437011434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000b6d0 con 0x7f67e0161840 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e4b1f700 1 -- 172.21.15.46:0/1437011434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d000b830 con 0x7f67e0161840 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e4b1f700 1 -- 172.21.15.46:0/1437011434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000bbd0 con 0x7f67e0161840 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 -- 172.21.15.46:0/1437011434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161840 msgr2=0x7f67e00caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161840 0x7f67e00caa10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098e0 tx=0x7f67d00091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 -- 172.21.15.46:0/1437011434 shutdown_connections 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 --2- 172.21.15.46:0/1437011434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161840 0x7f67e00caa10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 -- 172.21.15.46:0/1437011434 >> 172.21.15.46:0/1437011434 conn(0x7f67e00cbe30 msgr2=0x7f67e00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 -- 172.21.15.46:0/1437011434 shutdown_connections 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.559+0000 7f67e7d85700 1 -- 172.21.15.46:0/1437011434 wait complete. 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 Processor -- start 2023-10-06T11:18:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 -- start start 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e0161840 0x7f67e00c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00af370 con 0x7f67e0161470 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e015ce70 con 0x7f67e0161840 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e7d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e015d110 con 0x7f67e00caf50 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44682/0 (socket says 172.21.15.46:44682) 2023-10-06T11:18:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 -- 172.21.15.46:0/2927317462 learned_addr learned my addr 172.21.15.46:0/2927317462 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5b21700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e6322700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e0161840 0x7f67e00c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 -- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 msgr2=0x7f67e014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 -- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e0161840 msgr2=0x7f67e00c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e0161840 0x7f67e00c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.563+0000 7f67e5320700 1 -- 172.21.15.46:0/2927317462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0009420 con 0x7f67e0161470 2023-10-06T11:18:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67e5320700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f67d00093f0 tx=0x7f67d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000bbb0 con 0x7f67e0161470 2023-10-06T11:18:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0011420 con 0x7f67e0161470 2023-10-06T11:18:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00117c0 con 0x7f67e0161470 2023-10-06T11:18:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e015d3f0 con 0x7f67e0161470 2023-10-06T11:18:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e015d9b0 con 0x7f67e0161470 2023-10-06T11:18:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.567+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e015dbd0 con 0x7f67e0161470 2023-10-06T11:18:47.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.571+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f67d0011bf0 con 0x7f67e0161470 2023-10-06T11:18:47.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.571+0000 7f67d5ffb700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 0x7f67c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.571+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f67d007e2b0 con 0x7f67e0161470 2023-10-06T11:18:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.571+0000 7f67e5b21700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 0x7f67c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.571+0000 7f67e5b21700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 0x7f67c4062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67dc001230 tx=0x7f67dc007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.579+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f67d004e6a0 con 0x7f67e0161470 2023-10-06T11:18:47.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.755+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e0063f90 con 0x7f67e0161470 2023-10-06T11:18:47.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.755+0000 7f67d5ffb700 1 -- 172.21.15.46:0/2927317462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f67d0051cc0 con 0x7f67e0161470 2023-10-06T11:18:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:47.761 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:47.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 msgr2=0x7f67c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 0x7f67c4062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67dc001230 tx=0x7f67dc007d20 comp rx=0 tx=0).stop 2023-10-06T11:18:47.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 msgr2=0x7f67e006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f67d00093f0 tx=0x7f67d0017000 comp rx=0 tx=0).stop 2023-10-06T11:18:47.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 shutdown_connections 2023-10-06T11:18:47.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f67c4060550 0x7f67c4062a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e0161840 0x7f67e00c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e0161470 0x7f67e006e080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 --2- 172.21.15.46:0/2927317462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67e00caf50 0x7f67e014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 >> 172.21.15.46:0/2927317462 conn(0x7f67e00cbe30 msgr2=0x7f67e00c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 shutdown_connections 2023-10-06T11:18:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.759+0000 7f67e7d85700 1 -- 172.21.15.46:0/2927317462 wait complete. 2023-10-06T11:18:47.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:47.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.947+0000 7fb1e502e700 1 Processor -- start 2023-10-06T11:18:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.947+0000 7fb1e502e700 1 -- start start 2023-10-06T11:18:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1e502e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb1e00b23c0 0x7fb1e00b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1e502e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1e502e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e00cc510 con 0x7fb1e00cae40 2023-10-06T11:18:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1e502e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e00cb900 con 0x7fb1e00b23c0 2023-10-06T11:18:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1e502e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00cbbe0 con 0x7fb1e00cd250 2023-10-06T11:18:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55092/0 (socket says 172.21.15.46:55092) 2023-10-06T11:18:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 -- 172.21.15.46:0/3743300671 learned_addr learned my addr 172.21.15.46:0/3743300671 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1de59c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb1e00cae40 0x7fb1e00ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:41840/0 (socket says 172.21.15.46:41840) 2023-10-06T11:18:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1ded9d700 1 --2- 172.21.15.46:0/3743300671 >> v2:172.21.15.132:3300/0 conn(0x7fb1e00b23c0 0x7fb1e00b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 -- 172.21.15.46:0/3743300671 >> v2:172.21.15.132:3300/0 conn(0x7fb1e00b23c0 msgr2=0x7fb1e00b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 --2- 172.21.15.46:0/3743300671 >> v2:172.21.15.132:3300/0 conn(0x7fb1e00b23c0 0x7fb1e00b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 -- 172.21.15.46:0/3743300671 >> v1:172.21.15.132:6789/0 conn(0x7fb1e00cae40 legacy=0x7fb1e00ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 -- 172.21.15.46:0/3743300671 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00cbec0 con 0x7fb1e00cd250 2023-10-06T11:18:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.951+0000 7fb1df59e700 1 --2- 172.21.15.46:0/3743300671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009f20 tx=0x7fb1d8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2e912497bfbb96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1ddd9b700 1 -- 172.21.15.46:0/3743300671 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800e070 con 0x7fb1e00cd250 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1ddd9b700 1 -- 172.21.15.46:0/3743300671 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d800b9d0 con 0x7fb1e00cd250 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1ddd9b700 1 -- 172.21.15.46:0/3743300671 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80092e0 con 0x7fb1e00cd250 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 -- 172.21.15.46:0/3743300671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 msgr2=0x7fb1e00cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3743300671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009f20 tx=0x7fb1d8009840 comp rx=0 tx=0).stop 2023-10-06T11:18:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 -- 172.21.15.46:0/3743300671 shutdown_connections 2023-10-06T11:18:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3743300671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00cd250 0x7fb1e00cb290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3743300671 >> v2:172.21.15.132:3300/0 conn(0x7fb1e00b23c0 0x7fb1e00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 -- 172.21.15.46:0/3743300671 >> 172.21.15.46:0/3743300671 conn(0x7fb1e0099480 msgr2=0x7fb1e006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 -- 172.21.15.46:0/3743300671 shutdown_connections 2023-10-06T11:18:47.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.955+0000 7fb1e502e700 1 -- 172.21.15.46:0/3743300671 wait complete. 2023-10-06T11:18:47.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 Processor -- start 2023-10-06T11:18:47.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 -- start start 2023-10-06T11:18:47.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00b23c0 0x7fb1e00c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 0x7fb1e016da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c7660 con 0x7fb1e00cd250 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c7940 con 0x7fb1e00cae40 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1e502e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c7c20 con 0x7fb1e00b23c0 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1df59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44688/0 (socket says 172.21.15.46:44688) 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1df59e700 1 -- 172.21.15.46:0/3636737005 learned_addr learned my addr 172.21.15.46:0/3636737005 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1de59c700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 0x7fb1e016da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.959+0000 7fb1ded9d700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00b23c0 0x7fb1e00c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 -- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00b23c0 msgr2=0x7fb1e00c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00b23c0 0x7fb1e00c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 -- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 msgr2=0x7fb1e00b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 -- 172.21.15.46:0/3636737005 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8009ae0 con 0x7fb1e00cae40 2023-10-06T11:18:47.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1df59e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1de59c700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 0x7fb1e016da30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c7c0 tx=0x7fb1d4007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4005a70 con 0x7fb1e00cae40 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d4005bd0 con 0x7fb1e00cae40 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40070d0 con 0x7fb1e00cae40 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e00c7f00 con 0x7fb1e00cae40 2023-10-06T11:18:47.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e00c72f0 con 0x7fb1e00cae40 2023-10-06T11:18:47.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.963+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e00660f0 con 0x7fb1e00cae40 2023-10-06T11:18:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.967+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb1d4007480 con 0x7fb1e00cae40 2023-10-06T11:18:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.967+0000 7fb1c6ffd700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 0x7fb1cc062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.967+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb1d407bd80 con 0x7fb1e00cae40 2023-10-06T11:18:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.967+0000 7fb1ded9d700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 0x7fb1cc062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:47.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.971+0000 7fb1ded9d700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 0x7fb1cc062ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0157780 tx=0x7fb1d0009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:47.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:47.975+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb1d404c1f0 con 0x7fb1e00cae40 2023-10-06T11:18:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.147+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1e00c7a80 con 0x7fb1e00cae40 2023-10-06T11:18:48.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.147+0000 7fb1c6ffd700 1 -- 172.21.15.46:0/3636737005 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1e00c7a80 con 0x7fb1e00cae40 2023-10-06T11:18:48.152 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:48.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 msgr2=0x7fb1cc062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 0x7fb1cc062ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0157780 tx=0x7fb1d0009120 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 msgr2=0x7fb1e016da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 0x7fb1e016da30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c7c0 tx=0x7fb1d4007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 shutdown_connections 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb1cc060620 0x7fb1cc062ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e00cd250 0x7fb1e00b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1e00cae40 0x7fb1e016da30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 --2- 172.21.15.46:0/3636737005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb1e00b23c0 0x7fb1e00c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 >> 172.21.15.46:0/3636737005 conn(0x7fb1e0099480 msgr2=0x7fb1e0067c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 shutdown_connections 2023-10-06T11:18:48.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.151+0000 7fb1e502e700 1 -- 172.21.15.46:0/3636737005 wait complete. 2023-10-06T11:18:48.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:48.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.359+0000 7f98bf08e700 1 Processor -- start 2023-10-06T11:18:48.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.359+0000 7f98bf08e700 1 -- start start 2023-10-06T11:18:48.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bf08e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98b80abd90 0x7f98b80add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bf08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b80a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bf08e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b806f650 con 0x7f98b80a69c0 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bf08e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80b02d0 con 0x7f98b80abd90 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bf08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80aecd0 con 0x7f98b806ef00 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98be08c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98b80a69c0 0x7f98b80ad630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50296/0 (socket says 172.21.15.46:50296) 2023-10-06T11:18:48.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98be08c700 1 -- 172.21.15.46:0/1124465908 learned_addr learned my addr 172.21.15.46:0/1124465908 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98be88d700 1 --2- 172.21.15.46:0/1124465908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b80a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bd88b700 1 --2- 172.21.15.46:0/1124465908 >> v2:172.21.15.132:3300/0 conn(0x7f98b80abd90 0x7f98b80add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bd08a700 1 -- 172.21.15.46:0/1124465908 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268918522 0 0) 0x7f98b806f650 con 0x7f98b80a69c0 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.363+0000 7f98bd08a700 1 -- 172.21.15.46:0/1124465908 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a4003570 con 0x7f98b80a69c0 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98be88d700 1 -- 172.21.15.46:0/1124465908 >> v2:172.21.15.132:3300/0 conn(0x7f98b80abd90 msgr2=0x7f98b80add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98be88d700 1 --2- 172.21.15.46:0/1124465908 >> v2:172.21.15.132:3300/0 conn(0x7f98b80abd90 0x7f98b80add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98be88d700 1 -- 172.21.15.46:0/1124465908 >> v1:172.21.15.46:6789/0 conn(0x7f98b80a69c0 legacy=0x7f98b80ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98be88d700 1 -- 172.21.15.46:0/1124465908 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80a72d0 con 0x7f98b806ef00 2023-10-06T11:18:48.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98be88d700 1 --2- 172.21.15.46:0/1124465908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b80a6d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009e60 tx=0x7f98b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbee12d34676cba9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bd08a700 1 -- 172.21.15.46:0/1124465908 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b000e070 con 0x7f98b806ef00 2023-10-06T11:18:48.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bd08a700 1 -- 172.21.15.46:0/1124465908 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b000bab0 con 0x7f98b806ef00 2023-10-06T11:18:48.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bd08a700 1 -- 172.21.15.46:0/1124465908 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b000be70 con 0x7f98b806ef00 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 -- 172.21.15.46:0/1124465908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 msgr2=0x7f98b80a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 --2- 172.21.15.46:0/1124465908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b80a6d90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009e60 tx=0x7f98b0009780 comp rx=0 tx=0).stop 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 -- 172.21.15.46:0/1124465908 shutdown_connections 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 --2- 172.21.15.46:0/1124465908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b80a6d90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 --2- 172.21.15.46:0/1124465908 >> v2:172.21.15.132:3300/0 conn(0x7f98b80abd90 0x7f98b80add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 -- 172.21.15.46:0/1124465908 >> 172.21.15.46:0/1124465908 conn(0x7f98b80afd10 msgr2=0x7f98b806e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 -- 172.21.15.46:0/1124465908 shutdown_connections 2023-10-06T11:18:48.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 -- 172.21.15.46:0/1124465908 wait complete. 2023-10-06T11:18:48.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.367+0000 7f98bf08e700 1 Processor -- start 2023-10-06T11:18:48.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- start start 2023-10-06T11:18:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b816a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b80a69c0 0x7f98b8169860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b816b1f0 con 0x7f98b80abd90 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8169da0 con 0x7f98b80a69c0 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b816a5e0 con 0x7f98b806ef00 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44690/0 (socket says 172.21.15.46:44690) 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 -- 172.21.15.46:0/2433520059 learned_addr learned my addr 172.21.15.46:0/2433520059 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:48.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be08c700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bd88b700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b80a69c0 0x7f98b8169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 -- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 msgr2=0x7f98b816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 -- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b80a69c0 msgr2=0x7f98b8169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b80a69c0 0x7f98b8169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 -- 172.21.15.46:0/2433520059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b0009a20 con 0x7f98b80abd90 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98be88d700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f98b00097c0 tx=0x7f98b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0027470 con 0x7f98b80abd90 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00275d0 con 0x7f98b80abd90 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b816a8c0 con 0x7f98b80abd90 2023-10-06T11:18:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0027990 con 0x7f98b80abd90 2023-10-06T11:18:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.371+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b816ae80 con 0x7f98b80abd90 2023-10-06T11:18:48.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.375+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b816b3a0 con 0x7f98b80abd90 2023-10-06T11:18:48.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.375+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f98b0024070 con 0x7f98b80abd90 2023-10-06T11:18:48.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.375+0000 7f98ae7fc700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 0x7f989c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.375+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98b007f3e0 con 0x7f98b80abd90 2023-10-06T11:18:48.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.375+0000 7f98be08c700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 0x7f989c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.379+0000 7f98be08c700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 0x7f989c062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f98b40058d0 tx=0x7f98b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.383+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98b816b3a0 con 0x7f98b80abd90 2023-10-06T11:18:48.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.551+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98b8064590 con 0x7f98b80abd90 2023-10-06T11:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.551+0000 7f98ae7fc700 1 -- 172.21.15.46:0/2433520059 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98b004f850 con 0x7f98b80abd90 2023-10-06T11:18:48.557 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 msgr2=0x7f989c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 0x7f989c062a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f98b40058d0 tx=0x7f98b4009040 comp rx=0 tx=0).stop 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 msgr2=0x7f98b8166a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f98b00097c0 tx=0x7f98b0019000 comp rx=0 tx=0).stop 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 shutdown_connections 2023-10-06T11:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f989c060550 0x7f989c062a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b80abd90 0x7f98b8166a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b80a69c0 0x7f98b8169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 --2- 172.21.15.46:0/2433520059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b806ef00 0x7f98b816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 >> 172.21.15.46:0/2433520059 conn(0x7f98b80afd10 msgr2=0x7f98b80698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 shutdown_connections 2023-10-06T11:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.555+0000 7f98bf08e700 1 -- 172.21.15.46:0/2433520059 wait complete. 2023-10-06T11:18:48.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:48.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.739+0000 7f34d9ee3700 1 Processor -- start 2023-10-06T11:18:48.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.739+0000 7f34d9ee3700 1 -- start start 2023-10-06T11:18:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d9ee3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34d4154880 0x7f34d40c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d9ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 0x7f34d406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d9ee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d40c6cf0 con 0x7f34d40c7f20 2023-10-06T11:18:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d9ee3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d40c2a30 con 0x7f34d4154880 2023-10-06T11:18:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d9ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40c83e0 con 0x7f34d4154c50 2023-10-06T11:18:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34cbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 0x7f34d406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d8ee1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34d4154880 0x7f34d40c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d96e2700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f34d40c7f20 0x7f34d415aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:41854/0 (socket says 172.21.15.46:41854) 2023-10-06T11:18:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d96e2700 1 -- 172.21.15.46:0/2975319060 learned_addr learned my addr 172.21.15.46:0/2975319060 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d8ee1700 1 -- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 msgr2=0x7f34d406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d8ee1700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 0x7f34d406c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d8ee1700 1 -- 172.21.15.46:0/2975319060 >> v1:172.21.15.132:6789/0 conn(0x7f34d40c7f20 legacy=0x7f34d415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34d8ee1700 1 -- 172.21.15.46:0/2975319060 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d406e5d0 con 0x7f34d4154880 2023-10-06T11:18:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.743+0000 7f34cbfff700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 0x7f34d406c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d8ee1700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154880 0x7f34d40c7180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f34d00098e0 tx=0x7f34d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad6a51338862653 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34cb7fe700 1 -- 172.21.15.46:0/2975319060 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000b640 con 0x7f34d4154880 2023-10-06T11:18:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34cb7fe700 1 -- 172.21.15.46:0/2975319060 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000b7a0 con 0x7f34d4154880 2023-10-06T11:18:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34cb7fe700 1 -- 172.21.15.46:0/2975319060 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000bb40 con 0x7f34d4154880 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 -- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154880 msgr2=0x7f34d40c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154880 0x7f34d40c7180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f34d00098e0 tx=0x7f34d00091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 -- 172.21.15.46:0/2975319060 shutdown_connections 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154c50 0x7f34d406c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/2975319060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154880 0x7f34d40c7180 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 -- 172.21.15.46:0/2975319060 >> 172.21.15.46:0/2975319060 conn(0x7f34d40cbc10 msgr2=0x7f34d40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 -- 172.21.15.46:0/2975319060 shutdown_connections 2023-10-06T11:18:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.747+0000 7f34d9ee3700 1 -- 172.21.15.46:0/2975319060 wait complete. 2023-10-06T11:18:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 Processor -- start 2023-10-06T11:18:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 -- start start 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 0x7f34d40c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154c50 0x7f34d40c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40c6a40 con 0x7f34d40c7f20 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4156370 con 0x7f34d4154c50 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d9ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4156610 con 0x7f34d4154880 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d8ee1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d8ee1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44698/0 (socket says 172.21.15.46:44698) 2023-10-06T11:18:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d8ee1700 1 -- 172.21.15.46:0/3597220110 learned_addr learned my addr 172.21.15.46:0/3597220110 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34d96e2700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154c50 0x7f34d40c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.751+0000 7f34cbfff700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 0x7f34d40c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 -- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 msgr2=0x7f34d40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 0x7f34d40c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 -- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154c50 msgr2=0x7f34d40c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154c50 0x7f34d40c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 -- 172.21.15.46:0/3597220110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4009420 con 0x7f34d40c7f20 2023-10-06T11:18:48.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34cbfff700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 0x7f34d40c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d8ee1700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c9790 tx=0x7f34d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d001bc40 con 0x7f34d40c7f20 2023-10-06T11:18:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d001bde0 con 0x7f34d40c7f20 2023-10-06T11:18:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d0009420 con 0x7f34d40c7f20 2023-10-06T11:18:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0011670 con 0x7f34d40c7f20 2023-10-06T11:18:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d4156bd0 con 0x7f34d40c7f20 2023-10-06T11:18:48.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.755+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d4064590 con 0x7f34d40c7f20 2023-10-06T11:18:48.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.759+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f34d00119e0 con 0x7f34d40c7f20 2023-10-06T11:18:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.759+0000 7f34c8ff9700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 0x7f34c0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.759+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f34d007de80 con 0x7f34d40c7f20 2023-10-06T11:18:48.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.759+0000 7f34cbfff700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 0x7f34c0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:48.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.759+0000 7f34cbfff700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 0x7f34c0062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34d4148fa0 tx=0x7f34c40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.767+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f34d004e270 con 0x7f34d40c7f20 2023-10-06T11:18:48.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.939+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34d406e8b0 con 0x7f34d40c7f20 2023-10-06T11:18:48.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.939+0000 7f34c8ff9700 1 -- 172.21.15.46:0/3597220110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34d0051890 con 0x7f34d40c7f20 2023-10-06T11:18:48.945 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 msgr2=0x7f34c0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 0x7f34c0062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34d4148fa0 tx=0x7f34c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 msgr2=0x7f34d40ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34d40c9790 tx=0x7f34d0017000 comp rx=0 tx=0).stop 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 shutdown_connections 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f34c0060550 0x7f34c0062a00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34d4154c50 0x7f34d40c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34d4154880 0x7f34d40c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 --2- 172.21.15.46:0/3597220110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34d40c7f20 0x7f34d40ca9e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 >> 172.21.15.46:0/3597220110 conn(0x7f34d40cbc10 msgr2=0x7f34d40b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 shutdown_connections 2023-10-06T11:18:48.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:48.943+0000 7f34d9ee3700 1 -- 172.21.15.46:0/3597220110 wait complete. 2023-10-06T11:18:48.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.131+0000 7facb6fcc700 1 Processor -- start 2023-10-06T11:18:49.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.131+0000 7facb6fcc700 1 -- start start 2023-10-06T11:18:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.131+0000 7facb6fcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7facb00c97e0 0x7facb00c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.131+0000 7facb6fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c2220 0x7facb00c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.131+0000 7facb6fcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb00c1d90 con 0x7facb00c6250 2023-10-06T11:18:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb6fcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00c1460 con 0x7facb00c97e0 2023-10-06T11:18:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb6fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb00c1740 con 0x7facb00c2220 2023-10-06T11:18:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb57c9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7facb00c6250 0x7facb006c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50300/0 (socket says 172.21.15.46:50300) 2023-10-06T11:18:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb57c9700 1 -- 172.21.15.46:0/3104654080 learned_addr learned my addr 172.21.15.46:0/3104654080 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 --2- 172.21.15.46:0/3104654080 >> v2:172.21.15.46:3300/0 conn(0x7facb00c97e0 0x7facb00c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb67cb700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c2220 0x7facb00c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb4fc8700 1 -- 172.21.15.46:0/3104654080 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403619723 0 0) 0x7facb00c1d90 con 0x7facb00c6250 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb4fc8700 1 -- 172.21.15.46:0/3104654080 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac9c003570 con 0x7facb00c6250 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 -- 172.21.15.46:0/3104654080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c2220 msgr2=0x7facb00c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c2220 0x7facb00c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 -- 172.21.15.46:0/3104654080 >> v1:172.21.15.46:6789/0 conn(0x7facb00c6250 legacy=0x7facb006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 -- 172.21.15.46:0/3104654080 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00c1a20 con 0x7facb00c97e0 2023-10-06T11:18:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb5fca700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c97e0 0x7facb00c7d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facac0098e0 tx=0x7facac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eabd6a1ec81a400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb4fc8700 1 -- 172.21.15.46:0/3104654080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac011630 con 0x7facb00c97e0 2023-10-06T11:18:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb4fc8700 1 -- 172.21.15.46:0/3104654080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac011790 con 0x7facb00c97e0 2023-10-06T11:18:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.135+0000 7facb4fc8700 1 -- 172.21.15.46:0/3104654080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac011b30 con 0x7facb00c97e0 2023-10-06T11:18:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- 172.21.15.46:0/3104654080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c97e0 msgr2=0x7facb00c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c97e0 0x7facb00c7d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facac0098e0 tx=0x7facac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- 172.21.15.46:0/3104654080 shutdown_connections 2023-10-06T11:18:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c2220 0x7facb00c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 --2- 172.21.15.46:0/3104654080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c97e0 0x7facb00c7d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- 172.21.15.46:0/3104654080 >> 172.21.15.46:0/3104654080 conn(0x7facb00cbc10 msgr2=0x7facb00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:49.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- 172.21.15.46:0/3104654080 shutdown_connections 2023-10-06T11:18:49.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- 172.21.15.46:0/3104654080 wait complete. 2023-10-06T11:18:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 Processor -- start 2023-10-06T11:18:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.139+0000 7facb6fcc700 1 -- start start 2023-10-06T11:18:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 0x7facb016d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 0x7facb015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0159ec0 con 0x7facb00c6250 2023-10-06T11:18:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb00ab600 con 0x7facb00c2220 2023-10-06T11:18:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb016e040 con 0x7facb00c97e0 2023-10-06T11:18:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44724/0 (socket says 172.21.15.46:44724) 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 -- 172.21.15.46:0/991235939 learned_addr learned my addr 172.21.15.46:0/991235939 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb67cb700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 0x7facb015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 -- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 msgr2=0x7facb015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 0x7facb015ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb5fca700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 0x7facb016d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 -- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 msgr2=0x7facb016d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 0x7facb016d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 -- 172.21.15.46:0/991235939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac009420 con 0x7facb00c6250 2023-10-06T11:18:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb67cb700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 0x7facb015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb5fca700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 0x7facb016d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb57c9700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faca000c590 tx=0x7faca0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0005b30 con 0x7facb00c6250 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca0005cb0 con 0x7facb00c6250 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb016e320 con 0x7facb00c6250 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0007320 con 0x7facb00c6250 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb016e6b0 con 0x7facb00c6250 2023-10-06T11:18:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.143+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb00af2e0 con 0x7facb00c6250 2023-10-06T11:18:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.147+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faca001b3b0 con 0x7facb00c6250 2023-10-06T11:18:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.147+0000 7faca5ffb700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 0x7fac94062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.147+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faca007bf70 con 0x7facb00c6250 2023-10-06T11:18:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.147+0000 7facb5fca700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 0x7fac94062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.147+0000 7facb5fca700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 0x7fac94062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7facb0163110 tx=0x7facac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.151+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faca004c360 con 0x7facb00c6250 2023-10-06T11:18:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.315+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facb0064590 con 0x7facb00c6250 2023-10-06T11:18:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7faca5ffb700 1 -- 172.21.15.46:0/991235939 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7faca0017020 con 0x7facb00c6250 2023-10-06T11:18:49.322 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:49.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 msgr2=0x7fac94062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 0x7fac94062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7facb0163110 tx=0x7facac017000 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 msgr2=0x7facb0157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faca000c590 tx=0x7faca0007e00 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 shutdown_connections 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac94060550 0x7fac94062a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facb00c97e0 0x7facb015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb00c6250 0x7facb0157dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 --2- 172.21.15.46:0/991235939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facb00c2220 0x7facb016d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 >> 172.21.15.46:0/991235939 conn(0x7facb00cbc10 msgr2=0x7facb00c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.319+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 shutdown_connections 2023-10-06T11:18:49.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.323+0000 7facb6fcc700 1 -- 172.21.15.46:0/991235939 wait complete. 2023-10-06T11:18:49.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.495+0000 7f2fba145700 1 Processor -- start 2023-10-06T11:18:49.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.495+0000 7f2fba145700 1 -- start start 2023-10-06T11:18:49.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fba145700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 0x7f2fb406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fba145700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fb40ad6c0 0x7f2fb40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fba145700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb406d460 con 0x7f2fb415ab60 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fba145700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb406ed40 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fba145700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb40c7000 con 0x7f2fb406e8a0 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb37fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2fb415ab60 0x7f2fb415af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50308/0 (socket says 172.21.15.46:50308) 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb37fe700 1 -- 172.21.15.46:0/1179899628 learned_addr learned my addr 172.21.15.46:0/1179899628 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/1179899628 >> v2:172.21.15.46:3300/0 conn(0x7f2fb40ad6c0 0x7f2fb40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fabfff700 1 --2- 172.21.15.46:0/1179899628 >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 0x7f2fb406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb2ffd700 1 -- 172.21.15.46:0/1179899628 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225908164 0 0) 0x7f2fb406d460 con 0x7f2fb415ab60 2023-10-06T11:18:49.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb2ffd700 1 -- 172.21.15.46:0/1179899628 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa0003570 con 0x7f2fb415ab60 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 -- 172.21.15.46:0/1179899628 >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 msgr2=0x7f2fb406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/1179899628 >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 0x7f2fb406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 -- 172.21.15.46:0/1179899628 >> v1:172.21.15.46:6789/0 conn(0x7f2fb415ab60 legacy=0x7f2fb415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 -- 172.21.15.46:0/1179899628 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40c72c0 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fabfff700 1 --2- 172.21.15.46:0/1179899628 >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 0x7f2fb406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/1179899628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb40adaf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fb406f0b0 tx=0x7f2fac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b503c006be5bdde4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb2ffd700 1 -- 172.21.15.46:0/1179899628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00e070 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb2ffd700 1 -- 172.21.15.46:0/1179899628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00bb60 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.499+0000 7f2fb2ffd700 1 -- 172.21.15.46:0/1179899628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac012430 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 -- 172.21.15.46:0/1179899628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb40ad6c0 msgr2=0x7f2fb40adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 --2- 172.21.15.46:0/1179899628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb40adaf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2fb406f0b0 tx=0x7f2fac009780 comp rx=0 tx=0).stop 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 -- 172.21.15.46:0/1179899628 shutdown_connections 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 --2- 172.21.15.46:0/1179899628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb40adaf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 --2- 172.21.15.46:0/1179899628 >> v2:172.21.15.132:3300/0 conn(0x7f2fb406e8a0 0x7f2fb406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 -- 172.21.15.46:0/1179899628 >> 172.21.15.46:0/1179899628 conn(0x7f2fb40cbe30 msgr2=0x7f2fb40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 -- 172.21.15.46:0/1179899628 shutdown_connections 2023-10-06T11:18:49.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.503+0000 7f2fba145700 1 -- 172.21.15.46:0/1179899628 wait complete. 2023-10-06T11:18:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 Processor -- start 2023-10-06T11:18:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 -- start start 2023-10-06T11:18:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb406e8a0 0x7f2fb40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb41677d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4167d10 con 0x7f2fb415ab60 2023-10-06T11:18:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40af340 con 0x7f2fb40ad6c0 2023-10-06T11:18:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fba145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40af620 con 0x7f2fb406e8a0 2023-10-06T11:18:49.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44742/0 (socket says 172.21.15.46:44742) 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 -- 172.21.15.46:0/132132960 learned_addr learned my addr 172.21.15.46:0/132132960 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fabfff700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb41677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb37fe700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb406e8a0 0x7f2fb40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 -- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb406e8a0 msgr2=0x7f2fb40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb406e8a0 0x7f2fb40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 -- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb40ad6c0 msgr2=0x7f2fb41677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb41677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.507+0000 7f2fb3fff700 1 -- 172.21.15.46:0/132132960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009a20 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fb3fff700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009fd0 tx=0x7f2fac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac027470 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac0275f0 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb40af900 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0279b0 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb40afec0 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb406ee80 con 0x7f2fb415ab60 2023-10-06T11:18:49.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.511+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2fac024070 con 0x7f2fb415ab60 2023-10-06T11:18:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.515+0000 7f2fab7fe700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 0x7f2f94062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.515+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2fac07f310 con 0x7f2fb415ab60 2023-10-06T11:18:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.515+0000 7f2fb37fe700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 0x7f2f94062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.515+0000 7f2fb37fe700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 0x7f2f94062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4001360 tx=0x7f2fa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.523+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2fb406ee80 con 0x7f2fb415ab60 2023-10-06T11:18:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.691+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fb406ee80 con 0x7f2fb415ab60 2023-10-06T11:18:49.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.695+0000 7f2fab7fe700 1 -- 172.21.15.46:0/132132960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f2fb406ee80 con 0x7f2fb415ab60 2023-10-06T11:18:49.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:49.812 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 msgr2=0x7f2f94062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 0x7f2f94062a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4001360 tx=0x7f2fa4009040 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 msgr2=0x7f2fb414f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009fd0 tx=0x7f2fac019000 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 shutdown_connections 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f94060550 0x7f2f94062a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb415ab60 0x7f2fb414f6b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb40ad6c0 0x7f2fb41677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 --2- 172.21.15.46:0/132132960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fb406e8a0 0x7f2fb40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 >> 172.21.15.46:0/132132960 conn(0x7f2fb40cbe30 msgr2=0x7f2fb4068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 shutdown_connections 2023-10-06T11:18:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.699+0000 7f2fba145700 1 -- 172.21.15.46:0/132132960 wait complete. 2023-10-06T11:18:49.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:49.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.983+0000 7f688f3cc700 1 Processor -- start 2023-10-06T11:18:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.983+0000 7f688f3cc700 1 -- start start 2023-10-06T11:18:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688f3cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c97e0 0x7f68880c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688f3cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68880c2220 0x7f68880c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688f3cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68880c1d90 con 0x7f68880c6250 2023-10-06T11:18:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688f3cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68880c1460 con 0x7f68880c2220 2023-10-06T11:18:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688f3cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880c1740 con 0x7f68880c97e0 2023-10-06T11:18:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68880c2220 0x7f68880c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68880c2220 0x7f68880c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44756/0 (socket says 172.21.15.46:44756) 2023-10-06T11:18:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 -- 172.21.15.46:0/933642952 learned_addr learned my addr 172.21.15.46:0/933642952 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:49.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688e3ca700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c97e0 0x7f68880c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 -- 172.21.15.46:0/933642952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c97e0 msgr2=0x7f68880c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c97e0 0x7f68880c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 -- 172.21.15.46:0/933642952 >> v1:172.21.15.132:6789/0 conn(0x7f68880c6250 legacy=0x7f688806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 -- 172.21.15.46:0/933642952 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880c1a20 con 0x7f68880c2220 2023-10-06T11:18:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688d3c8700 1 -- 172.21.15.46:0/933642952 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124303977 0 0) 0x7f68880c1d90 con 0x7f68880c6250 2023-10-06T11:18:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688ebcb700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c2220 0x7f68880c2670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f68800099b0 tx=0x7f6880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4b4c4cb3d0ee04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688d3c8700 1 -- 172.21.15.46:0/933642952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009ea0 con 0x7f68880c2220 2023-10-06T11:18:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688d3c8700 1 -- 172.21.15.46:0/933642952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000b860 con 0x7f68880c2220 2023-10-06T11:18:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.987+0000 7f688d3c8700 1 -- 172.21.15.46:0/933642952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000bc00 con 0x7f68880c2220 2023-10-06T11:18:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 -- 172.21.15.46:0/933642952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c2220 msgr2=0x7f68880c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c2220 0x7f68880c2670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f68800099b0 tx=0x7f6880009250 comp rx=0 tx=0).stop 2023-10-06T11:18:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 -- 172.21.15.46:0/933642952 shutdown_connections 2023-10-06T11:18:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c2220 0x7f68880c2670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 --2- 172.21.15.46:0/933642952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c97e0 0x7f68880c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 -- 172.21.15.46:0/933642952 >> 172.21.15.46:0/933642952 conn(0x7f68880cbc10 msgr2=0x7f68880cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 -- 172.21.15.46:0/933642952 shutdown_connections 2023-10-06T11:18:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.991+0000 7f688f3cc700 1 -- 172.21.15.46:0/933642952 wait complete. 2023-10-06T11:18:50.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 Processor -- start 2023-10-06T11:18:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 -- start start 2023-10-06T11:18:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68880c2220 0x7f6888151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c6250 0x7f68880ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688815aed0 con 0x7f68880c97e0 2023-10-06T11:18:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881527a0 con 0x7f68880c2220 2023-10-06T11:18:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688f3cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880c5280 con 0x7f68880c6250 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688ebcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688ebcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44760/0 (socket says 172.21.15.46:44760) 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688ebcb700 1 -- 172.21.15.46:0/2254581458 learned_addr learned my addr 172.21.15.46:0/2254581458 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688e3ca700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68880c2220 0x7f6888151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.995+0000 7f688dbc9700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c6250 0x7f68880ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 -- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c6250 msgr2=0x7f68880ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c6250 0x7f68880ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 -- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68880c2220 msgr2=0x7f6888151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68880c2220 0x7f6888151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 -- 172.21.15.46:0/2254581458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800094f0 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688ebcb700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f688000b3c0 tx=0x7f6880017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000baa0 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880025420 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000bc00 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68880c5560 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688815ab60 con 0x7f68880c97e0 2023-10-06T11:18:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:49.999+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688815b080 con 0x7f68880c97e0 2023-10-06T11:18:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.003+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6880023070 con 0x7f68880c97e0 2023-10-06T11:18:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.003+0000 7f687e7fc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 0x7f686c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.003+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f688007dee0 con 0x7f68880c97e0 2023-10-06T11:18:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.003+0000 7f688e3ca700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 0x7f686c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.003+0000 7f688e3ca700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 0x7f686c062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6884009fd0 tx=0x7f688400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.007+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f688815b080 con 0x7f68880c97e0 2023-10-06T11:18:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.179+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f688815b080 con 0x7f68880c97e0 2023-10-06T11:18:50.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.179+0000 7f687e7fc700 1 -- 172.21.15.46:0/2254581458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f688815b080 con 0x7f68880c97e0 2023-10-06T11:18:50.183 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 msgr2=0x7f686c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 0x7f686c062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6884009fd0 tx=0x7f688400b040 comp rx=0 tx=0).stop 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 msgr2=0x7f68880aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f688000b3c0 tx=0x7f6880017000 comp rx=0 tx=0).stop 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 shutdown_connections 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f686c060550 0x7f686c062a00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68880c97e0 0x7f68880aba10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68880c6250 0x7f68880ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 --2- 172.21.15.46:0/2254581458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68880c2220 0x7f6888151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 >> 172.21.15.46:0/2254581458 conn(0x7f68880cbc10 msgr2=0x7f6888068530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 shutdown_connections 2023-10-06T11:18:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.183+0000 7f688f3cc700 1 -- 172.21.15.46:0/2254581458 wait complete. 2023-10-06T11:18:50.206 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:50.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 Processor -- start 2023-10-06T11:18:50.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 -- start start 2023-10-06T11:18:50.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca980e64e0 0x7fca98053200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca98054860 con 0x7fca98053740 2023-10-06T11:18:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca98053c50 con 0x7fca980e68b0 2023-10-06T11:18:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa608e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca98053f30 con 0x7fca980e64e0 2023-10-06T11:18:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa588d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca98053740 0x7fca9804ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50320/0 (socket says 172.21.15.46:50320) 2023-10-06T11:18:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa588d700 1 -- 172.21.15.46:0/1985674950 learned_addr learned my addr 172.21.15.46:0/1985674950 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.367+0000 7fcaa508c700 1 --2- 172.21.15.46:0/1985674950 >> v2:172.21.15.132:3300/0 conn(0x7fca980e64e0 0x7fca98053200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158181762 0 0) 0x7fca98053c50 con 0x7fca980e68b0 2023-10-06T11:18:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003570 con 0x7fca980e68b0 2023-10-06T11:18:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375074267 0 0) 0x7fca98054860 con 0x7fca98053740 2023-10-06T11:18:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca98053c50 con 0x7fca98053740 2023-10-06T11:18:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa508c700 1 -- 172.21.15.46:0/1985674950 >> v1:172.21.15.132:6789/0 conn(0x7fca980e68b0 legacy=0x7fca98006460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa508c700 1 -- 172.21.15.46:0/1985674950 >> v1:172.21.15.46:6789/0 conn(0x7fca98053740 legacy=0x7fca9804ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa508c700 1 -- 172.21.15.46:0/1985674950 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98054210 con 0x7fca980e64e0 2023-10-06T11:18:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa508c700 1 --2- 172.21.15.46:0/1985674950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca980e64e0 0x7fca98053200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098e0 tx=0x7fca9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a046f86ca5bb4f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011630 con 0x7fca980e64e0 2023-10-06T11:18:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c011790 con 0x7fca980e64e0 2023-10-06T11:18:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fca97fff700 1 -- 172.21.15.46:0/1985674950 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c011b30 con 0x7fca980e64e0 2023-10-06T11:18:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa608e700 1 -- 172.21.15.46:0/1985674950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca980e64e0 msgr2=0x7fca98053200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa608e700 1 --2- 172.21.15.46:0/1985674950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca980e64e0 0x7fca98053200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0098e0 tx=0x7fca9c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa608e700 1 -- 172.21.15.46:0/1985674950 shutdown_connections 2023-10-06T11:18:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa608e700 1 --2- 172.21.15.46:0/1985674950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca980e64e0 0x7fca98053200 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.371+0000 7fcaa608e700 1 -- 172.21.15.46:0/1985674950 >> 172.21.15.46:0/1985674950 conn(0x7fca98055630 msgr2=0x7fca98055a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- 172.21.15.46:0/1985674950 shutdown_connections 2023-10-06T11:18:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- 172.21.15.46:0/1985674950 wait complete. 2023-10-06T11:18:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 Processor -- start 2023-10-06T11:18:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- start start 2023-10-06T11:18:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca98053740 0x7fca980e2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 0x7fca9804a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9804f4f0 con 0x7fca980e64e0 2023-10-06T11:18:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca980522c0 con 0x7fca98053740 2023-10-06T11:18:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.375+0000 7fcaa608e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9804af40 con 0x7fca980e68b0 2023-10-06T11:18:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa488b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 0x7fca9804a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa488b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 0x7fca9804a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44764/0 (socket says 172.21.15.46:44764) 2023-10-06T11:18:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa488b700 1 -- 172.21.15.46:0/3845889779 learned_addr learned my addr 172.21.15.46:0/3845889779 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55214/0 (socket says 172.21.15.46:55214) 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa508c700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca98053740 0x7fca980e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 -- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca98053740 msgr2=0x7fca980e2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca98053740 0x7fca980e2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 -- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 msgr2=0x7fca9804a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 0x7fca9804a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 -- 172.21.15.46:0/3845889779 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009420 con 0x7fca980e68b0 2023-10-06T11:18:50.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa588d700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcaa006fbe0 tx=0x7fcaa0066e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa006f250 con 0x7fca980e68b0 2023-10-06T11:18:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa006f3b0 con 0x7fca980e68b0 2023-10-06T11:18:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9804b220 con 0x7fca980e68b0 2023-10-06T11:18:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa006f770 con 0x7fca980e68b0 2023-10-06T11:18:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.379+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9804b7e0 con 0x7fca980e68b0 2023-10-06T11:18:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.383+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca98053250 con 0x7fca980e68b0 2023-10-06T11:18:50.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.383+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcaa0062d50 con 0x7fca980e68b0 2023-10-06T11:18:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.383+0000 7fca957fa700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 0x7fca84062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.383+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcaa00f3200 con 0x7fca980e68b0 2023-10-06T11:18:50.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.383+0000 7fcaa508c700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 0x7fca84062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.387+0000 7fcaa508c700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 0x7fca84062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0091e0 tx=0x7fca9c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.391+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fca98053250 con 0x7fca980e68b0 2023-10-06T11:18:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.563+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca98053250 con 0x7fca980e68b0 2023-10-06T11:18:50.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.563+0000 7fca957fa700 1 -- 172.21.15.46:0/3845889779 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca98053250 con 0x7fca980e68b0 2023-10-06T11:18:50.569 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 msgr2=0x7fca84062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 0x7fca84062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0091e0 tx=0x7fca9c017000 comp rx=0 tx=0).stop 2023-10-06T11:18:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 msgr2=0x7fca9804a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcaa006fbe0 tx=0x7fcaa0066e50 comp rx=0 tx=0).stop 2023-10-06T11:18:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 shutdown_connections 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca84060550 0x7fca84062a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca980e68b0 0x7fca9804a910 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca980e64e0 0x7fca9804a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 --2- 172.21.15.46:0/3845889779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca98053740 0x7fca980e2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 >> 172.21.15.46:0/3845889779 conn(0x7fca98055630 msgr2=0x7fca98055a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.567+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 shutdown_connections 2023-10-06T11:18:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.571+0000 7fcaa608e700 1 -- 172.21.15.46:0/3845889779 wait complete. 2023-10-06T11:18:50.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 Processor -- start 2023-10-06T11:18:50.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 -- start start 2023-10-06T11:18:50.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 0x7f3ea00b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea015d090 con 0x7f3ea0142790 2023-10-06T11:18:50.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea0068890 con 0x7f3ea0142b60 2023-10-06T11:18:50.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3ea58f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0068310 con 0x7f3ea00b1340 2023-10-06T11:18:50.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3e9effd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ea0142790 0x7f3ea0159710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50332/0 (socket says 172.21.15.46:50332) 2023-10-06T11:18:50.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3e9effd700 1 -- 172.21.15.46:0/1523054160 learned_addr learned my addr 172.21.15.46:0/1523054160 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:50.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.759+0000 7f3e9f7fe700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 0x7f3ea00b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975407231 0 0) 0x7f3ea0068890 con 0x7f3ea0142b60 2023-10-06T11:18:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8c003570 con 0x7f3ea0142b60 2023-10-06T11:18:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944848569 0 0) 0x7f3ea015d090 con 0x7f3ea0142790 2023-10-06T11:18:50.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea0068890 con 0x7f3ea0142790 2023-10-06T11:18:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298299151 0 0) 0x7f3e8c003570 con 0x7f3ea0142b60 2023-10-06T11:18:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 msgr2=0x7f3ea00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 0x7f3ea00b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 >> v1:172.21.15.46:6789/0 conn(0x7f3ea0142790 legacy=0x7f3ea0159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00b38e0 con 0x7f3ea0142b60 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3e88001470 con 0x7f3ea0142b60 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 >> v1:172.21.15.132:6789/0 conn(0x7f3ea0142b60 legacy=0x7f3ea00b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e8c003bf0 0x7f3e8c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c004810 0x7f3e8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 0x7f3e8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0068310 con 0x7f3e8c003bf0 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c00bb40 con 0x7f3e8c004810 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c00bd00 con 0x7f3ea00b1340 2023-10-06T11:18:50.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e8c003bf0 0x7f3e8c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e97fff700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c004810 0x7f3e8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 -- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 msgr2=0x7f3e8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ea00b1340 0x7f3e8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 -- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c004810 msgr2=0x7f3e8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c004810 0x7f3e8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 -- 172.21.15.46:0/1523054160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c003570 con 0x7f3e8c003bf0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9effd700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e8c003bf0 0x7f3e8c003fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e90001af0 tx=0x7f3e90007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492aaef5fddfc6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900048f0 con 0x7f3e8c003bf0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e90004a50 con 0x7f3e8c003bf0 2023-10-06T11:18:50.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9e7fc700 1 -- 172.21.15.46:0/1523054160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90004df0 con 0x7f3e8c003bf0 2023-10-06T11:18:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 -- 172.21.15.46:0/1523054160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e8c003bf0 msgr2=0x7f3e8c003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/1523054160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e8c003bf0 0x7f3e8c003fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e90001af0 tx=0x7f3e90007e20 comp rx=0 tx=0).stop 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3e9f7fe700 1 -- 172.21.15.46:0/1523054160 reap_dead start 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 -- 172.21.15.46:0/1523054160 shutdown_connections 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 -- 172.21.15.46:0/1523054160 >> 172.21.15.46:0/1523054160 conn(0x7f3ea0067b20 msgr2=0x7f3ea006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 -- 172.21.15.46:0/1523054160 shutdown_connections 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.763+0000 7f3ea58f0700 1 -- 172.21.15.46:0/1523054160 wait complete. 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 Processor -- start 2023-10-06T11:18:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 -- start start 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c003bf0 0x7f3ea00ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 0x7f3ea0169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015b330 con 0x7f3ea013f8e0 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00ce900 con 0x7f3ea0169600 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3ea58f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00caef0 con 0x7f3e8c003bf0 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e97fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e97fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44784/0 (socket says 172.21.15.46:44784) 2023-10-06T11:18:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e97fff700 1 -- 172.21.15.46:0/2095143351 learned_addr learned my addr 172.21.15.46:0/2095143351 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e9effd700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c003bf0 0x7f3ea00ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e9f7fe700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 0x7f3ea0169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.767+0000 7f3e97fff700 1 -- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c003bf0 msgr2=0x7f3ea00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e97fff700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c003bf0 0x7f3ea00ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e97fff700 1 -- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 msgr2=0x7f3ea0169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e97fff700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 0x7f3ea0169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e97fff700 1 -- 172.21.15.46:0/2095143351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00ca2e0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e9f7fe700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 0x7f3ea0169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e97fff700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007fd0 tx=0x7f3e88009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880136c0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88013820 con 0x7f3ea013f8e0 2023-10-06T11:18:50.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88013be0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00ca5c0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00cab80 con 0x7f3ea013f8e0 2023-10-06T11:18:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.771+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea0064590 con 0x7f3ea013f8e0 2023-10-06T11:18:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.775+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3e88014480 con 0x7f3ea013f8e0 2023-10-06T11:18:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.775+0000 7f3e96ffd700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 0x7f3e80062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.775+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3e8807cfd0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.775+0000 7f3e9effd700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 0x7f3e80062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:50.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.775+0000 7f3e9effd700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 0x7f3e80062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e90007e60 tx=0x7f3e90015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:50.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.783+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3e8804d480 con 0x7f3ea013f8e0 2023-10-06T11:18:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.955+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ea00cb030 con 0x7f3ea013f8e0 2023-10-06T11:18:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.955+0000 7f3e96ffd700 1 -- 172.21.15.46:0/2095143351 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e88050aa0 con 0x7f3ea013f8e0 2023-10-06T11:18:50.961 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.955+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 msgr2=0x7f3e80062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 0x7f3e80062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e90007e60 tx=0x7f3e90015000 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 msgr2=0x7f3ea00c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007fd0 tx=0x7f3e88009040 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 shutdown_connections 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3e80060550 0x7f3e80062a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ea0169600 0x7f3ea0169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ea013f8e0 0x7f3ea00c9a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 --2- 172.21.15.46:0/2095143351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c003bf0 0x7f3ea00ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 >> 172.21.15.46:0/2095143351 conn(0x7f3ea0067b20 msgr2=0x7f3ea015d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 shutdown_connections 2023-10-06T11:18:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:50.959+0000 7f3ea58f0700 1 -- 172.21.15.46:0/2095143351 wait complete. 2023-10-06T11:18:50.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:51.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.139+0000 7f9cb062e700 1 Processor -- start 2023-10-06T11:18:51.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.139+0000 7f9cb062e700 1 -- start start 2023-10-06T11:18:51.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cb062e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cb062e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ca80697e0 0x7f9ca815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cb062e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca80ce470 con 0x7f9ca8158d10 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cb062e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca8069c50 con 0x7f9ca80697e0 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cb062e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815a890 con 0x7f9ca8069410 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ca80697e0 0x7f9ca815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ca80697e0 0x7f9ca815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44796/0 (socket says 172.21.15.46:44796) 2023-10-06T11:18:51.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 -- 172.21.15.46:0/3292366500 learned_addr learned my addr 172.21.15.46:0/3292366500 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:51.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cae3ca700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 -- 172.21.15.46:0/3292366500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 msgr2=0x7f9ca815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cad3c8700 1 -- 172.21.15.46:0/3292366500 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165562301 0 0) 0x7f9ca80ce470 con 0x7f9ca8158d10 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 -- 172.21.15.46:0/3292366500 >> v1:172.21.15.132:6789/0 conn(0x7f9ca8158d10 legacy=0x7f9ca8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.143+0000 7f9cadbc9700 1 -- 172.21.15.46:0/3292366500 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca815d470 con 0x7f9ca80697e0 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cadbc9700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca815a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c980099b0 tx=0x7f9c98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f0028c08afaece server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cad3c8700 1 -- 172.21.15.46:0/3292366500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98009ea0 con 0x7f9ca80697e0 2023-10-06T11:18:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cad3c8700 1 -- 172.21.15.46:0/3292366500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800b860 con 0x7f9ca80697e0 2023-10-06T11:18:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cad3c8700 1 -- 172.21.15.46:0/3292366500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800bc00 con 0x7f9ca80697e0 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 -- 172.21.15.46:0/3292366500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 msgr2=0x7f9ca815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca815a350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c980099b0 tx=0x7f9c98009250 comp rx=0 tx=0).stop 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 -- 172.21.15.46:0/3292366500 shutdown_connections 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca815a350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 --2- 172.21.15.46:0/3292366500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 -- 172.21.15.46:0/3292366500 >> 172.21.15.46:0/3292366500 conn(0x7f9ca80d0210 msgr2=0x7f9ca80d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 -- 172.21.15.46:0/3292366500 shutdown_connections 2023-10-06T11:18:51.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.147+0000 7f9cb062e700 1 -- 172.21.15.46:0/3292366500 wait complete. 2023-10-06T11:18:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 Processor -- start 2023-10-06T11:18:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 -- start start 2023-10-06T11:18:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca80b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 0x7f9ca815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca816a5b0 con 0x7f9ca80697e0 2023-10-06T11:18:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca816aa40 con 0x7f9ca8158d10 2023-10-06T11:18:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cb062e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80b34f0 con 0x7f9ca8069410 2023-10-06T11:18:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44812/0 (socket says 172.21.15.46:44812) 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 -- 172.21.15.46:0/1037238756 learned_addr learned my addr 172.21.15.46:0/1037238756 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cae3ca700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca80b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9caebcb700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 0x7f9ca815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 -- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 msgr2=0x7f9ca80b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca80b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 -- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 msgr2=0x7f9ca815bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.151+0000 7f9cadbc9700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 0x7f9ca815bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9cadbc9700 1 -- 172.21.15.46:0/1037238756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980094f0 con 0x7f9ca80697e0 2023-10-06T11:18:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9caebcb700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 0x7f9ca815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9cadbc9700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8088000 tx=0x7f9c98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800bbe0 con 0x7f9ca80697e0 2023-10-06T11:18:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98011420 con 0x7f9ca80697e0 2023-10-06T11:18:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca8169c80 con 0x7f9ca80697e0 2023-10-06T11:18:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980117c0 con 0x7f9ca80697e0 2023-10-06T11:18:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca816a240 con 0x7f9ca80697e0 2023-10-06T11:18:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.155+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca8169a50 con 0x7f9ca80697e0 2023-10-06T11:18:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.159+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9c98011bb0 con 0x7f9ca80697e0 2023-10-06T11:18:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.159+0000 7f9c9e7fc700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 0x7f9c8c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.159+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c9807dde0 con 0x7f9ca80697e0 2023-10-06T11:18:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.159+0000 7f9cae3ca700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 0x7f9c8c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.159+0000 7f9cae3ca700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 0x7f9c8c062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009db0 tx=0x7f9ca40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.167+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c9804e2f0 con 0x7f9ca80697e0 2023-10-06T11:18:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.339+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ca8088870 con 0x7f9ca80697e0 2023-10-06T11:18:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9c9e7fc700 1 -- 172.21.15.46:0/1037238756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c98015020 con 0x7f9ca80697e0 2023-10-06T11:18:51.346 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 msgr2=0x7f9c8c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 0x7f9c8c062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009db0 tx=0x7f9ca40091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 msgr2=0x7f9ca8154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8088000 tx=0x7f9c98017000 comp rx=0 tx=0).stop 2023-10-06T11:18:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 shutdown_connections 2023-10-06T11:18:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c8c060550 0x7f9c8c062a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ca8158d10 0x7f9ca815bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca80697e0 0x7f9ca8154f90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 --2- 172.21.15.46:0/1037238756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ca8069410 0x7f9ca80b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 >> 172.21.15.46:0/1037238756 conn(0x7f9ca80d0210 msgr2=0x7f9ca806fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 shutdown_connections 2023-10-06T11:18:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.343+0000 7f9cb062e700 1 -- 172.21.15.46:0/1037238756 wait complete. 2023-10-06T11:18:51.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:51.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 Processor -- start 2023-10-06T11:18:51.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 -- start start 2023-10-06T11:18:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff5e40c7fc0 0x7ff5e40c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40c6470 con 0x7ff5e40c7b50 2023-10-06T11:18:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e40c7760 con 0x7ff5e4156c80 2023-10-06T11:18:51.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.523+0000 7ff5ea6ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e40c66c0 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e37fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff5e40c7b50 0x7ff5e40c0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50334/0 (socket says 172.21.15.46:50334) 2023-10-06T11:18:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e37fe700 1 -- 172.21.15.46:0/2495507103 learned_addr learned my addr 172.21.15.46:0/2495507103 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:51.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e8c4b700 1 --2- 172.21.15.46:0/2495507103 >> v2:172.21.15.132:3300/0 conn(0x7ff5e40c7fc0 0x7ff5e40c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023688544 0 0) 0x7ff5e40c6470 con 0x7ff5e40c7b50 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003570 con 0x7ff5e40c7b50 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975332758 0 0) 0x7ff5e40c7760 con 0x7ff5e4156c80 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e40c6470 con 0x7ff5e4156c80 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2495507103 >> v1:172.21.15.132:6789/0 conn(0x7ff5e4156c80 legacy=0x7ff5e40ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2495507103 >> v1:172.21.15.46:6789/0 conn(0x7ff5e40c7b50 legacy=0x7ff5e40c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2495507103 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e40c69a0 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e8c4b700 1 --2- 172.21.15.46:0/2495507103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e40c12e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009e60 tx=0x7ff5dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59436b854d83c740 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00e070 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc00bab0 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5e2ffd700 1 -- 172.21.15.46:0/2495507103 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00be70 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2495507103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 msgr2=0x7ff5e40c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2495507103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e40c12e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009e60 tx=0x7ff5dc009780 comp rx=0 tx=0).stop 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2495507103 shutdown_connections 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2495507103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e40c12e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.527+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2495507103 >> 172.21.15.46:0/2495507103 conn(0x7ff5e40cbe30 msgr2=0x7ff5e40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.531+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2495507103 shutdown_connections 2023-10-06T11:18:51.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.531+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2495507103 wait complete. 2023-10-06T11:18:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.531+0000 7ff5ea6ae700 1 Processor -- start 2023-10-06T11:18:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 -- start start 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5e40c7b50 0x7ff5e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40ab7a0 con 0x7ff5e4156c80 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40abd60 con 0x7ff5e40c7fc0 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40aeef0 con 0x7ff5e40c7b50 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44818/0 (socket says 172.21.15.46:44818) 2023-10-06T11:18:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2867949396 learned_addr learned my addr 172.21.15.46:0/2867949396 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e37fe700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e3fff700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5e40c7b50 0x7ff5e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5e40c7b50 msgr2=0x7ff5e406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5e40c7b50 0x7ff5e406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 msgr2=0x7ff5e406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 -- 172.21.15.46:0/2867949396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc009a20 con 0x7ff5e4156c80 2023-10-06T11:18:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5e8c4b700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b610 tx=0x7ff5dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc012de0 con 0x7ff5e4156c80 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc026480 con 0x7ff5e4156c80 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.535+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e406e120 con 0x7ff5e4156c80 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.539+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc026840 con 0x7ff5e4156c80 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.539+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e40ab9f0 con 0x7ff5e4156c80 2023-10-06T11:18:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.539+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e406c880 con 0x7ff5e4156c80 2023-10-06T11:18:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.539+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff5dc01c400 con 0x7ff5e4156c80 2023-10-06T11:18:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.543+0000 7ff5cbfff700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 0x7ff5d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.543+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5dc07e8d0 con 0x7ff5e4156c80 2023-10-06T11:18:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.543+0000 7ff5e3fff700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 0x7ff5d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.543+0000 7ff5e3fff700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 0x7ff5d0062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4003a40 tx=0x7ff5d4007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.551+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff5e406c880 con 0x7ff5e4156c80 2023-10-06T11:18:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.719+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e406c880 con 0x7ff5e4156c80 2023-10-06T11:18:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.723+0000 7ff5cbfff700 1 -- 172.21.15.46:0/2867949396 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7ff5e406c880 con 0x7ff5e4156c80 2023-10-06T11:18:51.727 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:51.728 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 msgr2=0x7ff5d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 0x7ff5d0062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4003a40 tx=0x7ff5d4007d40 comp rx=0 tx=0).stop 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 msgr2=0x7ff5e40b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00b610 tx=0x7ff5dc019000 comp rx=0 tx=0).stop 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 shutdown_connections 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5d0060550 0x7ff5d0062a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e4156c80 0x7ff5e40b0650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e40c7fc0 0x7ff5e406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 --2- 172.21.15.46:0/2867949396 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5e40c7b50 0x7ff5e406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 >> 172.21.15.46:0/2867949396 conn(0x7ff5e40cbe30 msgr2=0x7ff5e414db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 shutdown_connections 2023-10-06T11:18:51.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.727+0000 7ff5ea6ae700 1 -- 172.21.15.46:0/2867949396 wait complete. 2023-10-06T11:18:51.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:51.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.919+0000 7f2f7386e700 1 Processor -- start 2023-10-06T11:18:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.919+0000 7f2f7386e700 1 -- start start 2023-10-06T11:18:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7386e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 0x7f2f6c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7386e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f6c06ed10 0x7f2f6c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7386e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c0c1bf0 con 0x7f2f6c0c6470 2023-10-06T11:18:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7386e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c72c0 con 0x7f2f6c06ed10 2023-10-06T11:18:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7386e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c06cf80 con 0x7f2f6c06e8a0 2023-10-06T11:18:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7160a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f6c0c6470 0x7f2f6c0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50340/0 (socket says 172.21.15.46:50340) 2023-10-06T11:18:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f70e09700 1 --2- 172.21.15.46:0/1077580236 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 0x7f2f6c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f7160a700 1 -- 172.21.15.46:0/1077580236 learned_addr learned my addr 172.21.15.46:0/1077580236 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 --2- 172.21.15.46:0/1077580236 >> v2:172.21.15.46:3300/0 conn(0x7f2f6c06ed10 0x7f2f6c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 -- 172.21.15.46:0/1077580236 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 msgr2=0x7f2f6c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 --2- 172.21.15.46:0/1077580236 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 0x7f2f6c0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 -- 172.21.15.46:0/1077580236 >> v1:172.21.15.46:6789/0 conn(0x7f2f6c0c6470 legacy=0x7f2f6c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 -- 172.21.15.46:0/1077580236 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c06d260 con 0x7f2f6c06ed10 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f70e09700 1 --2- 172.21.15.46:0/1077580236 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 0x7f2f6c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f71e0b700 1 --2- 172.21.15.46:0/1077580236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c6d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06c360 tx=0x7f2f64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62223907e13f0df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f63fff700 1 -- 172.21.15.46:0/1077580236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6400e070 con 0x7f2f6c06ed10 2023-10-06T11:18:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f63fff700 1 -- 172.21.15.46:0/1077580236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6400bb60 con 0x7f2f6c06ed10 2023-10-06T11:18:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.923+0000 7f2f63fff700 1 -- 172.21.15.46:0/1077580236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64012430 con 0x7f2f6c06ed10 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 -- 172.21.15.46:0/1077580236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 msgr2=0x7f2f6c0c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 --2- 172.21.15.46:0/1077580236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c6d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06c360 tx=0x7f2f64009780 comp rx=0 tx=0).stop 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 -- 172.21.15.46:0/1077580236 shutdown_connections 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 --2- 172.21.15.46:0/1077580236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c6d80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 --2- 172.21.15.46:0/1077580236 >> v2:172.21.15.132:3300/0 conn(0x7f2f6c06e8a0 0x7f2f6c0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 -- 172.21.15.46:0/1077580236 >> 172.21.15.46:0/1077580236 conn(0x7f2f6c0cbe30 msgr2=0x7f2f6c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 -- 172.21.15.46:0/1077580236 shutdown_connections 2023-10-06T11:18:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 -- 172.21.15.46:0/1077580236 wait complete. 2023-10-06T11:18:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.927+0000 7f2f7386e700 1 Processor -- start 2023-10-06T11:18:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 -- start start 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c06e8a0 0x7f2f6c13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0c6470 0x7f2f6c0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0ab140 con 0x7f2f6c06ed10 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0aa530 con 0x7f2f6c0c6470 2023-10-06T11:18:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7386e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0aa810 con 0x7f2f6c06e8a0 2023-10-06T11:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44826/0 (socket says 172.21.15.46:44826) 2023-10-06T11:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 -- 172.21.15.46:0/3989437757 learned_addr learned my addr 172.21.15.46:0/3989437757 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f7160a700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c06e8a0 0x7f2f6c13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f71e0b700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0c6470 0x7f2f6c0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 -- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c06e8a0 msgr2=0x7f2f6c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c06e8a0 0x7f2f6c13c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 -- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0c6470 msgr2=0x7f2f6c0c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0c6470 0x7f2f6c0c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.931+0000 7f2f70e09700 1 -- 172.21.15.46:0/3989437757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c009420 con 0x7f2f6c06ed10 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f70e09700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00ec80 tx=0x7f2f5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00c850 con 0x7f2f6c06ed10 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00c9b0 con 0x7f2f6c06ed10 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f64009a20 con 0x7f2f6c06ed10 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00cd30 con 0x7f2f6c06ed10 2023-10-06T11:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c0aadd0 con 0x7f2f6c06ed10 2023-10-06T11:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c0660f0 con 0x7f2f6c06ed10 2023-10-06T11:18:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.935+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2f5c013680 con 0x7f2f6c06ed10 2023-10-06T11:18:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.939+0000 7f2f617fa700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 0x7f2f50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.939+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f5c07cf50 con 0x7f2f6c06ed10 2023-10-06T11:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.939+0000 7f2f7160a700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 0x7f2f50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.939+0000 7f2f7160a700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 0x7f2f50062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f68007130 tx=0x7f2f68007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:51.947+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f5c050dd0 con 0x7f2f6c06ed10 2023-10-06T11:18:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.119+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f6c04b5c0 con 0x7f2f6c06ed10 2023-10-06T11:18:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.119+0000 7f2f617fa700 1 -- 172.21.15.46:0/3989437757 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f5c050960 con 0x7f2f6c06ed10 2023-10-06T11:18:52.125 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.123+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 msgr2=0x7f2f50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.123+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 0x7f2f50062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f68007130 tx=0x7f2f68007eb0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 msgr2=0x7f2f6c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00ec80 tx=0x7f2f5c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 shutdown_connections 2023-10-06T11:18:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f50060550 0x7f2f50062a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0c6470 0x7f2f6c0c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f6c06ed10 0x7f2f6c0c54a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 --2- 172.21.15.46:0/3989437757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f6c06e8a0 0x7f2f6c13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 >> 172.21.15.46:0/3989437757 conn(0x7f2f6c0cbe30 msgr2=0x7f2f6c15c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 shutdown_connections 2023-10-06T11:18:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.127+0000 7f2f7386e700 1 -- 172.21.15.46:0/3989437757 wait complete. 2023-10-06T11:18:52.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.307+0000 7f0ab2105700 1 Processor -- start 2023-10-06T11:18:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 -- start start 2023-10-06T11:18:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0aac0697e0 0x7f0aac15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac0ce470 con 0x7f0aac069410 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac15a890 con 0x7f0aac0697e0 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.311+0000 7f0ab2105700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac069c40 con 0x7f0aac159420 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aab7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0aac069410 0x7f0aac158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50354/0 (socket says 172.21.15.46:50354) 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aab7fe700 1 -- 172.21.15.46:0/3231621697 learned_addr learned my addr 172.21.15.46:0/3231621697 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 --2- 172.21.15.46:0/3231621697 >> v2:172.21.15.46:3300/0 conn(0x7f0aac0697e0 0x7f0aac15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aaaffd700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 -- 172.21.15.46:0/3231621697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 msgr2=0x7f0aac15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 -- 172.21.15.46:0/3231621697 >> v1:172.21.15.46:6789/0 conn(0x7f0aac069410 legacy=0x7f0aac158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aaa7fc700 1 -- 172.21.15.46:0/3231621697 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481658377 0 0) 0x7f0aac0ce470 con 0x7f0aac069410 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 -- 172.21.15.46:0/3231621697 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac15d470 con 0x7f0aac0697e0 2023-10-06T11:18:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aabfff700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac0697e0 0x7f0aac15a350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e60 tx=0x7f0aa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431cdf712cb0b51a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aaa7fc700 1 -- 172.21.15.46:0/3231621697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400e070 con 0x7f0aac0697e0 2023-10-06T11:18:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aaa7fc700 1 -- 172.21.15.46:0/3231621697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa400bb60 con 0x7f0aac0697e0 2023-10-06T11:18:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0aaa7fc700 1 -- 172.21.15.46:0/3231621697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4012430 con 0x7f0aac0697e0 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 -- 172.21.15.46:0/3231621697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac0697e0 msgr2=0x7f0aac15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac0697e0 0x7f0aac15a350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e60 tx=0x7f0aa4009780 comp rx=0 tx=0).stop 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 -- 172.21.15.46:0/3231621697 shutdown_connections 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac0697e0 0x7f0aac15a350 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 --2- 172.21.15.46:0/3231621697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 -- 172.21.15.46:0/3231621697 >> 172.21.15.46:0/3231621697 conn(0x7f0aac0d0210 msgr2=0x7f0aac0d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 -- 172.21.15.46:0/3231621697 shutdown_connections 2023-10-06T11:18:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.315+0000 7f0ab2105700 1 -- 172.21.15.46:0/3231621697 wait complete. 2023-10-06T11:18:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.319+0000 7f0ab2105700 1 Processor -- start 2023-10-06T11:18:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.319+0000 7f0ab2105700 1 -- start start 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac0697e0 0x7f0aac0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac14e5e0 con 0x7f0aac069410 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0c6900 con 0x7f0aac0697e0 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac16d4a0 con 0x7f0aac159420 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44832/0 (socket says 172.21.15.46:44832) 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 -- 172.21.15.46:0/38851898 learned_addr learned my addr 172.21.15.46:0/38851898 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:52.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aabfff700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aaaffd700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac0697e0 0x7f0aac0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 -- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 msgr2=0x7f0aac16d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac16d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 -- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac0697e0 msgr2=0x7f0aac0c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac0697e0 0x7f0aac0c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 -- 172.21.15.46:0/38851898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009a20 con 0x7f0aac069410 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0aab7fe700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c810 tx=0x7f0a9c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c005b30 con 0x7f0aac069410 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c005c90 con 0x7f0aac069410 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c007300 con 0x7f0aac069410 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac16de70 con 0x7f0aac069410 2023-10-06T11:18:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.323+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac0c7e90 con 0x7f0aac069410 2023-10-06T11:18:52.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.327+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac088870 con 0x7f0aac069410 2023-10-06T11:18:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.327+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0a9c01b3b0 con 0x7f0aac069410 2023-10-06T11:18:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.327+0000 7f0a937fe700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 0x7f0a98062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.327+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0a9c07bf20 con 0x7f0aac069410 2023-10-06T11:18:52.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.331+0000 7f0aaaffd700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 0x7f0a98062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.331+0000 7f0aaaffd700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 0x7f0a98062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ca0 tx=0x7f0aa0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.335+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0a9c04c310 con 0x7f0aac069410 2023-10-06T11:18:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0aac15d750 con 0x7f0aac069410 2023-10-06T11:18:52.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0a937fe700 1 -- 172.21.15.46:0/38851898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0a9c04f930 con 0x7f0aac069410 2023-10-06T11:18:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 msgr2=0x7f0a98062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 0x7f0a98062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ca0 tx=0x7f0aa0008040 comp rx=0 tx=0).stop 2023-10-06T11:18:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 msgr2=0x7f0aac0c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c810 tx=0x7f0a9c007df0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 shutdown_connections 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0a98060550 0x7f0a98062a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0aac159420 0x7f0aac16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.507+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac0697e0 0x7f0aac0c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.511+0000 7f0ab2105700 1 --2- 172.21.15.46:0/38851898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aac069410 0x7f0aac0c75b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.511+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 >> 172.21.15.46:0/38851898 conn(0x7f0aac0d0210 msgr2=0x7f0aac0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.511+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 shutdown_connections 2023-10-06T11:18:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.511+0000 7f0ab2105700 1 -- 172.21.15.46:0/38851898 wait complete. 2023-10-06T11:18:52.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 Processor -- start 2023-10-06T11:18:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 -- start start 2023-10-06T11:18:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03e00b2790 0x7f03e00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00cc510 con 0x7f03e00b23c0 2023-10-06T11:18:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00b2be0 con 0x7f03e00cd080 2023-10-06T11:18:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03e5345700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e00cd500 con 0x7f03e00b2790 2023-10-06T11:18:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03de7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03e00b2790 0x7f03e00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03de7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03e00b2790 0x7f03e00ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44834/0 (socket says 172.21.15.46:44834) 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03de7fc700 1 -- 172.21.15.46:0/3866849702 learned_addr learned my addr 172.21.15.46:0/3866849702 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.699+0000 7f03deffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03e00b23c0 0x7f03e00cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50356/0 (socket says 172.21.15.46:50356) 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03de7fc700 1 -- 172.21.15.46:0/3866849702 >> v1:172.21.15.132:6789/0 conn(0x7f03e00cd080 legacy=0x7f03e00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03de7fc700 1 -- 172.21.15.46:0/3866849702 >> v1:172.21.15.46:6789/0 conn(0x7f03e00b23c0 legacy=0x7f03e00cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03de7fc700 1 -- 172.21.15.46:0/3866849702 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00cb900 con 0x7f03e00b2790 2023-10-06T11:18:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03de7fc700 1 --2- 172.21.15.46:0/3866849702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00b2790 0x7f03e00ccb40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03d40099b0 tx=0x7f03d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936330135c4842dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03ddffb700 1 -- 172.21.15.46:0/3866849702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400b860 con 0x7f03e00b2790 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03ddffb700 1 -- 172.21.15.46:0/3866849702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d400b9c0 con 0x7f03e00b2790 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03ddffb700 1 -- 172.21.15.46:0/3866849702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400bd60 con 0x7f03e00b2790 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 -- 172.21.15.46:0/3866849702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00b2790 msgr2=0x7f03e00ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 --2- 172.21.15.46:0/3866849702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00b2790 0x7f03e00ccb40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03d40099b0 tx=0x7f03d4009250 comp rx=0 tx=0).stop 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 -- 172.21.15.46:0/3866849702 shutdown_connections 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 --2- 172.21.15.46:0/3866849702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00b2790 0x7f03e00ccb40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 -- 172.21.15.46:0/3866849702 >> 172.21.15.46:0/3866849702 conn(0x7f03e0099480 msgr2=0x7f03e006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 -- 172.21.15.46:0/3866849702 shutdown_connections 2023-10-06T11:18:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.703+0000 7f03e5345700 1 -- 172.21.15.46:0/3866849702 wait complete. 2023-10-06T11:18:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 Processor -- start 2023-10-06T11:18:52.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 -- start start 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03e00b23c0 0x7f03e0169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03e00b2790 0x7f03e01512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015ca50 con 0x7f03e00cd080 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015b7d0 con 0x7f03e00b2790 2023-10-06T11:18:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03e5345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015abc0 con 0x7f03e00b23c0 2023-10-06T11:18:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44846/0 (socket says 172.21.15.46:44846) 2023-10-06T11:18:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 -- 172.21.15.46:0/621758289 learned_addr learned my addr 172.21.15.46:0/621758289 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03deffd700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03e00b23c0 0x7f03e0169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03de7fc700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03e00b2790 0x7f03e01512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 -- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03e00b23c0 msgr2=0x7f03e0169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03e00b23c0 0x7f03e0169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 -- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03e00b2790 msgr2=0x7f03e01512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03e00b2790 0x7f03e01512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.707+0000 7f03df7fe700 1 -- 172.21.15.46:0/621758289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40094f0 con 0x7f03e00cd080 2023-10-06T11:18:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03df7fe700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c8e0 tx=0x7f03d80016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8012630 con 0x7f03e00cd080 2023-10-06T11:18:52.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d80127b0 con 0x7f03e00cd080 2023-10-06T11:18:52.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e015aea0 con 0x7f03e00cd080 2023-10-06T11:18:52.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8012b70 con 0x7f03e00cd080 2023-10-06T11:18:52.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e015b460 con 0x7f03e00cd080 2023-10-06T11:18:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e00660f0 con 0x7f03e00cd080 2023-10-06T11:18:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.711+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f03d8003030 con 0x7f03e00cd080 2023-10-06T11:18:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.715+0000 7f03c6ffd700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 0x7f03cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.715+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f03d807c9b0 con 0x7f03e00cd080 2023-10-06T11:18:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.715+0000 7f03deffd700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 0x7f03cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:52.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.715+0000 7f03deffd700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 0x7f03cc062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f03d0002930 tx=0x7f03d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:52.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.723+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f03d804cda0 con 0x7f03e00cd080 2023-10-06T11:18:52.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.891+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03e00cbbe0 con 0x7f03e00cd080 2023-10-06T11:18:52.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.895+0000 7f03c6ffd700 1 -- 172.21.15.46:0/621758289 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03d80503c0 con 0x7f03e00cd080 2023-10-06T11:18:52.899 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.895+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 msgr2=0x7f03cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.895+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 0x7f03cc062a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f03d0002930 tx=0x7f03d0009040 comp rx=0 tx=0).stop 2023-10-06T11:18:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.895+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 msgr2=0x7f03e014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.895+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c8e0 tx=0x7f03d80016e0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 shutdown_connections 2023-10-06T11:18:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03cc060550 0x7f03cc062a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03e00cd080 0x7f03e014e7d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03e00b2790 0x7f03e01512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 --2- 172.21.15.46:0/621758289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03e00b23c0 0x7f03e0169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 >> 172.21.15.46:0/621758289 conn(0x7f03e0099480 msgr2=0x7f03e006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 shutdown_connections 2023-10-06T11:18:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:52.899+0000 7f03e5345700 1 -- 172.21.15.46:0/621758289 wait complete. 2023-10-06T11:18:52.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.079+0000 7f043f46b700 1 Processor -- start 2023-10-06T11:18:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.079+0000 7f043f46b700 1 -- start start 2023-10-06T11:18:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043f46b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04380adbe0 0x7f04380d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043f46b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 0x7f04380b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043f46b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380b1740 con 0x7f04380d0d60 2023-10-06T11:18:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043f46b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0438159740 con 0x7f04380adbe0 2023-10-06T11:18:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043f46b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380afd20 con 0x7f04380b0dc0 2023-10-06T11:18:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.083+0000 7f043dc68700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f04380d0d60 0x7f04380b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39874/0 (socket says 172.21.15.46:39874) 2023-10-06T11:18:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043dc68700 1 -- 172.21.15.46:0/688657802 learned_addr learned my addr 172.21.15.46:0/688657802 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043e469700 1 --2- 172.21.15.46:0/688657802 >> v2:172.21.15.132:3300/0 conn(0x7f04380adbe0 0x7f04380d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 --2- 172.21.15.46:0/688657802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 0x7f04380b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043d467700 1 -- 172.21.15.46:0/688657802 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034335899 0 0) 0x7f0428001110 con 0x7f04380d0d60 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043d467700 1 -- 172.21.15.46:0/688657802 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0424003570 con 0x7f04380d0d60 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 -- 172.21.15.46:0/688657802 >> v2:172.21.15.132:3300/0 conn(0x7f04380adbe0 msgr2=0x7f04380d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 --2- 172.21.15.46:0/688657802 >> v2:172.21.15.132:3300/0 conn(0x7f04380adbe0 0x7f04380d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 -- 172.21.15.46:0/688657802 >> v1:172.21.15.46:6789/0 conn(0x7f04380d0d60 legacy=0x7f04380b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 -- 172.21.15.46:0/688657802 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380b0000 con 0x7f04380b0dc0 2023-10-06T11:18:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043ec6a700 1 --2- 172.21.15.46:0/688657802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 0x7f04380b1200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0430009e60 tx=0x7f0430009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd60885863b026a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043d467700 1 -- 172.21.15.46:0/688657802 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000e070 con 0x7f04380b0dc0 2023-10-06T11:18:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043d467700 1 -- 172.21.15.46:0/688657802 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043000bab0 con 0x7f04380b0dc0 2023-10-06T11:18:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043d467700 1 -- 172.21.15.46:0/688657802 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000be70 con 0x7f04380b0dc0 2023-10-06T11:18:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043f46b700 1 -- 172.21.15.46:0/688657802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 msgr2=0x7f04380b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.087+0000 7f043f46b700 1 --2- 172.21.15.46:0/688657802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 0x7f04380b1200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0430009e60 tx=0x7f0430009780 comp rx=0 tx=0).stop 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 -- 172.21.15.46:0/688657802 shutdown_connections 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 --2- 172.21.15.46:0/688657802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380b0dc0 0x7f04380b1200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 --2- 172.21.15.46:0/688657802 >> v2:172.21.15.132:3300/0 conn(0x7f04380adbe0 0x7f04380d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 -- 172.21.15.46:0/688657802 >> 172.21.15.46:0/688657802 conn(0x7f043806ef60 msgr2=0x7f04380d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 -- 172.21.15.46:0/688657802 shutdown_connections 2023-10-06T11:18:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 -- 172.21.15.46:0/688657802 wait complete. 2023-10-06T11:18:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 Processor -- start 2023-10-06T11:18:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.091+0000 7f043f46b700 1 -- start start 2023-10-06T11:18:53.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380adbe0 0x7f043814c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 0x7f0438068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043815c740 con 0x7f04380b0dc0 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438153bc0 con 0x7f04380d0d60 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0438153ea0 con 0x7f04380adbe0 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50852/0 (socket says 172.21.15.46:50852) 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 -- 172.21.15.46:0/1649058154 learned_addr learned my addr 172.21.15.46:0/1649058154 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043e469700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380adbe0 0x7f043814c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043ec6a700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 0x7f0438068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 -- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380adbe0 msgr2=0x7f043814c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380adbe0 0x7f043814c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 -- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 msgr2=0x7f0438068990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 0x7f0438068990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 -- 172.21.15.46:0/1649058154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430009a20 con 0x7f04380b0dc0 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043ec6a700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 0x7f0438068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043dc68700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f042800c7c0 tx=0x7f0428007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428005b30 con 0x7f04380b0dc0 2023-10-06T11:18:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428005c90 con 0x7f04380b0dc0 2023-10-06T11:18:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428007320 con 0x7f04380b0dc0 2023-10-06T11:18:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438154180 con 0x7f04380b0dc0 2023-10-06T11:18:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.095+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043814cc40 con 0x7f04380b0dc0 2023-10-06T11:18:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.099+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438064590 con 0x7f04380b0dc0 2023-10-06T11:18:53.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.099+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f042801a360 con 0x7f04380b0dc0 2023-10-06T11:18:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.103+0000 7f042e7fc700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 0x7f041c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.103+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f042807bc50 con 0x7f04380b0dc0 2023-10-06T11:18:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.103+0000 7f043e469700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 0x7f041c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.103+0000 7f043e469700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 0x7f041c062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d10 tx=0x7f04340091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.107+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f042804c040 con 0x7f04380b0dc0 2023-10-06T11:18:53.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.279+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04380b1740 con 0x7f04380b0dc0 2023-10-06T11:18:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.279+0000 7f042e7fc700 1 -- 172.21.15.46:0/1649058154 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04380b1740 con 0x7f04380b0dc0 2023-10-06T11:18:53.285 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 msgr2=0x7f041c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 0x7f041c062a00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d10 tx=0x7f04340091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 msgr2=0x7f0438068450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f042800c7c0 tx=0x7f0428007c10 comp rx=0 tx=0).stop 2023-10-06T11:18:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 shutdown_connections 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060550 0x7f041c062a00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04380d0d60 0x7f0438068990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04380b0dc0 0x7f0438068450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 --2- 172.21.15.46:0/1649058154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04380adbe0 0x7f043814c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 >> 172.21.15.46:0/1649058154 conn(0x7f043806ef60 msgr2=0x7f04380b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 shutdown_connections 2023-10-06T11:18:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.283+0000 7f043f46b700 1 -- 172.21.15.46:0/1649058154 wait complete. 2023-10-06T11:18:53.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:53.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.467+0000 7f9f27cae700 1 Processor -- start 2023-10-06T11:18:53.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 -- start start 2023-10-06T11:18:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f20161470 0x7f9f20161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 0x7f9f200caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200c18f0 con 0x7f9f200c1d80 2023-10-06T11:18:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200c2660 con 0x7f9f200c21e0 2023-10-06T11:18:53.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f27cae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200c0ce0 con 0x7f9f20161470 2023-10-06T11:18:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f25249700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f200c1d80 0x7f9f2006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39876/0 (socket says 172.21.15.46:39876) 2023-10-06T11:18:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f25249700 1 -- 172.21.15.46:0/1284375959 learned_addr learned my addr 172.21.15.46:0/1284375959 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f2624b700 1 --2- 172.21.15.46:0/1284375959 >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 0x7f9f200caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.471+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/1284375959 >> v2:172.21.15.132:3300/0 conn(0x7f9f20161470 0x7f9f20161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f24a48700 1 -- 172.21.15.46:0/1284375959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048835451 0 0) 0x7f9f200c18f0 con 0x7f9f200c1d80 2023-10-06T11:18:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f24a48700 1 -- 172.21.15.46:0/1284375959 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c003570 con 0x7f9f200c1d80 2023-10-06T11:18:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f25a4a700 1 -- 172.21.15.46:0/1284375959 >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 msgr2=0x7f9f200caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/1284375959 >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 0x7f9f200caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f25a4a700 1 -- 172.21.15.46:0/1284375959 >> v1:172.21.15.46:6789/0 conn(0x7f9f200c1d80 legacy=0x7f9f2006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f25a4a700 1 -- 172.21.15.46:0/1284375959 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f200c0fc0 con 0x7f9f20161470 2023-10-06T11:18:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f2624b700 1 --2- 172.21.15.46:0/1284375959 >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 0x7f9f200caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:53.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/1284375959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f20161470 0x7f9f20161840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f20063c10 tx=0x7f9f1c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d123a289b2d8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f24a48700 1 -- 172.21.15.46:0/1284375959 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c005c10 con 0x7f9f20161470 2023-10-06T11:18:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f24a48700 1 -- 172.21.15.46:0/1284375959 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c005d70 con 0x7f9f20161470 2023-10-06T11:18:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f24a48700 1 -- 172.21.15.46:0/1284375959 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c006260 con 0x7f9f20161470 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 -- 172.21.15.46:0/1284375959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f20161470 msgr2=0x7f9f20161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 --2- 172.21.15.46:0/1284375959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f20161470 0x7f9f20161840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f20063c10 tx=0x7f9f1c009120 comp rx=0 tx=0).stop 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 -- 172.21.15.46:0/1284375959 shutdown_connections 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 --2- 172.21.15.46:0/1284375959 >> v2:172.21.15.46:3300/0 conn(0x7f9f200c21e0 0x7f9f200caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 --2- 172.21.15.46:0/1284375959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f20161470 0x7f9f20161840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 -- 172.21.15.46:0/1284375959 >> 172.21.15.46:0/1284375959 conn(0x7f9f200cbe30 msgr2=0x7f9f200cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 -- 172.21.15.46:0/1284375959 shutdown_connections 2023-10-06T11:18:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.475+0000 7f9f27cae700 1 -- 172.21.15.46:0/1284375959 wait complete. 2023-10-06T11:18:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.479+0000 7f9f27cae700 1 Processor -- start 2023-10-06T11:18:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.479+0000 7f9f27cae700 1 -- start start 2023-10-06T11:18:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.479+0000 7f9f27cae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f200c21e0 0x7f9f2015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015d1e0 con 0x7f9f20161470 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015ff70 con 0x7f9f200c1d80 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20160250 con 0x7f9f200c21e0 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f2624b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43788/0 (socket says 172.21.15.46:43788) 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f2624b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50868/0 (socket says 172.21.15.46:50868) 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 -- 172.21.15.46:0/2164859464 learned_addr learned my addr 172.21.15.46:0/2164859464 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25249700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f200c21e0 0x7f9f2015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 -- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f200c21e0 msgr2=0x7f9f2015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f200c21e0 0x7f9f2015f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 -- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 msgr2=0x7f9f2016d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 -- 172.21.15.46:0/2164859464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009a20 con 0x7f9f200c1d80 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f2624b700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f25a4a700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009160 tx=0x7f9f1c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c01d8e0 con 0x7f9f200c1d80 2023-10-06T11:18:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c01da40 con 0x7f9f200c1d80 2023-10-06T11:18:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c01de00 con 0x7f9f200c1d80 2023-10-06T11:18:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0093c0 con 0x7f9f200c1d80 2023-10-06T11:18:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.483+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2015ce70 con 0x7f9f200c1d80 2023-10-06T11:18:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.487+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f20063f90 con 0x7f9f200c1d80 2023-10-06T11:18:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.487+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9f1c01b690 con 0x7f9f200c1d80 2023-10-06T11:18:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.491+0000 7f9f15ffb700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 0x7f9f04062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.491+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f1c07e5b0 con 0x7f9f200c1d80 2023-10-06T11:18:53.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.491+0000 7f9f25249700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 0x7f9f04062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.491+0000 7f9f25249700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 0x7f9f04062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f20068bc0 tx=0x7f9f10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.499+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f1c04ea60 con 0x7f9f200c1d80 2023-10-06T11:18:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.667+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f200c12a0 con 0x7f9f200c1d80 2023-10-06T11:18:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.671+0000 7f9f15ffb700 1 -- 172.21.15.46:0/2164859464 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f9f1c02a090 con 0x7f9f200c1d80 2023-10-06T11:18:53.676 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:53.677 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.675+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 msgr2=0x7f9f04062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.675+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 0x7f9f04062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f20068bc0 tx=0x7f9f10009040 comp rx=0 tx=0).stop 2023-10-06T11:18:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.675+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 msgr2=0x7f9f2015c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.675+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009160 tx=0x7f9f1c016000 comp rx=0 tx=0).stop 2023-10-06T11:18:53.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 shutdown_connections 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f04060580 0x7f9f04062a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f20161470 0x7f9f2016d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f200c21e0 0x7f9f2015f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 --2- 172.21.15.46:0/2164859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f200c1d80 0x7f9f2015c2d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 >> 172.21.15.46:0/2164859464 conn(0x7f9f200cbe30 msgr2=0x7f9f200ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 shutdown_connections 2023-10-06T11:18:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.679+0000 7f9f27cae700 1 -- 172.21.15.46:0/2164859464 wait complete. 2023-10-06T11:18:53.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:53.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.863+0000 7fac51f63700 1 Processor -- start 2023-10-06T11:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 -- start start 2023-10-06T11:18:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac4c0af030 0x7fac4c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0b04e0 con 0x7fac4c0abc40 2023-10-06T11:18:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0afbb0 con 0x7fac4c0af030 2023-10-06T11:18:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.867+0000 7fac51f63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0afe90 con 0x7fac4c15aba0 2023-10-06T11:18:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac4c0af030 0x7fac4c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51762700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51762700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43794/0 (socket says 172.21.15.46:43794) 2023-10-06T11:18:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51762700 1 -- 172.21.15.46:0/58142487 learned_addr learned my addr 172.21.15.46:0/58142487 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 -- 172.21.15.46:0/58142487 >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 msgr2=0x7fac4c15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 --2- 172.21.15.46:0/58142487 >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 -- 172.21.15.46:0/58142487 >> v1:172.21.15.132:6789/0 conn(0x7fac4c0abc40 legacy=0x7fac4c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 -- 172.21.15.46:0/58142487 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0b0170 con 0x7fac4c0af030 2023-10-06T11:18:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51762700 1 --2- 172.21.15.46:0/58142487 >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4bfff700 1 --2- 172.21.15.46:0/58142487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0af030 0x7fac4c0b0970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06d9c0 tx=0x7fac380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1dff09462ea639 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4b7fe700 1 -- 172.21.15.46:0/58142487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38011630 con 0x7fac4c0af030 2023-10-06T11:18:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4b7fe700 1 -- 172.21.15.46:0/58142487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38011790 con 0x7fac4c0af030 2023-10-06T11:18:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac4b7fe700 1 -- 172.21.15.46:0/58142487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38011b30 con 0x7fac4c0af030 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 -- 172.21.15.46:0/58142487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0af030 msgr2=0x7fac4c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 --2- 172.21.15.46:0/58142487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0af030 0x7fac4c0b0970 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06d9c0 tx=0x7fac380091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 -- 172.21.15.46:0/58142487 shutdown_connections 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 --2- 172.21.15.46:0/58142487 >> v2:172.21.15.132:3300/0 conn(0x7fac4c15aba0 0x7fac4c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 --2- 172.21.15.46:0/58142487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0af030 0x7fac4c0b0970 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 -- 172.21.15.46:0/58142487 >> 172.21.15.46:0/58142487 conn(0x7fac4c0ccd50 msgr2=0x7fac4c0cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 -- 172.21.15.46:0/58142487 shutdown_connections 2023-10-06T11:18:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.871+0000 7fac51f63700 1 -- 172.21.15.46:0/58142487 wait complete. 2023-10-06T11:18:53.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 Processor -- start 2023-10-06T11:18:53.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 -- start start 2023-10-06T11:18:53.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 0x7fac4c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 0x7fac4c16e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c159890 con 0x7fac4c0abc40 2023-10-06T11:18:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c15a4e0 con 0x7fac4c15aba0 2023-10-06T11:18:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51f63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c14f2a0 con 0x7fac4c0af030 2023-10-06T11:18:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac50f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac50f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50898/0 (socket says 172.21.15.46:50898) 2023-10-06T11:18:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac50f61700 1 -- 172.21.15.46:0/1098280648 learned_addr learned my addr 172.21.15.46:0/1098280648 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac51762700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 0x7fac4c16e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.875+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 0x7fac4c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 -- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 msgr2=0x7fac4c16e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 0x7fac4c16e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 -- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 msgr2=0x7fac4c0ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 -- 172.21.15.46:0/1098280648 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009ae0 con 0x7fac4c0af030 2023-10-06T11:18:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac50f61700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac51762700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 0x7fac4c16e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 0x7fac4c14ed60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac4c070ee0 tx=0x7fac38017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38000d10 con 0x7fac4c0af030 2023-10-06T11:18:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38011b10 con 0x7fac4c0af030 2023-10-06T11:18:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38025660 con 0x7fac4c0af030 2023-10-06T11:18:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac38009420 con 0x7fac4c0af030 2023-10-06T11:18:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0ad880 con 0x7fac4c0af030 2023-10-06T11:18:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.879+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0ad6e0 con 0x7fac4c0af030 2023-10-06T11:18:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.883+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fac3802e450 con 0x7fac4c0af030 2023-10-06T11:18:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.883+0000 7fac48ff9700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 0x7fac30062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.883+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac3807e0d0 con 0x7fac4c0af030 2023-10-06T11:18:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.883+0000 7fac50f61700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 0x7fac30062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:53.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.887+0000 7fac50f61700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 0x7fac30062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0cc9a0 tx=0x7fac40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:53.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:53.895+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fac4c0ad6e0 con 0x7fac4c0af030 2023-10-06T11:18:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.063+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac4c0ad6e0 con 0x7fac4c0af030 2023-10-06T11:18:54.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.063+0000 7fac48ff9700 1 -- 172.21.15.46:0/1098280648 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac4c0ad6e0 con 0x7fac4c0af030 2023-10-06T11:18:54.070 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:54.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 msgr2=0x7fac30062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 0x7fac30062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0cc9a0 tx=0x7fac40009040 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 msgr2=0x7fac4c14ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 0x7fac4c14ed60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac4c070ee0 tx=0x7fac38017000 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 shutdown_connections 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac30060550 0x7fac30062a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac4c15aba0 0x7fac4c16e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac4c0af030 0x7fac4c14ed60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 --2- 172.21.15.46:0/1098280648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0abc40 0x7fac4c0ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 >> 172.21.15.46:0/1098280648 conn(0x7fac4c0ccd50 msgr2=0x7fac4c155710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 shutdown_connections 2023-10-06T11:18:54.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.071+0000 7fac51f63700 1 -- 172.21.15.46:0/1098280648 wait complete. 2023-10-06T11:18:54.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 Processor -- start 2023-10-06T11:18:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 -- start start 2023-10-06T11:18:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f556406d340 0x7f55640caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55640c1980 con 0x7f55640c2350 2023-10-06T11:18:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55640cb1e0 con 0x7f556406d340 2023-10-06T11:18:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.255+0000 7f556c6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55640c0d70 con 0x7f5564146eb0 2023-10-06T11:18:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f5569c8b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f55640c2350 0x7f556406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:59030/0 (socket says 172.21.15.46:59030) 2023-10-06T11:18:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f5569c8b700 1 -- 172.21.15.46:0/4055675109 learned_addr learned my addr 172.21.15.46:0/4055675109 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556ac8d700 1 --2- 172.21.15.46:0/4055675109 >> v2:172.21.15.132:3300/0 conn(0x7f556406d340 0x7f55640caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 --2- 172.21.15.46:0/4055675109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556948a700 1 -- 172.21.15.46:0/4055675109 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299669174 0 0) 0x7f55640c1980 con 0x7f55640c2350 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556948a700 1 -- 172.21.15.46:0/4055675109 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5550003570 con 0x7f55640c2350 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 -- 172.21.15.46:0/4055675109 >> v2:172.21.15.132:3300/0 conn(0x7f556406d340 msgr2=0x7f55640caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 --2- 172.21.15.46:0/4055675109 >> v2:172.21.15.132:3300/0 conn(0x7f556406d340 0x7f55640caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 -- 172.21.15.46:0/4055675109 >> v1:172.21.15.132:6789/0 conn(0x7f55640c2350 legacy=0x7f556406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 -- 172.21.15.46:0/4055675109 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640c1050 con 0x7f5564146eb0 2023-10-06T11:18:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556a48c700 1 --2- 172.21.15.46:0/4055675109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c1e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55600098e0 tx=0x7f55600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20ca32dcf2dbe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556948a700 1 -- 172.21.15.46:0/4055675109 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560011630 con 0x7f5564146eb0 2023-10-06T11:18:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556948a700 1 -- 172.21.15.46:0/4055675109 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5560011790 con 0x7f5564146eb0 2023-10-06T11:18:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556948a700 1 -- 172.21.15.46:0/4055675109 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560011b30 con 0x7f5564146eb0 2023-10-06T11:18:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556c6f0700 1 -- 172.21.15.46:0/4055675109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 msgr2=0x7f55640c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.259+0000 7f556c6f0700 1 --2- 172.21.15.46:0/4055675109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c1e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f55600098e0 tx=0x7f55600091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 -- 172.21.15.46:0/4055675109 shutdown_connections 2023-10-06T11:18:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 --2- 172.21.15.46:0/4055675109 >> v2:172.21.15.132:3300/0 conn(0x7f556406d340 0x7f55640caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 --2- 172.21.15.46:0/4055675109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c1e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 -- 172.21.15.46:0/4055675109 >> 172.21.15.46:0/4055675109 conn(0x7f55640cbe30 msgr2=0x7f55640cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 -- 172.21.15.46:0/4055675109 shutdown_connections 2023-10-06T11:18:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 -- 172.21.15.46:0/4055675109 wait complete. 2023-10-06T11:18:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.263+0000 7f556c6f0700 1 Processor -- start 2023-10-06T11:18:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 -- start start 2023-10-06T11:18:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556406d340 0x7f55640aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5564160cc0 con 0x7f55640c2350 2023-10-06T11:18:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55640af110 con 0x7f556406d340 2023-10-06T11:18:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556c6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55640afbc0 con 0x7f5564146eb0 2023-10-06T11:18:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556a48c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556406d340 0x7f55640aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50904/0 (socket says 172.21.15.46:50904) 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 -- 172.21.15.46:0/354515727 learned_addr learned my addr 172.21.15.46:0/354515727 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f556ac8d700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 -- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 msgr2=0x7f55640c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 -- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556406d340 msgr2=0x7f55640aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556406d340 0x7f55640aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.267+0000 7f5569c8b700 1 -- 172.21.15.46:0/354515727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560009420 con 0x7f55640c2350 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f5569c8b700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f555400c7c0 tx=0x7f5554007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554005cd0 con 0x7f55640c2350 2023-10-06T11:18:54.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55640b0260 con 0x7f55640c2350 2023-10-06T11:18:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554005e30 con 0x7f55640c2350 2023-10-06T11:18:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554007320 con 0x7f55640c2350 2023-10-06T11:18:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55640af600 con 0x7f55640c2350 2023-10-06T11:18:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.271+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55640b03f0 con 0x7f55640c2350 2023-10-06T11:18:54.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.275+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f555401b3b0 con 0x7f55640c2350 2023-10-06T11:18:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.275+0000 7f555a7fc700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 0x7f5548062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.275+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f555407c090 con 0x7f55640c2350 2023-10-06T11:18:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.275+0000 7f556a48c700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 0x7f5548062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.279+0000 7f556a48c700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 0x7f5548062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55600091e0 tx=0x7f5560017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.283+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f555404c480 con 0x7f55640c2350 2023-10-06T11:18:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.455+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55640660f0 con 0x7f55640c2350 2023-10-06T11:18:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.455+0000 7f555a7fc700 1 -- 172.21.15.46:0/354515727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55540170a0 con 0x7f55640c2350 2023-10-06T11:18:54.461 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 msgr2=0x7f5548062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 0x7f5548062a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55600091e0 tx=0x7f5560017000 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 msgr2=0x7f5564165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f555400c7c0 tx=0x7f5554007e00 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 shutdown_connections 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5548060550 0x7f5548062a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5564146eb0 0x7f55640c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55640c2350 0x7f5564165770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 --2- 172.21.15.46:0/354515727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556406d340 0x7f55640aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 >> 172.21.15.46:0/354515727 conn(0x7f55640cbe30 msgr2=0x7f55640c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 shutdown_connections 2023-10-06T11:18:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.459+0000 7f556c6f0700 1 -- 172.21.15.46:0/354515727 wait complete. 2023-10-06T11:18:54.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.647+0000 7fd7f40f0700 1 Processor -- start 2023-10-06T11:18:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 -- start start 2023-10-06T11:18:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7ec06e8a0 0x7fd7ec0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 0x7fd7ec0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c1bf0 con 0x7fd7ec06ed10 2023-10-06T11:18:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c72c0 con 0x7fd7ec06e8a0 2023-10-06T11:18:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f40f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0abbe0 con 0x7fd7ec0c6470 2023-10-06T11:18:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f1e8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f1e8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:39866/0 (socket says 172.21.15.46:39866) 2023-10-06T11:18:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f1e8c700 1 -- 172.21.15.46:0/1011629480 learned_addr learned my addr 172.21.15.46:0/1011629480 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f268d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 0x7fd7ec0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.651+0000 7fd7f168b700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.132:3300/0 conn(0x7fd7ec06e8a0 0x7fd7ec0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 -- 172.21.15.46:0/1011629480 >> v2:172.21.15.132:3300/0 conn(0x7fd7ec06e8a0 msgr2=0x7fd7ec0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.132:3300/0 conn(0x7fd7ec06e8a0 0x7fd7ec0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 -- 172.21.15.46:0/1011629480 >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 msgr2=0x7fd7ec0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 0x7fd7ec0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 -- 172.21.15.46:0/1011629480 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec06cf80 con 0x7fd7ec0c6470 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f268d700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 0x7fd7ec0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/1011629480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cbd20 tx=0x7fd7e8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3281b5dcfc0f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f0e8a700 1 -- 172.21.15.46:0/1011629480 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800e070 con 0x7fd7ec0c6470 2023-10-06T11:18:54.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f0e8a700 1 -- 172.21.15.46:0/1011629480 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800b970 con 0x7fd7ec0c6470 2023-10-06T11:18:54.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f0e8a700 1 -- 172.21.15.46:0/1011629480 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80092e0 con 0x7fd7ec0c6470 2023-10-06T11:18:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 -- 172.21.15.46:0/1011629480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 msgr2=0x7fd7ec0ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/1011629480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0cbd20 tx=0x7fd7e8009840 comp rx=0 tx=0).stop 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 -- 172.21.15.46:0/1011629480 shutdown_connections 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.132:3300/0 conn(0x7fd7ec06e8a0 0x7fd7ec0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/1011629480 >> v2:172.21.15.46:3300/0 conn(0x7fd7ec06ed10 0x7fd7ec0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/1011629480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0ab6a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 -- 172.21.15.46:0/1011629480 >> 172.21.15.46:0/1011629480 conn(0x7fd7ec0cbe30 msgr2=0x7fd7ec0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 -- 172.21.15.46:0/1011629480 shutdown_connections 2023-10-06T11:18:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.655+0000 7fd7f40f0700 1 -- 172.21.15.46:0/1011629480 wait complete. 2023-10-06T11:18:54.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.659+0000 7fd7f40f0700 1 Processor -- start 2023-10-06T11:18:54.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.659+0000 7fd7f40f0700 1 -- start start 2023-10-06T11:18:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ec06e8a0 0x7fd7ec13c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec167840 con 0x7fd7ec06ed10 2023-10-06T11:18:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c41d0 con 0x7fd7ec06e8a0 2023-10-06T11:18:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec0c3c10 con 0x7fd7ec0c6470 2023-10-06T11:18:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50918/0 (socket says 172.21.15.46:50918) 2023-10-06T11:18:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 -- 172.21.15.46:0/3045851680 learned_addr learned my addr 172.21.15.46:0/3045851680 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ec06e8a0 0x7fd7ec13c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f268d700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 -- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 msgr2=0x7fd7ec0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 -- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ec06e8a0 msgr2=0x7fd7ec13c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ec06e8a0 0x7fd7ec13c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 -- 172.21.15.46:0/3045851680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4009420 con 0x7fd7ec06ed10 2023-10-06T11:18:54.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f168b700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00e930 tx=0x7fd7dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc013680 con 0x7fd7ec06ed10 2023-10-06T11:18:54.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0137e0 con 0x7fd7ec06ed10 2023-10-06T11:18:54.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e8009ae0 con 0x7fd7ec06ed10 2023-10-06T11:18:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc013b80 con 0x7fd7ec06ed10 2023-10-06T11:18:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.663+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec0aec70 con 0x7fd7ec06ed10 2023-10-06T11:18:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.667+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec0660f0 con 0x7fd7ec06ed10 2023-10-06T11:18:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.667+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd7dc00c460 con 0x7fd7ec06ed10 2023-10-06T11:18:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.667+0000 7fd7e1ffb700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 0x7fd7d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.667+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd7dc07ca90 con 0x7fd7ec06ed10 2023-10-06T11:18:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.667+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 0x7fd7d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.671+0000 7fd7f1e8c700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 0x7fd7d0062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009880 tx=0x7fd7e8018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:54.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.675+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7dc04cf40 con 0x7fd7ec06ed10 2023-10-06T11:18:54.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.855+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7ec0c61e0 con 0x7fd7ec06ed10 2023-10-06T11:18:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.855+0000 7fd7e1ffb700 1 -- 172.21.15.46:0/3045851680 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7dc050560 con 0x7fd7ec06ed10 2023-10-06T11:18:54.860 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 msgr2=0x7fd7d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 0x7fd7d0062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009880 tx=0x7fd7e8018000 comp rx=0 tx=0).stop 2023-10-06T11:18:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 msgr2=0x7fd7ec0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00e930 tx=0x7fd7dc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 shutdown_connections 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7d0060550 0x7fd7d0062a00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ec0c6470 0x7fd7ec0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ec06ed10 0x7fd7ec0c49d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 --2- 172.21.15.46:0/3045851680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ec06e8a0 0x7fd7ec13c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 >> 172.21.15.46:0/3045851680 conn(0x7fd7ec0cbe30 msgr2=0x7fd7ec15c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 shutdown_connections 2023-10-06T11:18:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:54.859+0000 7fd7f40f0700 1 -- 172.21.15.46:0/3045851680 wait complete. 2023-10-06T11:18:54.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:55.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 Processor -- start 2023-10-06T11:18:55.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 -- start start 2023-10-06T11:18:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0b23c0 0x7fa10c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa10c0cae40 0x7fa10c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10c0cc510 con 0x7fa10c0ccb40 2023-10-06T11:18:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0cb900 con 0x7fa10c0cae40 2023-10-06T11:18:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa110cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0cbbe0 con 0x7fa10c0b23c0 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa109d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa10c0cae40 0x7fa10c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa10ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa10c0ccb40 0x7fa10c15cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39880/0 (socket says 172.21.15.46:39880) 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa109d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa10c0cae40 0x7fa10c0cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43856/0 (socket says 172.21.15.46:43856) 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa10ad9d700 1 -- 172.21.15.46:0/2821862295 learned_addr learned my addr 172.21.15.46:0/2821862295 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.051+0000 7fa10a59c700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0b23c0 0x7fa10c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa109d9b700 1 -- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0b23c0 msgr2=0x7fa10c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa109d9b700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0b23c0 0x7fa10c0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa109d9b700 1 -- 172.21.15.46:0/2821862295 >> v1:172.21.15.46:6789/0 conn(0x7fa10c0ccb40 legacy=0x7fa10c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa109d9b700 1 -- 172.21.15.46:0/2821862295 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0cbec0 con 0x7fa10c0cae40 2023-10-06T11:18:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa109d9b700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0cae40 0x7fa10c0cb290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1000098e0 tx=0x7fa1000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71db40aa1ba338e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa10959a700 1 -- 172.21.15.46:0/2821862295 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000b640 con 0x7fa10c0cae40 2023-10-06T11:18:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa10959a700 1 -- 172.21.15.46:0/2821862295 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000b7a0 con 0x7fa10c0cae40 2023-10-06T11:18:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa10959a700 1 -- 172.21.15.46:0/2821862295 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000bb40 con 0x7fa10c0cae40 2023-10-06T11:18:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 -- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0cae40 msgr2=0x7fa10c0cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0cae40 0x7fa10c0cb290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1000098e0 tx=0x7fa1000091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 -- 172.21.15.46:0/2821862295 shutdown_connections 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0cae40 0x7fa10c0cb290 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 --2- 172.21.15.46:0/2821862295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0b23c0 0x7fa10c0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 -- 172.21.15.46:0/2821862295 >> 172.21.15.46:0/2821862295 conn(0x7fa10c099480 msgr2=0x7fa10c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 -- 172.21.15.46:0/2821862295 shutdown_connections 2023-10-06T11:18:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.055+0000 7fa110cf0700 1 -- 172.21.15.46:0/2821862295 wait complete. 2023-10-06T11:18:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.059+0000 7fa110cf0700 1 Processor -- start 2023-10-06T11:18:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.059+0000 7fa110cf0700 1 -- start start 2023-10-06T11:18:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0b23c0 0x7fa10c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0cae40 0x7fa10c15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c15a780 con 0x7fa10c0ccb40 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c159b70 con 0x7fa10c0b23c0 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c159e50 con 0x7fa10c0cae40 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50932/0 (socket says 172.21.15.46:50932) 2023-10-06T11:18:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 -- 172.21.15.46:0/723003920 learned_addr learned my addr 172.21.15.46:0/723003920 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10a59c700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0b23c0 0x7fa10c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa109d9b700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0cae40 0x7fa10c15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 -- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0cae40 msgr2=0x7fa10c15be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0cae40 0x7fa10c15be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 -- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0b23c0 msgr2=0x7fa10c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0b23c0 0x7fa10c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 -- 172.21.15.46:0/723003920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009420 con 0x7fa10c0ccb40 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa10ad9d700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10400cc40 tx=0x7fa104001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400a9c0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10400ab20 con 0x7fa10c0ccb40 2023-10-06T11:18:55.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c15a130 con 0x7fa10c0ccb40 2023-10-06T11:18:55.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1040070f0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.063+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c0c93f0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.067+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c15a8f0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.067+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa1040074e0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.067+0000 7fa0fa7fc700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 0x7fa0ec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.067+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa10407cde0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.067+0000 7fa10a59c700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 0x7fa0ec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.071+0000 7fa10a59c700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 0x7fa0ec062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa10c15da00 tx=0x7fa0fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.075+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa10404d250 con 0x7fa10c0ccb40 2023-10-06T11:18:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.243+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa10c0660f0 con 0x7fa10c0ccb40 2023-10-06T11:18:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.247+0000 7fa0fa7fc700 1 -- 172.21.15.46:0/723003920 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa104050870 con 0x7fa10c0ccb40 2023-10-06T11:18:55.250 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:55.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 msgr2=0x7fa0ec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 0x7fa0ec062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa10c15da00 tx=0x7fa0fc008040 comp rx=0 tx=0).stop 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 msgr2=0x7fa10c169aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10400cc40 tx=0x7fa104001a50 comp rx=0 tx=0).stop 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 shutdown_connections 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0ec060550 0x7fa0ec062a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10c0ccb40 0x7fa10c169aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa10c0cae40 0x7fa10c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 --2- 172.21.15.46:0/723003920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0b23c0 0x7fa10c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 >> 172.21.15.46:0/723003920 conn(0x7fa10c099480 msgr2=0x7fa10c0cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 shutdown_connections 2023-10-06T11:18:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.251+0000 7fa110cf0700 1 -- 172.21.15.46:0/723003920 wait complete. 2023-10-06T11:18:55.314 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-06T11:18:55.314 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:18:55.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-06T11:18:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 Processor -- start 2023-10-06T11:18:55.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 -- start start 2023-10-06T11:18:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80b40bf1a0 0x7f80b40c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40c8480 con 0x7f80b40bedd0 2023-10-06T11:18:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b4069660 con 0x7f80b40bf1a0 2023-10-06T11:18:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4068ac0 con 0x7f80b4067760 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.475+0000 7f80b259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f80b40bedd0 0x7f80b4067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39886/0 (socket says 172.21.15.46:39886) 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b259c700 1 -- 172.21.15.46:0/1163720640 learned_addr learned my addr 172.21.15.46:0/1163720640 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/1163720640 >> v2:172.21.15.46:3300/0 conn(0x7f80b40bf1a0 0x7f80b40c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80a9d9b700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b1d9b700 1 -- 172.21.15.46:0/1163720640 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379133853 0 0) 0x7f80a4001110 con 0x7f80b40bedd0 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b1d9b700 1 -- 172.21.15.46:0/1163720640 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a0003570 con 0x7f80b40bedd0 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 -- 172.21.15.46:0/1163720640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 msgr2=0x7f80b40c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 -- 172.21.15.46:0/1163720640 >> v1:172.21.15.46:6789/0 conn(0x7f80b40bedd0 legacy=0x7f80b4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 -- 172.21.15.46:0/1163720640 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4068d30 con 0x7f80b40bf1a0 2023-10-06T11:18:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c7f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009e60 tx=0x7f80ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae3d39c5b27b1c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b1d9b700 1 -- 172.21.15.46:0/1163720640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00e070 con 0x7f80b40bf1a0 2023-10-06T11:18:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b1d9b700 1 -- 172.21.15.46:0/1163720640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00bab0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b1d9b700 1 -- 172.21.15.46:0/1163720640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00be70 con 0x7f80b40bf1a0 2023-10-06T11:18:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 -- 172.21.15.46:0/1163720640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 msgr2=0x7f80b40c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c7f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009e60 tx=0x7f80ac009780 comp rx=0 tx=0).stop 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 -- 172.21.15.46:0/1163720640 shutdown_connections 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c7f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 --2- 172.21.15.46:0/1163720640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 -- 172.21.15.46:0/1163720640 >> 172.21.15.46:0/1163720640 conn(0x7f80b4062ef0 msgr2=0x7f80b40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 -- 172.21.15.46:0/1163720640 shutdown_connections 2023-10-06T11:18:55.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.479+0000 7f80b359e700 1 -- 172.21.15.46:0/1163720640 wait complete. 2023-10-06T11:18:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.483+0000 7f80b359e700 1 Processor -- start 2023-10-06T11:18:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.483+0000 7f80b359e700 1 -- start start 2023-10-06T11:18:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40bb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b40bedd0 0x7f80b415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40bdfd0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40bb650 con 0x7f80b40bedd0 2023-10-06T11:18:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415e610 con 0x7f80b4067760 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50944/0 (socket says 172.21.15.46:50944) 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 -- 172.21.15.46:0/808752977 learned_addr learned my addr 172.21.15.46:0/808752977 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b259c700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80a9d9b700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b40bedd0 0x7f80b415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 -- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 msgr2=0x7f80b40bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 -- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b40bedd0 msgr2=0x7f80b415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b40bedd0 0x7f80b415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 -- 172.21.15.46:0/808752977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac009a20 con 0x7f80b40bf1a0 2023-10-06T11:18:55.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b2d9d700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80ac013000 tx=0x7f80ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac027470 con 0x7f80b40bf1a0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac0275f0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0279b0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b415e8f0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.487+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b415eeb0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.491+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b40cb530 con 0x7f80b40bf1a0 2023-10-06T11:18:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.491+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f80ac024070 con 0x7f80b40bf1a0 2023-10-06T11:18:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.491+0000 7f80aaffd700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 0x7f8094062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.491+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80ac07f2e0 con 0x7f80b40bf1a0 2023-10-06T11:18:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.491+0000 7f80b259c700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 0x7f8094062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:55.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.495+0000 7f80b259c700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 0x7f8094062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001270 tx=0x7f80a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:55.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.499+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f80ac04f770 con 0x7f80b40bf1a0 2023-10-06T11:18:55.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:55.727+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f80b4064590 con 0x7f80b40bf1a0 2023-10-06T11:18:56.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.295+0000 7f80aaffd700 1 -- 172.21.15.46:0/808752977 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f80ac052d90 con 0x7f80b40bf1a0 2023-10-06T11:18:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 msgr2=0x7f8094062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 0x7f8094062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f80a4001270 tx=0x7f80a4009040 comp rx=0 tx=0).stop 2023-10-06T11:18:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 msgr2=0x7f80b40c06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80ac013000 tx=0x7f80ac019000 comp rx=0 tx=0).stop 2023-10-06T11:18:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 shutdown_connections 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8094060550 0x7f8094062a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80b40bf1a0 0x7f80b40c06b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80b40bedd0 0x7f80b415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 --2- 172.21.15.46:0/808752977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80b4067760 0x7f80b40bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 >> 172.21.15.46:0/808752977 conn(0x7f80b4062ef0 msgr2=0x7f80b415cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 shutdown_connections 2023-10-06T11:18:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.299+0000 7f80b359e700 1 -- 172.21.15.46:0/808752977 wait complete. 2023-10-06T11:18:56.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:18:56.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.495+0000 7f4181ce3700 1 Processor -- start 2023-10-06T11:18:56.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.495+0000 7f4181ce3700 1 -- start start 2023-10-06T11:18:56.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4181ce3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f417c0adbe0 0x7f417c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4181ce3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f417c0b0dc0 0x7f417c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4181ce3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c0b1700 con 0x7f417c0d0d60 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4181ce3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c159740 con 0x7f417c0adbe0 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4181ce3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c0afd20 con 0x7f417c0b0dc0 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4180ce1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f417c0adbe0 0x7f417c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4180ce1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f417c0adbe0 0x7f417c0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50958/0 (socket says 172.21.15.46:50958) 2023-10-06T11:18:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f4180ce1700 1 -- 172.21.15.46:0/3971561815 learned_addr learned my addr 172.21.15.46:0/3971561815 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.499+0000 7f41814e2700 1 --2- 172.21.15.46:0/3971561815 >> v2:172.21.15.132:3300/0 conn(0x7f417c0b0dc0 0x7f417c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4180ce1700 1 -- 172.21.15.46:0/3971561815 >> v2:172.21.15.132:3300/0 conn(0x7f417c0b0dc0 msgr2=0x7f417c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4180ce1700 1 --2- 172.21.15.46:0/3971561815 >> v2:172.21.15.132:3300/0 conn(0x7f417c0b0dc0 0x7f417c0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4180ce1700 1 -- 172.21.15.46:0/3971561815 >> v1:172.21.15.132:6789/0 conn(0x7f417c0d0d60 legacy=0x7f417c0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4180ce1700 1 -- 172.21.15.46:0/3971561815 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0b0000 con 0x7f417c0adbe0 2023-10-06T11:18:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4180ce1700 1 --2- 172.21.15.46:0/3971561815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0adbe0 0x7f417c0d0820 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f417c0899b0 tx=0x7f4170009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f144adb7e55b15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f417b7fe700 1 -- 172.21.15.46:0/3971561815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000b860 con 0x7f417c0adbe0 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f417b7fe700 1 -- 172.21.15.46:0/3971561815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417000b9c0 con 0x7f417c0adbe0 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f417b7fe700 1 -- 172.21.15.46:0/3971561815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000bd60 con 0x7f417c0adbe0 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 -- 172.21.15.46:0/3971561815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0adbe0 msgr2=0x7f417c0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3971561815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0adbe0 0x7f417c0d0820 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f417c0899b0 tx=0x7f4170009250 comp rx=0 tx=0).stop 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 -- 172.21.15.46:0/3971561815 shutdown_connections 2023-10-06T11:18:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3971561815 >> v2:172.21.15.132:3300/0 conn(0x7f417c0b0dc0 0x7f417c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3971561815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0adbe0 0x7f417c0d0820 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 -- 172.21.15.46:0/3971561815 >> 172.21.15.46:0/3971561815 conn(0x7f417c06ef60 msgr2=0x7f417c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:56.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 -- 172.21.15.46:0/3971561815 shutdown_connections 2023-10-06T11:18:56.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.503+0000 7f4181ce3700 1 -- 172.21.15.46:0/3971561815 wait complete. 2023-10-06T11:18:56.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.507+0000 7f4181ce3700 1 Processor -- start 2023-10-06T11:18:56.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.507+0000 7f4181ce3700 1 -- start start 2023-10-06T11:18:56.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.507+0000 7f4181ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.507+0000 7f4181ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 0x7f417c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0d0d60 0x7f417c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15b820 con 0x7f417c0d0d60 2023-10-06T11:18:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15ac10 con 0x7f417c0adbe0 2023-10-06T11:18:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15aef0 con 0x7f417c0b0dc0 2023-10-06T11:18:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43880/0 (socket says 172.21.15.46:43880) 2023-10-06T11:18:56.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 -- 172.21.15.46:0/3564479339 learned_addr learned my addr 172.21.15.46:0/3564479339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f41814e2700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0d0d60 0x7f417c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f417bfff700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 0x7f417c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 -- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 msgr2=0x7f417c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 0x7f417c15a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 -- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0d0d60 msgr2=0x7f417c068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0d0d60 0x7f417c068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 -- 172.21.15.46:0/3564479339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700094f0 con 0x7f417c0adbe0 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f417bfff700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 0x7f417c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:56.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4180ce1700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f417c088270 tx=0x7f4170017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:56.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170024440 con 0x7f417c0adbe0 2023-10-06T11:18:56.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41700245a0 con 0x7f417c0adbe0 2023-10-06T11:18:56.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c15b1d0 con 0x7f417c0adbe0 2023-10-06T11:18:56.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170024960 con 0x7f417c0adbe0 2023-10-06T11:18:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.511+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c15a7e0 con 0x7f417c0adbe0 2023-10-06T11:18:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.515+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c09b630 con 0x7f417c0adbe0 2023-10-06T11:18:56.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.515+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4170024d50 con 0x7f417c0adbe0 2023-10-06T11:18:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.515+0000 7f4178ff9700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 0x7f4160062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:56.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.515+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f417007e590 con 0x7f417c0adbe0 2023-10-06T11:18:56.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.515+0000 7f417bfff700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 0x7f4160062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.519+0000 7f417bfff700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 0x7f4160062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4168007fd0 tx=0x7f4168009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:56.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.523+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f417004e980 con 0x7f417c0adbe0 2023-10-06T11:18:56.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.695+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f417c064590 con 0x7f417c0adbe0 2023-10-06T11:18:56.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.707+0000 7f4178ff9700 1 -- 172.21.15.46:0/3564479339 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4170051fa0 con 0x7f417c0adbe0 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 msgr2=0x7f4160062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 0x7f4160062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4168007fd0 tx=0x7f4168009040 comp rx=0 tx=0).stop 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 msgr2=0x7f417c09d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f417c088270 tx=0x7f4170017000 comp rx=0 tx=0).stop 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 shutdown_connections 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4160060550 0x7f4160062a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f417c0d0d60 0x7f417c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f417c0b0dc0 0x7f417c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 --2- 172.21.15.46:0/3564479339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f417c0adbe0 0x7f417c09d6a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 >> 172.21.15.46:0/3564479339 conn(0x7f417c06ef60 msgr2=0x7f417c158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 shutdown_connections 2023-10-06T11:18:56.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:56.711+0000 7f4181ce3700 1 -- 172.21.15.46:0/3564479339 wait complete. 2023-10-06T11:18:56.736 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:18:56.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:18:56.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:18:56.821 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:18:56.822 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:18:56.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:18:56.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T11:18:56.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:18:56.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:56.915 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:18:56.943 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T11:18:56.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:18:56.943 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T11:18:57.034 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:18:57.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:57.035 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T11:18:57.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T11:18:57.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:18:57.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T11:18:57.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:18:57.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:57.123 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T11:18:57.143 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T11:18:57.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:18:57.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T11:18:57.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T11:18:57.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:18:57.232 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T11:18:57.252 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:18:57.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.415+0000 7f3aa2545700 1 Processor -- start 2023-10-06T11:18:57.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.415+0000 7f3aa2545700 1 -- start start 2023-10-06T11:18:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa2545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a9c0c5ff0 0x7f3a9c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa2545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a9c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa2545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0cc150 con 0x7f3a9c15c770 2023-10-06T11:18:57.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa2545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cfbb0 con 0x7f3a9c0c5ff0 2023-10-06T11:18:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa2545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cdbf0 con 0x7f3a9c0ccb50 2023-10-06T11:18:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3a9bfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3a9c15c770 0x7f3a9c0cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39890/0 (socket says 172.21.15.46:39890) 2023-10-06T11:18:57.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3a9bfff700 1 -- 172.21.15.46:0/556380496 learned_addr learned my addr 172.21.15.46:0/556380496 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3a937fe700 1 --2- 172.21.15.46:0/556380496 >> v2:172.21.15.46:3300/0 conn(0x7f3a9c0c5ff0 0x7f3a9c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3aa0ae2700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a9c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587281195 0 0) 0x7f3a9c0cc150 con 0x7f3a9c15c770 2023-10-06T11:18:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.419+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a88003570 con 0x7f3a9c15c770 2023-10-06T11:18:57.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008256728 0 0) 0x7f3a88003570 con 0x7f3a9c15c770 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 msgr2=0x7f3a9c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a9c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 >> v2:172.21.15.46:3300/0 conn(0x7f3a9c0c5ff0 msgr2=0x7f3a9c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 --2- 172.21.15.46:0/556380496 >> v2:172.21.15.46:3300/0 conn(0x7f3a9c0c5ff0 0x7f3a9c0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0cb540 con 0x7f3a9c15c770 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3a8c0015f0 con 0x7f3a9c15c770 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 >> v1:172.21.15.46:6789/0 conn(0x7f3a9c15c770 legacy=0x7f3a9c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a88004810 0x7f3a88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cfbb0 con 0x7f3a88003a60 2023-10-06T11:18:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cdbf0 con 0x7f3a88004810 2023-10-06T11:18:57.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8800ba60 con 0x7f3a9c0ccb50 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3aa0ae2700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a937fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a88004810 0x7f3a88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 -- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 msgr2=0x7f3a880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0ccb50 0x7f3a880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 -- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a88004810 msgr2=0x7f3a88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a88004810 0x7f3a88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 -- 172.21.15.46:0/556380496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88003570 con 0x7f3a88003a60 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a937fe700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a88004810 0x7f3a88004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a88003e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c3f0 tx=0x7f3a8c007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ae11e6b57dd086 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00cc80 con 0x7f3a88003a60 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c00cde0 con 0x7f3a88003a60 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.423+0000 7f3a9b7fe700 1 -- 172.21.15.46:0/556380496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c007320 con 0x7f3a88003a60 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- 172.21.15.46:0/556380496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 msgr2=0x7f3a88003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 --2- 172.21.15.46:0/556380496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a88003e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c3f0 tx=0x7f3a8c007920 comp rx=0 tx=0).stop 2023-10-06T11:18:57.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa0ae2700 1 -- 172.21.15.46:0/556380496 reap_dead start 2023-10-06T11:18:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- 172.21.15.46:0/556380496 shutdown_connections 2023-10-06T11:18:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- 172.21.15.46:0/556380496 >> 172.21.15.46:0/556380496 conn(0x7f3a9c099480 msgr2=0x7f3a9c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- 172.21.15.46:0/556380496 shutdown_connections 2023-10-06T11:18:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- 172.21.15.46:0/556380496 wait complete. 2023-10-06T11:18:57.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 Processor -- start 2023-10-06T11:18:57.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.427+0000 7f3aa2545700 1 -- start start 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0c9ae0 0x7f3a9c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a9c0c9f50 0x7f3a9c0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c171e90 con 0x7f3a88003a60 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c171280 con 0x7f3a9c0c9f50 2023-10-06T11:18:57.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c171560 con 0x7f3a9c0c9ae0 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50986/0 (socket says 172.21.15.46:50986) 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 -- 172.21.15.46:0/402668379 learned_addr learned my addr 172.21.15.46:0/402668379 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a937fe700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0c9ae0 0x7f3a9c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa0ae2700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a9c0c9f50 0x7f3a9c0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 -- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0c9ae0 msgr2=0x7f3a9c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0c9ae0 0x7f3a9c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 -- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a9c0c9f50 msgr2=0x7f3a9c0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a9c0c9f50 0x7f3a9c0d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 -- 172.21.15.46:0/402668379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c171840 con 0x7f3a88003a60 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a9bfff700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c011000 tx=0x7f3a8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00cc80 con 0x7f3a88003a60 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0010d0 con 0x7f3a88003a60 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c171b20 con 0x7f3a88003a60 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c001de0 con 0x7f3a88003a60 2023-10-06T11:18:57.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.431+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0b40a0 con 0x7f3a88003a60 2023-10-06T11:18:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.435+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c0ca960 con 0x7f3a88003a60 2023-10-06T11:18:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.435+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3a8c019030 con 0x7f3a88003a60 2023-10-06T11:18:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.435+0000 7f3a98ff9700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 0x7f3a7c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.435+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3a8c07df20 con 0x7f3a88003a60 2023-10-06T11:18:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.435+0000 7f3a937fe700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 0x7f3a7c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.439+0000 7f3a937fe700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 0x7f3a7c062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006dc0 tx=0x7f3a84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.443+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3a9c0ca960 con 0x7f3a88003a60 2023-10-06T11:18:57.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.647+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a9c0ca960 con 0x7f3a88003a60 2023-10-06T11:18:57.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3a98ff9700 1 -- 172.21.15.46:0/402668379 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f3a9c0ca960 con 0x7f3a88003a60 2023-10-06T11:18:57.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:57.654 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:18:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 msgr2=0x7f3a7c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 0x7f3a7c062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006dc0 tx=0x7f3a84008040 comp rx=0 tx=0).stop 2023-10-06T11:18:57.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 msgr2=0x7f3a9c153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c011000 tx=0x7f3a8c015000 comp rx=0 tx=0).stop 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 shutdown_connections 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3a7c060550 0x7f3a7c062a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a9c0c9f50 0x7f3a9c0d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a9c0c9ae0 0x7f3a9c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 --2- 172.21.15.46:0/402668379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a88003a60 0x7f3a9c153090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 >> 172.21.15.46:0/402668379 conn(0x7f3a9c099480 msgr2=0x7f3a9c15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 shutdown_connections 2023-10-06T11:18:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.651+0000 7f3aa2545700 1 -- 172.21.15.46:0/402668379 wait complete. 2023-10-06T11:18:57.660 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:18:57.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:18:57.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.839+0000 7f623242e700 1 Processor -- start 2023-10-06T11:18:57.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.839+0000 7f623242e700 1 -- start start 2023-10-06T11:18:57.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f623242e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f622c06e8a0 0x7f622c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f623242e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f622c0ad6c0 0x7f622c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f623242e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c06d460 con 0x7f622c15ab60 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f623242e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c06ed40 con 0x7f622c06e8a0 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f623242e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c0c7000 con 0x7f622c0ad6c0 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f622c06e8a0 0x7f622c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f622c06e8a0 0x7f622c06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50998/0 (socket says 172.21.15.46:50998) 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 -- 172.21.15.46:0/2033837331 learned_addr learned my addr 172.21.15.46:0/2033837331 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:57.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f622bfff700 1 --1- 172.21.15.46:0/2033837331 >> v1:172.21.15.46:6789/0 conn(0x7f622c15ab60 0x7f622c15af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39902/0 (socket says 172.21.15.46:39902) 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62309cb700 1 --2- 172.21.15.46:0/2033837331 >> v2:172.21.15.132:3300/0 conn(0x7f622c0ad6c0 0x7f622c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 -- 172.21.15.46:0/2033837331 >> v2:172.21.15.132:3300/0 conn(0x7f622c0ad6c0 msgr2=0x7f622c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 --2- 172.21.15.46:0/2033837331 >> v2:172.21.15.132:3300/0 conn(0x7f622c0ad6c0 0x7f622c0adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 -- 172.21.15.46:0/2033837331 >> v1:172.21.15.46:6789/0 conn(0x7f622c15ab60 legacy=0x7f622c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 -- 172.21.15.46:0/2033837331 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0c72c0 con 0x7f622c06e8a0 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.843+0000 7f62237fe700 1 --2- 172.21.15.46:0/2033837331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c06e8a0 0x7f622c06cc30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f622c15b640 tx=0x7f6214009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0103df6947cf9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f622b7fe700 1 -- 172.21.15.46:0/2033837331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400b860 con 0x7f622c06e8a0 2023-10-06T11:18:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f622b7fe700 1 -- 172.21.15.46:0/2033837331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621400b9c0 con 0x7f622c06e8a0 2023-10-06T11:18:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f622b7fe700 1 -- 172.21.15.46:0/2033837331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400bd60 con 0x7f622c06e8a0 2023-10-06T11:18:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 -- 172.21.15.46:0/2033837331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c06e8a0 msgr2=0x7f622c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 --2- 172.21.15.46:0/2033837331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c06e8a0 0x7f622c06cc30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f622c15b640 tx=0x7f6214009250 comp rx=0 tx=0).stop 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 -- 172.21.15.46:0/2033837331 shutdown_connections 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 --2- 172.21.15.46:0/2033837331 >> v2:172.21.15.132:3300/0 conn(0x7f622c0ad6c0 0x7f622c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 --2- 172.21.15.46:0/2033837331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c06e8a0 0x7f622c06cc30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 -- 172.21.15.46:0/2033837331 >> 172.21.15.46:0/2033837331 conn(0x7f622c0cbe30 msgr2=0x7f622c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 -- 172.21.15.46:0/2033837331 shutdown_connections 2023-10-06T11:18:57.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 -- 172.21.15.46:0/2033837331 wait complete. 2023-10-06T11:18:57.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.847+0000 7f623242e700 1 Processor -- start 2023-10-06T11:18:57.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 -- start start 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f622c0ad6c0 0x7f622c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0c3210 con 0x7f622c15ab60 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0af260 con 0x7f622c06e8a0 2023-10-06T11:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f623242e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0aef60 con 0x7f622c0ad6c0 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f622bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f622bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43920/0 (socket says 172.21.15.46:43920) 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51008/0 (socket says 172.21.15.46:51008) 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f622bfff700 1 -- 172.21.15.46:0/486379755 learned_addr learned my addr 172.21.15.46:0/486379755 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62237fe700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f622c0ad6c0 0x7f622c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 -- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f622c0ad6c0 msgr2=0x7f622c0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f622c0ad6c0 0x7f622c0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 -- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 msgr2=0x7f622c145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 -- 172.21.15.46:0/486379755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62140094f0 con 0x7f622c15ab60 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f622bfff700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.851+0000 7f62309cb700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f622400d890 tx=0x7f6224008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240125f0 con 0x7f622c15ab60 2023-10-06T11:18:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224012750 con 0x7f622c15ab60 2023-10-06T11:18:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c0cad00 con 0x7f622c15ab60 2023-10-06T11:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224012af0 con 0x7f622c15ab60 2023-10-06T11:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c0cb2c0 con 0x7f622c15ab60 2023-10-06T11:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c0af3f0 con 0x7f622c15ab60 2023-10-06T11:18:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.855+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6224013480 con 0x7f622c15ab60 2023-10-06T11:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.859+0000 7f6228ff9700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 0x7f620c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:57.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.859+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f622407cc50 con 0x7f622c15ab60 2023-10-06T11:18:57.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.859+0000 7f622bfff700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 0x7f620c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.859+0000 7f622bfff700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 0x7f620c062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f621c007fd0 tx=0x7f621c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:57.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:57.867+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f622404d040 con 0x7f622c15ab60 2023-10-06T11:18:58.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.039+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f622c063f90 con 0x7f622c15ab60 2023-10-06T11:18:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.043+0000 7f6228ff9700 1 -- 172.21.15.46:0/486379755 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f6224050660 con 0x7f622c15ab60 2023-10-06T11:18:58.048 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:18:58.048 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:18:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 msgr2=0x7f620c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 0x7f620c062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f621c007fd0 tx=0x7f621c009040 comp rx=0 tx=0).stop 2023-10-06T11:18:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 msgr2=0x7f622c0ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f622400d890 tx=0x7f6224008040 comp rx=0 tx=0).stop 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 shutdown_connections 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f620c060550 0x7f620c062a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f622c15ab60 0x7f622c0ca610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f622c0ad6c0 0x7f622c0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 --2- 172.21.15.46:0/486379755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c06e8a0 0x7f622c145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 >> 172.21.15.46:0/486379755 conn(0x7f622c0cbe30 msgr2=0x7f622c069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 shutdown_connections 2023-10-06T11:18:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.047+0000 7f623242e700 1 -- 172.21.15.46:0/486379755 wait complete. 2023-10-06T11:18:58.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:18:21.260824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:18:22.204974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:18:58.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:18:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.239+0000 7f9497aae700 1 Processor -- start 2023-10-06T11:18:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.239+0000 7f9497aae700 1 -- start start 2023-10-06T11:18:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9497aae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 0x7f94900c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9497aae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94900c2350 0x7f949006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9497aae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94900c1980 con 0x7f9490161fc0 2023-10-06T11:18:58.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9497aae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94900c0d70 con 0x7f9490146eb0 2023-10-06T11:18:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9497aae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94900c1050 con 0x7f94900c2350 2023-10-06T11:18:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f949584a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 0x7f94900c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9495049700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94900c2350 0x7f949006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f949604b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f9490161fc0 0x7f9490161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:59042/0 (socket says 172.21.15.46:59042) 2023-10-06T11:18:58.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f949604b700 1 -- 172.21.15.46:0/2829893061 learned_addr learned my addr 172.21.15.46:0/2829893061 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9495049700 1 -- 172.21.15.46:0/2829893061 >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 msgr2=0x7f94900c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9495049700 1 --2- 172.21.15.46:0/2829893061 >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 0x7f94900c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9495049700 1 -- 172.21.15.46:0/2829893061 >> v1:172.21.15.132:6789/0 conn(0x7f9490161fc0 legacy=0x7f9490161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9495049700 1 -- 172.21.15.46:0/2829893061 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900c1330 con 0x7f94900c2350 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f949584a700 1 --2- 172.21.15.46:0/2829893061 >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 0x7f94900c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.243+0000 7f9494848700 1 -- 172.21.15.46:0/2829893061 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894557521 0 0) 0x7f94900c1980 con 0x7f9490161fc0 2023-10-06T11:18:58.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9495049700 1 --2- 172.21.15.46:0/2829893061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949006cc30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94900c2e70 tx=0x7f94800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7e8eecf3bbfbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9494848700 1 -- 172.21.15.46:0/2829893061 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480011630 con 0x7f94900c2350 2023-10-06T11:18:58.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9494848700 1 -- 172.21.15.46:0/2829893061 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480011790 con 0x7f94900c2350 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9494848700 1 -- 172.21.15.46:0/2829893061 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480011b30 con 0x7f94900c2350 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 -- 172.21.15.46:0/2829893061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 msgr2=0x7f949006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 --2- 172.21.15.46:0/2829893061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949006cc30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94900c2e70 tx=0x7f94800091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 -- 172.21.15.46:0/2829893061 shutdown_connections 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 --2- 172.21.15.46:0/2829893061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949006cc30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 --2- 172.21.15.46:0/2829893061 >> v2:172.21.15.46:3300/0 conn(0x7f9490146eb0 0x7f94900c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 -- 172.21.15.46:0/2829893061 >> 172.21.15.46:0/2829893061 conn(0x7f94900cbe30 msgr2=0x7f94900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:58.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 -- 172.21.15.46:0/2829893061 shutdown_connections 2023-10-06T11:18:58.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.247+0000 7f9497aae700 1 -- 172.21.15.46:0/2829893061 wait complete. 2023-10-06T11:18:58.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 Processor -- start 2023-10-06T11:18:58.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 -- start start 2023-10-06T11:18:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9490161fc0 0x7f9490069f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94900c5790 con 0x7f9490146eb0 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490160950 con 0x7f94900c2350 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9497aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949015eec0 con 0x7f9490161fc0 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51020/0 (socket says 172.21.15.46:51020) 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 -- 172.21.15.46:0/3914294205 learned_addr learned my addr 172.21.15.46:0/3914294205 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f949604b700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9490161fc0 0x7f9490069f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f949584a700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 -- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9490161fc0 msgr2=0x7f9490069f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9490161fc0 0x7f9490069f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 -- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 msgr2=0x7f949015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.251+0000 7f9495049700 1 -- 172.21.15.46:0/3914294205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c009420 con 0x7f9490146eb0 2023-10-06T11:18:58.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9495049700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9480013000 tx=0x7f9480017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480000d10 con 0x7f9490146eb0 2023-10-06T11:18:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480011b10 con 0x7f9490146eb0 2023-10-06T11:18:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480025640 con 0x7f9490146eb0 2023-10-06T11:18:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9480009420 con 0x7f9490146eb0 2023-10-06T11:18:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949015f480 con 0x7f9490146eb0 2023-10-06T11:18:58.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.255+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94900660f0 con 0x7f9490146eb0 2023-10-06T11:18:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.259+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9480025a30 con 0x7f9490146eb0 2023-10-06T11:18:58.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.259+0000 7f9485ffb700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 0x7f9474062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.259+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f948007e0d0 con 0x7f9490146eb0 2023-10-06T11:18:58.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.259+0000 7f949584a700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 0x7f9474062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.263+0000 7f949584a700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 0x7f9474062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f948c009eb0 tx=0x7f948c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.267+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f948004e500 con 0x7f9490146eb0 2023-10-06T11:18:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.435+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94900c1610 con 0x7f9490146eb0 2023-10-06T11:18:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9485ffb700 1 -- 172.21.15.46:0/3914294205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9480051ae0 con 0x7f9490146eb0 2023-10-06T11:18:58.442 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:18:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 msgr2=0x7f9474062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 0x7f9474062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f948c009eb0 tx=0x7f948c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 msgr2=0x7f94900c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9480013000 tx=0x7f9480017000 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 shutdown_connections 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9474060550 0x7f9474062a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9490161fc0 0x7f9490069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9490146eb0 0x7f94900c4cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 --2- 172.21.15.46:0/3914294205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94900c2350 0x7f949015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.439+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 >> 172.21.15.46:0/3914294205 conn(0x7f94900cbe30 msgr2=0x7f94901459d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.443+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 shutdown_connections 2023-10-06T11:18:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.443+0000 7f9497aae700 1 -- 172.21.15.46:0/3914294205 wait complete. 2023-10-06T11:18:58.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:18:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.627+0000 7f06895cc700 1 Processor -- start 2023-10-06T11:18:58.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.627+0000 7f06895cc700 1 -- start start 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06895cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154880 0x7f06840c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06895cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0684154c50 0x7f068406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06895cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840c6cf0 con 0x7f06840c7f20 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06895cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06840c2a30 con 0x7f0684154c50 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06895cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840c83e0 con 0x7f0684154880 2023-10-06T11:18:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f0688dcb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f06840c7f20 0x7f068415aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39906/0 (socket says 172.21.15.46:39906) 2023-10-06T11:18:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f06837fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0684154c50 0x7f068406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f0688dcb700 1 -- 172.21.15.46:0/4218631278 learned_addr learned my addr 172.21.15.46:0/4218631278 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.631+0000 7f0683fff700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154880 0x7f06840c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06837fe700 1 -- 172.21.15.46:0/4218631278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154880 msgr2=0x7f06840c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06837fe700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154880 0x7f06840c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06837fe700 1 -- 172.21.15.46:0/4218631278 >> v1:172.21.15.46:6789/0 conn(0x7f06840c7f20 legacy=0x7f068415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06837fe700 1 -- 172.21.15.46:0/4218631278 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068406e5d0 con 0x7f0684154c50 2023-10-06T11:18:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f0682ffd700 1 -- 172.21.15.46:0/4218631278 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477512415 0 0) 0x7f06840c6cf0 con 0x7f06840c7f20 2023-10-06T11:18:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06837fe700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0684154c50 0x7f068406c9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06780098e0 tx=0x7f06780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659f2bc2d368de34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f0682ffd700 1 -- 172.21.15.46:0/4218631278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800b640 con 0x7f0684154c50 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f0682ffd700 1 -- 172.21.15.46:0/4218631278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800b7a0 con 0x7f0684154c50 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f0682ffd700 1 -- 172.21.15.46:0/4218631278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800bb40 con 0x7f0684154c50 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 -- 172.21.15.46:0/4218631278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0684154c50 msgr2=0x7f068406c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0684154c50 0x7f068406c9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06780098e0 tx=0x7f06780091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 -- 172.21.15.46:0/4218631278 shutdown_connections 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0684154c50 0x7f068406c9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 --2- 172.21.15.46:0/4218631278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154880 0x7f06840c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 -- 172.21.15.46:0/4218631278 >> 172.21.15.46:0/4218631278 conn(0x7f06840cbc10 msgr2=0x7f06840cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 -- 172.21.15.46:0/4218631278 shutdown_connections 2023-10-06T11:18:58.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.635+0000 7f06895cc700 1 -- 172.21.15.46:0/4218631278 wait complete. 2023-10-06T11:18:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 Processor -- start 2023-10-06T11:18:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 -- start start 2023-10-06T11:18:58.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 0x7f0684145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154c50 0x7f06840c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06840c68e0 con 0x7f06840c7f20 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0684145d90 con 0x7f0684154880 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06895cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840c60a0 con 0x7f0684154c50 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51036/0 (socket says 172.21.15.46:51036) 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 -- 172.21.15.46:0/4274334731 learned_addr learned my addr 172.21.15.46:0/4274334731 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0688dcb700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154c50 0x7f06840c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06837fe700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 0x7f0684145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 -- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154c50 msgr2=0x7f06840c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154c50 0x7f06840c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 -- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 msgr2=0x7f0684145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 0x7f0684145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 -- 172.21.15.46:0/4274334731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009420 con 0x7f06840c7f20 2023-10-06T11:18:58.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f06837fe700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 0x7f0684145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f0683fff700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f067400ec80 tx=0x7f06740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.639+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400c9d0 con 0x7f06840c7f20 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067400cb30 con 0x7f06840c7f20 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06840c6320 con 0x7f06840c7f20 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674013470 con 0x7f06840c7f20 2023-10-06T11:18:58.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06841551f0 con 0x7f06840c7f20 2023-10-06T11:18:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0684155050 con 0x7f06840c7f20 2023-10-06T11:18:58.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.643+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0674013820 con 0x7f06840c7f20 2023-10-06T11:18:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.647+0000 7f066bfff700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 0x7f0670062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:58.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.647+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f067407ce00 con 0x7f06840c7f20 2023-10-06T11:18:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.647+0000 7f06837fe700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 0x7f0670062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:58.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.647+0000 7f06837fe700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 0x7f0670062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0684155020 tx=0x7f0678017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:58.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.651+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0684155050 con 0x7f06840c7f20 2023-10-06T11:18:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.823+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0684155050 con 0x7f06840c7f20 2023-10-06T11:18:58.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.823+0000 7f066bfff700 1 -- 172.21.15.46:0/4274334731 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0684155050 con 0x7f06840c7f20 2023-10-06T11:18:58.829 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:18:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 msgr2=0x7f0670062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 0x7f0670062a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0684155020 tx=0x7f0678017000 comp rx=0 tx=0).stop 2023-10-06T11:18:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 msgr2=0x7f06840c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f067400ec80 tx=0x7f06740091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 shutdown_connections 2023-10-06T11:18:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0670060550 0x7f0670062a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0684154c50 0x7f06840c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0684154880 0x7f0684145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 --2- 172.21.15.46:0/4274334731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06840c7f20 0x7f06840c37b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 >> 172.21.15.46:0/4274334731 conn(0x7f06840cbc10 msgr2=0x7f06840af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 shutdown_connections 2023-10-06T11:18:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:58.827+0000 7f06895cc700 1 -- 172.21.15.46:0/4274334731 wait complete. 2023-10-06T11:18:58.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:18:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.019+0000 7fe58dbcc700 1 Processor -- start 2023-10-06T11:18:59.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.019+0000 7fe58dbcc700 1 -- start start 2023-10-06T11:18:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58dbcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe5880b30f0 0x7fe5880b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58dbcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880b3570 0x7fe5880b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58dbcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880afd20 con 0x7fe5880adbe0 2023-10-06T11:18:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58dbcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe588159740 con 0x7fe5880b3570 2023-10-06T11:18:59.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58dbcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880b0000 con 0x7fe5880b30f0 2023-10-06T11:18:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880b3570 0x7fe5880b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5880b3570 0x7fe5880b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51038/0 (socket says 172.21.15.46:51038) 2023-10-06T11:18:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 -- 172.21.15.46:0/94447099 learned_addr learned my addr 172.21.15.46:0/94447099 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:59.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe587fff700 1 --2- 172.21.15.46:0/94447099 >> v2:172.21.15.132:3300/0 conn(0x7fe5880b30f0 0x7fe5880b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 -- 172.21.15.46:0/94447099 >> v2:172.21.15.132:3300/0 conn(0x7fe5880b30f0 msgr2=0x7fe5880b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 --2- 172.21.15.46:0/94447099 >> v2:172.21.15.132:3300/0 conn(0x7fe5880b30f0 0x7fe5880b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.023+0000 7fe58d3cb700 1 -- 172.21.15.46:0/94447099 >> v1:172.21.15.132:6789/0 conn(0x7fe5880adbe0 legacy=0x7fe5880d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58d3cb700 1 -- 172.21.15.46:0/94447099 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880b02e0 con 0x7fe5880b3570 2023-10-06T11:18:59.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58d3cb700 1 --2- 172.21.15.46:0/94447099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880b3570 0x7fe5880b1300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5880881c0 tx=0x7fe580009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf882e88ad53d43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe5877fe700 1 -- 172.21.15.46:0/94447099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000e070 con 0x7fe5880b3570 2023-10-06T11:18:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe5877fe700 1 -- 172.21.15.46:0/94447099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58000ba20 con 0x7fe5880b3570 2023-10-06T11:18:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe5877fe700 1 -- 172.21.15.46:0/94447099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800092e0 con 0x7fe5880b3570 2023-10-06T11:18:59.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 -- 172.21.15.46:0/94447099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880b3570 msgr2=0x7fe5880b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/94447099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880b3570 0x7fe5880b1300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5880881c0 tx=0x7fe580009840 comp rx=0 tx=0).stop 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 -- 172.21.15.46:0/94447099 shutdown_connections 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/94447099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880b3570 0x7fe5880b1300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/94447099 >> v2:172.21.15.132:3300/0 conn(0x7fe5880b30f0 0x7fe5880b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 -- 172.21.15.46:0/94447099 >> 172.21.15.46:0/94447099 conn(0x7fe58806ef60 msgr2=0x7fe5880d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 -- 172.21.15.46:0/94447099 shutdown_connections 2023-10-06T11:18:59.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 -- 172.21.15.46:0/94447099 wait complete. 2023-10-06T11:18:59.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.027+0000 7fe58dbcc700 1 Processor -- start 2023-10-06T11:18:59.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 -- start start 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 0x7fe58815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5880b3570 0x7fe588153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588099ee0 con 0x7fe5880adbe0 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58815a340 con 0x7fe5880b30f0 2023-10-06T11:18:59.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588150d10 con 0x7fe5880b3570 2023-10-06T11:18:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51048/0 (socket says 172.21.15.46:51048) 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 -- 172.21.15.46:0/1867952795 learned_addr learned my addr 172.21.15.46:0/1867952795 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe587fff700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 0x7fe58815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58d3cb700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5880b3570 0x7fe588153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 -- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5880b3570 msgr2=0x7fe588153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5880b3570 0x7fe588153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 -- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 msgr2=0x7fe58815bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 0x7fe58815bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 -- 172.21.15.46:0/1867952795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009ae0 con 0x7fe5880adbe0 2023-10-06T11:18:59.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe587fff700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 0x7fe58815bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:18:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58cbca700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c510 tx=0x7fe57c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c011590 con 0x7fe5880adbe0 2023-10-06T11:18:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c011710 con 0x7fe5880adbe0 2023-10-06T11:18:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c011ad0 con 0x7fe5880adbe0 2023-10-06T11:18:59.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.031+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58815a5b0 con 0x7fe5880adbe0 2023-10-06T11:18:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.035+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58815ab70 con 0x7fe5880adbe0 2023-10-06T11:18:59.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.035+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588064590 con 0x7fe5880adbe0 2023-10-06T11:18:59.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.039+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe57c01b3b0 con 0x7fe5880adbe0 2023-10-06T11:18:59.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.039+0000 7fe584ff9700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 0x7fe56c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.039+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe57c07bf20 con 0x7fe5880adbe0 2023-10-06T11:18:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.039+0000 7fe587fff700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 0x7fe56c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.039+0000 7fe587fff700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 0x7fe56c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe588069970 tx=0x7fe574008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.043+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe57c04c310 con 0x7fe5880adbe0 2023-10-06T11:18:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.211+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5880675d0 con 0x7fe5880adbe0 2023-10-06T11:18:59.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.211+0000 7fe584ff9700 1 -- 172.21.15.46:0/1867952795 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe57c017020 con 0x7fe5880adbe0 2023-10-06T11:18:59.216 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:18:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 msgr2=0x7fe56c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 0x7fe56c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe588069970 tx=0x7fe574008040 comp rx=0 tx=0).stop 2023-10-06T11:18:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 msgr2=0x7fe5881507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c510 tx=0x7fe57c00c8d0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 shutdown_connections 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe56c060550 0x7fe56c062a00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5880b3570 0x7fe588153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5880b30f0 0x7fe58815bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 --2- 172.21.15.46:0/1867952795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5880adbe0 0x7fe5881507d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 >> 172.21.15.46:0/1867952795 conn(0x7fe58806ef60 msgr2=0x7fe588151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 shutdown_connections 2023-10-06T11:18:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.215+0000 7fe58dbcc700 1 -- 172.21.15.46:0/1867952795 wait complete. 2023-10-06T11:18:59.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:18:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.411+0000 7ff666678700 1 Processor -- start 2023-10-06T11:18:59.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.411+0000 7ff666678700 1 -- start start 2023-10-06T11:18:59.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff666678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6600ca430 0x7ff6600c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff666678700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600c1630 con 0x7ff6600a9840 2023-10-06T11:18:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff666678700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6600c0d00 con 0x7ff6600c7d80 2023-10-06T11:18:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff666678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600c0fe0 con 0x7ff6600ca430 2023-10-06T11:18:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff664c15700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6600a9840 0x7ff6600c1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39910/0 (socket says 172.21.15.46:39910) 2023-10-06T11:18:59.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff664c15700 1 -- 172.21.15.46:0/1058143020 learned_addr learned my addr 172.21.15.46:0/1058143020 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:59.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6600ca430 0x7ff6600c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197964069 0 0) 0x7ff6600c1630 con 0x7ff6600a9840 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff648003570 con 0x7ff6600a9840 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975378144 0 0) 0x7ff6600c0d00 con 0x7ff6600c7d80 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6600c1630 con 0x7ff6600c7d80 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128545484 0 0) 0x7ff648003570 con 0x7ff6600a9840 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6600ca430 msgr2=0x7ff6600c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6600ca430 0x7ff6600c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 >> v1:172.21.15.132:6789/0 conn(0x7ff6600c7d80 legacy=0x7ff6600c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600c12c0 con 0x7ff6600a9840 2023-10-06T11:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7ff6580017f0 con 0x7ff6600a9840 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 >> v1:172.21.15.46:6789/0 conn(0x7ff6600a9840 legacy=0x7ff6600c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff648004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff648004810 0x7ff648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6480051c0 0x7ff64800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6600c0fe0 con 0x7ff648003c20 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64800c020 con 0x7ff648004810 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64800c1e0 con 0x7ff6480051c0 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.415+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6600ca430 0x7ff6600c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff664c15700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6480051c0 0x7ff64800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff648004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff648004810 0x7ff648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 -- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6480051c0 msgr2=0x7ff64800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6480051c0 0x7ff64800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 -- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff648004810 msgr2=0x7ff648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff648004810 0x7ff648004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 -- 172.21.15.46:0/1058143020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648003570 con 0x7ff648003c20 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65ffff700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff648004010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff650001a10 tx=0x7ff650007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a54a8db4ee23325 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000c5d0 con 0x7ff648003c20 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000c730 con 0x7ff648003c20 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff65effd700 1 -- 172.21.15.46:0/1058143020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000cad0 con 0x7ff648003c20 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 -- 172.21.15.46:0/1058143020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 msgr2=0x7ff648004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 --2- 172.21.15.46:0/1058143020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff648004010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff650001a10 tx=0x7ff650007d30 comp rx=0 tx=0).stop 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff664c15700 1 -- 172.21.15.46:0/1058143020 reap_dead start 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 -- 172.21.15.46:0/1058143020 shutdown_connections 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 -- 172.21.15.46:0/1058143020 >> 172.21.15.46:0/1058143020 conn(0x7ff6600cbe30 msgr2=0x7ff6600cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 -- 172.21.15.46:0/1058143020 shutdown_connections 2023-10-06T11:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.419+0000 7ff666678700 1 -- 172.21.15.46:0/1058143020 wait complete. 2023-10-06T11:18:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 Processor -- start 2023-10-06T11:18:59.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 -- start start 2023-10-06T11:18:59.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff660155f50 0x7ff66015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66006e640 0x7ff660149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66016d4b0 con 0x7ff648003c20 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66016cea0 con 0x7ff66006e640 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.423+0000 7ff666678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66015b480 con 0x7ff660155f50 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51052/0 (socket says 172.21.15.46:51052) 2023-10-06T11:18:59.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 -- 172.21.15.46:0/3865299132 learned_addr learned my addr 172.21.15.46:0/3865299132 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:59.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff664c15700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66006e640 0x7ff660149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff660155f50 0x7ff66015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 -- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff660155f50 msgr2=0x7ff66015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff660155f50 0x7ff66015ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 -- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66006e640 msgr2=0x7ff660149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66006e640 0x7ff660149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 -- 172.21.15.46:0/3865299132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66015b760 con 0x7ff648003c20 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff65ffff700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff64800bff0 tx=0x7ff650015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65000c5d0 con 0x7ff648003c20 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65000cab0 con 0x7ff648003c20 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650006a70 con 0x7ff648003c20 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66016d140 con 0x7ff648003c20 2023-10-06T11:18:59.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.427+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66006ea80 con 0x7ff648003c20 2023-10-06T11:18:59.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.431+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6600660f0 con 0x7ff648003c20 2023-10-06T11:18:59.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.431+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff650019070 con 0x7ff648003c20 2023-10-06T11:18:59.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.431+0000 7ff647fff700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 0x7ff64c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.435+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff65007dfa0 con 0x7ff648003c20 2023-10-06T11:18:59.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.435+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 0x7ff64c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.435+0000 7ff65f7fe700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 0x7ff64c062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6600ac980 tx=0x7ff6540091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.439+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff65004e450 con 0x7ff648003c20 2023-10-06T11:18:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.611+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff66004b5c0 con 0x7ff648003c20 2023-10-06T11:18:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.611+0000 7ff647fff700 1 -- 172.21.15.46:0/3865299132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff65002c0a0 con 0x7ff648003c20 2023-10-06T11:18:59.615 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:18:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 msgr2=0x7ff64c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 0x7ff64c062a00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6600ac980 tx=0x7ff6540091a0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 msgr2=0x7ff66016d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff64800bff0 tx=0x7ff650015000 comp rx=0 tx=0).stop 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 shutdown_connections 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff64c060550 0x7ff64c062a00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff66006e640 0x7ff660149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff660155f50 0x7ff66015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 --2- 172.21.15.46:0/3865299132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff648003c20 0x7ff66016d940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 >> 172.21.15.46:0/3865299132 conn(0x7ff6600cbe30 msgr2=0x7ff660068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 shutdown_connections 2023-10-06T11:18:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.615+0000 7ff666678700 1 -- 172.21.15.46:0/3865299132 wait complete. 2023-10-06T11:18:59.638 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:18:59.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.791+0000 7f528046e700 1 Processor -- start 2023-10-06T11:18:59.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.791+0000 7f528046e700 1 -- start start 2023-10-06T11:18:59.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f528046e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5278155480 0x7f52780c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f528046e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527806dd30 con 0x7f52780c2840 2023-10-06T11:18:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f528046e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527815a080 con 0x7f5278155850 2023-10-06T11:18:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f528046e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527806ce80 con 0x7f5278155480 2023-10-06T11:18:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f527ea0b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52780c2840 0x7f5278159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39922/0 (socket says 172.21.15.46:39922) 2023-10-06T11:18:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f527ea0b700 1 -- 172.21.15.46:0/2972577776 learned_addr learned my addr 172.21.15.46:0/2972577776 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:18:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f527e20a700 1 --2- 172.21.15.46:0/2972577776 >> v2:172.21.15.132:3300/0 conn(0x7f5278155480 0x7f52780c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348991249 0 0) 0x7f527806dd30 con 0x7f52780c2840 2023-10-06T11:18:59.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.795+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5264003570 con 0x7f52780c2840 2023-10-06T11:18:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527e20a700 1 -- 172.21.15.46:0/2972577776 >> v1:172.21.15.132:6789/0 conn(0x7f5278155850 legacy=0x7f5278159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644915252 0 0) 0x7f527815a080 con 0x7f5278155850 2023-10-06T11:18:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527e20a700 1 -- 172.21.15.46:0/2972577776 >> v1:172.21.15.46:6789/0 conn(0x7f52780c2840 legacy=0x7f5278159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527e20a700 1 -- 172.21.15.46:0/2972577776 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527806d120 con 0x7f5278155480 2023-10-06T11:18:59.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527e20a700 1 --2- 172.21.15.46:0/2972577776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155480 0x7f52780c2300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52740098d0 tx=0x7f5274009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8507558434e39827 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274005c10 con 0x7f5278155480 2023-10-06T11:18:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274005d70 con 0x7f5278155480 2023-10-06T11:18:59.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f527d208700 1 -- 172.21.15.46:0/2972577776 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274006280 con 0x7f5278155480 2023-10-06T11:18:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 -- 172.21.15.46:0/2972577776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155480 msgr2=0x7f52780c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 --2- 172.21.15.46:0/2972577776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155480 0x7f52780c2300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52740098d0 tx=0x7f5274009120 comp rx=0 tx=0).stop 2023-10-06T11:18:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 -- 172.21.15.46:0/2972577776 shutdown_connections 2023-10-06T11:18:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 --2- 172.21.15.46:0/2972577776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155480 0x7f52780c2300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 -- 172.21.15.46:0/2972577776 >> 172.21.15.46:0/2972577776 conn(0x7f52780cbe30 msgr2=0x7f52780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:18:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 -- 172.21.15.46:0/2972577776 shutdown_connections 2023-10-06T11:18:59.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.799+0000 7f528046e700 1 -- 172.21.15.46:0/2972577776 wait complete. 2023-10-06T11:18:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 Processor -- start 2023-10-06T11:18:59.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 -- start start 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52780c2840 0x7f527816c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 0x7f52780abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780ab6a0 con 0x7f5278155480 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780ab980 con 0x7f5278155850 2023-10-06T11:18:59.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f528046e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527816cf30 con 0x7f52780c2840 2023-10-06T11:18:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51068/0 (socket says 172.21.15.46:51068) 2023-10-06T11:18:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 -- 172.21.15.46:0/1825780892 learned_addr learned my addr 172.21.15.46:0/1825780892 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:18:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527e20a700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52780c2840 0x7f527816c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527ea0b700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 0x7f52780abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 -- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52780c2840 msgr2=0x7f527816c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52780c2840 0x7f527816c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 -- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 msgr2=0x7f52780abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 0x7f52780abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 -- 172.21.15.46:0/1825780892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740093c0 con 0x7f5278155480 2023-10-06T11:18:59.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527ea0b700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 0x7f52780abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.803+0000 7f527da09700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f526800c560 tx=0x7f526800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268011590 con 0x7f5278155480 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268011710 con 0x7f5278155480 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527816d1b0 con 0x7f5278155480 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268011ad0 con 0x7f5278155480 2023-10-06T11:18:59.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527816d770 con 0x7f5278155480 2023-10-06T11:18:59.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.807+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5278063f90 con 0x7f5278155480 2023-10-06T11:18:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.811+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f526801b3b0 con 0x7f5278155480 2023-10-06T11:18:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.811+0000 7f526e7fc700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 0x7f525c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:18:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.811+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f526807bf10 con 0x7f5278155480 2023-10-06T11:18:59.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.811+0000 7f527e20a700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 0x7f525c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:18:59.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.811+0000 7f527e20a700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 0x7f525c062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5274009390 tx=0x7f5274016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:18:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:18:59.815+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f526804c300 con 0x7f5278155480 2023-10-06T11:19:00.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.023+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f527806d400 con 0x7f5278155480 2023-10-06T11:19:00.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.023+0000 7f526e7fc700 1 -- 172.21.15.46:0/1825780892 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f5268017020 con 0x7f5278155480 2023-10-06T11:19:00.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:00.028 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:19:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 msgr2=0x7f525c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 0x7f525c062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5274009390 tx=0x7f5274016000 comp rx=0 tx=0).stop 2023-10-06T11:19:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 msgr2=0x7f5278069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f526800c560 tx=0x7f526800c920 comp rx=0 tx=0).stop 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 shutdown_connections 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f525c060550 0x7f525c062a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5278155850 0x7f52780abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5278155480 0x7f5278069ac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 --2- 172.21.15.46:0/1825780892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52780c2840 0x7f527816c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 >> 172.21.15.46:0/1825780892 conn(0x7f52780cbe30 msgr2=0x7f52780c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 shutdown_connections 2023-10-06T11:19:00.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.027+0000 7f528046e700 1 -- 172.21.15.46:0/1825780892 wait complete. 2023-10-06T11:19:00.033 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:19:00.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.215+0000 7fb7f180c700 1 Processor -- start 2023-10-06T11:19:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.215+0000 7fb7f180c700 1 -- start start 2023-10-06T11:19:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f180c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 0x7fb7ec0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f180c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 0x7fb7ec0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f180c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec0cf100 con 0x7fb7ec0af3d0 2023-10-06T11:19:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f180c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec0ce4f0 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f180c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0ce7d0 con 0x7fb7ec0af000 2023-10-06T11:19:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e3fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 0x7fb7ec0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 0x7fb7ec0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e3fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 0x7fb7ec0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44002/0 (socket says 172.21.15.46:44002) 2023-10-06T11:19:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e3fff700 1 -- 172.21.15.46:0/276308523 learned_addr learned my addr 172.21.15.46:0/276308523 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f100b700 1 --1- 172.21.15.46:0/276308523 >> v1:172.21.15.132:6789/0 conn(0x7fb7ec0af3d0 0x7fb7ec0690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:59092/0 (socket says 172.21.15.46:59092) 2023-10-06T11:19:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 -- 172.21.15.46:0/276308523 >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 msgr2=0x7fb7ec0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 --2- 172.21.15.46:0/276308523 >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 0x7fb7ec0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 -- 172.21.15.46:0/276308523 >> v1:172.21.15.132:6789/0 conn(0x7fb7ec0af3d0 legacy=0x7fb7ec0690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 -- 172.21.15.46:0/276308523 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0ceab0 con 0x7fb7ec0af000 2023-10-06T11:19:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7f080a700 1 --2- 172.21.15.46:0/276308523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 0x7fb7ec0cf590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009e60 tx=0x7fb7e8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777b61b72cace859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e37fe700 1 -- 172.21.15.46:0/276308523 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800e070 con 0x7fb7ec0af000 2023-10-06T11:19:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e37fe700 1 -- 172.21.15.46:0/276308523 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800bb60 con 0x7fb7ec0af000 2023-10-06T11:19:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.219+0000 7fb7e37fe700 1 -- 172.21.15.46:0/276308523 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8012430 con 0x7fb7ec0af000 2023-10-06T11:19:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- 172.21.15.46:0/276308523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 msgr2=0x7fb7ec0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 --2- 172.21.15.46:0/276308523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 0x7fb7ec0cf590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009e60 tx=0x7fb7e8009780 comp rx=0 tx=0).stop 2023-10-06T11:19:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- 172.21.15.46:0/276308523 shutdown_connections 2023-10-06T11:19:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 --2- 172.21.15.46:0/276308523 >> v2:172.21.15.132:3300/0 conn(0x7fb7ec0cb4c0 0x7fb7ec0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 --2- 172.21.15.46:0/276308523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af000 0x7fb7ec0cf590 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- 172.21.15.46:0/276308523 >> 172.21.15.46:0/276308523 conn(0x7fb7ec0683b0 msgr2=0x7fb7ec0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- 172.21.15.46:0/276308523 shutdown_connections 2023-10-06T11:19:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- 172.21.15.46:0/276308523 wait complete. 2023-10-06T11:19:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 Processor -- start 2023-10-06T11:19:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.223+0000 7fb7f180c700 1 -- start start 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7ec0af000 0x7fb7ec16d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af3d0 0x7fb7ec15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec16e000 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec16d550 con 0x7fb7ec0af000 2023-10-06T11:19:00.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1598e0 con 0x7fb7ec0af3d0 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51072/0 (socket says 172.21.15.46:51072) 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 -- 172.21.15.46:0/1698752829 learned_addr learned my addr 172.21.15.46:0/1698752829 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7e3fff700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af3d0 0x7fb7ec15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f080a700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7ec0af000 0x7fb7ec16d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 -- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af3d0 msgr2=0x7fb7ec15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af3d0 0x7fb7ec15bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 -- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7ec0af000 msgr2=0x7fb7ec16d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7ec0af000 0x7fb7ec16d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 -- 172.21.15.46:0/1698752829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8009a20 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f100b700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c590 tx=0x7fb7e4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4005b40 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4005cc0 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec15c1f0 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4007320 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.227+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec16da40 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.231+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec064590 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.231+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb7e401b3b0 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.235+0000 7fb7e0ff9700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.235+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb7e407bf00 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.235+0000 7fb7f080a700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.235+0000 7fb7f080a700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 0x7fb7d8062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80097c0 tx=0x7fb7e8019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.239+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb7e404c2f0 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.415+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7ec0ced90 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.415+0000 7fb7e0ff9700 1 -- 172.21.15.46:0/1698752829 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fb7e4017090 con 0x7fb7ec0cb4c0 2023-10-06T11:19:00.420 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:00.421 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 msgr2=0x7fb7d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 0x7fb7d8062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80097c0 tx=0x7fb7e8019000 comp rx=0 tx=0).stop 2023-10-06T11:19:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 msgr2=0x7fb7ec156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c590 tx=0x7fb7e4007ef0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 shutdown_connections 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb7d8060550 0x7fb7d8062a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7ec0cb4c0 0x7fb7ec156e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7ec0af3d0 0x7fb7ec15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 --2- 172.21.15.46:0/1698752829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7ec0af000 0x7fb7ec16d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 >> 172.21.15.46:0/1698752829 conn(0x7fb7ec0683b0 msgr2=0x7fb7ec0c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 shutdown_connections 2023-10-06T11:19:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.419+0000 7fb7f180c700 1 -- 172.21.15.46:0/1698752829 wait complete. 2023-10-06T11:19:00.445 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:19:00.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:19:00.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 Processor -- start 2023-10-06T11:19:00.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 -- start start 2023-10-06T11:19:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72e00d07b0 0x7f72e00b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 0x7f72e00ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e00b15b0 con 0x7f72e00d07b0 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e00ae340 con 0x7f72e00b1130 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72e4c8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b09a0 con 0x7f72e0159290 2023-10-06T11:19:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72deffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72e00d07b0 0x7f72e00b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72df7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.603+0000 7f72dffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 0x7f72e00ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:40034/0 (socket says 172.21.15.46:40034) 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72dffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 0x7f72e00ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44018/0 (socket says 172.21.15.46:44018) 2023-10-06T11:19:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 -- 172.21.15.46:0/2541936339 learned_addr learned my addr 172.21.15.46:0/2541936339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 -- 172.21.15.46:0/2541936339 >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 msgr2=0x7f72e00ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2541936339 >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 0x7f72e00ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 -- 172.21.15.46:0/2541936339 >> v2:172.21.15.46:3300/0 conn(0x7f72e00d07b0 msgr2=0x7f72e00b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2541936339 >> v2:172.21.15.46:3300/0 conn(0x7f72e00d07b0 0x7f72e00b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 -- 172.21.15.46:0/2541936339 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0159660 con 0x7f72e0159290 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2541936339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72e00cffd0 tx=0x7f72d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635ca5eea1b2b09b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72de7fc700 1 -- 172.21.15.46:0/2541936339 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000b6d0 con 0x7f72e0159290 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72de7fc700 1 -- 172.21.15.46:0/2541936339 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000b830 con 0x7f72e0159290 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72de7fc700 1 -- 172.21.15.46:0/2541936339 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000bbf0 con 0x7f72e0159290 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2541936339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 msgr2=0x7f72e00b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2541936339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72e00cffd0 tx=0x7f72d0009250 comp rx=0 tx=0).stop 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2541936339 shutdown_connections 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2541936339 >> v2:172.21.15.132:3300/0 conn(0x7f72e00b1130 0x7f72e00ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2541936339 >> v2:172.21.15.46:3300/0 conn(0x7f72e00d07b0 0x7f72e00b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2541936339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e0159290 0x7f72e00b2f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2541936339 >> 172.21.15.46:0/2541936339 conn(0x7f72e006ef60 msgr2=0x7f72e00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2541936339 shutdown_connections 2023-10-06T11:19:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.607+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2541936339 wait complete. 2023-10-06T11:19:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.611+0000 7f72e4c8e700 1 Processor -- start 2023-10-06T11:19:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.611+0000 7f72e4c8e700 1 -- start start 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.611+0000 7f72e4c8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.611+0000 7f72e4c8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e00d07b0 0x7f72e01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 0x7f72e00b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e015a620 con 0x7f72e00b1130 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0151340 con 0x7f72e0159290 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0151620 con 0x7f72e00d07b0 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51098/0 (socket says 172.21.15.46:51098) 2023-10-06T11:19:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 -- 172.21.15.46:0/2973224676 learned_addr learned my addr 172.21.15.46:0/2973224676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72deffd700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e00d07b0 0x7f72e01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72dffff700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 0x7f72e00b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 -- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e00d07b0 msgr2=0x7f72e01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e00d07b0 0x7f72e01548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 -- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 msgr2=0x7f72e00b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 0x7f72e00b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 -- 172.21.15.46:0/2973224676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00094f0 con 0x7f72e00b1130 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72dffff700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 0x7f72e00b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72df7fe700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72e00b0ae0 tx=0x7f72d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000bbb0 con 0x7f72e00b1130 2023-10-06T11:19:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d001a430 con 0x7f72e00b1130 2023-10-06T11:19:00.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e0151900 con 0x7f72e00b1130 2023-10-06T11:19:00.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d001a7d0 con 0x7f72e00b1130 2023-10-06T11:19:00.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.615+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e014b440 con 0x7f72e00b1130 2023-10-06T11:19:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.619+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e0064590 con 0x7f72e00b1130 2023-10-06T11:19:00.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.619+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f72d001ab40 con 0x7f72e00b1130 2023-10-06T11:19:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.619+0000 7f72c77fe700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 0x7f72cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.619+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72d007e670 con 0x7f72e00b1130 2023-10-06T11:19:00.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.619+0000 7f72deffd700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 0x7f72cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.623+0000 7f72deffd700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 0x7f72cc062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f72e014d7d0 tx=0x7f72d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.623+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f72d004eae0 con 0x7f72e00b1130 2023-10-06T11:19:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.827+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e00675d0 con 0x7f72e00b1130 2023-10-06T11:19:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.831+0000 7f72c77fe700 1 -- 172.21.15.46:0/2973224676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f72d0052100 con 0x7f72e00b1130 2023-10-06T11:19:00.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:00.835 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 msgr2=0x7f72cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 0x7f72cc062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f72e014d7d0 tx=0x7f72d4008040 comp rx=0 tx=0).stop 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 msgr2=0x7f72e014af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72e00b0ae0 tx=0x7f72d0017000 comp rx=0 tx=0).stop 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 shutdown_connections 2023-10-06T11:19:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f72cc060550 0x7f72cc062a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72e0159290 0x7f72e00b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72e00d07b0 0x7f72e01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 --2- 172.21.15.46:0/2973224676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72e00b1130 0x7f72e014af00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 >> 172.21.15.46:0/2973224676 conn(0x7f72e006ef60 msgr2=0x7f72e0067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 shutdown_connections 2023-10-06T11:19:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:00.835+0000 7f72e4c8e700 1 -- 172.21.15.46:0/2973224676 wait complete. 2023-10-06T11:19:00.843 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:19:00.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:19:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.015+0000 7f1c7c8f0700 1 Processor -- start 2023-10-06T11:19:01.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c7c8f0700 1 -- start start 2023-10-06T11:19:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c7c8f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 0x7f1c78161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c7c8f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780c2630 con 0x7f1c780c1d80 2023-10-06T11:19:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c7c8f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780c18f0 con 0x7f1c780c21b0 2023-10-06T11:19:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c7c8f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780cb120 con 0x7f1c78161470 2023-10-06T11:19:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c75d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c780c1d80 0x7f1c7806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39928/0 (socket says 172.21.15.46:39928) 2023-10-06T11:19:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.019+0000 7f1c75d9b700 1 -- 172.21.15.46:0/3648431107 learned_addr learned my addr 172.21.15.46:0/3648431107 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7659c700 1 --2- 172.21.15.46:0/3648431107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 0x7f1c78161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758587279 0 0) 0x7f1c780c2630 con 0x7f1c780c1d80 2023-10-06T11:19:01.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c60003570 con 0x7f1c780c1d80 2023-10-06T11:19:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455147239 0 0) 0x7f1c780c18f0 con 0x7f1c780c21b0 2023-10-06T11:19:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c780c2630 con 0x7f1c780c21b0 2023-10-06T11:19:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7659c700 1 -- 172.21.15.46:0/3648431107 >> v1:172.21.15.132:6789/0 conn(0x7f1c780c21b0 legacy=0x7f1c780caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7659c700 1 -- 172.21.15.46:0/3648431107 >> v1:172.21.15.46:6789/0 conn(0x7f1c780c1d80 legacy=0x7f1c7806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091506995 0 0) 0x7f1c60003570 con 0x7f1c780c1d80 2023-10-06T11:19:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7659c700 1 -- 172.21.15.46:0/3648431107 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c780c0ce0 con 0x7f1c78161470 2023-10-06T11:19:01.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7659c700 1 --2- 172.21.15.46:0/3648431107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 0x7f1c78161840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009900 tx=0x7f1c68009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d49e9ea4a247a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c001470 con 0x7f1c78161470 2023-10-06T11:19:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011420 con 0x7f1c78161470 2023-10-06T11:19:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.023+0000 7f1c7559a700 1 -- 172.21.15.46:0/3648431107 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680117c0 con 0x7f1c78161470 2023-10-06T11:19:01.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/3648431107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 msgr2=0x7f1c78161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/3648431107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 0x7f1c78161840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009900 tx=0x7f1c68009120 comp rx=0 tx=0).stop 2023-10-06T11:19:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/3648431107 shutdown_connections 2023-10-06T11:19:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/3648431107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c78161470 0x7f1c78161840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/3648431107 >> 172.21.15.46:0/3648431107 conn(0x7f1c780cbe30 msgr2=0x7f1c780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:01.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/3648431107 shutdown_connections 2023-10-06T11:19:01.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/3648431107 wait complete. 2023-10-06T11:19:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 Processor -- start 2023-10-06T11:19:01.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.027+0000 7f1c7c8f0700 1 -- start start 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c780c21b0 0x7f1c780b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c78161470 0x7f1c780ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780b0720 con 0x7f1c780c1d80 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780c3cf0 con 0x7f1c78161470 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780b0f00 con 0x7f1c780c21b0 2023-10-06T11:19:01.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51114/0 (socket says 172.21.15.46:51114) 2023-10-06T11:19:01.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 -- 172.21.15.46:0/1764972775 learned_addr learned my addr 172.21.15.46:0/1764972775 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:01.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c75d9b700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c780c21b0 0x7f1c780b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c76d9d700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c78161470 0x7f1c780ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 -- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c780c21b0 msgr2=0x7f1c780b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c780c21b0 0x7f1c780b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 -- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c78161470 msgr2=0x7f1c780ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c78161470 0x7f1c780ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 -- 172.21.15.46:0/1764972775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680093c0 con 0x7f1c780c1d80 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7659c700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009850 tx=0x7f1c68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6801cc90 con 0x7f1c780c1d80 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6801cdf0 con 0x7f1c780c1d80 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6801a6a0 con 0x7f1c780c1d80 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c780c3450 con 0x7f1c780c1d80 2023-10-06T11:19:01.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.031+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c780c3a10 con 0x7f1c780c1d80 2023-10-06T11:19:01.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c78063f90 con 0x7f1c780c1d80 2023-10-06T11:19:01.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1c6801aa10 con 0x7f1c780c1d80 2023-10-06T11:19:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c667fc700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 0x7f1c58062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c6807e2f0 con 0x7f1c780c1d80 2023-10-06T11:19:01.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c75d9b700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 0x7f1c58062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.035+0000 7f1c75d9b700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 0x7f1c58062a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0015a0 tx=0x7f1c6c000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.043+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c6804e6e0 con 0x7f1c780c1d80 2023-10-06T11:19:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.251+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c780c0fc0 con 0x7f1c780c1d80 2023-10-06T11:19:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c667fc700 1 -- 172.21.15.46:0/1764972775 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f1c68051d00 con 0x7f1c780c1d80 2023-10-06T11:19:01.258 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:01.258 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/347894045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":347894045},{"type":"v1","addr":"172.21.15.132:6833","nonce":347894045}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/4036589104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4036589104},{"type":"v1","addr":"172.21.15.46:6836","nonce":4036589104}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3096324679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3096324679},{"type":"v1","addr":"172.21.15.46:6837","nonce":3096324679}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:18:23.206297+0000","modified":"2023-10-06T11:18:27.079467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1894883507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1894883507},{"type":"v1","addr":"172.21.15.132:6835","nonce":1894883507}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:19:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 msgr2=0x7f1c58062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 0x7f1c58062a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c0015a0 tx=0x7f1c6c000bc0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 msgr2=0x7f1c7814d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009850 tx=0x7f1c68016000 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 shutdown_connections 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c58060550 0x7f1c58062a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c78161470 0x7f1c780ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c780c21b0 0x7f1c780b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 --2- 172.21.15.46:0/1764972775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c780c1d80 0x7f1c7814d270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 >> 172.21.15.46:0/1764972775 conn(0x7f1c780cbe30 msgr2=0x7f1c780cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 shutdown_connections 2023-10-06T11:19:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.255+0000 7f1c7c8f0700 1 -- 172.21.15.46:0/1764972775 wait complete. 2023-10-06T11:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2023-10-06T11:19:01.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 Processor -- start 2023-10-06T11:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 -- start start 2023-10-06T11:19:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff130156e20 0x7ff13006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1300aaf30 0x7ff130067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13006d790 con 0x7ff130067590 2023-10-06T11:19:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff130067a10 con 0x7ff130156e20 2023-10-06T11:19:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300acd60 con 0x7ff1300aaf30 2023-10-06T11:19:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12ed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff130067590 0x7ff13006de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39936/0 (socket says 172.21.15.46:39936) 2023-10-06T11:19:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.435+0000 7ff12ed9d700 1 -- 172.21.15.46:0/5052215 learned_addr learned my addr 172.21.15.46:0/5052215 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 --2- 172.21.15.46:0/5052215 >> v2:172.21.15.132:3300/0 conn(0x7ff130156e20 0x7ff13006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff125d9b700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1300aaf30 0x7ff130067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030213775 0 0) 0x7ff13006d790 con 0x7ff130067590 2023-10-06T11:19:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c003570 con 0x7ff130067590 2023-10-06T11:19:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540146565 0 0) 0x7ff11c003570 con 0x7ff130067590 2023-10-06T11:19:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 -- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1300aaf30 msgr2=0x7ff130067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1300aaf30 0x7ff130067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 -- 172.21.15.46:0/5052215 >> v1:172.21.15.46:6789/0 conn(0x7ff130067590 legacy=0x7ff13006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 -- 172.21.15.46:0/5052215 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300ad040 con 0x7ff130156e20 2023-10-06T11:19:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12e59c700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff13006d250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff1200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d8bc66790eef206 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128001810 con 0x7ff130156e20 2023-10-06T11:19:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff120011420 con 0x7ff130156e20 2023-10-06T11:19:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12dd9b700 1 -- 172.21.15.46:0/5052215 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200117c0 con 0x7ff130156e20 2023-10-06T11:19:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12f59e700 1 -- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 msgr2=0x7ff13006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.439+0000 7ff12f59e700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff13006d250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098e0 tx=0x7ff1200091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 -- 172.21.15.46:0/5052215 shutdown_connections 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1300aaf30 0x7ff130067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 --2- 172.21.15.46:0/5052215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff13006d250 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 -- 172.21.15.46:0/5052215 >> 172.21.15.46:0/5052215 conn(0x7ff130062ef0 msgr2=0x7ff1300cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 -- 172.21.15.46:0/5052215 shutdown_connections 2023-10-06T11:19:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 -- 172.21.15.46:0/5052215 wait complete. 2023-10-06T11:19:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.443+0000 7ff12f59e700 1 Processor -- start 2023-10-06T11:19:01.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 -- start start 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff130067590 0x7ff1301554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff1300a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c5240 con 0x7ff1300aaf30 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c4db0 con 0x7ff130156e20 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300c7870 con 0x7ff130067590 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51116/0 (socket says 172.21.15.46:51116) 2023-10-06T11:19:01.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 -- 172.21.15.46:0/3693681309 learned_addr learned my addr 172.21.15.46:0/3693681309 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12ed9d700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff1300a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12e59c700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff130067590 0x7ff1301554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 -- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff130067590 msgr2=0x7ff1301554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff130067590 0x7ff1301554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 -- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 msgr2=0x7ff1300a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff1300a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 -- 172.21.15.46:0/3693681309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009420 con 0x7ff1300aaf30 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff12ed9d700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff1300a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.447+0000 7ff125d9b700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ec80 tx=0x7ff1180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800c9d0 con 0x7ff1300aaf30 2023-10-06T11:19:01.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800cb30 con 0x7ff1300aaf30 2023-10-06T11:19:01.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130155a30 con 0x7ff1300aaf30 2023-10-06T11:19:01.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118013470 con 0x7ff1300aaf30 2023-10-06T11:19:01.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1300c4760 con 0x7ff1300aaf30 2023-10-06T11:19:01.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1300c4f60 con 0x7ff1300aaf30 2023-10-06T11:19:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.451+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff118013820 con 0x7ff1300aaf30 2023-10-06T11:19:01.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.455+0000 7ff126ffd700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 0x7ff110062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.455+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff11807cbc0 con 0x7ff1300aaf30 2023-10-06T11:19:01.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.455+0000 7ff12e59c700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 0x7ff110062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.455+0000 7ff12e59c700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 0x7ff110062a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1200030c0 tx=0x7ff120016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.459+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff1300c4f60 con 0x7ff1300aaf30 2023-10-06T11:19:01.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.631+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1300c4f60 con 0x7ff1300aaf30 2023-10-06T11:19:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.631+0000 7ff126ffd700 1 -- 172.21.15.46:0/3693681309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7ff1300c4f60 con 0x7ff1300aaf30 2023-10-06T11:19:01.636 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:01.636 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:18:24.767201+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:18:21.260824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:18:22.204974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 msgr2=0x7ff110062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 0x7ff110062a30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1200030c0 tx=0x7ff120016000 comp rx=0 tx=0).stop 2023-10-06T11:19:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 msgr2=0x7ff1300c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff11800ec80 tx=0x7ff1180091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 shutdown_connections 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff110060580 0x7ff110062a30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff130156e20 0x7ff1300a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300aaf30 0x7ff1300c6b10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 --2- 172.21.15.46:0/3693681309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff130067590 0x7ff1301554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 >> 172.21.15.46:0/3693681309 conn(0x7ff130062ef0 msgr2=0x7ff130155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 shutdown_connections 2023-10-06T11:19:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.635+0000 7ff12f59e700 1 -- 172.21.15.46:0/3693681309 wait complete. 2023-10-06T11:19:01.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:19:01.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 Processor -- start 2023-10-06T11:19:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 -- start start 2023-10-06T11:19:01.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 0x7fd39c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c0c8a40 con 0x7fd39c068290 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c0cb470 con 0x7fd39c0ca280 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.823+0000 7fd3a1345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0ca860 con 0x7fd39c068660 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39affd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd39c068290 0x7fd39c0abfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39938/0 (socket says 172.21.15.46:39938) 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39affd700 1 -- 172.21.15.46:0/2783878735 learned_addr learned my addr 172.21.15.46:0/2783878735 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:01.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39b7fe700 1 --2- 172.21.15.46:0/2783878735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 0x7fd39c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211292994 0 0) 0x7fd39c0c8a40 con 0x7fd39c068290 2023-10-06T11:19:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd388003570 con 0x7fd39c068290 2023-10-06T11:19:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163261199 0 0) 0x7fd39c0cb470 con 0x7fd39c0ca280 2023-10-06T11:19:01.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd39c0c8a40 con 0x7fd39c0ca280 2023-10-06T11:19:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3181572646 0 0) 0x7fd388003570 con 0x7fd39c068290 2023-10-06T11:19:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39b7fe700 1 -- 172.21.15.46:0/2783878735 >> v1:172.21.15.132:6789/0 conn(0x7fd39c0ca280 legacy=0x7fd39c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39b7fe700 1 -- 172.21.15.46:0/2783878735 >> v1:172.21.15.46:6789/0 conn(0x7fd39c068290 legacy=0x7fd39c0abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39b7fe700 1 -- 172.21.15.46:0/2783878735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c0cab40 con 0x7fd39c068660 2023-10-06T11:19:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39b7fe700 1 --2- 172.21.15.46:0/2783878735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 0x7fd39c1570e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd394009e90 tx=0x7fd394009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff204021403d93d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400e070 con 0x7fd39c068660 2023-10-06T11:19:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39400b870 con 0x7fd39c068660 2023-10-06T11:19:01.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.827+0000 7fd39a7fc700 1 -- 172.21.15.46:0/2783878735 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c001470 con 0x7fd39c068660 2023-10-06T11:19:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- 172.21.15.46:0/2783878735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 msgr2=0x7fd39c1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 --2- 172.21.15.46:0/2783878735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 0x7fd39c1570e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd394009e90 tx=0x7fd394009780 comp rx=0 tx=0).stop 2023-10-06T11:19:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- 172.21.15.46:0/2783878735 shutdown_connections 2023-10-06T11:19:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 --2- 172.21.15.46:0/2783878735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068660 0x7fd39c1570e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- 172.21.15.46:0/2783878735 >> 172.21.15.46:0/2783878735 conn(0x7fd39c06d490 msgr2=0x7fd39c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- 172.21.15.46:0/2783878735 shutdown_connections 2023-10-06T11:19:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- 172.21.15.46:0/2783878735 wait complete. 2023-10-06T11:19:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 Processor -- start 2023-10-06T11:19:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.831+0000 7fd3a1345700 1 -- start start 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068290 0x7fd39c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 0x7fd39c0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0aca00 con 0x7fd39c0ca280 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0ac6d0 con 0x7fd39c068660 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c0b1800 con 0x7fd39c068290 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51122/0 (socket says 172.21.15.46:51122) 2023-10-06T11:19:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 -- 172.21.15.46:0/525619295 learned_addr learned my addr 172.21.15.46:0/525619295 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39affd700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068290 0x7fd39c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd393fff700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 0x7fd39c0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 -- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068290 msgr2=0x7fd39c0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068290 0x7fd39c0ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 -- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 msgr2=0x7fd39c0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 0x7fd39c0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 -- 172.21.15.46:0/525619295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd394009a20 con 0x7fd39c0ca280 2023-10-06T11:19:01.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd393fff700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 0x7fd39c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd39b7fe700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097c0 tx=0x7fd394017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394012cf0 con 0x7fd39c0ca280 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394012e90 con 0x7fd39c0ca280 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39401e6a0 con 0x7fd39c0ca280 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c0b1ae0 con 0x7fd39c0ca280 2023-10-06T11:19:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.835+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c0b1dc0 con 0x7fd39c0ca280 2023-10-06T11:19:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.839+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c06c110 con 0x7fd39c0ca280 2023-10-06T11:19:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.839+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd39401ead0 con 0x7fd39c0ca280 2023-10-06T11:19:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.839+0000 7fd392ffd700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 0x7fd37c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.839+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd39407ec40 con 0x7fd39c0ca280 2023-10-06T11:19:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.839+0000 7fd39affd700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 0x7fd37c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:01.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.843+0000 7fd39affd700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 0x7fd37c062a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd38c001ed0 tx=0x7fd38c000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:01.847+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd39404f030 con 0x7fd39c0ca280 2023-10-06T11:19:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.055+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd39c0acb40 con 0x7fd39c0ca280 2023-10-06T11:19:02.061 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:19:02.055+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:19:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.067+0000 7fd392ffd700 1 -- 172.21.15.46:0/525619295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd394052650 con 0x7fd39c0ca280 2023-10-06T11:19:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 msgr2=0x7fd37c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 0x7fd37c062a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd38c001ed0 tx=0x7fd38c000bc0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 msgr2=0x7fd39c146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097c0 tx=0x7fd394017000 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 shutdown_connections 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd37c060550 0x7fd37c062a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd39c0ca280 0x7fd39c146e20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39c068660 0x7fd39c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 --2- 172.21.15.46:0/525619295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39c068290 0x7fd39c0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 >> 172.21.15.46:0/525619295 conn(0x7fd39c06d490 msgr2=0x7fd39c158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 shutdown_connections 2023-10-06T11:19:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.071+0000 7fd3a1345700 1 -- 172.21.15.46:0/525619295 wait complete. 2023-10-06T11:19:02.078 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:19:02.080 INFO:tasks.ceph.mds.d.smithi132.stderr: -1853> 2023-10-06T11:18:28.855+0000 7fb76e1a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:19:02.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:19:02.130 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:19:02.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.255+0000 7f3808aae700 1 Processor -- start 2023-10-06T11:19:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.255+0000 7f3808aae700 1 -- start start 2023-10-06T11:19:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3808aae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3804161fc0 0x7f38040caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3808aae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3804147280 0x7f380406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3808aae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38040c1980 con 0x7f3804146eb0 2023-10-06T11:19:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3808aae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38040c2520 con 0x7f3804161fc0 2023-10-06T11:19:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3808aae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38040cb1e0 con 0x7f3804147280 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3804146eb0 0x7f38040c1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39946/0 (socket says 172.21.15.46:39946) 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380259c700 1 -- 172.21.15.46:0/197963614 learned_addr learned my addr 172.21.15.46:0/197963614 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 --2- 172.21.15.46:0/197963614 >> v2:172.21.15.46:3300/0 conn(0x7f3804161fc0 0x7f38040caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3802d9d700 1 --2- 172.21.15.46:0/197963614 >> v2:172.21.15.132:3300/0 conn(0x7f3804147280 0x7f380406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380159a700 1 -- 172.21.15.46:0/197963614 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957260394 0 0) 0x7f38040c1980 con 0x7f3804146eb0 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380159a700 1 -- 172.21.15.46:0/197963614 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003570 con 0x7f3804146eb0 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 -- 172.21.15.46:0/197963614 >> v2:172.21.15.132:3300/0 conn(0x7f3804147280 msgr2=0x7f380406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 --2- 172.21.15.46:0/197963614 >> v2:172.21.15.132:3300/0 conn(0x7f3804147280 0x7f380406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 -- 172.21.15.46:0/197963614 >> v1:172.21.15.46:6789/0 conn(0x7f3804146eb0 legacy=0x7f38040c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 -- 172.21.15.46:0/197963614 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040c0d70 con 0x7f3804161fc0 2023-10-06T11:19:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f3801d9b700 1 --2- 172.21.15.46:0/197963614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804161fc0 0x7f38040caca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38040c2e40 tx=0x7f37ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e2087b53b12177 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380159a700 1 -- 172.21.15.46:0/197963614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011630 con 0x7f3804161fc0 2023-10-06T11:19:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380159a700 1 -- 172.21.15.46:0/197963614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec0117b0 con 0x7f3804161fc0 2023-10-06T11:19:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.259+0000 7f380159a700 1 -- 172.21.15.46:0/197963614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011b30 con 0x7f3804161fc0 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- 172.21.15.46:0/197963614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804161fc0 msgr2=0x7f38040caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 --2- 172.21.15.46:0/197963614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804161fc0 0x7f38040caca0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38040c2e40 tx=0x7f37ec0091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- 172.21.15.46:0/197963614 shutdown_connections 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 --2- 172.21.15.46:0/197963614 >> v2:172.21.15.132:3300/0 conn(0x7f3804147280 0x7f380406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 --2- 172.21.15.46:0/197963614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804161fc0 0x7f38040caca0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- 172.21.15.46:0/197963614 >> 172.21.15.46:0/197963614 conn(0x7f38040cbe30 msgr2=0x7f38040cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:02.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- 172.21.15.46:0/197963614 shutdown_connections 2023-10-06T11:19:02.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- 172.21.15.46:0/197963614 wait complete. 2023-10-06T11:19:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 Processor -- start 2023-10-06T11:19:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.263+0000 7f3808aae700 1 -- start start 2023-10-06T11:19:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 0x7f3804141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3804161fc0 0x7f38040c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c46f0 con 0x7f3804146eb0 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c49d0 con 0x7f3804147280 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040c4cb0 con 0x7f3804161fc0 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51134/0 (socket says 172.21.15.46:51134) 2023-10-06T11:19:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 -- 172.21.15.46:0/3829628999 learned_addr learned my addr 172.21.15.46:0/3829628999 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3801d9b700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 0x7f3804141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3802d9d700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3804161fc0 0x7f38040c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 -- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3804161fc0 msgr2=0x7f38040c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3804161fc0 0x7f38040c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 -- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 msgr2=0x7f3804141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 0x7f3804141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 -- 172.21.15.46:0/3829628999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009420 con 0x7f3804146eb0 2023-10-06T11:19:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3801d9b700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 0x7f3804141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f380259c700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37f400cd00 tx=0x7f37f4001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400aba0 con 0x7f3804146eb0 2023-10-06T11:19:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f400ad00 con 0x7f3804146eb0 2023-10-06T11:19:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38040c4f90 con 0x7f3804146eb0 2023-10-06T11:19:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4007300 con 0x7f3804146eb0 2023-10-06T11:19:02.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.267+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38040c5cd0 con 0x7f3804146eb0 2023-10-06T11:19:02.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.271+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38040c5470 con 0x7f3804146eb0 2023-10-06T11:19:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.271+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f37f40076b0 con 0x7f3804146eb0 2023-10-06T11:19:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.275+0000 7f37fa7fc700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 0x7f37e4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.275+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f37f407d020 con 0x7f3804146eb0 2023-10-06T11:19:02.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.275+0000 7f3801d9b700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 0x7f37e4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:02.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.275+0000 7f3801d9b700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 0x7f37e4062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38040c2e40 tx=0x7f37ec017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:02.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.279+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f37f404d3e0 con 0x7f3804146eb0 2023-10-06T11:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:02.487+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f38040660f0 con 0x7f3804146eb0 2023-10-06T11:19:03.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.079+0000 7f37fa7fc700 1 -- 172.21.15.46:0/3829628999 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f37f4050a00 con 0x7f3804146eb0 2023-10-06T11:19:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 msgr2=0x7f37e4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 0x7f37e4062a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38040c2e40 tx=0x7f37ec017000 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 msgr2=0x7f38040c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37f400cd00 tx=0x7f37f4001b00 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 shutdown_connections 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37e4060550 0x7f37e4062a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3804161fc0 0x7f38040c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3804147280 0x7f3804141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 --2- 172.21.15.46:0/3829628999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3804146eb0 0x7f38040c5790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 >> 172.21.15.46:0/3829628999 conn(0x7f38040cbe30 msgr2=0x7f38041617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 shutdown_connections 2023-10-06T11:19:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.083+0000 7f3808aae700 1 -- 172.21.15.46:0/3829628999 wait complete. 2023-10-06T11:19:03.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:19:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 Processor -- start 2023-10-06T11:19:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 -- start start 2023-10-06T11:19:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6464159230 0x7f6464068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f64640a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64640aab80 con 0x7f6464068290 2023-10-06T11:19:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64640a9d80 con 0x7f6464159230 2023-10-06T11:19:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.271+0000 7f646886b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6464063420 con 0x7f6464062fe0 2023-10-06T11:19:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6464159230 0x7f6464068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6464159230 0x7f6464068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51516/0 (socket says 172.21.15.46:51516) 2023-10-06T11:19:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f64637fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6464068290 0x7f6464158b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55884/0 (socket says 172.21.15.46:55884) 2023-10-06T11:19:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 -- 172.21.15.46:0/2046650409 learned_addr learned my addr 172.21.15.46:0/2046650409 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f6463fff700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f64640a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 -- 172.21.15.46:0/2046650409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 msgr2=0x7f64640a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f64640a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 -- 172.21.15.46:0/2046650409 >> v1:172.21.15.132:6789/0 conn(0x7f6464068290 legacy=0x7f6464158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 -- 172.21.15.46:0/2046650409 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640a9f70 con 0x7f6464159230 2023-10-06T11:19:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f645bfff700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464159230 0x7f6464068660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f644c0099b0 tx=0x7f644c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d24ec5771b97b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f6462ffd700 1 -- 172.21.15.46:0/2046650409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c009ea0 con 0x7f6464159230 2023-10-06T11:19:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f6462ffd700 1 -- 172.21.15.46:0/2046650409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c00b860 con 0x7f6464159230 2023-10-06T11:19:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f6462ffd700 1 -- 172.21.15.46:0/2046650409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00bc00 con 0x7f6464159230 2023-10-06T11:19:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f646886b700 1 -- 172.21.15.46:0/2046650409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464159230 msgr2=0x7f6464068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f646886b700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464159230 0x7f6464068660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f644c0099b0 tx=0x7f644c009250 comp rx=0 tx=0).stop 2023-10-06T11:19:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.275+0000 7f646886b700 1 -- 172.21.15.46:0/2046650409 shutdown_connections 2023-10-06T11:19:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f64640a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 --2- 172.21.15.46:0/2046650409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464159230 0x7f6464068660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 -- 172.21.15.46:0/2046650409 >> 172.21.15.46:0/2046650409 conn(0x7f646406ab10 msgr2=0x7f646406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 -- 172.21.15.46:0/2046650409 shutdown_connections 2023-10-06T11:19:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 -- 172.21.15.46:0/2046650409 wait complete. 2023-10-06T11:19:03.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 Processor -- start 2023-10-06T11:19:03.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.279+0000 7f646886b700 1 -- start start 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f6464153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 0x7f64640caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464153650 con 0x7f6464068290 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464169e80 con 0x7f6464159230 2023-10-06T11:19:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640c95d0 con 0x7f6464062fe0 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51520/0 (socket says 172.21.15.46:51520) 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 -- 172.21.15.46:0/2532762842 learned_addr learned my addr 172.21.15.46:0/2532762842 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f6463fff700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 0x7f64640caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f64637fe700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f6464153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 -- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 msgr2=0x7f6464153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f6464153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 -- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 msgr2=0x7f64640caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 0x7f64640caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 -- 172.21.15.46:0/2532762842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0094f0 con 0x7f6464068290 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f6463fff700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 0x7f64640caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:03.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f64637fe700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f6464153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645bfff700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f646406c850 tx=0x7f644c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c00bbe0 con 0x7f6464068290 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c01a450 con 0x7f6464068290 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c01a7f0 con 0x7f6464068290 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640c98b0 con 0x7f6464068290 2023-10-06T11:19:03.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.283+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64640c9e70 con 0x7f6464068290 2023-10-06T11:19:03.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.287+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640ae190 con 0x7f6464068290 2023-10-06T11:19:03.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.287+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f644c01ab60 con 0x7f6464068290 2023-10-06T11:19:03.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.291+0000 7f645b7fe700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 0x7f6444062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:03.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.291+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f644c07ddb0 con 0x7f6464068290 2023-10-06T11:19:03.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.291+0000 7f64637fe700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 0x7f6444062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:03.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.291+0000 7f64637fe700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 0x7f6444062a30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6454007120 tx=0x7f6454007d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:03.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.295+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64640ae190 con 0x7f6464068290 2023-10-06T11:19:03.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:03.467+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f64640ae190 con 0x7f6464068290 2023-10-06T11:19:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.355+0000 7f645b7fe700 1 -- 172.21.15.46:0/2532762842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f64640ae190 con 0x7f6464068290 2023-10-06T11:19:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 msgr2=0x7f6444062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 0x7f6444062a30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6454007120 tx=0x7f6454007d80 comp rx=0 tx=0).stop 2023-10-06T11:19:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 msgr2=0x7f64640caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f646406c850 tx=0x7f644c017000 comp rx=0 tx=0).stop 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 shutdown_connections 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6444060580 0x7f6444062a30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464159230 0x7f64640caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6464068290 0x7f64640caa00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 --2- 172.21.15.46:0/2532762842 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6464062fe0 0x7f6464153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 >> 172.21.15.46:0/2532762842 conn(0x7f646406ab10 msgr2=0x7f64640ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 shutdown_connections 2023-10-06T11:19:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.363+0000 7f646886b700 1 -- 172.21.15.46:0/2532762842 wait complete. 2023-10-06T11:19:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:19:04.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:19:04.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.555+0000 7fa28cf63700 1 Processor -- start 2023-10-06T11:19:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 -- start start 2023-10-06T11:19:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2881591b0 0x7fa288069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880b5eb0 con 0x7fa2880695a0 2023-10-06T11:19:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa288069a20 con 0x7fa2881591b0 2023-10-06T11:19:04.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa28cf63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880cef30 con 0x7fa288158de0 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa286ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2881591b0 0x7fa288069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa286ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2881591b0 0x7fa288069060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51526/0 (socket says 172.21.15.46:51526) 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.559+0000 7fa286ffd700 1 -- 172.21.15.46:0/1660246867 learned_addr learned my addr 172.21.15.46:0/1660246867 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa286ffd700 1 -- 172.21.15.46:0/1660246867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 msgr2=0x7fa2880b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:04.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa286ffd700 1 -- 172.21.15.46:0/1660246867 >> v1:172.21.15.132:6789/0 conn(0x7fa2880695a0 legacy=0x7fa28815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa286ffd700 1 -- 172.21.15.46:0/1660246867 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880cf4f0 con 0x7fa2881591b0 2023-10-06T11:19:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa288069060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0099b0 tx=0x7fa27c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5e5f49080dbc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:04.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa2867fc700 1 -- 172.21.15.46:0/1660246867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00b860 con 0x7fa2881591b0 2023-10-06T11:19:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa2867fc700 1 -- 172.21.15.46:0/1660246867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00b9c0 con 0x7fa2881591b0 2023-10-06T11:19:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa2867fc700 1 -- 172.21.15.46:0/1660246867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00bd60 con 0x7fa2881591b0 2023-10-06T11:19:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 -- 172.21.15.46:0/1660246867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 msgr2=0x7fa288069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa288069060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0099b0 tx=0x7fa27c009250 comp rx=0 tx=0).stop 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 -- 172.21.15.46:0/1660246867 shutdown_connections 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa288069060 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1660246867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 -- 172.21.15.46:0/1660246867 >> 172.21.15.46:0/1660246867 conn(0x7fa288068390 msgr2=0x7fa2880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 -- 172.21.15.46:0/1660246867 shutdown_connections 2023-10-06T11:19:04.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.563+0000 7fa28cf63700 1 -- 172.21.15.46:0/1660246867 wait complete. 2023-10-06T11:19:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 Processor -- start 2023-10-06T11:19:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 -- start start 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2880695a0 0x7fa2880c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa288154480 con 0x7fa2881591b0 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880c6e70 con 0x7fa2880695a0 2023-10-06T11:19:04.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa28cf63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880c70c0 con 0x7fa288158de0 2023-10-06T11:19:04.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.567+0000 7fa287fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51532/0 (socket says 172.21.15.46:51532) 2023-10-06T11:19:04.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 -- 172.21.15.46:0/1196568688 learned_addr learned my addr 172.21.15.46:0/1196568688 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:04.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2880695a0 0x7fa2880c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 -- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 msgr2=0x7fa2880cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 -- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2880695a0 msgr2=0x7fa2880c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2880695a0 0x7fa2880c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:04.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 -- 172.21.15.46:0/1196568688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0094f0 con 0x7fa2881591b0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa287fff700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c8e0 tx=0x7fa280001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280012630 con 0x7fa2881591b0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa280012790 con 0x7fa2881591b0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280012b50 con 0x7fa2881591b0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2880c73a0 con 0x7fa2881591b0 2023-10-06T11:19:04.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2880c7960 con 0x7fa2881591b0 2023-10-06T11:19:04.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.571+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2880b2010 con 0x7fa2881591b0 2023-10-06T11:19:04.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.575+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa28001a3a0 con 0x7fa2881591b0 2023-10-06T11:19:04.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.575+0000 7fa26f7fe700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 0x7fa274062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:04.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.575+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5764+0+0 (secure 0 0 0) 0x7fa28007c840 con 0x7fa2881591b0 2023-10-06T11:19:04.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.575+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 0x7fa274062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:04.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.579+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 0x7fa274062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa278006fd0 tx=0x7fa278008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:04.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.583+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa2880b2010 con 0x7fa2881591b0 2023-10-06T11:19:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:04.759+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2880b5eb0 con 0x7fa2881591b0 2023-10-06T11:19:05.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.367+0000 7fa26f7fe700 1 -- 172.21.15.46:0/1196568688 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa2880b5eb0 con 0x7fa2881591b0 2023-10-06T11:19:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.375+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 msgr2=0x7fa274062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.375+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 0x7fa274062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa278006fd0 tx=0x7fa278008040 comp rx=0 tx=0).stop 2023-10-06T11:19:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.375+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 msgr2=0x7fa2880d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.375+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa28000c8e0 tx=0x7fa280001710 comp rx=0 tx=0).stop 2023-10-06T11:19:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 shutdown_connections 2023-10-06T11:19:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060550 0x7fa274062a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2881591b0 0x7fa2880d0310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa288158de0 0x7fa2880cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 --2- 172.21.15.46:0/1196568688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2880695a0 0x7fa2880c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 >> 172.21.15.46:0/1196568688 conn(0x7fa288068390 msgr2=0x7fa2880b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 shutdown_connections 2023-10-06T11:19:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.379+0000 7fa28cf63700 1 -- 172.21.15.46:0/1196568688 wait complete. 2023-10-06T11:19:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:19:05.404 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:19:05.405 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:19:05.405 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:19:05.405 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:19:05.405 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:19:05.406 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:19:05.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:19:05.406 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:19:05.406 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:19:05.406 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:19:05.406 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:19:05.406 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:19:05.407 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:19:05.407 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:19:05.407 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:19:05.407 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:19:05.424 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:19:05.424 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:19:05.426 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:19:05.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:19:05.427 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:19:05.427 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:19:05.429 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:19:05.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:19:05.474 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:19:05.475 DEBUG:teuthology.parallel:result is None 2023-10-06T11:19:05.480 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:19:05.480 DEBUG:teuthology.parallel:result is None 2023-10-06T11:19:05.483 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:19:05.484 DEBUG:teuthology.parallel:result is None 2023-10-06T11:19:05.487 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:19:05.488 DEBUG:teuthology.parallel:result is None 2023-10-06T11:19:05.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:19:05.584 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:19:05.590 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:19:05.595 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:19:05.596 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:19:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 Processor -- start 2023-10-06T11:19:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 -- start start 2023-10-06T11:19:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45e0156e20 0x7f45e006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f45e00aaf30 0x7f45e0067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e006d790 con 0x7f45e0067590 2023-10-06T11:19:05.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00679a0 con 0x7f45e0156e20 2023-10-06T11:19:05.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45e4c6b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00acd60 con 0x7f45e00aaf30 2023-10-06T11:19:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45df7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45e0156e20 0x7f45e006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45dffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f45e0067590 0x7f45e006de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55898/0 (socket says 172.21.15.46:55898) 2023-10-06T11:19:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45dffff700 1 -- 172.21.15.46:0/977307056 learned_addr learned my addr 172.21.15.46:0/977307056 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.643+0000 7f45deffd700 1 --2- 172.21.15.46:0/977307056 >> v2:172.21.15.132:3300/0 conn(0x7f45e00aaf30 0x7f45e0067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45de7fc700 1 -- 172.21.15.46:0/977307056 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031485784 0 0) 0x7f45e006d790 con 0x7f45e0067590 2023-10-06T11:19:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45df7fe700 1 -- 172.21.15.46:0/977307056 >> v2:172.21.15.132:3300/0 conn(0x7f45e00aaf30 msgr2=0x7f45e0067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45df7fe700 1 --2- 172.21.15.46:0/977307056 >> v2:172.21.15.132:3300/0 conn(0x7f45e00aaf30 0x7f45e0067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45df7fe700 1 -- 172.21.15.46:0/977307056 >> v1:172.21.15.132:6789/0 conn(0x7f45e0067590 legacy=0x7f45e006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45df7fe700 1 -- 172.21.15.46:0/977307056 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e00ad040 con 0x7f45e0156e20 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45df7fe700 1 --2- 172.21.15.46:0/977307056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e006d250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45e00635d0 tx=0x7f45d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88a5c6da54dcc48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45de7fc700 1 -- 172.21.15.46:0/977307056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0011630 con 0x7f45e0156e20 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45de7fc700 1 -- 172.21.15.46:0/977307056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0011790 con 0x7f45e0156e20 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45de7fc700 1 -- 172.21.15.46:0/977307056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0011b30 con 0x7f45e0156e20 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 -- 172.21.15.46:0/977307056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 msgr2=0x7f45e006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/977307056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e006d250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45e00635d0 tx=0x7f45d00091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 -- 172.21.15.46:0/977307056 shutdown_connections 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/977307056 >> v2:172.21.15.132:3300/0 conn(0x7f45e00aaf30 0x7f45e0067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/977307056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e006d250 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 -- 172.21.15.46:0/977307056 >> 172.21.15.46:0/977307056 conn(0x7f45e0062ef0 msgr2=0x7f45e00cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 -- 172.21.15.46:0/977307056 shutdown_connections 2023-10-06T11:19:05.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.647+0000 7f45e4c6b700 1 -- 172.21.15.46:0/977307056 wait complete. 2023-10-06T11:19:05.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.651+0000 7f45e4c6b700 1 Processor -- start 2023-10-06T11:19:05.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.651+0000 7f45e4c6b700 1 -- start start 2023-10-06T11:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 0x7f45e0157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 0x7f45e00c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e01551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00c3d40 con 0x7f45e0156e20 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00c4020 con 0x7f45e0067590 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00c4300 con 0x7f45e00aaf30 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45df7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 0x7f45e0157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e01551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 0x7f45e00c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45df7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 0x7f45e0157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:53862/0 (socket says 172.21.15.46:53862) 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45df7fe700 1 -- 172.21.15.46:0/283277755 learned_addr learned my addr 172.21.15.46:0/283277755 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 -- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 msgr2=0x7f45e0157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 0x7f45e0157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 -- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 msgr2=0x7f45e01551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e01551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 -- 172.21.15.46:0/283277755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0009420 con 0x7f45e00aaf30 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45deffd700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 0x7f45e00c7560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45d400ea50 tx=0x7f45d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d400ee90 con 0x7f45e00aaf30 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d400c7f0 con 0x7f45e00aaf30 2023-10-06T11:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e00c45e0 con 0x7f45e00aaf30 2023-10-06T11:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d400cb90 con 0x7f45e00aaf30 2023-10-06T11:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.655+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e0158220 con 0x7f45e00aaf30 2023-10-06T11:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.659+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e00c4820 con 0x7f45e00aaf30 2023-10-06T11:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.659+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f45d4013470 con 0x7f45e00aaf30 2023-10-06T11:19:05.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.663+0000 7f45c77fe700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 0x7f45cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:05.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.663+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f45d407c800 con 0x7f45e00aaf30 2023-10-06T11:19:05.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.663+0000 7f45df7fe700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 0x7f45cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.663+0000 7f45df7fe700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 0x7f45cc062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f45d00091e0 tx=0x7f45d0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.671+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f45e00c4820 con 0x7f45e00aaf30 2023-10-06T11:19:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:05.859+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45e00c47d0 con 0x7f45e00aaf30 2023-10-06T11:19:06.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.395+0000 7f45c77fe700 1 -- 172.21.15.46:0/283277755 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45d404cef0 con 0x7f45e00aaf30 2023-10-06T11:19:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 msgr2=0x7f45cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 0x7f45cc062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f45d00091e0 tx=0x7f45d0017000 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 msgr2=0x7f45e00c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 0x7f45e00c7560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45d400ea50 tx=0x7f45d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 shutdown_connections 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45cc060550 0x7f45cc062a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45e0156e20 0x7f45e01551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e00aaf30 0x7f45e00c7560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 --2- 172.21.15.46:0/283277755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e0067590 0x7f45e0157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 >> 172.21.15.46:0/283277755 conn(0x7f45e0062ef0 msgr2=0x7f45e00c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 shutdown_connections 2023-10-06T11:19:06.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.399+0000 7f45e4c6b700 1 -- 172.21.15.46:0/283277755 wait complete. 2023-10-06T11:19:06.405 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:19:06.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:19:06.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.583+0000 7f78181c5700 1 Processor -- start 2023-10-06T11:19:06.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.583+0000 7f78181c5700 1 -- start start 2023-10-06T11:19:06.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f78181c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f781006cc30 0x7f781006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f78181c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78100c18f0 con 0x7f7810161470 2023-10-06T11:19:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f78181c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78100cb120 con 0x7f7810161840 2023-10-06T11:19:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f78181c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78100c0ce0 con 0x7f781006cc30 2023-10-06T11:19:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7815f61700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7810161470 0x7f78100c1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48126/0 (socket says 172.21.15.46:48126) 2023-10-06T11:19:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7815f61700 1 -- 172.21.15.46:0/4148208755 learned_addr learned my addr 172.21.15.46:0/4148208755 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7816762700 1 --2- 172.21.15.46:0/4148208755 >> v2:172.21.15.132:3300/0 conn(0x7f781006cc30 0x7f781006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358918089 0 0) 0x7f78100c18f0 con 0x7f7810161470 2023-10-06T11:19:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77fc003570 con 0x7f7810161470 2023-10-06T11:19:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204408179 0 0) 0x7f78100cb120 con 0x7f7810161840 2023-10-06T11:19:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78100c18f0 con 0x7f7810161840 2023-10-06T11:19:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2341278400 0 0) 0x7f77fc003570 con 0x7f7810161470 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 >> v2:172.21.15.132:3300/0 conn(0x7f781006cc30 msgr2=0x7f781006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 --2- 172.21.15.46:0/4148208755 >> v2:172.21.15.132:3300/0 conn(0x7f781006cc30 0x7f781006d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 >> v1:172.21.15.132:6789/0 conn(0x7f7810161840 legacy=0x7f78100caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100c0fc0 con 0x7f7810161470 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f780c004ee0 con 0x7f7810161470 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 >> v1:172.21.15.46:6789/0 conn(0x7f7810161470 legacy=0x7f78100c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f77fc005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77fc0037b0 0x7f77fc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77fc004160 0x7f77fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7800001230 con 0x7f77fc004c80 2023-10-06T11:19:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78100c0ce0 con 0x7f77fc004160 2023-10-06T11:19:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.587+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc00c100 con 0x7f77fc0037b0 2023-10-06T11:19:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f77fc005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815760700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77fc0037b0 0x7f77fc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7816762700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77fc004160 0x7f77fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 -- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77fc0037b0 msgr2=0x7f77fc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77fc0037b0 0x7f77fc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 -- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77fc004160 msgr2=0x7f77fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77fc004160 0x7f77fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 -- 172.21.15.46:0/4148208755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc003570 con 0x7f77fc004c80 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7816762700 1 -- 172.21.15.46:0/4148208755 reap_dead start 2023-10-06T11:19:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7815f61700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f77fc005050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f780c0058d0 tx=0x7f780c005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85047d34bf6062a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c00cba0 con 0x7f77fc004c80 2023-10-06T11:19:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c00cd00 con 0x7f77fc004c80 2023-10-06T11:19:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f7814f5f700 1 -- 172.21.15.46:0/4148208755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c005b70 con 0x7f77fc004c80 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 -- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 msgr2=0x7f77fc005050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f77fc005050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f780c0058d0 tx=0x7f780c005860 comp rx=0 tx=0).stop 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 -- 172.21.15.46:0/4148208755 shutdown_connections 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 --2- 172.21.15.46:0/4148208755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f77fc005050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 -- 172.21.15.46:0/4148208755 >> 172.21.15.46:0/4148208755 conn(0x7f78100cbe30 msgr2=0x7f78100cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 -- 172.21.15.46:0/4148208755 shutdown_connections 2023-10-06T11:19:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.591+0000 7f78181c5700 1 -- 172.21.15.46:0/4148208755 wait complete. 2023-10-06T11:19:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 Processor -- start 2023-10-06T11:19:06.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 -- start start 2023-10-06T11:19:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78100b07b0 0x7f78100c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 0x7f781006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78101587e0 con 0x7f77fc004c80 2023-10-06T11:19:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78100b0320 con 0x7f78100b07b0 2023-10-06T11:19:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f78181c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78100b1430 con 0x7f78100c50b0 2023-10-06T11:19:06.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51592/0 (socket says 172.21.15.46:51592) 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 -- 172.21.15.46:0/1785871814 learned_addr learned my addr 172.21.15.46:0/1785871814 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815760700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78100b07b0 0x7f78100c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7816762700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 0x7f781006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 -- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 msgr2=0x7f781006a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 0x7f781006a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 -- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78100b07b0 msgr2=0x7f78100c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78100b07b0 0x7f78100c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.595+0000 7f7815f61700 1 -- 172.21.15.46:0/1785871814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100af9f0 con 0x7f77fc004c80 2023-10-06T11:19:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f7816762700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 0x7f781006a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f7815f61700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c050 tx=0x7f780c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c004830 con 0x7f77fc004c80 2023-10-06T11:19:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c0072e0 con 0x7f77fc004c80 2023-10-06T11:19:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0076a0 con 0x7f77fc004c80 2023-10-06T11:19:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78100afcd0 con 0x7f77fc004c80 2023-10-06T11:19:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78100b0c20 con 0x7f77fc004c80 2023-10-06T11:19:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.599+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78100c5480 con 0x7f77fc004c80 2023-10-06T11:19:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.603+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f780c019030 con 0x7f77fc004c80 2023-10-06T11:19:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.603+0000 7f7805ffb700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 0x7f77f4062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.603+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f780c07dc90 con 0x7f77fc004c80 2023-10-06T11:19:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.603+0000 7f7815760700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 0x7f77f4062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:06.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.603+0000 7f7815760700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 0x7f77f4062ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f78100c4770 tx=0x7f7800009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:06.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.611+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f780c04e450 con 0x7f77fc004c80 2023-10-06T11:19:06.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.847+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7810063f90 con 0x7f77fc004c80 2023-10-06T11:19:06.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.847+0000 7f7805ffb700 1 -- 172.21.15.46:0/1785871814 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f780c017090 con 0x7f77fc004c80 2023-10-06T11:19:06.852 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:06.852 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.854 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:19:06.855 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.857 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:19:06.859 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.860 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:19:06.862 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.863 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:19:06.864 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:19:06.865 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:19:06.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lBjTfEBAAo6yUr0DQUgioOaYnCW3Zmw==", 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lTUHfEBAAS48LvMdPWQCQLVBf6jhOSw==", 2023-10-06T11:19:06.867 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lYk7fEBAA2zh7UggrYelmGL9AfTpFfQ==", 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.868 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lelnfEBAA+IPHd90S3VT9VrUCZKb+Vw==", 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lb2TfEBAA+SfCy1GdI8ToUBk03a455g==", 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDl7B9lwG/fEBAAqKQxbXLVLjfc80h6uOJhAA==", 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.870 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.871 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:06.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:19:06.872 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:19:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 msgr2=0x7f77f4062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 0x7f77f4062ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f78100c4770 tx=0x7f7800009040 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 msgr2=0x7f78101582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c050 tx=0x7f780c015000 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 shutdown_connections 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77f4060620 0x7f77f4062ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78100c50b0 0x7f781006a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78100b07b0 0x7f78100c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 --2- 172.21.15.46:0/1785871814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77fc004c80 0x7f78101582a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 >> 172.21.15.46:0/1785871814 conn(0x7f78100cbe30 msgr2=0x7f78100ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 shutdown_connections 2023-10-06T11:19:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:06.851+0000 7f78181c5700 1 -- 172.21.15.46:0/1785871814 wait complete. 2023-10-06T11:19:06.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T11:19:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.035+0000 7f05192cd700 1 Processor -- start 2023-10-06T11:19:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.035+0000 7f05192cd700 1 -- start start 2023-10-06T11:19:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f05140aef50 0x7f05140af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05140cf5d0 con 0x7f05140cfa60 2023-10-06T11:19:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05140af860 con 0x7f0514069090 2023-10-06T11:19:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140ce9c0 con 0x7f05140aef50 2023-10-06T11:19:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05127fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f05140cfa60 0x7f05140caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48128/0 (socket says 172.21.15.46:48128) 2023-10-06T11:19:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05127fc700 1 -- 172.21.15.46:0/1677769632 learned_addr learned my addr 172.21.15.46:0/1677769632 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0512ffd700 1 --2- 172.21.15.46:0/1677769632 >> v2:172.21.15.132:3300/0 conn(0x7f05140aef50 0x7f05140af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0511ffb700 1 -- 172.21.15.46:0/1677769632 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817985312 0 0) 0x7f05140cf5d0 con 0x7f05140cfa60 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0511ffb700 1 -- 172.21.15.46:0/1677769632 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04fc003570 con 0x7f05140cfa60 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0512ffd700 1 -- 172.21.15.46:0/1677769632 >> v1:172.21.15.132:6789/0 conn(0x7f0514069090 legacy=0x7f05140694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0512ffd700 1 -- 172.21.15.46:0/1677769632 >> v1:172.21.15.46:6789/0 conn(0x7f05140cfa60 legacy=0x7f05140caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0512ffd700 1 -- 172.21.15.46:0/1677769632 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140ceca0 con 0x7f05140aef50 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0512ffd700 1 --2- 172.21.15.46:0/1677769632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05140aef50 0x7f05140af320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f05040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09d0cea4b042b94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0511ffb700 1 -- 172.21.15.46:0/1677769632 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400b640 con 0x7f05140aef50 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0511ffb700 1 -- 172.21.15.46:0/1677769632 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050400b7a0 con 0x7f05140aef50 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f0511ffb700 1 -- 172.21.15.46:0/1677769632 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400bb40 con 0x7f05140aef50 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 -- 172.21.15.46:0/1677769632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05140aef50 msgr2=0x7f05140af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.039+0000 7f05192cd700 1 --2- 172.21.15.46:0/1677769632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05140aef50 0x7f05140af320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05040098e0 tx=0x7f05040091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- 172.21.15.46:0/1677769632 shutdown_connections 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 --2- 172.21.15.46:0/1677769632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05140aef50 0x7f05140af320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- 172.21.15.46:0/1677769632 >> 172.21.15.46:0/1677769632 conn(0x7f051406f200 msgr2=0x7f05140d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- 172.21.15.46:0/1677769632 shutdown_connections 2023-10-06T11:19:07.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- 172.21.15.46:0/1677769632 wait complete. 2023-10-06T11:19:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 Processor -- start 2023-10-06T11:19:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- start start 2023-10-06T11:19:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 0x7f0514169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05140aef50 0x7f051414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 0x7f0514155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b5e70 con 0x7f05140cfa60 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051416d2a0 con 0x7f0514069090 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05192cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b5b90 con 0x7f05140aef50 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.043+0000 7f05137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 0x7f0514155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 0x7f0514155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51600/0 (socket says 172.21.15.46:51600) 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05137fe700 1 -- 172.21.15.46:0/144235901 learned_addr learned my addr 172.21.15.46:0/144235901 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 0x7f0514169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05127fc700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05140aef50 0x7f051414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 -- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05140aef50 msgr2=0x7f051414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05140aef50 0x7f051414c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 -- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 msgr2=0x7f0514155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 0x7f0514155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 -- 172.21.15.46:0/144235901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504009420 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f0512ffd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 0x7f0514169d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0504016000 tx=0x7f0504017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504000d10 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050400bb20 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504011670 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051416cc50 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140b57d0 con 0x7f0514069090 2023-10-06T11:19:07.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.047+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0514064590 con 0x7f0514069090 2023-10-06T11:19:07.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.051+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f05040119e0 con 0x7f0514069090 2023-10-06T11:19:07.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.051+0000 7f04faffd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 0x7f0500062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.051+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f050407e120 con 0x7f0514069090 2023-10-06T11:19:07.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.051+0000 7f05127fc700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 0x7f0500062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.055+0000 7f05127fc700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 0x7f0500062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05141549e0 tx=0x7f0508009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.059+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f050404e8e0 con 0x7f0514069090 2023-10-06T11:19:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.283+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f05140cef80 con 0x7f0514069090 2023-10-06T11:19:07.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.295+0000 7f04faffd700 1 -- 172.21.15.46:0/144235901 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0504051f00 con 0x7f0514069090 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 msgr2=0x7f0500062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 0x7f0500062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f05141549e0 tx=0x7f0508009040 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 msgr2=0x7f0514169d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 0x7f0514169d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0504016000 tx=0x7f0504017000 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 shutdown_connections 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0500060550 0x7f0500062a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05140cfa60 0x7f0514155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05140aef50 0x7f051414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 --2- 172.21.15.46:0/144235901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0514069090 0x7f0514169d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 >> 172.21.15.46:0/144235901 conn(0x7f051406f200 msgr2=0x7f051414d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 shutdown_connections 2023-10-06T11:19:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.299+0000 7f05192cd700 1 -- 172.21.15.46:0/144235901 wait complete. 2023-10-06T11:19:07.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T11:19:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 Processor -- start 2023-10-06T11:19:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 -- start start 2023-10-06T11:19:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72600acf50 0x7f72601595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 0x7f7260068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7260153b60 con 0x7f7260159b30 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7260159fa0 con 0x7f72600acf50 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.491+0000 7f72658f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72600689c0 con 0x7f72600685a0 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725e7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 0x7f7260068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72600acf50 0x7f72601595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:34656/0 (socket says 172.21.15.46:34656) 2023-10-06T11:19:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72600acf50 0x7f72601595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:53930/0 (socket says 172.21.15.46:53930) 2023-10-06T11:19:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725f7fe700 1 -- 172.21.15.46:0/3229036269 learned_addr learned my addr 172.21.15.46:0/3229036269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 -- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 msgr2=0x7f726015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 -- 172.21.15.46:0/3229036269 >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 msgr2=0x7f7260068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 --2- 172.21.15.46:0/3229036269 >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 0x7f7260068060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 -- 172.21.15.46:0/3229036269 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600aec30 con 0x7f72600acf50 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725f7fe700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725e7fc700 1 --2- 172.21.15.46:0/3229036269 >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 0x7f7260068060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725effd700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72600acf50 0x7f72601595f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f726006c7f0 tx=0x7f72500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9224e8c121f238f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725dffb700 1 -- 172.21.15.46:0/3229036269 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250011630 con 0x7f72600acf50 2023-10-06T11:19:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725dffb700 1 -- 172.21.15.46:0/3229036269 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72500117b0 con 0x7f72600acf50 2023-10-06T11:19:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f725dffb700 1 -- 172.21.15.46:0/3229036269 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250011b70 con 0x7f72600acf50 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f72658f0700 1 -- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72600acf50 msgr2=0x7f72601595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.495+0000 7f72658f0700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72600acf50 0x7f72601595f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f726006c7f0 tx=0x7f72500091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 -- 172.21.15.46:0/3229036269 shutdown_connections 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 --2- 172.21.15.46:0/3229036269 >> v2:172.21.15.46:3300/0 conn(0x7f7260159b30 0x7f7260068060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 --2- 172.21.15.46:0/3229036269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72600acf50 0x7f72601595f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 -- 172.21.15.46:0/3229036269 >> 172.21.15.46:0/3229036269 conn(0x7f726006a8e0 msgr2=0x7f726006ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 -- 172.21.15.46:0/3229036269 shutdown_connections 2023-10-06T11:19:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 -- 172.21.15.46:0/3229036269 wait complete. 2023-10-06T11:19:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 Processor -- start 2023-10-06T11:19:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.499+0000 7f72658f0700 1 -- start start 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726014dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 0x7f72601546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600afe80 con 0x7f72600acf50 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726014e0f0 con 0x7f7260159b30 2023-10-06T11:19:07.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72600af270 con 0x7f72600685a0 2023-10-06T11:19:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51616/0 (socket says 172.21.15.46:51616) 2023-10-06T11:19:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 -- 172.21.15.46:0/703317716 learned_addr learned my addr 172.21.15.46:0/703317716 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725effd700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726014dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725f7fe700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 0x7f72601546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 -- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 msgr2=0x7f726014dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726014dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 -- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 msgr2=0x7f72601546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 0x7f72601546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 -- 172.21.15.46:0/703317716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258009ae0 con 0x7f72600acf50 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725f7fe700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 0x7f72601546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f725e7fc700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f725400ecb0 tx=0x7f72540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400c920 con 0x7f72600acf50 2023-10-06T11:19:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725400ca80 con 0x7f72600acf50 2023-10-06T11:19:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400ce00 con 0x7f72600acf50 2023-10-06T11:19:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.503+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7254009420 con 0x7f72600acf50 2023-10-06T11:19:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.507+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7250009420 con 0x7f72600acf50 2023-10-06T11:19:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.507+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72600b0030 con 0x7f72600acf50 2023-10-06T11:19:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.507+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7254013680 con 0x7f72600acf50 2023-10-06T11:19:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.511+0000 7f7246ffd700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 0x7f724c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.511+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f725407c8a0 con 0x7f72600acf50 2023-10-06T11:19:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.511+0000 7f725effd700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 0x7f724c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.511+0000 7f725effd700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 0x7f724c062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f726006c7f0 tx=0x7f7250017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.515+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f72600b0030 con 0x7f72600acf50 2023-10-06T11:19:07.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.747+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f72600b0030 con 0x7f72600acf50 2023-10-06T11:19:07.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f7246ffd700 1 -- 172.21.15.46:0/703317716 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f72600b0030 con 0x7f72600acf50 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 msgr2=0x7f724c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 0x7f724c062a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f726006c7f0 tx=0x7f7250017000 comp rx=0 tx=0).stop 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 msgr2=0x7f72601491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f725400ecb0 tx=0x7f72540091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 shutdown_connections 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f724c060550 0x7f724c062a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7260159b30 0x7f72601546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72600acf50 0x7f72601491d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 --2- 172.21.15.46:0/703317716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72600685a0 0x7f726014dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 >> 172.21.15.46:0/703317716 conn(0x7f726006a8e0 msgr2=0x7f72601571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 shutdown_connections 2023-10-06T11:19:07.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.755+0000 7f72658f0700 1 -- 172.21.15.46:0/703317716 wait complete. 2023-10-06T11:19:07.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T11:19:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.935+0000 7fe34e22e700 1 Processor -- start 2023-10-06T11:19:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.935+0000 7fe34e22e700 1 -- start start 2023-10-06T11:19:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe34e22e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3480ca280 0x7fe348068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe34e22e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe34815b620 0x7fe3481569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe34e22e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480c8a40 con 0x7fe348068290 2023-10-06T11:19:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe34e22e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3480cb470 con 0x7fe3480ca280 2023-10-06T11:19:07.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe34e22e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3480ca860 con 0x7fe34815b620 2023-10-06T11:19:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3480ca280 0x7fe348068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3480ca280 0x7fe348068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51630/0 (socket says 172.21.15.46:51630) 2023-10-06T11:19:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 -- 172.21.15.46:0/672757095 learned_addr learned my addr 172.21.15.46:0/672757095 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe347fff700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe34815b620 0x7fe3481569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 -- 172.21.15.46:0/672757095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe34815b620 msgr2=0x7fe3481569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe34815b620 0x7fe3481569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 -- 172.21.15.46:0/672757095 >> v1:172.21.15.132:6789/0 conn(0x7fe348068290 legacy=0x7fe3480abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 -- 172.21.15.46:0/672757095 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480cab40 con 0x7fe3480ca280 2023-10-06T11:19:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.939+0000 7fe33ffff700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3480ca280 0x7fe348068660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe3300099b0 tx=0x7fe330009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253a6c4da84fa063 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe346ffd700 1 -- 172.21.15.46:0/672757095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330009ea0 con 0x7fe3480ca280 2023-10-06T11:19:07.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe346ffd700 1 -- 172.21.15.46:0/672757095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33000b860 con 0x7fe3480ca280 2023-10-06T11:19:07.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe346ffd700 1 -- 172.21.15.46:0/672757095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33000bc00 con 0x7fe3480ca280 2023-10-06T11:19:07.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 -- 172.21.15.46:0/672757095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3480ca280 msgr2=0x7fe348068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3480ca280 0x7fe348068660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe3300099b0 tx=0x7fe330009250 comp rx=0 tx=0).stop 2023-10-06T11:19:07.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 -- 172.21.15.46:0/672757095 shutdown_connections 2023-10-06T11:19:07.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe34815b620 0x7fe3481569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 --2- 172.21.15.46:0/672757095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3480ca280 0x7fe348068660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 -- 172.21.15.46:0/672757095 >> 172.21.15.46:0/672757095 conn(0x7fe34806d490 msgr2=0x7fe34806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 -- 172.21.15.46:0/672757095 shutdown_connections 2023-10-06T11:19:07.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.943+0000 7fe34e22e700 1 -- 172.21.15.46:0/672757095 wait complete. 2023-10-06T11:19:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 Processor -- start 2023-10-06T11:19:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 -- start start 2023-10-06T11:19:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 0x7fe3480acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 0x7fe3480ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3481537a0 con 0x7fe348068290 2023-10-06T11:19:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348152790 con 0x7fe34815b620 2023-10-06T11:19:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe34e22e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe348153310 con 0x7fe3480ca280 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe3477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe3477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51642/0 (socket says 172.21.15.46:51642) 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe3477fe700 1 -- 172.21.15.46:0/4231436909 learned_addr learned my addr 172.21.15.46:0/4231436909 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.947+0000 7fe347fff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 0x7fe3480ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe33ffff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 0x7fe3480acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 -- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 msgr2=0x7fe3480acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 0x7fe3480acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 -- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 msgr2=0x7fe3480ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 0x7fe3480ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 -- 172.21.15.46:0/4231436909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300094f0 con 0x7fe348068290 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe347fff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 0x7fe3480ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe33ffff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 0x7fe3480acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe3477fe700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c5c0 tx=0x7fe33800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338011590 con 0x7fe348068290 2023-10-06T11:19:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338011710 con 0x7fe348068290 2023-10-06T11:19:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338011ad0 con 0x7fe348068290 2023-10-06T11:19:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3481529e0 con 0x7fe348068290 2023-10-06T11:19:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe348152fa0 con 0x7fe348068290 2023-10-06T11:19:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.951+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34806c110 con 0x7fe348068290 2023-10-06T11:19:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.955+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe33801b3b0 con 0x7fe348068290 2023-10-06T11:19:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.955+0000 7fe33f7fe700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 0x7fe328062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.955+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe33807bc10 con 0x7fe348068290 2023-10-06T11:19:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.955+0000 7fe33ffff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 0x7fe328062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.955+0000 7fe33ffff700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 0x7fe328062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3300099b0 tx=0x7fe330017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:07.963+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe33804c350 con 0x7fe348068290 2023-10-06T11:19:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.191+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe3480cae20 con 0x7fe348068290 2023-10-06T11:19:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.203+0000 7fe33f7fe700 1 -- 172.21.15.46:0/4231436909 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe338017020 con 0x7fe348068290 2023-10-06T11:19:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.203+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 msgr2=0x7fe328062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.203+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 0x7fe328062a00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3300099b0 tx=0x7fe330017000 comp rx=0 tx=0).stop 2023-10-06T11:19:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.203+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 msgr2=0x7fe3481681c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.203+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c5c0 tx=0x7fe33800c980 comp rx=0 tx=0).stop 2023-10-06T11:19:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 shutdown_connections 2023-10-06T11:19:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe328060550 0x7fe328062a00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe34815b620 0x7fe3480ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe3480ca280 0x7fe3480acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 --2- 172.21.15.46:0/4231436909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe348068290 0x7fe3481681c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 >> 172.21.15.46:0/4231436909 conn(0x7fe34806d490 msgr2=0x7fe3480cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 shutdown_connections 2023-10-06T11:19:08.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.207+0000 7fe34e22e700 1 -- 172.21.15.46:0/4231436909 wait complete. 2023-10-06T11:19:08.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T11:19:08.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.395+0000 7fac41d23700 1 Processor -- start 2023-10-06T11:19:08.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac41d23700 1 -- start start 2023-10-06T11:19:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac41d23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac3c0af030 0x7fac3c0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac41d23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3c15aa40 con 0x7fac3c156ff0 2023-10-06T11:19:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac41d23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3c15ad20 con 0x7fac3c156c20 2023-10-06T11:19:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac41d23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3c0ca3f0 con 0x7fac3c0af030 2023-10-06T11:19:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.399+0000 7fac3bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac3c0af030 0x7fac3c0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41522700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fac3c156ff0 0x7fac3c0c9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48130/0 (socket says 172.21.15.46:48130) 2023-10-06T11:19:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41522700 1 -- 172.21.15.46:0/263883548 learned_addr learned my addr 172.21.15.46:0/263883548 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291237360 0 0) 0x7fac3c15ad20 con 0x7fac3c156c20 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac2c003570 con 0x7fac3c156c20 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3bfff700 1 -- 172.21.15.46:0/263883548 >> v1:172.21.15.132:6789/0 conn(0x7fac3c156c20 legacy=0x7fac3c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831055763 0 0) 0x7fac3c15aa40 con 0x7fac3c156ff0 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3bfff700 1 -- 172.21.15.46:0/263883548 >> v1:172.21.15.46:6789/0 conn(0x7fac3c156ff0 legacy=0x7fac3c0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3bfff700 1 -- 172.21.15.46:0/263883548 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c15b000 con 0x7fac3c0af030 2023-10-06T11:19:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3bfff700 1 --2- 172.21.15.46:0/263883548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0c8bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fac280098e0 tx=0x7fac280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11af6831df06083f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800b640 con 0x7fac3c0af030 2023-10-06T11:19:08.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2800b7a0 con 0x7fac3c0af030 2023-10-06T11:19:08.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac3b7fe700 1 -- 172.21.15.46:0/263883548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800bb40 con 0x7fac3c0af030 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 -- 172.21.15.46:0/263883548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 msgr2=0x7fac3c0c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 --2- 172.21.15.46:0/263883548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0c8bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fac280098e0 tx=0x7fac280091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 -- 172.21.15.46:0/263883548 shutdown_connections 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 --2- 172.21.15.46:0/263883548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0c8bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 -- 172.21.15.46:0/263883548 >> 172.21.15.46:0/263883548 conn(0x7fac3c0ccd50 msgr2=0x7fac3c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 -- 172.21.15.46:0/263883548 shutdown_connections 2023-10-06T11:19:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.403+0000 7fac41d23700 1 -- 172.21.15.46:0/263883548 wait complete. 2023-10-06T11:19:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 Processor -- start 2023-10-06T11:19:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 -- start start 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac3c156c20 0x7fac3c145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 0x7fac3c167010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c150a20 con 0x7fac3c0af030 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c0a5970 con 0x7fac3c156c20 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac41d23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c0b0500 con 0x7fac3c156ff0 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac40d21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac40d21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51660/0 (socket says 172.21.15.46:51660) 2023-10-06T11:19:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.407+0000 7fac40d21700 1 -- 172.21.15.46:0/1567736093 learned_addr learned my addr 172.21.15.46:0/1567736093 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:08.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac41522700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 0x7fac3c167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 -- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 msgr2=0x7fac3c167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 0x7fac3c167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 -- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac3c156c20 msgr2=0x7fac3c145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac3c156c20 0x7fac3c145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 -- 172.21.15.46:0/1567736093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac28009420 con 0x7fac3c0af030 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac41522700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 0x7fac3c167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac40d21700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fac3000cb10 tx=0x7fac30002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000a760 con 0x7fac3c0af030 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000a8c0 con 0x7fac3c0af030 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000ac60 con 0x7fac3c0af030 2023-10-06T11:19:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c0b1080 con 0x7fac3c0af030 2023-10-06T11:19:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c0b0a30 con 0x7fac3c0af030 2023-10-06T11:19:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.411+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac3c064590 con 0x7fac3c0af030 2023-10-06T11:19:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.415+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fac30012630 con 0x7fac3c0af030 2023-10-06T11:19:08.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.415+0000 7fac38ff9700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 0x7fac20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.415+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fac3007ca30 con 0x7fac3c0af030 2023-10-06T11:19:08.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.415+0000 7fac3bfff700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 0x7fac20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.419+0000 7fac3bfff700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 0x7fac20062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac280098e0 tx=0x7fac28017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.423+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fac3004d170 con 0x7fac3c0af030 2023-10-06T11:19:08.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.655+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fac3c15b2e0 con 0x7fac3c0af030 2023-10-06T11:19:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.663+0000 7fac38ff9700 1 -- 172.21.15.46:0/1567736093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fac30050790 con 0x7fac3c0af030 2023-10-06T11:19:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 msgr2=0x7fac20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 0x7fac20062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac280098e0 tx=0x7fac28017000 comp rx=0 tx=0).stop 2023-10-06T11:19:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 msgr2=0x7fac3c0a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fac3000cb10 tx=0x7fac30002820 comp rx=0 tx=0).stop 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 shutdown_connections 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac20060550 0x7fac20062a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac3c156ff0 0x7fac3c167010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac3c156c20 0x7fac3c145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 --2- 172.21.15.46:0/1567736093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac3c0af030 0x7fac3c0a5430 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 >> 172.21.15.46:0/1567736093 conn(0x7fac3c0ccd50 msgr2=0x7fac3c151770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 shutdown_connections 2023-10-06T11:19:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.667+0000 7fac41d23700 1 -- 172.21.15.46:0/1567736093 wait complete. 2023-10-06T11:19:08.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T11:19:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.855+0000 7f3cd386e700 1 Processor -- start 2023-10-06T11:19:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd386e700 1 -- start start 2023-10-06T11:19:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd386e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3ccc0c7b50 0x7f3ccc0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd386e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0c8500 con 0x7f3ccc156c80 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd386e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0c6470 con 0x7f3ccc0c0bd0 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd386e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c7760 con 0x7f3ccc0c7b50 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd160a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ccc156c80 0x7f3ccc0ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48142/0 (socket says 172.21.15.46:48142) 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd160a700 1 -- 172.21.15.46:0/2499758977 learned_addr learned my addr 172.21.15.46:0/2499758977 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cd0e09700 1 --2- 172.21.15.46:0/2499758977 >> v2:172.21.15.132:3300/0 conn(0x7f3ccc0c7b50 0x7f3ccc0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369450550 0 0) 0x7f3ccc0c8500 con 0x7f3ccc156c80 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb8003570 con 0x7f3ccc156c80 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100922344 0 0) 0x7f3ccc0c6470 con 0x7f3ccc0c0bd0 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ccc0c8500 con 0x7f3ccc0c0bd0 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.859+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 961472303 0 0) 0x7f3cb8003570 con 0x7f3ccc156c80 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 >> v2:172.21.15.132:3300/0 conn(0x7f3ccc0c7b50 msgr2=0x7f3ccc0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:08.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 --2- 172.21.15.46:0/2499758977 >> v2:172.21.15.132:3300/0 conn(0x7f3ccc0c7b50 0x7f3ccc0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 >> v1:172.21.15.132:6789/0 conn(0x7f3ccc0c0bd0 legacy=0x7f3ccc0c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0c66c0 con 0x7f3ccc156c80 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3cc8004ee0 con 0x7f3ccc156c80 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 >> v1:172.21.15.46:6789/0 conn(0x7f3ccc156c80 legacy=0x7f3ccc0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cb8003bf0 0x7f3cb8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb8004810 0x7f3cb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 0x7f3cb800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c7760 con 0x7f3cb80051c0 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb800c0b0 con 0x7f3cb8003bf0 2023-10-06T11:19:08.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb800c270 con 0x7f3cb8004810 2023-10-06T11:19:08.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 0x7f3cb800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd160a700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cb8003bf0 0x7f3cb8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd0e09700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb8004810 0x7f3cb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 -- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb8004810 msgr2=0x7f3cb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb8004810 0x7f3cb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 -- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cb8003bf0 msgr2=0x7f3cb8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cb8003bf0 0x7f3cb8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 -- 172.21.15.46:0/2499758977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8003570 con 0x7f3cb80051c0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 -- 172.21.15.46:0/2499758977 reap_dead start 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 0x7f3cb800ba60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4002120 tx=0x7f3cc40017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7720d2b63b520eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400c980 con 0x7f3cb80051c0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc400cae0 con 0x7f3cb80051c0 2023-10-06T11:19:08.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cc3fff700 1 -- 172.21.15.46:0/2499758977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400ce80 con 0x7f3cb80051c0 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd386e700 1 -- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 msgr2=0x7f3cb800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd386e700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 0x7f3cb800ba60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4002120 tx=0x7f3cc40017a0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd386e700 1 -- 172.21.15.46:0/2499758977 shutdown_connections 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd386e700 1 --2- 172.21.15.46:0/2499758977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cb80051c0 0x7f3cb800ba60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.863+0000 7f3cd386e700 1 -- 172.21.15.46:0/2499758977 >> 172.21.15.46:0/2499758977 conn(0x7f3ccc0cbe30 msgr2=0x7f3ccc0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.867+0000 7f3cd386e700 1 -- 172.21.15.46:0/2499758977 shutdown_connections 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.867+0000 7f3cd386e700 1 -- 172.21.15.46:0/2499758977 wait complete. 2023-10-06T11:19:08.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.867+0000 7f3cd386e700 1 Processor -- start 2023-10-06T11:19:08.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.867+0000 7f3cd386e700 1 -- start start 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 0x7f3ccc0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 0x7f3ccc0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc14c240 con 0x7f3ccc0ad970 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b1860 con 0x7f3ccc16dae0 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc06e2a0 con 0x7f3cb80051c0 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51678/0 (socket says 172.21.15.46:51678) 2023-10-06T11:19:08.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 -- 172.21.15.46:0/1438724780 learned_addr learned my addr 172.21.15.46:0/1438724780 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd160a700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 0x7f3ccc0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 0x7f3ccc0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 -- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 msgr2=0x7f3ccc0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 0x7f3ccc0b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 -- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 msgr2=0x7f3ccc0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 0x7f3ccc0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 -- 172.21.15.46:0/1438724780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc06e580 con 0x7f3ccc0ad970 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd1e0b700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 0x7f3ccc0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd160a700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 0x7f3ccc0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd0e09700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006ac0 tx=0x7f3cbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00f9b0 con 0x7f3ccc0ad970 2023-10-06T11:19:08.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00fb10 con 0x7f3ccc0ad970 2023-10-06T11:19:08.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc06e860 con 0x7f3ccc0ad970 2023-10-06T11:19:08.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00fe90 con 0x7f3ccc0ad970 2023-10-06T11:19:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc06ee20 con 0x7f3ccc0ad970 2023-10-06T11:19:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.871+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc063f90 con 0x7f3ccc0ad970 2023-10-06T11:19:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.875+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3cbc014650 con 0x7f3ccc0ad970 2023-10-06T11:19:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.875+0000 7f3cc17fa700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 0x7f3cb0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.875+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3cbc018030 con 0x7f3ccc0ad970 2023-10-06T11:19:08.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.875+0000 7f3cd160a700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 0x7f3cb0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:08.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.879+0000 7f3cd160a700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 0x7f3cb0062a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8000e10 tx=0x7f3cc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:08.883+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3cbc04d760 con 0x7f3ccc0ad970 2023-10-06T11:19:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.115+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3ccc04b5c0 con 0x7f3ccc0ad970 2023-10-06T11:19:09.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.119+0000 7f3cc17fa700 1 -- 172.21.15.46:0/1438724780 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3cbc022020 con 0x7f3ccc0ad970 2023-10-06T11:19:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 msgr2=0x7f3cb0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 0x7f3cb0062a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8000e10 tx=0x7f3cc8009040 comp rx=0 tx=0).stop 2023-10-06T11:19:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 msgr2=0x7f3ccc16d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006ac0 tx=0x7f3cbc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 shutdown_connections 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3cb0060580 0x7f3cb0062a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ccc16dae0 0x7f3ccc0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ccc0ad970 0x7f3ccc16d5a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 --2- 172.21.15.46:0/1438724780 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cb80051c0 0x7f3ccc0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 >> 172.21.15.46:0/1438724780 conn(0x7f3ccc0cbe30 msgr2=0x7f3ccc068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 shutdown_connections 2023-10-06T11:19:09.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.123+0000 7f3cd386e700 1 -- 172.21.15.46:0/1438724780 wait complete. 2023-10-06T11:19:09.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T11:19:09.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.311+0000 7f8eb6c2e700 1 Processor -- start 2023-10-06T11:19:09.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.311+0000 7f8eb6c2e700 1 -- start start 2023-10-06T11:19:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb6c2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8eb00c1ec0 0x7f8eb00ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb6c2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb006d460 con 0x7f8eb015ab60 2023-10-06T11:19:09.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb6c2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb00c7000 con 0x7f8eb006e8a0 2023-10-06T11:19:09.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb6c2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb006ed40 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8eb006e8a0 0x7f8eb006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55942/0 (socket says 172.21.15.46:55942) 2023-10-06T11:19:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaffff700 1 -- 172.21.15.46:0/3575763995 learned_addr learned my addr 172.21.15.46:0/3575763995 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb49ca700 1 --1- 172.21.15.46:0/3575763995 >> v1:172.21.15.46:6789/0 conn(0x7f8eb015ab60 0x7f8eb015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48156/0 (socket says 172.21.15.46:48156) 2023-10-06T11:19:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb51cb700 1 --2- 172.21.15.46:0/3575763995 >> v2:172.21.15.132:3300/0 conn(0x7f8eb00c1ec0 0x7f8eb00ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385542002 0 0) 0x7f8eb006d460 con 0x7f8eb015ab60 2023-10-06T11:19:09.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea0003570 con 0x7f8eb015ab60 2023-10-06T11:19:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477102085 0 0) 0x7f8eb00c7000 con 0x7f8eb006e8a0 2023-10-06T11:19:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb51cb700 1 -- 172.21.15.46:0/3575763995 >> v1:172.21.15.132:6789/0 conn(0x7f8eb006e8a0 legacy=0x7f8eb006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb51cb700 1 -- 172.21.15.46:0/3575763995 >> v1:172.21.15.46:6789/0 conn(0x7f8eb015ab60 legacy=0x7f8eb015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb51cb700 1 -- 172.21.15.46:0/3575763995 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00c72c0 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eb51cb700 1 --2- 172.21.15.46:0/3575763995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00ad6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009e60 tx=0x7f8ea8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a733ad2ec69193a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.315+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800e070 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea800bb60 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eaf7fe700 1 -- 172.21.15.46:0/3575763995 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8012430 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3575763995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb00c1ec0 msgr2=0x7f8eb00ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3575763995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00ad6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009e60 tx=0x7f8ea8009780 comp rx=0 tx=0).stop 2023-10-06T11:19:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3575763995 shutdown_connections 2023-10-06T11:19:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3575763995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00ad6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3575763995 >> 172.21.15.46:0/3575763995 conn(0x7f8eb00cbe30 msgr2=0x7f8eb00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3575763995 shutdown_connections 2023-10-06T11:19:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.319+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3575763995 wait complete. 2023-10-06T11:19:09.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 Processor -- start 2023-10-06T11:19:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 -- start start 2023-10-06T11:19:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8eb006e8a0 0x7f8eb014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb015ab60 0x7f8eb013f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb013f170 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00c5200 con 0x7f8eb015ab60 2023-10-06T11:19:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb6c2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00c45f0 con 0x7f8eb006e8a0 2023-10-06T11:19:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51684/0 (socket says 172.21.15.46:51684) 2023-10-06T11:19:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 -- 172.21.15.46:0/3200892252 learned_addr learned my addr 172.21.15.46:0/3200892252 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb49ca700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8eb006e8a0 0x7f8eb014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eb51cb700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb015ab60 0x7f8eb013f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 -- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8eb006e8a0 msgr2=0x7f8eb014f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8eb006e8a0 0x7f8eb014f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 -- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb015ab60 msgr2=0x7f8eb013f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb015ab60 0x7f8eb013f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.323+0000 7f8eaffff700 1 -- 172.21.15.46:0/3200892252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea8009a20 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eaffff700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c5b0 tx=0x7f8e9c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c005b30 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c005c90 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb00c48d0 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c007300 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb00c4e90 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.327+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb0063f90 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.331+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8e9c01b3b0 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.331+0000 7f8eacff9700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 0x7f8e94062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.331+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8e9c07b9f0 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.331+0000 7f8eb49ca700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 0x7f8e94062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.331+0000 7f8eb49ca700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 0x7f8e94062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007fd0 tx=0x7f8ea4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.335+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8e9c04c1d0 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.571+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8eb00c5340 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.579+0000 7f8eacff9700 1 -- 172.21.15.46:0/3200892252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8e9c053e60 con 0x7f8eb00c1ec0 2023-10-06T11:19:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 msgr2=0x7f8e94062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 0x7f8e94062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007fd0 tx=0x7f8ea4009040 comp rx=0 tx=0).stop 2023-10-06T11:19:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 msgr2=0x7f8eb00c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c5b0 tx=0x7f8e9c001070 comp rx=0 tx=0).stop 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 shutdown_connections 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8e94060550 0x7f8e94062a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8eb015ab60 0x7f8eb013f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8eb00c1ec0 0x7f8eb00c3be0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 --2- 172.21.15.46:0/3200892252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8eb006e8a0 0x7f8eb014f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 >> 172.21.15.46:0/3200892252 conn(0x7f8eb00cbe30 msgr2=0x7f8eb0158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 shutdown_connections 2023-10-06T11:19:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.583+0000 7f8eb6c2e700 1 -- 172.21.15.46:0/3200892252 wait complete. 2023-10-06T11:19:09.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.775+0000 7f3de140c700 1 Processor -- start 2023-10-06T11:19:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.775+0000 7f3de140c700 1 -- start start 2023-10-06T11:19:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3de140c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3ddc0d07b0 0x7f3ddc0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3de140c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc0ae340 con 0x7f3ddc159290 2023-10-06T11:19:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3de140c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc0b09a0 con 0x7f3ddc0b1130 2023-10-06T11:19:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3de140c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b15b0 con 0x7f3ddc0d07b0 2023-10-06T11:19:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddbfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ddc159290 0x7f3ddc0b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48170/0 (socket says 172.21.15.46:48170) 2023-10-06T11:19:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddbfff700 1 -- 172.21.15.46:0/3482450949 learned_addr learned my addr 172.21.15.46:0/3482450949 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/3482450949 >> v2:172.21.15.132:3300/0 conn(0x7f3ddc0d07b0 0x7f3ddc0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513397242 0 0) 0x7f3ddc0ae340 con 0x7f3ddc159290 2023-10-06T11:19:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc8003570 con 0x7f3ddc159290 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437784286 0 0) 0x7f3ddc0b09a0 con 0x7f3ddc0b1130 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ddc0ae340 con 0x7f3ddc0b1130 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273590106 0 0) 0x7f3dc8003570 con 0x7f3ddc159290 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 >> v2:172.21.15.132:3300/0 conn(0x7f3ddc0d07b0 msgr2=0x7f3ddc0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 --2- 172.21.15.46:0/3482450949 >> v2:172.21.15.132:3300/0 conn(0x7f3ddc0d07b0 0x7f3ddc0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 >> v1:172.21.15.132:6789/0 conn(0x7f3ddc0b1130 legacy=0x7f3ddc0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc159660 con 0x7f3ddc159290 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3dcc001470 con 0x7f3ddc159290 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 >> v1:172.21.15.46:6789/0 conn(0x7f3ddc159290 legacy=0x7f3ddc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc8003bf0 0x7f3dc8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80051c0 0x7f3dc800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b15b0 con 0x7f3dc8004810 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc800c0b0 con 0x7f3dc80051c0 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc800c270 con 0x7f3dc8003bf0 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.779+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3ddbfff700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc8003bf0 0x7f3dc8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de0c0b700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80051c0 0x7f3dc800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 -- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc8003bf0 msgr2=0x7f3dc8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dc8003bf0 0x7f3dc8003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 -- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80051c0 msgr2=0x7f3dc800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80051c0 0x7f3dc800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 -- 172.21.15.46:0/3482450949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8003570 con 0x7f3dc8004810 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de0c0b700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80051c0 0x7f3dc800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de0c0b700 1 -- 172.21.15.46:0/3482450949 reap_dead start 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3dc8004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009c00 tx=0x7f3dc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf0b61b56c8171b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400e710 con 0x7f3dc8004810 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400e870 con 0x7f3dc8004810 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3ddb7fe700 1 -- 172.21.15.46:0/3482450949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400ec30 con 0x7f3dc8004810 2023-10-06T11:19:09.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 -- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 msgr2=0x7f3dc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3dc8004c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009c00 tx=0x7f3dc40091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 -- 172.21.15.46:0/3482450949 shutdown_connections 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 --2- 172.21.15.46:0/3482450949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3dc8004c80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 -- 172.21.15.46:0/3482450949 >> 172.21.15.46:0/3482450949 conn(0x7f3ddc06ef60 msgr2=0x7f3ddc0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 -- 172.21.15.46:0/3482450949 shutdown_connections 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.783+0000 7f3de140c700 1 -- 172.21.15.46:0/3482450949 wait complete. 2023-10-06T11:19:09.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 Processor -- start 2023-10-06T11:19:09.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 -- start start 2023-10-06T11:19:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ddc15da80 0x7f3ddc16cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ddc16d0b0 0x7f3ddc132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc154a70 con 0x7f3dc8004810 2023-10-06T11:19:09.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc15def0 con 0x7f3ddc16d0b0 2023-10-06T11:19:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de140c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc16d520 con 0x7f3ddc15da80 2023-10-06T11:19:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51694/0 (socket says 172.21.15.46:51694) 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 -- 172.21.15.46:0/2176398219 learned_addr learned my addr 172.21.15.46:0/2176398219 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3de0c0b700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ddc16d0b0 0x7f3ddc132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ddc15da80 0x7f3ddc16cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 -- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ddc15da80 msgr2=0x7f3ddc16cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ddc15da80 0x7f3ddc16cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 -- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ddc16d0b0 msgr2=0x7f3ddc132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ddc16d0b0 0x7f3ddc132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.787+0000 7f3ddbfff700 1 -- 172.21.15.46:0/2176398219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc15e120 con 0x7f3dc8004810 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3ddbfff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc007390 tx=0x7f3dcc001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007770 con 0x7f3dc8004810 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc00f8d0 con 0x7f3dc8004810 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc15e400 con 0x7f3dc8004810 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00fc90 con 0x7f3dc8004810 2023-10-06T11:19:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc169c20 con 0x7f3dc8004810 2023-10-06T11:19:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.791+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc169e70 con 0x7f3dc8004810 2023-10-06T11:19:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.795+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3dcc0133b0 con 0x7f3dc8004810 2023-10-06T11:19:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.795+0000 7f3dd8ff9700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 0x7f3dbc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.795+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3dcc07bfb0 con 0x7f3dc8004810 2023-10-06T11:19:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.795+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 0x7f3dbc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.795+0000 7f3dd3fff700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 0x7f3dbc062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40091e0 tx=0x7f3dc401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.803+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3dcc050180 con 0x7f3dc8004810 2023-10-06T11:19:09.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.970+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ddc064590 con 0x7f3dc8004810 2023-10-06T11:19:09.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3dd8ff9700 1 -- 172.21.15.46:0/2176398219 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+13650 (secure 0 0 0) 0x7f3dcc04fd10 con 0x7f3dc8004810 2023-10-06T11:19:09.979 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:09.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:19:06.401728+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 msgr2=0x7f3dbc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 0x7f3dbc062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40091e0 tx=0x7f3dc401a000 comp rx=0 tx=0).stop 2023-10-06T11:19:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 msgr2=0x7f3ddc15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc007390 tx=0x7f3dcc001e10 comp rx=0 tx=0).stop 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 shutdown_connections 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3dbc060550 0x7f3dbc062a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ddc16d0b0 0x7f3ddc132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ddc15da80 0x7f3ddc16cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 --2- 172.21.15.46:0/2176398219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8004810 0x7f3ddc15d540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.974+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 >> 172.21.15.46:0/2176398219 conn(0x7f3ddc06ef60 msgr2=0x7f3ddc1408e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.978+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 shutdown_connections 2023-10-06T11:19:09.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:09.978+0000 7f3de140c700 1 -- 172.21.15.46:0/2176398219 wait complete. 2023-10-06T11:19:10.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:19:10.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:19:10.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 Processor -- start 2023-10-06T11:19:10.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 -- start start 2023-10-06T11:19:10.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f33840ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f338406ed10 0x7f33840c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840c1bf0 con 0x7f338406e8a0 2023-10-06T11:19:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33840abbe0 con 0x7f338406ed10 2023-10-06T11:19:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f338a8cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338406cf80 con 0x7f33840c6470 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f3388e6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f338406ed10 0x7f33840c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f3388e6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f338406ed10 0x7f33840c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51706/0 (socket says 172.21.15.46:51706) 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f3388e6a700 1 -- 172.21.15.46:0/2654181970 learned_addr learned my addr 172.21.15.46:0/2654181970 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.166+0000 7f3383fff700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f33840ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3388e6a700 1 -- 172.21.15.46:0/2654181970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 msgr2=0x7f33840ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3388e6a700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f33840ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3388e6a700 1 -- 172.21.15.46:0/2654181970 >> v1:172.21.15.132:6789/0 conn(0x7f338406e8a0 legacy=0x7f33840c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3388e6a700 1 -- 172.21.15.46:0/2654181970 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338406d260 con 0x7f338406ed10 2023-10-06T11:19:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3383fff700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f33840ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f3388e6a700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f33840c2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f337c0099b0 tx=0x7f337c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c557bb68f04d707 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f33837fe700 1 -- 172.21.15.46:0/2654181970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00b860 con 0x7f338406ed10 2023-10-06T11:19:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f33837fe700 1 -- 172.21.15.46:0/2654181970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00b9c0 con 0x7f338406ed10 2023-10-06T11:19:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f33837fe700 1 -- 172.21.15.46:0/2654181970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00bd60 con 0x7f338406ed10 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 -- 172.21.15.46:0/2654181970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 msgr2=0x7f33840c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f33840c2080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f337c0099b0 tx=0x7f337c009250 comp rx=0 tx=0).stop 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 -- 172.21.15.46:0/2654181970 shutdown_connections 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f33840c2080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 --2- 172.21.15.46:0/2654181970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f33840ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 -- 172.21.15.46:0/2654181970 >> 172.21.15.46:0/2654181970 conn(0x7f33840cbe30 msgr2=0x7f33840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 -- 172.21.15.46:0/2654181970 shutdown_connections 2023-10-06T11:19:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.170+0000 7f338a8cd700 1 -- 172.21.15.46:0/2654181970 wait complete. 2023-10-06T11:19:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 Processor -- start 2023-10-06T11:19:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 -- start start 2023-10-06T11:19:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f338406e8a0 0x7f338413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f338415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338406a340 con 0x7f338406ed10 2023-10-06T11:19:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33840b1540 con 0x7f338406e8a0 2023-10-06T11:19:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.174+0000 7f338a8cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840b0f60 con 0x7f33840c6470 2023-10-06T11:19:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f337b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f338415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3383fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f338406e8a0 0x7f338413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:34748/0 (socket says 172.21.15.46:34748) 2023-10-06T11:19:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 -- 172.21.15.46:0/728602653 learned_addr learned my addr 172.21.15.46:0/728602653 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 -- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f338406e8a0 msgr2=0x7f338413d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f338406e8a0 0x7f338413d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 -- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 msgr2=0x7f338415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f338415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 -- 172.21.15.46:0/728602653 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c0094f0 con 0x7f33840c6470 2023-10-06T11:19:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f337b7fe700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f338415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3388e6a700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f337c006fd0 tx=0x7f337c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c01a630 con 0x7f33840c6470 2023-10-06T11:19:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c01a7d0 con 0x7f33840c6470 2023-10-06T11:19:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384159e60 con 0x7f33840c6470 2023-10-06T11:19:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c01ab90 con 0x7f33840c6470 2023-10-06T11:19:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.178+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33840b19d0 con 0x7f33840c6470 2023-10-06T11:19:10.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.182+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33840b16f0 con 0x7f33840c6470 2023-10-06T11:19:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.182+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f337c042920 con 0x7f33840c6470 2023-10-06T11:19:10.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.186+0000 7f3380ff9700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 0x7f3364062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.186+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f337c07e190 con 0x7f33840c6470 2023-10-06T11:19:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.186+0000 7f3383fff700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 0x7f3364062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.186+0000 7f3383fff700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 0x7f3364062ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f337400d6e0 tx=0x7f3374007850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.190+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f337c04e9d0 con 0x7f33840c6470 2023-10-06T11:19:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.362+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33840b16f0 con 0x7f33840c6470 2023-10-06T11:19:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.366+0000 7f3380ff9700 1 -- 172.21.15.46:0/728602653 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33840b16f0 con 0x7f33840c6470 2023-10-06T11:19:10.370 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 msgr2=0x7f3364062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 0x7f3364062ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f337400d6e0 tx=0x7f3374007850 comp rx=0 tx=0).stop 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 msgr2=0x7f338413f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f337c006fd0 tx=0x7f337c017000 comp rx=0 tx=0).stop 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 shutdown_connections 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3364060620 0x7f3364062ad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33840c6470 0x7f338413f670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338406ed10 0x7f338415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 --2- 172.21.15.46:0/728602653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f338406e8a0 0x7f338413d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 >> 172.21.15.46:0/728602653 conn(0x7f33840cbe30 msgr2=0x7f338415ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 shutdown_connections 2023-10-06T11:19:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.370+0000 7f338a8cd700 1 -- 172.21.15.46:0/728602653 wait complete. 2023-10-06T11:19:10.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:19:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 Processor -- start 2023-10-06T11:19:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 -- start start 2023-10-06T11:19:10.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2740b2790 0x7fe2740ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740cc510 con 0x7fe2740b23c0 2023-10-06T11:19:10.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740b2be0 con 0x7fe2740cd080 2023-10-06T11:19:10.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.558+0000 7fe27bacd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2740cd500 con 0x7fe2740b2790 2023-10-06T11:19:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe279068700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2740b2790 0x7fe2740ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27a06a700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe2740cd080 0x7fe2740c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55980/0 (socket says 172.21.15.46:55980) 2023-10-06T11:19:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27a06a700 1 -- 172.21.15.46:0/1139042090 learned_addr learned my addr 172.21.15.46:0/1139042090 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe279068700 1 -- 172.21.15.46:0/1139042090 >> v1:172.21.15.132:6789/0 conn(0x7fe2740cd080 legacy=0x7fe2740c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe279068700 1 -- 172.21.15.46:0/1139042090 >> v1:172.21.15.46:6789/0 conn(0x7fe2740b23c0 legacy=0x7fe2740cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe279068700 1 -- 172.21.15.46:0/1139042090 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740cb900 con 0x7fe2740b2790 2023-10-06T11:19:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe278867700 1 -- 172.21.15.46:0/1139042090 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110462567 0 0) 0x7fe2740cc510 con 0x7fe2740b23c0 2023-10-06T11:19:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe279068700 1 --2- 172.21.15.46:0/1139042090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b2790 0x7fe2740ccb40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe2640098e0 tx=0x7fe2640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3707bc86ddf2d4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe278867700 1 -- 172.21.15.46:0/1139042090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264009e80 con 0x7fe2740b2790 2023-10-06T11:19:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe278867700 1 -- 172.21.15.46:0/1139042090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26400b860 con 0x7fe2740b2790 2023-10-06T11:19:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe278867700 1 -- 172.21.15.46:0/1139042090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400bbe0 con 0x7fe2740b2790 2023-10-06T11:19:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27bacd700 1 -- 172.21.15.46:0/1139042090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b2790 msgr2=0x7fe2740ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27bacd700 1 --2- 172.21.15.46:0/1139042090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b2790 0x7fe2740ccb40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe2640098e0 tx=0x7fe2640091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27bacd700 1 -- 172.21.15.46:0/1139042090 shutdown_connections 2023-10-06T11:19:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27bacd700 1 --2- 172.21.15.46:0/1139042090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b2790 0x7fe2740ccb40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.562+0000 7fe27bacd700 1 -- 172.21.15.46:0/1139042090 >> 172.21.15.46:0/1139042090 conn(0x7fe274099480 msgr2=0x7fe27406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- 172.21.15.46:0/1139042090 shutdown_connections 2023-10-06T11:19:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- 172.21.15.46:0/1139042090 wait complete. 2023-10-06T11:19:10.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 Processor -- start 2023-10-06T11:19:10.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- start start 2023-10-06T11:19:10.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2740b2790 0x7fe274069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2740cd080 0x7fe2740d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c9090 con 0x7fe2740b23c0 2023-10-06T11:19:10.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274069750 con 0x7fe2740b2790 2023-10-06T11:19:10.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.566+0000 7fe27bacd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27416cce0 con 0x7fe2740cd080 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51732/0 (socket says 172.21.15.46:51732) 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 -- 172.21.15.46:0/113207143 learned_addr learned my addr 172.21.15.46:0/113207143 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe27a06a700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2740cd080 0x7fe2740d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279068700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2740b2790 0x7fe274069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 -- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2740cd080 msgr2=0x7fe2740d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2740cd080 0x7fe2740d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 -- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2740b2790 msgr2=0x7fe274069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2740b2790 0x7fe274069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 -- 172.21.15.46:0/113207143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe264009420 con 0x7fe2740b23c0 2023-10-06T11:19:10.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe279869700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c5b0 tx=0x7fe270007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000e4a0 con 0x7fe2740b23c0 2023-10-06T11:19:10.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27000e600 con 0x7fe2740b23c0 2023-10-06T11:19:10.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe274169be0 con 0x7fe2740b23c0 2023-10-06T11:19:10.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000e9c0 con 0x7fe2740b23c0 2023-10-06T11:19:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.570+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe274169f70 con 0x7fe2740b23c0 2023-10-06T11:19:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.574+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2740660f0 con 0x7fe2740b23c0 2023-10-06T11:19:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.574+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe27001b3b0 con 0x7fe2740b23c0 2023-10-06T11:19:10.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.574+0000 7fe269ffb700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 0x7fe258062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.574+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe27007bd00 con 0x7fe2740b23c0 2023-10-06T11:19:10.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.574+0000 7fe279068700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 0x7fe258062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.578+0000 7fe279068700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 0x7fe258062a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe2640093f0 tx=0x7fe264017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.582+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe27004c440 con 0x7fe2740b23c0 2023-10-06T11:19:10.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.754+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2740cbbe0 con 0x7fe2740b23c0 2023-10-06T11:19:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.754+0000 7fe269ffb700 1 -- 172.21.15.46:0/113207143 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe270017070 con 0x7fe2740b23c0 2023-10-06T11:19:10.759 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 msgr2=0x7fe258062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 0x7fe258062a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe2640093f0 tx=0x7fe264017000 comp rx=0 tx=0).stop 2023-10-06T11:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 msgr2=0x7fe2741506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c5b0 tx=0x7fe270007f60 comp rx=0 tx=0).stop 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 shutdown_connections 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe258060580 0x7fe258062a30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2740cd080 0x7fe2740d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2740b2790 0x7fe274069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 --2- 172.21.15.46:0/113207143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2740b23c0 0x7fe2741506b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 >> 172.21.15.46:0/113207143 conn(0x7fe274099480 msgr2=0x7fe2740c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 shutdown_connections 2023-10-06T11:19:10.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.758+0000 7fe27bacd700 1 -- 172.21.15.46:0/113207143 wait complete. 2023-10-06T11:19:10.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:19:10.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 Processor -- start 2023-10-06T11:19:10.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 -- start start 2023-10-06T11:19:10.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdba40aaf30 0x7fdba4067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba406d790 con 0x7fdba4067590 2023-10-06T11:19:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba4067a10 con 0x7fdba40aaf30 2023-10-06T11:19:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.934+0000 7fdbab28e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba40acd60 con 0x7fdba4156e20 2023-10-06T11:19:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:34758/0 (socket says 172.21.15.46:34758) 2023-10-06T11:19:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 -- 172.21.15.46:0/2447979608 learned_addr learned my addr 172.21.15.46:0/2447979608 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdba9a8b700 1 --2- 172.21.15.46:0/2447979608 >> v2:172.21.15.132:3300/0 conn(0x7fdba40aaf30 0x7fdba4067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 -- 172.21.15.46:0/2447979608 >> v2:172.21.15.132:3300/0 conn(0x7fdba40aaf30 msgr2=0x7fdba4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/2447979608 >> v2:172.21.15.132:3300/0 conn(0x7fdba40aaf30 0x7fdba4067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 -- 172.21.15.46:0/2447979608 >> v1:172.21.15.132:6789/0 conn(0x7fdba4067590 legacy=0x7fdba406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 -- 172.21.15.46:0/2447979608 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40ad040 con 0x7fdba4156e20 2023-10-06T11:19:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/2447979608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdba00099b0 tx=0x7fdba0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442c84e2fcdfdf25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdba928a700 1 -- 172.21.15.46:0/2447979608 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0009ea0 con 0x7fdba4156e20 2023-10-06T11:19:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdba928a700 1 -- 172.21.15.46:0/2447979608 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba000b860 con 0x7fdba4156e20 2023-10-06T11:19:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.938+0000 7fdba928a700 1 -- 172.21.15.46:0/2447979608 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba000bc00 con 0x7fdba4156e20 2023-10-06T11:19:10.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 -- 172.21.15.46:0/2447979608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 msgr2=0x7fdba406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 --2- 172.21.15.46:0/2447979608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdba00099b0 tx=0x7fdba0009250 comp rx=0 tx=0).stop 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 -- 172.21.15.46:0/2447979608 shutdown_connections 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 --2- 172.21.15.46:0/2447979608 >> v2:172.21.15.132:3300/0 conn(0x7fdba40aaf30 0x7fdba4067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 --2- 172.21.15.46:0/2447979608 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba406d250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 -- 172.21.15.46:0/2447979608 >> 172.21.15.46:0/2447979608 conn(0x7fdba4062ef0 msgr2=0x7fdba40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 -- 172.21.15.46:0/2447979608 shutdown_connections 2023-10-06T11:19:10.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.942+0000 7fdbab28e700 1 -- 172.21.15.46:0/2447979608 wait complete. 2023-10-06T11:19:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 Processor -- start 2023-10-06T11:19:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 -- start start 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 0x7fdba40c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba40c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4153ed0 con 0x7fdba40aaf30 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4154c90 con 0x7fdba4067590 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbab28e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba4154080 con 0x7fdba4156e20 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdba9a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdba9a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51734/0 (socket says 172.21.15.46:51734) 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdba9a8b700 1 -- 172.21.15.46:0/3159397590 learned_addr learned my addr 172.21.15.46:0/3159397590 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 0x7fdba40c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.946+0000 7fdbaaa8d700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba40c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 -- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 msgr2=0x7fdba40c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba40c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 -- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 msgr2=0x7fdba40c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 0x7fdba40c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 -- 172.21.15.46:0/3159397590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00094f0 con 0x7fdba40aaf30 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 0x7fdba40c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdba9a8b700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400ec50 tx=0x7fdb940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400c850 con 0x7fdba40aaf30 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9400c9b0 con 0x7fdba40aaf30 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9400cd70 con 0x7fdba40aaf30 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba4154360 con 0x7fdba40aaf30 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba4154920 con 0x7fdba40aaf30 2023-10-06T11:19:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.950+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba4154e20 con 0x7fdba40aaf30 2023-10-06T11:19:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.954+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdb94013680 con 0x7fdba40aaf30 2023-10-06T11:19:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.954+0000 7fdb9a7fc700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 0x7fdb88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.954+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdb9407c760 con 0x7fdba40aaf30 2023-10-06T11:19:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.954+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 0x7fdb88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.954+0000 7fdbaa28c700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 0x7fdb88062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdba00099b0 tx=0x7fdba0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:10.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:10.962+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdb9404cf40 con 0x7fdba40aaf30 2023-10-06T11:19:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.134+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdba4064590 con 0x7fdba40aaf30 2023-10-06T11:19:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.134+0000 7fdb9a7fc700 1 -- 172.21.15.46:0/3159397590 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdb94050560 con 0x7fdba40aaf30 2023-10-06T11:19:11.140 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:19:11.141 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:11.142 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:19:11.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 msgr2=0x7fdb88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 0x7fdb88062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdba00099b0 tx=0x7fdba0017000 comp rx=0 tx=0).stop 2023-10-06T11:19:11.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 msgr2=0x7fdba4153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400ec50 tx=0x7fdb940091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 shutdown_connections 2023-10-06T11:19:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdb88060550 0x7fdb88062a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba4156e20 0x7fdba40c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba40aaf30 0x7fdba4153990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 --2- 172.21.15.46:0/3159397590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba4067590 0x7fdba40c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 >> 172.21.15.46:0/3159397590 conn(0x7fdba4062ef0 msgr2=0x7fdba40c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:11.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 shutdown_connections 2023-10-06T11:19:11.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.138+0000 7fdbab28e700 1 -- 172.21.15.46:0/3159397590 wait complete. 2023-10-06T11:19:11.163 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:19:11.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:19:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.326+0000 7f6c41d85700 1 Processor -- start 2023-10-06T11:19:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.326+0000 7f6c41d85700 1 -- start start 2023-10-06T11:19:11.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c41d85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c3c0cb190 0x7f6c3c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:11.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c41d85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c0c54d0 con 0x7f6c3c0af4d0 2023-10-06T11:19:11.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c41d85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c0cbaa0 con 0x7f6c3c0c6970 2023-10-06T11:19:11.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c41d85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0cc2b0 con 0x7f6c3c0cb190 2023-10-06T11:19:11.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c3c0cb190 0x7f6c3c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:11.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c3c0cb190 0x7f6c3c0cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51742/0 (socket says 172.21.15.46:51742) 2023-10-06T11:19:11.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/2375324574 learned_addr learned my addr 172.21.15.46:0/2375324574 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/2375324574 >> v1:172.21.15.132:6789/0 conn(0x7f6c3c0c6970 legacy=0x7f6c3c161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/2375324574 >> v1:172.21.15.46:6789/0 conn(0x7f6c3c0af4d0 legacy=0x7f6c3c1615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.330+0000 7f6c3b7fe700 1 -- 172.21.15.46:0/2375324574 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c162400 con 0x7f6c3c0cb190 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/2375324574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0cb560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009990 tx=0x7f6c2c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336d423ec67c1b43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c3a7fc700 1 -- 172.21.15.46:0/2375324574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c005c10 con 0x7f6c3c0cb190 2023-10-06T11:19:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c3a7fc700 1 -- 172.21.15.46:0/2375324574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c005d70 con 0x7f6c3c0cb190 2023-10-06T11:19:11.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c3a7fc700 1 -- 172.21.15.46:0/2375324574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00b840 con 0x7f6c3c0cb190 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 -- 172.21.15.46:0/2375324574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0cb190 msgr2=0x7f6c3c0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 --2- 172.21.15.46:0/2375324574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0cb560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009990 tx=0x7f6c2c009220 comp rx=0 tx=0).stop 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 -- 172.21.15.46:0/2375324574 shutdown_connections 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 --2- 172.21.15.46:0/2375324574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0cb560 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 -- 172.21.15.46:0/2375324574 >> 172.21.15.46:0/2375324574 conn(0x7f6c3c06ef60 msgr2=0x7f6c3c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 -- 172.21.15.46:0/2375324574 shutdown_connections 2023-10-06T11:19:11.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.334+0000 7f6c41d85700 1 -- 172.21.15.46:0/2375324574 wait complete. 2023-10-06T11:19:11.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 Processor -- start 2023-10-06T11:19:11.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 -- start start 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c3c0af4d0 0x7f6c3c16db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b3f90 con 0x7f6c3c0c6970 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0b3c60 con 0x7f6c3c0cb190 2023-10-06T11:19:11.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c41d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0c73b0 con 0x7f6c3c0af4d0 2023-10-06T11:19:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c3affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51744/0 (socket says 172.21.15.46:51744) 2023-10-06T11:19:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.338+0000 7f6c3affd700 1 -- 172.21.15.46:0/3621516542 learned_addr learned my addr 172.21.15.46:0/3621516542 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c3c0af4d0 0x7f6c3c16db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3bfff700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 -- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c3c0af4d0 msgr2=0x7f6c3c16db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c3c0af4d0 0x7f6c3c16db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 -- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0cb190 msgr2=0x7f6c3c0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 -- 172.21.15.46:0/3621516542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c009480 con 0x7f6c3c0c6970 2023-10-06T11:19:11.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c3affd700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c5b0 tx=0x7f6c30007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30016070 con 0x7f6c3c0c6970 2023-10-06T11:19:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30006820 con 0x7f6c3c0c6970 2023-10-06T11:19:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30006c00 con 0x7f6c3c0c6970 2023-10-06T11:19:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0c7690 con 0x7f6c3c0c6970 2023-10-06T11:19:11.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c0b3770 con 0x7f6c3c0c6970 2023-10-06T11:19:11.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.342+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c06f330 con 0x7f6c3c0c6970 2023-10-06T11:19:11.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.346+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6c30006d60 con 0x7f6c3c0c6970 2023-10-06T11:19:11.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.346+0000 7f6c237fe700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 0x7f6c28062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:11.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.346+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6c3007bec0 con 0x7f6c3c0c6970 2023-10-06T11:19:11.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.346+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 0x7f6c28062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:11.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.350+0000 7f6c3b7fe700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 0x7f6c28062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009260 tx=0x7f6c2c015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:11.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.354+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6c3c06f330 con 0x7f6c3c0c6970 2023-10-06T11:19:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:11.530+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c3c06f330 con 0x7f6c3c0c6970 2023-10-06T11:19:12.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.430+0000 7f6c237fe700 1 -- 172.21.15.46:0/3621516542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c3c06f330 con 0x7f6c3c0c6970 2023-10-06T11:19:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 msgr2=0x7f6c28062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 0x7f6c28062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009260 tx=0x7f6c2c015000 comp rx=0 tx=0).stop 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 msgr2=0x7f6c3c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c5b0 tx=0x7f6c30007ef0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 shutdown_connections 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c28060550 0x7f6c28062a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0cb190 0x7f6c3c0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c3c0c6970 0x7f6c3c0b3230 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 --2- 172.21.15.46:0/3621516542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c3c0af4d0 0x7f6c3c16db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.438+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 >> 172.21.15.46:0/3621516542 conn(0x7f6c3c06ef60 msgr2=0x7f6c3c0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.442+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 shutdown_connections 2023-10-06T11:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.442+0000 7f6c41d85700 1 -- 172.21.15.46:0/3621516542 wait complete. 2023-10-06T11:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:19:12.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:19:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.634+0000 7fa17ce6e700 1 Processor -- start 2023-10-06T11:19:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.634+0000 7fa17ce6e700 1 -- start start 2023-10-06T11:19:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.638+0000 7fa17ce6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1780b2790 0x7fa1780ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.638+0000 7fa17ce6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1780cd080 0x7fa1780c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.638+0000 7fa17ce6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780cc510 con 0x7fa1780b23c0 2023-10-06T11:19:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.638+0000 7fa17ce6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780cd500 con 0x7fa1780cd080 2023-10-06T11:19:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.638+0000 7fa17ce6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780b2be0 con 0x7fa1780b2790 2023-10-06T11:19:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1780cd080 0x7fa1780c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa175d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1780b2790 0x7fa1780ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1780cd080 0x7fa1780c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51754/0 (socket says 172.21.15.46:51754) 2023-10-06T11:19:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 -- 172.21.15.46:0/3978762346 learned_addr learned my addr 172.21.15.46:0/3978762346 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17659c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa1780b23c0 0x7fa1780cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55998/0 (socket says 172.21.15.46:55998) 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 -- 172.21.15.46:0/3978762346 >> v2:172.21.15.132:3300/0 conn(0x7fa1780b2790 msgr2=0x7fa1780ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 --2- 172.21.15.46:0/3978762346 >> v2:172.21.15.132:3300/0 conn(0x7fa1780b2790 0x7fa1780ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 -- 172.21.15.46:0/3978762346 >> v1:172.21.15.132:6789/0 conn(0x7fa1780b23c0 legacy=0x7fa1780cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17559a700 1 -- 172.21.15.46:0/3978762346 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006879195 0 0) 0x7fa1780cc510 con 0x7fa1780b23c0 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 -- 172.21.15.46:0/3978762346 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780cb900 con 0x7fa1780cd080 2023-10-06T11:19:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa176d9d700 1 --2- 172.21.15.46:0/3978762346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780c4fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa178089420 tx=0x7fa170009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946609999c6ab6a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17559a700 1 -- 172.21.15.46:0/3978762346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000e070 con 0x7fa1780cd080 2023-10-06T11:19:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17559a700 1 -- 172.21.15.46:0/3978762346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17000ba20 con 0x7fa1780cd080 2023-10-06T11:19:12.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17559a700 1 -- 172.21.15.46:0/3978762346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700092e0 con 0x7fa1780cd080 2023-10-06T11:19:12.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17ce6e700 1 -- 172.21.15.46:0/3978762346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 msgr2=0x7fa1780c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.642+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/3978762346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780c4fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa178089420 tx=0x7fa170009840 comp rx=0 tx=0).stop 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 -- 172.21.15.46:0/3978762346 shutdown_connections 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/3978762346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780c4fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/3978762346 >> v2:172.21.15.132:3300/0 conn(0x7fa1780b2790 0x7fa1780ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 -- 172.21.15.46:0/3978762346 >> 172.21.15.46:0/3978762346 conn(0x7fa178099480 msgr2=0x7fa17806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 -- 172.21.15.46:0/3978762346 shutdown_connections 2023-10-06T11:19:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 -- 172.21.15.46:0/3978762346 wait complete. 2023-10-06T11:19:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 Processor -- start 2023-10-06T11:19:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.646+0000 7fa17ce6e700 1 -- start start 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1780b23c0 0x7fa17809d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780b2790 0x7fa17816d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa178153650 con 0x7fa1780cd080 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17815ad70 con 0x7fa1780b2790 2023-10-06T11:19:12.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17815a8e0 con 0x7fa1780b23c0 2023-10-06T11:19:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51758/0 (socket says 172.21.15.46:51758) 2023-10-06T11:19:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 -- 172.21.15.46:0/638041281 learned_addr learned my addr 172.21.15.46:0/638041281 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17659c700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1780b23c0 0x7fa17809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa175d9b700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780b2790 0x7fa17816d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 -- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1780b23c0 msgr2=0x7fa17809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1780b23c0 0x7fa17809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 -- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780b2790 msgr2=0x7fa17816d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780b2790 0x7fa17816d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 -- 172.21.15.46:0/638041281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170009ae0 con 0x7fa1780cd080 2023-10-06T11:19:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa176d9d700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa170009880 tx=0x7fa170018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170008c20 con 0x7fa1780cd080 2023-10-06T11:19:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170008da0 con 0x7fa1780cd080 2023-10-06T11:19:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17816dcd0 con 0x7fa1780cd080 2023-10-06T11:19:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700266b0 con 0x7fa1780cd080 2023-10-06T11:19:12.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.650+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa178159cd0 con 0x7fa1780cd080 2023-10-06T11:19:12.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.654+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1780660f0 con 0x7fa1780cd080 2023-10-06T11:19:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.654+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa170003040 con 0x7fa1780cd080 2023-10-06T11:19:12.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.658+0000 7fa1667fc700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 0x7fa158062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:12.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.658+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fa17007e1b0 con 0x7fa1780cd080 2023-10-06T11:19:12.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.658+0000 7fa17659c700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 0x7fa158062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.658+0000 7fa17659c700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 0x7fa158062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa168007250 tx=0x7fa168007e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:12.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.662+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa17004e780 con 0x7fa1780cd080 2023-10-06T11:19:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:12.834+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa17815aa20 con 0x7fa1780cd080 2023-10-06T11:19:13.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.446+0000 7fa1667fc700 1 -- 172.21.15.46:0/638041281 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa170051da0 con 0x7fa1780cd080 2023-10-06T11:19:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.470+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 msgr2=0x7fa158062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.470+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 0x7fa158062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa168007250 tx=0x7fa168007e60 comp rx=0 tx=0).stop 2023-10-06T11:19:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.470+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 msgr2=0x7fa1780b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.470+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa170009880 tx=0x7fa170018000 comp rx=0 tx=0).stop 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 shutdown_connections 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa158060550 0x7fa158062a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1780cd080 0x7fa1780b5500 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780b2790 0x7fa17816d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 --2- 172.21.15.46:0/638041281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1780b23c0 0x7fa17809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 >> 172.21.15.46:0/638041281 conn(0x7fa178099480 msgr2=0x7fa178157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 shutdown_connections 2023-10-06T11:19:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.474+0000 7fa17ce6e700 1 -- 172.21.15.46:0/638041281 wait complete. 2023-10-06T11:19:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:19:13.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:19:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 Processor -- start 2023-10-06T11:19:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 -- start start 2023-10-06T11:19:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f200ca430 0x7f8f200c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 0x7f8f200c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f200c1630 con 0x7f8f200c7d80 2023-10-06T11:19:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f200c0d00 con 0x7f8f200a9840 2023-10-06T11:19:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.658+0000 7f8f2582e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f200c0fe0 con 0x7f8f200ca430 2023-10-06T11:19:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 0x7f8f200c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 0x7f8f200c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52954/0 (socket says 172.21.15.46:52954) 2023-10-06T11:19:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1390513993 >> v2:172.21.15.132:3300/0 conn(0x7f8f200ca430 0x7f8f200c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1f7fe700 1 -- 172.21.15.46:0/1390513993 learned_addr learned my addr 172.21.15.46:0/1390513993 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1390513993 >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 msgr2=0x7f8f200c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1390513993 >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 0x7f8f200c1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1390513993 >> v1:172.21.15.132:6789/0 conn(0x7f8f200c7d80 legacy=0x7f8f200c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1390513993 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200c12c0 con 0x7f8f200ca430 2023-10-06T11:19:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1390513993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c8150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f200cba50 tx=0x7f8f140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f7b51f394d4b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1dffb700 1 -- 172.21.15.46:0/1390513993 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14011630 con 0x7f8f200ca430 2023-10-06T11:19:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1dffb700 1 -- 172.21.15.46:0/1390513993 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14011790 con 0x7f8f200ca430 2023-10-06T11:19:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f1dffb700 1 -- 172.21.15.46:0/1390513993 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14011b30 con 0x7f8f200ca430 2023-10-06T11:19:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 -- 172.21.15.46:0/1390513993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 msgr2=0x7f8f200c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1390513993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c8150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8f200cba50 tx=0x7f8f140091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 -- 172.21.15.46:0/1390513993 shutdown_connections 2023-10-06T11:19:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1390513993 >> v2:172.21.15.46:3300/0 conn(0x7f8f200a9840 0x7f8f200c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1390513993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c8150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 -- 172.21.15.46:0/1390513993 >> 172.21.15.46:0/1390513993 conn(0x7f8f200cbe30 msgr2=0x7f8f200cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.662+0000 7f8f2582e700 1 -- 172.21.15.46:0/1390513993 shutdown_connections 2023-10-06T11:19:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.666+0000 7f8f2582e700 1 -- 172.21.15.46:0/1390513993 wait complete. 2023-10-06T11:19:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.666+0000 7f8f2582e700 1 Processor -- start 2023-10-06T11:19:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.666+0000 7f8f2582e700 1 -- start start 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200a9840 0x7f8f20153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20157580 con 0x7f8f200c7d80 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20157a10 con 0x7f8f200ca430 2023-10-06T11:19:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20156970 con 0x7f8f200a9840 2023-10-06T11:19:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52958/0 (socket says 172.21.15.46:52958) 2023-10-06T11:19:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1941762769 learned_addr learned my addr 172.21.15.46:0/1941762769 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1f7fe700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1effd700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200a9840 0x7f8f20153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200a9840 msgr2=0x7f8f20153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200a9840 0x7f8f20153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 msgr2=0x7f8f200c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 -- 172.21.15.46:0/1941762769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14009420 con 0x7f8f200c7d80 2023-10-06T11:19:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f1e7fc700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009f50 tx=0x7f8f14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14000d10 con 0x7f8f200c7d80 2023-10-06T11:19:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14011900 con 0x7f8f200c7d80 2023-10-06T11:19:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14025420 con 0x7f8f200c7d80 2023-10-06T11:19:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f20156c50 con 0x7f8f200c7d80 2023-10-06T11:19:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.670+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f20157210 con 0x7f8f200c7d80 2023-10-06T11:19:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.674+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f20157730 con 0x7f8f200c7d80 2023-10-06T11:19:13.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.674+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8f14023070 con 0x7f8f200c7d80 2023-10-06T11:19:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.674+0000 7f8f06ffd700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 0x7f8f0c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.674+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f8f1407dd60 con 0x7f8f200c7d80 2023-10-06T11:19:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.674+0000 7f8f1effd700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 0x7f8f0c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.678+0000 7f8f1effd700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 0x7f8f0c062a00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f10003aa0 tx=0x7f8f10000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.682+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8f20157730 con 0x7f8f200c7d80 2023-10-06T11:19:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:13.890+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8f20157730 con 0x7f8f200c7d80 2023-10-06T11:19:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.458+0000 7f8f06ffd700 1 -- 172.21.15.46:0/1941762769 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8f20157730 con 0x7f8f200c7d80 2023-10-06T11:19:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.478+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 msgr2=0x7f8f0c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.478+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 0x7f8f0c062a00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f10003aa0 tx=0x7f8f10000bc0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.478+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 msgr2=0x7f8f20156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.478+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009f50 tx=0x7f8f14017000 comp rx=0 tx=0).stop 2023-10-06T11:19:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 shutdown_connections 2023-10-06T11:19:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8f0c060550 0x7f8f0c062a00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f200ca430 0x7f8f200c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f200c7d80 0x7f8f20156070 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 --2- 172.21.15.46:0/1941762769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200a9840 0x7f8f20153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.490+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 >> 172.21.15.46:0/1941762769 conn(0x7f8f200cbe30 msgr2=0x7f8f200c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.494+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 shutdown_connections 2023-10-06T11:19:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.494+0000 7f8f2582e700 1 -- 172.21.15.46:0/1941762769 wait complete. 2023-10-06T11:19:14.501 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T11:19:14.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.678+0000 7f1c1cc8e700 1 Processor -- start 2023-10-06T11:19:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.678+0000 7f1c1cc8e700 1 -- start start 2023-10-06T11:19:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c1cc8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c18155c70 0x7f1c1806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c1cc8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c18067050 0x7f1c18067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c1cc8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c180c74d0 con 0x7f1c180c6150 2023-10-06T11:19:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c1cc8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c180679d0 con 0x7f1c18155c70 2023-10-06T11:19:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c1cc8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c180c68c0 con 0x7f1c18067050 2023-10-06T11:19:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c16ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c180c6150 0x7f1c180c7960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34100/0 (socket says 172.21.15.46:34100) 2023-10-06T11:19:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c16ffd700 1 -- 172.21.15.46:0/919346195 learned_addr learned my addr 172.21.15.46:0/919346195 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c177fe700 1 --2- 172.21.15.46:0/919346195 >> v2:172.21.15.46:3300/0 conn(0x7f1c18155c70 0x7f1c1806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c17fff700 1 --2- 172.21.15.46:0/919346195 >> v2:172.21.15.132:3300/0 conn(0x7f1c18067050 0x7f1c18067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311539603 0 0) 0x7f1c180c74d0 con 0x7f1c180c6150 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c00003570 con 0x7f1c180c6150 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c177fe700 1 -- 172.21.15.46:0/919346195 >> v2:172.21.15.132:3300/0 conn(0x7f1c18067050 msgr2=0x7f1c18067490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c177fe700 1 --2- 172.21.15.46:0/919346195 >> v2:172.21.15.132:3300/0 conn(0x7f1c18067050 0x7f1c18067490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c177fe700 1 -- 172.21.15.46:0/919346195 >> v1:172.21.15.46:6789/0 conn(0x7f1c180c6150 legacy=0x7f1c180c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683000394 0 0) 0x7f1c00003570 con 0x7f1c180c6150 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.682+0000 7f1c177fe700 1 -- 172.21.15.46:0/919346195 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180c6ba0 con 0x7f1c18155c70 2023-10-06T11:19:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c177fe700 1 --2- 172.21.15.46:0/919346195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c18155c70 0x7f1c1806da60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c1806c920 tx=0x7f1c080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d3ed01798b910e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c001470 con 0x7f1c18155c70 2023-10-06T11:19:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08011420 con 0x7f1c18155c70 2023-10-06T11:19:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c167fc700 1 -- 172.21.15.46:0/919346195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080117c0 con 0x7f1c18155c70 2023-10-06T11:19:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/919346195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c18155c70 msgr2=0x7f1c1806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/919346195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c18155c70 0x7f1c1806da60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c1806c920 tx=0x7f1c080091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/919346195 shutdown_connections 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/919346195 >> v2:172.21.15.132:3300/0 conn(0x7f1c18067050 0x7f1c18067490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/919346195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c18155c70 0x7f1c1806da60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/919346195 >> 172.21.15.46:0/919346195 conn(0x7f1c18062ef0 msgr2=0x7f1c180cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/919346195 shutdown_connections 2023-10-06T11:19:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/919346195 wait complete. 2023-10-06T11:19:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.686+0000 7f1c1cc8e700 1 Processor -- start 2023-10-06T11:19:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- start start 2023-10-06T11:19:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c18067050 0x7f1c18152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c180c6150 0x7f1c180c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c0a60 con 0x7f1c180c25f0 2023-10-06T11:19:14.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180ae960 con 0x7f1c18067050 2023-10-06T11:19:14.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180af7c0 con 0x7f1c180c6150 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52968/0 (socket says 172.21.15.46:52968) 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 -- 172.21.15.46:0/2875343161 learned_addr learned my addr 172.21.15.46:0/2875343161 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c177fe700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c18067050 0x7f1c18152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c16ffd700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c180c6150 0x7f1c180c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 -- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c180c6150 msgr2=0x7f1c180c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c180c6150 0x7f1c180c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 -- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c18067050 msgr2=0x7f1c18152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c18067050 0x7f1c18152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 -- 172.21.15.46:0/2875343161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08009420 con 0x7f1c180c25f0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c17fff700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ec0 tx=0x7f1c1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000cb70 con 0x7f1c180c25f0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000f040 con 0x7f1c180c25f0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10014480 con 0x7f1c180c25f0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c18152c40 con 0x7f1c180c25f0 2023-10-06T11:19:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.690+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c180aee90 con 0x7f1c180c25f0 2023-10-06T11:19:14.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.694+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c18064590 con 0x7f1c180c25f0 2023-10-06T11:19:14.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.694+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1c10014830 con 0x7f1c180c25f0 2023-10-06T11:19:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.698+0000 7f1bff7fe700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 0x7f1c04062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.698+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c1007d2b0 con 0x7f1c180c25f0 2023-10-06T11:19:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.698+0000 7f1c177fe700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 0x7f1c04062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:14.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.698+0000 7f1c177fe700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 0x7f1c04062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c1806c920 tx=0x7f1c08016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.702+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c1004d6a0 con 0x7f1c180c25f0 2023-10-06T11:19:14.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:19:14.830 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:19:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.870+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c180af900 con 0x7f1c180c25f0 2023-10-06T11:19:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.870+0000 7f1bff7fe700 1 -- 172.21.15.46:0/2875343161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f1c10050cc0 con 0x7f1c180c25f0 2023-10-06T11:19:14.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:14.878 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:19:13.894155+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 msgr2=0x7f1c04062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 0x7f1c04062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1c1806c920 tx=0x7f1c08016000 comp rx=0 tx=0).stop 2023-10-06T11:19:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 msgr2=0x7f1c180ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ec0 tx=0x7f1c1000b040 comp rx=0 tx=0).stop 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 shutdown_connections 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c04060550 0x7f1c04062a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c180c25f0 0x7f1c180ad260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c180c6150 0x7f1c180c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 --2- 172.21.15.46:0/2875343161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c18067050 0x7f1c18152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 >> 172.21.15.46:0/2875343161 conn(0x7f1c18062ef0 msgr2=0x7f1c18068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 shutdown_connections 2023-10-06T11:19:14.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:14.874+0000 7f1c1cc8e700 1 -- 172.21.15.46:0/2875343161 wait complete. 2023-10-06T11:19:14.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:15.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.094+0000 7f6965c6e700 1 Processor -- start 2023-10-06T11:19:15.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.094+0000 7f6965c6e700 1 -- start start 2023-10-06T11:19:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6965c6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69600ce160 0x7f69600ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6965c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6965c6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69600c9e80 con 0x7f696015b160 2023-10-06T11:19:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6965c6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696006ba20 con 0x7f69600ce160 2023-10-06T11:19:15.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6965c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696015b790 con 0x7f69600ca310 2023-10-06T11:19:15.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6956ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695f7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69600ce160 0x7f69600ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6956ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55410/0 (socket says 172.21.15.46:55410) 2023-10-06T11:19:15.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6956ffd700 1 -- 172.21.15.46:0/3445820345 learned_addr learned my addr 172.21.15.46:0/3445820345 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:15.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695ffff700 1 --1- 172.21.15.46:0/3445820345 >> v1:172.21.15.132:6789/0 conn(0x7f696015b160 0x7f6960156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42208/0 (socket says 172.21.15.46:42208) 2023-10-06T11:19:15.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695f7fe700 1 -- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 msgr2=0x7f696015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:15.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695f7fe700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695f7fe700 1 -- 172.21.15.46:0/3445820345 >> v1:172.21.15.132:6789/0 conn(0x7f696015b160 legacy=0x7f6960156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f695f7fe700 1 -- 172.21.15.46:0/3445820345 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696015ba70 con 0x7f69600ce160 2023-10-06T11:19:15.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.098+0000 7f6956ffd700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:15.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f695f7fe700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ce160 0x7f69600ce530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69500098e0 tx=0x7f69500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2966815b6bd5b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f695effd700 1 -- 172.21.15.46:0/3445820345 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000b640 con 0x7f69600ce160 2023-10-06T11:19:15.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f695effd700 1 -- 172.21.15.46:0/3445820345 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000b7a0 con 0x7f69600ce160 2023-10-06T11:19:15.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f695effd700 1 -- 172.21.15.46:0/3445820345 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000bb40 con 0x7f69600ce160 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 -- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ce160 msgr2=0x7f69600ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ce160 0x7f69600ce530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69500098e0 tx=0x7f69500091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 -- 172.21.15.46:0/3445820345 shutdown_connections 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600ca310 0x7f696015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3445820345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ce160 0x7f69600ce530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 -- 172.21.15.46:0/3445820345 >> 172.21.15.46:0/3445820345 conn(0x7f6960062d50 msgr2=0x7f6960063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 -- 172.21.15.46:0/3445820345 shutdown_connections 2023-10-06T11:19:15.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.102+0000 7f6965c6e700 1 -- 172.21.15.46:0/3445820345 wait complete. 2023-10-06T11:19:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 Processor -- start 2023-10-06T11:19:15.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 -- start start 2023-10-06T11:19:15.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 0x7f69600c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696015b160 0x7f69600c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600c8010 con 0x7f69600ce160 2023-10-06T11:19:15.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600c62f0 con 0x7f69600ca310 2023-10-06T11:19:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6965c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69600c56e0 con 0x7f696015b160 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6956ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f695f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 0x7f69600c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6956ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52984/0 (socket says 172.21.15.46:52984) 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6956ffd700 1 -- 172.21.15.46:0/3598851288 learned_addr learned my addr 172.21.15.46:0/3598851288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f695ffff700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696015b160 0x7f69600c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.106+0000 7f6956ffd700 1 -- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696015b160 msgr2=0x7f69600c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6956ffd700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696015b160 0x7f69600c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6956ffd700 1 -- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 msgr2=0x7f69600c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6956ffd700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 0x7f69600c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6956ffd700 1 -- 172.21.15.46:0/3598851288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480094f0 con 0x7f69600ce160 2023-10-06T11:19:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f695f7fe700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 0x7f69600c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6956ffd700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694800ec50 tx=0x7f6948009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800c850 con 0x7f69600ce160 2023-10-06T11:19:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694800c9b0 con 0x7f69600ce160 2023-10-06T11:19:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800cd30 con 0x7f69600ce160 2023-10-06T11:19:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6950009420 con 0x7f69600ce160 2023-10-06T11:19:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69600c5ca0 con 0x7f69600ce160 2023-10-06T11:19:15.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.110+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69600c6480 con 0x7f69600ce160 2023-10-06T11:19:15.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.114+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6948013680 con 0x7f69600ce160 2023-10-06T11:19:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.114+0000 7f6957fff700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 0x7f6940062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.114+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f694807cc10 con 0x7f69600ce160 2023-10-06T11:19:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.114+0000 7f695f7fe700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 0x7f6940062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.118+0000 7f695f7fe700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 0x7f6940062a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69500093f0 tx=0x7f6950017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.122+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f694804d000 con 0x7f69600ce160 2023-10-06T11:19:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.294+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69600ad320 con 0x7f69600ce160 2023-10-06T11:19:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.294+0000 7f6957fff700 1 -- 172.21.15.46:0/3598851288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f6948050620 con 0x7f69600ce160 2023-10-06T11:19:15.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:15.302 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:19:13.894155+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 msgr2=0x7f6940062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 0x7f6940062a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69500093f0 tx=0x7f6950017000 comp rx=0 tx=0).stop 2023-10-06T11:19:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 msgr2=0x7f6960155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694800ec50 tx=0x7f6948009250 comp rx=0 tx=0).stop 2023-10-06T11:19:15.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 shutdown_connections 2023-10-06T11:19:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6940060580 0x7f6940062a30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696015b160 0x7f69600c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69600ce160 0x7f6960155460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 --2- 172.21.15.46:0/3598851288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600ca310 0x7f69600c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 >> 172.21.15.46:0/3598851288 conn(0x7f6960062d50 msgr2=0x7f6960068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 shutdown_connections 2023-10-06T11:19:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.298+0000 7f6965c6e700 1 -- 172.21.15.46:0/3598851288 wait complete. 2023-10-06T11:19:15.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:19:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.490+0000 7f115d8cd700 1 Processor -- start 2023-10-06T11:19:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.490+0000 7f115d8cd700 1 -- start start 2023-10-06T11:19:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f115d8cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f115806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f115d8cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f115806abb0 0x7f1158067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f115d8cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115815d120 con 0x7f115806a7e0 2023-10-06T11:19:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f115d8cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115815dc00 con 0x7f115806abb0 2023-10-06T11:19:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f115d8cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115815d360 con 0x7f115815ccd0 2023-10-06T11:19:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1156ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f115806a7e0 0x7f1158069b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34112/0 (socket says 172.21.15.46:34112) 2023-10-06T11:19:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1156ffd700 1 -- 172.21.15.46:0/1273316059 learned_addr learned my addr 172.21.15.46:0/1273316059 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 --2- 172.21.15.46:0/1273316059 >> v2:172.21.15.46:3300/0 conn(0x7f115806abb0 0x7f1158067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11567fc700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f115806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 -- 172.21.15.46:0/1273316059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 msgr2=0x7f115806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f115806e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1155ffb700 1 -- 172.21.15.46:0/1273316059 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395246896 0 0) 0x7f1148001110 con 0x7f115806a7e0 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 -- 172.21.15.46:0/1273316059 >> v1:172.21.15.46:6789/0 conn(0x7f115806a7e0 legacy=0x7f1158069b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 -- 172.21.15.46:0/1273316059 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115815d640 con 0x7f115806abb0 2023-10-06T11:19:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f11577fe700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158067050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1150009e60 tx=0x7f1150009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dc4e514dc51fcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1155ffb700 1 -- 172.21.15.46:0/1273316059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000e070 con 0x7f115806abb0 2023-10-06T11:19:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1155ffb700 1 -- 172.21.15.46:0/1273316059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000bc30 con 0x7f115806abb0 2023-10-06T11:19:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.494+0000 7f1155ffb700 1 -- 172.21.15.46:0/1273316059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150012430 con 0x7f115806abb0 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 -- 172.21.15.46:0/1273316059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 msgr2=0x7f1158067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158067050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1150009e60 tx=0x7f1150009780 comp rx=0 tx=0).stop 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 -- 172.21.15.46:0/1273316059 shutdown_connections 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158067050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 --2- 172.21.15.46:0/1273316059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f115806e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 -- 172.21.15.46:0/1273316059 >> 172.21.15.46:0/1273316059 conn(0x7f1158068160 msgr2=0x7f1158068560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 -- 172.21.15.46:0/1273316059 shutdown_connections 2023-10-06T11:19:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.498+0000 7f115d8cd700 1 -- 172.21.15.46:0/1273316059 wait complete. 2023-10-06T11:19:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 Processor -- start 2023-10-06T11:19:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 -- start start 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115806a7e0 0x7f115815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f1158142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115815c590 con 0x7f115806abb0 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158151440 con 0x7f115806a7e0 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f115d8cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115815afb0 con 0x7f115815ccd0 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f11567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f11567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52994/0 (socket says 172.21.15.46:52994) 2023-10-06T11:19:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f11567fc700 1 -- 172.21.15.46:0/464623207 learned_addr learned my addr 172.21.15.46:0/464623207 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f1156ffd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115806a7e0 0x7f115815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.502+0000 7f11577fe700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f1158142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 -- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 msgr2=0x7f1158142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f1158142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 -- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115806a7e0 msgr2=0x7f115815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115806a7e0 0x7f115815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 -- 172.21.15.46:0/464623207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009a20 con 0x7f115806abb0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f11567fc700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d860 tx=0x7f114c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c009610 con 0x7f115806abb0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115815b290 con 0x7f115806abb0 2023-10-06T11:19:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c009770 con 0x7f115806abb0 2023-10-06T11:19:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c009b30 con 0x7f115806abb0 2023-10-06T11:19:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115815b620 con 0x7f115806abb0 2023-10-06T11:19:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.506+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1158064590 con 0x7f115806abb0 2023-10-06T11:19:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.510+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f114c01e460 con 0x7f115806abb0 2023-10-06T11:19:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.510+0000 7f113effd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 0x7f1144062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.510+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f114c07cff0 con 0x7f115806abb0 2023-10-06T11:19:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.510+0000 7f1156ffd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 0x7f1144062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.510+0000 7f1156ffd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 0x7f1144062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11480018a0 tx=0x7f1148009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.514+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f114c0acbe0 con 0x7f115806abb0 2023-10-06T11:19:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:15.730+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f115804b5c0 con 0x7f115806abb0 2023-10-06T11:19:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.490+0000 7f113effd700 1 -- 172.21.15.46:0/464623207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f114c04d400 con 0x7f115806abb0 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 msgr2=0x7f1144062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 0x7f1144062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11480018a0 tx=0x7f1148009040 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 msgr2=0x7f1158153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f114c00d860 tx=0x7f114c008040 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 shutdown_connections 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1144060550 0x7f1144062a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115815ccd0 0x7f1158142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f115806abb0 0x7f1158153f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 --2- 172.21.15.46:0/464623207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115806a7e0 0x7f115815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 >> 172.21.15.46:0/464623207 conn(0x7f1158068160 msgr2=0x7f11581507d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 shutdown_connections 2023-10-06T11:19:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.494+0000 7f115d8cd700 1 -- 172.21.15.46:0/464623207 wait complete. 2023-10-06T11:19:16.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:19:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.686+0000 7f84d96e3700 1 Processor -- start 2023-10-06T11:19:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.686+0000 7f84d96e3700 1 -- start start 2023-10-06T11:19:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d96e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40cb2d0 0x7f84d4158860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d96e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f84d40c97e0 0x7f84d40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d96e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d40cae40 con 0x7f84d406c9f0 2023-10-06T11:19:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d96e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d40ca230 con 0x7f84d40c97e0 2023-10-06T11:19:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d96e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40ca510 con 0x7f84d40cb2d0 2023-10-06T11:19:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d8ee2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84d406c9f0 0x7f84d406ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34126/0 (socket says 172.21.15.46:34126) 2023-10-06T11:19:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d8ee2700 1 -- 172.21.15.46:0/2735493349 learned_addr learned my addr 172.21.15.46:0/2735493349 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/2735493349 >> v2:172.21.15.132:3300/0 conn(0x7f84d40c97e0 0x7f84d40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d3fff700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40cb2d0 0x7f84d4158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d37fe700 1 -- 172.21.15.46:0/2735493349 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523016073 0 0) 0x7f84d40cae40 con 0x7f84d406c9f0 2023-10-06T11:19:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84d37fe700 1 -- 172.21.15.46:0/2735493349 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c0003570 con 0x7f84d406c9f0 2023-10-06T11:19:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84cb7fe700 1 -- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40cb2d0 msgr2=0x7f84d4158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40cb2d0 0x7f84d4158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84cb7fe700 1 -- 172.21.15.46:0/2735493349 >> v1:172.21.15.46:6789/0 conn(0x7f84d406c9f0 legacy=0x7f84d406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.690+0000 7f84cb7fe700 1 -- 172.21.15.46:0/2735493349 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40ca7f0 con 0x7f84d40c97e0 2023-10-06T11:19:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40c97e0 0x7f84d40c9c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009e60 tx=0x7f84bc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f756d9b9f088ed9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d37fe700 1 -- 172.21.15.46:0/2735493349 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00e070 con 0x7f84d40c97e0 2023-10-06T11:19:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d37fe700 1 -- 172.21.15.46:0/2735493349 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc00bab0 con 0x7f84d40c97e0 2023-10-06T11:19:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d37fe700 1 -- 172.21.15.46:0/2735493349 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00be70 con 0x7f84d40c97e0 2023-10-06T11:19:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 -- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40c97e0 msgr2=0x7f84d40c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40c97e0 0x7f84d40c9c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009e60 tx=0x7f84bc009780 comp rx=0 tx=0).stop 2023-10-06T11:19:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 -- 172.21.15.46:0/2735493349 shutdown_connections 2023-10-06T11:19:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40c97e0 0x7f84d40c9c30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 --2- 172.21.15.46:0/2735493349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40cb2d0 0x7f84d4158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 -- 172.21.15.46:0/2735493349 >> 172.21.15.46:0/2735493349 conn(0x7f84d40cbc10 msgr2=0x7f84d40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 -- 172.21.15.46:0/2735493349 shutdown_connections 2023-10-06T11:19:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.694+0000 7f84d96e3700 1 -- 172.21.15.46:0/2735493349 wait complete. 2023-10-06T11:19:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 Processor -- start 2023-10-06T11:19:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 -- start start 2023-10-06T11:19:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d406c9f0 0x7f84d414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40cb2d0 0x7f84d406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d4167750 con 0x7f84d40c97e0 2023-10-06T11:19:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d414ba80 con 0x7f84d40cb2d0 2023-10-06T11:19:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d96e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d414bd60 con 0x7f84d406c9f0 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53006/0 (socket says 172.21.15.46:53006) 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1386529826 learned_addr learned my addr 172.21.15.46:0/1386529826 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d8ee2700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40cb2d0 0x7f84d406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84d3fff700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d406c9f0 0x7f84d414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d406c9f0 msgr2=0x7f84d414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d406c9f0 0x7f84d414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40cb2d0 msgr2=0x7f84d406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40cb2d0 0x7f84d406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 -- 172.21.15.46:0/1386529826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc009a20 con 0x7f84d40c97e0 2023-10-06T11:19:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.698+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097c0 tx=0x7f84bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc012ba0 con 0x7f84d40c97e0 2023-10-06T11:19:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc012d20 con 0x7f84d40c97e0 2023-10-06T11:19:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d414c040 con 0x7f84d40c97e0 2023-10-06T11:19:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0276c0 con 0x7f84d40c97e0 2023-10-06T11:19:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d414a890 con 0x7f84d40c97e0 2023-10-06T11:19:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d40c5f00 con 0x7f84d40c97e0 2023-10-06T11:19:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.702+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f84bc024070 con 0x7f84d40c97e0 2023-10-06T11:19:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.706+0000 7f84d0ff9700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 0x7f84b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.706+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84bc07ee60 con 0x7f84d40c97e0 2023-10-06T11:19:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.706+0000 7f84d3fff700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 0x7f84b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.706+0000 7f84d3fff700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 0x7f84b4062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009db0 tx=0x7f84c40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:16.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.710+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f84d40c5f00 con 0x7f84d40c97e0 2023-10-06T11:19:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:16.914+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84d40c5f00 con 0x7f84d40c97e0 2023-10-06T11:19:17.398 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-10-06T11:19:17.394+0000 7ff4e5179700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:19:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.502+0000 7f84d0ff9700 1 -- 172.21.15.46:0/1386529826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84d40c5f00 con 0x7f84d40c97e0 2023-10-06T11:19:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 msgr2=0x7f84b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 0x7f84b4062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009db0 tx=0x7f84c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 msgr2=0x7f84d406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0097c0 tx=0x7f84bc019000 comp rx=0 tx=0).stop 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 shutdown_connections 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84b4060550 0x7f84b4062a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d40cb2d0 0x7f84d406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d40c97e0 0x7f84d406d000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 --2- 172.21.15.46:0/1386529826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d406c9f0 0x7f84d414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 >> 172.21.15.46:0/1386529826 conn(0x7f84d40cbc10 msgr2=0x7f84d40b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 shutdown_connections 2023-10-06T11:19:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.506+0000 7f84d96e3700 1 -- 172.21.15.46:0/1386529826 wait complete. 2023-10-06T11:19:17.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:19:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.702+0000 7f314500c700 1 Processor -- start 2023-10-06T11:19:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.702+0000 7f314500c700 1 -- start start 2023-10-06T11:19:17.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314500c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 0x7f3140068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314500c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f31401591a0 0x7f3140069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314500c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314015d6c0 con 0x7f3140158dd0 2023-10-06T11:19:17.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314500c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31400699c0 con 0x7f31401591a0 2023-10-06T11:19:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314500c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314015db50 con 0x7f3140067760 2023-10-06T11:19:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f313ffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3140158dd0 0x7f3140067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42224/0 (socket says 172.21.15.46:42224) 2023-10-06T11:19:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f313ffff700 1 -- 172.21.15.46:0/578287739 learned_addr learned my addr 172.21.15.46:0/578287739 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314480b700 1 --2- 172.21.15.46:0/578287739 >> v2:172.21.15.46:3300/0 conn(0x7f31401591a0 0x7f3140069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f3137fff700 1 --2- 172.21.15.46:0/578287739 >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 0x7f3140068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f313f7fe700 1 -- 172.21.15.46:0/578287739 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675979222 0 0) 0x7f314015d6c0 con 0x7f3140158dd0 2023-10-06T11:19:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f313f7fe700 1 -- 172.21.15.46:0/578287739 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f312c003570 con 0x7f3140158dd0 2023-10-06T11:19:17.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314480b700 1 -- 172.21.15.46:0/578287739 >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 msgr2=0x7f3140068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314480b700 1 --2- 172.21.15.46:0/578287739 >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 0x7f3140068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314480b700 1 -- 172.21.15.46:0/578287739 >> v1:172.21.15.132:6789/0 conn(0x7f3140158dd0 legacy=0x7f3140067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f314480b700 1 -- 172.21.15.46:0/578287739 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314015cd90 con 0x7f31401591a0 2023-10-06T11:19:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.706+0000 7f3137fff700 1 --2- 172.21.15.46:0/578287739 >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 0x7f3140068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314480b700 1 --2- 172.21.15.46:0/578287739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31401591a0 0x7f3140069130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f314006a410 tx=0x7f3138009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e729a43d213c151 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f313f7fe700 1 -- 172.21.15.46:0/578287739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800e070 con 0x7f31401591a0 2023-10-06T11:19:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f313f7fe700 1 -- 172.21.15.46:0/578287739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800ba70 con 0x7f31401591a0 2023-10-06T11:19:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f313f7fe700 1 -- 172.21.15.46:0/578287739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800be30 con 0x7f31401591a0 2023-10-06T11:19:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 -- 172.21.15.46:0/578287739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31401591a0 msgr2=0x7f3140069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 --2- 172.21.15.46:0/578287739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31401591a0 0x7f3140069130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f314006a410 tx=0x7f3138009780 comp rx=0 tx=0).stop 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 -- 172.21.15.46:0/578287739 shutdown_connections 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 --2- 172.21.15.46:0/578287739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31401591a0 0x7f3140069130 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 --2- 172.21.15.46:0/578287739 >> v2:172.21.15.132:3300/0 conn(0x7f3140067760 0x7f3140068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 -- 172.21.15.46:0/578287739 >> 172.21.15.46:0/578287739 conn(0x7f3140062ef0 msgr2=0x7f31400cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 -- 172.21.15.46:0/578287739 shutdown_connections 2023-10-06T11:19:17.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.710+0000 7f314500c700 1 -- 172.21.15.46:0/578287739 wait complete. 2023-10-06T11:19:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 Processor -- start 2023-10-06T11:19:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 -- start start 2023-10-06T11:19:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3140158dd0 0x7f31400c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31401591a0 0x7f314015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314015a190 con 0x7f3140067760 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3140158480 con 0x7f31401591a0 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314500c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31400c4550 con 0x7f3140158dd0 2023-10-06T11:19:17.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53024/0 (socket says 172.21.15.46:53024) 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 -- 172.21.15.46:0/3801170121 learned_addr learned my addr 172.21.15.46:0/3801170121 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f314480b700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31401591a0 0x7f314015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f3137fff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3140158dd0 0x7f31400c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 -- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3140158dd0 msgr2=0x7f31400c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3140158dd0 0x7f31400c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 -- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31401591a0 msgr2=0x7f314015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31401591a0 0x7f314015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 -- 172.21.15.46:0/3801170121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009420 con 0x7f3140067760 2023-10-06T11:19:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313ffff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f313000c7f0 tx=0x7f3130007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.714+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130005b30 con 0x7f3140067760 2023-10-06T11:19:17.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3130005c90 con 0x7f3140067760 2023-10-06T11:19:17.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3138009a20 con 0x7f3140067760 2023-10-06T11:19:17.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130007340 con 0x7f3140067760 2023-10-06T11:19:17.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31400c3f90 con 0x7f3140067760 2023-10-06T11:19:17.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3140064590 con 0x7f3140067760 2023-10-06T11:19:17.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.718+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f313001b400 con 0x7f3140067760 2023-10-06T11:19:17.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.722+0000 7f313cff9700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 0x7f3120062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:17.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.722+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f313007c4d0 con 0x7f3140067760 2023-10-06T11:19:17.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.722+0000 7f3137fff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 0x7f3120062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:17.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.722+0000 7f3137fff700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 0x7f3120062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31280093f0 tx=0x7f3128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:17.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.730+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f313004c940 con 0x7f3140067760 2023-10-06T11:19:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.938+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f314015d070 con 0x7f3140067760 2023-10-06T11:19:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.938+0000 7f313cff9700 1 -- 172.21.15.46:0/3801170121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f31300076f0 con 0x7f3140067760 2023-10-06T11:19:17.945 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:17.946 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:19:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 msgr2=0x7f3120062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 0x7f3120062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31280093f0 tx=0x7f3128008040 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 msgr2=0x7f3140157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f313000c7f0 tx=0x7f3130007e00 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 shutdown_connections 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3120060550 0x7f3120062a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31401591a0 0x7f314015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3140158dd0 0x7f31400c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 --2- 172.21.15.46:0/3801170121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3140067760 0x7f3140157f40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 >> 172.21.15.46:0/3801170121 conn(0x7f3140062ef0 msgr2=0x7f31400c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 shutdown_connections 2023-10-06T11:19:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:17.946+0000 7f314500c700 1 -- 172.21.15.46:0/3801170121 wait complete. 2023-10-06T11:19:17.954 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:19:17.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:19:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 Processor -- start 2023-10-06T11:19:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 -- start start 2023-10-06T11:19:18.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 0x7f4c38069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c38159980 0x7f4c380695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3815a5d0 con 0x7f4c380b5d00 2023-10-06T11:19:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c38159e00 con 0x7f4c38159980 2023-10-06T11:19:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c3d62e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380ce950 con 0x7f4c38158ea0 2023-10-06T11:19:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c36ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 0x7f4c38069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c377fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c38159980 0x7f4c380695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c2ffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4c380b5d00 0x7f4c38159270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42236/0 (socket says 172.21.15.46:42236) 2023-10-06T11:19:18.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.130+0000 7f4c2ffff700 1 -- 172.21.15.46:0/3709558916 learned_addr learned my addr 172.21.15.46:0/3709558916 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c367fc700 1 -- 172.21.15.46:0/3709558916 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412371135 0 0) 0x7f4c3815a5d0 con 0x7f4c380b5d00 2023-10-06T11:19:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c367fc700 1 -- 172.21.15.46:0/3709558916 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c24003570 con 0x7f4c380b5d00 2023-10-06T11:19:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c36ffd700 1 -- 172.21.15.46:0/3709558916 >> v2:172.21.15.132:3300/0 conn(0x7f4c38159980 msgr2=0x7f4c380695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c36ffd700 1 --2- 172.21.15.46:0/3709558916 >> v2:172.21.15.132:3300/0 conn(0x7f4c38159980 0x7f4c380695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c36ffd700 1 -- 172.21.15.46:0/3709558916 >> v1:172.21.15.132:6789/0 conn(0x7f4c380b5d00 legacy=0x7f4c38159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c36ffd700 1 -- 172.21.15.46:0/3709558916 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380cfdf0 con 0x7f4c38158ea0 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c36ffd700 1 --2- 172.21.15.46:0/3709558916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 0x7f4c38069060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c280098e0 tx=0x7f4c280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc1f4c6ee560c88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c367fc700 1 -- 172.21.15.46:0/3709558916 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28011630 con 0x7f4c38158ea0 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c367fc700 1 -- 172.21.15.46:0/3709558916 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28011790 con 0x7f4c38158ea0 2023-10-06T11:19:18.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c367fc700 1 -- 172.21.15.46:0/3709558916 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c28011b30 con 0x7f4c38158ea0 2023-10-06T11:19:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c3d62e700 1 -- 172.21.15.46:0/3709558916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 msgr2=0x7f4c38069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/3709558916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 0x7f4c38069060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c280098e0 tx=0x7f4c280091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c3d62e700 1 -- 172.21.15.46:0/3709558916 shutdown_connections 2023-10-06T11:19:18.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/3709558916 >> v2:172.21.15.132:3300/0 conn(0x7f4c38159980 0x7f4c380695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.134+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/3709558916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c38158ea0 0x7f4c38069060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.138+0000 7f4c3d62e700 1 -- 172.21.15.46:0/3709558916 >> 172.21.15.46:0/3709558916 conn(0x7f4c38068390 msgr2=0x7f4c380b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:18.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.138+0000 7f4c3d62e700 1 -- 172.21.15.46:0/3709558916 shutdown_connections 2023-10-06T11:19:18.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.138+0000 7f4c3d62e700 1 -- 172.21.15.46:0/3709558916 wait complete. 2023-10-06T11:19:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.138+0000 7f4c3d62e700 1 Processor -- start 2023-10-06T11:19:18.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 -- start start 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c380b5d00 0x7f4c380cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38159980 0x7f4c380cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3816c670 con 0x7f4c38158ea0 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38169590 con 0x7f4c38159980 2023-10-06T11:19:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38169870 con 0x7f4c380b5d00 2023-10-06T11:19:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53036/0 (socket says 172.21.15.46:53036) 2023-10-06T11:19:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 -- 172.21.15.46:0/1891494199 learned_addr learned my addr 172.21.15.46:0/1891494199 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c36ffd700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c380b5d00 0x7f4c380cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c377fe700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38159980 0x7f4c380cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 -- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c380b5d00 msgr2=0x7f4c380cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c380b5d00 0x7f4c380cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 -- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38159980 msgr2=0x7f4c380cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38159980 0x7f4c380cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 -- 172.21.15.46:0/1891494199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28009420 con 0x7f4c38158ea0 2023-10-06T11:19:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2ffff700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000c590 tx=0x7f4c20007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20005b30 con 0x7f4c38158ea0 2023-10-06T11:19:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20005cb0 con 0x7f4c38158ea0 2023-10-06T11:19:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c38169b50 con 0x7f4c38158ea0 2023-10-06T11:19:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.142+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20007320 con 0x7f4c38158ea0 2023-10-06T11:19:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.146+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c380cba40 con 0x7f4c38158ea0 2023-10-06T11:19:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.146+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c380cbc60 con 0x7f4c38158ea0 2023-10-06T11:19:18.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.146+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4c2001b3b0 con 0x7f4c38158ea0 2023-10-06T11:19:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.150+0000 7f4c2effd700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 0x7f4c18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.150+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4c2007be10 con 0x7f4c38158ea0 2023-10-06T11:19:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.150+0000 7f4c36ffd700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 0x7f4c18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.150+0000 7f4c36ffd700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 0x7f4c18062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c280091e0 tx=0x7f4c28017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.158+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c2007f030 con 0x7f4c38158ea0 2023-10-06T11:19:18.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.334+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c38064590 con 0x7f4c38158ea0 2023-10-06T11:19:18.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.334+0000 7f4c2effd700 1 -- 172.21.15.46:0/1891494199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f4c2004c200 con 0x7f4c38158ea0 2023-10-06T11:19:18.341 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:18.341 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:19:15.449399+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 msgr2=0x7f4c18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 0x7f4c18062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c280091e0 tx=0x7f4c28017000 comp rx=0 tx=0).stop 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 msgr2=0x7f4c3816a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000c590 tx=0x7f4c20007e00 comp rx=0 tx=0).stop 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 shutdown_connections 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c18060550 0x7f4c18062a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c38159980 0x7f4c380cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c38158ea0 0x7f4c3816a110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 --2- 172.21.15.46:0/1891494199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c380b5d00 0x7f4c380cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 >> 172.21.15.46:0/1891494199 conn(0x7f4c38068390 msgr2=0x7f4c380b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 shutdown_connections 2023-10-06T11:19:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.338+0000 7f4c3d62e700 1 -- 172.21.15.46:0/1891494199 wait complete. 2023-10-06T11:19:18.366 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:19:18.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:19:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 Processor -- start 2023-10-06T11:19:18.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 -- start start 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6680c32b0 0x7fd6680cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66806e5a0 0x7fd6680cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680671f0 con 0x7fd6680b02b0 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680ce770 con 0x7fd6680c32b0 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd66d82e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cded0 con 0x7fd66806e5a0 2023-10-06T11:19:18.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd666ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6680b02b0 0x7fd6680c2ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34136/0 (socket says 172.21.15.46:34136) 2023-10-06T11:19:18.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd65e7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6680c32b0 0x7fd6680cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.522+0000 7fd666ffd700 1 -- 172.21.15.46:0/2036301347 learned_addr learned my addr 172.21.15.46:0/2036301347 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:18.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd667fff700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66806e5a0 0x7fd6680cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd65e7fc700 1 -- 172.21.15.46:0/2036301347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66806e5a0 msgr2=0x7fd6680cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd65e7fc700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66806e5a0 0x7fd6680cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd65e7fc700 1 -- 172.21.15.46:0/2036301347 >> v1:172.21.15.46:6789/0 conn(0x7fd6680b02b0 legacy=0x7fd6680c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd65e7fc700 1 -- 172.21.15.46:0/2036301347 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680ce1b0 con 0x7fd6680c32b0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd6667fc700 1 -- 172.21.15.46:0/2036301347 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167107449 0 0) 0x7fd6680671f0 con 0x7fd6680b02b0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd65e7fc700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd6680cc450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098e0 tx=0x7fd6500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9531112f3f985d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd6667fc700 1 -- 172.21.15.46:0/2036301347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000b640 con 0x7fd6680c32b0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd6667fc700 1 -- 172.21.15.46:0/2036301347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65000b7a0 con 0x7fd6680c32b0 2023-10-06T11:19:18.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd6667fc700 1 -- 172.21.15.46:0/2036301347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000bb40 con 0x7fd6680c32b0 2023-10-06T11:19:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 -- 172.21.15.46:0/2036301347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 msgr2=0x7fd6680cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd6680cc450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd6500098e0 tx=0x7fd6500091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 -- 172.21.15.46:0/2036301347 shutdown_connections 2023-10-06T11:19:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66806e5a0 0x7fd6680cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 --2- 172.21.15.46:0/2036301347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd6680cc450 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 -- 172.21.15.46:0/2036301347 >> 172.21.15.46:0/2036301347 conn(0x7fd6680b0cc0 msgr2=0x7fd6680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:18.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 -- 172.21.15.46:0/2036301347 shutdown_connections 2023-10-06T11:19:18.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.526+0000 7fd66d82e700 1 -- 172.21.15.46:0/2036301347 wait complete. 2023-10-06T11:19:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.530+0000 7fd66d82e700 1 Processor -- start 2023-10-06T11:19:18.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.530+0000 7fd66d82e700 1 -- start start 2023-10-06T11:19:18.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 0x7fd6680989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 0x7fd66815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668099c60 con 0x7fd6680c32b0 2023-10-06T11:19:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668099050 con 0x7fd66806e5a0 2023-10-06T11:19:18.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668099330 con 0x7fd6680b02b0 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53052/0 (socket says 172.21.15.46:53052) 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 -- 172.21.15.46:0/111745022 learned_addr learned my addr 172.21.15.46:0/111745022 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd65e7fc700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 0x7fd66815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd666ffd700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 0x7fd6680989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 -- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 msgr2=0x7fd66815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 0x7fd66815c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 -- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 msgr2=0x7fd6680989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 0x7fd6680989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 -- 172.21.15.46:0/111745022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650009420 con 0x7fd6680c32b0 2023-10-06T11:19:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd65e7fc700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 0x7fd66815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:18.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd666ffd700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 0x7fd6680989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd667fff700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e90 tx=0x7fd66000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000cc20 con 0x7fd6680c32b0 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000f040 con 0x7fd6680c32b0 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668099610 con 0x7fd6680c32b0 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660014480 con 0x7fd6680c32b0 2023-10-06T11:19:18.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.534+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66816d0b0 con 0x7fd6680c32b0 2023-10-06T11:19:18.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.538+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66816d300 con 0x7fd6680c32b0 2023-10-06T11:19:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.538+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd6600147f0 con 0x7fd6680c32b0 2023-10-06T11:19:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.538+0000 7fd65f7fe700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 0x7fd648062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.538+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd66007d730 con 0x7fd6680c32b0 2023-10-06T11:19:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.542+0000 7fd666ffd700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 0x7fd648062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.542+0000 7fd666ffd700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 0x7fd648062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6680c3f30 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.546+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6600515b0 con 0x7fd6680c32b0 2023-10-06T11:19:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.718+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd668087f60 con 0x7fd648060550 2023-10-06T11:19:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.734+0000 7fd65f7fe700 1 -- 172.21.15.46:0/111745022 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306041 (secure 0 0 0) 0x7fd668087f60 con 0x7fd648060550 2023-10-06T11:19:18.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 msgr2=0x7fd648062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 0x7fd648062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6680c3f30 tx=0x7fd658009040 comp rx=0 tx=0).stop 2023-10-06T11:19:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 msgr2=0x7fd66816cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd660009e90 tx=0x7fd66000b040 comp rx=0 tx=0).stop 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 shutdown_connections 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd648060550 0x7fd648062a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6680c32b0 0x7fd66816cb70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6680b02b0 0x7fd66815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 --2- 172.21.15.46:0/111745022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66806e5a0 0x7fd6680989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 >> 172.21.15.46:0/111745022 conn(0x7fd6680b0cc0 msgr2=0x7fd6680c2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 shutdown_connections 2023-10-06T11:19:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.746+0000 7fd66d82e700 1 -- 172.21.15.46:0/111745022 wait complete. 2023-10-06T11:19:18.752 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:19:18.768 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T11:19:18.500216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398776,"kb_used_data":4088,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575944,"statfs":{"total":773094113280,"available":770637766656,"internally_reserved":0,"allocated":4186112,"data_stored":2380526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001607"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.471925+0000","last_change":"2023-10-06T11:19:14.471925+0000","last_active":"2023-10-06T11:19:14.471925+0000","last_peered":"2023-10-06T11:19:14.471925+0000","last_clean":"2023-10-06T11:19:14.471925+0000","last_became_active":"2023-10-06T11:19:14.471635+0000","last_became_peered":"2023-10-06T11:19:14.471635+0000","last_unstale":"2023-10-06T11:19:14.471925+0000","last_undegraded":"2023-10-06T11:19:14.471925+0000","last_fullsized":"2023-10-06T11:19:14.471925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485456+0000","last_change":"2023-10-06T11:19:14.485456+0000","last_active":"2023-10-06T11:19:14.485456+0000","last_peered":"2023-10-06T11:19:14.485456+0000","last_clean":"2023-10-06T11:19:14.485456+0000","last_became_active":"2023-10-06T11:19:14.482268+0000","last_became_peered":"2023-10-06T11:19:14.482268+0000","last_unstale":"2023-10-06T11:19:14.485456+0000","last_undegraded":"2023-10-06T11:19:14.485456+0000","last_fullsized":"2023-10-06T11:19:14.485456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490196+0000","last_change":"2023-10-06T11:19:14.490196+0000","last_active":"2023-10-06T11:19:14.490196+0000","last_peered":"2023-10-06T11:19:14.490196+0000","last_clean":"2023-10-06T11:19:14.490196+0000","last_became_active":"2023-10-06T11:19:14.489831+0000","last_became_peered":"2023-10-06T11:19:14.489831+0000","last_unstale":"2023-10-06T11:19:14.490196+0000","last_undegraded":"2023-10-06T11:19:14.490196+0000","last_fullsized":"2023-10-06T11:19:14.490196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490930+0000","last_change":"2023-10-06T11:19:14.490930+0000","last_active":"2023-10-06T11:19:14.490930+0000","last_peered":"2023-10-06T11:19:14.490930+0000","last_clean":"2023-10-06T11:19:14.490930+0000","last_became_active":"2023-10-06T11:19:14.489432+0000","last_became_peered":"2023-10-06T11:19:14.489432+0000","last_unstale":"2023-10-06T11:19:14.490930+0000","last_undegraded":"2023-10-06T11:19:14.490930+0000","last_fullsized":"2023-10-06T11:19:14.490930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:34.096691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477790+0000","last_change":"2023-10-06T11:19:14.477790+0000","last_active":"2023-10-06T11:19:14.477790+0000","last_peered":"2023-10-06T11:19:14.477790+0000","last_clean":"2023-10-06T11:19:14.477790+0000","last_became_active":"2023-10-06T11:19:14.477500+0000","last_became_peered":"2023-10-06T11:19:14.477500+0000","last_unstale":"2023-10-06T11:19:14.477790+0000","last_undegraded":"2023-10-06T11:19:14.477790+0000","last_fullsized":"2023-10-06T11:19:14.477790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:34.588402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487680+0000","last_change":"2023-10-06T11:19:14.487680+0000","last_active":"2023-10-06T11:19:14.487680+0000","last_peered":"2023-10-06T11:19:14.487680+0000","last_clean":"2023-10-06T11:19:14.487680+0000","last_became_active":"2023-10-06T11:19:14.487388+0000","last_became_peered":"2023-10-06T11:19:14.487388+0000","last_unstale":"2023-10-06T11:19:14.487680+0000","last_undegraded":"2023-10-06T11:19:14.487680+0000","last_fullsized":"2023-10-06T11:19:14.487680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.609253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492999+0000","last_change":"2023-10-06T11:19:14.492999+0000","last_active":"2023-10-06T11:19:14.492999+0000","last_peered":"2023-10-06T11:19:14.492999+0000","last_clean":"2023-10-06T11:19:14.492999+0000","last_became_active":"2023-10-06T11:19:14.492704+0000","last_became_peered":"2023-10-06T11:19:14.492704+0000","last_unstale":"2023-10-06T11:19:14.492999+0000","last_undegraded":"2023-10-06T11:19:14.492999+0000","last_fullsized":"2023-10-06T11:19:14.492999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481394+0000","last_change":"2023-10-06T11:19:14.481394+0000","last_active":"2023-10-06T11:19:14.481394+0000","last_peered":"2023-10-06T11:19:14.481394+0000","last_clean":"2023-10-06T11:19:14.481394+0000","last_became_active":"2023-10-06T11:19:14.481076+0000","last_became_peered":"2023-10-06T11:19:14.481076+0000","last_unstale":"2023-10-06T11:19:14.481394+0000","last_undegraded":"2023-10-06T11:19:14.481394+0000","last_fullsized":"2023-10-06T11:19:14.481394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482441+0000","last_change":"2023-10-06T11:19:14.482441+0000","last_active":"2023-10-06T11:19:14.482441+0000","last_peered":"2023-10-06T11:19:14.482441+0000","last_clean":"2023-10-06T11:19:14.482441+0000","last_became_active":"2023-10-06T11:19:14.481997+0000","last_became_peered":"2023-10-06T11:19:14.481997+0000","last_unstale":"2023-10-06T11:19:14.482441+0000","last_undegraded":"2023-10-06T11:19:14.482441+0000","last_fullsized":"2023-10-06T11:19:14.482441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479690+0000","last_change":"2023-10-06T11:19:14.479690+0000","last_active":"2023-10-06T11:19:14.479690+0000","last_peered":"2023-10-06T11:19:14.479690+0000","last_clean":"2023-10-06T11:19:14.479690+0000","last_became_active":"2023-10-06T11:19:14.479355+0000","last_became_peered":"2023-10-06T11:19:14.479355+0000","last_unstale":"2023-10-06T11:19:14.479690+0000","last_undegraded":"2023-10-06T11:19:14.479690+0000","last_fullsized":"2023-10-06T11:19:14.479690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:32.732866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479003+0000","last_change":"2023-10-06T11:19:14.479003+0000","last_active":"2023-10-06T11:19:14.479003+0000","last_peered":"2023-10-06T11:19:14.479003+0000","last_clean":"2023-10-06T11:19:14.479003+0000","last_became_active":"2023-10-06T11:19:14.478688+0000","last_became_peered":"2023-10-06T11:19:14.478688+0000","last_unstale":"2023-10-06T11:19:14.479003+0000","last_undegraded":"2023-10-06T11:19:14.479003+0000","last_fullsized":"2023-10-06T11:19:14.479003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479839+0000","last_change":"2023-10-06T11:19:14.479839+0000","last_active":"2023-10-06T11:19:14.479839+0000","last_peered":"2023-10-06T11:19:14.479839+0000","last_clean":"2023-10-06T11:19:14.479839+0000","last_became_active":"2023-10-06T11:19:14.479572+0000","last_became_peered":"2023-10-06T11:19:14.479572+0000","last_unstale":"2023-10-06T11:19:14.479839+0000","last_undegraded":"2023-10-06T11:19:14.479839+0000","last_fullsized":"2023-10-06T11:19:14.479839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.496930+0000","last_change":"2023-10-06T11:19:14.496930+0000","last_active":"2023-10-06T11:19:14.496930+0000","last_peered":"2023-10-06T11:19:14.496930+0000","last_clean":"2023-10-06T11:19:14.496930+0000","last_became_active":"2023-10-06T11:19:14.496578+0000","last_became_peered":"2023-10-06T11:19:14.496578+0000","last_unstale":"2023-10-06T11:19:14.496930+0000","last_undegraded":"2023-10-06T11:19:14.496930+0000","last_fullsized":"2023-10-06T11:19:14.496930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483089+0000","last_change":"2023-10-06T11:19:14.483089+0000","last_active":"2023-10-06T11:19:14.483089+0000","last_peered":"2023-10-06T11:19:14.483089+0000","last_clean":"2023-10-06T11:19:14.483089+0000","last_became_active":"2023-10-06T11:19:14.482798+0000","last_became_peered":"2023-10-06T11:19:14.482798+0000","last_unstale":"2023-10-06T11:19:14.483089+0000","last_undegraded":"2023-10-06T11:19:14.483089+0000","last_fullsized":"2023-10-06T11:19:14.483089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:53.890649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490604+0000","last_change":"2023-10-06T11:19:14.490604+0000","last_active":"2023-10-06T11:19:14.490604+0000","last_peered":"2023-10-06T11:19:14.490604+0000","last_clean":"2023-10-06T11:19:14.490604+0000","last_became_active":"2023-10-06T11:19:14.490331+0000","last_became_peered":"2023-10-06T11:19:14.490331+0000","last_unstale":"2023-10-06T11:19:14.490604+0000","last_undegraded":"2023-10-06T11:19:14.490604+0000","last_fullsized":"2023-10-06T11:19:14.490604+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492291+0000","last_change":"2023-10-06T11:19:14.492291+0000","last_active":"2023-10-06T11:19:14.492291+0000","last_peered":"2023-10-06T11:19:14.492291+0000","last_clean":"2023-10-06T11:19:14.492291+0000","last_became_active":"2023-10-06T11:19:14.492004+0000","last_became_peered":"2023-10-06T11:19:14.492004+0000","last_unstale":"2023-10-06T11:19:14.492291+0000","last_undegraded":"2023-10-06T11:19:14.492291+0000","last_fullsized":"2023-10-06T11:19:14.492291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485054+0000","last_change":"2023-10-06T11:19:14.485054+0000","last_active":"2023-10-06T11:19:14.485054+0000","last_peered":"2023-10-06T11:19:14.485054+0000","last_clean":"2023-10-06T11:19:14.485054+0000","last_became_active":"2023-10-06T11:19:14.484769+0000","last_became_peered":"2023-10-06T11:19:14.484769+0000","last_unstale":"2023-10-06T11:19:14.485054+0000","last_undegraded":"2023-10-06T11:19:14.485054+0000","last_fullsized":"2023-10-06T11:19:14.485054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:58.083133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474524+0000","last_change":"2023-10-06T11:19:14.474524+0000","last_active":"2023-10-06T11:19:14.474524+0000","last_peered":"2023-10-06T11:19:14.474524+0000","last_clean":"2023-10-06T11:19:14.474524+0000","last_became_active":"2023-10-06T11:19:14.474158+0000","last_became_peered":"2023-10-06T11:19:14.474158+0000","last_unstale":"2023-10-06T11:19:14.474524+0000","last_undegraded":"2023-10-06T11:19:14.474524+0000","last_fullsized":"2023-10-06T11:19:14.474524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:13.626160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487553+0000","last_change":"2023-10-06T11:19:14.487553+0000","last_active":"2023-10-06T11:19:14.487553+0000","last_peered":"2023-10-06T11:19:14.487553+0000","last_clean":"2023-10-06T11:19:14.487553+0000","last_became_active":"2023-10-06T11:19:14.483336+0000","last_became_peered":"2023-10-06T11:19:14.483336+0000","last_unstale":"2023-10-06T11:19:14.487553+0000","last_undegraded":"2023-10-06T11:19:14.487553+0000","last_fullsized":"2023-10-06T11:19:14.487553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:41.757732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474773+0000","last_change":"2023-10-06T11:19:14.474773+0000","last_active":"2023-10-06T11:19:14.474773+0000","last_peered":"2023-10-06T11:19:14.474773+0000","last_clean":"2023-10-06T11:19:14.474773+0000","last_became_active":"2023-10-06T11:19:14.474125+0000","last_became_peered":"2023-10-06T11:19:14.474125+0000","last_unstale":"2023-10-06T11:19:14.474773+0000","last_undegraded":"2023-10-06T11:19:14.474773+0000","last_fullsized":"2023-10-06T11:19:14.474773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:34.588402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.488905+0000","last_change":"2023-10-06T11:19:14.488905+0000","last_active":"2023-10-06T11:19:14.488905+0000","last_peered":"2023-10-06T11:19:14.488905+0000","last_clean":"2023-10-06T11:19:14.488905+0000","last_became_active":"2023-10-06T11:19:14.488654+0000","last_became_peered":"2023-10-06T11:19:14.488654+0000","last_unstale":"2023-10-06T11:19:14.488905+0000","last_undegraded":"2023-10-06T11:19:14.488905+0000","last_fullsized":"2023-10-06T11:19:14.488905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481693+0000","last_change":"2023-10-06T11:19:14.481693+0000","last_active":"2023-10-06T11:19:14.481693+0000","last_peered":"2023-10-06T11:19:14.481693+0000","last_clean":"2023-10-06T11:19:14.481693+0000","last_became_active":"2023-10-06T11:19:14.481221+0000","last_became_peered":"2023-10-06T11:19:14.481221+0000","last_unstale":"2023-10-06T11:19:14.481693+0000","last_undegraded":"2023-10-06T11:19:14.481693+0000","last_fullsized":"2023-10-06T11:19:14.481693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:32.732866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485668+0000","last_change":"2023-10-06T11:19:14.485668+0000","last_active":"2023-10-06T11:19:14.485668+0000","last_peered":"2023-10-06T11:19:14.485668+0000","last_clean":"2023-10-06T11:19:14.485668+0000","last_became_active":"2023-10-06T11:19:14.483270+0000","last_became_peered":"2023-10-06T11:19:14.483270+0000","last_unstale":"2023-10-06T11:19:14.485668+0000","last_undegraded":"2023-10-06T11:19:14.485668+0000","last_fullsized":"2023-10-06T11:19:14.485668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:36.804886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482373+0000","last_change":"2023-10-06T11:19:14.482373+0000","last_active":"2023-10-06T11:19:14.482373+0000","last_peered":"2023-10-06T11:19:14.482373+0000","last_clean":"2023-10-06T11:19:14.482373+0000","last_became_active":"2023-10-06T11:19:14.481735+0000","last_became_peered":"2023-10-06T11:19:14.481735+0000","last_unstale":"2023-10-06T11:19:14.482373+0000","last_undegraded":"2023-10-06T11:19:14.482373+0000","last_fullsized":"2023-10-06T11:19:14.482373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:13.626160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.475676+0000","last_change":"2023-10-06T11:19:14.475676+0000","last_active":"2023-10-06T11:19:14.475676+0000","last_peered":"2023-10-06T11:19:14.475676+0000","last_clean":"2023-10-06T11:19:14.475676+0000","last_became_active":"2023-10-06T11:19:14.475349+0000","last_became_peered":"2023-10-06T11:19:14.475349+0000","last_unstale":"2023-10-06T11:19:14.475676+0000","last_undegraded":"2023-10-06T11:19:14.475676+0000","last_fullsized":"2023-10-06T11:19:14.475676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:35.966826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494543+0000","last_change":"2023-10-06T11:19:14.494543+0000","last_active":"2023-10-06T11:19:14.494543+0000","last_peered":"2023-10-06T11:19:14.494543+0000","last_clean":"2023-10-06T11:19:14.494543+0000","last_became_active":"2023-10-06T11:19:14.494222+0000","last_became_peered":"2023-10-06T11:19:14.494222+0000","last_unstale":"2023-10-06T11:19:14.494543+0000","last_undegraded":"2023-10-06T11:19:14.494543+0000","last_fullsized":"2023-10-06T11:19:14.494543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:57.918616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481645+0000","last_change":"2023-10-06T11:19:14.481645+0000","last_active":"2023-10-06T11:19:14.481645+0000","last_peered":"2023-10-06T11:19:14.481645+0000","last_clean":"2023-10-06T11:19:14.481645+0000","last_became_active":"2023-10-06T11:19:14.481241+0000","last_became_peered":"2023-10-06T11:19:14.481241+0000","last_unstale":"2023-10-06T11:19:14.481645+0000","last_undegraded":"2023-10-06T11:19:14.481645+0000","last_fullsized":"2023-10-06T11:19:14.481645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:52.498302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492622+0000","last_change":"2023-10-06T11:19:14.492622+0000","last_active":"2023-10-06T11:19:14.492622+0000","last_peered":"2023-10-06T11:19:14.492622+0000","last_clean":"2023-10-06T11:19:14.492622+0000","last_became_active":"2023-10-06T11:19:14.492274+0000","last_became_peered":"2023-10-06T11:19:14.492274+0000","last_unstale":"2023-10-06T11:19:14.492622+0000","last_undegraded":"2023-10-06T11:19:14.492622+0000","last_fullsized":"2023-10-06T11:19:14.492622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482508+0000","last_change":"2023-10-06T11:19:14.482508+0000","last_active":"2023-10-06T11:19:14.482508+0000","last_peered":"2023-10-06T11:19:14.482508+0000","last_clean":"2023-10-06T11:19:14.482508+0000","last_became_active":"2023-10-06T11:19:14.482233+0000","last_became_peered":"2023-10-06T11:19:14.482233+0000","last_unstale":"2023-10-06T11:19:14.482508+0000","last_undegraded":"2023-10-06T11:19:14.482508+0000","last_fullsized":"2023-10-06T11:19:14.482508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:04:07.720382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.480621+0000","last_change":"2023-10-06T11:19:14.480621+0000","last_active":"2023-10-06T11:19:14.480621+0000","last_peered":"2023-10-06T11:19:14.480621+0000","last_clean":"2023-10-06T11:19:14.480621+0000","last_became_active":"2023-10-06T11:19:14.480305+0000","last_became_peered":"2023-10-06T11:19:14.480305+0000","last_unstale":"2023-10-06T11:19:14.480621+0000","last_undegraded":"2023-10-06T11:19:14.480621+0000","last_fullsized":"2023-10-06T11:19:14.480621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:56.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482707+0000","last_change":"2023-10-06T11:19:14.482707+0000","last_active":"2023-10-06T11:19:14.482707+0000","last_peered":"2023-10-06T11:19:14.482707+0000","last_clean":"2023-10-06T11:19:14.482707+0000","last_became_active":"2023-10-06T11:19:14.482414+0000","last_became_peered":"2023-10-06T11:19:14.482414+0000","last_unstale":"2023-10-06T11:19:14.482707+0000","last_undegraded":"2023-10-06T11:19:14.482707+0000","last_fullsized":"2023-10-06T11:19:14.482707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:36.953814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482141+0000","last_change":"2023-10-06T11:19:14.482141+0000","last_active":"2023-10-06T11:19:14.482141+0000","last_peered":"2023-10-06T11:19:14.482141+0000","last_clean":"2023-10-06T11:19:14.482141+0000","last_became_active":"2023-10-06T11:19:14.479449+0000","last_became_peered":"2023-10-06T11:19:14.479449+0000","last_unstale":"2023-10-06T11:19:14.482141+0000","last_undegraded":"2023-10-06T11:19:14.482141+0000","last_fullsized":"2023-10-06T11:19:14.482141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:40.785050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.491815+0000","last_change":"2023-10-06T11:19:14.491815+0000","last_active":"2023-10-06T11:19:14.491815+0000","last_peered":"2023-10-06T11:19:14.491815+0000","last_clean":"2023-10-06T11:19:14.491815+0000","last_became_active":"2023-10-06T11:19:14.491445+0000","last_became_peered":"2023-10-06T11:19:14.491445+0000","last_unstale":"2023-10-06T11:19:14.491815+0000","last_undegraded":"2023-10-06T11:19:14.491815+0000","last_fullsized":"2023-10-06T11:19:14.491815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:07.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499136+0000","last_change":"2023-10-06T11:19:13.475202+0000","last_active":"2023-10-06T11:19:14.499136+0000","last_peered":"2023-10-06T11:19:14.499136+0000","last_clean":"2023-10-06T11:19:14.499136+0000","last_became_active":"2023-10-06T11:19:13.474873+0000","last_became_peered":"2023-10-06T11:19:13.474873+0000","last_unstale":"2023-10-06T11:19:14.499136+0000","last_undegraded":"2023-10-06T11:19:14.499136+0000","last_fullsized":"2023-10-06T11:19:14.499136+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:06.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481269+0000","last_change":"2023-10-06T11:19:14.481269+0000","last_active":"2023-10-06T11:19:14.481269+0000","last_peered":"2023-10-06T11:19:14.481269+0000","last_clean":"2023-10-06T11:19:14.481269+0000","last_became_active":"2023-10-06T11:19:14.479998+0000","last_became_peered":"2023-10-06T11:19:14.479998+0000","last_unstale":"2023-10-06T11:19:14.481269+0000","last_undegraded":"2023-10-06T11:19:14.481269+0000","last_fullsized":"2023-10-06T11:19:14.481269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:41.975314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.464281+0000","last_change":"2023-10-06T11:19:13.473063+0000","last_active":"2023-10-06T11:19:14.464281+0000","last_peered":"2023-10-06T11:19:14.464281+0000","last_clean":"2023-10-06T11:19:14.464281+0000","last_became_active":"2023-10-06T11:19:13.472394+0000","last_became_peered":"2023-10-06T11:19:13.472394+0000","last_unstale":"2023-10-06T11:19:14.464281+0000","last_undegraded":"2023-10-06T11:19:14.464281+0000","last_fullsized":"2023-10-06T11:19:14.464281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:56.903816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.476811+0000","last_change":"2023-10-06T11:19:14.476811+0000","last_active":"2023-10-06T11:19:14.476811+0000","last_peered":"2023-10-06T11:19:14.476811+0000","last_clean":"2023-10-06T11:19:14.476811+0000","last_became_active":"2023-10-06T11:19:14.476485+0000","last_became_peered":"2023-10-06T11:19:14.476485+0000","last_unstale":"2023-10-06T11:19:14.476811+0000","last_undegraded":"2023-10-06T11:19:14.476811+0000","last_fullsized":"2023-10-06T11:19:14.476811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:01.227294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499928+0000","last_change":"2023-10-06T11:19:13.470307+0000","last_active":"2023-10-06T11:19:14.499928+0000","last_peered":"2023-10-06T11:19:14.499928+0000","last_clean":"2023-10-06T11:19:14.499928+0000","last_became_active":"2023-10-06T11:19:13.470002+0000","last_became_peered":"2023-10-06T11:19:13.470002+0000","last_unstale":"2023-10-06T11:19:14.499928+0000","last_undegraded":"2023-10-06T11:19:14.499928+0000","last_fullsized":"2023-10-06T11:19:14.499928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:34.952026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478182+0000","last_change":"2023-10-06T11:19:14.478182+0000","last_active":"2023-10-06T11:19:14.478182+0000","last_peered":"2023-10-06T11:19:14.478182+0000","last_clean":"2023-10-06T11:19:14.478182+0000","last_became_active":"2023-10-06T11:19:14.475164+0000","last_became_peered":"2023-10-06T11:19:14.475164+0000","last_unstale":"2023-10-06T11:19:14.478182+0000","last_undegraded":"2023-10-06T11:19:14.478182+0000","last_fullsized":"2023-10-06T11:19:14.478182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492335+0000","last_change":"2023-10-06T11:19:13.467957+0000","last_active":"2023-10-06T11:19:14.492335+0000","last_peered":"2023-10-06T11:19:14.492335+0000","last_clean":"2023-10-06T11:19:14.492335+0000","last_became_active":"2023-10-06T11:19:13.467664+0000","last_became_peered":"2023-10-06T11:19:13.467664+0000","last_unstale":"2023-10-06T11:19:14.492335+0000","last_undegraded":"2023-10-06T11:19:14.492335+0000","last_fullsized":"2023-10-06T11:19:14.492335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:48.810920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485521+0000","last_change":"2023-10-06T11:19:14.485521+0000","last_active":"2023-10-06T11:19:14.485521+0000","last_peered":"2023-10-06T11:19:14.485521+0000","last_clean":"2023-10-06T11:19:14.485521+0000","last_became_active":"2023-10-06T11:19:14.485225+0000","last_became_peered":"2023-10-06T11:19:14.485225+0000","last_unstale":"2023-10-06T11:19:14.485521+0000","last_undegraded":"2023-10-06T11:19:14.485521+0000","last_fullsized":"2023-10-06T11:19:14.485521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:07.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473703+0000","last_change":"2023-10-06T11:19:13.475244+0000","last_active":"2023-10-06T11:19:14.473703+0000","last_peered":"2023-10-06T11:19:14.473703+0000","last_clean":"2023-10-06T11:19:14.473703+0000","last_became_active":"2023-10-06T11:19:13.474883+0000","last_became_peered":"2023-10-06T11:19:13.474883+0000","last_unstale":"2023-10-06T11:19:14.473703+0000","last_undegraded":"2023-10-06T11:19:14.473703+0000","last_fullsized":"2023-10-06T11:19:14.473703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:02.929291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484013+0000","last_change":"2023-10-06T11:19:14.484013+0000","last_active":"2023-10-06T11:19:14.484013+0000","last_peered":"2023-10-06T11:19:14.484013+0000","last_clean":"2023-10-06T11:19:14.484013+0000","last_became_active":"2023-10-06T11:19:14.483705+0000","last_became_peered":"2023-10-06T11:19:14.483705+0000","last_unstale":"2023-10-06T11:19:14.484013+0000","last_undegraded":"2023-10-06T11:19:14.484013+0000","last_fullsized":"2023-10-06T11:19:14.484013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:54.897036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.463079+0000","last_change":"2023-10-06T11:19:13.479925+0000","last_active":"2023-10-06T11:19:14.463079+0000","last_peered":"2023-10-06T11:19:14.463079+0000","last_clean":"2023-10-06T11:19:14.463079+0000","last_became_active":"2023-10-06T11:19:13.479575+0000","last_became_peered":"2023-10-06T11:19:13.479575+0000","last_unstale":"2023-10-06T11:19:14.463079+0000","last_undegraded":"2023-10-06T11:19:14.463079+0000","last_fullsized":"2023-10-06T11:19:14.463079+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:55.699197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.493849+0000","last_change":"2023-10-06T11:19:14.493849+0000","last_active":"2023-10-06T11:19:14.493849+0000","last_peered":"2023-10-06T11:19:14.493849+0000","last_clean":"2023-10-06T11:19:14.493849+0000","last_became_active":"2023-10-06T11:19:14.491624+0000","last_became_peered":"2023-10-06T11:19:14.491624+0000","last_unstale":"2023-10-06T11:19:14.493849+0000","last_undegraded":"2023-10-06T11:19:14.493849+0000","last_fullsized":"2023-10-06T11:19:14.493849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477975+0000","last_change":"2023-10-06T11:19:13.476765+0000","last_active":"2023-10-06T11:19:13.477975+0000","last_peered":"2023-10-06T11:19:13.477975+0000","last_clean":"2023-10-06T11:19:13.477975+0000","last_became_active":"2023-10-06T11:19:13.476280+0000","last_became_peered":"2023-10-06T11:19:13.476280+0000","last_unstale":"2023-10-06T11:19:13.477975+0000","last_undegraded":"2023-10-06T11:19:13.477975+0000","last_fullsized":"2023-10-06T11:19:13.477975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:55.699197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.469515+0000","last_change":"2023-10-06T11:19:14.469515+0000","last_active":"2023-10-06T11:19:14.469515+0000","last_peered":"2023-10-06T11:19:14.469515+0000","last_clean":"2023-10-06T11:19:14.469515+0000","last_became_active":"2023-10-06T11:19:14.469196+0000","last_became_peered":"2023-10-06T11:19:14.469196+0000","last_unstale":"2023-10-06T11:19:14.469515+0000","last_undegraded":"2023-10-06T11:19:14.469515+0000","last_fullsized":"2023-10-06T11:19:14.469515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.479312+0000","last_change":"2023-10-06T11:19:13.473986+0000","last_active":"2023-10-06T11:19:13.479312+0000","last_peered":"2023-10-06T11:19:13.479312+0000","last_clean":"2023-10-06T11:19:13.479312+0000","last_became_active":"2023-10-06T11:19:13.473583+0000","last_became_peered":"2023-10-06T11:19:13.473583+0000","last_unstale":"2023-10-06T11:19:13.479312+0000","last_undegraded":"2023-10-06T11:19:13.479312+0000","last_fullsized":"2023-10-06T11:19:13.479312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:35.790086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483232+0000","last_change":"2023-10-06T11:19:14.483232+0000","last_active":"2023-10-06T11:19:14.483232+0000","last_peered":"2023-10-06T11:19:14.483232+0000","last_clean":"2023-10-06T11:19:14.483232+0000","last_became_active":"2023-10-06T11:19:14.480608+0000","last_became_peered":"2023-10-06T11:19:14.480608+0000","last_unstale":"2023-10-06T11:19:14.483232+0000","last_undegraded":"2023-10-06T11:19:14.483232+0000","last_fullsized":"2023-10-06T11:19:14.483232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:52.498302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.471197+0000","last_change":"2023-10-06T11:19:13.468283+0000","last_active":"2023-10-06T11:19:13.471197+0000","last_peered":"2023-10-06T11:19:13.471197+0000","last_clean":"2023-10-06T11:19:13.471197+0000","last_became_active":"2023-10-06T11:19:13.467919+0000","last_became_peered":"2023-10-06T11:19:13.467919+0000","last_unstale":"2023-10-06T11:19:13.471197+0000","last_undegraded":"2023-10-06T11:19:13.471197+0000","last_fullsized":"2023-10-06T11:19:13.471197+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:33.081892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481824+0000","last_change":"2023-10-06T11:19:14.481824+0000","last_active":"2023-10-06T11:19:14.481824+0000","last_peered":"2023-10-06T11:19:14.481824+0000","last_clean":"2023-10-06T11:19:14.481824+0000","last_became_active":"2023-10-06T11:19:14.481293+0000","last_became_peered":"2023-10-06T11:19:14.481293+0000","last_unstale":"2023-10-06T11:19:14.481824+0000","last_undegraded":"2023-10-06T11:19:14.481824+0000","last_fullsized":"2023-10-06T11:19:14.481824+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:16.928629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476654+0000","last_change":"2023-10-06T11:19:13.467272+0000","last_active":"2023-10-06T11:19:13.476654+0000","last_peered":"2023-10-06T11:19:13.476654+0000","last_clean":"2023-10-06T11:19:13.476654+0000","last_became_active":"2023-10-06T11:19:13.466979+0000","last_became_peered":"2023-10-06T11:19:13.466979+0000","last_unstale":"2023-10-06T11:19:13.476654+0000","last_undegraded":"2023-10-06T11:19:13.476654+0000","last_fullsized":"2023-10-06T11:19:13.476654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:40.742932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479351+0000","last_change":"2023-10-06T11:19:14.479351+0000","last_active":"2023-10-06T11:19:14.479351+0000","last_peered":"2023-10-06T11:19:14.479351+0000","last_clean":"2023-10-06T11:19:14.479351+0000","last_became_active":"2023-10-06T11:19:14.476894+0000","last_became_peered":"2023-10-06T11:19:14.476894+0000","last_unstale":"2023-10-06T11:19:14.479351+0000","last_undegraded":"2023-10-06T11:19:14.479351+0000","last_fullsized":"2023-10-06T11:19:14.479351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.489710+0000","last_change":"2023-10-06T11:19:13.469819+0000","last_active":"2023-10-06T11:19:14.489710+0000","last_peered":"2023-10-06T11:19:14.489710+0000","last_clean":"2023-10-06T11:19:14.489710+0000","last_became_active":"2023-10-06T11:19:13.469503+0000","last_became_peered":"2023-10-06T11:19:13.469503+0000","last_unstale":"2023-10-06T11:19:14.489710+0000","last_undegraded":"2023-10-06T11:19:14.489710+0000","last_fullsized":"2023-10-06T11:19:14.489710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:12.611360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482655+0000","last_change":"2023-10-06T11:19:14.482655+0000","last_active":"2023-10-06T11:19:14.482655+0000","last_peered":"2023-10-06T11:19:14.482655+0000","last_clean":"2023-10-06T11:19:14.482655+0000","last_became_active":"2023-10-06T11:19:14.482041+0000","last_became_peered":"2023-10-06T11:19:14.482041+0000","last_unstale":"2023-10-06T11:19:14.482655+0000","last_undegraded":"2023-10-06T11:19:14.482655+0000","last_fullsized":"2023-10-06T11:19:14.482655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:54.897036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499458+0000","last_change":"2023-10-06T11:19:13.472336+0000","last_active":"2023-10-06T11:19:14.499458+0000","last_peered":"2023-10-06T11:19:14.499458+0000","last_clean":"2023-10-06T11:19:14.499458+0000","last_became_active":"2023-10-06T11:19:13.471508+0000","last_became_peered":"2023-10-06T11:19:13.471508+0000","last_unstale":"2023-10-06T11:19:14.499458+0000","last_undegraded":"2023-10-06T11:19:14.499458+0000","last_fullsized":"2023-10-06T11:19:14.499458+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:39.770250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490282+0000","last_change":"2023-10-06T11:19:14.490282+0000","last_active":"2023-10-06T11:19:14.490282+0000","last_peered":"2023-10-06T11:19:14.490282+0000","last_clean":"2023-10-06T11:19:14.490282+0000","last_became_active":"2023-10-06T11:19:14.489983+0000","last_became_peered":"2023-10-06T11:19:14.489983+0000","last_unstale":"2023-10-06T11:19:14.490282+0000","last_undegraded":"2023-10-06T11:19:14.490282+0000","last_fullsized":"2023-10-06T11:19:14.490282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.498135+0000","last_change":"2023-10-06T11:19:13.472249+0000","last_active":"2023-10-06T11:19:14.498135+0000","last_peered":"2023-10-06T11:19:14.498135+0000","last_clean":"2023-10-06T11:19:14.498135+0000","last_became_active":"2023-10-06T11:19:13.471939+0000","last_became_peered":"2023-10-06T11:19:13.471939+0000","last_unstale":"2023-10-06T11:19:14.498135+0000","last_undegraded":"2023-10-06T11:19:14.498135+0000","last_fullsized":"2023-10-06T11:19:14.498135+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:40.742932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473109+0000","last_change":"2023-10-06T11:19:14.473109+0000","last_active":"2023-10-06T11:19:14.473109+0000","last_peered":"2023-10-06T11:19:14.473109+0000","last_clean":"2023-10-06T11:19:14.473109+0000","last_became_active":"2023-10-06T11:19:14.472795+0000","last_became_peered":"2023-10-06T11:19:14.472795+0000","last_unstale":"2023-10-06T11:19:14.473109+0000","last_undegraded":"2023-10-06T11:19:14.473109+0000","last_fullsized":"2023-10-06T11:19:14.473109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477950+0000","last_change":"2023-10-06T11:19:13.476564+0000","last_active":"2023-10-06T11:19:13.477950+0000","last_peered":"2023-10-06T11:19:13.477950+0000","last_clean":"2023-10-06T11:19:13.477950+0000","last_became_active":"2023-10-06T11:19:13.476213+0000","last_became_peered":"2023-10-06T11:19:13.476213+0000","last_unstale":"2023-10-06T11:19:13.477950+0000","last_undegraded":"2023-10-06T11:19:13.477950+0000","last_fullsized":"2023-10-06T11:19:13.477950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:33.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478506+0000","last_change":"2023-10-06T11:19:14.478506+0000","last_active":"2023-10-06T11:19:14.478506+0000","last_peered":"2023-10-06T11:19:14.478506+0000","last_clean":"2023-10-06T11:19:14.478506+0000","last_became_active":"2023-10-06T11:19:14.478070+0000","last_became_peered":"2023-10-06T11:19:14.478070+0000","last_unstale":"2023-10-06T11:19:14.478506+0000","last_undegraded":"2023-10-06T11:19:14.478506+0000","last_fullsized":"2023-10-06T11:19:14.478506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:49.825720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.485540+0000","last_change":"2023-10-06T11:19:13.477740+0000","last_active":"2023-10-06T11:19:13.485540+0000","last_peered":"2023-10-06T11:19:13.485540+0000","last_clean":"2023-10-06T11:19:13.485540+0000","last_became_active":"2023-10-06T11:19:13.477254+0000","last_became_peered":"2023-10-06T11:19:13.477254+0000","last_unstale":"2023-10-06T11:19:13.485540+0000","last_undegraded":"2023-10-06T11:19:13.485540+0000","last_fullsized":"2023-10-06T11:19:13.485540+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:43.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473907+0000","last_change":"2023-10-06T11:19:14.473907+0000","last_active":"2023-10-06T11:19:14.473907+0000","last_peered":"2023-10-06T11:19:14.473907+0000","last_clean":"2023-10-06T11:19:14.473907+0000","last_became_active":"2023-10-06T11:19:14.473576+0000","last_became_peered":"2023-10-06T11:19:14.473576+0000","last_unstale":"2023-10-06T11:19:14.473907+0000","last_undegraded":"2023-10-06T11:19:14.473907+0000","last_fullsized":"2023-10-06T11:19:14.473907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.501069+0000","last_change":"2023-10-06T11:19:13.478397+0000","last_active":"2023-10-06T11:19:14.501069+0000","last_peered":"2023-10-06T11:19:14.501069+0000","last_clean":"2023-10-06T11:19:14.501069+0000","last_became_active":"2023-10-06T11:19:13.478080+0000","last_became_peered":"2023-10-06T11:19:13.478080+0000","last_unstale":"2023-10-06T11:19:14.501069+0000","last_undegraded":"2023-10-06T11:19:14.501069+0000","last_fullsized":"2023-10-06T11:19:14.501069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:34.952026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477753+0000","last_change":"2023-10-06T11:19:14.477753+0000","last_active":"2023-10-06T11:19:14.477753+0000","last_peered":"2023-10-06T11:19:14.477753+0000","last_clean":"2023-10-06T11:19:14.477753+0000","last_became_active":"2023-10-06T11:19:14.475585+0000","last_became_peered":"2023-10-06T11:19:14.475585+0000","last_unstale":"2023-10-06T11:19:14.477753+0000","last_undegraded":"2023-10-06T11:19:14.477753+0000","last_fullsized":"2023-10-06T11:19:14.477753+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.471120+0000","last_change":"2023-10-06T11:19:13.470451+0000","last_active":"2023-10-06T11:19:14.471120+0000","last_peered":"2023-10-06T11:19:14.471120+0000","last_clean":"2023-10-06T11:19:14.471120+0000","last_became_active":"2023-10-06T11:19:13.469244+0000","last_became_peered":"2023-10-06T11:19:13.469244+0000","last_unstale":"2023-10-06T11:19:14.471120+0000","last_undegraded":"2023-10-06T11:19:14.471120+0000","last_fullsized":"2023-10-06T11:19:14.471120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:53.887022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484246+0000","last_change":"2023-10-06T11:19:14.484246+0000","last_active":"2023-10-06T11:19:14.484246+0000","last_peered":"2023-10-06T11:19:14.484246+0000","last_clean":"2023-10-06T11:19:14.484246+0000","last_became_active":"2023-10-06T11:19:14.483931+0000","last_became_peered":"2023-10-06T11:19:14.483931+0000","last_unstale":"2023-10-06T11:19:14.484246+0000","last_undegraded":"2023-10-06T11:19:14.484246+0000","last_fullsized":"2023-10-06T11:19:14.484246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.497436+0000","last_change":"2023-10-06T11:19:13.469735+0000","last_active":"2023-10-06T11:19:14.497436+0000","last_peered":"2023-10-06T11:19:14.497436+0000","last_clean":"2023-10-06T11:19:14.497436+0000","last_became_active":"2023-10-06T11:19:13.469443+0000","last_became_peered":"2023-10-06T11:19:13.469443+0000","last_unstale":"2023-10-06T11:19:14.497436+0000","last_undegraded":"2023-10-06T11:19:14.497436+0000","last_fullsized":"2023-10-06T11:19:14.497436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:35.939015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483573+0000","last_change":"2023-10-06T11:19:14.483573+0000","last_active":"2023-10-06T11:19:14.483573+0000","last_peered":"2023-10-06T11:19:14.483573+0000","last_clean":"2023-10-06T11:19:14.483573+0000","last_became_active":"2023-10-06T11:19:14.481611+0000","last_became_peered":"2023-10-06T11:19:14.481611+0000","last_unstale":"2023-10-06T11:19:14.483573+0000","last_undegraded":"2023-10-06T11:19:14.483573+0000","last_fullsized":"2023-10-06T11:19:14.483573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474595+0000","last_change":"2023-10-06T11:19:13.472260+0000","last_active":"2023-10-06T11:19:14.474595+0000","last_peered":"2023-10-06T11:19:14.474595+0000","last_clean":"2023-10-06T11:19:14.474595+0000","last_became_active":"2023-10-06T11:19:13.471887+0000","last_became_peered":"2023-10-06T11:19:13.471887+0000","last_unstale":"2023-10-06T11:19:14.474595+0000","last_undegraded":"2023-10-06T11:19:14.474595+0000","last_fullsized":"2023-10-06T11:19:14.474595+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:52.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477486+0000","last_change":"2023-10-06T11:19:14.477486+0000","last_active":"2023-10-06T11:19:14.477486+0000","last_peered":"2023-10-06T11:19:14.477486+0000","last_clean":"2023-10-06T11:19:14.477486+0000","last_became_active":"2023-10-06T11:19:14.477142+0000","last_became_peered":"2023-10-06T11:19:14.477142+0000","last_unstale":"2023-10-06T11:19:14.477486+0000","last_undegraded":"2023-10-06T11:19:14.477486+0000","last_fullsized":"2023-10-06T11:19:14.477486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:54.901822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476180+0000","last_change":"2023-10-06T11:19:13.470843+0000","last_active":"2023-10-06T11:19:13.476180+0000","last_peered":"2023-10-06T11:19:13.476180+0000","last_clean":"2023-10-06T11:19:13.476180+0000","last_became_active":"2023-10-06T11:19:13.470514+0000","last_became_peered":"2023-10-06T11:19:13.470514+0000","last_unstale":"2023-10-06T11:19:13.476180+0000","last_undegraded":"2023-10-06T11:19:13.476180+0000","last_fullsized":"2023-10-06T11:19:13.476180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:51.483503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478358+0000","last_change":"2023-10-06T11:19:14.478358+0000","last_active":"2023-10-06T11:19:14.478358+0000","last_peered":"2023-10-06T11:19:14.478358+0000","last_clean":"2023-10-06T11:19:14.478358+0000","last_became_active":"2023-10-06T11:19:14.477924+0000","last_became_peered":"2023-10-06T11:19:14.477924+0000","last_unstale":"2023-10-06T11:19:14.478358+0000","last_undegraded":"2023-10-06T11:19:14.478358+0000","last_fullsized":"2023-10-06T11:19:14.478358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494938+0000","last_change":"2023-10-06T11:19:13.473220+0000","last_active":"2023-10-06T11:19:14.494938+0000","last_peered":"2023-10-06T11:19:14.494938+0000","last_clean":"2023-10-06T11:19:14.494938+0000","last_became_active":"2023-10-06T11:19:13.472906+0000","last_became_peered":"2023-10-06T11:19:13.472906+0000","last_unstale":"2023-10-06T11:19:14.494938+0000","last_undegraded":"2023-10-06T11:19:14.494938+0000","last_fullsized":"2023-10-06T11:19:14.494938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:27.417289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484184+0000","last_change":"2023-10-06T11:19:14.484184+0000","last_active":"2023-10-06T11:19:14.484184+0000","last_peered":"2023-10-06T11:19:14.484184+0000","last_clean":"2023-10-06T11:19:14.484184+0000","last_became_active":"2023-10-06T11:19:14.477283+0000","last_became_peered":"2023-10-06T11:19:14.477283+0000","last_unstale":"2023-10-06T11:19:14.484184+0000","last_undegraded":"2023-10-06T11:19:14.484184+0000","last_fullsized":"2023-10-06T11:19:14.484184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.474386+0000","last_change":"2023-10-06T11:19:13.471276+0000","last_active":"2023-10-06T11:19:13.474386+0000","last_peered":"2023-10-06T11:19:13.474386+0000","last_clean":"2023-10-06T11:19:13.474386+0000","last_became_active":"2023-10-06T11:19:13.470848+0000","last_became_peered":"2023-10-06T11:19:13.470848+0000","last_unstale":"2023-10-06T11:19:13.474386+0000","last_undegraded":"2023-10-06T11:19:13.474386+0000","last_fullsized":"2023-10-06T11:19:13.474386+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:20.673213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487464+0000","last_change":"2023-10-06T11:19:14.487464+0000","last_active":"2023-10-06T11:19:14.487464+0000","last_peered":"2023-10-06T11:19:14.487464+0000","last_clean":"2023-10-06T11:19:14.487464+0000","last_became_active":"2023-10-06T11:19:14.487070+0000","last_became_peered":"2023-10-06T11:19:14.487070+0000","last_unstale":"2023-10-06T11:19:14.487464+0000","last_undegraded":"2023-10-06T11:19:14.487464+0000","last_fullsized":"2023-10-06T11:19:14.487464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.480157+0000","last_change":"2023-10-06T11:19:13.473242+0000","last_active":"2023-10-06T11:19:13.480157+0000","last_peered":"2023-10-06T11:19:13.480157+0000","last_clean":"2023-10-06T11:19:13.480157+0000","last_became_active":"2023-10-06T11:19:13.472924+0000","last_became_peered":"2023-10-06T11:19:13.472924+0000","last_unstale":"2023-10-06T11:19:13.480157+0000","last_undegraded":"2023-10-06T11:19:13.480157+0000","last_fullsized":"2023-10-06T11:19:13.480157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:51.483503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.468138+0000","last_change":"2023-10-06T11:19:14.468138+0000","last_active":"2023-10-06T11:19:14.468138+0000","last_peered":"2023-10-06T11:19:14.468138+0000","last_clean":"2023-10-06T11:19:14.468138+0000","last_became_active":"2023-10-06T11:19:14.467817+0000","last_became_peered":"2023-10-06T11:19:14.467817+0000","last_unstale":"2023-10-06T11:19:14.468138+0000","last_undegraded":"2023-10-06T11:19:14.468138+0000","last_fullsized":"2023-10-06T11:19:14.468138+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.474469+0000","last_change":"2023-10-06T11:19:13.471416+0000","last_active":"2023-10-06T11:19:13.474469+0000","last_peered":"2023-10-06T11:19:13.474469+0000","last_clean":"2023-10-06T11:19:13.474469+0000","last_became_active":"2023-10-06T11:19:13.471091+0000","last_became_peered":"2023-10-06T11:19:13.471091+0000","last_unstale":"2023-10-06T11:19:13.474469+0000","last_undegraded":"2023-10-06T11:19:13.474469+0000","last_fullsized":"2023-10-06T11:19:13.474469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:43.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.493407+0000","last_change":"2023-10-06T11:19:14.493407+0000","last_active":"2023-10-06T11:19:14.493407+0000","last_peered":"2023-10-06T11:19:14.493407+0000","last_clean":"2023-10-06T11:19:14.493407+0000","last_became_active":"2023-10-06T11:19:14.493126+0000","last_became_peered":"2023-10-06T11:19:14.493126+0000","last_unstale":"2023-10-06T11:19:14.493407+0000","last_undegraded":"2023-10-06T11:19:14.493407+0000","last_fullsized":"2023-10-06T11:19:14.493407+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.485290+0000","last_change":"2023-10-06T11:19:13.476441+0000","last_active":"2023-10-06T11:19:13.485290+0000","last_peered":"2023-10-06T11:19:13.485290+0000","last_clean":"2023-10-06T11:19:13.485290+0000","last_became_active":"2023-10-06T11:19:13.476001+0000","last_became_peered":"2023-10-06T11:19:13.476001+0000","last_unstale":"2023-10-06T11:19:13.485290+0000","last_undegraded":"2023-10-06T11:19:13.485290+0000","last_fullsized":"2023-10-06T11:19:13.485290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:04:06.705583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464878+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:19:13.464878+0000","last_peered":"2023-10-06T11:19:13.464878+0000","last_clean":"2023-10-06T11:19:13.464878+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:19:13.464878+0000","last_undegraded":"2023-10-06T11:19:13.464878+0000","last_fullsized":"2023-10-06T11:19:13.464878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.458364+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:19:13.458364+0000","last_peered":"2023-10-06T11:19:13.458364+0000","last_clean":"2023-10-06T11:19:13.458364+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:19:13.458364+0000","last_undegraded":"2023-10-06T11:19:13.458364+0000","last_fullsized":"2023-10-06T11:19:13.458364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494547+0000","last_change":"2023-10-06T11:19:14.494547+0000","last_active":"2023-10-06T11:19:14.494547+0000","last_peered":"2023-10-06T11:19:14.494547+0000","last_clean":"2023-10-06T11:19:14.494547+0000","last_became_active":"2023-10-06T11:19:14.494223+0000","last_became_peered":"2023-10-06T11:19:14.494223+0000","last_unstale":"2023-10-06T11:19:14.494547+0000","last_undegraded":"2023-10-06T11:19:14.494547+0000","last_fullsized":"2023-10-06T11:19:14.494547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:14.127084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.488439+0000","last_change":"2023-10-06T11:19:13.470668+0000","last_active":"2023-10-06T11:19:14.488439+0000","last_peered":"2023-10-06T11:19:14.488439+0000","last_clean":"2023-10-06T11:19:14.488439+0000","last_became_active":"2023-10-06T11:19:13.469191+0000","last_became_peered":"2023-10-06T11:19:13.469191+0000","last_unstale":"2023-10-06T11:19:14.488439+0000","last_undegraded":"2023-10-06T11:19:14.488439+0000","last_fullsized":"2023-10-06T11:19:14.488439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:33.081892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464756+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:19:13.464756+0000","last_peered":"2023-10-06T11:19:13.464756+0000","last_clean":"2023-10-06T11:19:13.464756+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:19:13.464756+0000","last_undegraded":"2023-10-06T11:19:13.464756+0000","last_fullsized":"2023-10-06T11:19:13.464756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481622+0000","last_change":"2023-10-06T11:19:14.481622+0000","last_active":"2023-10-06T11:19:14.481622+0000","last_peered":"2023-10-06T11:19:14.481622+0000","last_clean":"2023-10-06T11:19:14.481622+0000","last_became_active":"2023-10-06T11:19:14.480923+0000","last_became_peered":"2023-10-06T11:19:14.480923+0000","last_unstale":"2023-10-06T11:19:14.481622+0000","last_undegraded":"2023-10-06T11:19:14.481622+0000","last_fullsized":"2023-10-06T11:19:14.481622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:41.975314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.470221+0000","last_change":"2023-10-06T11:19:13.468516+0000","last_active":"2023-10-06T11:19:13.470221+0000","last_peered":"2023-10-06T11:19:13.470221+0000","last_clean":"2023-10-06T11:19:13.470221+0000","last_became_active":"2023-10-06T11:19:13.468083+0000","last_became_peered":"2023-10-06T11:19:13.468083+0000","last_unstale":"2023-10-06T11:19:13.470221+0000","last_undegraded":"2023-10-06T11:19:13.470221+0000","last_fullsized":"2023-10-06T11:19:13.470221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:06.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.460312+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:19:13.460312+0000","last_peered":"2023-10-06T11:19:13.460312+0000","last_clean":"2023-10-06T11:19:13.460312+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:19:13.460312+0000","last_undegraded":"2023-10-06T11:19:13.460312+0000","last_fullsized":"2023-10-06T11:19:13.460312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474063+0000","last_change":"2023-10-06T11:19:14.474063+0000","last_active":"2023-10-06T11:19:14.474063+0000","last_peered":"2023-10-06T11:19:14.474063+0000","last_clean":"2023-10-06T11:19:14.474063+0000","last_became_active":"2023-10-06T11:19:14.473731+0000","last_became_peered":"2023-10-06T11:19:14.473731+0000","last_unstale":"2023-10-06T11:19:14.474063+0000","last_undegraded":"2023-10-06T11:19:14.474063+0000","last_fullsized":"2023-10-06T11:19:14.474063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:36.804886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.472493+0000","last_change":"2023-10-06T11:19:13.466424+0000","last_active":"2023-10-06T11:19:13.472493+0000","last_peered":"2023-10-06T11:19:13.472493+0000","last_clean":"2023-10-06T11:19:13.472493+0000","last_became_active":"2023-10-06T11:19:13.465815+0000","last_became_peered":"2023-10-06T11:19:13.465815+0000","last_unstale":"2023-10-06T11:19:13.472493+0000","last_undegraded":"2023-10-06T11:19:13.472493+0000","last_fullsized":"2023-10-06T11:19:13.472493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:39.770250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.458001+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:19:13.458001+0000","last_peered":"2023-10-06T11:19:13.458001+0000","last_clean":"2023-10-06T11:19:13.458001+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:19:13.458001+0000","last_undegraded":"2023-10-06T11:19:13.458001+0000","last_fullsized":"2023-10-06T11:19:13.458001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.476065+0000","last_change":"2023-10-06T11:19:14.476065+0000","last_active":"2023-10-06T11:19:14.476065+0000","last_peered":"2023-10-06T11:19:14.476065+0000","last_clean":"2023-10-06T11:19:14.476065+0000","last_became_active":"2023-10-06T11:19:14.475782+0000","last_became_peered":"2023-10-06T11:19:14.475782+0000","last_unstale":"2023-10-06T11:19:14.476065+0000","last_undegraded":"2023-10-06T11:19:14.476065+0000","last_fullsized":"2023-10-06T11:19:14.476065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477044+0000","last_change":"2023-10-06T11:19:13.473457+0000","last_active":"2023-10-06T11:19:13.477044+0000","last_peered":"2023-10-06T11:19:13.477044+0000","last_clean":"2023-10-06T11:19:13.477044+0000","last_became_active":"2023-10-06T11:19:13.472598+0000","last_became_peered":"2023-10-06T11:19:13.472598+0000","last_unstale":"2023-10-06T11:19:13.477044+0000","last_undegraded":"2023-10-06T11:19:13.477044+0000","last_fullsized":"2023-10-06T11:19:13.477044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:46.594453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.457661+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:19:13.457661+0000","last_peered":"2023-10-06T11:19:13.457661+0000","last_clean":"2023-10-06T11:19:13.457661+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:19:13.457661+0000","last_undegraded":"2023-10-06T11:19:13.457661+0000","last_fullsized":"2023-10-06T11:19:13.457661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481589+0000","last_change":"2023-10-06T11:19:14.481589+0000","last_active":"2023-10-06T11:19:14.481589+0000","last_peered":"2023-10-06T11:19:14.481589+0000","last_clean":"2023-10-06T11:19:14.481589+0000","last_became_active":"2023-10-06T11:19:14.481291+0000","last_became_peered":"2023-10-06T11:19:14.481291+0000","last_unstale":"2023-10-06T11:19:14.481589+0000","last_undegraded":"2023-10-06T11:19:14.481589+0000","last_fullsized":"2023-10-06T11:19:14.481589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:32:33.919569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.482708+0000","last_change":"2023-10-06T11:19:13.475430+0000","last_active":"2023-10-06T11:19:13.482708+0000","last_peered":"2023-10-06T11:19:13.482708+0000","last_clean":"2023-10-06T11:19:13.482708+0000","last_became_active":"2023-10-06T11:19:13.473911+0000","last_became_peered":"2023-10-06T11:19:13.473911+0000","last_unstale":"2023-10-06T11:19:13.482708+0000","last_undegraded":"2023-10-06T11:19:13.482708+0000","last_fullsized":"2023-10-06T11:19:13.482708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:20.673213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464859+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:19:13.464859+0000","last_peered":"2023-10-06T11:19:13.464859+0000","last_clean":"2023-10-06T11:19:13.464859+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:19:13.464859+0000","last_undegraded":"2023-10-06T11:19:13.464859+0000","last_fullsized":"2023-10-06T11:19:13.464859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.489251+0000","last_change":"2023-10-06T11:19:14.489251+0000","last_active":"2023-10-06T11:19:14.489251+0000","last_peered":"2023-10-06T11:19:14.489251+0000","last_clean":"2023-10-06T11:19:14.489251+0000","last_became_active":"2023-10-06T11:19:14.488938+0000","last_became_peered":"2023-10-06T11:19:14.488938+0000","last_unstale":"2023-10-06T11:19:14.489251+0000","last_undegraded":"2023-10-06T11:19:14.489251+0000","last_fullsized":"2023-10-06T11:19:14.489251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476870+0000","last_change":"2023-10-06T11:19:13.469260+0000","last_active":"2023-10-06T11:19:13.476870+0000","last_peered":"2023-10-06T11:19:13.476870+0000","last_clean":"2023-10-06T11:19:13.476870+0000","last_became_active":"2023-10-06T11:19:13.468925+0000","last_became_peered":"2023-10-06T11:19:13.468925+0000","last_unstale":"2023-10-06T11:19:13.476870+0000","last_undegraded":"2023-10-06T11:19:13.476870+0000","last_fullsized":"2023-10-06T11:19:13.476870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:16.389400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.453775+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:19:13.453775+0000","last_peered":"2023-10-06T11:19:13.453775+0000","last_clean":"2023-10-06T11:19:13.453775+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:19:13.453775+0000","last_undegraded":"2023-10-06T11:19:13.453775+0000","last_fullsized":"2023-10-06T11:19:13.453775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.480272+0000","last_change":"2023-10-06T11:19:14.480272+0000","last_active":"2023-10-06T11:19:14.480272+0000","last_peered":"2023-10-06T11:19:14.480272+0000","last_clean":"2023-10-06T11:19:14.480272+0000","last_became_active":"2023-10-06T11:19:14.479902+0000","last_became_peered":"2023-10-06T11:19:14.479902+0000","last_unstale":"2023-10-06T11:19:14.480272+0000","last_undegraded":"2023-10-06T11:19:14.480272+0000","last_fullsized":"2023-10-06T11:19:14.480272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:17.404200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.467902+0000","last_change":"2023-10-06T11:19:13.466291+0000","last_active":"2023-10-06T11:19:13.467902+0000","last_peered":"2023-10-06T11:19:13.467902+0000","last_clean":"2023-10-06T11:19:13.467902+0000","last_became_active":"2023-10-06T11:19:13.465971+0000","last_became_peered":"2023-10-06T11:19:13.465971+0000","last_unstale":"2023-10-06T11:19:13.467902+0000","last_undegraded":"2023-10-06T11:19:13.467902+0000","last_fullsized":"2023-10-06T11:19:13.467902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:02.929291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.468315+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:19:13.468315+0000","last_peered":"2023-10-06T11:19:13.468315+0000","last_clean":"2023-10-06T11:19:13.468315+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:19:13.468315+0000","last_undegraded":"2023-10-06T11:19:13.468315+0000","last_fullsized":"2023-10-06T11:19:13.468315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":739441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.504},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.494}]},{"osd":1,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.262}]},{"osd":2,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.3069999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":1.155},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.649}]},{"osd":5,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.413},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":1.3799999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":2.0870000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4750000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.246}]},{"osd":3,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.605},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.532}]},{"osd":4,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.125},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.8140000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"max":{"1min":3.891,"5min":3.891,"15min":3.891},"last":1.5589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.429},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.1719999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.288},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.329}]},{"osd":2,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":3.609,"5min":3.609,"15min":3.609},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.405}]},{"osd":3,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.7250000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":3.4100000000000001,"5min":3.4100000000000001,"15min":3.4100000000000001},"last":1.512}]},{"osd":6,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.72,"5min":1.72,"15min":1.72},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":1.4890000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":3.5070000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.7869999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.101},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0009999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.395},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.264}]},{"osd":2,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.137,"5min":1.137,"15min":1.137},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.4390000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.02}]},{"osd":2,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.429},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.304}]},{"osd":5,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.407},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.627}]},{"osd":7,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.734},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.26}]},{"osd":1,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":1.081,"5min":1.081,"15min":1.081},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.335}]},{"osd":5,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.2},{"interface":"front","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.548},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.371}]},{"osd":7,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.5},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.4259999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301764,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070076,"statfs":{"total":96636764160,"available":96327757824,"internally_reserved":0,"allocated":389120,"data_stored":156279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302328,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069512,"statfs":{"total":96636764160,"available":96327180288,"internally_reserved":0,"allocated":966656,"data_stored":739441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.292},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.0209999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":1.054},{"interface":"front","average":{"1min":1.536,"5min":1.536,"15min":1.536},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.161}]},{"osd":7,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:19:18.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.962+0000 7fb050ce3700 1 Processor -- start 2023-10-06T11:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.962+0000 7fb050ce3700 1 -- start start 2023-10-06T11:19:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb050ce3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 0x7fb04c067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb050ce3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb04c067590 0x7fb04c069300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb050ce3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c0679f0 con 0x7fb04c1591a0 2023-10-06T11:19:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb050ce3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c0699c0 con 0x7fb04c067590 2023-10-06T11:19:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb050ce3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c15db50 con 0x7fb04c158dd0 2023-10-06T11:19:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb04c067590 0x7fb04c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb04c067590 0x7fb04c069300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53064/0 (socket says 172.21.15.46:53064) 2023-10-06T11:19:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 -- 172.21.15.46:0/569981758 learned_addr learned my addr 172.21.15.46:0/569981758 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb042ffd700 1 --1- 172.21.15.46:0/569981758 >> v1:172.21.15.46:6789/0 conn(0x7fb04c1591a0 0x7fb04c068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34144/0 (socket says 172.21.15.46:34144) 2023-10-06T11:19:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/569981758 >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 0x7fb04c067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 -- 172.21.15.46:0/569981758 >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 msgr2=0x7fb04c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 --2- 172.21.15.46:0/569981758 >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 0x7fb04c067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 -- 172.21.15.46:0/569981758 >> v1:172.21.15.46:6789/0 conn(0x7fb04c1591a0 legacy=0x7fb04c068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 -- 172.21.15.46:0/569981758 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c15d6c0 con 0x7fb04c067590 2023-10-06T11:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/569981758 >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 0x7fb04c067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04bfff700 1 --2- 172.21.15.46:0/569981758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c069300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb04c06ab40 tx=0x7fb044009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ab756a16a67bad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04affd700 1 -- 172.21.15.46:0/569981758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400e070 con 0x7fb04c067590 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04affd700 1 -- 172.21.15.46:0/569981758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400ba50 con 0x7fb04c067590 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.966+0000 7fb04affd700 1 -- 172.21.15.46:0/569981758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440092e0 con 0x7fb04c067590 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 -- 172.21.15.46:0/569981758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 msgr2=0x7fb04c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 --2- 172.21.15.46:0/569981758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c069300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb04c06ab40 tx=0x7fb044009840 comp rx=0 tx=0).stop 2023-10-06T11:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 -- 172.21.15.46:0/569981758 shutdown_connections 2023-10-06T11:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 --2- 172.21.15.46:0/569981758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c069300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 --2- 172.21.15.46:0/569981758 >> v2:172.21.15.132:3300/0 conn(0x7fb04c158dd0 0x7fb04c067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 -- 172.21.15.46:0/569981758 >> 172.21.15.46:0/569981758 conn(0x7fb04c062ef0 msgr2=0x7fb04c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 -- 172.21.15.46:0/569981758 shutdown_connections 2023-10-06T11:19:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 -- 172.21.15.46:0/569981758 wait complete. 2023-10-06T11:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.970+0000 7fb050ce3700 1 Processor -- start 2023-10-06T11:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- start start 2023-10-06T11:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 0x7fb04c0a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04c1591a0 0x7fb04c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c4cf0 con 0x7fb04c067590 2023-10-06T11:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c16a1f0 con 0x7fb04c158dd0 2023-10-06T11:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c38a0 con 0x7fb04c1591a0 2023-10-06T11:19:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53066/0 (socket says 172.21.15.46:53066) 2023-10-06T11:19:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 -- 172.21.15.46:0/2981540878 learned_addr learned my addr 172.21.15.46:0/2981540878 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04bfff700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04c1591a0 0x7fb04c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb042ffd700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 0x7fb04c0a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 -- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04c1591a0 msgr2=0x7fb04c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04c1591a0 0x7fb04c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 -- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 msgr2=0x7fb04c0a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 0x7fb04c0a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 -- 172.21.15.46:0/2981540878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009ae0 con 0x7fb04c067590 2023-10-06T11:19:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb042ffd700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 0x7fb04c0a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e900 tx=0x7fb03c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00eea0 con 0x7fb04c067590 2023-10-06T11:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00c5e0 con 0x7fb04c067590 2023-10-06T11:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00c980 con 0x7fb04c067590 2023-10-06T11:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0c40e0 con 0x7fb04c067590 2023-10-06T11:19:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.974+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0c46a0 con 0x7fb04c067590 2023-10-06T11:19:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.978+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c064590 con 0x7fb04c067590 2023-10-06T11:19:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.978+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb03c00cd30 con 0x7fb04c067590 2023-10-06T11:19:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.978+0000 7fb043fff700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 0x7fb02c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.978+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb03c07c690 con 0x7fb04c067590 2023-10-06T11:19:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.978+0000 7fb042ffd700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 0x7fb02c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.982+0000 7fb042ffd700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 0x7fb02c062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb034007fd0 tx=0x7fb034007e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:18.986+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb03c0505b0 con 0x7fb04c067590 2023-10-06T11:19:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.150+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb04c15cd90 con 0x7fb02c060580 2023-10-06T11:19:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.170+0000 7fb043fff700 1 -- 172.21.15.46:0/2981540878 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+306041 (secure 0 0 0) 0x7fb04c15cd90 con 0x7fb02c060580 2023-10-06T11:19:19.178 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 msgr2=0x7fb02c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 0x7fb02c062a30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb034007fd0 tx=0x7fb034007e10 comp rx=0 tx=0).stop 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 msgr2=0x7fb04c169cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e900 tx=0x7fb03c008040 comp rx=0 tx=0).stop 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 shutdown_connections 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb02c060580 0x7fb02c062a30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04c1591a0 0x7fb04c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb04c158dd0 0x7fb04c0a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 --2- 172.21.15.46:0/2981540878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c067590 0x7fb04c169cb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 >> 172.21.15.46:0/2981540878 conn(0x7fb04c062ef0 msgr2=0x7fb04c158050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 shutdown_connections 2023-10-06T11:19:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.178+0000 7fb050ce3700 1 -- 172.21.15.46:0/2981540878 wait complete. 2023-10-06T11:19:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:19:19.205 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T11:19:18.500216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398776,"kb_used_data":4088,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575944,"statfs":{"total":773094113280,"available":770637766656,"internally_reserved":0,"allocated":4186112,"data_stored":2380526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001607"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.471925+0000","last_change":"2023-10-06T11:19:14.471925+0000","last_active":"2023-10-06T11:19:14.471925+0000","last_peered":"2023-10-06T11:19:14.471925+0000","last_clean":"2023-10-06T11:19:14.471925+0000","last_became_active":"2023-10-06T11:19:14.471635+0000","last_became_peered":"2023-10-06T11:19:14.471635+0000","last_unstale":"2023-10-06T11:19:14.471925+0000","last_undegraded":"2023-10-06T11:19:14.471925+0000","last_fullsized":"2023-10-06T11:19:14.471925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485456+0000","last_change":"2023-10-06T11:19:14.485456+0000","last_active":"2023-10-06T11:19:14.485456+0000","last_peered":"2023-10-06T11:19:14.485456+0000","last_clean":"2023-10-06T11:19:14.485456+0000","last_became_active":"2023-10-06T11:19:14.482268+0000","last_became_peered":"2023-10-06T11:19:14.482268+0000","last_unstale":"2023-10-06T11:19:14.485456+0000","last_undegraded":"2023-10-06T11:19:14.485456+0000","last_fullsized":"2023-10-06T11:19:14.485456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490196+0000","last_change":"2023-10-06T11:19:14.490196+0000","last_active":"2023-10-06T11:19:14.490196+0000","last_peered":"2023-10-06T11:19:14.490196+0000","last_clean":"2023-10-06T11:19:14.490196+0000","last_became_active":"2023-10-06T11:19:14.489831+0000","last_became_peered":"2023-10-06T11:19:14.489831+0000","last_unstale":"2023-10-06T11:19:14.490196+0000","last_undegraded":"2023-10-06T11:19:14.490196+0000","last_fullsized":"2023-10-06T11:19:14.490196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490930+0000","last_change":"2023-10-06T11:19:14.490930+0000","last_active":"2023-10-06T11:19:14.490930+0000","last_peered":"2023-10-06T11:19:14.490930+0000","last_clean":"2023-10-06T11:19:14.490930+0000","last_became_active":"2023-10-06T11:19:14.489432+0000","last_became_peered":"2023-10-06T11:19:14.489432+0000","last_unstale":"2023-10-06T11:19:14.490930+0000","last_undegraded":"2023-10-06T11:19:14.490930+0000","last_fullsized":"2023-10-06T11:19:14.490930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:34.096691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477790+0000","last_change":"2023-10-06T11:19:14.477790+0000","last_active":"2023-10-06T11:19:14.477790+0000","last_peered":"2023-10-06T11:19:14.477790+0000","last_clean":"2023-10-06T11:19:14.477790+0000","last_became_active":"2023-10-06T11:19:14.477500+0000","last_became_peered":"2023-10-06T11:19:14.477500+0000","last_unstale":"2023-10-06T11:19:14.477790+0000","last_undegraded":"2023-10-06T11:19:14.477790+0000","last_fullsized":"2023-10-06T11:19:14.477790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:34.588402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487680+0000","last_change":"2023-10-06T11:19:14.487680+0000","last_active":"2023-10-06T11:19:14.487680+0000","last_peered":"2023-10-06T11:19:14.487680+0000","last_clean":"2023-10-06T11:19:14.487680+0000","last_became_active":"2023-10-06T11:19:14.487388+0000","last_became_peered":"2023-10-06T11:19:14.487388+0000","last_unstale":"2023-10-06T11:19:14.487680+0000","last_undegraded":"2023-10-06T11:19:14.487680+0000","last_fullsized":"2023-10-06T11:19:14.487680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.609253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492999+0000","last_change":"2023-10-06T11:19:14.492999+0000","last_active":"2023-10-06T11:19:14.492999+0000","last_peered":"2023-10-06T11:19:14.492999+0000","last_clean":"2023-10-06T11:19:14.492999+0000","last_became_active":"2023-10-06T11:19:14.492704+0000","last_became_peered":"2023-10-06T11:19:14.492704+0000","last_unstale":"2023-10-06T11:19:14.492999+0000","last_undegraded":"2023-10-06T11:19:14.492999+0000","last_fullsized":"2023-10-06T11:19:14.492999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481394+0000","last_change":"2023-10-06T11:19:14.481394+0000","last_active":"2023-10-06T11:19:14.481394+0000","last_peered":"2023-10-06T11:19:14.481394+0000","last_clean":"2023-10-06T11:19:14.481394+0000","last_became_active":"2023-10-06T11:19:14.481076+0000","last_became_peered":"2023-10-06T11:19:14.481076+0000","last_unstale":"2023-10-06T11:19:14.481394+0000","last_undegraded":"2023-10-06T11:19:14.481394+0000","last_fullsized":"2023-10-06T11:19:14.481394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482441+0000","last_change":"2023-10-06T11:19:14.482441+0000","last_active":"2023-10-06T11:19:14.482441+0000","last_peered":"2023-10-06T11:19:14.482441+0000","last_clean":"2023-10-06T11:19:14.482441+0000","last_became_active":"2023-10-06T11:19:14.481997+0000","last_became_peered":"2023-10-06T11:19:14.481997+0000","last_unstale":"2023-10-06T11:19:14.482441+0000","last_undegraded":"2023-10-06T11:19:14.482441+0000","last_fullsized":"2023-10-06T11:19:14.482441+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479690+0000","last_change":"2023-10-06T11:19:14.479690+0000","last_active":"2023-10-06T11:19:14.479690+0000","last_peered":"2023-10-06T11:19:14.479690+0000","last_clean":"2023-10-06T11:19:14.479690+0000","last_became_active":"2023-10-06T11:19:14.479355+0000","last_became_peered":"2023-10-06T11:19:14.479355+0000","last_unstale":"2023-10-06T11:19:14.479690+0000","last_undegraded":"2023-10-06T11:19:14.479690+0000","last_fullsized":"2023-10-06T11:19:14.479690+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:32.732866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479003+0000","last_change":"2023-10-06T11:19:14.479003+0000","last_active":"2023-10-06T11:19:14.479003+0000","last_peered":"2023-10-06T11:19:14.479003+0000","last_clean":"2023-10-06T11:19:14.479003+0000","last_became_active":"2023-10-06T11:19:14.478688+0000","last_became_peered":"2023-10-06T11:19:14.478688+0000","last_unstale":"2023-10-06T11:19:14.479003+0000","last_undegraded":"2023-10-06T11:19:14.479003+0000","last_fullsized":"2023-10-06T11:19:14.479003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479839+0000","last_change":"2023-10-06T11:19:14.479839+0000","last_active":"2023-10-06T11:19:14.479839+0000","last_peered":"2023-10-06T11:19:14.479839+0000","last_clean":"2023-10-06T11:19:14.479839+0000","last_became_active":"2023-10-06T11:19:14.479572+0000","last_became_peered":"2023-10-06T11:19:14.479572+0000","last_unstale":"2023-10-06T11:19:14.479839+0000","last_undegraded":"2023-10-06T11:19:14.479839+0000","last_fullsized":"2023-10-06T11:19:14.479839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.496930+0000","last_change":"2023-10-06T11:19:14.496930+0000","last_active":"2023-10-06T11:19:14.496930+0000","last_peered":"2023-10-06T11:19:14.496930+0000","last_clean":"2023-10-06T11:19:14.496930+0000","last_became_active":"2023-10-06T11:19:14.496578+0000","last_became_peered":"2023-10-06T11:19:14.496578+0000","last_unstale":"2023-10-06T11:19:14.496930+0000","last_undegraded":"2023-10-06T11:19:14.496930+0000","last_fullsized":"2023-10-06T11:19:14.496930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483089+0000","last_change":"2023-10-06T11:19:14.483089+0000","last_active":"2023-10-06T11:19:14.483089+0000","last_peered":"2023-10-06T11:19:14.483089+0000","last_clean":"2023-10-06T11:19:14.483089+0000","last_became_active":"2023-10-06T11:19:14.482798+0000","last_became_peered":"2023-10-06T11:19:14.482798+0000","last_unstale":"2023-10-06T11:19:14.483089+0000","last_undegraded":"2023-10-06T11:19:14.483089+0000","last_fullsized":"2023-10-06T11:19:14.483089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:53.890649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490604+0000","last_change":"2023-10-06T11:19:14.490604+0000","last_active":"2023-10-06T11:19:14.490604+0000","last_peered":"2023-10-06T11:19:14.490604+0000","last_clean":"2023-10-06T11:19:14.490604+0000","last_became_active":"2023-10-06T11:19:14.490331+0000","last_became_peered":"2023-10-06T11:19:14.490331+0000","last_unstale":"2023-10-06T11:19:14.490604+0000","last_undegraded":"2023-10-06T11:19:14.490604+0000","last_fullsized":"2023-10-06T11:19:14.490604+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492291+0000","last_change":"2023-10-06T11:19:14.492291+0000","last_active":"2023-10-06T11:19:14.492291+0000","last_peered":"2023-10-06T11:19:14.492291+0000","last_clean":"2023-10-06T11:19:14.492291+0000","last_became_active":"2023-10-06T11:19:14.492004+0000","last_became_peered":"2023-10-06T11:19:14.492004+0000","last_unstale":"2023-10-06T11:19:14.492291+0000","last_undegraded":"2023-10-06T11:19:14.492291+0000","last_fullsized":"2023-10-06T11:19:14.492291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485054+0000","last_change":"2023-10-06T11:19:14.485054+0000","last_active":"2023-10-06T11:19:14.485054+0000","last_peered":"2023-10-06T11:19:14.485054+0000","last_clean":"2023-10-06T11:19:14.485054+0000","last_became_active":"2023-10-06T11:19:14.484769+0000","last_became_peered":"2023-10-06T11:19:14.484769+0000","last_unstale":"2023-10-06T11:19:14.485054+0000","last_undegraded":"2023-10-06T11:19:14.485054+0000","last_fullsized":"2023-10-06T11:19:14.485054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:58.083133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474524+0000","last_change":"2023-10-06T11:19:14.474524+0000","last_active":"2023-10-06T11:19:14.474524+0000","last_peered":"2023-10-06T11:19:14.474524+0000","last_clean":"2023-10-06T11:19:14.474524+0000","last_became_active":"2023-10-06T11:19:14.474158+0000","last_became_peered":"2023-10-06T11:19:14.474158+0000","last_unstale":"2023-10-06T11:19:14.474524+0000","last_undegraded":"2023-10-06T11:19:14.474524+0000","last_fullsized":"2023-10-06T11:19:14.474524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:13.626160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487553+0000","last_change":"2023-10-06T11:19:14.487553+0000","last_active":"2023-10-06T11:19:14.487553+0000","last_peered":"2023-10-06T11:19:14.487553+0000","last_clean":"2023-10-06T11:19:14.487553+0000","last_became_active":"2023-10-06T11:19:14.483336+0000","last_became_peered":"2023-10-06T11:19:14.483336+0000","last_unstale":"2023-10-06T11:19:14.487553+0000","last_undegraded":"2023-10-06T11:19:14.487553+0000","last_fullsized":"2023-10-06T11:19:14.487553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:41.757732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474773+0000","last_change":"2023-10-06T11:19:14.474773+0000","last_active":"2023-10-06T11:19:14.474773+0000","last_peered":"2023-10-06T11:19:14.474773+0000","last_clean":"2023-10-06T11:19:14.474773+0000","last_became_active":"2023-10-06T11:19:14.474125+0000","last_became_peered":"2023-10-06T11:19:14.474125+0000","last_unstale":"2023-10-06T11:19:14.474773+0000","last_undegraded":"2023-10-06T11:19:14.474773+0000","last_fullsized":"2023-10-06T11:19:14.474773+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:34.588402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.488905+0000","last_change":"2023-10-06T11:19:14.488905+0000","last_active":"2023-10-06T11:19:14.488905+0000","last_peered":"2023-10-06T11:19:14.488905+0000","last_clean":"2023-10-06T11:19:14.488905+0000","last_became_active":"2023-10-06T11:19:14.488654+0000","last_became_peered":"2023-10-06T11:19:14.488654+0000","last_unstale":"2023-10-06T11:19:14.488905+0000","last_undegraded":"2023-10-06T11:19:14.488905+0000","last_fullsized":"2023-10-06T11:19:14.488905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481693+0000","last_change":"2023-10-06T11:19:14.481693+0000","last_active":"2023-10-06T11:19:14.481693+0000","last_peered":"2023-10-06T11:19:14.481693+0000","last_clean":"2023-10-06T11:19:14.481693+0000","last_became_active":"2023-10-06T11:19:14.481221+0000","last_became_peered":"2023-10-06T11:19:14.481221+0000","last_unstale":"2023-10-06T11:19:14.481693+0000","last_undegraded":"2023-10-06T11:19:14.481693+0000","last_fullsized":"2023-10-06T11:19:14.481693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:32.732866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485668+0000","last_change":"2023-10-06T11:19:14.485668+0000","last_active":"2023-10-06T11:19:14.485668+0000","last_peered":"2023-10-06T11:19:14.485668+0000","last_clean":"2023-10-06T11:19:14.485668+0000","last_became_active":"2023-10-06T11:19:14.483270+0000","last_became_peered":"2023-10-06T11:19:14.483270+0000","last_unstale":"2023-10-06T11:19:14.485668+0000","last_undegraded":"2023-10-06T11:19:14.485668+0000","last_fullsized":"2023-10-06T11:19:14.485668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:36.804886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482373+0000","last_change":"2023-10-06T11:19:14.482373+0000","last_active":"2023-10-06T11:19:14.482373+0000","last_peered":"2023-10-06T11:19:14.482373+0000","last_clean":"2023-10-06T11:19:14.482373+0000","last_became_active":"2023-10-06T11:19:14.481735+0000","last_became_peered":"2023-10-06T11:19:14.481735+0000","last_unstale":"2023-10-06T11:19:14.482373+0000","last_undegraded":"2023-10-06T11:19:14.482373+0000","last_fullsized":"2023-10-06T11:19:14.482373+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:13.626160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.475676+0000","last_change":"2023-10-06T11:19:14.475676+0000","last_active":"2023-10-06T11:19:14.475676+0000","last_peered":"2023-10-06T11:19:14.475676+0000","last_clean":"2023-10-06T11:19:14.475676+0000","last_became_active":"2023-10-06T11:19:14.475349+0000","last_became_peered":"2023-10-06T11:19:14.475349+0000","last_unstale":"2023-10-06T11:19:14.475676+0000","last_undegraded":"2023-10-06T11:19:14.475676+0000","last_fullsized":"2023-10-06T11:19:14.475676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:35.966826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494543+0000","last_change":"2023-10-06T11:19:14.494543+0000","last_active":"2023-10-06T11:19:14.494543+0000","last_peered":"2023-10-06T11:19:14.494543+0000","last_clean":"2023-10-06T11:19:14.494543+0000","last_became_active":"2023-10-06T11:19:14.494222+0000","last_became_peered":"2023-10-06T11:19:14.494222+0000","last_unstale":"2023-10-06T11:19:14.494543+0000","last_undegraded":"2023-10-06T11:19:14.494543+0000","last_fullsized":"2023-10-06T11:19:14.494543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:57.918616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481645+0000","last_change":"2023-10-06T11:19:14.481645+0000","last_active":"2023-10-06T11:19:14.481645+0000","last_peered":"2023-10-06T11:19:14.481645+0000","last_clean":"2023-10-06T11:19:14.481645+0000","last_became_active":"2023-10-06T11:19:14.481241+0000","last_became_peered":"2023-10-06T11:19:14.481241+0000","last_unstale":"2023-10-06T11:19:14.481645+0000","last_undegraded":"2023-10-06T11:19:14.481645+0000","last_fullsized":"2023-10-06T11:19:14.481645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:52.498302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492622+0000","last_change":"2023-10-06T11:19:14.492622+0000","last_active":"2023-10-06T11:19:14.492622+0000","last_peered":"2023-10-06T11:19:14.492622+0000","last_clean":"2023-10-06T11:19:14.492622+0000","last_became_active":"2023-10-06T11:19:14.492274+0000","last_became_peered":"2023-10-06T11:19:14.492274+0000","last_unstale":"2023-10-06T11:19:14.492622+0000","last_undegraded":"2023-10-06T11:19:14.492622+0000","last_fullsized":"2023-10-06T11:19:14.492622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:04:47.509154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482508+0000","last_change":"2023-10-06T11:19:14.482508+0000","last_active":"2023-10-06T11:19:14.482508+0000","last_peered":"2023-10-06T11:19:14.482508+0000","last_clean":"2023-10-06T11:19:14.482508+0000","last_became_active":"2023-10-06T11:19:14.482233+0000","last_became_peered":"2023-10-06T11:19:14.482233+0000","last_unstale":"2023-10-06T11:19:14.482508+0000","last_undegraded":"2023-10-06T11:19:14.482508+0000","last_fullsized":"2023-10-06T11:19:14.482508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:04:07.720382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.480621+0000","last_change":"2023-10-06T11:19:14.480621+0000","last_active":"2023-10-06T11:19:14.480621+0000","last_peered":"2023-10-06T11:19:14.480621+0000","last_clean":"2023-10-06T11:19:14.480621+0000","last_became_active":"2023-10-06T11:19:14.480305+0000","last_became_peered":"2023-10-06T11:19:14.480305+0000","last_unstale":"2023-10-06T11:19:14.480621+0000","last_undegraded":"2023-10-06T11:19:14.480621+0000","last_fullsized":"2023-10-06T11:19:14.480621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:56.713996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482707+0000","last_change":"2023-10-06T11:19:14.482707+0000","last_active":"2023-10-06T11:19:14.482707+0000","last_peered":"2023-10-06T11:19:14.482707+0000","last_clean":"2023-10-06T11:19:14.482707+0000","last_became_active":"2023-10-06T11:19:14.482414+0000","last_became_peered":"2023-10-06T11:19:14.482414+0000","last_unstale":"2023-10-06T11:19:14.482707+0000","last_undegraded":"2023-10-06T11:19:14.482707+0000","last_fullsized":"2023-10-06T11:19:14.482707+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:36.953814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482141+0000","last_change":"2023-10-06T11:19:14.482141+0000","last_active":"2023-10-06T11:19:14.482141+0000","last_peered":"2023-10-06T11:19:14.482141+0000","last_clean":"2023-10-06T11:19:14.482141+0000","last_became_active":"2023-10-06T11:19:14.479449+0000","last_became_peered":"2023-10-06T11:19:14.479449+0000","last_unstale":"2023-10-06T11:19:14.482141+0000","last_undegraded":"2023-10-06T11:19:14.482141+0000","last_fullsized":"2023-10-06T11:19:14.482141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:40.785050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.491815+0000","last_change":"2023-10-06T11:19:14.491815+0000","last_active":"2023-10-06T11:19:14.491815+0000","last_peered":"2023-10-06T11:19:14.491815+0000","last_clean":"2023-10-06T11:19:14.491815+0000","last_became_active":"2023-10-06T11:19:14.491445+0000","last_became_peered":"2023-10-06T11:19:14.491445+0000","last_unstale":"2023-10-06T11:19:14.491815+0000","last_undegraded":"2023-10-06T11:19:14.491815+0000","last_fullsized":"2023-10-06T11:19:14.491815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:07.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499136+0000","last_change":"2023-10-06T11:19:13.475202+0000","last_active":"2023-10-06T11:19:14.499136+0000","last_peered":"2023-10-06T11:19:14.499136+0000","last_clean":"2023-10-06T11:19:14.499136+0000","last_became_active":"2023-10-06T11:19:13.474873+0000","last_became_peered":"2023-10-06T11:19:13.474873+0000","last_unstale":"2023-10-06T11:19:14.499136+0000","last_undegraded":"2023-10-06T11:19:14.499136+0000","last_fullsized":"2023-10-06T11:19:14.499136+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:06.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481269+0000","last_change":"2023-10-06T11:19:14.481269+0000","last_active":"2023-10-06T11:19:14.481269+0000","last_peered":"2023-10-06T11:19:14.481269+0000","last_clean":"2023-10-06T11:19:14.481269+0000","last_became_active":"2023-10-06T11:19:14.479998+0000","last_became_peered":"2023-10-06T11:19:14.479998+0000","last_unstale":"2023-10-06T11:19:14.481269+0000","last_undegraded":"2023-10-06T11:19:14.481269+0000","last_fullsized":"2023-10-06T11:19:14.481269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:41.975314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.464281+0000","last_change":"2023-10-06T11:19:13.473063+0000","last_active":"2023-10-06T11:19:14.464281+0000","last_peered":"2023-10-06T11:19:14.464281+0000","last_clean":"2023-10-06T11:19:14.464281+0000","last_became_active":"2023-10-06T11:19:13.472394+0000","last_became_peered":"2023-10-06T11:19:13.472394+0000","last_unstale":"2023-10-06T11:19:14.464281+0000","last_undegraded":"2023-10-06T11:19:14.464281+0000","last_fullsized":"2023-10-06T11:19:14.464281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:57:56.903816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.476811+0000","last_change":"2023-10-06T11:19:14.476811+0000","last_active":"2023-10-06T11:19:14.476811+0000","last_peered":"2023-10-06T11:19:14.476811+0000","last_clean":"2023-10-06T11:19:14.476811+0000","last_became_active":"2023-10-06T11:19:14.476485+0000","last_became_peered":"2023-10-06T11:19:14.476485+0000","last_unstale":"2023-10-06T11:19:14.476811+0000","last_undegraded":"2023-10-06T11:19:14.476811+0000","last_fullsized":"2023-10-06T11:19:14.476811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:01.227294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499928+0000","last_change":"2023-10-06T11:19:13.470307+0000","last_active":"2023-10-06T11:19:14.499928+0000","last_peered":"2023-10-06T11:19:14.499928+0000","last_clean":"2023-10-06T11:19:14.499928+0000","last_became_active":"2023-10-06T11:19:13.470002+0000","last_became_peered":"2023-10-06T11:19:13.470002+0000","last_unstale":"2023-10-06T11:19:14.499928+0000","last_undegraded":"2023-10-06T11:19:14.499928+0000","last_fullsized":"2023-10-06T11:19:14.499928+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:34.952026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478182+0000","last_change":"2023-10-06T11:19:14.478182+0000","last_active":"2023-10-06T11:19:14.478182+0000","last_peered":"2023-10-06T11:19:14.478182+0000","last_clean":"2023-10-06T11:19:14.478182+0000","last_became_active":"2023-10-06T11:19:14.475164+0000","last_became_peered":"2023-10-06T11:19:14.475164+0000","last_unstale":"2023-10-06T11:19:14.478182+0000","last_undegraded":"2023-10-06T11:19:14.478182+0000","last_fullsized":"2023-10-06T11:19:14.478182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:11:13.084621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.492335+0000","last_change":"2023-10-06T11:19:13.467957+0000","last_active":"2023-10-06T11:19:14.492335+0000","last_peered":"2023-10-06T11:19:14.492335+0000","last_clean":"2023-10-06T11:19:14.492335+0000","last_became_active":"2023-10-06T11:19:13.467664+0000","last_became_peered":"2023-10-06T11:19:13.467664+0000","last_unstale":"2023-10-06T11:19:14.492335+0000","last_undegraded":"2023-10-06T11:19:14.492335+0000","last_fullsized":"2023-10-06T11:19:14.492335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:48.810920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.485521+0000","last_change":"2023-10-06T11:19:14.485521+0000","last_active":"2023-10-06T11:19:14.485521+0000","last_peered":"2023-10-06T11:19:14.485521+0000","last_clean":"2023-10-06T11:19:14.485521+0000","last_became_active":"2023-10-06T11:19:14.485225+0000","last_became_peered":"2023-10-06T11:19:14.485225+0000","last_unstale":"2023-10-06T11:19:14.485521+0000","last_undegraded":"2023-10-06T11:19:14.485521+0000","last_fullsized":"2023-10-06T11:19:14.485521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:07.835866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473703+0000","last_change":"2023-10-06T11:19:13.475244+0000","last_active":"2023-10-06T11:19:14.473703+0000","last_peered":"2023-10-06T11:19:14.473703+0000","last_clean":"2023-10-06T11:19:14.473703+0000","last_became_active":"2023-10-06T11:19:13.474883+0000","last_became_peered":"2023-10-06T11:19:13.474883+0000","last_unstale":"2023-10-06T11:19:14.473703+0000","last_undegraded":"2023-10-06T11:19:14.473703+0000","last_fullsized":"2023-10-06T11:19:14.473703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:02.929291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484013+0000","last_change":"2023-10-06T11:19:14.484013+0000","last_active":"2023-10-06T11:19:14.484013+0000","last_peered":"2023-10-06T11:19:14.484013+0000","last_clean":"2023-10-06T11:19:14.484013+0000","last_became_active":"2023-10-06T11:19:14.483705+0000","last_became_peered":"2023-10-06T11:19:14.483705+0000","last_unstale":"2023-10-06T11:19:14.484013+0000","last_undegraded":"2023-10-06T11:19:14.484013+0000","last_fullsized":"2023-10-06T11:19:14.484013+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:54.897036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.463079+0000","last_change":"2023-10-06T11:19:13.479925+0000","last_active":"2023-10-06T11:19:14.463079+0000","last_peered":"2023-10-06T11:19:14.463079+0000","last_clean":"2023-10-06T11:19:14.463079+0000","last_became_active":"2023-10-06T11:19:13.479575+0000","last_became_peered":"2023-10-06T11:19:13.479575+0000","last_unstale":"2023-10-06T11:19:14.463079+0000","last_undegraded":"2023-10-06T11:19:14.463079+0000","last_fullsized":"2023-10-06T11:19:14.463079+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:55.699197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.493849+0000","last_change":"2023-10-06T11:19:14.493849+0000","last_active":"2023-10-06T11:19:14.493849+0000","last_peered":"2023-10-06T11:19:14.493849+0000","last_clean":"2023-10-06T11:19:14.493849+0000","last_became_active":"2023-10-06T11:19:14.491624+0000","last_became_peered":"2023-10-06T11:19:14.491624+0000","last_unstale":"2023-10-06T11:19:14.493849+0000","last_undegraded":"2023-10-06T11:19:14.493849+0000","last_fullsized":"2023-10-06T11:19:14.493849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477975+0000","last_change":"2023-10-06T11:19:13.476765+0000","last_active":"2023-10-06T11:19:13.477975+0000","last_peered":"2023-10-06T11:19:13.477975+0000","last_clean":"2023-10-06T11:19:13.477975+0000","last_became_active":"2023-10-06T11:19:13.476280+0000","last_became_peered":"2023-10-06T11:19:13.476280+0000","last_unstale":"2023-10-06T11:19:13.477975+0000","last_undegraded":"2023-10-06T11:19:13.477975+0000","last_fullsized":"2023-10-06T11:19:13.477975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:29:55.699197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.469515+0000","last_change":"2023-10-06T11:19:14.469515+0000","last_active":"2023-10-06T11:19:14.469515+0000","last_peered":"2023-10-06T11:19:14.469515+0000","last_clean":"2023-10-06T11:19:14.469515+0000","last_became_active":"2023-10-06T11:19:14.469196+0000","last_became_peered":"2023-10-06T11:19:14.469196+0000","last_unstale":"2023-10-06T11:19:14.469515+0000","last_undegraded":"2023-10-06T11:19:14.469515+0000","last_fullsized":"2023-10-06T11:19:14.469515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.479312+0000","last_change":"2023-10-06T11:19:13.473986+0000","last_active":"2023-10-06T11:19:13.479312+0000","last_peered":"2023-10-06T11:19:13.479312+0000","last_clean":"2023-10-06T11:19:13.479312+0000","last_became_active":"2023-10-06T11:19:13.473583+0000","last_became_peered":"2023-10-06T11:19:13.473583+0000","last_unstale":"2023-10-06T11:19:13.479312+0000","last_undegraded":"2023-10-06T11:19:13.479312+0000","last_fullsized":"2023-10-06T11:19:13.479312+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:35.790086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483232+0000","last_change":"2023-10-06T11:19:14.483232+0000","last_active":"2023-10-06T11:19:14.483232+0000","last_peered":"2023-10-06T11:19:14.483232+0000","last_clean":"2023-10-06T11:19:14.483232+0000","last_became_active":"2023-10-06T11:19:14.480608+0000","last_became_peered":"2023-10-06T11:19:14.480608+0000","last_unstale":"2023-10-06T11:19:14.483232+0000","last_undegraded":"2023-10-06T11:19:14.483232+0000","last_fullsized":"2023-10-06T11:19:14.483232+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:52.498302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.471197+0000","last_change":"2023-10-06T11:19:13.468283+0000","last_active":"2023-10-06T11:19:13.471197+0000","last_peered":"2023-10-06T11:19:13.471197+0000","last_clean":"2023-10-06T11:19:13.471197+0000","last_became_active":"2023-10-06T11:19:13.467919+0000","last_became_peered":"2023-10-06T11:19:13.467919+0000","last_unstale":"2023-10-06T11:19:13.471197+0000","last_undegraded":"2023-10-06T11:19:13.471197+0000","last_fullsized":"2023-10-06T11:19:13.471197+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:33.081892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481824+0000","last_change":"2023-10-06T11:19:14.481824+0000","last_active":"2023-10-06T11:19:14.481824+0000","last_peered":"2023-10-06T11:19:14.481824+0000","last_clean":"2023-10-06T11:19:14.481824+0000","last_became_active":"2023-10-06T11:19:14.481293+0000","last_became_peered":"2023-10-06T11:19:14.481293+0000","last_unstale":"2023-10-06T11:19:14.481824+0000","last_undegraded":"2023-10-06T11:19:14.481824+0000","last_fullsized":"2023-10-06T11:19:14.481824+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:16.928629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476654+0000","last_change":"2023-10-06T11:19:13.467272+0000","last_active":"2023-10-06T11:19:13.476654+0000","last_peered":"2023-10-06T11:19:13.476654+0000","last_clean":"2023-10-06T11:19:13.476654+0000","last_became_active":"2023-10-06T11:19:13.466979+0000","last_became_peered":"2023-10-06T11:19:13.466979+0000","last_unstale":"2023-10-06T11:19:13.476654+0000","last_undegraded":"2023-10-06T11:19:13.476654+0000","last_fullsized":"2023-10-06T11:19:13.476654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:40.742932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.479351+0000","last_change":"2023-10-06T11:19:14.479351+0000","last_active":"2023-10-06T11:19:14.479351+0000","last_peered":"2023-10-06T11:19:14.479351+0000","last_clean":"2023-10-06T11:19:14.479351+0000","last_became_active":"2023-10-06T11:19:14.476894+0000","last_became_peered":"2023-10-06T11:19:14.476894+0000","last_unstale":"2023-10-06T11:19:14.479351+0000","last_undegraded":"2023-10-06T11:19:14.479351+0000","last_fullsized":"2023-10-06T11:19:14.479351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.489710+0000","last_change":"2023-10-06T11:19:13.469819+0000","last_active":"2023-10-06T11:19:14.489710+0000","last_peered":"2023-10-06T11:19:14.489710+0000","last_clean":"2023-10-06T11:19:14.489710+0000","last_became_active":"2023-10-06T11:19:13.469503+0000","last_became_peered":"2023-10-06T11:19:13.469503+0000","last_unstale":"2023-10-06T11:19:14.489710+0000","last_undegraded":"2023-10-06T11:19:14.489710+0000","last_fullsized":"2023-10-06T11:19:14.489710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:12.611360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.482655+0000","last_change":"2023-10-06T11:19:14.482655+0000","last_active":"2023-10-06T11:19:14.482655+0000","last_peered":"2023-10-06T11:19:14.482655+0000","last_clean":"2023-10-06T11:19:14.482655+0000","last_became_active":"2023-10-06T11:19:14.482041+0000","last_became_peered":"2023-10-06T11:19:14.482041+0000","last_unstale":"2023-10-06T11:19:14.482655+0000","last_undegraded":"2023-10-06T11:19:14.482655+0000","last_fullsized":"2023-10-06T11:19:14.482655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:54.897036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.499458+0000","last_change":"2023-10-06T11:19:13.472336+0000","last_active":"2023-10-06T11:19:14.499458+0000","last_peered":"2023-10-06T11:19:14.499458+0000","last_clean":"2023-10-06T11:19:14.499458+0000","last_became_active":"2023-10-06T11:19:13.471508+0000","last_became_peered":"2023-10-06T11:19:13.471508+0000","last_unstale":"2023-10-06T11:19:14.499458+0000","last_undegraded":"2023-10-06T11:19:14.499458+0000","last_fullsized":"2023-10-06T11:19:14.499458+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:39.770250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.490282+0000","last_change":"2023-10-06T11:19:14.490282+0000","last_active":"2023-10-06T11:19:14.490282+0000","last_peered":"2023-10-06T11:19:14.490282+0000","last_clean":"2023-10-06T11:19:14.490282+0000","last_became_active":"2023-10-06T11:19:14.489983+0000","last_became_peered":"2023-10-06T11:19:14.489983+0000","last_unstale":"2023-10-06T11:19:14.490282+0000","last_undegraded":"2023-10-06T11:19:14.490282+0000","last_fullsized":"2023-10-06T11:19:14.490282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.498135+0000","last_change":"2023-10-06T11:19:13.472249+0000","last_active":"2023-10-06T11:19:14.498135+0000","last_peered":"2023-10-06T11:19:14.498135+0000","last_clean":"2023-10-06T11:19:14.498135+0000","last_became_active":"2023-10-06T11:19:13.471939+0000","last_became_peered":"2023-10-06T11:19:13.471939+0000","last_unstale":"2023-10-06T11:19:14.498135+0000","last_undegraded":"2023-10-06T11:19:14.498135+0000","last_fullsized":"2023-10-06T11:19:14.498135+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:40.742932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473109+0000","last_change":"2023-10-06T11:19:14.473109+0000","last_active":"2023-10-06T11:19:14.473109+0000","last_peered":"2023-10-06T11:19:14.473109+0000","last_clean":"2023-10-06T11:19:14.473109+0000","last_became_active":"2023-10-06T11:19:14.472795+0000","last_became_peered":"2023-10-06T11:19:14.472795+0000","last_unstale":"2023-10-06T11:19:14.473109+0000","last_undegraded":"2023-10-06T11:19:14.473109+0000","last_fullsized":"2023-10-06T11:19:14.473109+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:52.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477950+0000","last_change":"2023-10-06T11:19:13.476564+0000","last_active":"2023-10-06T11:19:13.477950+0000","last_peered":"2023-10-06T11:19:13.477950+0000","last_clean":"2023-10-06T11:19:13.477950+0000","last_became_active":"2023-10-06T11:19:13.476213+0000","last_became_peered":"2023-10-06T11:19:13.476213+0000","last_unstale":"2023-10-06T11:19:13.477950+0000","last_undegraded":"2023-10-06T11:19:13.477950+0000","last_fullsized":"2023-10-06T11:19:13.477950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:33.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478506+0000","last_change":"2023-10-06T11:19:14.478506+0000","last_active":"2023-10-06T11:19:14.478506+0000","last_peered":"2023-10-06T11:19:14.478506+0000","last_clean":"2023-10-06T11:19:14.478506+0000","last_became_active":"2023-10-06T11:19:14.478070+0000","last_became_peered":"2023-10-06T11:19:14.478070+0000","last_unstale":"2023-10-06T11:19:14.478506+0000","last_undegraded":"2023-10-06T11:19:14.478506+0000","last_fullsized":"2023-10-06T11:19:14.478506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:49.825720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.485540+0000","last_change":"2023-10-06T11:19:13.477740+0000","last_active":"2023-10-06T11:19:13.485540+0000","last_peered":"2023-10-06T11:19:13.485540+0000","last_clean":"2023-10-06T11:19:13.485540+0000","last_became_active":"2023-10-06T11:19:13.477254+0000","last_became_peered":"2023-10-06T11:19:13.477254+0000","last_unstale":"2023-10-06T11:19:13.485540+0000","last_undegraded":"2023-10-06T11:19:13.485540+0000","last_fullsized":"2023-10-06T11:19:13.485540+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:43.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.473907+0000","last_change":"2023-10-06T11:19:14.473907+0000","last_active":"2023-10-06T11:19:14.473907+0000","last_peered":"2023-10-06T11:19:14.473907+0000","last_clean":"2023-10-06T11:19:14.473907+0000","last_became_active":"2023-10-06T11:19:14.473576+0000","last_became_peered":"2023-10-06T11:19:14.473576+0000","last_unstale":"2023-10-06T11:19:14.473907+0000","last_undegraded":"2023-10-06T11:19:14.473907+0000","last_fullsized":"2023-10-06T11:19:14.473907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.501069+0000","last_change":"2023-10-06T11:19:13.478397+0000","last_active":"2023-10-06T11:19:14.501069+0000","last_peered":"2023-10-06T11:19:14.501069+0000","last_clean":"2023-10-06T11:19:14.501069+0000","last_became_active":"2023-10-06T11:19:13.478080+0000","last_became_peered":"2023-10-06T11:19:13.478080+0000","last_unstale":"2023-10-06T11:19:14.501069+0000","last_undegraded":"2023-10-06T11:19:14.501069+0000","last_fullsized":"2023-10-06T11:19:14.501069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:34.952026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477753+0000","last_change":"2023-10-06T11:19:14.477753+0000","last_active":"2023-10-06T11:19:14.477753+0000","last_peered":"2023-10-06T11:19:14.477753+0000","last_clean":"2023-10-06T11:19:14.477753+0000","last_became_active":"2023-10-06T11:19:14.475585+0000","last_became_peered":"2023-10-06T11:19:14.475585+0000","last_unstale":"2023-10-06T11:19:14.477753+0000","last_undegraded":"2023-10-06T11:19:14.477753+0000","last_fullsized":"2023-10-06T11:19:14.477753+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.471120+0000","last_change":"2023-10-06T11:19:13.470451+0000","last_active":"2023-10-06T11:19:14.471120+0000","last_peered":"2023-10-06T11:19:14.471120+0000","last_clean":"2023-10-06T11:19:14.471120+0000","last_became_active":"2023-10-06T11:19:13.469244+0000","last_became_peered":"2023-10-06T11:19:13.469244+0000","last_unstale":"2023-10-06T11:19:14.471120+0000","last_undegraded":"2023-10-06T11:19:14.471120+0000","last_fullsized":"2023-10-06T11:19:14.471120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:53.887022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484246+0000","last_change":"2023-10-06T11:19:14.484246+0000","last_active":"2023-10-06T11:19:14.484246+0000","last_peered":"2023-10-06T11:19:14.484246+0000","last_clean":"2023-10-06T11:19:14.484246+0000","last_became_active":"2023-10-06T11:19:14.483931+0000","last_became_peered":"2023-10-06T11:19:14.483931+0000","last_unstale":"2023-10-06T11:19:14.484246+0000","last_undegraded":"2023-10-06T11:19:14.484246+0000","last_fullsized":"2023-10-06T11:19:14.484246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:15.829353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.497436+0000","last_change":"2023-10-06T11:19:13.469735+0000","last_active":"2023-10-06T11:19:14.497436+0000","last_peered":"2023-10-06T11:19:14.497436+0000","last_clean":"2023-10-06T11:19:14.497436+0000","last_became_active":"2023-10-06T11:19:13.469443+0000","last_became_peered":"2023-10-06T11:19:13.469443+0000","last_unstale":"2023-10-06T11:19:14.497436+0000","last_undegraded":"2023-10-06T11:19:14.497436+0000","last_fullsized":"2023-10-06T11:19:14.497436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:35.939015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.483573+0000","last_change":"2023-10-06T11:19:14.483573+0000","last_active":"2023-10-06T11:19:14.483573+0000","last_peered":"2023-10-06T11:19:14.483573+0000","last_clean":"2023-10-06T11:19:14.483573+0000","last_became_active":"2023-10-06T11:19:14.481611+0000","last_became_peered":"2023-10-06T11:19:14.481611+0000","last_unstale":"2023-10-06T11:19:14.483573+0000","last_undegraded":"2023-10-06T11:19:14.483573+0000","last_fullsized":"2023-10-06T11:19:14.483573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474595+0000","last_change":"2023-10-06T11:19:13.472260+0000","last_active":"2023-10-06T11:19:14.474595+0000","last_peered":"2023-10-06T11:19:14.474595+0000","last_clean":"2023-10-06T11:19:14.474595+0000","last_became_active":"2023-10-06T11:19:13.471887+0000","last_became_peered":"2023-10-06T11:19:13.471887+0000","last_unstale":"2023-10-06T11:19:14.474595+0000","last_undegraded":"2023-10-06T11:19:14.474595+0000","last_fullsized":"2023-10-06T11:19:14.474595+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:52.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.477486+0000","last_change":"2023-10-06T11:19:14.477486+0000","last_active":"2023-10-06T11:19:14.477486+0000","last_peered":"2023-10-06T11:19:14.477486+0000","last_clean":"2023-10-06T11:19:14.477486+0000","last_became_active":"2023-10-06T11:19:14.477142+0000","last_became_peered":"2023-10-06T11:19:14.477142+0000","last_unstale":"2023-10-06T11:19:14.477486+0000","last_undegraded":"2023-10-06T11:19:14.477486+0000","last_fullsized":"2023-10-06T11:19:14.477486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:23:54.901822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476180+0000","last_change":"2023-10-06T11:19:13.470843+0000","last_active":"2023-10-06T11:19:13.476180+0000","last_peered":"2023-10-06T11:19:13.476180+0000","last_clean":"2023-10-06T11:19:13.476180+0000","last_became_active":"2023-10-06T11:19:13.470514+0000","last_became_peered":"2023-10-06T11:19:13.470514+0000","last_unstale":"2023-10-06T11:19:13.476180+0000","last_undegraded":"2023-10-06T11:19:13.476180+0000","last_fullsized":"2023-10-06T11:19:13.476180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:51.483503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.478358+0000","last_change":"2023-10-06T11:19:14.478358+0000","last_active":"2023-10-06T11:19:14.478358+0000","last_peered":"2023-10-06T11:19:14.478358+0000","last_clean":"2023-10-06T11:19:14.478358+0000","last_became_active":"2023-10-06T11:19:14.477924+0000","last_became_peered":"2023-10-06T11:19:14.477924+0000","last_unstale":"2023-10-06T11:19:14.478358+0000","last_undegraded":"2023-10-06T11:19:14.478358+0000","last_fullsized":"2023-10-06T11:19:14.478358+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:18:31.386658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494938+0000","last_change":"2023-10-06T11:19:13.473220+0000","last_active":"2023-10-06T11:19:14.494938+0000","last_peered":"2023-10-06T11:19:14.494938+0000","last_clean":"2023-10-06T11:19:14.494938+0000","last_became_active":"2023-10-06T11:19:13.472906+0000","last_became_peered":"2023-10-06T11:19:13.472906+0000","last_unstale":"2023-10-06T11:19:14.494938+0000","last_undegraded":"2023-10-06T11:19:14.494938+0000","last_fullsized":"2023-10-06T11:19:14.494938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:48:27.417289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.484184+0000","last_change":"2023-10-06T11:19:14.484184+0000","last_active":"2023-10-06T11:19:14.484184+0000","last_peered":"2023-10-06T11:19:14.484184+0000","last_clean":"2023-10-06T11:19:14.484184+0000","last_became_active":"2023-10-06T11:19:14.477283+0000","last_became_peered":"2023-10-06T11:19:14.477283+0000","last_unstale":"2023-10-06T11:19:14.484184+0000","last_undegraded":"2023-10-06T11:19:14.484184+0000","last_fullsized":"2023-10-06T11:19:14.484184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.474386+0000","last_change":"2023-10-06T11:19:13.471276+0000","last_active":"2023-10-06T11:19:13.474386+0000","last_peered":"2023-10-06T11:19:13.474386+0000","last_clean":"2023-10-06T11:19:13.474386+0000","last_became_active":"2023-10-06T11:19:13.470848+0000","last_became_peered":"2023-10-06T11:19:13.470848+0000","last_unstale":"2023-10-06T11:19:13.474386+0000","last_undegraded":"2023-10-06T11:19:13.474386+0000","last_fullsized":"2023-10-06T11:19:13.474386+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:20.673213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.487464+0000","last_change":"2023-10-06T11:19:14.487464+0000","last_active":"2023-10-06T11:19:14.487464+0000","last_peered":"2023-10-06T11:19:14.487464+0000","last_clean":"2023-10-06T11:19:14.487464+0000","last_became_active":"2023-10-06T11:19:14.487070+0000","last_became_peered":"2023-10-06T11:19:14.487070+0000","last_unstale":"2023-10-06T11:19:14.487464+0000","last_undegraded":"2023-10-06T11:19:14.487464+0000","last_fullsized":"2023-10-06T11:19:14.487464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:06.937531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.480157+0000","last_change":"2023-10-06T11:19:13.473242+0000","last_active":"2023-10-06T11:19:13.480157+0000","last_peered":"2023-10-06T11:19:13.480157+0000","last_clean":"2023-10-06T11:19:13.480157+0000","last_became_active":"2023-10-06T11:19:13.472924+0000","last_became_peered":"2023-10-06T11:19:13.472924+0000","last_unstale":"2023-10-06T11:19:13.480157+0000","last_undegraded":"2023-10-06T11:19:13.480157+0000","last_fullsized":"2023-10-06T11:19:13.480157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:51.483503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.468138+0000","last_change":"2023-10-06T11:19:14.468138+0000","last_active":"2023-10-06T11:19:14.468138+0000","last_peered":"2023-10-06T11:19:14.468138+0000","last_clean":"2023-10-06T11:19:14.468138+0000","last_became_active":"2023-10-06T11:19:14.467817+0000","last_became_peered":"2023-10-06T11:19:14.467817+0000","last_unstale":"2023-10-06T11:19:14.468138+0000","last_undegraded":"2023-10-06T11:19:14.468138+0000","last_fullsized":"2023-10-06T11:19:14.468138+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:52:23.654115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.474469+0000","last_change":"2023-10-06T11:19:13.471416+0000","last_active":"2023-10-06T11:19:13.474469+0000","last_peered":"2023-10-06T11:19:13.474469+0000","last_clean":"2023-10-06T11:19:13.474469+0000","last_became_active":"2023-10-06T11:19:13.471091+0000","last_became_peered":"2023-10-06T11:19:13.471091+0000","last_unstale":"2023-10-06T11:19:13.474469+0000","last_undegraded":"2023-10-06T11:19:13.474469+0000","last_fullsized":"2023-10-06T11:19:13.474469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:43:43.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.493407+0000","last_change":"2023-10-06T11:19:14.493407+0000","last_active":"2023-10-06T11:19:14.493407+0000","last_peered":"2023-10-06T11:19:14.493407+0000","last_clean":"2023-10-06T11:19:14.493407+0000","last_became_active":"2023-10-06T11:19:14.493126+0000","last_became_peered":"2023-10-06T11:19:14.493126+0000","last_unstale":"2023-10-06T11:19:14.493407+0000","last_undegraded":"2023-10-06T11:19:14.493407+0000","last_fullsized":"2023-10-06T11:19:14.493407+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.485290+0000","last_change":"2023-10-06T11:19:13.476441+0000","last_active":"2023-10-06T11:19:13.485290+0000","last_peered":"2023-10-06T11:19:13.485290+0000","last_clean":"2023-10-06T11:19:13.485290+0000","last_became_active":"2023-10-06T11:19:13.476001+0000","last_became_peered":"2023-10-06T11:19:13.476001+0000","last_unstale":"2023-10-06T11:19:13.485290+0000","last_undegraded":"2023-10-06T11:19:13.485290+0000","last_fullsized":"2023-10-06T11:19:13.485290+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:04:06.705583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464878+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:19:13.464878+0000","last_peered":"2023-10-06T11:19:13.464878+0000","last_clean":"2023-10-06T11:19:13.464878+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:19:13.464878+0000","last_undegraded":"2023-10-06T11:19:13.464878+0000","last_fullsized":"2023-10-06T11:19:13.464878+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.458364+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:19:13.458364+0000","last_peered":"2023-10-06T11:19:13.458364+0000","last_clean":"2023-10-06T11:19:13.458364+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:19:13.458364+0000","last_undegraded":"2023-10-06T11:19:13.458364+0000","last_fullsized":"2023-10-06T11:19:13.458364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.494547+0000","last_change":"2023-10-06T11:19:14.494547+0000","last_active":"2023-10-06T11:19:14.494547+0000","last_peered":"2023-10-06T11:19:14.494547+0000","last_clean":"2023-10-06T11:19:14.494547+0000","last_became_active":"2023-10-06T11:19:14.494223+0000","last_became_peered":"2023-10-06T11:19:14.494223+0000","last_unstale":"2023-10-06T11:19:14.494547+0000","last_undegraded":"2023-10-06T11:19:14.494547+0000","last_fullsized":"2023-10-06T11:19:14.494547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:14.127084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.488439+0000","last_change":"2023-10-06T11:19:13.470668+0000","last_active":"2023-10-06T11:19:14.488439+0000","last_peered":"2023-10-06T11:19:14.488439+0000","last_clean":"2023-10-06T11:19:14.488439+0000","last_became_active":"2023-10-06T11:19:13.469191+0000","last_became_peered":"2023-10-06T11:19:13.469191+0000","last_unstale":"2023-10-06T11:19:14.488439+0000","last_undegraded":"2023-10-06T11:19:14.488439+0000","last_fullsized":"2023-10-06T11:19:14.488439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:33.081892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464756+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:19:13.464756+0000","last_peered":"2023-10-06T11:19:13.464756+0000","last_clean":"2023-10-06T11:19:13.464756+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:19:13.464756+0000","last_undegraded":"2023-10-06T11:19:13.464756+0000","last_fullsized":"2023-10-06T11:19:13.464756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481622+0000","last_change":"2023-10-06T11:19:14.481622+0000","last_active":"2023-10-06T11:19:14.481622+0000","last_peered":"2023-10-06T11:19:14.481622+0000","last_clean":"2023-10-06T11:19:14.481622+0000","last_became_active":"2023-10-06T11:19:14.480923+0000","last_became_peered":"2023-10-06T11:19:14.480923+0000","last_unstale":"2023-10-06T11:19:14.481622+0000","last_undegraded":"2023-10-06T11:19:14.481622+0000","last_fullsized":"2023-10-06T11:19:14.481622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:41:41.975314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.470221+0000","last_change":"2023-10-06T11:19:13.468516+0000","last_active":"2023-10-06T11:19:13.470221+0000","last_peered":"2023-10-06T11:19:13.470221+0000","last_clean":"2023-10-06T11:19:13.470221+0000","last_became_active":"2023-10-06T11:19:13.468083+0000","last_became_peered":"2023-10-06T11:19:13.468083+0000","last_unstale":"2023-10-06T11:19:13.470221+0000","last_undegraded":"2023-10-06T11:19:13.470221+0000","last_fullsized":"2023-10-06T11:19:13.470221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:06.997572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.460312+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:19:13.460312+0000","last_peered":"2023-10-06T11:19:13.460312+0000","last_clean":"2023-10-06T11:19:13.460312+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:19:13.460312+0000","last_undegraded":"2023-10-06T11:19:13.460312+0000","last_fullsized":"2023-10-06T11:19:13.460312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.474063+0000","last_change":"2023-10-06T11:19:14.474063+0000","last_active":"2023-10-06T11:19:14.474063+0000","last_peered":"2023-10-06T11:19:14.474063+0000","last_clean":"2023-10-06T11:19:14.474063+0000","last_became_active":"2023-10-06T11:19:14.473731+0000","last_became_peered":"2023-10-06T11:19:14.473731+0000","last_unstale":"2023-10-06T11:19:14.474063+0000","last_undegraded":"2023-10-06T11:19:14.474063+0000","last_fullsized":"2023-10-06T11:19:14.474063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:36.804886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.472493+0000","last_change":"2023-10-06T11:19:13.466424+0000","last_active":"2023-10-06T11:19:13.472493+0000","last_peered":"2023-10-06T11:19:13.472493+0000","last_clean":"2023-10-06T11:19:13.472493+0000","last_became_active":"2023-10-06T11:19:13.465815+0000","last_became_peered":"2023-10-06T11:19:13.465815+0000","last_unstale":"2023-10-06T11:19:13.472493+0000","last_undegraded":"2023-10-06T11:19:13.472493+0000","last_fullsized":"2023-10-06T11:19:13.472493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:39.770250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.458001+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:19:13.458001+0000","last_peered":"2023-10-06T11:19:13.458001+0000","last_clean":"2023-10-06T11:19:13.458001+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:19:13.458001+0000","last_undegraded":"2023-10-06T11:19:13.458001+0000","last_fullsized":"2023-10-06T11:19:13.458001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.476065+0000","last_change":"2023-10-06T11:19:14.476065+0000","last_active":"2023-10-06T11:19:14.476065+0000","last_peered":"2023-10-06T11:19:14.476065+0000","last_clean":"2023-10-06T11:19:14.476065+0000","last_became_active":"2023-10-06T11:19:14.475782+0000","last_became_peered":"2023-10-06T11:19:14.475782+0000","last_unstale":"2023-10-06T11:19:14.476065+0000","last_undegraded":"2023-10-06T11:19:14.476065+0000","last_fullsized":"2023-10-06T11:19:14.476065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:54.643473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.477044+0000","last_change":"2023-10-06T11:19:13.473457+0000","last_active":"2023-10-06T11:19:13.477044+0000","last_peered":"2023-10-06T11:19:13.477044+0000","last_clean":"2023-10-06T11:19:13.477044+0000","last_became_active":"2023-10-06T11:19:13.472598+0000","last_became_peered":"2023-10-06T11:19:13.472598+0000","last_unstale":"2023-10-06T11:19:13.477044+0000","last_undegraded":"2023-10-06T11:19:13.477044+0000","last_fullsized":"2023-10-06T11:19:13.477044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:46.594453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.457661+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:19:13.457661+0000","last_peered":"2023-10-06T11:19:13.457661+0000","last_clean":"2023-10-06T11:19:13.457661+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:19:13.457661+0000","last_undegraded":"2023-10-06T11:19:13.457661+0000","last_fullsized":"2023-10-06T11:19:13.457661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.481589+0000","last_change":"2023-10-06T11:19:14.481589+0000","last_active":"2023-10-06T11:19:14.481589+0000","last_peered":"2023-10-06T11:19:14.481589+0000","last_clean":"2023-10-06T11:19:14.481589+0000","last_became_active":"2023-10-06T11:19:14.481291+0000","last_became_peered":"2023-10-06T11:19:14.481291+0000","last_unstale":"2023-10-06T11:19:14.481589+0000","last_undegraded":"2023-10-06T11:19:14.481589+0000","last_fullsized":"2023-10-06T11:19:14.481589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:32:33.919569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.482708+0000","last_change":"2023-10-06T11:19:13.475430+0000","last_active":"2023-10-06T11:19:13.482708+0000","last_peered":"2023-10-06T11:19:13.482708+0000","last_clean":"2023-10-06T11:19:13.482708+0000","last_became_active":"2023-10-06T11:19:13.473911+0000","last_became_peered":"2023-10-06T11:19:13.473911+0000","last_unstale":"2023-10-06T11:19:13.482708+0000","last_undegraded":"2023-10-06T11:19:13.482708+0000","last_fullsized":"2023-10-06T11:19:13.482708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:20.673213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.464859+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:19:13.464859+0000","last_peered":"2023-10-06T11:19:13.464859+0000","last_clean":"2023-10-06T11:19:13.464859+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:19:13.464859+0000","last_undegraded":"2023-10-06T11:19:13.464859+0000","last_fullsized":"2023-10-06T11:19:13.464859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.489251+0000","last_change":"2023-10-06T11:19:14.489251+0000","last_active":"2023-10-06T11:19:14.489251+0000","last_peered":"2023-10-06T11:19:14.489251+0000","last_clean":"2023-10-06T11:19:14.489251+0000","last_became_active":"2023-10-06T11:19:14.488938+0000","last_became_peered":"2023-10-06T11:19:14.488938+0000","last_unstale":"2023-10-06T11:19:14.489251+0000","last_undegraded":"2023-10-06T11:19:14.489251+0000","last_fullsized":"2023-10-06T11:19:14.489251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:10.196620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.476870+0000","last_change":"2023-10-06T11:19:13.469260+0000","last_active":"2023-10-06T11:19:13.476870+0000","last_peered":"2023-10-06T11:19:13.476870+0000","last_clean":"2023-10-06T11:19:13.476870+0000","last_became_active":"2023-10-06T11:19:13.468925+0000","last_became_peered":"2023-10-06T11:19:13.468925+0000","last_unstale":"2023-10-06T11:19:13.476870+0000","last_undegraded":"2023-10-06T11:19:13.476870+0000","last_fullsized":"2023-10-06T11:19:13.476870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:16.389400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.453775+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:19:13.453775+0000","last_peered":"2023-10-06T11:19:13.453775+0000","last_clean":"2023-10-06T11:19:13.453775+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:19:13.453775+0000","last_undegraded":"2023-10-06T11:19:13.453775+0000","last_fullsized":"2023-10-06T11:19:13.453775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:19:14.480272+0000","last_change":"2023-10-06T11:19:14.480272+0000","last_active":"2023-10-06T11:19:14.480272+0000","last_peered":"2023-10-06T11:19:14.480272+0000","last_clean":"2023-10-06T11:19:14.480272+0000","last_became_active":"2023-10-06T11:19:14.479902+0000","last_became_peered":"2023-10-06T11:19:14.479902+0000","last_unstale":"2023-10-06T11:19:14.480272+0000","last_undegraded":"2023-10-06T11:19:14.480272+0000","last_fullsized":"2023-10-06T11:19:14.480272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:13.430600+0000","last_clean_scrub_stamp":"2023-10-06T11:19:13.430600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:36:17.404200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.467902+0000","last_change":"2023-10-06T11:19:13.466291+0000","last_active":"2023-10-06T11:19:13.467902+0000","last_peered":"2023-10-06T11:19:13.467902+0000","last_clean":"2023-10-06T11:19:13.467902+0000","last_became_active":"2023-10-06T11:19:13.465971+0000","last_became_peered":"2023-10-06T11:19:13.465971+0000","last_unstale":"2023-10-06T11:19:13.467902+0000","last_undegraded":"2023-10-06T11:19:13.467902+0000","last_fullsized":"2023-10-06T11:19:13.467902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:19:12.415800+0000","last_clean_scrub_stamp":"2023-10-06T11:19:12.415800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:02.929291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T11:19:13.468315+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:19:13.468315+0000","last_peered":"2023-10-06T11:19:13.468315+0000","last_clean":"2023-10-06T11:19:13.468315+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:19:13.468315+0000","last_undegraded":"2023-10-06T11:19:13.468315+0000","last_fullsized":"2023-10-06T11:19:13.468315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":739441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.504},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.494}]},{"osd":1,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.262}]},{"osd":2,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.1000000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.665,"5min":2.665,"15min":2.665},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.3069999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.7970000000000002,"5min":2.7970000000000002,"15min":2.7970000000000002},"last":1.155},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.649}]},{"osd":5,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.413},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":3.105,"5min":3.105,"15min":3.105},"last":1.3799999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.9119999999999999,"5min":2.9119999999999999,"15min":2.9119999999999999},"last":2.0870000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.4750000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.7789999999999999,"5min":2.7789999999999999,"15min":2.7789999999999999},"last":1.246}]},{"osd":3,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.605},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.532}]},{"osd":4,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.125},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.8140000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"max":{"1min":3.891,"5min":3.891,"15min":3.891},"last":1.5589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.429},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.1719999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.288},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":1.329}]},{"osd":2,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":3.609,"5min":3.609,"15min":3.609},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.405}]},{"osd":3,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.7250000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":3.4100000000000001,"5min":3.4100000000000001,"15min":3.4100000000000001},"last":1.512}]},{"osd":6,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.72,"5min":1.72,"15min":1.72},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":1.4890000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":3.5070000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.7869999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.101},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0009999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.395},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":1.264}]},{"osd":2,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.367,"15min":1.367},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":2.8250000000000002,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.597,"15min":1.597},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.137,"5min":1.137,"15min":1.137},"max":{"1min":2.464,"5min":2.464,"15min":2.464},"last":1.4390000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":1.02}]},{"osd":2,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.429},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.304}]},{"osd":5,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.407},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.627}]},{"osd":7,"last update":"Fri Oct 6 11:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.734},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301748,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070092,"statfs":{"total":96636764160,"available":96327774208,"internally_reserved":0,"allocated":372736,"data_stored":149073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.26}]},{"osd":1,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.4610000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":1.081,"5min":1.081,"15min":1.081},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.335}]},{"osd":5,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.2},{"interface":"front","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.548},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":1.371}]},{"osd":7,"last update":"Fri Oct 6 11:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.5},{"interface":"front","average":{"1min":1.611,"5min":1.611,"15min":1.611},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.4259999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301764,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070076,"statfs":{"total":96636764160,"available":96327757824,"internally_reserved":0,"allocated":389120,"data_stored":156279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302328,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069512,"statfs":{"total":96636764160,"available":96327180288,"internally_reserved":0,"allocated":966656,"data_stored":739441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.752}]},{"osd":2,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.3530000000000002,"5min":2.3530000000000002,"15min":2.3530000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.292},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.0209999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.573,"5min":2.573,"15min":2.573},"last":1.054},{"interface":"front","average":{"1min":1.536,"5min":1.536,"15min":1.536},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.161}]},{"osd":7,"last update":"Fri Oct 6 11:19:10 2023","interfaces":[{"interface":"back","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":1.01,"5min":1.01,"15min":1.01},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:19:19.213 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:19:19.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:19:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.422+0000 7fa7793cc700 1 Processor -- start 2023-10-06T11:19:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.422+0000 7fa7793cc700 1 -- start start 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa7741590b0 0x7fa7740cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 0x7fa774067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa774067e50 0x7fa7740691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740682d0 con 0x7fa7740b2420 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740cc8b0 con 0x7fa7741590b0 2023-10-06T11:19:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7793cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740cbca0 con 0x7fa774067e50 2023-10-06T11:19:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7737fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 0x7fa774067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa7741590b0 0x7fa7740cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa7741590b0 0x7fa7740cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:58202/0 (socket says 172.21.15.46:58202) 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 -- 172.21.15.46:0/3020023654 learned_addr learned my addr 172.21.15.46:0/3020023654 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa778bcb700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa774067e50 0x7fa7740691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 -- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa774067e50 msgr2=0x7fa7740691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa774067e50 0x7fa7740691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 -- 172.21.15.46:0/3020023654 >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 msgr2=0x7fa774067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 --2- 172.21.15.46:0/3020023654 >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 0x7fa774067910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa773fff700 1 -- 172.21.15.46:0/3020023654 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740cbf80 con 0x7fa7741590b0 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.426+0000 7fa7737fe700 1 --2- 172.21.15.46:0/3020023654 >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 0x7fa774067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa773fff700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cb600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa7740685e0 tx=0x7fa764009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d1bc1089de9378 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa772ffd700 1 -- 172.21.15.46:0/3020023654 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400e070 con 0x7fa7741590b0 2023-10-06T11:19:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa772ffd700 1 -- 172.21.15.46:0/3020023654 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76400ba20 con 0x7fa7741590b0 2023-10-06T11:19:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa772ffd700 1 -- 172.21.15.46:0/3020023654 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640092e0 con 0x7fa7741590b0 2023-10-06T11:19:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 msgr2=0x7fa7740cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cb600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa7740685e0 tx=0x7fa764009840 comp rx=0 tx=0).stop 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- 172.21.15.46:0/3020023654 shutdown_connections 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa774067e50 0x7fa7740691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 --2- 172.21.15.46:0/3020023654 >> v2:172.21.15.46:3300/0 conn(0x7fa7740b2420 0x7fa774067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 --2- 172.21.15.46:0/3020023654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cb600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- 172.21.15.46:0/3020023654 >> 172.21.15.46:0/3020023654 conn(0x7fa7740cfff0 msgr2=0x7fa7740d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- 172.21.15.46:0/3020023654 shutdown_connections 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- 172.21.15.46:0/3020023654 wait complete. 2023-10-06T11:19:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 Processor -- start 2023-10-06T11:19:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.430+0000 7fa7793cc700 1 -- start start 2023-10-06T11:19:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7740b2420 0x7fa77415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774155060 con 0x7fa774067e50 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c5090 con 0x7fa7741590b0 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c5370 con 0x7fa7740b2420 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53086/0 (socket says 172.21.15.46:53086) 2023-10-06T11:19:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 -- 172.21.15.46:0/1826382417 learned_addr learned my addr 172.21.15.46:0/1826382417 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7737fe700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7740b2420 0x7fa77415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa778bcb700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 -- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7740b2420 msgr2=0x7fa77415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7740b2420 0x7fa77415bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 -- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 msgr2=0x7fa7740cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 -- 172.21.15.46:0/1826382417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009420 con 0x7fa774067e50 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa773fff700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7740cc9f0 tx=0x7fa764018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764008c30 con 0x7fa774067e50 2023-10-06T11:19:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764008db0 con 0x7fa774067e50 2023-10-06T11:19:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640256b0 con 0x7fa774067e50 2023-10-06T11:19:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa764009ae0 con 0x7fa774067e50 2023-10-06T11:19:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.434+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7740c8860 con 0x7fa774067e50 2023-10-06T11:19:19.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.438+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774064590 con 0x7fa774067e50 2023-10-06T11:19:19.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.438+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa764003040 con 0x7fa774067e50 2023-10-06T11:19:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.438+0000 7fa75bfff700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 0x7fa760062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.438+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa76407dfe0 con 0x7fa774067e50 2023-10-06T11:19:19.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.442+0000 7fa7737fe700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 0x7fa760062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.442+0000 7fa7737fe700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 0x7fa760062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7680093f0 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.446+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa76404e480 con 0x7fa774067e50 2023-10-06T11:19:19.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.678+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa7740cc260 con 0x7fa774067e50 2023-10-06T11:19:19.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.690+0000 7fa75bfff700 1 -- 172.21.15.46:0/1826382417 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa764051aa0 con 0x7fa774067e50 2023-10-06T11:19:19.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 msgr2=0x7fa760062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 0x7fa760062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7680093f0 tx=0x7fa768008040 comp rx=0 tx=0).stop 2023-10-06T11:19:19.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 msgr2=0x7fa77416a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7740cc9f0 tx=0x7fa764018000 comp rx=0 tx=0).stop 2023-10-06T11:19:19.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 shutdown_connections 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa760060550 0x7fa760062a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7741590b0 0x7fa7740cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa7740b2420 0x7fa77415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 --2- 172.21.15.46:0/1826382417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa774067e50 0x7fa77416a640 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 >> 172.21.15.46:0/1826382417 conn(0x7fa7740cfff0 msgr2=0x7fa7740b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 shutdown_connections 2023-10-06T11:19:19.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.694+0000 7fa7793cc700 1 -- 172.21.15.46:0/1826382417 wait complete. 2023-10-06T11:19:19.702 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:19:19.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:19:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.882+0000 7fa2fbecd700 1 Processor -- start 2023-10-06T11:19:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 -- start start 2023-10-06T11:19:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa2f41502f0 0x7fa2f40c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2f41506c0 0x7fa2f4159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f40c67e0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f406e810 con 0x7fa2f41506c0 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fbecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f406eaf0 con 0x7fa2f41502f0 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fa46a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2f406e2a0 0x7fa2f4156490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34154/0 (socket says 172.21.15.46:34154) 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2fa46a700 1 -- 172.21.15.46:0/840637512 learned_addr learned my addr 172.21.15.46:0/840637512 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/840637512 >> v2:172.21.15.132:3300/0 conn(0x7fa2f41502f0 0x7fa2f40c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.886+0000 7fa2f9468700 1 --2- 172.21.15.46:0/840637512 >> v2:172.21.15.46:3300/0 conn(0x7fa2f41506c0 0x7fa2f4159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f9c69700 1 -- 172.21.15.46:0/840637512 >> v2:172.21.15.46:3300/0 conn(0x7fa2f41506c0 msgr2=0x7fa2f4159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/840637512 >> v2:172.21.15.46:3300/0 conn(0x7fa2f41506c0 0x7fa2f4159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f8c67700 1 -- 172.21.15.46:0/840637512 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980446324 0 0) 0x7fa2f40c67e0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f9c69700 1 -- 172.21.15.46:0/840637512 >> v1:172.21.15.46:6789/0 conn(0x7fa2f406e2a0 legacy=0x7fa2f4156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f9c69700 1 -- 172.21.15.46:0/840637512 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f406edd0 con 0x7fa2f41502f0 2023-10-06T11:19:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/840637512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41502f0 0x7fa2f40c0bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2f406ca30 tx=0x7fa2f0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d3675bc90205b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f8c67700 1 -- 172.21.15.46:0/840637512 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0005c10 con 0x7fa2f41502f0 2023-10-06T11:19:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f8c67700 1 -- 172.21.15.46:0/840637512 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0005d70 con 0x7fa2f41502f0 2023-10-06T11:19:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2f8c67700 1 -- 172.21.15.46:0/840637512 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0006260 con 0x7fa2f41502f0 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 -- 172.21.15.46:0/840637512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41502f0 msgr2=0x7fa2f40c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/840637512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41502f0 0x7fa2f40c0bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2f406ca30 tx=0x7fa2f0009120 comp rx=0 tx=0).stop 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 -- 172.21.15.46:0/840637512 shutdown_connections 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/840637512 >> v2:172.21.15.46:3300/0 conn(0x7fa2f41506c0 0x7fa2f4159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/840637512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41502f0 0x7fa2f40c0bd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 -- 172.21.15.46:0/840637512 >> 172.21.15.46:0/840637512 conn(0x7fa2f40cbe30 msgr2=0x7fa2f40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 -- 172.21.15.46:0/840637512 shutdown_connections 2023-10-06T11:19:19.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.890+0000 7fa2fbecd700 1 -- 172.21.15.46:0/840637512 wait complete. 2023-10-06T11:19:19.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.894+0000 7fa2fbecd700 1 Processor -- start 2023-10-06T11:19:19.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.894+0000 7fa2fbecd700 1 -- start start 2023-10-06T11:19:19.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2f41502f0 0x7fa2f414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 0x7fa2f41575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f414ac10 con 0x7fa2f406e2a0 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40acb80 con 0x7fa2f41506c0 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f40ace60 con 0x7fa2f41502f0 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53104/0 (socket says 172.21.15.46:53104) 2023-10-06T11:19:19.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 -- 172.21.15.46:0/3324171319 learned_addr learned my addr 172.21.15.46:0/3324171319 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fa46a700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 0x7fa2f41575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 -- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2f41502f0 msgr2=0x7fa2f414ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2f41502f0 0x7fa2f414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 -- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 msgr2=0x7fa2f41575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 0x7fa2f41575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 -- 172.21.15.46:0/3324171319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f00093c0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fa46a700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 0x7fa2f41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2f9c69700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000a010 tx=0x7fa2f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000f870 con 0x7fa2f406e2a0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f000f9f0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000fdb0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f40ad140 con 0x7fa2f406e2a0 2023-10-06T11:19:19.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.898+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f40ad700 con 0x7fa2f406e2a0 2023-10-06T11:19:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f40afaa0 con 0x7fa2f406e2a0 2023-10-06T11:19:19.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa2f0025450 con 0x7fa2f406e2a0 2023-10-06T11:19:19.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2e9ffb700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 0x7fa2d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:19.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa2f0014030 con 0x7fa2f406e2a0 2023-10-06T11:19:19.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2f9468700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 0x7fa2d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:19.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.902+0000 7fa2f9468700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 0x7fa2d8062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2f41570c0 tx=0x7fa2e40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:19.910+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa2f0050750 con 0x7fa2f406e2a0 2023-10-06T11:19:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.118+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2f40c6470 con 0x7fa2f406e2a0 2023-10-06T11:19:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.118+0000 7fa2e9ffb700 1 -- 172.21.15.46:0/3324171319 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fa2f002a020 con 0x7fa2f406e2a0 2023-10-06T11:19:20.123 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:19:20.123 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:19:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 msgr2=0x7fa2d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 0x7fa2d8062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2f41570c0 tx=0x7fa2e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 msgr2=0x7fa2f414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000a010 tx=0x7fa2f0018000 comp rx=0 tx=0).stop 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 shutdown_connections 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa2d8060550 0x7fa2d8062a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2f41506c0 0x7fa2f41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2f41502f0 0x7fa2f414ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 --2- 172.21.15.46:0/3324171319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2f406e2a0 0x7fa2f414a6d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 >> 172.21.15.46:0/3324171319 conn(0x7fa2f40cbe30 msgr2=0x7fa2f4069020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 shutdown_connections 2023-10-06T11:19:20.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.122+0000 7fa2fbecd700 1 -- 172.21.15.46:0/3324171319 wait complete. 2023-10-06T11:19:20.129 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:19:20.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:19:13.894505+0000', 'modified': '2023-10-06T11:19:17.491302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4995}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4995': {'gid': 4995, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6837/1323459348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1323459348}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 1323459348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:19:20.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:19:20.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:19:20.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.810+0000 7f16e42f0700 1 Processor -- start 2023-10-06T11:19:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.810+0000 7f16e42f0700 1 -- start start 2023-10-06T11:19:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e42f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc15ab60 0x7f16dc06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e42f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16dc0ad6c0 0x7f16dc0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e42f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16dc06d460 con 0x7f16dc15af30 2023-10-06T11:19:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e42f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16dc0c7000 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e42f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c72c0 con 0x7f16dc15ab60 2023-10-06T11:19:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16dc0ad6c0 0x7f16dc0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e188b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16dc15af30 0x7f16dc06cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34160/0 (socket says 172.21.15.46:34160) 2023-10-06T11:19:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16dc0ad6c0 0x7f16dc0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53118/0 (socket says 172.21.15.46:53118) 2023-10-06T11:19:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e188b700 1 -- 172.21.15.46:0/4093376285 learned_addr learned my addr 172.21.15.46:0/4093376285 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e208c700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc15ab60 0x7f16dc06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 -- 172.21.15.46:0/4093376285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc15ab60 msgr2=0x7f16dc06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc15ab60 0x7f16dc06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 -- 172.21.15.46:0/4093376285 >> v1:172.21.15.46:6789/0 conn(0x7f16dc15af30 legacy=0x7f16dc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 -- 172.21.15.46:0/4093376285 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0c75a0 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e288d700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc0ad6c0 0x7f16dc0adb20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009e60 tx=0x7f16d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941883b65a7ed9b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e108a700 1 -- 172.21.15.46:0/4093376285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d400e070 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e108a700 1 -- 172.21.15.46:0/4093376285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d400bb60 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.814+0000 7f16e108a700 1 -- 172.21.15.46:0/4093376285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4012430 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 -- 172.21.15.46:0/4093376285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc0ad6c0 msgr2=0x7f16dc0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc0ad6c0 0x7f16dc0adb20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009e60 tx=0x7f16d4009780 comp rx=0 tx=0).stop 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 -- 172.21.15.46:0/4093376285 shutdown_connections 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc0ad6c0 0x7f16dc0adb20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 --2- 172.21.15.46:0/4093376285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc15ab60 0x7f16dc06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 -- 172.21.15.46:0/4093376285 >> 172.21.15.46:0/4093376285 conn(0x7f16dc0cbe30 msgr2=0x7f16dc0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 -- 172.21.15.46:0/4093376285 shutdown_connections 2023-10-06T11:19:20.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 -- 172.21.15.46:0/4093376285 wait complete. 2023-10-06T11:19:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.818+0000 7f16e42f0700 1 Processor -- start 2023-10-06T11:19:20.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 -- start start 2023-10-06T11:19:20.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc0ad6c0 0x7f16dc156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16dc15af30 0x7f16dc0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c57b0 con 0x7f16dc15ab60 2023-10-06T11:19:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0ca7c0 con 0x7f16dc15af30 2023-10-06T11:19:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e42f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0c51f0 con 0x7f16dc0ad6c0 2023-10-06T11:19:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53126/0 (socket says 172.21.15.46:53126) 2023-10-06T11:19:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 -- 172.21.15.46:0/3491098219 learned_addr learned my addr 172.21.15.46:0/3491098219 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e208c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc0ad6c0 0x7f16dc156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e288d700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16dc15af30 0x7f16dc0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 -- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc0ad6c0 msgr2=0x7f16dc156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc0ad6c0 0x7f16dc156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 -- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16dc15af30 msgr2=0x7f16dc0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16dc15af30 0x7f16dc0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.822+0000 7f16e188b700 1 -- 172.21.15.46:0/3491098219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4009a20 con 0x7f16dc15ab60 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16e188b700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c790 tx=0x7f16cc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16d27fc700 1 -- 172.21.15.46:0/3491098219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc005af0 con 0x7f16dc15ab60 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16d27fc700 1 -- 172.21.15.46:0/3491098219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc005c50 con 0x7f16dc15ab60 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16d27fc700 1 -- 172.21.15.46:0/3491098219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0070d0 con 0x7f16dc15ab60 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc0c54d0 con 0x7f16dc15ab60 2023-10-06T11:19:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.826+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc157410 con 0x7f16dc15ab60 2023-10-06T11:19:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.830+0000 7f16d27fc700 1 -- 172.21.15.46:0/3491098219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f16cc0074c0 con 0x7f16dc15ab60 2023-10-06T11:19:20.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.830+0000 7f16d27fc700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 0x7f16c0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16d27fc700 1 -- 172.21.15.46:0/3491098219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f16cc07bef0 con 0x7f16dc15ab60 2023-10-06T11:19:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e208c700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 0x7f16c0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 Processor -- start 2023-10-06T11:19:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 -- start start 2023-10-06T11:19:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e208c700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 0x7f16c0062a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009d00 tx=0x7f16d8009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x179db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x179e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x16cbf30 con 0x1520ae0 2023-10-06T11:19:20.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c7ed0 con 0x156b790 2023-10-06T11:19:20.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x168e0b0 con 0x1520710 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53138/0 (socket says 172.21.15.46:53138) 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e208c700 1 -- 172.21.15.46:0/2971986216 learned_addr learned my addr 172.21.15.46:0/2971986216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e288d700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x179e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.834+0000 7f16e188b700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x179db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 -- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 msgr2=0x179db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x179db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 -- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 msgr2=0x179e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x179e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 -- 172.21.15.46:0/2971986216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x156c9e0 con 0x1520ae0 2023-10-06T11:19:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e188b700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x179db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e208c700 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011e80 tx=0x7f16d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81feb75feb7930fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16bf7fe700 1 -- 172.21.15.46:0/2971986216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011eb0 con 0x1520ae0 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16bf7fe700 1 -- 172.21.15.46:0/2971986216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8005d90 con 0x1520ae0 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16bf7fe700 1 -- 172.21.15.46:0/2971986216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d80066f0 con 0x1520ae0 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- 172.21.15.46:0/2971986216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 msgr2=0x179d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f16d8011e80 tx=0x7f16d8018000 comp rx=0 tx=0).stop 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- 172.21.15.46:0/2971986216 shutdown_connections 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x179e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x179db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 --2- 172.21.15.46:0/2971986216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x179d630 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- 172.21.15.46:0/2971986216 >> 172.21.15.46:0/2971986216 conn(0x1520eb0 msgr2=0x175a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- 172.21.15.46:0/2971986216 shutdown_connections 2023-10-06T11:19:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- 172.21.15.46:0/2971986216 wait complete. 2023-10-06T11:19:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 -- start start 2023-10-06T11:19:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.838+0000 7f16e65dc740 1 Processor -- start 2023-10-06T11:19:20.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 0x17a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x156b790 0x1757c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x168dd50 con 0x1520710 2023-10-06T11:19:20.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1666970 con 0x1520ae0 2023-10-06T11:19:20.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x14f7060 con 0x156b790 2023-10-06T11:19:20.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 10 client.0 _ll_register_callbacks cb 0x166ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53140/0 (socket says 172.21.15.46:53140) 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 -- 172.21.15.46:0/4221518601 learned_addr learned my addr 172.21.15.46:0/4221518601 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e188b700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 0x17a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e288d700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x156b790 0x1757c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 -- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x156b790 msgr2=0x1757c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x156b790 0x1757c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 -- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 msgr2=0x17a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 0x17a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 -- 172.21.15.46:0/4221518601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8011ab0 con 0x1520710 2023-10-06T11:19:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e188b700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 0x17a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e208c700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f24195fb976ff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8030070 con 0x1520710 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f16d80068e0 con 0x1520710 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1666cc0 con 0x1520710 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d8009a40 con 0x1520710 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 10 client.5175 fetch_fsmap learned FSMap version 17 2023-10-06T11:19:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1696370 con 0x1520710 2023-10-06T11:19:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1666cc0 con 0x1520710 2023-10-06T11:19:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f16d8014440 con 0x1520710 2023-10-06T11:19:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 10 client.5175 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:19:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.842+0000 7f16e65dc740 10 client.5175 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 20 client.5175 populate_metadata read hostname 'smithi046' 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17592d0 0x17acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 4 client.5175 mds_command: new command op to 4995 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 --> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x166a3e0 con 0x17592d0 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e188b700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17592d0 0x17acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e188b700 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17592d0 0x17acde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89134d5b802c8719 server_cookie=bf5acddac799e72 in_seq=0 out_seq=0 2023-10-06T11:19:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16bcff9700 10 client.5175 ms_handle_connect on v2:172.21.15.46:6834/1323459348 2023-10-06T11:19:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16bcff9700 1 -- 172.21.15.46:0/4221518601 <== mds.0 v2:172.21.15.46:6834/1323459348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x166a3e0 con 0x17592d0 2023-10-06T11:19:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16bcff9700 10 client.5175 handle_command_reply: tid=0 2023-10-06T11:19:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 client.5175 shutdown 2023-10-06T11:19:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 msgr2=0x17a1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009fd0 tx=0x7f16d8022000 comp rx=0 tx=0).stop 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 shutdown_connections 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17592d0 0x17acde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x156b790 0x1757c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1520ae0 0x17a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 --2- 172.21.15.46:0/4221518601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17a1c40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16bcff9700 0 client.5175 ms_handle_reset on v2:172.21.15.46:6834/1323459348 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 >> 172.21.15.46:0/4221518601 conn(0x1520eb0 msgr2=0x17811e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 shutdown_connections 2023-10-06T11:19:20.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.846+0000 7f16e65dc740 1 -- 172.21.15.46:0/4221518601 wait complete. 2023-10-06T11:19:20.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.850+0000 7f16e65dc740 20 client.5175 trim_cache size 0 max 16384 2023-10-06T11:19:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 Processor -- start 2023-10-06T11:19:20.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 -- start start 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x17526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x177ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c5890 con 0x1520ae0 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x169b040 con 0x156b790 2023-10-06T11:19:20.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x169d010 con 0x1520710 2023-10-06T11:19:20.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e188b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e188b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53144/0 (socket says 172.21.15.46:53144) 2023-10-06T11:19:20.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e188b700 1 -- 172.21.15.46:0/750056718 learned_addr learned my addr 172.21.15.46:0/750056718 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:20.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x17526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e288d700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x177ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 -- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 msgr2=0x177ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x177ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 -- 172.21.15.46:0/750056718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 msgr2=0x1752bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 -- 172.21.15.46:0/750056718 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0cb4f0 con 0x1520710 2023-10-06T11:19:20.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e188b700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.870+0000 7f16e208c700 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x17526b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006330 tx=0x7f16d802d070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a079a722a3d0204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16bffff700 1 -- 172.21.15.46:0/750056718 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8030070 con 0x1520710 2023-10-06T11:19:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16bffff700 1 -- 172.21.15.46:0/750056718 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8009a20 con 0x1520710 2023-10-06T11:19:20.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16bffff700 1 -- 172.21.15.46:0/750056718 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8014950 con 0x1520710 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 msgr2=0x17526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x17526b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006330 tx=0x7f16d802d070 comp rx=0 tx=0).stop 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- 172.21.15.46:0/750056718 shutdown_connections 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x177ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520ae0 0x1752bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 --2- 172.21.15.46:0/750056718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520710 0x17526b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- 172.21.15.46:0/750056718 >> 172.21.15.46:0/750056718 conn(0x1520eb0 msgr2=0x17596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- 172.21.15.46:0/750056718 shutdown_connections 2023-10-06T11:19:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- 172.21.15.46:0/750056718 wait complete. 2023-10-06T11:19:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 -- start start 2023-10-06T11:19:20.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.874+0000 7f16e65dc740 1 Processor -- start 2023-10-06T11:19:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520ae0 0x17fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x17abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ce400 con 0x1520710 2023-10-06T11:19:20.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c76e0 con 0x156b790 2023-10-06T11:19:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x16cc810 con 0x1520ae0 2023-10-06T11:19:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 10 client.0 _ll_register_callbacks cb 0x1693110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:19:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53154/0 (socket says 172.21.15.46:53154) 2023-10-06T11:19:20.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 -- 172.21.15.46:0/1364139470 learned_addr learned my addr 172.21.15.46:0/1364139470 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e188b700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520ae0 0x17fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e288d700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x17abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 -- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520ae0 msgr2=0x17fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520ae0 0x17fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 -- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 msgr2=0x17abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x17abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 -- 172.21.15.46:0/1364139470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc157410 con 0x1520710 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e208c700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16d8023010 tx=0x7f16d8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ee80201881c05f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:19:20.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8030040 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16d8005d90 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f16d8011e50 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d80068e0 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 10 client.5181 fetch_fsmap learned FSMap version 17 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x1696370 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8005d90 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f16d8046030 con 0x1520710 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 10 client.5181 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 10 client.5181 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 20 client.5181 populate_metadata read hostname 'smithi046' 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17ac730 0x17acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:20.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 4 client.5181 mds_command: new command op to 4995 tid=0[{"prefix": "status"}] 2023-10-06T11:19:20.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 --> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x169c410 con 0x17ac730 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.878+0000 7f16e188b700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17ac730 0x17acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e188b700 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17ac730 0x17acb00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe85a751776c258 server_cookie=4f201b354d3595ba in_seq=0 out_seq=0 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16bcff9700 10 client.5181 ms_handle_connect on v2:172.21.15.46:6834/1323459348 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16bcff9700 1 -- 172.21.15.46:0/1364139470 <== mds.0 v2:172.21.15.46:6834/1323459348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x169c410 con 0x17ac730 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16bcff9700 10 client.5181 handle_command_reply: tid=0 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 client.5181 shutdown 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 msgr2=0x17ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16d8023010 tx=0x7f16d8023040 comp rx=0 tx=0).stop 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 shutdown_connections 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:6834/1323459348,v1:172.21.15.46:6837/1323459348] conn(0x17ac730 0x17acb00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x156b790 0x17abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1520ae0 0x17fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 --2- 172.21.15.46:0/1364139470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1520710 0x17ff9d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16bcff9700 0 client.5181 ms_handle_reset on v2:172.21.15.46:6834/1323459348 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 >> 172.21.15.46:0/1364139470 conn(0x1520eb0 msgr2=0x17a0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 shutdown_connections 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 1 -- 172.21.15.46:0/1364139470 wait complete. 2023-10-06T11:19:20.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e65dc740 20 client.5181 trim_cache size 0 max 16384 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4995, 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.4335399080000002, 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 17, 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 35, 2023-10-06T11:19:20.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 35, 2023-10-06T11:19:20.891 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.285626493000001 2023-10-06T11:19:20.891 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:19:20.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 msgr2=0x7f16c0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 0x7f16c0062a30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009d00 tx=0x7f16d8009120 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 msgr2=0x7f16dc16a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00c790 tx=0x7f16cc007f60 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 shutdown_connections 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16c0060580 0x7f16c0062a30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16dc15af30 0x7f16dc0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc15ab60 0x7f16dc16a270 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 --2- 172.21.15.46:0/3491098219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16dc0ad6c0 0x7f16dc156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 >> 172.21.15.46:0/3491098219 conn(0x7f16dc0cbe30 msgr2=0x7f16dc0acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 shutdown_connections 2023-10-06T11:19:20.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:20.882+0000 7f16e42f0700 1 -- 172.21.15.46:0/3491098219 wait complete. 2023-10-06T11:19:20.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:19:21.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.062+0000 7f6728a4c700 1 Processor -- start 2023-10-06T11:19:21.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 -- start start 2023-10-06T11:19:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240b02a0 0x7f67240c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240ca7c0 0x7f67240cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67240c90b0 con 0x7f6724158100 2023-10-06T11:19:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240cb180 con 0x7f67240b02a0 2023-10-06T11:19:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f6728a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cc980 con 0x7f67240ca7c0 2023-10-06T11:19:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.066+0000 7f67237fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240b02a0 0x7f67240c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67240b02a0 0x7f67240c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53166/0 (socket says 172.21.15.46:53166) 2023-10-06T11:19:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 -- 172.21.15.46:0/2010498370 learned_addr learned my addr 172.21.15.46:0/2010498370 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6723fff700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240ca7c0 0x7f67240cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f671bfff700 1 --1- 172.21.15.46:0/2010498370 >> v1:172.21.15.46:6789/0 conn(0x7f6724158100 0x7f67240cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34162/0 (socket says 172.21.15.46:34162) 2023-10-06T11:19:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 -- 172.21.15.46:0/2010498370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240ca7c0 msgr2=0x7f67240cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240ca7c0 0x7f67240cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 -- 172.21.15.46:0/2010498370 >> v1:172.21.15.46:6789/0 conn(0x7f6724158100 legacy=0x7f67240cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 -- 172.21.15.46:0/2010498370 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240ccc60 con 0x7f67240b02a0 2023-10-06T11:19:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f67237fe700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240b02a0 0x7f67240c2c20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6714009990 tx=0x7f6714009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da63323be59852 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6722ffd700 1 -- 172.21.15.46:0/2010498370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714009e80 con 0x7f67240b02a0 2023-10-06T11:19:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6722ffd700 1 -- 172.21.15.46:0/2010498370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714005c10 con 0x7f67240b02a0 2023-10-06T11:19:21.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6722ffd700 1 -- 172.21.15.46:0/2010498370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400b650 con 0x7f67240b02a0 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 -- 172.21.15.46:0/2010498370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240b02a0 msgr2=0x7f67240c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240b02a0 0x7f67240c2c20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6714009990 tx=0x7f6714009220 comp rx=0 tx=0).stop 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 -- 172.21.15.46:0/2010498370 shutdown_connections 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240ca7c0 0x7f67240cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 --2- 172.21.15.46:0/2010498370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67240b02a0 0x7f67240c2c20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 -- 172.21.15.46:0/2010498370 >> 172.21.15.46:0/2010498370 conn(0x7f67240970c0 msgr2=0x7f6724067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 -- 172.21.15.46:0/2010498370 shutdown_connections 2023-10-06T11:19:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.070+0000 7f6728a4c700 1 -- 172.21.15.46:0/2010498370 wait complete. 2023-10-06T11:19:21.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 Processor -- start 2023-10-06T11:19:21.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 -- start start 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 0x7f6724097d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67240ca7c0 0x7f67240b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240b2960 con 0x7f6724158100 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240b1d90 con 0x7f67240ca7c0 2023-10-06T11:19:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6728a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240b2030 con 0x7f67240b02a0 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6723fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6723fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53168/0 (socket says 172.21.15.46:53168) 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f6723fff700 1 -- 172.21.15.46:0/268926121 learned_addr learned my addr 172.21.15.46:0/268926121 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.074+0000 7f67237fe700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 0x7f6724097d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f671bfff700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67240ca7c0 0x7f67240b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 -- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 msgr2=0x7f6724097d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 0x7f6724097d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 -- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67240ca7c0 msgr2=0x7f67240b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67240ca7c0 0x7f67240b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 -- 172.21.15.46:0/268926121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714009480 con 0x7f6724158100 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f67237fe700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 0x7f6724097d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6723fff700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f671c009df0 tx=0x7f671c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00cb50 con 0x7f6724158100 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c00f040 con 0x7f6724158100 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c014480 con 0x7f6724158100 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67240b2310 con 0x7f6724158100 2023-10-06T11:19:21.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672416d8f0 con 0x7f6724158100 2023-10-06T11:19:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.078+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67240b2b10 con 0x7f6724158100 2023-10-06T11:19:21.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.082+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f671c014830 con 0x7f6724158100 2023-10-06T11:19:21.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.082+0000 7f671b7fe700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 0x7f6704062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:21.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.082+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f671c07d5c0 con 0x7f6724158100 2023-10-06T11:19:21.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.082+0000 7f67237fe700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 0x7f6704062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:21.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.082+0000 7f67237fe700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 0x7f6704062a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67240afa90 tx=0x7f6714016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.086+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f67240b2b10 con 0x7f6724158100 2023-10-06T11:19:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.294+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6724064590 con 0x7f6724158100 2023-10-06T11:19:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.294+0000 7f671b7fe700 1 -- 172.21.15.46:0/268926121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f671c051440 con 0x7f6724158100 2023-10-06T11:19:21.301 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:19:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 msgr2=0x7f6704062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 0x7f6704062a30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f67240afa90 tx=0x7f6714016000 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 msgr2=0x7f67240c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f671c009df0 tx=0x7f671c00b040 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 shutdown_connections 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6704060580 0x7f6704062a30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6724158100 0x7f67240c5550 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67240ca7c0 0x7f67240b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 --2- 172.21.15.46:0/268926121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67240b02a0 0x7f6724097d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 >> 172.21.15.46:0/268926121 conn(0x7f67240970c0 msgr2=0x7f672406ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 shutdown_connections 2023-10-06T11:19:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:21.298+0000 7f6728a4c700 1 -- 172.21.15.46:0/268926121 wait complete. 2023-10-06T11:19:21.326 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:19:21.326 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:19:21.326 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:19:21.326 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:19:21.327 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:19:21.327 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:19:21.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:19:21.327 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:19:21.327 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:19:21.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.327 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:19:21.343 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:19:21.343 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:19:21.343 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:19:21.343 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:19:21.344 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42369sec preferred_lft 42369sec 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:19:21.345 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:19:21.346 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:19:21.347 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.348 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:19:21.348 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:19:21.348 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:19:21.348 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:19:21.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.350 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:19:21.431 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:19:21.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.435 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:19:21.499 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:19:21.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.500 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:19:21.500 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:19:21.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:19:21.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:19:21.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:19:21.501 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:19:21.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.639 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:19:21.653 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:19:21.653 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:19:21.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:19:21.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:21.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:19:21.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:19:21.731 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:19:21.791 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:19:21.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:19:21.792 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:19:21.850 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:19:21.850 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:19:21.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:19:21.974 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:19:21.975 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:19:21.975 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:19:21.996 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:19:21.997 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:19:21.997 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:19:21.997 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:19:21.997 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:19:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:19:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:19:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:19:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:19:22.012 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:19:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:19:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:19:22.013 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:19:22.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:19:22.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:19:22.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:19:22.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:19:22.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.258+0000 7f1062a6e700 1 Processor -- start 2023-10-06T11:19:22.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.258+0000 7f1062a6e700 1 -- start start 2023-10-06T11:19:22.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f105c161840 0x7f105c0caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105c0caf50 0x7f105c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c0c18f0 con 0x7f105c161470 2023-10-06T11:19:22.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0c0ce0 con 0x7f105c0caf50 2023-10-06T11:19:22.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0c0fc0 con 0x7f105c161840 2023-10-06T11:19:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105c0caf50 0x7f105c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f105c0caf50 0x7f105c06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53172/0 (socket says 172.21.15.46:53172) 2023-10-06T11:19:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 -- 172.21.15.46:0/1278799288 learned_addr learned my addr 172.21.15.46:0/1278799288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f105bfff700 1 --2- 172.21.15.46:0/1278799288 >> v2:172.21.15.132:3300/0 conn(0x7f105c161840 0x7f105c0caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 -- 172.21.15.46:0/1278799288 >> v2:172.21.15.132:3300/0 conn(0x7f105c161840 msgr2=0x7f105c0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 --2- 172.21.15.46:0/1278799288 >> v2:172.21.15.132:3300/0 conn(0x7f105c161840 0x7f105c0caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 -- 172.21.15.46:0/1278799288 >> v1:172.21.15.132:6789/0 conn(0x7f105c161470 legacy=0x7f105c0c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 -- 172.21.15.46:0/1278799288 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c0c12a0 con 0x7f105c0caf50 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f106100b700 1 --2- 172.21.15.46:0/1278799288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c0caf50 0x7f105c06cc30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f105c06ba90 tx=0x7f1054009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7d962bea27a546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f105b7fe700 1 -- 172.21.15.46:0/1278799288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400e070 con 0x7f105c0caf50 2023-10-06T11:19:22.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f105b7fe700 1 -- 172.21.15.46:0/1278799288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105400baa0 con 0x7f105c0caf50 2023-10-06T11:19:22.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f105b7fe700 1 -- 172.21.15.46:0/1278799288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540092e0 con 0x7f105c0caf50 2023-10-06T11:19:22.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- 172.21.15.46:0/1278799288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c0caf50 msgr2=0x7f105c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 --2- 172.21.15.46:0/1278799288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c0caf50 0x7f105c06cc30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f105c06ba90 tx=0x7f1054009840 comp rx=0 tx=0).stop 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- 172.21.15.46:0/1278799288 shutdown_connections 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 --2- 172.21.15.46:0/1278799288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c0caf50 0x7f105c06cc30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 --2- 172.21.15.46:0/1278799288 >> v2:172.21.15.132:3300/0 conn(0x7f105c161840 0x7f105c0caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.262+0000 7f1062a6e700 1 -- 172.21.15.46:0/1278799288 >> 172.21.15.46:0/1278799288 conn(0x7f105c0cbe30 msgr2=0x7f105c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.266+0000 7f1062a6e700 1 -- 172.21.15.46:0/1278799288 shutdown_connections 2023-10-06T11:19:22.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.266+0000 7f1062a6e700 1 -- 172.21.15.46:0/1278799288 wait complete. 2023-10-06T11:19:22.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.266+0000 7f1062a6e700 1 Processor -- start 2023-10-06T11:19:22.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.266+0000 7f1062a6e700 1 -- start start 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 0x7f105c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 0x7f105c16d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c16e7f0 con 0x7f105c161840 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0acb10 con 0x7f105c161470 2023-10-06T11:19:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c16e230 con 0x7f105c0caf50 2023-10-06T11:19:22.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53178/0 (socket says 172.21.15.46:53178) 2023-10-06T11:19:22.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 -- 172.21.15.46:0/973979569 learned_addr learned my addr 172.21.15.46:0/973979569 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106080a700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 0x7f105c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f105bfff700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 0x7f105c16d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 -- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 msgr2=0x7f105c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 0x7f105c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 -- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 msgr2=0x7f105c16d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 0x7f105c16d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 -- 172.21.15.46:0/973979569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009ae0 con 0x7f105c161840 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f105bfff700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 0x7f105c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106080a700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 0x7f105c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f106100b700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f105c06c150 tx=0x7f1054017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540032f0 con 0x7f105c161840 2023-10-06T11:19:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054012e60 con 0x7f105c161840 2023-10-06T11:19:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c16e510 con 0x7f105c161840 2023-10-06T11:19:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105401d670 con 0x7f105c161840 2023-10-06T11:19:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.270+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c0ac850 con 0x7f105c161840 2023-10-06T11:19:22.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.274+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c0acca0 con 0x7f105c161840 2023-10-06T11:19:22.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.274+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f105401da60 con 0x7f105c161840 2023-10-06T11:19:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.278+0000 7f1058ff9700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 0x7f1040062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.278+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f105407e210 con 0x7f105c161840 2023-10-06T11:19:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.278+0000 7f106080a700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 0x7f1040062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.278+0000 7f106080a700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 0x7f1040062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f105c15a970 tx=0x7f1050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.286+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f105404e600 con 0x7f105c161840 2023-10-06T11:19:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.482+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f105c063f90 con 0x7f1040060550 2023-10-06T11:19:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.482+0000 7f1058ff9700 1 -- 172.21.15.46:0/973979569 <== mgr.4105 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f105c063f90 con 0x7f1040060550 2023-10-06T11:19:22.488 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:19:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:19:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:19:22.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:19:22.488 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:19:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 msgr2=0x7f1040062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 0x7f1040062a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f105c15a970 tx=0x7f1050009040 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 msgr2=0x7f105c157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f105c06c150 tx=0x7f1054017000 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 shutdown_connections 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1040060550 0x7f1040062a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f105c161840 0x7f105c157db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105c161470 0x7f105c16d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 --2- 172.21.15.46:0/973979569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f105c0caf50 0x7f105c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 >> 172.21.15.46:0/973979569 conn(0x7f105c0cbe30 msgr2=0x7f105c06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 shutdown_connections 2023-10-06T11:19:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.486+0000 7f1062a6e700 1 -- 172.21.15.46:0/973979569 wait complete. 2023-10-06T11:19:22.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:19:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.666+0000 7fc19f59e700 1 Processor -- start 2023-10-06T11:19:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.666+0000 7fc19f59e700 1 -- start start 2023-10-06T11:19:22.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.670+0000 7fc19f59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc1a0159340 0x7fc1a00ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.670+0000 7fc19f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a00ab550 con 0x7fc1a0067760 2023-10-06T11:19:22.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.670+0000 7fc19f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a00aa980 con 0x7fc1a0158f70 2023-10-06T11:19:22.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.670+0000 7fc19f59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00aac20 con 0x7fc1a0159340 2023-10-06T11:19:22.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19e59c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc1a0158f70 0x7fc1a0067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42256/0 (socket says 172.21.15.46:42256) 2023-10-06T11:19:22.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19e59c700 1 -- 172.21.15.46:0/1209991292 learned_addr learned my addr 172.21.15.46:0/1209991292 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:22.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19dd9b700 1 --1- 172.21.15.46:0/1209991292 >> v1:172.21.15.46:6789/0 conn(0x7fc1a0067760 0x7fc1a00ac990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34174/0 (socket says 172.21.15.46:34174) 2023-10-06T11:19:22.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19ed9d700 1 --2- 172.21.15.46:0/1209991292 >> v2:172.21.15.132:3300/0 conn(0x7fc1a0159340 0x7fc1a00ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98119538 0 0) 0x7fc1a00ab550 con 0x7fc1a0067760 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc188003570 con 0x7fc1a0067760 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792782850 0 0) 0x7fc1a00aa980 con 0x7fc1a0158f70 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a00ab550 con 0x7fc1a0158f70 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1278157625 0 0) 0x7fc188003570 con 0x7fc1a0067760 2023-10-06T11:19:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1209991292 >> v1:172.21.15.132:6789/0 conn(0x7fc1a0158f70 legacy=0x7fc1a0067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1209991292 >> v1:172.21.15.46:6789/0 conn(0x7fc1a0067760 legacy=0x7fc1a00ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1209991292 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00aaf00 con 0x7fc1a0159340 2023-10-06T11:19:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19ed9d700 1 --2- 172.21.15.46:0/1209991292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0159340 0x7fc1a00ad0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc198009e60 tx=0x7fc198009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad39fa5ebde379b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19800e070 con 0x7fc1a0159340 2023-10-06T11:19:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19800b7a0 con 0x7fc1a0159340 2023-10-06T11:19:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19d59a700 1 -- 172.21.15.46:0/1209991292 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194001470 con 0x7fc1a0159340 2023-10-06T11:19:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19f59e700 1 -- 172.21.15.46:0/1209991292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0159340 msgr2=0x7fc1a00ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.674+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1209991292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0159340 0x7fc1a00ad0a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc198009e60 tx=0x7fc198009780 comp rx=0 tx=0).stop 2023-10-06T11:19:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 -- 172.21.15.46:0/1209991292 shutdown_connections 2023-10-06T11:19:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1209991292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0159340 0x7fc1a00ad0a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 -- 172.21.15.46:0/1209991292 >> 172.21.15.46:0/1209991292 conn(0x7fc1a00ccd50 msgr2=0x7fc1a00cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 -- 172.21.15.46:0/1209991292 shutdown_connections 2023-10-06T11:19:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 -- 172.21.15.46:0/1209991292 wait complete. 2023-10-06T11:19:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 Processor -- start 2023-10-06T11:19:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.678+0000 7fc19f59e700 1 -- start start 2023-10-06T11:19:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1a0067760 0x7fc1a016da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0158f70 0x7fc1a00b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a016df50 con 0x7fc1a0159340 2023-10-06T11:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00cc4f0 con 0x7fc1a0158f70 2023-10-06T11:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00cb8e0 con 0x7fc1a0067760 2023-10-06T11:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53188/0 (socket says 172.21.15.46:53188) 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1622654269 learned_addr learned my addr 172.21.15.46:0/1622654269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19e59c700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1a0067760 0x7fc1a016da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19dd9b700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0158f70 0x7fc1a00b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1a0067760 msgr2=0x7fc1a016da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1a0067760 0x7fc1a016da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0158f70 msgr2=0x7fc1a00b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0158f70 0x7fc1a00b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 -- 172.21.15.46:0/1622654269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009a20 con 0x7fc1a0159340 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.682+0000 7fc19ed9d700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1a006bc50 tx=0x7fc198017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198012c20 con 0x7fc1a0159340 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc198012d80 con 0x7fc1a0159340 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19801d6c0 con 0x7fc1a0159340 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a00cbbc0 con 0x7fc1a0159340 2023-10-06T11:19:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00cc180 con 0x7fc1a0159340 2023-10-06T11:19:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a00cc680 con 0x7fc1a0159340 2023-10-06T11:19:22.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.686+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc19801da30 con 0x7fc1a0159340 2023-10-06T11:19:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.690+0000 7fc18e7fc700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 0x7fc180062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.690+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc19807e5a0 con 0x7fc1a0159340 2023-10-06T11:19:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.690+0000 7fc19e59c700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 0x7fc180062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.690+0000 7fc19e59c700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 0x7fc180062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc190007fd0 tx=0x7fc190009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.694+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc19804ea50 con 0x7fc1a0159340 2023-10-06T11:19:22.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:22.906+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc1a0064590 con 0x7fc1a0159340 2023-10-06T11:19:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.094+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7fc19801dce0 con 0x7fc1a0159340 2023-10-06T11:19:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.094+0000 7fc18e7fc700 1 -- 172.21.15.46:0/1622654269 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc19804ea50 con 0x7fc1a0159340 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 msgr2=0x7fc180062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 0x7fc180062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc190007fd0 tx=0x7fc190009040 comp rx=0 tx=0).stop 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 msgr2=0x7fc1a0153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc1a006bc50 tx=0x7fc198017000 comp rx=0 tx=0).stop 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 shutdown_connections 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc180060550 0x7fc180062a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a0159340 0x7fc1a0153c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1a0158f70 0x7fc1a00b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 --2- 172.21.15.46:0/1622654269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1a0067760 0x7fc1a016da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 >> 172.21.15.46:0/1622654269 conn(0x7fc1a00ccd50 msgr2=0x7fc1a0156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 shutdown_connections 2023-10-06T11:19:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.098+0000 7fc19f59e700 1 -- 172.21.15.46:0/1622654269 wait complete. 2023-10-06T11:19:23.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:19:23.284 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:23.278+0000 7f2c5f549000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:19:23.285 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:23.282+0000 7f183fbce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:19:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.286+0000 7ff9596e3700 1 Processor -- start 2023-10-06T11:19:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.286+0000 7ff9596e3700 1 -- start start 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff9540b30f0 0x7ff9540b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9540b3570 0x7ff9540b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540afd20 con 0x7ff9540adbe0 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff954159740 con 0x7ff9540b3570 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9540b0000 con 0x7ff9540b30f0 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9540b3570 0x7ff9540b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9540b3570 0x7ff9540b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55028/0 (socket says 172.21.15.46:55028) 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 -- 172.21.15.46:0/1211047153 learned_addr learned my addr 172.21.15.46:0/1211047153 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff94bfff700 1 --2- 172.21.15.46:0/1211047153 >> v2:172.21.15.132:3300/0 conn(0x7ff9540b30f0 0x7ff9540b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 -- 172.21.15.46:0/1211047153 >> v2:172.21.15.132:3300/0 conn(0x7ff9540b30f0 msgr2=0x7ff9540b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 --2- 172.21.15.46:0/1211047153 >> v2:172.21.15.132:3300/0 conn(0x7ff9540b30f0 0x7ff9540b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 -- 172.21.15.46:0/1211047153 >> v1:172.21.15.46:6789/0 conn(0x7ff9540adbe0 legacy=0x7ff9540d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 -- 172.21.15.46:0/1211047153 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540b02e0 con 0x7ff9540b3570 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff958ee2700 1 --2- 172.21.15.46:0/1211047153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff9540b1300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9540881c0 tx=0x7ff94c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6319cb51d88127f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9537fe700 1 -- 172.21.15.46:0/1211047153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00e070 con 0x7ff9540b3570 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9537fe700 1 -- 172.21.15.46:0/1211047153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00ba20 con 0x7ff9540b3570 2023-10-06T11:19:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9537fe700 1 -- 172.21.15.46:0/1211047153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0092e0 con 0x7ff9540b3570 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- 172.21.15.46:0/1211047153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 msgr2=0x7ff9540b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1211047153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff9540b1300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9540881c0 tx=0x7ff94c009840 comp rx=0 tx=0).stop 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- 172.21.15.46:0/1211047153 shutdown_connections 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1211047153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff9540b1300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1211047153 >> v2:172.21.15.132:3300/0 conn(0x7ff9540b30f0 0x7ff9540b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- 172.21.15.46:0/1211047153 >> 172.21.15.46:0/1211047153 conn(0x7ff95406ef60 msgr2=0x7ff9540d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.290+0000 7ff9596e3700 1 -- 172.21.15.46:0/1211047153 shutdown_connections 2023-10-06T11:19:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 -- 172.21.15.46:0/1211047153 wait complete. 2023-10-06T11:19:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 Processor -- start 2023-10-06T11:19:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 -- start start 2023-10-06T11:19:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9540b30f0 0x7ff95415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff954153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff954099ee0 con 0x7ff9540b3570 2023-10-06T11:19:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95415a340 con 0x7ff9540adbe0 2023-10-06T11:19:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff9596e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff954150d10 con 0x7ff9540b30f0 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff958ee2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff954153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff953fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff94bfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9540b30f0 0x7ff95415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff953fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:56562/0 (socket says 172.21.15.46:56562) 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.294+0000 7ff953fff700 1 -- 172.21.15.46:0/1031516678 learned_addr learned my addr 172.21.15.46:0/1031516678 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 -- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9540b30f0 msgr2=0x7ff95415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9540b30f0 0x7ff95415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 -- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 msgr2=0x7ff9541507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 -- 172.21.15.46:0/1031516678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009ae0 con 0x7ff9540b3570 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff953fff700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff958ee2700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff954153bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff95406a0c0 tx=0x7ff94c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c008a80 con 0x7ff9540b3570 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c008c00 con 0x7ff9540b3570 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95415a5b0 con 0x7ff9540b3570 2023-10-06T11:19:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c01d420 con 0x7ff9540b3570 2023-10-06T11:19:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95415ab70 con 0x7ff9540b3570 2023-10-06T11:19:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954064590 con 0x7ff9540b3570 2023-10-06T11:19:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.298+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7ff94c003040 con 0x7ff9540b3570 2023-10-06T11:19:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.302+0000 7ff950ff9700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 0x7ff934062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.302+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff94c07e430 con 0x7ff9540b3570 2023-10-06T11:19:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.302+0000 7ff953fff700 1 -- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 msgr2=0x7ff934062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.302+0000 7ff953fff700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 0x7ff934062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:19:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.306+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff94c04e820 con 0x7ff9540b3570 2023-10-06T11:19:23.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.486+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff9540675d0 con 0x7ff9540b3570 2023-10-06T11:19:23.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.494+0000 7ff950ff9700 1 -- 172.21.15.46:0/1031516678 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff94c051e40 con 0x7ff9540b3570 2023-10-06T11:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 msgr2=0x7ff934062a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 0x7ff934062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 msgr2=0x7ff954153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff954153bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff95406a0c0 tx=0x7ff94c018000 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 shutdown_connections 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9340605a0 0x7ff934062a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9540b3570 0x7ff954153bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9540b30f0 0x7ff95415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 --2- 172.21.15.46:0/1031516678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9540adbe0 0x7ff9541507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 >> 172.21.15.46:0/1031516678 conn(0x7ff95406ef60 msgr2=0x7ff954151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 shutdown_connections 2023-10-06T11:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.498+0000 7ff9596e3700 1 -- 172.21.15.46:0/1031516678 wait complete. 2023-10-06T11:19:23.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:19:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.686+0000 7f5b640cd700 1 Processor -- start 2023-10-06T11:19:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.686+0000 7f5b640cd700 1 -- start start 2023-10-06T11:19:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b640cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5b5c06ed10 0x7f5b5c0c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b640cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0c1bf0 con 0x7f5b5c0c6470 2023-10-06T11:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b640cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c06cf80 con 0x7f5b5c06e8a0 2023-10-06T11:19:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b640cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c06d260 con 0x7f5b5c06ed10 2023-10-06T11:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b61e69700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5b5c0c6470 0x7f5b5c0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53628/0 (socket says 172.21.15.46:53628) 2023-10-06T11:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b61e69700 1 -- 172.21.15.46:0/3273832549 learned_addr learned my addr 172.21.15.46:0/3273832549 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b6266a700 1 --2- 172.21.15.46:0/3273832549 >> v2:172.21.15.132:3300/0 conn(0x7f5b5c06ed10 0x7f5b5c0c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b60e67700 1 -- 172.21.15.46:0/3273832549 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690871407 0 0) 0x7f5b5c0c1bf0 con 0x7f5b5c0c6470 2023-10-06T11:19:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b60e67700 1 -- 172.21.15.46:0/3273832549 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b48003570 con 0x7f5b5c0c6470 2023-10-06T11:19:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b6266a700 1 -- 172.21.15.46:0/3273832549 >> v1:172.21.15.132:6789/0 conn(0x7f5b5c06e8a0 legacy=0x7f5b5c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b6266a700 1 -- 172.21.15.46:0/3273832549 >> v1:172.21.15.46:6789/0 conn(0x7f5b5c0c6470 legacy=0x7f5b5c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.690+0000 7f5b6266a700 1 -- 172.21.15.46:0/3273832549 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0c1880 con 0x7f5b5c06ed10 2023-10-06T11:19:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b6266a700 1 --2- 172.21.15.46:0/3273832549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c0c2080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009e60 tx=0x7f5b54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84855bac0c77d732 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b60e67700 1 -- 172.21.15.46:0/3273832549 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400e070 con 0x7f5b5c06ed10 2023-10-06T11:19:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b60e67700 1 -- 172.21.15.46:0/3273832549 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5400bb60 con 0x7f5b5c06ed10 2023-10-06T11:19:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b60e67700 1 -- 172.21.15.46:0/3273832549 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54012430 con 0x7f5b5c06ed10 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 -- 172.21.15.46:0/3273832549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06ed10 msgr2=0x7f5b5c0c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 --2- 172.21.15.46:0/3273832549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c0c2080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009e60 tx=0x7f5b54009780 comp rx=0 tx=0).stop 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 -- 172.21.15.46:0/3273832549 shutdown_connections 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 --2- 172.21.15.46:0/3273832549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c0c2080 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 -- 172.21.15.46:0/3273832549 >> 172.21.15.46:0/3273832549 conn(0x7f5b5c0cbe30 msgr2=0x7f5b5c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 -- 172.21.15.46:0/3273832549 shutdown_connections 2023-10-06T11:19:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.694+0000 7f5b640cd700 1 -- 172.21.15.46:0/3273832549 wait complete. 2023-10-06T11:19:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 Processor -- start 2023-10-06T11:19:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 -- start start 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06e8a0 0x7f5b5c16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b5c0c6470 0x7f5b5c069030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0aba10 con 0x7f5b5c06ed10 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0c5160 con 0x7f5b5c06e8a0 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0c5440 con 0x7f5b5c0c6470 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55046/0 (socket says 172.21.15.46:55046) 2023-10-06T11:19:23.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 -- 172.21.15.46:0/1429476942 learned_addr learned my addr 172.21.15.46:0/1429476942 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b6266a700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b5c0c6470 0x7f5b5c069030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61e69700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06e8a0 0x7f5b5c16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b5c0c6470 msgr2=0x7f5b5c069030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b5c0c6470 0x7f5b5c069030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06e8a0 msgr2=0x7f5b5c16a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06e8a0 0x7f5b5c16a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 -- 172.21.15.46:0/1429476942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54009a20 con 0x7f5b5c06ed10 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b61668700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00c590 tx=0x7f5b4c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011590 con 0x7f5b5c06ed10 2023-10-06T11:19:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c011710 con 0x7f5b5c06ed10 2023-10-06T11:19:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011ad0 con 0x7f5b5c06ed10 2023-10-06T11:19:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.698+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c0c5720 con 0x7f5b5c06ed10 2023-10-06T11:19:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.702+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c16aa20 con 0x7f5b5c06ed10 2023-10-06T11:19:23.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.702+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c0abbc0 con 0x7f5b5c06ed10 2023-10-06T11:19:23.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.702+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7f5b4c01b3b0 con 0x7f5b5c06ed10 2023-10-06T11:19:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.706+0000 7f5b51ffb700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 0x7f5b40062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.706+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5b4c07be30 con 0x7f5b5c06ed10 2023-10-06T11:19:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.706+0000 7f5b61e69700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 msgr2=0x7f5b40062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.706+0000 7f5b61e69700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 0x7f5b40062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:19:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.710+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5b5c0abbc0 con 0x7f5b5c06ed10 2023-10-06T11:19:23.818 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:23.814+0000 7f183fbce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:19:23.882 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:23.878+0000 7f2c5f549000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:19:23.890 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:23.886+0000 7f183fbce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:19:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.910+0000 7f5b61e69700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 msgr2=0x7f5b40062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.910+0000 7f5b61e69700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 0x7f5b40062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:19:23.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.918+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f5b5c0660f0 con 0x7f5b5c06ed10 2023-10-06T11:19:23.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.926+0000 7f5b51ffb700 1 -- 172.21.15.46:0/1429476942 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5b4c017020 con 0x7f5b5c06ed10 2023-10-06T11:19:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 msgr2=0x7f5b40062a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:19:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 0x7f5b40062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 msgr2=0x7f5b5c068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00c590 tx=0x7f5b4c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 shutdown_connections 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5b400605a0 0x7f5b40062a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b5c0c6470 0x7f5b5c069030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b5c06ed10 0x7f5b5c068af0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 --2- 172.21.15.46:0/1429476942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b5c06e8a0 0x7f5b5c16a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 >> 172.21.15.46:0/1429476942 conn(0x7f5b5c0cbe30 msgr2=0x7f5b5c0c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 shutdown_connections 2023-10-06T11:19:23.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:23.930+0000 7f5b640cd700 1 -- 172.21.15.46:0/1429476942 wait complete. 2023-10-06T11:19:23.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:23.958 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:19:23.963 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:23.958+0000 7f2c5f549000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:19:23.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:23.987 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-06T11:19:24.023 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.018+0000 7f183fbce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:19:24.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T11:19:24.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.106+0000 7f2c5f549000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:19:24.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.250+0000 7f195eca3700 1 Processor -- start 2023-10-06T11:19:24.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.254+0000 7f195eca3700 1 -- start start 2023-10-06T11:19:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.254+0000 7f195eca3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1958140720 0x7f19580ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.254+0000 7f195eca3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1958140af0 0x7f19580aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19580afff0 con 0x7f19580cb3d0 2023-10-06T11:19:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19580cc150 con 0x7f1958140af0 2023-10-06T11:19:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19580cc430 con 0x7f1958140720 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1958140af0 0x7f19580aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195e4a2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f19580cb3d0 0x7f19580cb870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53644/0 (socket says 172.21.15.46:53644) 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195e4a2700 1 -- 172.21.15.46:0/227025431 learned_addr learned my addr 172.21.15.46:0/227025431 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195dca1700 1 --2- 172.21.15.46:0/227025431 >> v2:172.21.15.132:3300/0 conn(0x7f1958140720 0x7f19580ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 -- 172.21.15.46:0/227025431 >> v2:172.21.15.132:3300/0 conn(0x7f1958140720 msgr2=0x7f19580ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 --2- 172.21.15.46:0/227025431 >> v2:172.21.15.132:3300/0 conn(0x7f1958140720 0x7f19580ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 -- 172.21.15.46:0/227025431 >> v1:172.21.15.46:6789/0 conn(0x7f19580cb3d0 legacy=0x7f19580cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 -- 172.21.15.46:0/227025431 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580cc710 con 0x7f1958140af0 2023-10-06T11:19:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195cc9f700 1 -- 172.21.15.46:0/227025431 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499214700 0 0) 0x7f19580afff0 con 0x7f19580cb3d0 2023-10-06T11:19:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195d4a0700 1 --2- 172.21.15.46:0/227025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1958140af0 0x7f19580aee50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1958064f30 tx=0x7f19480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b5cd36fc91e0b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195cc9f700 1 -- 172.21.15.46:0/227025431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948011630 con 0x7f1958140af0 2023-10-06T11:19:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195cc9f700 1 -- 172.21.15.46:0/227025431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1948011790 con 0x7f1958140af0 2023-10-06T11:19:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195cc9f700 1 -- 172.21.15.46:0/227025431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948011b30 con 0x7f1958140af0 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- 172.21.15.46:0/227025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1958140af0 msgr2=0x7f19580aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 --2- 172.21.15.46:0/227025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1958140af0 0x7f19580aee50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1958064f30 tx=0x7f19480091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- 172.21.15.46:0/227025431 shutdown_connections 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 --2- 172.21.15.46:0/227025431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1958140af0 0x7f19580aee50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 --2- 172.21.15.46:0/227025431 >> v2:172.21.15.132:3300/0 conn(0x7f1958140720 0x7f19580ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.258+0000 7f195eca3700 1 -- 172.21.15.46:0/227025431 >> 172.21.15.46:0/227025431 conn(0x7f19580ccd40 msgr2=0x7f19580cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.262+0000 7f195eca3700 1 -- 172.21.15.46:0/227025431 shutdown_connections 2023-10-06T11:19:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.262+0000 7f195eca3700 1 -- 172.21.15.46:0/227025431 wait complete. 2023-10-06T11:19:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.262+0000 7f195eca3700 1 Processor -- start 2023-10-06T11:19:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.262+0000 7f195eca3700 1 -- start start 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958140720 0x7f19580c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19580c4820 0x7f195815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580c4ca0 con 0x7f19580cb3d0 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19580c6340 con 0x7f1958140720 2023-10-06T11:19:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19580c1b80 con 0x7f19580c4820 2023-10-06T11:19:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55072/0 (socket says 172.21.15.46:55072) 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 -- 172.21.15.46:0/390017310 learned_addr learned my addr 172.21.15.46:0/390017310 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958140720 0x7f19580c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195e4a2700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19580c4820 0x7f195815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19580c4820 msgr2=0x7f195815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19580c4820 0x7f195815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958140720 msgr2=0x7f19580c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958140720 0x7f19580c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1948009420 con 0x7f19580cb3d0 2023-10-06T11:19:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195dca1700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f195400ec80 tx=0x7f19540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:24.273 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.270+0000 7f183fbce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:19:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400c850 con 0x7f19580cb3d0 2023-10-06T11:19:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19580c1e60 con 0x7f19580cb3d0 2023-10-06T11:19:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195400c9b0 con 0x7f19580cb3d0 2023-10-06T11:19:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400cd50 con 0x7f19580cb3d0 2023-10-06T11:19:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.266+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19580c2420 con 0x7f19580cb3d0 2023-10-06T11:19:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1958064590 con 0x7f19580cb3d0 2023-10-06T11:19:24.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 6) v1 ==== 79976+0+0 (secure 0 0 0) 0x7f1954013680 con 0x7f19580cb3d0 2023-10-06T11:19:24.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f194dffb700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:24.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f195407cfa0 con 0x7f19580cb3d0 2023-10-06T11:19:24.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f195d4a0700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:24.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.270+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:19:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.278+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19540acbe0 con 0x7f19580cb3d0 2023-10-06T11:19:24.343 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.338+0000 7f183fbce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:19:24.398 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.394+0000 7f2c5f549000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:19:24.472 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.466+0000 7f2c5f549000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:19:24.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.470+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f195806b690 con 0x7f193c0605a0 2023-10-06T11:19:24.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.474+0000 7f195d4a0700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:24.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.474+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:19:24.543 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.538+0000 7f183fbce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:19:24.624 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.622+0000 7f183fbce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:19:24.697 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.690+0000 7f2c5f549000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:19:24.706 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:24.702+0000 7f183fbce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:19:24.781 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.774+0000 7f2c5f549000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:19:24.873 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:24.866+0000 7f2c5f549000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:19:24.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.878+0000 7f195d4a0700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:24.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:24.878+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:19:25.066 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.062+0000 7f183fbce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:19:25.218 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.214+0000 7f183fbce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:19:25.286 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:25.282+0000 7f2c5f549000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:19:25.460 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:25.454+0000 7f2c5f549000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:19:25.553 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.550+0000 7f183fbce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:19:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:25.682+0000 7f195d4a0700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:25.682+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:19:25.769 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.766+0000 7f183fbce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:19:25.841 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:25.834+0000 7f2c5f549000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:19:25.858 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.854+0000 7f183fbce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:19:25.933 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:25.930+0000 7f183fbce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:19:26.011 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.006+0000 7f183fbce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:19:26.072 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.066+0000 7f2c5f549000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:19:26.169 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.162+0000 7f2c5f549000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:19:26.247 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.242+0000 7f183fbce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:19:26.250 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.246+0000 7f2c5f549000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:19:26.323 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.318+0000 7f183fbce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:19:26.335 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.330+0000 7f2c5f549000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:19:26.411 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.406+0000 7f183fbce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:19:26.569 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.562+0000 7f2c5f549000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:19:26.613 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.610+0000 7f183fbce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:19:26.649 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.642+0000 7f2c5f549000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:19:26.697 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:26.694+0000 7f183fbce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:19:26.742 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.738+0000 7f2c5f549000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:19:26.948 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:26.942+0000 7f2c5f549000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:19:27.037 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:27.030+0000 7f2c5f549000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:19:27.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:27.286+0000 7f195d4a0700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:19:27.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:27.286+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:19:27.420 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:19:27.418+0000 7f183fbce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:19:27.796 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:27.790+0000 7f2c5f549000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:19:27.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:27.810+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) v1 ==== 79687+0+0 (secure 0 0 0) 0x7f195404d800 con 0x7f19580cb3d0 2023-10-06T11:19:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:27.810+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:19:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:27.810+0000 7f194dffb700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.009 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.002+0000 7f2c1f412700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.009 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.002+0000 7f2c1f412700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.009 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.002+0000 7f2c1f412700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.009 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.002+0000 7f2c1f412700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.009 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.002+0000 7f2c1f412700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.010 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.006+0000 7f2c21c17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.010 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.006+0000 7f2c21c17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.010 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.006+0000 7f2c21c17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.010 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.006+0000 7f2c21c17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.010 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:19:28.006+0000 7f2c21c17700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:19:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.822+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 8) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f1954051390 con 0x7f19580cb3d0 2023-10-06T11:19:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.822+0000 7f194dffb700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.822+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f195806b690 con 0x7f193c0605a0 2023-10-06T11:19:28.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.826+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:28.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.830+0000 7f195d4a0700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19480091e0 tx=0x7f1948017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.954+0000 7f194dffb700 1 -- 172.21.15.46:0/390017310 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f195806b690 con 0x7f193c0605a0 2023-10-06T11:19:28.959 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 msgr2=0x7f193c062a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f19480091e0 tx=0x7f1948017000 comp rx=0 tx=0).stop 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 msgr2=0x7f1958156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f195400ec80 tx=0x7f19540091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 shutdown_connections 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f193c0605a0 0x7f193c062a50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19580c4820 0x7f195815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1958140720 0x7f19580c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 --2- 172.21.15.46:0/390017310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19580cb3d0 0x7f1958156870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 >> 172.21.15.46:0/390017310 conn(0x7f19580ccd40 msgr2=0x7f19580b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 shutdown_connections 2023-10-06T11:19:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:28.958+0000 7f195eca3700 1 -- 172.21.15.46:0/390017310 wait complete. 2023-10-06T11:19:28.985 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-06T11:19:28.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:28.986 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-06T11:19:29.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T11:19:29.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.270+0000 7f55c8625700 1 Processor -- start 2023-10-06T11:19:29.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.274+0000 7f55c8625700 1 -- start start 2023-10-06T11:19:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c8625700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 0x7f55c0068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c8625700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55c00aba90 0x7f55c00abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c8625700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c0159f20 con 0x7f55c00aeba0 2023-10-06T11:19:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c8625700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c00ad650 con 0x7f55c00aba90 2023-10-06T11:19:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c8625700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00ac450 con 0x7f55c0068290 2023-10-06T11:19:29.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55c00aba90 0x7f55c00abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55c00aba90 0x7f55c00abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55202/0 (socket says 172.21.15.46:55202) 2023-10-06T11:19:29.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 -- 172.21.15.46:0/3623470575 learned_addr learned my addr 172.21.15.46:0/3623470575 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:29.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c63c1700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 0x7f55c0068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 -- 172.21.15.46:0/3623470575 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 msgr2=0x7f55c0068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 0x7f55c0068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 -- 172.21.15.46:0/3623470575 >> v1:172.21.15.132:6789/0 conn(0x7f55c00aeba0 legacy=0x7f55c0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c53bf700 1 -- 172.21.15.46:0/3623470575 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088600356 0 0) 0x7f55b0001110 con 0x7f55c00aeba0 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 -- 172.21.15.46:0/3623470575 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015a160 con 0x7f55c00aba90 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c63c1700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 0x7f55c0068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:29.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c6bc2700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c00abf10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009f20 tx=0x7f55b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44f0810affc201f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.278+0000 7f55c53bf700 1 -- 172.21.15.46:0/3623470575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800e070 con 0x7f55c00aba90 2023-10-06T11:19:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c53bf700 1 -- 172.21.15.46:0/3623470575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800b9d0 con 0x7f55c00aba90 2023-10-06T11:19:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c53bf700 1 -- 172.21.15.46:0/3623470575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80092e0 con 0x7f55c00aba90 2023-10-06T11:19:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 -- 172.21.15.46:0/3623470575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 msgr2=0x7f55c00abf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c00abf10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009f20 tx=0x7f55b8009840 comp rx=0 tx=0).stop 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 -- 172.21.15.46:0/3623470575 shutdown_connections 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c00abf10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 --2- 172.21.15.46:0/3623470575 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c0068290 0x7f55c0068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 -- 172.21.15.46:0/3623470575 >> 172.21.15.46:0/3623470575 conn(0x7f55c006d490 msgr2=0x7f55c006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 -- 172.21.15.46:0/3623470575 shutdown_connections 2023-10-06T11:19:29.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 -- 172.21.15.46:0/3623470575 wait complete. 2023-10-06T11:19:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.282+0000 7f55c8625700 1 Processor -- start 2023-10-06T11:19:29.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 -- start start 2023-10-06T11:19:29.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55c0068290 0x7f55c0141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c00aeba0 0x7f55c00aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0141ff0 con 0x7f55c00aba90 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0157d20 con 0x7f55c0068290 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0157110 con 0x7f55c00aeba0 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55206/0 (socket says 172.21.15.46:55206) 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 -- 172.21.15.46:0/4290371807 learned_addr learned my addr 172.21.15.46:0/4290371807 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c63c1700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55c0068290 0x7f55c0141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c6bc2700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c00aeba0 0x7f55c00aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 -- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c00aeba0 msgr2=0x7f55c00aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c00aeba0 0x7f55c00aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 -- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55c0068290 msgr2=0x7f55c0141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55c0068290 0x7f55c0141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 -- 172.21.15.46:0/4290371807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc009420 con 0x7f55c00aba90 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c5bc0700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c530 tx=0x7f55b000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0011590 con 0x7f55c00aba90 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0011710 con 0x7f55c00aba90 2023-10-06T11:19:29.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b8009ae0 con 0x7f55c00aba90 2023-10-06T11:19:29.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.286+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0011ad0 con 0x7f55c00aba90 2023-10-06T11:19:29.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.290+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01576d0 con 0x7f55c00aba90 2023-10-06T11:19:29.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.290+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c00660f0 con 0x7f55c00aba90 2023-10-06T11:19:29.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.290+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f55b001b3b0 con 0x7f55c00aba90 2023-10-06T11:19:29.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.294+0000 7f55b67fc700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 0x7f55a40628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.294+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f55b007bae0 con 0x7f55c00aba90 2023-10-06T11:19:29.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.294+0000 7f55c63c1700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 0x7f55a40628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.294+0000 7f55c63c1700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 0x7f55a40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f40 tx=0x7f55bc0091a0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.298+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f55b004c050 con 0x7f55c00aba90 2023-10-06T11:19:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.490+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c0159f20 con 0x7f55a40603f0 2023-10-06T11:19:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.598+0000 7f55b67fc700 1 -- 172.21.15.46:0/4290371807 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f55c0159f20 con 0x7f55a40603f0 2023-10-06T11:19:29.606 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-06T11:19:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.602+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 msgr2=0x7f55a40628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.602+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 0x7f55a40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009f40 tx=0x7f55bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.602+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 msgr2=0x7f55c0142270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.602+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c530 tx=0x7f55b000c840 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 shutdown_connections 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f55a40603f0 0x7f55a40628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f55c00aeba0 0x7f55c00aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55c00aba90 0x7f55c0142270 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 --2- 172.21.15.46:0/4290371807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55c0068290 0x7f55c0141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 >> 172.21.15.46:0/4290371807 conn(0x7f55c006d490 msgr2=0x7f55c015b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 shutdown_connections 2023-10-06T11:19:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.606+0000 7f55c8625700 1 -- 172.21.15.46:0/4290371807 wait complete. 2023-10-06T11:19:29.632 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-06T11:19:29.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:29.633 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-06T11:19:29.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T11:19:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.866+0000 7fe6d6923700 1 Processor -- start 2023-10-06T11:19:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.866+0000 7fe6d6923700 1 -- start start 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.866+0000 7fe6d6923700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6d0158de0 0x7fe6d00b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.866+0000 7fe6d6923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6d0069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00b5eb0 con 0x7fe6d00695a0 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6923700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d0069a20 con 0x7fe6d0158de0 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00cef30 con 0x7fe6d01591b0 2023-10-06T11:19:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6122700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe6d00695a0 0x7fe6d015cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53660/0 (socket says 172.21.15.46:53660) 2023-10-06T11:19:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6122700 1 -- 172.21.15.46:0/3795679152 learned_addr learned my addr 172.21.15.46:0/3795679152 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6d0069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5921700 1 --2- 172.21.15.46:0/3795679152 >> v2:172.21.15.132:3300/0 conn(0x7fe6d0158de0 0x7fe6d00b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169352252 0 0) 0x7fe6c8001490 con 0x7fe6d00695a0 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6bc003570 con 0x7fe6d00695a0 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1392918187 0 0) 0x7fe6bc003570 con 0x7fe6d00695a0 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 msgr2=0x7fe6d0069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6d0069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 >> v2:172.21.15.132:3300/0 conn(0x7fe6d0158de0 msgr2=0x7fe6d00b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 --2- 172.21.15.46:0/3795679152 >> v2:172.21.15.132:3300/0 conn(0x7fe6d0158de0 0x7fe6d00b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00cf4f0 con 0x7fe6d00695a0 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fe6c8001980 con 0x7fe6d00695a0 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 >> v1:172.21.15.46:6789/0 conn(0x7fe6d00695a0 legacy=0x7fe6d015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6bc004810 0x7fe6bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0069a20 con 0x7fe6bc004810 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00cef30 con 0x7fe6bc003a60 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc00ba60 con 0x7fe6d01591b0 2023-10-06T11:19:29.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6bc004810 0x7fe6bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5921700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d6122700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 -- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 msgr2=0x7fe6bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d01591b0 0x7fe6bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 -- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc003a60 msgr2=0x7fe6bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc003a60 0x7fe6bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 -- 172.21.15.46:0/3795679152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc003570 con 0x7fe6bc004810 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6bc004810 0x7fe6bc004c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009c30 tx=0x7fe6c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377cc87ee4e20b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000e710 con 0x7fe6bc004810 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c000e870 con 0x7fe6bc004810 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.870+0000 7fe6d491f700 1 -- 172.21.15.46:0/3795679152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ec30 con 0x7fe6bc004810 2023-10-06T11:19:29.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 -- 172.21.15.46:0/3795679152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6bc004810 msgr2=0x7fe6bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3795679152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6bc004810 0x7fe6bc004c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009c30 tx=0x7fe6c00091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6122700 1 -- 172.21.15.46:0/3795679152 reap_dead start 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 -- 172.21.15.46:0/3795679152 shutdown_connections 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 -- 172.21.15.46:0/3795679152 >> 172.21.15.46:0/3795679152 conn(0x7fe6d0068390 msgr2=0x7fe6d00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 -- 172.21.15.46:0/3795679152 shutdown_connections 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.874+0000 7fe6d6923700 1 -- 172.21.15.46:0/3795679152 wait complete. 2023-10-06T11:19:29.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 Processor -- start 2023-10-06T11:19:29.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 -- start start 2023-10-06T11:19:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc004810 0x7fe6d0171a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d00b48d0 0x7fe6d00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b5290 con 0x7fe6d006b430 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d006e070 con 0x7fe6bc004810 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0171f40 con 0x7fe6d00b48d0 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55224/0 (socket says 172.21.15.46:55224) 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 -- 172.21.15.46:0/3330935948 learned_addr learned my addr 172.21.15.46:0/3330935948 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d6122700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d00b48d0 0x7fe6d00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5921700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc004810 0x7fe6d0171a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 -- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d00b48d0 msgr2=0x7fe6d00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d00b48d0 0x7fe6d00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 -- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc004810 msgr2=0x7fe6d0171a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc004810 0x7fe6d0171a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 -- 172.21.15.46:0/3330935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d006b8c0 con 0x7fe6d006b430 2023-10-06T11:19:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.878+0000 7fe6d5120700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e4b0 tx=0x7fe6c001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0023070 con 0x7fe6d006b430 2023-10-06T11:19:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c000e9e0 con 0x7fe6d006b430 2023-10-06T11:19:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d006bba0 con 0x7fe6d006b430 2023-10-06T11:19:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c001e030 con 0x7fe6d006b430 2023-10-06T11:19:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d006c160 con 0x7fe6d006b430 2023-10-06T11:19:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d0157640 con 0x7fe6d006b430 2023-10-06T11:19:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.882+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fe6c001e1b0 con 0x7fe6d006b430 2023-10-06T11:19:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.886+0000 7fe6c5ffb700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 0x7fe6b4062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.886+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe6c007e5d0 con 0x7fe6d006b430 2023-10-06T11:19:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.886+0000 7fe6d5921700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 0x7fe6b4062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.886+0000 7fe6d5921700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 0x7fe6b4062c00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009ee0 tx=0x7fe6cc008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:29.890+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe6d0157640 con 0x7fe6d006b430 2023-10-06T11:19:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.082+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6d0157640 con 0x7fe6b4060750 2023-10-06T11:19:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.182+0000 7fe6c5ffb700 1 -- 172.21.15.46:0/3330935948 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe6d0157640 con 0x7fe6b4060750 2023-10-06T11:19:30.190 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 msgr2=0x7fe6b4062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 0x7fe6b4062c00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009ee0 tx=0x7fe6cc008040 comp rx=0 tx=0).stop 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 msgr2=0x7fe6d014deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000e4b0 tx=0x7fe6c001a000 comp rx=0 tx=0).stop 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 shutdown_connections 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe6b4060750 0x7fe6b4062c00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6d00b48d0 0x7fe6d00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6d006b430 0x7fe6d014deb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 --2- 172.21.15.46:0/3330935948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6bc004810 0x7fe6d0171a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.186+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 >> 172.21.15.46:0/3330935948 conn(0x7fe6d0068390 msgr2=0x7fe6d0156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.190+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 shutdown_connections 2023-10-06T11:19:30.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.190+0000 7fe6d6923700 1 -- 172.21.15.46:0/3330935948 wait complete. 2023-10-06T11:19:30.216 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-06T11:19:30.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:30.216 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-06T11:19:30.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T11:19:30.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.446+0000 7f04acdcc700 1 Processor -- start 2023-10-06T11:19:30.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.450+0000 7f04acdcc700 1 -- start start 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.450+0000 7f04acdcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04a8158de0 0x7f04a80b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.450+0000 7f04acdcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.450+0000 7f04acdcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04a80695a0 0x7f04a815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.450+0000 7f04acdcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a8069a20 con 0x7f04a80695a0 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80b5eb0 con 0x7f04a8158de0 2023-10-06T11:19:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80cef30 con 0x7f04a81591b0 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04a80695a0 0x7f04a815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04a80695a0 0x7f04a815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55238/0 (socket says 172.21.15.46:55238) 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 -- 172.21.15.46:0/66346094 learned_addr learned my addr 172.21.15.46:0/66346094 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a77fe700 1 --2- 172.21.15.46:0/66346094 >> v2:172.21.15.132:3300/0 conn(0x7f04a8158de0 0x7f04a80b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a6ffd700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 -- 172.21.15.46:0/66346094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 msgr2=0x7f04a8069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 -- 172.21.15.46:0/66346094 >> v2:172.21.15.132:3300/0 conn(0x7f04a8158de0 msgr2=0x7f04a80b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 --2- 172.21.15.46:0/66346094 >> v2:172.21.15.132:3300/0 conn(0x7f04a8158de0 0x7f04a80b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 -- 172.21.15.46:0/66346094 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a80cf4f0 con 0x7f04a80695a0 2023-10-06T11:19:30.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a7fff700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a815cc50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04a8088830 tx=0x7f04a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6114844d9225c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:30.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a67fc700 1 -- 172.21.15.46:0/66346094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000e070 con 0x7f04a80695a0 2023-10-06T11:19:30.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a67fc700 1 -- 172.21.15.46:0/66346094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a000b970 con 0x7f04a80695a0 2023-10-06T11:19:30.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04a67fc700 1 -- 172.21.15.46:0/66346094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00092e0 con 0x7f04a80695a0 2023-10-06T11:19:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- 172.21.15.46:0/66346094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 msgr2=0x7f04a815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a815cc50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04a8088830 tx=0x7f04a0009840 comp rx=0 tx=0).stop 2023-10-06T11:19:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- 172.21.15.46:0/66346094 shutdown_connections 2023-10-06T11:19:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a815cc50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 --2- 172.21.15.46:0/66346094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 --2- 172.21.15.46:0/66346094 >> v2:172.21.15.132:3300/0 conn(0x7f04a8158de0 0x7f04a80b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- 172.21.15.46:0/66346094 >> 172.21.15.46:0/66346094 conn(0x7f04a8068390 msgr2=0x7f04a80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- 172.21.15.46:0/66346094 shutdown_connections 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.454+0000 7f04acdcc700 1 -- 172.21.15.46:0/66346094 wait complete. 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 Processor -- start 2023-10-06T11:19:30.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 -- start start 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a8158de0 0x7f04a80c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80cbe20 con 0x7f04a80695a0 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80cab50 con 0x7f04a8158de0 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04acdcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c6290 con 0x7f04a81591b0 2023-10-06T11:19:30.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55248/0 (socket says 172.21.15.46:55248) 2023-10-06T11:19:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.458+0000 7f04a77fe700 1 -- 172.21.15.46:0/1180325538 learned_addr learned my addr 172.21.15.46:0/1180325538 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a6ffd700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a8158de0 0x7f04a80c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a7fff700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 -- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 msgr2=0x7f04a8154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 -- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a8158de0 msgr2=0x7f04a80c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a8158de0 0x7f04a80c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 -- 172.21.15.46:0/1180325538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0009ae0 con 0x7f04a80695a0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04a77fe700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f049800ecb0 tx=0x7f04980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049800c9d0 con 0x7f04a80695a0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049800cb30 con 0x7f04a80695a0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80cb060 con 0x7f04a80695a0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049800ced0 con 0x7f04a80695a0 2023-10-06T11:19:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80cb620 con 0x7f04a80695a0 2023-10-06T11:19:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.462+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8064590 con 0x7f04a80695a0 2023-10-06T11:19:30.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.466+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f0498013680 con 0x7f04a80695a0 2023-10-06T11:19:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.466+0000 7f048f7fe700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 0x7f0494062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.466+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f049807d160 con 0x7f04a80695a0 2023-10-06T11:19:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.466+0000 7f04a6ffd700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 0x7f0494062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.470+0000 7f04a6ffd700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 0x7f0494062c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04a815cb80 tx=0x7f049c008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.474+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f049804d1d0 con 0x7f04a80695a0 2023-10-06T11:19:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.670+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f04a80cad80 con 0x7f0494060750 2023-10-06T11:19:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.774+0000 7f048f7fe700 1 -- 172.21.15.46:0/1180325538 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f049c009900 con 0x7f0494060750 2023-10-06T11:19:30.782 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-06T11:19:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 msgr2=0x7f0494062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 0x7f0494062c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f04a815cb80 tx=0x7f049c008040 comp rx=0 tx=0).stop 2023-10-06T11:19:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 msgr2=0x7f04a80ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f049800ecb0 tx=0x7f04980091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 shutdown_connections 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0494060750 0x7f0494062c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04a81591b0 0x7f04a8154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04a8158de0 0x7f04a80c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 --2- 172.21.15.46:0/1180325538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04a80695a0 0x7f04a80ca610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 >> 172.21.15.46:0/1180325538 conn(0x7f04a8068390 msgr2=0x7f04a8155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 shutdown_connections 2023-10-06T11:19:30.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:30.782+0000 7f04acdcc700 1 -- 172.21.15.46:0/1180325538 wait complete. 2023-10-06T11:19:30.809 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-06T11:19:30.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:30.810 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-06T11:19:30.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T11:19:31.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.050+0000 7fded1ae3700 1 Processor -- start 2023-10-06T11:19:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.050+0000 7fded1ae3700 1 -- start start 2023-10-06T11:19:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded1ae3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 0x7fdecc063000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded1ae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded1ae3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc0c5b70 con 0x7fdecc15d310 2023-10-06T11:19:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded1ae3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc0a96a0 con 0x7fdecc15cf40 2023-10-06T11:19:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded1ae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0a90e0 con 0x7fdecc15c080 2023-10-06T11:19:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fdecbfff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fdecc15d310 0x7fdecc15d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42058/0 (socket says 172.21.15.46:42058) 2023-10-06T11:19:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fdecbfff700 1 -- 172.21.15.46:0/4155598585 learned_addr learned my addr 172.21.15.46:0/4155598585 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded0ae1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 0x7fdecc063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fdecb7fe700 1 -- 172.21.15.46:0/4155598585 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629966939 0 0) 0x7fdecc0c5b70 con 0x7fdecc15d310 2023-10-06T11:19:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fdecb7fe700 1 -- 172.21.15.46:0/4155598585 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdebc003570 con 0x7fdecc15d310 2023-10-06T11:19:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 -- 172.21.15.46:0/4155598585 >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 msgr2=0x7fdecc063000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 --2- 172.21.15.46:0/4155598585 >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 0x7fdecc063000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 -- 172.21.15.46:0/4155598585 >> v1:172.21.15.132:6789/0 conn(0x7fdecc15d310 legacy=0x7fdecc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 -- 172.21.15.46:0/4155598585 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc0a93c0 con 0x7fdecc15c080 2023-10-06T11:19:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded0ae1700 1 --2- 172.21.15.46:0/4155598585 >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 0x7fdecc063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.054+0000 7fded12e2700 1 --2- 172.21.15.46:0/4155598585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15c520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009e60 tx=0x7fdec4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7a6c5f76db9350 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fdecb7fe700 1 -- 172.21.15.46:0/4155598585 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400e070 con 0x7fdecc15c080 2023-10-06T11:19:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fdecb7fe700 1 -- 172.21.15.46:0/4155598585 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec400bab0 con 0x7fdecc15c080 2023-10-06T11:19:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fdecb7fe700 1 -- 172.21.15.46:0/4155598585 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400be70 con 0x7fdecc15c080 2023-10-06T11:19:31.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 -- 172.21.15.46:0/4155598585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 msgr2=0x7fdecc15c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 --2- 172.21.15.46:0/4155598585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15c520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009e60 tx=0x7fdec4009780 comp rx=0 tx=0).stop 2023-10-06T11:19:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 -- 172.21.15.46:0/4155598585 shutdown_connections 2023-10-06T11:19:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 --2- 172.21.15.46:0/4155598585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15c520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 --2- 172.21.15.46:0/4155598585 >> v2:172.21.15.132:3300/0 conn(0x7fdecc15cf40 0x7fdecc063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 -- 172.21.15.46:0/4155598585 >> 172.21.15.46:0/4155598585 conn(0x7fdecc0afe40 msgr2=0x7fdecc06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 -- 172.21.15.46:0/4155598585 shutdown_connections 2023-10-06T11:19:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.058+0000 7fded1ae3700 1 -- 172.21.15.46:0/4155598585 wait complete. 2023-10-06T11:19:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 Processor -- start 2023-10-06T11:19:31.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 -- start start 2023-10-06T11:19:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc15cf40 0x7fdecc0abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc15d310 0x7fdecc16c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc16d600 con 0x7fdecc15cf40 2023-10-06T11:19:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc15a5c0 con 0x7fdecc15d310 2023-10-06T11:19:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded1ae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1599b0 con 0x7fdecc15c080 2023-10-06T11:19:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fdecbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc15cf40 0x7fdecc0abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded0ae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded0ae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:48586/0 (socket says 172.21.15.46:48586) 2023-10-06T11:19:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded0ae1700 1 -- 172.21.15.46:0/902996750 learned_addr learned my addr 172.21.15.46:0/902996750 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.062+0000 7fded12e2700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc15d310 0x7fdecc16c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 -- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc15d310 msgr2=0x7fdecc16c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc15d310 0x7fdecc16c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 -- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc15cf40 msgr2=0x7fdecc0abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc15cf40 0x7fdecc0abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 -- 172.21.15.46:0/902996750 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00093c0 con 0x7fdecc15c080 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded0ae1700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdec000ecb0 tx=0x7fdec0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0005a00 con 0x7fdecc15c080 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0005b60 con 0x7fdecc15c080 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec4009a20 con 0x7fdecc15c080 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000c850 con 0x7fdecc15c080 2023-10-06T11:19:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc159f70 con 0x7fdecc15c080 2023-10-06T11:19:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.066+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc064590 con 0x7fdecc15c080 2023-10-06T11:19:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.070+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fdec000cc00 con 0x7fdecc15c080 2023-10-06T11:19:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.070+0000 7fdec8ff9700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 0x7fdeb0062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.070+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fdec007d1a0 con 0x7fdecc15c080 2023-10-06T11:19:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.070+0000 7fdecbfff700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 0x7fdeb0062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.070+0000 7fdecbfff700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 0x7fdeb0062c30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cd390 tx=0x7fdeb8009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.078+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdec004d210 con 0x7fdecc15c080 2023-10-06T11:19:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.270+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdecc04b5c0 con 0x7fdeb0060780 2023-10-06T11:19:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.370+0000 7fdec8ff9700 1 -- 172.21.15.46:0/902996750 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdecc04b5c0 con 0x7fdeb0060780 2023-10-06T11:19:31.375 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 msgr2=0x7fdeb0062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 0x7fdeb0062c30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0cd390 tx=0x7fdeb8009040 comp rx=0 tx=0).stop 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 msgr2=0x7fdecc15aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdec000ecb0 tx=0x7fdec0009120 comp rx=0 tx=0).stop 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 shutdown_connections 2023-10-06T11:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdeb0060780 0x7fdeb0062c30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc15d310 0x7fdecc16c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc15cf40 0x7fdecc0abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 --2- 172.21.15.46:0/902996750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdecc15c080 0x7fdecc15aa50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 >> 172.21.15.46:0/902996750 conn(0x7fdecc0afe40 msgr2=0x7fdecc069350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 shutdown_connections 2023-10-06T11:19:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.374+0000 7fded1ae3700 1 -- 172.21.15.46:0/902996750 wait complete. 2023-10-06T11:19:31.402 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-06T11:19:31.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:31.403 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-06T11:19:31.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T11:19:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.646+0000 7f21c9bcc700 1 Processor -- start 2023-10-06T11:19:31.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.646+0000 7f21c9bcc700 1 -- start start 2023-10-06T11:19:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c9bcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f21c4067760 0x7f21c406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c9bcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40c6150 con 0x7f21c41537f0 2023-10-06T11:19:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c9bcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40c6430 con 0x7f21c4153bc0 2023-10-06T11:19:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c9bcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c4157420 con 0x7f21c4067760 2023-10-06T11:19:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c8bca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f21c41537f0 0x7f21c4067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53668/0 (socket says 172.21.15.46:53668) 2023-10-06T11:19:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c8bca700 1 -- 172.21.15.46:0/3721888792 learned_addr learned my addr 172.21.15.46:0/3721888792 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c3fff700 1 --2- 172.21.15.46:0/3721888792 >> v2:172.21.15.132:3300/0 conn(0x7f21c4067760 0x7f21c406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434905120 0 0) 0x7f21c40c6150 con 0x7f21c41537f0 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b4003570 con 0x7f21c41537f0 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.650+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099718337 0 0) 0x7f21c40c6430 con 0x7f21c4153bc0 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c40c6150 con 0x7f21c4153bc0 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c3fff700 1 -- 172.21.15.46:0/3721888792 >> v1:172.21.15.132:6789/0 conn(0x7f21c4153bc0 legacy=0x7f21c406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c3fff700 1 -- 172.21.15.46:0/3721888792 >> v1:172.21.15.46:6789/0 conn(0x7f21c41537f0 legacy=0x7f21c4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c3fff700 1 -- 172.21.15.46:0/3721888792 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c406d910 con 0x7f21c4067760 2023-10-06T11:19:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c3fff700 1 --2- 172.21.15.46:0/3721888792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c4067760 0x7f21c406cc20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21b00098e0 tx=0x7f21b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148c8e23dd5dd923 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0011630 con 0x7f21c4067760 2023-10-06T11:19:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0011790 con 0x7f21c4067760 2023-10-06T11:19:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c37fe700 1 -- 172.21.15.46:0/3721888792 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0011b30 con 0x7f21c4067760 2023-10-06T11:19:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3721888792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c4067760 msgr2=0x7f21c406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3721888792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c4067760 0x7f21c406cc20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21b00098e0 tx=0x7f21b00091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3721888792 shutdown_connections 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3721888792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c4067760 0x7f21c406cc20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3721888792 >> 172.21.15.46:0/3721888792 conn(0x7f21c40cbbb0 msgr2=0x7f21c40cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3721888792 shutdown_connections 2023-10-06T11:19:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.654+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3721888792 wait complete. 2023-10-06T11:19:31.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.658+0000 7f21c9bcc700 1 Processor -- start 2023-10-06T11:19:31.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.658+0000 7f21c9bcc700 1 -- start start 2023-10-06T11:19:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 0x7f21c40c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 0x7f21c40b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b0d10 con 0x7f21c4153bc0 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b0ff0 con 0x7f21c41537f0 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b12d0 con 0x7f21c4067760 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c93cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c8bca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 0x7f21c40c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 0x7f21c40b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c93cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55262/0 (socket says 172.21.15.46:55262) 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c93cb700 1 -- 172.21.15.46:0/3262378080 learned_addr learned my addr 172.21.15.46:0/3262378080 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c8bca700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 0x7f21c40c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:48598/0 (socket says 172.21.15.46:48598) 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 -- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 msgr2=0x7f21c40c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 0x7f21c40c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 -- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 msgr2=0x7f21c4068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 -- 172.21.15.46:0/3262378080 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b0009420 con 0x7f21c41537f0 2023-10-06T11:19:31.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c93cb700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:31.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c3fff700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 0x7f21c40b03c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21b0016000 tx=0x7f21b0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b001f070 con 0x7f21c41537f0 2023-10-06T11:19:31.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b0000bc0 con 0x7f21c41537f0 2023-10-06T11:19:31.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c40b15b0 con 0x7f21c41537f0 2023-10-06T11:19:31.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0011b30 con 0x7f21c41537f0 2023-10-06T11:19:31.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.662+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c40b0900 con 0x7f21c41537f0 2023-10-06T11:19:31.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.666+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4064590 con 0x7f21c41537f0 2023-10-06T11:19:31.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.666+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f21b0023070 con 0x7f21c41537f0 2023-10-06T11:19:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.670+0000 7f21c0ff9700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 0x7f21a8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.670+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f21b007dc10 con 0x7f21c41537f0 2023-10-06T11:19:31.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.670+0000 7f21c8bca700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 0x7f21a8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:31.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.670+0000 7f21c8bca700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 0x7f21a8062c00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007fd0 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:31.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.678+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f21b004dd30 con 0x7f21c41537f0 2023-10-06T11:19:31.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.874+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c406dbf0 con 0x7f21a8060750 2023-10-06T11:19:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.982+0000 7f21c0ff9700 1 -- 172.21.15.46:0/3262378080 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f21c406dbf0 con 0x7f21a8060750 2023-10-06T11:19:31.989 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-06T11:19:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 msgr2=0x7f21a8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 0x7f21a8062c00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f21b8007fd0 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2023-10-06T11:19:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 msgr2=0x7f21c40b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 0x7f21c40b03c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21b0016000 tx=0x7f21b0017000 comp rx=0 tx=0).stop 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 shutdown_connections 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21a8060750 0x7f21a8062c00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21c4153bc0 0x7f21c4068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c41537f0 0x7f21c40b03c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 --2- 172.21.15.46:0/3262378080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21c4067760 0x7f21c40c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 >> 172.21.15.46:0/3262378080 conn(0x7f21c40cbbb0 msgr2=0x7f21c406c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 shutdown_connections 2023-10-06T11:19:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:31.986+0000 7f21c9bcc700 1 -- 172.21.15.46:0/3262378080 wait complete. 2023-10-06T11:19:32.016 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-06T11:19:32.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:32.016 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-06T11:19:32.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T11:19:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.246+0000 7f64118f0700 1 Processor -- start 2023-10-06T11:19:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.250+0000 7f64118f0700 1 -- start start 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f64118f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f640c159500 0x7f640c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f64118f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c0ce2c0 con 0x7f640c069190 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f64118f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c0af570 con 0x7f640c159130 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f64118f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0afb30 con 0x7f640c159500 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f640c069190 0x7f640c0695b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53682/0 (socket says 172.21.15.46:53682) 2023-10-06T11:19:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640b7fe700 1 -- 172.21.15.46:0/1608570061 learned_addr learned my addr 172.21.15.46:0/1608570061 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640a7fc700 1 --2- 172.21.15.46:0/1608570061 >> v2:172.21.15.132:3300/0 conn(0x7f640c159500 0x7f640c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503206739 0 0) 0x7f640c0ce2c0 con 0x7f640c069190 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f4003570 con 0x7f640c069190 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617290019 0 0) 0x7f640c0af570 con 0x7f640c159130 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c0ce2c0 con 0x7f640c159130 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659074311 0 0) 0x7f63f4003570 con 0x7f640c069190 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 >> v2:172.21.15.132:3300/0 conn(0x7f640c159500 msgr2=0x7f640c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 --2- 172.21.15.46:0/1608570061 >> v2:172.21.15.132:3300/0 conn(0x7f640c159500 0x7f640c0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 >> v1:172.21.15.132:6789/0 conn(0x7f640c159130 legacy=0x7f640c0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0af850 con 0x7f640c069190 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f64040017f0 con 0x7f640c069190 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 >> v1:172.21.15.46:6789/0 conn(0x7f640c069190 legacy=0x7f640c0695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63f4003bf0 0x7f63f4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 0x7f63f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0afb30 con 0x7f63f40051c0 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400c0b0 con 0x7f63f4003bf0 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f400c270 con 0x7f63f4004810 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640b7fe700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 0x7f63f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640a7fc700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.254+0000 7f640affd700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63f4003bf0 0x7f63f4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 -- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f4004810 msgr2=0x7f63f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f4004810 0x7f63f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 -- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63f4003bf0 msgr2=0x7f63f4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63f4003bf0 0x7f63f4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 -- 172.21.15.46:0/1608570061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4003570 con 0x7f63f40051c0 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 -- 172.21.15.46:0/1608570061 reap_dead start 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f640b7fe700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 0x7f63f400ba60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f640400a8f0 tx=0x7f640c0ce230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717daa2c1c697160 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64040070d0 con 0x7f63f40051c0 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6404007250 con 0x7f63f40051c0 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f6409ffb700 1 -- 172.21.15.46:0/1608570061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404007610 con 0x7f63f40051c0 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 -- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 msgr2=0x7f63f400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 0x7f63f400ba60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f640400a8f0 tx=0x7f640c0ce230 comp rx=0 tx=0).stop 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 -- 172.21.15.46:0/1608570061 shutdown_connections 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 --2- 172.21.15.46:0/1608570061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f40051c0 0x7f63f400ba60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 -- 172.21.15.46:0/1608570061 >> 172.21.15.46:0/1608570061 conn(0x7f640c068510 msgr2=0x7f640c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 -- 172.21.15.46:0/1608570061 shutdown_connections 2023-10-06T11:19:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.258+0000 7f64118f0700 1 -- 172.21.15.46:0/1608570061 wait complete. 2023-10-06T11:19:32.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 Processor -- start 2023-10-06T11:19:32.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 -- start start 2023-10-06T11:19:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f40051c0 0x7f640c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640c0b5960 0x7f640c1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b2f80 con 0x7f640c0cea50 2023-10-06T11:19:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0cf400 con 0x7f640c0b5960 2023-10-06T11:19:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f64118f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0b3500 con 0x7f63f40051c0 2023-10-06T11:19:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f640b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f640b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55276/0 (socket says 172.21.15.46:55276) 2023-10-06T11:19:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f640b7fe700 1 -- 172.21.15.46:0/3611116856 learned_addr learned my addr 172.21.15.46:0/3611116856 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f640affd700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f40051c0 0x7f640c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.262+0000 7f640a7fc700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640c0b5960 0x7f640c1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 -- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f40051c0 msgr2=0x7f640c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f40051c0 0x7f640c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 -- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640c0b5960 msgr2=0x7f640c1417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640c0b5960 0x7f640c1417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 -- 172.21.15.46:0/3611116856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0c9bd0 con 0x7f640c0cea50 2023-10-06T11:19:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f640b7fe700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6404002250 tx=0x7f6404016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404004dc0 con 0x7f640c0cea50 2023-10-06T11:19:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640401d420 con 0x7f640c0cea50 2023-10-06T11:19:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640401d7e0 con 0x7f640c0cea50 2023-10-06T11:19:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c0c9eb0 con 0x7f640c0cea50 2023-10-06T11:19:32.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c141ce0 con 0x7f640c0cea50 2023-10-06T11:19:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.266+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c0ca0a0 con 0x7f640c0cea50 2023-10-06T11:19:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.270+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f640401dbd0 con 0x7f640c0cea50 2023-10-06T11:19:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.270+0000 7f63f2ffd700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 0x7f63f8062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.270+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f640407eff0 con 0x7f640c0cea50 2023-10-06T11:19:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.270+0000 7f640affd700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 0x7f63f8062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.270+0000 7f640affd700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 0x7f63f8062c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b57e0 tx=0x7f63fc007d30 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.274+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f640c0ca0a0 con 0x7f640c0cea50 2023-10-06T11:19:32.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.466+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f640c0ca0a0 con 0x7f63f8060780 2023-10-06T11:19:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.574+0000 7f63f2ffd700 1 -- 172.21.15.46:0/3611116856 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f640c0ca0a0 con 0x7f63f8060780 2023-10-06T11:19:32.580 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 msgr2=0x7f63f8062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 0x7f63f8062c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b57e0 tx=0x7f63fc007d30 comp rx=0 tx=0).stop 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 msgr2=0x7f640c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6404002250 tx=0x7f6404016000 comp rx=0 tx=0).stop 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 shutdown_connections 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f63f8060780 0x7f63f8062c30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f640c0cea50 0x7f640c0ceec0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640c0b5960 0x7f640c1417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 --2- 172.21.15.46:0/3611116856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63f40051c0 0x7f640c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 >> 172.21.15.46:0/3611116856 conn(0x7f640c068510 msgr2=0x7f640c155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 shutdown_connections 2023-10-06T11:19:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.578+0000 7f64118f0700 1 -- 172.21.15.46:0/3611116856 wait complete. 2023-10-06T11:19:32.604 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-06T11:19:32.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:32.605 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-06T11:19:32.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T11:19:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.850+0000 7f2f19523700 1 Processor -- start 2023-10-06T11:19:32.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.850+0000 7f2f19523700 1 -- start start 2023-10-06T11:19:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f19523700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f14158de0 0x7f2f140b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f19523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f14069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f19523700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f140b5eb0 con 0x7f2f140695a0 2023-10-06T11:19:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f19523700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f14069a20 con 0x7f2f14158de0 2023-10-06T11:19:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f19523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f140cef30 con 0x7f2f141591b0 2023-10-06T11:19:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f18d22700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f140695a0 0x7f2f1415cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53688/0 (socket says 172.21.15.46:53688) 2023-10-06T11:19:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f18d22700 1 -- 172.21.15.46:0/2611080588 learned_addr learned my addr 172.21.15.46:0/2611080588 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f137fe700 1 --2- 172.21.15.46:0/2611080588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f14069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f13fff700 1 --2- 172.21.15.46:0/2611080588 >> v2:172.21.15.132:3300/0 conn(0x7f2f14158de0 0x7f2f140b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f12ffd700 1 -- 172.21.15.46:0/2611080588 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634314384 0 0) 0x7f2f0c001490 con 0x7f2f140695a0 2023-10-06T11:19:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.854+0000 7f2f137fe700 1 -- 172.21.15.46:0/2611080588 >> v2:172.21.15.132:3300/0 conn(0x7f2f14158de0 msgr2=0x7f2f140b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f137fe700 1 --2- 172.21.15.46:0/2611080588 >> v2:172.21.15.132:3300/0 conn(0x7f2f14158de0 0x7f2f140b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f137fe700 1 -- 172.21.15.46:0/2611080588 >> v1:172.21.15.46:6789/0 conn(0x7f2f140695a0 legacy=0x7f2f1415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f137fe700 1 -- 172.21.15.46:0/2611080588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f140cf4f0 con 0x7f2f141591b0 2023-10-06T11:19:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f137fe700 1 --2- 172.21.15.46:0/2611080588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f14069060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f080098e0 tx=0x7f2f080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883f2d0fb6544c47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f12ffd700 1 -- 172.21.15.46:0/2611080588 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800b640 con 0x7f2f141591b0 2023-10-06T11:19:32.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f12ffd700 1 -- 172.21.15.46:0/2611080588 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0800b7a0 con 0x7f2f141591b0 2023-10-06T11:19:32.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f12ffd700 1 -- 172.21.15.46:0/2611080588 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800bb40 con 0x7f2f141591b0 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 -- 172.21.15.46:0/2611080588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 msgr2=0x7f2f14069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 --2- 172.21.15.46:0/2611080588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f14069060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f080098e0 tx=0x7f2f080091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 -- 172.21.15.46:0/2611080588 shutdown_connections 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 --2- 172.21.15.46:0/2611080588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f14069060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 --2- 172.21.15.46:0/2611080588 >> v2:172.21.15.132:3300/0 conn(0x7f2f14158de0 0x7f2f140b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 -- 172.21.15.46:0/2611080588 >> 172.21.15.46:0/2611080588 conn(0x7f2f14068390 msgr2=0x7f2f140b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 -- 172.21.15.46:0/2611080588 shutdown_connections 2023-10-06T11:19:32.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.858+0000 7f2f19523700 1 -- 172.21.15.46:0/2611080588 wait complete. 2023-10-06T11:19:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 Processor -- start 2023-10-06T11:19:32.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 -- start start 2023-10-06T11:19:32.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f14158de0 0x7f2f1406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f1406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1406a370 con 0x7f2f140695a0 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1406aba0 con 0x7f2f14158de0 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f19523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1415bda0 con 0x7f2f141591b0 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55280/0 (socket says 172.21.15.46:55280) 2023-10-06T11:19:32.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f13fff700 1 -- 172.21.15.46:0/1420084272 learned_addr learned my addr 172.21.15.46:0/1420084272 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f137fe700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f14158de0 0x7f2f1406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.862+0000 7f2f18d22700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f1406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 -- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 msgr2=0x7f2f1406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f1406ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 -- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f14158de0 msgr2=0x7f2f1406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f14158de0 0x7f2f1406c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 -- 172.21.15.46:0/1420084272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009420 con 0x7f2f140695a0 2023-10-06T11:19:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f13fff700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400ec80 tx=0x7f2f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400c850 con 0x7f2f140695a0 2023-10-06T11:19:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0400c9b0 con 0x7f2f140695a0 2023-10-06T11:19:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1406a550 con 0x7f2f140695a0 2023-10-06T11:19:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400cd50 con 0x7f2f140695a0 2023-10-06T11:19:32.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1414a690 con 0x7f2f140695a0 2023-10-06T11:19:32.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.866+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f140cce10 con 0x7f2f140695a0 2023-10-06T11:19:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.870+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f2f04013680 con 0x7f2f140695a0 2023-10-06T11:19:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.870+0000 7f2efbfff700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 0x7f2f00062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:32.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.870+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2f0407d070 con 0x7f2f140695a0 2023-10-06T11:19:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.870+0000 7f2f137fe700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 0x7f2f00062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.874+0000 7f2f137fe700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 0x7f2f00062c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f080093f0 tx=0x7f2f08017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:32.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:32.878+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f040acbe0 con 0x7f2f140695a0 2023-10-06T11:19:33.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.074+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f14064590 con 0x7f2f00060750 2023-10-06T11:19:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.178+0000 7f2efbfff700 1 -- 172.21.15.46:0/1420084272 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2f14064590 con 0x7f2f00060750 2023-10-06T11:19:33.182 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 msgr2=0x7f2f00062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 0x7f2f00062c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f080093f0 tx=0x7f2f08017000 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 msgr2=0x7f2f1414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400ec80 tx=0x7f2f040091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 shutdown_connections 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f00060750 0x7f2f00062c00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f141591b0 0x7f2f1406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f14158de0 0x7f2f1406c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 --2- 172.21.15.46:0/1420084272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140695a0 0x7f2f1414a150 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 >> 172.21.15.46:0/1420084272 conn(0x7f2f14068390 msgr2=0x7f2f140b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 shutdown_connections 2023-10-06T11:19:33.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.182+0000 7f2f19523700 1 -- 172.21.15.46:0/1420084272 wait complete. 2023-10-06T11:19:33.210 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-06T11:19:33.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:33.210 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-06T11:19:33.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T11:19:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.446+0000 7fbf636ae700 1 Processor -- start 2023-10-06T11:19:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.446+0000 7fbf636ae700 1 -- start start 2023-10-06T11:19:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf636ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 0x7fbf5c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf636ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbf5c0c82e0 0x7fbf5c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf636ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c0c71a0 con 0x7fbf5c159420 2023-10-06T11:19:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf636ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c0c1bd0 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf636ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0c6590 con 0x7fbf5c159050 2023-10-06T11:19:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf60c49700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbf5c159420 0x7fbf5c0c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39806/0 (socket says 172.21.15.46:39806) 2023-10-06T11:19:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf60c49700 1 -- 172.21.15.46:0/2599594592 learned_addr learned my addr 172.21.15.46:0/2599594592 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.450+0000 7fbf61c4b700 1 --2- 172.21.15.46:0/2599594592 >> v2:172.21.15.132:3300/0 conn(0x7fbf5c0c82e0 0x7fbf5c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf6144a700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 0x7fbf5c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf53fff700 1 -- 172.21.15.46:0/2599594592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722717237 0 0) 0x7fbf5c0c71a0 con 0x7fbf5c159420 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf53fff700 1 -- 172.21.15.46:0/2599594592 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf48003570 con 0x7fbf5c159420 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf61c4b700 1 -- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 msgr2=0x7fbf5c0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf61c4b700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 0x7fbf5c0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf61c4b700 1 -- 172.21.15.46:0/2599594592 >> v1:172.21.15.46:6789/0 conn(0x7fbf5c159420 legacy=0x7fbf5c0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf61c4b700 1 -- 172.21.15.46:0/2599594592 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0c6870 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf6144a700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 0x7fbf5c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf61c4b700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c0c82e0 0x7fbf5c0ca1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009e60 tx=0x7fbf54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa786cb2e68df62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf53fff700 1 -- 172.21.15.46:0/2599594592 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400e070 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf53fff700 1 -- 172.21.15.46:0/2599594592 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400bab0 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf53fff700 1 -- 172.21.15.46:0/2599594592 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400be70 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 -- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c0c82e0 msgr2=0x7fbf5c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c0c82e0 0x7fbf5c0ca1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009e60 tx=0x7fbf54009780 comp rx=0 tx=0).stop 2023-10-06T11:19:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 -- 172.21.15.46:0/2599594592 shutdown_connections 2023-10-06T11:19:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c0c82e0 0x7fbf5c0ca1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 --2- 172.21.15.46:0/2599594592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c159050 0x7fbf5c0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.454+0000 7fbf636ae700 1 -- 172.21.15.46:0/2599594592 >> 172.21.15.46:0/2599594592 conn(0x7fbf5c0cbe20 msgr2=0x7fbf5c0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- 172.21.15.46:0/2599594592 shutdown_connections 2023-10-06T11:19:33.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- 172.21.15.46:0/2599594592 wait complete. 2023-10-06T11:19:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 Processor -- start 2023-10-06T11:19:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- start start 2023-10-06T11:19:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 0x7fbf5c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c159420 0x7fbf5c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0c4240 con 0x7fbf5c159050 2023-10-06T11:19:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0c5cc0 con 0x7fbf5c159420 2023-10-06T11:19:33.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.458+0000 7fbf636ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0afd40 con 0x7fbf5c0c82e0 2023-10-06T11:19:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39246/0 (socket says 172.21.15.46:39246) 2023-10-06T11:19:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 -- 172.21.15.46:0/4193965571 learned_addr learned my addr 172.21.15.46:0/4193965571 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:33.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf6144a700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 0x7fbf5c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf61c4b700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c159420 0x7fbf5c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 -- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 msgr2=0x7fbf5c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 0x7fbf5c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 -- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c159420 msgr2=0x7fbf5c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c159420 0x7fbf5c0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 -- 172.21.15.46:0/4193965571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58009420 con 0x7fbf5c159050 2023-10-06T11:19:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf6144a700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 0x7fbf5c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf60c49700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c5c0 tx=0x7fbf4c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c005b30 con 0x7fbf5c159050 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c005cb0 con 0x7fbf5c159050 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c007320 con 0x7fbf5c159050 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf54009a20 con 0x7fbf5c159050 2023-10-06T11:19:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.462+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c0affc0 con 0x7fbf5c159050 2023-10-06T11:19:33.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.466+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c06b890 con 0x7fbf5c159050 2023-10-06T11:19:33.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.466+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fbf4c01b3b0 con 0x7fbf5c159050 2023-10-06T11:19:33.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.466+0000 7fbf517fa700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 0x7fbf40062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:33.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.470+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fbf4c07c220 con 0x7fbf5c159050 2023-10-06T11:19:33.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.470+0000 7fbf6144a700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 0x7fbf40062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.470+0000 7fbf6144a700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 0x7fbf40062c00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf58011600 tx=0x7fbf580091a0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:33.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.474+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbf4c04c290 con 0x7fbf5c159050 2023-10-06T11:19:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.662+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf5c04b5c0 con 0x7fbf40060750 2023-10-06T11:19:33.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.770+0000 7fbf517fa700 1 -- 172.21.15.46:0/4193965571 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbf5c04b5c0 con 0x7fbf40060750 2023-10-06T11:19:33.776 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-06T11:19:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 msgr2=0x7fbf40062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 0x7fbf40062c00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf58011600 tx=0x7fbf580091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 msgr2=0x7fbf5c0c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00c5c0 tx=0x7fbf4c007e00 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 shutdown_connections 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbf40060750 0x7fbf40062c00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5c159420 0x7fbf5c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf5c159050 0x7fbf5c0c50b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 --2- 172.21.15.46:0/4193965571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf5c0c82e0 0x7fbf5c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 >> 172.21.15.46:0/4193965571 conn(0x7fbf5c0cbe20 msgr2=0x7fbf5c152870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 shutdown_connections 2023-10-06T11:19:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:33.774+0000 7fbf636ae700 1 -- 172.21.15.46:0/4193965571 wait complete. 2023-10-06T11:19:33.802 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-06T11:19:33.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:19:33.802 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-06T11:19:33.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T11:19:34.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.038+0000 7f98913c5700 1 Processor -- start 2023-10-06T11:19:34.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.038+0000 7f98913c5700 1 -- start start 2023-10-06T11:19:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f98913c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f988c158f80 0x7f988c159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f98913c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c063270 0x7f988c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f98913c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c159b70 con 0x7f988c15d830 2023-10-06T11:19:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f98913c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c159890 con 0x7f988c158f80 2023-10-06T11:19:34.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f98913c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c15dcc0 con 0x7f988c063270 2023-10-06T11:19:34.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988a7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f988c15d830 0x7f988c15a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39808/0 (socket says 172.21.15.46:39808) 2023-10-06T11:19:34.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988a7fc700 1 -- 172.21.15.46:0/1394231302 learned_addr learned my addr 172.21.15.46:0/1394231302 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988affd700 1 --2- 172.21.15.46:0/1394231302 >> v2:172.21.15.46:3300/0 conn(0x7f988c158f80 0x7f988c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988b7fe700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c063270 0x7f988c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751673646 0 0) 0x7f988c159b70 con 0x7f988c15d830 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9874003570 con 0x7f988c15d830 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988affd700 1 -- 172.21.15.46:0/1394231302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c063270 msgr2=0x7f988c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:34.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1827326087 0 0) 0x7f9874003570 con 0x7f988c15d830 2023-10-06T11:19:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988affd700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c063270 0x7f988c06d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988affd700 1 -- 172.21.15.46:0/1394231302 >> v1:172.21.15.46:6789/0 conn(0x7f988c15d830 legacy=0x7f988c15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.042+0000 7f988affd700 1 -- 172.21.15.46:0/1394231302 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0abc00 con 0x7f988c158f80 2023-10-06T11:19:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f988affd700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c158f80 0x7f988c159350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f987c0098e0 tx=0x7f987c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ce65ff97ddfffb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800015f0 con 0x7f988c158f80 2023-10-06T11:19:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c011420 con 0x7f988c158f80 2023-10-06T11:19:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f9889ffb700 1 -- 172.21.15.46:0/1394231302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0117c0 con 0x7f988c158f80 2023-10-06T11:19:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 -- 172.21.15.46:0/1394231302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c158f80 msgr2=0x7f988c159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c158f80 0x7f988c159350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f987c0098e0 tx=0x7f987c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 -- 172.21.15.46:0/1394231302 shutdown_connections 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c063270 0x7f988c06d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 --2- 172.21.15.46:0/1394231302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c158f80 0x7f988c159350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 -- 172.21.15.46:0/1394231302 >> 172.21.15.46:0/1394231302 conn(0x7f988c069f50 msgr2=0x7f988c0ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 -- 172.21.15.46:0/1394231302 shutdown_connections 2023-10-06T11:19:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.046+0000 7f98913c5700 1 -- 172.21.15.46:0/1394231302 wait complete. 2023-10-06T11:19:34.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 Processor -- start 2023-10-06T11:19:34.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 -- start start 2023-10-06T11:19:34.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c158f80 0x7f988c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 0x7f988c16ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c16acc0 con 0x7f988c063270 2023-10-06T11:19:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0c5570 con 0x7f988c15d830 2023-10-06T11:19:34.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f98913c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c16a420 con 0x7f988c158f80 2023-10-06T11:19:34.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39258/0 (socket says 172.21.15.46:39258) 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 -- 172.21.15.46:0/2875999204 learned_addr learned my addr 172.21.15.46:0/2875999204 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988a7fc700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c158f80 0x7f988c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988b7fe700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 0x7f988c16ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 -- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c158f80 msgr2=0x7f988c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c158f80 0x7f988c0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 -- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 msgr2=0x7f988c16ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 0x7f988c16ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 -- 172.21.15.46:0/2875999204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c009420 con 0x7f988c063270 2023-10-06T11:19:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988b7fe700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 0x7f988c16ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:19:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.050+0000 7f988affd700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f987c0030c0 tx=0x7f987c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c01cc90 con 0x7f988c063270 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c01cdf0 con 0x7f988c063270 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c01a6a0 con 0x7f988c063270 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c16a700 con 0x7f988c063270 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c169cb0 con 0x7f988c063270 2023-10-06T11:19:34.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.054+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c0c39f0 con 0x7f988c063270 2023-10-06T11:19:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.058+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f987c01aad0 con 0x7f988c063270 2023-10-06T11:19:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.058+0000 7f9872ffd700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 0x7f9878062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.058+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f987c07e800 con 0x7f988c063270 2023-10-06T11:19:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.058+0000 7f988a7fc700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 0x7f9878062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.058+0000 7f988a7fc700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 0x7f9878062c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9880002ee0 tx=0x7f9880009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.066+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f987c04e870 con 0x7f988c063270 2023-10-06T11:19:34.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.258+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f988c0c56b0 con 0x7f9878060750 2023-10-06T11:19:34.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.378+0000 7f9872ffd700 1 -- 172.21.15.46:0/2875999204 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9880007870 con 0x7f9878060750 2023-10-06T11:19:34.382 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-06T11:19:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.378+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 msgr2=0x7f9878062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.378+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 0x7f9878062c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9880002ee0 tx=0x7f9880009040 comp rx=0 tx=0).stop 2023-10-06T11:19:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.378+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 msgr2=0x7f988c1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.378+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f987c0030c0 tx=0x7f987c016000 comp rx=0 tx=0).stop 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 shutdown_connections 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9878060750 0x7f9878062c00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c15d830 0x7f988c16ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c158f80 0x7f988c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 --2- 172.21.15.46:0/2875999204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c063270 0x7f988c1573c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 >> 172.21.15.46:0/2875999204 conn(0x7f988c069f50 msgr2=0x7f988c15a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 shutdown_connections 2023-10-06T11:19:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.382+0000 7f98913c5700 1 -- 172.21.15.46:0/2875999204 wait complete. 2023-10-06T11:19:34.408 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-06T11:19:34.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T11:19:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.570+0000 7f3db926e700 1 Processor -- start 2023-10-06T11:19:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.570+0000 7f3db926e700 1 -- start start 2023-10-06T11:19:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db926e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 0x7f3db40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db926e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db4069470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db926e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40cfc20 con 0x7f3db4069020 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db926e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40cee20 con 0x7f3db41591f0 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db926e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40699b0 con 0x7f3db40b40b0 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db2ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 0x7f3db40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db27fc700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3db4069020 0x7f3db415cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:38222/0 (socket says 172.21.15.46:38222) 2023-10-06T11:19:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.574+0000 7f3db27fc700 1 -- 172.21.15.46:0/3588992851 learned_addr learned my addr 172.21.15.46:0/3588992851 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db37fe700 1 -- 172.21.15.46:0/3588992851 >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 msgr2=0x7f3db40b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db1ffb700 1 -- 172.21.15.46:0/3588992851 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154047174 0 0) 0x7f3db40cfc20 con 0x7f3db4069020 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db37fe700 1 --2- 172.21.15.46:0/3588992851 >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 0x7f3db40b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db37fe700 1 -- 172.21.15.46:0/3588992851 >> v1:172.21.15.132:6789/0 conn(0x7f3db4069020 legacy=0x7f3db415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db37fe700 1 -- 172.21.15.46:0/3588992851 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db40cf010 con 0x7f3db40b40b0 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db2ffd700 1 --2- 172.21.15.46:0/3588992851 >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 0x7f3db40b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db37fe700 1 --2- 172.21.15.46:0/3588992851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db4069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009e60 tx=0x7f3dac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa8b069790a63b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db1ffb700 1 -- 172.21.15.46:0/3588992851 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00e070 con 0x7f3db40b40b0 2023-10-06T11:19:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db1ffb700 1 -- 172.21.15.46:0/3588992851 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac00bb60 con 0x7f3db40b40b0 2023-10-06T11:19:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db1ffb700 1 -- 172.21.15.46:0/3588992851 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac012430 con 0x7f3db40b40b0 2023-10-06T11:19:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 -- 172.21.15.46:0/3588992851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 msgr2=0x7f3db4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 --2- 172.21.15.46:0/3588992851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db4069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009e60 tx=0x7f3dac009780 comp rx=0 tx=0).stop 2023-10-06T11:19:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 -- 172.21.15.46:0/3588992851 shutdown_connections 2023-10-06T11:19:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 --2- 172.21.15.46:0/3588992851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db4069470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 --2- 172.21.15.46:0/3588992851 >> v2:172.21.15.132:3300/0 conn(0x7f3db41591f0 0x7f3db40b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 -- 172.21.15.46:0/3588992851 >> 172.21.15.46:0/3588992851 conn(0x7f3db40d0b20 msgr2=0x7f3db40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 -- 172.21.15.46:0/3588992851 shutdown_connections 2023-10-06T11:19:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.578+0000 7f3db926e700 1 -- 172.21.15.46:0/3588992851 wait complete. 2023-10-06T11:19:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 Processor -- start 2023-10-06T11:19:34.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 -- start start 2023-10-06T11:19:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 0x7f3db41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db416a8d0 con 0x7f3db41591f0 2023-10-06T11:19:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4159e80 con 0x7f3db4069020 2023-10-06T11:19:34.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db926e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db415ca60 con 0x7f3db40b40b0 2023-10-06T11:19:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39272/0 (socket says 172.21.15.46:39272) 2023-10-06T11:19:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db37fe700 1 -- 172.21.15.46:0/2151550874 learned_addr learned my addr 172.21.15.46:0/2151550874 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db2ffd700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 0x7f3db41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.582+0000 7f3db27fc700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 -- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 msgr2=0x7f3db415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db415c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 -- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 msgr2=0x7f3db40b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 -- 172.21.15.46:0/2151550874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4009420 con 0x7f3db4069020 2023-10-06T11:19:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db37fe700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db2ffd700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 0x7f3db41595c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3da400ea00 tx=0x7f3da40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400c6a0 con 0x7f3db4069020 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da400c800 con 0x7f3db4069020 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac009a20 con 0x7f3db4069020 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da400cba0 con 0x7f3db4069020 2023-10-06T11:19:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.586+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db4159ba0 con 0x7f3db4069020 2023-10-06T11:19:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.590+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4064590 con 0x7f3db4069020 2023-10-06T11:19:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.590+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f3da4013470 con 0x7f3db4069020 2023-10-06T11:19:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.590+0000 7f3d9affd700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 0x7f3da0062d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.590+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3da407cf20 con 0x7f3db4069020 2023-10-06T11:19:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.594+0000 7f3db27fc700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 0x7f3da0062d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.594+0000 7f3db27fc700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 0x7f3da0062d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3db40b4c60 tx=0x7f3da8007d30 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:34.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.598+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3da404d010 con 0x7f3db4069020 2023-10-06T11:19:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.794+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3db40cf2f0 con 0x7f3da0060850 2023-10-06T11:19:34.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.882+0000 7f3d9affd700 1 -- 172.21.15.46:0/2151550874 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3db40cf2f0 con 0x7f3da0060850 2023-10-06T11:19:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-06T11:19:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 msgr2=0x7f3da0062d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 0x7f3da0062d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3db40b4c60 tx=0x7f3da8007d30 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 msgr2=0x7f3db41595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 0x7f3db41595c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3da400ea00 tx=0x7f3da40091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 shutdown_connections 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3da0060850 0x7f3da0062d00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3db41591f0 0x7f3db40b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db40b40b0 0x7f3db415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 --2- 172.21.15.46:0/2151550874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db4069020 0x7f3db41595c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.886+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 >> 172.21.15.46:0/2151550874 conn(0x7f3db40d0b20 msgr2=0x7f3db4155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.890+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 shutdown_connections 2023-10-06T11:19:34.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:34.890+0000 7f3db926e700 1 -- 172.21.15.46:0/2151550874 wait complete. 2023-10-06T11:19:34.915 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-06T11:19:34.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T11:19:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.086+0000 7fde3206b700 1 Processor -- start 2023-10-06T11:19:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.086+0000 7fde3206b700 1 -- start start 2023-10-06T11:19:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde3206b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde2c158de0 0x7fde2c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde3206b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde2c1591b0 0x7fde2c069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde3206b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2c0b5eb0 con 0x7fde2c0695a0 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde3206b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c0cef30 con 0x7fde2c1591b0 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde3206b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c0cf4f0 con 0x7fde2c158de0 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde2c1591b0 0x7fde2c069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde2c1591b0 0x7fde2c069060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39280/0 (socket says 172.21.15.46:39280) 2023-10-06T11:19:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 -- 172.21.15.46:0/2051124315 learned_addr learned my addr 172.21.15.46:0/2051124315 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde31069700 1 --2- 172.21.15.46:0/2051124315 >> v2:172.21.15.132:3300/0 conn(0x7fde2c158de0 0x7fde2c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 -- 172.21.15.46:0/2051124315 >> v2:172.21.15.132:3300/0 conn(0x7fde2c158de0 msgr2=0x7fde2c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 --2- 172.21.15.46:0/2051124315 >> v2:172.21.15.132:3300/0 conn(0x7fde2c158de0 0x7fde2c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 -- 172.21.15.46:0/2051124315 >> v1:172.21.15.46:6789/0 conn(0x7fde2c0695a0 legacy=0x7fde2c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.090+0000 7fde30868700 1 -- 172.21.15.46:0/2051124315 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c0699c0 con 0x7fde2c1591b0 2023-10-06T11:19:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.094+0000 7fde30868700 1 --2- 172.21.15.46:0/2051124315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c069060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fde2c088830 tx=0x7fde1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291e770af9b9b7a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde23fff700 1 -- 172.21.15.46:0/2051124315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c009e70 con 0x7fde2c1591b0 2023-10-06T11:19:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde23fff700 1 -- 172.21.15.46:0/2051124315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c00b860 con 0x7fde2c1591b0 2023-10-06T11:19:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde23fff700 1 -- 172.21.15.46:0/2051124315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00bc00 con 0x7fde2c1591b0 2023-10-06T11:19:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 -- 172.21.15.46:0/2051124315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 msgr2=0x7fde2c069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 --2- 172.21.15.46:0/2051124315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c069060 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fde2c088830 tx=0x7fde1c009250 comp rx=0 tx=0).stop 2023-10-06T11:19:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 -- 172.21.15.46:0/2051124315 shutdown_connections 2023-10-06T11:19:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 --2- 172.21.15.46:0/2051124315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c069060 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 --2- 172.21.15.46:0/2051124315 >> v2:172.21.15.132:3300/0 conn(0x7fde2c158de0 0x7fde2c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 -- 172.21.15.46:0/2051124315 >> 172.21.15.46:0/2051124315 conn(0x7fde2c068390 msgr2=0x7fde2c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 -- 172.21.15.46:0/2051124315 shutdown_connections 2023-10-06T11:19:35.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.098+0000 7fde3206b700 1 -- 172.21.15.46:0/2051124315 wait complete. 2023-10-06T11:19:35.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.102+0000 7fde3206b700 1 Processor -- start 2023-10-06T11:19:35.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.102+0000 7fde3206b700 1 -- start start 2023-10-06T11:19:35.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.102+0000 7fde3206b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 0x7fde2c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c16cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0c5b70 con 0x7fde2c1591b0 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c1546b0 con 0x7fde2c158de0 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c15b110 con 0x7fde2c0695a0 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde31069700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 0x7fde2c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54718/0 (socket says 172.21.15.46:54718) 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 -- 172.21.15.46:0/733555948 learned_addr learned my addr 172.21.15.46:0/733555948 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:35.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3186a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c16cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 -- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 msgr2=0x7fde2c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 0x7fde2c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 -- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 msgr2=0x7fde2c16cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c16cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 -- 172.21.15.46:0/733555948 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0094f0 con 0x7fde2c158de0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde31069700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 0x7fde2c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3186a700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c16cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde30868700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00bbe0 con 0x7fde2c158de0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c011420 con 0x7fde2c158de0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0117e0 con 0x7fde2c158de0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c0cc800 con 0x7fde2c158de0 2023-10-06T11:19:35.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.106+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c15aac0 con 0x7fde2c158de0 2023-10-06T11:19:35.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.110+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2c064590 con 0x7fde2c158de0 2023-10-06T11:19:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.110+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fde1c011c10 con 0x7fde2c158de0 2023-10-06T11:19:35.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.110+0000 7fde217fa700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 0x7fde10062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.114+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fde1c07e590 con 0x7fde2c158de0 2023-10-06T11:19:35.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.114+0000 7fde31069700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 0x7fde10062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.114+0000 7fde31069700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 0x7fde10062c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde28011480 tx=0x7fde280091a0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.118+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fde1c04e600 con 0x7fde2c158de0 2023-10-06T11:19:35.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.310+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fde2c15b250 con 0x7fde10060750 2023-10-06T11:19:35.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.398+0000 7fde217fa700 1 -- 172.21.15.46:0/733555948 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fde2c15b250 con 0x7fde10060750 2023-10-06T11:19:35.402 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-06T11:19:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 msgr2=0x7fde10062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 0x7fde10062c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde28011480 tx=0x7fde280091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 msgr2=0x7fde2c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006fd0 tx=0x7fde1c017000 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 shutdown_connections 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fde10060750 0x7fde10062c00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde2c1591b0 0x7fde2c16cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde2c158de0 0x7fde2c0ca140 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 --2- 172.21.15.46:0/733555948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde2c0695a0 0x7fde2c0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 >> 172.21.15.46:0/733555948 conn(0x7fde2c068390 msgr2=0x7fde2c15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 shutdown_connections 2023-10-06T11:19:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.402+0000 7fde3206b700 1 -- 172.21.15.46:0/733555948 wait complete. 2023-10-06T11:19:35.428 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-06T11:19:35.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T11:19:35.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 Processor -- start 2023-10-06T11:19:35.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 -- start start 2023-10-06T11:19:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4204158de0 0x7f42040b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42041591b0 0x7f4204069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42040b5eb0 con 0x7f42040695a0 2023-10-06T11:19:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4204069a20 con 0x7f4204158de0 2023-10-06T11:19:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.598+0000 7f420a5cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cef30 con 0x7f42041591b0 2023-10-06T11:19:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4204158de0 0x7f42040b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4204158de0 0x7f42040b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39298/0 (socket says 172.21.15.46:39298) 2023-10-06T11:19:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 -- 172.21.15.46:0/2584328408 learned_addr learned my addr 172.21.15.46:0/2584328408 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f4208dc9700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42041591b0 0x7f4204069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 -- 172.21.15.46:0/2584328408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42041591b0 msgr2=0x7f4204069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42041591b0 0x7f4204069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 -- 172.21.15.46:0/2584328408 >> v1:172.21.15.46:6789/0 conn(0x7f42040695a0 legacy=0x7f420415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 -- 172.21.15.46:0/2584328408 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040cf4f0 con 0x7f4204158de0 2023-10-06T11:19:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f42095ca700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4204158de0 0x7f42040b5970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42000099b0 tx=0x7f4200009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75b4abd37a55c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f41fbfff700 1 -- 172.21.15.46:0/2584328408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200009ea0 con 0x7f4204158de0 2023-10-06T11:19:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f41fbfff700 1 -- 172.21.15.46:0/2584328408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420000b860 con 0x7f4204158de0 2023-10-06T11:19:35.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.602+0000 7f41fbfff700 1 -- 172.21.15.46:0/2584328408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000bc00 con 0x7f4204158de0 2023-10-06T11:19:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 -- 172.21.15.46:0/2584328408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4204158de0 msgr2=0x7f42040b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4204158de0 0x7f42040b5970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42000099b0 tx=0x7f4200009250 comp rx=0 tx=0).stop 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 -- 172.21.15.46:0/2584328408 shutdown_connections 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42041591b0 0x7f4204069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 --2- 172.21.15.46:0/2584328408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4204158de0 0x7f42040b5970 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 -- 172.21.15.46:0/2584328408 >> 172.21.15.46:0/2584328408 conn(0x7f4204068390 msgr2=0x7f42040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 -- 172.21.15.46:0/2584328408 shutdown_connections 2023-10-06T11:19:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.606+0000 7f420a5cc700 1 -- 172.21.15.46:0/2584328408 wait complete. 2023-10-06T11:19:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 Processor -- start 2023-10-06T11:19:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 -- start start 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040695a0 0x7f42040c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4204158de0 0x7f42040c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040c9590 con 0x7f42041591b0 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cb5f0 con 0x7f4204158de0 2023-10-06T11:19:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f420a5cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42040caa20 con 0x7f42040695a0 2023-10-06T11:19:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f4209dcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f4209dcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39314/0 (socket says 172.21.15.46:39314) 2023-10-06T11:19:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f4209dcb700 1 -- 172.21.15.46:0/3319374429 learned_addr learned my addr 172.21.15.46:0/3319374429 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f42095ca700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040695a0 0x7f42040c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.610+0000 7f4208dc9700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4204158de0 0x7f42040c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 -- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040695a0 msgr2=0x7f42040c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040695a0 0x7f42040c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 -- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4204158de0 msgr2=0x7f42040c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4204158de0 0x7f42040c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 -- 172.21.15.46:0/3319374429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000094f0 con 0x7f42041591b0 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f4209dcb700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c910 tx=0x7f41fc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc012630 con 0x7f42041591b0 2023-10-06T11:19:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc0127b0 con 0x7f42041591b0 2023-10-06T11:19:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc012b70 con 0x7f42041591b0 2023-10-06T11:19:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42040cacc0 con 0x7f42041591b0 2023-10-06T11:19:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42040cb280 con 0x7f42041591b0 2023-10-06T11:19:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42040cb780 con 0x7f42041591b0 2023-10-06T11:19:35.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.614+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f41fc003030 con 0x7f42041591b0 2023-10-06T11:19:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.618+0000 7f41f97fa700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 0x7f41e8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.618+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f41fc07cb50 con 0x7f42041591b0 2023-10-06T11:19:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.618+0000 7f42095ca700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 0x7f41e8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:35.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.618+0000 7f42095ca700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 0x7f41e8062c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42000099b0 tx=0x7f4200017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:35.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.622+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f41fc04cbc0 con 0x7f42041591b0 2023-10-06T11:19:35.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.818+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f42040b5eb0 con 0x7f41e8060750 2023-10-06T11:19:35.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.910+0000 7f41f97fa700 1 -- 172.21.15.46:0/3319374429 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f42040b5eb0 con 0x7f41e8060750 2023-10-06T11:19:35.944 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-06T11:19:35.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 msgr2=0x7f41e8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 0x7f41e8062c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f42000099b0 tx=0x7f4200017000 comp rx=0 tx=0).stop 2023-10-06T11:19:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 msgr2=0x7f4204154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00c910 tx=0x7f41fc001710 comp rx=0 tx=0).stop 2023-10-06T11:19:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 shutdown_connections 2023-10-06T11:19:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f41e8060750 0x7f41e8062c00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42041591b0 0x7f4204154a70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4204158de0 0x7f42040c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 --2- 172.21.15.46:0/3319374429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040695a0 0x7f42040c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 >> 172.21.15.46:0/3319374429 conn(0x7f4204068390 msgr2=0x7f42040aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 shutdown_connections 2023-10-06T11:19:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:35.914+0000 7f420a5cc700 1 -- 172.21.15.46:0/3319374429 wait complete. 2023-10-06T11:19:35.947 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-06T11:19:35.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T11:19:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.114+0000 7f8013fff700 1 Processor -- start 2023-10-06T11:19:36.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.114+0000 7f8013fff700 1 -- start start 2023-10-06T11:19:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8014156bf0 0x7f80140cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f801406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f801406c8f0 0x7f801415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140cdcd0 con 0x7f8014156bf0 2023-10-06T11:19:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801406cd70 con 0x7f801406c8f0 2023-10-06T11:19:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8013fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80140cd4c0 con 0x7f8014156fc0 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8014156bf0 0x7f80140cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8014156bf0 0x7f80140cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39318/0 (socket says 172.21.15.46:39318) 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 -- 172.21.15.46:0/4060805243 learned_addr learned my addr 172.21.15.46:0/4060805243 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f80127fc700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f801406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f80137fe700 1 --2- 172.21.15.46:0/4060805243 >> v2:172.21.15.132:3300/0 conn(0x7f801406c8f0 0x7f801415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 -- 172.21.15.46:0/4060805243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 msgr2=0x7f801406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f801406c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 -- 172.21.15.46:0/4060805243 >> v2:172.21.15.132:3300/0 conn(0x7f801406c8f0 msgr2=0x7f801415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:36.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 --2- 172.21.15.46:0/4060805243 >> v2:172.21.15.132:3300/0 conn(0x7f801406c8f0 0x7f801415aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.118+0000 7f8012ffd700 1 -- 172.21.15.46:0/4060805243 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140ca2d0 con 0x7f8014156bf0 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8012ffd700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014156bf0 0x7f80140cc410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8014157410 tx=0x7f8004009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ce9712ef091cc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8011ffb700 1 -- 172.21.15.46:0/4060805243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400b820 con 0x7f8014156bf0 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8011ffb700 1 -- 172.21.15.46:0/4060805243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800400b980 con 0x7f8014156bf0 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8011ffb700 1 -- 172.21.15.46:0/4060805243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400bd20 con 0x7f8014156bf0 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 -- 172.21.15.46:0/4060805243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014156bf0 msgr2=0x7f80140cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014156bf0 0x7f80140cc410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8014157410 tx=0x7f8004009250 comp rx=0 tx=0).stop 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 -- 172.21.15.46:0/4060805243 shutdown_connections 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 --2- 172.21.15.46:0/4060805243 >> v2:172.21.15.132:3300/0 conn(0x7f801406c8f0 0x7f801415aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f801406c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 --2- 172.21.15.46:0/4060805243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014156bf0 0x7f80140cc410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 -- 172.21.15.46:0/4060805243 >> 172.21.15.46:0/4060805243 conn(0x7f801406ad70 msgr2=0x7f801406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 -- 172.21.15.46:0/4060805243 shutdown_connections 2023-10-06T11:19:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.122+0000 7f8013fff700 1 -- 172.21.15.46:0/4060805243 wait complete. 2023-10-06T11:19:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 Processor -- start 2023-10-06T11:19:36.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 -- start start 2023-10-06T11:19:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 0x7f8014152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f80141554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80140af980 con 0x7f801406c8f0 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014153a30 con 0x7f8014156bf0 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.126+0000 7f8013fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80140ae8e0 con 0x7f8014156fc0 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39320/0 (socket says 172.21.15.46:39320) 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 -- 172.21.15.46:0/3595350129 learned_addr learned my addr 172.21.15.46:0/3595350129 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f80137fe700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f80141554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f80127fc700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 0x7f8014152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 -- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 msgr2=0x7f80141554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f80141554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 -- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 msgr2=0x7f8014152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 0x7f8014152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 -- 172.21.15.46:0/3595350129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80040094f0 con 0x7f801406c8f0 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f80127fc700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 0x7f8014152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8012ffd700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f80040094c0 tx=0x7f8004017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004011e00 con 0x7f801406c8f0 2023-10-06T11:19:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800401a420 con 0x7f801406c8f0 2023-10-06T11:19:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800401a7e0 con 0x7f801406c8f0 2023-10-06T11:19:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80140aebc0 con 0x7f801406c8f0 2023-10-06T11:19:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.130+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80140af180 con 0x7f801406c8f0 2023-10-06T11:19:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.134+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014064590 con 0x7f801406c8f0 2023-10-06T11:19:36.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.134+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f800401ab90 con 0x7f801406c8f0 2023-10-06T11:19:36.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.134+0000 7f7ffaffd700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 0x7f8000062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.134+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f800407e7a0 con 0x7f801406c8f0 2023-10-06T11:19:36.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.138+0000 7f80127fc700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 0x7f8000062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.138+0000 7f80127fc700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 0x7f8000062c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8014156170 tx=0x7f8008008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.142+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f800404e810 con 0x7f801406c8f0 2023-10-06T11:19:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.334+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8014063190 con 0x7f8000060750 2023-10-06T11:19:36.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.426+0000 7f7ffaffd700 1 -- 172.21.15.46:0/3595350129 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8014063190 con 0x7f8000060750 2023-10-06T11:19:36.432 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 msgr2=0x7f8000062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 0x7f8000062c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8014156170 tx=0x7f8008008040 comp rx=0 tx=0).stop 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 msgr2=0x7f80140b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f80040094c0 tx=0x7f8004017000 comp rx=0 tx=0).stop 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 shutdown_connections 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8000060750 0x7f8000062c00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8014156fc0 0x7f80141554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8014156bf0 0x7f8014152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 --2- 172.21.15.46:0/3595350129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801406c8f0 0x7f80140b2cc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 >> 172.21.15.46:0/3595350129 conn(0x7f801406ad70 msgr2=0x7f80140c2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 shutdown_connections 2023-10-06T11:19:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.430+0000 7f8013fff700 1 -- 172.21.15.46:0/3595350129 wait complete. 2023-10-06T11:19:36.459 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-06T11:19:36.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T11:19:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.638+0000 7f167cc2e700 1 Processor -- start 2023-10-06T11:19:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.638+0000 7f167cc2e700 1 -- start start 2023-10-06T11:19:36.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.642+0000 7f167cc2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f167815a570 0x7f16780cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.642+0000 7f167cc2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f16780c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167cc2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780ca530 con 0x7f16780c91f0 2023-10-06T11:19:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167cc2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780ccdb0 con 0x7f167815a570 2023-10-06T11:19:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167cc2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780c9670 con 0x7f16780cc960 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f167815a570 0x7f16780cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f1676d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16780c91f0 0x7f167815ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39822/0 (socket says 172.21.15.46:39822) 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f1676d9d700 1 -- 172.21.15.46:0/2555888913 learned_addr learned my addr 172.21.15.46:0/2555888913 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f1675d9b700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f16780c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167559a700 1 -- 172.21.15.46:0/2555888913 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500362553 0 0) 0x7f16780ca530 con 0x7f16780c91f0 2023-10-06T11:19:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 -- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 msgr2=0x7f16780c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f16780c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 -- 172.21.15.46:0/2555888913 >> v1:172.21.15.46:6789/0 conn(0x7f16780c91f0 legacy=0x7f167815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 -- 172.21.15.46:0/2555888913 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780c9920 con 0x7f167815a570 2023-10-06T11:19:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167659c700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f167815a570 0x7f16780cc420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16680098d0 tx=0x7f1668009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f73c96117afed7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167559a700 1 -- 172.21.15.46:0/2555888913 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668005c10 con 0x7f167815a570 2023-10-06T11:19:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167559a700 1 -- 172.21.15.46:0/2555888913 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668005d70 con 0x7f167815a570 2023-10-06T11:19:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.646+0000 7f167559a700 1 -- 172.21.15.46:0/2555888913 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668006280 con 0x7f167815a570 2023-10-06T11:19:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f167815a570 msgr2=0x7f16780cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f167815a570 0x7f16780cc420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16680098d0 tx=0x7f1668009120 comp rx=0 tx=0).stop 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- 172.21.15.46:0/2555888913 shutdown_connections 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f16780c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 --2- 172.21.15.46:0/2555888913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f167815a570 0x7f16780cc420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- 172.21.15.46:0/2555888913 >> 172.21.15.46:0/2555888913 conn(0x7f1678097290 msgr2=0x7f1678067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- 172.21.15.46:0/2555888913 shutdown_connections 2023-10-06T11:19:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- 172.21.15.46:0/2555888913 wait complete. 2023-10-06T11:19:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 Processor -- start 2023-10-06T11:19:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.650+0000 7f167cc2e700 1 -- start start 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b2e50 con 0x7f167815a570 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b2240 con 0x7f16780c91f0 2023-10-06T11:19:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b2520 con 0x7f16780cc960 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1675d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1675d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:45216/0 (socket says 172.21.15.46:45216) 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54766/0 (socket says 172.21.15.46:54766) 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39332/0 (socket says 172.21.15.46:39332) 2023-10-06T11:19:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1675d9b700 1 -- 172.21.15.46:0/719970278 learned_addr learned my addr 172.21.15.46:0/719970278 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 -- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 msgr2=0x7f1678151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 -- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 msgr2=0x7f1678158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 -- 172.21.15.46:0/719970278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680093c0 con 0x7f167815a570 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1675d9b700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167659c700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f1676d9d700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f167000cb90 tx=0x7f16700018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000a780 con 0x7f167815a570 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167000a8e0 con 0x7f167815a570 2023-10-06T11:19:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000aca0 con 0x7f167815a570 2023-10-06T11:19:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16780b2800 con 0x7f167815a570 2023-10-06T11:19:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.654+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16781521d0 con 0x7f167815a570 2023-10-06T11:19:36.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.658+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16780b2fc0 con 0x7f167815a570 2023-10-06T11:19:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.658+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f1670007300 con 0x7f167815a570 2023-10-06T11:19:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.662+0000 7f16667fc700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 0x7f1658062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.662+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f167007cfe0 con 0x7f167815a570 2023-10-06T11:19:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.662+0000 7f167659c700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 0x7f1658062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.662+0000 7f167659c700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 0x7f1658062c30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1668009160 tx=0x7f1668016000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:36.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.666+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f167004d0f0 con 0x7f167815a570 2023-10-06T11:19:36.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.858+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f167806cf20 con 0x7f1658060780 2023-10-06T11:19:36.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.950+0000 7f16667fc700 1 -- 172.21.15.46:0/719970278 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f167806cf20 con 0x7f1658060780 2023-10-06T11:19:36.955 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 msgr2=0x7f1658062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 0x7f1658062c30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1668009160 tx=0x7f1668016000 comp rx=0 tx=0).stop 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 msgr2=0x7f1678153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f167000cb90 tx=0x7f16700018f0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 shutdown_connections 2023-10-06T11:19:36.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1658060780 0x7f1658062c30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f167815a570 0x7f1678153b60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16780cc960 0x7f1678151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 --2- 172.21.15.46:0/719970278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16780c91f0 0x7f1678158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 >> 172.21.15.46:0/719970278 conn(0x7f1678097290 msgr2=0x7f16780c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 shutdown_connections 2023-10-06T11:19:36.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:36.954+0000 7f167cc2e700 1 -- 172.21.15.46:0/719970278 wait complete. 2023-10-06T11:19:36.981 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-06T11:19:36.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T11:19:37.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.154+0000 7fe5c1aa5700 1 Processor -- start 2023-10-06T11:19:37.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.154+0000 7fe5c1aa5700 1 -- start start 2023-10-06T11:19:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5c1aa5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe5bc0b2750 0x7fe5bc06e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5c1aa5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc06a110 con 0x7fe5bc157250 2023-10-06T11:19:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5c1aa5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc06a3f0 con 0x7fe5bc156e80 2023-10-06T11:19:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5c1aa5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc06ec70 con 0x7fe5bc0b2750 2023-10-06T11:19:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5bbfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe5bc157250 0x7fe5bc0caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39834/0 (socket says 172.21.15.46:39834) 2023-10-06T11:19:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5bbfff700 1 -- 172.21.15.46:0/1702192829 learned_addr learned my addr 172.21.15.46:0/1702192829 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:37.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.158+0000 7fe5baffd700 1 --2- 172.21.15.46:0/1702192829 >> v2:172.21.15.132:3300/0 conn(0x7fe5bc0b2750 0x7fe5bc06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164024302 0 0) 0x7fe5bc06a3f0 con 0x7fe5bc156e80 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a4003570 con 0x7fe5bc156e80 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390682485 0 0) 0x7fe5bc06a110 con 0x7fe5bc157250 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5bc06a3f0 con 0x7fe5bc157250 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5baffd700 1 -- 172.21.15.46:0/1702192829 >> v1:172.21.15.132:6789/0 conn(0x7fe5bc156e80 legacy=0x7fe5bc0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5baffd700 1 -- 172.21.15.46:0/1702192829 >> v1:172.21.15.46:6789/0 conn(0x7fe5bc157250 legacy=0x7fe5bc0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5baffd700 1 -- 172.21.15.46:0/1702192829 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc06a6d0 con 0x7fe5bc0b2750 2023-10-06T11:19:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5baffd700 1 --2- 172.21.15.46:0/1702192829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc06e730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009910 tx=0x7fe5b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101cd4fb6954a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011630 con 0x7fe5bc0b2750 2023-10-06T11:19:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0011790 con 0x7fe5bc0b2750 2023-10-06T11:19:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5ba7fc700 1 -- 172.21.15.46:0/1702192829 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011b30 con 0x7fe5bc0b2750 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/1702192829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 msgr2=0x7fe5bc06e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/1702192829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc06e730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009910 tx=0x7fe5b00091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/1702192829 shutdown_connections 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/1702192829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc06e730 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/1702192829 >> 172.21.15.46:0/1702192829 conn(0x7fe5bc068c30 msgr2=0x7fe5bc0ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/1702192829 shutdown_connections 2023-10-06T11:19:37.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.162+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/1702192829 wait complete. 2023-10-06T11:19:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 Processor -- start 2023-10-06T11:19:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 -- start start 2023-10-06T11:19:37.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 0x7fe5bc1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc14fff0 con 0x7fe5bc156e80 2023-10-06T11:19:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc14f3e0 con 0x7fe5bc0b2750 2023-10-06T11:19:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5c1aa5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc14f6c0 con 0x7fe5bc157250 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39342/0 (socket says 172.21.15.46:39342) 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5baffd700 1 -- 172.21.15.46:0/941815206 learned_addr learned my addr 172.21.15.46:0/941815206 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.166+0000 7fe5bb7fe700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 0x7fe5bc1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 -- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 msgr2=0x7fe5bc1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 0x7fe5bc1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 -- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 msgr2=0x7fe5bc14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 -- 172.21.15.46:0/941815206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0009420 con 0x7fe5bc156e80 2023-10-06T11:19:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 0x7fe5bc1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5baffd700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0013000 tx=0x7fe5b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001f070 con 0x7fe5bc156e80 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b0000bc0 con 0x7fe5bc156e80 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc14f9a0 con 0x7fe5bc156e80 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0011b30 con 0x7fe5bc156e80 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc14b030 con 0x7fe5bc156e80 2023-10-06T11:19:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.170+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc150160 con 0x7fe5bc156e80 2023-10-06T11:19:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.174+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fe5b0023070 con 0x7fe5bc156e80 2023-10-06T11:19:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.174+0000 7fe5a37fe700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 0x7fe5a8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.174+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe5b007e110 con 0x7fe5bc156e80 2023-10-06T11:19:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.174+0000 7fe5bb7fe700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 0x7fe5a8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.178+0000 7fe5bb7fe700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 0x7fe5a8062c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc153ea0 tx=0x7fe5ac009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.182+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe5b004e180 con 0x7fe5bc156e80 2023-10-06T11:19:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.374+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5bc0647c0 con 0x7fe5a8060750 2023-10-06T11:19:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.462+0000 7fe5a37fe700 1 -- 172.21.15.46:0/941815206 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe5bc0647c0 con 0x7fe5a8060750 2023-10-06T11:19:37.466 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-06T11:19:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.462+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 msgr2=0x7fe5a8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.462+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 0x7fe5a8062c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc153ea0 tx=0x7fe5ac009040 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 msgr2=0x7fe5bc14e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0013000 tx=0x7fe5b0017000 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 shutdown_connections 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5a8060750 0x7fe5a8062c00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5bc157250 0x7fe5bc1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc156e80 0x7fe5bc14e990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 --2- 172.21.15.46:0/941815206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5bc0b2750 0x7fe5bc14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 >> 172.21.15.46:0/941815206 conn(0x7fe5bc068c30 msgr2=0x7fe5bc0caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 shutdown_connections 2023-10-06T11:19:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.466+0000 7fe5c1aa5700 1 -- 172.21.15.46:0/941815206 wait complete. 2023-10-06T11:19:37.494 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-06T11:19:37.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T11:19:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 Processor -- start 2023-10-06T11:19:37.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 -- start start 2023-10-06T11:19:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4118067760 0x7f41181572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118069910 0x7f4118157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4118158450 con 0x7f4118156f20 2023-10-06T11:19:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41180adde0 con 0x7f4118067760 2023-10-06T11:19:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411ea8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118068b50 con 0x7f4118069910 2023-10-06T11:19:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411d28b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4118067760 0x7f41181572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411e28d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118069910 0x7f4118157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411d28b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4118067760 0x7f41181572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54788/0 (socket says 172.21.15.46:54788) 2023-10-06T11:19:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.662+0000 7f411d28b700 1 -- 172.21.15.46:0/4212481643 learned_addr learned my addr 172.21.15.46:0/4212481643 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411d28b700 1 -- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118069910 msgr2=0x7f4118157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411d28b700 1 --2- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118069910 0x7f4118157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411d28b700 1 -- 172.21.15.46:0/4212481643 >> v1:172.21.15.132:6789/0 conn(0x7f4118156f20 legacy=0x7f4118067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411d28b700 1 -- 172.21.15.46:0/4212481643 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118068e30 con 0x7f4118067760 2023-10-06T11:19:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411d28b700 1 --2- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118067760 0x7f41181572f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41080099b0 tx=0x7f4108009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb36d4338c7374cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ca8a700 1 -- 172.21.15.46:0/4212481643 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108009ea0 con 0x7f4118067760 2023-10-06T11:19:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ca8a700 1 -- 172.21.15.46:0/4212481643 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410800b860 con 0x7f4118067760 2023-10-06T11:19:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ca8a700 1 -- 172.21.15.46:0/4212481643 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410800bc20 con 0x7f4118067760 2023-10-06T11:19:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 -- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118067760 msgr2=0x7f41181572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 --2- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118067760 0x7f41181572f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41080099b0 tx=0x7f4108009250 comp rx=0 tx=0).stop 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 -- 172.21.15.46:0/4212481643 shutdown_connections 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 --2- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118069910 0x7f4118157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 --2- 172.21.15.46:0/4212481643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118067760 0x7f41181572f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 -- 172.21.15.46:0/4212481643 >> 172.21.15.46:0/4212481643 conn(0x7f41180cbbb0 msgr2=0x7f41180cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 -- 172.21.15.46:0/4212481643 shutdown_connections 2023-10-06T11:19:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.666+0000 7f411ea8e700 1 -- 172.21.15.46:0/4212481643 wait complete. 2023-10-06T11:19:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.670+0000 7f411ea8e700 1 Processor -- start 2023-10-06T11:19:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.670+0000 7f411ea8e700 1 -- start start 2023-10-06T11:19:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118067760 0x7f41180c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 0x7f4118158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118158b40 con 0x7f4118156f20 2023-10-06T11:19:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118158de0 con 0x7f4118069910 2023-10-06T11:19:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41181590c0 con 0x7f4118067760 2023-10-06T11:19:37.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39348/0 (socket says 172.21.15.46:39348) 2023-10-06T11:19:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 -- 172.21.15.46:0/3887891757 learned_addr learned my addr 172.21.15.46:0/3887891757 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411d28b700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 0x7f4118158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411da8c700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118067760 0x7f41180c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 -- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118067760 msgr2=0x7f41180c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118067760 0x7f41180c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 -- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 msgr2=0x7f4118158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 0x7f4118158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 -- 172.21.15.46:0/3887891757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080094f0 con 0x7f4118156f20 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411d28b700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 0x7f4118158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411e28d700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4110009ef0 tx=0x7f411000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000cc50 con 0x7f4118156f20 2023-10-06T11:19:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000f040 con 0x7f4118156f20 2023-10-06T11:19:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41181593a0 con 0x7f4118156f20 2023-10-06T11:19:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110014480 con 0x7f4118156f20 2023-10-06T11:19:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.674+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4118153570 con 0x7f4118156f20 2023-10-06T11:19:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.678+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41181595e0 con 0x7f4118156f20 2023-10-06T11:19:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.678+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f4110014830 con 0x7f4118156f20 2023-10-06T11:19:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.682+0000 7f410dffb700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 0x7f40fc062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.682+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f411007dca0 con 0x7f4118156f20 2023-10-06T11:19:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.682+0000 7f411da8c700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 0x7f40fc062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.682+0000 7f411da8c700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 0x7f40fc062c30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41140037c0 tx=0x7f4114009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.690+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f411004dc00 con 0x7f4118156f20 2023-10-06T11:19:37.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.878+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f41180a9850 con 0x7f40fc060780 2023-10-06T11:19:37.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.970+0000 7f410dffb700 1 -- 172.21.15.46:0/3887891757 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4114003410 con 0x7f40fc060780 2023-10-06T11:19:37.976 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 msgr2=0x7f40fc062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 0x7f40fc062c30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41140037c0 tx=0x7f4114009040 comp rx=0 tx=0).stop 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 msgr2=0x7f4118153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4110009ef0 tx=0x7f411000b040 comp rx=0 tx=0).stop 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 shutdown_connections 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f40fc060780 0x7f40fc062c30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4118156f20 0x7f4118153030 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4118069910 0x7f4118158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 --2- 172.21.15.46:0/3887891757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4118067760 0x7f41180c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 >> 172.21.15.46:0/3887891757 conn(0x7f41180cbbb0 msgr2=0x7f41181572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 shutdown_connections 2023-10-06T11:19:37.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:37.974+0000 7f411ea8e700 1 -- 172.21.15.46:0/3887891757 wait complete. 2023-10-06T11:19:38.002 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-06T11:19:38.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T11:19:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.162+0000 7f66e759e700 1 Processor -- start 2023-10-06T11:19:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.162+0000 7f66e759e700 1 -- start start 2023-10-06T11:19:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 0x7f66e80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66e80cc500 0x7f66e80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e80cf360 con 0x7f66e81591f0 2023-10-06T11:19:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e80cff00 con 0x7f66e80cc500 2023-10-06T11:19:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e80b03c0 con 0x7f66e81595c0 2023-10-06T11:19:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66e80cc500 0x7f66e80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66e80cc500 0x7f66e80afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39360/0 (socket says 172.21.15.46:39360) 2023-10-06T11:19:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e5d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 0x7f66e80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 -- 172.21.15.46:0/2389176764 learned_addr learned my addr 172.21.15.46:0/2389176764 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 -- 172.21.15.46:0/2389176764 >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 msgr2=0x7f66e80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 --2- 172.21.15.46:0/2389176764 >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 0x7f66e80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 -- 172.21.15.46:0/2389176764 >> v1:172.21.15.132:6789/0 conn(0x7f66e81591f0 legacy=0x7f66e80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e6d9d700 1 -- 172.21.15.46:0/2389176764 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e815dd60 con 0x7f66e80cc500 2023-10-06T11:19:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.166+0000 7f66e5d9b700 1 --2- 172.21.15.46:0/2389176764 >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 0x7f66e80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:19:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e6d9d700 1 --2- 172.21.15.46:0/2389176764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80cc500 0x7f66e80afe80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66e8088580 tx=0x7f66e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa32b671a6078d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e559a700 1 -- 172.21.15.46:0/2389176764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000e070 con 0x7f66e80cc500 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e559a700 1 -- 172.21.15.46:0/2389176764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e000bab0 con 0x7f66e80cc500 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e559a700 1 -- 172.21.15.46:0/2389176764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00092e0 con 0x7f66e80cc500 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- 172.21.15.46:0/2389176764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80cc500 msgr2=0x7f66e80afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 --2- 172.21.15.46:0/2389176764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80cc500 0x7f66e80afe80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66e8088580 tx=0x7f66e0009840 comp rx=0 tx=0).stop 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- 172.21.15.46:0/2389176764 shutdown_connections 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 --2- 172.21.15.46:0/2389176764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e80cc500 0x7f66e80afe80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 --2- 172.21.15.46:0/2389176764 >> v2:172.21.15.132:3300/0 conn(0x7f66e81595c0 0x7f66e80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- 172.21.15.46:0/2389176764 >> 172.21.15.46:0/2389176764 conn(0x7f66e806ef60 msgr2=0x7f66e80d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- 172.21.15.46:0/2389176764 shutdown_connections 2023-10-06T11:19:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- 172.21.15.46:0/2389176764 wait complete. 2023-10-06T11:19:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 Processor -- start 2023-10-06T11:19:38.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.170+0000 7f66e759e700 1 -- start start 2023-10-06T11:19:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e80cc500 0x7f66e815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66e81591f0 0x7f66e816ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8099dc0 con 0x7f66e8169ee0 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e816b570 con 0x7f66e80cc500 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e816a960 con 0x7f66e81591f0 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39372/0 (socket says 172.21.15.46:39372) 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 -- 172.21.15.46:0/545180844 learned_addr learned my addr 172.21.15.46:0/545180844 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e5d9b700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66e81591f0 0x7f66e816ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e659c700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e80cc500 0x7f66e815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 -- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66e81591f0 msgr2=0x7f66e816ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66e81591f0 0x7f66e816ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 -- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e80cc500 msgr2=0x7f66e815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e80cc500 0x7f66e815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 -- 172.21.15.46:0/545180844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc009420 con 0x7f66e8169ee0 2023-10-06T11:19:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e6d9d700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66e8088770 tx=0x7f66e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00032f0 con 0x7f66e8169ee0 2023-10-06T11:19:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e0008ce0 con 0x7f66e8169ee0 2023-10-06T11:19:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e0009ae0 con 0x7f66e8169ee0 2023-10-06T11:19:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e001c660 con 0x7f66e8169ee0 2023-10-06T11:19:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.174+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e816af20 con 0x7f66e8169ee0 2023-10-06T11:19:38.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.178+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e816b700 con 0x7f66e8169ee0 2023-10-06T11:19:38.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.178+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f66e0003040 con 0x7f66e8169ee0 2023-10-06T11:19:38.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.182+0000 7f66d67fc700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 0x7f66c8062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.182+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f66e007e620 con 0x7f66e8169ee0 2023-10-06T11:19:38.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.182+0000 7f66e659c700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 0x7f66c8062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.182+0000 7f66e659c700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 0x7f66c8062c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66e8166900 tx=0x7f66d8009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.186+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f66e004e580 con 0x7f66e8169ee0 2023-10-06T11:19:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.378+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e8064590 con 0x7f66c8060780 2023-10-06T11:19:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.466+0000 7f66d67fc700 1 -- 172.21.15.46:0/545180844 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f66e8064590 con 0x7f66c8060780 2023-10-06T11:19:38.471 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-06T11:19:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 msgr2=0x7f66c8062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 0x7f66c8062c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66e8166900 tx=0x7f66d8009040 comp rx=0 tx=0).stop 2023-10-06T11:19:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 msgr2=0x7f66e816a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66e8088770 tx=0x7f66e0018000 comp rx=0 tx=0).stop 2023-10-06T11:19:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 shutdown_connections 2023-10-06T11:19:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66c8060780 0x7f66c8062c30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e8169ee0 0x7f66e816a380 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66e81591f0 0x7f66e816ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 --2- 172.21.15.46:0/545180844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e80cc500 0x7f66e815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 >> 172.21.15.46:0/545180844 conn(0x7f66e806ef60 msgr2=0x7f66e815df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 shutdown_connections 2023-10-06T11:19:38.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.470+0000 7f66e759e700 1 -- 172.21.15.46:0/545180844 wait complete. 2023-10-06T11:19:38.500 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-06T11:19:38.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T11:19:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.666+0000 7fbbce745700 1 Processor -- start 2023-10-06T11:19:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.666+0000 7fbbce745700 1 -- start start 2023-10-06T11:19:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbce745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbce745700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbbc8069190 0x7fbbc8069610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbce745700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80ce2c0 con 0x7fbbc8159500 2023-10-06T11:19:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbce745700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80afb30 con 0x7fbbc8069190 2023-10-06T11:19:38.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbce745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80cc090 con 0x7fbbc8159130 2023-10-06T11:19:38.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbbc8069190 0x7fbbc8069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbc77fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbbc8159500 0x7fbbc80aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39848/0 (socket says 172.21.15.46:39848) 2023-10-06T11:19:38.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbc77fe700 1 -- 172.21.15.46:0/2134769366 learned_addr learned my addr 172.21.15.46:0/2134769366 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:38.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 -- 172.21.15.46:0/2134769366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 msgr2=0x7fbbc80d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 -- 172.21.15.46:0/2134769366 >> v1:172.21.15.46:6789/0 conn(0x7fbbc8159500 legacy=0x7fbbc80aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 -- 172.21.15.46:0/2134769366 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80af570 con 0x7fbbc8069190 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/2134769366 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621660302 0 0) 0x7fbbc80ce2c0 con 0x7fbbc8159500 2023-10-06T11:19:38.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.670+0000 7fbbccce2700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc8069610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009e60 tx=0x7fbbc0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486a6e1165181670 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/2134769366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000e070 con 0x7fbbc8069190 2023-10-06T11:19:38.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/2134769366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000bb60 con 0x7fbbc8069190 2023-10-06T11:19:38.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/2134769366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0012430 con 0x7fbbc8069190 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 -- 172.21.15.46:0/2134769366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 msgr2=0x7fbbc8069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc8069610 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009e60 tx=0x7fbbc0009780 comp rx=0 tx=0).stop 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 -- 172.21.15.46:0/2134769366 shutdown_connections 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc8069610 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 --2- 172.21.15.46:0/2134769366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 -- 172.21.15.46:0/2134769366 >> 172.21.15.46:0/2134769366 conn(0x7fbbc8068510 msgr2=0x7fbbc80d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 -- 172.21.15.46:0/2134769366 shutdown_connections 2023-10-06T11:19:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.674+0000 7fbbce745700 1 -- 172.21.15.46:0/2134769366 wait complete. 2023-10-06T11:19:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 Processor -- start 2023-10-06T11:19:38.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 -- start start 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8159500 0x7fbbc815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80b0610 con 0x7fbbc8069190 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815f280 con 0x7fbbc8159500 2023-10-06T11:19:38.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbce745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815f560 con 0x7fbbc8159130 2023-10-06T11:19:38.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39380/0 (socket says 172.21.15.46:39380) 2023-10-06T11:19:38.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.678+0000 7fbbc7fff700 1 -- 172.21.15.46:0/2211999006 learned_addr learned my addr 172.21.15.46:0/2211999006 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbccce2700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8159500 0x7fbbc815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 -- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 msgr2=0x7fbbc80c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 -- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8159500 msgr2=0x7fbbc815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8159500 0x7fbbc815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 -- 172.21.15.46:0/2211999006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009a20 con 0x7fbbc8069190 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800ec80 tx=0x7fbbb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800c850 con 0x7fbbc8069190 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb800c9b0 con 0x7fbbc8069190 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800cd50 con 0x7fbbc8069190 2023-10-06T11:19:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc815f840 con 0x7fbbc8069190 2023-10-06T11:19:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.682+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80c9dc0 con 0x7fbbc8069190 2023-10-06T11:19:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc8088540 con 0x7fbbc8069190 2023-10-06T11:19:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fbbb8013680 con 0x7fbbc8069190 2023-10-06T11:19:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbaffff700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 0x7fbbb4062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fbbb807d0b0 con 0x7fbbc8069190 2023-10-06T11:19:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 0x7fbbb4062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:38.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.686+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 0x7fbbb4062c00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80cda10 tx=0x7fbbbc007f50 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:38.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.694+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbbb804d120 con 0x7fbbc8069190 2023-10-06T11:19:38.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.890+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbc804b5c0 con 0x7fbbb4060750 2023-10-06T11:19:38.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.974+0000 7fbbaffff700 1 -- 172.21.15.46:0/2211999006 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbbc804b5c0 con 0x7fbbb4060750 2023-10-06T11:19:38.980 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-06T11:19:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 msgr2=0x7fbbb4062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 0x7fbbb4062c00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80cda10 tx=0x7fbbbc007f50 comp rx=0 tx=0).stop 2023-10-06T11:19:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 msgr2=0x7fbbc80c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800ec80 tx=0x7fbbb80091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 shutdown_connections 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbb4060750 0x7fbbb4062c00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8159500 0x7fbbc815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8159130 0x7fbbc80c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 --2- 172.21.15.46:0/2211999006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc8069190 0x7fbbc80c74a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 >> 172.21.15.46:0/2211999006 conn(0x7fbbc8068510 msgr2=0x7fbbc80ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 shutdown_connections 2023-10-06T11:19:38.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:38.978+0000 7fbbce745700 1 -- 172.21.15.46:0/2211999006 wait complete. 2023-10-06T11:19:39.007 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-06T11:19:39.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T11:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.182+0000 7f8a2d123700 1 Processor -- start 2023-10-06T11:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.182+0000 7f8a2d123700 1 -- start start 2023-10-06T11:19:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2d123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a281591f0 0x7f8a280cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2d123700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a281595c0 0x7f8a280af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2d123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a280cf360 con 0x7f8a280cc500 2023-10-06T11:19:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2d123700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2815dd60 con 0x7f8a281595c0 2023-10-06T11:19:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2d123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280cea30 con 0x7f8a281591f0 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a1ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a281595c0 0x7f8a280af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2c922700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8a280cc500 0x7f8a2815ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39850/0 (socket says 172.21.15.46:39850) 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a1ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a281595c0 0x7f8a280af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39392/0 (socket says 172.21.15.46:39392) 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.186+0000 7f8a2c922700 1 -- 172.21.15.46:0/2050050549 learned_addr learned my addr 172.21.15.46:0/2050050549 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a27fff700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a281591f0 0x7f8a280cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a1ffff700 1 -- 172.21.15.46:0/2050050549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a281591f0 msgr2=0x7f8a280cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:19:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a1ffff700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a281591f0 0x7f8a280cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a1ffff700 1 -- 172.21.15.46:0/2050050549 >> v1:172.21.15.46:6789/0 conn(0x7f8a280cc500 legacy=0x7f8a2815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a1ffff700 1 -- 172.21.15.46:0/2050050549 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280ced10 con 0x7f8a281595c0 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a1ffff700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a281595c0 0x7f8a280af940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8a100098e0 tx=0x7f8a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f620027b8689b8cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a277fe700 1 -- 172.21.15.46:0/2050050549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10009e80 con 0x7f8a281595c0 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a277fe700 1 -- 172.21.15.46:0/2050050549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000b8f0 con 0x7f8a281595c0 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a277fe700 1 -- 172.21.15.46:0/2050050549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000bc70 con 0x7f8a281595c0 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 -- 172.21.15.46:0/2050050549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a281595c0 msgr2=0x7f8a280af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a281595c0 0x7f8a280af940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8a100098e0 tx=0x7f8a100091a0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 -- 172.21.15.46:0/2050050549 shutdown_connections 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a281595c0 0x7f8a280af940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 --2- 172.21.15.46:0/2050050549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a281591f0 0x7f8a280cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 -- 172.21.15.46:0/2050050549 >> 172.21.15.46:0/2050050549 conn(0x7f8a2806ef60 msgr2=0x7f8a280d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 -- 172.21.15.46:0/2050050549 shutdown_connections 2023-10-06T11:19:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.190+0000 7f8a2d123700 1 -- 172.21.15.46:0/2050050549 wait complete. 2023-10-06T11:19:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 Processor -- start 2023-10-06T11:19:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 -- start start 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a281591f0 0x7f8a28149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a280b0820 0x7f8a280b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280b1f10 con 0x7f8a280cc500 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280b1300 con 0x7f8a281591f0 2023-10-06T11:19:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a2d123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280b15e0 con 0x7f8a280b0820 2023-10-06T11:19:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39394/0 (socket says 172.21.15.46:39394) 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.194+0000 7f8a27fff700 1 -- 172.21.15.46:0/604712525 learned_addr learned my addr 172.21.15.46:0/604712525 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a2c922700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a280b0820 0x7f8a280b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a1ffff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a281591f0 0x7f8a28149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 -- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a280b0820 msgr2=0x7f8a280b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a280b0820 0x7f8a280b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 -- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a281591f0 msgr2=0x7f8a28149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a281591f0 0x7f8a28149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 -- 172.21.15.46:0/604712525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10009420 con 0x7f8a280cc500 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a27fff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d890 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a180125f0 con 0x7f8a280cc500 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a18012750 con 0x7f8a280cc500 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a280b18c0 con 0x7f8a280cc500 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18012af0 con 0x7f8a280cc500 2023-10-06T11:19:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2814afb0 con 0x7f8a280cc500 2023-10-06T11:19:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.198+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a28064590 con 0x7f8a280cc500 2023-10-06T11:19:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.202+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f8a1801e460 con 0x7f8a280cc500 2023-10-06T11:19:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.202+0000 7f8a24ff9700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 0x7f8a0c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:19:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.202+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8a1807d3e0 con 0x7f8a280cc500 2023-10-06T11:19:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.202+0000 7f8a1ffff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 0x7f8a0c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:19:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.202+0000 7f8a1ffff700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 0x7f8a0c062c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a100091e0 tx=0x7f8a10017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:19:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.210+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a1804d450 con 0x7f8a280cc500 2023-10-06T11:19:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.406+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a280b2050 con 0x7f8a0c060750 2023-10-06T11:19:39.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.490+0000 7f8a24ff9700 1 -- 172.21.15.46:0/604712525 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8a1000bba0 con 0x7f8a0c060750 2023-10-06T11:19:39.497 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-06T11:19:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 msgr2=0x7f8a0c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 0x7f8a0c062c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a100091e0 tx=0x7f8a10017000 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 msgr2=0x7f8a280b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d890 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 shutdown_connections 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a0c060750 0x7f8a0c062c00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a280b0820 0x7f8a280b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a281591f0 0x7f8a28149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 --2- 172.21.15.46:0/604712525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a280cc500 0x7f8a280b23a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.494+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 >> 172.21.15.46:0/604712525 conn(0x7f8a2806ef60 msgr2=0x7f8a2815e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.498+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 shutdown_connections 2023-10-06T11:19:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:19:39.498+0000 7f8a2d123700 1 -- 172.21.15.46:0/604712525 wait complete. 2023-10-06T11:19:39.524 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-06T11:19:44.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:19:44.902 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:20:00.423 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:20:00.418+0000 7f2c123f8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:20:00.423 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:20:00.418+0000 7f2c123f8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:20:00.423 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:20:00.418+0000 7f2c123f8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:20:00.423 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:20:00.418+0000 7f2c123f8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:20:00.423 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:20:00.418+0000 7f2c123f8700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:20:14.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:20:14.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:20:45.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:20:45.069 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:15.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:15.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:45.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:21:45.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:15.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:15.343 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:45.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:22:45.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:15.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:15.616 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:45.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:23:45.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:15.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:15.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:45.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:24:45.872 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:25:00.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:25:00.493+0000 7f2c11bf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:25:00.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:25:00.493+0000 7f2c11bf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:25:00.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:25:00.493+0000 7f2c11bf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:25:00.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:25:00.493+0000 7f2c11bf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:25:00.501 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:25:00.493+0000 7f2c11bf7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:25:15.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:25:15.952 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:25:46.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:25:46.073 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:26:16.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:26:16.164 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:26:46.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:26:46.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:27:16.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:27:16.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:27:46.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:27:46.402 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:28:16.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:28:16.470 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:28:46.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:28:46.552 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:29:16.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:29:16.657 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:29:46.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:29:46.739 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:30:16.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:30:16.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:30:20.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T11:30:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.691+0000 7fc38feae700 1 Processor -- start 2023-10-06T11:30:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.691+0000 7fc38feae700 1 -- start start 2023-10-06T11:30:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38feae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 0x7fc3880d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38feae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3880ce2c0 con 0x7fc388069190 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38feae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3880af570 con 0x7fc388159130 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38feae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880afb30 con 0x7fc388159500 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38e44b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc388069190 0x7fc3880ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60882/0 (socket says 172.21.15.46:60882) 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38e44b700 1 -- 172.21.15.46:0/44493669 learned_addr learned my addr 172.21.15.46:0/44493669 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38d449700 1 --2- 172.21.15.46:0/44493669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 0x7fc3880d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38d449700 1 -- 172.21.15.46:0/44493669 >> v1:172.21.15.132:6789/0 conn(0x7fc388159130 legacy=0x7fc3880aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38d449700 1 -- 172.21.15.46:0/44493669 >> v1:172.21.15.46:6789/0 conn(0x7fc388069190 legacy=0x7fc3880ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38d449700 1 -- 172.21.15.46:0/44493669 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3880af850 con 0x7fc388159500 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38cc48700 1 -- 172.21.15.46:0/44493669 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240442537 0 0) 0x7fc3880ce2c0 con 0x7fc388069190 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.695+0000 7fc38d449700 1 --2- 172.21.15.46:0/44493669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 0x7fc3880d0d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc378009910 tx=0x7fc3780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15eb642f4a4f9c5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38cc48700 1 -- 172.21.15.46:0/44493669 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800b860 con 0x7fc388159500 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38cc48700 1 -- 172.21.15.46:0/44493669 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37800b9c0 con 0x7fc388159500 2023-10-06T11:30:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38cc48700 1 -- 172.21.15.46:0/44493669 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800bd60 con 0x7fc388159500 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 -- 172.21.15.46:0/44493669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 msgr2=0x7fc3880d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 --2- 172.21.15.46:0/44493669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 0x7fc3880d0d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc378009910 tx=0x7fc3780091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 -- 172.21.15.46:0/44493669 shutdown_connections 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 --2- 172.21.15.46:0/44493669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388159500 0x7fc3880d0d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 -- 172.21.15.46:0/44493669 >> 172.21.15.46:0/44493669 conn(0x7fc388068510 msgr2=0x7fc3880d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 -- 172.21.15.46:0/44493669 shutdown_connections 2023-10-06T11:30:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.699+0000 7fc38feae700 1 -- 172.21.15.46:0/44493669 wait complete. 2023-10-06T11:30:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 Processor -- start 2023-10-06T11:30:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 -- start start 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388069190 0x7fc3880b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc388159500 0x7fc3880c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3881599e0 con 0x7fc388159130 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815a840 con 0x7fc388159500 2023-10-06T11:30:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38feae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815acd0 con 0x7fc388069190 2023-10-06T11:30:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38d449700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38d449700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48790/0 (socket says 172.21.15.46:48790) 2023-10-06T11:30:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38d449700 1 -- 172.21.15.46:0/2791591868 learned_addr learned my addr 172.21.15.46:0/2791591868 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38dc4a700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388069190 0x7fc3880b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.703+0000 7fc38e44b700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc388159500 0x7fc3880c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 -- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388069190 msgr2=0x7fc3880b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388069190 0x7fc3880b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 -- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc388159500 msgr2=0x7fc3880c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc388159500 0x7fc3880c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 -- 172.21.15.46:0/2791591868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc378009420 con 0x7fc388159130 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38d449700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b3c0 tx=0x7fc378017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801f070 con 0x7fc388159130 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378002f70 con 0x7fc388159130 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc388159c30 con 0x7fc388159130 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378025420 con 0x7fc388159130 2023-10-06T11:30:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.707+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38815a1f0 con 0x7fc388159130 2023-10-06T11:30:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.711+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc388088540 con 0x7fc388159130 2023-10-06T11:30:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.711+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fc378023070 con 0x7fc388159130 2023-10-06T11:30:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.711+0000 7fc37dffb700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 0x7fc36c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.711+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fc37807e290 con 0x7fc388159130 2023-10-06T11:30:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.715+0000 7fc38dc4a700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 0x7fc36c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.715+0000 7fc38dc4a700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 0x7fc36c062c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ce8a0 tx=0x7fc384009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.719+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc37804e2e0 con 0x7fc388159130 2023-10-06T11:30:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:20.915+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc38815a980 con 0x7fc36c060750 2023-10-06T11:30:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.043+0000 7fc37dffb700 1 -- 172.21.15.46:0/2791591868 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc38815a980 con 0x7fc36c060750 2023-10-06T11:30:21.052 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 msgr2=0x7fc36c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 0x7fc36c062c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ce8a0 tx=0x7fc384009040 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 msgr2=0x7fc388152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b3c0 tx=0x7fc378017000 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 shutdown_connections 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc36c060750 0x7fc36c062c00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc388159500 0x7fc3880c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc388159130 0x7fc388152ca0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 --2- 172.21.15.46:0/2791591868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc388069190 0x7fc3880b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 >> 172.21.15.46:0/2791591868 conn(0x7fc388068510 msgr2=0x7fc388156170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:21.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 shutdown_connections 2023-10-06T11:30:21.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.047+0000 7fc38feae700 1 -- 172.21.15.46:0/2791591868 wait complete. 2023-10-06T11:30:21.080 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-06T11:30:21.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T11:30:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.259+0000 7f017afc5700 1 Processor -- start 2023-10-06T11:30:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.259+0000 7f017afc5700 1 -- start start 2023-10-06T11:30:21.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f017afc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f01740aeba0 0x7f0174068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f017afc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0174159f20 con 0x7f0174068290 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f017afc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f017415a160 con 0x7f01740aba90 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f017afc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01740ad650 con 0x7f01740aeba0 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0178d61700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0174068290 0x7f0174159810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60890/0 (socket says 172.21.15.46:60890) 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0178d61700 1 -- 172.21.15.46:0/2956133240 learned_addr learned my addr 172.21.15.46:0/2956133240 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0173fff700 1 --2- 172.21.15.46:0/2956133240 >> v2:172.21.15.132:3300/0 conn(0x7f01740aeba0 0x7f0174068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f01737fe700 1 -- 172.21.15.46:0/2956133240 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800646948 0 0) 0x7f0168001110 con 0x7f0174068290 2023-10-06T11:30:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f01737fe700 1 -- 172.21.15.46:0/2956133240 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0164003570 con 0x7f0174068290 2023-10-06T11:30:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0173fff700 1 -- 172.21.15.46:0/2956133240 >> v1:172.21.15.132:6789/0 conn(0x7f01740aba90 legacy=0x7f017414f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0173fff700 1 -- 172.21.15.46:0/2956133240 >> v1:172.21.15.46:6789/0 conn(0x7f0174068290 legacy=0x7f0174159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0173fff700 1 -- 172.21.15.46:0/2956133240 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017415a440 con 0x7f01740aeba0 2023-10-06T11:30:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f0173fff700 1 --2- 172.21.15.46:0/2956133240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f0174068660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01600098e0 tx=0x7f01600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b28ade7a2bff8198 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f01737fe700 1 -- 172.21.15.46:0/2956133240 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000b640 con 0x7f01740aeba0 2023-10-06T11:30:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f01737fe700 1 -- 172.21.15.46:0/2956133240 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016000b7a0 con 0x7f01740aeba0 2023-10-06T11:30:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.263+0000 7f01737fe700 1 -- 172.21.15.46:0/2956133240 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000bb40 con 0x7f01740aeba0 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 -- 172.21.15.46:0/2956133240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 msgr2=0x7f0174068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 --2- 172.21.15.46:0/2956133240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f0174068660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01600098e0 tx=0x7f01600091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 -- 172.21.15.46:0/2956133240 shutdown_connections 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 --2- 172.21.15.46:0/2956133240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f0174068660 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 -- 172.21.15.46:0/2956133240 >> 172.21.15.46:0/2956133240 conn(0x7f017406d490 msgr2=0x7f017406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 -- 172.21.15.46:0/2956133240 shutdown_connections 2023-10-06T11:30:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.267+0000 7f017afc5700 1 -- 172.21.15.46:0/2956133240 wait complete. 2023-10-06T11:30:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 Processor -- start 2023-10-06T11:30:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 -- start start 2023-10-06T11:30:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0174068290 0x7f017416d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f01740c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017414ff60 con 0x7f01740aba90 2023-10-06T11:30:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017416dd90 con 0x7f01740aeba0 2023-10-06T11:30:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f017afc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0174158200 con 0x7f0174068290 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48806/0 (socket says 172.21.15.46:48806) 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 -- 172.21.15.46:0/2088235070 learned_addr learned my addr 172.21.15.46:0/2088235070 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0178d61700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0174068290 0x7f017416d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0179562700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f01740c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 -- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0174068290 msgr2=0x7f017416d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0174068290 0x7f017416d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 -- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 msgr2=0x7f01740c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f01740c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.271+0000 7f0173fff700 1 -- 172.21.15.46:0/2088235070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160009420 con 0x7f01740aba90 2023-10-06T11:30:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f0173fff700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f017406cb90 tx=0x7f0160017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000bb20 con 0x7f01740aba90 2023-10-06T11:30:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0160000c90 con 0x7f01740aba90 2023-10-06T11:30:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160011650 con 0x7f01740aba90 2023-10-06T11:30:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0174158d20 con 0x7f01740aba90 2023-10-06T11:30:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0174158480 con 0x7f01740aba90 2023-10-06T11:30:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0174155b30 con 0x7f01740aba90 2023-10-06T11:30:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.275+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f01600119c0 con 0x7f01740aba90 2023-10-06T11:30:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.279+0000 7f0170ff9700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 0x7f0158062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.279+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f016007e0e0 con 0x7f01740aba90 2023-10-06T11:30:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.279+0000 7f0178d61700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 0x7f0158062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.279+0000 7f0178d61700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 0x7f0158062c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0168001270 tx=0x7f0168007bf0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.287+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0174155b30 con 0x7f01740aba90 2023-10-06T11:30:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.479+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0174155b30 con 0x7f0158060750 2023-10-06T11:30:21.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.603+0000 7f0170ff9700 1 -- 172.21.15.46:0/2088235070 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0174155b30 con 0x7f0158060750 2023-10-06T11:30:21.610 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 msgr2=0x7f0158062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 0x7f0158062c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0168001270 tx=0x7f0168007bf0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 msgr2=0x7f0174157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f017406cb90 tx=0x7f0160017000 comp rx=0 tx=0).stop 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 shutdown_connections 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0158060750 0x7f0158062c00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01740aeba0 0x7f01740c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01740aba90 0x7f0174157cc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 --2- 172.21.15.46:0/2088235070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0174068290 0x7f017416d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 >> 172.21.15.46:0/2088235070 conn(0x7f017406d490 msgr2=0x7f01740aa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 shutdown_connections 2023-10-06T11:30:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.607+0000 7f017afc5700 1 -- 172.21.15.46:0/2088235070 wait complete. 2023-10-06T11:30:21.637 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-06T11:30:21.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T11:30:21.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.803+0000 7fe954a43700 1 Processor -- start 2023-10-06T11:30:21.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.803+0000 7fe954a43700 1 -- start start 2023-10-06T11:30:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.803+0000 7fe954a43700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9501595c0 0x7fe9500af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9500cc500 0x7fe9500afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9500cf360 con 0x7fe9501591f0 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500cff00 con 0x7fe9501595c0 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500b03c0 con 0x7fe9500cc500 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9501595c0 0x7fe9500af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9501591f0 0x7fe9500cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60900/0 (socket says 172.21.15.46:60900) 2023-10-06T11:30:21.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94f7fe700 1 -- 172.21.15.46:0/3297000046 learned_addr learned my addr 172.21.15.46:0/3297000046 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94ffff700 1 --2- 172.21.15.46:0/3297000046 >> v2:172.21.15.132:3300/0 conn(0x7fe9500cc500 0x7fe9500afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 -- 172.21.15.46:0/3297000046 >> v2:172.21.15.132:3300/0 conn(0x7fe9500cc500 msgr2=0x7fe9500afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 --2- 172.21.15.46:0/3297000046 >> v2:172.21.15.132:3300/0 conn(0x7fe9500cc500 0x7fe9500afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94e7fc700 1 -- 172.21.15.46:0/3297000046 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527845204 0 0) 0x7fe9500cf360 con 0x7fe9501591f0 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 -- 172.21.15.46:0/3297000046 >> v1:172.21.15.46:6789/0 conn(0x7fe9501591f0 legacy=0x7fe9500cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 -- 172.21.15.46:0/3297000046 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95015dd60 con 0x7fe9501595c0 2023-10-06T11:30:21.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94effd700 1 --2- 172.21.15.46:0/3297000046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9501595c0 0x7fe9500af940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe9500acdd0 tx=0x7fe9440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0f6d8d4188e817 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94e7fc700 1 -- 172.21.15.46:0/3297000046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400b740 con 0x7fe9501595c0 2023-10-06T11:30:21.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94e7fc700 1 -- 172.21.15.46:0/3297000046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94400b8a0 con 0x7fe9501595c0 2023-10-06T11:30:21.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe94e7fc700 1 -- 172.21.15.46:0/3297000046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400bc40 con 0x7fe9501595c0 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- 172.21.15.46:0/3297000046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9501595c0 msgr2=0x7fe9500af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 --2- 172.21.15.46:0/3297000046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9501595c0 0x7fe9500af940 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe9500acdd0 tx=0x7fe9440091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- 172.21.15.46:0/3297000046 shutdown_connections 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 --2- 172.21.15.46:0/3297000046 >> v2:172.21.15.132:3300/0 conn(0x7fe9500cc500 0x7fe9500afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 --2- 172.21.15.46:0/3297000046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9501595c0 0x7fe9500af940 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- 172.21.15.46:0/3297000046 >> 172.21.15.46:0/3297000046 conn(0x7fe95006ef60 msgr2=0x7fe9500d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- 172.21.15.46:0/3297000046 shutdown_connections 2023-10-06T11:30:21.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.807+0000 7fe954a43700 1 -- 172.21.15.46:0/3297000046 wait complete. 2023-10-06T11:30:21.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 Processor -- start 2023-10-06T11:30:21.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 -- start start 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 0x7fe9501569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe95016c000 0x7fe950156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe950168df0 con 0x7fe9500cc500 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95016bb70 con 0x7fe9501591f0 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe954a43700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95016af60 con 0x7fe95016c000 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe94f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48832/0 (socket says 172.21.15.46:48832) 2023-10-06T11:30:21.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.811+0000 7fe94f7fe700 1 -- 172.21.15.46:0/2029767654 learned_addr learned my addr 172.21.15.46:0/2029767654 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94ffff700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe95016c000 0x7fe950156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94effd700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 0x7fe9501569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 -- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe95016c000 msgr2=0x7fe950156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe95016c000 0x7fe950156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 -- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 msgr2=0x7fe9501569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 0x7fe9501569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 -- 172.21.15.46:0/2029767654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe944009420 con 0x7fe9500cc500 2023-10-06T11:30:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94effd700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 0x7fe9501569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe94f7fe700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c7e0 tx=0x7fe940007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940005b30 con 0x7fe9500cc500 2023-10-06T11:30:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe940005c90 con 0x7fe9500cc500 2023-10-06T11:30:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940007300 con 0x7fe9500cc500 2023-10-06T11:30:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95016b240 con 0x7fe9500cc500 2023-10-06T11:30:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95016b800 con 0x7fe9500cc500 2023-10-06T11:30:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe950064590 con 0x7fe9500cc500 2023-10-06T11:30:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.815+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fe9400076b0 con 0x7fe9500cc500 2023-10-06T11:30:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.819+0000 7fe9377fe700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 0x7fe93c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:21.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.819+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fe94007c950 con 0x7fe9500cc500 2023-10-06T11:30:21.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.819+0000 7fe94effd700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 0x7fe93c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:21.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.819+0000 7fe94effd700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 0x7fe93c062c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe950067ce0 tx=0x7fe944017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:21.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:21.827+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe94004ca20 con 0x7fe9500cc500 2023-10-06T11:30:22.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.019+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe95004b5c0 con 0x7fe93c060750 2023-10-06T11:30:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.143+0000 7fe9377fe700 1 -- 172.21.15.46:0/2029767654 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe95004b5c0 con 0x7fe93c060750 2023-10-06T11:30:22.151 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 msgr2=0x7fe93c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 0x7fe93c062c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe950067ce0 tx=0x7fe944017000 comp rx=0 tx=0).stop 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 msgr2=0x7fe9501688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe94000c7e0 tx=0x7fe940007df0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 shutdown_connections 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe93c060750 0x7fe93c062c00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe95016c000 0x7fe950156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9501591f0 0x7fe9501569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 --2- 172.21.15.46:0/2029767654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9500cc500 0x7fe9501688b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 >> 172.21.15.46:0/2029767654 conn(0x7fe95006ef60 msgr2=0x7fe950157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 shutdown_connections 2023-10-06T11:30:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.147+0000 7fe954a43700 1 -- 172.21.15.46:0/2029767654 wait complete. 2023-10-06T11:30:22.176 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-06T11:30:22.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T11:30:22.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 Processor -- start 2023-10-06T11:30:22.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 -- start start 2023-10-06T11:30:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 0x7f4b900b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900b2600 con 0x7f4b90097c60 2023-10-06T11:30:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900b3a10 con 0x7f4b900b0500 2023-10-06T11:30:22.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.347+0000 7f4b9526e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900a8ab0 con 0x7f4b900b1090 2023-10-06T11:30:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8e7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b90097c60 0x7f4b900afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60916/0 (socket says 172.21.15.46:60916) 2023-10-06T11:30:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8e7fc700 1 -- 172.21.15.46:0/901004194 learned_addr learned my addr 172.21.15.46:0/901004194 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:22.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8effd700 1 --2- 172.21.15.46:0/901004194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 0x7f4b900b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286437179 0 0) 0x7f4b900b2600 con 0x7f4b90097c60 2023-10-06T11:30:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b78003570 con 0x7f4b90097c60 2023-10-06T11:30:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966227460 0 0) 0x7f4b900b3a10 con 0x7f4b900b0500 2023-10-06T11:30:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b900b2600 con 0x7f4b900b0500 2023-10-06T11:30:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8effd700 1 -- 172.21.15.46:0/901004194 >> v1:172.21.15.132:6789/0 conn(0x7f4b900b0500 legacy=0x7f4b900aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8effd700 1 -- 172.21.15.46:0/901004194 >> v1:172.21.15.46:6789/0 conn(0x7f4b90097c60 legacy=0x7f4b900afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8effd700 1 -- 172.21.15.46:0/901004194 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900b0980 con 0x7f4b900b1090 2023-10-06T11:30:22.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8effd700 1 --2- 172.21.15.46:0/901004194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 0x7f4b900b1460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009900 tx=0x7f4b80009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a7fd5e21375742 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80005c10 con 0x7f4b900b1090 2023-10-06T11:30:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b80005d70 con 0x7f4b900b1090 2023-10-06T11:30:22.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.351+0000 7f4b8dffb700 1 -- 172.21.15.46:0/901004194 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80006260 con 0x7f4b900b1090 2023-10-06T11:30:22.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 -- 172.21.15.46:0/901004194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 msgr2=0x7f4b900b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 --2- 172.21.15.46:0/901004194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 0x7f4b900b1460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009900 tx=0x7f4b80009120 comp rx=0 tx=0).stop 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 -- 172.21.15.46:0/901004194 shutdown_connections 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 --2- 172.21.15.46:0/901004194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b1090 0x7f4b900b1460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 -- 172.21.15.46:0/901004194 >> 172.21.15.46:0/901004194 conn(0x7f4b90067830 msgr2=0x7f4b900ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 -- 172.21.15.46:0/901004194 shutdown_connections 2023-10-06T11:30:22.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 -- 172.21.15.46:0/901004194 wait complete. 2023-10-06T11:30:22.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.355+0000 7f4b9526e700 1 Processor -- start 2023-10-06T11:30:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 -- start start 2023-10-06T11:30:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b0500 0x7f4b9015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b900b1090 0x7f4b900c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015bf00 con 0x7f4b90097c60 2023-10-06T11:30:22.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015d070 con 0x7f4b900b1090 2023-10-06T11:30:22.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9015b660 con 0x7f4b900b0500 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48844/0 (socket says 172.21.15.46:48844) 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 -- 172.21.15.46:0/3855239056 learned_addr learned my addr 172.21.15.46:0/3855239056 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8e7fc700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b0500 0x7f4b9015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8f7fe700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b900b1090 0x7f4b900c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 -- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b0500 msgr2=0x7f4b9015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b0500 0x7f4b9015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 -- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b900b1090 msgr2=0x7f4b900c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b900b1090 0x7f4b900c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 -- 172.21.15.46:0/3855239056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800093c0 con 0x7f4b90097c60 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b8effd700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009390 tx=0x7f4b80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8001d7b0 con 0x7f4b90097c60 2023-10-06T11:30:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8001d930 con 0x7f4b90097c60 2023-10-06T11:30:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9015b940 con 0x7f4b90097c60 2023-10-06T11:30:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.359+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8001dcf0 con 0x7f4b90097c60 2023-10-06T11:30:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.363+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9015b370 con 0x7f4b90097c60 2023-10-06T11:30:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.363+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9015ce40 con 0x7f4b90097c60 2023-10-06T11:30:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.363+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f4b8001b690 con 0x7f4b90097c60 2023-10-06T11:30:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.367+0000 7f4b76ffd700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 0x7f4b7c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.367+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f4b8007e990 con 0x7f4b90097c60 2023-10-06T11:30:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.367+0000 7f4b8e7fc700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 0x7f4b7c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.367+0000 7f4b8e7fc700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 0x7f4b7c062c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b9015a500 tx=0x7f4b84009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.375+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b9015ce40 con 0x7f4b90097c60 2023-10-06T11:30:22.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.567+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b900b3270 con 0x7f4b7c060750 2023-10-06T11:30:22.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.683+0000 7f4b76ffd700 1 -- 172.21.15.46:0/3855239056 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4b900b3270 con 0x7f4b7c060750 2023-10-06T11:30:22.694 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-06T11:30:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 msgr2=0x7f4b7c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 0x7f4b7c062c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b9015a500 tx=0x7f4b84009040 comp rx=0 tx=0).stop 2023-10-06T11:30:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 msgr2=0x7f4b9015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009390 tx=0x7f4b80016000 comp rx=0 tx=0).stop 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 shutdown_connections 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b7c060750 0x7f4b7c062c00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b900b1090 0x7f4b900c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b900b0500 0x7f4b9015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 --2- 172.21.15.46:0/3855239056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b90097c60 0x7f4b9015c860 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 >> 172.21.15.46:0/3855239056 conn(0x7f4b90067830 msgr2=0x7f4b9006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 shutdown_connections 2023-10-06T11:30:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.691+0000 7f4b9526e700 1 -- 172.21.15.46:0/3855239056 wait complete. 2023-10-06T11:30:22.724 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-06T11:30:22.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T11:30:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.891+0000 7f692866e700 1 Processor -- start 2023-10-06T11:30:22.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.891+0000 7f692866e700 1 -- start start 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f692866e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f69200b2600 0x7f69200ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f692866e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200cae50 0x7f69200c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f692866e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200cbff0 con 0x7f69200b29d0 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f692866e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69201598f0 con 0x7f69200cae50 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f692866e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200cb3e0 con 0x7f69200b2600 2023-10-06T11:30:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f6926c0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200cae50 0x7f69200c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f6926c0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69200cae50 0x7f69200c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48846/0 (socket says 172.21.15.46:48846) 2023-10-06T11:30:22.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.895+0000 7f6926c0b700 1 -- 172.21.15.46:0/3592216002 learned_addr learned my addr 172.21.15.46:0/3592216002 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:22.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692640a700 1 --2- 172.21.15.46:0/3592216002 >> v2:172.21.15.132:3300/0 conn(0x7f69200b2600 0x7f69200ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6926c0b700 1 -- 172.21.15.46:0/3592216002 >> v2:172.21.15.132:3300/0 conn(0x7f69200b2600 msgr2=0x7f69200ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6926c0b700 1 --2- 172.21.15.46:0/3592216002 >> v2:172.21.15.132:3300/0 conn(0x7f69200b2600 0x7f69200ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6926c0b700 1 -- 172.21.15.46:0/3592216002 >> v1:172.21.15.132:6789/0 conn(0x7f69200b29d0 legacy=0x7f69200ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6925408700 1 -- 172.21.15.46:0/3592216002 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728453363 0 0) 0x7f69200cbff0 con 0x7f69200b29d0 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6926c0b700 1 -- 172.21.15.46:0/3592216002 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200cb6c0 con 0x7f69200cae50 2023-10-06T11:30:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6926c0b700 1 --2- 172.21.15.46:0/3592216002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200cae50 0x7f69200c4ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f69200679d0 tx=0x7f6918009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28cd9170fe43a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6925408700 1 -- 172.21.15.46:0/3592216002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800e070 con 0x7f69200cae50 2023-10-06T11:30:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6925408700 1 -- 172.21.15.46:0/3592216002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691800ba20 con 0x7f69200cae50 2023-10-06T11:30:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f6925408700 1 -- 172.21.15.46:0/3592216002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180092e0 con 0x7f69200cae50 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 -- 172.21.15.46:0/3592216002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200cae50 msgr2=0x7f69200c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 --2- 172.21.15.46:0/3592216002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200cae50 0x7f69200c4ff0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f69200679d0 tx=0x7f6918009840 comp rx=0 tx=0).stop 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 -- 172.21.15.46:0/3592216002 shutdown_connections 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 --2- 172.21.15.46:0/3592216002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200cae50 0x7f69200c4ff0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 --2- 172.21.15.46:0/3592216002 >> v2:172.21.15.132:3300/0 conn(0x7f69200b2600 0x7f69200ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 -- 172.21.15.46:0/3592216002 >> 172.21.15.46:0/3592216002 conn(0x7f692006ef60 msgr2=0x7f69200d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:22.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 -- 172.21.15.46:0/3592216002 shutdown_connections 2023-10-06T11:30:22.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.899+0000 7f692866e700 1 -- 172.21.15.46:0/3592216002 wait complete. 2023-10-06T11:30:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 Processor -- start 2023-10-06T11:30:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 -- start start 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69200b29d0 0x7f69200b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 0x7f69200c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b4910 con 0x7f69200b2600 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b5be0 con 0x7f69200cae50 2023-10-06T11:30:22.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692866e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b4070 con 0x7f69200b29d0 2023-10-06T11:30:22.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f6926c0b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 0x7f69200c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48850/0 (socket says 172.21.15.46:48850) 2023-10-06T11:30:22.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 -- 172.21.15.46:0/1208910546 learned_addr learned my addr 172.21.15.46:0/1208910546 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:22.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f6925c09700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69200b29d0 0x7f69200b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 -- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69200b29d0 msgr2=0x7f69200b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69200b29d0 0x7f69200b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 -- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 msgr2=0x7f69200c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 0x7f69200c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:22.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f692640a700 1 -- 172.21.15.46:0/1208910546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6918009ae0 con 0x7f69200b2600 2023-10-06T11:30:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.903+0000 7f6926c0b700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 0x7f69200c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f692640a700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ea00 tx=0x7f691c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00c730 con 0x7f69200b2600 2023-10-06T11:30:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c00c890 con 0x7f69200b2600 2023-10-06T11:30:22.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69200b4350 con 0x7f69200b2600 2023-10-06T11:30:22.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00cc10 con 0x7f69200b2600 2023-10-06T11:30:22.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6920169290 con 0x7f69200b2600 2023-10-06T11:30:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.907+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6920069b50 con 0x7f69200b2600 2023-10-06T11:30:22.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.911+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f691c013470 con 0x7f69200b2600 2023-10-06T11:30:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.911+0000 7f69167fc700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 0x7f6904062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.911+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f691c07d430 con 0x7f69200b2600 2023-10-06T11:30:22.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.911+0000 7f6925c09700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 0x7f6904062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:22.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.911+0000 7f6925c09700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 0x7f6904062c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69200b2050 tx=0x7f6910007d30 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:22.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:22.919+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f691c04d480 con 0x7f69200b2600 2023-10-06T11:30:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.111+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f69200cb9a0 con 0x7f6904060750 2023-10-06T11:30:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.247+0000 7f69167fc700 1 -- 172.21.15.46:0/1208910546 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f69200cb9a0 con 0x7f6904060750 2023-10-06T11:30:23.255 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 msgr2=0x7f6904062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 0x7f6904062c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f69200b2050 tx=0x7f6910007d30 comp rx=0 tx=0).stop 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 msgr2=0x7f69200b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ea00 tx=0x7f691c009120 comp rx=0 tx=0).stop 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 shutdown_connections 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6904060750 0x7f6904062c00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200cae50 0x7f69200c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69200b29d0 0x7f69200b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 --2- 172.21.15.46:0/1208910546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69200b2600 0x7f69200b39a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 >> 172.21.15.46:0/1208910546 conn(0x7f692006ef60 msgr2=0x7f69200d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 shutdown_connections 2023-10-06T11:30:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.251+0000 7f692866e700 1 -- 172.21.15.46:0/1208910546 wait complete. 2023-10-06T11:30:23.280 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-06T11:30:23.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T11:30:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.443+0000 7f1c22fcc700 1 Processor -- start 2023-10-06T11:30:23.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.443+0000 7f1c22fcc700 1 -- start start 2023-10-06T11:30:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c22fcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c1c1595c0 0x7f1c1c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c22fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c22fcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1c0cf360 con 0x7f1c1c1591f0 2023-10-06T11:30:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c22fcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c0cff00 con 0x7f1c1c1595c0 2023-10-06T11:30:23.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c22fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0b03c0 con 0x7f1c1c0cc500 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c1c1595c0 0x7f1c1c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c1c1595c0 0x7f1c1c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34494/0 (socket says 172.21.15.46:34494) 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 -- 172.21.15.46:0/2566985027 learned_addr learned my addr 172.21.15.46:0/2566985027 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c227cb700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 -- 172.21.15.46:0/2566985027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 msgr2=0x7f1c1c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 -- 172.21.15.46:0/2566985027 >> v1:172.21.15.132:6789/0 conn(0x7f1c1c1591f0 legacy=0x7f1c1c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 -- 172.21.15.46:0/2566985027 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c15dd60 con 0x7f1c1c1595c0 2023-10-06T11:30:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c217c9700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c1595c0 0x7f1c1c0af940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0099b0 tx=0x7f1c0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f455ed7e9c2a0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c20fc8700 1 -- 172.21.15.46:0/2566985027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c009ea0 con 0x7f1c1c1595c0 2023-10-06T11:30:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c20fc8700 1 -- 172.21.15.46:0/2566985027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c00b8f0 con 0x7f1c1c1595c0 2023-10-06T11:30:23.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.447+0000 7f1c20fc8700 1 -- 172.21.15.46:0/2566985027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00bc90 con 0x7f1c1c1595c0 2023-10-06T11:30:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2566985027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c1595c0 msgr2=0x7f1c1c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c1595c0 0x7f1c1c0af940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0099b0 tx=0x7f1c0c009250 comp rx=0 tx=0).stop 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2566985027 shutdown_connections 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2566985027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c1595c0 0x7f1c1c0af940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2566985027 >> 172.21.15.46:0/2566985027 conn(0x7f1c1c06ef60 msgr2=0x7f1c1c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2566985027 shutdown_connections 2023-10-06T11:30:23.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2566985027 wait complete. 2023-10-06T11:30:23.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 Processor -- start 2023-10-06T11:30:23.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.451+0000 7f1c22fcc700 1 -- start start 2023-10-06T11:30:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 0x7f1c1c155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c156b80 con 0x7f1c1c157010 2023-10-06T11:30:23.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c155cd0 con 0x7f1c1c1591f0 2023-10-06T11:30:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c155f70 con 0x7f1c1c0cc500 2023-10-06T11:30:23.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34498/0 (socket says 172.21.15.46:34498) 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 -- 172.21.15.46:0/2328951445 learned_addr learned my addr 172.21.15.46:0/2328951445 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c21fca700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c217c9700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 0x7f1c1c155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 -- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 msgr2=0x7f1c1c158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 -- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 msgr2=0x7f1c1c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 0x7f1c1c155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 -- 172.21.15.46:0/2328951445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0094f0 con 0x7f1c1c157010 2023-10-06T11:30:23.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c217c9700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 0x7f1c1c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c227cb700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009df0 tx=0x7f1c1400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400caa0 con 0x7f1c1c157010 2023-10-06T11:30:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400f040 con 0x7f1c1c157010 2023-10-06T11:30:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c156250 con 0x7f1c1c157010 2023-10-06T11:30:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400ce70 con 0x7f1c1c157010 2023-10-06T11:30:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.455+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c156810 con 0x7f1c1c157010 2023-10-06T11:30:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.459+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c156cf0 con 0x7f1c1c157010 2023-10-06T11:30:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.459+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f1c14014690 con 0x7f1c1c157010 2023-10-06T11:30:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.459+0000 7f1c11ffb700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 0x7f1c00062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.459+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f1c1407d760 con 0x7f1c1c157010 2023-10-06T11:30:23.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.459+0000 7f1c21fca700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 0x7f1c00062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.463+0000 7f1c21fca700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 0x7f1c00062c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c18003950 tx=0x7f1c18007ec0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.467+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c14051130 con 0x7f1c1c157010 2023-10-06T11:30:23.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.655+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c1c064590 con 0x7f1c00060750 2023-10-06T11:30:23.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.779+0000 7f1c11ffb700 1 -- 172.21.15.46:0/2328951445 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1c1c064590 con 0x7f1c00060750 2023-10-06T11:30:23.787 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 msgr2=0x7f1c00062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 0x7f1c00062c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c18003950 tx=0x7f1c18007ec0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 msgr2=0x7f1c1c1574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009df0 tx=0x7f1c1400b040 comp rx=0 tx=0).stop 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 shutdown_connections 2023-10-06T11:30:23.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c00060750 0x7f1c00062c00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c1c157010 0x7f1c1c1574b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c1591f0 0x7f1c1c155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 --2- 172.21.15.46:0/2328951445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c1c0cc500 0x7f1c1c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 >> 172.21.15.46:0/2328951445 conn(0x7f1c1c06ef60 msgr2=0x7f1c1c0b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 shutdown_connections 2023-10-06T11:30:23.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.783+0000 7f1c22fcc700 1 -- 172.21.15.46:0/2328951445 wait complete. 2023-10-06T11:30:23.813 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-06T11:30:23.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T11:30:23.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.979+0000 7f6f12763700 1 Processor -- start 2023-10-06T11:30:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.983+0000 7f6f12763700 1 -- start start 2023-10-06T11:30:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f12763700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6f0c1591f0 0x7f6f0c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f12763700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f12763700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c0cf360 con 0x7f6f0c1595c0 2023-10-06T11:30:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f12763700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c15dd60 con 0x7f6f0c1591f0 2023-10-06T11:30:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f12763700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0cea30 con 0x7f6f0c0cc500 2023-10-06T11:30:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f10f60700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6f0c1595c0 0x7f6f0c0af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43388/0 (socket says 172.21.15.46:43388) 2023-10-06T11:30:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f10f60700 1 -- 172.21.15.46:0/1391060573 learned_addr learned my addr 172.21.15.46:0/1391060573 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 --2- 172.21.15.46:0/1391060573 >> v2:172.21.15.132:3300/0 conn(0x7f6f0c1591f0 0x7f6f0c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11f62700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 -- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 msgr2=0x7f6f0c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f03fff700 1 -- 172.21.15.46:0/1391060573 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029670443 0 0) 0x7f6f0c0cf360 con 0x7f6f0c1595c0 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c15ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 -- 172.21.15.46:0/1391060573 >> v1:172.21.15.46:6789/0 conn(0x7f6f0c1595c0 legacy=0x7f6f0c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 -- 172.21.15.46:0/1391060573 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0ced10 con 0x7f6f0c1591f0 2023-10-06T11:30:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11f62700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f11761700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c0cf7f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098e0 tx=0x7f6f080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ba32e359e08d02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f03fff700 1 -- 172.21.15.46:0/1391060573 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08009e80 con 0x7f6f0c1591f0 2023-10-06T11:30:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f03fff700 1 -- 172.21.15.46:0/1391060573 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0800b8f0 con 0x7f6f0c1591f0 2023-10-06T11:30:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.987+0000 7f6f03fff700 1 -- 172.21.15.46:0/1391060573 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800bc70 con 0x7f6f0c1591f0 2023-10-06T11:30:23.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 msgr2=0x7f6f0c0cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:23.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c0cf7f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098e0 tx=0x7f6f080091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- 172.21.15.46:0/1391060573 shutdown_connections 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- 172.21.15.46:0/1391060573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c0cf7f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- 172.21.15.46:0/1391060573 >> 172.21.15.46:0/1391060573 conn(0x7f6f0c06ef60 msgr2=0x7f6f0c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- 172.21.15.46:0/1391060573 shutdown_connections 2023-10-06T11:30:23.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- 172.21.15.46:0/1391060573 wait complete. 2023-10-06T11:30:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 Processor -- start 2023-10-06T11:30:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- start start 2023-10-06T11:30:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c09d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09c260 con 0x7f6f0c15afc0 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09c4b0 con 0x7f6f0c1591f0 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.991+0000 7f6f12763700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c09c790 con 0x7f6f0c0cc500 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34500/0 (socket says 172.21.15.46:34500) 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 -- 172.21.15.46:0/1977490149 learned_addr learned my addr 172.21.15.46:0/1977490149 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f10f60700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11761700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c09d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 -- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 msgr2=0x7f6f0c09d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c09d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 -- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 msgr2=0x7f6f0c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 -- 172.21.15.46:0/1977490149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08009420 con 0x7f6f0c15afc0 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f11f62700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009fc0 tx=0x7f6f040099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400dbe0 con 0x7f6f0c15afc0 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400f040 con 0x7f6f0c15afc0 2023-10-06T11:30:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04015440 con 0x7f6f0c15afc0 2023-10-06T11:30:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c09ca70 con 0x7f6f0c15afc0 2023-10-06T11:30:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.995+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c16dc70 con 0x7f6f0c15afc0 2023-10-06T11:30:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.999+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c064590 con 0x7f6f0c15afc0 2023-10-06T11:30:24.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.999+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f6f0401e3b0 con 0x7f6f0c15afc0 2023-10-06T11:30:24.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.999+0000 7f6f017fa700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 0x7f6ef0062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.999+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f6f0407d390 con 0x7f6f0c15afc0 2023-10-06T11:30:24.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:23.999+0000 7f6f11761700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 0x7f6ef0062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.003+0000 7f6f11761700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 0x7f6ef0062c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f080091e0 tx=0x7f6f08017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:24.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.007+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6f0404d2d0 con 0x7f6f0c15afc0 2023-10-06T11:30:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.203+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f0c0acbb0 con 0x7f6ef0060750 2023-10-06T11:30:24.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.323+0000 7f6f017fa700 1 -- 172.21.15.46:0/1977490149 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6f0c0acbb0 con 0x7f6ef0060750 2023-10-06T11:30:24.331 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-06T11:30:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 msgr2=0x7f6ef0062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 0x7f6ef0062c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f080091e0 tx=0x7f6f08017000 comp rx=0 tx=0).stop 2023-10-06T11:30:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 msgr2=0x7f6f0c159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009fc0 tx=0x7f6f040099b0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 shutdown_connections 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6ef0060750 0x7f6ef0062c00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f0c15afc0 0x7f6f0c159d70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c1591f0 0x7f6f0c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 --2- 172.21.15.46:0/1977490149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6f0c0cc500 0x7f6f0c09d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 >> 172.21.15.46:0/1977490149 conn(0x7f6f0c06ef60 msgr2=0x7f6f0c0d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 shutdown_connections 2023-10-06T11:30:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.327+0000 7f6f12763700 1 -- 172.21.15.46:0/1977490149 wait complete. 2023-10-06T11:30:24.361 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-06T11:30:24.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T11:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.527+0000 7f58218cd700 1 Processor -- start 2023-10-06T11:30:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.527+0000 7f58218cd700 1 -- start start 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159130 0x7f581c0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f581c159500 0x7f581c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f581c069190 0x7f581c069610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0afb30 con 0x7f581c069190 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0cc090 con 0x7f581c159500 2023-10-06T11:30:24.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f58218cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c069b50 con 0x7f581c159130 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f581c069190 0x7f581c069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f581c069190 0x7f581c069610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34508/0 (socket says 172.21.15.46:34508) 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 -- 172.21.15.46:0/1148565547 learned_addr learned my addr 172.21.15.46:0/1148565547 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581a7fc700 1 --2- 172.21.15.46:0/1148565547 >> v2:172.21.15.132:3300/0 conn(0x7f581c159500 0x7f581c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581affd700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159130 0x7f581c0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 -- 172.21.15.46:0/1148565547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159130 msgr2=0x7f581c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159130 0x7f581c0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 -- 172.21.15.46:0/1148565547 >> v2:172.21.15.132:3300/0 conn(0x7f581c159500 msgr2=0x7f581c0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 --2- 172.21.15.46:0/1148565547 >> v2:172.21.15.132:3300/0 conn(0x7f581c159500 0x7f581c0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 -- 172.21.15.46:0/1148565547 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0ce2c0 con 0x7f581c069190 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f581b7fe700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c069610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f581c088ff0 tx=0x7f5814009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a306f6c3b05cbea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:24.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f5819ffb700 1 -- 172.21.15.46:0/1148565547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400e070 con 0x7f581c069190 2023-10-06T11:30:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f5819ffb700 1 -- 172.21.15.46:0/1148565547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581400ba20 con 0x7f581c069190 2023-10-06T11:30:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.531+0000 7f5819ffb700 1 -- 172.21.15.46:0/1148565547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140092e0 con 0x7f581c069190 2023-10-06T11:30:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 -- 172.21.15.46:0/1148565547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 msgr2=0x7f581c069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c069610 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f581c088ff0 tx=0x7f5814009840 comp rx=0 tx=0).stop 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 -- 172.21.15.46:0/1148565547 shutdown_connections 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 --2- 172.21.15.46:0/1148565547 >> v2:172.21.15.132:3300/0 conn(0x7f581c159500 0x7f581c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c069610 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 --2- 172.21.15.46:0/1148565547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159130 0x7f581c0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 -- 172.21.15.46:0/1148565547 >> 172.21.15.46:0/1148565547 conn(0x7f581c068510 msgr2=0x7f581c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 -- 172.21.15.46:0/1148565547 shutdown_connections 2023-10-06T11:30:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 -- 172.21.15.46:0/1148565547 wait complete. 2023-10-06T11:30:24.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.535+0000 7f58218cd700 1 Processor -- start 2023-10-06T11:30:24.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 -- start start 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c159130 0x7f581c0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159500 0x7f581c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c15c850 con 0x7f581c069190 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b52d0 con 0x7f581c159130 2023-10-06T11:30:24.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f58218cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c155270 con 0x7f581c159500 2023-10-06T11:30:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34520/0 (socket says 172.21.15.46:34520) 2023-10-06T11:30:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 -- 172.21.15.46:0/3488748400 learned_addr learned my addr 172.21.15.46:0/3488748400 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581a7fc700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c159130 0x7f581c0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581b7fe700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159500 0x7f581c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 -- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159500 msgr2=0x7f581c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159500 0x7f581c0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 -- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c159130 msgr2=0x7f581c0b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c159130 0x7f581c0b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 -- 172.21.15.46:0/3488748400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814009ae0 con 0x7f581c069190 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f581affd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d890 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.539+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00b600 con 0x7f581c069190 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00b760 con 0x7f581c069190 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c15c290 con 0x7f581c069190 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00bae0 con 0x7f581c069190 2023-10-06T11:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c0b5570 con 0x7f581c069190 2023-10-06T11:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c064380 con 0x7f581c069190 2023-10-06T11:30:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.543+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f580c01e460 con 0x7f581c069190 2023-10-06T11:30:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.547+0000 7f5802ffd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 0x7f5808062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.547+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f580c07d540 con 0x7f581c069190 2023-10-06T11:30:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.547+0000 7f581a7fc700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 0x7f5808062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.547+0000 7f581a7fc700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 0x7f5808062c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f581c1562c0 tx=0x7f5810008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.551+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f581c064380 con 0x7f581c069190 2023-10-06T11:30:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.751+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f581c0b5150 con 0x7f5808060750 2023-10-06T11:30:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.867+0000 7f5802ffd700 1 -- 172.21.15.46:0/3488748400 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f58100098e0 con 0x7f5808060750 2023-10-06T11:30:24.875 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 msgr2=0x7f5808062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 0x7f5808062c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f581c1562c0 tx=0x7f5810008040 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 msgr2=0x7f581c0b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d890 tx=0x7f580c008040 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 shutdown_connections 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5808060750 0x7f5808062c00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c159500 0x7f581c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c159130 0x7f581c0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 --2- 172.21.15.46:0/3488748400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f581c069190 0x7f581c0b5a90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 >> 172.21.15.46:0/3488748400 conn(0x7f581c068510 msgr2=0x7f581c156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:24.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 shutdown_connections 2023-10-06T11:30:24.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:24.871+0000 7f58218cd700 1 -- 172.21.15.46:0/3488748400 wait complete. 2023-10-06T11:30:24.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-06T11:30:24.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T11:30:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.067+0000 7f74e500c700 1 Processor -- start 2023-10-06T11:30:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.067+0000 7f74e500c700 1 -- start start 2023-10-06T11:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e500c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f74e01591f0 0x7f74e00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e500c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 0x7f74e00af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e500c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e00cf360 con 0x7f74e00cc500 2023-10-06T11:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e500c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e015dd60 con 0x7f74e01591f0 2023-10-06T11:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e500c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00cea30 con 0x7f74e01595c0 2023-10-06T11:30:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e480b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f74e00cc500 0x7f74e015ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43398/0 (socket says 172.21.15.46:43398) 2023-10-06T11:30:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74e480b700 1 -- 172.21.15.46:0/2989242224 learned_addr learned my addr 172.21.15.46:0/2989242224 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.071+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2989242224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 0x7f74e00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74dffff700 1 --2- 172.21.15.46:0/2989242224 >> v2:172.21.15.132:3300/0 conn(0x7f74e01591f0 0x7f74e00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74deffd700 1 -- 172.21.15.46:0/2989242224 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680869685 0 0) 0x7f74e00cf360 con 0x7f74e00cc500 2023-10-06T11:30:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74deffd700 1 -- 172.21.15.46:0/2989242224 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c8003570 con 0x7f74e00cc500 2023-10-06T11:30:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74df7fe700 1 -- 172.21.15.46:0/2989242224 >> v2:172.21.15.132:3300/0 conn(0x7f74e01591f0 msgr2=0x7f74e00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2989242224 >> v2:172.21.15.132:3300/0 conn(0x7f74e01591f0 0x7f74e00cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74df7fe700 1 -- 172.21.15.46:0/2989242224 >> v1:172.21.15.46:6789/0 conn(0x7f74e00cc500 legacy=0x7f74e015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74df7fe700 1 -- 172.21.15.46:0/2989242224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00ced10 con 0x7f74e01595c0 2023-10-06T11:30:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2989242224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 0x7f74e00af940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74d40098e0 tx=0x7f74d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22300eacf9676b60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74deffd700 1 -- 172.21.15.46:0/2989242224 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4011630 con 0x7f74e01595c0 2023-10-06T11:30:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74deffd700 1 -- 172.21.15.46:0/2989242224 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d4011790 con 0x7f74e01595c0 2023-10-06T11:30:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74deffd700 1 -- 172.21.15.46:0/2989242224 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4011b30 con 0x7f74e01595c0 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 -- 172.21.15.46:0/2989242224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 msgr2=0x7f74e00af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 --2- 172.21.15.46:0/2989242224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 0x7f74e00af940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74d40098e0 tx=0x7f74d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 -- 172.21.15.46:0/2989242224 shutdown_connections 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 --2- 172.21.15.46:0/2989242224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01595c0 0x7f74e00af940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 --2- 172.21.15.46:0/2989242224 >> v2:172.21.15.132:3300/0 conn(0x7f74e01591f0 0x7f74e00cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 -- 172.21.15.46:0/2989242224 >> 172.21.15.46:0/2989242224 conn(0x7f74e006ef60 msgr2=0x7f74e00d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 -- 172.21.15.46:0/2989242224 shutdown_connections 2023-10-06T11:30:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.075+0000 7f74e500c700 1 -- 172.21.15.46:0/2989242224 wait complete. 2023-10-06T11:30:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 Processor -- start 2023-10-06T11:30:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 -- start start 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 0x7f74e00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01591f0 0x7f74e0171bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0172110 con 0x7f74e014ca10 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00b2020 con 0x7f74e00cc500 2023-10-06T11:30:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e500c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e00b1410 con 0x7f74e01591f0 2023-10-06T11:30:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e480b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e480b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34536/0 (socket says 172.21.15.46:34536) 2023-10-06T11:30:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74e480b700 1 -- 172.21.15.46:0/2734406064 learned_addr learned my addr 172.21.15.46:0/2734406064 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.079+0000 7f74dffff700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 0x7f74e00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01591f0 0x7f74e0171bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 -- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01591f0 msgr2=0x7f74e0171bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01591f0 0x7f74e0171bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 -- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 msgr2=0x7f74e00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 -- 172.21.15.46:0/2734406064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4009420 con 0x7f74e00cc500 2023-10-06T11:30:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74e480b700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74dffff700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 0x7f74e00b0d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ea80 tx=0x7f74d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d000eec0 con 0x7f74e00cc500 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d000c7f0 con 0x7f74e00cc500 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d000cb70 con 0x7f74e00cc500 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e00b16f0 con 0x7f74e00cc500 2023-10-06T11:30:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e00b1cb0 con 0x7f74e00cc500 2023-10-06T11:30:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.083+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e0064590 con 0x7f74e00cc500 2023-10-06T11:30:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.087+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f74d0013470 con 0x7f74e00cc500 2023-10-06T11:30:25.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.087+0000 7f74c7fff700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 0x7f74cc062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.087+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f74d007cfb0 con 0x7f74e00cc500 2023-10-06T11:30:25.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.087+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 0x7f74cc062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.087+0000 7f74df7fe700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 0x7f74cc062c00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74d40091e0 tx=0x7f74d4017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.095+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74d004d000 con 0x7f74e00cc500 2023-10-06T11:30:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.295+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e00b2160 con 0x7f74cc060750 2023-10-06T11:30:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.415+0000 7f74c7fff700 1 -- 172.21.15.46:0/2734406064 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f74d4011900 con 0x7f74cc060750 2023-10-06T11:30:25.425 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-06T11:30:25.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 msgr2=0x7f74cc062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 0x7f74cc062c00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74d40091e0 tx=0x7f74d4017000 comp rx=0 tx=0).stop 2023-10-06T11:30:25.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 msgr2=0x7f74e00b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 0x7f74e00b0d30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ea80 tx=0x7f74d00091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 shutdown_connections 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060750 0x7f74cc062c00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e014ca10 0x7f74e00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e01591f0 0x7f74e0171bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 --2- 172.21.15.46:0/2734406064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e00cc500 0x7f74e00b0d30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 >> 172.21.15.46:0/2734406064 conn(0x7f74e006ef60 msgr2=0x7f74e00b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 shutdown_connections 2023-10-06T11:30:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.419+0000 7f74e500c700 1 -- 172.21.15.46:0/2734406064 wait complete. 2023-10-06T11:30:25.452 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-06T11:30:25.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T11:30:25.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 Processor -- start 2023-10-06T11:30:25.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 -- start start 2023-10-06T11:30:25.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdc381591f0 0x7fdc380cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380cc500 0x7fdc3815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc380cf360 con 0x7fdc381595c0 2023-10-06T11:30:25.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3815dd60 con 0x7fdc381591f0 2023-10-06T11:30:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.615+0000 7fdc3f216700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380cea30 con 0x7fdc380cc500 2023-10-06T11:30:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3da13700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdc381595c0 0x7fdc380af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43414/0 (socket says 172.21.15.46:43414) 2023-10-06T11:30:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3da13700 1 -- 172.21.15.46:0/2882679468 learned_addr learned my addr 172.21.15.46:0/2882679468 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 --2- 172.21.15.46:0/2882679468 >> v2:172.21.15.132:3300/0 conn(0x7fdc381591f0 0x7fdc380cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3ea15700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380cc500 0x7fdc3815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3d212700 1 -- 172.21.15.46:0/2882679468 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173175769 0 0) 0x7fdc380cf360 con 0x7fdc381595c0 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3d212700 1 -- 172.21.15.46:0/2882679468 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc24003570 con 0x7fdc381595c0 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 -- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380cc500 msgr2=0x7fdc3815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380cc500 0x7fdc3815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 -- 172.21.15.46:0/2882679468 >> v1:172.21.15.46:6789/0 conn(0x7fdc381595c0 legacy=0x7fdc380af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 -- 172.21.15.46:0/2882679468 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380ced10 con 0x7fdc381591f0 2023-10-06T11:30:25.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3e214700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc380cf7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098e0 tx=0x7fdc340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bc992e94e87282 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3d212700 1 -- 172.21.15.46:0/2882679468 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34011630 con 0x7fdc381591f0 2023-10-06T11:30:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3d212700 1 -- 172.21.15.46:0/2882679468 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34011790 con 0x7fdc381591f0 2023-10-06T11:30:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3d212700 1 -- 172.21.15.46:0/2882679468 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34011b30 con 0x7fdc381591f0 2023-10-06T11:30:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 -- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 msgr2=0x7fdc380cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc380cf7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc340098e0 tx=0x7fdc340091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 -- 172.21.15.46:0/2882679468 shutdown_connections 2023-10-06T11:30:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380cc500 0x7fdc3815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 --2- 172.21.15.46:0/2882679468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc380cf7f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 -- 172.21.15.46:0/2882679468 >> 172.21.15.46:0/2882679468 conn(0x7fdc3806ef60 msgr2=0x7fdc380d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.619+0000 7fdc3f216700 1 -- 172.21.15.46:0/2882679468 shutdown_connections 2023-10-06T11:30:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 -- 172.21.15.46:0/2882679468 wait complete. 2023-10-06T11:30:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 Processor -- start 2023-10-06T11:30:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 -- start start 2023-10-06T11:30:25.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc381512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 0x7fdc38151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38151d60 con 0x7fdc380cc500 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3809bc60 con 0x7fdc381591f0 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3f216700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3809b3f0 con 0x7fdc3815aa20 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3e214700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3e214700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34538/0 (socket says 172.21.15.46:34538) 2023-10-06T11:30:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.623+0000 7fdc3e214700 1 -- 172.21.15.46:0/1962606189 learned_addr learned my addr 172.21.15.46:0/1962606189 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3ea15700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 0x7fdc38151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3da13700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc381512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 -- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 msgr2=0x7fdc38151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 0x7fdc38151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 -- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 msgr2=0x7fdc381512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc381512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 -- 172.21.15.46:0/1962606189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009420 con 0x7fdc380cc500 2023-10-06T11:30:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3ea15700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 0x7fdc38151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3e214700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc38088770 tx=0x7fdc34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34000d10 con 0x7fdc380cc500 2023-10-06T11:30:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34011b10 con 0x7fdc380cc500 2023-10-06T11:30:25.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34025660 con 0x7fdc380cc500 2023-10-06T11:30:25.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3809b6d0 con 0x7fdc380cc500 2023-10-06T11:30:25.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3809c190 con 0x7fdc380cc500 2023-10-06T11:30:25.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.627+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3809c3b0 con 0x7fdc380cc500 2023-10-06T11:30:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.631+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fdc34025a50 con 0x7fdc380cc500 2023-10-06T11:30:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.631+0000 7fdc2e7fc700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 0x7fdc1c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.631+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fdc3407e660 con 0x7fdc380cc500 2023-10-06T11:30:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.631+0000 7fdc3da13700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 0x7fdc1c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.631+0000 7fdc3da13700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 0x7fdc1c062c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b42b0 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.639+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdc3404e6b0 con 0x7fdc380cc500 2023-10-06T11:30:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.831+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc38064590 con 0x7fdc1c060750 2023-10-06T11:30:25.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.951+0000 7fdc2e7fc700 1 -- 172.21.15.46:0/1962606189 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdc38064590 con 0x7fdc1c060750 2023-10-06T11:30:25.961 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-06T11:30:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 msgr2=0x7fdc1c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 0x7fdc1c062c00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc380b42b0 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2023-10-06T11:30:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 msgr2=0x7fdc3815a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc38088770 tx=0x7fdc34017000 comp rx=0 tx=0).stop 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 shutdown_connections 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc1c060750 0x7fdc1c062c00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc3815aa20 0x7fdc38151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc381591f0 0x7fdc381512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 --2- 172.21.15.46:0/1962606189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc380cc500 0x7fdc3815a4e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 >> 172.21.15.46:0/1962606189 conn(0x7fdc3806ef60 msgr2=0x7fdc38152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 shutdown_connections 2023-10-06T11:30:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:25.955+0000 7fdc3f216700 1 -- 172.21.15.46:0/1962606189 wait complete. 2023-10-06T11:30:25.990 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-06T11:30:25.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:25.990 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-06T11:30:26.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:26.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:26.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:26.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:26.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:26.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:26.018 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_01_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:26.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-10-06T11:30:26.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.227+0000 7fd316923700 1 Processor -- start 2023-10-06T11:30:26.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.227+0000 7fd316923700 1 -- start start 2023-10-06T11:30:26.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd316923700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3101591f0 0x7fd3100cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd316923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100cf360 con 0x7fd3101595c0 2023-10-06T11:30:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd316923700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31015dd60 con 0x7fd3100cc500 2023-10-06T11:30:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd316923700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100cea30 con 0x7fd3101591f0 2023-10-06T11:30:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd315120700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3101595c0 0x7fd3100af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43424/0 (socket says 172.21.15.46:43424) 2023-10-06T11:30:26.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd315921700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3101591f0 0x7fd3100cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.231+0000 7fd315120700 1 -- 172.21.15.46:0/2888951404 learned_addr learned my addr 172.21.15.46:0/2888951404 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd315921700 1 -- 172.21.15.46:0/2888951404 >> v1:172.21.15.132:6789/0 conn(0x7fd3100cc500 legacy=0x7fd31015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd31491f700 1 -- 172.21.15.46:0/2888951404 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354702308 0 0) 0x7fd3100cf360 con 0x7fd3101595c0 2023-10-06T11:30:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd315921700 1 -- 172.21.15.46:0/2888951404 >> v1:172.21.15.46:6789/0 conn(0x7fd3101595c0 legacy=0x7fd3100af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd315921700 1 -- 172.21.15.46:0/2888951404 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100ced10 con 0x7fd3101591f0 2023-10-06T11:30:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd315921700 1 --2- 172.21.15.46:0/2888951404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd3100cf7f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098e0 tx=0x7fd30c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a2c703f6e644ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd31491f700 1 -- 172.21.15.46:0/2888951404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c009e80 con 0x7fd3101591f0 2023-10-06T11:30:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd31491f700 1 -- 172.21.15.46:0/2888951404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c00b8f0 con 0x7fd3101591f0 2023-10-06T11:30:26.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd31491f700 1 -- 172.21.15.46:0/2888951404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c00bc70 con 0x7fd3101591f0 2023-10-06T11:30:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 -- 172.21.15.46:0/2888951404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 msgr2=0x7fd3100cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 --2- 172.21.15.46:0/2888951404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd3100cf7f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098e0 tx=0x7fd30c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 -- 172.21.15.46:0/2888951404 shutdown_connections 2023-10-06T11:30:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 --2- 172.21.15.46:0/2888951404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd3100cf7f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 -- 172.21.15.46:0/2888951404 >> 172.21.15.46:0/2888951404 conn(0x7fd31006ef60 msgr2=0x7fd3100d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 -- 172.21.15.46:0/2888951404 shutdown_connections 2023-10-06T11:30:26.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.235+0000 7fd316923700 1 -- 172.21.15.46:0/2888951404 wait complete. 2023-10-06T11:30:26.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 Processor -- start 2023-10-06T11:30:26.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 -- start start 2023-10-06T11:30:26.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100cc500 0x7fd31015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100b1bc0 0x7fd3100b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100b2580 con 0x7fd3101591f0 2023-10-06T11:30:26.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31009cd00 con 0x7fd3100cc500 2023-10-06T11:30:26.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.239+0000 7fd316923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31009aca0 con 0x7fd3100b1bc0 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34562/0 (socket says 172.21.15.46:34562) 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 -- 172.21.15.46:0/1818358357 learned_addr learned my addr 172.21.15.46:0/1818358357 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315921700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100cc500 0x7fd31015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd316122700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100b1bc0 0x7fd3100b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 -- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100b1bc0 msgr2=0x7fd3100b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100b1bc0 0x7fd3100b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 -- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100cc500 msgr2=0x7fd31015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100cc500 0x7fd31015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 -- 172.21.15.46:0/1818358357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c009420 con 0x7fd3101591f0 2023-10-06T11:30:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd315120700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c560 tx=0x7fd300000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300005b30 con 0x7fd3101591f0 2023-10-06T11:30:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300005c90 con 0x7fd3101591f0 2023-10-06T11:30:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300007320 con 0x7fd3101591f0 2023-10-06T11:30:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31009c6b0 con 0x7fd3101591f0 2023-10-06T11:30:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.243+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31009b000 con 0x7fd3101591f0 2023-10-06T11:30:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.247+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31009c510 con 0x7fd3101591f0 2023-10-06T11:30:26.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.247+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fd30001c440 con 0x7fd3101591f0 2023-10-06T11:30:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.247+0000 7fd305ffb700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 0x7fd2f4062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.247+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fd30007c1c0 con 0x7fd3101591f0 2023-10-06T11:30:26.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.251+0000 7fd315921700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 0x7fd2f4062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.251+0000 7fd315921700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 0x7fd2f4062c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0093f0 tx=0x7fd30c017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.255+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd31009c510 con 0x7fd3101591f0 2023-10-06T11:30:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.447+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd31009c510 con 0x7fd2f4060780 2023-10-06T11:30:26.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.463+0000 7fd305ffb700 1 -- 172.21.15.46:0/1818358357 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd31009c510 con 0x7fd2f4060780 2023-10-06T11:30:26.473 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:26.473 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:28", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 msgr2=0x7fd2f4062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 0x7fd2f4062c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0093f0 tx=0x7fd30c017000 comp rx=0 tx=0).stop 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 msgr2=0x7fd310151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c560 tx=0x7fd300000fe0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 shutdown_connections 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd2f4060780 0x7fd2f4062c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100b1bc0 0x7fd3100b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3101591f0 0x7fd310151cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 --2- 172.21.15.46:0/1818358357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100cc500 0x7fd31015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 >> 172.21.15.46:0/1818358357 conn(0x7fd31006ef60 msgr2=0x7fd310152740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 shutdown_connections 2023-10-06T11:30:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.467+0000 7fd316923700 1 -- 172.21.15.46:0/1818358357 wait complete. 2023-10-06T11:30:26.498 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:28", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:26.498 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:28', 'first': '2023-10-06T11:20:01', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:26.498 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:26.498 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:26.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:26.498 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:26.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:26.517 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:26.517 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:26.517 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_00_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_01_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:26.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-10-06T11:30:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.723+0000 7f98a8d45700 1 Processor -- start 2023-10-06T11:30:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.723+0000 7f98a8d45700 1 -- start start 2023-10-06T11:30:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a8d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98a4140800 0x7f98a406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a8d45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98a40c6b70 0x7f98a406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a8d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40c25a0 con 0x7f98a4140bd0 2023-10-06T11:30:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a8d45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a406efa0 con 0x7f98a40c6b70 2023-10-06T11:30:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a8d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40c1c70 con 0x7f98a4140800 2023-10-06T11:30:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a1d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98a4140bd0 0x7f98a40c6460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43426/0 (socket says 172.21.15.46:43426) 2023-10-06T11:30:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a1d9b700 1 -- 172.21.15.46:0/1382014225 learned_addr learned my addr 172.21.15.46:0/1382014225 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a259c700 1 --2- 172.21.15.46:0/1382014225 >> v2:172.21.15.132:3300/0 conn(0x7f98a4140800 0x7f98a406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/1382014225 >> v2:172.21.15.46:3300/0 conn(0x7f98a40c6b70 0x7f98a406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758305576 0 0) 0x7f98a40c25a0 con 0x7f98a4140bd0 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.727+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988c003570 con 0x7f98a4140bd0 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1057141240 0 0) 0x7f988c003570 con 0x7f98a4140bd0 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 >> v2:172.21.15.132:3300/0 conn(0x7f98a4140800 msgr2=0x7f98a406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 --2- 172.21.15.46:0/1382014225 >> v2:172.21.15.132:3300/0 conn(0x7f98a4140800 0x7f98a406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 >> v2:172.21.15.46:3300/0 conn(0x7f98a40c6b70 msgr2=0x7f98a406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 --2- 172.21.15.46:0/1382014225 >> v2:172.21.15.46:3300/0 conn(0x7f98a40c6b70 0x7f98a406ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40c1f50 con 0x7f98a4140bd0 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f9898001470 con 0x7f98a4140bd0 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 >> v1:172.21.15.46:6789/0 conn(0x7f98a4140bd0 legacy=0x7f98a40c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c003a60 0x7f988c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c004810 0x7f988c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f988c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40c1c70 con 0x7f988c0051c0 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c00bfa0 con 0x7f988c003a60 2023-10-06T11:30:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c00c110 con 0x7f988c004810 2023-10-06T11:30:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f988c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a1d9b700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c004810 0x7f988c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a259c700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c003a60 0x7f988c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 -- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c004810 msgr2=0x7f988c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f988c004810 0x7f988c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 -- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c003a60 msgr2=0x7f988c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f988c003a60 0x7f988c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 -- 172.21.15.46:0/1382014225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c003570 con 0x7f988c0051c0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 -- 172.21.15.46:0/1382014225 reap_dead start 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f988c00ba60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f989c0094d0 tx=0x7f989c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be2666fc9ddff2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00ea00 con 0x7f988c0051c0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00f040 con 0x7f988c0051c0 2023-10-06T11:30:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a159a700 1 -- 172.21.15.46:0/1382014225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00edd0 con 0x7f988c0051c0 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a8d45700 1 -- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 msgr2=0x7f988c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a8d45700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f988c00ba60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f989c0094d0 tx=0x7f989c00b040 comp rx=0 tx=0).stop 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a8d45700 1 -- 172.21.15.46:0/1382014225 shutdown_connections 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a8d45700 1 --2- 172.21.15.46:0/1382014225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f988c00ba60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.731+0000 7f98a8d45700 1 -- 172.21.15.46:0/1382014225 >> 172.21.15.46:0/1382014225 conn(0x7f98a40cbe20 msgr2=0x7f98a40cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.735+0000 7f98a8d45700 1 -- 172.21.15.46:0/1382014225 shutdown_connections 2023-10-06T11:30:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.735+0000 7f98a8d45700 1 -- 172.21.15.46:0/1382014225 wait complete. 2023-10-06T11:30:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.735+0000 7f98a8d45700 1 Processor -- start 2023-10-06T11:30:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.735+0000 7f98a8d45700 1 -- start start 2023-10-06T11:30:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98a415bb50 0x7f98a415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98a40c55e0 0x7f98a40c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a415c500 con 0x7f988c0051c0 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40ad480 con 0x7f98a415bb50 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40cb9f0 con 0x7f98a40c55e0 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34580/0 (socket says 172.21.15.46:34580) 2023-10-06T11:30:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 -- 172.21.15.46:0/2535683485 learned_addr learned my addr 172.21.15.46:0/2535683485 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a2d9d700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98a40c55e0 0x7f98a40c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a1d9b700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98a415bb50 0x7f98a415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 -- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98a40c55e0 msgr2=0x7f98a40c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98a40c55e0 0x7f98a40c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 -- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98a415bb50 msgr2=0x7f98a415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98a415bb50 0x7f98a415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 -- 172.21.15.46:0/2535683485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40ac870 con 0x7f988c0051c0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a259c700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9894011750 tx=0x7f9894008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894015670 con 0x7f988c0051c0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940157d0 con 0x7f988c0051c0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40acb50 con 0x7f988c0051c0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894015b50 con 0x7f988c0051c0 2023-10-06T11:30:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.739+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a40ad110 con 0x7f988c0051c0 2023-10-06T11:30:26.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.743+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a406b890 con 0x7f988c0051c0 2023-10-06T11:30:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.743+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f9894013410 con 0x7f988c0051c0 2023-10-06T11:30:26.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.743+0000 7f98927fc700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 0x7f9884062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:26.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.743+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f989407d0e0 con 0x7f988c0051c0 2023-10-06T11:30:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.747+0000 7f98a1d9b700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 0x7f9884062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.747+0000 7f98a1d9b700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 0x7f9884062c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9898001620 tx=0x7f9898007040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:26.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.751+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f989404d130 con 0x7f988c0051c0 2023-10-06T11:30:26.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.939+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98a40ad5c0 con 0x7f9884060780 2023-10-06T11:30:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98927fc700 1 -- 172.21.15.46:0/2535683485 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9898010040 con 0x7f9884060780 2023-10-06T11:30:26.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:26.962 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:29", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 msgr2=0x7f9884062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 0x7f9884062c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9898001620 tx=0x7f9898007040 comp rx=0 tx=0).stop 2023-10-06T11:30:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 msgr2=0x7f98a40cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9894011750 tx=0x7f9894008040 comp rx=0 tx=0).stop 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 shutdown_connections 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9884060780 0x7f9884062c30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98a40c55e0 0x7f98a40c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98a415bb50 0x7f98a415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 --2- 172.21.15.46:0/2535683485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988c0051c0 0x7f98a40cb4b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 >> 172.21.15.46:0/2535683485 conn(0x7f98a40cbe20 msgr2=0x7f98a406a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 shutdown_connections 2023-10-06T11:30:26.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:26.955+0000 7f98a8d45700 1 -- 172.21.15.46:0/2535683485 wait complete. 2023-10-06T11:30:26.986 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:29", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:26.986 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:29', 'first': '2023-10-06T11:20:00', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:26.987 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:26.987 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:26.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:26.987 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:27.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:27.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:27.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:27.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:27.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:27.010 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_01_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_01_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:27.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-10-06T11:30:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.219+0000 7fa579745700 1 Processor -- start 2023-10-06T11:30:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.219+0000 7fa579745700 1 -- start start 2023-10-06T11:30:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa579745700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa574159050 0x7fa5740c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa579745700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5740c82e0 0x7fa5740ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa579745700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740c71a0 con 0x7fa574159420 2023-10-06T11:30:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa579745700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740c86e0 con 0x7fa574159050 2023-10-06T11:30:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa579745700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5740c1bd0 con 0x7fa5740c82e0 2023-10-06T11:30:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa574159050 0x7fa5740c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa574159050 0x7fa5740c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34586/0 (socket says 172.21.15.46:34586) 2023-10-06T11:30:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 -- 172.21.15.46:0/878095373 learned_addr learned my addr 172.21.15.46:0/878095373 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa5737fe700 1 --2- 172.21.15.46:0/878095373 >> v2:172.21.15.132:3300/0 conn(0x7fa5740c82e0 0x7fa5740ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 -- 172.21.15.46:0/878095373 >> v2:172.21.15.132:3300/0 conn(0x7fa5740c82e0 msgr2=0x7fa5740ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 --2- 172.21.15.46:0/878095373 >> v2:172.21.15.132:3300/0 conn(0x7fa5740c82e0 0x7fa5740ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 -- 172.21.15.46:0/878095373 >> v1:172.21.15.132:6789/0 conn(0x7fa574159420 legacy=0x7fa5740c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.223+0000 7fa572ffd700 1 -- 172.21.15.46:0/878095373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740c6590 con 0x7fa574159050 2023-10-06T11:30:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa572ffd700 1 --2- 172.21.15.46:0/878095373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa5740c9c90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c63c0 tx=0x7fa564009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5fbbf504de8ae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa571ffb700 1 -- 172.21.15.46:0/878095373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56400b860 con 0x7fa574159050 2023-10-06T11:30:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa571ffb700 1 -- 172.21.15.46:0/878095373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56400b9c0 con 0x7fa574159050 2023-10-06T11:30:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa571ffb700 1 -- 172.21.15.46:0/878095373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56400bd60 con 0x7fa574159050 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 -- 172.21.15.46:0/878095373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 msgr2=0x7fa5740c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 --2- 172.21.15.46:0/878095373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa5740c9c90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c63c0 tx=0x7fa564009250 comp rx=0 tx=0).stop 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 -- 172.21.15.46:0/878095373 shutdown_connections 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 --2- 172.21.15.46:0/878095373 >> v2:172.21.15.132:3300/0 conn(0x7fa5740c82e0 0x7fa5740ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 --2- 172.21.15.46:0/878095373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa5740c9c90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 -- 172.21.15.46:0/878095373 >> 172.21.15.46:0/878095373 conn(0x7fa5740cbe20 msgr2=0x7fa5740cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 -- 172.21.15.46:0/878095373 shutdown_connections 2023-10-06T11:30:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.227+0000 7fa579745700 1 -- 172.21.15.46:0/878095373 wait complete. 2023-10-06T11:30:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 Processor -- start 2023-10-06T11:30:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 -- start start 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 0x7fa57416aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 0x7fa5740c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740afbc0 con 0x7fa574159050 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740afea0 con 0x7fa574159420 2023-10-06T11:30:27.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa579745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b0180 con 0x7fa5740c82e0 2023-10-06T11:30:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34600/0 (socket says 172.21.15.46:34600) 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 -- 172.21.15.46:0/4175557819 learned_addr learned my addr 172.21.15.46:0/4175557819 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa572ffd700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 0x7fa57416aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5737fe700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 0x7fa5740c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 -- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 msgr2=0x7fa57416aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 0x7fa57416aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 -- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 msgr2=0x7fa5740c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 0x7fa5740c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 -- 172.21.15.46:0/4175557819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640094f0 con 0x7fa574159050 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5737fe700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 0x7fa5740c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa572ffd700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 0x7fa57416aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.231+0000 7fa5727fc700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c590 tx=0x7fa56800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011590 con 0x7fa574159050 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680116f0 con 0x7fa574159050 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5740b0460 con 0x7fa574159050 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568011ab0 con 0x7fa574159050 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57416b000 con 0x7fa574159050 2023-10-06T11:30:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57406b890 con 0x7fa574159050 2023-10-06T11:30:27.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.235+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fa56801b3b0 con 0x7fa574159050 2023-10-06T11:30:27.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.239+0000 7fa55affd700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 0x7fa560062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.239+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fa56807c3c0 con 0x7fa574159050 2023-10-06T11:30:27.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.239+0000 7fa572ffd700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 0x7fa560062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.239+0000 7fa572ffd700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 0x7fa560062c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c63c0 tx=0x7fa564017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.247+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa56804c4b0 con 0x7fa574159050 2023-10-06T11:30:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.439+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa57404b5c0 con 0x7fa560060750 2023-10-06T11:30:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.455+0000 7fa55affd700 1 -- 172.21.15.46:0/4175557819 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa57404b5c0 con 0x7fa560060750 2023-10-06T11:30:27.464 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:27.464 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:30", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 msgr2=0x7fa560062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 0x7fa560062c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5740c63c0 tx=0x7fa564017000 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 msgr2=0x7fa574152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c590 tx=0x7fa56800c950 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 shutdown_connections 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa560060750 0x7fa560062c00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa574159420 0x7fa5740c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa574159050 0x7fa574152300 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 --2- 172.21.15.46:0/4175557819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5740c82e0 0x7fa57416aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 >> 172.21.15.46:0/4175557819 conn(0x7fa5740cbe20 msgr2=0x7fa5740c7630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 shutdown_connections 2023-10-06T11:30:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.459+0000 7fa579745700 1 -- 172.21.15.46:0/4175557819 wait complete. 2023-10-06T11:30:27.487 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:30", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:27.488 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:30', 'first': '2023-10-06T11:20:01', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:27.488 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:27.488 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:27.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:27.488 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-06T11:30:27.509 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:27.509 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:27.509 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_01_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:27.510 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:27.511 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_00_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_01_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_01_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:27.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-10-06T11:30:27.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.719+0000 7fc21f523700 1 Processor -- start 2023-10-06T11:30:27.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.719+0000 7fc21f523700 1 -- start start 2023-10-06T11:30:27.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21f523700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2180cc500 0x7fc21815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21f523700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180cf360 con 0x7fc2181595c0 2023-10-06T11:30:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21f523700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21815dd60 con 0x7fc2181591f0 2023-10-06T11:30:27.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21f523700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180cff00 con 0x7fc2180cc500 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21ed22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2180cc500 0x7fc21815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21e521700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc2181591f0 0x7fc2180cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:33896/0 (socket says 172.21.15.46:33896) 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21e521700 1 -- 172.21.15.46:0/163605165 learned_addr learned my addr 172.21.15.46:0/163605165 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21ed22700 1 -- 172.21.15.46:0/163605165 >> v1:172.21.15.132:6789/0 conn(0x7fc2181591f0 legacy=0x7fc2180cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21ed22700 1 -- 172.21.15.46:0/163605165 >> v1:172.21.15.46:6789/0 conn(0x7fc2181595c0 legacy=0x7fc2180af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.723+0000 7fc21ed22700 1 -- 172.21.15.46:0/163605165 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180cea30 con 0x7fc2180cc500 2023-10-06T11:30:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21ed22700 1 --2- 172.21.15.46:0/163605165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2180cc500 0x7fc21815ceb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc210009e60 tx=0x7fc210009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37499e3aa773f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21d51f700 1 -- 172.21.15.46:0/163605165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000e070 con 0x7fc2180cc500 2023-10-06T11:30:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21d51f700 1 -- 172.21.15.46:0/163605165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000bcc0 con 0x7fc2180cc500 2023-10-06T11:30:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21d51f700 1 -- 172.21.15.46:0/163605165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210012680 con 0x7fc2180cc500 2023-10-06T11:30:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- 172.21.15.46:0/163605165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2180cc500 msgr2=0x7fc21815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 --2- 172.21.15.46:0/163605165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2180cc500 0x7fc21815ceb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc210009e60 tx=0x7fc210009780 comp rx=0 tx=0).stop 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- 172.21.15.46:0/163605165 shutdown_connections 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 --2- 172.21.15.46:0/163605165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2180cc500 0x7fc21815ceb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- 172.21.15.46:0/163605165 >> 172.21.15.46:0/163605165 conn(0x7fc21806ef60 msgr2=0x7fc2180d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- 172.21.15.46:0/163605165 shutdown_connections 2023-10-06T11:30:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- 172.21.15.46:0/163605165 wait complete. 2023-10-06T11:30:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 Processor -- start 2023-10-06T11:30:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.727+0000 7fc21f523700 1 -- start start 2023-10-06T11:30:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 0x7fc21815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2181591f0 0x7fc21813e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815cfd0 con 0x7fc21815a370 2023-10-06T11:30:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815ba00 con 0x7fc2181591f0 2023-10-06T11:30:27.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21813e6e0 con 0x7fc2180cc500 2023-10-06T11:30:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34616/0 (socket says 172.21.15.46:34616) 2023-10-06T11:30:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 -- 172.21.15.46:0/3558361631 learned_addr learned my addr 172.21.15.46:0/3558361631 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:27.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21dd20700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2181591f0 0x7fc21813e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21e521700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 0x7fc21815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 -- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 msgr2=0x7fc21815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 0x7fc21815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 -- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2181591f0 msgr2=0x7fc21813e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2181591f0 0x7fc21813e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 -- 172.21.15.46:0/3558361631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009a20 con 0x7fc21815a370 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21e521700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 0x7fc21815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21ed22700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc218088770 tx=0x7fc210019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21001d630 con 0x7fc21815a370 2023-10-06T11:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21001d7d0 con 0x7fc21815a370 2023-10-06T11:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21815adf0 con 0x7fc21815a370 2023-10-06T11:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21001db90 con 0x7fc21815a370 2023-10-06T11:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.731+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21815b3b0 con 0x7fc21815a370 2023-10-06T11:30:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.735+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc218064590 con 0x7fc21815a370 2023-10-06T11:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.735+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fc210024070 con 0x7fc21815a370 2023-10-06T11:30:27.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.735+0000 7fc20e7fc700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 0x7fc1fc062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:27.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.735+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fc21007f840 con 0x7fc21815a370 2023-10-06T11:30:27.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.735+0000 7fc21e521700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 0x7fc1fc062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:27.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.739+0000 7fc21e521700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 0x7fc1fc062c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2140018c0 tx=0x7fc214009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:27.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.743+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc21004f940 con 0x7fc21815a370 2023-10-06T11:30:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.935+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc21815bb40 con 0x7fc1fc060750 2023-10-06T11:30:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.951+0000 7fc20e7fc700 1 -- 172.21.15.46:0/3558361631 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc21815bb40 con 0x7fc1fc060750 2023-10-06T11:30:27.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:27.962 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:30", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 msgr2=0x7fc1fc062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 0x7fc1fc062c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc2140018c0 tx=0x7fc214009040 comp rx=0 tx=0).stop 2023-10-06T11:30:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 msgr2=0x7fc21815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc218088770 tx=0x7fc210019000 comp rx=0 tx=0).stop 2023-10-06T11:30:27.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 shutdown_connections 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1fc060750 0x7fc1fc062c00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc21815a370 0x7fc21815a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2181591f0 0x7fc21813e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 --2- 172.21.15.46:0/3558361631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2180cc500 0x7fc21815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 >> 172.21.15.46:0/3558361631 conn(0x7fc21806ef60 msgr2=0x7fc2180b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 shutdown_connections 2023-10-06T11:30:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:27.955+0000 7fc21f523700 1 -- 172.21.15.46:0/3558361631 wait complete. 2023-10-06T11:30:27.985 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:30", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:27.986 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:30', 'first': '2023-10-06T11:20:00', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:27.986 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:27.986 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:27.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:27.987 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-06T11:30:28.008 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:28.009 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:28.011 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_01_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_01_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:28.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-10-06T11:30:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.219+0000 7f59c106b700 1 Processor -- start 2023-10-06T11:30:28.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.219+0000 7f59c106b700 1 -- start start 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 0x7f59bc159150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 0x7f59bc0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59bc0aec80 0x7f59bc0af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0adc20 con 0x7f59bc0aec80 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0adec0 con 0x7f59bc158d80 2023-10-06T11:30:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c106b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0ae1a0 con 0x7f59bc0d0770 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59bc0aec80 0x7f59bc0af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59bc0aec80 0x7f59bc0af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34630/0 (socket says 172.21.15.46:34630) 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 -- 172.21.15.46:0/2763856114 learned_addr learned my addr 172.21.15.46:0/2763856114 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59bb7fe700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 0x7f59bc0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 -- 172.21.15.46:0/2763856114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 msgr2=0x7f59bc0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2763856114 >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 0x7f59bc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 0x7f59bc0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 -- 172.21.15.46:0/2763856114 >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 msgr2=0x7f59bc159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.223+0000 7f59c086a700 1 --2- 172.21.15.46:0/2763856114 >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 0x7f59bc159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c086a700 1 -- 172.21.15.46:0/2763856114 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0ae480 con 0x7f59bc0aec80 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59bb7fe700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 0x7f59bc0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2763856114 >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 0x7f59bc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c086a700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc0aec80 0x7f59bc0af100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59bc063cd0 tx=0x7f59b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9646bb1fa3251f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59baffd700 1 -- 172.21.15.46:0/2763856114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400e070 con 0x7f59bc0aec80 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59baffd700 1 -- 172.21.15.46:0/2763856114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b400ba50 con 0x7f59bc0aec80 2023-10-06T11:30:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59baffd700 1 -- 172.21.15.46:0/2763856114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40092e0 con 0x7f59bc0aec80 2023-10-06T11:30:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 -- 172.21.15.46:0/2763856114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc0aec80 msgr2=0x7f59bc0af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc0aec80 0x7f59bc0af100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59bc063cd0 tx=0x7f59b4009840 comp rx=0 tx=0).stop 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 -- 172.21.15.46:0/2763856114 shutdown_connections 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc0aec80 0x7f59bc0af100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 --2- 172.21.15.46:0/2763856114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0d0770 0x7f59bc0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 --2- 172.21.15.46:0/2763856114 >> v2:172.21.15.132:3300/0 conn(0x7f59bc158d80 0x7f59bc159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 -- 172.21.15.46:0/2763856114 >> 172.21.15.46:0/2763856114 conn(0x7f59bc06ef60 msgr2=0x7f59bc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 -- 172.21.15.46:0/2763856114 shutdown_connections 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 -- 172.21.15.46:0/2763856114 wait complete. 2023-10-06T11:30:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.227+0000 7f59c106b700 1 Processor -- start 2023-10-06T11:30:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 -- start start 2023-10-06T11:30:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 0x7f59bc14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59bc0d0770 0x7f59bc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc14e100 con 0x7f59bc158d80 2023-10-06T11:30:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc06cb10 con 0x7f59bc0d0770 2023-10-06T11:30:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c106b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc09d5f0 con 0x7f59bc0aec80 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59bb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59bc0d0770 0x7f59bc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34632/0 (socket says 172.21.15.46:34632) 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 -- 172.21.15.46:0/2546062315 learned_addr learned my addr 172.21.15.46:0/2546062315 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 0x7f59bc14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 -- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 msgr2=0x7f59bc14dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 0x7f59bc14dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 -- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59bc0d0770 msgr2=0x7f59bc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59bc0d0770 0x7f59bc0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 -- 172.21.15.46:0/2546062315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009ae0 con 0x7f59bc158d80 2023-10-06T11:30:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 0x7f59bc14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.231+0000 7f59c086a700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009fd0 tx=0x7f59b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4008dc0 con 0x7f59bc158d80 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4026460 con 0x7f59bc158d80 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc06d670 con 0x7f59bc158d80 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4026820 con 0x7f59bc158d80 2023-10-06T11:30:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc06cdf0 con 0x7f59bc158d80 2023-10-06T11:30:28.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc06d800 con 0x7f59bc158d80 2023-10-06T11:30:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.235+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f59b4003040 con 0x7f59bc158d80 2023-10-06T11:30:28.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.239+0000 7f59a3fff700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 0x7f59a8062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.239+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f59b407ea30 con 0x7f59bc158d80 2023-10-06T11:30:28.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.239+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 0x7f59a8062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.239+0000 7f59bbfff700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 0x7f59a8062c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.243+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f59b404e970 con 0x7f59bc158d80 2023-10-06T11:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.439+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59bc0ae760 con 0x7f59a8060780 2023-10-06T11:30:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.455+0000 7f59a3fff700 1 -- 172.21.15.46:0/2546062315 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f59bc0ae760 con 0x7f59a8060780 2023-10-06T11:30:28.463 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:28.463 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:31", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 msgr2=0x7f59a8062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 0x7f59a8062c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 msgr2=0x7f59bc09cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009fd0 tx=0x7f59b4018000 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 shutdown_connections 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59a8060780 0x7f59a8062c30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59bc158d80 0x7f59bc09cb50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59bc0d0770 0x7f59bc0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 --2- 172.21.15.46:0/2546062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59bc0aec80 0x7f59bc14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 >> 172.21.15.46:0/2546062315 conn(0x7f59bc06ef60 msgr2=0x7f59bc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 shutdown_connections 2023-10-06T11:30:28.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.459+0000 7f59c106b700 1 -- 172.21.15.46:0/2546062315 wait complete. 2023-10-06T11:30:28.490 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:31", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:28.490 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:31', 'first': '2023-10-06T11:20:01', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:28.490 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:28.490 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:28.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:28.490 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:28.515 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_01_UTC 2023-10-06T11:30:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:28.516 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:28.517 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_00_UTC', 'scheduled-2023-10-06-11_21_01_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_01_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_01_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:28.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-10-06T11:30:28.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.731+0000 7f73258a5700 1 Processor -- start 2023-10-06T11:30:28.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.731+0000 7f73258a5700 1 -- start start 2023-10-06T11:30:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73200ccb30 0x7f73200c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73200ca530 con 0x7f732015a570 2023-10-06T11:30:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73200c9670 con 0x7f73200ccb30 2023-10-06T11:30:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200c9920 con 0x7f73200c91f0 2023-10-06T11:30:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731effd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f732015a570 0x7f73200cc420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43450/0 (socket says 172.21.15.46:43450) 2023-10-06T11:30:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731effd700 1 -- 172.21.15.46:0/2453471443 learned_addr learned my addr 172.21.15.46:0/2453471443 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 --2- 172.21.15.46:0/2453471443 >> v2:172.21.15.132:3300/0 conn(0x7f73200ccb30 0x7f73200c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731f7fe700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731dffb700 1 -- 172.21.15.46:0/2453471443 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870279725 0 0) 0x7f73200ca530 con 0x7f732015a570 2023-10-06T11:30:28.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731dffb700 1 -- 172.21.15.46:0/2453471443 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7308003570 con 0x7f732015a570 2023-10-06T11:30:28.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 -- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 msgr2=0x7f73200c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 -- 172.21.15.46:0/2453471443 >> v1:172.21.15.46:6789/0 conn(0x7f732015a570 legacy=0x7f73200cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 -- 172.21.15.46:0/2453471443 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200c9c00 con 0x7f73200ccb30 2023-10-06T11:30:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731e7fc700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f73200c8cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73140098e0 tx=0x7f73140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ef9afb1594a669 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731dffb700 1 -- 172.21.15.46:0/2453471443 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011630 con 0x7f73200ccb30 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731dffb700 1 -- 172.21.15.46:0/2453471443 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314011790 con 0x7f73200ccb30 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f731dffb700 1 -- 172.21.15.46:0/2453471443 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011b30 con 0x7f73200ccb30 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 -- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 msgr2=0x7f73200c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.735+0000 7f73258a5700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f73200c8cb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73140098e0 tx=0x7f73140091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- 172.21.15.46:0/2453471443 shutdown_connections 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 --2- 172.21.15.46:0/2453471443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f73200c8cb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- 172.21.15.46:0/2453471443 >> 172.21.15.46:0/2453471443 conn(0x7f7320097290 msgr2=0x7f7320067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- 172.21.15.46:0/2453471443 shutdown_connections 2023-10-06T11:30:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- 172.21.15.46:0/2453471443 wait complete. 2023-10-06T11:30:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 Processor -- start 2023-10-06T11:30:28.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- start start 2023-10-06T11:30:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f7320156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320098d50 con 0x7f732015a570 2023-10-06T11:30:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732009a780 con 0x7f73200ccb30 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.739+0000 7f73258a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732009aa60 con 0x7f73200c91f0 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34642/0 (socket says 172.21.15.46:34642) 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 -- 172.21.15.46:0/4072313603 learned_addr learned my addr 172.21.15.46:0/4072313603 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731e7fc700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f7320156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731effd700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 -- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 msgr2=0x7f73200c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 -- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 msgr2=0x7f7320156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f7320156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 -- 172.21.15.46:0/4072313603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009420 con 0x7f732015a570 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f731f7fe700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7318009e90 tx=0x7f731800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731800cb70 con 0x7f732015a570 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731800f040 con 0x7f732015a570 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318014480 con 0x7f732015a570 2023-10-06T11:30:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732009ad40 con 0x7f732015a570 2023-10-06T11:30:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.743+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7320156cb0 con 0x7f732015a570 2023-10-06T11:30:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.747+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732009af30 con 0x7f732015a570 2023-10-06T11:30:28.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.747+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f7318014830 con 0x7f732015a570 2023-10-06T11:30:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.747+0000 7f7306ffd700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 0x7f730c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.751+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f731807d930 con 0x7f732015a570 2023-10-06T11:30:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.751+0000 7f731effd700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 0x7f730c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:28.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.751+0000 7f731effd700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 0x7f730c062c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7310001360 tx=0x7f7310009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.755+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f731804d920 con 0x7f732015a570 2023-10-06T11:30:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.943+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f732006cf20 con 0x7f730c060750 2023-10-06T11:30:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.959+0000 7f7306ffd700 1 -- 172.21.15.46:0/4072313603 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f732006cf20 con 0x7f730c060750 2023-10-06T11:30:28.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:28.969 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:31", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 msgr2=0x7f730c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 0x7f730c062c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7310001360 tx=0x7f7310009040 comp rx=0 tx=0).stop 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 msgr2=0x7f7320098370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7318009e90 tx=0x7f731800b040 comp rx=0 tx=0).stop 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 shutdown_connections 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f730c060750 0x7f730c062c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f732015a570 0x7f7320098370 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73200ccb30 0x7f7320156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 --2- 172.21.15.46:0/4072313603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73200c91f0 0x7f73200c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 >> 172.21.15.46:0/4072313603 conn(0x7f7320097290 msgr2=0x7f7320157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 shutdown_connections 2023-10-06T11:30:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:28.963+0000 7f73258a5700 1 -- 172.21.15.46:0/4072313603 wait complete. 2023-10-06T11:30:28.992 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:31", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:28.993 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:31', 'first': '2023-10-06T11:20:00', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:28.993 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:28.993 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:28.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:28.993 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:29.011 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:29.012 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:29.012 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:29.012 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:29.012 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:29.013 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_01_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:29.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-10-06T11:30:29.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.215+0000 7fb78f523700 1 Processor -- start 2023-10-06T11:30:29.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.215+0000 7fb78f523700 1 -- start start 2023-10-06T11:30:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78f523700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb7881591f0 0x7fb7880cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78f523700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7880cc500 0x7fb78815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78f523700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7880cf360 con 0x7fb7881595c0 2023-10-06T11:30:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78f523700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78815dd60 con 0x7fb7880cc500 2023-10-06T11:30:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78f523700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7880cea30 con 0x7fb7881591f0 2023-10-06T11:30:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7880cc500 0x7fb78815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78dd20700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb7881595c0 0x7fb7880af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43454/0 (socket says 172.21.15.46:43454) 2023-10-06T11:30:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78dd20700 1 -- 172.21.15.46:0/3923556728 learned_addr learned my addr 172.21.15.46:0/3923556728 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78e521700 1 --2- 172.21.15.46:0/3923556728 >> v2:172.21.15.132:3300/0 conn(0x7fb7881591f0 0x7fb7880cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 -- 172.21.15.46:0/3923556728 >> v2:172.21.15.132:3300/0 conn(0x7fb7881591f0 msgr2=0x7fb7880cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 --2- 172.21.15.46:0/3923556728 >> v2:172.21.15.132:3300/0 conn(0x7fb7881591f0 0x7fb7880cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 -- 172.21.15.46:0/3923556728 >> v1:172.21.15.46:6789/0 conn(0x7fb7881595c0 legacy=0x7fb7880af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 -- 172.21.15.46:0/3923556728 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880ced10 con 0x7fb7880cc500 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78ed22700 1 --2- 172.21.15.46:0/3923556728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880cc500 0x7fb78815ceb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7880892e0 tx=0x7fb780009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3068d933257496b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78d51f700 1 -- 172.21.15.46:0/3923556728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78000e070 con 0x7fb7880cc500 2023-10-06T11:30:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78d51f700 1 -- 172.21.15.46:0/3923556728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78000bcc0 con 0x7fb7880cc500 2023-10-06T11:30:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.219+0000 7fb78d51f700 1 -- 172.21.15.46:0/3923556728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780012680 con 0x7fb7880cc500 2023-10-06T11:30:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- 172.21.15.46:0/3923556728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880cc500 msgr2=0x7fb78815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 --2- 172.21.15.46:0/3923556728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880cc500 0x7fb78815ceb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7880892e0 tx=0x7fb780009780 comp rx=0 tx=0).stop 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- 172.21.15.46:0/3923556728 shutdown_connections 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 --2- 172.21.15.46:0/3923556728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7880cc500 0x7fb78815ceb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 --2- 172.21.15.46:0/3923556728 >> v2:172.21.15.132:3300/0 conn(0x7fb7881591f0 0x7fb7880cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- 172.21.15.46:0/3923556728 >> 172.21.15.46:0/3923556728 conn(0x7fb78806ef60 msgr2=0x7fb7880d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- 172.21.15.46:0/3923556728 shutdown_connections 2023-10-06T11:30:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- 172.21.15.46:0/3923556728 wait complete. 2023-10-06T11:30:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 Processor -- start 2023-10-06T11:30:29.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.223+0000 7fb78f523700 1 -- start start 2023-10-06T11:30:29.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7880cc500 0x7fb788155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7881591f0 0x7fb7880af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788156fa0 con 0x7fb788141a90 2023-10-06T11:30:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880afe50 con 0x7fb7880cc500 2023-10-06T11:30:29.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7880b0ed0 con 0x7fb7881591f0 2023-10-06T11:30:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34660/0 (socket says 172.21.15.46:34660) 2023-10-06T11:30:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 -- 172.21.15.46:0/4179111085 learned_addr learned my addr 172.21.15.46:0/4179111085 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78e521700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7880cc500 0x7fb788155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78dd20700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7881591f0 0x7fb7880af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 -- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7881591f0 msgr2=0x7fb7880af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7881591f0 0x7fb7880af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 -- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7880cc500 msgr2=0x7fb788155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7880cc500 0x7fb788155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 -- 172.21.15.46:0/4179111085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780009a20 con 0x7fb788141a90 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78ed22700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb788088770 tx=0x7fb780019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780027470 con 0x7fb788141a90 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7800275f0 con 0x7fb788141a90 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7880b0080 con 0x7fb788141a90 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800279b0 con 0x7fb788141a90 2023-10-06T11:30:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.227+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7880b0640 con 0x7fb788141a90 2023-10-06T11:30:29.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.231+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880b1080 con 0x7fb788141a90 2023-10-06T11:30:29.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.231+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fb780024070 con 0x7fb788141a90 2023-10-06T11:30:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.235+0000 7fb77e7fc700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 0x7fb76c062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.235+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fb78007f900 con 0x7fb788141a90 2023-10-06T11:30:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.235+0000 7fb78e521700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 0x7fb76c062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.235+0000 7fb78e521700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 0x7fb76c062c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.239+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb7880b1080 con 0x7fb788141a90 2023-10-06T11:30:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.431+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7880b0860 con 0x7fb76c0607d0 2023-10-06T11:30:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.447+0000 7fb77e7fc700 1 -- 172.21.15.46:0/4179111085 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fb7880b0860 con 0x7fb76c0607d0 2023-10-06T11:30:29.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:29.456 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:32", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:29.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 msgr2=0x7fb76c062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 0x7fb76c062c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 msgr2=0x7fb7880689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb788088770 tx=0x7fb780019000 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 shutdown_connections 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb76c0607d0 0x7fb76c062c80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb788141a90 0x7fb7880689d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7881591f0 0x7fb7880af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 --2- 172.21.15.46:0/4179111085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7880cc500 0x7fb788155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 >> 172.21.15.46:0/4179111085 conn(0x7fb78806ef60 msgr2=0x7fb7881530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 shutdown_connections 2023-10-06T11:30:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.451+0000 7fb78f523700 1 -- 172.21.15.46:0/4179111085 wait complete. 2023-10-06T11:30:29.481 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:32", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:29.482 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:32', 'first': '2023-10-06T11:20:01', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:29.482 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:29.482 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:29.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:29.482 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-06T11:30:29.506 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:29.507 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:29.508 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:29.508 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_01_UTC 2023-10-06T11:30:29.508 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:29.508 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:29.509 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_00_UTC', 'scheduled-2023-10-06-11_21_01_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_01_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_01_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_01_UTC'] 2023-10-06T11:30:29.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-10-06T11:30:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.715+0000 7fa17f0ae700 1 Processor -- start 2023-10-06T11:30:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.715+0000 7fa17f0ae700 1 -- start start 2023-10-06T11:30:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17f0ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1780ca3a0 0x7fa1780c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17f0ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159420 0x7fa1780c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17f0ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780c71a0 con 0x7fa178159050 2023-10-06T11:30:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17f0ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780c8650 con 0x7fa1780ca3a0 2023-10-06T11:30:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17f0ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780c1bd0 con 0x7fa178159420 2023-10-06T11:30:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa177fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1780ca3a0 0x7fa1780c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17ce4a700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa178159050 0x7fa1780c9c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:33900/0 (socket says 172.21.15.46:33900) 2023-10-06T11:30:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17ce4a700 1 -- 172.21.15.46:0/1365272046 learned_addr learned my addr 172.21.15.46:0/1365272046 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa17d64b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159420 0x7fa1780c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa177fff700 1 -- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159420 msgr2=0x7fa1780c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa177fff700 1 --2- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159420 0x7fa1780c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa177fff700 1 -- 172.21.15.46:0/1365272046 >> v1:172.21.15.132:6789/0 conn(0x7fa178159050 legacy=0x7fa1780c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.719+0000 7fa177fff700 1 -- 172.21.15.46:0/1365272046 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780c6590 con 0x7fa1780ca3a0 2023-10-06T11:30:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa177fff700 1 --2- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa1780c7bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098e0 tx=0x7fa1640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc4efac0192fdf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa1777fe700 1 -- 172.21.15.46:0/1365272046 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400b640 con 0x7fa1780ca3a0 2023-10-06T11:30:29.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa1777fe700 1 -- 172.21.15.46:0/1365272046 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16400b7a0 con 0x7fa1780ca3a0 2023-10-06T11:30:29.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa1777fe700 1 -- 172.21.15.46:0/1365272046 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400bb40 con 0x7fa1780ca3a0 2023-10-06T11:30:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 msgr2=0x7fa1780c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa1780c7bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa1640098e0 tx=0x7fa1640091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1365272046 shutdown_connections 2023-10-06T11:30:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159420 0x7fa1780c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1365272046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa1780c7bd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1365272046 >> 172.21.15.46:0/1365272046 conn(0x7fa1780cbe20 msgr2=0x7fa1780cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1365272046 shutdown_connections 2023-10-06T11:30:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.723+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1365272046 wait complete. 2023-10-06T11:30:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 Processor -- start 2023-10-06T11:30:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 -- start start 2023-10-06T11:30:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa17814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159050 0x7fa1780ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780addd0 con 0x7fa178159420 2023-10-06T11:30:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781541f0 con 0x7fa1780ca3a0 2023-10-06T11:30:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17f0ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780ad1c0 con 0x7fa178159050 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17d64b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.727+0000 7fa17d64b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34674/0 (socket says 172.21.15.46:34674) 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 -- 172.21.15.46:0/1338371335 learned_addr learned my addr 172.21.15.46:0/1338371335 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17ce4a700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa17814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa177fff700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159050 0x7fa1780ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 -- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159050 msgr2=0x7fa1780ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159050 0x7fa1780ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 -- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 msgr2=0x7fa17814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa17814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 -- 172.21.15.46:0/1338371335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164009420 con 0x7fa178159420 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17d64b700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa170009d40 tx=0x7fa17000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170009ea0 con 0x7fa178159420 2023-10-06T11:30:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17000f040 con 0x7fa178159420 2023-10-06T11:30:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000ccf0 con 0x7fa178159420 2023-10-06T11:30:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1780ad4a0 con 0x7fa178159420 2023-10-06T11:30:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780ada60 con 0x7fa178159420 2023-10-06T11:30:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.731+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1780adf40 con 0x7fa178159420 2023-10-06T11:30:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.735+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fa170014690 con 0x7fa178159420 2023-10-06T11:30:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.735+0000 7fa174ff9700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 0x7fa15c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.735+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fa17007da00 con 0x7fa178159420 2023-10-06T11:30:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.735+0000 7fa17ce4a700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 0x7fa15c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.735+0000 7fa17ce4a700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 0x7fa15c062c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007130 tx=0x7fa16c007eb0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.743+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa17004d940 con 0x7fa178159420 2023-10-06T11:30:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.939+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa17806eed0 con 0x7fa15c060750 2023-10-06T11:30:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.951+0000 7fa174ff9700 1 -- 172.21.15.46:0/1338371335 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa17806eed0 con 0x7fa15c060750 2023-10-06T11:30:29.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:29.962 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:33", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 msgr2=0x7fa15c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 0x7fa15c062c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007130 tx=0x7fa16c007eb0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 msgr2=0x7fa1780af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa170009d40 tx=0x7fa17000b040 comp rx=0 tx=0).stop 2023-10-06T11:30:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 shutdown_connections 2023-10-06T11:30:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa15c060750 0x7fa15c062c00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa178159420 0x7fa1780af560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa178159050 0x7fa1780ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 --2- 172.21.15.46:0/1338371335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1780ca3a0 0x7fa17814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 >> 172.21.15.46:0/1338371335 conn(0x7fa1780cbe20 msgr2=0x7fa1780a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 shutdown_connections 2023-10-06T11:30:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:29.955+0000 7fa17f0ae700 1 -- 172.21.15.46:0/1338371335 wait complete. 2023-10-06T11:30:29.988 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:33", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:29.988 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:33', 'first': '2023-10-06T11:20:00', 'last': '2023-10-06T11:30:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:29.988 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:29.988 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:29.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:29.989 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-06T11:30:30.015 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:30.015 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_01_UTC 2023-10-06T11:30:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:30.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:30.017 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:30.018 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_01_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_01_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_00_UTC'] 2023-10-06T11:30:30.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-10-06T11:30:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.235+0000 7f872d2e3700 1 Processor -- start 2023-10-06T11:30:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.235+0000 7f872d2e3700 1 -- start start 2023-10-06T11:30:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f872d2e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87281595c0 0x7f87280af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f872d2e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87280cf360 con 0x7f87281591f0 2023-10-06T11:30:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f872d2e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872815dd60 con 0x7f87280cc500 2023-10-06T11:30:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f872d2e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280cff00 con 0x7f87281595c0 2023-10-06T11:30:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8727fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87281591f0 0x7f87280cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43466/0 (socket says 172.21.15.46:43466) 2023-10-06T11:30:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8727fff700 1 -- 172.21.15.46:0/1070310001 learned_addr learned my addr 172.21.15.46:0/1070310001 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87281595c0 0x7f87280af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438852466 0 0) 0x7f87280cf360 con 0x7f87281591f0 2023-10-06T11:30:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8710003570 con 0x7f87281591f0 2023-10-06T11:30:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210022373 0 0) 0x7f872815dd60 con 0x7f87280cc500 2023-10-06T11:30:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87280cf360 con 0x7f87280cc500 2023-10-06T11:30:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187220456 0 0) 0x7f8710003570 con 0x7f87281591f0 2023-10-06T11:30:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87281595c0 msgr2=0x7f87280af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87281595c0 0x7f87280af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 >> v1:172.21.15.132:6789/0 conn(0x7f87280cc500 legacy=0x7f872815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280cea30 con 0x7f87281591f0 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f87180017f0 con 0x7f87281591f0 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 >> v1:172.21.15.46:6789/0 conn(0x7f87281591f0 legacy=0x7f87280cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8710003c20 0x7f8710004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f8710004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87100051c0 0x7f871000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280cff00 con 0x7f8710004810 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871000bfa0 con 0x7f8710003c20 2023-10-06T11:30:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871000c0e0 con 0x7f87100051c0 2023-10-06T11:30:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f8710004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f8727fff700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8710003c20 0x7f8710004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f872cae2700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87100051c0 0x7f871000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 -- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87100051c0 msgr2=0x7f871000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87100051c0 0x7f871000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 -- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8710003c20 msgr2=0x7f8710004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8710003c20 0x7f8710004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.239+0000 7f87277fe700 1 -- 172.21.15.46:0/1070310001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710003570 con 0x7f8710004810 2023-10-06T11:30:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f87277fe700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f8710004c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f871c009c30 tx=0x7f871c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a1080305a841f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00e710 con 0x7f8710004810 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00e870 con 0x7f8710004810 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f8726ffd700 1 -- 172.21.15.46:0/1070310001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00ec30 con 0x7f8710004810 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 -- 172.21.15.46:0/1070310001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 msgr2=0x7f8710004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 --2- 172.21.15.46:0/1070310001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f8710004c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f871c009c30 tx=0x7f871c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872cae2700 1 -- 172.21.15.46:0/1070310001 reap_dead start 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 -- 172.21.15.46:0/1070310001 shutdown_connections 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 -- 172.21.15.46:0/1070310001 >> 172.21.15.46:0/1070310001 conn(0x7f872806ef60 msgr2=0x7f87280d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 -- 172.21.15.46:0/1070310001 shutdown_connections 2023-10-06T11:30:30.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.243+0000 7f872d2e3700 1 -- 172.21.15.46:0/1070310001 wait complete. 2023-10-06T11:30:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 Processor -- start 2023-10-06T11:30:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 -- start start 2023-10-06T11:30:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f872814ca10 0x7f87280b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87280b1780 0x7f87280b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872816e860 con 0x7f8710004810 2023-10-06T11:30:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872816dc50 con 0x7f87280b1780 2023-10-06T11:30:30.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872d2e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872816df30 con 0x7f872814ca10 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34684/0 (socket says 172.21.15.46:34684) 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 -- 172.21.15.46:0/3383783180 learned_addr learned my addr 172.21.15.46:0/3383783180 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f87277fe700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f872814ca10 0x7f87280b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f872cae2700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87280b1780 0x7f87280b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 -- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f872814ca10 msgr2=0x7f87280b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f872814ca10 0x7f87280b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 -- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87280b1780 msgr2=0x7f87280b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87280b1780 0x7f87280b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.247+0000 7f8727fff700 1 -- 172.21.15.46:0/3383783180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872816e210 con 0x7f8710004810 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f8727fff700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8718007ac0 tx=0x7f8718002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871800acc0 con 0x7f8710004810 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871800ae20 con 0x7f8710004810 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718013640 con 0x7f8710004810 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872816e4f0 con 0x7f8710004810 2023-10-06T11:30:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872816d870 con 0x7f8710004810 2023-10-06T11:30:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.251+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8728064590 con 0x7f8710004810 2023-10-06T11:30:30.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.255+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f8718013a30 con 0x7f8710004810 2023-10-06T11:30:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.255+0000 7f870ffff700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 0x7f8714062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.255+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f871807da20 con 0x7f8710004810 2023-10-06T11:30:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.255+0000 7f87277fe700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 0x7f8714062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.255+0000 7f87277fe700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 0x7f8714062c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f871c0091e0 tx=0x7f871c01a000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.263+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f871804da70 con 0x7f8710004810 2023-10-06T11:30:30.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.447+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f872806e560 con 0x7f8714060750 2023-10-06T11:30:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.467+0000 7f870ffff700 1 -- 172.21.15.46:0/3383783180 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f872806e560 con 0x7f8714060750 2023-10-06T11:30:30.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:30.475 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:33", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:30.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 msgr2=0x7f8714062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 0x7f8714062c00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f871c0091e0 tx=0x7f871c01a000 comp rx=0 tx=0).stop 2023-10-06T11:30:30.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 msgr2=0x7f872814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8718007ac0 tx=0x7f8718002190 comp rx=0 tx=0).stop 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 shutdown_connections 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8714060750 0x7f8714062c00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87280b1780 0x7f87280b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f872814ca10 0x7f87280b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 --2- 172.21.15.46:0/3383783180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8710004810 0x7f872814d350 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 >> 172.21.15.46:0/3383783180 conn(0x7f872806ef60 msgr2=0x7f87280b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 shutdown_connections 2023-10-06T11:30:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.471+0000 7f872d2e3700 1 -- 172.21.15.46:0/3383783180 wait complete. 2023-10-06T11:30:30.501 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:33", "first": "2023-10-06T11:20:01", "last": "2023-10-06T11:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:30.501 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:33', 'first': '2023-10-06T11:20:01', 'last': '2023-10-06T11:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:30.501 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:30.501 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:30.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:30.502 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-06T11:30:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:30.524 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:30.524 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:30.524 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:30.525 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_20_00_UTC', 'scheduled-2023-10-06-11_21_00_UTC', 'scheduled-2023-10-06-11_22_00_UTC', 'scheduled-2023-10-06-11_23_00_UTC', 'scheduled-2023-10-06-11_24_00_UTC', 'scheduled-2023-10-06-11_25_00_UTC', 'scheduled-2023-10-06-11_26_00_UTC', 'scheduled-2023-10-06-11_27_00_UTC', 'scheduled-2023-10-06-11_28_00_UTC', 'scheduled-2023-10-06-11_29_00_UTC', 'scheduled-2023-10-06-11_30_01_UTC'] 2023-10-06T11:30:30.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-10-06T11:30:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.735+0000 7fa28d3cc700 1 Processor -- start 2023-10-06T11:30:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.735+0000 7fa28d3cc700 1 -- start start 2023-10-06T11:30:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28d3cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28d3cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880b5eb0 con 0x7fa288069770 2023-10-06T11:30:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28d3cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2880cef30 con 0x7fa2881591b0 2023-10-06T11:30:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28d3cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2880cf4f0 con 0x7fa288158de0 2023-10-06T11:30:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28cbcb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa288069770 0x7fa2880b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43482/0 (socket says 172.21.15.46:43482) 2023-10-06T11:30:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28cbcb700 1 -- 172.21.15.46:0/1590299969 learned_addr learned my addr 172.21.15.46:0/1590299969 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa287fff700 1 --2- 172.21.15.46:0/1590299969 >> v2:172.21.15.132:3300/0 conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848472016 0 0) 0x7fa280001490 con 0x7fa288069770 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa270003570 con 0x7fa288069770 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172576612 0 0) 0x7fa2880cef30 con 0x7fa2881591b0 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa280001490 con 0x7fa2881591b0 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214921729 0 0) 0x7fa270003570 con 0x7fa288069770 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 >> v2:172.21.15.132:3300/0 conn(0x7fa288158de0 msgr2=0x7fa2880b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1590299969 >> v2:172.21.15.132:3300/0 conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 >> v1:172.21.15.132:6789/0 conn(0x7fa2881591b0 legacy=0x7fa288069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880ccac0 con 0x7fa288069770 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fa280001980 con 0x7fa288069770 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa287fff700 1 --2- 172.21.15.46:0/1590299969 >> v2:172.21.15.132:3300/0 conn(0x7fa288158de0 0x7fa2880b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 >> v1:172.21.15.46:6789/0 conn(0x7fa288069770 legacy=0x7fa2880b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa270003c40 0x7fa270004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa270005210 0x7fa27000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2880cf4f0 con 0x7fa270004810 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27000c130 con 0x7fa270003c40 2023-10-06T11:30:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27000c2c0 con 0x7fa270005210 2023-10-06T11:30:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa287fff700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa270003c40 0x7fa270004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa28cbcb700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa270005210 0x7fa27000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa2877fe700 1 -- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa270005210 msgr2=0x7fa27000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa270005210 0x7fa27000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa2877fe700 1 -- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa270003c40 msgr2=0x7fa270004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.739+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa270003c40 0x7fa270004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28cbcb700 1 -- 172.21.15.46:0/1590299969 reap_dead start 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa2877fe700 1 -- 172.21.15.46:0/1590299969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270003570 con 0x7fa270004810 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa2877fe700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa270004c80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0019e0 tx=0x7fa27c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98a03114179a311 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00c550 con 0x7fa270004810 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00c6b0 con 0x7fa270004810 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa286ffd700 1 -- 172.21.15.46:0/1590299969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00ca50 con 0x7fa270004810 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 -- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 msgr2=0x7fa270004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa270004c80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0019e0 tx=0x7fa27c007d30 comp rx=0 tx=0).stop 2023-10-06T11:30:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 -- 172.21.15.46:0/1590299969 shutdown_connections 2023-10-06T11:30:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/1590299969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa270004c80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 -- 172.21.15.46:0/1590299969 >> 172.21.15.46:0/1590299969 conn(0x7fa288068390 msgr2=0x7fa2880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 -- 172.21.15.46:0/1590299969 shutdown_connections 2023-10-06T11:30:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.743+0000 7fa28d3cc700 1 -- 172.21.15.46:0/1590299969 wait complete. 2023-10-06T11:30:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 Processor -- start 2023-10-06T11:30:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 -- start start 2023-10-06T11:30:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28815b540 0x7fa28816c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa28816c880 0x7fa28816ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28815c020 con 0x7fa270004810 2023-10-06T11:30:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28816a700 con 0x7fa28815b540 2023-10-06T11:30:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa28d3cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28815bd40 con 0x7fa28816c880 2023-10-06T11:30:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa287fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa287fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34704/0 (socket says 172.21.15.46:34704) 2023-10-06T11:30:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa287fff700 1 -- 172.21.15.46:0/3893636914 learned_addr learned my addr 172.21.15.46:0/3893636914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.747+0000 7fa2877fe700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28815b540 0x7fa28816c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa28cbcb700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa28816c880 0x7fa28816ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 -- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa28816c880 msgr2=0x7fa28816ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa28816c880 0x7fa28816ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 -- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28815b540 msgr2=0x7fa28816c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28815b540 0x7fa28816c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 -- 172.21.15.46:0/3893636914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28816a0b0 con 0x7fa270004810 2023-10-06T11:30:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa287fff700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa278011d20 tx=0x7fa2780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278011ed0 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27800fcd0 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278014670 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28816a390 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2881685a0 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.751+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa288169f10 con 0x7fa270004810 2023-10-06T11:30:30.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.755+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fa278014aa0 con 0x7fa270004810 2023-10-06T11:30:30.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.755+0000 7fa26ffff700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 0x7fa274062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:30.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.755+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fa278018030 con 0x7fa270004810 2023-10-06T11:30:30.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.755+0000 7fa2877fe700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 0x7fa274062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.759+0000 7fa2877fe700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 0x7fa274062cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c015000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:30.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.763+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa288169f10 con 0x7fa270004810 2023-10-06T11:30:30.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.951+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa288169f10 con 0x7fa274060800 2023-10-06T11:30:30.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.967+0000 7fa26ffff700 1 -- 172.21.15.46:0/3893636914 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa288169f10 con 0x7fa274060800 2023-10-06T11:30:30.976 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:30.976 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:34", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 msgr2=0x7fa274062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 0x7fa274062cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c015000 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 msgr2=0x7fa28815b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa278011d20 tx=0x7fa2780091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 shutdown_connections 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa274060800 0x7fa274062cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa28816c880 0x7fa28816ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa28815b540 0x7fa28816c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 --2- 172.21.15.46:0/3893636914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa270004810 0x7fa28815b000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:30.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 >> 172.21.15.46:0/3893636914 conn(0x7fa288068390 msgr2=0x7fa28815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:30.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 shutdown_connections 2023-10-06T11:30:30.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:30.971+0000 7fa28d3cc700 1 -- 172.21.15.46:0/3893636914 wait complete. 2023-10-06T11:30:31.001 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:19:34", "first": "2023-10-06T11:20:00", "last": "2023-10-06T11:30:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T11:30:31.001 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:19:34', 'first': '2023-10-06T11:20:00', 'last': '2023-10-06T11:30:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T11:30:31.001 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T11:30:31.001 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T11:30:31.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T11:30:31.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.167+0000 7f6559e6e700 1 Processor -- start 2023-10-06T11:30:31.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.167+0000 7f6559e6e700 1 -- start start 2023-10-06T11:30:31.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6559e6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6554153870 0x7f655406e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:31.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6559e6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65540c6870 con 0x7f6554156bf0 2023-10-06T11:30:31.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6559e6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655406e8a0 con 0x7f6554153c40 2023-10-06T11:30:31.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6559e6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655406e690 con 0x7f6554153870 2023-10-06T11:30:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6553fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6554156bf0 0x7f65540c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43496/0 (socket says 172.21.15.46:43496) 2023-10-06T11:30:31.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f6553fff700 1 -- 172.21.15.46:0/2982581466 learned_addr learned my addr 172.21.15.46:0/2982581466 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.171+0000 7f65537fe700 1 --2- 172.21.15.46:0/2982581466 >> v2:172.21.15.132:3300/0 conn(0x7f6554153870 0x7f655406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164791002 0 0) 0x7f65540c6870 con 0x7f6554156bf0 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c003570 con 0x7f6554156bf0 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962681252 0 0) 0x7f655406e8a0 con 0x7f6554153c40 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65540c6870 con 0x7f6554153c40 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2174427281 0 0) 0x7f653c003570 con 0x7f6554156bf0 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65537fe700 1 -- 172.21.15.46:0/2982581466 >> v1:172.21.15.132:6789/0 conn(0x7f6554153c40 legacy=0x7f65541564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65537fe700 1 -- 172.21.15.46:0/2982581466 >> v1:172.21.15.46:6789/0 conn(0x7f6554156bf0 legacy=0x7f65540c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65537fe700 1 -- 172.21.15.46:0/2982581466 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655406eb80 con 0x7f6554153870 2023-10-06T11:30:31.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65537fe700 1 --2- 172.21.15.46:0/2982581466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153870 0x7f655406e150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65440098e0 tx=0x7f65440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2246690bf3773b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0017f0 con 0x7f6554153870 2023-10-06T11:30:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544011420 con 0x7f6554153870 2023-10-06T11:30:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f65527fc700 1 -- 172.21.15.46:0/2982581466 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440117a0 con 0x7f6554153870 2023-10-06T11:30:31.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 -- 172.21.15.46:0/2982581466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153870 msgr2=0x7f655406e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 --2- 172.21.15.46:0/2982581466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153870 0x7f655406e150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65440098e0 tx=0x7f65440091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 -- 172.21.15.46:0/2982581466 shutdown_connections 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 --2- 172.21.15.46:0/2982581466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153870 0x7f655406e150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 -- 172.21.15.46:0/2982581466 >> 172.21.15.46:0/2982581466 conn(0x7f65540cbe20 msgr2=0x7f65540cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 -- 172.21.15.46:0/2982581466 shutdown_connections 2023-10-06T11:30:31.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.175+0000 7f6559e6e700 1 -- 172.21.15.46:0/2982581466 wait complete. 2023-10-06T11:30:31.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.179+0000 7f6559e6e700 1 Processor -- start 2023-10-06T11:30:31.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.179+0000 7f6559e6e700 1 -- start start 2023-10-06T11:30:31.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153c40 0x7f65540c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 0x7f65540c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554159610 con 0x7f6554153870 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65541593e0 con 0x7f6554153c40 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655416dbd0 con 0x7f6554156bf0 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34708/0 (socket says 172.21.15.46:34708) 2023-10-06T11:30:31.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 -- 172.21.15.46:0/1157099402 learned_addr learned my addr 172.21.15.46:0/1157099402 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:31.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6552ffd700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153c40 0x7f65540c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:31.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6553fff700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 0x7f65540c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 -- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 msgr2=0x7f65540c1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 0x7f65540c1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 -- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153c40 msgr2=0x7f65540c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153c40 0x7f65540c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 -- 172.21.15.46:0/1157099402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6544009420 con 0x7f6554153870 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6553fff700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 0x7f65540c1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f65537fe700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65440093f0 tx=0x7f6544016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654401cbe0 con 0x7f6554153870 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654401cd60 con 0x7f6554153870 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654401a6a0 con 0x7f6554153870 2023-10-06T11:30:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655416ce30 con 0x7f6554153870 2023-10-06T11:30:31.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.183+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655416d3f0 con 0x7f6554153870 2023-10-06T11:30:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.187+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655406b890 con 0x7f6554153870 2023-10-06T11:30:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.187+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f654401aa90 con 0x7f6554153870 2023-10-06T11:30:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.187+0000 7f653b7fe700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 0x7f6540062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.187+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f654407e230 con 0x7f6554153870 2023-10-06T11:30:31.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.187+0000 7f6552ffd700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 0x7f6540062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:31.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.191+0000 7f6552ffd700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 0x7f6540062c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65540c28d0 tx=0x7f6548007d50 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.195+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f654404e330 con 0x7f6554153870 2023-10-06T11:30:31.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.391+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f655404b5c0 con 0x7f6540060780 2023-10-06T11:30:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.527+0000 7f653b7fe700 1 -- 172.21.15.46:0/1157099402 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f655404b5c0 con 0x7f6540060780 2023-10-06T11:30:31.535 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 msgr2=0x7f6540062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 0x7f6540062c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65540c28d0 tx=0x7f6548007d50 comp rx=0 tx=0).stop 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 msgr2=0x7f6554158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65440093f0 tx=0x7f6544016000 comp rx=0 tx=0).stop 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 shutdown_connections 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6540060780 0x7f6540062c30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6554156bf0 0x7f65540c1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6554153c40 0x7f65540c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 --2- 172.21.15.46:0/1157099402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6554153870 0x7f6554158ea0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 >> 172.21.15.46:0/1157099402 conn(0x7f65540cbe20 msgr2=0x7f65540cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 shutdown_connections 2023-10-06T11:30:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:31.531+0000 7f6559e6e700 1 -- 172.21.15.46:0/1157099402 wait complete. 2023-10-06T11:30:31.563 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-06T11:30:31.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.564 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-06T11:30:31.584 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:31.585 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:31.586 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:31.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.587 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_20_01_UTC) 2023-10-06T11:30:31.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:31.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.666 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:31.747 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:31.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.748 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:31.827 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:31.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.828 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:31.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:31.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.912 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:31.993 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:31.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:31.993 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:32.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:32.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.080 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:32.167 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:32.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.168 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:32.248 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:32.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.249 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:32.330 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:32.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.331 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:32.405 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:32.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.405 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:32.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:32.481 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-06T11:30:32.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T11:30:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.755+0000 7fdc5d345700 1 Processor -- start 2023-10-06T11:30:32.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.759+0000 7fdc5d345700 1 -- start start 2023-10-06T11:30:32.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.759+0000 7fdc5d345700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc58158e60 0x7fdc58068f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdc580694c0 0x7fdc580a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc580c6280 con 0x7fdc580a9d80 2023-10-06T11:30:32.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580c64d0 con 0x7fdc58158e60 2023-10-06T11:30:32.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580c67b0 con 0x7fdc580694c0 2023-10-06T11:30:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc58158e60 0x7fdc58068f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc577fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdc580a9d80 0x7fdc58069c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43510/0 (socket says 172.21.15.46:43510) 2023-10-06T11:30:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc577fe700 1 -- 172.21.15.46:0/2629596764 learned_addr learned my addr 172.21.15.46:0/2629596764 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:32.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc567fc700 1 --2- 172.21.15.46:0/2629596764 >> v2:172.21.15.132:3300/0 conn(0x7fdc580694c0 0x7fdc580a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 -- 172.21.15.46:0/2629596764 >> v2:172.21.15.132:3300/0 conn(0x7fdc580694c0 msgr2=0x7fdc580a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 --2- 172.21.15.46:0/2629596764 >> v2:172.21.15.132:3300/0 conn(0x7fdc580694c0 0x7fdc580a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 -- 172.21.15.46:0/2629596764 >> v1:172.21.15.46:6789/0 conn(0x7fdc580a9d80 legacy=0x7fdc58069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 -- 172.21.15.46:0/2629596764 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580c6a90 con 0x7fdc58158e60 2023-10-06T11:30:32.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc55ffb700 1 -- 172.21.15.46:0/2629596764 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113526375 0 0) 0x7fdc580c6280 con 0x7fdc580a9d80 2023-10-06T11:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc56ffd700 1 --2- 172.21.15.46:0/2629596764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc58158e60 0x7fdc58068f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc580cc560 tx=0x7fdc480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6486456dc458c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc55ffb700 1 -- 172.21.15.46:0/2629596764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011630 con 0x7fdc58158e60 2023-10-06T11:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc55ffb700 1 -- 172.21.15.46:0/2629596764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48011790 con 0x7fdc58158e60 2023-10-06T11:30:32.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc55ffb700 1 -- 172.21.15.46:0/2629596764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011b30 con 0x7fdc58158e60 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- 172.21.15.46:0/2629596764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc58158e60 msgr2=0x7fdc58068f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 --2- 172.21.15.46:0/2629596764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc58158e60 0x7fdc58068f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc580cc560 tx=0x7fdc480091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- 172.21.15.46:0/2629596764 shutdown_connections 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 --2- 172.21.15.46:0/2629596764 >> v2:172.21.15.132:3300/0 conn(0x7fdc580694c0 0x7fdc580a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 --2- 172.21.15.46:0/2629596764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc58158e60 0x7fdc58068f80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.763+0000 7fdc5d345700 1 -- 172.21.15.46:0/2629596764 >> 172.21.15.46:0/2629596764 conn(0x7fdc58067ae0 msgr2=0x7fdc580cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:32.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.767+0000 7fdc5d345700 1 -- 172.21.15.46:0/2629596764 shutdown_connections 2023-10-06T11:30:32.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.767+0000 7fdc5d345700 1 -- 172.21.15.46:0/2629596764 wait complete. 2023-10-06T11:30:32.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 Processor -- start 2023-10-06T11:30:32.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 -- start start 2023-10-06T11:30:32.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 0x7fdc58153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc58158e60 0x7fdc5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5814e100 con 0x7fdc580694c0 2023-10-06T11:30:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580c3aa0 con 0x7fdc58158e60 2023-10-06T11:30:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc5d345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580abac0 con 0x7fdc580a9d80 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34722/0 (socket says 172.21.15.46:34722) 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 -- 172.21.15.46:0/1131213299 learned_addr learned my addr 172.21.15.46:0/1131213299 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc577fe700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc58158e60 0x7fdc5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc567fc700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 0x7fdc58153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 -- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 msgr2=0x7fdc58153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 0x7fdc58153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 -- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc58158e60 msgr2=0x7fdc5806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:32.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc58158e60 0x7fdc5806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:32.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 -- 172.21.15.46:0/1131213299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc48009420 con 0x7fdc580694c0 2023-10-06T11:30:32.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc567fc700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 0x7fdc58153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc56ffd700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdc580ab4d0 tx=0x7fdc48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.771+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48011b10 con 0x7fdc580694c0 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480253f0 con 0x7fdc580694c0 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc580abda0 con 0x7fdc580694c0 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48025790 con 0x7fdc580694c0 2023-10-06T11:30:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc58154270 con 0x7fdc580694c0 2023-10-06T11:30:32.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc580c38e0 con 0x7fdc580694c0 2023-10-06T11:30:32.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.775+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7fdc48025b40 con 0x7fdc580694c0 2023-10-06T11:30:32.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.779+0000 7fdc3effd700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 0x7fdc44062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:32.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.779+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fdc4807e5d0 con 0x7fdc580694c0 2023-10-06T11:30:32.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.779+0000 7fdc567fc700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 0x7fdc44062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:32.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.779+0000 7fdc567fc700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 0x7fdc44062c00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc5806eab0 tx=0x7fdc4c008040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:32.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.787+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdc580c38e0 con 0x7fdc580694c0 2023-10-06T11:30:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:32.975+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc580c38e0 con 0x7fdc44060750 2023-10-06T11:30:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.119+0000 7fdc3effd700 1 -- 172.21.15.46:0/1131213299 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fdc580c38e0 con 0x7fdc44060750 2023-10-06T11:30:33.127 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 msgr2=0x7fdc44062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 0x7fdc44062c00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc5806eab0 tx=0x7fdc4c008040 comp rx=0 tx=0).stop 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 msgr2=0x7fdc5814dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdc580ab4d0 tx=0x7fdc48017000 comp rx=0 tx=0).stop 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 shutdown_connections 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdc44060750 0x7fdc44062c00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc58158e60 0x7fdc5806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc580a9d80 0x7fdc58153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:33.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 --2- 172.21.15.46:0/1131213299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc580694c0 0x7fdc5814dbc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 >> 172.21.15.46:0/1131213299 conn(0x7fdc58067ae0 msgr2=0x7fdc580aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 shutdown_connections 2023-10-06T11:30:33.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:33.123+0000 7fdc5d345700 1 -- 172.21.15.46:0/1131213299 wait complete. 2023-10-06T11:30:33.155 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-06T11:30:33.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.155 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-06T11:30:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:33.177 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:33.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:33.179 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:33.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.179 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_20_00_UTC) 2023-10-06T11:30:33.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:33.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.259 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:33.339 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:33.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.340 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:33.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:33.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.415 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:33.490 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:33.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.491 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:33.567 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:33.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.567 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_25_01_UTC) 2023-10-06T11:30:33.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:33.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.645 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:33.720 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:33.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.720 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:33.796 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:33.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.797 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:33.871 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:33.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.871 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:33.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:33.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:33.949 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:34.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.023 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-06T11:30:34.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T11:30:34.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 Processor -- start 2023-10-06T11:30:34.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 -- start start 2023-10-06T11:30:34.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 0x7f6e48069470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e480b30a0 0x7f6e480b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480cfc20 con 0x7f6e481591f0 2023-10-06T11:30:34.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480cee20 con 0x7f6e480b30a0 2023-10-06T11:30:34.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e4d62e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480699b0 con 0x7f6e48069020 2023-10-06T11:30:34.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e46ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6e481591f0 0x7f6e480b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46376/0 (socket says 172.21.15.46:46376) 2023-10-06T11:30:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e46ffd700 1 -- 172.21.15.46:0/4093151843 learned_addr learned my addr 172.21.15.46:0/4093151843 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e467fc700 1 --2- 172.21.15.46:0/4093151843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 0x7f6e48069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e477fe700 1 --2- 172.21.15.46:0/4093151843 >> v2:172.21.15.132:3300/0 conn(0x7f6e480b30a0 0x7f6e480b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.299+0000 7f6e45ffb700 1 -- 172.21.15.46:0/4093151843 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028287452 0 0) 0x7f6e480cfc20 con 0x7f6e481591f0 2023-10-06T11:30:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e45ffb700 1 -- 172.21.15.46:0/4093151843 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e30003570 con 0x7f6e481591f0 2023-10-06T11:30:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e467fc700 1 -- 172.21.15.46:0/4093151843 >> v2:172.21.15.132:3300/0 conn(0x7f6e480b30a0 msgr2=0x7f6e480b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e467fc700 1 --2- 172.21.15.46:0/4093151843 >> v2:172.21.15.132:3300/0 conn(0x7f6e480b30a0 0x7f6e480b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e467fc700 1 -- 172.21.15.46:0/4093151843 >> v1:172.21.15.46:6789/0 conn(0x7f6e481591f0 legacy=0x7f6e480b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e467fc700 1 -- 172.21.15.46:0/4093151843 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480cf010 con 0x7f6e48069020 2023-10-06T11:30:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e467fc700 1 --2- 172.21.15.46:0/4093151843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 0x7f6e48069470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0098e0 tx=0x7f6e3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd9c8a0ea9e90925 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e45ffb700 1 -- 172.21.15.46:0/4093151843 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011630 con 0x7f6e48069020 2023-10-06T11:30:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e45ffb700 1 -- 172.21.15.46:0/4093151843 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c011790 con 0x7f6e48069020 2023-10-06T11:30:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e45ffb700 1 -- 172.21.15.46:0/4093151843 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c011b30 con 0x7f6e48069020 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 -- 172.21.15.46:0/4093151843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 msgr2=0x7f6e48069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/4093151843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 0x7f6e48069470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0098e0 tx=0x7f6e3c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 -- 172.21.15.46:0/4093151843 shutdown_connections 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/4093151843 >> v2:172.21.15.132:3300/0 conn(0x7f6e480b30a0 0x7f6e480b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/4093151843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e48069020 0x7f6e48069470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 -- 172.21.15.46:0/4093151843 >> 172.21.15.46:0/4093151843 conn(0x7f6e480d0b20 msgr2=0x7f6e480b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 -- 172.21.15.46:0/4093151843 shutdown_connections 2023-10-06T11:30:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.303+0000 7f6e4d62e700 1 -- 172.21.15.46:0/4093151843 wait complete. 2023-10-06T11:30:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 Processor -- start 2023-10-06T11:30:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 -- start start 2023-10-06T11:30:34.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 0x7f6e480c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e481591f0 0x7f6e480c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815f3a0 con 0x7f6e48069020 2023-10-06T11:30:34.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815d020 con 0x7f6e480b30a0 2023-10-06T11:30:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e4d62e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815c130 con 0x7f6e481591f0 2023-10-06T11:30:34.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53148/0 (socket says 172.21.15.46:53148) 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 -- 172.21.15.46:0/120141401 learned_addr learned my addr 172.21.15.46:0/120141401 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e477fe700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e481591f0 0x7f6e480c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e467fc700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 0x7f6e480c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 -- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e481591f0 msgr2=0x7f6e480c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e481591f0 0x7f6e480c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 -- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 msgr2=0x7f6e480c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 0x7f6e480c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e46ffd700 1 -- 172.21.15.46:0/120141401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c009420 con 0x7f6e48069020 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.307+0000 7f6e467fc700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 0x7f6e480c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e46ffd700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c790 tx=0x7f6e38007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38005b30 con 0x7f6e48069020 2023-10-06T11:30:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e38005c90 con 0x7f6e48069020 2023-10-06T11:30:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38007320 con 0x7f6e48069020 2023-10-06T11:30:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4815c410 con 0x7f6e48069020 2023-10-06T11:30:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4815c9d0 con 0x7f6e48069020 2023-10-06T11:30:34.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48064590 con 0x7f6e48069020 2023-10-06T11:30:34.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.311+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f6e38007690 con 0x7f6e48069020 2023-10-06T11:30:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.315+0000 7f6e2effd700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 0x7f6e34062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.315+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f6e3807c330 con 0x7f6e48069020 2023-10-06T11:30:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.315+0000 7f6e467fc700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 0x7f6e34062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.315+0000 7f6e467fc700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 0x7f6e34062c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0091e0 tx=0x7f6e3c017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.323+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6e3804c380 con 0x7f6e48069020 2023-10-06T11:30:34.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.523+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4815d160 con 0x7f6e34060750 2023-10-06T11:30:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.663+0000 7f6e2effd700 1 -- 172.21.15.46:0/120141401 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6e3c011900 con 0x7f6e34060750 2023-10-06T11:30:34.672 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-06T11:30:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 msgr2=0x7f6e34062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 0x7f6e34062c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0091e0 tx=0x7f6e3c017000 comp rx=0 tx=0).stop 2023-10-06T11:30:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 msgr2=0x7f6e4815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800c790 tx=0x7f6e38007df0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 shutdown_connections 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e34060750 0x7f6e34062c00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e481591f0 0x7f6e480c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e480b30a0 0x7f6e480c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 --2- 172.21.15.46:0/120141401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e48069020 0x7f6e4815d4b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 >> 172.21.15.46:0/120141401 conn(0x7f6e480d0b20 msgr2=0x7f6e480b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 shutdown_connections 2023-10-06T11:30:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:34.667+0000 7f6e4d62e700 1 -- 172.21.15.46:0/120141401 wait complete. 2023-10-06T11:30:34.699 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-06T11:30:34.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.700 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-06T11:30:34.723 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:34.724 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:34.724 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:34.724 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:34.724 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:34.724 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:34.725 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:34.727 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:34.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.727 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_20_01_UTC) 2023-10-06T11:30:34.813 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:34.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.813 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:34.887 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:34.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.888 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:34.964 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:34.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:34.964 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:35.040 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:35.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.041 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:35.116 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:35.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.117 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:35.191 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:35.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.191 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_26_01_UTC) 2023-10-06T11:30:35.269 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:35.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.269 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:35.342 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:35.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.343 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:35.421 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:35.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.421 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:35.494 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:35.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.495 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:35.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:35.572 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-06T11:30:35.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T11:30:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.847+0000 7f189759e700 1 Processor -- start 2023-10-06T11:30:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.847+0000 7f189759e700 1 -- start start 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18981537f0 0x7f189806ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18980c6150 con 0x7f1898153bc0 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18980c6430 con 0x7f1898067050 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1898156c00 con 0x7f18981537f0 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18981537f0 0x7f189806ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18981537f0 0x7f189806ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53154/0 (socket says 172.21.15.46:53154) 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 -- 172.21.15.46:0/3919394154 learned_addr learned my addr 172.21.15.46:0/3919394154 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f1895d9b700 1 --1- 172.21.15.46:0/3919394154 >> v1:172.21.15.46:6789/0 conn(0x7f1898153bc0 0x7f1898156360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46386/0 (socket says 172.21.15.46:46386) 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 -- 172.21.15.46:0/3919394154 >> v1:172.21.15.132:6789/0 conn(0x7f1898067050 legacy=0x7f18980c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 -- 172.21.15.46:0/3919394154 >> v1:172.21.15.46:6789/0 conn(0x7f1898153bc0 legacy=0x7f1898156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189659c700 1 -- 172.21.15.46:0/3919394154 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1898156ee0 con 0x7f18981537f0 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.851+0000 7f189559a700 1 -- 172.21.15.46:0/3919394154 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143831365 0 0) 0x7f18980c6430 con 0x7f1898067050 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189659c700 1 --2- 172.21.15.46:0/3919394154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f189806ceb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f188800bc50 tx=0x7f188800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c4d512b1f0c0f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189559a700 1 -- 172.21.15.46:0/3919394154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800be60 con 0x7f18981537f0 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189559a700 1 -- 172.21.15.46:0/3919394154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188800de00 con 0x7f18981537f0 2023-10-06T11:30:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189559a700 1 -- 172.21.15.46:0/3919394154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880196b0 con 0x7f18981537f0 2023-10-06T11:30:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 -- 172.21.15.46:0/3919394154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 msgr2=0x7f189806ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 --2- 172.21.15.46:0/3919394154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f189806ceb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f188800bc50 tx=0x7f188800b4f0 comp rx=0 tx=0).stop 2023-10-06T11:30:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 -- 172.21.15.46:0/3919394154 shutdown_connections 2023-10-06T11:30:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 --2- 172.21.15.46:0/3919394154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f189806ceb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 -- 172.21.15.46:0/3919394154 >> 172.21.15.46:0/3919394154 conn(0x7f18980cbbb0 msgr2=0x7f18980cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 -- 172.21.15.46:0/3919394154 shutdown_connections 2023-10-06T11:30:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.855+0000 7f189759e700 1 -- 172.21.15.46:0/3919394154 wait complete. 2023-10-06T11:30:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 Processor -- start 2023-10-06T11:30:35.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 -- start start 2023-10-06T11:30:35.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1898067050 0x7f18980c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1898169530 con 0x7f18981537f0 2023-10-06T11:30:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980a9d90 con 0x7f1898153bc0 2023-10-06T11:30:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1898157660 con 0x7f1898067050 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1896d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1896d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:50628/0 (socket says 172.21.15.46:50628) 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53164/0 (socket says 172.21.15.46:53164) 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1896d9d700 1 -- 172.21.15.46:0/3168044793 learned_addr learned my addr 172.21.15.46:0/3168044793 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f189659c700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1898067050 0x7f18980c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 -- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1898067050 msgr2=0x7f18980c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1898067050 0x7f18980c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 -- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 msgr2=0x7f18980a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 -- 172.21.15.46:0/3168044793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188800b790 con 0x7f18981537f0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1896d9d700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.859+0000 7f1895d9b700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c5c0 tx=0x7f188c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c005a70 con 0x7f18981537f0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c005bf0 con 0x7f18981537f0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0070d0 con 0x7f18981537f0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980a9fc0 con 0x7f18981537f0 2023-10-06T11:30:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980aa580 con 0x7f18981537f0 2023-10-06T11:30:35.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898064590 con 0x7f18981537f0 2023-10-06T11:30:35.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.863+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f188c007480 con 0x7f18981537f0 2023-10-06T11:30:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.867+0000 7f18867fc700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 0x7f1878062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.867+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f188c07c190 con 0x7f18981537f0 2023-10-06T11:30:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.867+0000 7f189659c700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 0x7f1878062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.867+0000 7f189659c700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 0x7f1878062c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f188800bc50 tx=0x7f188801b000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:35.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:35.875+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f188c04c1e0 con 0x7f18981537f0 2023-10-06T11:30:36.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.071+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f18980c7f50 con 0x7f1878060750 2023-10-06T11:30:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.207+0000 7f18867fc700 1 -- 172.21.15.46:0/3168044793 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f18980c7f50 con 0x7f1878060750 2023-10-06T11:30:36.216 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-06T11:30:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 msgr2=0x7f1878062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 0x7f1878062c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f188800bc50 tx=0x7f188801b000 comp rx=0 tx=0).stop 2023-10-06T11:30:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 msgr2=0x7f18981590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c5c0 tx=0x7f188c007eb0 comp rx=0 tx=0).stop 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 shutdown_connections 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1878060750 0x7f1878062c00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1898153bc0 0x7f18980a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981537f0 0x7f18981590c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 --2- 172.21.15.46:0/3168044793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1898067050 0x7f18980c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 >> 172.21.15.46:0/3168044793 conn(0x7f18980cbbb0 msgr2=0x7f18980c8380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 shutdown_connections 2023-10-06T11:30:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:36.211+0000 7f189759e700 1 -- 172.21.15.46:0/3168044793 wait complete. 2023-10-06T11:30:36.243 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-06T11:30:36.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.243 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_01_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:36.266 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:36.267 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:36.267 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:36.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.268 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_20_00_UTC) 2023-10-06T11:30:36.371 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:36.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.371 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:36.409 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:36.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.410 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:36.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:36.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.481 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:36.549 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_24_01_UTC 2023-10-06T11:30:36.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.550 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_24_01_UTC) 2023-10-06T11:30:36.619 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:36.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.619 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:36.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:36.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.685 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:36.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:36.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.761 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:36.830 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:36.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.883 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:36.912 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:36.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:36.912 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_29_01_UTC) 2023-10-06T11:30:36.989 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:37.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.033 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:37.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.065 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-06T11:30:37.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T11:30:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 Processor -- start 2023-10-06T11:30:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 -- start start 2023-10-06T11:30:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 0x7f3b7c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3b7c0b29d0 0x7f3b7c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0cbff0 con 0x7f3b7c0cae50 2023-10-06T11:30:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c1598f0 con 0x7f3b7c0b29d0 2023-10-06T11:30:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.331+0000 7f3b82865700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0cb3e0 con 0x7f3b7c0b2600 2023-10-06T11:30:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3b7c0cae50 0x7f3b7c15cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46388/0 (socket says 172.21.15.46:46388) 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 -- 172.21.15.46:0/981263638 learned_addr learned my addr 172.21.15.46:0/981263638 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7bfff700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 0x7f3b7c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7b7fe700 1 --2- 172.21.15.46:0/981263638 >> v2:172.21.15.132:3300/0 conn(0x7f3b7c0b29d0 0x7f3b7c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783422757 0 0) 0x7f3b7c0cbff0 con 0x7f3b7c0cae50 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b64003570 con 0x7f3b7c0cae50 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3698255311 0 0) 0x7f3b64003570 con 0x7f3b7c0cae50 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 msgr2=0x7f3b7c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 0x7f3b7c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 >> v2:172.21.15.132:3300/0 conn(0x7f3b7c0b29d0 msgr2=0x7f3b7c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 --2- 172.21.15.46:0/981263638 >> v2:172.21.15.132:3300/0 conn(0x7f3b7c0b29d0 0x7f3b7c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0cb6c0 con 0x7f3b7c0cae50 2023-10-06T11:30:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3b740017f0 con 0x7f3b7c0cae50 2023-10-06T11:30:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 >> v1:172.21.15.46:6789/0 conn(0x7f3b7c0cae50 legacy=0x7f3b7c15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b64003a60 0x7f3b64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 0x7f3b64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1598f0 con 0x7f3b64004d50 2023-10-06T11:30:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0cb3e0 con 0x7f3b64003a60 2023-10-06T11:30:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6400bae0 con 0x7f3b7c0b2600 2023-10-06T11:30:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7b7fe700 1 --2- 172.21.15.46:0/981263638 >> v2:172.21.15.132:3300/0 conn(0x7f3b7c0b29d0 0x7f3b7c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 -- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 msgr2=0x7f3b64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b2600 0x7f3b64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 -- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b64003a60 msgr2=0x7f3b64003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b64003a60 0x7f3b64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 -- 172.21.15.46:0/981263638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64003570 con 0x7f3b64004d50 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b80e02700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b640051c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400a970 tx=0x7f3b74002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7454f7751efcc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b740070d0 con 0x7f3b64004d50 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b74007270 con 0x7f3b64004d50 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.335+0000 7f3b7affd700 1 -- 172.21.15.46:0/981263638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74007630 con 0x7f3b64004d50 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 -- 172.21.15.46:0/981263638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 msgr2=0x7f3b640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 --2- 172.21.15.46:0/981263638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b640051c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400a970 tx=0x7f3b74002210 comp rx=0 tx=0).stop 2023-10-06T11:30:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b80e02700 1 -- 172.21.15.46:0/981263638 reap_dead start 2023-10-06T11:30:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 -- 172.21.15.46:0/981263638 shutdown_connections 2023-10-06T11:30:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 -- 172.21.15.46:0/981263638 >> 172.21.15.46:0/981263638 conn(0x7f3b7c06ef60 msgr2=0x7f3b7c0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 -- 172.21.15.46:0/981263638 shutdown_connections 2023-10-06T11:30:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 -- 172.21.15.46:0/981263638 wait complete. 2023-10-06T11:30:37.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.339+0000 7f3b82865700 1 Processor -- start 2023-10-06T11:30:37.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 -- start start 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b1920 0x7f3b7c09c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b7c09c960 0x7f3b7c147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c156f50 con 0x7f3b64004d50 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c09b380 con 0x7f3b7c09c960 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c09b660 con 0x7f3b7c0b1920 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53178/0 (socket says 172.21.15.46:53178) 2023-10-06T11:30:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 -- 172.21.15.46:0/3326049635 learned_addr learned my addr 172.21.15.46:0/3326049635 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7b7fe700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b1920 0x7f3b7c09c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b80e02700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b7c09c960 0x7f3b7c147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 -- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b1920 msgr2=0x7f3b7c09c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b1920 0x7f3b7c09c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 -- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b7c09c960 msgr2=0x7f3b7c147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b7c09c960 0x7f3b7c147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 -- 172.21.15.46:0/3326049635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c09b940 con 0x7f3b64004d50 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b7bfff700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00ea50 tx=0x7f3b6c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00eee0 con 0x7f3b64004d50 2023-10-06T11:30:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c00c7d0 con 0x7f3b64004d50 2023-10-06T11:30:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00cb90 con 0x7f3b64004d50 2023-10-06T11:30:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.343+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c09bc20 con 0x7f3b64004d50 2023-10-06T11:30:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.347+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c147fd0 con 0x7f3b64004d50 2023-10-06T11:30:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.347+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c09cd30 con 0x7f3b64004d50 2023-10-06T11:30:37.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.347+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f3b6c013470 con 0x7f3b64004d50 2023-10-06T11:30:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.351+0000 7f3b63fff700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 0x7f3b68062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.351+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f3b6c07d510 con 0x7f3b64004d50 2023-10-06T11:30:37.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.351+0000 7f3b7b7fe700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 0x7f3b68062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.351+0000 7f3b7b7fe700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 0x7f3b68062c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c068f20 tx=0x7f3b70006ec0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.355+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3b7c09cd30 con 0x7f3b64004d50 2023-10-06T11:30:37.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.559+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b7c09cd30 con 0x7f3b68060780 2023-10-06T11:30:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.703+0000 7f3b63fff700 1 -- 172.21.15.46:0/3326049635 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3b7c09cd30 con 0x7f3b68060780 2023-10-06T11:30:37.714 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-06T11:30:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.707+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 msgr2=0x7f3b68062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 0x7f3b68062c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c068f20 tx=0x7f3b70006ec0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 msgr2=0x7f3b7c0b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00ea50 tx=0x7f3b6c009120 comp rx=0 tx=0).stop 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 shutdown_connections 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3b68060780 0x7f3b68062c30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b7c09c960 0x7f3b7c147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b7c0b1920 0x7f3b7c09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 --2- 172.21.15.46:0/3326049635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b64004d50 0x7f3b7c0b13e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 >> 172.21.15.46:0/3326049635 conn(0x7f3b7c06ef60 msgr2=0x7f3b7c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 shutdown_connections 2023-10-06T11:30:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:37.711+0000 7f3b82865700 1 -- 172.21.15.46:0/3326049635 wait complete. 2023-10-06T11:30:37.742 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-06T11:30:37.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.742 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-06T11:30:37.767 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:37.768 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:37.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:37.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.770 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_20_01_UTC) 2023-10-06T11:30:37.850 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:37.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.850 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:37.931 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:37.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:37.931 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:38.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:38.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.008 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:38.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:38.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.088 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:38.164 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_25_01_UTC 2023-10-06T11:30:38.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.164 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_25_01_UTC) 2023-10-06T11:30:38.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:38.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.241 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:38.337 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:38.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.338 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:38.386 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:38.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.386 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:38.466 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:38.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.466 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:38.549 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:38.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.549 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:38.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:38.617 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-06T11:30:38.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T11:30:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.891+0000 7f9143ef0700 1 Processor -- start 2023-10-06T11:30:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.891+0000 7f9143ef0700 1 -- start start 2023-10-06T11:30:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9143ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9143ef0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c06dde0 con 0x7f913c155960 2023-10-06T11:30:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9143ef0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c06d1d0 con 0x7f913c0c1b60 2023-10-06T11:30:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9143ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0c7150 con 0x7f913c155590 2023-10-06T11:30:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f914148b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f913c155960 0x7f913c0c1450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46398/0 (socket says 172.21.15.46:46398) 2023-10-06T11:30:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f914148b700 1 -- 172.21.15.46:0/554407654 learned_addr learned my addr 172.21.15.46:0/554407654 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9141c8c700 1 --2- 172.21.15.46:0/554407654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.895+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112990170 0 0) 0x7f913c06dde0 con 0x7f913c155960 2023-10-06T11:30:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9128004680 con 0x7f913c155960 2023-10-06T11:30:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299956303 0 0) 0x7f913c06d1d0 con 0x7f913c0c1b60 2023-10-06T11:30:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f913c06dde0 con 0x7f913c0c1b60 2023-10-06T11:30:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9141c8c700 1 -- 172.21.15.46:0/554407654 >> v1:172.21.15.132:6789/0 conn(0x7f913c0c1b60 legacy=0x7f913c0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9141c8c700 1 -- 172.21.15.46:0/554407654 >> v1:172.21.15.46:6789/0 conn(0x7f913c155960 legacy=0x7f913c0c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9141c8c700 1 -- 172.21.15.46:0/554407654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c06d4b0 con 0x7f913c155590 2023-10-06T11:30:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9141c8c700 1 --2- 172.21.15.46:0/554407654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c06e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9138009910 tx=0x7f91380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb9b8066edab341d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138011630 con 0x7f913c155590 2023-10-06T11:30:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138011790 con 0x7f913c155590 2023-10-06T11:30:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9140c8a700 1 -- 172.21.15.46:0/554407654 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138011b30 con 0x7f913c155590 2023-10-06T11:30:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 -- 172.21.15.46:0/554407654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 msgr2=0x7f913c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 --2- 172.21.15.46:0/554407654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c06e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9138009910 tx=0x7f91380091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 -- 172.21.15.46:0/554407654 shutdown_connections 2023-10-06T11:30:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 --2- 172.21.15.46:0/554407654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c06e770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 -- 172.21.15.46:0/554407654 >> 172.21.15.46:0/554407654 conn(0x7f913c0cbe20 msgr2=0x7f913c0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 -- 172.21.15.46:0/554407654 shutdown_connections 2023-10-06T11:30:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.899+0000 7f9143ef0700 1 -- 172.21.15.46:0/554407654 wait complete. 2023-10-06T11:30:38.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.903+0000 7f9143ef0700 1 Processor -- start 2023-10-06T11:30:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.903+0000 7f9143ef0700 1 -- start start 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c0c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 0x7f913c0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c158a00 con 0x7f913c0c1b60 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0c48d0 con 0x7f913c159950 2023-10-06T11:30:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0c45a0 con 0x7f913c155590 2023-10-06T11:30:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53188/0 (socket says 172.21.15.46:53188) 2023-10-06T11:30:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 -- 172.21.15.46:0/2588132944 learned_addr learned my addr 172.21.15.46:0/2588132944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f914248d700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 0x7f913c0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 -- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 msgr2=0x7f913c0c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f914148b700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c0c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c0c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:38.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 -- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 msgr2=0x7f913c0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 0x7f913c0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 -- 172.21.15.46:0/2588132944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009420 con 0x7f913c0c1b60 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f914148b700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c0c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f914248d700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 0x7f913c0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9141c8c700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f91380091e0 tx=0x7f9138017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913801f070 con 0x7f913c0c1b60 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c0c4070 con 0x7f913c0c1b60 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138000bc0 con 0x7f913c0c1b60 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.907+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138025420 con 0x7f913c0c1b60 2023-10-06T11:30:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.911+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c0c3d90 con 0x7f913c0c1b60 2023-10-06T11:30:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.911+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c06b890 con 0x7f913c0c1b60 2023-10-06T11:30:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.911+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f9138023070 con 0x7f913c0c1b60 2023-10-06T11:30:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.915+0000 7f9131ffb700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 0x7f9120062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.915+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7057+0+0 (secure 0 0 0) 0x7f913807e4d0 con 0x7f913c0c1b60 2023-10-06T11:30:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.915+0000 7f914148b700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 0x7f9120062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.915+0000 7f914148b700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 0x7f9120062c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:38.923+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f913804e250 con 0x7f913c0c1b60 2023-10-06T11:30:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.115+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f913c0c4a10 con 0x7f9120060750 2023-10-06T11:30:39.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.251+0000 7f9131ffb700 1 -- 172.21.15.46:0/2588132944 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f912c001470 con 0x7f9120060750 2023-10-06T11:30:39.261 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 msgr2=0x7f9120062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 0x7f9120062c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).stop 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 msgr2=0x7f913c159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f91380091e0 tx=0x7f9138017000 comp rx=0 tx=0).stop 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 shutdown_connections 2023-10-06T11:30:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9120060750 0x7f9120062c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f913c159950 0x7f913c0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f913c155590 0x7f913c0c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 --2- 172.21.15.46:0/2588132944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f913c0c1b60 0x7f913c159410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 >> 172.21.15.46:0/2588132944 conn(0x7f913c0cbe20 msgr2=0x7f913c150a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 shutdown_connections 2023-10-06T11:30:39.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:39.255+0000 7f9143ef0700 1 -- 172.21.15.46:0/2588132944 wait complete. 2023-10-06T11:30:39.294 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-06T11:30:39.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.294 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-06T11:30:39.309 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_01_UTC 2023-10-06T11:30:39.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:39.311 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:39.311 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:39.312 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:39.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.312 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_20_00_UTC) 2023-10-06T11:30:39.402 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:39.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.402 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_21_01_UTC) 2023-10-06T11:30:39.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:39.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.483 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:39.561 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:39.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.561 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:39.635 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:39.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.635 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:39.711 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:39.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.712 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:39.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:39.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.781 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:39.854 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_27_01_UTC 2023-10-06T11:30:39.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.855 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_27_01_UTC) 2023-10-06T11:30:39.933 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:39.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:39.933 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:40.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_29_01_UTC 2023-10-06T11:30:40.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.011 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_29_01_UTC) 2023-10-06T11:30:40.088 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:40.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.089 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:40.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.166 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-06T11:30:40.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T11:30:40.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.455+0000 7f11c484c700 1 Processor -- start 2023-10-06T11:30:40.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.455+0000 7f11c484c700 1 -- start start 2023-10-06T11:30:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c0158de0 0x7f11c00b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f11c01591b0 0x7f11c0069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11c00695a0 0x7f11c015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c0069a20 con 0x7f11c00695a0 2023-10-06T11:30:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c00b5eb0 con 0x7f11c01591b0 2023-10-06T11:30:40.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11c484c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00cef30 con 0x7f11c0158de0 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11c00695a0 0x7f11c015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11beffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f11c01591b0 0x7f11c0069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bf7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c0158de0 0x7f11c00b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11c00695a0 0x7f11c015cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53198/0 (socket says 172.21.15.46:53198) 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 -- 172.21.15.46:0/2701168195 learned_addr learned my addr 172.21.15.46:0/2701168195 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:40.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 -- 172.21.15.46:0/2701168195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c0158de0 msgr2=0x7f11c00b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:40.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 --2- 172.21.15.46:0/2701168195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c0158de0 0x7f11c00b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 -- 172.21.15.46:0/2701168195 >> v2:172.21.15.132:3300/0 conn(0x7f11c01591b0 msgr2=0x7f11c0069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:40.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 --2- 172.21.15.46:0/2701168195 >> v2:172.21.15.132:3300/0 conn(0x7f11c01591b0 0x7f11c0069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.459+0000 7f11bffff700 1 -- 172.21.15.46:0/2701168195 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00cf4f0 con 0x7f11c00695a0 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11bffff700 1 --2- 172.21.15.46:0/2701168195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c00695a0 0x7f11c015cc50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11c00687e0 tx=0x7f11b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36d7ee8659a6df2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11be7fc700 1 -- 172.21.15.46:0/2701168195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800e070 con 0x7f11c00695a0 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11be7fc700 1 -- 172.21.15.46:0/2701168195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b800b970 con 0x7f11c00695a0 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11be7fc700 1 -- 172.21.15.46:0/2701168195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80092e0 con 0x7f11c00695a0 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 -- 172.21.15.46:0/2701168195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c00695a0 msgr2=0x7f11c015cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:40.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 --2- 172.21.15.46:0/2701168195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c00695a0 0x7f11c015cc50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f11c00687e0 tx=0x7f11b8009840 comp rx=0 tx=0).stop 2023-10-06T11:30:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 -- 172.21.15.46:0/2701168195 shutdown_connections 2023-10-06T11:30:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 --2- 172.21.15.46:0/2701168195 >> v2:172.21.15.132:3300/0 conn(0x7f11c01591b0 0x7f11c0069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 --2- 172.21.15.46:0/2701168195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c00695a0 0x7f11c015cc50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 --2- 172.21.15.46:0/2701168195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c0158de0 0x7f11c00b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 -- 172.21.15.46:0/2701168195 >> 172.21.15.46:0/2701168195 conn(0x7f11c0068390 msgr2=0x7f11c00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 -- 172.21.15.46:0/2701168195 shutdown_connections 2023-10-06T11:30:40.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.463+0000 7f11c484c700 1 -- 172.21.15.46:0/2701168195 wait complete. 2023-10-06T11:30:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 Processor -- start 2023-10-06T11:30:40.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 -- start start 2023-10-06T11:30:40.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c00695a0 0x7f11c0155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 0x7f11c0159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00c5020 con 0x7f11c01591b0 2023-10-06T11:30:40.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c015a960 con 0x7f11c0158de0 2023-10-06T11:30:40.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.467+0000 7f11c484c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c015a0c0 con 0x7f11c00695a0 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 0x7f11c0159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53202/0 (socket says 172.21.15.46:53202) 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11bffff700 1 -- 172.21.15.46:0/4191612158 learned_addr learned my addr 172.21.15.46:0/4191612158 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11bf7fe700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c00695a0 0x7f11c0155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 -- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c00695a0 msgr2=0x7f11c0155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:40.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c00695a0 0x7f11c0155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 -- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 msgr2=0x7f11c00b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:40.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 -- 172.21.15.46:0/4191612158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8009ae0 con 0x7f11c0158de0 2023-10-06T11:30:40.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11bffff700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:40.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11beffd700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 0x7f11c0159670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e900 tx=0x7f11b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4013680 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b40137e0 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4013b80 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c015a3a0 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.471+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c015b4d0 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.475+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c00c4df0 con 0x7f11c0158de0 2023-10-06T11:30:40.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.475+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f11b40123d0 con 0x7f11c0158de0 2023-10-06T11:30:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.475+0000 7f11a77fe700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 0x7f11ac062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.475+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6945+0+0 (secure 0 0 0) 0x7f11b407d040 con 0x7f11c0158de0 2023-10-06T11:30:40.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.475+0000 7f11bf7fe700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 0x7f11ac062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:40.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.479+0000 7f11bf7fe700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 0x7f11ac062c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009db0 tx=0x7f11b00091a0 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.487+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11b40acbe0 con 0x7f11c0158de0 2023-10-06T11:30:40.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.699+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c00c5190 con 0x7f11ac060780 2023-10-06T11:30:40.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.839+0000 7f11a77fe700 1 -- 172.21.15.46:0/4191612158 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f11c00c5190 con 0x7f11ac060780 2023-10-06T11:30:40.847 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-06T11:30:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 msgr2=0x7f11ac062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 0x7f11ac062c30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009db0 tx=0x7f11b00091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 msgr2=0x7f11c0159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 0x7f11c0159670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e900 tx=0x7f11b40091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 shutdown_connections 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11ac060780 0x7f11ac062c30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11c01591b0 0x7f11c00b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11c0158de0 0x7f11c0159670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 --2- 172.21.15.46:0/4191612158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11c00695a0 0x7f11c0155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 >> 172.21.15.46:0/4191612158 conn(0x7f11c0068390 msgr2=0x7f11c0156430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 shutdown_connections 2023-10-06T11:30:40.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:40.843+0000 7f11c484c700 1 -- 172.21.15.46:0/4191612158 wait complete. 2023-10-06T11:30:40.879 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-06T11:30:40.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.879 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-06T11:30:40.895 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:40.895 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:40.895 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:40.895 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:40.896 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:40.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.897 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_20_01_UTC) 2023-10-06T11:30:40.969 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:40.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:40.970 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:41.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:41.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.050 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:41.128 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:41.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.128 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:41.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:41.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.204 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:41.283 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:41.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.284 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:41.361 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:41.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.361 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:41.425 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:41.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.425 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:41.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:41.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.511 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:41.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:41.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.575 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:41.651 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:41.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.652 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:41.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:41.727 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-06T11:30:41.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T11:30:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.007+0000 7f18f2a2e700 1 Processor -- start 2023-10-06T11:30:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.007+0000 7f18f2a2e700 1 -- start start 2023-10-06T11:30:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18f2a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18ec1596a0 0x7f18ec0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18f2a2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c5db0 con 0x7f18ec1592d0 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18f2a2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c5ab0 con 0x7f18ec0c6780 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18f2a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0d0aa0 con 0x7f18ec1596a0 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18ebfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f18ec1592d0 0x7f18ec0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46414/0 (socket says 172.21.15.46:46414) 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18ebfff700 1 -- 172.21.15.46:0/2336290311 learned_addr learned my addr 172.21.15.46:0/2336290311 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eb7fe700 1 --2- 172.21.15.46:0/2336290311 >> v2:172.21.15.46:3300/0 conn(0x7f18ec1596a0 0x7f18ec0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656348941 0 0) 0x7f18ec0c5db0 con 0x7f18ec1592d0 2023-10-06T11:30:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d4003570 con 0x7f18ec1592d0 2023-10-06T11:30:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666722448 0 0) 0x7f18ec0c5ab0 con 0x7f18ec0c6780 2023-10-06T11:30:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ec0c5db0 con 0x7f18ec0c6780 2023-10-06T11:30:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eb7fe700 1 -- 172.21.15.46:0/2336290311 >> v1:172.21.15.132:6789/0 conn(0x7f18ec0c6780 legacy=0x7f18ec0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eb7fe700 1 -- 172.21.15.46:0/2336290311 >> v1:172.21.15.46:6789/0 conn(0x7f18ec1592d0 legacy=0x7f18ec0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eb7fe700 1 -- 172.21.15.46:0/2336290311 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0d0d80 con 0x7f18ec1596a0 2023-10-06T11:30:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eb7fe700 1 --2- 172.21.15.46:0/2336290311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c6240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f18e00098d0 tx=0x7f18e0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d3f47c11637934 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.011+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0005c10 con 0x7f18ec1596a0 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e0005d70 con 0x7f18ec1596a0 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18eaffd700 1 -- 172.21.15.46:0/2336290311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0006280 con 0x7f18ec1596a0 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 -- 172.21.15.46:0/2336290311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 msgr2=0x7f18ec0c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/2336290311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c6240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f18e00098d0 tx=0x7f18e0009120 comp rx=0 tx=0).stop 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 -- 172.21.15.46:0/2336290311 shutdown_connections 2023-10-06T11:30:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/2336290311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c6240 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 -- 172.21.15.46:0/2336290311 >> 172.21.15.46:0/2336290311 conn(0x7f18ec06ef60 msgr2=0x7f18ec0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 -- 172.21.15.46:0/2336290311 shutdown_connections 2023-10-06T11:30:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.015+0000 7f18f2a2e700 1 -- 172.21.15.46:0/2336290311 wait complete. 2023-10-06T11:30:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 Processor -- start 2023-10-06T11:30:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 -- start start 2023-10-06T11:30:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 0x7f18ec0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ec1592d0 0x7f18ec1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec16a240 con 0x7f18ec1596a0 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec169910 con 0x7f18ec1592d0 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f2a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec169bf0 con 0x7f18ec0c6780 2023-10-06T11:30:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f0fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 0x7f18ec0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f0fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53224/0 (socket says 172.21.15.46:53224) 2023-10-06T11:30:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18ebfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 0x7f18ec0c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:51484/0 (socket says 172.21.15.46:51484) 2023-10-06T11:30:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18f0fcb700 1 -- 172.21.15.46:0/1402776512 learned_addr learned my addr 172.21.15.46:0/1402776512 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.019+0000 7f18eb7fe700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ec1592d0 0x7f18ec1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 -- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 msgr2=0x7f18ec0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 0x7f18ec0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 -- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ec1592d0 msgr2=0x7f18ec1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ec1592d0 0x7f18ec1554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 -- 172.21.15.46:0/1402776512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e00093c0 con 0x7f18ec1596a0 2023-10-06T11:30:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f0fcb700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c860 tx=0x7f18e4002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4002030 con 0x7f18ec1596a0 2023-10-06T11:30:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e40021b0 con 0x7f18ec1596a0 2023-10-06T11:30:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec169ed0 con 0x7f18ec1596a0 2023-10-06T11:30:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4012630 con 0x7f18ec1596a0 2023-10-06T11:30:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec0c9590 con 0x7f18ec1596a0 2023-10-06T11:30:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.023+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec0c6460 con 0x7f18ec1596a0 2023-10-06T11:30:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.027+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f18e4012a20 con 0x7f18ec1596a0 2023-10-06T11:30:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.027+0000 7f18d3fff700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 0x7f18d8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.027+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 7025+0+0 (secure 0 0 0) 0x7f18e407cff0 con 0x7f18ec1596a0 2023-10-06T11:30:42.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.027+0000 7f18ebfff700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 0x7f18d8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:42.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.031+0000 7f18ebfff700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 0x7f18d8062c00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c8f50 tx=0x7f18dc009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:42.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.035+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f18ec0c6460 con 0x7f18ec1596a0 2023-10-06T11:30:42.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.231+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f18ec169a80 con 0x7f18d8060750 2023-10-06T11:30:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.511+0000 7f18d3fff700 1 -- 172.21.15.46:0/1402776512 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f18ec169a80 con 0x7f18d8060750 2023-10-06T11:30:42.519 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-06T11:30:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 msgr2=0x7f18d8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 0x7f18d8062c00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0c8f50 tx=0x7f18dc009040 comp rx=0 tx=0).stop 2023-10-06T11:30:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 msgr2=0x7f18ec0c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c860 tx=0x7f18e4002700 comp rx=0 tx=0).stop 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 shutdown_connections 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18d8060750 0x7f18d8062c00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18ec1596a0 0x7f18ec0c7310 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18ec1592d0 0x7f18ec1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 --2- 172.21.15.46:0/1402776512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18ec0c6780 0x7f18ec0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 >> 172.21.15.46:0/1402776512 conn(0x7f18ec06ef60 msgr2=0x7f18ec0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 shutdown_connections 2023-10-06T11:30:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:42.515+0000 7f18f2a2e700 1 -- 172.21.15.46:0/1402776512 wait complete. 2023-10-06T11:30:42.544 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-06T11:30:42.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.544 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-06T11:30:42.575 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_01_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:42.576 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:42.577 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:42.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.578 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_20_00_UTC) 2023-10-06T11:30:42.658 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_21_01_UTC 2023-10-06T11:30:42.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.659 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_21_01_UTC) 2023-10-06T11:30:42.740 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:42.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.741 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:42.818 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:42.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.818 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:42.900 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:42.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.901 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:42.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:42.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:42.974 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:43.048 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_26_01_UTC 2023-10-06T11:30:43.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.048 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_26_01_UTC) 2023-10-06T11:30:43.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:43.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.125 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:43.202 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_28_01_UTC 2023-10-06T11:30:43.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.202 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_28_01_UTC) 2023-10-06T11:30:43.280 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:43.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.280 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:43.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:43.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.361 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-11_30_01_UTC) 2023-10-06T11:30:43.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:43.435 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-06T11:30:43.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T11:30:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.719+0000 7f309f24c700 1 Processor -- start 2023-10-06T11:30:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.719+0000 7f309f24c700 1 -- start start 2023-10-06T11:30:43.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.723+0000 7f309f24c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30981595c0 0x7f30980af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:43.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.723+0000 7f309f24c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30980cf360 con 0x7f30981591f0 2023-10-06T11:30:43.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.723+0000 7f309f24c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309815dd60 con 0x7f30980cc500 2023-10-06T11:30:43.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.723+0000 7f309f24c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30980cff00 con 0x7f30981595c0 2023-10-06T11:30:43.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.723+0000 7f309da49700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30981595c0 0x7f30980af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:43.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309e24a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f30981591f0 0x7f30980cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53900/0 (socket says 172.21.15.46:53900) 2023-10-06T11:30:43.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309e24a700 1 -- 172.21.15.46:0/3545358550 learned_addr learned my addr 172.21.15.46:0/3545358550 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:43.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309ea4b700 1 --1- 172.21.15.46:0/3545358550 >> v1:172.21.15.132:6789/0 conn(0x7f30980cc500 0x7f309815ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34496/0 (socket says 172.21.15.46:34496) 2023-10-06T11:30:43.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309da49700 1 -- 172.21.15.46:0/3545358550 >> v1:172.21.15.132:6789/0 conn(0x7f30980cc500 legacy=0x7f309815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:43.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309d248700 1 -- 172.21.15.46:0/3545358550 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498198032 0 0) 0x7f30980cf360 con 0x7f30981591f0 2023-10-06T11:30:43.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309da49700 1 -- 172.21.15.46:0/3545358550 >> v1:172.21.15.46:6789/0 conn(0x7f30981591f0 legacy=0x7f30980cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:43.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309da49700 1 -- 172.21.15.46:0/3545358550 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980cea30 con 0x7f30981595c0 2023-10-06T11:30:43.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309da49700 1 --2- 172.21.15.46:0/3545358550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30981595c0 0x7f30980af940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3088009860 tx=0x7f30880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81125b25b3f4d947 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:43.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309d248700 1 -- 172.21.15.46:0/3545358550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088011630 con 0x7f30981595c0 2023-10-06T11:30:43.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309d248700 1 -- 172.21.15.46:0/3545358550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088011790 con 0x7f30981595c0 2023-10-06T11:30:43.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309d248700 1 -- 172.21.15.46:0/3545358550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088011b50 con 0x7f30981595c0 2023-10-06T11:30:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 -- 172.21.15.46:0/3545358550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30981595c0 msgr2=0x7f30980af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:43.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 --2- 172.21.15.46:0/3545358550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30981595c0 0x7f30980af940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3088009860 tx=0x7f30880091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 -- 172.21.15.46:0/3545358550 shutdown_connections 2023-10-06T11:30:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 --2- 172.21.15.46:0/3545358550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30981595c0 0x7f30980af940 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 -- 172.21.15.46:0/3545358550 >> 172.21.15.46:0/3545358550 conn(0x7f309806ef60 msgr2=0x7f30980d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 -- 172.21.15.46:0/3545358550 shutdown_connections 2023-10-06T11:30:43.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.727+0000 7f309f24c700 1 -- 172.21.15.46:0/3545358550 wait complete. 2023-10-06T11:30:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 Processor -- start 2023-10-06T11:30:43.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 -- start start 2023-10-06T11:30:43.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30980cc500 0x7f30981508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30981591f0 0x7f30980b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3098150e00 con 0x7f30980b10a0 2023-10-06T11:30:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30980afac0 con 0x7f30980cc500 2023-10-06T11:30:43.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309f24c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30980afda0 con 0x7f30981591f0 2023-10-06T11:30:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58024/0 (socket says 172.21.15.46:58024) 2023-10-06T11:30:43.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 -- 172.21.15.46:0/3011635269 learned_addr learned my addr 172.21.15.46:0/3011635269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309e24a700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30980cc500 0x7f30981508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309da49700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30981591f0 0x7f30980b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 -- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30981591f0 msgr2=0x7f30980b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30981591f0 0x7f30980b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 -- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30980cc500 msgr2=0x7f30981508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.731+0000 7f309ea4b700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30980cc500 0x7f30981508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f309ea4b700 1 -- 172.21.15.46:0/3011635269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088009420 con 0x7f30980b10a0 2023-10-06T11:30:43.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f309ea4b700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f309000cd80 tx=0x7f3090001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090005df0 con 0x7f30980b10a0 2023-10-06T11:30:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309000a960 con 0x7f30980b10a0 2023-10-06T11:30:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30980b0080 con 0x7f30980b10a0 2023-10-06T11:30:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309000ad20 con 0x7f30980b10a0 2023-10-06T11:30:43.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30980b0640 con 0x7f30980b10a0 2023-10-06T11:30:43.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.735+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30980b0830 con 0x7f30980b10a0 2023-10-06T11:30:43.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.739+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f3090007300 con 0x7f30980b10a0 2023-10-06T11:30:43.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.739+0000 7f308e7fc700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 0x7f307c062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:43.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.739+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f309007d3c0 con 0x7f30980b10a0 2023-10-06T11:30:43.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.739+0000 7f309e24a700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 0x7f307c062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:43.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.739+0000 7f309e24a700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 0x7f307c062c00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30940011e0 tx=0x7f3094009040 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.747+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f309004d1a0 con 0x7f30980b10a0 2023-10-06T11:30:43.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:43.939+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3098064590 con 0x7f307c060750 2023-10-06T11:30:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.087+0000 7f308e7fc700 1 -- 172.21.15.46:0/3011635269 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3098064590 con 0x7f307c060750 2023-10-06T11:30:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 msgr2=0x7f307c062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 0x7f307c062c00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30940011e0 tx=0x7f3094009040 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 msgr2=0x7f309815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f309000cd80 tx=0x7f3090001b40 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 shutdown_connections 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f307c060750 0x7f307c062c00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30980b10a0 0x7f309815abd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30981591f0 0x7f30980b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 --2- 172.21.15.46:0/3011635269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30980cc500 0x7f30981508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 >> 172.21.15.46:0/3011635269 conn(0x7f309806ef60 msgr2=0x7f30980b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 shutdown_connections 2023-10-06T11:30:44.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:44.091+0000 7f309f24c700 1 -- 172.21.15.46:0/3011635269 wait complete. 2023-10-06T11:30:44.122 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-06T11:30:44.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.122 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-06T11:30:44.146 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:44.146 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:44.146 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_01_UTC 2023-10-06T11:30:44.146 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:44.146 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:44.147 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:44.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_20_01_UTC 2023-10-06T11:30:44.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.148 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_20_01_UTC) 2023-10-06T11:30:44.229 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:44.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.229 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:44.304 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_22_01_UTC 2023-10-06T11:30:44.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.304 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_22_01_UTC) 2023-10-06T11:30:44.379 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:44.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.379 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:44.454 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:44.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.454 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:44.536 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:44.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.536 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:44.616 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:44.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.617 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:44.693 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:44.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.693 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:44.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:44.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.767 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:44.842 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:44.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.842 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:44.915 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_30_00_UTC 2023-10-06T11:30:44.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.915 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-11_30_00_UTC) 2023-10-06T11:30:44.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:44.994 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-06T11:30:45.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T11:30:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.275+0000 7f84cd7c3700 1 Processor -- start 2023-10-06T11:30:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.275+0000 7f84cd7c3700 1 -- start start 2023-10-06T11:30:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.275+0000 7f84cd7c3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84c80aea70 0x7f84c80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.275+0000 7f84cd7c3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80ab500 con 0x7f84c806f1f0 2023-10-06T11:30:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c8067f70 con 0x7f84c8068400 2023-10-06T11:30:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c8067a10 con 0x7f84c80aea70 2023-10-06T11:30:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84c80aea70 0x7f84c80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84bf7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84c806f1f0 0x7f84c80cc980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53912/0 (socket says 172.21.15.46:53912) 2023-10-06T11:30:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84bf7fe700 1 -- 172.21.15.46:0/3931300725 learned_addr learned my addr 172.21.15.46:0/3931300725 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c7fff700 1 -- 172.21.15.46:0/3931300725 >> v1:172.21.15.132:6789/0 conn(0x7f84c8068400 legacy=0x7f84c80aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c7fff700 1 -- 172.21.15.46:0/3931300725 >> v1:172.21.15.46:6789/0 conn(0x7f84c806f1f0 legacy=0x7f84c80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c7fff700 1 -- 172.21.15.46:0/3931300725 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80aee40 con 0x7f84c80aea70 2023-10-06T11:30:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c7fff700 1 --2- 172.21.15.46:0/3931300725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80cc440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f84b80098d0 tx=0x7f84b8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7f1272b92fc0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c77fe700 1 -- 172.21.15.46:0/3931300725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8009e70 con 0x7f84c80aea70 2023-10-06T11:30:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c77fe700 1 -- 172.21.15.46:0/3931300725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8005c10 con 0x7f84c80aea70 2023-10-06T11:30:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84c77fe700 1 -- 172.21.15.46:0/3931300725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800b6e0 con 0x7f84c80aea70 2023-10-06T11:30:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- 172.21.15.46:0/3931300725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 msgr2=0x7f84c80cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/3931300725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80cc440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f84b80098d0 tx=0x7f84b8009120 comp rx=0 tx=0).stop 2023-10-06T11:30:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- 172.21.15.46:0/3931300725 shutdown_connections 2023-10-06T11:30:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/3931300725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80cc440 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- 172.21.15.46:0/3931300725 >> 172.21.15.46:0/3931300725 conn(0x7f84c80afe40 msgr2=0x7f84c806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- 172.21.15.46:0/3931300725 shutdown_connections 2023-10-06T11:30:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.279+0000 7f84cd7c3700 1 -- 172.21.15.46:0/3931300725 wait complete. 2023-10-06T11:30:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 Processor -- start 2023-10-06T11:30:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 -- start start 2023-10-06T11:30:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84c8068400 0x7f84c81694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84c806f1f0 0x7f84c80c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c32e0 con 0x7f84c80aea70 2023-10-06T11:30:45.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c2430 con 0x7f84c806f1f0 2023-10-06T11:30:45.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84cd7c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80c26d0 con 0x7f84c8068400 2023-10-06T11:30:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84ccfc2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84ccfc2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58046/0 (socket says 172.21.15.46:58046) 2023-10-06T11:30:45.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84ccfc2700 1 -- 172.21.15.46:0/1438718413 learned_addr learned my addr 172.21.15.46:0/1438718413 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84c7fff700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84c8068400 0x7f84c81694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.283+0000 7f84bf7fe700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84c806f1f0 0x7f84c80c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 -- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84c8068400 msgr2=0x7f84c81694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84c8068400 0x7f84c81694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 -- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84c806f1f0 msgr2=0x7f84c80c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84c806f1f0 0x7f84c80c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 -- 172.21.15.46:0/1438718413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80093c0 con 0x7f84c80aea70 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84ccfc2700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c840 tx=0x7f84c0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0015c40 con 0x7f84c80aea70 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c0015dc0 con 0x7f84c80aea70 2023-10-06T11:30:45.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c001d680 con 0x7f84c80aea70 2023-10-06T11:30:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c80c29b0 con 0x7f84c80aea70 2023-10-06T11:30:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c80c2f70 con 0x7f84c80aea70 2023-10-06T11:30:45.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.287+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c80c3450 con 0x7f84c80aea70 2023-10-06T11:30:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.291+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f84c0003030 con 0x7f84c80aea70 2023-10-06T11:30:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.291+0000 7f84c4ff9700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 0x7f84a8062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.291+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f84c0085180 con 0x7f84c80aea70 2023-10-06T11:30:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.291+0000 7f84c7fff700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 0x7f84a8062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.291+0000 7f84c7fff700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 0x7f84a8062c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009fd0 tx=0x7f84b8016000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.299+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f84c0054d90 con 0x7f84c80aea70 2023-10-06T11:30:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.491+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f84c8064590 con 0x7f84a8060750 2023-10-06T11:30:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.635+0000 7f84c4ff9700 1 -- 172.21.15.46:0/1438718413 <== mgr.5211 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f84c8064590 con 0x7f84a8060750 2023-10-06T11:30:45.645 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-06T11:30:45.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 msgr2=0x7f84a8062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 0x7f84a8062c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009fd0 tx=0x7f84b8016000 comp rx=0 tx=0).stop 2023-10-06T11:30:45.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 msgr2=0x7f84c80c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:45.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c840 tx=0x7f84c0001670 comp rx=0 tx=0).stop 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 shutdown_connections 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f84a8060750 0x7f84a8062c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84c80aea70 0x7f84c80c4920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84c806f1f0 0x7f84c80c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 --2- 172.21.15.46:0/1438718413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84c8068400 0x7f84c81694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 >> 172.21.15.46:0/1438718413 conn(0x7f84c80afe40 msgr2=0x7f84c806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 shutdown_connections 2023-10-06T11:30:45.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:45.639+0000 7f84cd7c3700 1 -- 172.21.15.46:0/1438718413 wait complete. 2023-10-06T11:30:45.671 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-06T11:30:45.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:45.671 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-06T11:30:45.697 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:45.698 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:45.699 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_20_00_UTC 2023-10-06T11:30:45.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:45.700 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_20_00_UTC) 2023-10-06T11:30:45.781 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_21_00_UTC 2023-10-06T11:30:45.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:45.781 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_21_00_UTC) 2023-10-06T11:30:45.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_22_00_UTC 2023-10-06T11:30:45.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:45.856 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_22_00_UTC) 2023-10-06T11:30:45.935 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_23_00_UTC 2023-10-06T11:30:45.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:45.935 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_23_00_UTC) 2023-10-06T11:30:46.009 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_24_00_UTC 2023-10-06T11:30:46.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.010 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_24_00_UTC) 2023-10-06T11:30:46.093 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_25_00_UTC 2023-10-06T11:30:46.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.093 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_25_00_UTC) 2023-10-06T11:30:46.169 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_26_00_UTC 2023-10-06T11:30:46.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_26_00_UTC) 2023-10-06T11:30:46.245 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_27_00_UTC 2023-10-06T11:30:46.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.245 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_27_00_UTC) 2023-10-06T11:30:46.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_28_00_UTC 2023-10-06T11:30:46.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.321 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_28_00_UTC) 2023-10-06T11:30:46.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_29_00_UTC 2023-10-06T11:30:46.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.542 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_29_00_UTC) 2023-10-06T11:30:46.587 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_30_01_UTC 2023-10-06T11:30:46.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.588 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-11_30_01_UTC) 2023-10-06T11:30:46.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:46.660 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-06T11:30:46.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:30:46.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:30:46.926 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:30:46.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 Processor -- start 2023-10-06T11:30:46.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 -- start start 2023-10-06T11:30:46.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fcc0b2360 0x7f8fcc0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0af6f0 con 0x7f8fcc0aee00 2023-10-06T11:30:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0b43e0 con 0x7f8fcc159550 2023-10-06T11:30:46.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.935+0000 7f8fd1678700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0b04b0 con 0x7f8fcc0b2360 2023-10-06T11:30:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fca7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8fcc0aee00 0x7f8fcc158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53928/0 (socket says 172.21.15.46:53928) 2023-10-06T11:30:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fca7fc700 1 -- 172.21.15.46:0/1682492731 learned_addr learned my addr 172.21.15.46:0/1682492731 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fcaffd700 1 --2- 172.21.15.46:0/1682492731 >> v2:172.21.15.132:3300/0 conn(0x7f8fcc0b2360 0x7f8fcc0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228766159 0 0) 0x7f8fcc0af6f0 con 0x7f8fcc0aee00 2023-10-06T11:30:46.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb4003570 con 0x7f8fcc0aee00 2023-10-06T11:30:46.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364109681 0 0) 0x7f8fcc0b43e0 con 0x7f8fcc159550 2023-10-06T11:30:46.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc0af6f0 con 0x7f8fcc159550 2023-10-06T11:30:46.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fcaffd700 1 -- 172.21.15.46:0/1682492731 >> v1:172.21.15.132:6789/0 conn(0x7f8fcc159550 legacy=0x7f8fcc0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fcaffd700 1 -- 172.21.15.46:0/1682492731 >> v1:172.21.15.46:6789/0 conn(0x7f8fcc0aee00 legacy=0x7f8fcc158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fcaffd700 1 -- 172.21.15.46:0/1682492731 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0b0790 con 0x7f8fcc0b2360 2023-10-06T11:30:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fcaffd700 1 --2- 172.21.15.46:0/1682492731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0b2360 0x7f8fcc0b2730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098e0 tx=0x7f8fbc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370b68b13fdfd392 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011630 con 0x7f8fcc0b2360 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc011790 con 0x7f8fcc0b2360 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.939+0000 7f8fc9ffb700 1 -- 172.21.15.46:0/1682492731 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011b30 con 0x7f8fcc0b2360 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- 172.21.15.46:0/1682492731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0b2360 msgr2=0x7f8fcc0b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 --2- 172.21.15.46:0/1682492731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0b2360 0x7f8fcc0b2730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0098e0 tx=0x7f8fbc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- 172.21.15.46:0/1682492731 shutdown_connections 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 --2- 172.21.15.46:0/1682492731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0b2360 0x7f8fcc0b2730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- 172.21.15.46:0/1682492731 >> 172.21.15.46:0/1682492731 conn(0x7f8fcc06e780 msgr2=0x7f8fcc06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- 172.21.15.46:0/1682492731 shutdown_connections 2023-10-06T11:30:46.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- 172.21.15.46:0/1682492731 wait complete. 2023-10-06T11:30:46.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 Processor -- start 2023-10-06T11:30:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- start start 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0aee00 0x7f8fcc157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 0x7f8fcc15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 0x7f8fcc0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc15ac40 con 0x7f8fcc159550 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0c9980 con 0x7f8fcc0aee00 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fd1678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0c94f0 con 0x7f8fcc0b2360 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fca7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 0x7f8fcc15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.943+0000 7f8fcb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 0x7f8fcc0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fca7fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 0x7f8fcc15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38848/0 (socket says 172.21.15.46:38848) 2023-10-06T11:30:46.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fca7fc700 1 -- 172.21.15.46:0/3788110769 learned_addr learned my addr 172.21.15.46:0/3788110769 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcaffd700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0aee00 0x7f8fcc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 -- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 msgr2=0x7f8fcc15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 0x7f8fcc15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 -- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0aee00 msgr2=0x7f8fcc157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0aee00 0x7f8fcc157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 -- 172.21.15.46:0/3788110769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc009420 con 0x7f8fcc159550 2023-10-06T11:30:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fcb7fe700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 0x7f8fcc0cc700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c910 tx=0x7f8fc40026f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4012630 con 0x7f8fcc159550 2023-10-06T11:30:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc4012790 con 0x7f8fcc159550 2023-10-06T11:30:46.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4012b50 con 0x7f8fcc159550 2023-10-06T11:30:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc157c90 con 0x7f8fcc159550 2023-10-06T11:30:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0c8c40 con 0x7f8fcc159550 2023-10-06T11:30:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.947+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc0c96a0 con 0x7f8fcc159550 2023-10-06T11:30:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.951+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 80147+0+0 (secure 0 0 0) 0x7f8fc401a3b0 con 0x7f8fcc159550 2023-10-06T11:30:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.951+0000 7f8fb2ffd700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 0x7f8fb8062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.951+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7217+0+0 (secure 0 0 0) 0x7f8fc407d270 con 0x7f8fcc159550 2023-10-06T11:30:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.951+0000 7f8fcaffd700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 0x7f8fb8062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.951+0000 7f8fcaffd700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 0x7f8fb8062c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0093f0 tx=0x7f8fbc017000 comp rx=0 tx=0).ready entity=mgr.5211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:46.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:46.955+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fcc0c96a0 con 0x7f8fcc159550 2023-10-06T11:30:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.167+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8fcc0c9660 con 0x7f8fcc159550 2023-10-06T11:30:47.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.363+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7f8fc404d320 con 0x7f8fcc159550 2023-10-06T11:30:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.367+0000 7f8fb2ffd700 1 -- 172.21.15.46:0/3788110769 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v13) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8fc40509e0 con 0x7f8fcc159550 2023-10-06T11:30:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 msgr2=0x7f8fb8062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 0x7f8fb8062c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0093f0 tx=0x7f8fbc017000 comp rx=0 tx=0).stop 2023-10-06T11:30:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 msgr2=0x7f8fcc0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 0x7f8fcc0cc700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c910 tx=0x7f8fc40026f0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 shutdown_connections 2023-10-06T11:30:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8fb8060780 0x7f8fb8062c30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fcc159550 0x7f8fcc0cc700 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fcc0b2360 0x7f8fcc15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 --2- 172.21.15.46:0/3788110769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc0aee00 0x7f8fcc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 >> 172.21.15.46:0/3788110769 conn(0x7f8fcc06e780 msgr2=0x7f8fcc0b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 shutdown_connections 2023-10-06T11:30:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.375+0000 7f8fd1678700 1 -- 172.21.15.46:0/3788110769 wait complete. 2023-10-06T11:30:47.414 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:30:47.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:30:47.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:30:47.493 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:30:47.493 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:30:47.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:30:47.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:30:47.609 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:47.606+0000 7f25783ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:30:47.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:30:47.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:30:47.618 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:30:47.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:30:47.670 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:47.659+0000 7fd0b75c7000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:30:47.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.835+0000 7fb142c25700 1 Processor -- start 2023-10-06T11:30:47.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.835+0000 7fb142c25700 1 -- start start 2023-10-06T11:30:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb142c25700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13c0cb4e0 0x7fb13c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb142c25700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0cb8b0 con 0x7fb13c0ce640 2023-10-06T11:30:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb142c25700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0c8570 con 0x7fb13c069640 2023-10-06T11:30:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb142c25700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c069180 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13c0cb4e0 0x7fb13c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:47.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb13c0cb4e0 0x7fb13c068c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58106/0 (socket says 172.21.15.46:58106) 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 -- 172.21.15.46:0/1195485895 learned_addr learned my addr 172.21.15.46:0/1195485895 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1195485895 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993943549 0 0) 0x7fb13c0c8570 con 0x7fb13c069640 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1195485895 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12c003570 con 0x7fb13c069640 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 -- 172.21.15.46:0/1195485895 >> v1:172.21.15.132:6789/0 conn(0x7fb13c069640 legacy=0x7fb13c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 -- 172.21.15.46:0/1195485895 >> v1:172.21.15.46:6789/0 conn(0x7fb13c0ce640 legacy=0x7fb13c14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 -- 172.21.15.46:0/1195485895 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0687b0 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb1409c1700 1 --2- 172.21.15.46:0/1195485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c068c40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb1300099b0 tx=0x7fb130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901a56d2a20d2fb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1195485895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130009ea0 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1195485895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000b860 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.839+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1195485895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000bc00 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- 172.21.15.46:0/1195485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0cb4e0 msgr2=0x7fb13c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 --2- 172.21.15.46:0/1195485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c068c40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb1300099b0 tx=0x7fb130009250 comp rx=0 tx=0).stop 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- 172.21.15.46:0/1195485895 shutdown_connections 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 --2- 172.21.15.46:0/1195485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c068c40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- 172.21.15.46:0/1195485895 >> 172.21.15.46:0/1195485895 conn(0x7fb13c062d50 msgr2=0x7fb13c063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- 172.21.15.46:0/1195485895 shutdown_connections 2023-10-06T11:30:47.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- 172.21.15.46:0/1195485895 wait complete. 2023-10-06T11:30:47.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 Processor -- start 2023-10-06T11:30:47.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.843+0000 7fb142c25700 1 -- start start 2023-10-06T11:30:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb13c069640 0x7fb13c0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c1690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:47.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0b23e0 con 0x7fb13c0ce640 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0b17d0 con 0x7fb13c0cb4e0 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0b1ab0 con 0x7fb13c069640 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58110/0 (socket says 172.21.15.46:58110) 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 -- 172.21.15.46:0/3300709927 learned_addr learned my addr 172.21.15.46:0/3300709927 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb13bfff700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c1690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:47.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1409c1700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb13c069640 0x7fb13c0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:47.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb13c069640 msgr2=0x7fb13c0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb13c069640 0x7fb13c0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb13c0cb4e0 msgr2=0x7fb13c1690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c1690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 -- 172.21.15.46:0/3300709927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300094f0 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb1411c2700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c910 tx=0x7fb1340016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134012630 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134012790 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c0b1d90 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.847+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134012b50 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.851+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c0b12c0 con 0x7fb13c0ce640 2023-10-06T11:30:47.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.851+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0b2590 con 0x7fb13c0ce640 2023-10-06T11:30:47.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.851+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7fb13401b450 con 0x7fb13c0ce640 2023-10-06T11:30:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.851+0000 7fb138ff9700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 0x7fb120062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.855+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7fb13407d140 con 0x7fb13c0ce640 2023-10-06T11:30:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.855+0000 7fb1409c1700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 msgr2=0x7fb120062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.855+0000 7fb1409c1700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 0x7fb120062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:47.859+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb13c0b2590 con 0x7fb13c0ce640 2023-10-06T11:30:48.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.059+0000 7fb1409c1700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 msgr2=0x7fb120062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:48.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.059+0000 7fb1409c1700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 0x7fb120062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:30:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.083+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb13c0cb8b0 con 0x7fb13c0ce640 2023-10-06T11:30:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb138ff9700 1 -- 172.21.15.46:0/3300709927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb13c0cb8b0 con 0x7fb13c0ce640 2023-10-06T11:30:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:48.095 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:30:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 msgr2=0x7fb120062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 0x7fb120062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 msgr2=0x7fb13c1695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c910 tx=0x7fb1340016f0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 shutdown_connections 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb120060700 0x7fb120062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb13c0ce640 0x7fb13c1695e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb13c0cb4e0 0x7fb13c1690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 --2- 172.21.15.46:0/3300709927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb13c069640 0x7fb13c0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.087+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 >> 172.21.15.46:0/3300709927 conn(0x7fb13c062d50 msgr2=0x7fb13c0cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.091+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 shutdown_connections 2023-10-06T11:30:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.091+0000 7fb142c25700 1 -- 172.21.15.46:0/3300709927 wait complete. 2023-10-06T11:30:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:30:48.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:30:48.174 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.174+0000 7f25783ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:30:48.252 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.250+0000 7f25783ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:30:48.268 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:48.259+0000 7fd0b75c7000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:30:48.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.267+0000 7fd76ddcc700 1 Processor -- start 2023-10-06T11:30:48.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.267+0000 7fd76ddcc700 1 -- start start 2023-10-06T11:30:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.267+0000 7fd76ddcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 0x7fd7680b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.267+0000 7fd76ddcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7680d07b0 0x7fd7680b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680ae340 con 0x7fd7680b1130 2023-10-06T11:30:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680b15b0 con 0x7fd7680d07b0 2023-10-06T11:30:48.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680b09a0 con 0x7fd768159290 2023-10-06T11:30:48.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7680d07b0 0x7fd7680b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7680d07b0 0x7fd7680b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58124/0 (socket says 172.21.15.46:58124) 2023-10-06T11:30:48.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 -- 172.21.15.46:0/126350651 learned_addr learned my addr 172.21.15.46:0/126350651 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:48.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76cdca700 1 --2- 172.21.15.46:0/126350651 >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 0x7fd7680b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 -- 172.21.15.46:0/126350651 >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 msgr2=0x7fd7680b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 --2- 172.21.15.46:0/126350651 >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 0x7fd7680b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 -- 172.21.15.46:0/126350651 >> v1:172.21.15.132:6789/0 conn(0x7fd7680b1130 legacy=0x7fd7680ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 -- 172.21.15.46:0/126350651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768159660 con 0x7fd7680d07b0 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76cdca700 1 --2- 172.21.15.46:0/126350651 >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 0x7fd7680b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:48.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd767fff700 1 --2- 172.21.15.46:0/126350651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7680d07b0 0x7fd7680b0bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd768071530 tx=0x7fd754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20c5c0c6ab01700 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:48.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd7677fe700 1 -- 172.21.15.46:0/126350651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400b8f0 con 0x7fd7680d07b0 2023-10-06T11:30:48.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd7677fe700 1 -- 172.21.15.46:0/126350651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75400ba50 con 0x7fd7680d07b0 2023-10-06T11:30:48.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd7677fe700 1 -- 172.21.15.46:0/126350651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400bdf0 con 0x7fd7680d07b0 2023-10-06T11:30:48.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- 172.21.15.46:0/126350651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7680d07b0 msgr2=0x7fd7680b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/126350651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7680d07b0 0x7fd7680b0bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd768071530 tx=0x7fd754009250 comp rx=0 tx=0).stop 2023-10-06T11:30:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- 172.21.15.46:0/126350651 shutdown_connections 2023-10-06T11:30:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/126350651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7680d07b0 0x7fd7680b0bf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/126350651 >> v2:172.21.15.132:3300/0 conn(0x7fd768159290 0x7fd7680b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.271+0000 7fd76ddcc700 1 -- 172.21.15.46:0/126350651 >> 172.21.15.46:0/126350651 conn(0x7fd76806ef60 msgr2=0x7fd7680d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:48.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.275+0000 7fd76ddcc700 1 -- 172.21.15.46:0/126350651 shutdown_connections 2023-10-06T11:30:48.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.275+0000 7fd76ddcc700 1 -- 172.21.15.46:0/126350651 wait complete. 2023-10-06T11:30:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.275+0000 7fd76ddcc700 1 Processor -- start 2023-10-06T11:30:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.275+0000 7fd76ddcc700 1 -- start start 2023-10-06T11:30:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7680b1130 0x7fd768159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7680d07b0 0x7fd76814b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76814b550 con 0x7fd768159290 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76815aa40 con 0x7fd7680d07b0 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd768159e30 con 0x7fd7680b1130 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58132/0 (socket says 172.21.15.46:58132) 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 -- 172.21.15.46:0/1198208476 learned_addr learned my addr 172.21.15.46:0/1198208476 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd767fff700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7680d07b0 0x7fd76814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76cdca700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7680b1130 0x7fd768159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 -- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7680b1130 msgr2=0x7fd768159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7680b1130 0x7fd768159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 -- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7680d07b0 msgr2=0x7fd76814b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7680d07b0 0x7fd76814b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 -- 172.21.15.46:0/1198208476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7540094f0 con 0x7fd768159290 2023-10-06T11:30:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76d5cb700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c840 tx=0x7fd760001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012630 con 0x7fd768159290 2023-10-06T11:30:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7600127b0 con 0x7fd768159290 2023-10-06T11:30:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76815a110 con 0x7fd768159290 2023-10-06T11:30:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012b70 con 0x7fd768159290 2023-10-06T11:30:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.279+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76815a6d0 con 0x7fd768159290 2023-10-06T11:30:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76815abb0 con 0x7fd768159290 2023-10-06T11:30:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7fd76001a3a0 con 0x7fd768159290 2023-10-06T11:30:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd764ff9700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 0x7fd74c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7fd76007d300 con 0x7fd768159290 2023-10-06T11:30:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd76cdca700 1 -- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 msgr2=0x7fd74c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:48.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.283+0000 7fd76cdca700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 0x7fd74c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.291+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd76004cf90 con 0x7fd768159290 2023-10-06T11:30:48.350 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:48.343+0000 7fd0b75c7000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:30:48.396 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.394+0000 7f25783ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:30:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.463+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd768064590 con 0x7fd768159290 2023-10-06T11:30:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.467+0000 7fd764ff9700 1 -- 172.21.15.46:0/1198208476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18763 (secure 0 0 0) 0x7fd7600505b0 con 0x7fd768159290 2023-10-06T11:30:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:48.476 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:30:47.374792+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/32974":"2023-10-07T11:19:27.796424+0000","172.21.15.46:6832/32974":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/3406707913":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/2277270196":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/568326314":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/3506103189":"2023-10-07T11:19:27.796424+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":2},{"begin":188,"length":33}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":72,"length":1},{"begin":83,"length":1},{"begin":97,"length":1},{"begin":218,"length":3}]}],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":4,"length":1},{"begin":9,"length":2},{"begin":15,"length":2},{"begin":20,"length":1},{"begin":25,"length":1},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":46,"length":1},{"begin":48,"length":1},{"begin":52,"length":1},{"begin":54,"length":1},{"begin":60,"length":1},{"begin":63,"length":2},{"begin":67,"length":1},{"begin":76,"length":1},{"begin":78,"length":1},{"begin":81,"length":2},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":101,"length":1},{"begin":105,"length":3},{"begin":153,"length":35}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 msgr2=0x7fd74c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 0x7fd74c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 msgr2=0x7fd76814cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c840 tx=0x7fd760001670 comp rx=0 tx=0).stop 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 shutdown_connections 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd74c060700 0x7fd74c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd768159290 0x7fd76814cb80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7680d07b0 0x7fd76814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 --2- 172.21.15.46:0/1198208476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7680b1130 0x7fd768159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 >> 172.21.15.46:0/1198208476 conn(0x7fd76806ef60 msgr2=0x7fd76814de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 shutdown_connections 2023-10-06T11:30:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.471+0000 7fd76ddcc700 1 -- 172.21.15.46:0/1198208476 wait complete. 2023-10-06T11:30:48.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:19:11.591550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:19:12.896650+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:30:48.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:30:48.504 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:48.491+0000 7fd0b75c7000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:30:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.651+0000 7f1902723700 1 Processor -- start 2023-10-06T11:30:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.651+0000 7f1902723700 1 -- start start 2023-10-06T11:30:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1902723700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1902723700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 0x7f18fc0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1902723700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0aca00 con 0x7f18fc0abfc0 2023-10-06T11:30:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1902723700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc0ada20 con 0x7f18fc06f1d0 2023-10-06T11:30:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1902723700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0633b0 con 0x7f18fc0ae890 2023-10-06T11:30:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1901f22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 0x7f18fc0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1900f20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1900f20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38912/0 (socket says 172.21.15.46:38912) 2023-10-06T11:30:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1900f20700 1 -- 172.21.15.46:0/3051051057 learned_addr learned my addr 172.21.15.46:0/3051051057 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.655+0000 7f1901f22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 0x7f18fc0ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58148/0 (socket says 172.21.15.46:58148) 2023-10-06T11:30:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1900f20700 1 -- 172.21.15.46:0/3051051057 >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 msgr2=0x7f18fc0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1900f20700 1 --2- 172.21.15.46:0/3051051057 >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 0x7f18fc0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1900f20700 1 -- 172.21.15.46:0/3051051057 >> v1:172.21.15.132:6789/0 conn(0x7f18fc0abfc0 legacy=0x7f18fc096140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1900f20700 1 -- 172.21.15.46:0/3051051057 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc062df0 con 0x7f18fc0ae890 2023-10-06T11:30:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1900f20700 1 --2- 172.21.15.46:0/3051051057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0099b0 tx=0x7f18ec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ce949f77c7f20c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f18f3fff700 1 -- 172.21.15.46:0/3051051057 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec009ea0 con 0x7f18fc0ae890 2023-10-06T11:30:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f18f3fff700 1 -- 172.21.15.46:0/3051051057 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec00b8f0 con 0x7f18fc0ae890 2023-10-06T11:30:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f18f3fff700 1 -- 172.21.15.46:0/3051051057 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00bc90 con 0x7f18fc0ae890 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 -- 172.21.15.46:0/3051051057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 msgr2=0x7f18fc0ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 --2- 172.21.15.46:0/3051051057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0099b0 tx=0x7f18ec009250 comp rx=0 tx=0).stop 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 -- 172.21.15.46:0/3051051057 shutdown_connections 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 --2- 172.21.15.46:0/3051051057 >> v2:172.21.15.46:3300/0 conn(0x7f18fc06f1d0 0x7f18fc0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 --2- 172.21.15.46:0/3051051057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc0ae890 0x7f18fc0ac390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 -- 172.21.15.46:0/3051051057 >> 172.21.15.46:0/3051051057 conn(0x7f18fc0afd10 msgr2=0x7f18fc06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 -- 172.21.15.46:0/3051051057 shutdown_connections 2023-10-06T11:30:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.659+0000 7f1902723700 1 -- 172.21.15.46:0/3051051057 wait complete. 2023-10-06T11:30:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 Processor -- start 2023-10-06T11:30:48.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 -- start start 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 0x7f18fc1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18fc0abfc0 0x7f18fc0c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc15c910 con 0x7f18fc0ae890 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0c4e10 con 0x7f18fc0abfc0 2023-10-06T11:30:48.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1902723700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0c50f0 con 0x7f18fc06f1d0 2023-10-06T11:30:48.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58150/0 (socket says 172.21.15.46:58150) 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 -- 172.21.15.46:0/4254013142 learned_addr learned my addr 172.21.15.46:0/4254013142 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1900f20700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18fc0abfc0 0x7f18fc0c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901721700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 0x7f18fc1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 -- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 msgr2=0x7f18fc1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 0x7f18fc1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 -- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18fc0abfc0 msgr2=0x7f18fc0c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18fc0abfc0 0x7f18fc0c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 -- 172.21.15.46:0/4254013142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0094f0 con 0x7f18fc0ae890 2023-10-06T11:30:48.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901721700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 0x7f18fc1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:48.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f1901f22700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009e90 tx=0x7f18f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:48.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.663+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400cc20 con 0x7f18fc0ae890 2023-10-06T11:30:48.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400f040 con 0x7f18fc0ae890 2023-10-06T11:30:48.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc0c53d0 con 0x7f18fc0ae890 2023-10-06T11:30:48.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4014480 con 0x7f18fc0ae890 2023-10-06T11:30:48.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc1566f0 con 0x7f18fc0ae890 2023-10-06T11:30:48.676 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.674+0000 7f25783ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:30:48.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc064590 con 0x7f18fc0ae890 2023-10-06T11:30:48.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.667+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7f18f4014830 con 0x7f18fc0ae890 2023-10-06T11:30:48.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.671+0000 7f18f17fa700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 0x7f18e0062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:48.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.671+0000 7f1901721700 1 -- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 msgr2=0x7f18e0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:48.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.671+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7f18f407dd20 con 0x7f18fc0ae890 2023-10-06T11:30:48.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.671+0000 7f1901721700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 0x7f18e0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:48.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.675+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f18f4051330 con 0x7f18fc0ae890 2023-10-06T11:30:48.749 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.746+0000 7f25783ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:30:48.797 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:48.787+0000 7fd0b75c7000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:30:48.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.855+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18fc0630d0 con 0x7f18fc0ae890 2023-10-06T11:30:48.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.855+0000 7f18f17fa700 1 -- 172.21.15.46:0/4254013142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18f4018020 con 0x7f18fc0ae890 2023-10-06T11:30:48.866 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:30:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 msgr2=0x7f18e0062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 0x7f18e0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 msgr2=0x7f18fc0c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009e90 tx=0x7f18f400b040 comp rx=0 tx=0).stop 2023-10-06T11:30:48.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 shutdown_connections 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18e0060700 0x7f18e0062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18fc0ae890 0x7f18fc0c2d30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18fc0abfc0 0x7f18fc0c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 --2- 172.21.15.46:0/4254013142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18fc06f1d0 0x7f18fc1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 >> 172.21.15.46:0/4254013142 conn(0x7f18fc0afd10 msgr2=0x7f18fc06acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 shutdown_connections 2023-10-06T11:30:48.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:48.859+0000 7f1902723700 1 -- 172.21.15.46:0/4254013142 wait complete. 2023-10-06T11:30:48.876 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:48.867+0000 7fd0b75c7000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:30:48.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:30:48.971 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:48.970+0000 7f25783ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:30:49.052 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:49.050+0000 7f25783ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:30:49.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.051+0000 7f0f7e46e700 1 Processor -- start 2023-10-06T11:30:49.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.051+0000 7f0f7e46e700 1 -- start start 2023-10-06T11:30:49.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0f78069410 0x7f0f7815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f780697e0 0x7f0f7815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f780ce470 con 0x7f0f78158d10 2023-10-06T11:30:49.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7815a890 con 0x7f0f780697e0 2023-10-06T11:30:49.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7815d470 con 0x7f0f78069410 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f780697e0 0x7f0f7815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f780697e0 0x7f0f7815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58162/0 (socket says 172.21.15.46:58162) 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 -- 172.21.15.46:0/1534451832 learned_addr learned my addr 172.21.15.46:0/1534451832 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f77fff700 1 --2- 172.21.15.46:0/1534451832 >> v2:172.21.15.132:3300/0 conn(0x7f0f78069410 0x7f0f7815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 -- 172.21.15.46:0/1534451832 >> v2:172.21.15.132:3300/0 conn(0x7f0f78069410 msgr2=0x7f0f7815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:49.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 --2- 172.21.15.46:0/1534451832 >> v2:172.21.15.132:3300/0 conn(0x7f0f78069410 0x7f0f7815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 -- 172.21.15.46:0/1534451832 >> v1:172.21.15.132:6789/0 conn(0x7f0f78158d10 legacy=0x7f0f78159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 -- 172.21.15.46:0/1534451832 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7815d750 con 0x7f0f780697e0 2023-10-06T11:30:49.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f777fe700 1 --2- 172.21.15.46:0/1534451832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f780697e0 0x7f0f7815a350 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f780cdd90 tx=0x7f0f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdca596f6844b899 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f76ffd700 1 -- 172.21.15.46:0/1534451832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c009e70 con 0x7f0f780697e0 2023-10-06T11:30:49.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f76ffd700 1 -- 172.21.15.46:0/1534451832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c00b860 con 0x7f0f780697e0 2023-10-06T11:30:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f76ffd700 1 -- 172.21.15.46:0/1534451832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00bc00 con 0x7f0f780697e0 2023-10-06T11:30:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 -- 172.21.15.46:0/1534451832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f780697e0 msgr2=0x7f0f7815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/1534451832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f780697e0 0x7f0f7815a350 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f780cdd90 tx=0x7f0f6c009250 comp rx=0 tx=0).stop 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 -- 172.21.15.46:0/1534451832 shutdown_connections 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/1534451832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f780697e0 0x7f0f7815a350 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.055+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/1534451832 >> v2:172.21.15.132:3300/0 conn(0x7f0f78069410 0x7f0f7815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.059+0000 7f0f7e46e700 1 -- 172.21.15.46:0/1534451832 >> 172.21.15.46:0/1534451832 conn(0x7f0f780d0210 msgr2=0x7f0f780d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.059+0000 7f0f7e46e700 1 -- 172.21.15.46:0/1534451832 shutdown_connections 2023-10-06T11:30:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.059+0000 7f0f7e46e700 1 -- 172.21.15.46:0/1534451832 wait complete. 2023-10-06T11:30:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.059+0000 7f0f7e46e700 1 Processor -- start 2023-10-06T11:30:49.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.059+0000 7f0f7e46e700 1 -- start start 2023-10-06T11:30:49.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f78069410 0x7f0f780c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f780697e0 0x7f0f780caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780b3260 con 0x7f0f78158d10 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780b3540 con 0x7f0f780697e0 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780b3820 con 0x7f0f78069410 2023-10-06T11:30:49.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58164/0 (socket says 172.21.15.46:58164) 2023-10-06T11:30:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 -- 172.21.15.46:0/858796660 learned_addr learned my addr 172.21.15.46:0/858796660 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f77fff700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f78069410 0x7f0f780c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f777fe700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f780697e0 0x7f0f780caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 -- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f78069410 msgr2=0x7f0f780c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f78069410 0x7f0f780c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 -- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f780697e0 msgr2=0x7f0f780caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f780697e0 0x7f0f780caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 -- 172.21.15.46:0/858796660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0094f0 con 0x7f0f78158d10 2023-10-06T11:30:49.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7ca0b700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000c8e0 tx=0x7f0f70001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70012630 con 0x7f0f78158d10 2023-10-06T11:30:49.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70012790 con 0x7f0f78158d10 2023-10-06T11:30:49.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f780b3b00 con 0x7f0f78158d10 2023-10-06T11:30:49.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70012b50 con 0x7f0f78158d10 2023-10-06T11:30:49.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.063+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f780cafa0 con 0x7f0f78158d10 2023-10-06T11:30:49.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.067+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7815a490 con 0x7f0f78158d10 2023-10-06T11:30:49.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.067+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7f0f7001a3a0 con 0x7f0f78158d10 2023-10-06T11:30:49.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.067+0000 7f0f5ffff700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 0x7f0f64062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.067+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7f0f7007d2b0 con 0x7f0f78158d10 2023-10-06T11:30:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.067+0000 7f0f77fff700 1 -- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 msgr2=0x7f0f64062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.071+0000 7f0f77fff700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 0x7f0f64062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:49.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.075+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0f7815a490 con 0x7f0f78158d10 2023-10-06T11:30:49.110 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:49.103+0000 7fd0b75c7000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:30:49.144 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:49.142+0000 7f25783ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:30:49.198 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:49.187+0000 7fd0b75c7000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:30:49.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.243+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0f7816e2e0 con 0x7f0f78158d10 2023-10-06T11:30:49.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.243+0000 7f0f5ffff700 1 -- 172.21.15.46:0/858796660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0f7816e2e0 con 0x7f0f78158d10 2023-10-06T11:30:49.252 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:30:49.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 msgr2=0x7f0f64062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 0x7f0f64062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 msgr2=0x7f0f7816b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000c8e0 tx=0x7f0f70001710 comp rx=0 tx=0).stop 2023-10-06T11:30:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 shutdown_connections 2023-10-06T11:30:49.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0f64060700 0x7f0f64062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f78158d10 0x7f0f7816b250 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f780697e0 0x7f0f780caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 --2- 172.21.15.46:0/858796660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f78069410 0x7f0f780c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 >> 172.21.15.46:0/858796660 conn(0x7f0f780d0210 msgr2=0x7f0f780cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 shutdown_connections 2023-10-06T11:30:49.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.247+0000 7f0f7e46e700 1 -- 172.21.15.46:0/858796660 wait complete. 2023-10-06T11:30:49.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:30:49.292 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:49.283+0000 7fd0b75c7000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:30:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 Processor -- start 2023-10-06T11:30:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 -- start start 2023-10-06T11:30:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80880b2790 0x7f80880ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80880cc510 con 0x7f80880b23c0 2023-10-06T11:30:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80880b2be0 con 0x7f80880cd080 2023-10-06T11:30:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f80907c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80880cd500 con 0x7f80880b2790 2023-10-06T11:30:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f808e561700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f80880b23c0 0x7f80880cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53946/0 (socket says 172.21.15.46:53946) 2023-10-06T11:30:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f808e561700 1 -- 172.21.15.46:0/3160460648 learned_addr learned my addr 172.21.15.46:0/3160460648 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.435+0000 7f808dd60700 1 --2- 172.21.15.46:0/3160460648 >> v2:172.21.15.46:3300/0 conn(0x7f80880b2790 0x7f80880ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808d55f700 1 -- 172.21.15.46:0/3160460648 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900790014 0 0) 0x7f80880b2be0 con 0x7f80880cd080 2023-10-06T11:30:49.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808dd60700 1 -- 172.21.15.46:0/3160460648 >> v1:172.21.15.132:6789/0 conn(0x7f80880cd080 legacy=0x7f80880c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808dd60700 1 -- 172.21.15.46:0/3160460648 >> v1:172.21.15.46:6789/0 conn(0x7f80880b23c0 legacy=0x7f80880cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808dd60700 1 -- 172.21.15.46:0/3160460648 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880cb900 con 0x7f80880b2790 2023-10-06T11:30:49.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808dd60700 1 --2- 172.21.15.46:0/3160460648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b2790 0x7f80880ccb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80780098e0 tx=0x7f80780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9da1494f195919c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808d55f700 1 -- 172.21.15.46:0/3160460648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078009e80 con 0x7f80880b2790 2023-10-06T11:30:49.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808d55f700 1 -- 172.21.15.46:0/3160460648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807800b860 con 0x7f80880b2790 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f808d55f700 1 -- 172.21.15.46:0/3160460648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800bbe0 con 0x7f80880b2790 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 -- 172.21.15.46:0/3160460648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b2790 msgr2=0x7f80880ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 --2- 172.21.15.46:0/3160460648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b2790 0x7f80880ccb40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80780098e0 tx=0x7f80780091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 -- 172.21.15.46:0/3160460648 shutdown_connections 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 --2- 172.21.15.46:0/3160460648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b2790 0x7f80880ccb40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 -- 172.21.15.46:0/3160460648 >> 172.21.15.46:0/3160460648 conn(0x7f8088099480 msgr2=0x7f808806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 -- 172.21.15.46:0/3160460648 shutdown_connections 2023-10-06T11:30:49.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.439+0000 7f80907c5700 1 -- 172.21.15.46:0/3160460648 wait complete. 2023-10-06T11:30:49.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 Processor -- start 2023-10-06T11:30:49.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 -- start start 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 0x7f808815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80880cd080 0x7f8088068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80880b30f0 con 0x7f80880b23c0 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80880b33d0 con 0x7f80880cd080 2023-10-06T11:30:49.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.443+0000 7f80907c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80880b36b0 con 0x7f80880b2790 2023-10-06T11:30:49.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808dd60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 0x7f808815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808dd60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 0x7f808815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38930/0 (socket says 172.21.15.46:38930) 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808dd60700 1 -- 172.21.15.46:0/2564830373 learned_addr learned my addr 172.21.15.46:0/2564830373 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58186/0 (socket says 172.21.15.46:58186) 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808ed62700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80880cd080 0x7f8088068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 -- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 msgr2=0x7f808815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 0x7f808815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 -- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80880cd080 msgr2=0x7f8088068d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80880cd080 0x7f8088068d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 -- 172.21.15.46:0/2564830373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078009420 con 0x7f80880b23c0 2023-10-06T11:30:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f808e561700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f808400c810 tx=0x7f8084007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084005d80 con 0x7f80880b23c0 2023-10-06T11:30:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80840070f0 con 0x7f80880b23c0 2023-10-06T11:30:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80880b3990 con 0x7f80880b23c0 2023-10-06T11:30:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840074b0 con 0x7f80880b23c0 2023-10-06T11:30:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808815b3f0 con 0x7f80880b23c0 2023-10-06T11:30:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.447+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80880660f0 con 0x7f80880b23c0 2023-10-06T11:30:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.451+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7f80840078e0 con 0x7f80880b23c0 2023-10-06T11:30:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.451+0000 7f807e7fc700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 0x7f806c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.451+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7f808407cc80 con 0x7f80880b23c0 2023-10-06T11:30:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.451+0000 7f808dd60700 1 -- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 msgr2=0x7f806c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.451+0000 7f808dd60700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 0x7f806c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:49.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.459+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f808404c9b0 con 0x7f80880b23c0 2023-10-06T11:30:49.540 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:49.538+0000 7f25783ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:30:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.635+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80880cbbe0 con 0x7f80880b23c0 2023-10-06T11:30:49.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.635+0000 7f807e7fc700 1 -- 172.21.15.46:0/2564830373 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f808404ffd0 con 0x7f80880b23c0 2023-10-06T11:30:49.643 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:30:49.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 msgr2=0x7f806c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 0x7f806c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 msgr2=0x7f808816a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f808400c810 tx=0x7f8084007f90 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 shutdown_connections 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f806c060700 0x7f806c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80880cd080 0x7f8088068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80880b2790 0x7f808815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 --2- 172.21.15.46:0/2564830373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80880b23c0 0x7f808816a420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 >> 172.21.15.46:0/2564830373 conn(0x7f8088099480 msgr2=0x7f80881580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 shutdown_connections 2023-10-06T11:30:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.639+0000 7f80907c5700 1 -- 172.21.15.46:0/2564830373 wait complete. 2023-10-06T11:30:49.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:30:49.694 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:49.690+0000 7f25783ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:30:49.721 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:49.707+0000 7fd0b75c7000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:30:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.871+0000 7fbbf0478700 1 Processor -- start 2023-10-06T11:30:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 -- start start 2023-10-06T11:30:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbbe8069410 0x7fbbe815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbbe8159420 0x7fbbe815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80ce470 con 0x7fbbe80697e0 2023-10-06T11:30:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe815a890 con 0x7fbbe8159420 2023-10-06T11:30:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbf0478700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe815d470 con 0x7fbbe8069410 2023-10-06T11:30:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbeea15700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbbe8159420 0x7fbbe815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbeea15700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbbe8159420 0x7fbbe815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58192/0 (socket says 172.21.15.46:58192) 2023-10-06T11:30:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbeea15700 1 -- 172.21.15.46:0/303497502 learned_addr learned my addr 172.21.15.46:0/303497502 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.875+0000 7fbbee214700 1 --2- 172.21.15.46:0/303497502 >> v2:172.21.15.132:3300/0 conn(0x7fbbe8069410 0x7fbbe815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbeea15700 1 -- 172.21.15.46:0/303497502 >> v2:172.21.15.132:3300/0 conn(0x7fbbe8069410 msgr2=0x7fbbe815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbeea15700 1 --2- 172.21.15.46:0/303497502 >> v2:172.21.15.132:3300/0 conn(0x7fbbe8069410 0x7fbbe815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbeea15700 1 -- 172.21.15.46:0/303497502 >> v1:172.21.15.46:6789/0 conn(0x7fbbe80697e0 legacy=0x7fbbe8158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbeea15700 1 -- 172.21.15.46:0/303497502 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe815d750 con 0x7fbbe8159420 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbeea15700 1 --2- 172.21.15.46:0/303497502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe815a350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80cdd90 tx=0x7fbbe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4192dd091bc4968 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbed212700 1 -- 172.21.15.46:0/303497502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000e070 con 0x7fbbe8159420 2023-10-06T11:30:49.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbed212700 1 -- 172.21.15.46:0/303497502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe000ba20 con 0x7fbbe8159420 2023-10-06T11:30:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbed212700 1 -- 172.21.15.46:0/303497502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00092e0 con 0x7fbbe8159420 2023-10-06T11:30:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 -- 172.21.15.46:0/303497502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 msgr2=0x7fbbe815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 --2- 172.21.15.46:0/303497502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe815a350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80cdd90 tx=0x7fbbe0009840 comp rx=0 tx=0).stop 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 -- 172.21.15.46:0/303497502 shutdown_connections 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 --2- 172.21.15.46:0/303497502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe815a350 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 --2- 172.21.15.46:0/303497502 >> v2:172.21.15.132:3300/0 conn(0x7fbbe8069410 0x7fbbe815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 -- 172.21.15.46:0/303497502 >> 172.21.15.46:0/303497502 conn(0x7fbbe80d0210 msgr2=0x7fbbe80d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 -- 172.21.15.46:0/303497502 shutdown_connections 2023-10-06T11:30:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.879+0000 7fbbf0478700 1 -- 172.21.15.46:0/303497502 wait complete. 2023-10-06T11:30:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 Processor -- start 2023-10-06T11:30:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 -- start start 2023-10-06T11:30:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe8069410 0x7fbbe80c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbe80697e0 0x7fbbe80c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8155f00 con 0x7fbbe8159420 2023-10-06T11:30:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c5640 con 0x7fbbe8069410 2023-10-06T11:30:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbf0478700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c5920 con 0x7fbbe80697e0 2023-10-06T11:30:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbeea15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbeea15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58196/0 (socket says 172.21.15.46:58196) 2023-10-06T11:30:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbeea15700 1 -- 172.21.15.46:0/3113889753 learned_addr learned my addr 172.21.15.46:0/3113889753 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbee214700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe8069410 0x7fbbe80c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.883+0000 7fbbeda13700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbe80697e0 0x7fbbe80c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 -- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbe80697e0 msgr2=0x7fbbe80c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbe80697e0 0x7fbbe80c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 -- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe8069410 msgr2=0x7fbbe80c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe8069410 0x7fbbe80c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 -- 172.21.15.46:0/3113889753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009ae0 con 0x7fbbe8159420 2023-10-06T11:30:49.896 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fd0b75c7000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:30:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbeea15700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009880 tx=0x7fbbe0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0008b30 con 0x7fbbe8159420 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0008c90 con 0x7fbbe8159420 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe80c5c00 con 0x7fbbe8159420 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00266b0 con 0x7fbbe8159420 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe80affc0 con 0x7fbbe8159420 2023-10-06T11:30:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.887+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe80cc4f0 con 0x7fbbe8159420 2023-10-06T11:30:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.891+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7fbbe0003040 con 0x7fbbe8159420 2023-10-06T11:30:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.891+0000 7fbbde7fc700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 0x7fbbcc062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.891+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7fbbe007f090 con 0x7fbbe8159420 2023-10-06T11:30:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.891+0000 7fbbee214700 1 -- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 msgr2=0x7fbbcc062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:49.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.891+0000 7fbbee214700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 0x7fbbcc062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:49.899+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbbe80cc4f0 con 0x7fbbe8159420 2023-10-06T11:30:50.033 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.030+0000 7f25783ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:30:50.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.067+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbe80cc4f0 con 0x7fbbe8159420 2023-10-06T11:30:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.067+0000 7fbbde7fc700 1 -- 172.21.15.46:0/3113889753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbbe80cc4f0 con 0x7fbbe8159420 2023-10-06T11:30:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:30:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 msgr2=0x7fbbcc062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 0x7fbbcc062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 msgr2=0x7fbbe81559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009880 tx=0x7fbbe0018000 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 shutdown_connections 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbbcc060730 0x7fbbcc062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbe8159420 0x7fbbe81559c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbe80697e0 0x7fbbe80c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 --2- 172.21.15.46:0/3113889753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe8069410 0x7fbbe80c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 >> 172.21.15.46:0/3113889753 conn(0x7fbbe80d0210 msgr2=0x7fbbe815a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 shutdown_connections 2023-10-06T11:30:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.071+0000 7fbbf0478700 1 -- 172.21.15.46:0/3113889753 wait complete. 2023-10-06T11:30:50.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:30:50.240 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.238+0000 7f25783ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:30:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.251+0000 7efda9723700 1 Processor -- start 2023-10-06T11:30:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.251+0000 7efda9723700 1 -- start start 2023-10-06T11:30:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda9723700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efda40aec70 0x7efda40ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda9723700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda40ccfd0 con 0x7efda40ae2d0 2023-10-06T11:30:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda9723700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda40633b0 con 0x7efda406f1f0 2023-10-06T11:30:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda9723700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40cd460 con 0x7efda40aec70 2023-10-06T11:30:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda3fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efda40ae2d0 0x7efda4067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53974/0 (socket says 172.21.15.46:53974) 2023-10-06T11:30:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda3fff700 1 -- 172.21.15.46:0/2992560116 learned_addr learned my addr 172.21.15.46:0/2992560116 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efd9bfff700 1 --2- 172.21.15.46:0/2992560116 >> v2:172.21.15.132:3300/0 conn(0x7efda40aec70 0x7efda40ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efda37fe700 1 -- 172.21.15.46:0/2992560116 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039202876 0 0) 0x7efda40ccfd0 con 0x7efda40ae2d0 2023-10-06T11:30:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efd9bfff700 1 -- 172.21.15.46:0/2992560116 >> v1:172.21.15.132:6789/0 conn(0x7efda406f1f0 legacy=0x7efda4067760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efd9bfff700 1 -- 172.21.15.46:0/2992560116 >> v1:172.21.15.46:6789/0 conn(0x7efda40ae2d0 legacy=0x7efda4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.255+0000 7efd9bfff700 1 -- 172.21.15.46:0/2992560116 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda4067e70 con 0x7efda40aec70 2023-10-06T11:30:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efd9bfff700 1 --2- 172.21.15.46:0/2992560116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda40aec70 0x7efda40ae6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009910 tx=0x7efd8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42384719f0e1975 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda37fe700 1 -- 172.21.15.46:0/2992560116 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00b820 con 0x7efda40aec70 2023-10-06T11:30:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda37fe700 1 -- 172.21.15.46:0/2992560116 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c00b980 con 0x7efda40aec70 2023-10-06T11:30:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda37fe700 1 -- 172.21.15.46:0/2992560116 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00bd20 con 0x7efda40aec70 2023-10-06T11:30:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 -- 172.21.15.46:0/2992560116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda40aec70 msgr2=0x7efda40ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 --2- 172.21.15.46:0/2992560116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda40aec70 0x7efda40ae6a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009910 tx=0x7efd8c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 -- 172.21.15.46:0/2992560116 shutdown_connections 2023-10-06T11:30:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 --2- 172.21.15.46:0/2992560116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda40aec70 0x7efda40ae6a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 -- 172.21.15.46:0/2992560116 >> 172.21.15.46:0/2992560116 conn(0x7efda40afd40 msgr2=0x7efda406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 -- 172.21.15.46:0/2992560116 shutdown_connections 2023-10-06T11:30:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.259+0000 7efda9723700 1 -- 172.21.15.46:0/2992560116 wait complete. 2023-10-06T11:30:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 Processor -- start 2023-10-06T11:30:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 -- start start 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 0x7efda40ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efda40ae2d0 0x7efda4067280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40a9ee0 con 0x7efda40aec70 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.263+0000 7efda9723700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40aa1c0 con 0x7efda406f1f0 2023-10-06T11:30:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda9723700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40aa4a0 con 0x7efda40ae2d0 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58208/0 (socket says 172.21.15.46:58208) 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 -- 172.21.15.46:0/1274195319 learned_addr learned my addr 172.21.15.46:0/1274195319 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efd9bfff700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efda40ae2d0 0x7efda4067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda3fff700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 0x7efda40ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efda40ae2d0 msgr2=0x7efda4067280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efda40ae2d0 0x7efda4067280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 msgr2=0x7efda40ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 0x7efda40ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 -- 172.21.15.46:0/1274195319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c009420 con 0x7efda40aec70 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda3fff700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 0x7efda40ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda8f22700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c840 tx=0x7efd9c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012630 con 0x7efda40aec70 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c0127b0 con 0x7efda40aec70 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda40aa780 con 0x7efda40aec70 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012b70 con 0x7efda40aec70 2023-10-06T11:30:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda40aad40 con 0x7efda40aec70 2023-10-06T11:30:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.267+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda4064590 con 0x7efda40aec70 2023-10-06T11:30:50.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.271+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7efd9c003030 con 0x7efda40aec70 2023-10-06T11:30:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.271+0000 7efda0ff9700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 0x7efd84062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.271+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7efd9c07d0e0 con 0x7efda40aec70 2023-10-06T11:30:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.271+0000 7efda3fff700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 msgr2=0x7efd84062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:50.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.271+0000 7efda3fff700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 0x7efd84062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:50.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.275+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efd9c050800 con 0x7efda40aec70 2023-10-06T11:30:50.287 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:50.279+0000 7fd0b75c7000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:30:50.330 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.330+0000 7f25783ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:30:50.405 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.402+0000 7f25783ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:30:50.477 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.474+0000 7f25783ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:30:50.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.475+0000 7efda3fff700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 msgr2=0x7efd84062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:50.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.475+0000 7efda3fff700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 0x7efd84062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:30:50.644 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:50.635+0000 7fd0b75c7000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:30:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.675+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda404b5c0 con 0x7efda40aec70 2023-10-06T11:30:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.675+0000 7efda0ff9700 1 -- 172.21.15.46:0/1274195319 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7efd9c050390 con 0x7efda40aec70 2023-10-06T11:30:50.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:50.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:30:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 msgr2=0x7efd84062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 0x7efd84062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 msgr2=0x7efda40c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00c840 tx=0x7efd9c001670 comp rx=0 tx=0).stop 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 shutdown_connections 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd84060730 0x7efd84062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda40aec70 0x7efda40c7a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efda40ae2d0 0x7efda4067280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 --2- 172.21.15.46:0/1274195319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda406f1f0 0x7efda40ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 >> 172.21.15.46:0/1274195319 conn(0x7efda40afd40 msgr2=0x7efda4159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 shutdown_connections 2023-10-06T11:30:50.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.679+0000 7efda9723700 1 -- 172.21.15.46:0/1274195319 wait complete. 2023-10-06T11:30:50.691 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:30:50.693 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.690+0000 7f25783ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:30:50.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:30:50.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:50.751+0000 7fd0b75c7000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:30:50.767 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.766+0000 7f25783ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:30:50.841 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:50.831+0000 7fd0b75c7000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:30:50.847 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:50.846+0000 7f25783ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:30:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 Processor -- start 2023-10-06T11:30:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 -- start start 2023-10-06T11:30:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefe80af000 0x7fefe80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80cb4c0 0x7fefe80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe80cf100 con 0x7fefe80af3d0 2023-10-06T11:30:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe80ce4f0 con 0x7fefe80af000 2023-10-06T11:30:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.855+0000 7fefe759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80ce7d0 con 0x7fefe80cb4c0 2023-10-06T11:30:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefe80af000 0x7fefe80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefe80af000 0x7fefe80cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58210/0 (socket says 172.21.15.46:58210) 2023-10-06T11:30:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 -- 172.21.15.46:0/2959927765 learned_addr learned my addr 172.21.15.46:0/2959927765 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe5d9b700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80cb4c0 0x7fefe80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 -- 172.21.15.46:0/2959927765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80cb4c0 msgr2=0x7fefe80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80cb4c0 0x7fefe80cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 -- 172.21.15.46:0/2959927765 >> v1:172.21.15.132:6789/0 conn(0x7fefe80af3d0 legacy=0x7fefe80690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 -- 172.21.15.46:0/2959927765 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe80ceab0 con 0x7fefe80af000 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe659c700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80af000 0x7fefe80cf590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fefd80099b0 tx=0x7fefd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bab27a314c27dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe559a700 1 -- 172.21.15.46:0/2959927765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8009ea0 con 0x7fefe80af000 2023-10-06T11:30:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe559a700 1 -- 172.21.15.46:0/2959927765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800b860 con 0x7fefe80af000 2023-10-06T11:30:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe559a700 1 -- 172.21.15.46:0/2959927765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800bc00 con 0x7fefe80af000 2023-10-06T11:30:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe759e700 1 -- 172.21.15.46:0/2959927765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80af000 msgr2=0x7fefe80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.859+0000 7fefe759e700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80af000 0x7fefe80cf590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fefd80099b0 tx=0x7fefd8009250 comp rx=0 tx=0).stop 2023-10-06T11:30:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 -- 172.21.15.46:0/2959927765 shutdown_connections 2023-10-06T11:30:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80cb4c0 0x7fefe80cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 --2- 172.21.15.46:0/2959927765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80af000 0x7fefe80cf590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 -- 172.21.15.46:0/2959927765 >> 172.21.15.46:0/2959927765 conn(0x7fefe80683b0 msgr2=0x7fefe80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 -- 172.21.15.46:0/2959927765 shutdown_connections 2023-10-06T11:30:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 -- 172.21.15.46:0/2959927765 wait complete. 2023-10-06T11:30:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 Processor -- start 2023-10-06T11:30:50.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.863+0000 7fefe759e700 1 -- start start 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefe80af000 0x7fefe80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 0x7fefe816a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b5cc0 con 0x7fefe80cb4c0 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b3ea0 con 0x7fefe80af000 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b4180 con 0x7fefe80af3d0 2023-10-06T11:30:50.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 0x7fefe816a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe5d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe5d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38964/0 (socket says 172.21.15.46:38964) 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe5d9b700 1 -- 172.21.15.46:0/76534075 learned_addr learned my addr 172.21.15.46:0/76534075 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe659c700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefe80af000 0x7fefe80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 -- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 msgr2=0x7fefe8169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 -- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefe80af000 msgr2=0x7fefe80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefe80af000 0x7fefe80b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 -- 172.21.15.46:0/76534075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80094f0 con 0x7fefe80cb4c0 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe5d9b700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe6d9d700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 0x7fefe816a0e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c830 tx=0x7fefe0001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0012630 con 0x7fefe80cb4c0 2023-10-06T11:30:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0012790 con 0x7fefe80cb4c0 2023-10-06T11:30:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0012b50 con 0x7fefe80cb4c0 2023-10-06T11:30:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe80b4460 con 0x7fefe80cb4c0 2023-10-06T11:30:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.867+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe80b2f90 con 0x7fefe80cb4c0 2023-10-06T11:30:50.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.871+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe8064590 con 0x7fefe80cb4c0 2023-10-06T11:30:50.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.871+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7fefe0011380 con 0x7fefe80cb4c0 2023-10-06T11:30:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.871+0000 7fefd67fc700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 0x7fefc8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.875+0000 7fefe659c700 1 -- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 msgr2=0x7fefc8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.875+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7fefe007d330 con 0x7fefe80cb4c0 2023-10-06T11:30:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.875+0000 7fefe659c700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 0x7fefc8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:50.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:50.879+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fefe004cfc0 con 0x7fefe80cb4c0 2023-10-06T11:30:50.928 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:50.919+0000 7fd0b75c7000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:30:51.031 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:51.030+0000 7f25783ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:30:51.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.051+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefe80ced90 con 0x7fefe80cb4c0 2023-10-06T11:30:51.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.051+0000 7fefd67fc700 1 -- 172.21.15.46:0/76534075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18763 (secure 0 0 0) 0x7fefe00505e0 con 0x7fefe80cb4c0 2023-10-06T11:30:51.062 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:51.062 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:30:47.374792+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/32974":"2023-10-07T11:19:27.796424+0000","172.21.15.46:6832/32974":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/3406707913":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/2277270196":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/568326314":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/3506103189":"2023-10-07T11:19:27.796424+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":2},{"begin":188,"length":33}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":72,"length":1},{"begin":83,"length":1},{"begin":97,"length":1},{"begin":218,"length":3}]}],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":4,"length":1},{"begin":9,"length":2},{"begin":15,"length":2},{"begin":20,"length":1},{"begin":25,"length":1},{"begin":29,"length":2},{"begin":33,"length":1},{"begin":35,"length":1},{"begin":39,"length":1},{"begin":42,"length":1},{"begin":46,"length":1},{"begin":48,"length":1},{"begin":52,"length":1},{"begin":54,"length":1},{"begin":60,"length":1},{"begin":63,"length":2},{"begin":67,"length":1},{"begin":76,"length":1},{"begin":78,"length":1},{"begin":81,"length":2},{"begin":87,"length":1},{"begin":90,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":101,"length":1},{"begin":105,"length":3},{"begin":153,"length":35}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.055+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 msgr2=0x7fefc8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.055+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 0x7fefc8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.055+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 msgr2=0x7fefe816a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.055+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 0x7fefe816a0e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c830 tx=0x7fefe0001690 comp rx=0 tx=0).stop 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 shutdown_connections 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fefc8060700 0x7fefc8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefe80cb4c0 0x7fefe816a0e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefe80af3d0 0x7fefe8169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 --2- 172.21.15.46:0/76534075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefe80af000 0x7fefe80b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 >> 172.21.15.46:0/76534075 conn(0x7fefe80683b0 msgr2=0x7fefe80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 shutdown_connections 2023-10-06T11:30:51.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.059+0000 7fefe759e700 1 -- 172.21.15.46:0/76534075 wait complete. 2023-10-06T11:30:51.088 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:30:51.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:30:51.113 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:51.110+0000 7f25783ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:30:51.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:51.135+0000 7fd0b75c7000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:30:51.226 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:51.215+0000 7fd0b75c7000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:30:51.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.271+0000 7f905966e700 1 Processor -- start 2023-10-06T11:30:51.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.271+0000 7f905966e700 1 -- start start 2023-10-06T11:30:51.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9054156c80 0x7f90540ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90540c7b50 0x7f90540c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f905415cde0 0x7f905415d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540c8500 con 0x7f9054156c80 2023-10-06T11:30:51.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540c6470 con 0x7f905415cde0 2023-10-06T11:30:51.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f905966e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540c7760 con 0x7f90540c7b50 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9054156c80 0x7f90540ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9054156c80 0x7f90540ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58220/0 (socket says 172.21.15.46:58220) 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 -- 172.21.15.46:0/355683805 learned_addr learned my addr 172.21.15.46:0/355683805 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f904a7fc700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90540c7b50 0x7f90540c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f90537fe700 1 --2- 172.21.15.46:0/355683805 >> v2:172.21.15.132:3300/0 conn(0x7f905415cde0 0x7f905415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 -- 172.21.15.46:0/355683805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90540c7b50 msgr2=0x7f90540c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90540c7b50 0x7f90540c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 -- 172.21.15.46:0/355683805 >> v2:172.21.15.132:3300/0 conn(0x7f905415cde0 msgr2=0x7f905415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 --2- 172.21.15.46:0/355683805 >> v2:172.21.15.132:3300/0 conn(0x7f905415cde0 0x7f905415d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 -- 172.21.15.46:0/355683805 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540ca710 con 0x7f9054156c80 2023-10-06T11:30:51.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f9052ffd700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540ca1d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f905406bf00 tx=0x7f9044009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41edbef0376f4b99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:51.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.275+0000 7f90527fc700 1 -- 172.21.15.46:0/355683805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044005c10 con 0x7f9054156c80 2023-10-06T11:30:51.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f90527fc700 1 -- 172.21.15.46:0/355683805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9044005d70 con 0x7f9054156c80 2023-10-06T11:30:51.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f90527fc700 1 -- 172.21.15.46:0/355683805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044006260 con 0x7f9054156c80 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- 172.21.15.46:0/355683805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 msgr2=0x7f90540ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540ca1d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f905406bf00 tx=0x7f9044009220 comp rx=0 tx=0).stop 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- 172.21.15.46:0/355683805 shutdown_connections 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 --2- 172.21.15.46:0/355683805 >> v2:172.21.15.132:3300/0 conn(0x7f905415cde0 0x7f905415d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90540c7b50 0x7f90540c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 --2- 172.21.15.46:0/355683805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540ca1d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- 172.21.15.46:0/355683805 >> 172.21.15.46:0/355683805 conn(0x7f90540cbe30 msgr2=0x7f90540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- 172.21.15.46:0/355683805 shutdown_connections 2023-10-06T11:30:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- 172.21.15.46:0/355683805 wait complete. 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 Processor -- start 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.279+0000 7f905966e700 1 -- start start 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90540c7b50 0x7f9054155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 0x7f9054150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540ac9c0 con 0x7f9054156c80 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90540ad6a0 con 0x7f90540c7b50 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540acd70 con 0x7f905415cde0 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58234/0 (socket says 172.21.15.46:58234) 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 -- 172.21.15.46:0/2397803425 learned_addr learned my addr 172.21.15.46:0/2397803425 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f9052ffd700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90540c7b50 0x7f9054155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f90537fe700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 0x7f9054150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 msgr2=0x7f9054150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 0x7f9054150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90540c7b50 msgr2=0x7f9054155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90540c7b50 0x7f9054155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 -- 172.21.15.46:0/2397803425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044009480 con 0x7f9054156c80 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f90537fe700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 0x7f9054150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904a7fc700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f903c00e930 tx=0x7f903c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00eed0 con 0x7f9054156c80 2023-10-06T11:30:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c00c850 con 0x7f9054156c80 2023-10-06T11:30:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00cbf0 con 0x7f9054156c80 2023-10-06T11:30:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90540ad050 con 0x7f9054156c80 2023-10-06T11:30:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9054151320 con 0x7f9054156c80 2023-10-06T11:30:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.283+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90540ad850 con 0x7f9054156c80 2023-10-06T11:30:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.287+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7f903c013470 con 0x7f9054156c80 2023-10-06T11:30:51.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.287+0000 7f904b7fe700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 0x7f9034062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.287+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7309+0+0 (secure 0 0 0) 0x7f903c07d400 con 0x7f9054156c80 2023-10-06T11:30:51.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.287+0000 7f9052ffd700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 msgr2=0x7f9034062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:51.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.287+0000 7f9052ffd700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 0x7f9034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.295+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f90540ad850 con 0x7f9054156c80 2023-10-06T11:30:51.311 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:51.303+0000 7fd0b75c7000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:30:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.491+0000 7f9052ffd700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 msgr2=0x7f9034062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.491+0000 7f9052ffd700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 0x7f9034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:30:51.511 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:51.503+0000 7fd0b75c7000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:30:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.503+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90540ad810 con 0x7f9054156c80 2023-10-06T11:30:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f904b7fe700 1 -- 172.21.15.46:0/2397803425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f903c04d090 con 0x7f9054156c80 2023-10-06T11:30:51.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:51.516 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:30:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 msgr2=0x7f9034062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 0x7f9034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 msgr2=0x7f90540b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f903c00e930 tx=0x7f903c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 shutdown_connections 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9034060700 0x7f9034062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f905415cde0 0x7f9054150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9054156c80 0x7f90540b1590 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 --2- 172.21.15.46:0/2397803425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90540c7b50 0x7f9054155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.507+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 >> 172.21.15.46:0/2397803425 conn(0x7f90540cbe30 msgr2=0x7f905415d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.511+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 shutdown_connections 2023-10-06T11:30:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.511+0000 7f905966e700 1 -- 172.21.15.46:0/2397803425 wait complete. 2023-10-06T11:30:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:30:51.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:30:51.599 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:51.591+0000 7fd0b75c7000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:30:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 Processor -- start 2023-10-06T11:30:51.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 -- start start 2023-10-06T11:30:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efff0156c80 0x7efff00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff00c6470 con 0x7efff00c7b50 2023-10-06T11:30:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff00c7760 con 0x7efff0156c80 2023-10-06T11:30:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.719+0000 7efff5225700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff00ca710 con 0x7efff00c7fc0 2023-10-06T11:30:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38988/0 (socket says 172.21.15.46:38988) 2023-10-06T11:30:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 -- 172.21.15.46:0/2206095856 learned_addr learned my addr 172.21.15.46:0/2206095856 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:51.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effeed9d700 1 --2- 172.21.15.46:0/2206095856 >> v2:172.21.15.46:3300/0 conn(0x7efff0156c80 0x7efff00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 -- 172.21.15.46:0/2206095856 >> v2:172.21.15.46:3300/0 conn(0x7efff0156c80 msgr2=0x7efff00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 --2- 172.21.15.46:0/2206095856 >> v2:172.21.15.46:3300/0 conn(0x7efff0156c80 0x7efff00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 -- 172.21.15.46:0/2206095856 >> v1:172.21.15.132:6789/0 conn(0x7efff00c7b50 legacy=0x7efff015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 -- 172.21.15.46:0/2206095856 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00c66c0 con 0x7efff00c7fc0 2023-10-06T11:30:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effef59e700 1 --2- 172.21.15.46:0/2206095856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7effe8009f20 tx=0x7effe8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc8c6a4cfb3b020 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:51.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effedd9b700 1 -- 172.21.15.46:0/2206095856 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800e070 con 0x7efff00c7fc0 2023-10-06T11:30:51.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effedd9b700 1 -- 172.21.15.46:0/2206095856 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe800b9d0 con 0x7efff00c7fc0 2023-10-06T11:30:51.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7effedd9b700 1 -- 172.21.15.46:0/2206095856 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe80092e0 con 0x7efff00c7fc0 2023-10-06T11:30:51.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 -- 172.21.15.46:0/2206095856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 msgr2=0x7efff015d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 --2- 172.21.15.46:0/2206095856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7effe8009f20 tx=0x7effe8009840 comp rx=0 tx=0).stop 2023-10-06T11:30:51.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 -- 172.21.15.46:0/2206095856 shutdown_connections 2023-10-06T11:30:51.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 --2- 172.21.15.46:0/2206095856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7fc0 0x7efff015d4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 --2- 172.21.15.46:0/2206095856 >> v2:172.21.15.46:3300/0 conn(0x7efff0156c80 0x7efff00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 -- 172.21.15.46:0/2206095856 >> 172.21.15.46:0/2206095856 conn(0x7efff00cbe30 msgr2=0x7efff00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 -- 172.21.15.46:0/2206095856 shutdown_connections 2023-10-06T11:30:51.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.723+0000 7efff5225700 1 -- 172.21.15.46:0/2206095856 wait complete. 2023-10-06T11:30:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 Processor -- start 2023-10-06T11:30:51.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 -- start start 2023-10-06T11:30:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7b50 0x7efff0158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efff0156c80 0x7efff01686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff0159610 con 0x7efff00c7fc0 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff0169f00 con 0x7efff0156c80 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.727+0000 7efff5225700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff00adc60 con 0x7efff00c7b50 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effeed9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7b50 0x7efff0158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58258/0 (socket says 172.21.15.46:58258) 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 -- 172.21.15.46:0/1292828551 learned_addr learned my addr 172.21.15.46:0/1292828551 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:51.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effef59e700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efff0156c80 0x7efff01686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7b50 msgr2=0x7efff0158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7b50 0x7efff0158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efff0156c80 msgr2=0x7efff01686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efff0156c80 0x7efff01686c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 -- 172.21.15.46:0/1292828551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8009ae0 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effee59c700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7effe400c4e0 tx=0x7effe400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4011590 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4011710 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff00ac270 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4011ad0 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff00adf00 con 0x7efff00c7fc0 2023-10-06T11:30:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.731+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff0063f90 con 0x7efff00c7fc0 2023-10-06T11:30:51.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.735+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7effe4001020 con 0x7efff00c7fc0 2023-10-06T11:30:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.735+0000 7effd6ffd700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 0x7effdc062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.735+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6833+0+0 (secure 0 0 0) 0x7effe407c320 con 0x7efff00c7fc0 2023-10-06T11:30:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.735+0000 7effeed9d700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 msgr2=0x7effdc062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.735+0000 7effeed9d700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 0x7effdc062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.739+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7effe404c180 con 0x7efff00c7fc0 2023-10-06T11:30:51.821 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:30:51.818+0000 7f25783ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:30:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.939+0000 7effeed9d700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 msgr2=0x7effdc062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.939+0000 7effeed9d700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 0x7effdc062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:30:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.939+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efff004b5c0 con 0x7efff00c7fc0 2023-10-06T11:30:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.939+0000 7effd6ffd700 1 -- 172.21.15.46:0/1292828551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4603 (secure 0 0 0) 0x7effe404f7a0 con 0x7efff00c7fc0 2023-10-06T11:30:51.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:51.949 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:19:13.894505+0000","modified":"2023-10-06T11:19:17.491302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6837/1323459348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1323459348},{"type":"v1","addr":"172.21.15.46:6837","nonce":1323459348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 msgr2=0x7effdc062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 0x7effdc062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 msgr2=0x7efff01699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7effe400c4e0 tx=0x7effe400c7f0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 shutdown_connections 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7effdc060700 0x7effdc062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efff0156c80 0x7efff01686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efff00c7fc0 0x7efff01699c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 --2- 172.21.15.46:0/1292828551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efff00c7b50 0x7efff0158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 >> 172.21.15.46:0/1292828551 conn(0x7efff00cbe30 msgr2=0x7efff00c1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 shutdown_connections 2023-10-06T11:30:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:51.943+0000 7efff5225700 1 -- 172.21.15.46:0/1292828551 wait complete. 2023-10-06T11:30:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2023-10-06T11:30:51.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:30:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 Processor -- start 2023-10-06T11:30:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 -- start start 2023-10-06T11:30:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe97c15ab60 0x7fe97c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c06d460 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c0c7000 con 0x7fe97c15af30 2023-10-06T11:30:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe9842f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c0c72c0 con 0x7fe97c15ab60 2023-10-06T11:30:52.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe98288d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe97c0ad6c0 0x7fe97c0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53982/0 (socket says 172.21.15.46:53982) 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.179+0000 7fe98288d700 1 -- 172.21.15.46:0/4241581079 learned_addr learned my addr 172.21.15.46:0/4241581079 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98208c700 1 --2- 172.21.15.46:0/4241581079 >> v2:172.21.15.132:3300/0 conn(0x7fe97c15ab60 0x7fe97c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60176315 0 0) 0x7fe97c06d460 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe968003570 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470672132 0 0) 0x7fe97c0c7000 con 0x7fe97c15af30 2023-10-06T11:30:52.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe97c06d460 con 0x7fe97c15af30 2023-10-06T11:30:52.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98208c700 1 -- 172.21.15.46:0/4241581079 >> v1:172.21.15.132:6789/0 conn(0x7fe97c15af30 legacy=0x7fe97c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966090340 0 0) 0x7fe968003570 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98208c700 1 -- 172.21.15.46:0/4241581079 >> v1:172.21.15.46:6789/0 conn(0x7fe97c0ad6c0 legacy=0x7fe97c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98208c700 1 -- 172.21.15.46:0/4241581079 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0c75a0 con 0x7fe97c15ab60 2023-10-06T11:30:52.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98208c700 1 --2- 172.21.15.46:0/4241581079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c06e8a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0c71f0 tx=0x7fe978009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edd36517ebfc625 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:52.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740017f0 con 0x7fe97c15ab60 2023-10-06T11:30:52.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978005a00 con 0x7fe97c15ab60 2023-10-06T11:30:52.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe98108a700 1 -- 172.21.15.46:0/4241581079 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978005d80 con 0x7fe97c15ab60 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe9842f0700 1 -- 172.21.15.46:0/4241581079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 msgr2=0x7fe97c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe9842f0700 1 --2- 172.21.15.46:0/4241581079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c06e8a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0c71f0 tx=0x7fe978009120 comp rx=0 tx=0).stop 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe9842f0700 1 -- 172.21.15.46:0/4241581079 shutdown_connections 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe9842f0700 1 --2- 172.21.15.46:0/4241581079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c06e8a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.183+0000 7fe9842f0700 1 -- 172.21.15.46:0/4241581079 >> 172.21.15.46:0/4241581079 conn(0x7fe97c0cbe30 msgr2=0x7fe97c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.187+0000 7fe9842f0700 1 -- 172.21.15.46:0/4241581079 shutdown_connections 2023-10-06T11:30:52.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.187+0000 7fe9842f0700 1 -- 172.21.15.46:0/4241581079 wait complete. 2023-10-06T11:30:52.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.187+0000 7fe9842f0700 1 Processor -- start 2023-10-06T11:30:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.187+0000 7fe9842f0700 1 -- start start 2023-10-06T11:30:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe97c15af30 0x7fe97c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c150da0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c0b04d0 con 0x7fe97c15ab60 2023-10-06T11:30:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c15f420 con 0x7fe97c15af30 2023-10-06T11:30:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58266/0 (socket says 172.21.15.46:58266) 2023-10-06T11:30:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 -- 172.21.15.46:0/1652078736 learned_addr learned my addr 172.21.15.46:0/1652078736 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98288d700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe97c15af30 0x7fe97c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98188b700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 -- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe97c15af30 msgr2=0x7fe97c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe97c15af30 0x7fe97c0b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 -- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 msgr2=0x7fe97c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c14fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 -- 172.21.15.46:0/1652078736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780093c0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98188b700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe98208c700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe978003080 tx=0x7fe978015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780117b0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978011930 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c0afec0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011cf0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.191+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c0b0160 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.195+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c063f90 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.195+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 80130+0+0 (secure 0 0 0) 0x7fe97801d640 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.195+0000 7fe9727fc700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 0x7fe960062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.195+0000 7fe98188b700 1 -- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 msgr2=0x7fe960062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:30:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.195+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fe97807ebd0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.199+0000 7fe98188b700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 0x7fe960062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:30:52.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.203+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe97804ea30 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.357 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.347+0000 7fd0b75c7000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:30:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.367+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 14) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fe97804ee00 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.367+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 msgr2=0x7fe960062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:30:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.367+0000 7fe9727fc700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 0x7fe960062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.383+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe97c0c69e0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.387+0000 7fe9727fc700 1 -- 172.21.15.46:0/1652078736 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18410 (secure 0 0 0) 0x7fe978054bd0 con 0x7fe97c0ad6c0 2023-10-06T11:30:52.396 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:52.397 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:30:52.353782+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:19:11.591550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:19:12.896650+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/3334240510":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/2687662961":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/3207803962":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/3281805880":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/1116752397":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/3502847244":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/1751428573":"2023-10-07T11:30:52.353701+0000","172.21.15.46:0/3506103189":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/568326314":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/2277270196":"2023-10-07T11:19:27.796424+0000","172.21.15.46:0/3406707913":"2023-10-07T11:19:27.796424+0000","172.21.15.46:6832/32974":"2023-10-07T11:19:27.796424+0000","172.21.15.46:6833/32974":"2023-10-07T11:19:27.796424+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:30:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 msgr2=0x7fe97c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe978003080 tx=0x7fe978015000 comp rx=0 tx=0).stop 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 shutdown_connections 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe960060700 0x7fe960062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe97c15af30 0x7fe97c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe97c15ab60 0x7fe97c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 --2- 172.21.15.46:0/1652078736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe97c0ad6c0 0x7fe97c15ea30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 >> 172.21.15.46:0/1652078736 conn(0x7fe97c0cbe30 msgr2=0x7fe97c0c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 shutdown_connections 2023-10-06T11:30:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.391+0000 7fe9842f0700 1 -- 172.21.15.46:0/1652078736 wait complete. 2023-10-06T11:30:52.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:30:52.476 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd08319b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.476 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd08319b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.476 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd08319b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.476 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd08319b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.476 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd08319b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd07f193700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd07f193700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd07f193700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd07f193700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:30:52.467+0000 7fd07f193700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:30:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.575+0000 7f4cbb3c5700 1 Processor -- start 2023-10-06T11:30:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.575+0000 7f4cbb3c5700 1 -- start start 2023-10-06T11:30:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cb415ab60 0x7f4cb406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cb415af30 0x7f4cb406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb406d460 con 0x7f4cb415ab60 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb40c7000 con 0x7f4cb415af30 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40c72c0 con 0x7f4cb40ad6c0 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cb415ab60 0x7f4cb406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cb415ab60 0x7f4cb406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58384/0 (socket says 172.21.15.46:58384) 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 -- 172.21.15.46:0/1373016657 learned_addr learned my addr 172.21.15.46:0/1373016657 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 -- 172.21.15.46:0/1373016657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 msgr2=0x7f4cb40adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 --2- 172.21.15.46:0/1373016657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 -- 172.21.15.46:0/1373016657 >> v2:172.21.15.132:3300/0 conn(0x7f4cb415af30 msgr2=0x7f4cb406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 --2- 172.21.15.46:0/1373016657 >> v2:172.21.15.132:3300/0 conn(0x7f4cb415af30 0x7f4cb406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 -- 172.21.15.46:0/1373016657 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40c75a0 con 0x7f4cb415ab60 2023-10-06T11:30:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb9161700 1 --2- 172.21.15.46:0/1373016657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb406e8a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4cb406f0b0 tx=0x7f4ca4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecc8a5151eb878e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb3fff700 1 -- 172.21.15.46:0/1373016657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4005c10 con 0x7f4cb415ab60 2023-10-06T11:30:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb3fff700 1 -- 172.21.15.46:0/1373016657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4005d70 con 0x7f4cb415ab60 2023-10-06T11:30:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cb3fff700 1 -- 172.21.15.46:0/1373016657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400b880 con 0x7f4cb415ab60 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/1373016657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 msgr2=0x7f4cb406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/1373016657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb406e8a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4cb406f0b0 tx=0x7f4ca4009220 comp rx=0 tx=0).stop 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/1373016657 shutdown_connections 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/1373016657 >> v2:172.21.15.132:3300/0 conn(0x7f4cb415af30 0x7f4cb406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/1373016657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/1373016657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb406e8a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/1373016657 >> 172.21.15.46:0/1373016657 conn(0x7f4cb40cbe30 msgr2=0x7f4cb40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/1373016657 shutdown_connections 2023-10-06T11:30:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.579+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/1373016657 wait complete. 2023-10-06T11:30:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.583+0000 7f4cbb3c5700 1 Processor -- start 2023-10-06T11:30:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.583+0000 7f4cbb3c5700 1 -- start start 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 0x7f4cb41505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40aebd0 con 0x7f4cb415ab60 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40b1360 con 0x7f4cb415af30 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40b1640 con 0x7f4cb40ad6c0 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58388/0 (socket says 172.21.15.46:58388) 2023-10-06T11:30:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 -- 172.21.15.46:0/3583641766 learned_addr learned my addr 172.21.15.46:0/3583641766 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb9161700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb9962700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 0x7f4cb41505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 -- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 msgr2=0x7f4cb40a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 -- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 msgr2=0x7f4cb41505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 0x7f4cb41505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 -- 172.21.15.46:0/3583641766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca4009480 con 0x7f4cb415ab60 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb9962700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 0x7f4cb41505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb8960700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b810 tx=0x7f4ca80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800f570 con 0x7f4cb415ab60 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca800f6f0 con 0x7f4cb415ab60 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb40b1920 con 0x7f4cb415ab60 2023-10-06T11:30:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800fab0 con 0x7f4cb415ab60 2023-10-06T11:30:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.587+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb40a9490 con 0x7f4cb415ab60 2023-10-06T11:30:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.591+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb40a96b0 con 0x7f4cb415ab60 2023-10-06T11:30:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.591+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 14) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f4ca801a460 con 0x7f4cb415ab60 2023-10-06T11:30:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.595+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7090+0+0 (secure 0 0 0) 0x7f4ca807c680 con 0x7f4cb415ab60 2023-10-06T11:30:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.599+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4ca804ca50 con 0x7f4cb415ab60 2023-10-06T11:30:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.807+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4cb4063f90 con 0x7f4cb415ab60 2023-10-06T11:30:52.815 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:30:52.807+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:30:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.819+0000 7f4cb17fa700 1 -- 172.21.15.46:0/3583641766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4ca8050070 con 0x7f4cb415ab60 2023-10-06T11:30:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 msgr2=0x7f4cb40c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b810 tx=0x7f4ca80091d0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 shutdown_connections 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb415af30 0x7f4cb41505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cb415ab60 0x7f4cb40c4920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 --2- 172.21.15.46:0/3583641766 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cb40ad6c0 0x7f4cb40a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 >> 172.21.15.46:0/3583641766 conn(0x7f4cb40cbe30 msgr2=0x7f4cb406e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 shutdown_connections 2023-10-06T11:30:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.823+0000 7f4cbb3c5700 1 -- 172.21.15.46:0/3583641766 wait complete. 2023-10-06T11:30:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:30:52.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:30:52.894 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:30:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 Processor -- start 2023-10-06T11:30:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 -- start start 2023-10-06T11:30:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8146eb0 0x7fd4e80c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e80caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80c1980 con 0x7fd4e80c2350 2023-10-06T11:30:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80cb1e0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:52.999+0000 7fd4efaf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c0d70 con 0x7fd4e806d340 2023-10-06T11:30:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8146eb0 0x7fd4e80c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8146eb0 0x7fd4e80c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:41826/0 (socket says 172.21.15.46:41826) 2023-10-06T11:30:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 -- 172.21.15.46:0/760108625 learned_addr learned my addr 172.21.15.46:0/760108625 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed08b700 1 --1- 172.21.15.46:0/760108625 >> v1:172.21.15.46:6789/0 conn(0x7fd4e80c2350 0x7fd4e806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54002/0 (socket says 172.21.15.46:54002) 2023-10-06T11:30:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ee08d700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e80caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 -- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 msgr2=0x7fd4e80caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e80caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 -- 172.21.15.46:0/760108625 >> v1:172.21.15.46:6789/0 conn(0x7fd4e80c2350 legacy=0x7fd4e806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 -- 172.21.15.46:0/760108625 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80c1050 con 0x7fd4e8146eb0 2023-10-06T11:30:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ed88c700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c1e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40099b0 tx=0x7fd4e4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8defddacc756bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ec88a700 1 -- 172.21.15.46:0/760108625 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009ea0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ec88a700 1 -- 172.21.15.46:0/760108625 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400b860 con 0x7fd4e8146eb0 2023-10-06T11:30:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4ec88a700 1 -- 172.21.15.46:0/760108625 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400bc00 con 0x7fd4e8146eb0 2023-10-06T11:30:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 -- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8146eb0 msgr2=0x7fd4e80c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c1e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40099b0 tx=0x7fd4e4009250 comp rx=0 tx=0).stop 2023-10-06T11:30:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 -- 172.21.15.46:0/760108625 shutdown_connections 2023-10-06T11:30:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e80caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/760108625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c1e10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 -- 172.21.15.46:0/760108625 >> 172.21.15.46:0/760108625 conn(0x7fd4e80cbe30 msgr2=0x7fd4e80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 -- 172.21.15.46:0/760108625 shutdown_connections 2023-10-06T11:30:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.003+0000 7fd4efaf0700 1 -- 172.21.15.46:0/760108625 wait complete. 2023-10-06T11:30:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 Processor -- start 2023-10-06T11:30:53.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 -- start start 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e8171a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80c2350 0x7fd4e815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80aef40 con 0x7fd4e8146eb0 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8171f90 con 0x7fd4e80c2350 2023-10-06T11:30:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4efaf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e815d500 con 0x7fd4e806d340 2023-10-06T11:30:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4ee08d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4ee08d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58404/0 (socket says 172.21.15.46:58404) 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.007+0000 7fd4ee08d700 1 -- 172.21.15.46:0/444768417 learned_addr learned my addr 172.21.15.46:0/444768417 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ed88c700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e8171a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ed08b700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80c2350 0x7fd4e815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 -- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 msgr2=0x7fd4e8171a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e8171a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 -- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80c2350 msgr2=0x7fd4e815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80c2350 0x7fd4e815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 -- 172.21.15.46:0/444768417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40094f0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ee08d700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009ef0 tx=0x7fd4e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cc20 con 0x7fd4e8146eb0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000f040 con 0x7fd4e8146eb0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e8172210 con 0x7fd4e8146eb0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0014480 con 0x7fd4e8146eb0 2023-10-06T11:30:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e81727d0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.011+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80af0f0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.015+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 14) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fd4e00147f0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.015+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7176+0+0 (secure 0 0 0) 0x7fd4e007d5c0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.019+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd4e80af0f0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.227+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd4e80af0f0 con 0x7fd4e8146eb0 2023-10-06T11:30:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.403+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 15) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fd4e004d370 con 0x7fd4e8146eb0 2023-10-06T11:30:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.403+0000 7fd4ddffb700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 0x7fd4cc068d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.407+0000 7fd4ed88c700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 0x7fd4cc068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:53.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.407+0000 7fd4ed88c700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 0x7fd4cc068d20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40099b0 tx=0x7fd4e4017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.819+0000 7fd4ddffb700 1 -- 172.21.15.46:0/444768417 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd4e0068690 con 0x7fd4e8146eb0 2023-10-06T11:30:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 msgr2=0x7fd4cc068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 0x7fd4cc068d20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40099b0 tx=0x7fd4e4017000 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 msgr2=0x7fd4e80c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009ef0 tx=0x7fd4e000b040 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 shutdown_connections 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4cc066940 0x7fd4cc068d20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e8146eb0 0x7fd4e80c2ee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80c2350 0x7fd4e815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 --2- 172.21.15.46:0/444768417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e806d340 0x7fd4e8171a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 >> 172.21.15.46:0/444768417 conn(0x7fd4e80cbe30 msgr2=0x7fd4e80ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 shutdown_connections 2023-10-06T11:30:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:53.827+0000 7fd4efaf0700 1 -- 172.21.15.46:0/444768417 wait complete. 2023-10-06T11:30:53.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:30:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.075+0000 7f2170c6b700 1 Processor -- start 2023-10-06T11:30:54.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.075+0000 7f2170c6b700 1 -- start start 2023-10-06T11:30:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f2170c6b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f216c069770 0x7f216c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f2170c6b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0b5eb0 con 0x7f216c158de0 2023-10-06T11:30:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f2170c6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0cef30 con 0x7f216c1591b0 2023-10-06T11:30:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f2170c6b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0cf4f0 con 0x7f216c069770 2023-10-06T11:30:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f216c158de0 0x7f216c069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52862/0 (socket says 172.21.15.46:52862) 2023-10-06T11:30:54.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216b7fe700 1 -- 172.21.15.46:0/3354772433 learned_addr learned my addr 172.21.15.46:0/3354772433 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:54.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216affd700 1 --2- 172.21.15.46:0/3354772433 >> v2:172.21.15.132:3300/0 conn(0x7f216c069770 0x7f216c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002025228 0 0) 0x7f215c001110 con 0x7f216c158de0 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154003570 con 0x7f216c158de0 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809028691 0 0) 0x7f216c0cef30 con 0x7f216c1591b0 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f215c001110 con 0x7f216c1591b0 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216affd700 1 -- 172.21.15.46:0/3354772433 >> v1:172.21.15.132:6789/0 conn(0x7f216c1591b0 legacy=0x7f216c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216affd700 1 -- 172.21.15.46:0/3354772433 >> v1:172.21.15.46:6789/0 conn(0x7f216c158de0 legacy=0x7f216c069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.079+0000 7f216affd700 1 -- 172.21.15.46:0/3354772433 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0ccac0 con 0x7f216c069770 2023-10-06T11:30:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f216affd700 1 --2- 172.21.15.46:0/3354772433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c0b5970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21600098e0 tx=0x7f21600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108264aeff36fa2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011630 con 0x7f216c069770 2023-10-06T11:30:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011790 con 0x7f216c069770 2023-10-06T11:30:54.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f216a7fc700 1 -- 172.21.15.46:0/3354772433 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011b30 con 0x7f216c069770 2023-10-06T11:30:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 -- 172.21.15.46:0/3354772433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 msgr2=0x7f216c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 --2- 172.21.15.46:0/3354772433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c0b5970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f21600098e0 tx=0x7f21600091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 -- 172.21.15.46:0/3354772433 shutdown_connections 2023-10-06T11:30:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 --2- 172.21.15.46:0/3354772433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c0b5970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 -- 172.21.15.46:0/3354772433 >> 172.21.15.46:0/3354772433 conn(0x7f216c068390 msgr2=0x7f216c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 -- 172.21.15.46:0/3354772433 shutdown_connections 2023-10-06T11:30:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.083+0000 7f2170c6b700 1 -- 172.21.15.46:0/3354772433 wait complete. 2023-10-06T11:30:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.087+0000 7f2170c6b700 1 Processor -- start 2023-10-06T11:30:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.087+0000 7f2170c6b700 1 -- start start 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.087+0000 7f2170c6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.087+0000 7f2170c6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.087+0000 7f2170c6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f216c1591b0 0x7f216c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f2170c6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15bf70 con 0x7f216c158de0 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f2170c6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c145fd0 con 0x7f216c069770 2023-10-06T11:30:54.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f2170c6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15b6d0 con 0x7f216c1591b0 2023-10-06T11:30:54.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53508/0 (socket says 172.21.15.46:53508) 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 -- 172.21.15.46:0/226686729 learned_addr learned my addr 172.21.15.46:0/226686729 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216b7fe700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216bfff700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f216c1591b0 0x7f216c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 -- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f216c1591b0 msgr2=0x7f216c0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f216c1591b0 0x7f216c0c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 -- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 msgr2=0x7f216c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 -- 172.21.15.46:0/226686729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160009420 con 0x7f216c158de0 2023-10-06T11:30:54.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f216affd700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f216c088340 tx=0x7f2160017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160000d10 con 0x7f216c158de0 2023-10-06T11:30:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011900 con 0x7f216c158de0 2023-10-06T11:30:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c15b9b0 con 0x7f216c158de0 2023-10-06T11:30:54.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160025420 con 0x7f216c158de0 2023-10-06T11:30:54.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.091+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c0c53b0 con 0x7f216c158de0 2023-10-06T11:30:54.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.095+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c0c55a0 con 0x7f216c158de0 2023-10-06T11:30:54.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.095+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f2160023070 con 0x7f216c158de0 2023-10-06T11:30:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.095+0000 7f21537fe700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 0x7f2158062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.095+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f216007df70 con 0x7f216c158de0 2023-10-06T11:30:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.095+0000 7f216b7fe700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 0x7f2158062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.099+0000 7f216b7fe700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 0x7f2158062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.103+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f216004e180 con 0x7f216c158de0 2023-10-06T11:30:54.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.279+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f216c064590 con 0x7f216c158de0 2023-10-06T11:30:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.831+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f21600570d0 con 0x7f216c158de0 2023-10-06T11:30:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.839+0000 7f21537fe700 1 -- 172.21.15.46:0/226686729 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f21600674b0 con 0x7f216c158de0 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 msgr2=0x7f2158062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 0x7f2158062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c009040 comp rx=0 tx=0).stop 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 msgr2=0x7f216c145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f216c088340 tx=0x7f2160017000 comp rx=0 tx=0).stop 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 shutdown_connections 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.843+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f21580603a0 0x7f2158062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f216c1591b0 0x7f216c0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216c158de0 0x7f216c145a90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 --2- 172.21.15.46:0/226686729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f216c069770 0x7f216c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 >> 172.21.15.46:0/226686729 conn(0x7f216c068390 msgr2=0x7f216c156430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 shutdown_connections 2023-10-06T11:30:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:54.847+0000 7f2170c6b700 1 -- 172.21.15.46:0/226686729 wait complete. 2023-10-06T11:30:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:30:54.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:30:55.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.039+0000 7f6cbb123700 1 Processor -- start 2023-10-06T11:30:55.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.039+0000 7f6cbb123700 1 -- start start 2023-10-06T11:30:55.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cbb123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:55.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cbb123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb40c6150 con 0x7f6cb4161b50 2023-10-06T11:30:55.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cbb123700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb406e840 con 0x7f6cb4067280 2023-10-06T11:30:55.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cbb123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb406eb20 con 0x7f6cb4067650 2023-10-06T11:30:55.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cba922700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6cb4161b50 0x7f6cb40c6bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52870/0 (socket says 172.21.15.46:52870) 2023-10-06T11:30:55.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cba922700 1 -- 172.21.15.46:0/2665075033 learned_addr learned my addr 172.21.15.46:0/2665075033 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:55.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb9920700 1 --2- 172.21.15.46:0/2665075033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:55.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625181362 0 0) 0x7f6cb40c6150 con 0x7f6cb4161b50 2023-10-06T11:30:55.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca0003570 con 0x7f6cb4161b50 2023-10-06T11:30:55.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955685208 0 0) 0x7f6cb406e840 con 0x7f6cb4067280 2023-10-06T11:30:55.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb40c6150 con 0x7f6cb4067280 2023-10-06T11:30:55.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb9920700 1 -- 172.21.15.46:0/2665075033 >> v1:172.21.15.132:6789/0 conn(0x7f6cb4067280 legacy=0x7f6cb415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb9920700 1 -- 172.21.15.46:0/2665075033 >> v1:172.21.15.46:6789/0 conn(0x7f6cb4161b50 legacy=0x7f6cb40c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb9920700 1 -- 172.21.15.46:0/2665075033 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40c63f0 con 0x7f6cb4067650 2023-10-06T11:30:55.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb9920700 1 --2- 172.21.15.46:0/2665075033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415d690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009910 tx=0x7f6ca40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75020cb35f1946a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:55.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011630 con 0x7f6cb4067650 2023-10-06T11:30:55.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca4011790 con 0x7f6cb4067650 2023-10-06T11:30:55.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.043+0000 7f6cb911f700 1 -- 172.21.15.46:0/2665075033 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4011b30 con 0x7f6cb4067650 2023-10-06T11:30:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- 172.21.15.46:0/2665075033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 msgr2=0x7f6cb415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 --2- 172.21.15.46:0/2665075033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415d690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009910 tx=0x7f6ca40091a0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- 172.21.15.46:0/2665075033 shutdown_connections 2023-10-06T11:30:55.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 --2- 172.21.15.46:0/2665075033 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415d690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- 172.21.15.46:0/2665075033 >> 172.21.15.46:0/2665075033 conn(0x7f6cb40cbbb0 msgr2=0x7f6cb40cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- 172.21.15.46:0/2665075033 shutdown_connections 2023-10-06T11:30:55.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- 172.21.15.46:0/2665075033 wait complete. 2023-10-06T11:30:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 Processor -- start 2023-10-06T11:30:55.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.047+0000 7f6cbb123700 1 -- start start 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cb4067280 0x7f6cb414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb415ec50 con 0x7f6cb4161b50 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb415f7d0 con 0x7f6cb4067280 2023-10-06T11:30:55.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb415eea0 con 0x7f6cb4067650 2023-10-06T11:30:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53518/0 (socket says 172.21.15.46:53518) 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 -- 172.21.15.46:0/3066285800 learned_addr learned my addr 172.21.15.46:0/3066285800 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cb9920700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba121700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cb4067280 0x7f6cb414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 -- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 msgr2=0x7f6cb415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 -- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cb4067280 msgr2=0x7f6cb414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cb4067280 0x7f6cb414e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 -- 172.21.15.46:0/3066285800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009420 con 0x7f6cb4161b50 2023-10-06T11:30:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cba922700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00cc50 tx=0x7f6cac001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00a810 con 0x7f6cb4161b50 2023-10-06T11:30:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00a990 con 0x7f6cb4161b50 2023-10-06T11:30:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb415f180 con 0x7f6cb4161b50 2023-10-06T11:30:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00ad50 con 0x7f6cb4161b50 2023-10-06T11:30:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.051+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb40c0fe0 con 0x7f6cb4161b50 2023-10-06T11:30:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.055+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb415f960 con 0x7f6cb4161b50 2023-10-06T11:30:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.055+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6cac007300 con 0x7f6cb4161b50 2023-10-06T11:30:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.059+0000 7f6caa7fc700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 0x7f6c98062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.059+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f6cac07d070 con 0x7f6cb4161b50 2023-10-06T11:30:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.059+0000 7f6cba121700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 0x7f6c98062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.059+0000 7f6cba121700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 0x7f6c98062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007fd0 tx=0x7f6cb0007d50 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.063+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6cac04d120 con 0x7f6cb4161b50 2023-10-06T11:30:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.235+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6cb4064590 con 0x7f6cb4161b50 2023-10-06T11:30:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.847+0000 7f6caa7fc700 1 -- 172.21.15.46:0/3066285800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6cac050740 con 0x7f6cb4161b50 2023-10-06T11:30:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.851+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 msgr2=0x7f6c98062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.851+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 0x7f6c98062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007fd0 tx=0x7f6cb0007d50 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 msgr2=0x7f6cb40c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00cc50 tx=0x7f6cac001a40 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 shutdown_connections 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c98060670 0x7f6c98062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cb4161b50 0x7f6cb40c7a20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cb4067650 0x7f6cb415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 --2- 172.21.15.46:0/3066285800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cb4067280 0x7f6cb414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 >> 172.21.15.46:0/3066285800 conn(0x7f6cb40cbbb0 msgr2=0x7f6cb40cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 shutdown_connections 2023-10-06T11:30:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:55.855+0000 7f6cbb123700 1 -- 172.21.15.46:0/3066285800 wait complete. 2023-10-06T11:30:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:30:55.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:30:56.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.043+0000 7ffb931c5700 1 Processor -- start 2023-10-06T11:30:56.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.043+0000 7ffb931c5700 1 -- start start 2023-10-06T11:30:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb931c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb8c15d490 0x7ffb8c0b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb931c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0b2370 con 0x7ffb8c0d0180 2023-10-06T11:30:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb931c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c15eaa0 con 0x7ffb8c15e450 2023-10-06T11:30:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb931c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c15e890 con 0x7ffb8c15d490 2023-10-06T11:30:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb8c15d490 0x7ffb8c0b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb8c15d490 0x7ffb8c0b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53522/0 (socket says 172.21.15.46:53522) 2023-10-06T11:30:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 -- 172.21.15.46:0/4261879187 learned_addr learned my addr 172.21.15.46:0/4261879187 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 -- 172.21.15.46:0/4261879187 >> v1:172.21.15.132:6789/0 conn(0x7ffb8c15e450 legacy=0x7ffb8c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 -- 172.21.15.46:0/4261879187 >> v1:172.21.15.46:6789/0 conn(0x7ffb8c0d0180 legacy=0x7ffb8c15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 -- 172.21.15.46:0/4261879187 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b1a40 con 0x7ffb8c15d490 2023-10-06T11:30:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.047+0000 7ffb90f61700 1 --2- 172.21.15.46:0/4261879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c0b2800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009f20 tx=0x7ffb80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f48f2cfdf7cf5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb8b7fe700 1 -- 172.21.15.46:0/4261879187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000e070 con 0x7ffb8c15d490 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb8b7fe700 1 -- 172.21.15.46:0/4261879187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8000ba40 con 0x7ffb8c15d490 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb8b7fe700 1 -- 172.21.15.46:0/4261879187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb800092e0 con 0x7ffb8c15d490 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 -- 172.21.15.46:0/4261879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 msgr2=0x7ffb8c0b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 --2- 172.21.15.46:0/4261879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c0b2800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009f20 tx=0x7ffb80009840 comp rx=0 tx=0).stop 2023-10-06T11:30:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 -- 172.21.15.46:0/4261879187 shutdown_connections 2023-10-06T11:30:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 --2- 172.21.15.46:0/4261879187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c0b2800 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 -- 172.21.15.46:0/4261879187 >> 172.21.15.46:0/4261879187 conn(0x7ffb8c06e780 msgr2=0x7ffb8c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 -- 172.21.15.46:0/4261879187 shutdown_connections 2023-10-06T11:30:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.051+0000 7ffb931c5700 1 -- 172.21.15.46:0/4261879187 wait complete. 2023-10-06T11:30:56.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 Processor -- start 2023-10-06T11:30:56.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 -- start start 2023-10-06T11:30:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb8c0d0180 0x7ffb8c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb8c15e450 0x7ffb8c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b5660 con 0x7ffb8c15d490 2023-10-06T11:30:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b4a90 con 0x7ffb8c0d0180 2023-10-06T11:30:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb931c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1728f0 con 0x7ffb8c15e450 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53532/0 (socket says 172.21.15.46:53532) 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 -- 172.21.15.46:0/439372350 learned_addr learned my addr 172.21.15.46:0/439372350 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb90f61700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb8c0d0180 0x7ffb8c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb91762700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb8c15e450 0x7ffb8c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 -- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb8c15e450 msgr2=0x7ffb8c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb8c15e450 0x7ffb8c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 -- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb8c0d0180 msgr2=0x7ffb8c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb8c0d0180 0x7ffb8c0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.055+0000 7ffb8bfff700 1 -- 172.21.15.46:0/439372350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80009ae0 con 0x7ffb8c15d490 2023-10-06T11:30:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb8bfff700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800c5c0 tx=0x7ffb7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78011590 con 0x7ffb8c15d490 2023-10-06T11:30:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78011710 con 0x7ffb8c15d490 2023-10-06T11:30:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78011ad0 con 0x7ffb8c15d490 2023-10-06T11:30:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0b4cc0 con 0x7ffb8c15d490 2023-10-06T11:30:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0b5280 con 0x7ffb8c15d490 2023-10-06T11:30:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.059+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0b57a0 con 0x7ffb8c15d490 2023-10-06T11:30:56.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.063+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ffb7801b3b0 con 0x7ffb8c15d490 2023-10-06T11:30:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.063+0000 7ffb88ff9700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 0x7ffb70062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.063+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7ffb7807c080 con 0x7ffb8c15d490 2023-10-06T11:30:56.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.063+0000 7ffb90f61700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 0x7ffb70062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.063+0000 7ffb90f61700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 0x7ffb70062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009880 tx=0x7ffb80017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.071+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffb8c0b57a0 con 0x7ffb8c15d490 2023-10-06T11:30:56.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.247+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ffb8c0b57a0 con 0x7ffb8c15d490 2023-10-06T11:30:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.255+0000 7ffb88ff9700 1 -- 172.21.15.46:0/439372350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffb8c0b57a0 con 0x7ffb8c15d490 2023-10-06T11:30:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 msgr2=0x7ffb70062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 0x7ffb70062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009880 tx=0x7ffb80017000 comp rx=0 tx=0).stop 2023-10-06T11:30:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 msgr2=0x7ffb8c16db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800c5c0 tx=0x7ffb7800c8d0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 shutdown_connections 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb70060670 0x7ffb70062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb8c15e450 0x7ffb8c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb8c15d490 0x7ffb8c16db20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 --2- 172.21.15.46:0/439372350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb8c0d0180 0x7ffb8c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 >> 172.21.15.46:0/439372350 conn(0x7ffb8c06e780 msgr2=0x7ffb8c0cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 shutdown_connections 2023-10-06T11:30:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.259+0000 7ffb931c5700 1 -- 172.21.15.46:0/439372350 wait complete. 2023-10-06T11:30:56.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:30:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 Processor -- start 2023-10-06T11:30:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 -- start start 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b30f0 0x7f82000b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82000b0dc0 0x7f82000b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82000afd20 con 0x7f82000adbe0 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8200159740 con 0x7f82000b0dc0 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000b1760 con 0x7f82000b30f0 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82000b0dc0 0x7f82000b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82000b0dc0 0x7f82000b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53538/0 (socket says 172.21.15.46:53538) 2023-10-06T11:30:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 -- 172.21.15.46:0/2263863146 learned_addr learned my addr 172.21.15.46:0/2263863146 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8205268700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b30f0 0x7f82000b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 -- 172.21.15.46:0/2263863146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b30f0 msgr2=0x7f82000b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b30f0 0x7f82000b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 -- 172.21.15.46:0/2263863146 >> v1:172.21.15.132:6789/0 conn(0x7f82000adbe0 legacy=0x7f82000d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 -- 172.21.15.46:0/2263863146 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000b0000 con 0x7f82000b0dc0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f820626a700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b0dc0 0x7f82000b1220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009f50 tx=0x7f81f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8641dd35033e6594 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8204a67700 1 -- 172.21.15.46:0/2263863146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800e070 con 0x7f82000b0dc0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8204a67700 1 -- 172.21.15.46:0/2263863146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f800b9c0 con 0x7f82000b0dc0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8204a67700 1 -- 172.21.15.46:0/2263863146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f80092e0 con 0x7f82000b0dc0 2023-10-06T11:30:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 -- 172.21.15.46:0/2263863146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b0dc0 msgr2=0x7f82000b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.439+0000 7f8206a6b700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b0dc0 0x7f82000b1220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009f50 tx=0x7f81f8009840 comp rx=0 tx=0).stop 2023-10-06T11:30:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- 172.21.15.46:0/2263863146 shutdown_connections 2023-10-06T11:30:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b0dc0 0x7f82000b1220 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 --2- 172.21.15.46:0/2263863146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b30f0 0x7f82000b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- 172.21.15.46:0/2263863146 >> 172.21.15.46:0/2263863146 conn(0x7f820006ef60 msgr2=0x7f82000d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- 172.21.15.46:0/2263863146 shutdown_connections 2023-10-06T11:30:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- 172.21.15.46:0/2263863146 wait complete. 2023-10-06T11:30:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 Processor -- start 2023-10-06T11:30:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- start start 2023-10-06T11:30:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000adbe0 0x7f820014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b0dc0 0x7f8200154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820015b200 con 0x7f82000b30f0 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820015a5f0 con 0x7f82000adbe0 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.443+0000 7f8206a6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820015a8d0 con 0x7f82000b0dc0 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f8205268700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b0dc0 0x7f8200154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53550/0 (socket says 172.21.15.46:53550) 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 -- 172.21.15.46:0/703474634 learned_addr learned my addr 172.21.15.46:0/703474634 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f8205a69700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000adbe0 0x7f820014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 -- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b0dc0 msgr2=0x7f8200154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b0dc0 0x7f8200154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 -- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000adbe0 msgr2=0x7f820014fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000adbe0 0x7f820014fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 -- 172.21.15.46:0/703474634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009ae0 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f820626a700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009880 tx=0x7f81f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8008ba0 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8008d00 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8026690 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820015abb0 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820015a240 con 0x7f82000b30f0 2023-10-06T11:30:56.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.447+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820015b3b0 con 0x7f82000b30f0 2023-10-06T11:30:56.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.451+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f81f8003040 con 0x7f82000b30f0 2023-10-06T11:30:56.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.451+0000 7f81f5ffb700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 0x7f81e4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.451+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f81f807e550 con 0x7f82000b30f0 2023-10-06T11:30:56.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.451+0000 7f8205a69700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 0x7f81e4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.451+0000 7f8205a69700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 0x7f81e4062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0039e0 tx=0x7f81fc007ec0 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.459+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f820015b3b0 con 0x7f82000b30f0 2023-10-06T11:30:56.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.630+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f8200064590 con 0x7f82000b30f0 2023-10-06T11:30:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.634+0000 7f81f5ffb700 1 -- 172.21.15.46:0/703474634 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f81f804e6e0 con 0x7f82000b30f0 2023-10-06T11:30:56.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 msgr2=0x7f81e4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 0x7f81e4062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0039e0 tx=0x7f81fc007ec0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 msgr2=0x7f8200157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009880 tx=0x7f81f8018000 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 shutdown_connections 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f81e40606a0 0x7f81e4062b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000b30f0 0x7f8200157400 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82000b0dc0 0x7f8200154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 --2- 172.21.15.46:0/703474634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000adbe0 0x7f820014fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 >> 172.21.15.46:0/703474634 conn(0x7f820006ef60 msgr2=0x7f82000adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.638+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 shutdown_connections 2023-10-06T11:30:56.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.642+0000 7f8206a6b700 1 -- 172.21.15.46:0/703474634 wait complete. 2023-10-06T11:30:56.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 Processor -- start 2023-10-06T11:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 -- start start 2023-10-06T11:30:56.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9780c7d40 0x7fb97806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9780c1d90 con 0x7fb9780c97e0 2023-10-06T11:30:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780c8190 con 0x7fb97806cf30 2023-10-06T11:30:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.830+0000 7fb97d9cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780c1460 con 0x7fb9780c7d40 2023-10-06T11:30:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb97d1cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9780c7d40 0x7fb97806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9780c7d40 0x7fb97806c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:39638/0 (socket says 172.21.15.46:39638) 2023-10-06T11:30:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 -- 172.21.15.46:0/727281962 learned_addr learned my addr 172.21.15.46:0/727281962 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb97d1cb700 1 --2- 172.21.15.46:0/727281962 >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53552/0 (socket says 172.21.15.46:53552) 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 -- 172.21.15.46:0/727281962 >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 msgr2=0x7fb9780c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 --2- 172.21.15.46:0/727281962 >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 -- 172.21.15.46:0/727281962 >> v1:172.21.15.132:6789/0 conn(0x7fb9780c97e0 legacy=0x7fb9780c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 -- 172.21.15.46:0/727281962 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9780c1740 con 0x7fb9780c7d40 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb97d1cb700 1 --2- 172.21.15.46:0/727281962 >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb9777fe700 1 -- 172.21.15.46:0/727281962 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422188588 0 0) 0x7fb9780c1d90 con 0x7fb9780c97e0 2023-10-06T11:30:56.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb977fff700 1 --2- 172.21.15.46:0/727281962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9780c7d40 0x7fb97806c9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb9780c9690 tx=0x7fb964009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae9167fcde5c752 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb9777fe700 1 -- 172.21.15.46:0/727281962 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964009e70 con 0x7fb9780c7d40 2023-10-06T11:30:56.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb9777fe700 1 -- 172.21.15.46:0/727281962 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400b860 con 0x7fb9780c7d40 2023-10-06T11:30:56.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb9777fe700 1 -- 172.21.15.46:0/727281962 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400bc00 con 0x7fb9780c7d40 2023-10-06T11:30:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb97d9cc700 1 -- 172.21.15.46:0/727281962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9780c7d40 msgr2=0x7fb97806c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.834+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/727281962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9780c7d40 0x7fb97806c9f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb9780c9690 tx=0x7fb964009250 comp rx=0 tx=0).stop 2023-10-06T11:30:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 -- 172.21.15.46:0/727281962 shutdown_connections 2023-10-06T11:30:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/727281962 >> v2:172.21.15.46:3300/0 conn(0x7fb97806cf30 0x7fb9780c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/727281962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9780c7d40 0x7fb97806c9f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 -- 172.21.15.46:0/727281962 >> 172.21.15.46:0/727281962 conn(0x7fb9780cbc10 msgr2=0x7fb9780cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 -- 172.21.15.46:0/727281962 shutdown_connections 2023-10-06T11:30:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 -- 172.21.15.46:0/727281962 wait complete. 2023-10-06T11:30:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 Processor -- start 2023-10-06T11:30:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.838+0000 7fb97d9cc700 1 -- start start 2023-10-06T11:30:56.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97806cf30 0x7fb97815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9780c7d40 0x7fb97814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97815e610 con 0x7fb9780c97e0 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97815da00 con 0x7fb97806cf30 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d9cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97815dce0 con 0x7fb9780c7d40 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53558/0 (socket says 172.21.15.46:53558) 2023-10-06T11:30:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 -- 172.21.15.46:0/3152593163 learned_addr learned my addr 172.21.15.46:0/3152593163 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97c9ca700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97806cf30 0x7fb97815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb977fff700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9780c7d40 0x7fb97814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 -- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9780c7d40 msgr2=0x7fb97814cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9780c7d40 0x7fb97814cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 -- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97806cf30 msgr2=0x7fb97815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97806cf30 0x7fb97815eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 -- 172.21.15.46:0/3152593163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009a20 con 0x7fb9780c97e0 2023-10-06T11:30:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.842+0000 7fb97d1cb700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb970009fc0 tx=0x7fb97000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000cc80 con 0x7fb9780c97e0 2023-10-06T11:30:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97000f040 con 0x7fb9780c97e0 2023-10-06T11:30:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970014480 con 0x7fb9780c97e0 2023-10-06T11:30:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9640094f0 con 0x7fb9780c97e0 2023-10-06T11:30:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97815e2a0 con 0x7fb9780c97e0 2023-10-06T11:30:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.850+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97806f690 con 0x7fb9780c97e0 2023-10-06T11:30:56.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.854+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb970003020 con 0x7fb9780c97e0 2023-10-06T11:30:56.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.854+0000 7fb974ff9700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 0x7fb95c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:56.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.854+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7fb97007d730 con 0x7fb9780c97e0 2023-10-06T11:30:56.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.854+0000 7fb97c9ca700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 0x7fb95c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.854+0000 7fb97c9ca700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 0x7fb95c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb96c003a40 tx=0x7fb96c007ec0 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:56.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:56.862+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb97806f690 con 0x7fb9780c97e0 2023-10-06T11:30:57.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.030+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb97806f690 con 0x7fb9780c97e0 2023-10-06T11:30:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.038+0000 7fb974ff9700 1 -- 172.21.15.46:0/3152593163 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb97806f690 con 0x7fb9780c97e0 2023-10-06T11:30:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 msgr2=0x7fb95c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 0x7fb95c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb96c003a40 tx=0x7fb96c007ec0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 msgr2=0x7fb9780ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb970009fc0 tx=0x7fb97000b040 comp rx=0 tx=0).stop 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 shutdown_connections 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c060670 0x7fb95c062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9780c97e0 0x7fb9780ad5a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9780c7d40 0x7fb97814cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 --2- 172.21.15.46:0/3152593163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97806cf30 0x7fb97815eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 >> 172.21.15.46:0/3152593163 conn(0x7fb9780cbc10 msgr2=0x7fb97815c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 shutdown_connections 2023-10-06T11:30:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.042+0000 7fb97d9cc700 1 -- 172.21.15.46:0/3152593163 wait complete. 2023-10-06T11:30:57.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-06T11:30:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 Processor -- start 2023-10-06T11:30:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 -- start start 2023-10-06T11:30:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5380d07b0 0x7ff5380b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5380b1130 0x7ff5380ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380ae340 con 0x7ff538159290 2023-10-06T11:30:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5380b15b0 con 0x7ff5380b1130 2023-10-06T11:30:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.230+0000 7ff53eca3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380b09a0 con 0x7ff5380d07b0 2023-10-06T11:30:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5380b1130 0x7ff5380ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5380b1130 0x7ff5380ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53564/0 (socket says 172.21.15.46:53564) 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 -- 172.21.15.46:0/406106686 learned_addr learned my addr 172.21.15.46:0/406106686 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53d4a0700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5380d07b0 0x7ff5380b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 -- 172.21.15.46:0/406106686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5380d07b0 msgr2=0x7ff5380b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5380d07b0 0x7ff5380b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 -- 172.21.15.46:0/406106686 >> v1:172.21.15.46:6789/0 conn(0x7ff538159290 legacy=0x7ff5380b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 -- 172.21.15.46:0/406106686 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538159660 con 0x7ff5380b1130 2023-10-06T11:30:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53e4a2700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff5380ae7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099b0 tx=0x7ff530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f794d2c2b90004 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53cc9f700 1 -- 172.21.15.46:0/406106686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530009ea0 con 0x7ff5380b1130 2023-10-06T11:30:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53cc9f700 1 -- 172.21.15.46:0/406106686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000b8f0 con 0x7ff5380b1130 2023-10-06T11:30:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53cc9f700 1 -- 172.21.15.46:0/406106686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000bc90 con 0x7ff5380b1130 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 -- 172.21.15.46:0/406106686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 msgr2=0x7ff5380ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff5380ae7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099b0 tx=0x7ff530009250 comp rx=0 tx=0).stop 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 -- 172.21.15.46:0/406106686 shutdown_connections 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff5380ae7d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 --2- 172.21.15.46:0/406106686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5380d07b0 0x7ff5380b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 -- 172.21.15.46:0/406106686 >> 172.21.15.46:0/406106686 conn(0x7ff53806ef60 msgr2=0x7ff5380d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 -- 172.21.15.46:0/406106686 shutdown_connections 2023-10-06T11:30:57.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.234+0000 7ff53eca3700 1 -- 172.21.15.46:0/406106686 wait complete. 2023-10-06T11:30:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.238+0000 7ff53eca3700 1 Processor -- start 2023-10-06T11:30:57.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.238+0000 7ff53eca3700 1 -- start start 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 0x7ff538149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 0x7ff5380ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53814ac60 con 0x7ff5380b1130 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53814dac0 con 0x7ff5380d07b0 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff538156d80 con 0x7ff538159290 2023-10-06T11:30:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53576/0 (socket says 172.21.15.46:53576) 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 -- 172.21.15.46:0/1593674848 learned_addr learned my addr 172.21.15.46:0/1593674848 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53d4a0700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 0x7ff538149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53e4a2700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 0x7ff5380ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 -- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 msgr2=0x7ff5380ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 0x7ff5380ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 -- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 msgr2=0x7ff538149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 0x7ff538149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 -- 172.21.15.46:0/1593674848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300094f0 con 0x7ff5380b1130 2023-10-06T11:30:57.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53e4a2700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 0x7ff5380ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53d4a0700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 0x7ff538149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53dca1700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c910 tx=0x7ff534001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400a7a0 con 0x7ff5380b1130 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53400a920 con 0x7ff5380b1130 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53814d500 con 0x7ff5380b1130 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400ace0 con 0x7ff5380b1130 2023-10-06T11:30:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.242+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53814a8f0 con 0x7ff5380b1130 2023-10-06T11:30:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.246+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff538064590 con 0x7ff5380b1130 2023-10-06T11:30:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.246+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff534003030 con 0x7ff5380b1130 2023-10-06T11:30:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.250+0000 7ff52dffb700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 0x7ff51c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.250+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7ff53407cc40 con 0x7ff5380b1130 2023-10-06T11:30:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.250+0000 7ff53d4a0700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 0x7ff51c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.250+0000 7ff53d4a0700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 0x7ff51c062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff5380d0520 tx=0x7ff528008040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.254+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff53404ce30 con 0x7ff5380b1130 2023-10-06T11:30:57.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.486+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7ff5380b06c0 con 0x7ff5380b1130 2023-10-06T11:30:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.854+0000 7ff52dffb700 1 -- 172.21.15.46:0/1593674848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v457) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff534050450 con 0x7ff5380b1130 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 msgr2=0x7ff51c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 0x7ff51c062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff5380d0520 tx=0x7ff528008040 comp rx=0 tx=0).stop 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 msgr2=0x7ff538156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c910 tx=0x7ff534001710 comp rx=0 tx=0).stop 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 shutdown_connections 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff51c060670 0x7ff51c062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff538159290 0x7ff5380ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5380d07b0 0x7ff538149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 --2- 172.21.15.46:0/1593674848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5380b1130 0x7ff538156840 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 >> 172.21.15.46:0/1593674848 conn(0x7ff53806ef60 msgr2=0x7ff5380b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 shutdown_connections 2023-10-06T11:30:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:57.858+0000 7ff53eca3700 1 -- 172.21.15.46:0/1593674848 wait complete. 2023-10-06T11:30:57.891 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-06T11:30:57.891 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-06T11:30:57.891 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:30:57.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-06T11:30:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.050+0000 7f6c3b105700 1 Processor -- start 2023-10-06T11:30:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.050+0000 7f6c3b105700 1 -- start start 2023-10-06T11:30:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c3b105700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c34159050 0x7f6c34159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c3b105700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6c340d0aa0 0x7f6c340b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c3b105700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340b4f80 con 0x7f6c340b39b0 2023-10-06T11:30:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c3b105700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340b4370 con 0x7f6c34159050 2023-10-06T11:30:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c3b105700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340b4650 con 0x7f6c340d0aa0 2023-10-06T11:30:58.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c34159050 0x7f6c34159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6c34159050 0x7f6c34159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53578/0 (socket says 172.21.15.46:53578) 2023-10-06T11:30:58.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 -- 172.21.15.46:0/1658427111 learned_addr learned my addr 172.21.15.46:0/1658427111 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:58.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c396a2700 1 --1- 172.21.15.46:0/1658427111 >> v1:172.21.15.46:6789/0 conn(0x7f6c340b39b0 0x7f6c340b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52912/0 (socket says 172.21.15.46:52912) 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c2bfff700 1 --2- 172.21.15.46:0/1658427111 >> v2:172.21.15.132:3300/0 conn(0x7f6c340d0aa0 0x7f6c340b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 -- 172.21.15.46:0/1658427111 >> v2:172.21.15.132:3300/0 conn(0x7f6c340d0aa0 msgr2=0x7f6c340b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/1658427111 >> v2:172.21.15.132:3300/0 conn(0x7f6c340d0aa0 0x7f6c340b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 -- 172.21.15.46:0/1658427111 >> v1:172.21.15.46:6789/0 conn(0x7f6c340b39b0 legacy=0x7f6c340b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.054+0000 7f6c38ea1700 1 -- 172.21.15.46:0/1658427111 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340b4930 con 0x7f6c34159050 2023-10-06T11:30:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/1658427111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c34159050 0x7f6c34159420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c340892d0 tx=0x7f6c30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136d002de499ece0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c2b7fe700 1 -- 172.21.15.46:0/1658427111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000e070 con 0x7f6c34159050 2023-10-06T11:30:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c2b7fe700 1 -- 172.21.15.46:0/1658427111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3000ba50 con 0x7f6c34159050 2023-10-06T11:30:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c2b7fe700 1 -- 172.21.15.46:0/1658427111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c300092e0 con 0x7f6c34159050 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- 172.21.15.46:0/1658427111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c34159050 msgr2=0x7f6c34159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 --2- 172.21.15.46:0/1658427111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c34159050 0x7f6c34159420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c340892d0 tx=0x7f6c30009840 comp rx=0 tx=0).stop 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- 172.21.15.46:0/1658427111 shutdown_connections 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 --2- 172.21.15.46:0/1658427111 >> v2:172.21.15.132:3300/0 conn(0x7f6c340d0aa0 0x7f6c340b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 --2- 172.21.15.46:0/1658427111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c34159050 0x7f6c34159420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- 172.21.15.46:0/1658427111 >> 172.21.15.46:0/1658427111 conn(0x7f6c3406e780 msgr2=0x7f6c3406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- 172.21.15.46:0/1658427111 shutdown_connections 2023-10-06T11:30:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- 172.21.15.46:0/1658427111 wait complete. 2023-10-06T11:30:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 Processor -- start 2023-10-06T11:30:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.058+0000 7f6c3b105700 1 -- start start 2023-10-06T11:30:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 0x7f6c3415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c34159050 0x7f6c341493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415a8e0 con 0x7f6c340d0aa0 2023-10-06T11:30:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340c97d0 con 0x7f6c34159050 2023-10-06T11:30:58.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415aad0 con 0x7f6c340b39b0 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53588/0 (socket says 172.21.15.46:53588) 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 -- 172.21.15.46:0/2121885458 learned_addr learned my addr 172.21.15.46:0/2121885458 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c396a2700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c34159050 0x7f6c341493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 0x7f6c3415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 -- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 msgr2=0x7f6c3415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 0x7f6c3415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 -- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c34159050 msgr2=0x7f6c341493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c34159050 0x7f6c341493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 -- 172.21.15.46:0/2121885458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009ae0 con 0x7f6c340d0aa0 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 0x7f6c3415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c2bfff700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d8c0 tx=0x7f6c24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240125f0 con 0x7f6c340d0aa0 2023-10-06T11:30:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c340c72f0 con 0x7f6c340d0aa0 2023-10-06T11:30:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24012750 con 0x7f6c340d0aa0 2023-10-06T11:30:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24012af0 con 0x7f6c340d0aa0 2023-10-06T11:30:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.062+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c340c7b90 con 0x7f6c340d0aa0 2023-10-06T11:30:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.066+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34088820 con 0x7f6c340d0aa0 2023-10-06T11:30:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.066+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6c2401e460 con 0x7f6c340d0aa0 2023-10-06T11:30:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.070+0000 7f6c28ff9700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 0x7f6c20062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.070+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f6c2407d250 con 0x7f6c340d0aa0 2023-10-06T11:30:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.070+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 0x7f6c20062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:58.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.070+0000 7f6c38ea1700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 0x7f6c20062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009880 tx=0x7f6c30018000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.074+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6c2404d440 con 0x7f6c340d0aa0 2023-10-06T11:30:58.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.302+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f6c340b4c10 con 0x7f6c340d0aa0 2023-10-06T11:30:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.862+0000 7f6c28ff9700 1 -- 172.21.15.46:0/2121885458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v458) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6c24050a60 con 0x7f6c340d0aa0 2023-10-06T11:30:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 msgr2=0x7f6c20062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 0x7f6c20062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009880 tx=0x7f6c30018000 comp rx=0 tx=0).stop 2023-10-06T11:30:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 msgr2=0x7f6c340c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d8c0 tx=0x7f6c24008040 comp rx=0 tx=0).stop 2023-10-06T11:30:58.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 shutdown_connections 2023-10-06T11:30:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6c20060670 0x7f6c20062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c34159050 0x7f6c341493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c340d0aa0 0x7f6c340c46b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 --2- 172.21.15.46:0/2121885458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c340b39b0 0x7f6c3415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:58.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 >> 172.21.15.46:0/2121885458 conn(0x7f6c3406e780 msgr2=0x7f6c340b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:58.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 shutdown_connections 2023-10-06T11:30:58.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:58.866+0000 7f6c3b105700 1 -- 172.21.15.46:0/2121885458 wait complete. 2023-10-06T11:30:58.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:30:59.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.066+0000 7f7363f85700 1 Processor -- start 2023-10-06T11:30:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.066+0000 7f7363f85700 1 -- start start 2023-10-06T11:30:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7363f85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f735c0b23c0 0x7f735c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7363f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f735c0cae40 0x7f735c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7363f85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c0cc510 con 0x7f735c0ccb40 2023-10-06T11:30:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7363f85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0cb900 con 0x7f735c0b23c0 2023-10-06T11:30:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7363f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0cbbe0 con 0x7f735c0cae40 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f735c0b23c0 0x7f735c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f735c0b23c0 0x7f735c0b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53602/0 (socket says 172.21.15.46:53602) 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 -- 172.21.15.46:0/1717926572 learned_addr learned my addr 172.21.15.46:0/1717926572 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361520700 1 --2- 172.21.15.46:0/1717926572 >> v2:172.21.15.132:3300/0 conn(0x7f735c0cae40 0x7f735c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 -- 172.21.15.46:0/1717926572 >> v2:172.21.15.132:3300/0 conn(0x7f735c0cae40 msgr2=0x7f735c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 --2- 172.21.15.46:0/1717926572 >> v2:172.21.15.132:3300/0 conn(0x7f735c0cae40 0x7f735c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 -- 172.21.15.46:0/1717926572 >> v1:172.21.15.132:6789/0 conn(0x7f735c0ccb40 legacy=0x7f735c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.070+0000 7f7361d21700 1 -- 172.21.15.46:0/1717926572 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0cbec0 con 0x7f735c0b23c0 2023-10-06T11:30:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7361d21700 1 --2- 172.21.15.46:0/1717926572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0b23c0 0x7f735c0b2790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f735c089420 tx=0x7f7358009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fca53d59979a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7360d1f700 1 -- 172.21.15.46:0/1717926572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358005c10 con 0x7f735c0b23c0 2023-10-06T11:30:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7360d1f700 1 -- 172.21.15.46:0/1717926572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358005d70 con 0x7f735c0b23c0 2023-10-06T11:30:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7360d1f700 1 -- 172.21.15.46:0/1717926572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800b840 con 0x7f735c0b23c0 2023-10-06T11:30:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 -- 172.21.15.46:0/1717926572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0b23c0 msgr2=0x7f735c0b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 --2- 172.21.15.46:0/1717926572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0b23c0 0x7f735c0b2790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f735c089420 tx=0x7f7358009220 comp rx=0 tx=0).stop 2023-10-06T11:30:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 -- 172.21.15.46:0/1717926572 shutdown_connections 2023-10-06T11:30:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 --2- 172.21.15.46:0/1717926572 >> v2:172.21.15.132:3300/0 conn(0x7f735c0cae40 0x7f735c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 --2- 172.21.15.46:0/1717926572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0b23c0 0x7f735c0b2790 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 -- 172.21.15.46:0/1717926572 >> 172.21.15.46:0/1717926572 conn(0x7f735c099480 msgr2=0x7f735c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 -- 172.21.15.46:0/1717926572 shutdown_connections 2023-10-06T11:30:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.074+0000 7f7363f85700 1 -- 172.21.15.46:0/1717926572 wait complete. 2023-10-06T11:30:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 Processor -- start 2023-10-06T11:30:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 -- start start 2023-10-06T11:30:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 0x7f735c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 0x7f735c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c159f30 con 0x7f735c0ccb40 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c09dd10 con 0x7f735c0cae40 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7363f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0b4560 con 0x7f735c0b23c0 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7361d21700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 0x7f735c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7361520700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 0x7f735c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53604/0 (socket says 172.21.15.46:53604) 2023-10-06T11:30:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 -- 172.21.15.46:0/3075751291 learned_addr learned my addr 172.21.15.46:0/3075751291 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 -- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 msgr2=0x7f735c0b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 0x7f735c0b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 -- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 msgr2=0x7f735c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 0x7f735c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7362522700 1 -- 172.21.15.46:0/3075751291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358009480 con 0x7f735c0ccb40 2023-10-06T11:30:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7361d21700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 0x7f735c0b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:59.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.078+0000 7f7361520700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 0x7f735c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7362522700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f735400c930 tx=0x7f73540026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354012630 con 0x7f735c0ccb40 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73540127b0 con 0x7f735c0ccb40 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354012b70 con 0x7f735c0ccb40 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c0b4840 con 0x7f735c0ccb40 2023-10-06T11:30:59.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c0b54e0 con 0x7f735c0ccb40 2023-10-06T11:30:59.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c159a20 con 0x7f735c0ccb40 2023-10-06T11:30:59.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.082+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f735401b400 con 0x7f735c0ccb40 2023-10-06T11:30:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.086+0000 7f7351ffb700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 0x7f7340062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.086+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f735407ca60 con 0x7f735c0ccb40 2023-10-06T11:30:59.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.086+0000 7f7361d21700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 0x7f7340062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.086+0000 7f7361d21700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 0x7f7340062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73580030c0 tx=0x7f7358015000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.094+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f735c159a20 con 0x7f735c0ccb40 2023-10-06T11:30:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.262+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f735c159a20 con 0x7f735c0ccb40 2023-10-06T11:30:59.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.274+0000 7f7351ffb700 1 -- 172.21.15.46:0/3075751291 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f735c159a20 con 0x7f735c0ccb40 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 msgr2=0x7f7340062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 0x7f7340062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73580030c0 tx=0x7f7358015000 comp rx=0 tx=0).stop 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 msgr2=0x7f735c0b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f735400c930 tx=0x7f73540026e0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 shutdown_connections 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f73400606a0 0x7f7340062b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f735c0ccb40 0x7f735c0b3ee0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f735c0cae40 0x7f735c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 --2- 172.21.15.46:0/3075751291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f735c0b23c0 0x7f735c0b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 >> 172.21.15.46:0/3075751291 conn(0x7f735c099480 msgr2=0x7f735c0c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 shutdown_connections 2023-10-06T11:30:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.278+0000 7f7363f85700 1 -- 172.21.15.46:0/3075751291 wait complete. 2023-10-06T11:30:59.309 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:30:59.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:30:59.354 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:30:59.397 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:30:59.397 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:30:59.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:30:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.558+0000 7f30caa03700 1 Processor -- start 2023-10-06T11:30:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.558+0000 7f30caa03700 1 -- start start 2023-10-06T11:30:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30caa03700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30c4067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30caa03700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c4069660 con 0x7f30c40bf1a0 2023-10-06T11:30:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30caa03700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c4068ac0 con 0x7f30c40c8110 2023-10-06T11:30:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30caa03700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c4068d30 con 0x7f30c40bedd0 2023-10-06T11:30:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c9200700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f30c40bf1a0 0x7f30c40c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52918/0 (socket says 172.21.15.46:52918) 2023-10-06T11:30:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c9200700 1 -- 172.21.15.46:0/2359859358 learned_addr learned my addr 172.21.15.46:0/2359859358 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:30:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c9a01700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30c4067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957834148 0 0) 0x7f30c4069660 con 0x7f30c40bf1a0 2023-10-06T11:30:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b0003570 con 0x7f30c40bf1a0 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2317707552 0 0) 0x7f30b0003570 con 0x7f30c40bf1a0 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 msgr2=0x7f30c4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30c4067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 >> v1:172.21.15.132:6789/0 conn(0x7f30c40c8110 legacy=0x7f30c4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4069010 con 0x7f30c40bf1a0 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f30b4001470 con 0x7f30c40bf1a0 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 >> v1:172.21.15.46:6789/0 conn(0x7f30c40bf1a0 legacy=0x7f30c40c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30b0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b0003fd0 0x7f30b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30b0004d50 0x7f30b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c4068d30 con 0x7f30b0004d50 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b000bb40 con 0x7f30b0003fd0 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b000bd00 con 0x7f30c40bedd0 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30ca202700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30b0004d50 0x7f30b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.562+0000 7f30c9200700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b0003fd0 0x7f30b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30c9a01700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30b0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 -- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 msgr2=0x7f30b0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30b0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 -- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b0003fd0 msgr2=0x7f30b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30b0003fd0 0x7f30b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 -- 172.21.15.46:0/2359859358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b0003570 con 0x7f30b0004d50 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30c9a01700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40bedd0 0x7f30b0003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:30:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30b0004d50 0x7f30b00051c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0019a0 tx=0x7f30bc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7a8b8924d52a2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00c5d0 con 0x7f30b0004d50 2023-10-06T11:30:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc00c750 con 0x7f30b0004d50 2023-10-06T11:30:59.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30c89ff700 1 -- 172.21.15.46:0/2359859358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00cb10 con 0x7f30b0004d50 2023-10-06T11:30:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 -- 172.21.15.46:0/2359859358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30b0004d50 msgr2=0x7f30b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 --2- 172.21.15.46:0/2359859358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30b0004d50 0x7f30b00051c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0019a0 tx=0x7f30bc007d10 comp rx=0 tx=0).stop 2023-10-06T11:30:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30ca202700 1 -- 172.21.15.46:0/2359859358 reap_dead start 2023-10-06T11:30:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 -- 172.21.15.46:0/2359859358 shutdown_connections 2023-10-06T11:30:59.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 -- 172.21.15.46:0/2359859358 >> 172.21.15.46:0/2359859358 conn(0x7f30c4062ef0 msgr2=0x7f30c40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 -- 172.21.15.46:0/2359859358 shutdown_connections 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.566+0000 7f30caa03700 1 -- 172.21.15.46:0/2359859358 wait complete. 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 Processor -- start 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 -- start start 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c4156960 0x7f30c4158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40c88c0 0x7f30c40c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40c92a0 con 0x7f30c40af650 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c4157580 con 0x7f30c4156960 2023-10-06T11:30:59.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30caa03700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41580c0 con 0x7f30c40c88c0 2023-10-06T11:30:59.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30c9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.570+0000 7f30c9a01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c4156960 0x7f30c4158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53630/0 (socket says 172.21.15.46:53630) 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 -- 172.21.15.46:0/2057460604 learned_addr learned my addr 172.21.15.46:0/2057460604 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30ca202700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40c88c0 0x7f30c40c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 -- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40c88c0 msgr2=0x7f30c40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40c88c0 0x7f30c40c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 -- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c4156960 msgr2=0x7f30c4158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c4156960 0x7f30c4158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 -- 172.21.15.46:0/2057460604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4157790 con 0x7f30c40af650 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30c9200700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f30b40073c0 tx=0x7f30b4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b400f640 con 0x7f30c40af650 2023-10-06T11:30:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b400f7a0 con 0x7f30c40af650 2023-10-06T11:30:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b400fb60 con 0x7f30c40af650 2023-10-06T11:30:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c4157a70 con 0x7f30c40af650 2023-10-06T11:30:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c40b0b10 con 0x7f30c40af650 2023-10-06T11:30:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.574+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c4158270 con 0x7f30c40af650 2023-10-06T11:30:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.578+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f30b401e3b0 con 0x7f30c40af650 2023-10-06T11:30:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.578+0000 7f30b9ffb700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 0x7f30a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:30:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.578+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f30b407c940 con 0x7f30c40af650 2023-10-06T11:30:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.578+0000 7f30c9a01700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 0x7f30a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:30:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.582+0000 7f30c9a01700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 0x7f30a8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f30c40beb70 tx=0x7f30c000b040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:30:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.586+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f30c4158270 con 0x7f30c40af650 2023-10-06T11:30:59.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.798+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c4158230 con 0x7f30c40af650 2023-10-06T11:30:59.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.798+0000 7f30b9ffb700 1 -- 172.21.15.46:0/2057460604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f30b401a020 con 0x7f30c40af650 2023-10-06T11:30:59.809 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:30:59.809 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1137106739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1137106739},{"type":"v1","addr":"172.21.15.132:6833","nonce":1137106739}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1470331264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1470331264},{"type":"v1","addr":"172.21.15.132:6835","nonce":1470331264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4992,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3962966234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":3962966234},{"type":"v1","addr":"172.21.15.46:6836","nonce":3962966234}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5760,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1421432490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1421432490},{"type":"v1","addr":"172.21.15.46:6837","nonce":1421432490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-06T11:30:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 msgr2=0x7f30a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 0x7f30a8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f30c40beb70 tx=0x7f30c000b040 comp rx=0 tx=0).stop 2023-10-06T11:30:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 msgr2=0x7f30c40b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:30:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f30b40073c0 tx=0x7f30b4001e70 comp rx=0 tx=0).stop 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 shutdown_connections 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f30a8060670 0x7f30a8062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30c40c88c0 0x7f30c40c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c40af650 0x7f30c40b05d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 --2- 172.21.15.46:0/2057460604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30c4156960 0x7f30c4158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.802+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 >> 172.21.15.46:0/2057460604 conn(0x7f30c4062ef0 msgr2=0x7f30c415c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.806+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 shutdown_connections 2023-10-06T11:30:59.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:30:59.806+0000 7f30caa03700 1 -- 172.21.15.46:0/2057460604 wait complete. 2023-10-06T11:30:59.817 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2023-10-06T11:30:59.835 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:30:59.835 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:30:59.835 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:30:59.835 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:30:59.836 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:30:59.836 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:30:59.836 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:30:59.836 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:30:59.836 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:30:59.836 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:30:59.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:30:59.837 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:30:59.837 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:30:59.837 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:30:59.837 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:30:59.837 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:30:59.854 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:30:59.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:30:59.856 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:30:59.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:30:59.857 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:30:59.858 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:30:59.860 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:30:59.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:30:59.903 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:30:59.903 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:59.907 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:30:59.907 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:59.908 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:30:59.908 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:59.912 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:30:59.912 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:59.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:31:00.011 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:31:00.013 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:31:00.016 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:31:00.017 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:31:00.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 Processor -- start 2023-10-06T11:31:00.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 -- start start 2023-10-06T11:31:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47cc158e70 0x7f47cc0abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc067760 0x7f47cc15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc0abdb0 con 0x7f47cc159240 2023-10-06T11:31:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc15dc00 con 0x7f47cc158e70 2023-10-06T11:31:00.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47d16f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc15d360 con 0x7f47cc067760 2023-10-06T11:31:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47caffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47cc158e70 0x7f47cc0abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47caffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47cc158e70 0x7f47cc0abfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:39772/0 (socket says 172.21.15.46:39772) 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47caffd700 1 -- 172.21.15.46:0/2832980203 learned_addr learned my addr 172.21.15.46:0/2832980203 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.070+0000 7f47cb7fe700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc067760 0x7f47cc15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47caffd700 1 -- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc067760 msgr2=0x7f47cc15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47caffd700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc067760 0x7f47cc15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47caffd700 1 -- 172.21.15.46:0/2832980203 >> v1:172.21.15.132:6789/0 conn(0x7f47cc159240 legacy=0x7f47cc067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47caffd700 1 -- 172.21.15.46:0/2832980203 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc15d640 con 0x7f47cc158e70 2023-10-06T11:31:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47caffd700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc158e70 0x7f47cc0abfd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0099b0 tx=0x7f47bc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97723711395dd18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47c9ffb700 1 -- 172.21.15.46:0/2832980203 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc009ea0 con 0x7f47cc158e70 2023-10-06T11:31:00.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47c9ffb700 1 -- 172.21.15.46:0/2832980203 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc00b860 con 0x7f47cc158e70 2023-10-06T11:31:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47c9ffb700 1 -- 172.21.15.46:0/2832980203 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00bc20 con 0x7f47cc158e70 2023-10-06T11:31:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 -- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc158e70 msgr2=0x7f47cc0abfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc158e70 0x7f47cc0abfd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0099b0 tx=0x7f47bc009250 comp rx=0 tx=0).stop 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 -- 172.21.15.46:0/2832980203 shutdown_connections 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc067760 0x7f47cc15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 --2- 172.21.15.46:0/2832980203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc158e70 0x7f47cc0abfd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 -- 172.21.15.46:0/2832980203 >> 172.21.15.46:0/2832980203 conn(0x7f47cc0ccd50 msgr2=0x7f47cc0cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 -- 172.21.15.46:0/2832980203 shutdown_connections 2023-10-06T11:31:00.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.074+0000 7f47d16f0700 1 -- 172.21.15.46:0/2832980203 wait complete. 2023-10-06T11:31:00.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 Processor -- start 2023-10-06T11:31:00.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 -- start start 2023-10-06T11:31:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc158e70 0x7f47cc16a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc159240 0x7f47cc15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0c8b80 con 0x7f47cc067760 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc1540a0 con 0x7f47cc159240 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47d16f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc0ca270 con 0x7f47cc158e70 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53670/0 (socket says 172.21.15.46:53670) 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 -- 172.21.15.46:0/80625884 learned_addr learned my addr 172.21.15.46:0/80625884 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47cb7fe700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc159240 0x7f47cc15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47ca7fc700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc158e70 0x7f47cc16a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 -- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc158e70 msgr2=0x7f47cc16a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc158e70 0x7f47cc16a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 -- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc159240 msgr2=0x7f47cc15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc159240 0x7f47cc15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.078+0000 7f47caffd700 1 -- 172.21.15.46:0/80625884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0094f0 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47caffd700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006fd0 tx=0x7f47bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc01f070 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc000bc0 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc16ab80 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc025420 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc159ad0 con 0x7f47cc067760 2023-10-06T11:31:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.082+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc159cf0 con 0x7f47cc067760 2023-10-06T11:31:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.086+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f47bc023070 con 0x7f47cc067760 2023-10-06T11:31:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.086+0000 7f47b2ffd700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 0x7f47b8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.086+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f47bc07df80 con 0x7f47cc067760 2023-10-06T11:31:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.086+0000 7f47ca7fc700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 0x7f47b8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.086+0000 7f47ca7fc700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 0x7f47b8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47cc06aaf0 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.090+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f47bc04e230 con 0x7f47cc067760 2023-10-06T11:31:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.278+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f47cc064590 con 0x7f47cc067760 2023-10-06T11:31:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.886+0000 7f47b2ffd700 1 -- 172.21.15.46:0/80625884 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f47bc015080 con 0x7f47cc067760 2023-10-06T11:31:00.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.890+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 msgr2=0x7f47b8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.890+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 0x7f47b8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47cc06aaf0 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.890+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 msgr2=0x7f47cc153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.890+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006fd0 tx=0x7f47bc017000 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 shutdown_connections 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f47b8060670 0x7f47b8062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47cc159240 0x7f47cc15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47cc158e70 0x7f47cc16a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 --2- 172.21.15.46:0/80625884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47cc067760 0x7f47cc153b60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 >> 172.21.15.46:0/80625884 conn(0x7f47cc0ccd50 msgr2=0x7f47cc0a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 shutdown_connections 2023-10-06T11:31:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:00.894+0000 7f47d16f0700 1 -- 172.21.15.46:0/80625884 wait complete. 2023-10-06T11:31:00.904 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:31:00.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:31:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.082+0000 7fe245aae700 1 Processor -- start 2023-10-06T11:31:01.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.082+0000 7fe245aae700 1 -- start start 2023-10-06T11:31:01.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe245aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe24006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe245aae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24006d460 con 0x7fe24015af30 2023-10-06T11:31:01.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe245aae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400c7000 con 0x7fe2400ad6c0 2023-10-06T11:31:01.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe245aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c72c0 con 0x7fe24015ab60 2023-10-06T11:31:01.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe237fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe24015af30 0x7fe24006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52934/0 (socket says 172.21.15.46:52934) 2023-10-06T11:31:01.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe237fff700 1 -- 172.21.15.46:0/1409597571 learned_addr learned my addr 172.21.15.46:0/1409597571 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe24006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777507999 0 0) 0x7fe24006d460 con 0x7fe24015af30 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c003570 con 0x7fe24015af30 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902684533 0 0) 0x7fe2400c7000 con 0x7fe2400ad6c0 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24006d460 con 0x7fe2400ad6c0 2023-10-06T11:31:01.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836633265 0 0) 0x7fe22c003570 con 0x7fe24015af30 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 msgr2=0x7fe24006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe24006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 >> v1:172.21.15.132:6789/0 conn(0x7fe2400ad6c0 legacy=0x7fe2400adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400c75a0 con 0x7fe24015af30 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fe228001490 con 0x7fe24015af30 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 >> v1:172.21.15.46:6789/0 conn(0x7fe24015af30 legacy=0x7fe24006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe22c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe22c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe22c004d50 0x7fe22c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c72c0 con 0x7fe22c003c20 2023-10-06T11:31:01.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c00bb40 con 0x7fe22c004d50 2023-10-06T11:31:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c010110 con 0x7fe24015ab60 2023-10-06T11:31:01.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe22c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe237fff700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe22c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23ffff700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe22c004d50 0x7fe22c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 msgr2=0x7fe22c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.086+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24015ab60 0x7fe22c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe22c004d50 msgr2=0x7fe22c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe22c004d50 0x7fe22c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1409597571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c003570 con 0x7fe22c003c20 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23ffff700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe22c004d50 0x7fe22c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe22c004010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe230009bf0 tx=0x7fe230009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906924ee8c4d181d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230005a00 con 0x7fe22c003c20 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe230005b60 con 0x7fe22c003c20 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23effd700 1 -- 172.21.15.46:0/1409597571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000e710 con 0x7fe22c003c20 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 -- 172.21.15.46:0/1409597571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 msgr2=0x7fe22c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 --2- 172.21.15.46:0/1409597571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe22c004010 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe230009bf0 tx=0x7fe230009120 comp rx=0 tx=0).stop 2023-10-06T11:31:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe23ffff700 1 -- 172.21.15.46:0/1409597571 reap_dead start 2023-10-06T11:31:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 -- 172.21.15.46:0/1409597571 shutdown_connections 2023-10-06T11:31:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 -- 172.21.15.46:0/1409597571 >> 172.21.15.46:0/1409597571 conn(0x7fe2400cbe30 msgr2=0x7fe2400cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 -- 172.21.15.46:0/1409597571 shutdown_connections 2023-10-06T11:31:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.090+0000 7fe245aae700 1 -- 172.21.15.46:0/1409597571 wait complete. 2023-10-06T11:31:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 Processor -- start 2023-10-06T11:31:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 -- start start 2023-10-06T11:31:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2400b1150 0x7fe2400b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 0x7fe2400c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c3890 con 0x7fe22c003c20 2023-10-06T11:31:01.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c5320 con 0x7fe2400c2ee0 2023-10-06T11:31:01.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.094+0000 7fe245aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400c4780 con 0x7fe2400b1150 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53694/0 (socket says 172.21.15.46:53694) 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1002255088 learned_addr learned my addr 172.21.15.46:0/1002255088 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe237fff700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2400b1150 0x7fe2400b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23ffff700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 0x7fe2400c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2400b1150 msgr2=0x7fe2400b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2400b1150 0x7fe2400b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 msgr2=0x7fe2400c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 0x7fe2400c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 -- 172.21.15.46:0/1002255088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400c49f0 con 0x7fe22c003c20 2023-10-06T11:31:01.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23ffff700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 0x7fe2400c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe23f7fe700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe23000f000 tx=0x7fe230019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230005a00 con 0x7fe22c003c20 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe230006240 con 0x7fe22c003c20 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2400c4cd0 con 0x7fe22c003c20 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230006600 con 0x7fe22c003c20 2023-10-06T11:31:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe240166130 con 0x7fe22c003c20 2023-10-06T11:31:01.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.098+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240063f90 con 0x7fe22c003c20 2023-10-06T11:31:01.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.102+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe230013470 con 0x7fe22c003c20 2023-10-06T11:31:01.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.102+0000 7fe2377fe700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 0x7fe220062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.102+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fe23007e080 con 0x7fe22c003c20 2023-10-06T11:31:01.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.102+0000 7fe237fff700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 0x7fe220062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.102+0000 7fe237fff700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 0x7fe220062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe240158400 tx=0x7fe228007040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.110+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe23004e650 con 0x7fe22c003c20 2023-10-06T11:31:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.338+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2400c3890 con 0x7fe22c003c20 2023-10-06T11:31:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.338+0000 7fe2377fe700 1 -- 172.21.15.46:0/1002255088 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe230051bb0 con 0x7fe22c003c20 2023-10-06T11:31:01.348 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:01.348 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:31:01.348 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.349 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:31:01.350 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:31:01.351 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:31:01.353 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.354 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.355 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:31:01.358 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:31:01.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.361 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:31:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:31:01.363 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:31:01.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:01.363 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:31:01.363 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 msgr2=0x7fe220062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 0x7fe220062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe240158400 tx=0x7fe228007040 comp rx=0 tx=0).stop 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 msgr2=0x7fe2400b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe23000f000 tx=0x7fe230019000 comp rx=0 tx=0).stop 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 shutdown_connections 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe220060670 0x7fe220062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2400c2ee0 0x7fe2400c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2400b1150 0x7fe2400b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 --2- 172.21.15.46:0/1002255088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe22c003c20 0x7fe2400b0450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 >> 172.21.15.46:0/1002255088 conn(0x7fe2400cbe30 msgr2=0x7fe240069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 shutdown_connections 2023-10-06T11:31:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.342+0000 7fe245aae700 1 -- 172.21.15.46:0/1002255088 wait complete. 2023-10-06T11:31:01.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:31:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.518+0000 7f7b316ae700 1 Processor -- start 2023-10-06T11:31:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.518+0000 7f7b316ae700 1 -- start start 2023-10-06T11:31:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b316ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b2c156c80 0x7f7b2c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b316ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7fc0 0x7f7b2c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b316ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c0c6470 con 0x7f7b2c0c7b50 2023-10-06T11:31:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b316ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c0c7760 con 0x7f7b2c156c80 2023-10-06T11:31:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b316ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0ca710 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2a7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7b2c0c7b50 0x7f7b2c15cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52942/0 (socket says 172.21.15.46:52942) 2023-10-06T11:31:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2a7fc700 1 -- 172.21.15.46:0/262281419 learned_addr learned my addr 172.21.15.46:0/262281419 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2affd700 1 --2- 172.21.15.46:0/262281419 >> v2:172.21.15.132:3300/0 conn(0x7f7b2c156c80 0x7f7b2c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2b7fe700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7fc0 0x7f7b2c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b29ffb700 1 -- 172.21.15.46:0/262281419 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844662397 0 0) 0x7f7b2c0c6470 con 0x7f7b2c0c7b50 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b29ffb700 1 -- 172.21.15.46:0/262281419 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b14003570 con 0x7f7b2c0c7b50 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2affd700 1 -- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7fc0 msgr2=0x7f7b2c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2affd700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7fc0 0x7f7b2c15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2affd700 1 -- 172.21.15.46:0/262281419 >> v1:172.21.15.46:6789/0 conn(0x7f7b2c0c7b50 legacy=0x7f7b2c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.522+0000 7f7b2affd700 1 -- 172.21.15.46:0/262281419 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0c66c0 con 0x7f7b2c156c80 2023-10-06T11:31:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b2affd700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0ca1d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0098d0 tx=0x7f7b1c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4daa14b4fbc3f195 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b29ffb700 1 -- 172.21.15.46:0/262281419 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c005c10 con 0x7f7b2c156c80 2023-10-06T11:31:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b29ffb700 1 -- 172.21.15.46:0/262281419 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c005d70 con 0x7f7b2c156c80 2023-10-06T11:31:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b29ffb700 1 -- 172.21.15.46:0/262281419 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c006260 con 0x7f7b2c156c80 2023-10-06T11:31:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 -- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 msgr2=0x7f7b2c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0ca1d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0098d0 tx=0x7f7b1c009120 comp rx=0 tx=0).stop 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 -- 172.21.15.46:0/262281419 shutdown_connections 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7fc0 0x7f7b2c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 --2- 172.21.15.46:0/262281419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0ca1d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 -- 172.21.15.46:0/262281419 >> 172.21.15.46:0/262281419 conn(0x7f7b2c0cbe30 msgr2=0x7f7b2c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 -- 172.21.15.46:0/262281419 shutdown_connections 2023-10-06T11:31:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.526+0000 7f7b316ae700 1 -- 172.21.15.46:0/262281419 wait complete. 2023-10-06T11:31:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 Processor -- start 2023-10-06T11:31:01.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 -- start start 2023-10-06T11:31:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7b50 0x7f7b2c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b1200 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b05f0 con 0x7f7b2c156c80 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b316ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b08d0 con 0x7f7b2c0c7b50 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53704/0 (socket says 172.21.15.46:53704) 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b2a7fc700 1 -- 172.21.15.46:0/2060809400 learned_addr learned my addr 172.21.15.46:0/2060809400 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b2affd700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7b50 0x7f7b2c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.530+0000 7f7b2b7fe700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 -- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7b50 msgr2=0x7f7b2c0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7b50 0x7f7b2c0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 -- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 msgr2=0x7f7b2c0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 -- 172.21.15.46:0/2060809400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0093c0 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b2a7fc700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7b200011e0 tx=0x7f7b20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20005b30 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20005c90 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20007300 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c0b0bb0 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c154600 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.534+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c063f90 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.538+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7b20007670 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.538+0000 7f7b12ffd700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 0x7f7b18062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.538+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f7b2007bd70 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.538+0000 7f7b2affd700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 0x7f7b18062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.542+0000 7f7b2affd700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 0x7f7b18062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009160 tx=0x7f7b1c016000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.546+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b2004c1e0 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.714+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b2c04b5c0 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.718+0000 7f7b12ffd700 1 -- 172.21.15.46:0/2060809400 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+14321 (secure 0 0 0) 0x7f7b2004f800 con 0x7f7b2c0c7fc0 2023-10-06T11:31:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:01.729 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":58,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:31:00.901215+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:31:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 msgr2=0x7f7b18062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 0x7f7b18062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009160 tx=0x7f7b1c016000 comp rx=0 tx=0).stop 2023-10-06T11:31:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 msgr2=0x7f7b2c0ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7b200011e0 tx=0x7f7b20007df0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 shutdown_connections 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7b18060670 0x7f7b18062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b2c156c80 0x7f7b2c0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b2c0c7fc0 0x7f7b2c0ab6f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 --2- 172.21.15.46:0/2060809400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b2c0c7b50 0x7f7b2c0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 >> 172.21.15.46:0/2060809400 conn(0x7f7b2c0cbe30 msgr2=0x7f7b2c155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 shutdown_connections 2023-10-06T11:31:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.722+0000 7f7b316ae700 1 -- 172.21.15.46:0/2060809400 wait complete. 2023-10-06T11:31:01.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:31:01.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:31:01.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.902+0000 7efda02ae700 1 Processor -- start 2023-10-06T11:31:01.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.902+0000 7efda02ae700 1 -- start start 2023-10-06T11:31:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efd980b23c0 0x7efd980b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980cae40 0x7efd980cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd980ccb40 0x7efd980ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd980cd500 con 0x7efd980ccb40 2023-10-06T11:31:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd980cc510 con 0x7efd980b23c0 2023-10-06T11:31:01.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efda02ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd980cb900 con 0x7efd980cae40 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd980ccb40 0x7efd980ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd980ccb40 0x7efd980ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53708/0 (socket says 172.21.15.46:53708) 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 -- 172.21.15.46:0/86187621 learned_addr learned my addr 172.21.15.46:0/86187621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9d849700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980cae40 0x7efd980cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e04a700 1 --2- 172.21.15.46:0/86187621 >> v2:172.21.15.132:3300/0 conn(0x7efd980b23c0 0x7efd980b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 -- 172.21.15.46:0/86187621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980cae40 msgr2=0x7efd980cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980cae40 0x7efd980cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 -- 172.21.15.46:0/86187621 >> v2:172.21.15.132:3300/0 conn(0x7efd980b23c0 msgr2=0x7efd980b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 --2- 172.21.15.46:0/86187621 >> v2:172.21.15.132:3300/0 conn(0x7efd980b23c0 0x7efd980b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 -- 172.21.15.46:0/86187621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd980cbbe0 con 0x7efd980ccb40 2023-10-06T11:31:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9e84b700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980ccb40 0x7efd980ccfc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efd9806a030 tx=0x7efd90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020ff994c5830ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9d048700 1 -- 172.21.15.46:0/86187621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9000e070 con 0x7efd980ccb40 2023-10-06T11:31:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9d048700 1 -- 172.21.15.46:0/86187621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9000b970 con 0x7efd980ccb40 2023-10-06T11:31:01.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.906+0000 7efd9d048700 1 -- 172.21.15.46:0/86187621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd900092e0 con 0x7efd980ccb40 2023-10-06T11:31:01.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- 172.21.15.46:0/86187621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980ccb40 msgr2=0x7efd980ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980ccb40 0x7efd980ccfc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efd9806a030 tx=0x7efd90009840 comp rx=0 tx=0).stop 2023-10-06T11:31:01.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- 172.21.15.46:0/86187621 shutdown_connections 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980ccb40 0x7efd980ccfc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 --2- 172.21.15.46:0/86187621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980cae40 0x7efd980cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 --2- 172.21.15.46:0/86187621 >> v2:172.21.15.132:3300/0 conn(0x7efd980b23c0 0x7efd980b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- 172.21.15.46:0/86187621 >> 172.21.15.46:0/86187621 conn(0x7efd98099480 msgr2=0x7efd9806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- 172.21.15.46:0/86187621 shutdown_connections 2023-10-06T11:31:01.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- 172.21.15.46:0/86187621 wait complete. 2023-10-06T11:31:01.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 Processor -- start 2023-10-06T11:31:01.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.910+0000 7efda02ae700 1 -- start start 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 0x7efd980c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 0x7efd9815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980c96c0 con 0x7efd980cae40 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815a9c0 con 0x7efd980ccb40 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815aca0 con 0x7efd980b23c0 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53712/0 (socket says 172.21.15.46:53712) 2023-10-06T11:31:01.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 -- 172.21.15.46:0/3390608658 learned_addr learned my addr 172.21.15.46:0/3390608658 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9e04a700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 0x7efd980c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9e84b700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 0x7efd9815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 -- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 msgr2=0x7efd980c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 0x7efd980c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 -- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 msgr2=0x7efd9815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 0x7efd9815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 -- 172.21.15.46:0/3390608658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd90009ae0 con 0x7efd980cae40 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9e04a700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 0x7efd980c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9e84b700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 0x7efd9815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd9d849700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efd8800ec80 tx=0x7efd880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8800c9d0 con 0x7efd980cae40 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8800cb30 con 0x7efd980cae40 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88013470 con 0x7efd980cae40 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.914+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9815af80 con 0x7efd980cae40 2023-10-06T11:31:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.918+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9815b540 con 0x7efd980cae40 2023-10-06T11:31:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.918+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd980660f0 con 0x7efd980cae40 2023-10-06T11:31:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.918+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7efd88013820 con 0x7efd980cae40 2023-10-06T11:31:01.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.922+0000 7efd8e7fc700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 0x7efd7c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:01.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.922+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7efd8807cd60 con 0x7efd980cae40 2023-10-06T11:31:01.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.922+0000 7efd9e04a700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 0x7efd7c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:01.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.922+0000 7efd9e04a700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 0x7efd7c062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd94009cd0 tx=0x7efd94008040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:01.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:01.926+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efd8804d1d0 con 0x7efd980cae40 2023-10-06T11:31:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.098+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd980cbec0 con 0x7efd980cae40 2023-10-06T11:31:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.098+0000 7efd8e7fc700 1 -- 172.21.15.46:0/3390608658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd88054e60 con 0x7efd980cae40 2023-10-06T11:31:02.109 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:31:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 msgr2=0x7efd7c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 0x7efd7c062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd94009cd0 tx=0x7efd94008040 comp rx=0 tx=0).stop 2023-10-06T11:31:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 msgr2=0x7efd980d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efd8800ec80 tx=0x7efd880091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 shutdown_connections 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efd7c060670 0x7efd7c062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980ccb40 0x7efd9815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd980cae40 0x7efd980d0200 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 --2- 172.21.15.46:0/3390608658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980b23c0 0x7efd980c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 >> 172.21.15.46:0/3390608658 conn(0x7efd98099480 msgr2=0x7efd98157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 shutdown_connections 2023-10-06T11:31:02.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.102+0000 7efda02ae700 1 -- 172.21.15.46:0/3390608658 wait complete. 2023-10-06T11:31:02.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.290+0000 7f7eac785700 1 Processor -- start 2023-10-06T11:31:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.290+0000 7f7eac785700 1 -- start start 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 0x7f7ea40ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 0x7f7ea40c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea406ed10 0x7f7ea40c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40c1bf0 con 0x7f7ea406ed10 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40c72c0 con 0x7f7ea40c6470 2023-10-06T11:31:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eac785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40abbe0 con 0x7f7ea406e8a0 2023-10-06T11:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea406ed10 0x7f7ea40c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea406ed10 0x7f7ea40c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53728/0 (socket says 172.21.15.46:53728) 2023-10-06T11:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 -- 172.21.15.46:0/376007702 learned_addr learned my addr 172.21.15.46:0/376007702 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7ea9d20700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 0x7f7ea40c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaa521700 1 --2- 172.21.15.46:0/376007702 >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 0x7f7ea40ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 -- 172.21.15.46:0/376007702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 msgr2=0x7f7ea40c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 0x7f7ea40c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 -- 172.21.15.46:0/376007702 >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 msgr2=0x7f7ea40ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 --2- 172.21.15.46:0/376007702 >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 0x7f7ea40ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 -- 172.21.15.46:0/376007702 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea406cf80 con 0x7f7ea406ed10 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7ea9d20700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 0x7f7ea40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaa521700 1 --2- 172.21.15.46:0/376007702 >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 0x7f7ea40ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7eaad22700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea40c6d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40c2e90 tx=0x7f7e9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a3d331ec7016e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7ea951f700 1 -- 172.21.15.46:0/376007702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00e070 con 0x7f7ea406ed10 2023-10-06T11:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7ea951f700 1 -- 172.21.15.46:0/376007702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00ba50 con 0x7f7ea406ed10 2023-10-06T11:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.294+0000 7f7ea951f700 1 -- 172.21.15.46:0/376007702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0092e0 con 0x7f7ea406ed10 2023-10-06T11:31:02.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- 172.21.15.46:0/376007702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 msgr2=0x7f7ea40c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea40c6d80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40c2e90 tx=0x7f7e9c009840 comp rx=0 tx=0).stop 2023-10-06T11:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- 172.21.15.46:0/376007702 shutdown_connections 2023-10-06T11:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea40c6d80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 --2- 172.21.15.46:0/376007702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea406e8a0 0x7f7ea40c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 --2- 172.21.15.46:0/376007702 >> v2:172.21.15.132:3300/0 conn(0x7f7ea40c6470 0x7f7ea40ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- 172.21.15.46:0/376007702 >> 172.21.15.46:0/376007702 conn(0x7f7ea40cbe30 msgr2=0x7f7ea40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:02.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- 172.21.15.46:0/376007702 shutdown_connections 2023-10-06T11:31:02.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- 172.21.15.46:0/376007702 wait complete. 2023-10-06T11:31:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 Processor -- start 2023-10-06T11:31:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.298+0000 7f7eac785700 1 -- start start 2023-10-06T11:31:02.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ea406e8a0 0x7f7ea415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 0x7f7ea40af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40a98e0 con 0x7f7ea406ed10 2023-10-06T11:31:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40a9bc0 con 0x7f7ea406e8a0 2023-10-06T11:31:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40a9ea0 con 0x7f7ea40c6470 2023-10-06T11:31:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53742/0 (socket says 172.21.15.46:53742) 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 -- 172.21.15.46:0/1109798271 learned_addr learned my addr 172.21.15.46:0/1109798271 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eaa521700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ea406e8a0 0x7f7ea415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eaad22700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 0x7f7ea40af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 -- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 msgr2=0x7f7ea40af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 0x7f7ea40af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 -- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ea406e8a0 msgr2=0x7f7ea415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ea406e8a0 0x7f7ea415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 -- 172.21.15.46:0/1109798271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c009ae0 con 0x7f7ea406ed10 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eaad22700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 0x7f7ea40af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:02.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7ea9d20700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d860 tx=0x7f7e94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940125f0 con 0x7f7ea406ed10 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94012750 con 0x7f7ea406ed10 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40aa180 con 0x7f7ea406ed10 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94012af0 con 0x7f7ea406ed10 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.302+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea415b480 con 0x7f7ea406ed10 2023-10-06T11:31:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.306+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea40660f0 con 0x7f7ea406ed10 2023-10-06T11:31:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.306+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7e94014430 con 0x7f7ea406ed10 2023-10-06T11:31:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.306+0000 7f7e9a7fc700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 0x7f7e88062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.306+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f7e9407cd60 con 0x7f7ea406ed10 2023-10-06T11:31:02.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.310+0000 7f7eaa521700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 0x7f7e88062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.310+0000 7f7eaa521700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 0x7f7e88062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.314+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7e9404d1d0 con 0x7f7ea406ed10 2023-10-06T11:31:02.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7ea40a9a20 con 0x7f7ea406ed10 2023-10-06T11:31:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7e9a7fc700 1 -- 172.21.15.46:0/1109798271 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v58) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ea40a9a20 con 0x7f7ea406ed10 2023-10-06T11:31:02.495 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 msgr2=0x7f7e88062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 0x7f7e88062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).stop 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 msgr2=0x7f7ea415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d860 tx=0x7f7e94008040 comp rx=0 tx=0).stop 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 shutdown_connections 2023-10-06T11:31:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e88060670 0x7f7e88062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ea40c6470 0x7f7ea40af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea406ed10 0x7f7ea415a2c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 --2- 172.21.15.46:0/1109798271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ea406e8a0 0x7f7ea415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.486+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 >> 172.21.15.46:0/1109798271 conn(0x7f7ea40cbe30 msgr2=0x7f7ea40addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.490+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 shutdown_connections 2023-10-06T11:31:02.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.490+0000 7f7eac785700 1 -- 172.21.15.46:0/1109798271 wait complete. 2023-10-06T11:31:02.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:31:02.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.670+0000 7fbed308e700 1 Processor -- start 2023-10-06T11:31:02.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.670+0000 7fbed308e700 1 -- start start 2023-10-06T11:31:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed308e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbecc158dd0 0x7fbecc067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed308e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc067590 0x7fbecc069300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed308e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc15d6c0 con 0x7fbecc1591a0 2023-10-06T11:31:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed308e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0699c0 con 0x7fbecc158dd0 2023-10-06T11:31:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed308e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15db50 con 0x7fbecc067590 2023-10-06T11:31:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbecc158dd0 0x7fbecc067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed188b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbecc1591a0 0x7fbecc068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52946/0 (socket says 172.21.15.46:52946) 2023-10-06T11:31:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbecc158dd0 0x7fbecc067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53748/0 (socket says 172.21.15.46:53748) 2023-10-06T11:31:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed188b700 1 -- 172.21.15.46:0/2107745782 learned_addr learned my addr 172.21.15.46:0/2107745782 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed288d700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc067590 0x7fbecc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 -- 172.21.15.46:0/2107745782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc067590 msgr2=0x7fbecc069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc067590 0x7fbecc069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 -- 172.21.15.46:0/2107745782 >> v1:172.21.15.46:6789/0 conn(0x7fbecc1591a0 legacy=0x7fbecc068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.674+0000 7fbed208c700 1 -- 172.21.15.46:0/2107745782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc15cd90 con 0x7fbecc158dd0 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed208c700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc158dd0 0x7fbecc067050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009910 tx=0x7fbec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca0408bc75aab7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed108a700 1 -- 172.21.15.46:0/2107745782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800b820 con 0x7fbecc158dd0 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed108a700 1 -- 172.21.15.46:0/2107745782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800b980 con 0x7fbecc158dd0 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed108a700 1 -- 172.21.15.46:0/2107745782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800bd20 con 0x7fbecc158dd0 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 -- 172.21.15.46:0/2107745782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc158dd0 msgr2=0x7fbecc067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc158dd0 0x7fbecc067050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009910 tx=0x7fbec80091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 -- 172.21.15.46:0/2107745782 shutdown_connections 2023-10-06T11:31:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc067590 0x7fbecc069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 --2- 172.21.15.46:0/2107745782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc158dd0 0x7fbecc067050 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 -- 172.21.15.46:0/2107745782 >> 172.21.15.46:0/2107745782 conn(0x7fbecc062ef0 msgr2=0x7fbecc0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 -- 172.21.15.46:0/2107745782 shutdown_connections 2023-10-06T11:31:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.678+0000 7fbed308e700 1 -- 172.21.15.46:0/2107745782 wait complete. 2023-10-06T11:31:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 Processor -- start 2023-10-06T11:31:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- start start 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc158dd0 0x7fbecc14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbecc1591a0 0x7fbecc16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc157500 con 0x7fbecc067590 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15a7b0 con 0x7fbecc1591a0 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc15aa90 con 0x7fbecc158dd0 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53754/0 (socket says 172.21.15.46:53754) 2023-10-06T11:31:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 -- 172.21.15.46:0/4261790715 learned_addr learned my addr 172.21.15.46:0/4261790715 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed288d700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbecc1591a0 0x7fbecc16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed188b700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc158dd0 0x7fbecc14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 -- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc158dd0 msgr2=0x7fbecc14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc158dd0 0x7fbecc14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 -- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbecc1591a0 msgr2=0x7fbecc16d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbecc1591a0 0x7fbecc16d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 -- 172.21.15.46:0/4261790715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009420 con 0x7fbecc067590 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed208c700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbecc06ab70 tx=0x7fbec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8011e90 con 0x7fbecc067590 2023-10-06T11:31:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec801a420 con 0x7fbecc067590 2023-10-06T11:31:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec801a7e0 con 0x7fbecc067590 2023-10-06T11:31:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc15ad70 con 0x7fbecc067590 2023-10-06T11:31:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.682+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc15b330 con 0x7fbecc067590 2023-10-06T11:31:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.686+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc15b550 con 0x7fbecc067590 2023-10-06T11:31:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.686+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbec801ac10 con 0x7fbecc067590 2023-10-06T11:31:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.686+0000 7fbec27fc700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 0x7fbeb0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.690+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fbec807e340 con 0x7fbecc067590 2023-10-06T11:31:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.690+0000 7fbed188b700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 0x7fbeb0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.690+0000 7fbed188b700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 0x7fbeb0062b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001780 tx=0x7fbebc009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:02.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.698+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbec804e7b0 con 0x7fbecc067590 2023-10-06T11:31:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbecc064590 con 0x7fbecc067590 2023-10-06T11:31:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbec27fc700 1 -- 172.21.15.46:0/4261790715 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbec8051dd0 con 0x7fbecc067590 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:31:02.880 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:31:02.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:02.881 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:31:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 msgr2=0x7fbeb0062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 0x7fbeb0062b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001780 tx=0x7fbebc009040 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 msgr2=0x7fbecc15c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.870+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbecc06ab70 tx=0x7fbec8017000 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 shutdown_connections 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbeb00606a0 0x7fbeb0062b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbecc1591a0 0x7fbecc16d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbecc158dd0 0x7fbecc14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 --2- 172.21.15.46:0/4261790715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbecc067590 0x7fbecc15c260 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 >> 172.21.15.46:0/4261790715 conn(0x7fbecc062ef0 msgr2=0x7fbecc158050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 shutdown_connections 2023-10-06T11:31:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:02.874+0000 7fbed308e700 1 -- 172.21.15.46:0/4261790715 wait complete. 2023-10-06T11:31:02.903 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:31:02.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:31:03.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.062+0000 7fe343ef0700 1 Processor -- start 2023-10-06T11:31:03.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.062+0000 7fe343ef0700 1 -- start start 2023-10-06T11:31:03.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe343ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 0x7fe33c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe343ef0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe33c1596a0 0x7fe33c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe343ef0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33c0c5db0 con 0x7fe33c0c6780 2023-10-06T11:31:03.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe343ef0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33c0c5ab0 con 0x7fe33c1596a0 2023-10-06T11:31:03.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe343ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0ad2f0 con 0x7fe33c1592d0 2023-10-06T11:31:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe34248d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe33c0c6780 0x7fe33c0cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52952/0 (socket says 172.21.15.46:52952) 2023-10-06T11:31:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe34248d700 1 -- 172.21.15.46:0/2029864947 learned_addr learned my addr 172.21.15.46:0/2029864947 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe341c8c700 1 --2- 172.21.15.46:0/2029864947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 0x7fe33c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe34148b700 1 --2- 172.21.15.46:0/2029864947 >> v2:172.21.15.132:3300/0 conn(0x7fe33c1596a0 0x7fe33c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe340c8a700 1 -- 172.21.15.46:0/2029864947 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676245647 0 0) 0x7fe33c0c5db0 con 0x7fe33c0c6780 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe340c8a700 1 -- 172.21.15.46:0/2029864947 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe328003570 con 0x7fe33c0c6780 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe341c8c700 1 -- 172.21.15.46:0/2029864947 >> v2:172.21.15.132:3300/0 conn(0x7fe33c1596a0 msgr2=0x7fe33c0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe341c8c700 1 --2- 172.21.15.46:0/2029864947 >> v2:172.21.15.132:3300/0 conn(0x7fe33c1596a0 0x7fe33c0c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe341c8c700 1 -- 172.21.15.46:0/2029864947 >> v1:172.21.15.46:6789/0 conn(0x7fe33c0c6780 legacy=0x7fe33c0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.066+0000 7fe341c8c700 1 -- 172.21.15.46:0/2029864947 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0d0aa0 con 0x7fe33c1592d0 2023-10-06T11:31:03.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe341c8c700 1 --2- 172.21.15.46:0/2029864947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 0x7fe33c0acdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098d0 tx=0x7fe338009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946782e85deb347a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:03.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe340c8a700 1 -- 172.21.15.46:0/2029864947 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338005c10 con 0x7fe33c1592d0 2023-10-06T11:31:03.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe340c8a700 1 -- 172.21.15.46:0/2029864947 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338005d70 con 0x7fe33c1592d0 2023-10-06T11:31:03.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe340c8a700 1 -- 172.21.15.46:0/2029864947 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338006280 con 0x7fe33c1592d0 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 -- 172.21.15.46:0/2029864947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 msgr2=0x7fe33c0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 --2- 172.21.15.46:0/2029864947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 0x7fe33c0acdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098d0 tx=0x7fe338009120 comp rx=0 tx=0).stop 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 -- 172.21.15.46:0/2029864947 shutdown_connections 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 --2- 172.21.15.46:0/2029864947 >> v2:172.21.15.132:3300/0 conn(0x7fe33c1596a0 0x7fe33c0c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 --2- 172.21.15.46:0/2029864947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1592d0 0x7fe33c0acdb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 -- 172.21.15.46:0/2029864947 >> 172.21.15.46:0/2029864947 conn(0x7fe33c06ef60 msgr2=0x7fe33c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 -- 172.21.15.46:0/2029864947 shutdown_connections 2023-10-06T11:31:03.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 -- 172.21.15.46:0/2029864947 wait complete. 2023-10-06T11:31:03.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.070+0000 7fe343ef0700 1 Processor -- start 2023-10-06T11:31:03.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 -- start start 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe33c0c6780 0x7fe33c0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 0x7fe33c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15df40 con 0x7fe33c1592d0 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c0ca5a0 con 0x7fe33c0c6780 2023-10-06T11:31:03.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe343ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c15d330 con 0x7fe33c1596a0 2023-10-06T11:31:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34594/0 (socket says 172.21.15.46:34594) 2023-10-06T11:31:03.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 -- 172.21.15.46:0/1625016162 learned_addr learned my addr 172.21.15.46:0/1625016162 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe341c8c700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe33c0c6780 0x7fe33c0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34248d700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 0x7fe33c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 -- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 msgr2=0x7fe33c159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 0x7fe33c159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 -- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe33c0c6780 msgr2=0x7fe33c0c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe33c0c6780 0x7fe33c0c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34148b700 1 -- 172.21.15.46:0/1625016162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380093c0 con 0x7fe33c1592d0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.074+0000 7fe34248d700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 0x7fe33c159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe34148b700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00ea00 tx=0x7fe32c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00c850 con 0x7fe33c1592d0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c00c9b0 con 0x7fe33c1592d0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c15d610 con 0x7fe33c1592d0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00cd30 con 0x7fe33c1592d0 2023-10-06T11:31:03.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c15dbd0 con 0x7fe33c1592d0 2023-10-06T11:31:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c15e0f0 con 0x7fe33c1592d0 2023-10-06T11:31:03.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.078+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe32c013680 con 0x7fe33c1592d0 2023-10-06T11:31:03.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.082+0000 7fe331ffb700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 0x7fe320062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:03.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.082+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fe32c07cff0 con 0x7fe33c1592d0 2023-10-06T11:31:03.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.082+0000 7fe341c8c700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 0x7fe320062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:03.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.082+0000 7fe341c8c700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 0x7fe320062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098d0 tx=0x7fe338016000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.086+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe32c0acbe0 con 0x7fe33c1592d0 2023-10-06T11:31:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.258+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe33c069e00 con 0x7fe33c1592d0 2023-10-06T11:31:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.914+0000 7fe331ffb700 1 -- 172.21.15.46:0/1625016162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe32c0170b0 con 0x7fe33c1592d0 2023-10-06T11:31:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.922+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 msgr2=0x7fe320062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.922+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 0x7fe320062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098d0 tx=0x7fe338016000 comp rx=0 tx=0).stop 2023-10-06T11:31:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.922+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 msgr2=0x7fe33c0ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:03.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.922+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00ea00 tx=0x7fe32c009120 comp rx=0 tx=0).stop 2023-10-06T11:31:03.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 shutdown_connections 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe320060670 0x7fe320062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe33c1596a0 0x7fe33c159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c1592d0 0x7fe33c0ca060 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 --2- 172.21.15.46:0/1625016162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe33c0c6780 0x7fe33c0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 >> 172.21.15.46:0/1625016162 conn(0x7fe33c06ef60 msgr2=0x7fe33c158300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 shutdown_connections 2023-10-06T11:31:03.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:03.926+0000 7fe343ef0700 1 -- 172.21.15.46:0/1625016162 wait complete. 2023-10-06T11:31:03.939 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:31:03.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:31:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 Processor -- start 2023-10-06T11:31:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 -- start start 2023-10-06T11:31:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f871406ed10 0x7f87140c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87140c1bf0 con 0x7f87140c6470 2023-10-06T11:31:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871406cf80 con 0x7f871406e8a0 2023-10-06T11:31:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.122+0000 7f8718c78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871406d260 con 0x7f871406ed10 2023-10-06T11:31:04.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87140c6470 0x7f87140ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49168/0 (socket says 172.21.15.46:49168) 2023-10-06T11:31:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871259c700 1 -- 172.21.15.46:0/2041141860 learned_addr learned my addr 172.21.15.46:0/2041141860 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8712d9d700 1 --2- 172.21.15.46:0/2041141860 >> v2:172.21.15.132:3300/0 conn(0x7f871406ed10 0x7f87140c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871159a700 1 -- 172.21.15.46:0/2041141860 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068036178 0 0) 0x7f87140c1bf0 con 0x7f87140c6470 2023-10-06T11:31:04.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871159a700 1 -- 172.21.15.46:0/2041141860 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc003570 con 0x7f87140c6470 2023-10-06T11:31:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8712d9d700 1 -- 172.21.15.46:0/2041141860 >> v1:172.21.15.132:6789/0 conn(0x7f871406e8a0 legacy=0x7f87140c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8712d9d700 1 -- 172.21.15.46:0/2041141860 >> v1:172.21.15.46:6789/0 conn(0x7f87140c6470 legacy=0x7f87140ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8712d9d700 1 -- 172.21.15.46:0/2041141860 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140c1880 con 0x7f871406ed10 2023-10-06T11:31:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8712d9d700 1 --2- 172.21.15.46:0/2041141860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87140c2080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f870c009e60 tx=0x7f870c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f951ec54fdc546a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871159a700 1 -- 172.21.15.46:0/2041141860 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00e070 con 0x7f871406ed10 2023-10-06T11:31:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871159a700 1 -- 172.21.15.46:0/2041141860 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00bb60 con 0x7f871406ed10 2023-10-06T11:31:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f871159a700 1 -- 172.21.15.46:0/2041141860 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c012430 con 0x7f871406ed10 2023-10-06T11:31:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8718c78700 1 -- 172.21.15.46:0/2041141860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 msgr2=0x7f87140c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.126+0000 7f8718c78700 1 --2- 172.21.15.46:0/2041141860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87140c2080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f870c009e60 tx=0x7f870c009780 comp rx=0 tx=0).stop 2023-10-06T11:31:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- 172.21.15.46:0/2041141860 shutdown_connections 2023-10-06T11:31:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 --2- 172.21.15.46:0/2041141860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87140c2080 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- 172.21.15.46:0/2041141860 >> 172.21.15.46:0/2041141860 conn(0x7f87140cbe30 msgr2=0x7f87140cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- 172.21.15.46:0/2041141860 shutdown_connections 2023-10-06T11:31:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- 172.21.15.46:0/2041141860 wait complete. 2023-10-06T11:31:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 Processor -- start 2023-10-06T11:31:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- start start 2023-10-06T11:31:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87141306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87140c6470 0x7f87140690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87140c5d30 con 0x7f871406e8a0 2023-10-06T11:31:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87140c54d0 con 0x7f871406ed10 2023-10-06T11:31:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.130+0000 7f8718c78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87140c5770 con 0x7f87140c6470 2023-10-06T11:31:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34608/0 (socket says 172.21.15.46:34608) 2023-10-06T11:31:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 -- 172.21.15.46:0/3416355066 learned_addr learned my addr 172.21.15.46:0/3416355066 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:04.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f8712d9d700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87140c6470 0x7f87140690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:04.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f8711d9b700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87141306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 -- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87140c6470 msgr2=0x7f87140690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87140c6470 0x7f87140690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 -- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 msgr2=0x7f87141306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87141306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 -- 172.21.15.46:0/3416355066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c009a20 con 0x7f871406e8a0 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f8711d9b700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87141306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f871259c700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f870400c590 tx=0x7f8704007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:04.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704005cd0 con 0x7f871406e8a0 2023-10-06T11:31:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704005e50 con 0x7f871406e8a0 2023-10-06T11:31:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87140c5a50 con 0x7f871406e8a0 2023-10-06T11:31:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704007320 con 0x7f871406e8a0 2023-10-06T11:31:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.134+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87140c3420 con 0x7f871406e8a0 2023-10-06T11:31:04.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.138+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87140660f0 con 0x7f871406e8a0 2023-10-06T11:31:04.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.138+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f870401b3b0 con 0x7f871406e8a0 2023-10-06T11:31:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.138+0000 7f87027fc700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 0x7f86f4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.138+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6020+0+0 (secure 0 0 0) 0x7f870407c1c0 con 0x7f871406e8a0 2023-10-06T11:31:04.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.138+0000 7f8711d9b700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 0x7f86f4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:04.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.142+0000 7f8711d9b700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 0x7f86f4062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8708007fd0 tx=0x7f8708009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.146+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f870404c4c0 con 0x7f871406e8a0 2023-10-06T11:31:04.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.322+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f871404b5c0 con 0x7f871406e8a0 2023-10-06T11:31:04.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.934+0000 7f87027fc700 1 -- 172.21.15.46:0/3416355066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f870404fae0 con 0x7f871406e8a0 2023-10-06T11:31:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.946+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 msgr2=0x7f86f4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.946+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 0x7f86f4062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8708007fd0 tx=0x7f8708009040 comp rx=0 tx=0).stop 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.946+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 msgr2=0x7f87140c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.946+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f870400c590 tx=0x7f8704007e00 comp rx=0 tx=0).stop 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 shutdown_connections 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86f4060670 0x7f86f4062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87140c6470 0x7f87140690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f871406ed10 0x7f87141306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 --2- 172.21.15.46:0/3416355066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f871406e8a0 0x7f87140c2ee0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 >> 172.21.15.46:0/3416355066 conn(0x7f87140cbe30 msgr2=0x7f87140aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.950+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 shutdown_connections 2023-10-06T11:31:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:04.954+0000 7f8718c78700 1 -- 172.21.15.46:0/3416355066 wait complete. 2023-10-06T11:31:04.973 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:31:04.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:31:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.146+0000 7fd862e6e700 1 Processor -- start 2023-10-06T11:31:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.146+0000 7fd862e6e700 1 -- start start 2023-10-06T11:31:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd862e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 0x7fd85c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd862e6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0cc510 con 0x7fd85c0b23c0 2023-10-06T11:31:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd862e6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0b2be0 con 0x7fd85c0cd080 2023-10-06T11:31:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd862e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0cd500 con 0x7fd85c0b2790 2023-10-06T11:31:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd860c0a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd85c0b23c0 0x7fd85c0cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49182/0 (socket says 172.21.15.46:49182) 2023-10-06T11:31:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd860c0a700 1 -- 172.21.15.46:0/1174220837 learned_addr learned my addr 172.21.15.46:0/1174220837 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.150+0000 7fd85bfff700 1 --2- 172.21.15.46:0/1174220837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 0x7fd85c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084658194 0 0) 0x7fd85c0cc510 con 0x7fd85c0b23c0 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84c003570 con 0x7fd85c0b23c0 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878972401 0 0) 0x7fd85c0b2be0 con 0x7fd85c0cd080 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd85c0cc510 con 0x7fd85c0cd080 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85bfff700 1 -- 172.21.15.46:0/1174220837 >> v1:172.21.15.132:6789/0 conn(0x7fd85c0cd080 legacy=0x7fd85c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85bfff700 1 -- 172.21.15.46:0/1174220837 >> v1:172.21.15.46:6789/0 conn(0x7fd85c0b23c0 legacy=0x7fd85c0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573752824 0 0) 0x7fd84c003570 con 0x7fd85c0b23c0 2023-10-06T11:31:05.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85bfff700 1 -- 172.21.15.46:0/1174220837 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0cb900 con 0x7fd85c0b2790 2023-10-06T11:31:05.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85bfff700 1 --2- 172.21.15.46:0/1174220837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 0x7fd85c0ccb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd848009910 tx=0x7fd8480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f16b87ffcfb2621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:05.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500015f0 con 0x7fd85c0b2790 2023-10-06T11:31:05.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848011420 con 0x7fd85c0b2790 2023-10-06T11:31:05.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd85b7fe700 1 -- 172.21.15.46:0/1174220837 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480117c0 con 0x7fd85c0b2790 2023-10-06T11:31:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 -- 172.21.15.46:0/1174220837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 msgr2=0x7fd85c0ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 --2- 172.21.15.46:0/1174220837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 0x7fd85c0ccb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd848009910 tx=0x7fd8480091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 -- 172.21.15.46:0/1174220837 shutdown_connections 2023-10-06T11:31:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 --2- 172.21.15.46:0/1174220837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0b2790 0x7fd85c0ccb40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 -- 172.21.15.46:0/1174220837 >> 172.21.15.46:0/1174220837 conn(0x7fd85c099480 msgr2=0x7fd85c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 -- 172.21.15.46:0/1174220837 shutdown_connections 2023-10-06T11:31:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.154+0000 7fd862e6e700 1 -- 172.21.15.46:0/1174220837 wait complete. 2023-10-06T11:31:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 Processor -- start 2023-10-06T11:31:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 -- start start 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd85c0b23c0 0x7fd85c0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0cd080 0x7fd85c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c15a320 con 0x7fd85c0b2790 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c16e1a0 con 0x7fd85c0b23c0 2023-10-06T11:31:05.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd862e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c1599f0 con 0x7fd85c0cd080 2023-10-06T11:31:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34610/0 (socket says 172.21.15.46:34610) 2023-10-06T11:31:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd85bfff700 1 -- 172.21.15.46:0/2772470296 learned_addr learned my addr 172.21.15.46:0/2772470296 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd860c0a700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd85c0b23c0 0x7fd85c0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.158+0000 7fd86140b700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0cd080 0x7fd85c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 -- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0cd080 msgr2=0x7fd85c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0cd080 0x7fd85c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 -- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd85c0b23c0 msgr2=0x7fd85c0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd85c0b23c0 0x7fd85c0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 -- 172.21.15.46:0/2772470296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009420 con 0x7fd85c0b2790 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd85bfff700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd8480030c0 tx=0x7fd848016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84801ccb0 con 0x7fd85c0b2790 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84801ce10 con 0x7fd85c0b2790 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84801a6a0 con 0x7fd85c0b2790 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c159cd0 con 0x7fd85c0b2790 2023-10-06T11:31:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0c6870 con 0x7fd85c0b2790 2023-10-06T11:31:05.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.162+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c0660f0 con 0x7fd85c0b2790 2023-10-06T11:31:05.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.166+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd84801aad0 con 0x7fd85c0b2790 2023-10-06T11:31:05.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.166+0000 7fd858ff9700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 0x7fd840062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:05.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.170+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6402+0+0 (secure 0 0 0) 0x7fd84807ea90 con 0x7fd85c0b2790 2023-10-06T11:31:05.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.170+0000 7fd860c0a700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 0x7fd840062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:05.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.170+0000 7fd860c0a700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 0x7fd840062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500016e0 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.174+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd84804ec10 con 0x7fd85c0b2790 2023-10-06T11:31:05.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.386+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd85c0cbbe0 con 0x7fd85c0b2790 2023-10-06T11:31:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.962+0000 7fd858ff9700 1 -- 172.21.15.46:0/2772470296 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd848052230 con 0x7fd85c0b2790 2023-10-06T11:31:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.970+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 msgr2=0x7fd840062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.970+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 0x7fd840062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500016e0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.970+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 msgr2=0x7fd85c155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.970+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd8480030c0 tx=0x7fd848016000 comp rx=0 tx=0).stop 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 shutdown_connections 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd840060670 0x7fd840062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85c0cd080 0x7fd85c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd85c0b2790 0x7fd85c155b60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 --2- 172.21.15.46:0/2772470296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd85c0b23c0 0x7fd85c0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 >> 172.21.15.46:0/2772470296 conn(0x7fd85c099480 msgr2=0x7fd85c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 shutdown_connections 2023-10-06T11:31:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:05.978+0000 7fd862e6e700 1 -- 172.21.15.46:0/2772470296 wait complete. 2023-10-06T11:31:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T11:31:06.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:31:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.150+0000 7f98eb678700 1 Processor -- start 2023-10-06T11:31:06.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98eb678700 1 -- start start 2023-10-06T11:31:06.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98eb678700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98e40b2750 0x7f98e406a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98eb678700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e415bb70 con 0x7f98e4156ea0 2023-10-06T11:31:06.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98eb678700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40cab20 con 0x7f98e40af3f0 2023-10-06T11:31:06.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98eb678700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e415aff0 con 0x7f98e40b2750 2023-10-06T11:31:06.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98e9414700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98e4156ea0 0x7f98e40b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49190/0 (socket says 172.21.15.46:49190) 2023-10-06T11:31:06.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98e9414700 1 -- 172.21.15.46:0/1456311837 learned_addr learned my addr 172.21.15.46:0/1456311837 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:06.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.154+0000 7f98e9c15700 1 --2- 172.21.15.46:0/1456311837 >> v2:172.21.15.132:3300/0 conn(0x7f98e40b2750 0x7f98e406a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98dbfff700 1 -- 172.21.15.46:0/1456311837 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790678220 0 0) 0x7f98e415bb70 con 0x7f98e4156ea0 2023-10-06T11:31:06.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98e9c15700 1 -- 172.21.15.46:0/1456311837 >> v1:172.21.15.132:6789/0 conn(0x7f98e40af3f0 legacy=0x7f98e406a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98e9c15700 1 -- 172.21.15.46:0/1456311837 >> v1:172.21.15.46:6789/0 conn(0x7f98e4156ea0 legacy=0x7f98e40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98e9c15700 1 -- 172.21.15.46:0/1456311837 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e415b2d0 con 0x7f98e40b2750 2023-10-06T11:31:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98e9c15700 1 --2- 172.21.15.46:0/1456311837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b2750 0x7f98e406a800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009de0 tx=0x7f98dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984097be10547ea3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98dbfff700 1 -- 172.21.15.46:0/1456311837 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00e070 con 0x7f98e40b2750 2023-10-06T11:31:06.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98dbfff700 1 -- 172.21.15.46:0/1456311837 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc00bbb0 con 0x7f98e40b2750 2023-10-06T11:31:06.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98dbfff700 1 -- 172.21.15.46:0/1456311837 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc012430 con 0x7f98e40b2750 2023-10-06T11:31:06.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 -- 172.21.15.46:0/1456311837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b2750 msgr2=0x7f98e406a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 --2- 172.21.15.46:0/1456311837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b2750 0x7f98e406a800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009de0 tx=0x7f98dc009780 comp rx=0 tx=0).stop 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 -- 172.21.15.46:0/1456311837 shutdown_connections 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 --2- 172.21.15.46:0/1456311837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40b2750 0x7f98e406a800 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 -- 172.21.15.46:0/1456311837 >> 172.21.15.46:0/1456311837 conn(0x7f98e4068bf0 msgr2=0x7f98e40ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 -- 172.21.15.46:0/1456311837 shutdown_connections 2023-10-06T11:31:06.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.158+0000 7f98eb678700 1 -- 172.21.15.46:0/1456311837 wait complete. 2023-10-06T11:31:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 Processor -- start 2023-10-06T11:31:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 -- start start 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40af3f0 0x7f98e4169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b2750 0x7f98e414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40adaf0 con 0x7f98e4156ea0 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e416a010 con 0x7f98e40af3f0 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98eb678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4151580 con 0x7f98e40b2750 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34620/0 (socket says 172.21.15.46:34620) 2023-10-06T11:31:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 -- 172.21.15.46:0/2270363499 learned_addr learned my addr 172.21.15.46:0/2270363499 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:06.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9414700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40af3f0 0x7f98e4169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e8c13700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b2750 0x7f98e414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 -- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b2750 msgr2=0x7f98e414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b2750 0x7f98e414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 -- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40af3f0 msgr2=0x7f98e4169ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40af3f0 0x7f98e4169ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.162+0000 7f98e9c15700 1 -- 172.21.15.46:0/2270363499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc009a20 con 0x7f98e4156ea0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98e9c15700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f98e406d6e0 tx=0x7f98dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0124b0 con 0x7f98e4156ea0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc021070 con 0x7f98e4156ea0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc012ea0 con 0x7f98e4156ea0 2023-10-06T11:31:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e41512a0 con 0x7f98e4156ea0 2023-10-06T11:31:06.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e40ad780 con 0x7f98e4156ea0 2023-10-06T11:31:06.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e406cc30 con 0x7f98e4156ea0 2023-10-06T11:31:06.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.166+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f98dc023030 con 0x7f98e4156ea0 2023-10-06T11:31:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.170+0000 7f98d97fa700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 0x7f98c8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.170+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f98dc07ef40 con 0x7f98e4156ea0 2023-10-06T11:31:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.170+0000 7f98e9414700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 0x7f98c8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.170+0000 7f98e9414700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 0x7f98c8062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98e4156180 tx=0x7f98e0009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.174+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98dc04f100 con 0x7f98e4156ea0 2023-10-06T11:31:06.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.346+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98e404b5c0 con 0x7f98e4156ea0 2023-10-06T11:31:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.346+0000 7f98d97fa700 1 -- 172.21.15.46:0/2270363499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17550 (secure 0 0 0) 0x7f98dc052720 con 0x7f98e4156ea0 2023-10-06T11:31:06.356 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:06.356 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:31:05.389604+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.346+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 msgr2=0x7f98c8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.346+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 0x7f98c8062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98e4156180 tx=0x7f98e0009040 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 msgr2=0x7f98e4150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f98e406d6e0 tx=0x7f98dc018000 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 shutdown_connections 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98c8060670 0x7f98c8062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98e4156ea0 0x7f98e4150330 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98e40b2750 0x7f98e414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 --2- 172.21.15.46:0/2270363499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98e40af3f0 0x7f98e4169ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 >> 172.21.15.46:0/2270363499 conn(0x7f98e4068bf0 msgr2=0x7f98e40b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:06.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 shutdown_connections 2023-10-06T11:31:06.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.350+0000 7f98eb678700 1 -- 172.21.15.46:0/2270363499 wait complete. 2023-10-06T11:31:06.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:31:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.526+0000 7f1c251c5700 1 Processor -- start 2023-10-06T11:31:06.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.526+0000 7f1c251c5700 1 -- start start 2023-10-06T11:31:06.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c251c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c201539d0 0x7f1c2006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c251c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201572a0 0x7f1c20069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c251c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c20156e10 con 0x7f1c200af0b0 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c251c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200af520 con 0x7f1c201539d0 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c251c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c201567c0 con 0x7f1c201572a0 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c1e59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c200af0b0 0x7f1c200b1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49204/0 (socket says 172.21.15.46:49204) 2023-10-06T11:31:06.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c1e59c700 1 -- 172.21.15.46:0/328726100 learned_addr learned my addr 172.21.15.46:0/328726100 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.530+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/328726100 >> v2:172.21.15.132:3300/0 conn(0x7f1c201539d0 0x7f1c2006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1f59e700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201572a0 0x7f1c20069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1dd9b700 1 -- 172.21.15.46:0/328726100 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877773549 0 0) 0x7f1c20156e10 con 0x7f1c200af0b0 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1dd9b700 1 -- 172.21.15.46:0/328726100 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c08003570 con 0x7f1c200af0b0 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201572a0 msgr2=0x7f1c20069d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:06.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201572a0 0x7f1c20069d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/328726100 >> v1:172.21.15.46:6789/0 conn(0x7f1c200af0b0 legacy=0x7f1c200b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/328726100 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c20156aa0 con 0x7f1c201539d0 2023-10-06T11:31:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201539d0 0x7f1c2006e6e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c100098e0 tx=0x7f1c100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1956cabf3cd6f580 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1dd9b700 1 -- 172.21.15.46:0/328726100 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000f6b0 con 0x7f1c201539d0 2023-10-06T11:31:06.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1dd9b700 1 -- 172.21.15.46:0/328726100 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000f810 con 0x7f1c201539d0 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c1dd9b700 1 -- 172.21.15.46:0/328726100 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000fbb0 con 0x7f1c201539d0 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 -- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201539d0 msgr2=0x7f1c2006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201539d0 0x7f1c2006e6e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c100098e0 tx=0x7f1c100091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 -- 172.21.15.46:0/328726100 shutdown_connections 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201572a0 0x7f1c20069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 --2- 172.21.15.46:0/328726100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201539d0 0x7f1c2006e6e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 -- 172.21.15.46:0/328726100 >> 172.21.15.46:0/328726100 conn(0x7f1c20068bf0 msgr2=0x7f1c200ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 -- 172.21.15.46:0/328726100 shutdown_connections 2023-10-06T11:31:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.534+0000 7f1c251c5700 1 -- 172.21.15.46:0/328726100 wait complete. 2023-10-06T11:31:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 Processor -- start 2023-10-06T11:31:06.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 -- start start 2023-10-06T11:31:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 0x7f1c200ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 0x7f1c20149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200cb040 con 0x7f1c200af0b0 2023-10-06T11:31:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200cb1d0 con 0x7f1c201572a0 2023-10-06T11:31:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c251c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200ab650 con 0x7f1c201539d0 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34634/0 (socket says 172.21.15.46:34634) 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/4185040594 learned_addr learned my addr 172.21.15.46:0/4185040594 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1e59c700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 0x7f1c200ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1f59e700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 0x7f1c20149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 msgr2=0x7f1c200ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 0x7f1c200ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 msgr2=0x7f1c20149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 0x7f1c20149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1ed9d700 1 -- 172.21.15.46:0/4185040594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10009420 con 0x7f1c200af0b0 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1e59c700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 0x7f1c200ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.538+0000 7f1c1f59e700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 0x7f1c20149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c1ed9d700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ed0 tx=0x7f1c10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10000d10 con 0x7f1c200af0b0 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000fb90 con 0x7f1c200af0b0 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200ab8d0 con 0x7f1c200af0b0 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c100106c0 con 0x7f1c200af0b0 2023-10-06T11:31:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c200abe90 con 0x7f1c200af0b0 2023-10-06T11:31:06.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.542+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2006cc30 con 0x7f1c200af0b0 2023-10-06T11:31:06.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.546+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1c100243f0 con 0x7f1c200af0b0 2023-10-06T11:31:06.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.546+0000 7f1c06ffd700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 0x7f1c0c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.546+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f1c10014070 con 0x7f1c200af0b0 2023-10-06T11:31:06.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.546+0000 7f1c1e59c700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 0x7f1c0c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.546+0000 7f1c1e59c700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 0x7f1c0c062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c2014be30 tx=0x7f1c14009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.554+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c1004e8c0 con 0x7f1c200af0b0 2023-10-06T11:31:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.726+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c200ac230 con 0x7f1c200af0b0 2023-10-06T11:31:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.726+0000 7f1c06ffd700 1 -- 172.21.15.46:0/4185040594 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17550 (secure 0 0 0) 0x7f1c10051ee0 con 0x7f1c200af0b0 2023-10-06T11:31:06.738 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:06.738 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:31:05.389604+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 msgr2=0x7f1c0c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 0x7f1c0c062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c2014be30 tx=0x7f1c14009040 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 msgr2=0x7f1c2014b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ed0 tx=0x7f1c10019000 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 shutdown_connections 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c0c0606a0 0x7f1c0c062b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c201572a0 0x7f1c20149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c201539d0 0x7f1c200ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 --2- 172.21.15.46:0/4185040594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c200af0b0 0x7f1c2014b2d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 >> 172.21.15.46:0/4185040594 conn(0x7f1c20068bf0 msgr2=0x7f1c20152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 shutdown_connections 2023-10-06T11:31:06.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.730+0000 7f1c251c5700 1 -- 172.21.15.46:0/4185040594 wait complete. 2023-10-06T11:31:06.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:31:06.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.914+0000 7f0d6f08e700 1 Processor -- start 2023-10-06T11:31:06.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.914+0000 7f0d6f08e700 1 -- start start 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0d680d0d60 0x7f0d680b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d680b3630 0x7f0d680b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d68159740 con 0x7f0d680b3630 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680afd20 con 0x7f0d680d0d60 2023-10-06T11:31:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6f08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680b0000 con 0x7f0d680adbe0 2023-10-06T11:31:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e88d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d680b3630 0x7f0d680b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:45508/0 (socket says 172.21.15.46:45508) 2023-10-06T11:31:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 -- 172.21.15.46:0/897993064 learned_addr learned my addr 172.21.15.46:0/897993064 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6d88b700 1 --2- 172.21.15.46:0/897993064 >> v2:172.21.15.132:3300/0 conn(0x7f0d680d0d60 0x7f0d680b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 -- 172.21.15.46:0/897993064 >> v2:172.21.15.132:3300/0 conn(0x7f0d680d0d60 msgr2=0x7f0d680b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/897993064 >> v2:172.21.15.132:3300/0 conn(0x7f0d680d0d60 0x7f0d680b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 -- 172.21.15.46:0/897993064 >> v2:172.21.15.46:3300/0 conn(0x7f0d680b3630 msgr2=0x7f0d680b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/897993064 >> v2:172.21.15.46:3300/0 conn(0x7f0d680b3630 0x7f0d680b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.918+0000 7f0d6e08c700 1 -- 172.21.15.46:0/897993064 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680b02e0 con 0x7f0d680adbe0 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/897993064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d6806a130 tx=0x7f0d64009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd5d0e8a9b7159d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6d08a700 1 -- 172.21.15.46:0/897993064 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400b640 con 0x7f0d680adbe0 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6d08a700 1 -- 172.21.15.46:0/897993064 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400b7a0 con 0x7f0d680adbe0 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6d08a700 1 -- 172.21.15.46:0/897993064 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400bb60 con 0x7f0d680adbe0 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 -- 172.21.15.46:0/897993064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 msgr2=0x7f0d680d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/897993064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d6806a130 tx=0x7f0d64009250 comp rx=0 tx=0).stop 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 -- 172.21.15.46:0/897993064 shutdown_connections 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/897993064 >> v2:172.21.15.132:3300/0 conn(0x7f0d680d0d60 0x7f0d680b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/897993064 >> v2:172.21.15.46:3300/0 conn(0x7f0d680b3630 0x7f0d680b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/897993064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d680d0820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 -- 172.21.15.46:0/897993064 >> 172.21.15.46:0/897993064 conn(0x7f0d6806ef60 msgr2=0x7f0d680d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 -- 172.21.15.46:0/897993064 shutdown_connections 2023-10-06T11:31:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.922+0000 7f0d6f08e700 1 -- 172.21.15.46:0/897993064 wait complete. 2023-10-06T11:31:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 Processor -- start 2023-10-06T11:31:06.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 -- start start 2023-10-06T11:31:06.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d681694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d680b3630 0x7f0d68155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6815b9c0 con 0x7f0d680d0d60 2023-10-06T11:31:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68169a30 con 0x7f0d680b3630 2023-10-06T11:31:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6f08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6809b4d0 con 0x7f0d680adbe0 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34656/0 (socket says 172.21.15.46:34656) 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 -- 172.21.15.46:0/1973522797 learned_addr learned my addr 172.21.15.46:0/1973522797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d681694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6d88b700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d680b3630 0x7f0d68155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 -- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 msgr2=0x7f0d681694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d681694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 -- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d680b3630 msgr2=0x7f0d68155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d680b3630 0x7f0d68155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 -- 172.21.15.46:0/1973522797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640094f0 con 0x7f0d680d0d60 2023-10-06T11:31:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.926+0000 7f0d6e88d700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009ec0 tx=0x7f0d6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000cb70 con 0x7f0d680d0d60 2023-10-06T11:31:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6815b090 con 0x7f0d680d0d60 2023-10-06T11:31:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x7f0d680d0d60 2023-10-06T11:31:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60014480 con 0x7f0d680d0d60 2023-10-06T11:31:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6815b650 con 0x7f0d680d0d60 2023-10-06T11:31:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.930+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6815bb70 con 0x7f0d680d0d60 2023-10-06T11:31:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.934+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0d600147f0 con 0x7f0d680d0d60 2023-10-06T11:31:06.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.934+0000 7f0d5e7fc700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 0x7f0d4c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:06.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.934+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0d6007da50 con 0x7f0d680d0d60 2023-10-06T11:31:06.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.934+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 0x7f0d4c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:06.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.934+0000 7f0d6e08c700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 0x7f0d4c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d6806a130 tx=0x7f0d64017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:06.942+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d6004da60 con 0x7f0d680d0d60 2023-10-06T11:31:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.154+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0d68064590 con 0x7f0d680d0d60 2023-10-06T11:31:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.970+0000 7f0d5e7fc700 1 -- 172.21.15.46:0/1973522797 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0d600180b0 con 0x7f0d680d0d60 2023-10-06T11:31:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 msgr2=0x7f0d4c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 0x7f0d4c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d6806a130 tx=0x7f0d64017000 comp rx=0 tx=0).stop 2023-10-06T11:31:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 msgr2=0x7f0d680adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:07.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009ec0 tx=0x7f0d6000b040 comp rx=0 tx=0).stop 2023-10-06T11:31:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 shutdown_connections 2023-10-06T11:31:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0d4c060670 0x7f0d4c062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:07.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d680d0d60 0x7f0d680adfb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d680b3630 0x7f0d68155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 --2- 172.21.15.46:0/1973522797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d680adbe0 0x7f0d681694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 >> 172.21.15.46:0/1973522797 conn(0x7f0d6806ef60 msgr2=0x7f0d681589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.974+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 shutdown_connections 2023-10-06T11:31:07.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:07.978+0000 7f0d6f08e700 1 -- 172.21.15.46:0/1973522797 wait complete. 2023-10-06T11:31:08.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:31:08.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.174+0000 7f617ce6e700 1 Processor -- start 2023-10-06T11:31:08.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.174+0000 7f617ce6e700 1 -- start start 2023-10-06T11:31:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617ce6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6178161fc0 0x7f61780caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617ce6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780c1980 con 0x7f6178147280 2023-10-06T11:31:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617ce6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780c2520 con 0x7f6178146eb0 2023-10-06T11:31:08.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617ce6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780cb1e0 con 0x7f6178161fc0 2023-10-06T11:31:08.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f6176d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6178147280 0x7f617806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49212/0 (socket says 172.21.15.46:49212) 2023-10-06T11:31:08.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f6176d9d700 1 -- 172.21.15.46:0/123652777 learned_addr learned my addr 172.21.15.46:0/123652777 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:08.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f6175d9b700 1 --2- 172.21.15.46:0/123652777 >> v2:172.21.15.46:3300/0 conn(0x7f6178161fc0 0x7f61780caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:08.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152852455 0 0) 0x7f61780c1980 con 0x7f6178147280 2023-10-06T11:31:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6160003570 con 0x7f6178147280 2023-10-06T11:31:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759240363 0 0) 0x7f61780c2520 con 0x7f6178146eb0 2023-10-06T11:31:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61780c1980 con 0x7f6178146eb0 2023-10-06T11:31:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f6175d9b700 1 -- 172.21.15.46:0/123652777 >> v1:172.21.15.132:6789/0 conn(0x7f6178146eb0 legacy=0x7f61780c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.178+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2531740932 0 0) 0x7f6160003570 con 0x7f6178147280 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f6175d9b700 1 -- 172.21.15.46:0/123652777 >> v1:172.21.15.46:6789/0 conn(0x7f6178147280 legacy=0x7f617806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f6175d9b700 1 -- 172.21.15.46:0/123652777 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780c0d70 con 0x7f6178161fc0 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f6175d9b700 1 --2- 172.21.15.46:0/123652777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178161fc0 0x7f61780caca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f616c0098e0 tx=0x7f616c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68acadf69ef1205e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700017f0 con 0x7f6178161fc0 2023-10-06T11:31:08.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c011420 con 0x7f6178161fc0 2023-10-06T11:31:08.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617559a700 1 -- 172.21.15.46:0/123652777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c0117c0 con 0x7f6178161fc0 2023-10-06T11:31:08.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 -- 172.21.15.46:0/123652777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178161fc0 msgr2=0x7f61780caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 --2- 172.21.15.46:0/123652777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178161fc0 0x7f61780caca0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f616c0098e0 tx=0x7f616c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 -- 172.21.15.46:0/123652777 shutdown_connections 2023-10-06T11:31:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 --2- 172.21.15.46:0/123652777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178161fc0 0x7f61780caca0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 -- 172.21.15.46:0/123652777 >> 172.21.15.46:0/123652777 conn(0x7f61780cbe30 msgr2=0x7f61780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 -- 172.21.15.46:0/123652777 shutdown_connections 2023-10-06T11:31:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.182+0000 7f617ce6e700 1 -- 172.21.15.46:0/123652777 wait complete. 2023-10-06T11:31:08.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 Processor -- start 2023-10-06T11:31:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 -- start start 2023-10-06T11:31:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178146eb0 0x7f6178171fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 0x7f6178141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61781724f0 con 0x7f6178147280 2023-10-06T11:31:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617815fcc0 con 0x7f6178146eb0 2023-10-06T11:31:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f617ce6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6178160b00 con 0x7f6178161fc0 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f6175d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f6175d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34678/0 (socket says 172.21.15.46:34678) 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.186+0000 7f6175d9b700 1 -- 172.21.15.46:0/3446598944 learned_addr learned my addr 172.21.15.46:0/3446598944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f617659c700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178146eb0 0x7f6178171fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6176d9d700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 0x7f6178141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 -- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 msgr2=0x7f6178141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 0x7f6178141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 -- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178146eb0 msgr2=0x7f6178171fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178146eb0 0x7f6178171fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 -- 172.21.15.46:0/3446598944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c009420 con 0x7f6178147280 2023-10-06T11:31:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6176d9d700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 0x7f6178141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:31:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f6175d9b700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f617806c150 tx=0x7f616c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c01cc90 con 0x7f6178147280 2023-10-06T11:31:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c01cdf0 con 0x7f6178147280 2023-10-06T11:31:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617815fef0 con 0x7f6178147280 2023-10-06T11:31:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c01a6a0 con 0x7f6178147280 2023-10-06T11:31:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.190+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61781604b0 con 0x7f6178147280 2023-10-06T11:31:08.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.194+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61780c2510 con 0x7f6178147280 2023-10-06T11:31:08.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.194+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f616c01aa50 con 0x7f6178147280 2023-10-06T11:31:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.194+0000 7f61667fc700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 0x7f6158062c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.194+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f616c07e630 con 0x7f6178147280 2023-10-06T11:31:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.194+0000 7f617659c700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 0x7f6158062c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.198+0000 7f617659c700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 0x7f6158062c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6168003870 tx=0x7f6168009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:08.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.198+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f61780c2510 con 0x7f6178147280 2023-10-06T11:31:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.402+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61780c2510 con 0x7f6178147280 2023-10-06T11:31:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.982+0000 7f61667fc700 1 -- 172.21.15.46:0/3446598944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f616c04e7d0 con 0x7f6178147280 2023-10-06T11:31:08.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.986+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 msgr2=0x7f6158062c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.986+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 0x7f6158062c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6168003870 tx=0x7f6168009040 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.986+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 msgr2=0x7f617815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.986+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f617806c150 tx=0x7f616c016000 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 shutdown_connections 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6158060770 0x7f6158062c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6178161fc0 0x7f6178141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6178147280 0x7f617815f780 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 --2- 172.21.15.46:0/3446598944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6178146eb0 0x7f6178171fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:08.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 >> 172.21.15.46:0/3446598944 conn(0x7f61780cbe30 msgr2=0x7f61781427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 shutdown_connections 2023-10-06T11:31:09.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:08.990+0000 7f617ce6e700 1 -- 172.21.15.46:0/3446598944 wait complete. 2023-10-06T11:31:09.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:31:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 Processor -- start 2023-10-06T11:31:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 -- start start 2023-10-06T11:31:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 0x7fca300caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca300caf50 0x7fca3006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300c18f0 con 0x7fca30161470 2023-10-06T11:31:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300c0ce0 con 0x7fca300caf50 2023-10-06T11:31:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.174+0000 7fca35cf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300c0fc0 con 0x7fca30161840 2023-10-06T11:31:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca300caf50 0x7fca3006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca30161470 0x7fca300c1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49228/0 (socket says 172.21.15.46:49228) 2023-10-06T11:31:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2f7fe700 1 -- 172.21.15.46:0/901106565 learned_addr learned my addr 172.21.15.46:0/901106565 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:09.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca26ffd700 1 --2- 172.21.15.46:0/901106565 >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 0x7fca300caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 -- 172.21.15.46:0/901106565 >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 msgr2=0x7fca300caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 --2- 172.21.15.46:0/901106565 >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 0x7fca300caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 -- 172.21.15.46:0/901106565 >> v1:172.21.15.46:6789/0 conn(0x7fca30161470 legacy=0x7fca300c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 -- 172.21.15.46:0/901106565 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300c12a0 con 0x7fca300caf50 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca26ffd700 1 --2- 172.21.15.46:0/901106565 >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 0x7fca300caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2effd700 1 -- 172.21.15.46:0/901106565 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435472002 0 0) 0x7fca300c18f0 con 0x7fca30161470 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2ffff700 1 --2- 172.21.15.46:0/901106565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300caf50 0x7fca3006cc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fca3006ba90 tx=0x7fca28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1441114326d8283 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2effd700 1 -- 172.21.15.46:0/901106565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2800e070 con 0x7fca300caf50 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2effd700 1 -- 172.21.15.46:0/901106565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2800bcc0 con 0x7fca300caf50 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca2effd700 1 -- 172.21.15.46:0/901106565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28012680 con 0x7fca300caf50 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 -- 172.21.15.46:0/901106565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300caf50 msgr2=0x7fca3006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 --2- 172.21.15.46:0/901106565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300caf50 0x7fca3006cc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fca3006ba90 tx=0x7fca28009780 comp rx=0 tx=0).stop 2023-10-06T11:31:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 -- 172.21.15.46:0/901106565 shutdown_connections 2023-10-06T11:31:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 --2- 172.21.15.46:0/901106565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300caf50 0x7fca3006cc30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 --2- 172.21.15.46:0/901106565 >> v2:172.21.15.132:3300/0 conn(0x7fca30161840 0x7fca300caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.178+0000 7fca35cf0700 1 -- 172.21.15.46:0/901106565 >> 172.21.15.46:0/901106565 conn(0x7fca300cbe30 msgr2=0x7fca300cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.182+0000 7fca35cf0700 1 -- 172.21.15.46:0/901106565 shutdown_connections 2023-10-06T11:31:09.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.182+0000 7fca35cf0700 1 -- 172.21.15.46:0/901106565 wait complete. 2023-10-06T11:31:09.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.182+0000 7fca35cf0700 1 Processor -- start 2023-10-06T11:31:09.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- start start 2023-10-06T11:31:09.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 0x7fca30148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca30161470 0x7fca300b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015a8b0 con 0x7fca30161840 2023-10-06T11:31:09.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015dc20 con 0x7fca30161470 2023-10-06T11:31:09.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015d010 con 0x7fca300caf50 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34700/0 (socket says 172.21.15.46:34700) 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 -- 172.21.15.46:0/1290332822 learned_addr learned my addr 172.21.15.46:0/1290332822 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca26ffd700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca30161470 0x7fca300b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2f7fe700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 0x7fca30148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 -- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 msgr2=0x7fca30148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 0x7fca30148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 -- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca30161470 msgr2=0x7fca300b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca30161470 0x7fca300b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 -- 172.21.15.46:0/1290332822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28009a20 con 0x7fca30161840 2023-10-06T11:31:09.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2f7fe700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 0x7fca30148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:31:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca2ffff700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fca280099f0 tx=0x7fca28019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28012e70 con 0x7fca30161840 2023-10-06T11:31:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2801d630 con 0x7fca30161840 2023-10-06T11:31:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2801da10 con 0x7fca30161840 2023-10-06T11:31:09.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3015d2f0 con 0x7fca30161840 2023-10-06T11:31:09.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.186+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3015d8b0 con 0x7fca30161840 2023-10-06T11:31:09.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.190+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca30063f90 con 0x7fca30161840 2023-10-06T11:31:09.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.190+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fca28024070 con 0x7fca30161840 2023-10-06T11:31:09.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.194+0000 7fca27fff700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 0x7fca10062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.194+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fca2807f5d0 con 0x7fca30161840 2023-10-06T11:31:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.194+0000 7fca2f7fe700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 0x7fca10062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.194+0000 7fca2f7fe700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 0x7fca10062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca20007fd0 tx=0x7fca20009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.198+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fca2804f6f0 con 0x7fca30161840 2023-10-06T11:31:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.406+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca300c1580 con 0x7fca30161840 2023-10-06T11:31:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.406+0000 7fca27fff700 1 -- 172.21.15.46:0/1290332822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fca2802c090 con 0x7fca30161840 2023-10-06T11:31:09.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:09.417 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:31:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 msgr2=0x7fca10062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 0x7fca10062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca20007fd0 tx=0x7fca20009040 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 msgr2=0x7fca300ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fca280099f0 tx=0x7fca28019000 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 shutdown_connections 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fca10060670 0x7fca10062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30161840 0x7fca300ca170 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca30161470 0x7fca300b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 --2- 172.21.15.46:0/1290332822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca300caf50 0x7fca30148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 >> 172.21.15.46:0/1290332822 conn(0x7fca300cbe30 msgr2=0x7fca3006e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 shutdown_connections 2023-10-06T11:31:09.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.414+0000 7fca35cf0700 1 -- 172.21.15.46:0/1290332822 wait complete. 2023-10-06T11:31:09.425 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:31:09.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:31:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 Processor -- start 2023-10-06T11:31:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 -- start start 2023-10-06T11:31:09.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faa180c5ff0 0x7faa180ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 0x7faa180cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180cc150 con 0x7faa1815c770 2023-10-06T11:31:09.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180cfbb0 con 0x7faa180ccb50 2023-10-06T11:31:09.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.594+0000 7faa1e8cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180cdbf0 con 0x7faa180c5ff0 2023-10-06T11:31:09.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1ce6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 0x7faa180cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa17fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faa1815c770 0x7faa180cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49238/0 (socket says 172.21.15.46:49238) 2023-10-06T11:31:09.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa17fff700 1 -- 172.21.15.46:0/2647888329 learned_addr learned my addr 172.21.15.46:0/2647888329 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:09.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 --2- 172.21.15.46:0/2647888329 >> v2:172.21.15.132:3300/0 conn(0x7faa180c5ff0 0x7faa180ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 -- 172.21.15.46:0/2647888329 >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 msgr2=0x7faa180cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 --2- 172.21.15.46:0/2647888329 >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 0x7faa180cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 -- 172.21.15.46:0/2647888329 >> v1:172.21.15.46:6789/0 conn(0x7faa1815c770 legacy=0x7faa180cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa16ffd700 1 -- 172.21.15.46:0/2647888329 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051479543 0 0) 0x7faa180cc150 con 0x7faa1815c770 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 -- 172.21.15.46:0/2647888329 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa180cb540 con 0x7faa180c5ff0 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1ce6a700 1 --2- 172.21.15.46:0/2647888329 >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 0x7faa180cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:31:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa177fe700 1 --2- 172.21.15.46:0/2647888329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa180ce4f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa18099b90 tx=0x7faa0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e378ef84a6f0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa16ffd700 1 -- 172.21.15.46:0/2647888329 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011630 con 0x7faa180c5ff0 2023-10-06T11:31:09.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa16ffd700 1 -- 172.21.15.46:0/2647888329 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c011790 con 0x7faa180c5ff0 2023-10-06T11:31:09.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa16ffd700 1 -- 172.21.15.46:0/2647888329 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011b30 con 0x7faa180c5ff0 2023-10-06T11:31:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1e8cd700 1 -- 172.21.15.46:0/2647888329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 msgr2=0x7faa180ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/2647888329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa180ce4f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa18099b90 tx=0x7faa0c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1e8cd700 1 -- 172.21.15.46:0/2647888329 shutdown_connections 2023-10-06T11:31:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/2647888329 >> v2:172.21.15.46:3300/0 conn(0x7faa180ccb50 0x7faa180cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.598+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/2647888329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa180ce4f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- 172.21.15.46:0/2647888329 >> 172.21.15.46:0/2647888329 conn(0x7faa18099480 msgr2=0x7faa1806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- 172.21.15.46:0/2647888329 shutdown_connections 2023-10-06T11:31:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- 172.21.15.46:0/2647888329 wait complete. 2023-10-06T11:31:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 Processor -- start 2023-10-06T11:31:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- start start 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa18140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa1815c770 0x7faa180b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815e130 con 0x7faa180ccb50 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815d520 con 0x7faa180c5ff0 2023-10-06T11:31:09.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.602+0000 7faa1e8cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1815d800 con 0x7faa1815c770 2023-10-06T11:31:09.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34714/0 (socket says 172.21.15.46:34714) 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 -- 172.21.15.46:0/4135346066 learned_addr learned my addr 172.21.15.46:0/4135346066 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa17fff700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa18140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa1ce6a700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa1815c770 0x7faa180b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 -- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa1815c770 msgr2=0x7faa180b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa1815c770 0x7faa180b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 -- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 msgr2=0x7faa18140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa18140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 -- 172.21.15.46:0/4135346066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10009a20 con 0x7faa180ccb50 2023-10-06T11:31:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa177fe700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faa180cfcf0 tx=0x7faa0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011b10 con 0x7faa180ccb50 2023-10-06T11:31:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0253f0 con 0x7faa180ccb50 2023-10-06T11:31:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c025790 con 0x7faa180ccb50 2023-10-06T11:31:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c009420 con 0x7faa180ccb50 2023-10-06T11:31:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1815ddc0 con 0x7faa180ccb50 2023-10-06T11:31:09.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.606+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1815e2a0 con 0x7faa180ccb50 2023-10-06T11:31:09.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.610+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7faa0c025b80 con 0x7faa180ccb50 2023-10-06T11:31:09.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.610+0000 7fa9fffff700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 0x7faa04062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.610+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7faa0c07e4e0 con 0x7faa180ccb50 2023-10-06T11:31:09.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.610+0000 7faa17fff700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 0x7faa04062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.614+0000 7faa17fff700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 0x7faa04062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa08002930 tx=0x7faa08009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.618+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faa0c04e600 con 0x7faa180ccb50 2023-10-06T11:31:09.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.790+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa180660f0 con 0x7faa180ccb50 2023-10-06T11:31:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.790+0000 7fa9fffff700 1 -- 172.21.15.46:0/4135346066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+17537 (secure 0 0 0) 0x7faa0c051c20 con 0x7faa180ccb50 2023-10-06T11:31:09.801 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:09.801 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:31:06.936927+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:31:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 msgr2=0x7faa04062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 0x7faa04062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa08002930 tx=0x7faa08009040 comp rx=0 tx=0).stop 2023-10-06T11:31:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 msgr2=0x7faa1815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faa180cfcf0 tx=0x7faa0c017000 comp rx=0 tx=0).stop 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 shutdown_connections 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa040606a0 0x7faa04062b50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa1815c770 0x7faa180b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa180ccb50 0x7faa1815cb40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 --2- 172.21.15.46:0/4135346066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa180c5ff0 0x7faa18140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 >> 172.21.15.46:0/4135346066 conn(0x7faa18099480 msgr2=0x7faa180b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 shutdown_connections 2023-10-06T11:31:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.794+0000 7faa1e8cd700 1 -- 172.21.15.46:0/4135346066 wait complete. 2023-10-06T11:31:09.826 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:31:09.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:31:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.974+0000 7fb9828cd700 1 Processor -- start 2023-10-06T11:31:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.974+0000 7fb9828cd700 1 -- start start 2023-10-06T11:31:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb9828cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb97c1502f0 0x7fb97c0c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb9828cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0c67e0 con 0x7fb97c15a1a0 2023-10-06T11:31:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb9828cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c06e810 con 0x7fb97c1506c0 2023-10-06T11:31:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb9828cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c06eaf0 con 0x7fb97c1502f0 2023-10-06T11:31:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb97bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb97c1502f0 0x7fb97c0c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb980e6a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb97c15a1a0 0x7fb97c0c6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49240/0 (socket says 172.21.15.46:49240) 2023-10-06T11:31:09.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb980e6a700 1 -- 172.21.15.46:0/2101986199 learned_addr learned my addr 172.21.15.46:0/2101986199 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb97affd700 1 -- 172.21.15.46:0/2101986199 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747536627 0 0) 0x7fb97c0c67e0 con 0x7fb97c15a1a0 2023-10-06T11:31:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.978+0000 7fb97affd700 1 -- 172.21.15.46:0/2101986199 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb968003570 con 0x7fb97c15a1a0 2023-10-06T11:31:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97bfff700 1 -- 172.21.15.46:0/2101986199 >> v1:172.21.15.132:6789/0 conn(0x7fb97c1506c0 legacy=0x7fb97c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97bfff700 1 -- 172.21.15.46:0/2101986199 >> v1:172.21.15.46:6789/0 conn(0x7fb97c15a1a0 legacy=0x7fb97c0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97bfff700 1 -- 172.21.15.46:0/2101986199 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c06edd0 con 0x7fb97c1502f0 2023-10-06T11:31:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97bfff700 1 --2- 172.21.15.46:0/2101986199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c1502f0 0x7fb97c0c0bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0098d0 tx=0x7fb96c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4f65556fcfc28f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97affd700 1 -- 172.21.15.46:0/2101986199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00f6b0 con 0x7fb97c1502f0 2023-10-06T11:31:09.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97affd700 1 -- 172.21.15.46:0/2101986199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00f810 con 0x7fb97c1502f0 2023-10-06T11:31:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb97affd700 1 -- 172.21.15.46:0/2101986199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00fbd0 con 0x7fb97c1502f0 2023-10-06T11:31:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 -- 172.21.15.46:0/2101986199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c1502f0 msgr2=0x7fb97c0c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 --2- 172.21.15.46:0/2101986199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c1502f0 0x7fb97c0c0bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0098d0 tx=0x7fb96c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 -- 172.21.15.46:0/2101986199 shutdown_connections 2023-10-06T11:31:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 --2- 172.21.15.46:0/2101986199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c1502f0 0x7fb97c0c0bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 -- 172.21.15.46:0/2101986199 >> 172.21.15.46:0/2101986199 conn(0x7fb97c0cbe30 msgr2=0x7fb97c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 -- 172.21.15.46:0/2101986199 shutdown_connections 2023-10-06T11:31:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.982+0000 7fb9828cd700 1 -- 172.21.15.46:0/2101986199 wait complete. 2023-10-06T11:31:09.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 Processor -- start 2023-10-06T11:31:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 -- start start 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb97c1502f0 0x7fb97c14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97c1506c0 0x7fb97c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c14e770 con 0x7fb97c15a1a0 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c14ea50 con 0x7fb97c1506c0 2023-10-06T11:31:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb9828cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c14ed30 con 0x7fb97c1502f0 2023-10-06T11:31:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34736/0 (socket says 172.21.15.46:34736) 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 -- 172.21.15.46:0/1885688790 learned_addr learned my addr 172.21.15.46:0/1885688790 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb97bfff700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb97c1502f0 0x7fb97c14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb97b7fe700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97c1506c0 0x7fb97c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 -- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb97c1502f0 msgr2=0x7fb97c14dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb97c1502f0 0x7fb97c14dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 -- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97c1506c0 msgr2=0x7fb97c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97c1506c0 0x7fb97c151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 -- 172.21.15.46:0/1885688790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0093c0 con 0x7fb97c15a1a0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.986+0000 7fb980e6a700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb97400cc40 tx=0x7fb974001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400a8b0 con 0x7fb97c15a1a0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400aa10 con 0x7fb97c15a1a0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400ad90 con 0x7fb97c15a1a0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c14f010 con 0x7fb97c15a1a0 2023-10-06T11:31:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c16d3b0 con 0x7fb97c15a1a0 2023-10-06T11:31:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.990+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c063f90 con 0x7fb97c15a1a0 2023-10-06T11:31:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.994+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb974007300 con 0x7fb97c15a1a0 2023-10-06T11:31:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.994+0000 7fb973fff700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 0x7fb95c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.994+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fb97407cfd0 con 0x7fb97c15a1a0 2023-10-06T11:31:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.994+0000 7fb97bfff700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 0x7fb95c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.994+0000 7fb97bfff700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 0x7fb95c062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009390 tx=0x7fb96c019000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:10.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:09.998+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb97404d0e0 con 0x7fb97c15a1a0 2023-10-06T11:31:10.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.166+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb97c0a9840 con 0x7fb95c0606a0 2023-10-06T11:31:10.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.182+0000 7fb973fff700 1 -- 172.21.15.46:0/1885688790 <== mgr.5637 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+294035 (secure 0 0 0) 0x7fb97c0a9840 con 0x7fb95c0606a0 2023-10-06T11:31:10.199 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:10.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 msgr2=0x7fb95c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 0x7fb95c062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009390 tx=0x7fb96c019000 comp rx=0 tx=0).stop 2023-10-06T11:31:10.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 msgr2=0x7fb97c1403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb97400cc40 tx=0x7fb974001a00 comp rx=0 tx=0).stop 2023-10-06T11:31:10.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 shutdown_connections 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb95c0606a0 0x7fb95c062b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb97c15a1a0 0x7fb97c1403b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb97c1506c0 0x7fb97c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 --2- 172.21.15.46:0/1885688790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb97c1502f0 0x7fb97c14dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.190+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 >> 172.21.15.46:0/1885688790 conn(0x7fb97c0cbe30 msgr2=0x7fb97c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.194+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 shutdown_connections 2023-10-06T11:31:10.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.194+0000 7fb9828cd700 1 -- 172.21.15.46:0/1885688790 wait complete. 2023-10-06T11:31:10.204 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:31:10.223 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:31:08.387540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2438088,"kb_used_data":6536,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752536632,"statfs":{"total":773094113280,"available":770597511168,"internally_reserved":0,"allocated":6692864,"data_stored":3831728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.470483"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"51'139","reported_seq":274,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.884764+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:31:00.884764+0000","last_peered":"2023-10-06T11:31:00.884764+0000","last_clean":"2023-10-06T11:31:00.884764+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:31:00.884764+0000","last_undegraded":"2023-10-06T11:31:00.884764+0000","last_fullsized":"2023-10-06T11:31:00.884764+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.909217+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:31:00.909217+0000","last_peered":"2023-10-06T11:31:00.909217+0000","last_clean":"2023-10-06T11:31:00.909217+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:31:00.909217+0000","last_undegraded":"2023-10-06T11:31:00.909217+0000","last_fullsized":"2023-10-06T11:31:00.909217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.884659+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:31:00.884659+0000","last_peered":"2023-10-06T11:31:00.884659+0000","last_clean":"2023-10-06T11:31:00.884659+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:31:00.884659+0000","last_undegraded":"2023-10-06T11:31:00.884659+0000","last_fullsized":"2023-10-06T11:31:00.884659+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-06T11:30:56.142019+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:30:56.142019+0000","last_peered":"2023-10-06T11:30:56.142019+0000","last_clean":"2023-10-06T11:30:56.142019+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:30:56.142019+0000","last_undegraded":"2023-10-06T11:30:56.142019+0000","last_fullsized":"2023-10-06T11:30:56.142019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.909199+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:31:00.909199+0000","last_peered":"2023-10-06T11:31:00.909199+0000","last_clean":"2023-10-06T11:31:00.909199+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:31:00.909199+0000","last_undegraded":"2023-10-06T11:31:00.909199+0000","last_fullsized":"2023-10-06T11:31:00.909199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.906414+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:31:00.906414+0000","last_peered":"2023-10-06T11:31:00.906414+0000","last_clean":"2023-10-06T11:31:00.906414+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:31:00.906414+0000","last_undegraded":"2023-10-06T11:31:00.906414+0000","last_fullsized":"2023-10-06T11:31:00.906414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.884701+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:31:00.884701+0000","last_peered":"2023-10-06T11:31:00.884701+0000","last_clean":"2023-10-06T11:31:00.884701+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:31:00.884701+0000","last_undegraded":"2023-10-06T11:31:00.884701+0000","last_fullsized":"2023-10-06T11:31:00.884701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.906420+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:31:00.906420+0000","last_peered":"2023-10-06T11:31:00.906420+0000","last_clean":"2023-10-06T11:31:00.906420+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:31:00.906420+0000","last_undegraded":"2023-10-06T11:31:00.906420+0000","last_fullsized":"2023-10-06T11:31:00.906420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:04.917143+0000","last_change":"2023-10-06T11:31:04.917143+0000","last_active":"2023-10-06T11:31:04.917143+0000","last_peered":"2023-10-06T11:31:04.917143+0000","last_clean":"2023-10-06T11:31:04.917143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:04.917143+0000","last_undegraded":"2023-10-06T11:31:04.917143+0000","last_fullsized":"2023-10-06T11:31:04.917143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:31:03.901714+0000","last_change":"2023-10-06T11:31:03.901714+0000","last_active":"2023-10-06T11:31:03.901714+0000","last_peered":"2023-10-06T11:31:03.901714+0000","last_clean":"2023-10-06T11:31:03.901714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:31:03.901714+0000","last_undegraded":"2023-10-06T11:31:03.901714+0000","last_fullsized":"2023-10-06T11:31:03.901714+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-06T11:31:00.899768+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:31:00.899768+0000","last_peered":"2023-10-06T11:31:00.899768+0000","last_clean":"2023-10-06T11:31:00.899768+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:31:00.899768+0000","last_undegraded":"2023-10-06T11:31:00.899768+0000","last_fullsized":"2023-10-06T11:31:00.899768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310828,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061012,"statfs":{"total":96636764160,"available":96318476288,"internally_reserved":0,"allocated":1282048,"data_stored":921742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.157,"15min":1.1419999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.51,"5min":1.6499999999999999,"15min":1.827},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.127,"15min":1.153},"min":{"1min":0.749,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.649,"5min":1.649,"15min":2.0030000000000001},"last":1.476}]},{"osd":1,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3049999999999999,"15min":1.325},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.6059999999999999},"last":1.871},{"interface":"front","average":{"1min":1.175,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.502,"5min":1.6379999999999999,"15min":1.931},"last":1.278}]},{"osd":2,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2350000000000001,"15min":1.2110000000000001},"min":{"1min":1.0229999999999999,"5min":0.66300000000000003,"15min":0.65500000000000003},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":2.3799999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.79100000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.528,"5min":1.7210000000000001,"15min":2.2850000000000001},"last":1.6639999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.274,"15min":1.286},"min":{"1min":0.79600000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5169999999999999,"5min":1.6970000000000001,"15min":2.665},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3029999999999999,"15min":1.351},"min":{"1min":0.97699999999999998,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.8570000000000002},"last":1.597}]},{"osd":4,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.101,"15min":1.163},"min":{"1min":0.751,"5min":0.65900000000000003,"15min":0.63600000000000001},"max":{"1min":2.149,"5min":2.149,"15min":2.7970000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5600000000000001,"5min":1.8580000000000001,"15min":2.5339999999999998},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2769999999999999,"15min":1.3560000000000001},"min":{"1min":0.89700000000000002,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7030000000000001,"5min":1.7709999999999999,"15min":2.9529999999999998},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2829999999999999,"15min":1.3069999999999999},"min":{"1min":0.81000000000000005,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":3.105},"last":1.718}]},{"osd":6,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":1.0920000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":3.004},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4059999999999999,"15min":1.4259999999999999},"min":{"1min":0.93300000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.913,"5min":2.0579999999999998,"15min":2.9119999999999999},"last":1.6319999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302056,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.7030000000000001,"5min":1.73,"15min":2.1379999999999999},"last":1.468},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3029999999999999,"15min":1.3169999999999999},"min":{"1min":0.76100000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":2.1920000000000002},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.92300000000000004,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.4670000000000001,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2310000000000001,"15min":1.204},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5669999999999999,"5min":1.669,"15min":1.8480000000000001},"last":0.97199999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.74299999999999999},"max":{"1min":1.4630000000000001,"5min":1.5980000000000001,"15min":1.6919999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.2370000000000001,"15min":1.244},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.73799999999999999},"max":{"1min":1.587,"5min":1.615,"15min":2.7789999999999999},"last":1.0189999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3029999999999999,"15min":1.3129999999999999},"min":{"1min":0.97699999999999998,"5min":0.91900000000000004,"15min":0.78400000000000003},"max":{"1min":1.764,"5min":1.764,"15min":1.9850000000000001},"last":1.415},{"interface":"front","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":0.997,"15min":0.83199999999999996},"max":{"1min":1.5109999999999999,"5min":1.724,"15min":2.109},"last":1.1639999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2130000000000001,"15min":1.2350000000000001},"min":{"1min":0.97199999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6539999999999999,"5min":1.8,"15min":1.956},"last":1.206},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2070000000000001,"15min":1.252},"min":{"1min":0.89100000000000001,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.7,"5min":1.8009999999999999,"15min":1.8879999999999999},"last":1.431}]},{"osd":5,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.242,"15min":1.26},"min":{"1min":0.92500000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.52,"5min":1.72,"15min":2.198},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.361,"15min":1.397},"min":{"1min":0.92600000000000005,"5min":0.85199999999999998,"15min":0.71799999999999997},"max":{"1min":1.827,"5min":1.879,"15min":2.1509999999999998},"last":1.333}]},{"osd":7,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.429,"15min":1.472},"min":{"1min":1.052,"5min":0.878,"15min":0.878},"max":{"1min":1.7809999999999999,"5min":1.927,"15min":2.2919999999999998},"last":1.26},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4730000000000001},"min":{"1min":0.751,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6639999999999999,"5min":1.867,"15min":3.891},"last":1.2410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302056,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.754,"5min":0.67800000000000005,"15min":0.60399999999999998},"max":{"1min":1.617,"5min":1.734,"15min":2.306},"last":1.23},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.292,"15min":1.3080000000000001},"min":{"1min":1.0669999999999999,"5min":0.80800000000000005,"15min":0.64900000000000002},"max":{"1min":1.536,"5min":1.6870000000000001,"15min":2.544},"last":1.0640000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.177,"15min":1.196},"min":{"1min":0.98499999999999999,"5min":0.64300000000000002,"15min":0.53200000000000003},"max":{"1min":1.516,"5min":1.673,"15min":2.4209999999999998},"last":1.153},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.129,"15min":1.208},"min":{"1min":0.83199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4610000000000001,"5min":1.589,"15min":2.9910000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.159,"15min":1.2410000000000001},"min":{"1min":0.77500000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.494,"5min":1.5840000000000001,"15min":3.609},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2310000000000001,"15min":1.264},"min":{"1min":0.88200000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.49,"5min":1.738,"15min":2.6949999999999998},"last":1.371}]},{"osd":3,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.208,"15min":1.294},"min":{"1min":0.94499999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.587,"5min":1.7270000000000001,"15min":3.1429999999999998},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3220000000000001,"15min":1.3959999999999999},"min":{"1min":1.0469999999999999,"5min":0.90000000000000002,"15min":0.77700000000000002},"max":{"1min":1.5509999999999999,"5min":1.6439999999999999,"15min":3.7770000000000001},"last":1.202}]},{"osd":4,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2090000000000001,"15min":1.262},"min":{"1min":0.89400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5649999999999999,"5min":1.73,"15min":3.0800000000000001},"last":1.431},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.278,"15min":1.4039999999999999},"min":{"1min":0.93100000000000005,"5min":0.63500000000000001,"15min":0.59799999999999998},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":3.4100000000000001},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2210000000000001,"15min":1.367},"min":{"1min":0.89000000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.6879999999999999,"5min":1.7629999999999999,"15min":3.8999999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2749999999999999,"15min":1.375},"min":{"1min":0.81999999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.571,"5min":1.7529999999999999,"15min":3.3530000000000002},"last":1.6359999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3169999999999999,"15min":1.431},"min":{"1min":1.1140000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":3.5070000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.363,"15min":1.496},"min":{"1min":1.0329999999999999,"5min":0.84599999999999997,"15min":0.65400000000000003},"max":{"1min":1.782,"5min":1.8109999999999999,"15min":4.0529999999999999},"last":1.2669999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1519999999999999},"min":{"1min":0.66400000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5189999999999999,"5min":1.633,"15min":2.1789999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.169,"5min":1.1319999999999999,"15min":1.169},"min":{"1min":0.81699999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.444,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3200000000000001,"15min":1.353},"min":{"1min":0.90400000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.607,"5min":1.6830000000000001,"15min":2.1989999999999998},"last":1.448},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":1.099,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.851},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1859999999999999,"15min":1.238},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69999999999999996},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":2.2269999999999999},"last":1.194},{"interface":"front","average":{"1min":1.276,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.0229999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.488,"5min":1.5840000000000001,"15min":2.1150000000000002},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":1.052,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.671,"5min":1.671,"15min":1.6839999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.339,"5min":1.278,"15min":1.3280000000000001},"min":{"1min":0.98999999999999999,"5min":0.82999999999999996,"15min":0.77400000000000002},"max":{"1min":1.657,"5min":1.6930000000000001,"15min":2.258},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.173,"15min":1.2669999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.7390000000000001,"15min":2.8250000000000002},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1930000000000001,"15min":1.212},"min":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.24,"15min":1.2989999999999999},"min":{"1min":0.91200000000000003,"5min":0.66200000000000003,"15min":0.54900000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.4039999999999999},"min":{"1min":0.97699999999999998,"5min":0.83899999999999997,"15min":0.77000000000000002},"max":{"1min":1.667,"5min":1.8069999999999999,"15min":2.4329999999999998},"last":1.71}]},{"osd":7,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.339,"15min":1.3919999999999999},"min":{"1min":1.083,"5min":0.70299999999999996,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":1.841,"15min":2.653},"last":1.173},{"interface":"front","average":{"1min":1.496,"5min":1.369,"15min":1.4410000000000001},"min":{"1min":1.262,"5min":0.75600000000000001,"15min":0.70499999999999996},"max":{"1min":1.7949999999999999,"5min":1.8029999999999999,"15min":2.464},"last":1.51}]}]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306152,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065688,"statfs":{"total":96636764160,"available":96323264512,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.88900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90100000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.998,"15min":0.98499999999999999},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.437},"max":{"1min":1.3380000000000001,"5min":1.5289999999999999,"15min":1.627},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.58399999999999996},"max":{"1min":1.284,"5min":1.448,"15min":1.516},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.089,"15min":1.083},"min":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":1.004}]},{"osd":2,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.76000000000000001,"5min":0.752,"15min":0.64700000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.7030000000000001},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1950000000000001,"15min":1.258},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.4430000000000001,"5min":1.639,"15min":2.6640000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.3799999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.569,"5min":1.9730000000000001,"15min":2.0840000000000001},"last":1.159}]},{"osd":5,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.381,"15min":1.4330000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.89000000000000001},"max":{"1min":1.883,"5min":2.1309999999999998,"15min":2.1699999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.395,"15min":1.429},"min":{"1min":1.0880000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.605,"5min":2.113,"15min":2.1179999999999999},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4510000000000001,"15min":1.4670000000000001},"min":{"1min":1.1850000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.663,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.357,"15min":1.405},"min":{"1min":1.0169999999999999,"5min":0.93999999999999995,"15min":0.89800000000000002},"max":{"1min":1.7589999999999999,"5min":1.8180000000000001,"15min":2.137},"last":1.256}]},{"osd":7,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.417,"15min":1.4730000000000001},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"max":{"1min":1.7290000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.1539999999999999,"5min":1.018,"15min":0.93000000000000005},"max":{"1min":2.004,"5min":2.266,"15min":2.266},"last":1.1160000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306152,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065688,"statfs":{"total":96636764160,"available":96323264512,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.748},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.46400000000000002},"max":{"1min":1.3169999999999999,"5min":1.5629999999999999,"15min":1.9650000000000001},"last":0.879}]},{"osd":1,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.94899999999999995,"15min":0.93100000000000005},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.54900000000000004},"max":{"1min":1.631,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.623},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.048,"15min":1.0449999999999999},"min":{"1min":0.82799999999999996,"5min":0.54500000000000004,"15min":0.439},"max":{"1min":1.6779999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1899999999999999,"15min":1.1970000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8,"5min":1.8240000000000001,"15min":1.917},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.327,"15min":1.3120000000000001},"min":{"1min":0.96999999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.631,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.84699999999999998,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.595,"5min":1.754,"15min":2.1899999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2689999999999999,"15min":1.331},"min":{"1min":0.93200000000000005,"5min":0.86199999999999999,"15min":0.84399999999999997},"max":{"1min":1.6819999999999999,"5min":2.024,"15min":2.6920000000000002},"last":1.0509999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.369,"15min":1.3879999999999999},"min":{"1min":1.0489999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.601},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.403,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.91500000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.6859999999999999,"5min":1.778,"15min":2.6400000000000001},"last":1.1499999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.417,"15min":1.4530000000000001},"min":{"1min":1.0229999999999999,"5min":0.996,"15min":0.996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":2.8340000000000001},"last":1.02},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4590000000000001,"15min":1.4670000000000001},"min":{"1min":1.2769999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9910000000000001,"5min":2.617,"15min":2.617},"last":1.1160000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3740000000000001,"15min":1.4079999999999999},"min":{"1min":1.002,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":1.863,"15min":2.9020000000000001},"last":1.071},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":1.0940000000000001,"5min":0.93799999999999994,"15min":0.85199999999999998},"max":{"1min":1.8100000000000001,"5min":2.2829999999999999,"15min":3.2240000000000002},"last":0.96799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302056,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":688128,"data_stored":331374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58099999999999996},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.65300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.319,"5min":1.4259999999999999,"15min":1.855},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.21,"5min":1.133,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.97599999999999998}]},{"osd":3,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.2609999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.508},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.77300000000000002,"5min":0.67300000000000004,"15min":0.58199999999999996},"max":{"1min":1.417,"5min":1.7210000000000001,"15min":4.0830000000000002},"last":1.1080000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.23,"15min":1.3069999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.3919999999999999,"15min":1.4419999999999999},"min":{"1min":1.0189999999999999,"5min":0.96399999999999997,"15min":0.78000000000000003},"max":{"1min":1.6759999999999999,"5min":1.865,"15min":3.5609999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.296,"15min":1.363},"min":{"1min":0.95299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.704,"5min":1.716,"15min":3.2559999999999998},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.333,"15min":1.3879999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.85999999999999999},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.766},"last":1.421}]},{"osd":6,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.48,"15min":1.512},"min":{"1min":1.2390000000000001,"5min":1.0680000000000001,"15min":0.94299999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.399,"15min":1.4530000000000001},"min":{"1min":1.036,"5min":1.0109999999999999,"15min":0.97999999999999998},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.363}]},{"osd":7,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4950000000000001,"15min":1.5600000000000001},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.782,"5min":1.8460000000000001,"15min":3.9620000000000002},"last":1.645},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.421,"15min":1.502},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.708,"5min":1.9079999999999999,"15min":2.8740000000000001},"last":1.1970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306732,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065108,"statfs":{"total":96636764160,"available":96322670592,"internally_reserved":0,"allocated":1282048,"data_stored":921742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.90500000000000003,"15min":0.88900000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.1439999999999999,"5min":1.571,"15min":1.571},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.94799999999999995,"15min":0.96799999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.492,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.98199999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"max":{"1min":1.351,"5min":1.5289999999999999,"15min":2.1989999999999998},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0700000000000001,"15min":1.093},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.327,"5min":1.6160000000000001,"15min":2.7719999999999998},"last":1.179}]},{"osd":3,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0740000000000001,"15min":1.0660000000000001},"min":{"1min":0.78600000000000003,"5min":0.68000000000000005,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.7030000000000001,"15min":1.919},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.327,"15min":1.284},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.61499999999999999},"max":{"1min":1.5600000000000001,"5min":1.7789999999999999,"15min":2.3530000000000002},"last":1.256}]},{"osd":4,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3620000000000001,"15min":1.3500000000000001},"min":{"1min":1.0489999999999999,"5min":1.018,"15min":0.68999999999999995},"max":{"1min":1.627,"5min":2.294,"15min":2.294},"last":1.109},{"interface":"front","average":{"1min":1.234,"5min":1.3049999999999999,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.597,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3180000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3600000000000001,"15min":1.3819999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6220000000000001,"5min":1.881,"15min":2.573},"last":1.349},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3740000000000001,"15min":1.4359999999999999},"min":{"1min":1.135,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.659,"5min":1.982,"15min":2.8650000000000002},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3280000000000001,"15min":1.337},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.88900000000000001},"max":{"1min":1.6799999999999999,"5min":1.9410000000000001,"15min":2.52},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4410000000000001,"15min":1.4730000000000001},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.7210000000000001,"5min":1.944,"15min":2.742},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.454,"15min":1.5329999999999999},"min":{"1min":1.024,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.73,"5min":1.9219999999999999,"15min":4.1159999999999997},"last":1.522},{"interface":"front","average":{"1min":1.329,"5min":1.3799999999999999,"15min":1.4339999999999999},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.75,"5min":1.8340000000000001,"15min":2.3980000000000001},"last":1.7609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:31:10.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:31:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.498+0000 7fce591c3700 1 Processor -- start 2023-10-06T11:31:10.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.498+0000 7fce591c3700 1 -- start start 2023-10-06T11:31:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce591c3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fce5406da60 0x7fce540c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce591c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54067050 0x7fce540674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce591c3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540c6690 con 0x7fce54155c70 2023-10-06T11:31:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce591c3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5406dee0 con 0x7fce5406da60 2023-10-06T11:31:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce591c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce54067a10 con 0x7fce54067050 2023-10-06T11:31:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fce5406da60 0x7fce540c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fce5406da60 0x7fce540c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45626/0 (socket says 172.21.15.46:45626) 2023-10-06T11:31:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 -- 172.21.15.46:0/290073973 learned_addr learned my addr 172.21.15.46:0/290073973 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce589c2700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54067050 0x7fce540674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 -- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54067050 msgr2=0x7fce540674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54067050 0x7fce540674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 -- 172.21.15.46:0/290073973 >> v1:172.21.15.132:6789/0 conn(0x7fce54155c70 legacy=0x7fce540c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 -- 172.21.15.46:0/290073973 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540c74d0 con 0x7fce5406da60 2023-10-06T11:31:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce537fe700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5406da60 0x7fce540c6150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce480099b0 tx=0x7fce48009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5733137257b896a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce52ffd700 1 -- 172.21.15.46:0/290073973 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48009ea0 con 0x7fce5406da60 2023-10-06T11:31:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce52ffd700 1 -- 172.21.15.46:0/290073973 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800b860 con 0x7fce5406da60 2023-10-06T11:31:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.502+0000 7fce52ffd700 1 -- 172.21.15.46:0/290073973 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4800bc00 con 0x7fce5406da60 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5406da60 msgr2=0x7fce540c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5406da60 0x7fce540c6150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce480099b0 tx=0x7fce48009250 comp rx=0 tx=0).stop 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- 172.21.15.46:0/290073973 shutdown_connections 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54067050 0x7fce540674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 --2- 172.21.15.46:0/290073973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce5406da60 0x7fce540c6150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- 172.21.15.46:0/290073973 >> 172.21.15.46:0/290073973 conn(0x7fce54062ef0 msgr2=0x7fce540cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:10.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- 172.21.15.46:0/290073973 shutdown_connections 2023-10-06T11:31:10.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- 172.21.15.46:0/290073973 wait complete. 2023-10-06T11:31:10.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 Processor -- start 2023-10-06T11:31:10.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.506+0000 7fce591c3700 1 -- start start 2023-10-06T11:31:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54067050 0x7fce540c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54155c70 0x7fce540c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c0b30 con 0x7fce5406da60 2023-10-06T11:31:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c0e10 con 0x7fce54067050 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540c10f0 con 0x7fce54155c70 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34740/0 (socket says 172.21.15.46:34740) 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 -- 172.21.15.46:0/2305222879 learned_addr learned my addr 172.21.15.46:0/2305222879 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce53fff700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54067050 0x7fce540c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce589c2700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54155c70 0x7fce540c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 -- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54155c70 msgr2=0x7fce540c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54155c70 0x7fce540c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 -- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54067050 msgr2=0x7fce540c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54067050 0x7fce540c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 -- 172.21.15.46:0/2305222879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480094f0 con 0x7fce5406da60 2023-10-06T11:31:10.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce537fe700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce5406ab70 tx=0x7fce48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:10.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48000d10 con 0x7fce5406da60 2023-10-06T11:31:10.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4800bbe0 con 0x7fce5406da60 2023-10-06T11:31:10.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce540c13d0 con 0x7fce5406da60 2023-10-06T11:31:10.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.510+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48011670 con 0x7fce5406da60 2023-10-06T11:31:10.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.514+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540c0770 con 0x7fce5406da60 2023-10-06T11:31:10.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.514+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54064590 con 0x7fce5406da60 2023-10-06T11:31:10.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.514+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fce48011aa0 con 0x7fce5406da60 2023-10-06T11:31:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.518+0000 7fce3bfff700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 0x7fce40062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.518+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fce4807e570 con 0x7fce5406da60 2023-10-06T11:31:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.518+0000 7fce53fff700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 0x7fce40062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.518+0000 7fce53fff700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 0x7fce40062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce540b1a80 tx=0x7fce44009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.522+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce4804e690 con 0x7fce5406da60 2023-10-06T11:31:10.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.686+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce540c68c0 con 0x7fce40060670 2023-10-06T11:31:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.702+0000 7fce3bfff700 1 -- 172.21.15.46:0/2305222879 <== mgr.5637 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308652 (secure 0 0 0) 0x7fce540c68c0 con 0x7fce40060670 2023-10-06T11:31:10.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 msgr2=0x7fce40062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 0x7fce40062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fce540b1a80 tx=0x7fce44009040 comp rx=0 tx=0).stop 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 msgr2=0x7fce540bc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce5406ab70 tx=0x7fce48017000 comp rx=0 tx=0).stop 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 shutdown_connections 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce40060670 0x7fce40062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce54155c70 0x7fce540c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce5406da60 0x7fce540bc4e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 --2- 172.21.15.46:0/2305222879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce54067050 0x7fce540c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 >> 172.21.15.46:0/2305222879 conn(0x7fce54062ef0 msgr2=0x7fce540c4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 shutdown_connections 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.710+0000 7fce591c3700 1 -- 172.21.15.46:0/2305222879 wait complete. 2023-10-06T11:31:10.723 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:31:10.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T11:31:10.388202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2438536,"kb_used_data":6984,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752536184,"statfs":{"total":773094113280,"available":770597052416,"internally_reserved":0,"allocated":7151616,"data_stored":4062796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001533"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964198+0000","last_change":"2023-10-06T11:31:05.961231+0000","last_active":"2023-10-06T11:31:05.964198+0000","last_peered":"2023-10-06T11:31:05.964198+0000","last_clean":"2023-10-06T11:31:05.964198+0000","last_became_active":"2023-10-06T11:31:05.960705+0000","last_became_peered":"2023-10-06T11:31:05.960705+0000","last_unstale":"2023-10-06T11:31:05.964198+0000","last_undegraded":"2023-10-06T11:31:05.964198+0000","last_fullsized":"2023-10-06T11:31:05.964198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:04.918783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.982691+0000","last_change":"2023-10-06T11:31:05.978703+0000","last_active":"2023-10-06T11:31:05.982691+0000","last_peered":"2023-10-06T11:31:05.982691+0000","last_clean":"2023-10-06T11:31:05.982691+0000","last_became_active":"2023-10-06T11:31:05.978069+0000","last_became_peered":"2023-10-06T11:31:05.978069+0000","last_unstale":"2023-10-06T11:31:05.982691+0000","last_undegraded":"2023-10-06T11:31:05.982691+0000","last_fullsized":"2023-10-06T11:31:05.982691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:28.340213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976810+0000","last_change":"2023-10-06T11:31:05.971040+0000","last_active":"2023-10-06T11:31:05.976810+0000","last_peered":"2023-10-06T11:31:05.976810+0000","last_clean":"2023-10-06T11:31:05.976810+0000","last_became_active":"2023-10-06T11:31:05.970703+0000","last_became_peered":"2023-10-06T11:31:05.970703+0000","last_unstale":"2023-10-06T11:31:05.976810+0000","last_undegraded":"2023-10-06T11:31:05.976810+0000","last_fullsized":"2023-10-06T11:31:05.976810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:30.465148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979838+0000","last_change":"2023-10-06T11:31:05.973806+0000","last_active":"2023-10-06T11:31:05.979838+0000","last_peered":"2023-10-06T11:31:05.979838+0000","last_clean":"2023-10-06T11:31:05.979838+0000","last_became_active":"2023-10-06T11:31:05.973497+0000","last_became_peered":"2023-10-06T11:31:05.973497+0000","last_unstale":"2023-10-06T11:31:05.979838+0000","last_undegraded":"2023-10-06T11:31:05.979838+0000","last_fullsized":"2023-10-06T11:31:05.979838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:06.955369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.982819+0000","last_change":"2023-10-06T11:31:05.967877+0000","last_active":"2023-10-06T11:31:05.982819+0000","last_peered":"2023-10-06T11:31:05.982819+0000","last_clean":"2023-10-06T11:31:05.982819+0000","last_became_active":"2023-10-06T11:31:05.967593+0000","last_became_peered":"2023-10-06T11:31:05.967593+0000","last_unstale":"2023-10-06T11:31:05.982819+0000","last_undegraded":"2023-10-06T11:31:05.982819+0000","last_fullsized":"2023-10-06T11:31:05.982819+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:28.340213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966273+0000","last_change":"2023-10-06T11:31:05.964402+0000","last_active":"2023-10-06T11:31:05.966273+0000","last_peered":"2023-10-06T11:31:05.966273+0000","last_clean":"2023-10-06T11:31:05.966273+0000","last_became_active":"2023-10-06T11:31:05.963955+0000","last_became_peered":"2023-10-06T11:31:05.963955+0000","last_unstale":"2023-10-06T11:31:05.966273+0000","last_undegraded":"2023-10-06T11:31:05.966273+0000","last_fullsized":"2023-10-06T11:31:05.966273+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:44.147552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.978679+0000","last_change":"2023-10-06T11:31:05.976060+0000","last_active":"2023-10-06T11:31:05.978679+0000","last_peered":"2023-10-06T11:31:05.978679+0000","last_clean":"2023-10-06T11:31:05.978679+0000","last_became_active":"2023-10-06T11:31:05.975787+0000","last_became_peered":"2023-10-06T11:31:05.975787+0000","last_unstale":"2023-10-06T11:31:05.978679+0000","last_undegraded":"2023-10-06T11:31:05.978679+0000","last_fullsized":"2023-10-06T11:31:05.978679+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:07.953002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.990607+0000","last_change":"2023-10-06T11:31:05.983071+0000","last_active":"2023-10-06T11:31:05.990607+0000","last_peered":"2023-10-06T11:31:05.990607+0000","last_clean":"2023-10-06T11:31:05.990607+0000","last_became_active":"2023-10-06T11:31:05.982736+0000","last_became_peered":"2023-10-06T11:31:05.982736+0000","last_unstale":"2023-10-06T11:31:05.990607+0000","last_undegraded":"2023-10-06T11:31:05.990607+0000","last_fullsized":"2023-10-06T11:31:05.990607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:58.634581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.990574+0000","last_change":"2023-10-06T11:31:05.982605+0000","last_active":"2023-10-06T11:31:05.990574+0000","last_peered":"2023-10-06T11:31:05.990574+0000","last_clean":"2023-10-06T11:31:05.990574+0000","last_became_active":"2023-10-06T11:31:05.982224+0000","last_became_peered":"2023-10-06T11:31:05.982224+0000","last_unstale":"2023-10-06T11:31:05.990574+0000","last_undegraded":"2023-10-06T11:31:05.990574+0000","last_fullsized":"2023-10-06T11:31:05.990574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:44.671829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.974176+0000","last_change":"2023-10-06T11:31:05.972402+0000","last_active":"2023-10-06T11:31:05.974176+0000","last_peered":"2023-10-06T11:31:05.974176+0000","last_clean":"2023-10-06T11:31:05.974176+0000","last_became_active":"2023-10-06T11:31:05.972047+0000","last_became_peered":"2023-10-06T11:31:05.972047+0000","last_unstale":"2023-10-06T11:31:05.974176+0000","last_undegraded":"2023-10-06T11:31:05.974176+0000","last_fullsized":"2023-10-06T11:31:05.974176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.422633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.973797+0000","last_change":"2023-10-06T11:31:05.961341+0000","last_active":"2023-10-06T11:31:05.973797+0000","last_peered":"2023-10-06T11:31:05.973797+0000","last_clean":"2023-10-06T11:31:05.973797+0000","last_became_active":"2023-10-06T11:31:05.961025+0000","last_became_peered":"2023-10-06T11:31:05.961025+0000","last_unstale":"2023-10-06T11:31:05.973797+0000","last_undegraded":"2023-10-06T11:31:05.973797+0000","last_fullsized":"2023-10-06T11:31:05.973797+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:04.918783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.971585+0000","last_change":"2023-10-06T11:31:05.965780+0000","last_active":"2023-10-06T11:31:05.971585+0000","last_peered":"2023-10-06T11:31:05.971585+0000","last_clean":"2023-10-06T11:31:05.971585+0000","last_became_active":"2023-10-06T11:31:05.965364+0000","last_became_peered":"2023-10-06T11:31:05.965364+0000","last_unstale":"2023-10-06T11:31:05.971585+0000","last_undegraded":"2023-10-06T11:31:05.971585+0000","last_fullsized":"2023-10-06T11:31:05.971585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:11.827016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.975996+0000","last_change":"2023-10-06T11:31:05.973633+0000","last_active":"2023-10-06T11:31:05.975996+0000","last_peered":"2023-10-06T11:31:05.975996+0000","last_clean":"2023-10-06T11:31:05.975996+0000","last_became_active":"2023-10-06T11:31:05.973353+0000","last_became_peered":"2023-10-06T11:31:05.973353+0000","last_unstale":"2023-10-06T11:31:05.975996+0000","last_undegraded":"2023-10-06T11:31:05.975996+0000","last_fullsized":"2023-10-06T11:31:05.975996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:03.709215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976993+0000","last_change":"2023-10-06T11:31:05.971643+0000","last_active":"2023-10-06T11:31:05.976993+0000","last_peered":"2023-10-06T11:31:05.976993+0000","last_clean":"2023-10-06T11:31:05.976993+0000","last_became_active":"2023-10-06T11:31:05.971373+0000","last_became_peered":"2023-10-06T11:31:05.971373+0000","last_unstale":"2023-10-06T11:31:05.976993+0000","last_undegraded":"2023-10-06T11:31:05.976993+0000","last_fullsized":"2023-10-06T11:31:05.976993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:22.250952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979905+0000","last_change":"2023-10-06T11:31:05.974721+0000","last_active":"2023-10-06T11:31:05.979905+0000","last_peered":"2023-10-06T11:31:05.979905+0000","last_clean":"2023-10-06T11:31:05.979905+0000","last_became_active":"2023-10-06T11:31:05.974452+0000","last_became_peered":"2023-10-06T11:31:05.974452+0000","last_unstale":"2023-10-06T11:31:05.979905+0000","last_undegraded":"2023-10-06T11:31:05.979905+0000","last_fullsized":"2023-10-06T11:31:05.979905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:21.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.974153+0000","last_change":"2023-10-06T11:31:05.972615+0000","last_active":"2023-10-06T11:31:05.974153+0000","last_peered":"2023-10-06T11:31:05.974153+0000","last_clean":"2023-10-06T11:31:05.974153+0000","last_became_active":"2023-10-06T11:31:05.971944+0000","last_became_peered":"2023-10-06T11:31:05.971944+0000","last_unstale":"2023-10-06T11:31:05.974153+0000","last_undegraded":"2023-10-06T11:31:05.974153+0000","last_fullsized":"2023-10-06T11:31:05.974153+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:48:28.503794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.978997+0000","last_change":"2023-10-06T11:31:05.971151+0000","last_active":"2023-10-06T11:31:05.978997+0000","last_peered":"2023-10-06T11:31:05.978997+0000","last_clean":"2023-10-06T11:31:05.978997+0000","last_became_active":"2023-10-06T11:31:05.970849+0000","last_became_peered":"2023-10-06T11:31:05.970849+0000","last_unstale":"2023-10-06T11:31:05.978997+0000","last_undegraded":"2023-10-06T11:31:05.978997+0000","last_fullsized":"2023-10-06T11:31:05.978997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:45.666669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.982672+0000","last_change":"2023-10-06T11:31:05.978490+0000","last_active":"2023-10-06T11:31:05.982672+0000","last_peered":"2023-10-06T11:31:05.982672+0000","last_clean":"2023-10-06T11:31:05.982672+0000","last_became_active":"2023-10-06T11:31:05.978101+0000","last_became_peered":"2023-10-06T11:31:05.978101+0000","last_unstale":"2023-10-06T11:31:05.982672+0000","last_undegraded":"2023-10-06T11:31:05.982672+0000","last_fullsized":"2023-10-06T11:31:05.982672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:43:41.587201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.985822+0000","last_change":"2023-10-06T11:31:05.981305+0000","last_active":"2023-10-06T11:31:05.985822+0000","last_peered":"2023-10-06T11:31:05.985822+0000","last_clean":"2023-10-06T11:31:05.985822+0000","last_became_active":"2023-10-06T11:31:05.980984+0000","last_became_peered":"2023-10-06T11:31:05.980984+0000","last_unstale":"2023-10-06T11:31:05.985822+0000","last_undegraded":"2023-10-06T11:31:05.985822+0000","last_fullsized":"2023-10-06T11:31:05.985822+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:37.353722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.982876+0000","last_change":"2023-10-06T11:31:05.978789+0000","last_active":"2023-10-06T11:31:05.982876+0000","last_peered":"2023-10-06T11:31:05.982876+0000","last_clean":"2023-10-06T11:31:05.982876+0000","last_became_active":"2023-10-06T11:31:05.978514+0000","last_became_peered":"2023-10-06T11:31:05.978514+0000","last_unstale":"2023-10-06T11:31:05.982876+0000","last_undegraded":"2023-10-06T11:31:05.982876+0000","last_fullsized":"2023-10-06T11:31:05.982876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:47.418804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.977525+0000","last_change":"2023-10-06T11:31:05.974567+0000","last_active":"2023-10-06T11:31:05.977525+0000","last_peered":"2023-10-06T11:31:05.977525+0000","last_clean":"2023-10-06T11:31:05.977525+0000","last_became_active":"2023-10-06T11:31:05.974228+0000","last_became_peered":"2023-10-06T11:31:05.974228+0000","last_unstale":"2023-10-06T11:31:05.977525+0000","last_undegraded":"2023-10-06T11:31:05.977525+0000","last_fullsized":"2023-10-06T11:31:05.977525+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:58.508008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.970363+0000","last_change":"2023-10-06T11:31:05.966599+0000","last_active":"2023-10-06T11:31:05.970363+0000","last_peered":"2023-10-06T11:31:05.970363+0000","last_clean":"2023-10-06T11:31:05.970363+0000","last_became_active":"2023-10-06T11:31:05.966232+0000","last_became_peered":"2023-10-06T11:31:05.966232+0000","last_unstale":"2023-10-06T11:31:05.970363+0000","last_undegraded":"2023-10-06T11:31:05.970363+0000","last_fullsized":"2023-10-06T11:31:05.970363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:07:53.572929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.973856+0000","last_change":"2023-10-06T11:31:05.962120+0000","last_active":"2023-10-06T11:31:05.973856+0000","last_peered":"2023-10-06T11:31:05.973856+0000","last_clean":"2023-10-06T11:31:05.973856+0000","last_became_active":"2023-10-06T11:31:05.961660+0000","last_became_peered":"2023-10-06T11:31:05.961660+0000","last_unstale":"2023-10-06T11:31:05.973856+0000","last_undegraded":"2023-10-06T11:31:05.973856+0000","last_fullsized":"2023-10-06T11:31:05.973856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:47.418804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976701+0000","last_change":"2023-10-06T11:31:05.962067+0000","last_active":"2023-10-06T11:31:05.976701+0000","last_peered":"2023-10-06T11:31:05.976701+0000","last_clean":"2023-10-06T11:31:05.976701+0000","last_became_active":"2023-10-06T11:31:05.961789+0000","last_became_peered":"2023-10-06T11:31:05.961789+0000","last_unstale":"2023-10-06T11:31:05.976701+0000","last_undegraded":"2023-10-06T11:31:05.976701+0000","last_fullsized":"2023-10-06T11:31:05.976701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:45.665029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.985688+0000","last_change":"2023-10-06T11:31:05.978832+0000","last_active":"2023-10-06T11:31:05.985688+0000","last_peered":"2023-10-06T11:31:05.985688+0000","last_clean":"2023-10-06T11:31:05.985688+0000","last_became_active":"2023-10-06T11:31:05.978544+0000","last_became_peered":"2023-10-06T11:31:05.978544+0000","last_unstale":"2023-10-06T11:31:05.985688+0000","last_undegraded":"2023-10-06T11:31:05.985688+0000","last_fullsized":"2023-10-06T11:31:05.985688+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:06.619441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.965271+0000","last_change":"2023-10-06T11:31:05.958607+0000","last_active":"2023-10-06T11:31:05.965271+0000","last_peered":"2023-10-06T11:31:05.965271+0000","last_clean":"2023-10-06T11:31:05.965271+0000","last_became_active":"2023-10-06T11:31:05.957337+0000","last_became_peered":"2023-10-06T11:31:05.957337+0000","last_unstale":"2023-10-06T11:31:05.965271+0000","last_undegraded":"2023-10-06T11:31:05.965271+0000","last_fullsized":"2023-10-06T11:31:05.965271+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:29.289659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992703+0000","last_change":"2023-10-06T11:31:05.978142+0000","last_active":"2023-10-06T11:31:05.992703+0000","last_peered":"2023-10-06T11:31:05.992703+0000","last_clean":"2023-10-06T11:31:05.992703+0000","last_became_active":"2023-10-06T11:31:05.977835+0000","last_became_peered":"2023-10-06T11:31:05.977835+0000","last_unstale":"2023-10-06T11:31:05.992703+0000","last_undegraded":"2023-10-06T11:31:05.992703+0000","last_fullsized":"2023-10-06T11:31:05.992703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:54:24.695341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992900+0000","last_change":"2023-10-06T11:31:05.989635+0000","last_active":"2023-10-06T11:31:05.992900+0000","last_peered":"2023-10-06T11:31:05.992900+0000","last_clean":"2023-10-06T11:31:05.992900+0000","last_became_active":"2023-10-06T11:31:05.989346+0000","last_became_peered":"2023-10-06T11:31:05.989346+0000","last_unstale":"2023-10-06T11:31:05.992900+0000","last_undegraded":"2023-10-06T11:31:05.992900+0000","last_fullsized":"2023-10-06T11:31:05.992900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.422633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976643+0000","last_change":"2023-10-06T11:31:05.963636+0000","last_active":"2023-10-06T11:31:05.976643+0000","last_peered":"2023-10-06T11:31:05.976643+0000","last_clean":"2023-10-06T11:31:05.976643+0000","last_became_active":"2023-10-06T11:31:05.963354+0000","last_became_peered":"2023-10-06T11:31:05.963354+0000","last_unstale":"2023-10-06T11:31:05.976643+0000","last_undegraded":"2023-10-06T11:31:05.976643+0000","last_fullsized":"2023-10-06T11:31:05.976643+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:20.101137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979854+0000","last_change":"2023-10-06T11:31:05.973720+0000","last_active":"2023-10-06T11:31:05.979854+0000","last_peered":"2023-10-06T11:31:05.979854+0000","last_clean":"2023-10-06T11:31:05.979854+0000","last_became_active":"2023-10-06T11:31:05.973391+0000","last_became_peered":"2023-10-06T11:31:05.973391+0000","last_unstale":"2023-10-06T11:31:05.979854+0000","last_undegraded":"2023-10-06T11:31:05.979854+0000","last_fullsized":"2023-10-06T11:31:05.979854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:29.289659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.978664+0000","last_change":"2023-10-06T11:31:05.973288+0000","last_active":"2023-10-06T11:31:05.978664+0000","last_peered":"2023-10-06T11:31:05.978664+0000","last_clean":"2023-10-06T11:31:05.978664+0000","last_became_active":"2023-10-06T11:31:05.972306+0000","last_became_peered":"2023-10-06T11:31:05.972306+0000","last_unstale":"2023-10-06T11:31:05.978664+0000","last_undegraded":"2023-10-06T11:31:05.978664+0000","last_fullsized":"2023-10-06T11:31:05.978664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:54:25.163668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.965427+0000","last_change":"2023-10-06T11:31:05.960657+0000","last_active":"2023-10-06T11:31:05.965427+0000","last_peered":"2023-10-06T11:31:05.965427+0000","last_clean":"2023-10-06T11:31:05.965427+0000","last_became_active":"2023-10-06T11:31:05.960289+0000","last_became_peered":"2023-10-06T11:31:05.960289+0000","last_unstale":"2023-10-06T11:31:05.965427+0000","last_undegraded":"2023-10-06T11:31:05.965427+0000","last_fullsized":"2023-10-06T11:31:05.965427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:11.827016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.982613+0000","last_change":"2023-10-06T11:31:05.978387+0000","last_active":"2023-10-06T11:31:05.982613+0000","last_peered":"2023-10-06T11:31:05.982613+0000","last_clean":"2023-10-06T11:31:05.982613+0000","last_became_active":"2023-10-06T11:31:05.978070+0000","last_became_peered":"2023-10-06T11:31:05.978070+0000","last_unstale":"2023-10-06T11:31:05.982613+0000","last_undegraded":"2023-10-06T11:31:05.982613+0000","last_fullsized":"2023-10-06T11:31:05.982613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:49.569367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"61'4","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.990659+0000","last_change":"2023-10-06T11:31:04.959422+0000","last_active":"2023-10-06T11:31:05.990659+0000","last_peered":"2023-10-06T11:31:05.990659+0000","last_clean":"2023-10-06T11:31:05.990659+0000","last_became_active":"2023-10-06T11:31:04.959024+0000","last_became_peered":"2023-10-06T11:31:04.959024+0000","last_unstale":"2023-10-06T11:31:05.990659+0000","last_undegraded":"2023-10-06T11:31:05.990659+0000","last_fullsized":"2023-10-06T11:31:05.990659+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:13.085653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979759+0000","last_change":"2023-10-06T11:31:05.971666+0000","last_active":"2023-10-06T11:31:05.979759+0000","last_peered":"2023-10-06T11:31:05.979759+0000","last_clean":"2023-10-06T11:31:05.979759+0000","last_became_active":"2023-10-06T11:31:05.969354+0000","last_became_peered":"2023-10-06T11:31:05.969354+0000","last_unstale":"2023-10-06T11:31:05.979759+0000","last_undegraded":"2023-10-06T11:31:05.979759+0000","last_fullsized":"2023-10-06T11:31:05.979759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:21.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"61'1","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992900+0000","last_change":"2023-10-06T11:31:04.964614+0000","last_active":"2023-10-06T11:31:05.992900+0000","last_peered":"2023-10-06T11:31:05.992900+0000","last_clean":"2023-10-06T11:31:05.992900+0000","last_became_active":"2023-10-06T11:31:04.964295+0000","last_became_peered":"2023-10-06T11:31:04.964295+0000","last_unstale":"2023-10-06T11:31:05.992900+0000","last_undegraded":"2023-10-06T11:31:05.992900+0000","last_fullsized":"2023-10-06T11:31:05.992900+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:10.244202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.980295+0000","last_change":"2023-10-06T11:31:05.974764+0000","last_active":"2023-10-06T11:31:05.980295+0000","last_peered":"2023-10-06T11:31:05.980295+0000","last_clean":"2023-10-06T11:31:05.980295+0000","last_became_active":"2023-10-06T11:31:05.974388+0000","last_became_peered":"2023-10-06T11:31:05.974388+0000","last_unstale":"2023-10-06T11:31:05.980295+0000","last_undegraded":"2023-10-06T11:31:05.980295+0000","last_fullsized":"2023-10-06T11:31:05.980295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:03.709215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.993004+0000","last_change":"2023-10-06T11:31:04.956699+0000","last_active":"2023-10-06T11:31:05.993004+0000","last_peered":"2023-10-06T11:31:05.993004+0000","last_clean":"2023-10-06T11:31:05.993004+0000","last_became_active":"2023-10-06T11:31:04.956399+0000","last_became_peered":"2023-10-06T11:31:04.956399+0000","last_unstale":"2023-10-06T11:31:05.993004+0000","last_undegraded":"2023-10-06T11:31:05.993004+0000","last_fullsized":"2023-10-06T11:31:05.993004+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:06.420284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.959336+0000","last_change":"2023-10-06T11:31:05.954893+0000","last_active":"2023-10-06T11:31:05.959336+0000","last_peered":"2023-10-06T11:31:05.959336+0000","last_clean":"2023-10-06T11:31:05.959336+0000","last_became_active":"2023-10-06T11:31:05.954553+0000","last_became_peered":"2023-10-06T11:31:05.954553+0000","last_unstale":"2023-10-06T11:31:05.959336+0000","last_undegraded":"2023-10-06T11:31:05.959336+0000","last_fullsized":"2023-10-06T11:31:05.959336+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:20.101137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.997832+0000","last_change":"2023-10-06T11:31:04.969769+0000","last_active":"2023-10-06T11:31:05.997832+0000","last_peered":"2023-10-06T11:31:05.997832+0000","last_clean":"2023-10-06T11:31:05.997832+0000","last_became_active":"2023-10-06T11:31:04.969381+0000","last_became_peered":"2023-10-06T11:31:04.969381+0000","last_unstale":"2023-10-06T11:31:05.997832+0000","last_undegraded":"2023-10-06T11:31:05.997832+0000","last_fullsized":"2023-10-06T11:31:05.997832+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:29.760673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.990338+0000","last_change":"2023-10-06T11:31:05.987761+0000","last_active":"2023-10-06T11:31:05.990338+0000","last_peered":"2023-10-06T11:31:05.990338+0000","last_clean":"2023-10-06T11:31:05.990338+0000","last_became_active":"2023-10-06T11:31:05.987443+0000","last_became_peered":"2023-10-06T11:31:05.987443+0000","last_unstale":"2023-10-06T11:31:05.990338+0000","last_undegraded":"2023-10-06T11:31:05.990338+0000","last_fullsized":"2023-10-06T11:31:05.990338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:29.289659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"61'1","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.993618+0000","last_change":"2023-10-06T11:31:04.957929+0000","last_active":"2023-10-06T11:31:05.993618+0000","last_peered":"2023-10-06T11:31:05.993618+0000","last_clean":"2023-10-06T11:31:05.993618+0000","last_became_active":"2023-10-06T11:31:04.957595+0000","last_became_peered":"2023-10-06T11:31:04.957595+0000","last_unstale":"2023-10-06T11:31:05.993618+0000","last_undegraded":"2023-10-06T11:31:05.993618+0000","last_fullsized":"2023-10-06T11:31:05.993618+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:11:26.413860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.977547+0000","last_change":"2023-10-06T11:31:05.973065+0000","last_active":"2023-10-06T11:31:05.977547+0000","last_peered":"2023-10-06T11:31:05.977547+0000","last_clean":"2023-10-06T11:31:05.977547+0000","last_became_active":"2023-10-06T11:31:05.972786+0000","last_became_peered":"2023-10-06T11:31:05.972786+0000","last_unstale":"2023-10-06T11:31:05.977547+0000","last_undegraded":"2023-10-06T11:31:05.977547+0000","last_fullsized":"2023-10-06T11:31:05.977547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:01:11.827016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"61'1","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.993048+0000","last_change":"2023-10-06T11:31:04.958089+0000","last_active":"2023-10-06T11:31:05.993048+0000","last_peered":"2023-10-06T11:31:05.993048+0000","last_clean":"2023-10-06T11:31:05.993048+0000","last_became_active":"2023-10-06T11:31:04.957774+0000","last_became_peered":"2023-10-06T11:31:04.957774+0000","last_unstale":"2023-10-06T11:31:05.993048+0000","last_undegraded":"2023-10-06T11:31:05.993048+0000","last_fullsized":"2023-10-06T11:31:05.993048+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:11.746894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.955498+0000","last_change":"2023-10-06T11:31:05.953300+0000","last_active":"2023-10-06T11:31:05.955498+0000","last_peered":"2023-10-06T11:31:05.955498+0000","last_clean":"2023-10-06T11:31:05.955498+0000","last_became_active":"2023-10-06T11:31:05.952961+0000","last_became_peered":"2023-10-06T11:31:05.952961+0000","last_unstale":"2023-10-06T11:31:05.955498+0000","last_undegraded":"2023-10-06T11:31:05.955498+0000","last_fullsized":"2023-10-06T11:31:05.955498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:21.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966150+0000","last_change":"2023-10-06T11:31:04.962190+0000","last_active":"2023-10-06T11:31:05.966150+0000","last_peered":"2023-10-06T11:31:05.966150+0000","last_clean":"2023-10-06T11:31:05.966150+0000","last_became_active":"2023-10-06T11:31:04.961065+0000","last_became_peered":"2023-10-06T11:31:04.961065+0000","last_unstale":"2023-10-06T11:31:05.966150+0000","last_undegraded":"2023-10-06T11:31:05.966150+0000","last_fullsized":"2023-10-06T11:31:05.966150+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:11.525745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.983183+0000","last_change":"2023-10-06T11:31:05.981400+0000","last_active":"2023-10-06T11:31:05.983183+0000","last_peered":"2023-10-06T11:31:05.983183+0000","last_clean":"2023-10-06T11:31:05.983183+0000","last_became_active":"2023-10-06T11:31:05.981106+0000","last_became_peered":"2023-10-06T11:31:05.981106+0000","last_unstale":"2023-10-06T11:31:05.983183+0000","last_undegraded":"2023-10-06T11:31:05.983183+0000","last_fullsized":"2023-10-06T11:31:05.983183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:20.101137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.948344+0000","last_change":"2023-10-06T11:31:04.956738+0000","last_active":"2023-10-06T11:31:05.948344+0000","last_peered":"2023-10-06T11:31:05.948344+0000","last_clean":"2023-10-06T11:31:05.948344+0000","last_became_active":"2023-10-06T11:31:04.954002+0000","last_became_peered":"2023-10-06T11:31:04.954002+0000","last_unstale":"2023-10-06T11:31:05.948344+0000","last_undegraded":"2023-10-06T11:31:05.948344+0000","last_fullsized":"2023-10-06T11:31:05.948344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:43.398733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.978021+0000","last_change":"2023-10-06T11:31:05.966121+0000","last_active":"2023-10-06T11:31:05.978021+0000","last_peered":"2023-10-06T11:31:05.978021+0000","last_clean":"2023-10-06T11:31:05.978021+0000","last_became_active":"2023-10-06T11:31:05.965829+0000","last_became_peered":"2023-10-06T11:31:05.965829+0000","last_unstale":"2023-10-06T11:31:05.978021+0000","last_undegraded":"2023-10-06T11:31:05.978021+0000","last_fullsized":"2023-10-06T11:31:05.978021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:21.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964090+0000","last_change":"2023-10-06T11:31:04.960057+0000","last_active":"2023-10-06T11:31:05.964090+0000","last_peered":"2023-10-06T11:31:05.964090+0000","last_clean":"2023-10-06T11:31:05.964090+0000","last_became_active":"2023-10-06T11:31:04.959717+0000","last_became_peered":"2023-10-06T11:31:04.959717+0000","last_unstale":"2023-10-06T11:31:05.964090+0000","last_undegraded":"2023-10-06T11:31:05.964090+0000","last_fullsized":"2023-10-06T11:31:05.964090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:03.903354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966703+0000","last_change":"2023-10-06T11:31:05.962952+0000","last_active":"2023-10-06T11:31:05.966703+0000","last_peered":"2023-10-06T11:31:05.966703+0000","last_clean":"2023-10-06T11:31:05.966703+0000","last_became_active":"2023-10-06T11:31:05.962662+0000","last_became_peered":"2023-10-06T11:31:05.962662+0000","last_unstale":"2023-10-06T11:31:05.966703+0000","last_undegraded":"2023-10-06T11:31:05.966703+0000","last_fullsized":"2023-10-06T11:31:05.966703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:43:41.587201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.954170+0000","last_change":"2023-10-06T11:31:04.968200+0000","last_active":"2023-10-06T11:31:05.954170+0000","last_peered":"2023-10-06T11:31:05.954170+0000","last_clean":"2023-10-06T11:31:05.954170+0000","last_became_active":"2023-10-06T11:31:04.967899+0000","last_became_peered":"2023-10-06T11:31:04.967899+0000","last_unstale":"2023-10-06T11:31:05.954170+0000","last_undegraded":"2023-10-06T11:31:05.954170+0000","last_fullsized":"2023-10-06T11:31:05.954170+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:11:26.413860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976971+0000","last_change":"2023-10-06T11:31:05.962886+0000","last_active":"2023-10-06T11:31:05.976971+0000","last_peered":"2023-10-06T11:31:05.976971+0000","last_clean":"2023-10-06T11:31:05.976971+0000","last_became_active":"2023-10-06T11:31:05.962577+0000","last_became_peered":"2023-10-06T11:31:05.962577+0000","last_unstale":"2023-10-06T11:31:05.976971+0000","last_undegraded":"2023-10-06T11:31:05.976971+0000","last_fullsized":"2023-10-06T11:31:05.976971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:48.109910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"61'3","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:06.000376+0000","last_change":"2023-10-06T11:31:04.972270+0000","last_active":"2023-10-06T11:31:06.000376+0000","last_peered":"2023-10-06T11:31:06.000376+0000","last_clean":"2023-10-06T11:31:06.000376+0000","last_became_active":"2023-10-06T11:31:04.971963+0000","last_became_peered":"2023-10-06T11:31:04.971963+0000","last_unstale":"2023-10-06T11:31:06.000376+0000","last_undegraded":"2023-10-06T11:31:06.000376+0000","last_fullsized":"2023-10-06T11:31:06.000376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:43.398733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979461+0000","last_change":"2023-10-06T11:31:05.964892+0000","last_active":"2023-10-06T11:31:05.979461+0000","last_peered":"2023-10-06T11:31:05.979461+0000","last_clean":"2023-10-06T11:31:05.979461+0000","last_became_active":"2023-10-06T11:31:05.964610+0000","last_became_peered":"2023-10-06T11:31:05.964610+0000","last_unstale":"2023-10-06T11:31:05.979461+0000","last_undegraded":"2023-10-06T11:31:05.979461+0000","last_fullsized":"2023-10-06T11:31:05.979461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:06.955369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"61'6","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.997271+0000","last_change":"2023-10-06T11:31:04.954455+0000","last_active":"2023-10-06T11:31:05.997271+0000","last_peered":"2023-10-06T11:31:05.997271+0000","last_clean":"2023-10-06T11:31:05.997271+0000","last_became_active":"2023-10-06T11:31:04.953860+0000","last_became_peered":"2023-10-06T11:31:04.953860+0000","last_unstale":"2023-10-06T11:31:05.997271+0000","last_undegraded":"2023-10-06T11:31:05.997271+0000","last_fullsized":"2023-10-06T11:31:05.997271+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:32.407204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.977319+0000","last_change":"2023-10-06T11:31:05.969054+0000","last_active":"2023-10-06T11:31:05.977319+0000","last_peered":"2023-10-06T11:31:05.977319+0000","last_clean":"2023-10-06T11:31:05.977319+0000","last_became_active":"2023-10-06T11:31:05.968784+0000","last_became_peered":"2023-10-06T11:31:05.968784+0000","last_unstale":"2023-10-06T11:31:05.977319+0000","last_undegraded":"2023-10-06T11:31:05.977319+0000","last_fullsized":"2023-10-06T11:31:05.977319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:29.289659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.990333+0000","last_change":"2023-10-06T11:31:04.959440+0000","last_active":"2023-10-06T11:31:05.990333+0000","last_peered":"2023-10-06T11:31:05.990333+0000","last_clean":"2023-10-06T11:31:05.990333+0000","last_became_active":"2023-10-06T11:31:04.959164+0000","last_became_peered":"2023-10-06T11:31:04.959164+0000","last_unstale":"2023-10-06T11:31:05.990333+0000","last_undegraded":"2023-10-06T11:31:05.990333+0000","last_fullsized":"2023-10-06T11:31:05.990333+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:11.746894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976828+0000","last_change":"2023-10-06T11:31:05.972260+0000","last_active":"2023-10-06T11:31:05.976828+0000","last_peered":"2023-10-06T11:31:05.976828+0000","last_clean":"2023-10-06T11:31:05.976828+0000","last_became_active":"2023-10-06T11:31:05.971901+0000","last_became_peered":"2023-10-06T11:31:05.971901+0000","last_unstale":"2023-10-06T11:31:05.976828+0000","last_undegraded":"2023-10-06T11:31:05.976828+0000","last_fullsized":"2023-10-06T11:31:05.976828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:14.637348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966860+0000","last_change":"2023-10-06T11:31:04.965863+0000","last_active":"2023-10-06T11:31:05.966860+0000","last_peered":"2023-10-06T11:31:05.966860+0000","last_clean":"2023-10-06T11:31:05.966860+0000","last_became_active":"2023-10-06T11:31:04.965546+0000","last_became_peered":"2023-10-06T11:31:04.965546+0000","last_unstale":"2023-10-06T11:31:05.966860+0000","last_undegraded":"2023-10-06T11:31:05.966860+0000","last_fullsized":"2023-10-06T11:31:05.966860+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:04:10.244202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.970296+0000","last_change":"2023-10-06T11:31:05.965505+0000","last_active":"2023-10-06T11:31:05.970296+0000","last_peered":"2023-10-06T11:31:05.970296+0000","last_clean":"2023-10-06T11:31:05.970296+0000","last_became_active":"2023-10-06T11:31:05.965066+0000","last_became_peered":"2023-10-06T11:31:05.965066+0000","last_unstale":"2023-10-06T11:31:05.970296+0000","last_undegraded":"2023-10-06T11:31:05.970296+0000","last_fullsized":"2023-10-06T11:31:05.970296+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:06.955369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.965534+0000","last_change":"2023-10-06T11:31:04.959773+0000","last_active":"2023-10-06T11:31:05.965534+0000","last_peered":"2023-10-06T11:31:05.965534+0000","last_clean":"2023-10-06T11:31:05.965534+0000","last_became_active":"2023-10-06T11:31:04.959396+0000","last_became_peered":"2023-10-06T11:31:04.959396+0000","last_unstale":"2023-10-06T11:31:05.965534+0000","last_undegraded":"2023-10-06T11:31:05.965534+0000","last_fullsized":"2023-10-06T11:31:05.965534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:54:23.679912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.979019+0000","last_change":"2023-10-06T11:31:05.971161+0000","last_active":"2023-10-06T11:31:05.979019+0000","last_peered":"2023-10-06T11:31:05.979019+0000","last_clean":"2023-10-06T11:31:05.979019+0000","last_became_active":"2023-10-06T11:31:05.970883+0000","last_became_peered":"2023-10-06T11:31:05.970883+0000","last_unstale":"2023-10-06T11:31:05.979019+0000","last_undegraded":"2023-10-06T11:31:05.979019+0000","last_fullsized":"2023-10-06T11:31:05.979019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:42.472466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"61'8","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.995577+0000","last_change":"2023-10-06T11:31:04.953282+0000","last_active":"2023-10-06T11:31:05.995577+0000","last_peered":"2023-10-06T11:31:05.995577+0000","last_clean":"2023-10-06T11:31:05.995577+0000","last_became_active":"2023-10-06T11:31:04.953000+0000","last_became_peered":"2023-10-06T11:31:04.953000+0000","last_unstale":"2023-10-06T11:31:05.995577+0000","last_undegraded":"2023-10-06T11:31:05.995577+0000","last_fullsized":"2023-10-06T11:31:05.995577+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:49:03.903354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.967121+0000","last_change":"2023-10-06T11:31:05.959367+0000","last_active":"2023-10-06T11:31:05.967121+0000","last_peered":"2023-10-06T11:31:05.967121+0000","last_clean":"2023-10-06T11:31:05.967121+0000","last_became_active":"2023-10-06T11:31:05.959052+0000","last_became_peered":"2023-10-06T11:31:05.959052+0000","last_unstale":"2023-10-06T11:31:05.967121+0000","last_undegraded":"2023-10-06T11:31:05.967121+0000","last_fullsized":"2023-10-06T11:31:05.967121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:25:19.156295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"61'1","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992277+0000","last_change":"2023-10-06T11:31:04.966377+0000","last_active":"2023-10-06T11:31:05.992277+0000","last_peered":"2023-10-06T11:31:05.992277+0000","last_clean":"2023-10-06T11:31:05.992277+0000","last_became_active":"2023-10-06T11:31:04.966095+0000","last_became_peered":"2023-10-06T11:31:04.966095+0000","last_unstale":"2023-10-06T11:31:05.992277+0000","last_undegraded":"2023-10-06T11:31:05.992277+0000","last_fullsized":"2023-10-06T11:31:05.992277+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:17:29.760673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992716+0000","last_change":"2023-10-06T11:31:05.978206+0000","last_active":"2023-10-06T11:31:05.992716+0000","last_peered":"2023-10-06T11:31:05.992716+0000","last_clean":"2023-10-06T11:31:05.992716+0000","last_became_active":"2023-10-06T11:31:05.977908+0000","last_became_peered":"2023-10-06T11:31:05.977908+0000","last_unstale":"2023-10-06T11:31:05.992716+0000","last_undegraded":"2023-10-06T11:31:05.992716+0000","last_fullsized":"2023-10-06T11:31:05.992716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:49.814373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.993440+0000","last_change":"2023-10-06T11:31:04.950129+0000","last_active":"2023-10-06T11:31:05.993440+0000","last_peered":"2023-10-06T11:31:05.993440+0000","last_clean":"2023-10-06T11:31:05.993440+0000","last_became_active":"2023-10-06T11:31:04.949269+0000","last_became_peered":"2023-10-06T11:31:04.949269+0000","last_unstale":"2023-10-06T11:31:05.993440+0000","last_undegraded":"2023-10-06T11:31:05.993440+0000","last_fullsized":"2023-10-06T11:31:05.993440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:48:27.488365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.986120+0000","last_change":"2023-10-06T11:31:05.982032+0000","last_active":"2023-10-06T11:31:05.986120+0000","last_peered":"2023-10-06T11:31:05.986120+0000","last_clean":"2023-10-06T11:31:05.986120+0000","last_became_active":"2023-10-06T11:31:05.981700+0000","last_became_peered":"2023-10-06T11:31:05.981700+0000","last_unstale":"2023-10-06T11:31:05.986120+0000","last_undegraded":"2023-10-06T11:31:05.986120+0000","last_fullsized":"2023-10-06T11:31:05.986120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:20.101137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"61'1","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.999769+0000","last_change":"2023-10-06T11:31:04.972328+0000","last_active":"2023-10-06T11:31:05.999769+0000","last_peered":"2023-10-06T11:31:05.999769+0000","last_clean":"2023-10-06T11:31:05.999769+0000","last_became_active":"2023-10-06T11:31:04.972001+0000","last_became_peered":"2023-10-06T11:31:04.972001+0000","last_unstale":"2023-10-06T11:31:05.999769+0000","last_undegraded":"2023-10-06T11:31:05.999769+0000","last_fullsized":"2023-10-06T11:31:05.999769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:21.287736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964154+0000","last_change":"2023-10-06T11:31:05.960942+0000","last_active":"2023-10-06T11:31:05.964154+0000","last_peered":"2023-10-06T11:31:05.964154+0000","last_clean":"2023-10-06T11:31:05.964154+0000","last_became_active":"2023-10-06T11:31:05.960648+0000","last_became_peered":"2023-10-06T11:31:05.960648+0000","last_unstale":"2023-10-06T11:31:05.964154+0000","last_undegraded":"2023-10-06T11:31:05.964154+0000","last_fullsized":"2023-10-06T11:31:05.964154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:47.418804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966418+0000","last_change":"2023-10-06T11:31:04.970998+0000","last_active":"2023-10-06T11:31:05.966418+0000","last_peered":"2023-10-06T11:31:05.966418+0000","last_clean":"2023-10-06T11:31:05.966418+0000","last_became_active":"2023-10-06T11:31:04.970643+0000","last_became_peered":"2023-10-06T11:31:04.970643+0000","last_unstale":"2023-10-06T11:31:05.966418+0000","last_undegraded":"2023-10-06T11:31:05.966418+0000","last_fullsized":"2023-10-06T11:31:05.966418+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:43.730174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.969569+0000","last_change":"2023-10-06T11:31:05.963580+0000","last_active":"2023-10-06T11:31:05.969569+0000","last_peered":"2023-10-06T11:31:05.969569+0000","last_clean":"2023-10-06T11:31:05.969569+0000","last_became_active":"2023-10-06T11:31:05.963305+0000","last_became_peered":"2023-10-06T11:31:05.963305+0000","last_unstale":"2023-10-06T11:31:05.969569+0000","last_undegraded":"2023-10-06T11:31:05.969569+0000","last_fullsized":"2023-10-06T11:31:05.969569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:28.340213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992911+0000","last_change":"2023-10-06T11:31:04.959794+0000","last_active":"2023-10-06T11:31:05.992911+0000","last_peered":"2023-10-06T11:31:05.992911+0000","last_clean":"2023-10-06T11:31:05.992911+0000","last_became_active":"2023-10-06T11:31:04.959512+0000","last_became_peered":"2023-10-06T11:31:04.959512+0000","last_unstale":"2023-10-06T11:31:05.992911+0000","last_undegraded":"2023-10-06T11:31:05.992911+0000","last_fullsized":"2023-10-06T11:31:05.992911+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:17.543900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.988103+0000","last_change":"2023-10-06T11:31:05.983644+0000","last_active":"2023-10-06T11:31:05.988103+0000","last_peered":"2023-10-06T11:31:05.988103+0000","last_clean":"2023-10-06T11:31:05.988103+0000","last_became_active":"2023-10-06T11:31:05.983292+0000","last_became_peered":"2023-10-06T11:31:05.983292+0000","last_unstale":"2023-10-06T11:31:05.988103+0000","last_undegraded":"2023-10-06T11:31:05.988103+0000","last_fullsized":"2023-10-06T11:31:05.988103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:44.147552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.950285+0000","last_change":"2023-10-06T11:31:04.955142+0000","last_active":"2023-10-06T11:31:05.950285+0000","last_peered":"2023-10-06T11:31:05.950285+0000","last_clean":"2023-10-06T11:31:05.950285+0000","last_became_active":"2023-10-06T11:31:04.952665+0000","last_became_peered":"2023-10-06T11:31:04.952665+0000","last_unstale":"2023-10-06T11:31:05.950285+0000","last_undegraded":"2023-10-06T11:31:05.950285+0000","last_fullsized":"2023-10-06T11:31:05.950285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:43.132123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976634+0000","last_change":"2023-10-06T11:31:05.968996+0000","last_active":"2023-10-06T11:31:05.976634+0000","last_peered":"2023-10-06T11:31:05.976634+0000","last_clean":"2023-10-06T11:31:05.976634+0000","last_became_active":"2023-10-06T11:31:05.968715+0000","last_became_peered":"2023-10-06T11:31:05.968715+0000","last_unstale":"2023-10-06T11:31:05.976634+0000","last_undegraded":"2023-10-06T11:31:05.976634+0000","last_fullsized":"2023-10-06T11:31:05.976634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:22.694289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.958523+0000","last_change":"2023-10-06T11:31:04.966363+0000","last_active":"2023-10-06T11:31:05.958523+0000","last_peered":"2023-10-06T11:31:05.958523+0000","last_clean":"2023-10-06T11:31:05.958523+0000","last_became_active":"2023-10-06T11:31:04.966138+0000","last_became_peered":"2023-10-06T11:31:04.966138+0000","last_unstale":"2023-10-06T11:31:05.958523+0000","last_undegraded":"2023-10-06T11:31:05.958523+0000","last_fullsized":"2023-10-06T11:31:05.958523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:29.382230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964179+0000","last_change":"2023-10-06T11:31:05.961033+0000","last_active":"2023-10-06T11:31:05.964179+0000","last_peered":"2023-10-06T11:31:05.964179+0000","last_clean":"2023-10-06T11:31:05.964179+0000","last_became_active":"2023-10-06T11:31:05.960321+0000","last_became_peered":"2023-10-06T11:31:05.960321+0000","last_unstale":"2023-10-06T11:31:05.964179+0000","last_undegraded":"2023-10-06T11:31:05.964179+0000","last_fullsized":"2023-10-06T11:31:05.964179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:58.508008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.940578+0000","last_change":"2023-10-06T11:31:04.954490+0000","last_active":"2023-10-06T11:31:05.940578+0000","last_peered":"2023-10-06T11:31:05.940578+0000","last_clean":"2023-10-06T11:31:05.940578+0000","last_became_active":"2023-10-06T11:31:04.953934+0000","last_became_peered":"2023-10-06T11:31:04.953934+0000","last_unstale":"2023-10-06T11:31:05.940578+0000","last_undegraded":"2023-10-06T11:31:05.940578+0000","last_fullsized":"2023-10-06T11:31:05.940578+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:02.693786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.975087+0000","last_change":"2023-10-06T11:31:05.972787+0000","last_active":"2023-10-06T11:31:05.975087+0000","last_peered":"2023-10-06T11:31:05.975087+0000","last_clean":"2023-10-06T11:31:05.975087+0000","last_became_active":"2023-10-06T11:31:05.972428+0000","last_became_peered":"2023-10-06T11:31:05.972428+0000","last_unstale":"2023-10-06T11:31:05.975087+0000","last_undegraded":"2023-10-06T11:31:05.975087+0000","last_fullsized":"2023-10-06T11:31:05.975087+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:48:28.503794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.956430+0000","last_change":"2023-10-06T11:31:04.960135+0000","last_active":"2023-10-06T11:31:05.956430+0000","last_peered":"2023-10-06T11:31:05.956430+0000","last_clean":"2023-10-06T11:31:05.956430+0000","last_became_active":"2023-10-06T11:31:04.958785+0000","last_became_peered":"2023-10-06T11:31:04.958785+0000","last_unstale":"2023-10-06T11:31:05.956430+0000","last_undegraded":"2023-10-06T11:31:05.956430+0000","last_fullsized":"2023-10-06T11:31:05.956430+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:30.429172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"51'139","reported_seq":278,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964226+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:31:05.964226+0000","last_peered":"2023-10-06T11:31:05.964226+0000","last_clean":"2023-10-06T11:31:05.964226+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:31:05.964226+0000","last_undegraded":"2023-10-06T11:31:05.964226+0000","last_fullsized":"2023-10-06T11:31:05.964226+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.954031+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:31:05.954031+0000","last_peered":"2023-10-06T11:31:05.954031+0000","last_clean":"2023-10-06T11:31:05.954031+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:31:05.954031+0000","last_undegraded":"2023-10-06T11:31:05.954031+0000","last_fullsized":"2023-10-06T11:31:05.954031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.992863+0000","last_change":"2023-10-06T11:31:05.983176+0000","last_active":"2023-10-06T11:31:05.992863+0000","last_peered":"2023-10-06T11:31:05.992863+0000","last_clean":"2023-10-06T11:31:05.992863+0000","last_became_active":"2023-10-06T11:31:05.982849+0000","last_became_peered":"2023-10-06T11:31:05.982849+0000","last_unstale":"2023-10-06T11:31:05.992863+0000","last_undegraded":"2023-10-06T11:31:05.992863+0000","last_fullsized":"2023-10-06T11:31:05.992863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:38.388915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.965688+0000","last_change":"2023-10-06T11:31:04.960120+0000","last_active":"2023-10-06T11:31:05.965688+0000","last_peered":"2023-10-06T11:31:05.965688+0000","last_clean":"2023-10-06T11:31:05.965688+0000","last_became_active":"2023-10-06T11:31:04.959804+0000","last_became_peered":"2023-10-06T11:31:04.959804+0000","last_unstale":"2023-10-06T11:31:05.965688+0000","last_undegraded":"2023-10-06T11:31:05.965688+0000","last_fullsized":"2023-10-06T11:31:05.965688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:48.798944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.964144+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:31:05.964144+0000","last_peered":"2023-10-06T11:31:05.964144+0000","last_clean":"2023-10-06T11:31:05.964144+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:31:05.964144+0000","last_undegraded":"2023-10-06T11:31:05.964144+0000","last_fullsized":"2023-10-06T11:31:05.964144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.970226+0000","last_change":"2023-10-06T11:31:05.963808+0000","last_active":"2023-10-06T11:31:05.970226+0000","last_peered":"2023-10-06T11:31:05.970226+0000","last_clean":"2023-10-06T11:31:05.970226+0000","last_became_active":"2023-10-06T11:31:05.959888+0000","last_became_peered":"2023-10-06T11:31:05.959888+0000","last_unstale":"2023-10-06T11:31:05.970226+0000","last_undegraded":"2023-10-06T11:31:05.970226+0000","last_fullsized":"2023-10-06T11:31:05.970226+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:00:28.340213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.950215+0000","last_change":"2023-10-06T11:31:04.957337+0000","last_active":"2023-10-06T11:31:05.950215+0000","last_peered":"2023-10-06T11:31:05.950215+0000","last_clean":"2023-10-06T11:31:05.950215+0000","last_became_active":"2023-10-06T11:31:04.957058+0000","last_became_peered":"2023-10-06T11:31:04.957058+0000","last_unstale":"2023-10-06T11:31:05.950215+0000","last_undegraded":"2023-10-06T11:31:05.950215+0000","last_fullsized":"2023-10-06T11:31:05.950215+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:48.798944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.940702+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:31:05.940702+0000","last_peered":"2023-10-06T11:31:05.940702+0000","last_clean":"2023-10-06T11:31:05.940702+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:31:05.940702+0000","last_undegraded":"2023-10-06T11:31:05.940702+0000","last_fullsized":"2023-10-06T11:31:05.940702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966534+0000","last_change":"2023-10-06T11:31:05.962569+0000","last_active":"2023-10-06T11:31:05.966534+0000","last_peered":"2023-10-06T11:31:05.966534+0000","last_clean":"2023-10-06T11:31:05.966534+0000","last_became_active":"2023-10-06T11:31:05.961613+0000","last_became_peered":"2023-10-06T11:31:05.961613+0000","last_unstale":"2023-10-06T11:31:05.966534+0000","last_undegraded":"2023-10-06T11:31:05.966534+0000","last_fullsized":"2023-10-06T11:31:05.966534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:20.101137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"61'2","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.993012+0000","last_change":"2023-10-06T11:31:04.967346+0000","last_active":"2023-10-06T11:31:05.993012+0000","last_peered":"2023-10-06T11:31:05.993012+0000","last_clean":"2023-10-06T11:31:05.993012+0000","last_became_active":"2023-10-06T11:31:04.966999+0000","last_became_peered":"2023-10-06T11:31:04.966999+0000","last_unstale":"2023-10-06T11:31:05.993012+0000","last_undegraded":"2023-10-06T11:31:05.993012+0000","last_fullsized":"2023-10-06T11:31:05.993012+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:12.399626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.953844+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:31:05.953844+0000","last_peered":"2023-10-06T11:31:05.953844+0000","last_clean":"2023-10-06T11:31:05.953844+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:31:05.953844+0000","last_undegraded":"2023-10-06T11:31:05.953844+0000","last_fullsized":"2023-10-06T11:31:05.953844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.976752+0000","last_change":"2023-10-06T11:31:05.972605+0000","last_active":"2023-10-06T11:31:05.976752+0000","last_peered":"2023-10-06T11:31:05.976752+0000","last_clean":"2023-10-06T11:31:05.976752+0000","last_became_active":"2023-10-06T11:31:05.970501+0000","last_became_peered":"2023-10-06T11:31:05.970501+0000","last_unstale":"2023-10-06T11:31:05.976752+0000","last_undegraded":"2023-10-06T11:31:05.976752+0000","last_fullsized":"2023-10-06T11:31:05.976752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:27:33.008848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.953706+0000","last_change":"2023-10-06T11:31:04.969690+0000","last_active":"2023-10-06T11:31:05.953706+0000","last_peered":"2023-10-06T11:31:05.953706+0000","last_clean":"2023-10-06T11:31:05.953706+0000","last_became_active":"2023-10-06T11:31:04.969311+0000","last_became_peered":"2023-10-06T11:31:04.969311+0000","last_unstale":"2023-10-06T11:31:05.953706+0000","last_undegraded":"2023-10-06T11:31:05.953706+0000","last_fullsized":"2023-10-06T11:31:05.953706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:05:30.429172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.953797+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:31:05.953797+0000","last_peered":"2023-10-06T11:31:05.953797+0000","last_clean":"2023-10-06T11:31:05.953797+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:31:05.953797+0000","last_undegraded":"2023-10-06T11:31:05.953797+0000","last_fullsized":"2023-10-06T11:31:05.953797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.955337+0000","last_change":"2023-10-06T11:31:05.952209+0000","last_active":"2023-10-06T11:31:05.955337+0000","last_peered":"2023-10-06T11:31:05.955337+0000","last_clean":"2023-10-06T11:31:05.955337+0000","last_became_active":"2023-10-06T11:31:05.951829+0000","last_became_peered":"2023-10-06T11:31:05.951829+0000","last_unstale":"2023-10-06T11:31:05.955337+0000","last_undegraded":"2023-10-06T11:31:05.955337+0000","last_fullsized":"2023-10-06T11:31:05.955337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:30:49.569367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.954371+0000","last_change":"2023-10-06T11:31:04.962444+0000","last_active":"2023-10-06T11:31:05.954371+0000","last_peered":"2023-10-06T11:31:05.954371+0000","last_clean":"2023-10-06T11:31:05.954371+0000","last_became_active":"2023-10-06T11:31:04.962170+0000","last_became_peered":"2023-10-06T11:31:04.962170+0000","last_unstale":"2023-10-06T11:31:05.954371+0000","last_undegraded":"2023-10-06T11:31:05.954371+0000","last_fullsized":"2023-10-06T11:31:05.954371+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:29.382230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.965764+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:31:05.965764+0000","last_peered":"2023-10-06T11:31:05.965764+0000","last_clean":"2023-10-06T11:31:05.965764+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:31:05.965764+0000","last_undegraded":"2023-10-06T11:31:05.965764+0000","last_fullsized":"2023-10-06T11:31:05.965764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.978093+0000","last_change":"2023-10-06T11:31:05.975151+0000","last_active":"2023-10-06T11:31:05.978093+0000","last_peered":"2023-10-06T11:31:05.978093+0000","last_clean":"2023-10-06T11:31:05.978093+0000","last_became_active":"2023-10-06T11:31:05.974718+0000","last_became_peered":"2023-10-06T11:31:05.974718+0000","last_unstale":"2023-10-06T11:31:05.978093+0000","last_undegraded":"2023-10-06T11:31:05.978093+0000","last_fullsized":"2023-10-06T11:31:05.978093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:40.905512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.950115+0000","last_change":"2023-10-06T11:31:04.953565+0000","last_active":"2023-10-06T11:31:05.950115+0000","last_peered":"2023-10-06T11:31:05.950115+0000","last_clean":"2023-10-06T11:31:05.950115+0000","last_became_active":"2023-10-06T11:31:04.952602+0000","last_became_peered":"2023-10-06T11:31:04.952602+0000","last_unstale":"2023-10-06T11:31:05.950115+0000","last_undegraded":"2023-10-06T11:31:05.950115+0000","last_fullsized":"2023-10-06T11:31:05.950115+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:14:46.403375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.956470+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:31:05.956470+0000","last_peered":"2023-10-06T11:31:05.956470+0000","last_clean":"2023-10-06T11:31:05.956470+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:31:05.956470+0000","last_undegraded":"2023-10-06T11:31:05.956470+0000","last_fullsized":"2023-10-06T11:31:05.956470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.977387+0000","last_change":"2023-10-06T11:31:05.969713+0000","last_active":"2023-10-06T11:31:05.977387+0000","last_peered":"2023-10-06T11:31:05.977387+0000","last_clean":"2023-10-06T11:31:05.977387+0000","last_became_active":"2023-10-06T11:31:05.969367+0000","last_became_peered":"2023-10-06T11:31:05.969367+0000","last_unstale":"2023-10-06T11:31:05.977387+0000","last_undegraded":"2023-10-06T11:31:05.977387+0000","last_fullsized":"2023-10-06T11:31:05.977387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:04.917143+0000","last_clean_scrub_stamp":"2023-10-06T11:31:04.917143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:06.955369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.950397+0000","last_change":"2023-10-06T11:31:04.956355+0000","last_active":"2023-10-06T11:31:05.950397+0000","last_peered":"2023-10-06T11:31:05.950397+0000","last_clean":"2023-10-06T11:31:05.950397+0000","last_became_active":"2023-10-06T11:31:04.956027+0000","last_became_peered":"2023-10-06T11:31:04.956027+0000","last_unstale":"2023-10-06T11:31:05.950397+0000","last_undegraded":"2023-10-06T11:31:05.950397+0000","last_fullsized":"2023-10-06T11:31:05.950397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:31:03.901714+0000","last_clean_scrub_stamp":"2023-10-06T11:31:03.901714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:57.492579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-06T11:31:05.966764+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:31:05.966764+0000","last_peered":"2023-10-06T11:31:05.966764+0000","last_clean":"2023-10-06T11:31:05.966764+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:31:05.966764+0000","last_undegraded":"2023-10-06T11:31:05.966764+0000","last_fullsized":"2023-10-06T11:31:05.966764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310884,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060956,"statfs":{"total":96636764160,"available":96318418944,"internally_reserved":0,"allocated":1339392,"data_stored":950173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.157,"15min":1.1419999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.51,"5min":1.6499999999999999,"15min":1.827},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.127,"15min":1.153},"min":{"1min":0.749,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.649,"5min":1.649,"15min":2.0030000000000001},"last":1.4550000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.3049999999999999,"15min":1.325},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.6059999999999999},"last":1.292},{"interface":"front","average":{"1min":1.175,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.502,"5min":1.6379999999999999,"15min":1.931},"last":1.2669999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2350000000000001,"15min":1.2110000000000001},"min":{"1min":1.0229999999999999,"5min":0.66300000000000003,"15min":0.65500000000000003},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":2.3799999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.79100000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.528,"5min":1.7210000000000001,"15min":2.2850000000000001},"last":1.522}]},{"osd":3,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.274,"15min":1.286},"min":{"1min":0.79600000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5169999999999999,"5min":1.6970000000000001,"15min":2.665},"last":1.536},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.3029999999999999,"15min":1.351},"min":{"1min":0.97699999999999998,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.5980000000000001,"5min":1.764,"15min":2.8570000000000002},"last":1.1759999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.101,"15min":1.163},"min":{"1min":0.751,"5min":0.65900000000000003,"15min":0.63600000000000001},"max":{"1min":2.149,"5min":2.149,"15min":2.7970000000000002},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5600000000000001,"5min":1.8580000000000001,"15min":2.5339999999999998},"last":1.3320000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2769999999999999,"15min":1.3560000000000001},"min":{"1min":0.89700000000000002,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7030000000000001,"5min":1.7709999999999999,"15min":2.9529999999999998},"last":1.363},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2829999999999999,"15min":1.3069999999999999},"min":{"1min":0.81000000000000005,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":3.105},"last":1.115}]},{"osd":6,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":1.0920000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":3.004},"last":1.214},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4059999999999999,"15min":1.4259999999999999},"min":{"1min":0.93300000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.913,"5min":2.0579999999999998,"15min":2.9119999999999999},"last":1.49}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302112,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069728,"statfs":{"total":96636764160,"available":96327401472,"internally_reserved":0,"allocated":745472,"data_stored":360305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.7030000000000001,"5min":1.73,"15min":2.1379999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3029999999999999,"15min":1.3169999999999999},"min":{"1min":0.76100000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":2.1920000000000002},"last":1.4079999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.92300000000000004,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.4670000000000001,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2310000000000001,"15min":1.204},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5669999999999999,"5min":1.669,"15min":1.8480000000000001},"last":1.1399999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.74299999999999999},"max":{"1min":1.4630000000000001,"5min":1.5980000000000001,"15min":1.6919999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.2370000000000001,"15min":1.244},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.73799999999999999},"max":{"1min":1.587,"5min":1.615,"15min":2.7789999999999999},"last":1.1950000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3029999999999999,"15min":1.3129999999999999},"min":{"1min":0.97699999999999998,"5min":0.91900000000000004,"15min":0.78400000000000003},"max":{"1min":1.764,"5min":1.764,"15min":1.9850000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":0.997,"15min":0.83199999999999996},"max":{"1min":1.5109999999999999,"5min":1.724,"15min":2.109},"last":1.5649999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2130000000000001,"15min":1.2350000000000001},"min":{"1min":0.97199999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.6539999999999999,"5min":1.8,"15min":1.956},"last":1.335},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2070000000000001,"15min":1.252},"min":{"1min":0.89100000000000001,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.7,"5min":1.8009999999999999,"15min":1.8879999999999999},"last":1.6499999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.242,"15min":1.26},"min":{"1min":0.92500000000000004,"5min":0.75,"15min":0.75},"max":{"1min":1.52,"5min":1.72,"15min":2.198},"last":1.375},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.361,"15min":1.397},"min":{"1min":0.92600000000000005,"5min":0.85199999999999998,"15min":0.71799999999999997},"max":{"1min":1.827,"5min":1.879,"15min":2.1509999999999998},"last":1.534}]},{"osd":7,"last update":"Fri Oct 6 11:30:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.429,"15min":1.472},"min":{"1min":1.052,"5min":0.878,"15min":0.878},"max":{"1min":1.7809999999999999,"5min":1.927,"15min":2.2919999999999998},"last":1.71},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4730000000000001},"min":{"1min":0.751,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6639999999999999,"5min":1.867,"15min":3.891},"last":1.5129999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302104,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069736,"statfs":{"total":96636764160,"available":96327409664,"internally_reserved":0,"allocated":737280,"data_stored":359681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2350000000000001,"15min":1.1990000000000001},"min":{"1min":0.754,"5min":0.67800000000000005,"15min":0.60399999999999998},"max":{"1min":1.617,"5min":1.734,"15min":2.306},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.292,"15min":1.3080000000000001},"min":{"1min":1.0669999999999999,"5min":0.80800000000000005,"15min":0.64900000000000002},"max":{"1min":1.536,"5min":1.6870000000000001,"15min":2.544},"last":1.498}]},{"osd":1,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.177,"15min":1.196},"min":{"1min":0.98499999999999999,"5min":0.64300000000000002,"15min":0.53200000000000003},"max":{"1min":1.516,"5min":1.673,"15min":2.4209999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.129,"15min":1.208},"min":{"1min":0.83199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4610000000000001,"5min":1.589,"15min":2.9910000000000001},"last":1.794}]},{"osd":2,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.159,"15min":1.2410000000000001},"min":{"1min":0.77500000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.494,"5min":1.5840000000000001,"15min":3.609},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2310000000000001,"15min":1.264},"min":{"1min":0.88200000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.49,"5min":1.738,"15min":2.6949999999999998},"last":1.4570000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.208,"15min":1.294},"min":{"1min":0.94499999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.587,"5min":1.7270000000000001,"15min":3.1429999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3220000000000001,"15min":1.3959999999999999},"min":{"1min":1.0469999999999999,"5min":0.90000000000000002,"15min":0.77700000000000002},"max":{"1min":1.5509999999999999,"5min":1.6439999999999999,"15min":3.7770000000000001},"last":1.734}]},{"osd":4,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2090000000000001,"15min":1.262},"min":{"1min":0.89400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5649999999999999,"5min":1.73,"15min":3.0800000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.278,"15min":1.4039999999999999},"min":{"1min":0.93100000000000005,"5min":0.63500000000000001,"15min":0.59799999999999998},"max":{"1min":1.669,"5min":1.6719999999999999,"15min":3.4100000000000001},"last":1.9099999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2210000000000001,"15min":1.367},"min":{"1min":0.89000000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.6879999999999999,"5min":1.7629999999999999,"15min":3.8999999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2749999999999999,"15min":1.375},"min":{"1min":0.81999999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.571,"5min":1.7529999999999999,"15min":3.3530000000000002},"last":1.5369999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3169999999999999,"15min":1.431},"min":{"1min":1.1140000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":3.5070000000000001},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.363,"15min":1.496},"min":{"1min":1.0329999999999999,"5min":0.84599999999999997,"15min":0.65400000000000003},"max":{"1min":1.782,"5min":1.8109999999999999,"15min":4.0529999999999999},"last":1.873}]}]},{"osd":4,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302108,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069732,"statfs":{"total":96636764160,"available":96327405568,"internally_reserved":0,"allocated":741376,"data_stored":360235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.133,"15min":1.1519999999999999},"min":{"1min":0.66400000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5189999999999999,"5min":1.633,"15min":2.1789999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.169,"5min":1.1319999999999999,"15min":1.169},"min":{"1min":0.81699999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.444,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3200000000000001,"15min":1.353},"min":{"1min":0.90400000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.607,"5min":1.6830000000000001,"15min":2.1989999999999998},"last":1.448},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2609999999999999,"15min":1.27},"min":{"1min":1.099,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.851},"last":1.0609999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1859999999999999,"15min":1.238},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69999999999999996},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":2.2269999999999999},"last":1.194},{"interface":"front","average":{"1min":1.276,"5min":1.244,"15min":1.2769999999999999},"min":{"1min":1.0229999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.488,"5min":1.5840000000000001,"15min":2.1150000000000002},"last":1.5860000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2170000000000001,"15min":1.22},"min":{"1min":1.052,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.671,"5min":1.671,"15min":1.6839999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.339,"5min":1.278,"15min":1.3280000000000001},"min":{"1min":0.98999999999999999,"5min":0.82999999999999996,"15min":0.77400000000000002},"max":{"1min":1.657,"5min":1.6930000000000001,"15min":2.258},"last":1.0960000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.173,"15min":1.2669999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.556,"5min":1.7390000000000001,"15min":2.8250000000000002},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1930000000000001,"15min":1.212},"min":{"1min":0.84499999999999997,"5min":0.75800000000000001,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.24,"15min":1.2989999999999999},"min":{"1min":0.91200000000000003,"5min":0.66200000000000003,"15min":0.54900000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3460000000000001,"15min":1.4039999999999999},"min":{"1min":0.97699999999999998,"5min":0.83899999999999997,"15min":0.77000000000000002},"max":{"1min":1.667,"5min":1.8069999999999999,"15min":2.4329999999999998},"last":1.71}]},{"osd":7,"last update":"Fri Oct 6 11:30:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.339,"15min":1.3919999999999999},"min":{"1min":1.083,"5min":0.70299999999999996,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":1.841,"15min":2.653},"last":1.173},{"interface":"front","average":{"1min":1.496,"5min":1.369,"15min":1.4410000000000001},"min":{"1min":1.262,"5min":0.75600000000000001,"15min":0.70499999999999996},"max":{"1min":1.7949999999999999,"5min":1.8029999999999999,"15min":2.464},"last":1.51}]}]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306216,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065624,"statfs":{"total":96636764160,"available":96323198976,"internally_reserved":0,"allocated":753664,"data_stored":361239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.85999999999999999,"15min":0.88900000000000001},"min":{"1min":0.51000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90100000000000002,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.196},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.998,"15min":0.98499999999999999},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.437},"max":{"1min":1.3380000000000001,"5min":1.5289999999999999,"15min":1.627},"last":1.1200000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.92500000000000004,"15min":0.95599999999999996},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.58399999999999996},"max":{"1min":1.284,"5min":1.448,"15min":1.516},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.089,"15min":1.083},"min":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.135},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.1890000000000001,"15min":1.22},"min":{"1min":0.76000000000000001,"5min":0.752,"15min":0.64700000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.7030000000000001},"last":1.075}]},{"osd":4,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1950000000000001,"15min":1.258},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.4430000000000001,"5min":1.639,"15min":2.6640000000000001},"last":1.24},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.3799999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.569,"5min":1.9730000000000001,"15min":2.0840000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.381,"15min":1.4330000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.89000000000000001},"max":{"1min":1.883,"5min":2.1309999999999998,"15min":2.1699999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.395,"15min":1.429},"min":{"1min":1.0880000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.605,"5min":2.113,"15min":2.1179999999999999},"last":1.3160000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.4510000000000001,"15min":1.4670000000000001},"min":{"1min":1.1850000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.663,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.357,"15min":1.405},"min":{"1min":1.0169999999999999,"5min":0.93999999999999995,"15min":0.89800000000000002},"max":{"1min":1.7589999999999999,"5min":1.8180000000000001,"15min":2.137},"last":1.5409999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.417,"15min":1.4730000000000001},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0089999999999999},"max":{"1min":1.7290000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.466,"15min":1.5049999999999999},"min":{"1min":1.1539999999999999,"5min":1.018,"15min":0.93000000000000005},"max":{"1min":2.004,"5min":2.266,"15min":2.266},"last":1.2909999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306212,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065628,"statfs":{"total":96636764160,"available":96323203072,"internally_reserved":0,"allocated":749568,"data_stored":361217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.97999999999999998,"15min":0.96999999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.42699999999999999},"max":{"1min":1.4430000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.206},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.46400000000000002},"max":{"1min":1.3169999999999999,"5min":1.5629999999999999,"15min":1.9650000000000001},"last":0.92100000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.94899999999999995,"15min":0.93100000000000005},"min":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.54900000000000004},"max":{"1min":1.631,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.048,"15min":1.0449999999999999},"min":{"1min":0.82799999999999996,"5min":0.54500000000000004,"15min":0.439},"max":{"1min":1.6779999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.3200000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1899999999999999,"15min":1.1970000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8,"5min":1.8240000000000001,"15min":1.917},"last":1.075},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.327,"15min":1.3120000000000001},"min":{"1min":0.96999999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.631,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.548}]},{"osd":4,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.216,"15min":1.2250000000000001},"min":{"1min":0.84699999999999998,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.595,"5min":1.754,"15min":2.1899999999999999},"last":1.615},{"interface":"front","average":{"1min":1.25,"5min":1.2689999999999999,"15min":1.331},"min":{"1min":0.93200000000000005,"5min":0.86199999999999999,"15min":0.84399999999999997},"max":{"1min":1.6819999999999999,"5min":2.024,"15min":2.6920000000000002},"last":1.661}]},{"osd":5,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.369,"15min":1.3879999999999999},"min":{"1min":1.0489999999999999,"5min":0.86899999999999999,"15min":0.86099999999999999},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.601},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.91500000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.6859999999999999,"5min":1.778,"15min":2.6400000000000001},"last":1.4259999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.417,"15min":1.4530000000000001},"min":{"1min":1.0229999999999999,"5min":0.996,"15min":0.996},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":2.8340000000000001},"last":1.486},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4590000000000001,"15min":1.4670000000000001},"min":{"1min":1.2769999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9910000000000001,"5min":2.617,"15min":2.617},"last":1.702}]},{"osd":7,"last update":"Fri Oct 6 11:30:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3740000000000001,"15min":1.4079999999999999},"min":{"1min":1.002,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.726,"5min":1.863,"15min":2.9020000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4690000000000001,"15min":1.498},"min":{"1min":1.0940000000000001,"5min":0.93799999999999994,"15min":0.85199999999999998},"max":{"1min":1.8100000000000001,"5min":2.2829999999999999,"15min":3.2240000000000002},"last":1.28}]}]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302112,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069728,"statfs":{"total":96636764160,"available":96327401472,"internally_reserved":0,"allocated":745472,"data_stored":359841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.623,"5min":0.60999999999999999,"15min":0.58099999999999996},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.65300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.319,"5min":1.4259999999999999,"15min":1.855},"last":1.401}]},{"osd":2,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.996,"15min":0.98199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.21,"5min":1.133,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.0149999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.2609999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.452},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1240000000000001,"15min":1.141},"min":{"1min":0.77300000000000002,"5min":0.67300000000000004,"15min":0.58199999999999996},"max":{"1min":1.417,"5min":1.7210000000000001,"15min":4.0830000000000002},"last":1.1919999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.23,"15min":1.3069999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.117},{"interface":"front","average":{"1min":1.399,"5min":1.3919999999999999,"15min":1.4419999999999999},"min":{"1min":1.0189999999999999,"5min":0.96399999999999997,"15min":0.78000000000000003},"max":{"1min":1.6759999999999999,"5min":1.865,"15min":3.5609999999999999},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.296,"15min":1.363},"min":{"1min":0.95299999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.704,"5min":1.716,"15min":3.2559999999999998},"last":1.512},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.333,"15min":1.3879999999999999},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.85999999999999999},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.766},"last":1.22}]},{"osd":6,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.48,"15min":1.512},"min":{"1min":1.2390000000000001,"5min":1.0680000000000001,"15min":0.94299999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.258},{"interface":"front","average":{"1min":1.413,"5min":1.399,"15min":1.4530000000000001},"min":{"1min":1.036,"5min":1.0109999999999999,"15min":0.97999999999999998},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.2969999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:30:42 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4950000000000001,"15min":1.5600000000000001},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.782,"5min":1.8460000000000001,"15min":3.9620000000000002},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.421,"15min":1.502},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.708,"5min":1.9079999999999999,"15min":2.8740000000000001},"last":1.534}]}]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306788,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065052,"statfs":{"total":96636764160,"available":96322613248,"internally_reserved":0,"allocated":1339392,"data_stored":950105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.90500000000000003,"15min":0.88900000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.1439999999999999,"5min":1.571,"15min":1.571},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.94799999999999995,"15min":0.96799999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.492,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.113}]},{"osd":2,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"max":{"1min":1.351,"5min":1.5289999999999999,"15min":2.1989999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0700000000000001,"15min":1.093},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.327,"5min":1.6160000000000001,"15min":2.7719999999999998},"last":0.89000000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0740000000000001,"15min":1.0660000000000001},"min":{"1min":0.78600000000000003,"5min":0.68000000000000005,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.7030000000000001,"15min":1.919},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.244,"5min":1.327,"15min":1.284},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.61499999999999999},"max":{"1min":1.5600000000000001,"5min":1.7789999999999999,"15min":2.3530000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3620000000000001,"15min":1.3500000000000001},"min":{"1min":1.0489999999999999,"5min":1.018,"15min":0.68999999999999995},"max":{"1min":1.627,"5min":2.294,"15min":2.294},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.3049999999999999,"15min":1.292},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.597,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.232}]},{"osd":5,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3600000000000001,"15min":1.3819999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6220000000000001,"5min":1.881,"15min":2.573},"last":1.2},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3740000000000001,"15min":1.4359999999999999},"min":{"1min":1.135,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.659,"5min":1.982,"15min":2.8650000000000002},"last":1.3340000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3280000000000001,"15min":1.337},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.88900000000000001},"max":{"1min":1.6799999999999999,"5min":1.9410000000000001,"15min":2.52},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4410000000000001,"15min":1.4730000000000001},"min":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.7210000000000001,"5min":1.944,"15min":2.742},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:30:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.454,"15min":1.5329999999999999},"min":{"1min":1.024,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.73,"5min":1.9219999999999999,"15min":4.1159999999999997},"last":1.262},{"interface":"front","average":{"1min":1.329,"5min":1.3799999999999999,"15min":1.4339999999999999},"min":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.75,"5min":1.8340000000000001,"15min":2.3980000000000001},"last":1.3029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:31:10.746 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:31:10.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:31:10.902 INFO:tasks.ceph.mds.c.smithi046.stderr:2023-10-06T11:31:10.890+0000 7f7e440f3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:31:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 Processor -- start 2023-10-06T11:31:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 -- start start 2023-10-06T11:31:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9180b2320 con 0x7fa9180b1ed0 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180af930 con 0x7fa9180af4b0 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91f745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa918159d70 con 0x7fa918068290 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91dce2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91d4e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91dce2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34752/0 (socket says 172.21.15.46:34752) 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91dce2700 1 -- 172.21.15.46:0/2971280565 learned_addr learned my addr 172.21.15.46:0/2971280565 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.950+0000 7fa91cce0700 1 --1- 172.21.15.46:0/2971280565 >> v1:172.21.15.46:6789/0 conn(0x7fa9180b1ed0 0x7fa9180adaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49256/0 (socket says 172.21.15.46:49256) 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91d4e1700 1 -- 172.21.15.46:0/2971280565 >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 msgr2=0x7fa9180ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/2971280565 >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91d4e1700 1 -- 172.21.15.46:0/2971280565 >> v1:172.21.15.46:6789/0 conn(0x7fa9180b1ed0 legacy=0x7fa9180adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91d4e1700 1 -- 172.21.15.46:0/2971280565 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180ad810 con 0x7fa918068290 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa90ffff700 1 -- 172.21.15.46:0/2971280565 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591166826 0 0) 0x7fa9180b2320 con 0x7fa9180b1ed0 2023-10-06T11:31:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91dce2700 1 --2- 172.21.15.46:0/2971280565 >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/2971280565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918068660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098e0 tx=0x7fa9140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73c77b568bf441c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa90ffff700 1 -- 172.21.15.46:0/2971280565 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400b640 con 0x7fa918068290 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa90ffff700 1 -- 172.21.15.46:0/2971280565 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91400b7a0 con 0x7fa918068290 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa90ffff700 1 -- 172.21.15.46:0/2971280565 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400bb40 con 0x7fa918068290 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 -- 172.21.15.46:0/2971280565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 msgr2=0x7fa918068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 --2- 172.21.15.46:0/2971280565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918068660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098e0 tx=0x7fa9140091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 -- 172.21.15.46:0/2971280565 shutdown_connections 2023-10-06T11:31:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 --2- 172.21.15.46:0/2971280565 >> v2:172.21.15.46:3300/0 conn(0x7fa9180af4b0 0x7fa9180ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 --2- 172.21.15.46:0/2971280565 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918068660 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 -- 172.21.15.46:0/2971280565 >> 172.21.15.46:0/2971280565 conn(0x7fa91806d490 msgr2=0x7fa91806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 -- 172.21.15.46:0/2971280565 shutdown_connections 2023-10-06T11:31:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.954+0000 7fa91f745700 1 -- 172.21.15.46:0/2971280565 wait complete. 2023-10-06T11:31:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.958+0000 7fa91f745700 1 Processor -- start 2023-10-06T11:31:11.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.958+0000 7fa91f745700 1 -- start start 2023-10-06T11:31:11.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9180b1ed0 0x7fa918145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918146a70 con 0x7fa9180af4b0 2023-10-06T11:31:11.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181539c0 con 0x7fa9180b1ed0 2023-10-06T11:31:11.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181533b0 con 0x7fa918068290 2023-10-06T11:31:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34768/0 (socket says 172.21.15.46:34768) 2023-10-06T11:31:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 -- 172.21.15.46:0/1296699577 learned_addr learned my addr 172.21.15.46:0/1296699577 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91dce2700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9180b1ed0 0x7fa918145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 -- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 msgr2=0x7fa918145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 -- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9180b1ed0 msgr2=0x7fa918145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9180b1ed0 0x7fa918145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 -- 172.21.15.46:0/1296699577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009ae0 con 0x7fa9180af4b0 2023-10-06T11:31:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91cce0700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c5b0 tx=0x7fa908001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908005a70 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908005bd0 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080070d0 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa914009420 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.962+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa918153650 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.966+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa918153b30 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.966+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa908007480 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.970+0000 7fa90d7fa700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 0x7fa8fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.970+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa90807c240 con 0x7fa9180af4b0 2023-10-06T11:31:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.970+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 0x7fa8fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.970+0000 7fa91d4e1700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 0x7fa8fc062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9140093f0 tx=0x7fa914017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:10.974+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa90804c360 con 0x7fa9180af4b0 2023-10-06T11:31:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.210+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa9180afb40 con 0x7fa9180af4b0 2023-10-06T11:31:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.222+0000 7fa90d7fa700 1 -- 172.21.15.46:0/1296699577 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa90804f980 con 0x7fa9180af4b0 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 msgr2=0x7fa8fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 0x7fa8fc062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9140093f0 tx=0x7fa914017000 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 msgr2=0x7fa918146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c5b0 tx=0x7fa908001040 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 shutdown_connections 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8fc060670 0x7fa8fc062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9180b1ed0 0x7fa918145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9180af4b0 0x7fa918146be0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 --2- 172.21.15.46:0/1296699577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa918068290 0x7fa918145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 >> 172.21.15.46:0/1296699577 conn(0x7fa91806d490 msgr2=0x7fa91806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 shutdown_connections 2023-10-06T11:31:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.226+0000 7fa91f745700 1 -- 172.21.15.46:0/1296699577 wait complete. 2023-10-06T11:31:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:31:11.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:31:11.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.514+0000 7f30c1078700 1 Processor -- start 2023-10-06T11:31:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.514+0000 7f30c1078700 1 -- start start 2023-10-06T11:31:11.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30c1078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30c1078700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30bc15af30 0x7f30bc06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30c1078700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30bc06d460 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30c1078700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30bc0c7000 con 0x7f30bc15af30 2023-10-06T11:31:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30c1078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0c72c0 con 0x7f30bc15ab60 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30bc15af30 0x7f30bc06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30bad9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30bc15af30 0x7f30bc06cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34772/0 (socket says 172.21.15.46:34772) 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30bb59e700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f30bc0ad6c0 0x7f30bc0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:41384/0 (socket says 172.21.15.46:41384) 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 -- 172.21.15.46:0/2754743864 learned_addr learned my addr 172.21.15.46:0/2754743864 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 -- 172.21.15.46:0/2754743864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 msgr2=0x7f30bc06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 --2- 172.21.15.46:0/2754743864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 -- 172.21.15.46:0/2754743864 >> v1:172.21.15.132:6789/0 conn(0x7f30bc0ad6c0 legacy=0x7f30bc0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30ba59c700 1 -- 172.21.15.46:0/2754743864 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963235079 0 0) 0x7f30bc06d460 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.518+0000 7f30b259c700 1 -- 172.21.15.46:0/2754743864 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc0c75a0 con 0x7f30bc15af30 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30b259c700 1 --2- 172.21.15.46:0/2754743864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc15af30 0x7f30bc06cc30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30a40099b0 tx=0x7f30a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96607b14d70d92c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30ba59c700 1 -- 172.21.15.46:0/2754743864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4009ea0 con 0x7f30bc15af30 2023-10-06T11:31:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30ba59c700 1 -- 172.21.15.46:0/2754743864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a400b860 con 0x7f30bc15af30 2023-10-06T11:31:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30ba59c700 1 -- 172.21.15.46:0/2754743864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400bc00 con 0x7f30bc15af30 2023-10-06T11:31:11.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 -- 172.21.15.46:0/2754743864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc15af30 msgr2=0x7f30bc06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 --2- 172.21.15.46:0/2754743864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc15af30 0x7f30bc06cc30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30a40099b0 tx=0x7f30a4009250 comp rx=0 tx=0).stop 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 -- 172.21.15.46:0/2754743864 shutdown_connections 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 --2- 172.21.15.46:0/2754743864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc15af30 0x7f30bc06cc30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 --2- 172.21.15.46:0/2754743864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 -- 172.21.15.46:0/2754743864 >> 172.21.15.46:0/2754743864 conn(0x7f30bc0cbe30 msgr2=0x7f30bc0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 -- 172.21.15.46:0/2754743864 shutdown_connections 2023-10-06T11:31:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.522+0000 7f30c1078700 1 -- 172.21.15.46:0/2754743864 wait complete. 2023-10-06T11:31:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 Processor -- start 2023-10-06T11:31:11.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 -- start start 2023-10-06T11:31:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc0c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30bc15af30 0x7f30bc0c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0af740 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0af130 con 0x7f30bc15af30 2023-10-06T11:31:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30c1078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc0c5de0 con 0x7f30bc15ab60 2023-10-06T11:31:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34786/0 (socket says 172.21.15.46:34786) 2023-10-06T11:31:11.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 -- 172.21.15.46:0/1311795219 learned_addr learned my addr 172.21.15.46:0/1311795219 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:11.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30b259c700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc0c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bb59e700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30bc15af30 0x7f30bc0c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 -- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 msgr2=0x7f30bc0c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc0c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 -- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30bc15af30 msgr2=0x7f30bc0c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.526+0000 7f30bad9d700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30bc15af30 0x7f30bc0c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30bad9d700 1 -- 172.21.15.46:0/1311795219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40094f0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30b259c700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc0c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30bad9d700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00ec80 tx=0x7f30ac009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00c8e0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00ca40 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc0c60c0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00cdc0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30bc143990 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.530+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30bc063f90 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.534+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f30ac013680 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.534+0000 7f30b37fe700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 0x7f309c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:11.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.534+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f30ac07d390 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.534+0000 7f30b259c700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 0x7f309c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:11.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.534+0000 7f30b259c700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 0x7f309c062b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30a40099b0 tx=0x7f30a4017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:11.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.538+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f30ac04d4b0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.750+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30bc04b5c0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.750+0000 7f30b37fe700 1 -- 172.21.15.46:0/1311795219 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f30ac050ad0 con 0x7f30bc0ad6c0 2023-10-06T11:31:11.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:11.761 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:31:11.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 msgr2=0x7f309c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 0x7f309c062b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30a40099b0 tx=0x7f30a4017000 comp rx=0 tx=0).stop 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 msgr2=0x7f30bc143450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00ec80 tx=0x7f30ac009120 comp rx=0 tx=0).stop 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 shutdown_connections 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f309c0606a0 0x7f309c062b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30bc15af30 0x7f30bc0c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30bc15ab60 0x7f30bc0c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 --2- 172.21.15.46:0/1311795219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30bc0ad6c0 0x7f30bc143450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 >> 172.21.15.46:0/1311795219 conn(0x7f30bc0cbe30 msgr2=0x7f30bc0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 shutdown_connections 2023-10-06T11:31:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:11.754+0000 7f30c1078700 1 -- 172.21.15.46:0/1311795219 wait complete. 2023-10-06T11:31:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:31:12.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:31:05.390358+0000', 'modified': '2023-10-06T11:31:08.971806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5844}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5844': {'gid': 5844, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/3641487371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3641487371}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 3641487371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:31:12.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:31:12.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T11:31:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 Processor -- start 2023-10-06T11:31:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 -- start start 2023-10-06T11:31:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f604815ab60 0x7f604806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f604815af30 0x7f604806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604806d460 con 0x7f60480ad6c0 2023-10-06T11:31:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60480c7000 con 0x7f604815af30 2023-10-06T11:31:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f604db85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60480c72c0 con 0x7f604815ab60 2023-10-06T11:31:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.342+0000 7f603ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f604815af30 0x7f604806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f604815af30 0x7f604806cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34802/0 (socket says 172.21.15.46:34802) 2023-10-06T11:31:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 -- 172.21.15.46:0/1979671630 learned_addr learned my addr 172.21.15.46:0/1979671630 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f60477fe700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f604815ab60 0x7f604806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 -- 172.21.15.46:0/1979671630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f604815ab60 msgr2=0x7f604806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f604815ab60 0x7f604806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 -- 172.21.15.46:0/1979671630 >> v1:172.21.15.46:6789/0 conn(0x7f60480ad6c0 legacy=0x7f60480adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 -- 172.21.15.46:0/1979671630 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480c75a0 con 0x7f604815af30 2023-10-06T11:31:12.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f6046ffd700 1 -- 172.21.15.46:0/1979671630 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044339393 0 0) 0x7f604806d460 con 0x7f60480ad6c0 2023-10-06T11:31:12.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f603ffff700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815af30 0x7f604806cc30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f60300099b0 tx=0x7f6030009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be7db2e762fee48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f6046ffd700 1 -- 172.21.15.46:0/1979671630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009ea0 con 0x7f604815af30 2023-10-06T11:31:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f6046ffd700 1 -- 172.21.15.46:0/1979671630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000b860 con 0x7f604815af30 2023-10-06T11:31:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f6046ffd700 1 -- 172.21.15.46:0/1979671630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000bc00 con 0x7f604815af30 2023-10-06T11:31:12.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 -- 172.21.15.46:0/1979671630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815af30 msgr2=0x7f604806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815af30 0x7f604806cc30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f60300099b0 tx=0x7f6030009250 comp rx=0 tx=0).stop 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 -- 172.21.15.46:0/1979671630 shutdown_connections 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815af30 0x7f604806cc30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 --2- 172.21.15.46:0/1979671630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f604815ab60 0x7f604806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 -- 172.21.15.46:0/1979671630 >> 172.21.15.46:0/1979671630 conn(0x7f60480cbe30 msgr2=0x7f60480cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 -- 172.21.15.46:0/1979671630 shutdown_connections 2023-10-06T11:31:12.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.346+0000 7f604db85700 1 -- 172.21.15.46:0/1979671630 wait complete. 2023-10-06T11:31:12.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 Processor -- start 2023-10-06T11:31:12.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 -- start start 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60480ad6c0 0x7f60481697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f604815af30 0x7f6048069df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048169d10 con 0x7f604815ab60 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60480afa40 con 0x7f604815af30 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.350+0000 7f604db85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60480c5e50 con 0x7f60480ad6c0 2023-10-06T11:31:12.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34806/0 (socket says 172.21.15.46:34806) 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 -- 172.21.15.46:0/628965062 learned_addr learned my addr 172.21.15.46:0/628965062 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f60477fe700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60480ad6c0 0x7f60481697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f6047fff700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f604815af30 0x7f6048069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 -- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60480ad6c0 msgr2=0x7f60481697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60480ad6c0 0x7f60481697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 -- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f604815af30 msgr2=0x7f6048069df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f604815af30 0x7f6048069df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 -- 172.21.15.46:0/628965062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300094f0 con 0x7f604815ab60 2023-10-06T11:31:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603ffff700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f604806bf10 tx=0x7f6030017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603f7fe700 1 -- 172.21.15.46:0/628965062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000bbe0 con 0x7f604815ab60 2023-10-06T11:31:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603f7fe700 1 -- 172.21.15.46:0/628965062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603001a450 con 0x7f604815ab60 2023-10-06T11:31:12.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60480af3f0 con 0x7f604815ab60 2023-10-06T11:31:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f603f7fe700 1 -- 172.21.15.46:0/628965062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603001a810 con 0x7f604815ab60 2023-10-06T11:31:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.354+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6048157b70 con 0x7f604815ab60 2023-10-06T11:31:12.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.358+0000 7f603f7fe700 1 -- 172.21.15.46:0/628965062 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f603001ac00 con 0x7f604815ab60 2023-10-06T11:31:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.358+0000 7f603f7fe700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 0x7f6028062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.358+0000 7f603f7fe700 1 -- 172.21.15.46:0/628965062 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f603007e1a0 con 0x7f604815ab60 2023-10-06T11:31:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.358+0000 7f60477fe700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 0x7f6028062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.358+0000 7f60477fe700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 0x7f6028062b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6038009d30 tx=0x7f6038008040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 Processor -- start 2023-10-06T11:31:12.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 -- start start 2023-10-06T11:31:12.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2727630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x2655f30 con 0x24aa710 2023-10-06T11:31:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2651ed0 con 0x24f5790 2023-10-06T11:31:12.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x26180b0 con 0x24aaae0 2023-10-06T11:31:12.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34808/0 (socket says 172.21.15.46:34808) 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 -- 172.21.15.46:0/2245663098 learned_addr learned my addr 172.21.15.46:0/2245663098 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f60477fe700 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2727630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f6047fff700 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 -- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 msgr2=0x2727630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2727630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 -- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 msgr2=0x27280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 -- 172.21.15.46:0/2245663098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24f69e0 con 0x24aa710 2023-10-06T11:31:12.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.362+0000 7f603ffff700 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f603000b3c0 tx=0x7f6030009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2289de9008beac5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f603cff9700 1 -- 172.21.15.46:0/2245663098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300526c0 con 0x24aa710 2023-10-06T11:31:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f603cff9700 1 -- 172.21.15.46:0/2245663098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030055210 con 0x24aa710 2023-10-06T11:31:12.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f603cff9700 1 -- 172.21.15.46:0/2245663098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030058f60 con 0x24aa710 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- 172.21.15.46:0/2245663098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 msgr2=0x2727b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f603000b3c0 tx=0x7f6030009400 comp rx=0 tx=0).stop 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- 172.21.15.46:0/2245663098 shutdown_connections 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2727b70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 --2- 172.21.15.46:0/2245663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2727630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- 172.21.15.46:0/2245663098 >> 172.21.15.46:0/2245663098 conn(0x24aaeb0 msgr2=0x26e4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- 172.21.15.46:0/2245663098 shutdown_connections 2023-10-06T11:31:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- 172.21.15.46:0/2245663098 wait complete. 2023-10-06T11:31:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 -- start start 2023-10-06T11:31:12.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.366+0000 7f604fe66740 1 Processor -- start 2023-10-06T11:31:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x272c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x26e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x2617d50 con 0x24aa710 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x25f0970 con 0x24f5790 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x260b430 con 0x24aaae0 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 10 client.0 _ll_register_callbacks cb 0x25f4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34810/0 (socket says 172.21.15.46:34810) 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 -- 172.21.15.46:0/894532653 learned_addr learned my addr 172.21.15.46:0/894532653 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6047fff700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x26e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f603ffff700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x272c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 -- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 msgr2=0x272c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x272c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 -- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 msgr2=0x26e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x26e1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 -- 172.21.15.46:0/894532653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6048157b70 con 0x24aa710 2023-10-06T11:31:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f60477fe700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60380158c0 tx=0x7f6038018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee8fa25e75d9898 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038014870 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25f0cc0 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f60380149d0 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6038014da0 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 10 client.5994 fetch_fsmap learned FSMap version 26 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x2620370 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25f0cc0 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6038022420 con 0x24aa710 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 10 client.5994 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 10 client.5994 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:31:12.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 20 client.5994 populate_metadata read hostname 'smithi046' 2023-10-06T11:31:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x26e33b0 0x2736de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 4 client.5994 mds_command: new command op to 5844 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:31:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.370+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 --> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25f43e0 con 0x26e33b0 2023-10-06T11:31:12.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f603ffff700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x26e33b0 0x2736de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f603ffff700 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x26e33b0 0x2736de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc270b905e371775 server_cookie=71838c6da0e45cd5 in_seq=0 out_seq=0 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f6025ffb700 10 client.5994 ms_handle_connect on v2:172.21.15.46:6834/3641487371 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f6025ffb700 1 -- 172.21.15.46:0/894532653 <== mds.0 v2:172.21.15.46:6834/3641487371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x25f43e0 con 0x26e33b0 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f6025ffb700 10 client.5994 handle_command_reply: tid=0 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 client.5994 shutdown 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 msgr2=0x272bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60380158c0 tx=0x7f6038018000 comp rx=0 tx=0).stop 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 shutdown_connections 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x26e33b0 0x2736de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x26e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f6025ffb700 0 client.5994 ms_handle_reset on v2:172.21.15.46:6834/3641487371 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x272c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 --2- 172.21.15.46:0/894532653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x272bc40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 >> 172.21.15.46:0/894532653 conn(0x24aaeb0 msgr2=0x270b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 shutdown_connections 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 1 -- 172.21.15.46:0/894532653 wait complete. 2023-10-06T11:31:12.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.374+0000 7f604fe66740 20 client.5994 trim_cache size 0 max 16384 2023-10-06T11:31:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.394+0000 7f604fe66740 1 Processor -- start 2023-10-06T11:31:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.394+0000 7f604fe66740 1 -- start start 2023-10-06T11:31:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.394+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 0x26e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x2734dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x26555d0 con 0x24aaae0 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2654590 con 0x24f5790 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x2656ef0 con 0x24aa710 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34816/0 (socket says 172.21.15.46:34816) 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 -- 172.21.15.46:0/2517676148 learned_addr learned my addr 172.21.15.46:0/2517676148 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f60477fe700 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 0x26e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 -- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 msgr2=0x26e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 0x26e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 -- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 msgr2=0x2734dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x2734dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 -- 172.21.15.46:0/2517676148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2606d80 con 0x24aaae0 2023-10-06T11:31:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f60477fe700 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 0x26e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:31:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603ffff700 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f603000b3c0 tx=0x7f6030026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0b1ef8a210ccc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603d7fa700 1 -- 172.21.15.46:0/2517676148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603006b070 con 0x24aaae0 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603d7fa700 1 -- 172.21.15.46:0/2517676148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030058f60 con 0x24aaae0 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f603d7fa700 1 -- 172.21.15.46:0/2517676148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300500c0 con 0x24aaae0 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- 172.21.15.46:0/2517676148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 msgr2=0x26e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f603000b3c0 tx=0x7f6030026040 comp rx=0 tx=0).stop 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- 172.21.15.46:0/2517676148 shutdown_connections 2023-10-06T11:31:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x2734dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aaae0 0x26e3cc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 --2- 172.21.15.46:0/2517676148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aa710 0x26e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- 172.21.15.46:0/2517676148 >> 172.21.15.46:0/2517676148 conn(0x24aaeb0 msgr2=0x272a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- 172.21.15.46:0/2517676148 shutdown_connections 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- 172.21.15.46:0/2517676148 wait complete. 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 -- start start 2023-10-06T11:31:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.398+0000 7f604fe66740 1 Processor -- start 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2781b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x2650d60 con 0x24aa710 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x2627010 con 0x24f5790 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x2652770 con 0x24aaae0 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 10 client.0 _ll_register_callbacks cb 0x261d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:31:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34820/0 (socket says 172.21.15.46:34820) 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 -- 172.21.15.46:0/3738338071 learned_addr learned my addr 172.21.15.46:0/3738338071 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f6047fff700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f603ffff700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2781b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 -- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 msgr2=0x2781b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2781b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 -- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 msgr2=0x27050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 -- 172.21.15.46:0/3738338071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x24f69e0 con 0x24aa710 2023-10-06T11:31:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f60477fe700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6038009510 tx=0x7f6038023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee9e774410df6f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f6025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:31:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038022980 con 0x24aa710 2023-10-06T11:31:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6038015440 con 0x24aa710 2023-10-06T11:31:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6030055210 con 0x24aa710 2023-10-06T11:31:12.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60380146c0 con 0x24aa710 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 10 client.6000 fetch_fsmap learned FSMap version 26 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.402+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x25fb130 con 0x24aa710 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030055210 con 0x24aa710 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6038014820 con 0x24aa710 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 10 client.6000 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 10 client.6000 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:31:12.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 20 client.6000 populate_metadata read hostname 'smithi046' 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x27884e0 0x278a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 4 client.6000 mds_command: new command op to 5844 tid=0[{"prefix": "status"}] 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 --> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2626410 con 0x27884e0 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f603ffff700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x27884e0 0x278a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f603ffff700 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x27884e0 0x278a8c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665666c74b933bbd server_cookie=94b2d79c00098b2a in_seq=0 out_seq=0 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 10 client.6000 ms_handle_connect on v2:172.21.15.46:6834/3641487371 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 1 -- 172.21.15.46:0/3738338071 <== mds.0 v2:172.21.15.46:6834/3641487371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x2626410 con 0x27884e0 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 10 client.6000 handle_command_reply: tid=0 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 client.6000 shutdown 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 msgr2=0x2781650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6038009510 tx=0x7f6038023000 comp rx=0 tx=0).stop 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 shutdown_connections 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:6834/3641487371,v1:172.21.15.46:6836/3641487371] conn(0x27884e0 0x278a8c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x24f5790 0x27050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x24aaae0 0x2781b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 --2- 172.21.15.46:0/3738338071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x24aa710 0x2781650 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 >> 172.21.15.46:0/3738338071 conn(0x24aaeb0 msgr2=0x272a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f6025ffb700 0 client.6000 ms_handle_reset on v2:172.21.15.46:6834/3641487371 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 shutdown_connections 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 1 -- 172.21.15.46:0/3738338071 wait complete. 2023-10-06T11:31:12.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.406+0000 7f604fe66740 20 client.6000 trim_cache size 0 max 16384 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5844, 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:31:12.421 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.4498595529999996, 2023-10-06T11:31:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2023-10-06T11:31:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 62, 2023-10-06T11:31:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 62, 2023-10-06T11:31:12.422 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.398776209999999 2023-10-06T11:31:12.422 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 msgr2=0x7f6028062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 0x7f6028062b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6038009d30 tx=0x7f6038008040 comp rx=0 tx=0).stop 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 msgr2=0x7f6048157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f604806bf10 tx=0x7f6030017000 comp rx=0 tx=0).stop 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 shutdown_connections 2023-10-06T11:31:12.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f60280606a0 0x7f6028062b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f604815af30 0x7f6048069df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604815ab60 0x7f6048157630 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 --2- 172.21.15.46:0/628965062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60480ad6c0 0x7f60481697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 >> 172.21.15.46:0/628965062 conn(0x7f60480cbe30 msgr2=0x7f60480aab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 shutdown_connections 2023-10-06T11:31:12.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.410+0000 7f604db85700 1 -- 172.21.15.46:0/628965062 wait complete. 2023-10-06T11:31:12.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:31:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 Processor -- start 2023-10-06T11:31:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 -- start start 2023-10-06T11:31:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa62c15c770 0x7fa62c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0cc150 con 0x7fa62c0ccb50 2023-10-06T11:31:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62c0cfbb0 con 0x7fa62c15c770 2023-10-06T11:31:12.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.594+0000 7fa6344f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c0cdbf0 con 0x7fa62c0cec00 2023-10-06T11:31:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa62c15c770 0x7fa62c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa62c15c770 0x7fa62c0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34832/0 (socket says 172.21.15.46:34832) 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 -- 172.21.15.46:0/3453555818 learned_addr learned my addr 172.21.15.46:0/3453555818 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa632a8d700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 -- 172.21.15.46:0/3453555818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 msgr2=0x7fa62c162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 -- 172.21.15.46:0/3453555818 >> v1:172.21.15.132:6789/0 conn(0x7fa62c0ccb50 legacy=0x7fa62c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63128a700 1 -- 172.21.15.46:0/3453555818 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904817018 0 0) 0x7fa62c0cc150 con 0x7fa62c0ccb50 2023-10-06T11:31:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 -- 172.21.15.46:0/3453555818 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0cb280 con 0x7fa62c15c770 2023-10-06T11:31:12.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63228c700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0cad40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0cfcf0 tx=0x7fa628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6ed3ed6e7e3728 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63128a700 1 -- 172.21.15.46:0/3453555818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628009e60 con 0x7fa62c15c770 2023-10-06T11:31:12.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63128a700 1 -- 172.21.15.46:0/3453555818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800b860 con 0x7fa62c15c770 2023-10-06T11:31:12.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa63128a700 1 -- 172.21.15.46:0/3453555818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800bc20 con 0x7fa62c15c770 2023-10-06T11:31:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa6344f0700 1 -- 172.21.15.46:0/3453555818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 msgr2=0x7fa62c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa6344f0700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0cad40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0cfcf0 tx=0x7fa628009250 comp rx=0 tx=0).stop 2023-10-06T11:31:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa6344f0700 1 -- 172.21.15.46:0/3453555818 shutdown_connections 2023-10-06T11:31:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.598+0000 7fa6344f0700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 --2- 172.21.15.46:0/3453555818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0cad40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- 172.21.15.46:0/3453555818 >> 172.21.15.46:0/3453555818 conn(0x7fa62c099480 msgr2=0x7fa62c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- 172.21.15.46:0/3453555818 shutdown_connections 2023-10-06T11:31:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- 172.21.15.46:0/3453555818 wait complete. 2023-10-06T11:31:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 Processor -- start 2023-10-06T11:31:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- start start 2023-10-06T11:31:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 0x7fa62c169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c09bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c16a120 con 0x7fa62c15c770 2023-10-06T11:31:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c09aa30 con 0x7fa62c0ccb50 2023-10-06T11:31:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa6344f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c09ad10 con 0x7fa62c0cec00 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa632a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa632a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34834/0 (socket says 172.21.15.46:34834) 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.602+0000 7fa632a8d700 1 -- 172.21.15.46:0/572869166 learned_addr learned my addr 172.21.15.46:0/572869166 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa63228c700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 0x7fa62c169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa631a8b700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c09bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 -- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 msgr2=0x7fa62c09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c09bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 -- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 msgr2=0x7fa62c169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 0x7fa62c169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 -- 172.21.15.46:0/572869166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009490 con 0x7fa62c15c770 2023-10-06T11:31:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa63228c700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 0x7fa62c169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa632a8d700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ef0 tx=0x7fa62400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62400cc50 con 0x7fa62c15c770 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62400f040 con 0x7fa62c15c770 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624014480 con 0x7fa62c15c770 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c09aff0 con 0x7fa62c15c770 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c09b5b0 con 0x7fa62c15c770 2023-10-06T11:31:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.606+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c0660f0 con 0x7fa62c15c770 2023-10-06T11:31:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.610+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa624014830 con 0x7fa62c15c770 2023-10-06T11:31:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.610+0000 7fa6227fc700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 0x7fa610062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.610+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa62407dbd0 con 0x7fa62c15c770 2023-10-06T11:31:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.610+0000 7fa63228c700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 0x7fa610062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.614+0000 7fa63228c700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 0x7fa610062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0cfcf0 tx=0x7fa628016000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.618+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa62404dbe0 con 0x7fa62c15c770 2023-10-06T11:31:12.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.826+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa62c0cb540 con 0x7fa62c15c770 2023-10-06T11:31:12.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.826+0000 7fa6227fc700 1 -- 172.21.15.46:0/572869166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa624018070 con 0x7fa62c15c770 2023-10-06T11:31:12.836 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:31:12.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 msgr2=0x7fa610062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 0x7fa610062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0cfcf0 tx=0x7fa628016000 comp rx=0 tx=0).stop 2023-10-06T11:31:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 msgr2=0x7fa62c0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:12.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ef0 tx=0x7fa62400b040 comp rx=0 tx=0).stop 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 shutdown_connections 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa610060670 0x7fa610062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c15c770 0x7fa62c0c9860 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa62c0cec00 0x7fa62c09bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 --2- 172.21.15.46:0/572869166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ccb50 0x7fa62c169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 >> 172.21.15.46:0/572869166 conn(0x7fa62c099480 msgr2=0x7fa62c068b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 shutdown_connections 2023-10-06T11:31:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:12.830+0000 7fa6344f0700 1 -- 172.21.15.46:0/572869166 wait complete. 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:31:12.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:31:12.862 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:31:12.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:12.862 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:31:12.880 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:31:12.881 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41657sec preferred_lft 41657sec 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:31:12.882 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:31:12.883 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:31:12.884 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.885 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:31:12.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:31:12.885 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:31:12.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:31:12.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:12.886 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:31:12.969 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:31:12.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:12.972 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:31:13.035 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:31:13.035 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:31:13.036 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:31:13.036 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:31:13.036 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:31:13.036 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:31:13.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:31:13.037 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:31:13.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:13.179 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:31:13.196 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:31:13.196 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:31:13.197 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:31:13.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:13.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:31:13.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:31:13.274 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:31:13.340 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:31:13.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:31:13.340 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:31:13.404 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:31:13.404 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:31:13.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:31:13.527 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:31:13.527 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:31:13.528 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:31:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:31:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:31:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:31:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:31:13.550 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:31:13.565 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:31:13.565 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:31:13.565 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:31:13.566 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:31:13.566 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:31:13.566 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:31:13.566 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:31:13.566 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:31:13.571 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:31:13.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:31:13.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:31:13.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:31:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 Processor -- start 2023-10-06T11:31:13.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 -- start start 2023-10-06T11:31:13.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1406e8a0 0x7f9f1406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f140ad6c0 0x7f9f140adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1406d460 con 0x7f9f1415ab60 2023-10-06T11:31:13.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1406ed10 con 0x7f9f140ad6c0 2023-10-06T11:31:13.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f195c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c7000 con 0x7f9f1406e8a0 2023-10-06T11:31:13.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f12ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f1415ab60 0x7f9f1415af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41936/0 (socket says 172.21.15.46:41936) 2023-10-06T11:31:13.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f12ffd700 1 -- 172.21.15.46:0/3776132540 learned_addr learned my addr 172.21.15.46:0/3776132540 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:13.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f137fe700 1 --2- 172.21.15.46:0/3776132540 >> v2:172.21.15.132:3300/0 conn(0x7f9f140ad6c0 0x7f9f140adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.802+0000 7f9f127fc700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1406e8a0 0x7f9f1406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f11ffb700 1 -- 172.21.15.46:0/3776132540 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953306188 0 0) 0x7f9f1406d460 con 0x7f9f1415ab60 2023-10-06T11:31:13.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f11ffb700 1 -- 172.21.15.46:0/3776132540 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9efc003570 con 0x7f9f1415ab60 2023-10-06T11:31:13.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f137fe700 1 -- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1406e8a0 msgr2=0x7f9f1406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f137fe700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1406e8a0 0x7f9f1406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f137fe700 1 -- 172.21.15.46:0/3776132540 >> v1:172.21.15.46:6789/0 conn(0x7f9f1415ab60 legacy=0x7f9f1415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f137fe700 1 -- 172.21.15.46:0/3776132540 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f140c72c0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f137fe700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f140ad6c0 0x7f9f140adb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009e60 tx=0x7f9f0c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8adbdd56a5d6ac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:13.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f11ffb700 1 -- 172.21.15.46:0/3776132540 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00e070 con 0x7f9f140ad6c0 2023-10-06T11:31:13.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f11ffb700 1 -- 172.21.15.46:0/3776132540 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c00bab0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f11ffb700 1 -- 172.21.15.46:0/3776132540 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00be70 con 0x7f9f140ad6c0 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 -- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f140ad6c0 msgr2=0x7f9f140adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f140ad6c0 0x7f9f140adb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009e60 tx=0x7f9f0c009780 comp rx=0 tx=0).stop 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 -- 172.21.15.46:0/3776132540 shutdown_connections 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f140ad6c0 0x7f9f140adb20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3776132540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1406e8a0 0x7f9f1406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 -- 172.21.15.46:0/3776132540 >> 172.21.15.46:0/3776132540 conn(0x7f9f140cbe30 msgr2=0x7f9f140cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 -- 172.21.15.46:0/3776132540 shutdown_connections 2023-10-06T11:31:13.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.806+0000 7f9f195c5700 1 -- 172.21.15.46:0/3776132540 wait complete. 2023-10-06T11:31:13.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.810+0000 7f9f195c5700 1 Processor -- start 2023-10-06T11:31:13.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.810+0000 7f9f195c5700 1 -- start start 2023-10-06T11:31:13.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1406e8a0 0x7f9f140b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1415ab60 0x7f9f14150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c4d30 con 0x7f9f140ad6c0 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f14158320 con 0x7f9f1406e8a0 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f140c46e0 con 0x7f9f1415ab60 2023-10-06T11:31:13.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41180/0 (socket says 172.21.15.46:41180) 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 -- 172.21.15.46:0/3109908610 learned_addr learned my addr 172.21.15.46:0/3109908610 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f12ffd700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1406e8a0 0x7f9f140b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f137fe700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1415ab60 0x7f9f14150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 -- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1415ab60 msgr2=0x7f9f14150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1415ab60 0x7f9f14150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 -- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1406e8a0 msgr2=0x7f9f140b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1406e8a0 0x7f9f140b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:13.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 -- 172.21.15.46:0/3109908610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c009a20 con 0x7f9f140ad6c0 2023-10-06T11:31:13.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f127fc700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e930 tx=0x7f9f080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08013680 con 0x7f9f140ad6c0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f080137e0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f140c49c0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08013b80 con 0x7f9f140ad6c0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.814+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f140c4300 con 0x7f9f140ad6c0 2023-10-06T11:31:13.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.818+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1406e1a0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.818+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9f0800c3e0 con 0x7f9f140ad6c0 2023-10-06T11:31:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.818+0000 7f9efaffd700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 0x7f9f00062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.818+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f9f0807cd20 con 0x7f9f140ad6c0 2023-10-06T11:31:13.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.822+0000 7f9f12ffd700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 0x7f9f00062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:13.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.822+0000 7f9f12ffd700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 0x7f9f00062af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f04007fd0 tx=0x7f9f04009040 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:13.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:13.826+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f1406e1a0 con 0x7f9f140ad6c0 2023-10-06T11:31:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.022+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f1406e1a0 con 0x7f9f00060640 2023-10-06T11:31:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.026+0000 7f9efaffd700 1 -- 172.21.15.46:0/3109908610 <== mgr.5637 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9f1406e1a0 con 0x7f9f00060640 2023-10-06T11:31:14.036 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:31:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:31:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:31:14.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:31:14.037 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:31:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 msgr2=0x7f9f00062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 0x7f9f00062af0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f04007fd0 tx=0x7f9f04009040 comp rx=0 tx=0).stop 2023-10-06T11:31:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 msgr2=0x7f9f140aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e930 tx=0x7f9f080091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 shutdown_connections 2023-10-06T11:31:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f00060640 0x7f9f00062af0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f1415ab60 0x7f9f14150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f140ad6c0 0x7f9f140aebd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 --2- 172.21.15.46:0/3109908610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f1406e8a0 0x7f9f140b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 >> 172.21.15.46:0/3109908610 conn(0x7f9f140cbe30 msgr2=0x7f9f140c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 shutdown_connections 2023-10-06T11:31:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.030+0000 7f9f195c5700 1 -- 172.21.15.46:0/3109908610 wait complete. 2023-10-06T11:31:14.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:31:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 Processor -- start 2023-10-06T11:31:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 -- start start 2023-10-06T11:31:14.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff43c06e8a0 0x7ff43c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0c1bf0 con 0x7ff43c0c6470 2023-10-06T11:31:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c0c72c0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4428f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c06cf80 con 0x7ff43c06ed10 2023-10-06T11:31:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.222+0000 7ff4337fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff43c06e8a0 0x7ff43c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff43c06e8a0 0x7ff43c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41182/0 (socket says 172.21.15.46:41182) 2023-10-06T11:31:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43bfff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff43c0c6470 0x7ff43c0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40262/0 (socket says 172.21.15.46:40262) 2023-10-06T11:31:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 -- 172.21.15.46:0/2310582383 learned_addr learned my addr 172.21.15.46:0/2310582383 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:14.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff440e8d700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43b7fe700 1 -- 172.21.15.46:0/2310582383 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397836935 0 0) 0x7ff43c0c1bf0 con 0x7ff43c0c6470 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43b7fe700 1 -- 172.21.15.46:0/2310582383 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff428003570 con 0x7ff43c0c6470 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 -- 172.21.15.46:0/2310582383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 msgr2=0x7ff43c0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 -- 172.21.15.46:0/2310582383 >> v1:172.21.15.132:6789/0 conn(0x7ff43c0c6470 legacy=0x7ff43c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 -- 172.21.15.46:0/2310582383 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c06d260 con 0x7ff43c06e8a0 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff440e8d700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:31:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4337fe700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c0c6840 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff4240099b0 tx=0x7ff424009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f75b889b58faaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43b7fe700 1 -- 172.21.15.46:0/2310582383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400b680 con 0x7ff43c06e8a0 2023-10-06T11:31:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43b7fe700 1 -- 172.21.15.46:0/2310582383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400b7e0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff43b7fe700 1 -- 172.21.15.46:0/2310582383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400bb80 con 0x7ff43c06e8a0 2023-10-06T11:31:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 -- 172.21.15.46:0/2310582383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 msgr2=0x7ff43c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c0c6840 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff4240099b0 tx=0x7ff424009250 comp rx=0 tx=0).stop 2023-10-06T11:31:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 -- 172.21.15.46:0/2310582383 shutdown_connections 2023-10-06T11:31:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 --2- 172.21.15.46:0/2310582383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c0c6840 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 -- 172.21.15.46:0/2310582383 >> 172.21.15.46:0/2310582383 conn(0x7ff43c0cbe30 msgr2=0x7ff43c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.226+0000 7ff4428f0700 1 -- 172.21.15.46:0/2310582383 shutdown_connections 2023-10-06T11:31:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.230+0000 7ff4428f0700 1 -- 172.21.15.46:0/2310582383 wait complete. 2023-10-06T11:31:14.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.230+0000 7ff4428f0700 1 Processor -- start 2023-10-06T11:31:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.230+0000 7ff4428f0700 1 -- start start 2023-10-06T11:31:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 0x7ff43c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c13de00 con 0x7ff43c06e8a0 2023-10-06T11:31:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c16a360 con 0x7ff43c0c6470 2023-10-06T11:31:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0ad570 con 0x7ff43c06ed10 2023-10-06T11:31:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41184/0 (socket says 172.21.15.46:41184) 2023-10-06T11:31:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 -- 172.21.15.46:0/1499086829 learned_addr learned my addr 172.21.15.46:0/1499086829 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff440e8d700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 0x7ff43c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4337fe700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 -- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 msgr2=0x7ff43c0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 -- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 msgr2=0x7ff43c13f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 0x7ff43c13f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 -- 172.21.15.46:0/1499086829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434009a20 con 0x7ff43c06e8a0 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff440e8d700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 0x7ff43c13f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff43bfff700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c7c0 tx=0x7ff42c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005a70 con 0x7ff43c06e8a0 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c005bf0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0070d0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.234+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4240094f0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.238+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c169d10 con 0x7ff43c06e8a0 2023-10-06T11:31:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.238+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c0660f0 con 0x7ff43c06e8a0 2023-10-06T11:31:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.238+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff42c007480 con 0x7ff43c06e8a0 2023-10-06T11:31:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.242+0000 7ff438ff9700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 0x7ff41c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.242+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ff42c07c130 con 0x7ff43c06e8a0 2023-10-06T11:31:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.242+0000 7ff4337fe700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 0x7ff41c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:14.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.242+0000 7ff4337fe700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 0x7ff41c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4240099b0 tx=0x7ff424017000 comp rx=0 tx=0).ready entity=mgr.5637 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.246+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff42c04c310 con 0x7ff43c06e8a0 2023-10-06T11:31:14.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:14.454+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff43c04b5c0 con 0x7ff43c06e8a0 2023-10-06T11:31:15.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.422+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 17) v1 ==== 80078+0+0 (secure 0 0 0) 0x7ff42c04c6e0 con 0x7ff43c06e8a0 2023-10-06T11:31:15.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.422+0000 7ff438ff9700 1 -- 172.21.15.46:0/1499086829 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v17) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff42c04ffd0 con 0x7ff43c06e8a0 2023-10-06T11:31:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 msgr2=0x7ff41c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 0x7ff41c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4240099b0 tx=0x7ff424017000 comp rx=0 tx=0).stop 2023-10-06T11:31:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 msgr2=0x7ff43c13d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c7c0 tx=0x7ff42c007e00 comp rx=0 tx=0).stop 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 shutdown_connections 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff41c060670 0x7ff41c062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff43c0c6470 0x7ff43c13f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff43c06ed10 0x7ff43c0ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 --2- 172.21.15.46:0/1499086829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff43c06e8a0 0x7ff43c13d8c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 >> 172.21.15.46:0/1499086829 conn(0x7ff43c0cbe30 msgr2=0x7ff43c0c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 shutdown_connections 2023-10-06T11:31:15.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.426+0000 7ff4428f0700 1 -- 172.21.15.46:0/1499086829 wait complete. 2023-10-06T11:31:15.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:31:15.607 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:15.606+0000 7f67186eb000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:31:15.630 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:15.618+0000 7fae32dc3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:31:15.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.626+0000 7f32230e3700 1 Processor -- start 2023-10-06T11:31:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.626+0000 7f32230e3700 1 -- start start 2023-10-06T11:31:15.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32230e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 0x7f321c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:15.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32230e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c15b6d0 con 0x7f321c0abc40 2023-10-06T11:31:15.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32230e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0b04e0 con 0x7f321c0af030 2023-10-06T11:31:15.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32230e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0afbb0 con 0x7f321c15b2b0 2023-10-06T11:31:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32220e1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f321c0abc40 0x7f321c0ae920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41942/0 (socket says 172.21.15.46:41942) 2023-10-06T11:31:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32220e1700 1 -- 172.21.15.46:0/3367995172 learned_addr learned my addr 172.21.15.46:0/3367995172 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32228e2700 1 --2- 172.21.15.46:0/3367995172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 0x7f321c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:15.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499175583 0 0) 0x7f3218001110 con 0x7f321c0abc40 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3208003570 con 0x7f321c0abc40 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334663291 0 0) 0x7f321c0b04e0 con 0x7f321c0af030 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3218001110 con 0x7f321c0af030 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32228e2700 1 -- 172.21.15.46:0/3367995172 >> v1:172.21.15.132:6789/0 conn(0x7f321c0af030 legacy=0x7f321c15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32228e2700 1 -- 172.21.15.46:0/3367995172 >> v1:172.21.15.46:6789/0 conn(0x7f321c0abc40 legacy=0x7f321c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32228e2700 1 -- 172.21.15.46:0/3367995172 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0afe90 con 0x7f321c15b2b0 2023-10-06T11:31:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32228e2700 1 --2- 172.21.15.46:0/3367995172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 0x7f321c0b0970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e90 tx=0x7f3214009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8804fe13dfaea40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400e070 con 0x7f321c15b2b0 2023-10-06T11:31:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321400bb80 con 0x7f321c15b2b0 2023-10-06T11:31:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.630+0000 7f32210df700 1 -- 172.21.15.46:0/3367995172 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214012430 con 0x7f321c15b2b0 2023-10-06T11:31:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 -- 172.21.15.46:0/3367995172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 msgr2=0x7f321c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 --2- 172.21.15.46:0/3367995172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 0x7f321c0b0970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e90 tx=0x7f3214009780 comp rx=0 tx=0).stop 2023-10-06T11:31:15.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 -- 172.21.15.46:0/3367995172 shutdown_connections 2023-10-06T11:31:15.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 --2- 172.21.15.46:0/3367995172 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c15b2b0 0x7f321c0b0970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 -- 172.21.15.46:0/3367995172 >> 172.21.15.46:0/3367995172 conn(0x7f321c0ccd50 msgr2=0x7f321c0cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:15.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 -- 172.21.15.46:0/3367995172 shutdown_connections 2023-10-06T11:31:15.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.634+0000 7f32230e3700 1 -- 172.21.15.46:0/3367995172 wait complete. 2023-10-06T11:31:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 Processor -- start 2023-10-06T11:31:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 -- start start 2023-10-06T11:31:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c0af030 0x7f321c0a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321c15b2b0 0x7f321c0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0a5b20 con 0x7f321c0abc40 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0a9220 con 0x7f321c15b2b0 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32230e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0a9500 con 0x7f321c0af030 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41210/0 (socket says 172.21.15.46:41210) 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 -- 172.21.15.46:0/2197117334 learned_addr learned my addr 172.21.15.46:0/2197117334 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32218e0700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c0af030 0x7f321c0a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32228e2700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321c15b2b0 0x7f321c0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 -- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c0af030 msgr2=0x7f321c0a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c0af030 0x7f321c0a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 -- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321c15b2b0 msgr2=0x7f321c0b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321c15b2b0 0x7f321c0b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.638+0000 7f32220e1700 1 -- 172.21.15.46:0/2197117334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009a20 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32220e1700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f321800c710 tx=0x7f3218007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218005c00 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218005d60 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0a54d0 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218007320 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c0c8990 con 0x7f321c0abc40 2023-10-06T11:31:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.642+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c064590 con 0x7f321c0abc40 2023-10-06T11:31:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.646+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f321801b3b0 con 0x7f321c0abc40 2023-10-06T11:31:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.646+0000 7f32127fc700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 0x7f3200062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.646+0000 7f32218e0700 1 -- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 msgr2=0x7f3200062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.646+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f321807c0a0 con 0x7f321c0abc40 2023-10-06T11:31:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.646+0000 7f32218e0700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 0x7f3200062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:31:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.654+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f321804c1c0 con 0x7f321c0abc40 2023-10-06T11:31:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.826+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f321c15b6d0 con 0x7f321c0abc40 2023-10-06T11:31:15.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.838+0000 7f32127fc700 1 -- 172.21.15.46:0/2197117334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f321c15b6d0 con 0x7f321c0abc40 2023-10-06T11:31:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.838+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 msgr2=0x7f3200062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:31:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.838+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 0x7f3200062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.838+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 msgr2=0x7f321c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.838+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f321800c710 tx=0x7f3218007e00 comp rx=0 tx=0).stop 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 shutdown_connections 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32000606c0 0x7f3200062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321c15b2b0 0x7f321c0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321c0af030 0x7f321c0a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 --2- 172.21.15.46:0/2197117334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0abc40 0x7f321c155dc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 >> 172.21.15.46:0/2197117334 conn(0x7f321c0ccd50 msgr2=0x7f321c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 shutdown_connections 2023-10-06T11:31:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:15.842+0000 7f32230e3700 1 -- 172.21.15.46:0/2197117334 wait complete. 2023-10-06T11:31:15.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:31:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.030+0000 7f6b41185700 1 Processor -- start 2023-10-06T11:31:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.030+0000 7f6b41185700 1 -- start start 2023-10-06T11:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b41185700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b3c0b23c0 0x7f6b3c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b41185700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0cae40 0x7f6b3c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b41185700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3c0cc510 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b41185700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0cb900 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b41185700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0cbbe0 con 0x7f6b3c0cae40 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b3c0b23c0 0x7f6b3c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3b59e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6b3c0ccb40 0x7f6b3c15cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41958/0 (socket says 172.21.15.46:41958) 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3b59e700 1 -- 172.21.15.46:0/2870986189 learned_addr learned my addr 172.21.15.46:0/2870986189 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3a59c700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0cae40 0x7f6b3c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 -- 172.21.15.46:0/2870986189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0cae40 msgr2=0x7f6b3c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0cae40 0x7f6b3c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 -- 172.21.15.46:0/2870986189 >> v1:172.21.15.46:6789/0 conn(0x7f6b3c0ccb40 legacy=0x7f6b3c15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 -- 172.21.15.46:0/2870986189 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0cbec0 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.034+0000 7f6b3ad9d700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0b23c0 0x7f6b3c0b2790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0098d0 tx=0x7f6b2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b60116972e8ca3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b39d9b700 1 -- 172.21.15.46:0/2870986189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00b640 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b39d9b700 1 -- 172.21.15.46:0/2870986189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c00b7a0 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b39d9b700 1 -- 172.21.15.46:0/2870986189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00bb60 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 -- 172.21.15.46:0/2870986189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0b23c0 msgr2=0x7f6b3c0b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0b23c0 0x7f6b3c0b2790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0098d0 tx=0x7f6b2c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 -- 172.21.15.46:0/2870986189 shutdown_connections 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0cae40 0x7f6b3c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 --2- 172.21.15.46:0/2870986189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0b23c0 0x7f6b3c0b2790 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 -- 172.21.15.46:0/2870986189 >> 172.21.15.46:0/2870986189 conn(0x7f6b3c099480 msgr2=0x7f6b3c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 -- 172.21.15.46:0/2870986189 shutdown_connections 2023-10-06T11:31:16.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.038+0000 7f6b41185700 1 -- 172.21.15.46:0/2870986189 wait complete. 2023-10-06T11:31:16.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 Processor -- start 2023-10-06T11:31:16.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 -- start start 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0b23c0 0x7f6b3c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 0x7f6b3c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c156810 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b3880 con 0x7f6b3c0cae40 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b41185700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c15ac00 con 0x7f6b3c0b23c0 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41222/0 (socket says 172.21.15.46:41222) 2023-10-06T11:31:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 -- 172.21.15.46:0/2340799265 learned_addr learned my addr 172.21.15.46:0/2340799265 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3a59c700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 0x7f6b3c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3ad9d700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0b23c0 0x7f6b3c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 -- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0b23c0 msgr2=0x7f6b3c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0b23c0 0x7f6b3c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 -- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 msgr2=0x7f6b3c155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 0x7f6b3c155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3b59e700 1 -- 172.21.15.46:0/2340799265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0093c0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.042+0000 7f6b3a59c700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 0x7f6b3c155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b3b59e700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400cc40 tx=0x7f6b34001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400a9c0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3400ab40 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c15a640 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340070f0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c0b3b20 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c0660f0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.046+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f6b340074e0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.050+0000 7f6b22ffd700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 0x7f6b28062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.050+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6b3407d140 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.050+0000 7f6b3ad9d700 1 -- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 msgr2=0x7f6b28062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:16.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.050+0000 7f6b3ad9d700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 0x7f6b28062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:31:16.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.058+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b3404d2e0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.134 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.130+0000 7f67186eb000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:31:16.199 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.190+0000 7fae32dc3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:31:16.205 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.202+0000 7f67186eb000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:31:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.230+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6b3c0cc1a0 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.238+0000 7f6b22ffd700 1 -- 172.21.15.46:0/2340799265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6b34050900 con 0x7f6b3c0ccb40 2023-10-06T11:31:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 msgr2=0x7f6b28062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:31:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 0x7f6b28062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 msgr2=0x7f6b3c0c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400cc40 tx=0x7f6b34001a50 comp rx=0 tx=0).stop 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 shutdown_connections 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b280606c0 0x7f6b28062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b3c0ccb40 0x7f6b3c0c5fb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0cae40 0x7f6b3c155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 --2- 172.21.15.46:0/2340799265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0b23c0 0x7f6b3c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 >> 172.21.15.46:0/2340799265 conn(0x7f6b3c099480 msgr2=0x7f6b3c0cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:16.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 shutdown_connections 2023-10-06T11:31:16.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.242+0000 7f6b41185700 1 -- 172.21.15.46:0/2340799265 wait complete. 2023-10-06T11:31:16.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:16.272 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:31:16.273 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.262+0000 7fae32dc3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:31:16.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T11:31:16.338 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.338+0000 7f67186eb000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:31:16.421 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.410+0000 7fae32dc3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:31:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.498+0000 7f0320bcc700 1 Processor -- start 2023-10-06T11:31:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.498+0000 7f0320bcc700 1 -- start start 2023-10-06T11:31:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f0320bcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f031c1591f0 0x7f031c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f0320bcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 0x7f031c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f0320bcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c0cf360 con 0x7f031c1595c0 2023-10-06T11:31:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f0320bcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031c15dd60 con 0x7f031c1591f0 2023-10-06T11:31:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f0320bcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0cea30 con 0x7f031c0cc500 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f031c1591f0 0x7f031c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f031c1591f0 0x7f031c0cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41228/0 (socket says 172.21.15.46:41228) 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 -- 172.21.15.46:0/571915629 learned_addr learned my addr 172.21.15.46:0/571915629 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031bfff700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 0x7f031c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 -- 172.21.15.46:0/571915629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 msgr2=0x7f031c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 0x7f031c15ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 -- 172.21.15.46:0/571915629 >> v1:172.21.15.132:6789/0 conn(0x7f031c1595c0 legacy=0x7f031c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 -- 172.21.15.46:0/571915629 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0ced10 con 0x7f031c1591f0 2023-10-06T11:31:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031bfff700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 0x7f031c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031b7fe700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c1591f0 0x7f031c0cf7f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f030c0099b0 tx=0x7f030c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41006e1582761bfd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031a7fc700 1 -- 172.21.15.46:0/571915629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00b8f0 con 0x7f031c1591f0 2023-10-06T11:31:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031a7fc700 1 -- 172.21.15.46:0/571915629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00ba50 con 0x7f031c1591f0 2023-10-06T11:31:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.502+0000 7f031a7fc700 1 -- 172.21.15.46:0/571915629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00bdf0 con 0x7f031c1591f0 2023-10-06T11:31:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- 172.21.15.46:0/571915629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c1591f0 msgr2=0x7f031c0cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c1591f0 0x7f031c0cf7f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f030c0099b0 tx=0x7f030c009250 comp rx=0 tx=0).stop 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- 172.21.15.46:0/571915629 shutdown_connections 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c0cc500 0x7f031c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 --2- 172.21.15.46:0/571915629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c1591f0 0x7f031c0cf7f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- 172.21.15.46:0/571915629 >> 172.21.15.46:0/571915629 conn(0x7f031c06ef60 msgr2=0x7f031c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- 172.21.15.46:0/571915629 shutdown_connections 2023-10-06T11:31:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- 172.21.15.46:0/571915629 wait complete. 2023-10-06T11:31:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 Processor -- start 2023-10-06T11:31:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.506+0000 7f0320bcc700 1 -- start start 2023-10-06T11:31:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f031c0cc500 0x7f031c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c1591f0 0x7f031c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c151f30 con 0x7f031c15d860 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c15ce90 con 0x7f031c0cc500 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c169ef0 con 0x7f031c1591f0 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031affd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c1591f0 0x7f031c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41244/0 (socket says 172.21.15.46:41244) 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 -- 172.21.15.46:0/3308081690 learned_addr learned my addr 172.21.15.46:0/3308081690 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f031c0cc500 0x7f031c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c1591f0 msgr2=0x7f031c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c1591f0 0x7f031c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f031c0cc500 msgr2=0x7f031c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f031c0cc500 0x7f031c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0094f0 con 0x7f031c15d860 2023-10-06T11:31:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f031bfff700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0314009fc0 tx=0x7f03140099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031400dbe0 con 0x7f031c15d860 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031400f040 con 0x7f031c15d860 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314015440 con 0x7f031c15d860 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c16a1d0 con 0x7f031c15d860 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.510+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c15cb20 con 0x7f031c15d860 2023-10-06T11:31:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c16a410 con 0x7f031c15d860 2023-10-06T11:31:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f031401e3b0 con 0x7f031c15d860 2023-10-06T11:31:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f03037fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f031407d2c0 con 0x7f031c15d860 2023-10-06T11:31:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f031b7fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.514+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:31:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.522+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f031c16a410 con 0x7f031c15d860 2023-10-06T11:31:16.592 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.590+0000 7f67186eb000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:31:16.661 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.658+0000 7f67186eb000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:31:16.696 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.686+0000 7fae32dc3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:31:16.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.718+0000 7f031b7fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:16.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.718+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:31:16.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:16.718+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f031c16a410 con 0x7f03080606c0 2023-10-06T11:31:16.778 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.766+0000 7fae32dc3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:31:16.865 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.861+0000 7f67186eb000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:31:16.945 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:16.941+0000 7f67186eb000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:31:17.004 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:16.994+0000 7fae32dc3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:31:17.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:31:17.014 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:31:17.030 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:17.025+0000 7f67186eb000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:31:17.095 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:17.086+0000 7fae32dc3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:31:17.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:17.122+0000 7f031b7fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:17.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:17.122+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:31:17.189 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:17.178+0000 7fae32dc3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:31:17.395 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:17.393+0000 7f67186eb000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:31:17.560 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:17.557+0000 7f67186eb000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:31:17.594 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:17.586+0000 7fae32dc3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:31:17.763 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:17.754+0000 7fae32dc3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:31:17.932 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:17.929+0000 7f67186eb000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:31:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:17.926+0000 7f031b7fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:17.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:17.926+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:31:18.140 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.130+0000 7fae32dc3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:31:18.148 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.145+0000 7f67186eb000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:31:18.237 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.233+0000 7f67186eb000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:31:18.314 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.309+0000 7f67186eb000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:31:18.355 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.346+0000 7fae32dc3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:31:18.388 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.385+0000 7f67186eb000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:31:18.448 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.438+0000 7fae32dc3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:31:18.525 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.514+0000 7fae32dc3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:31:18.599 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.597+0000 7f67186eb000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:31:18.604 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.594+0000 7fae32dc3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:31:18.677 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.673+0000 7f67186eb000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:31:18.761 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.757+0000 7f67186eb000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:31:18.823 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.814+0000 7fae32dc3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:31:18.898 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.886+0000 7fae32dc3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:31:18.941 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:18.937+0000 7f67186eb000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:31:18.987 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:18.978+0000 7fae32dc3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:31:19.023 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:19.021+0000 7f67186eb000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:31:19.179 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:19.170+0000 7fae32dc3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:31:19.260 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:19.250+0000 7fae32dc3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:31:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.530+0000 7f031b7fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:31:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.530+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:31:19.716 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:31:19.713+0000 7f67186eb000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:31:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.774+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 18) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f031404d6a0 con 0x7f031c15d860 2023-10-06T11:31:19.988 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:19.978+0000 7fae32dc3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:31:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.998+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 19) v1 ==== 79687+0+0 (secure 0 0 0) 0x7f031404ed40 con 0x7f031c15d860 2023-10-06T11:31:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.998+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:31:20.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:19.998+0000 7f03037fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf40ab700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf40ab700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf40ab700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf40ab700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf40ab700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.208 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf18a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.209 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf18a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.209 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf18a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.209 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf18a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:20.209 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:31:20.198+0000 7fadf18a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:31:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.010+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 20) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f031404df70 con 0x7f031c15d860 2023-10-06T11:31:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.010+0000 7f03037fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.010+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f031c16a410 con 0x7f03080606c0 2023-10-06T11:31:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.010+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.018+0000 7f031b7fe700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f030c0099b0 tx=0x7f030c017000 comp rx=0 tx=0).ready entity=mgr.6033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.130+0000 7f03037fe700 1 -- 172.21.15.46:0/3308081690 <== mgr.6033 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f031c16a410 con 0x7f03080606c0 2023-10-06T11:31:21.140 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T11:31:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 msgr2=0x7f0308062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f030c0099b0 tx=0x7f030c017000 comp rx=0 tx=0).stop 2023-10-06T11:31:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 msgr2=0x7f031c169430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0314009fc0 tx=0x7f03140099b0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 shutdown_connections 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f03080606c0 0x7f0308062af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031c15d860 0x7f031c169430 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f031c1591f0 0x7f031c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 --2- 172.21.15.46:0/3308081690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f031c0cc500 0x7f031c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 >> 172.21.15.46:0/3308081690 conn(0x7f031c06ef60 msgr2=0x7f031c0d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 shutdown_connections 2023-10-06T11:31:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.134+0000 7f0320bcc700 1 -- 172.21.15.46:0/3308081690 wait complete. 2023-10-06T11:31:21.167 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T11:31:21.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-10-06T11:31:21.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.382+0000 7f28ff0e3700 1 Processor -- start 2023-10-06T11:31:21.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.382+0000 7f28ff0e3700 1 -- start start 2023-10-06T11:31:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28ff0e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f28f8067280 0x7f28f8067650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28ff0e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28f815a840 0x7f28f815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28ff0e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80699b0 con 0x7f28f815c3f0 2023-10-06T11:31:21.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28ff0e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f815ac90 con 0x7f28f815a840 2023-10-06T11:31:21.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28ff0e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f8068da0 con 0x7f28f8067280 2023-10-06T11:31:21.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28f815a840 0x7f28f815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28f815a840 0x7f28f815beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41400/0 (socket says 172.21.15.46:41400) 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1513250112 learned_addr learned my addr 172.21.15.46:0/1513250112 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fe0e1700 1 --2- 172.21.15.46:0/1513250112 >> v2:172.21.15.132:3300/0 conn(0x7f28f8067280 0x7f28f8067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1513250112 >> v2:172.21.15.132:3300/0 conn(0x7f28f8067280 msgr2=0x7f28f8067650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 --2- 172.21.15.46:0/1513250112 >> v2:172.21.15.132:3300/0 conn(0x7f28f8067280 0x7f28f8067650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1513250112 >> v1:172.21.15.132:6789/0 conn(0x7f28f815c3f0 legacy=0x7f28f815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1513250112 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8069080 con 0x7f28f815a840 2023-10-06T11:31:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd8e0700 1 --2- 172.21.15.46:0/1513250112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f815beb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cb940 tx=0x7f28e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4397c253b2503291 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd0df700 1 -- 172.21.15.46:0/1513250112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e800b860 con 0x7f28f815a840 2023-10-06T11:31:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd0df700 1 -- 172.21.15.46:0/1513250112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e800b9c0 con 0x7f28f815a840 2023-10-06T11:31:21.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.386+0000 7f28fd0df700 1 -- 172.21.15.46:0/1513250112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e800bd60 con 0x7f28f815a840 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1513250112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 msgr2=0x7f28f815beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1513250112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f815beb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cb940 tx=0x7f28e8009250 comp rx=0 tx=0).stop 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1513250112 shutdown_connections 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1513250112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f815beb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1513250112 >> v2:172.21.15.132:3300/0 conn(0x7f28f8067280 0x7f28f8067650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1513250112 >> 172.21.15.46:0/1513250112 conn(0x7f28f80cbbb0 msgr2=0x7f28f80cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1513250112 shutdown_connections 2023-10-06T11:31:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1513250112 wait complete. 2023-10-06T11:31:21.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 Processor -- start 2023-10-06T11:31:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.390+0000 7f28ff0e3700 1 -- start start 2023-10-06T11:31:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28f8067280 0x7f28f80c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28f815c3f0 0x7f28f8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c4e80 con 0x7f28f815a840 2023-10-06T11:31:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8153240 con 0x7f28f815c3f0 2023-10-06T11:31:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80c3fb0 con 0x7f28f8067280 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41406/0 (socket says 172.21.15.46:41406) 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1897037443 learned_addr learned my addr 172.21.15.46:0/1897037443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fe8e2700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28f815c3f0 0x7f28f8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fe0e1700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28f8067280 0x7f28f80c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28f8067280 msgr2=0x7f28f80c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28f8067280 0x7f28f80c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28f815c3f0 msgr2=0x7f28f8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28f815c3f0 0x7f28f8154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 -- 172.21.15.46:0/1897037443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80094f0 con 0x7f28f815a840 2023-10-06T11:31:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28fd8e0700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cb970 tx=0x7f28e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8024440 con 0x7f28f815a840 2023-10-06T11:31:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80245a0 con 0x7f28f815a840 2023-10-06T11:31:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80c4550 con 0x7f28f815a840 2023-10-06T11:31:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8024960 con 0x7f28f815a840 2023-10-06T11:31:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.394+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80c4b10 con 0x7f28f815a840 2023-10-06T11:31:21.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.398+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8064590 con 0x7f28f815a840 2023-10-06T11:31:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.398+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f28e8024d50 con 0x7f28f815a840 2023-10-06T11:31:21.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.398+0000 7f28ee7fc700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 0x7f28dc0628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.402+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f28e807e570 con 0x7f28f815a840 2023-10-06T11:31:21.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.402+0000 7f28fe0e1700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 0x7f28dc0628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.402+0000 7f28fe0e1700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 0x7f28dc0628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006fd0 tx=0x7f28f4008040 comp rx=0 tx=0).ready entity=mgr.6033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.406+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f28e804e8d0 con 0x7f28f815a840 2023-10-06T11:31:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.598+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f8069360 con 0x7f28dc0603f0 2023-10-06T11:31:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.698+0000 7f28ee7fc700 1 -- 172.21.15.46:0/1897037443 <== mgr.6033 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28f8069360 con 0x7f28dc0603f0 2023-10-06T11:31:21.709 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T11:31:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 msgr2=0x7f28dc0628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 0x7f28dc0628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006fd0 tx=0x7f28f4008040 comp rx=0 tx=0).stop 2023-10-06T11:31:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 msgr2=0x7f28f80c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cb970 tx=0x7f28e8017000 comp rx=0 tx=0).stop 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 shutdown_connections 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f28dc0603f0 0x7f28dc0628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f28f815c3f0 0x7f28f8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28f815a840 0x7f28f80c5310 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 --2- 172.21.15.46:0/1897037443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28f8067280 0x7f28f80c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 >> 172.21.15.46:0/1897037443 conn(0x7f28f80cbbb0 msgr2=0x7f28f80ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 shutdown_connections 2023-10-06T11:31:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.702+0000 7f28ff0e3700 1 -- 172.21.15.46:0/1897037443 wait complete. 2023-10-06T11:31:21.735 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T11:31:21.735 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-11_32_00 in ~39s... 2023-10-06T11:31:21.735 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-11_33_00 in ~99s... 2023-10-06T11:31:21.735 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-10-06T11:31:21.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T11:31:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.898+0000 7f05960e3700 1 Processor -- start 2023-10-06T11:31:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.898+0000 7f05960e3700 1 -- start start 2023-10-06T11:31:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05960e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0590069770 0x7f05900b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05960e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 0x7f05900b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05960e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900b5eb0 con 0x7f0590158de0 2023-10-06T11:31:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05960e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05900cef30 con 0x7f0590069770 2023-10-06T11:31:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05960e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05900cf4f0 con 0x7f05901591b0 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0590069770 0x7f05900b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0590069770 0x7f05900b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41410/0 (socket says 172.21.15.46:41410) 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 -- 172.21.15.46:0/1598044901 learned_addr learned my addr 172.21.15.46:0/1598044901 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05958e2700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 0x7f05900b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 -- 172.21.15.46:0/1598044901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 msgr2=0x7f05900b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 0x7f05900b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 -- 172.21.15.46:0/1598044901 >> v1:172.21.15.46:6789/0 conn(0x7f0590158de0 legacy=0x7f0590069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05948e0700 1 -- 172.21.15.46:0/1598044901 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900ccac0 con 0x7f0590069770 2023-10-06T11:31:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.902+0000 7f05958e2700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 0x7f05900b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:31:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05948e0700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f05900b5970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f05800099b0 tx=0x7f0580009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0fcaaba747599a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f0587fff700 1 -- 172.21.15.46:0/1598044901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000b860 con 0x7f0590069770 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f0587fff700 1 -- 172.21.15.46:0/1598044901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058000b9c0 con 0x7f0590069770 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f0587fff700 1 -- 172.21.15.46:0/1598044901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000bd60 con 0x7f0590069770 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 -- 172.21.15.46:0/1598044901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 msgr2=0x7f05900b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f05900b5970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f05800099b0 tx=0x7f0580009250 comp rx=0 tx=0).stop 2023-10-06T11:31:21.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 -- 172.21.15.46:0/1598044901 shutdown_connections 2023-10-06T11:31:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05901591b0 0x7f05900b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 --2- 172.21.15.46:0/1598044901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f05900b5970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 -- 172.21.15.46:0/1598044901 >> 172.21.15.46:0/1598044901 conn(0x7f0590068390 msgr2=0x7f05900b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 -- 172.21.15.46:0/1598044901 shutdown_connections 2023-10-06T11:31:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.906+0000 7f05960e3700 1 -- 172.21.15.46:0/1598044901 wait complete. 2023-10-06T11:31:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 Processor -- start 2023-10-06T11:31:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 -- start start 2023-10-06T11:31:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0590158de0 0x7f059006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05901591b0 0x7f059006cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059015c560 con 0x7f0590069770 2023-10-06T11:31:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059015b950 con 0x7f05901591b0 2023-10-06T11:31:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05960e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059015bc30 con 0x7f0590158de0 2023-10-06T11:31:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.910+0000 7f05950e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05958e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05901591b0 0x7f059006cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05948e0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0590158de0 0x7f059006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41418/0 (socket says 172.21.15.46:41418) 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 -- 172.21.15.46:0/3798046128 learned_addr learned my addr 172.21.15.46:0/3798046128 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 -- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0590158de0 msgr2=0x7f059006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0590158de0 0x7f059006ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 -- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05901591b0 msgr2=0x7f059006cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05901591b0 0x7f059006cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 -- 172.21.15.46:0/3798046128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800094f0 con 0x7f0590069770 2023-10-06T11:31:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05950e1700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c590 tx=0x7f058c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011590 con 0x7f0590069770 2023-10-06T11:31:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c011710 con 0x7f0590069770 2023-10-06T11:31:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011ad0 con 0x7f0590069770 2023-10-06T11:31:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059015bf10 con 0x7f0590069770 2023-10-06T11:31:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059014a690 con 0x7f0590069770 2023-10-06T11:31:21.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.914+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0590064590 con 0x7f0590069770 2023-10-06T11:31:21.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.918+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 20) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f058c01b3b0 con 0x7f0590069770 2023-10-06T11:31:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.918+0000 7f05857fa700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 0x7f05740628d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:31:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.918+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f058c07bd10 con 0x7f0590069770 2023-10-06T11:31:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.918+0000 7f05948e0700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 0x7f05740628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:31:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.922+0000 7f05948e0700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 0x7f05740628d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f05800099b0 tx=0x7f0580017000 comp rx=0 tx=0).ready entity=mgr.6033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:31:21.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:21.926+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f058c081020 con 0x7f0590069770 2023-10-06T11:31:22.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.026+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 21) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f058c04c440 con 0x7f0590069770 2023-10-06T11:31:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.130+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05900b5eb0 con 0x7f0574060420 2023-10-06T11:31:22.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.142+0000 7f05857fa700 1 -- 172.21.15.46:0/3798046128 <== mgr.6033 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f05900b5eb0 con 0x7f0574060420 2023-10-06T11:31:22.153 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:31:22.153 INFO:teuthology.orchestra.run.smithi046.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-06T11:31:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 msgr2=0x7f05740628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 0x7f05740628d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f05800099b0 tx=0x7f0580017000 comp rx=0 tx=0).stop 2023-10-06T11:31:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 msgr2=0x7f059014a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:31:22.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c590 tx=0x7f058c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 shutdown_connections 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0574060420 0x7f05740628d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05901591b0 0x7f059006cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0590158de0 0x7f059006ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 --2- 172.21.15.46:0/3798046128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0590069770 0x7f059014a150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 >> 172.21.15.46:0/3798046128 conn(0x7f0590068390 msgr2=0x7f0590154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 shutdown_connections 2023-10-06T11:31:22.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:31:22.146+0000 7f05960e3700 1 -- 172.21.15.46:0/3798046128 wait complete. 2023-10-06T11:31:22.179 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-06T11:31:22.180 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-10-06T11:31:22.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:22.180 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:22.204 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:22.204 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:22.204 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:23.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:23.205 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:23.230 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:23.230 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:23.230 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:24.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:24.231 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:24.255 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:24.255 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:24.255 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:25.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:25.256 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:25.279 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:25.280 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:25.280 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:26.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:26.281 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:26.305 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:26.305 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:26.305 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:27.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:27.306 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:27.331 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:27.331 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:27.331 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:28.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:28.333 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:28.357 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:28.357 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:28.357 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:29.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:29.358 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:29.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:29.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:29.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:30.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:30.384 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:30.409 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:30.409 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:30.409 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:31.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:31.410 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:31.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:31.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:31.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:32.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:32.434 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:32.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:32.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:32.458 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:33.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:33.459 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:33.481 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:33.482 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:33.482 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:34.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:34.482 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:34.508 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:34.508 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:34.508 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:35.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:35.509 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:35.532 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:35.532 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:35.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:36.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:36.533 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:36.557 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:36.557 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:36.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:37.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:37.558 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:37.583 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:37.583 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:37.583 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:38.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:38.584 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:38.609 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:38.609 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:38.609 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:39.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:39.611 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:39.632 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:39.632 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:39.632 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:40.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:40.633 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:40.657 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:40.657 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:40.657 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:41.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:41.658 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:41.682 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:41.683 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:41.683 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:42.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:42.684 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:42.710 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:42.710 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:42.711 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:43.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:43.711 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:43.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:43.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:43.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:44.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:44.738 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:44.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:44.766 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:44.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:45.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:45.767 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:45.792 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:45.792 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:45.793 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:46.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:46.794 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:46.819 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:46.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:46.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:47.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:31:47.097 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:31:47.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:47.821 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:47.846 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:47.846 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:47.846 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:48.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:48.847 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:48.870 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:48.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:48.871 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:49.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:49.872 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:49.895 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:49.895 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:49.895 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:50.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:50.897 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:50.920 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:50.920 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:50.920 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:51.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:51.922 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:51.946 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:51.946 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:51.947 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:52.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:52.948 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:52.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:52.970 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:52.970 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:53.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:53.972 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:53.992 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:53.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:53.993 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:54.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:54.993 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:55.019 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:55.020 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:55.020 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:56.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:56.021 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:56.046 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:56.046 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:56.046 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:57.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:57.047 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:57.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:57.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:57.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:58.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:58.072 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:58.096 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:58.097 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:58.097 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:31:59.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:31:59.098 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:31:59.124 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:31:59.124 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:31:59.124 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:00.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:00.125 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:00.149 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:32:00.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:00.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:00.730 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:32:00.717+0000 7fade408b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:32:00.730 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:32:00.717+0000 7fade408b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:32:00.731 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:32:00.717+0000 7fade408b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:32:00.731 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:32:00.717+0000 7fade408b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:32:00.731 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:32:00.717+0000 7fade408b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:32:01.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:01.150 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:01.172 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:01.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:01.173 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:01.174 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:01.174 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-11_32_00_UTC'] 2023-10-06T11:32:01.174 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696591881.7351637, now = 1696591921.1743097, timo = 39 2023-10-06T11:32:02.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:02.175 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:02.200 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:02.201 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:02.201 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:02.201 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:03.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:03.202 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:03.225 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:03.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:03.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:03.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:04.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:04.228 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:04.253 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:04.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:04.254 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:04.254 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:05.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:05.256 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:05.277 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:05.278 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:05.279 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:05.279 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:06.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:06.281 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:06.305 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:06.306 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:06.306 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:06.307 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:07.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:07.308 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:07.330 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:07.332 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:07.332 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:07.332 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:08.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:08.333 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:08.355 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:08.357 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:08.357 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:08.357 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:09.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:09.357 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:09.382 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:09.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:09.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:09.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:10.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:10.386 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:10.410 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:10.412 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:10.412 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:10.412 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:11.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:11.413 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:11.437 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:11.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:11.439 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:11.439 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:12.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:12.440 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:12.462 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:12.464 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:12.464 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:12.464 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:13.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:13.466 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:13.488 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:13.490 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:13.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:13.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:14.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:14.491 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:14.511 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:14.513 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:14.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:14.513 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:15.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:15.513 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:15.536 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:15.538 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:15.538 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:15.538 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:16.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:16.539 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:16.561 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:16.563 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:16.563 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:16.563 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:17.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:32:17.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:32:17.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:17.564 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:17.586 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:17.588 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:17.588 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:17.588 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:18.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:18.589 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:18.612 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:18.614 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:18.614 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:18.614 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:19.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:19.615 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:19.639 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:19.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:19.641 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:19.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:20.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:20.642 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:20.664 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:20.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:20.666 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:20.666 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:21.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:21.667 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:21.692 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:21.692 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:21.692 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:22.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:22.693 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:22.718 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:22.719 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:22.719 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:22.719 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:23.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:23.721 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:23.745 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:23.746 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:23.746 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:23.746 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:24.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:24.747 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:24.770 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:24.772 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:24.772 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:24.772 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:25.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:25.773 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:25.807 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:25.809 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:25.809 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:25.809 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:26.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:26.810 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:26.833 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:26.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:26.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:26.836 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:27.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:27.837 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:27.859 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:27.859 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:27.860 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:28.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:28.860 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:28.883 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:28.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:28.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:28.885 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:29.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:29.886 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:29.910 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:29.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:29.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:29.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:30.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:30.913 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:30.937 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:30.939 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:30.939 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:30.939 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:31.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:31.940 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:31.963 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:31.965 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:31.965 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:31.965 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:32.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:32.966 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:32.983 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:32.985 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:32.985 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:32.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:33.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:33.986 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:34.004 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:34.006 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:34.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:34.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:35.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:35.008 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:35.033 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:35.034 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:35.034 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:35.034 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:36.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:36.035 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:36.053 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:36.054 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:36.054 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:36.054 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:37.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:37.055 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:37.078 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:37.080 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:37.080 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:37.080 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:38.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:38.081 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:38.103 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:38.104 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:38.105 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:38.105 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:39.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:39.105 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:39.126 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:39.128 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:39.128 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:39.128 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:40.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:40.129 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:40.151 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:40.153 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:40.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:40.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:41.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:41.155 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:41.177 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:41.179 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:41.179 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:41.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:42.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:42.180 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:42.205 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:42.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:42.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:42.206 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:43.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:43.207 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:43.231 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:43.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:43.233 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:43.233 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:44.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:44.234 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:44.256 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:44.258 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:44.258 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:44.258 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:45.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:45.259 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:45.282 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:45.284 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:45.284 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:45.285 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:46.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:46.286 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:46.310 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:46.312 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:46.312 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:46.312 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:47.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:32:47.245 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:32:47.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:47.313 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:47.335 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:47.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:47.337 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:47.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:48.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:48.338 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:48.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:48.363 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:48.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:48.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:49.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:49.365 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:49.388 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:49.390 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:49.390 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:49.390 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:50.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:50.391 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:50.415 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:50.416 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:50.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:50.417 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:51.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:51.418 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:51.441 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:51.443 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:51.443 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:51.443 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:52.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:52.445 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:52.468 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:52.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:52.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:52.470 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:53.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:53.472 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:53.497 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:53.499 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:53.499 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:53.499 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:54.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:54.500 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:54.523 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:54.525 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:54.525 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:54.525 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:55.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:55.526 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:55.545 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:55.546 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:55.547 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:55.547 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:56.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:56.548 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:56.570 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:56.572 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:56.572 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:56.572 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:57.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:57.573 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:57.597 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:57.599 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:57.599 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:57.599 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:58.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:58.600 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:58.625 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:58.627 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:58.627 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:58.627 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:32:59.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:32:59.628 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:32:59.652 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:32:59.653 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:32:59.654 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:32:59.654 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:33:00.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:00.654 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:33:00.678 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:33:00.679 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:33:00.679 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:33:00.680 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:33:01.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:01.680 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:33:01.700 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:33:01.700 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_33_00_UTC 2023-10-06T11:33:01.702 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_33_00_UTC', 'scheduled-2023-10-06-11_32_00_UTC'} 2023-10-06T11:33:01.702 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-11_33_00_UTC'} 2023-10-06T11:33:01.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:33:01.702 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-11_33_00_UTC'] 2023-10-06T11:33:01.702 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696591881.7351637, now = 1696591981.7029192, timo = 99 2023-10-06T11:33:02.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T11:33:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.872+0000 7f7f1759e700 1 Processor -- start 2023-10-06T11:33:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.872+0000 7f7f1759e700 1 -- start start 2023-10-06T11:33:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f18158de0 0x7f7f180b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f181591b0 0x7f7f18069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180b5eb0 con 0x7f7f180695a0 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f180cef30 con 0x7f7f181591b0 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f180cf4f0 con 0x7f7f18158de0 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f0ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f181591b0 0x7f7f18069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f16d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f180695a0 0x7f7f1815cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43758/0 (socket says 172.21.15.46:43758) 2023-10-06T11:33:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f1659c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f18158de0 0x7f7f180b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f16d9d700 1 -- 172.21.15.46:0/3461443763 learned_addr learned my addr 172.21.15.46:0/3461443763 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f15d9b700 1 -- 172.21.15.46:0/3461443763 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713029336 0 0) 0x7f7f10001490 con 0x7f7f180695a0 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f15d9b700 1 -- 172.21.15.46:0/3461443763 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f04003570 con 0x7f7f180695a0 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f0ffff700 1 -- 172.21.15.46:0/3461443763 >> v2:172.21.15.132:3300/0 conn(0x7f7f18158de0 msgr2=0x7f7f180b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/3461443763 >> v2:172.21.15.132:3300/0 conn(0x7f7f18158de0 0x7f7f180b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f0ffff700 1 -- 172.21.15.46:0/3461443763 >> v1:172.21.15.46:6789/0 conn(0x7f7f180695a0 legacy=0x7f7f1815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.876+0000 7f7f0ffff700 1 -- 172.21.15.46:0/3461443763 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180699c0 con 0x7f7f181591b0 2023-10-06T11:33:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/3461443763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f181591b0 0x7f7f18069060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7f18088830 tx=0x7f7f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece66b9e63b309b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f15d9b700 1 -- 172.21.15.46:0/3461443763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00011630 con 0x7f7f181591b0 2023-10-06T11:33:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f15d9b700 1 -- 172.21.15.46:0/3461443763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f000117b0 con 0x7f7f181591b0 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f15d9b700 1 -- 172.21.15.46:0/3461443763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00011b70 con 0x7f7f181591b0 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- 172.21.15.46:0/3461443763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f181591b0 msgr2=0x7f7f18069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 --2- 172.21.15.46:0/3461443763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f181591b0 0x7f7f18069060 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7f18088830 tx=0x7f7f000091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- 172.21.15.46:0/3461443763 shutdown_connections 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 --2- 172.21.15.46:0/3461443763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f181591b0 0x7f7f18069060 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 --2- 172.21.15.46:0/3461443763 >> v2:172.21.15.132:3300/0 conn(0x7f7f18158de0 0x7f7f180b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- 172.21.15.46:0/3461443763 >> 172.21.15.46:0/3461443763 conn(0x7f7f18068390 msgr2=0x7f7f180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- 172.21.15.46:0/3461443763 shutdown_connections 2023-10-06T11:33:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- 172.21.15.46:0/3461443763 wait complete. 2023-10-06T11:33:02.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 Processor -- start 2023-10-06T11:33:02.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.880+0000 7f7f1759e700 1 -- start start 2023-10-06T11:33:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 0x7f7f1816a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f181591b0 0x7f7f18169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1816a770 con 0x7f7f180695a0 2023-10-06T11:33:02.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f18169680 con 0x7f7f181591b0 2023-10-06T11:33:02.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1815c860 con 0x7f7f18158de0 2023-10-06T11:33:02.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53028/0 (socket says 172.21.15.46:53028) 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 -- 172.21.15.46:0/1015312451 learned_addr learned my addr 172.21.15.46:0/1015312451 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f16d9d700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f181591b0 0x7f7f18169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 0x7f7f1816a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 -- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 msgr2=0x7f7f1816a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 0x7f7f1816a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 -- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f181591b0 msgr2=0x7f7f18169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f181591b0 0x7f7f18169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 -- 172.21.15.46:0/1015312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00009420 con 0x7f7f180695a0 2023-10-06T11:33:02.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 0x7f7f1816a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1659c700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800ec80 tx=0x7f7f080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800c850 con 0x7f7f180695a0 2023-10-06T11:33:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0800c9b0 con 0x7f7f180695a0 2023-10-06T11:33:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1816adb0 con 0x7f7f180695a0 2023-10-06T11:33:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800cd50 con 0x7f7f180695a0 2023-10-06T11:33:02.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.884+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1816b370 con 0x7f7f180695a0 2023-10-06T11:33:02.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.888+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f18064590 con 0x7f7f180695a0 2023-10-06T11:33:02.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.888+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f7f08013680 con 0x7f7f180695a0 2023-10-06T11:33:02.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.892+0000 7f7f0e7fc700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 0x7f7ef8062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:02.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.892+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6904+0+0 (secure 0 0 0) 0x7f7f0807d4e0 con 0x7f7f180695a0 2023-10-06T11:33:02.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.892+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 0x7f7ef8062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:02.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.892+0000 7f7f0ffff700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 0x7f7ef8062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f18088830 tx=0x7f7f00017000 comp rx=0 tx=0).ready entity=mgr.6033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:02.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:02.896+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f0804d470 con 0x7f7f180695a0 2023-10-06T11:33:03.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.092+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f180ccac0 con 0x7f7ef80606c0 2023-10-06T11:33:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.208+0000 7f7f0e7fc700 1 -- 172.21.15.46:0/1015312451 <== mgr.6033 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7f180ccac0 con 0x7f7ef80606c0 2023-10-06T11:33:03.223 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path snap_test_dir1 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.212+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 msgr2=0x7f7ef8062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.212+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 0x7f7ef8062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f18088830 tx=0x7f7f00017000 comp rx=0 tx=0).stop 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.212+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 msgr2=0x7f7f18167440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.212+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800ec80 tx=0x7f7f080091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 shutdown_connections 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef80606c0 0x7f7ef8062b70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f181591b0 0x7f7f18169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f18158de0 0x7f7f1816a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 --2- 172.21.15.46:0/1015312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f180695a0 0x7f7f18167440 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 >> 172.21.15.46:0/1015312451 conn(0x7f7f18068390 msgr2=0x7f7f18154e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 shutdown_connections 2023-10-06T11:33:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.216+0000 7f7f1759e700 1 -- 172.21.15.46:0/1015312451 wait complete. 2023-10-06T11:33:03.249 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T11:33:03.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:03.250 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:33:03.271 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:33:03.271 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_33_00_UTC 2023-10-06T11:33:03.273 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_32_00_UTC 2023-10-06T11:33:03.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:03.273 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_32_00_UTC) 2023-10-06T11:33:03.346 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_33_00_UTC 2023-10-06T11:33:03.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:03.346 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_33_00_UTC) 2023-10-06T11:33:03.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:03.420 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T11:33:03.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:33:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.692+0000 7f8abf59e700 1 Processor -- start 2023-10-06T11:33:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.692+0000 7f8abf59e700 1 -- start start 2023-10-06T11:33:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abf59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac00cb740 con 0x7f8ac00c8dc0 2023-10-06T11:33:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abf59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac0157290 con 0x7f8ac0156ec0 2023-10-06T11:33:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abf59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00ab320 con 0x7f8ac00cb940 2023-10-06T11:33:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ac00c8dc0 0x7f8ac00cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54600/0 (socket says 172.21.15.46:54600) 2023-10-06T11:33:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abed9d700 1 -- 172.21.15.46:0/1737499015 learned_addr learned my addr 172.21.15.46:0/1737499015 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/1737499015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abd59a700 1 -- 172.21.15.46:0/1737499015 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633207069 0 0) 0x7f8ac00cb740 con 0x7f8ac00c8dc0 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.696+0000 7f8abd59a700 1 -- 172.21.15.46:0/1737499015 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa8003570 con 0x7f8ac00c8dc0 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abdd9b700 1 -- 172.21.15.46:0/1737499015 >> v1:172.21.15.132:6789/0 conn(0x7f8ac0156ec0 legacy=0x7f8ac00cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abdd9b700 1 -- 172.21.15.46:0/1737499015 >> v1:172.21.15.46:6789/0 conn(0x7f8ac00c8dc0 legacy=0x7f8ac00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abdd9b700 1 -- 172.21.15.46:0/1737499015 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00cbdd0 con 0x7f8ac00cb940 2023-10-06T11:33:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/1737499015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00aade0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009910 tx=0x7f8ab40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33782cc75fa7c043 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abd59a700 1 -- 172.21.15.46:0/1737499015 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400b680 con 0x7f8ac00cb940 2023-10-06T11:33:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abd59a700 1 -- 172.21.15.46:0/1737499015 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab400b7e0 con 0x7f8ac00cb940 2023-10-06T11:33:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abd59a700 1 -- 172.21.15.46:0/1737499015 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400bba0 con 0x7f8ac00cb940 2023-10-06T11:33:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 -- 172.21.15.46:0/1737499015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 msgr2=0x7f8ac00aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 --2- 172.21.15.46:0/1737499015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00aade0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009910 tx=0x7f8ab40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 -- 172.21.15.46:0/1737499015 shutdown_connections 2023-10-06T11:33:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 --2- 172.21.15.46:0/1737499015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00aade0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 -- 172.21.15.46:0/1737499015 >> 172.21.15.46:0/1737499015 conn(0x7f8ac0097060 msgr2=0x7f8ac0067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 -- 172.21.15.46:0/1737499015 shutdown_connections 2023-10-06T11:33:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.700+0000 7f8abf59e700 1 -- 172.21.15.46:0/1737499015 wait complete. 2023-10-06T11:33:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 Processor -- start 2023-10-06T11:33:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 -- start start 2023-10-06T11:33:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac00c8dc0 0x7f8ac00c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 0x7f8ac01699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0155bf0 con 0x7f8ac0156ec0 2023-10-06T11:33:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00c1220 con 0x7f8ac00c8dc0 2023-10-06T11:33:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abf59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac016a280 con 0x7f8ac00cb940 2023-10-06T11:33:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 0x7f8ac01699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 0x7f8ac01699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42982/0 (socket says 172.21.15.46:42982) 2023-10-06T11:33:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abed9d700 1 -- 172.21.15.46:0/348709897 learned_addr learned my addr 172.21.15.46:0/348709897 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.704+0000 7f8abe59c700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac00c8dc0 0x7f8ac00c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 -- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac00c8dc0 msgr2=0x7f8ac00c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac00c8dc0 0x7f8ac00c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 -- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 msgr2=0x7f8ac01699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 0x7f8ac01699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 -- 172.21.15.46:0/348709897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009420 con 0x7f8ac00cb940 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abdd9b700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00c0ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4016000 tx=0x7f8ab4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400bb60 con 0x7f8ac00cb940 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4000c90 con 0x7f8ac00cb940 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac016a560 con 0x7f8ac00cb940 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab401a660 con 0x7f8ac00cb940 2023-10-06T11:33:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00c1870 con 0x7f8ac00cb940 2023-10-06T11:33:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.708+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac0064590 con 0x7f8ac00cb940 2023-10-06T11:33:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.712+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 23) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f8ab401a9d0 con 0x7f8ac00cb940 2023-10-06T11:33:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.712+0000 7f8aae7fc700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 0x7f8aa0062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.712+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6904+0+0 (secure 0 0 0) 0x7f8ab407e850 con 0x7f8ac00cb940 2023-10-06T11:33:03.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.712+0000 7f8abe59c700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 0x7f8aa0062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.712+0000 7f8abe59c700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 0x7f8aa0062b70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0003a20 tx=0x7f8ab0007f10 comp rx=0 tx=0).ready entity=mgr.6033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:03.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.720+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ab404e7e0 con 0x7f8ac00cb940 2023-10-06T11:33:03.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:03.928+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8ac00cc0b0 con 0x7f8ac00cb940 2023-10-06T11:33:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.464+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f8ab404ebb0 con 0x7f8ac00cb940 2023-10-06T11:33:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8aae7fc700 1 -- 172.21.15.46:0/348709897 <== mon.2 v2:172.21.15.132:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v24) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8ab40263f0 con 0x7f8ac00cb940 2023-10-06T11:33:04.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 msgr2=0x7f8aa0062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 0x7f8aa0062b70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0003a20 tx=0x7f8ab0007f10 comp rx=0 tx=0).stop 2023-10-06T11:33:04.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 msgr2=0x7f8ac00c0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00c0ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4016000 tx=0x7f8ab4017000 comp rx=0 tx=0).stop 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 shutdown_connections 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8aa00606c0 0x7f8aa0062b70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac0156ec0 0x7f8ac01699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ac00cb940 0x7f8ac00c0ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 --2- 172.21.15.46:0/348709897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac00c8dc0 0x7f8ac00c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 >> 172.21.15.46:0/348709897 conn(0x7f8ac0097060 msgr2=0x7f8ac0067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 shutdown_connections 2023-10-06T11:33:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.468+0000 7f8abf59e700 1 -- 172.21.15.46:0/348709897 wait complete. 2023-10-06T11:33:04.502 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:33:04.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:04.541 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:04.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:33:04.585 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:33:04.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:04.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:04.654 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:04.647+0000 7f9122f5a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:04.666 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:33:04.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:04.666 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:33:04.674 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:04.660+0000 7f90415c3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:04.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 Processor -- start 2023-10-06T11:33:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 -- start start 2023-10-06T11:33:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ae406da60 0x7f1ae40c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4067050 0x7f1ae40674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae40c6690 con 0x7f1ae4155c70 2023-10-06T11:33:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae406dee0 con 0x7f1ae406da60 2023-10-06T11:33:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aeb04c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae4067a10 con 0x7f1ae4067050 2023-10-06T11:33:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1ae9849700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ae406da60 0x7f1ae40c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1ae9849700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ae406da60 0x7f1ae40c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54400/0 (socket says 172.21.15.46:54400) 2023-10-06T11:33:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1ae9849700 1 -- 172.21.15.46:0/2536101797 learned_addr learned my addr 172.21.15.46:0/2536101797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.872+0000 7f1aea84b700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4067050 0x7f1ae40674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9849700 1 -- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4067050 msgr2=0x7f1ae40674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9849700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4067050 0x7f1ae40674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9849700 1 -- 172.21.15.46:0/2536101797 >> v1:172.21.15.132:6789/0 conn(0x7f1ae4155c70 legacy=0x7f1ae40c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9849700 1 -- 172.21.15.46:0/2536101797 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40c74d0 con 0x7f1ae406da60 2023-10-06T11:33:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9849700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae406da60 0x7f1ae40c6150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40099b0 tx=0x7f1ad4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=213d15e65dddabe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9048700 1 -- 172.21.15.46:0/2536101797 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400b860 con 0x7f1ae406da60 2023-10-06T11:33:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9048700 1 -- 172.21.15.46:0/2536101797 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad400b9c0 con 0x7f1ae406da60 2023-10-06T11:33:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1ae9048700 1 -- 172.21.15.46:0/2536101797 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400bd60 con 0x7f1ae406da60 2023-10-06T11:33:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 -- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae406da60 msgr2=0x7f1ae40c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae406da60 0x7f1ae40c6150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40099b0 tx=0x7f1ad4009250 comp rx=0 tx=0).stop 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 -- 172.21.15.46:0/2536101797 shutdown_connections 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4067050 0x7f1ae40674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/2536101797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae406da60 0x7f1ae40c6150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 -- 172.21.15.46:0/2536101797 >> 172.21.15.46:0/2536101797 conn(0x7f1ae4062ef0 msgr2=0x7f1ae40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 -- 172.21.15.46:0/2536101797 shutdown_connections 2023-10-06T11:33:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.876+0000 7f1aeb04c700 1 -- 172.21.15.46:0/2536101797 wait complete. 2023-10-06T11:33:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 Processor -- start 2023-10-06T11:33:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 -- start start 2023-10-06T11:33:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae4067050 0x7f1ae40c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 0x7f1ae40c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40af180 con 0x7f1ae406da60 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40aeb00 con 0x7f1ae4067050 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.880+0000 7f1aeb04c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40c5320 con 0x7f1ae4155c70 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1aea84b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 0x7f1ae40c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43012/0 (socket says 172.21.15.46:43012) 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1aea84b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 0x7f1ae40c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42388/0 (socket says 172.21.15.46:42388) 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 -- 172.21.15.46:0/4097303923 learned_addr learned my addr 172.21.15.46:0/4097303923 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1aea04a700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae4067050 0x7f1ae40c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 msgr2=0x7f1ae40c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 0x7f1ae40c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae4067050 msgr2=0x7f1ae40c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae4067050 0x7f1ae40c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 -- 172.21.15.46:0/4097303923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40094f0 con 0x7f1ae406da60 2023-10-06T11:33:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ae9849700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009290 tx=0x7f1ad4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4011e00 con 0x7f1ae406da60 2023-10-06T11:33:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4024440 con 0x7f1ae406da60 2023-10-06T11:33:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae40c5040 con 0x7f1ae406da60 2023-10-06T11:33:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4024800 con 0x7f1ae406da60 2023-10-06T11:33:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.884+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae40c4ce0 con 0x7f1ae406da60 2023-10-06T11:33:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae40ae960 con 0x7f1ae406da60 2023-10-06T11:33:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f1ad401a420 con 0x7f1ae406da60 2023-10-06T11:33:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1ada7fc700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 0x7f1ac8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6904+0+0 (secure 0 0 0) 0x7f1ad407ed50 con 0x7f1ae406da60 2023-10-06T11:33:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1aea04a700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 msgr2=0x7f1ac8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.888+0000 7f1aea04a700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 0x7f1ac8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:04.896+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ae40ae960 con 0x7f1ae406da60 2023-10-06T11:33:05.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.092+0000 7f1aea04a700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 msgr2=0x7f1ac8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:05.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.092+0000 7f1aea04a700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 0x7f1ac8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.108+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ae40ae960 con 0x7f1ae406da60 2023-10-06T11:33:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.108+0000 7f1ada7fc700 1 -- 172.21.15.46:0/4097303923 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f1ae40ae960 con 0x7f1ae406da60 2023-10-06T11:33:05.122 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:05.122 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:33:05.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 msgr2=0x7f1ac8062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 0x7f1ac8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 msgr2=0x7f1ae40b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009290 tx=0x7f1ad4017000 comp rx=0 tx=0).stop 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 shutdown_connections 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ac8060670 0x7f1ac8062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ae4155c70 0x7f1ae40c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae406da60 0x7f1ae40b0e00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 --2- 172.21.15.46:0/4097303923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ae4067050 0x7f1ae40c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 >> 172.21.15.46:0/4097303923 conn(0x7f1ae4062ef0 msgr2=0x7f1ae40aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 shutdown_connections 2023-10-06T11:33:05.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.112+0000 7f1aeb04c700 1 -- 172.21.15.46:0/4097303923 wait complete. 2023-10-06T11:33:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:33:05.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:05.191 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.187+0000 7f9122f5a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:05.255 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:05.244+0000 7f90415c3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:05.259 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.255+0000 7f9122f5a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:05.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 Processor -- start 2023-10-06T11:33:05.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 -- start start 2023-10-06T11:33:05.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffa4c067760 0x7ffa4c0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa4c0bf1a0 0x7ffa4c0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0c8480 con 0x7ffa4c0bedd0 2023-10-06T11:33:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c069660 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa52fcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c068ac0 con 0x7ffa4c067760 2023-10-06T11:33:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa51fca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa4c0bedd0 0x7ffa4c067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54622/0 (socket says 172.21.15.46:54622) 2023-10-06T11:33:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa51fca700 1 -- 172.21.15.46:0/894328055 learned_addr learned my addr 172.21.15.46:0/894328055 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa527cb700 1 --2- 172.21.15.46:0/894328055 >> v2:172.21.15.46:3300/0 conn(0x7ffa4c0bf1a0 0x7ffa4c0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.288+0000 7ffa517c9700 1 --2- 172.21.15.46:0/894328055 >> v2:172.21.15.132:3300/0 conn(0x7ffa4c067760 0x7ffa4c0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa50fc8700 1 -- 172.21.15.46:0/894328055 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093454159 0 0) 0x7ffa48001110 con 0x7ffa4c0bedd0 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa50fc8700 1 -- 172.21.15.46:0/894328055 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa38003570 con 0x7ffa4c0bedd0 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa527cb700 1 -- 172.21.15.46:0/894328055 >> v2:172.21.15.132:3300/0 conn(0x7ffa4c067760 msgr2=0x7ffa4c0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa527cb700 1 --2- 172.21.15.46:0/894328055 >> v2:172.21.15.132:3300/0 conn(0x7ffa4c067760 0x7ffa4c0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa527cb700 1 -- 172.21.15.46:0/894328055 >> v1:172.21.15.46:6789/0 conn(0x7ffa4c0bedd0 legacy=0x7ffa4c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa527cb700 1 -- 172.21.15.46:0/894328055 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c068d30 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa527cb700 1 --2- 172.21.15.46:0/894328055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c7f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0cc5e0 tx=0x7ffa44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9e2a5ee9678f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:05.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa50fc8700 1 -- 172.21.15.46:0/894328055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400e070 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa50fc8700 1 -- 172.21.15.46:0/894328055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4400bab0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa50fc8700 1 -- 172.21.15.46:0/894328055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400be70 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 -- 172.21.15.46:0/894328055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bf1a0 msgr2=0x7ffa4c0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/894328055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c7f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0cc5e0 tx=0x7ffa44009780 comp rx=0 tx=0).stop 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 -- 172.21.15.46:0/894328055 shutdown_connections 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/894328055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c7f40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/894328055 >> v2:172.21.15.132:3300/0 conn(0x7ffa4c067760 0x7ffa4c0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 -- 172.21.15.46:0/894328055 >> 172.21.15.46:0/894328055 conn(0x7ffa4c062ef0 msgr2=0x7ffa4c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 -- 172.21.15.46:0/894328055 shutdown_connections 2023-10-06T11:33:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.292+0000 7ffa52fcc700 1 -- 172.21.15.46:0/894328055 wait complete. 2023-10-06T11:33:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 Processor -- start 2023-10-06T11:33:05.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 -- start start 2023-10-06T11:33:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa4c067760 0x7ffa4c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0af190 con 0x7ffa4c0bedd0 2023-10-06T11:33:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0af470 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa52fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0af750 con 0x7ffa4c067760 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa517c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa527cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa517c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43030/0 (socket says 172.21.15.46:43030) 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa517c9700 1 -- 172.21.15.46:0/1315764241 learned_addr learned my addr 172.21.15.46:0/1315764241 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.296+0000 7ffa51fca700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa4c067760 0x7ffa4c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa4c067760 msgr2=0x7ffa4c0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa4c067760 0x7ffa4c0ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 msgr2=0x7ffa4c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 -- 172.21.15.46:0/1315764241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa44009a20 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa517c9700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa527cb700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c1160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa44016000 tx=0x7ffa44019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44012dd0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa44027470 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44027870 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0afa30 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c0bb2f0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.300+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c064590 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.304+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7ffa44024070 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.304+0000 7ffa41ffb700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 0x7ffa30062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.304+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6904+0+0 (secure 0 0 0) 0x7ffa4407f7f0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.304+0000 7ffa51fca700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 msgr2=0x7ffa30062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.304+0000 7ffa51fca700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 0x7ffa30062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:05.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.312+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffa4404f780 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.337 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:05.324+0000 7f90415c3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:05.393 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.387+0000 7f9122f5a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:05.504 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:05.492+0000 7f90415c3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.504+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c04b5c0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.508+0000 7ffa51fca700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 msgr2=0x7ffa30062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:05.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.508+0000 7ffa51fca700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 0x7ffa30062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:05.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.508+0000 7ffa41ffb700 1 -- 172.21.15.46:0/1315764241 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18233 (secure 0 0 0) 0x7ffa44052da0 con 0x7ffa4c0bf1a0 2023-10-06T11:33:05.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:05.520 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:05.460548+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1691388410":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3927254994":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/2878818767":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6833/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3489019109":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6832/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1702922641":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1135435385":"2023-10-07T11:31:19.985120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:05.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 msgr2=0x7ffa30062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 0x7ffa30062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 msgr2=0x7ffa4c0c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c1160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa44016000 tx=0x7ffa44019000 comp rx=0 tx=0).stop 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 shutdown_connections 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa30060670 0x7ffa30062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa4c0bf1a0 0x7ffa4c0c1160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa4c0bedd0 0x7ffa4c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 --2- 172.21.15.46:0/1315764241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa4c067760 0x7ffa4c0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 >> 172.21.15.46:0/1315764241 conn(0x7ffa4c062ef0 msgr2=0x7ffa4c0cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 shutdown_connections 2023-10-06T11:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.512+0000 7ffa52fcc700 1 -- 172.21.15.46:0/1315764241 wait complete. 2023-10-06T11:33:05.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:31:03.320790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:31:04.384281+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:33:05.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:33:05.648 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.643+0000 7f9122f5a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:05.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 Processor -- start 2023-10-06T11:33:05.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 -- start start 2023-10-06T11:33:05.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c154880 0x7f6a5c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6a5c0c7f20 0x7f6a5c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0c6cf0 con 0x7f6a5c154c50 2023-10-06T11:33:05.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0c2a30 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.692+0000 7f6a5b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0c83e0 con 0x7f6a5c154880 2023-10-06T11:33:05.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a59d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a5c154c50 0x7f6a5c06c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54636/0 (socket says 172.21.15.46:54636) 2023-10-06T11:33:05.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a59d9b700 1 -- 172.21.15.46:0/3081399086 learned_addr learned my addr 172.21.15.46:0/3081399086 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 --2- 172.21.15.46:0/3081399086 >> v2:172.21.15.132:3300/0 conn(0x7f6a5c0c7f20 0x7f6a5c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5a59c700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c154880 0x7f6a5c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 -- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c154880 msgr2=0x7f6a5c0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c154880 0x7f6a5c0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 -- 172.21.15.46:0/3081399086 >> v1:172.21.15.46:6789/0 conn(0x7f6a5c154c50 legacy=0x7f6a5c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 -- 172.21.15.46:0/3081399086 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c06e5d0 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5959a700 1 -- 172.21.15.46:0/3081399086 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238933673 0 0) 0x7f6a5c0c6cf0 con 0x7f6a5c154c50 2023-10-06T11:33:05.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5ad9d700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c0c7f20 0x7f6a5c15aa30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e60 tx=0x7f6a54009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c1000ea2d4ca00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:05.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5959a700 1 -- 172.21.15.46:0/3081399086 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400e070 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5959a700 1 -- 172.21.15.46:0/3081399086 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400bb60 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5959a700 1 -- 172.21.15.46:0/3081399086 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54012430 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c0c7f20 msgr2=0x7f6a5c15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c0c7f20 0x7f6a5c15aa30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e60 tx=0x7f6a54009780 comp rx=0 tx=0).stop 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3081399086 shutdown_connections 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c0c7f20 0x7f6a5c15aa30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3081399086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c154880 0x7f6a5c0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3081399086 >> 172.21.15.46:0/3081399086 conn(0x7f6a5c0cbc10 msgr2=0x7f6a5c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:05.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3081399086 shutdown_connections 2023-10-06T11:33:05.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.696+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3081399086 wait complete. 2023-10-06T11:33:05.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.700+0000 7f6a5b59e700 1 Processor -- start 2023-10-06T11:33:05.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.700+0000 7f6a5b59e700 1 -- start start 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c0c7f20 0x7f6a5c0c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c154c50 0x7f6a5c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c151470 con 0x7f6a5c154880 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c150e40 con 0x7f6a5c154c50 2023-10-06T11:33:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0c5890 con 0x7f6a5c0c7f20 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43042/0 (socket says 172.21.15.46:43042) 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 -- 172.21.15.46:0/3601974878 learned_addr learned my addr 172.21.15.46:0/3601974878 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5a59c700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c0c7f20 0x7f6a5c0c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5ad9d700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c154c50 0x7f6a5c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 -- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c0c7f20 msgr2=0x7f6a5c0c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c0c7f20 0x7f6a5c0c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 -- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c154c50 msgr2=0x7f6a5c159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c154c50 0x7f6a5c159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 -- 172.21.15.46:0/3601974878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009a20 con 0x7f6a5c154880 2023-10-06T11:33:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a59d9b700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c5e0 tx=0x7f6a50000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50005b30 con 0x7f6a5c154880 2023-10-06T11:33:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50005c90 con 0x7f6a5c154880 2023-10-06T11:33:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50007300 con 0x7f6a5c154880 2023-10-06T11:33:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c0c5b70 con 0x7f6a5c154880 2023-10-06T11:33:05.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.704+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c157eb0 con 0x7f6a5c154880 2023-10-06T11:33:05.719 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.715+0000 7f9122f5a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:05.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.708+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c1515b0 con 0x7f6a5c154880 2023-10-06T11:33:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.708+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f6a5001b3b0 con 0x7f6a5c154880 2023-10-06T11:33:05.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.708+0000 7f6a4a7fc700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 0x7f6a3c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:05.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.708+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f6a5007c2a0 con 0x7f6a5c154880 2023-10-06T11:33:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.712+0000 7f6a5a59c700 1 -- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 msgr2=0x7f6a3c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.712+0000 7f6a5a59c700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 0x7f6a3c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.716+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6a5c1515b0 con 0x7f6a5c154880 2023-10-06T11:33:05.798 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:05.784+0000 7f90415c3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:05.876 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:05.864+0000 7f90415c3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:05.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.884+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a5c1515b0 con 0x7f6a5c154880 2023-10-06T11:33:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.884+0000 7f6a4a7fc700 1 -- 172.21.15.46:0/3601974878 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a5c1515b0 con 0x7f6a5c154880 2023-10-06T11:33:05.897 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 msgr2=0x7f6a3c062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 0x7f6a3c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 msgr2=0x7f6a5c159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c5e0 tx=0x7f6a50000fe0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 shutdown_connections 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c060670 0x7f6a3c062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5c154c50 0x7f6a5c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a5c154880 0x7f6a5c159920 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 --2- 172.21.15.46:0/3601974878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5c0c7f20 0x7f6a5c0c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 >> 172.21.15.46:0/3601974878 conn(0x7f6a5c0cbc10 msgr2=0x7f6a5c0af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 shutdown_connections 2023-10-06T11:33:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:05.888+0000 7f6a5b59e700 1 -- 172.21.15.46:0/3601974878 wait complete. 2023-10-06T11:33:05.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:33:05.921 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:05.915+0000 7f9122f5a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:06.007 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:06.003+0000 7f9122f5a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:06.089 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:06.083+0000 7f9122f5a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:06.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.076+0000 7fc362f63700 1 Processor -- start 2023-10-06T11:33:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 -- start start 2023-10-06T11:33:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 0x7fc35c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc35c159850 0x7fc35c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c0ce6e0 con 0x7fc35c158aa0 2023-10-06T11:33:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c15a5c0 con 0x7fc35c069270 2023-10-06T11:33:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.080+0000 7fc362f63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c0cddb0 con 0x7fc35c159850 2023-10-06T11:33:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc361f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 0x7fc35c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc361f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 0x7fc35c069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43054/0 (socket says 172.21.15.46:43054) 2023-10-06T11:33:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc361f61700 1 -- 172.21.15.46:0/3497579997 learned_addr learned my addr 172.21.15.46:0/3497579997 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 --2- 172.21.15.46:0/3497579997 >> v2:172.21.15.132:3300/0 conn(0x7fc35c159850 0x7fc35c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 -- 172.21.15.46:0/3497579997 >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 msgr2=0x7fc35c069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 --2- 172.21.15.46:0/3497579997 >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 0x7fc35c069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 -- 172.21.15.46:0/3497579997 >> v1:172.21.15.132:6789/0 conn(0x7fc35c158aa0 legacy=0x7fc35c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 -- 172.21.15.46:0/3497579997 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0ce090 con 0x7fc35c159850 2023-10-06T11:33:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362762700 1 --2- 172.21.15.46:0/3497579997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c159850 0x7fc35c159cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc35c067710 tx=0x7fc354009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b847adb6f29fa61d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc360f5f700 1 -- 172.21.15.46:0/3497579997 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400e070 con 0x7fc35c159850 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc360f5f700 1 -- 172.21.15.46:0/3497579997 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35400bb60 con 0x7fc35c159850 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc360f5f700 1 -- 172.21.15.46:0/3497579997 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354012430 con 0x7fc35c159850 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 -- 172.21.15.46:0/3497579997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c159850 msgr2=0x7fc35c159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 --2- 172.21.15.46:0/3497579997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c159850 0x7fc35c159cd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc35c067710 tx=0x7fc354009780 comp rx=0 tx=0).stop 2023-10-06T11:33:06.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 -- 172.21.15.46:0/3497579997 shutdown_connections 2023-10-06T11:33:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 --2- 172.21.15.46:0/3497579997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c159850 0x7fc35c159cd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 --2- 172.21.15.46:0/3497579997 >> v2:172.21.15.46:3300/0 conn(0x7fc35c069270 0x7fc35c069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 -- 172.21.15.46:0/3497579997 >> 172.21.15.46:0/3497579997 conn(0x7fc35c068390 msgr2=0x7fc35c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 -- 172.21.15.46:0/3497579997 shutdown_connections 2023-10-06T11:33:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.084+0000 7fc362f63700 1 -- 172.21.15.46:0/3497579997 wait complete. 2023-10-06T11:33:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 Processor -- start 2023-10-06T11:33:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 -- start start 2023-10-06T11:33:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 0x7fc35c14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c158aa0 0x7fc35c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 0x7fc35c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0afd50 con 0x7fc35c159850 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0b30f0 con 0x7fc35c069270 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362f63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0b33d0 con 0x7fc35c158aa0 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362762700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 0x7fc35c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362762700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 0x7fc35c1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43064/0 (socket says 172.21.15.46:43064) 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc362762700 1 -- 172.21.15.46:0/2677592335 learned_addr learned my addr 172.21.15.46:0/2677592335 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc361f61700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 0x7fc35c14ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.088+0000 7fc361760700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c158aa0 0x7fc35c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 -- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c158aa0 msgr2=0x7fc35c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c158aa0 0x7fc35c0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 -- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 msgr2=0x7fc35c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 0x7fc35c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 -- 172.21.15.46:0/2677592335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354009a20 con 0x7fc35c069270 2023-10-06T11:33:06.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc361f61700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 0x7fc35c14ba70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc358006d40 tx=0x7fc358009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358013680 con 0x7fc35c069270 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3580137e0 con 0x7fc35c069270 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358013b60 con 0x7fc35c069270 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c0b36b0 con 0x7fc35c069270 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c0b2de0 con 0x7fc35c069270 2023-10-06T11:33:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.092+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c064590 con 0x7fc35c069270 2023-10-06T11:33:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.096+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fc35800c3e0 con 0x7fc35c069270 2023-10-06T11:33:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.096+0000 7fc351ffb700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 0x7fc340062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.096+0000 7fc361760700 1 -- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 msgr2=0x7fc340062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.096+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fc35807cf20 con 0x7fc35c069270 2023-10-06T11:33:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.096+0000 7fc361760700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 0x7fc340062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:06.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:06.100+0000 7f90415c3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.104+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc35804cea0 con 0x7fc35c069270 2023-10-06T11:33:06.204 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:06.192+0000 7f90415c3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:06.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.272+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc35c0ce370 con 0x7fc35c069270 2023-10-06T11:33:06.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.272+0000 7fc351ffb700 1 -- 172.21.15.46:0/2677592335 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc3580210a0 con 0x7fc35c069270 2023-10-06T11:33:06.286 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:33:06.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 msgr2=0x7fc340062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:06.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 0x7fc340062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 msgr2=0x7fc35c14ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 0x7fc35c14ba70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc358006d40 tx=0x7fc358009250 comp rx=0 tx=0).stop 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 shutdown_connections 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc340060670 0x7fc340062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc35c159850 0x7fc35c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c158aa0 0x7fc35c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 --2- 172.21.15.46:0/2677592335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c069270 0x7fc35c14ba70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 >> 172.21.15.46:0/2677592335 conn(0x7fc35c068390 msgr2=0x7fc35c0c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:06.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 shutdown_connections 2023-10-06T11:33:06.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.280+0000 7fc362f63700 1 -- 172.21.15.46:0/2677592335 wait complete. 2023-10-06T11:33:06.297 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:06.284+0000 7f90415c3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:06.312 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:33:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.468+0000 7f259c923700 1 Processor -- start 2023-10-06T11:33:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.468+0000 7f259c923700 1 -- start start 2023-10-06T11:33:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f259c923700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2598068290 0x7f2598068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f259c923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f259c923700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2598159900 con 0x7f25980abd80 2023-10-06T11:33:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f259c923700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25981596f0 con 0x7f2598068290 2023-10-06T11:33:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f259c923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598159be0 con 0x7f25980ac200 2023-10-06T11:33:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2598068290 0x7f2598068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2598068290 0x7f2598068660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54472/0 (socket says 172.21.15.46:54472) 2023-10-06T11:33:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f2596ffd700 1 --1- 172.21.15.46:0/1264147599 >> v1:172.21.15.132:6789/0 conn(0x7f25980abd80 0x7f25980acba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:57474/0 (socket says 172.21.15.46:57474) 2023-10-06T11:33:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 -- 172.21.15.46:0/1264147599 learned_addr learned my addr 172.21.15.46:0/1264147599 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f2597fff700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 -- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 msgr2=0x7f25980a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 -- 172.21.15.46:0/1264147599 >> v1:172.21.15.132:6789/0 conn(0x7f25980abd80 legacy=0x7f25980acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 -- 172.21.15.46:0/1264147599 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598159ec0 con 0x7f2598068290 2023-10-06T11:33:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.472+0000 7f25977fe700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f2598068660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2588009990 tx=0x7f2588009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481548f6ba7e8b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f25967fc700 1 -- 172.21.15.46:0/1264147599 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588009e80 con 0x7f2598068290 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f25967fc700 1 -- 172.21.15.46:0/1264147599 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2588005c10 con 0x7f2598068290 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f25967fc700 1 -- 172.21.15.46:0/1264147599 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258800b650 con 0x7f2598068290 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 -- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 msgr2=0x7f2598068660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f2598068660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2588009990 tx=0x7f2588009220 comp rx=0 tx=0).stop 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 -- 172.21.15.46:0/1264147599 shutdown_connections 2023-10-06T11:33:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 --2- 172.21.15.46:0/1264147599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f2598068660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 -- 172.21.15.46:0/1264147599 >> 172.21.15.46:0/1264147599 conn(0x7f259806a880 msgr2=0x7f259806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 -- 172.21.15.46:0/1264147599 shutdown_connections 2023-10-06T11:33:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.476+0000 7f259c923700 1 -- 172.21.15.46:0/1264147599 wait complete. 2023-10-06T11:33:06.490 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:06.483+0000 7f9122f5a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 Processor -- start 2023-10-06T11:33:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 -- start start 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f259816a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259816a6a0 con 0x7f25980abd80 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259814d2e0 con 0x7f2598068290 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f259c923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259816ae70 con 0x7f25980ac200 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f25977fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f259816a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43066/0 (socket says 172.21.15.46:43066) 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 -- 172.21.15.46:0/2718710339 learned_addr learned my addr 172.21.15.46:0/2718710339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f25977fe700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f259816a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54474/0 (socket says 172.21.15.46:54474) 2023-10-06T11:33:06.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2597fff700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 -- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 msgr2=0x7f25980b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 -- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 msgr2=0x7f259816a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f259816a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.480+0000 7f2596ffd700 1 -- 172.21.15.46:0/2718710339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009480 con 0x7f25980abd80 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f2596ffd700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f258400c7c0 tx=0x7f2584007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584005a70 con 0x7f25980abd80 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584005bd0 con 0x7f25980abd80 2023-10-06T11:33:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840070d0 con 0x7f25980abd80 2023-10-06T11:33:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259816a840 con 0x7f25980abd80 2023-10-06T11:33:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980afe20 con 0x7f25980abd80 2023-10-06T11:33:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598064590 con 0x7f25980abd80 2023-10-06T11:33:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.484+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f2584007480 con 0x7f25980abd80 2023-10-06T11:33:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.488+0000 7f258f7fe700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 0x7f2580062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.488+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f258407c2a0 con 0x7f25980abd80 2023-10-06T11:33:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.488+0000 7f25977fe700 1 -- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 msgr2=0x7f2580062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.488+0000 7f25977fe700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 0x7f2580062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.492+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f258404c220 con 0x7f25980abd80 2023-10-06T11:33:06.656 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:06.651+0000 7f9122f5a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.664+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f259815a1a0 con 0x7f25980abd80 2023-10-06T11:33:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.664+0000 7f258f7fe700 1 -- 172.21.15.46:0/2718710339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f258404f840 con 0x7f25980abd80 2023-10-06T11:33:06.679 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:33:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.668+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 msgr2=0x7f2580062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.668+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 0x7f2580062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.668+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 msgr2=0x7f259814cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.668+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f258400c7c0 tx=0x7f2584007f70 comp rx=0 tx=0).stop 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 shutdown_connections 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2580060670 0x7f2580062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25980ac200 0x7f25980b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25980abd80 0x7f259814cda0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 --2- 172.21.15.46:0/2718710339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2598068290 0x7f259816a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 >> 172.21.15.46:0/2718710339 conn(0x7f259806a880 msgr2=0x7f25980ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 shutdown_connections 2023-10-06T11:33:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.672+0000 7f259c923700 1 -- 172.21.15.46:0/2718710339 wait complete. 2023-10-06T11:33:06.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:33:06.727 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:06.712+0000 7f90415c3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.852+0000 7fe1e726e700 1 Processor -- start 2023-10-06T11:33:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.852+0000 7fe1e726e700 1 -- start start 2023-10-06T11:33:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e726e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e006e8a0 0x7fe1e00c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e726e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1e006ed10 0x7fe1e00c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e726e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e00c1bf0 con 0x7fe1e00c6470 2023-10-06T11:33:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e726e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e00c72c0 con 0x7fe1e006ed10 2023-10-06T11:33:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e726e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e006cf80 con 0x7fe1e006e8a0 2023-10-06T11:33:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e500a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe1e00c6470 0x7fe1e00ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54648/0 (socket says 172.21.15.46:54648) 2023-10-06T11:33:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e500a700 1 -- 172.21.15.46:0/1436785869 learned_addr learned my addr 172.21.15.46:0/1436785869 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 --2- 172.21.15.46:0/1436785869 >> v2:172.21.15.46:3300/0 conn(0x7fe1e006ed10 0x7fe1e00c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e4809700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e006e8a0 0x7fe1e00c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1d7fff700 1 -- 172.21.15.46:0/1436785869 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098703662 0 0) 0x7fe1e00c1bf0 con 0x7fe1e00c6470 2023-10-06T11:33:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 -- 172.21.15.46:0/1436785869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e006e8a0 msgr2=0x7fe1e00c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e006e8a0 0x7fe1e00c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 -- 172.21.15.46:0/1436785869 >> v1:172.21.15.46:6789/0 conn(0x7fe1e00c6470 legacy=0x7fe1e00ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 -- 172.21.15.46:0/1436785869 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e006d260 con 0x7fe1e006ed10 2023-10-06T11:33:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1e580b700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006ed10 0x7fe1e00c6d80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009e60 tx=0x7fe1d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e518e0b1f7fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1d7fff700 1 -- 172.21.15.46:0/1436785869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800e070 con 0x7fe1e006ed10 2023-10-06T11:33:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1d7fff700 1 -- 172.21.15.46:0/1436785869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d800bb60 con 0x7fe1e006ed10 2023-10-06T11:33:06.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.856+0000 7fe1d7fff700 1 -- 172.21.15.46:0/1436785869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8012430 con 0x7fe1e006ed10 2023-10-06T11:33:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- 172.21.15.46:0/1436785869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006ed10 msgr2=0x7fe1e00c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006ed10 0x7fe1e00c6d80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009e60 tx=0x7fe1d8009780 comp rx=0 tx=0).stop 2023-10-06T11:33:06.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- 172.21.15.46:0/1436785869 shutdown_connections 2023-10-06T11:33:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006ed10 0x7fe1e00c6d80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1436785869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e006e8a0 0x7fe1e00c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- 172.21.15.46:0/1436785869 >> 172.21.15.46:0/1436785869 conn(0x7fe1e00cbe30 msgr2=0x7fe1e00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- 172.21.15.46:0/1436785869 shutdown_connections 2023-10-06T11:33:06.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- 172.21.15.46:0/1436785869 wait complete. 2023-10-06T11:33:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 Processor -- start 2023-10-06T11:33:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.860+0000 7fe1e726e700 1 -- start start 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e006ed10 0x7fe1e00b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e00c6470 0x7fe1e015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e015bac0 con 0x7fe1e006e8a0 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e016cb00 con 0x7fe1e006ed10 2023-10-06T11:33:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e016c7d0 con 0x7fe1e00c6470 2023-10-06T11:33:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43074/0 (socket says 172.21.15.46:43074) 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 -- 172.21.15.46:0/1916717844 learned_addr learned my addr 172.21.15.46:0/1916717844 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e580b700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e00c6470 0x7fe1e015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e4809700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e006ed10 0x7fe1e00b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 -- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e00c6470 msgr2=0x7fe1e015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e00c6470 0x7fe1e015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 -- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e006ed10 msgr2=0x7fe1e00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e006ed10 0x7fe1e00b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 -- 172.21.15.46:0/1916717844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009a20 con 0x7fe1e006e8a0 2023-10-06T11:33:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e500a700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00c810 tx=0x7fe1dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc005b30 con 0x7fe1e006e8a0 2023-10-06T11:33:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc005c90 con 0x7fe1e006e8a0 2023-10-06T11:33:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e00b1400 con 0x7fe1e006e8a0 2023-10-06T11:33:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc007300 con 0x7fe1e006e8a0 2023-10-06T11:33:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.864+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e00b19c0 con 0x7fe1e006e8a0 2023-10-06T11:33:06.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.868+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e013d030 con 0x7fe1e006e8a0 2023-10-06T11:33:06.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.868+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fe1dc01b3b0 con 0x7fe1e006e8a0 2023-10-06T11:33:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.868+0000 7fe1d57fa700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 0x7fe1c4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.872+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fe1dc07c360 con 0x7fe1e006e8a0 2023-10-06T11:33:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.872+0000 7fe1e4809700 1 -- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 msgr2=0x7fe1c4062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.872+0000 7fe1e4809700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 0x7fe1c4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:06.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:06.876+0000 7f90415c3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:06.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:06.876+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe1dc04c2e0 con 0x7fe1e006e8a0 2023-10-06T11:33:07.024 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.019+0000 7f9122f5a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.052+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1e00660f0 con 0x7fe1e006e8a0 2023-10-06T11:33:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.052+0000 7fe1d57fa700 1 -- 172.21.15.46:0/1916717844 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe1dc001d60 con 0x7fe1e006e8a0 2023-10-06T11:33:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:33:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 msgr2=0x7fe1c4062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 0x7fe1c4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 msgr2=0x7fe1e015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00c810 tx=0x7fe1dc007df0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 shutdown_connections 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe1c4060670 0x7fe1c4062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1e00c6470 0x7fe1e015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1e006ed10 0x7fe1e00b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 --2- 172.21.15.46:0/1916717844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e006e8a0 0x7fe1e015bc20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 >> 172.21.15.46:0/1916717844 conn(0x7fe1e00cbe30 msgr2=0x7fe1e00ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 shutdown_connections 2023-10-06T11:33:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.056+0000 7fe1e726e700 1 -- 172.21.15.46:0/1916717844 wait complete. 2023-10-06T11:33:07.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:07.236 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.231+0000 7f9122f5a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:07.280 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.268+0000 7f90415c3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:07.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.288+0000 7fc642523700 1 Processor -- start 2023-10-06T11:33:07.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 -- start start 2023-10-06T11:33:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc63c0c3160 0x7fc63c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0ccfa0 con 0x7fc63c0cb4f0 2023-10-06T11:33:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63c0cd970 con 0x7fc63c0c3160 2023-10-06T11:33:07.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc642523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0cc390 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc641d22700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc63c0cb4f0 0x7fc63c0c84c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54652/0 (socket says 172.21.15.46:54652) 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc641d22700 1 -- 172.21.15.46:0/3489164663 learned_addr learned my addr 172.21.15.46:0/3489164663 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc640d20700 1 --2- 172.21.15.46:0/3489164663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc641521700 1 --2- 172.21.15.46:0/3489164663 >> v2:172.21.15.132:3300/0 conn(0x7fc63c0c3160 0x7fc63c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986401372 0 0) 0x7fc63c0ccfa0 con 0x7fc63c0cb4f0 2023-10-06T11:33:07.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc628003570 con 0x7fc63c0cb4f0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc640d20700 1 -- 172.21.15.46:0/3489164663 >> v2:172.21.15.132:3300/0 conn(0x7fc63c0c3160 msgr2=0x7fc63c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc640d20700 1 --2- 172.21.15.46:0/3489164663 >> v2:172.21.15.132:3300/0 conn(0x7fc63c0c3160 0x7fc63c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc640d20700 1 -- 172.21.15.46:0/3489164663 >> v1:172.21.15.46:6789/0 conn(0x7fc63c0cb4f0 legacy=0x7fc63c0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641712895 0 0) 0x7fc628003570 con 0x7fc63c0cb4f0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.292+0000 7fc640d20700 1 -- 172.21.15.46:0/3489164663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0cc670 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc640d20700 1 --2- 172.21.15.46:0/3489164663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0c3530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0098e0 tx=0x7fc62c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984f041475f01333 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340017f0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c011420 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc633fff700 1 -- 172.21.15.46:0/3489164663 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0117c0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 -- 172.21.15.46:0/3489164663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 msgr2=0x7fc63c0c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 --2- 172.21.15.46:0/3489164663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0c3530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0098e0 tx=0x7fc62c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 -- 172.21.15.46:0/3489164663 shutdown_connections 2023-10-06T11:33:07.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 --2- 172.21.15.46:0/3489164663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0c3530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 --2- 172.21.15.46:0/3489164663 >> v2:172.21.15.132:3300/0 conn(0x7fc63c0c3160 0x7fc63c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 -- 172.21.15.46:0/3489164663 >> 172.21.15.46:0/3489164663 conn(0x7fc63c0970c0 msgr2=0x7fc63c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:07.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 -- 172.21.15.46:0/3489164663 shutdown_connections 2023-10-06T11:33:07.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.296+0000 7fc642523700 1 -- 172.21.15.46:0/3489164663 wait complete. 2023-10-06T11:33:07.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 Processor -- start 2023-10-06T11:33:07.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 -- start start 2023-10-06T11:33:07.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c0c3160 0x7fc63c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63c0cb4f0 0x7fc63c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c156330 con 0x7fc63c0cb4f0 2023-10-06T11:33:07.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0b34a0 con 0x7fc63c0c3160 2023-10-06T11:33:07.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc642523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0b3940 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc641d22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63c0cb4f0 0x7fc63c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc641521700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c0c3160 0x7fc63c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:42472/0 (socket says 172.21.15.46:42472) 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 -- 172.21.15.46:0/93543747 learned_addr learned my addr 172.21.15.46:0/93543747 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c0c3160 msgr2=0x7fc63c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c0c3160 0x7fc63c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63c0cb4f0 msgr2=0x7fc63c0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63c0cb4f0 0x7fc63c0b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 -- 172.21.15.46:0/93543747 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c009420 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc640d20700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0030c0 tx=0x7fc62c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.300+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c01cb30 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c01cc90 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c153e10 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c01a6a0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c155ce0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c064590 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.304+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fc62c01aad0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.308+0000 7fc6317fa700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 0x7fc620062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.308+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fc62c07e770 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.308+0000 7fc641521700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 msgr2=0x7fc620062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:07.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.308+0000 7fc641521700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 0x7fc620062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:07.323 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.319+0000 7f9122f5a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.312+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc62c04e6f0 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.395 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.391+0000 7f9122f5a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:07.468 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.463+0000 7f9122f5a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:07.510 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.496+0000 7f90415c3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.512+0000 7fc641521700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 msgr2=0x7fc620062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.512+0000 7fc641521700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 0x7fc620062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.520+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc63c0cc950 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.524+0000 7fc6317fa700 1 -- 172.21.15.46:0/93543747 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fc62c051d10 con 0x7fc63c0cb0a0 2023-10-06T11:33:07.536 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:07.536 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:33:07.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 msgr2=0x7fc620062b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:07.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 0x7fc620062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 msgr2=0x7fc63c0b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0030c0 tx=0x7fc62c016000 comp rx=0 tx=0).stop 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 shutdown_connections 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6200606a0 0x7fc620062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63c0cb4f0 0x7fc63c0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc63c0cb0a0 0x7fc63c0b2a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 --2- 172.21.15.46:0/93543747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63c0c3160 0x7fc63c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 >> 172.21.15.46:0/93543747 conn(0x7fc63c0970c0 msgr2=0x7fc63c06ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 shutdown_connections 2023-10-06T11:33:07.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.528+0000 7fc642523700 1 -- 172.21.15.46:0/93543747 wait complete. 2023-10-06T11:33:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:33:07.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:07.611 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.600+0000 7f90415c3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:07.681 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.675+0000 7f9122f5a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:07.688 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.676+0000 7f90415c3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:07.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.708+0000 7f1eb6d85700 1 Processor -- start 2023-10-06T11:33:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb6d85700 1 -- start start 2023-10-06T11:33:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb6d85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1eb00cc2d0 0x7f1eb00ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb6d85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00abe20 con 0x7f1eb00af0b0 2023-10-06T11:33:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb6d85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb015a780 con 0x7f1eb0067050 2023-10-06T11:33:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb6d85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb00ac0c0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eaffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1eb00af0b0 0x7f1eb015a070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54664/0 (socket says 172.21.15.46:54664) 2023-10-06T11:33:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eaffff700 1 -- 172.21.15.46:0/3472892566 learned_addr learned my addr 172.21.15.46:0/3472892566 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb4b21700 1 --2- 172.21.15.46:0/3472892566 >> v2:172.21.15.46:3300/0 conn(0x7f1eb00cc2d0 0x7f1eb00ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605456395 0 0) 0x7f1eb00abe20 con 0x7f1eb00af0b0 2023-10-06T11:33:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea0003570 con 0x7f1eb00af0b0 2023-10-06T11:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb4b21700 1 -- 172.21.15.46:0/3472892566 >> v1:172.21.15.132:6789/0 conn(0x7f1eb0067050 legacy=0x7f1eb0067420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893621665 0 0) 0x7f1eb015a780 con 0x7f1eb0067050 2023-10-06T11:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb4b21700 1 -- 172.21.15.46:0/3472892566 >> v1:172.21.15.46:6789/0 conn(0x7f1eb00af0b0 legacy=0x7f1eb015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.712+0000 7f1eb4b21700 1 -- 172.21.15.46:0/3472892566 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00ac3a0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb4b21700 1 --2- 172.21.15.46:0/3472892566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb00ab8e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1eb015a970 tx=0x7f1ea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b49317b578dd7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c001470 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea4011420 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eaf7fe700 1 -- 172.21.15.46:0/3472892566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40117c0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 -- 172.21.15.46:0/3472892566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 msgr2=0x7f1eb00ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/3472892566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb00ab8e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1eb015a970 tx=0x7f1ea40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 -- 172.21.15.46:0/3472892566 shutdown_connections 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/3472892566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb00ab8e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 -- 172.21.15.46:0/3472892566 >> 172.21.15.46:0/3472892566 conn(0x7f1eb0068bf0 msgr2=0x7f1eb00ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 -- 172.21.15.46:0/3472892566 shutdown_connections 2023-10-06T11:33:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 -- 172.21.15.46:0/3472892566 wait complete. 2023-10-06T11:33:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.716+0000 7f1eb6d85700 1 Processor -- start 2023-10-06T11:33:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- start start 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eb0067050 0x7f1eb00c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb00af0b0 0x7f1eb00cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00c2dc0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00c3250 con 0x7f1eb00af0b0 2023-10-06T11:33:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00c21b0 con 0x7f1eb0067050 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43108/0 (socket says 172.21.15.46:43108) 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 -- 172.21.15.46:0/507500044 learned_addr learned my addr 172.21.15.46:0/507500044 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eaffff700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb00af0b0 0x7f1eb00cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb4b21700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eb0067050 0x7f1eb00c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 -- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eb0067050 msgr2=0x7f1eb00c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eb0067050 0x7f1eb00c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 -- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb00af0b0 msgr2=0x7f1eb00cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb00af0b0 0x7f1eb00cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 -- 172.21.15.46:0/507500044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4009420 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb5322700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001560 tx=0x7f1ea8001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8004160 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea800a7e0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb00c2490 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800aba0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.720+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb00c2a50 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.724+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb00c2f70 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.724+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f1ea8012420 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.728+0000 7f1eacff9700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 0x7f1e94062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.728+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f1ea807cc70 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.728+0000 7f1eb4b21700 1 -- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 msgr2=0x7f1e94062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.728+0000 7f1eb4b21700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 0x7f1e94062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.732+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1eb00c2f70 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.758 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.755+0000 7f9122f5a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:07.775 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.764+0000 7f90415c3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:07.843 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:07.839+0000 7f9122f5a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.904+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eb00c2f00 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.908+0000 7f1eacff9700 1 -- 172.21.15.46:0/507500044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f1ea804cbf0 con 0x7f1eb00cc2d0 2023-10-06T11:33:07.921 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:07.921 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:05.460548+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1691388410":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3927254994":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/2878818767":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6833/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3489019109":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6832/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1702922641":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1135435385":"2023-10-07T11:31:19.985120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 msgr2=0x7f1e94062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 0x7f1e94062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 msgr2=0x7f1eb016d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001560 tx=0x7f1ea8001770 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 shutdown_connections 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e94060670 0x7f1e94062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb00cc2d0 0x7f1eb016d480 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb00af0b0 0x7f1eb00cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 --2- 172.21.15.46:0/507500044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1eb0067050 0x7f1eb00c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 >> 172.21.15.46:0/507500044 conn(0x7f1eb0068bf0 msgr2=0x7f1eb00ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 shutdown_connections 2023-10-06T11:33:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:07.912+0000 7f1eb6d85700 1 -- 172.21.15.46:0/507500044 wait complete. 2023-10-06T11:33:07.948 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:33:07.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:08.002 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:07.984+0000 7f90415c3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:08.031 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:08.027+0000 7f9122f5a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:08.072 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:08.060+0000 7f90415c3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:08.113 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:08.107+0000 7f9122f5a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.140+0000 7f95b16cd700 1 Processor -- start 2023-10-06T11:33:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.140+0000 7f95b16cd700 1 -- start start 2023-10-06T11:33:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95b16cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95ac153460 0x7f95ac06e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95b16cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac159a90 con 0x7f95ac159f20 2023-10-06T11:33:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95b16cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac159790 con 0x7f95ac153830 2023-10-06T11:33:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95b16cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0aa320 con 0x7f95ac153460 2023-10-06T11:33:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95ab7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95ac159f20 0x7f95ac067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54670/0 (socket says 172.21.15.46:54670) 2023-10-06T11:33:08.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95ab7fe700 1 -- 172.21.15.46:0/575875427 learned_addr learned my addr 172.21.15.46:0/575875427 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:08.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95aaffd700 1 --2- 172.21.15.46:0/575875427 >> v2:172.21.15.132:3300/0 conn(0x7f95ac153460 0x7f95ac06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555026553 0 0) 0x7f95ac159790 con 0x7f95ac153830 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9594003570 con 0x7f95ac153830 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096328768 0 0) 0x7f95ac159a90 con 0x7f95ac159f20 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ac159790 con 0x7f95ac159f20 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496079243 0 0) 0x7f95ac159790 con 0x7f95ac159f20 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 >> v2:172.21.15.132:3300/0 conn(0x7f95ac153460 msgr2=0x7f95ac06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 --2- 172.21.15.46:0/575875427 >> v2:172.21.15.132:3300/0 conn(0x7f95ac153460 0x7f95ac06e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 >> v1:172.21.15.132:6789/0 conn(0x7f95ac153830 legacy=0x7f95ac0ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0abf80 con 0x7f95ac159f20 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f95a4001770 con 0x7f95ac159f20 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 >> v1:172.21.15.46:6789/0 conn(0x7f95ac159f20 legacy=0x7f95ac067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9594004c50 0x7f9594005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940037b0 0x7f9594003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 0x7f959400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0aa320 con 0x7f95940037b0 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959400c160 con 0x7f9594004160 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959400c320 con 0x7f9594004c50 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95ab7fe700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 0x7f959400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95aa7fc700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940037b0 0x7f9594003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.144+0000 7f95aaffd700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9594004c50 0x7f9594005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 -- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9594004c50 msgr2=0x7f9594005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9594004c50 0x7f9594005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 -- 172.21.15.46:0/575875427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940037b0 msgr2=0x7f9594003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940037b0 0x7f9594003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 -- 172.21.15.46:0/575875427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac159790 con 0x7f9594004160 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95aa7fc700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95940037b0 0x7f9594003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 -- 172.21.15.46:0/575875427 reap_dead start 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95ab7fe700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 0x7f959400ba60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95a400a810 tx=0x7f95a40020d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afbf7981c89ad73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40070f0 con 0x7f9594004160 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4007290 con 0x7f9594004160 2023-10-06T11:33:08.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95a9ffb700 1 -- 172.21.15.46:0/575875427 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4007650 con 0x7f9594004160 2023-10-06T11:33:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 -- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 msgr2=0x7f959400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 0x7f959400ba60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95a400a810 tx=0x7f95a40020d0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 -- 172.21.15.46:0/575875427 shutdown_connections 2023-10-06T11:33:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 --2- 172.21.15.46:0/575875427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9594004160 0x7f959400ba60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 -- 172.21.15.46:0/575875427 >> 172.21.15.46:0/575875427 conn(0x7f95ac068160 msgr2=0x7f95ac068560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 -- 172.21.15.46:0/575875427 shutdown_connections 2023-10-06T11:33:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 -- 172.21.15.46:0/575875427 wait complete. 2023-10-06T11:33:08.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f95b16cd700 1 Processor -- start 2023-10-06T11:33:08.163 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:08.148+0000 7f90415c3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- start start 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95ac149f90 0x7f95ac1479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95ac0aeaf0 0x7f95ac0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac14f7b0 con 0x7f9594004160 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ad070 con 0x7f95ac0aeaf0 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ad5a0 con 0x7f95ac149f90 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43122/0 (socket says 172.21.15.46:43122) 2023-10-06T11:33:08.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 -- 172.21.15.46:0/2113258837 learned_addr learned my addr 172.21.15.46:0/2113258837 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aa7fc700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95ac149f90 0x7f95ac1479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95ab7fe700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95ac0aeaf0 0x7f95ac0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95ac149f90 msgr2=0x7f95ac1479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95ac149f90 0x7f95ac1479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95ac0aeaf0 msgr2=0x7f95ac0aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95ac0aeaf0 0x7f95ac0aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 -- 172.21.15.46:0/2113258837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac148800 con 0x7f9594004160 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95aaffd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f959c011730 tx=0x7f959c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c015670 con 0x7f9594004160 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0157d0 con 0x7f9594004160 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac147ee0 con 0x7f9594004160 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c015b50 con 0x7f9594004160 2023-10-06T11:33:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.152+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1481b0 con 0x7f9594004160 2023-10-06T11:33:08.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.156+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac064590 con 0x7f9594004160 2023-10-06T11:33:08.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.156+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f959c013410 con 0x7f9594004160 2023-10-06T11:33:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.156+0000 7f9592ffd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 0x7f9598062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.156+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7f959c07d520 con 0x7f9594004160 2023-10-06T11:33:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.160+0000 7f95aa7fc700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 msgr2=0x7f9598062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.160+0000 7f95aa7fc700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 0x7f9598062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:08.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.164+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f959c04d560 con 0x7f9594004160 2023-10-06T11:33:08.366 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:08.352+0000 7f90415c3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.364+0000 7f95aa7fc700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 msgr2=0x7f9598062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.364+0000 7f95aa7fc700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 0x7f9598062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.368+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95ac0ac260 con 0x7f9594004160 2023-10-06T11:33:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.368+0000 7f9592ffd700 1 -- 172.21.15.46:0/2113258837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f959c050b80 con 0x7f9594004160 2023-10-06T11:33:08.383 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:08.383 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 msgr2=0x7f9598062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 0x7f9598062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 msgr2=0x7f95ac14f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f959c011730 tx=0x7f959c008040 comp rx=0 tx=0).stop 2023-10-06T11:33:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 shutdown_connections 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9598060670 0x7f9598062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95ac0aeaf0 0x7f95ac0aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95ac149f90 0x7f95ac1479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 --2- 172.21.15.46:0/2113258837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9594004160 0x7f95ac14f270 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 >> 172.21.15.46:0/2113258837 conn(0x7f95ac068160 msgr2=0x7f95ac0b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 shutdown_connections 2023-10-06T11:33:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.372+0000 7f95b16cd700 1 -- 172.21.15.46:0/2113258837 wait complete. 2023-10-06T11:33:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:33:08.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:08.447 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:08.436+0000 7f90415c3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:08.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 Processor -- start 2023-10-06T11:33:08.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 -- start start 2023-10-06T11:33:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf8158670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdaf80ac7d0 0x7fdaf80acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf80ae3f0 con 0x7fdaf80ab990 2023-10-06T11:33:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf80ad160 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdafd545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf8158390 con 0x7fdaf80b3840 2023-10-06T11:33:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaf77fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdaf80ab990 0x7fdaf80abe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54674/0 (socket says 172.21.15.46:54674) 2023-10-06T11:33:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaf77fe700 1 -- 172.21.15.46:0/2325388467 learned_addr learned my addr 172.21.15.46:0/2325388467 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaeffff700 1 --2- 172.21.15.46:0/2325388467 >> v2:172.21.15.132:3300/0 conn(0x7fdaf80ac7d0 0x7fdaf80acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaf6ffd700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf8158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaf67fc700 1 -- 172.21.15.46:0/2325388467 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771618592 0 0) 0x7fdaf80ae3f0 con 0x7fdaf80ab990 2023-10-06T11:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.552+0000 7fdaf67fc700 1 -- 172.21.15.46:0/2325388467 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae4003570 con 0x7fdaf80ab990 2023-10-06T11:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaeffff700 1 -- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 msgr2=0x7fdaf8158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaeffff700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf8158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaeffff700 1 -- 172.21.15.46:0/2325388467 >> v1:172.21.15.46:6789/0 conn(0x7fdaf80ab990 legacy=0x7fdaf80abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaeffff700 1 -- 172.21.15.46:0/2325388467 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf80ad7e0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaf6ffd700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf8158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaeffff700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80acc20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098e0 tx=0x7fdae00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ba7c02a15d761c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaf67fc700 1 -- 172.21.15.46:0/2325388467 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0011630 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaf67fc700 1 -- 172.21.15.46:0/2325388467 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0011790 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdaf67fc700 1 -- 172.21.15.46:0/2325388467 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0011b30 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 -- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ac7d0 msgr2=0x7fdaf80acc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80acc20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098e0 tx=0x7fdae00091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 -- 172.21.15.46:0/2325388467 shutdown_connections 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80acc20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 --2- 172.21.15.46:0/2325388467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf8158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 -- 172.21.15.46:0/2325388467 >> 172.21.15.46:0/2325388467 conn(0x7fdaf8097100 msgr2=0x7fdaf8067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 -- 172.21.15.46:0/2325388467 shutdown_connections 2023-10-06T11:33:08.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.556+0000 7fdafd545700 1 -- 172.21.15.46:0/2325388467 wait complete. 2023-10-06T11:33:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 Processor -- start 2023-10-06T11:33:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 -- start start 2023-10-06T11:33:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ab990 0x7fdaf8169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf815d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c3160 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c3400 con 0x7fdaf80ab990 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdafd545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf80c36e0 con 0x7fdaf80b3840 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdaeffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdaeffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43138/0 (socket says 172.21.15.46:43138) 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdaeffff700 1 -- 172.21.15.46:0/3024032276 learned_addr learned my addr 172.21.15.46:0/3024032276 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdaf6ffd700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ab990 0x7fdaf8169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.560+0000 7fdaf77fe700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 msgr2=0x7fdaf815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf815d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ab990 msgr2=0x7fdaf8169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ab990 0x7fdaf8169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 -- 172.21.15.46:0/3024032276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80093c0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeffff700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806df80 tx=0x7fdae0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0011900 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0011a60 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0025640 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae0009420 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf8068d40 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.564+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8068ba0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.568+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fdae0025a30 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.568+0000 7fdaeeffd700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 0x7fdad8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.568+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fdae007e5a0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.568+0000 7fdaf6ffd700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 msgr2=0x7fdad8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.568+0000 7fdaf6ffd700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 0x7fdad8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:08.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.576+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdaf8068ba0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.772+0000 7fdaf6ffd700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 msgr2=0x7fdad8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.772+0000 7fdaf6ffd700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 0x7fdad8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:08.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.780+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaf8068ba0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.780+0000 7fdaeeffd700 1 -- 172.21.15.46:0/3024032276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fdaf8068ba0 con 0x7fdaf80ac7d0 2023-10-06T11:33:08.794 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:08.794 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:31:05.390358+0000","modified":"2023-10-06T11:31:08.971806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3641487371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3641487371},{"type":"v1","addr":"172.21.15.46:6836","nonce":3641487371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T11:33:08.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 msgr2=0x7fdad8062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 0x7fdad8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 msgr2=0x7fdaf80c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdaf806df80 tx=0x7fdae0017000 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 shutdown_connections 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdad8060670 0x7fdad8062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaf80b3840 0x7fdaf815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaf80ac7d0 0x7fdaf80c2c20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 --2- 172.21.15.46:0/3024032276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaf80ab990 0x7fdaf8169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:08.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 >> 172.21.15.46:0/3024032276 conn(0x7fdaf8097100 msgr2=0x7fdaf80b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.784+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 shutdown_connections 2023-10-06T11:33:08.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:08.788+0000 7fdafd545700 1 -- 172.21.15.46:0/3024032276 wait complete. 2023-10-06T11:33:08.801 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2023-10-06T11:33:08.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:08.819 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:08.815+0000 7f9122f5a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:09.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.004+0000 7fe8c62f0700 1 Processor -- start 2023-10-06T11:33:09.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.004+0000 7fe8c62f0700 1 -- start start 2023-10-06T11:33:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c62f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c00af0b0 0x7fe8c015a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c62f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c015a5b0 0x7fe8c0067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c62f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c00ac0c0 con 0x7fe8c00cc2d0 2023-10-06T11:33:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c62f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00ac3a0 con 0x7fe8c00af0b0 2023-10-06T11:33:09.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c62f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00ac680 con 0x7fe8c015a5b0 2023-10-06T11:33:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c00af0b0 0x7fe8c015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8c00af0b0 0x7fe8c015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43150/0 (socket says 172.21.15.46:43150) 2023-10-06T11:33:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 -- 172.21.15.46:0/1673359178 learned_addr learned my addr 172.21.15.46:0/1673359178 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:09.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8c488d700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c015a5b0 0x7fe8c0067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 -- 172.21.15.46:0/1673359178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c015a5b0 msgr2=0x7fe8c0067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c015a5b0 0x7fe8c0067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 -- 172.21.15.46:0/1673359178 >> v1:172.21.15.132:6789/0 conn(0x7fe8c00cc2d0 legacy=0x7fe8c00ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8beffd700 1 -- 172.21.15.46:0/1673359178 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369589584 0 0) 0x7fe8c00ac0c0 con 0x7fe8c00cc2d0 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 -- 172.21.15.46:0/1673359178 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c0159b50 con 0x7fe8c00af0b0 2023-10-06T11:33:09.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8bf7fe700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c00af0b0 0x7fe8c015a070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40099b0 tx=0x7fe8b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffd0e3b5d6f4439 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8beffd700 1 -- 172.21.15.46:0/1673359178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4009ea0 con 0x7fe8c00af0b0 2023-10-06T11:33:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8beffd700 1 -- 172.21.15.46:0/1673359178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400b860 con 0x7fe8c00af0b0 2023-10-06T11:33:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.008+0000 7fe8beffd700 1 -- 172.21.15.46:0/1673359178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400bc00 con 0x7fe8c00af0b0 2023-10-06T11:33:09.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1673359178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c00af0b0 msgr2=0x7fe8c015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c00af0b0 0x7fe8c015a070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40099b0 tx=0x7fe8b4009250 comp rx=0 tx=0).stop 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1673359178 shutdown_connections 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c015a5b0 0x7fe8c0067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1673359178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c00af0b0 0x7fe8c015a070 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1673359178 >> 172.21.15.46:0/1673359178 conn(0x7fe8c0068bf0 msgr2=0x7fe8c00ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:09.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1673359178 shutdown_connections 2023-10-06T11:33:09.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1673359178 wait complete. 2023-10-06T11:33:09.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.012+0000 7fe8c62f0700 1 Processor -- start 2023-10-06T11:33:09.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 -- start start 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 0x7fe8c016a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c00cc2d0 0x7fe8c016cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0169e00 con 0x7fe8c015a5b0 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c016dd90 con 0x7fe8c00cc2d0 2023-10-06T11:33:09.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c016d7d0 con 0x7fe8c00af0b0 2023-10-06T11:33:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43158/0 (socket says 172.21.15.46:43158) 2023-10-06T11:33:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 -- 172.21.15.46:0/1290463141 learned_addr learned my addr 172.21.15.46:0/1290463141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8bf7fe700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c00cc2d0 0x7fe8c016cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8bffff700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 0x7fe8c016a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 -- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 msgr2=0x7fe8c016a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 0x7fe8c016a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 -- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c00cc2d0 msgr2=0x7fe8c016cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c00cc2d0 0x7fe8c016cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 -- 172.21.15.46:0/1290463141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40094f0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8bffff700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 0x7fe8c016a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c488d700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009df0 tx=0x7fe8b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800cb50 con 0x7fe8c015a5b0 2023-10-06T11:33:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b800f040 con 0x7fe8c015a5b0 2023-10-06T11:33:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8014480 con 0x7fe8c015a5b0 2023-10-06T11:33:09.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.016+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c016dab0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.020+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c016a7d0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.020+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0169fb0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.020+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fe8b8014830 con 0x7fe8c015a5b0 2023-10-06T11:33:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.024+0000 7fe8a7fff700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 0x7fe8ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.024+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fe8b807d870 con 0x7fe8c015a5b0 2023-10-06T11:33:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.024+0000 7fe8bffff700 1 -- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 msgr2=0x7fe8ac062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.024+0000 7fe8bffff700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 0x7fe8ac062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:09.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.028+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe8c0169fb0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.204 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.192+0000 7f90415c3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.196+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8c0169fb0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.200+0000 7fe8a7fff700 1 -- 172.21.15.46:0/1290463141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18233 (secure 0 0 0) 0x7fe8c0169fb0 con 0x7fe8c015a5b0 2023-10-06T11:33:09.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:09.212 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:05.460548+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:31:03.320790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T11:31:04.384281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1f","mappings":[{"from":0,"to":6}]},{"pgid":"8.28","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1691388410":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3927254994":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/2878818767":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6833/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/3489019109":"2023-10-07T11:31:19.985120+0000","172.21.15.46:6832/32974":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1702922641":"2023-10-07T11:31:19.985120+0000","172.21.15.46:0/1135435385":"2023-10-07T11:31:19.985120+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.200+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 msgr2=0x7fe8ac062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.200+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 0x7fe8ac062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.200+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 msgr2=0x7fe8c00acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.200+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009df0 tx=0x7fe8b800b040 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 shutdown_connections 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8ac060670 0x7fe8ac062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8c015a5b0 0x7fe8c00acb90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8c00cc2d0 0x7fe8c016cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 --2- 172.21.15.46:0/1290463141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8c00af0b0 0x7fe8c016a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 >> 172.21.15.46:0/1290463141 conn(0x7fe8c0068bf0 msgr2=0x7fe8c0150ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 shutdown_connections 2023-10-06T11:33:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.204+0000 7fe8c62f0700 1 -- 172.21.15.46:0/1290463141 wait complete. 2023-10-06T11:33:09.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:33:09.315 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f900b209700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.315 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f900b209700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.315 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f900b209700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.315 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f900b209700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f900b209700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f9008a04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f9008a04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f9008a04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f9008a04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:09.304+0000 7f9008a04700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.384+0000 7fd98bf45700 1 Processor -- start 2023-10-06T11:33:09.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 -- start start 2023-10-06T11:33:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd98415ab60 0x7fd98406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd9840adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98406d460 con 0x7fd98415af30 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9840c7000 con 0x7fd98415ab60 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd98bf45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c72c0 con 0x7fd9840ad6c0 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd989ce1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd98415ab60 0x7fd98406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd989ce1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd98415ab60 0x7fd98406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43272/0 (socket says 172.21.15.46:43272) 2023-10-06T11:33:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.388+0000 7fd989ce1700 1 -- 172.21.15.46:0/631706385 learned_addr learned my addr 172.21.15.46:0/631706385 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd989ce1700 1 -- 172.21.15.46:0/631706385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 msgr2=0x7fd9840adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98a4e2700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd9840adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd989ce1700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd9840adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd989ce1700 1 -- 172.21.15.46:0/631706385 >> v1:172.21.15.132:6789/0 conn(0x7fd98415af30 legacy=0x7fd98406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd989ce1700 1 -- 172.21.15.46:0/631706385 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840c75a0 con 0x7fd98415ab60 2023-10-06T11:33:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98a4e2700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd9840adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd989ce1700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd98406e8a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd980009990 tx=0x7fd980009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3933df2c1679ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd988cdf700 1 -- 172.21.15.46:0/631706385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980005c10 con 0x7fd98415ab60 2023-10-06T11:33:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd988cdf700 1 -- 172.21.15.46:0/631706385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980005d70 con 0x7fd98415ab60 2023-10-06T11:33:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd988cdf700 1 -- 172.21.15.46:0/631706385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000b840 con 0x7fd98415ab60 2023-10-06T11:33:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 -- 172.21.15.46:0/631706385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 msgr2=0x7fd98406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd98406e8a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd980009990 tx=0x7fd980009220 comp rx=0 tx=0).stop 2023-10-06T11:33:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 -- 172.21.15.46:0/631706385 shutdown_connections 2023-10-06T11:33:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd9840adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 --2- 172.21.15.46:0/631706385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd98406e8a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 -- 172.21.15.46:0/631706385 >> 172.21.15.46:0/631706385 conn(0x7fd9840cbe30 msgr2=0x7fd9840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 -- 172.21.15.46:0/631706385 shutdown_connections 2023-10-06T11:33:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.392+0000 7fd98bf45700 1 -- 172.21.15.46:0/631706385 wait complete. 2023-10-06T11:33:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 Processor -- start 2023-10-06T11:33:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 -- start start 2023-10-06T11:33:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd984159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98415af30 0x7fd9840c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98416c9e0 con 0x7fd98415ab60 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98416cc80 con 0x7fd98415af30 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd98bf45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98416cf60 con 0x7fd9840ad6c0 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd9894e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd9894e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43288/0 (socket says 172.21.15.46:43288) 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd9894e0700 1 -- 172.21.15.46:0/2274988478 learned_addr learned my addr 172.21.15.46:0/2274988478 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.396+0000 7fd989ce1700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd984159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd98a4e2700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98415af30 0x7fd9840c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 -- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 msgr2=0x7fd984159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd984159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 -- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98415af30 msgr2=0x7fd9840c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98415af30 0x7fd9840c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 -- 172.21.15.46:0/2274988478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009480 con 0x7fd98415ab60 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd9894e0700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c5d0 tx=0x7fd974007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011590 con 0x7fd98415ab60 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974011710 con 0x7fd98415ab60 2023-10-06T11:33:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011ad0 con 0x7fd98415ab60 2023-10-06T11:33:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98416d240 con 0x7fd98415ab60 2023-10-06T11:33:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98415a6b0 con 0x7fd98415ab60 2023-10-06T11:33:09.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.400+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984063f90 con 0x7fd98415ab60 2023-10-06T11:33:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.404+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fd97401c460 con 0x7fd98415ab60 2023-10-06T11:33:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.404+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7190+0+0 (secure 0 0 0) 0x7fd97407c6c0 con 0x7fd98415ab60 2023-10-06T11:33:09.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.412+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd97404c9b0 con 0x7fd98415ab60 2023-10-06T11:33:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.612+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd98404b5c0 con 0x7fd98415ab60 2023-10-06T11:33:09.629 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:33:09.616+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:33:09.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.624+0000 7fd979ffb700 1 -- 172.21.15.46:0/2274988478 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd97404ffd0 con 0x7fd98415ab60 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 msgr2=0x7fd9840c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c5d0 tx=0x7fd974007f60 comp rx=0 tx=0).stop 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 shutdown_connections 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd98415af30 0x7fd9840c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98415ab60 0x7fd9840c9a00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 --2- 172.21.15.46:0/2274988478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9840ad6c0 0x7fd984159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 >> 172.21.15.46:0/2274988478 conn(0x7fd9840cbe30 msgr2=0x7fd98406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 shutdown_connections 2023-10-06T11:33:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.628+0000 7fd98bf45700 1 -- 172.21.15.46:0/2274988478 wait complete. 2023-10-06T11:33:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:33:09.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:33:09.699 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:33:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.804+0000 7f2073e6e700 1 Processor -- start 2023-10-06T11:33:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.804+0000 7f2073e6e700 1 -- start start 2023-10-06T11:33:09.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.808+0000 7f2073e6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f206c06e8a0 0x7f206c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c0c1bf0 con 0x7f206c0c6470 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071c0a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f206c0c6470 0x7f206c0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54702/0 (socket says 172.21.15.46:54702) 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071c0a700 1 -- 172.21.15.46:0/1661090647 learned_addr learned my addr 172.21.15.46:0/1661090647 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c06cf80 con 0x7f206c06ed10 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c06d260 con 0x7f206c06e8a0 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071409700 1 --2- 172.21.15.46:0/1661090647 >> v2:172.21.15.46:3300/0 conn(0x7f206c06e8a0 0x7f206c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151480608 0 0) 0x7f206c0c1bf0 con 0x7f206c0c6470 2023-10-06T11:33:09.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2058003570 con 0x7f206c0c6470 2023-10-06T11:33:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242994595 0 0) 0x7f206c06cf80 con 0x7f206c06ed10 2023-10-06T11:33:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f206c0c1bf0 con 0x7f206c06ed10 2023-10-06T11:33:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071409700 1 -- 172.21.15.46:0/1661090647 >> v1:172.21.15.132:6789/0 conn(0x7f206c06ed10 legacy=0x7f206c0c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071409700 1 -- 172.21.15.46:0/1661090647 >> v1:172.21.15.46:6789/0 conn(0x7f206c0c6470 legacy=0x7f206c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071409700 1 -- 172.21.15.46:0/1661090647 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0c1880 con 0x7f206c06e8a0 2023-10-06T11:33:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2071409700 1 --2- 172.21.15.46:0/1661090647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c6840 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f205c0098e0 tx=0x7f205c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb0a0dc0b62cb2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c011630 con 0x7f206c06e8a0 2023-10-06T11:33:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c011790 con 0x7f206c06e8a0 2023-10-06T11:33:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2070c08700 1 -- 172.21.15.46:0/1661090647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c011b30 con 0x7f206c06e8a0 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 msgr2=0x7f206c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 --2- 172.21.15.46:0/1661090647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c6840 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f205c0098e0 tx=0x7f205c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 shutdown_connections 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 --2- 172.21.15.46:0/1661090647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c6840 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 >> 172.21.15.46:0/1661090647 conn(0x7f206c0cbe30 msgr2=0x7f206c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 shutdown_connections 2023-10-06T11:33:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.812+0000 7f2073e6e700 1 -- 172.21.15.46:0/1661090647 wait complete. 2023-10-06T11:33:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 Processor -- start 2023-10-06T11:33:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 -- start start 2023-10-06T11:33:09.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 0x7f206c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f206c0c6470 0x7f206c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0b11b0 con 0x7f206c06e8a0 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c3e80 con 0x7f206c0c6470 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2073e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c8e80 con 0x7f206c06ed10 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2071c0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2071c0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43320/0 (socket says 172.21.15.46:43320) 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.816+0000 7f2071c0a700 1 -- 172.21.15.46:0/498976306 learned_addr learned my addr 172.21.15.46:0/498976306 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f207240b700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f206c0c6470 0x7f206c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071409700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 0x7f206c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 -- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 msgr2=0x7f206c0b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 0x7f206c0b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 -- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f206c0c6470 msgr2=0x7f206c15a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f206c0c6470 0x7f206c15a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 -- 172.21.15.46:0/498976306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c009420 con 0x7f206c06e8a0 2023-10-06T11:33:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071409700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 0x7f206c0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2071c0a700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f206800c790 tx=0x7f2068007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068005b30 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068005c90 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c0c9120 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068007300 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c0c35e0 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.820+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c0660f0 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.824+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f206801b3b0 con 0x7f206c06e8a0 2023-10-06T11:33:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.824+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7276+0+0 (secure 0 0 0) 0x7f206807be50 con 0x7f206c06e8a0 2023-10-06T11:33:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:09.832+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f206804c1e0 con 0x7f206c06e8a0 2023-10-06T11:33:10.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.036+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f206c0ae320 con 0x7f206c06e8a0 2023-10-06T11:33:10.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.224+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 26) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f20680550c0 con 0x7f206c06e8a0 2023-10-06T11:33:10.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.224+0000 7f2061ffb700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 0x7f2050068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.228+0000 7f2071409700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 0x7f2050068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.228+0000 7f2071409700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 0x7f2050068e60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f206c0c8c70 tx=0x7f205c017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:10.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.628+0000 7f2061ffb700 1 -- 172.21.15.46:0/498976306 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2068069af0 con 0x7f206c06e8a0 2023-10-06T11:33:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.632+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 msgr2=0x7f2050068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.632+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 0x7f2050068e60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f206c0c8c70 tx=0x7f205c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:10.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.632+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 msgr2=0x7f206c0c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.632+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f206800c790 tx=0x7f2068007ea0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 shutdown_connections 2023-10-06T11:33:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2050066a80 0x7f2050068e60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f206c0c6470 0x7f206c15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f206c06ed10 0x7f206c0b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 --2- 172.21.15.46:0/498976306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f206c06e8a0 0x7f206c0c2ee0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 >> 172.21.15.46:0/498976306 conn(0x7f206c0cbe30 msgr2=0x7f206c15b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 shutdown_connections 2023-10-06T11:33:10.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.636+0000 7f2073e6e700 1 -- 172.21.15.46:0/498976306 wait complete. 2023-10-06T11:33:10.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:33:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.876+0000 7f157e363700 1 Processor -- start 2023-10-06T11:33:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.876+0000 7f157e363700 1 -- start start 2023-10-06T11:33:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.880+0000 7f157e363700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f15780cc500 0x7f157815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.880+0000 7f157e363700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15780cf360 con 0x7f15781595c0 2023-10-06T11:33:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.880+0000 7f157e363700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157815dd60 con 0x7f15781591f0 2023-10-06T11:33:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.880+0000 7f157e363700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15780cff00 con 0x7f15780cc500 2023-10-06T11:33:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157cb60700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15781595c0 0x7f15780af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54710/0 (socket says 172.21.15.46:54710) 2023-10-06T11:33:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157cb60700 1 -- 172.21.15.46:0/648355055 learned_addr learned my addr 172.21.15.46:0/648355055 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157db62700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f15780cc500 0x7f157815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128962078 0 0) 0x7f15780cf360 con 0x7f15781595c0 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1564003570 con 0x7f15781595c0 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157db62700 1 -- 172.21.15.46:0/648355055 >> v1:172.21.15.132:6789/0 conn(0x7f15781591f0 legacy=0x7f15780cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858572287 0 0) 0x7f157815dd60 con 0x7f15781591f0 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157db62700 1 -- 172.21.15.46:0/648355055 >> v1:172.21.15.46:6789/0 conn(0x7f15781595c0 legacy=0x7f15780af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157db62700 1 -- 172.21.15.46:0/648355055 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780cea30 con 0x7f15780cc500 2023-10-06T11:33:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157db62700 1 --2- 172.21.15.46:0/648355055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15780cc500 0x7f157815ceb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1570009e60 tx=0x7f1570009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f96d6cd519c331 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000e070 con 0x7f15780cc500 2023-10-06T11:33:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157000bbf0 con 0x7f15780cc500 2023-10-06T11:33:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f156ffff700 1 -- 172.21.15.46:0/648355055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570012430 con 0x7f15780cc500 2023-10-06T11:33:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 -- 172.21.15.46:0/648355055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15780cc500 msgr2=0x7f157815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 --2- 172.21.15.46:0/648355055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15780cc500 0x7f157815ceb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1570009e60 tx=0x7f1570009780 comp rx=0 tx=0).stop 2023-10-06T11:33:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 -- 172.21.15.46:0/648355055 shutdown_connections 2023-10-06T11:33:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 --2- 172.21.15.46:0/648355055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15780cc500 0x7f157815ceb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 -- 172.21.15.46:0/648355055 >> 172.21.15.46:0/648355055 conn(0x7f157806ef60 msgr2=0x7f15780d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.884+0000 7f157e363700 1 -- 172.21.15.46:0/648355055 shutdown_connections 2023-10-06T11:33:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 -- 172.21.15.46:0/648355055 wait complete. 2023-10-06T11:33:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 Processor -- start 2023-10-06T11:33:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 -- start start 2023-10-06T11:33:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15780cc500 0x7f157816c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15781591f0 0x7f15780b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157816d250 con 0x7f157816cbc0 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578150d00 con 0x7f15780cc500 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.888+0000 7f157e363700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1578150fe0 con 0x7f15781591f0 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157d361700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15780cc500 0x7f157816c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43350/0 (socket says 172.21.15.46:43350) 2023-10-06T11:33:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157cb60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15781591f0 0x7f15780b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 -- 172.21.15.46:0/3692941511 learned_addr learned my addr 172.21.15.46:0/3692941511 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 -- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15781591f0 msgr2=0x7f15780b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15781591f0 0x7f15780b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 -- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15780cc500 msgr2=0x7f157816c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15780cc500 0x7f157816c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 -- 172.21.15.46:0/3692941511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009a20 con 0x7f157816cbc0 2023-10-06T11:33:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157db62700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f15700099f0 tx=0x7f1570019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570026480 con 0x7f157816cbc0 2023-10-06T11:33:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1570026600 con 0x7f157816cbc0 2023-10-06T11:33:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15781512c0 con 0x7f157816cbc0 2023-10-06T11:33:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700269c0 con 0x7f157816cbc0 2023-10-06T11:33:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.892+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15780b26e0 con 0x7f157816cbc0 2023-10-06T11:33:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.896+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15780b2990 con 0x7f157816cbc0 2023-10-06T11:33:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.896+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f157001c400 con 0x7f157816cbc0 2023-10-06T11:33:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.896+0000 7f156d7fa700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 0x7f155c062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.896+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7276+0+0 (secure 0 0 0) 0x7f157007ede0 con 0x7f157816cbc0 2023-10-06T11:33:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.896+0000 7f157d361700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 0x7f155c062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.900+0000 7f157d361700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 0x7f155c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1574007fd0 tx=0x7f1574007d30 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:10.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:10.904+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f15780b2990 con 0x7f157816cbc0 2023-10-06T11:33:11.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.080+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15780afe40 con 0x7f157816cbc0 2023-10-06T11:33:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.640+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f157004f360 con 0x7f157816cbc0 2023-10-06T11:33:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.648+0000 7f156d7fa700 1 -- 172.21.15.46:0/3692941511 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f157004ef90 con 0x7f157816cbc0 2023-10-06T11:33:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 msgr2=0x7f155c062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 0x7f155c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1574007fd0 tx=0x7f1574007d30 comp rx=0 tx=0).stop 2023-10-06T11:33:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 msgr2=0x7f157814d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f15700099f0 tx=0x7f1570019000 comp rx=0 tx=0).stop 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 shutdown_connections 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f155c0603a0 0x7f155c062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f157816cbc0 0x7f157814d760 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15781591f0 0x7f15780b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 --2- 172.21.15.46:0/3692941511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15780cc500 0x7f157816c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 >> 172.21.15.46:0/3692941511 conn(0x7f157806ef60 msgr2=0x7f15780b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 shutdown_connections 2023-10-06T11:33:11.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.652+0000 7f157e363700 1 -- 172.21.15.46:0/3692941511 wait complete. 2023-10-06T11:33:11.667 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:33:11.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:33:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.852+0000 7f80fd816700 1 Processor -- start 2023-10-06T11:33:11.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.852+0000 7f80fd816700 1 -- start start 2023-10-06T11:33:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80f8069770 0x7f80f80b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f80f81591b0 0x7f80f80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f80b5eb0 con 0x7f80f8158de0 2023-10-06T11:33:11.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80cef30 con 0x7f80f8069770 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f80cf4f0 con 0x7f80f81591b0 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f80f8069770 0x7f80f80b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fc814700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f80f8158de0 0x7f80f8069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54712/0 (socket says 172.21.15.46:54712) 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fc814700 1 -- 172.21.15.46:0/2803394565 learned_addr learned my addr 172.21.15.46:0/2803394565 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd015700 1 --2- 172.21.15.46:0/2803394565 >> v2:172.21.15.132:3300/0 conn(0x7f80f81591b0 0x7f80f80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 -- 172.21.15.46:0/2803394565 >> v2:172.21.15.132:3300/0 conn(0x7f80f81591b0 msgr2=0x7f80f80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 --2- 172.21.15.46:0/2803394565 >> v2:172.21.15.132:3300/0 conn(0x7f80f81591b0 0x7f80f80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 -- 172.21.15.46:0/2803394565 >> v1:172.21.15.46:6789/0 conn(0x7f80f8158de0 legacy=0x7f80f8069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f77fe700 1 -- 172.21.15.46:0/2803394565 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767523010 0 0) 0x7f80ec001110 con 0x7f80f8158de0 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 -- 172.21.15.46:0/2803394565 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f80ccac0 con 0x7f80f8069770 2023-10-06T11:33:11.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f7fff700 1 --2- 172.21.15.46:0/2803394565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8069770 0x7f80f80b5970 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f80f8088dd0 tx=0x7f80e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e33f1884fc9272 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f77fe700 1 -- 172.21.15.46:0/2803394565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400b6b0 con 0x7f80f8069770 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f77fe700 1 -- 172.21.15.46:0/2803394565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e400b810 con 0x7f80f8069770 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80f77fe700 1 -- 172.21.15.46:0/2803394565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400bbb0 con 0x7f80f8069770 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 -- 172.21.15.46:0/2803394565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8069770 msgr2=0x7f80f80b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.856+0000 7f80fd816700 1 --2- 172.21.15.46:0/2803394565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8069770 0x7f80f80b5970 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f80f8088dd0 tx=0x7f80e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 -- 172.21.15.46:0/2803394565 shutdown_connections 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 --2- 172.21.15.46:0/2803394565 >> v2:172.21.15.132:3300/0 conn(0x7f80f81591b0 0x7f80f80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 --2- 172.21.15.46:0/2803394565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8069770 0x7f80f80b5970 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 -- 172.21.15.46:0/2803394565 >> 172.21.15.46:0/2803394565 conn(0x7f80f8068390 msgr2=0x7f80f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 -- 172.21.15.46:0/2803394565 shutdown_connections 2023-10-06T11:33:11.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 -- 172.21.15.46:0/2803394565 wait complete. 2023-10-06T11:33:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.860+0000 7f80fd816700 1 Processor -- start 2023-10-06T11:33:11.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 -- start start 2023-10-06T11:33:11.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f8069770 0x7f80f815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f81591b0 0x7f80f806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f815aec0 con 0x7f80f8158de0 2023-10-06T11:33:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f814b760 con 0x7f80f81591b0 2023-10-06T11:33:11.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f815a2b0 con 0x7f80f8069770 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43368/0 (socket says 172.21.15.46:43368) 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 -- 172.21.15.46:0/1034630667 learned_addr learned my addr 172.21.15.46:0/1034630667 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fd015700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f81591b0 0x7f80f806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80fc814700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f8069770 0x7f80f815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 -- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f8069770 msgr2=0x7f80f815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f8069770 0x7f80f815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 -- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f81591b0 msgr2=0x7f80f806bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f81591b0 0x7f80f806bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 -- 172.21.15.46:0/1034630667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4009420 con 0x7f80f8158de0 2023-10-06T11:33:11.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.864+0000 7f80f7fff700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f80f8088340 tx=0x7f80e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400bb90 con 0x7f80f8158de0 2023-10-06T11:33:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e4011420 con 0x7f80f8158de0 2023-10-06T11:33:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f815a590 con 0x7f80f8158de0 2023-10-06T11:33:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40117c0 con 0x7f80f8158de0 2023-10-06T11:33:11.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f815ab50 con 0x7f80f8158de0 2023-10-06T11:33:11.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.868+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f80b5850 con 0x7f80f8158de0 2023-10-06T11:33:11.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.872+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f80e4011bb0 con 0x7f80f8158de0 2023-10-06T11:33:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.872+0000 7f80f4ff9700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 0x7f80dc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:11.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.872+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6838+0+0 (secure 0 0 0) 0x7f80e407e7b0 con 0x7f80f8158de0 2023-10-06T11:33:11.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.872+0000 7f80fc814700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 0x7f80dc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:11.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.872+0000 7f80fc814700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 0x7f80dc062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006fc0 tx=0x7f80ec007b90 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:11.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:11.880+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f80f80b5850 con 0x7f80f8158de0 2023-10-06T11:33:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.052+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80f80b5850 con 0x7f80f8158de0 2023-10-06T11:33:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.448+0000 7f80f4ff9700 1 -- 172.21.15.46:0/1034630667 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7f80f80b5850 con 0x7f80f8158de0 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 msgr2=0x7f80dc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 0x7f80dc062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006fc0 tx=0x7f80ec007b90 comp rx=0 tx=0).stop 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 msgr2=0x7f80f81597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f80f8088340 tx=0x7f80e4017000 comp rx=0 tx=0).stop 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 shutdown_connections 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f80dc060670 0x7f80dc062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80f81591b0 0x7f80f806bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f8158de0 0x7f80f81597f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 --2- 172.21.15.46:0/1034630667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80f8069770 0x7f80f815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 >> 172.21.15.46:0/1034630667 conn(0x7f80f8068390 msgr2=0x7f80f80af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:12.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 shutdown_connections 2023-10-06T11:33:12.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.456+0000 7f80fd816700 1 -- 172.21.15.46:0/1034630667 wait complete. 2023-10-06T11:33:12.471 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:33:12.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:33:12.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 Processor -- start 2023-10-06T11:33:12.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 -- start start 2023-10-06T11:33:12.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:12.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0cb740 con 0x7fb70c156ec0 2023-10-06T11:33:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c157290 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.648+0000 7fb712323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0ab320 con 0x7fb70c0cb940 2023-10-06T11:33:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb711321700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb70c156ec0 0x7fb70c0cb030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54728/0 (socket says 172.21.15.46:54728) 2023-10-06T11:33:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb711321700 1 -- 172.21.15.46:0/279075724 learned_addr learned my addr 172.21.15.46:0/279075724 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:12.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb710b20700 1 --2- 172.21.15.46:0/279075724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:12.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215840014 0 0) 0x7fb70c0cb740 con 0x7fb70c156ec0 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f8003570 con 0x7fb70c156ec0 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336693944 0 0) 0x7fb70c157290 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c0cb740 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb710b20700 1 -- 172.21.15.46:0/279075724 >> v1:172.21.15.132:6789/0 conn(0x7fb70c0c8dc0 legacy=0x7fb70c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb710b20700 1 -- 172.21.15.46:0/279075724 >> v1:172.21.15.46:6789/0 conn(0x7fb70c156ec0 legacy=0x7fb70c0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb710b20700 1 -- 172.21.15.46:0/279075724 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0cbdd0 con 0x7fb70c0cb940 2023-10-06T11:33:12.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb710b20700 1 --2- 172.21.15.46:0/279075724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0aade0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009910 tx=0x7fb6fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cc4491f623e3bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:12.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc011630 con 0x7fb70c0cb940 2023-10-06T11:33:12.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc011790 con 0x7fb70c0cb940 2023-10-06T11:33:12.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.652+0000 7fb703fff700 1 -- 172.21.15.46:0/279075724 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc011b30 con 0x7fb70c0cb940 2023-10-06T11:33:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- 172.21.15.46:0/279075724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 msgr2=0x7fb70c0aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 --2- 172.21.15.46:0/279075724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0aade0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009910 tx=0x7fb6fc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- 172.21.15.46:0/279075724 shutdown_connections 2023-10-06T11:33:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 --2- 172.21.15.46:0/279075724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0aade0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- 172.21.15.46:0/279075724 >> 172.21.15.46:0/279075724 conn(0x7fb70c097060 msgr2=0x7fb70c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:12.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- 172.21.15.46:0/279075724 shutdown_connections 2023-10-06T11:33:12.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- 172.21.15.46:0/279075724 wait complete. 2023-10-06T11:33:12.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 Processor -- start 2023-10-06T11:33:12.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.656+0000 7fb712323700 1 -- start start 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70c156ec0 0x7fb70c0c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c2010 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c5d10 con 0x7fb70c156ec0 2023-10-06T11:33:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c5470 con 0x7fb70c0cb940 2023-10-06T11:33:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43378/0 (socket says 172.21.15.46:43378) 2023-10-06T11:33:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 -- 172.21.15.46:0/1179793744 learned_addr learned my addr 172.21.15.46:0/1179793744 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb710b20700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711b22700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70c156ec0 0x7fb70c0c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 -- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 msgr2=0x7fb70c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 -- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70c156ec0 msgr2=0x7fb70c0c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70c156ec0 0x7fb70c0c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 -- 172.21.15.46:0/1179793744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc009420 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb710b20700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb711321700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c580 tx=0x7fb708007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708005b30 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708005c90 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c0c5750 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708007300 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.660+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c0c22c0 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.664+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c0c2450 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.664+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb70801b3b0 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.668+0000 7fb7017fa700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 0x7fb6f0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.668+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7fb70807bfc0 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.668+0000 7fb710b20700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 0x7fb6f0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.668+0000 7fb710b20700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 0x7fb6f0062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009910 tx=0x7fb6fc017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.672+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb70804c210 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.848+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb70c064590 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.856+0000 7fb7017fa700 1 -- 172.21.15.46:0/1179793744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb70804f830 con 0x7fb70c0c8dc0 2023-10-06T11:33:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 msgr2=0x7fb6f0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 0x7fb6f0062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009910 tx=0x7fb6fc017000 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 msgr2=0x7fb70c0c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c580 tx=0x7fb708007dd0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 shutdown_connections 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb6f0060670 0x7fb6f0062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb70c156ec0 0x7fb70c0c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb70c0cb940 0x7fb70c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 --2- 172.21.15.46:0/1179793744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb70c0c8dc0 0x7fb70c0c4e20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 >> 172.21.15.46:0/1179793744 conn(0x7fb70c097060 msgr2=0x7fb70c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 shutdown_connections 2023-10-06T11:33:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:12.860+0000 7fb712323700 1 -- 172.21.15.46:0/1179793744 wait complete. 2023-10-06T11:33:12.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:33:13.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 Processor -- start 2023-10-06T11:33:13.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 -- start start 2023-10-06T11:33:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f701591b0 0x7f2f70069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f70159850 0x7f2f70159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f700ce6e0 con 0x7f2f70069270 2023-10-06T11:33:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7015a5c0 con 0x7f2f70159850 2023-10-06T11:33:13.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7708e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f700cddb0 con 0x7f2f701591b0 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7688d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f70159850 0x7f2f70159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7688d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f70159850 0x7f2f70159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43390/0 (socket says 172.21.15.46:43390) 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.056+0000 7f2f7688d700 1 -- 172.21.15.46:0/1970950357 learned_addr learned my addr 172.21.15.46:0/1970950357 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7588b700 1 --2- 172.21.15.46:0/1970950357 >> v2:172.21.15.132:3300/0 conn(0x7f2f701591b0 0x7f2f70069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7688d700 1 -- 172.21.15.46:0/1970950357 >> v2:172.21.15.132:3300/0 conn(0x7f2f701591b0 msgr2=0x7f2f70069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7688d700 1 --2- 172.21.15.46:0/1970950357 >> v2:172.21.15.132:3300/0 conn(0x7f2f701591b0 0x7f2f70069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7688d700 1 -- 172.21.15.46:0/1970950357 >> v1:172.21.15.132:6789/0 conn(0x7f2f70069270 legacy=0x7f2f70158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7688d700 1 -- 172.21.15.46:0/1970950357 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700ce090 con 0x7f2f70159850 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7688d700 1 --2- 172.21.15.46:0/1970950357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70159850 0x7f2f70159cd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2f7015cb80 tx=0x7f2f68009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2acf6cc284c7f6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7508a700 1 -- 172.21.15.46:0/1970950357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800e070 con 0x7f2f70159850 2023-10-06T11:33:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7508a700 1 -- 172.21.15.46:0/1970950357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6800ba10 con 0x7f2f70159850 2023-10-06T11:33:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7508a700 1 -- 172.21.15.46:0/1970950357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680092e0 con 0x7f2f70159850 2023-10-06T11:33:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 -- 172.21.15.46:0/1970950357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70159850 msgr2=0x7f2f70159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 --2- 172.21.15.46:0/1970950357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70159850 0x7f2f70159cd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2f7015cb80 tx=0x7f2f68009840 comp rx=0 tx=0).stop 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 -- 172.21.15.46:0/1970950357 shutdown_connections 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 --2- 172.21.15.46:0/1970950357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70159850 0x7f2f70159cd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 --2- 172.21.15.46:0/1970950357 >> v2:172.21.15.132:3300/0 conn(0x7f2f701591b0 0x7f2f70069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 -- 172.21.15.46:0/1970950357 >> 172.21.15.46:0/1970950357 conn(0x7f2f70068390 msgr2=0x7f2f700d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 -- 172.21.15.46:0/1970950357 shutdown_connections 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.060+0000 7f2f7708e700 1 -- 172.21.15.46:0/1970950357 wait complete. 2023-10-06T11:33:13.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 Processor -- start 2023-10-06T11:33:13.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- start start 2023-10-06T11:33:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f701591b0 0x7f2f700c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f70159850 0x7f2f701542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700c7fc0 con 0x7f2f70069270 2023-10-06T11:33:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700c70f0 con 0x7f2f701591b0 2023-10-06T11:33:13.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700c7340 con 0x7f2f70159850 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43406/0 (socket says 172.21.15.46:43406) 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 -- 172.21.15.46:0/2667525825 learned_addr learned my addr 172.21.15.46:0/2667525825 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7688d700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f70159850 0x7f2f701542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7588b700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f701591b0 0x7f2f700c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 -- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f70159850 msgr2=0x7f2f701542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f70159850 0x7f2f701542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 -- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f701591b0 msgr2=0x7f2f700c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f701591b0 0x7f2f700c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 -- 172.21.15.46:0/2667525825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009ae0 con 0x7f2f70069270 2023-10-06T11:33:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7608c700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c600 tx=0x7f2f6c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011590 con 0x7f2f70069270 2023-10-06T11:33:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c011710 con 0x7f2f70069270 2023-10-06T11:33:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011ad0 con 0x7f2f70069270 2023-10-06T11:33:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f700c7620 con 0x7f2f70069270 2023-10-06T11:33:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.064+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f700c7be0 con 0x7f2f70069270 2023-10-06T11:33:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.068+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f70159f10 con 0x7f2f70069270 2023-10-06T11:33:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.068+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2f6c01c460 con 0x7f2f70069270 2023-10-06T11:33:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.072+0000 7f2f667fc700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 0x7f2f54062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.072+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f2f6c07c870 con 0x7f2f70069270 2023-10-06T11:33:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.072+0000 7f2f7588b700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 0x7f2f54062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.072+0000 7f2f7588b700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 0x7f2f54062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f700cd260 tx=0x7f2f60008040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.076+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f70159f10 con 0x7f2f70069270 2023-10-06T11:33:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.248+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2f70159f10 con 0x7f2f70069270 2023-10-06T11:33:13.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.260+0000 7f2f667fc700 1 -- 172.21.15.46:0/2667525825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2f70159f10 con 0x7f2f70069270 2023-10-06T11:33:13.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 msgr2=0x7f2f54062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 0x7f2f54062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f700cd260 tx=0x7f2f60008040 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 msgr2=0x7f2f700b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c600 tx=0x7f2f6c007f60 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 shutdown_connections 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2f54060670 0x7f2f54062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f70159850 0x7f2f701542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f701591b0 0x7f2f700c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 --2- 172.21.15.46:0/2667525825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f70069270 0x7f2f700b16e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 >> 172.21.15.46:0/2667525825 conn(0x7f2f70068390 msgr2=0x7f2f70155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:13.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 shutdown_connections 2023-10-06T11:33:13.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.264+0000 7f2f7708e700 1 -- 172.21.15.46:0/2667525825 wait complete. 2023-10-06T11:33:13.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:33:13.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.448+0000 7fbd0b59e700 1 Processor -- start 2023-10-06T11:33:13.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 -- start start 2023-10-06T11:33:13.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c2220 0x7fbd0c0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0c0c1d90 con 0x7fbd0c0c6250 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0c1460 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0c1740 con 0x7fbd0c0c2220 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51300/0 (socket says 172.21.15.46:51300) 2023-10-06T11:33:13.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.452+0000 7fbd0a59c700 1 -- 172.21.15.46:0/3030439527 learned_addr learned my addr 172.21.15.46:0/3030439527 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0a59c700 1 -- 172.21.15.46:0/3030439527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c2220 msgr2=0x7fbd0c0c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0a59c700 1 --2- 172.21.15.46:0/3030439527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c2220 0x7fbd0c0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0a59c700 1 -- 172.21.15.46:0/3030439527 >> v1:172.21.15.132:6789/0 conn(0x7fbd0c0c6250 legacy=0x7fbd0c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0a59c700 1 -- 172.21.15.46:0/3030439527 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0c1a20 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0a59c700 1 --2- 172.21.15.46:0/3030439527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0099b0 tx=0x7fbcfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19266d0e6eea4d24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd09d9b700 1 -- 172.21.15.46:0/3030439527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00b820 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd09d9b700 1 -- 172.21.15.46:0/3030439527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00b980 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd09d9b700 1 -- 172.21.15.46:0/3030439527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00bd20 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 -- 172.21.15.46:0/3030439527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c97e0 msgr2=0x7fbd0c0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/3030439527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0099b0 tx=0x7fbcfc009250 comp rx=0 tx=0).stop 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 -- 172.21.15.46:0/3030439527 shutdown_connections 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/3030439527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c2220 0x7fbd0c0c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/3030439527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c97e0 0x7fbd0c0c7d40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 -- 172.21.15.46:0/3030439527 >> 172.21.15.46:0/3030439527 conn(0x7fbd0c0cbc10 msgr2=0x7fbd0c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:13.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 -- 172.21.15.46:0/3030439527 shutdown_connections 2023-10-06T11:33:13.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.456+0000 7fbd0b59e700 1 -- 172.21.15.46:0/3030439527 wait complete. 2023-10-06T11:33:13.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.460+0000 7fbd0b59e700 1 Processor -- start 2023-10-06T11:33:13.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.460+0000 7fbd0b59e700 1 -- start start 2023-10-06T11:33:13.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 0x7fbd0c157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c97e0 0x7fbd0c1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0ac330 con 0x7fbd0c0c2220 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0abea0 con 0x7fbd0c0c6250 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c155e00 con 0x7fbd0c0c97e0 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51316/0 (socket says 172.21.15.46:51316) 2023-10-06T11:33:13.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 -- 172.21.15.46:0/4133010803 learned_addr learned my addr 172.21.15.46:0/4133010803 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd01d9b700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 0x7fbd0c157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0ad9d700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c97e0 0x7fbd0c1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 -- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c97e0 msgr2=0x7fbd0c1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c97e0 0x7fbd0c1583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 -- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 msgr2=0x7fbd0c157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 0x7fbd0c157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 -- 172.21.15.46:0/4133010803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0094f0 con 0x7fbd0c0c2220 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd01d9b700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 0x7fbd0c157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0a59c700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009290 tx=0x7fbcfc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc026480 con 0x7fbd0c0c2220 2023-10-06T11:33:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc01f070 con 0x7fbd0c0c2220 2023-10-06T11:33:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c0c4fb0 con 0x7fbd0c0c2220 2023-10-06T11:33:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc026830 con 0x7fbd0c0c2220 2023-10-06T11:33:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.464+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c0ab850 con 0x7fbd0c0c2220 2023-10-06T11:33:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.468+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c064590 con 0x7fbd0c0c2220 2023-10-06T11:33:13.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.468+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbcfc01a440 con 0x7fbd0c0c2220 2023-10-06T11:33:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.468+0000 7fbd02ffd700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 0x7fbcec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.468+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7fbcfc021070 con 0x7fbd0c0c2220 2023-10-06T11:33:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.472+0000 7fbd01d9b700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 0x7fbcec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.472+0000 7fbd01d9b700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 0x7fbcec062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4007fd0 tx=0x7fbcf4009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.476+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbcfc04f090 con 0x7fbd0c0c2220 2023-10-06T11:33:13.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.648+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fbd0c06d100 con 0x7fbd0c0c2220 2023-10-06T11:33:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.660+0000 7fbd02ffd700 1 -- 172.21.15.46:0/4133010803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fbcfc0526b0 con 0x7fbd0c0c2220 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 msgr2=0x7fbcec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 0x7fbcec062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4007fd0 tx=0x7fbcf4009040 comp rx=0 tx=0).stop 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 msgr2=0x7fbd0c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009290 tx=0x7fbcfc016000 comp rx=0 tx=0).stop 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 shutdown_connections 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbcec060670 0x7fbcec062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd0c0c97e0 0x7fbd0c1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd0c0c6250 0x7fbd0c157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 --2- 172.21.15.46:0/4133010803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd0c0c2220 0x7fbd0c1558c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 >> 172.21.15.46:0/4133010803 conn(0x7fbd0c0cbc10 msgr2=0x7fbd0c0cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 shutdown_connections 2023-10-06T11:33:13.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.664+0000 7fbd0b59e700 1 -- 172.21.15.46:0/4133010803 wait complete. 2023-10-06T11:33:13.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-06T11:33:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.844+0000 7f395906e700 1 Processor -- start 2023-10-06T11:33:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.844+0000 7f395906e700 1 -- start start 2023-10-06T11:33:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395906e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39540c5ff0 0x7f39540ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395906e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 0x7f39540cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395906e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39540cc150 con 0x7f395415c770 2023-10-06T11:33:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395906e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39540cfbb0 con 0x7f39540c5ff0 2023-10-06T11:33:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395906e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540cdbf0 con 0x7f39540ccb50 2023-10-06T11:33:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39540c5ff0 0x7f39540ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39540c5ff0 0x7f39540ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51324/0 (socket says 172.21.15.46:51324) 2023-10-06T11:33:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 -- 172.21.15.46:0/438857040 learned_addr learned my addr 172.21.15.46:0/438857040 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f3952d9d700 1 --1- 172.21.15.46:0/438857040 >> v1:172.21.15.46:6789/0 conn(0x7f395415c770 0x7f39540cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34984/0 (socket says 172.21.15.46:34984) 2023-10-06T11:33:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395359e700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 0x7f39540cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 -- 172.21.15.46:0/438857040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 msgr2=0x7f39540cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 0x7f39540cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 -- 172.21.15.46:0/438857040 >> v1:172.21.15.46:6789/0 conn(0x7f395415c770 legacy=0x7f39540cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 -- 172.21.15.46:0/438857040 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540cb540 con 0x7f39540c5ff0 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395359e700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 0x7f39540cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f395259c700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540ce4f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f39480099b0 tx=0x7f3948009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a02a3795e7e88f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f3951d9b700 1 -- 172.21.15.46:0/438857040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800b860 con 0x7f39540c5ff0 2023-10-06T11:33:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f3951d9b700 1 -- 172.21.15.46:0/438857040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394800b9c0 con 0x7f39540c5ff0 2023-10-06T11:33:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.848+0000 7f3951d9b700 1 -- 172.21.15.46:0/438857040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800bd60 con 0x7f39540c5ff0 2023-10-06T11:33:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 -- 172.21.15.46:0/438857040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 msgr2=0x7f39540ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540ce4f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f39480099b0 tx=0x7f3948009250 comp rx=0 tx=0).stop 2023-10-06T11:33:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 -- 172.21.15.46:0/438857040 shutdown_connections 2023-10-06T11:33:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39540ccb50 0x7f39540cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 --2- 172.21.15.46:0/438857040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540ce4f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 -- 172.21.15.46:0/438857040 >> 172.21.15.46:0/438857040 conn(0x7f3954099480 msgr2=0x7f395406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 -- 172.21.15.46:0/438857040 shutdown_connections 2023-10-06T11:33:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.852+0000 7f395906e700 1 -- 172.21.15.46:0/438857040 wait complete. 2023-10-06T11:33:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 Processor -- start 2023-10-06T11:33:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 -- start start 2023-10-06T11:33:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 0x7f395409d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395415c770 0x7f395409d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395409dc70 con 0x7f39540c5ff0 2023-10-06T11:33:13.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b5ee0 con 0x7f39540ccb50 2023-10-06T11:33:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395906e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b5640 con 0x7f395415c770 2023-10-06T11:33:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51340/0 (socket says 172.21.15.46:51340) 2023-10-06T11:33:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 -- 172.21.15.46:0/657797189 learned_addr learned my addr 172.21.15.46:0/657797189 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395359e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395415c770 0x7f395409d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395259c700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 0x7f395409d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 -- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395415c770 msgr2=0x7f395409d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395415c770 0x7f395409d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 -- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 msgr2=0x7f395409d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 0x7f395409d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f3952d9d700 1 -- 172.21.15.46:0/657797189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480094f0 con 0x7f39540c5ff0 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.856+0000 7f395259c700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 0x7f395409d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f3952d9d700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f394400c590 tx=0x7f3944007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944005a70 con 0x7f39540c5ff0 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3944005bd0 con 0x7f39540c5ff0 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440070d0 con 0x7f39540c5ff0 2023-10-06T11:33:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39540b5920 con 0x7f39540c5ff0 2023-10-06T11:33:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39540b1dc0 con 0x7f39540c5ff0 2023-10-06T11:33:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39540b5b10 con 0x7f39540c5ff0 2023-10-06T11:33:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.860+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3944007480 con 0x7f39540c5ff0 2023-10-06T11:33:13.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.864+0000 7f393affd700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 0x7f3940062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:13.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.864+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f394407c060 con 0x7f39540c5ff0 2023-10-06T11:33:13.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.864+0000 7f395259c700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 0x7f3940062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:13.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.864+0000 7f395259c700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 0x7f3940062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39480099b0 tx=0x7f3948017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:13.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:13.868+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f394404c210 con 0x7f39540c5ff0 2023-10-06T11:33:14.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.092+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f39540660f0 con 0x7f39540c5ff0 2023-10-06T11:33:14.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.500+0000 7f393affd700 1 -- 172.21.15.46:0/657797189 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v541) v1 ==== 154+0+0 (secure 0 0 0) 0x7f39440170e0 con 0x7f39540c5ff0 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 msgr2=0x7f3940062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 0x7f3940062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39480099b0 tx=0x7f3948017000 comp rx=0 tx=0).stop 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 msgr2=0x7f39540b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f394400c590 tx=0x7f3944007f70 comp rx=0 tx=0).stop 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 shutdown_connections 2023-10-06T11:33:14.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3940060670 0x7f3940062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f395415c770 0x7f395409d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540ccb50 0x7f395409d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 --2- 172.21.15.46:0/657797189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39540c5ff0 0x7f39540b1880 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 >> 172.21.15.46:0/657797189 conn(0x7f3954099480 msgr2=0x7f39540cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 shutdown_connections 2023-10-06T11:33:14.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.504+0000 7f395906e700 1 -- 172.21.15.46:0/657797189 wait complete. 2023-10-06T11:33:14.541 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-06T11:33:14.541 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-06T11:33:14.541 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:33:14.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-06T11:33:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.692+0000 7f12d84f0700 1 Processor -- start 2023-10-06T11:33:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.692+0000 7f12d84f0700 1 -- start start 2023-10-06T11:33:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d84f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12d00b26d0 0x7f12d0159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d84f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00b4f80 con 0x7f12d00d0aa0 2023-10-06T11:33:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d84f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00b40c0 con 0x7f12d0159050 2023-10-06T11:33:14.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d84f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d00b4370 con 0x7f12d00b26d0 2023-10-06T11:33:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d5a8b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f12d00d0aa0 0x7f12d00b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34986/0 (socket says 172.21.15.46:34986) 2023-10-06T11:33:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d5a8b700 1 -- 172.21.15.46:0/2650316254 learned_addr learned my addr 172.21.15.46:0/2650316254 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:14.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d6a8d700 1 --2- 172.21.15.46:0/2650316254 >> v2:172.21.15.132:3300/0 conn(0x7f12d00b26d0 0x7f12d0159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512938966 0 0) 0x7f12d00b4f80 con 0x7f12d00d0aa0 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12bc003570 con 0x7f12d00d0aa0 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216551689 0 0) 0x7f12d00b40c0 con 0x7f12d0159050 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d00b4f80 con 0x7f12d0159050 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d6a8d700 1 -- 172.21.15.46:0/2650316254 >> v1:172.21.15.132:6789/0 conn(0x7f12d0159050 legacy=0x7f12d00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d6a8d700 1 -- 172.21.15.46:0/2650316254 >> v1:172.21.15.46:6789/0 conn(0x7f12d00d0aa0 legacy=0x7f12d00b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d6a8d700 1 -- 172.21.15.46:0/2650316254 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00b4650 con 0x7f12d00b26d0 2023-10-06T11:33:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.696+0000 7f12d6a8d700 1 --2- 172.21.15.46:0/2650316254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d0159420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009e60 tx=0x7f12c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a586f86fea27d75b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800e070 con 0x7f12d00b26d0 2023-10-06T11:33:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800ba70 con 0x7f12d00b26d0 2023-10-06T11:33:14.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d528a700 1 -- 172.21.15.46:0/2650316254 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800be30 con 0x7f12d00b26d0 2023-10-06T11:33:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- 172.21.15.46:0/2650316254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 msgr2=0x7f12d0159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 --2- 172.21.15.46:0/2650316254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d0159420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009e60 tx=0x7f12c8009780 comp rx=0 tx=0).stop 2023-10-06T11:33:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- 172.21.15.46:0/2650316254 shutdown_connections 2023-10-06T11:33:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 --2- 172.21.15.46:0/2650316254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d0159420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- 172.21.15.46:0/2650316254 >> 172.21.15.46:0/2650316254 conn(0x7f12d006e780 msgr2=0x7f12d006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- 172.21.15.46:0/2650316254 shutdown_connections 2023-10-06T11:33:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- 172.21.15.46:0/2650316254 wait complete. 2023-10-06T11:33:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 Processor -- start 2023-10-06T11:33:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.700+0000 7f12d84f0700 1 -- start start 2023-10-06T11:33:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d00c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12d0159050 0x7f12d0159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0158240 con 0x7f12d00d0aa0 2023-10-06T11:33:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00c8240 con 0x7f12d00b26d0 2023-10-06T11:33:14.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00c7910 con 0x7f12d0159050 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51342/0 (socket says 172.21.15.46:51342) 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 -- 172.21.15.46:0/4150481098 learned_addr learned my addr 172.21.15.46:0/4150481098 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d6a8d700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12d0159050 0x7f12d0159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d628c700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d00c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 -- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12d0159050 msgr2=0x7f12d0159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12d0159050 0x7f12d0159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 -- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 msgr2=0x7f12d00c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d00c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 -- 172.21.15.46:0/4150481098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8009a20 con 0x7f12d00d0aa0 2023-10-06T11:33:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d5a8b700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c7f0 tx=0x7f12c0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0005b30 con 0x7f12d00d0aa0 2023-10-06T11:33:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0005c90 con 0x7f12d00d0aa0 2023-10-06T11:33:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00c7bf0 con 0x7f12d00d0aa0 2023-10-06T11:33:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0007320 con 0x7f12d00d0aa0 2023-10-06T11:33:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.704+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00c33b0 con 0x7f12d00d0aa0 2023-10-06T11:33:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.708+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d0088820 con 0x7f12d00d0aa0 2023-10-06T11:33:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.708+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f12c001b3b0 con 0x7f12d00d0aa0 2023-10-06T11:33:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.712+0000 7f12c67fc700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 0x7f12b4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.712+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f12c007c180 con 0x7f12d00d0aa0 2023-10-06T11:33:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.712+0000 7f12d628c700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 0x7f12b4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.712+0000 7f12d628c700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 0x7f12b4062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f12cc007fd0 tx=0x7f12cc007d30 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.716+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12c004c330 con 0x7f12d00d0aa0 2023-10-06T11:33:14.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:14.948+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f12d00c8070 con 0x7f12d00d0aa0 2023-10-06T11:33:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.508+0000 7f12c67fc700 1 -- 172.21.15.46:0/4150481098 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v542) v1 ==== 159+0+0 (secure 0 0 0) 0x7f12c004f950 con 0x7f12d00d0aa0 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 msgr2=0x7f12b4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 0x7f12b4062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f12cc007fd0 tx=0x7f12cc007d30 comp rx=0 tx=0).stop 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 msgr2=0x7f12d01599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c7f0 tx=0x7f12c0007e00 comp rx=0 tx=0).stop 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 shutdown_connections 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12b4060670 0x7f12b4062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12d0159050 0x7f12d0159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00d0aa0 0x7f12d01599c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 --2- 172.21.15.46:0/4150481098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12d00b26d0 0x7f12d00c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 >> 172.21.15.46:0/4150481098 conn(0x7f12d006e780 msgr2=0x7f12d0155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 shutdown_connections 2023-10-06T11:33:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.512+0000 7f12d84f0700 1 -- 172.21.15.46:0/4150481098 wait complete. 2023-10-06T11:33:15.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 Processor -- start 2023-10-06T11:33:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 -- start start 2023-10-06T11:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f108c1591b0 0x7f108c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0ce6e0 con 0x7f108c069270 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c15a5c0 con 0x7f108c1591b0 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.704+0000 7f109224c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cddb0 con 0x7f108c159850 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109124a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f108c069270 0x7f108c158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34998/0 (socket says 172.21.15.46:34998) 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109124a700 1 -- 172.21.15.46:0/3311681586 learned_addr learned my addr 172.21.15.46:0/3311681586 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 --2- 172.21.15.46:0/3311681586 >> v2:172.21.15.132:3300/0 conn(0x7f108c1591b0 0x7f108c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1091a4b700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1083fff700 1 -- 172.21.15.46:0/3311681586 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197437232 0 0) 0x7f108c0ce6e0 con 0x7f108c069270 2023-10-06T11:33:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1083fff700 1 -- 172.21.15.46:0/3311681586 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1078003570 con 0x7f108c069270 2023-10-06T11:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 -- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 msgr2=0x7f108c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 -- 172.21.15.46:0/3311681586 >> v1:172.21.15.46:6789/0 conn(0x7f108c069270 legacy=0x7f108c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 -- 172.21.15.46:0/3311681586 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0ce090 con 0x7f108c1591b0 2023-10-06T11:33:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1090a49700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c1591b0 0x7f108c069640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f107c0098e0 tx=0x7f107c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a0fd07e0079a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1083fff700 1 -- 172.21.15.46:0/3311681586 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011630 con 0x7f108c1591b0 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1083fff700 1 -- 172.21.15.46:0/3311681586 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011790 con 0x7f108c1591b0 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f1083fff700 1 -- 172.21.15.46:0/3311681586 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c011b30 con 0x7f108c1591b0 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 -- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c1591b0 msgr2=0x7f108c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c1591b0 0x7f108c069640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f107c0098e0 tx=0x7f107c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 -- 172.21.15.46:0/3311681586 shutdown_connections 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 --2- 172.21.15.46:0/3311681586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c1591b0 0x7f108c069640 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 -- 172.21.15.46:0/3311681586 >> 172.21.15.46:0/3311681586 conn(0x7f108c068390 msgr2=0x7f108c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 -- 172.21.15.46:0/3311681586 shutdown_connections 2023-10-06T11:33:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.708+0000 7f109224c700 1 -- 172.21.15.46:0/3311681586 wait complete. 2023-10-06T11:33:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 Processor -- start 2023-10-06T11:33:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 -- start start 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c069270 0x7f108c169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0b5c20 con 0x7f108c1591b0 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0c8e10 con 0x7f108c069270 2023-10-06T11:33:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109224c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0c90f0 con 0x7f108c159850 2023-10-06T11:33:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f1090a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f109124a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c069270 0x7f108c169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f1090a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51354/0 (socket says 172.21.15.46:51354) 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f1090a49700 1 -- 172.21.15.46:0/1615006632 learned_addr learned my addr 172.21.15.46:0/1615006632 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.712+0000 7f1091a4b700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 -- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 msgr2=0x7f108c0c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c0c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 -- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c069270 msgr2=0x7f108c169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c069270 0x7f108c169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 -- 172.21.15.46:0/1615006632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c009420 con 0x7f108c1591b0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f1090a49700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f108c0afab0 tx=0x7f107c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c000d10 con 0x7f108c1591b0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c011b10 con 0x7f108c1591b0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c025660 con 0x7f108c1591b0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c0c93d0 con 0x7f108c1591b0 2023-10-06T11:33:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c0c8ac0 con 0x7f108c1591b0 2023-10-06T11:33:15.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.716+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c064590 con 0x7f108c1591b0 2023-10-06T11:33:15.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.720+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f107c025a50 con 0x7f108c1591b0 2023-10-06T11:33:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.720+0000 7f10817fa700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 0x7f1070062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:15.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.720+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f107c07e420 con 0x7f108c1591b0 2023-10-06T11:33:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.720+0000 7f109124a700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 0x7f1070062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:15.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.724+0000 7f109124a700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 0x7f1070062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1088007fd0 tx=0x7f1088009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.728+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f107c04e5d0 con 0x7f108c1591b0 2023-10-06T11:33:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.904+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f108c0ce370 con 0x7f108c1591b0 2023-10-06T11:33:15.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.912+0000 7f10817fa700 1 -- 172.21.15.46:0/1615006632 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f107c051bf0 con 0x7f108c1591b0 2023-10-06T11:33:15.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 msgr2=0x7f1070062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 0x7f1070062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1088007fd0 tx=0x7f1088009040 comp rx=0 tx=0).stop 2023-10-06T11:33:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 msgr2=0x7f108c0b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:15.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f108c0afab0 tx=0x7f107c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 shutdown_connections 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1070060670 0x7f1070062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f108c159850 0x7f108c0c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f108c1591b0 0x7f108c0b56e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 --2- 172.21.15.46:0/1615006632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f108c069270 0x7f108c169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 >> 172.21.15.46:0/1615006632 conn(0x7f108c068390 msgr2=0x7f108c0b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:15.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 shutdown_connections 2023-10-06T11:33:15.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:15.916+0000 7f109224c700 1 -- 172.21.15.46:0/1615006632 wait complete. 2023-10-06T11:33:15.949 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:33:15.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:15.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:16.033 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:33:16.033 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:33:16.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 Processor -- start 2023-10-06T11:33:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 -- start start 2023-10-06T11:33:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6cd406e8a0 0x7f6cd406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd406d460 con 0x7f6cd415ab60 2023-10-06T11:33:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd406ed40 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd8d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40c7000 con 0x7f6cd406e8a0 2023-10-06T11:33:16.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd2d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd2d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51360/0 (socket says 172.21.15.46:51360) 2023-10-06T11:33:16.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6cd2d9d700 1 -- 172.21.15.46:0/4284897150 learned_addr learned my addr 172.21.15.46:0/4284897150 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:16.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.184+0000 7f6ccbfff700 1 --2- 172.21.15.46:0/4284897150 >> v2:172.21.15.132:3300/0 conn(0x7f6cd406e8a0 0x7f6cd406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd2d9d700 1 -- 172.21.15.46:0/4284897150 >> v2:172.21.15.132:3300/0 conn(0x7f6cd406e8a0 msgr2=0x7f6cd406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd2d9d700 1 --2- 172.21.15.46:0/4284897150 >> v2:172.21.15.132:3300/0 conn(0x7f6cd406e8a0 0x7f6cd406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd2d9d700 1 -- 172.21.15.46:0/4284897150 >> v1:172.21.15.132:6789/0 conn(0x7f6cd415ab60 legacy=0x7f6cd415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd2d9d700 1 -- 172.21.15.46:0/4284897150 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40c72c0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd2d9d700 1 --2- 172.21.15.46:0/4284897150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6cd406f0b0 tx=0x7f6ccc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aa79f38db13ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd1d9b700 1 -- 172.21.15.46:0/4284897150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00e070 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd1d9b700 1 -- 172.21.15.46:0/4284897150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00ba20 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd1d9b700 1 -- 172.21.15.46:0/4284897150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0092e0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 -- 172.21.15.46:0/4284897150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 msgr2=0x7f6cd40adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/4284897150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6cd406f0b0 tx=0x7f6ccc009840 comp rx=0 tx=0).stop 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 -- 172.21.15.46:0/4284897150 shutdown_connections 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/4284897150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40adaf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/4284897150 >> v2:172.21.15.132:3300/0 conn(0x7f6cd406e8a0 0x7f6cd406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 -- 172.21.15.46:0/4284897150 >> 172.21.15.46:0/4284897150 conn(0x7f6cd40cbe30 msgr2=0x7f6cd40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 -- 172.21.15.46:0/4284897150 shutdown_connections 2023-10-06T11:33:16.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.188+0000 7f6cd8d45700 1 -- 172.21.15.46:0/4284897150 wait complete. 2023-10-06T11:33:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 Processor -- start 2023-10-06T11:33:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 -- start start 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 0x7f6cd40c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cd415ab60 0x7f6cd4150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd416c890 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c6820 con 0x7f6cd415ab60 2023-10-06T11:33:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd8d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd416cae0 con 0x7f6cd406e8a0 2023-10-06T11:33:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6ccbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6ccbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51376/0 (socket says 172.21.15.46:51376) 2023-10-06T11:33:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6ccbfff700 1 -- 172.21.15.46:0/2714955753 learned_addr learned my addr 172.21.15.46:0/2714955753 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd259c700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 0x7f6cd40c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.192+0000 7f6cd2d9d700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cd415ab60 0x7f6cd4150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 -- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 msgr2=0x7f6cd40c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 0x7f6cd40c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 -- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cd415ab60 msgr2=0x7f6cd4150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cd415ab60 0x7f6cd4150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 -- 172.21.15.46:0/2714955753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc009ae0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cd259c700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 0x7f6cd40c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:16.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6ccbfff700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e960 tx=0x7f6cbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00c850 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00c9b0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd416cdc0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00cd50 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd416d380 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.196+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4063f90 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.200+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6cbc013680 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.200+0000 7f6cca7fc700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 0x7f6cb4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.200+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f6cbc07ce10 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.200+0000 7f6cd259c700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 0x7f6cb4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.200+0000 7f6cd259c700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 0x7f6cb4062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40018f0 tx=0x7f6cc4009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.208+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6cbc080030 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.416+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cd404b5c0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.420+0000 7f6cca7fc700 1 -- 172.21.15.46:0/2714955753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f6cbc017de0 con 0x7f6cd40ad6c0 2023-10-06T11:33:16.432 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:16.432 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/927401457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":927401457},{"type":"v1","addr":"172.21.15.46:6837","nonce":927401457}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5339,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/4039277897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4039277897},{"type":"v1","addr":"172.21.15.132:6835","nonce":4039277897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5803,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/2338102410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2338102410},{"type":"v1","addr":"172.21.15.132:6834","nonce":2338102410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/187915410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":187915410},{"type":"v1","addr":"172.21.15.46:6836","nonce":187915410}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 msgr2=0x7f6cb4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 0x7f6cb4062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40018f0 tx=0x7f6cc4009040 comp rx=0 tx=0).stop 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 msgr2=0x7f6cd40a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00e960 tx=0x7f6cbc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 shutdown_connections 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cb4060670 0x7f6cb4062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cd415ab60 0x7f6cd4150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ad6c0 0x7f6cd40a9aa0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 --2- 172.21.15.46:0/2714955753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cd406e8a0 0x7f6cd40c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 >> 172.21.15.46:0/2714955753 conn(0x7f6cd40cbe30 msgr2=0x7f6cd40ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 shutdown_connections 2023-10-06T11:33:16.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.424+0000 7f6cd8d45700 1 -- 172.21.15.46:0/2714955753 wait complete. 2023-10-06T11:33:16.439 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2023-10-06T11:33:16.458 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:33:16.458 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:33:16.458 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:33:16.458 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:16.458 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:33:16.458 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:33:16.459 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:33:16.459 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:16.459 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:33:16.459 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:33:16.459 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:33:16.459 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:16.459 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:33:16.460 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:33:16.460 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:33:16.460 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:16.477 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:33:16.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:33:16.480 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:33:16.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:33:16.482 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:33:16.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:33:16.483 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:33:16.483 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:33:16.528 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:33:16.528 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:16.530 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:33:16.530 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:16.532 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:33:16.532 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:16.533 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:33:16.533 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:16.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:33:16.634 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:33:16.635 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:33:16.635 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:33:16.635 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:33:16.685 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T11:33:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.680+0000 7f707a88e700 1 Processor -- start 2023-10-06T11:33:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.684+0000 7f707a88e700 1 -- start start 2023-10-06T11:33:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.684+0000 7f707a88e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7074158dd0 0x7f7074067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a88e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740679f0 con 0x7f70741591a0 2023-10-06T11:33:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a88e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707415d6c0 con 0x7f7074067590 2023-10-06T11:33:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a88e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70740699c0 con 0x7f7074158dd0 2023-10-06T11:33:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707988c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7074158dd0 0x7f7074067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a08d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7074067590 0x7f707406c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34492/0 (socket says 172.21.15.46:34492) 2023-10-06T11:33:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707908b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70741591a0 0x7f7074068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35018/0 (socket says 172.21.15.46:35018) 2023-10-06T11:33:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a08d700 1 -- 172.21.15.46:0/2938754928 learned_addr learned my addr 172.21.15.46:0/2938754928 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707988c700 1 -- 172.21.15.46:0/2938754928 >> v1:172.21.15.132:6789/0 conn(0x7f7074067590 legacy=0x7f707406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707988c700 1 -- 172.21.15.46:0/2938754928 >> v1:172.21.15.46:6789/0 conn(0x7f70741591a0 legacy=0x7f7074068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707988c700 1 -- 172.21.15.46:0/2938754928 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707415cd90 con 0x7f7074158dd0 2023-10-06T11:33:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707888a700 1 -- 172.21.15.46:0/2938754928 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190882716 0 0) 0x7f7064001110 con 0x7f70741591a0 2023-10-06T11:33:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707988c700 1 --2- 172.21.15.46:0/2938754928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f7074067050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70700098e0 tx=0x7f70700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2ce59c5479d244 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707888a700 1 -- 172.21.15.46:0/2938754928 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070009e80 con 0x7f7074158dd0 2023-10-06T11:33:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707888a700 1 -- 172.21.15.46:0/2938754928 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707000b860 con 0x7f7074158dd0 2023-10-06T11:33:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707888a700 1 -- 172.21.15.46:0/2938754928 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000bbe0 con 0x7f7074158dd0 2023-10-06T11:33:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a88e700 1 -- 172.21.15.46:0/2938754928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 msgr2=0x7f7074067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.688+0000 7f707a88e700 1 --2- 172.21.15.46:0/2938754928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f7074067050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70700098e0 tx=0x7f70700091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 -- 172.21.15.46:0/2938754928 shutdown_connections 2023-10-06T11:33:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 --2- 172.21.15.46:0/2938754928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f7074067050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 -- 172.21.15.46:0/2938754928 >> 172.21.15.46:0/2938754928 conn(0x7f7074062ef0 msgr2=0x7f70740cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 -- 172.21.15.46:0/2938754928 shutdown_connections 2023-10-06T11:33:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 -- 172.21.15.46:0/2938754928 wait complete. 2023-10-06T11:33:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 Processor -- start 2023-10-06T11:33:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.692+0000 7f707a88e700 1 -- start start 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f707415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70741591a0 0x7f707416ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70740abdd0 con 0x7f7074067590 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707416d8a0 con 0x7f7074158dd0 2023-10-06T11:33:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70740ac070 con 0x7f70741591a0 2023-10-06T11:33:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51420/0 (socket says 172.21.15.46:51420) 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 -- 172.21.15.46:0/4057889063 learned_addr learned my addr 172.21.15.46:0/4057889063 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707908b700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f707415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a08d700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70741591a0 0x7f707416ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 -- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70741591a0 msgr2=0x7f707416ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70741591a0 0x7f707416ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 -- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 msgr2=0x7f707415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f707415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 -- 172.21.15.46:0/4057889063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009420 con 0x7f7074067590 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707988c700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f707406ab70 tx=0x7f7070017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707001f070 con 0x7f7074067590 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070000c90 con 0x7f7074067590 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70740ac350 con 0x7f7074067590 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070025420 con 0x7f7074067590 2023-10-06T11:33:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.696+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707415be50 con 0x7f7074067590 2023-10-06T11:33:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.700+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70740abc30 con 0x7f7074067590 2023-10-06T11:33:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.700+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7070023070 con 0x7f7074067590 2023-10-06T11:33:16.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.700+0000 7f7069ffb700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 0x7f7058062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:16.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.704+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6366+0+0 (secure 0 0 0) 0x7f707007e070 con 0x7f7074067590 2023-10-06T11:33:16.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.704+0000 7f707908b700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 0x7f7058062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:16.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.704+0000 7f707908b700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 0x7f7058062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70740c9690 tx=0x7f7064007ef0 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:16.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.708+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f70700adbe0 con 0x7f7074067590 2023-10-06T11:33:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:16.888+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7074064590 con 0x7f7074067590 2023-10-06T11:33:17.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:33:17.399 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:33:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.480+0000 7f7069ffb700 1 -- 172.21.15.46:0/4057889063 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f70740abc30 con 0x7f7074067590 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 msgr2=0x7f7058062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 0x7f7058062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70740c9690 tx=0x7f7064007ef0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 msgr2=0x7f70740c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f707406ab70 tx=0x7f7070017000 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 shutdown_connections 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7058060670 0x7f7058062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70741591a0 0x7f707416ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7074158dd0 0x7f707415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 --2- 172.21.15.46:0/4057889063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7074067590 0x7f70740c4410 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 >> 172.21.15.46:0/4057889063 conn(0x7f7074062ef0 msgr2=0x7f707406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:18.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 shutdown_connections 2023-10-06T11:33:18.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.484+0000 7f707a88e700 1 -- 172.21.15.46:0/4057889063 wait complete. 2023-10-06T11:33:18.498 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:33:18.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:33:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.676+0000 7f082f6cd700 1 Processor -- start 2023-10-06T11:33:18.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.676+0000 7f082f6cd700 1 -- start start 2023-10-06T11:33:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 0x7f0828159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08280b4f80 con 0x7f0828159050 2023-10-06T11:33:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08280b40c0 con 0x7f08280b5410 2023-10-06T11:33:18.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08280b4370 con 0x7f08280d0aa0 2023-10-06T11:33:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082d469700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0828159050 0x7f08280b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35028/0 (socket says 172.21.15.46:35028) 2023-10-06T11:33:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082d469700 1 -- 172.21.15.46:0/4088978670 learned_addr learned my addr 172.21.15.46:0/4088978670 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082cc68700 1 --2- 172.21.15.46:0/4088978670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 0x7f0828159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221664747 0 0) 0x7f08280b4f80 con 0x7f0828159050 2023-10-06T11:33:18.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0814003570 con 0x7f0828159050 2023-10-06T11:33:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707681112 0 0) 0x7f08280b40c0 con 0x7f08280b5410 2023-10-06T11:33:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082cc68700 1 -- 172.21.15.46:0/4088978670 >> v1:172.21.15.132:6789/0 conn(0x7f08280b5410 legacy=0x7f08280b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082cc68700 1 -- 172.21.15.46:0/4088978670 >> v1:172.21.15.46:6789/0 conn(0x7f0828159050 legacy=0x7f08280b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082cc68700 1 -- 172.21.15.46:0/4088978670 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280b4650 con 0x7f08280d0aa0 2023-10-06T11:33:18.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082cc68700 1 --2- 172.21.15.46:0/4088978670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 0x7f0828159420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0818009910 tx=0x7f08180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f230eac70a4b7d16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800b680 con 0x7f08280d0aa0 2023-10-06T11:33:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800b7e0 con 0x7f08280d0aa0 2023-10-06T11:33:18.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f081ffff700 1 -- 172.21.15.46:0/4088978670 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800bba0 con 0x7f08280d0aa0 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 -- 172.21.15.46:0/4088978670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 msgr2=0x7f0828159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.680+0000 7f082f6cd700 1 --2- 172.21.15.46:0/4088978670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 0x7f0828159420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0818009910 tx=0x7f08180091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- 172.21.15.46:0/4088978670 shutdown_connections 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 --2- 172.21.15.46:0/4088978670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280d0aa0 0x7f0828159420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- 172.21.15.46:0/4088978670 >> 172.21.15.46:0/4088978670 conn(0x7f082806e780 msgr2=0x7f082806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- 172.21.15.46:0/4088978670 shutdown_connections 2023-10-06T11:33:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- 172.21.15.46:0/4088978670 wait complete. 2023-10-06T11:33:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 Processor -- start 2023-10-06T11:33:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- start start 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280b5410 0x7f0828156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08280d0aa0 0x7f08280c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08280c4130 con 0x7f0828159050 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08280c4410 con 0x7f08280d0aa0 2023-10-06T11:33:18.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.684+0000 7f082f6cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08280c46f0 con 0x7f08280b5410 2023-10-06T11:33:18.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51434/0 (socket says 172.21.15.46:51434) 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 -- 172.21.15.46:0/2853812810 learned_addr learned my addr 172.21.15.46:0/2853812810 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082d469700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280b5410 0x7f0828156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082cc68700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08280d0aa0 0x7f08280c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 -- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280b5410 msgr2=0x7f0828156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280b5410 0x7f0828156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 -- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08280d0aa0 msgr2=0x7f08280c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08280d0aa0 0x7f08280c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 -- 172.21.15.46:0/2853812810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009420 con 0x7f0828159050 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082dc6a700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0820001560 tx=0x7f0820001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000a780 con 0x7f0828159050 2023-10-06T11:33:18.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000a8e0 con 0x7f0828159050 2023-10-06T11:33:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000aca0 con 0x7f0828159050 2023-10-06T11:33:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08280c49d0 con 0x7f0828159050 2023-10-06T11:33:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08280c92c0 con 0x7f0828159050 2023-10-06T11:33:18.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.688+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08280c4d40 con 0x7f0828159050 2023-10-06T11:33:18.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.692+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0820012630 con 0x7f0828159050 2023-10-06T11:33:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.692+0000 7f081d7fa700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 0x7f080c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.696+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f082007cd40 con 0x7f0828159050 2023-10-06T11:33:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.696+0000 7f082d469700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 0x7f080c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:18.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.696+0000 7f082d469700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 0x7f080c062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08240058d0 tx=0x7f0824009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:18.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.700+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f08280c4d40 con 0x7f0828159050 2023-10-06T11:33:18.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.928+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08280c4d40 con 0x7f0828159050 2023-10-06T11:33:18.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.928+0000 7f081d7fa700 1 -- 172.21.15.46:0/2853812810 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08280c4d40 con 0x7f0828159050 2023-10-06T11:33:18.941 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:18.941 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:33:18.941 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:33:18.941 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.943 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.945 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.946 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.947 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:33:18.949 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:18.950 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:18.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.953 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:33:18.954 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:18.955 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:18.957 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:33:18.957 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:33:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 msgr2=0x7f080c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 0x7f080c062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08240058d0 tx=0x7f0824009040 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 msgr2=0x7f08280cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0820001560 tx=0x7f0820001770 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 shutdown_connections 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f080c060670 0x7f080c062b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828159050 0x7f08280cf390 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08280d0aa0 0x7f08280c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 --2- 172.21.15.46:0/2853812810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08280b5410 0x7f0828156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 >> 172.21.15.46:0/2853812810 conn(0x7f082806e780 msgr2=0x7f08280cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 shutdown_connections 2023-10-06T11:33:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:18.932+0000 7f082f6cd700 1 -- 172.21.15.46:0/2853812810 wait complete. 2023-10-06T11:33:18.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 Processor -- start 2023-10-06T11:33:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 -- start start 2023-10-06T11:33:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 0x7fd8c00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8c00cec00 0x7fd8c0162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c00cc150 con 0x7fd8c00ccb50 2023-10-06T11:33:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00cfbb0 con 0x7fd8c00cec00 2023-10-06T11:33:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c6dc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00cdbf0 con 0x7fd8c015c770 2023-10-06T11:33:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8bffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8c00ccb50 0x7fd8c00ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35040/0 (socket says 172.21.15.46:35040) 2023-10-06T11:33:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8bffff700 1 -- 172.21.15.46:0/1591716138 learned_addr learned my addr 172.21.15.46:0/1591716138 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1591716138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 0x7fd8c00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8c5362700 1 --2- 172.21.15.46:0/1591716138 >> v2:172.21.15.132:3300/0 conn(0x7fd8c00cec00 0x7fd8c0162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269167244 0 0) 0x7fd8c00cc150 con 0x7fd8c00ccb50 2023-10-06T11:33:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.116+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b0003570 con 0x7fd8c00ccb50 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1591716138 >> v2:172.21.15.132:3300/0 conn(0x7fd8c00cec00 msgr2=0x7fd8c0162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1591716138 >> v2:172.21.15.132:3300/0 conn(0x7fd8c00cec00 0x7fd8c0162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1591716138 >> v1:172.21.15.46:6789/0 conn(0x7fd8c00ccb50 legacy=0x7fd8c00ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683448576 0 0) 0x7fd8b0003570 con 0x7fd8c00ccb50 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1591716138 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00cb280 con 0x7fd8c015c770 2023-10-06T11:33:19.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1591716138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 0x7fd8c00cad40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00cfcf0 tx=0x7fd8b4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d4fdf8db892e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac001470 con 0x7fd8c015c770 2023-10-06T11:33:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4005a00 con 0x7fd8c015c770 2023-10-06T11:33:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8bf7fe700 1 -- 172.21.15.46:0/1591716138 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005d80 con 0x7fd8c015c770 2023-10-06T11:33:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1591716138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 msgr2=0x7fd8c00cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1591716138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 0x7fd8c00cad40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00cfcf0 tx=0x7fd8b4009120 comp rx=0 tx=0).stop 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1591716138 shutdown_connections 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1591716138 >> v2:172.21.15.132:3300/0 conn(0x7fd8c00cec00 0x7fd8c0162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1591716138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c015c770 0x7fd8c00cad40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1591716138 >> 172.21.15.46:0/1591716138 conn(0x7fd8c0099480 msgr2=0x7fd8c006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1591716138 shutdown_connections 2023-10-06T11:33:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.120+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1591716138 wait complete. 2023-10-06T11:33:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 Processor -- start 2023-10-06T11:33:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- start start 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00cec00 0x7fd8c00c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015c770 0x7fd8c009b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0159070 con 0x7fd8c00ccb50 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00c8f70 con 0x7fd8c015c770 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00c9400 con 0x7fd8c00cec00 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51450/0 (socket says 172.21.15.46:51450) 2023-10-06T11:33:19.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1686447533 learned_addr learned my addr 172.21.15.46:0/1686447533 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8bffff700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00cec00 0x7fd8c00c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c5362700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015c770 0x7fd8c009b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00cec00 msgr2=0x7fd8c00c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00cec00 0x7fd8c00c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015c770 msgr2=0x7fd8c009b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015c770 0x7fd8c009b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 -- 172.21.15.46:0/1686447533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40093c0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c4b61700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0064b20 tx=0x7fd8b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40117b0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4011930 con 0x7fd8c00ccb50 2023-10-06T11:33:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c00c8360 con 0x7fd8c00ccb50 2023-10-06T11:33:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4011cf0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.124+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00c8920 con 0x7fd8c00ccb50 2023-10-06T11:33:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.128+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c00660f0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.128+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd8b4023070 con 0x7fd8c00ccb50 2023-10-06T11:33:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.132+0000 7fd8bcff9700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 0x7fd8a4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.132+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fd8b407e1a0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.132+0000 7fd8bffff700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 0x7fd8a4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.132+0000 7fd8bffff700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 0x7fd8a4062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001620 tx=0x7fd8ac0014c0 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.136+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8b404e6b0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.312+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8c004b5c0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8bcff9700 1 -- 172.21.15.46:0/1686447533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+14321 (secure 0 0 0) 0x7fd8b4051cd0 con 0x7fd8c00ccb50 2023-10-06T11:33:19.328 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:19.328 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:18.474605+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 msgr2=0x7fd8a4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 0x7fd8a4062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac001620 tx=0x7fd8ac0014c0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 msgr2=0x7fd8c014cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0064b20 tx=0x7fd8b4015000 comp rx=0 tx=0).stop 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 shutdown_connections 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a4060670 0x7fd8a4062b20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015c770 0x7fd8c009b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00cec00 0x7fd8c00c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 --2- 172.21.15.46:0/1686447533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c00ccb50 0x7fd8c014cc90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.316+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 >> 172.21.15.46:0/1686447533 conn(0x7fd8c0099480 msgr2=0x7fd8c00c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.320+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 shutdown_connections 2023-10-06T11:33:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.320+0000 7fd8c6dc5700 1 -- 172.21.15.46:0/1686447533 wait complete. 2023-10-06T11:33:19.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:33:19.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:33:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.512+0000 7fa20482d700 1 Processor -- start 2023-10-06T11:33:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.512+0000 7fa20482d700 1 -- start start 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa20482d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2000b14d0 0x7fa2000adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa20482d700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000afd20 con 0x7fa2000adbe0 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa20482d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000b0000 con 0x7fa2000b30f0 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa20482d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa200159740 con 0x7fa2000b14d0 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1fed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2000adbe0 0x7fa2000d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35042/0 (socket says 172.21.15.46:35042) 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1fed9d700 1 -- 172.21.15.46:0/3498422654 learned_addr learned my addr 172.21.15.46:0/3498422654 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1ff59e700 1 --2- 172.21.15.46:0/3498422654 >> v2:172.21.15.46:3300/0 conn(0x7fa2000b14d0 0x7fa2000adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629348495 0 0) 0x7fa2000afd20 con 0x7fa2000adbe0 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e8003570 con 0x7fa2000adbe0 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1ff59e700 1 -- 172.21.15.46:0/3498422654 >> v1:172.21.15.132:6789/0 conn(0x7fa2000b30f0 legacy=0x7fa2000b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409400324 0 0) 0x7fa2000b0000 con 0x7fa2000b30f0 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1ff59e700 1 -- 172.21.15.46:0/3498422654 >> v1:172.21.15.46:6789/0 conn(0x7fa2000adbe0 legacy=0x7fa2000d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1ff59e700 1 -- 172.21.15.46:0/3498422654 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000b02e0 con 0x7fa2000b14d0 2023-10-06T11:33:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.516+0000 7fa1ff59e700 1 --2- 172.21.15.46:0/3498422654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000b14d0 0x7fa2000adfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009e60 tx=0x7fa1f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369ddcb49192e4ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800e070 con 0x7fa2000b14d0 2023-10-06T11:33:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800bb60 con 0x7fa2000b14d0 2023-10-06T11:33:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa1fdd9b700 1 -- 172.21.15.46:0/3498422654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8012430 con 0x7fa2000b14d0 2023-10-06T11:33:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 -- 172.21.15.46:0/3498422654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000b14d0 msgr2=0x7fa2000adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 --2- 172.21.15.46:0/3498422654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000b14d0 0x7fa2000adfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009e60 tx=0x7fa1f8009780 comp rx=0 tx=0).stop 2023-10-06T11:33:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 -- 172.21.15.46:0/3498422654 shutdown_connections 2023-10-06T11:33:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 --2- 172.21.15.46:0/3498422654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000b14d0 0x7fa2000adfb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 -- 172.21.15.46:0/3498422654 >> 172.21.15.46:0/3498422654 conn(0x7fa20006ef60 msgr2=0x7fa2000d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 -- 172.21.15.46:0/3498422654 shutdown_connections 2023-10-06T11:33:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.520+0000 7fa20482d700 1 -- 172.21.15.46:0/3498422654 wait complete. 2023-10-06T11:33:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 Processor -- start 2023-10-06T11:33:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- start start 2023-10-06T11:33:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2000b14d0 0x7fa20016a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2000b30f0 0x7fa200150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015bf10 con 0x7fa2000adbe0 2023-10-06T11:33:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015a360 con 0x7fa2000b14d0 2023-10-06T11:33:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa200159a30 con 0x7fa2000b30f0 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51468/0 (socket says 172.21.15.46:51468) 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 -- 172.21.15.46:0/3167808354 learned_addr learned my addr 172.21.15.46:0/3167808354 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 -- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2000b30f0 msgr2=0x7fa200150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2000b30f0 0x7fa200150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 -- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2000b14d0 msgr2=0x7fa20016a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2000b14d0 0x7fa20016a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 -- 172.21.15.46:0/3167808354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f8009a20 con 0x7fa2000adbe0 2023-10-06T11:33:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1fed9d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00011e0 tx=0x7fa1f0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0005b30 con 0x7fa2000adbe0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0005c90 con 0x7fa2000adbe0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa200159d10 con 0x7fa2000adbe0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0007320 con 0x7fa2000adbe0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.524+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20016a6e0 con 0x7fa2000adbe0 2023-10-06T11:33:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.528+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20015a190 con 0x7fa2000adbe0 2023-10-06T11:33:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.528+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa1f0007690 con 0x7fa2000adbe0 2023-10-06T11:33:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.528+0000 7fa1e6ffd700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 0x7fa1ec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.528+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fa1f007be90 con 0x7fa2000adbe0 2023-10-06T11:33:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.532+0000 7fa1fe59c700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 0x7fa1ec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.532+0000 7fa1fe59c700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 0x7fa1ec062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2001558d0 tx=0x7fa1f40079b0 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.540+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa1f004c300 con 0x7fa2000adbe0 2023-10-06T11:33:19.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.704+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa200064590 con 0x7fa2000adbe0 2023-10-06T11:33:19.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa1e6ffd700 1 -- 172.21.15.46:0/3167808354 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1f00170a0 con 0x7fa2000adbe0 2023-10-06T11:33:19.719 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 msgr2=0x7fa1ec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 0x7fa1ec062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2001558d0 tx=0x7fa1f40079b0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 msgr2=0x7fa20015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00011e0 tx=0x7fa1f0007df0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 shutdown_connections 2023-10-06T11:33:19.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa1ec060670 0x7fa1ec062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2000b30f0 0x7fa200150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2000b14d0 0x7fa20016a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 --2- 172.21.15.46:0/3167808354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2000adbe0 0x7fa20015b9d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 >> 172.21.15.46:0/3167808354 conn(0x7fa20006ef60 msgr2=0x7fa20014e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.708+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 shutdown_connections 2023-10-06T11:33:19.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.712+0000 7fa20482d700 1 -- 172.21.15.46:0/3167808354 wait complete. 2023-10-06T11:33:19.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:33:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.900+0000 7f96a628e700 1 Processor -- start 2023-10-06T11:33:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.900+0000 7f96a628e700 1 -- start start 2023-10-06T11:33:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a628e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96a0156c00 0x7f96a00ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a628e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00c97e0 con 0x7f96a00c3ce0 2023-10-06T11:33:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a628e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00c9ac0 con 0x7f96a0156fd0 2023-10-06T11:33:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a628e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00c9da0 con 0x7f96a0156c00 2023-10-06T11:33:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a5a8d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f96a00c3ce0 0x7f96a0160a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35046/0 (socket says 172.21.15.46:35046) 2023-10-06T11:33:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a5a8d700 1 -- 172.21.15.46:0/501794850 learned_addr learned my addr 172.21.15.46:0/501794850 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a528c700 1 --2- 172.21.15.46:0/501794850 >> v2:172.21.15.132:3300/0 conn(0x7f96a0156c00 0x7f96a00ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f9697fff700 1 -- 172.21.15.46:0/501794850 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513106491 0 0) 0x7f96a00c97e0 con 0x7f96a00c3ce0 2023-10-06T11:33:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f9697fff700 1 -- 172.21.15.46:0/501794850 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f968c003570 con 0x7f96a00c3ce0 2023-10-06T11:33:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a528c700 1 -- 172.21.15.46:0/501794850 >> v1:172.21.15.132:6789/0 conn(0x7f96a0156fd0 legacy=0x7f96a00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a528c700 1 -- 172.21.15.46:0/501794850 >> v1:172.21.15.46:6789/0 conn(0x7f96a00c3ce0 legacy=0x7f96a0160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.904+0000 7f96a528c700 1 -- 172.21.15.46:0/501794850 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a00ca080 con 0x7f96a0156c00 2023-10-06T11:33:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a528c700 1 --2- 172.21.15.46:0/501794850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00ca810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f969c0098e0 tx=0x7f969c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcda0986a9e9a4ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f9697fff700 1 -- 172.21.15.46:0/501794850 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00b640 con 0x7f96a0156c00 2023-10-06T11:33:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f9697fff700 1 -- 172.21.15.46:0/501794850 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c00b7a0 con 0x7f96a0156c00 2023-10-06T11:33:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f9697fff700 1 -- 172.21.15.46:0/501794850 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c00bb40 con 0x7f96a0156c00 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 -- 172.21.15.46:0/501794850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 msgr2=0x7f96a00ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 --2- 172.21.15.46:0/501794850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00ca810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f969c0098e0 tx=0x7f969c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 -- 172.21.15.46:0/501794850 shutdown_connections 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 --2- 172.21.15.46:0/501794850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00ca810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 -- 172.21.15.46:0/501794850 >> 172.21.15.46:0/501794850 conn(0x7f96a006f1c0 msgr2=0x7f96a0097140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 -- 172.21.15.46:0/501794850 shutdown_connections 2023-10-06T11:33:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.908+0000 7f96a628e700 1 -- 172.21.15.46:0/501794850 wait complete. 2023-10-06T11:33:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 Processor -- start 2023-10-06T11:33:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 -- start start 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a00c3ce0 0x7f96a00c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c6f70 con 0x7f96a0156fd0 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c6360 con 0x7f96a0156c00 2023-10-06T11:33:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a628e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c6640 con 0x7f96a00c3ce0 2023-10-06T11:33:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51484/0 (socket says 172.21.15.46:51484) 2023-10-06T11:33:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a528c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a00c3ce0 0x7f96a00c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 -- 172.21.15.46:0/1715512440 learned_addr learned my addr 172.21.15.46:0/1715512440 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a4a8b700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 -- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a00c3ce0 msgr2=0x7f96a00c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a00c3ce0 0x7f96a00c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 -- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 msgr2=0x7f96a00b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.912+0000 7f96a5a8d700 1 -- 172.21.15.46:0/1715512440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c009420 con 0x7f96a0156fd0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96a5a8d700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9698001460 tx=0x7f9698001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800a890 con 0x7f96a0156fd0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969800a9f0 con 0x7f96a0156fd0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a00c6920 con 0x7f96a0156fd0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800ad70 con 0x7f96a0156fd0 2023-10-06T11:33:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a00c5f80 con 0x7f96a0156fd0 2023-10-06T11:33:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a00c6da0 con 0x7f96a0156fd0 2023-10-06T11:33:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.916+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9698007300 con 0x7f96a0156fd0 2023-10-06T11:33:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.920+0000 7f96957fa700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 0x7f9684062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.920+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f969807cd00 con 0x7f96a0156fd0 2023-10-06T11:33:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.920+0000 7f96a528c700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 0x7f9684062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.920+0000 7f96a528c700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 0x7f9684062b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f969c0093f0 tx=0x7f969c017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:19.924+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f969804d180 con 0x7f96a0156fd0 2023-10-06T11:33:20.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.096+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96a0064590 con 0x7f96a0156fd0 2023-10-06T11:33:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.100+0000 7f96957fa700 1 -- 172.21.15.46:0/1715512440 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f96980507a0 con 0x7f96a0156fd0 2023-10-06T11:33:20.112 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:33:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.100+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 msgr2=0x7f9684062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.100+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 0x7f9684062b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f969c0093f0 tx=0x7f969c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.100+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 msgr2=0x7f96a006ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.100+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9698001460 tx=0x7f9698001ae0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 shutdown_connections 2023-10-06T11:33:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96840606a0 0x7f9684062b50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96a0156fd0 0x7f96a006ad20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96a0156c00 0x7f96a00b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 --2- 172.21.15.46:0/1715512440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96a00c3ce0 0x7f96a00c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 >> 172.21.15.46:0/1715512440 conn(0x7f96a006f1c0 msgr2=0x7f96a014bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 shutdown_connections 2023-10-06T11:33:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.104+0000 7f96a628e700 1 -- 172.21.15.46:0/1715512440 wait complete. 2023-10-06T11:33:20.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:33:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.280+0000 7f1af759e700 1 Processor -- start 2023-10-06T11:33:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af759e700 1 -- start start 2023-10-06T11:33:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 0x7f1af806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af806d790 con 0x7f1af8067760 2023-10-06T11:33:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af80acd60 con 0x7f1af80aaf30 2023-10-06T11:33:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80ad040 con 0x7f1af8156e20 2023-10-06T11:33:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af6d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1af8067760 0x7f1af8068a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35060/0 (socket says 172.21.15.46:35060) 2023-10-06T11:33:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.284+0000 7f1af6d9d700 1 -- 172.21.15.46:0/753125133 learned_addr learned my addr 172.21.15.46:0/753125133 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af659c700 1 --2- 172.21.15.46:0/753125133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 0x7f1af806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014506601 0 0) 0x7f1af806d790 con 0x7f1af8067760 2023-10-06T11:33:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae4003570 con 0x7f1af8067760 2023-10-06T11:33:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980349548 0 0) 0x7f1ae4003570 con 0x7f1af8067760 2023-10-06T11:33:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af659c700 1 -- 172.21.15.46:0/753125133 >> v1:172.21.15.132:6789/0 conn(0x7f1af80aaf30 legacy=0x7f1af8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af659c700 1 -- 172.21.15.46:0/753125133 >> v1:172.21.15.46:6789/0 conn(0x7f1af8067760 legacy=0x7f1af8068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af659c700 1 -- 172.21.15.46:0/753125133 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80ad320 con 0x7f1af8156e20 2023-10-06T11:33:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af659c700 1 --2- 172.21.15.46:0/753125133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 0x7f1af806d250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009910 tx=0x7f1ae80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc849d84daf36602 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af00017f0 con 0x7f1af8156e20 2023-10-06T11:33:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8011420 con 0x7f1af8156e20 2023-10-06T11:33:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af5d9b700 1 -- 172.21.15.46:0/753125133 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80117c0 con 0x7f1af8156e20 2023-10-06T11:33:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 -- 172.21.15.46:0/753125133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 msgr2=0x7f1af806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 --2- 172.21.15.46:0/753125133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 0x7f1af806d250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009910 tx=0x7f1ae80091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 -- 172.21.15.46:0/753125133 shutdown_connections 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 --2- 172.21.15.46:0/753125133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8156e20 0x7f1af806d250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 -- 172.21.15.46:0/753125133 >> 172.21.15.46:0/753125133 conn(0x7f1af8062ef0 msgr2=0x7f1af80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 -- 172.21.15.46:0/753125133 shutdown_connections 2023-10-06T11:33:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.288+0000 7f1af759e700 1 -- 172.21.15.46:0/753125133 wait complete. 2023-10-06T11:33:20.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 Processor -- start 2023-10-06T11:33:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 -- start start 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8067760 0x7f1af80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 0x7f1af80afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815a840 con 0x7f1af80aaf30 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80c3780 con 0x7f1af8156e20 2023-10-06T11:33:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815a560 con 0x7f1af8067760 2023-10-06T11:33:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51486/0 (socket says 172.21.15.46:51486) 2023-10-06T11:33:20.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1aeffff700 1 -- 172.21.15.46:0/2566603184 learned_addr learned my addr 172.21.15.46:0/2566603184 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.292+0000 7f1af6d9d700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 0x7f1af80afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1af659c700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8067760 0x7f1af80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 -- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8067760 msgr2=0x7f1af80b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8067760 0x7f1af80b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 -- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 msgr2=0x7f1af80afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 0x7f1af80afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 -- 172.21.15.46:0/2566603184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8009420 con 0x7f1af80aaf30 2023-10-06T11:33:20.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1af6d9d700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 0x7f1af80afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aeffff700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c7e0 tx=0x7f1ae0007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011660 con 0x7f1af80aaf30 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00117c0 con 0x7f1af80aaf30 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80c3130 con 0x7f1af80aaf30 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011b80 con 0x7f1af80aaf30 2023-10-06T11:33:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80b12e0 con 0x7f1af80aaf30 2023-10-06T11:33:20.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.296+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af80c2f90 con 0x7f1af80aaf30 2023-10-06T11:33:20.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.300+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1ae001b3b0 con 0x7f1af80aaf30 2023-10-06T11:33:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.300+0000 7f1aee7fc700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 0x7f1ad8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.300+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f1ae007bfe0 con 0x7f1af80aaf30 2023-10-06T11:33:20.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.300+0000 7f1af659c700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 0x7f1ad8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.300+0000 7f1af659c700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 0x7f1ad8062b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80093f0 tx=0x7f1ae8017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.304+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1af80c2f90 con 0x7f1af80aaf30 2023-10-06T11:33:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.476+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1af80c2f90 con 0x7f1af80aaf30 2023-10-06T11:33:20.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.476+0000 7f1aee7fc700 1 -- 172.21.15.46:0/2566603184 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1af80c2f90 con 0x7f1af80aaf30 2023-10-06T11:33:20.490 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:20.491 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 msgr2=0x7f1ad8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 0x7f1ad8062b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80093f0 tx=0x7f1ae8017000 comp rx=0 tx=0).stop 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 msgr2=0x7f1af80af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c7e0 tx=0x7f1ae0007e70 comp rx=0 tx=0).stop 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 shutdown_connections 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1ad80606a0 0x7f1ad8062b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af8156e20 0x7f1af80afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af80aaf30 0x7f1af80af930 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 --2- 172.21.15.46:0/2566603184 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af8067760 0x7f1af80b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 >> 172.21.15.46:0/2566603184 conn(0x7f1af8062ef0 msgr2=0x7f1af8155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 shutdown_connections 2023-10-06T11:33:20.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.480+0000 7f1af759e700 1 -- 172.21.15.46:0/2566603184 wait complete. 2023-10-06T11:33:20.513 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:33:20.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:33:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.676+0000 7f38148e3700 1 Processor -- start 2023-10-06T11:33:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.676+0000 7f38148e3700 1 -- start start 2023-10-06T11:33:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f38148e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 0x7f38100af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f38148e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38100cc500 0x7f38100afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f38148e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38100cf360 con 0x7f38101591f0 2023-10-06T11:33:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f38148e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38100cff00 con 0x7f38100cc500 2023-10-06T11:33:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f38148e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38100b03c0 con 0x7f38101595c0 2023-10-06T11:33:20.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38100cc500 0x7f38100afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38100cc500 0x7f38100afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51502/0 (socket says 172.21.15.46:51502) 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 -- 172.21.15.46:0/2869308849 learned_addr learned my addr 172.21.15.46:0/2869308849 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380effd700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 0x7f38100af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 -- 172.21.15.46:0/2869308849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 msgr2=0x7f38100af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 0x7f38100af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 -- 172.21.15.46:0/2869308849 >> v1:172.21.15.132:6789/0 conn(0x7f38101591f0 legacy=0x7f38100cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 -- 172.21.15.46:0/2869308849 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381015dd60 con 0x7f38100cc500 2023-10-06T11:33:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380effd700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 0x7f38100af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380ffff700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f38100afe80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3808009f20 tx=0x7f3808009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb0e28b4d0f930b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380e7fc700 1 -- 172.21.15.46:0/2869308849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800e070 con 0x7f38100cc500 2023-10-06T11:33:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380e7fc700 1 -- 172.21.15.46:0/2869308849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800bb10 con 0x7f38100cc500 2023-10-06T11:33:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.680+0000 7f380e7fc700 1 -- 172.21.15.46:0/2869308849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080092e0 con 0x7f38100cc500 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- 172.21.15.46:0/2869308849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 msgr2=0x7f38100afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f38100afe80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3808009f20 tx=0x7f3808009840 comp rx=0 tx=0).stop 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- 172.21.15.46:0/2869308849 shutdown_connections 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f38100afe80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 --2- 172.21.15.46:0/2869308849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101595c0 0x7f38100af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- 172.21.15.46:0/2869308849 >> 172.21.15.46:0/2869308849 conn(0x7f381006ef60 msgr2=0x7f38100d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- 172.21.15.46:0/2869308849 shutdown_connections 2023-10-06T11:33:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- 172.21.15.46:0/2869308849 wait complete. 2023-10-06T11:33:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 Processor -- start 2023-10-06T11:33:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 -- start start 2023-10-06T11:33:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.684+0000 7f38148e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101591f0 0x7f38100b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3810145ae0 0x7f381015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3810156890 con 0x7f38100cc500 2023-10-06T11:33:20.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38100b3d90 con 0x7f3810145ae0 2023-10-06T11:33:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38100b3ff0 con 0x7f38101591f0 2023-10-06T11:33:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51518/0 (socket says 172.21.15.46:51518) 2023-10-06T11:33:20.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 -- 172.21.15.46:0/78313767 learned_addr learned my addr 172.21.15.46:0/78313767 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380effd700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101591f0 0x7f38100b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380ffff700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3810145ae0 0x7f381015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 -- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101591f0 msgr2=0x7f38100b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101591f0 0x7f38100b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 -- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3810145ae0 msgr2=0x7f381015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3810145ae0 0x7f381015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 -- 172.21.15.46:0/78313767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808009ae0 con 0x7f38100cc500 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f380f7fe700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f380000c4e0 tx=0x7f380000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800011590 con 0x7f38100cc500 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800011710 con 0x7f38100cc500 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800011ad0 con 0x7f38100cc500 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38100b42d0 con 0x7f38100cc500 2023-10-06T11:33:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.688+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3810156dc0 con 0x7f38100cc500 2023-10-06T11:33:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3810156fe0 con 0x7f38100cc500 2023-10-06T11:33:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3800001020 con 0x7f38100cc500 2023-10-06T11:33:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f37f77fe700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 0x7f37fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f380007be10 con 0x7f38100cc500 2023-10-06T11:33:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f380effd700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 0x7f37fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:20.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.692+0000 7f380effd700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 0x7f37fc062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3810069d70 tx=0x7f3804006f90 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.700+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f380007f030 con 0x7f38100cc500 2023-10-06T11:33:20.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:20.872+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3810064590 con 0x7f38100cc500 2023-10-06T11:33:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.560+0000 7f37f77fe700 1 -- 172.21.15.46:0/78313767 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3800017020 con 0x7f38100cc500 2023-10-06T11:33:21.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 msgr2=0x7f37fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 0x7f37fc062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3810069d70 tx=0x7f3804006f90 comp rx=0 tx=0).stop 2023-10-06T11:33:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 msgr2=0x7f3810156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f380000c4e0 tx=0x7f380000c8a0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 shutdown_connections 2023-10-06T11:33:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37fc060670 0x7f37fc062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3810145ae0 0x7f381015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38101591f0 0x7f38100b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 --2- 172.21.15.46:0/78313767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38100cc500 0x7f3810156280 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 >> 172.21.15.46:0/78313767 conn(0x7f381006ef60 msgr2=0x7f38100af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 shutdown_connections 2023-10-06T11:33:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.564+0000 7f38148e3700 1 -- 172.21.15.46:0/78313767 wait complete. 2023-10-06T11:33:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:33:21.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:33:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.760+0000 7f7d52a8e700 1 Processor -- start 2023-10-06T11:33:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.760+0000 7f7d52a8e700 1 -- start start 2023-10-06T11:33:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d52a8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 0x7f7d4c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d52a8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d4c0cf4e0 0x7f7d4c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d52a8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4c0cf050 con 0x7f7d4c0cb020 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d52a8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4c0cf8f0 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d52a8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4c0ce440 con 0x7f7d4c0d0750 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d51a8c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7d4c0cb020 0x7f7d4c0cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35066/0 (socket says 172.21.15.46:35066) 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d51a8c700 1 -- 172.21.15.46:0/1449358749 learned_addr learned my addr 172.21.15.46:0/1449358749 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d5128b700 1 --2- 172.21.15.46:0/1449358749 >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 0x7f7d4c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.764+0000 7f7d5228d700 1 --2- 172.21.15.46:0/1449358749 >> v2:172.21.15.46:3300/0 conn(0x7f7d4c0cf4e0 0x7f7d4c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5228d700 1 -- 172.21.15.46:0/1449358749 >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 msgr2=0x7f7d4c0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5228d700 1 --2- 172.21.15.46:0/1449358749 >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 0x7f7d4c0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5228d700 1 -- 172.21.15.46:0/1449358749 >> v1:172.21.15.46:6789/0 conn(0x7f7d4c0cb020 legacy=0x7f7d4c0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d50a8a700 1 -- 172.21.15.46:0/1449358749 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697036961 0 0) 0x7f7d4c0cf050 con 0x7f7d4c0cb020 2023-10-06T11:33:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5228d700 1 -- 172.21.15.46:0/1449358749 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c0ce720 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5128b700 1 --2- 172.21.15.46:0/1449358749 >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 0x7f7d4c0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:21.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d5228d700 1 --2- 172.21.15.46:0/1449358749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c158bd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c088f20 tx=0x7f7d44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef55db539e10e845 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d50a8a700 1 -- 172.21.15.46:0/1449358749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400e070 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d50a8a700 1 -- 172.21.15.46:0/1449358749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4400bb60 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d50a8a700 1 -- 172.21.15.46:0/1449358749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44012430 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 -- 172.21.15.46:0/1449358749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 msgr2=0x7f7d4c158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/1449358749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c158bd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c088f20 tx=0x7f7d44009780 comp rx=0 tx=0).stop 2023-10-06T11:33:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 -- 172.21.15.46:0/1449358749 shutdown_connections 2023-10-06T11:33:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/1449358749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c158bd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/1449358749 >> v2:172.21.15.132:3300/0 conn(0x7f7d4c0d0750 0x7f7d4c0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 -- 172.21.15.46:0/1449358749 >> 172.21.15.46:0/1449358749 conn(0x7f7d4c0683d0 msgr2=0x7f7d4c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:21.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 -- 172.21.15.46:0/1449358749 shutdown_connections 2023-10-06T11:33:21.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.768+0000 7f7d52a8e700 1 -- 172.21.15.46:0/1449358749 wait complete. 2023-10-06T11:33:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 Processor -- start 2023-10-06T11:33:21.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 -- start start 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d4c0cb020 0x7f7d4c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d4c0d0750 0x7f7d4c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c144fd0 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c15cd30 con 0x7f7d4c0cb020 2023-10-06T11:33:21.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d52a8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c15b7c0 con 0x7f7d4c0d0750 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d5128b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d5128b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51532/0 (socket says 172.21.15.46:51532) 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d5128b700 1 -- 172.21.15.46:0/6710135 learned_addr learned my addr 172.21.15.46:0/6710135 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d51a8c700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d4c0cb020 0x7f7d4c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.772+0000 7f7d5228d700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d4c0d0750 0x7f7d4c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 -- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d4c0d0750 msgr2=0x7f7d4c159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d4c0d0750 0x7f7d4c159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 -- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d4c0cb020 msgr2=0x7f7d4c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d4c0cb020 0x7f7d4c155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 -- 172.21.15.46:0/6710135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c009420 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d5128b700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00ec80 tx=0x7f7d3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00c850 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c00c9b0 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00cd30 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d44009a20 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c155e10 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.776+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4c064590 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.780+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7d3c013680 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.780+0000 7f7d41ffb700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 0x7f7d30062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.780+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6020+0+0 (secure 0 0 0) 0x7f7d3c07d070 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.780+0000 7f7d51a8c700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 0x7f7d30062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:21.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.784+0000 7f7d51a8c700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 0x7f7d30062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d48007fd0 tx=0x7f7d48007d30 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.788+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d3c0acbe0 con 0x7f7d4c0cf4e0 2023-10-06T11:33:21.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:21.960+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7d4c0cea00 con 0x7f7d4c0cf4e0 2023-10-06T11:33:22.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.576+0000 7f7d41ffb700 1 -- 172.21.15.46:0/6710135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7d3c050b70 con 0x7f7d4c0cf4e0 2023-10-06T11:33:22.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 msgr2=0x7f7d30062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 0x7f7d30062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d48007fd0 tx=0x7f7d48007d30 comp rx=0 tx=0).stop 2023-10-06T11:33:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 msgr2=0x7f7d4c144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00ec80 tx=0x7f7d3c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 shutdown_connections 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d30060670 0x7f7d30062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d4c0d0750 0x7f7d4c159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d4c0cf4e0 0x7f7d4c144a90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 --2- 172.21.15.46:0/6710135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d4c0cb020 0x7f7d4c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.584+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 >> 172.21.15.46:0/6710135 conn(0x7f7d4c0683d0 msgr2=0x7f7d4c157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:22.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.588+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 shutdown_connections 2023-10-06T11:33:22.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.588+0000 7f7d52a8e700 1 -- 172.21.15.46:0/6710135 wait complete. 2023-10-06T11:33:22.608 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:33:22.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:33:22.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 Processor -- start 2023-10-06T11:33:22.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 -- start start 2023-10-06T11:33:22.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 0x7ff21806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180c6cf0 con 0x7ff2180c7f20 2023-10-06T11:33:22.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180c2a30 con 0x7ff218154c50 2023-10-06T11:33:22.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21e163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c83e0 con 0x7ff218154880 2023-10-06T11:33:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.784+0000 7ff21c960700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 0x7ff21806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21c960700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 0x7ff21806c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51536/0 (socket says 172.21.15.46:51536) 2023-10-06T11:33:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21c960700 1 -- 172.21.15.46:0/3286236032 learned_addr learned my addr 172.21.15.46:0/3286236032 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:22.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d962700 1 --1- 172.21.15.46:0/3286236032 >> v1:172.21.15.132:6789/0 conn(0x7ff2180c7f20 0x7ff21815aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34562/0 (socket says 172.21.15.46:34562) 2023-10-06T11:33:22.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 -- 172.21.15.46:0/3286236032 >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 msgr2=0x7ff21806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:22.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 --2- 172.21.15.46:0/3286236032 >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 0x7ff21806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 -- 172.21.15.46:0/3286236032 >> v1:172.21.15.132:6789/0 conn(0x7ff2180c7f20 legacy=0x7ff21815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 -- 172.21.15.46:0/3286236032 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21806e5d0 con 0x7ff218154880 2023-10-06T11:33:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21d161700 1 --2- 172.21.15.46:0/3286236032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c7180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff214009e60 tx=0x7ff214009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf3b64f1ec1381c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:22.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff20ffff700 1 -- 172.21.15.46:0/3286236032 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400e070 con 0x7ff218154880 2023-10-06T11:33:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff20ffff700 1 -- 172.21.15.46:0/3286236032 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21400bb60 con 0x7ff218154880 2023-10-06T11:33:22.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff20ffff700 1 -- 172.21.15.46:0/3286236032 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214012430 con 0x7ff218154880 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 -- 172.21.15.46:0/3286236032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 msgr2=0x7ff2180c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 --2- 172.21.15.46:0/3286236032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c7180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff214009e60 tx=0x7ff214009780 comp rx=0 tx=0).stop 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 -- 172.21.15.46:0/3286236032 shutdown_connections 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 --2- 172.21.15.46:0/3286236032 >> v2:172.21.15.46:3300/0 conn(0x7ff218154c50 0x7ff21806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 --2- 172.21.15.46:0/3286236032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c7180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.788+0000 7ff21e163700 1 -- 172.21.15.46:0/3286236032 >> 172.21.15.46:0/3286236032 conn(0x7ff2180cbc10 msgr2=0x7ff2180cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.792+0000 7ff21e163700 1 -- 172.21.15.46:0/3286236032 shutdown_connections 2023-10-06T11:33:22.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.792+0000 7ff21e163700 1 -- 172.21.15.46:0/3286236032 wait complete. 2023-10-06T11:33:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.792+0000 7ff21e163700 1 Processor -- start 2023-10-06T11:33:22.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.792+0000 7ff21e163700 1 -- start start 2023-10-06T11:33:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 0x7ff218157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218154c50 0x7ff218155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c3cf0 con 0x7ff2180c7f20 2023-10-06T11:33:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c3f20 con 0x7ff218154c50 2023-10-06T11:33:22.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c4200 con 0x7ff218154880 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21d161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 0x7ff218157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21d161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 0x7ff218157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51544/0 (socket says 172.21.15.46:51544) 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21d161700 1 -- 172.21.15.46:0/1505436852 learned_addr learned my addr 172.21.15.46:0/1505436852 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21d962700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218154c50 0x7ff218155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 -- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218154c50 msgr2=0x7ff218155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218154c50 0x7ff218155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 -- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 msgr2=0x7ff218157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 0x7ff218157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 -- 172.21.15.46:0/1505436852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214009a20 con 0x7ff218154880 2023-10-06T11:33:22.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21c960700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c37b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff208006fd0 tx=0x7ff208009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208013680 con 0x7ff218154880 2023-10-06T11:33:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2080137e0 con 0x7ff218154880 2023-10-06T11:33:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2180c44e0 con 0x7ff218154880 2023-10-06T11:33:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208013b60 con 0x7ff218154880 2023-10-06T11:33:22.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.796+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2180c4aa0 con 0x7ff218154880 2023-10-06T11:33:22.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.800+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff218064590 con 0x7ff218154880 2023-10-06T11:33:22.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.800+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff2080123d0 con 0x7ff218154880 2023-10-06T11:33:22.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.804+0000 7ff20d7fa700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 0x7ff1fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.804+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6402+0+0 (secure 0 0 0) 0x7ff20807cc30 con 0x7ff218154880 2023-10-06T11:33:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.804+0000 7ff21d161700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 0x7ff1fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.804+0000 7ff21d161700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 0x7ff1fc062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2140097c0 tx=0x7ff214019000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:22.812+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff20804cdb0 con 0x7ff218154880 2023-10-06T11:33:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.028+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff21806e8b0 con 0x7ff218154880 2023-10-06T11:33:23.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T11:33:23.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.612+0000 7ff20d7fa700 1 -- 172.21.15.46:0/1505436852 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff2080170c0 con 0x7ff218154880 2023-10-06T11:33:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 msgr2=0x7ff1fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 0x7ff1fc062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2140097c0 tx=0x7ff214019000 comp rx=0 tx=0).stop 2023-10-06T11:33:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 msgr2=0x7ff2180c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c37b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff208006fd0 tx=0x7ff208009250 comp rx=0 tx=0).stop 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 shutdown_connections 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1fc060670 0x7ff1fc062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218154c50 0x7ff218155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff218154880 0x7ff2180c37b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 --2- 172.21.15.46:0/1505436852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2180c7f20 0x7ff218157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 >> 172.21.15.46:0/1505436852 conn(0x7ff2180cbc10 msgr2=0x7ff2180c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 shutdown_connections 2023-10-06T11:33:23.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.620+0000 7ff21e163700 1 -- 172.21.15.46:0/1505436852 wait complete. 2023-10-06T11:33:23.635 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T11:33:23.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 Processor -- start 2023-10-06T11:33:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 -- start start 2023-10-06T11:33:23.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99c80bf1a0 0x7f99c8067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c8069660 con 0x7f99c80c7a00 2023-10-06T11:33:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c8068ac0 con 0x7f99c80bf1a0 2023-10-06T11:33:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.800+0000 7f99cefcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8068d30 con 0x7f99c80bedd0 2023-10-06T11:33:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ce7cb700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f99c80c7a00 0x7f99c80c7ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42312/0 (socket says 172.21.15.46:42312) 2023-10-06T11:33:23.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:59466/0 (socket says 172.21.15.46:59466) 2023-10-06T11:33:23.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ce7cb700 1 -- 172.21.15.46:0/817081469 learned_addr learned my addr 172.21.15.46:0/817081469 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:23.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cd7c9700 1 --2- 172.21.15.46:0/817081469 >> v2:172.21.15.46:3300/0 conn(0x7f99c80bf1a0 0x7f99c8067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 -- 172.21.15.46:0/817081469 >> v2:172.21.15.46:3300/0 conn(0x7f99c80bf1a0 msgr2=0x7f99c8067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:23.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 --2- 172.21.15.46:0/817081469 >> v2:172.21.15.46:3300/0 conn(0x7f99c80bf1a0 0x7f99c8067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ccfc8700 1 -- 172.21.15.46:0/817081469 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299854900 0 0) 0x7f99c8069660 con 0x7f99c80c7a00 2023-10-06T11:33:23.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 -- 172.21.15.46:0/817081469 >> v1:172.21.15.132:6789/0 conn(0x7f99c80c7a00 legacy=0x7f99c80c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 -- 172.21.15.46:0/817081469 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c8069010 con 0x7f99c80bedd0 2023-10-06T11:33:23.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99cdfca700 1 --2- 172.21.15.46:0/817081469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098e0 tx=0x7f99c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bae6120fc1cb70e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ccfc8700 1 -- 172.21.15.46:0/817081469 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400b640 con 0x7f99c80bedd0 2023-10-06T11:33:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ccfc8700 1 -- 172.21.15.46:0/817081469 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400b7a0 con 0x7f99c80bedd0 2023-10-06T11:33:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.804+0000 7f99ccfc8700 1 -- 172.21.15.46:0/817081469 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400bb40 con 0x7f99c80bedd0 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 -- 172.21.15.46:0/817081469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 msgr2=0x7f99c8067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 --2- 172.21.15.46:0/817081469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098e0 tx=0x7f99c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 -- 172.21.15.46:0/817081469 shutdown_connections 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 --2- 172.21.15.46:0/817081469 >> v2:172.21.15.46:3300/0 conn(0x7f99c80bf1a0 0x7f99c8067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 --2- 172.21.15.46:0/817081469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c8067050 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 -- 172.21.15.46:0/817081469 >> 172.21.15.46:0/817081469 conn(0x7f99c8062ef0 msgr2=0x7f99c80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:23.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 -- 172.21.15.46:0/817081469 shutdown_connections 2023-10-06T11:33:23.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 -- 172.21.15.46:0/817081469 wait complete. 2023-10-06T11:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.808+0000 7f99cefcc700 1 Processor -- start 2023-10-06T11:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 -- start start 2023-10-06T11:33:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 0x7f99c80c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80c60c0 con 0x7f99c80c7a00 2023-10-06T11:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c816d9c0 con 0x7f99c80bf1a0 2023-10-06T11:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cefcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c816dca0 con 0x7f99c80bedd0 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40326/0 (socket says 172.21.15.46:40326) 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 -- 172.21.15.46:0/1400448424 learned_addr learned my addr 172.21.15.46:0/1400448424 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cdfca700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cd7c9700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 0x7f99c80c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 -- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 msgr2=0x7f99c80c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c80c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 -- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 msgr2=0x7f99c80c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 0x7f99c80c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 -- 172.21.15.46:0/1400448424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009420 con 0x7f99c80c7a00 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99cd7c9700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 0x7f99c80c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99ce7cb700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f99c000cc90 tx=0x7f99c0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.812+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000a960 con 0x7f99c80c7a00 2023-10-06T11:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.816+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c000aac0 con 0x7f99c80c7a00 2023-10-06T11:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.816+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000ae80 con 0x7f99c80c7a00 2023-10-06T11:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.816+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c816df80 con 0x7f99c80c7a00 2023-10-06T11:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.816+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c816e540 con 0x7f99c80c7a00 2023-10-06T11:33:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.816+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c8064590 con 0x7f99c80c7a00 2023-10-06T11:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.820+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f99c0007300 con 0x7f99c80c7a00 2023-10-06T11:33:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.820+0000 7f99bdffb700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 0x7f99ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.820+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f99c007d150 con 0x7f99c80c7a00 2023-10-06T11:33:23.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.820+0000 7f99cdfca700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 0x7f99ac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.820+0000 7f99cdfca700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 0x7f99ac062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99c40093f0 tx=0x7f99c4017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:23.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:23.828+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99c004d2f0 con 0x7f99c80c7a00 2023-10-06T11:33:24.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.000+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c804b5c0 con 0x7f99c80c7a00 2023-10-06T11:33:24.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.004+0000 7f99bdffb700 1 -- 172.21.15.46:0/1400448424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17552 (secure 0 0 0) 0x7f99c0050910 con 0x7f99c80c7a00 2023-10-06T11:33:24.018 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:24.018 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:23.034363+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:24.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 msgr2=0x7f99ac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 0x7f99ac062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99c40093f0 tx=0x7f99c4017000 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 msgr2=0x7f99c80ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f99c000cc90 tx=0x7f99c0001a80 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 shutdown_connections 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f99ac060670 0x7f99ac062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c80c7a00 0x7f99c80ca8e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c80bf1a0 0x7f99c80c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 --2- 172.21.15.46:0/1400448424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c80bedd0 0x7f99c80c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 >> 172.21.15.46:0/1400448424 conn(0x7f99c8062ef0 msgr2=0x7f99c815c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 shutdown_connections 2023-10-06T11:33:24.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.008+0000 7f99cefcc700 1 -- 172.21.15.46:0/1400448424 wait complete. 2023-10-06T11:33:24.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.188+0000 7f5d174e3700 1 Processor -- start 2023-10-06T11:33:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.188+0000 7f5d174e3700 1 -- start start 2023-10-06T11:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d174e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d10156e20 0x7f5d1006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d174e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 0x7f5d10067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d174e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1006d790 con 0x7f5d10067590 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d174e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d10067a10 con 0x7f5d10156e20 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d174e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100acd60 con 0x7f5d100aaf30 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d10156e20 0x7f5d1006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d10156e20 0x7f5d1006d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40330/0 (socket says 172.21.15.46:40330) 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 -- 172.21.15.46:0/3286064139 learned_addr learned my addr 172.21.15.46:0/3286064139 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d15ce0700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 0x7f5d10067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 -- 172.21.15.46:0/3286064139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 msgr2=0x7f5d10067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 0x7f5d10067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 -- 172.21.15.46:0/3286064139 >> v1:172.21.15.46:6789/0 conn(0x7f5d10067590 legacy=0x7f5d1006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 -- 172.21.15.46:0/3286064139 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100ad040 con 0x7f5d10156e20 2023-10-06T11:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d15ce0700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 0x7f5d10067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d164e1700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10156e20 0x7f5d1006d250 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0099b0 tx=0x7f5d0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bd5af4d04a0be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d154df700 1 -- 172.21.15.46:0/3286064139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00b860 con 0x7f5d10156e20 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d154df700 1 -- 172.21.15.46:0/3286064139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c00b9c0 con 0x7f5d10156e20 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.192+0000 7f5d154df700 1 -- 172.21.15.46:0/3286064139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00bd60 con 0x7f5d10156e20 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 -- 172.21.15.46:0/3286064139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10156e20 msgr2=0x7f5d1006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10156e20 0x7f5d1006d250 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0099b0 tx=0x7f5d0c009250 comp rx=0 tx=0).stop 2023-10-06T11:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 -- 172.21.15.46:0/3286064139 shutdown_connections 2023-10-06T11:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d100aaf30 0x7f5d10067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 --2- 172.21.15.46:0/3286064139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10156e20 0x7f5d1006d250 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 -- 172.21.15.46:0/3286064139 >> 172.21.15.46:0/3286064139 conn(0x7f5d10062ef0 msgr2=0x7f5d100cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 -- 172.21.15.46:0/3286064139 shutdown_connections 2023-10-06T11:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.196+0000 7f5d174e3700 1 -- 172.21.15.46:0/3286064139 wait complete. 2023-10-06T11:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 Processor -- start 2023-10-06T11:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 -- start start 2023-10-06T11:33:24.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d100aaf30 0x7f5d100b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d10156e20 0x7f5d100b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1016da10 con 0x7f5d10067590 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10157cc0 con 0x7f5d100aaf30 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d174e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10157390 con 0x7f5d10156e20 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40342/0 (socket says 172.21.15.46:40342) 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 -- 172.21.15.46:0/4180262111 learned_addr learned my addr 172.21.15.46:0/4180262111 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d15ce0700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d100aaf30 0x7f5d100b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d16ce2700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d10156e20 0x7f5d100b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 -- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d10156e20 msgr2=0x7f5d100b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d10156e20 0x7f5d100b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 -- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d100aaf30 msgr2=0x7f5d100b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d100aaf30 0x7f5d100b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.200+0000 7f5d164e1700 1 -- 172.21.15.46:0/4180262111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0094f0 con 0x7f5d10067590 2023-10-06T11:33:24.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d164e1700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009290 tx=0x7f5d0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c024440 con 0x7f5d10067590 2023-10-06T11:33:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0245a0 con 0x7f5d10067590 2023-10-06T11:33:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c024960 con 0x7f5d10067590 2023-10-06T11:33:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d10157670 con 0x7f5d10067590 2023-10-06T11:33:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d101550c0 con 0x7f5d10067590 2023-10-06T11:33:24.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d10064590 con 0x7f5d10067590 2023-10-06T11:33:24.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.204+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5d0c01a420 con 0x7f5d10067590 2023-10-06T11:33:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.208+0000 7f5d067fc700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 0x7f5cf4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.208+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5d0c07ea60 con 0x7f5d10067590 2023-10-06T11:33:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.208+0000 7f5d15ce0700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 0x7f5cf4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.208+0000 7f5d15ce0700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 0x7f5cf4062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d100ab300 tx=0x7f5d00006f90 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.212+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d0c04ecb0 con 0x7f5d10067590 2023-10-06T11:33:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.388+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d100ad320 con 0x7f5d10067590 2023-10-06T11:33:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d067fc700 1 -- 172.21.15.46:0/4180262111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17552 (secure 0 0 0) 0x7f5d0c0522d0 con 0x7f5d10067590 2023-10-06T11:33:24.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:24.404 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:23.034363+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 msgr2=0x7f5cf4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 0x7f5cf4062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5d100ab300 tx=0x7f5d00006f90 comp rx=0 tx=0).stop 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 msgr2=0x7f5d10158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009290 tx=0x7f5d0c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 shutdown_connections 2023-10-06T11:33:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cf40606a0 0x7f5cf4062b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d10156e20 0x7f5d100b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d100aaf30 0x7f5d100b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 --2- 172.21.15.46:0/4180262111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d10067590 0x7f5d10158150 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 >> 172.21.15.46:0/4180262111 conn(0x7f5d10062ef0 msgr2=0x7f5d100c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 shutdown_connections 2023-10-06T11:33:24.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.392+0000 7f5d174e3700 1 -- 172.21.15.46:0/4180262111 wait complete. 2023-10-06T11:33:24.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:33:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.572+0000 7fe29e84c700 1 Processor -- start 2023-10-06T11:33:24.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.572+0000 7fe29e84c700 1 -- start start 2023-10-06T11:33:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e84c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe298159240 0x7fe2980a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e84c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe2980c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e84c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2980c10a0 con 0x7fe298158e70 2023-10-06T11:33:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e84c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2980ca240 con 0x7fe298159240 2023-10-06T11:33:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e84c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980ca520 con 0x7fe2980c72d0 2023-10-06T11:33:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29d84a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe298158e70 0x7fe2980c99f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33278/0 (socket says 172.21.15.46:33278) 2023-10-06T11:33:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29d84a700 1 -- 172.21.15.46:0/2973007303 learned_addr learned my addr 172.21.15.46:0/2973007303 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29d049700 1 --2- 172.21.15.46:0/2973007303 >> v2:172.21.15.132:3300/0 conn(0x7fe298159240 0x7fe2980a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29e04b700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe2980c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.576+0000 7fe29c848700 1 -- 172.21.15.46:0/2973007303 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928580992 0 0) 0x7fe2980c10a0 con 0x7fe298158e70 2023-10-06T11:33:24.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29c848700 1 -- 172.21.15.46:0/2973007303 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe284003570 con 0x7fe298158e70 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29d049700 1 -- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 msgr2=0x7fe2980c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29d049700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe2980c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29d049700 1 -- 172.21.15.46:0/2973007303 >> v1:172.21.15.46:6789/0 conn(0x7fe298158e70 legacy=0x7fe2980c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29d049700 1 -- 172.21.15.46:0/2973007303 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2980c0a50 con 0x7fe298159240 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e04b700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe2980c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:24.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29d049700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298159240 0x7fe2980a9840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe2880098e0 tx=0x7fe2880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6364b0bd4373f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29c848700 1 -- 172.21.15.46:0/2973007303 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288011630 con 0x7fe298159240 2023-10-06T11:33:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29c848700 1 -- 172.21.15.46:0/2973007303 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288011790 con 0x7fe298159240 2023-10-06T11:33:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29c848700 1 -- 172.21.15.46:0/2973007303 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288011b30 con 0x7fe298159240 2023-10-06T11:33:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 -- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298159240 msgr2=0x7fe2980a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298159240 0x7fe2980a9840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe2880098e0 tx=0x7fe2880091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 -- 172.21.15.46:0/2973007303 shutdown_connections 2023-10-06T11:33:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe2980c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 --2- 172.21.15.46:0/2973007303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298159240 0x7fe2980a9840 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 -- 172.21.15.46:0/2973007303 >> 172.21.15.46:0/2973007303 conn(0x7fe2980cbc10 msgr2=0x7fe2980cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:24.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 -- 172.21.15.46:0/2973007303 shutdown_connections 2023-10-06T11:33:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.580+0000 7fe29e84c700 1 -- 172.21.15.46:0/2973007303 wait complete. 2023-10-06T11:33:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 Processor -- start 2023-10-06T11:33:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 -- start start 2023-10-06T11:33:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298158e70 0x7fe2980c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29806e8d0 con 0x7fe298159240 2023-10-06T11:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29806dd20 con 0x7fe298158e70 2023-10-06T11:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e84c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980c44b0 con 0x7fe2980c72d0 2023-10-06T11:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29d84a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29d84a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:59528/0 (socket says 172.21.15.46:59528) 2023-10-06T11:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29d84a700 1 -- 172.21.15.46:0/3350030339 learned_addr learned my addr 172.21.15.46:0/3350030339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40346/0 (socket says 172.21.15.46:40346) 2023-10-06T11:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29d049700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298158e70 0x7fe2980c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 -- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 msgr2=0x7fe29806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 -- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298158e70 msgr2=0x7fe2980c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298158e70 0x7fe2980c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 -- 172.21.15.46:0/3350030339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe290009a20 con 0x7fe298159240 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29d84a700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe29e04b700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe290009f10 tx=0x7fe29000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.584+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000cb70 con 0x7fe298159240 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29000f040 con 0x7fe298159240 2023-10-06T11:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290014480 con 0x7fe298159240 2023-10-06T11:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe288009420 con 0x7fe298159240 2023-10-06T11:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2980c4a30 con 0x7fe298159240 2023-10-06T11:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe298064590 con 0x7fe298159240 2023-10-06T11:33:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.588+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe290003020 con 0x7fe298159240 2023-10-06T11:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.592+0000 7fe28dffb700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 0x7fe27c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.592+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fe29007d680 con 0x7fe298159240 2023-10-06T11:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.592+0000 7fe29d84a700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 0x7fe27c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:24.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.592+0000 7fe29d84a700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 0x7fe27c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe294001e40 tx=0x7fe294009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.596+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe29004d690 con 0x7fe298159240 2023-10-06T11:33:24.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:24.808+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe2980c5750 con 0x7fe298159240 2023-10-06T11:33:25.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.624+0000 7fe28dffb700 1 -- 172.21.15.46:0/3350030339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe290050cb0 con 0x7fe298159240 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 msgr2=0x7fe27c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 0x7fe27c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe294001e40 tx=0x7fe294009040 comp rx=0 tx=0).stop 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 msgr2=0x7fe298151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe290009f10 tx=0x7fe29000b040 comp rx=0 tx=0).stop 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 shutdown_connections 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe27c060670 0x7fe27c062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe298159240 0x7fe298151160 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe298158e70 0x7fe2980c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 --2- 172.21.15.46:0/3350030339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2980c72d0 0x7fe29806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 >> 172.21.15.46:0/3350030339 conn(0x7fe2980cbc10 msgr2=0x7fe298155730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 shutdown_connections 2023-10-06T11:33:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.628+0000 7fe29e84c700 1 -- 172.21.15.46:0/3350030339 wait complete. 2023-10-06T11:33:25.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:33:25.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.816+0000 7ff9b126b700 1 Processor -- start 2023-10-06T11:33:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.816+0000 7ff9b126b700 1 -- start start 2023-10-06T11:33:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b126b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff9ac069270 0x7ff9ac069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b126b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0ce6e0 con 0x7ff9ac159850 2023-10-06T11:33:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b126b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0cddb0 con 0x7ff9ac158aa0 2023-10-06T11:33:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b126b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac15a5c0 con 0x7ff9ac069270 2023-10-06T11:33:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b0a6a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9ac159850 0x7ff9ac159cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33280/0 (socket says 172.21.15.46:33280) 2023-10-06T11:33:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9b0a6a700 1 -- 172.21.15.46:0/934048487 learned_addr learned my addr 172.21.15.46:0/934048487 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9abfff700 1 --2- 172.21.15.46:0/934048487 >> v2:172.21.15.132:3300/0 conn(0x7ff9ac069270 0x7ff9ac069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615910574 0 0) 0x7ff9ac0ce6e0 con 0x7ff9ac159850 2023-10-06T11:33:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff998003570 con 0x7ff9ac159850 2023-10-06T11:33:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943256577 0 0) 0x7ff9ac0cddb0 con 0x7ff9ac158aa0 2023-10-06T11:33:25.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9ac0ce6e0 con 0x7ff9ac158aa0 2023-10-06T11:33:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9abfff700 1 -- 172.21.15.46:0/934048487 >> v1:172.21.15.132:6789/0 conn(0x7ff9ac158aa0 legacy=0x7ff9ac158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9abfff700 1 -- 172.21.15.46:0/934048487 >> v1:172.21.15.46:6789/0 conn(0x7ff9ac159850 legacy=0x7ff9ac159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9abfff700 1 -- 172.21.15.46:0/934048487 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0ce090 con 0x7ff9ac069270 2023-10-06T11:33:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9abfff700 1 --2- 172.21.15.46:0/934048487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac069640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0098e0 tx=0x7ff99c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6410bf5afec49031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:25.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011630 con 0x7ff9ac069270 2023-10-06T11:33:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c011790 con 0x7ff9ac069270 2023-10-06T11:33:25.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.820+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/934048487 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c011b30 con 0x7ff9ac069270 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- 172.21.15.46:0/934048487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 msgr2=0x7ff9ac069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 --2- 172.21.15.46:0/934048487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac069640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0098e0 tx=0x7ff99c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- 172.21.15.46:0/934048487 shutdown_connections 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 --2- 172.21.15.46:0/934048487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac069640 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- 172.21.15.46:0/934048487 >> 172.21.15.46:0/934048487 conn(0x7ff9ac068390 msgr2=0x7ff9ac0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- 172.21.15.46:0/934048487 shutdown_connections 2023-10-06T11:33:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- 172.21.15.46:0/934048487 wait complete. 2023-10-06T11:33:25.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 Processor -- start 2023-10-06T11:33:25.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.824+0000 7ff9b126b700 1 -- start start 2023-10-06T11:33:25.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:25.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:25.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 0x7ff9ac0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac15a3e0 con 0x7ff9ac158aa0 2023-10-06T11:33:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac16dd70 con 0x7ff9ac069270 2023-10-06T11:33:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac16d160 con 0x7ff9ac159850 2023-10-06T11:33:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40356/0 (socket says 172.21.15.46:40356) 2023-10-06T11:33:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 -- 172.21.15.46:0/830955471 learned_addr learned my addr 172.21.15.46:0/830955471 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9abfff700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b0a6a700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 0x7ff9ac0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 -- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 msgr2=0x7ff9ac0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 0x7ff9ac0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 -- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 msgr2=0x7ff9ac147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 -- 172.21.15.46:0/830955471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c009420 con 0x7ff9ac158aa0 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b0a6a700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 0x7ff9ac0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:25.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a37fe700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c790 tx=0x7ff994007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994010510 con 0x7ff9ac158aa0 2023-10-06T11:33:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994010670 con 0x7ff9ac158aa0 2023-10-06T11:33:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994010a30 con 0x7ff9ac158aa0 2023-10-06T11:33:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac16d440 con 0x7ff9ac158aa0 2023-10-06T11:33:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.828+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac16da00 con 0x7ff9ac158aa0 2023-10-06T11:33:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.832+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0b4330 con 0x7ff9ac158aa0 2023-10-06T11:33:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.832+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff99401b3b0 con 0x7ff9ac158aa0 2023-10-06T11:33:25.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.836+0000 7ff9a8ff9700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 0x7ff98c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.836+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ff99407c510 con 0x7ff9ac158aa0 2023-10-06T11:33:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.836+0000 7ff9abfff700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 0x7ff98c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:25.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.836+0000 7ff9abfff700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 0x7ff98c062b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0093f0 tx=0x7ff99c017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:25.840+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff9ac0b4330 con 0x7ff9ac158aa0 2023-10-06T11:33:26.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.052+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff9ac0b4330 con 0x7ff9ac158aa0 2023-10-06T11:33:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.628+0000 7ff9a8ff9700 1 -- 172.21.15.46:0/830955471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9ac0b4330 con 0x7ff9ac158aa0 2023-10-06T11:33:26.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 msgr2=0x7ff98c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 0x7ff98c062b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0093f0 tx=0x7ff99c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 msgr2=0x7ff9ac0c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c790 tx=0x7ff994007e10 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 shutdown_connections 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff98c0606a0 0x7ff98c062b50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac159850 0x7ff9ac0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac158aa0 0x7ff9ac0c70f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 --2- 172.21.15.46:0/830955471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac069270 0x7ff9ac147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 >> 172.21.15.46:0/830955471 conn(0x7ff9ac068390 msgr2=0x7ff9ac0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:26.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 shutdown_connections 2023-10-06T11:33:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.632+0000 7ff9b126b700 1 -- 172.21.15.46:0/830955471 wait complete. 2023-10-06T11:33:26.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.828+0000 7ffb983c5700 1 Processor -- start 2023-10-06T11:33:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.828+0000 7ffb983c5700 1 -- start start 2023-10-06T11:33:26.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb983c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb9006e8a0 0x7ffb9006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:26.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb983c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9006d460 con 0x7ffb9015ab60 2023-10-06T11:33:26.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb983c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb900c7000 con 0x7ffb900ad6c0 2023-10-06T11:33:26.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb983c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9006ed40 con 0x7ffb9006e8a0 2023-10-06T11:33:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb95960700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb9006e8a0 0x7ffb9006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb96161700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb9015ab60 0x7ffb9015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33282/0 (socket says 172.21.15.46:33282) 2023-10-06T11:33:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb96161700 1 -- 172.21.15.46:0/2509995278 learned_addr learned my addr 172.21.15.46:0/2509995278 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:26.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb95960700 1 -- 172.21.15.46:0/2509995278 >> v1:172.21.15.132:6789/0 conn(0x7ffb900ad6c0 legacy=0x7ffb900adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb95960700 1 -- 172.21.15.46:0/2509995278 >> v1:172.21.15.46:6789/0 conn(0x7ffb9015ab60 legacy=0x7ffb9015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.832+0000 7ffb95960700 1 -- 172.21.15.46:0/2509995278 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900c72c0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb95960700 1 --2- 172.21.15.46:0/2509995278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb9006cc30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098e0 tx=0x7ffb800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599804ecb07dba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb9515f700 1 -- 172.21.15.46:0/2509995278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80009e80 con 0x7ffb9006e8a0 2023-10-06T11:33:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb9515f700 1 -- 172.21.15.46:0/2509995278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8000b820 con 0x7ffb9006e8a0 2023-10-06T11:33:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb9515f700 1 -- 172.21.15.46:0/2509995278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000bba0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 -- 172.21.15.46:0/2509995278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 msgr2=0x7ffb9006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 --2- 172.21.15.46:0/2509995278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb9006cc30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffb800098e0 tx=0x7ffb800091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 -- 172.21.15.46:0/2509995278 shutdown_connections 2023-10-06T11:33:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 --2- 172.21.15.46:0/2509995278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb9006cc30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 -- 172.21.15.46:0/2509995278 >> 172.21.15.46:0/2509995278 conn(0x7ffb900cbe30 msgr2=0x7ffb900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 -- 172.21.15.46:0/2509995278 shutdown_connections 2023-10-06T11:33:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 -- 172.21.15.46:0/2509995278 wait complete. 2023-10-06T11:33:26.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.836+0000 7ffb983c5700 1 Processor -- start 2023-10-06T11:33:26.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- start start 2023-10-06T11:33:26.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:26.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 0x7ffb900ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 0x7ffb900acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901592e0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901595c0 con 0x7ffb9015ab60 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901598a0 con 0x7ffb900ad6c0 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40364/0 (socket says 172.21.15.46:40364) 2023-10-06T11:33:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 -- 172.21.15.46:0/349454626 learned_addr learned my addr 172.21.15.46:0/349454626 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb95960700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 0x7ffb900ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 -- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 msgr2=0x7ffb900ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 0x7ffb900ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 -- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 msgr2=0x7ffb900acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96962700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 0x7ffb900acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 0x7ffb900acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 -- 172.21.15.46:0/349454626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80009420 con 0x7ffb9006e8a0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb95960700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 0x7ffb900ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96962700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 0x7ffb900acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb96161700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c590 tx=0x7ffb8c007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0072e0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c007460 con 0x7ffb9006e8a0 2023-10-06T11:33:26.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb90159b80 con 0x7ffb9006e8a0 2023-10-06T11:33:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0057f0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.840+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb90143cd0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.844+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb90063f90 con 0x7ffb9006e8a0 2023-10-06T11:33:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.844+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ffb8c005ba0 con 0x7ffb9006e8a0 2023-10-06T11:33:26.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.848+0000 7ffb867fc700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 0x7ffb74062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:26.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.848+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ffb8c07c230 con 0x7ffb9006e8a0 2023-10-06T11:33:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.848+0000 7ffb95960700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 0x7ffb74062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.848+0000 7ffb95960700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 0x7ffb74062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb800030c0 tx=0x7ffb80016000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:26.852+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffb8c04c390 con 0x7ffb9006e8a0 2023-10-06T11:33:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.064+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb9004b5c0 con 0x7ffb9006e8a0 2023-10-06T11:33:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.064+0000 7ffb867fc700 1 -- 172.21.15.46:0/349454626 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ffb8c04f9b0 con 0x7ffb9006e8a0 2023-10-06T11:33:27.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:27.077 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 msgr2=0x7ffb74062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 0x7ffb74062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb800030c0 tx=0x7ffb80016000 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 msgr2=0x7ffb90143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c590 tx=0x7ffb8c007e40 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 shutdown_connections 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffb74060670 0x7ffb74062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb9015ab60 0x7ffb900acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb900ad6c0 0x7ffb900ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 --2- 172.21.15.46:0/349454626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9006e8a0 0x7ffb90143790 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 >> 172.21.15.46:0/349454626 conn(0x7ffb900cbe30 msgr2=0x7ffb900aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:27.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 shutdown_connections 2023-10-06T11:33:27.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.068+0000 7ffb983c5700 1 -- 172.21.15.46:0/349454626 wait complete. 2023-10-06T11:33:27.082 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:27.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.248+0000 7f76aac2e700 1 Processor -- start 2023-10-06T11:33:27.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.248+0000 7f76aac2e700 1 -- start start 2023-10-06T11:33:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76aac2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f76a40d0aa0 0x7f76a4159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76aac2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f76a40b5410 0x7f76a40b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76aac2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40b5830 con 0x7f76a4159050 2023-10-06T11:33:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76aac2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a40b4f80 con 0x7f76a40d0aa0 2023-10-06T11:33:27.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76aac2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a40b40c0 con 0x7f76a40b5410 2023-10-06T11:33:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a89ca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f76a4159050 0x7f76a40b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33286/0 (socket says 172.21.15.46:33286) 2023-10-06T11:33:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a89ca700 1 -- 172.21.15.46:0/3147171667 learned_addr learned my addr 172.21.15.46:0/3147171667 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a3fff700 1 --2- 172.21.15.46:0/3147171667 >> v2:172.21.15.46:3300/0 conn(0x7f76a40d0aa0 0x7f76a4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a91cb700 1 --2- 172.21.15.46:0/3147171667 >> v2:172.21.15.132:3300/0 conn(0x7f76a40b5410 0x7f76a40b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a37fe700 1 -- 172.21.15.46:0/3147171667 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730177862 0 0) 0x7f76a40b5830 con 0x7f76a4159050 2023-10-06T11:33:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a37fe700 1 -- 172.21.15.46:0/3147171667 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7694003570 con 0x7f76a4159050 2023-10-06T11:33:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a3fff700 1 -- 172.21.15.46:0/3147171667 >> v2:172.21.15.132:3300/0 conn(0x7f76a40b5410 msgr2=0x7f76a40b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a3fff700 1 --2- 172.21.15.46:0/3147171667 >> v2:172.21.15.132:3300/0 conn(0x7f76a40b5410 0x7f76a40b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a3fff700 1 -- 172.21.15.46:0/3147171667 >> v1:172.21.15.46:6789/0 conn(0x7f76a4159050 legacy=0x7f76a40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.252+0000 7f76a3fff700 1 -- 172.21.15.46:0/3147171667 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40b4370 con 0x7f76a40d0aa0 2023-10-06T11:33:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76a3fff700 1 --2- 172.21.15.46:0/3147171667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a40d0aa0 0x7f76a4159420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f76a40c7150 tx=0x7f76900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6448a3122a1d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76a37fe700 1 -- 172.21.15.46:0/3147171667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690011630 con 0x7f76a40d0aa0 2023-10-06T11:33:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76a37fe700 1 -- 172.21.15.46:0/3147171667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900117b0 con 0x7f76a40d0aa0 2023-10-06T11:33:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76a37fe700 1 -- 172.21.15.46:0/3147171667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690011b70 con 0x7f76a40d0aa0 2023-10-06T11:33:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 -- 172.21.15.46:0/3147171667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a40d0aa0 msgr2=0x7f76a4159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 --2- 172.21.15.46:0/3147171667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a40d0aa0 0x7f76a4159420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f76a40c7150 tx=0x7f76900091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 -- 172.21.15.46:0/3147171667 shutdown_connections 2023-10-06T11:33:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 --2- 172.21.15.46:0/3147171667 >> v2:172.21.15.132:3300/0 conn(0x7f76a40b5410 0x7f76a40b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 --2- 172.21.15.46:0/3147171667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a40d0aa0 0x7f76a4159420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 -- 172.21.15.46:0/3147171667 >> 172.21.15.46:0/3147171667 conn(0x7f76a406e780 msgr2=0x7f76a406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 -- 172.21.15.46:0/3147171667 shutdown_connections 2023-10-06T11:33:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.256+0000 7f76aac2e700 1 -- 172.21.15.46:0/3147171667 wait complete. 2023-10-06T11:33:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 Processor -- start 2023-10-06T11:33:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 -- start start 2023-10-06T11:33:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40b5410 0x7f76a4154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40d0aa0 0x7f76a415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40c8590 con 0x7f76a4159050 2023-10-06T11:33:27.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a415b2e0 con 0x7f76a40b5410 2023-10-06T11:33:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76aac2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a415b5c0 con 0x7f76a40d0aa0 2023-10-06T11:33:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40372/0 (socket says 172.21.15.46:40372) 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 -- 172.21.15.46:0/2008765183 learned_addr learned my addr 172.21.15.46:0/2008765183 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a3fff700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40d0aa0 0x7f76a415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a89ca700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40b5410 0x7f76a4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 -- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40d0aa0 msgr2=0x7f76a415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40d0aa0 0x7f76a415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 -- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40b5410 msgr2=0x7f76a4154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40b5410 0x7f76a4154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.260+0000 7f76a91cb700 1 -- 172.21.15.46:0/2008765183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76a4159050 2023-10-06T11:33:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76a91cb700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f769c009e90 tx=0x7f769c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00caa0 con 0x7f76a4159050 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00f040 con 0x7f76a4159050 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00ce50 con 0x7f76a4159050 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a415b8a0 con 0x7f76a4159050 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a40cb8b0 con 0x7f76a4159050 2023-10-06T11:33:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.264+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a40c8700 con 0x7f76a4159050 2023-10-06T11:33:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.268+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f769c014690 con 0x7f76a4159050 2023-10-06T11:33:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.268+0000 7f76a0ff9700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 0x7f7688062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.268+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f769c07d6c0 con 0x7f76a4159050 2023-10-06T11:33:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.268+0000 7f76a89ca700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 0x7f7688062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.268+0000 7f76a89ca700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 0x7f7688062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.276+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f76a40c8700 con 0x7f76a4159050 2023-10-06T11:33:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.452+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76a40c86d0 con 0x7f76a4159050 2023-10-06T11:33:27.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.452+0000 7f76a0ff9700 1 -- 172.21.15.46:0/2008765183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17539 (secure 0 0 0) 0x7f769c051160 con 0x7f76a4159050 2023-10-06T11:33:27.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:27.466 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:24.604042+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.456+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 msgr2=0x7f7688062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.456+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 0x7f7688062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698009040 comp rx=0 tx=0).stop 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.456+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 msgr2=0x7f76a40cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.456+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f769c009e90 tx=0x7f769c00b040 comp rx=0 tx=0).stop 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 shutdown_connections 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7688060670 0x7f7688062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76a4159050 0x7f76a40cb370 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76a40d0aa0 0x7f76a415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 --2- 172.21.15.46:0/2008765183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76a40b5410 0x7f76a4154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 >> 172.21.15.46:0/2008765183 conn(0x7f76a406e780 msgr2=0x7f76a41570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 shutdown_connections 2023-10-06T11:33:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.460+0000 7f76aac2e700 1 -- 172.21.15.46:0/2008765183 wait complete. 2023-10-06T11:33:27.491 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:33:27.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:33:27.544 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-10-06T11:33:27.532+0000 7f30e8288700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:33:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 Processor -- start 2023-10-06T11:33:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 -- start start 2023-10-06T11:33:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff2bc0af000 0x7ff2bc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 0x7ff2bc0690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2bc0cf100 con 0x7ff2bc0cb4c0 2023-10-06T11:33:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2bc0ce300 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2c080c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2bc0ce4f0 con 0x7ff2bc0af000 2023-10-06T11:33:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2baffd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff2bc0cb4c0 0x7ff2bc0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42328/0 (socket says 172.21.15.46:42328) 2023-10-06T11:33:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2baffd700 1 -- 172.21.15.46:0/375192060 learned_addr learned my addr 172.21.15.46:0/375192060 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/375192060 >> v2:172.21.15.132:3300/0 conn(0x7ff2bc0af000 0x7ff2bc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.640+0000 7ff2bbfff700 1 --2- 172.21.15.46:0/375192060 >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 0x7ff2bc0690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2ba7fc700 1 -- 172.21.15.46:0/375192060 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016387232 0 0) 0x7ff2bc0cf100 con 0x7ff2bc0cb4c0 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2ba7fc700 1 -- 172.21.15.46:0/375192060 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2a4003570 con 0x7ff2bc0cb4c0 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bb7fe700 1 -- 172.21.15.46:0/375192060 >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 msgr2=0x7ff2bc0690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/375192060 >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 0x7ff2bc0690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bb7fe700 1 -- 172.21.15.46:0/375192060 >> v1:172.21.15.132:6789/0 conn(0x7ff2bc0cb4c0 legacy=0x7ff2bc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bb7fe700 1 -- 172.21.15.46:0/375192060 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc0ce7d0 con 0x7ff2bc0af000 2023-10-06T11:33:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bbfff700 1 --2- 172.21.15.46:0/375192060 >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 0x7ff2bc0690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/375192060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc0cf590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc089020 tx=0x7ff2ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d455e45dd4b38777 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2ba7fc700 1 -- 172.21.15.46:0/375192060 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011630 con 0x7ff2bc0af000 2023-10-06T11:33:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2ba7fc700 1 -- 172.21.15.46:0/375192060 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac0117b0 con 0x7ff2bc0af000 2023-10-06T11:33:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2ba7fc700 1 -- 172.21.15.46:0/375192060 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011b70 con 0x7ff2bc0af000 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 -- 172.21.15.46:0/375192060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 msgr2=0x7ff2bc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 --2- 172.21.15.46:0/375192060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc0cf590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc089020 tx=0x7ff2ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 -- 172.21.15.46:0/375192060 shutdown_connections 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 --2- 172.21.15.46:0/375192060 >> v2:172.21.15.46:3300/0 conn(0x7ff2bc0af3d0 0x7ff2bc0690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 --2- 172.21.15.46:0/375192060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc0cf590 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 -- 172.21.15.46:0/375192060 >> 172.21.15.46:0/375192060 conn(0x7ff2bc0683b0 msgr2=0x7ff2bc0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 -- 172.21.15.46:0/375192060 shutdown_connections 2023-10-06T11:33:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.644+0000 7ff2c080c700 1 -- 172.21.15.46:0/375192060 wait complete. 2023-10-06T11:33:27.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 Processor -- start 2023-10-06T11:33:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 -- start start 2023-10-06T11:33:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2bc0cb4c0 0x7ff2bc0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0b2e60 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0b30d0 con 0x7ff2bc0af000 2023-10-06T11:33:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2c080c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc0b33b0 con 0x7ff2bc0cb4c0 2023-10-06T11:33:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40378/0 (socket says 172.21.15.46:40378) 2023-10-06T11:33:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 -- 172.21.15.46:0/1199260216 learned_addr learned my addr 172.21.15.46:0/1199260216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2bbfff700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2bc0cb4c0 0x7ff2bc0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 -- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2bc0cb4c0 msgr2=0x7ff2bc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2bc0cb4c0 0x7ff2bc0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 -- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 msgr2=0x7ff2bc149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.648+0000 7ff2baffd700 1 -- 172.21.15.46:0/1199260216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4009a20 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2baffd700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c560 tx=0x7ff2b0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0005c00 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b0005d60 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac009420 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0007300 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc0b3970 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc064590 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.652+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff2b00076b0 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.656+0000 7ff2a37fe700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 0x7ff2a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.656+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ff2b007c120 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.656+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 0x7ff2a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.656+0000 7ff2bb7fe700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 0x7ff2a8062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc089020 tx=0x7ff2ac017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.664+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff2b004c2e0 con 0x7ff2bc0af3d0 2023-10-06T11:33:27.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.832+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2bc0ceb00 con 0x7ff2a8060670 2023-10-06T11:33:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.848+0000 7ff2a37fe700 1 -- 172.21.15.46:0/1199260216 <== mgr.6150 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308791 (secure 0 0 0) 0x7ff2bc0ceb00 con 0x7ff2a8060670 2023-10-06T11:33:27.868 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 msgr2=0x7ff2a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 0x7ff2a8062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc089020 tx=0x7ff2ac017000 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 msgr2=0x7ff2bc0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000c560 tx=0x7ff2b0007dd0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 shutdown_connections 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff2a8060670 0x7ff2a8062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2bc0cb4c0 0x7ff2bc0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2bc0af3d0 0x7ff2bc0b3f60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 --2- 172.21.15.46:0/1199260216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2bc0af000 0x7ff2bc149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 >> 172.21.15.46:0/1199260216 conn(0x7ff2bc0683b0 msgr2=0x7ff2bc0c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 shutdown_connections 2023-10-06T11:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:27.856+0000 7ff2c080c700 1 -- 172.21.15.46:0/1199260216 wait complete. 2023-10-06T11:33:27.872 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:33:27.888 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:33:27.232519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448388,"kb_used_data":8644,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752526332,"statfs":{"total":773094113280,"available":770586963968,"internally_reserved":0,"allocated":8851456,"data_stored":5000181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669656"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624008+0000","last_change":"2023-10-06T11:33:23.614474+0000","last_active":"2023-10-06T11:33:23.624008+0000","last_peered":"2023-10-06T11:33:23.624008+0000","last_clean":"2023-10-06T11:33:23.624008+0000","last_became_active":"2023-10-06T11:33:23.614195+0000","last_became_peered":"2023-10-06T11:33:23.614195+0000","last_unstale":"2023-10-06T11:33:23.624008+0000","last_undegraded":"2023-10-06T11:33:23.624008+0000","last_fullsized":"2023-10-06T11:33:23.624008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:45.182698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609398+0000","last_change":"2023-10-06T11:33:23.602424+0000","last_active":"2023-10-06T11:33:23.609398+0000","last_peered":"2023-10-06T11:33:23.609398+0000","last_clean":"2023-10-06T11:33:23.609398+0000","last_became_active":"2023-10-06T11:33:23.602127+0000","last_became_peered":"2023-10-06T11:33:23.602127+0000","last_unstale":"2023-10-06T11:33:23.609398+0000","last_undegraded":"2023-10-06T11:33:23.609398+0000","last_fullsized":"2023-10-06T11:33:23.609398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:40:17.879667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614335+0000","last_change":"2023-10-06T11:33:23.610017+0000","last_active":"2023-10-06T11:33:23.614335+0000","last_peered":"2023-10-06T11:33:23.614335+0000","last_clean":"2023-10-06T11:33:23.614335+0000","last_became_active":"2023-10-06T11:33:23.609714+0000","last_became_peered":"2023-10-06T11:33:23.609714+0000","last_unstale":"2023-10-06T11:33:23.614335+0000","last_undegraded":"2023-10-06T11:33:23.614335+0000","last_fullsized":"2023-10-06T11:33:23.614335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622836+0000","last_change":"2023-10-06T11:33:23.614075+0000","last_active":"2023-10-06T11:33:23.622836+0000","last_peered":"2023-10-06T11:33:23.622836+0000","last_clean":"2023-10-06T11:33:23.622836+0000","last_became_active":"2023-10-06T11:33:23.612877+0000","last_became_peered":"2023-10-06T11:33:23.612877+0000","last_unstale":"2023-10-06T11:33:23.622836+0000","last_undegraded":"2023-10-06T11:33:23.622836+0000","last_fullsized":"2023-10-06T11:33:23.622836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:13.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622900+0000","last_change":"2023-10-06T11:33:23.611515+0000","last_active":"2023-10-06T11:33:23.622900+0000","last_peered":"2023-10-06T11:33:23.622900+0000","last_clean":"2023-10-06T11:33:23.622900+0000","last_became_active":"2023-10-06T11:33:23.611214+0000","last_became_peered":"2023-10-06T11:33:23.611214+0000","last_unstale":"2023-10-06T11:33:23.622900+0000","last_undegraded":"2023-10-06T11:33:23.622900+0000","last_fullsized":"2023-10-06T11:33:23.622900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:19.061615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.616555+0000","last_change":"2023-10-06T11:33:23.614534+0000","last_active":"2023-10-06T11:33:23.616555+0000","last_peered":"2023-10-06T11:33:23.616555+0000","last_clean":"2023-10-06T11:33:23.616555+0000","last_became_active":"2023-10-06T11:33:23.612294+0000","last_became_peered":"2023-10-06T11:33:23.612294+0000","last_unstale":"2023-10-06T11:33:23.616555+0000","last_undegraded":"2023-10-06T11:33:23.616555+0000","last_fullsized":"2023-10-06T11:33:23.616555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:59.265142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.613609+0000","last_change":"2023-10-06T11:33:23.604365+0000","last_active":"2023-10-06T11:33:23.613609+0000","last_peered":"2023-10-06T11:33:23.613609+0000","last_clean":"2023-10-06T11:33:23.613609+0000","last_became_active":"2023-10-06T11:33:23.604055+0000","last_became_peered":"2023-10-06T11:33:23.604055+0000","last_unstale":"2023-10-06T11:33:23.613609+0000","last_undegraded":"2023-10-06T11:33:23.613609+0000","last_fullsized":"2023-10-06T11:33:23.613609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:22:23.482421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633210+0000","last_change":"2023-10-06T11:33:23.625528+0000","last_active":"2023-10-06T11:33:23.633210+0000","last_peered":"2023-10-06T11:33:23.633210+0000","last_clean":"2023-10-06T11:33:23.633210+0000","last_became_active":"2023-10-06T11:33:23.625193+0000","last_became_peered":"2023-10-06T11:33:23.625193+0000","last_unstale":"2023-10-06T11:33:23.633210+0000","last_undegraded":"2023-10-06T11:33:23.633210+0000","last_fullsized":"2023-10-06T11:33:23.633210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627814+0000","last_change":"2023-10-06T11:33:23.622261+0000","last_active":"2023-10-06T11:33:23.627814+0000","last_peered":"2023-10-06T11:33:23.627814+0000","last_clean":"2023-10-06T11:33:23.627814+0000","last_became_active":"2023-10-06T11:33:23.621933+0000","last_became_peered":"2023-10-06T11:33:23.621933+0000","last_unstale":"2023-10-06T11:33:23.627814+0000","last_undegraded":"2023-10-06T11:33:23.627814+0000","last_fullsized":"2023-10-06T11:33:23.627814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:19.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615002+0000","last_change":"2023-10-06T11:33:23.612701+0000","last_active":"2023-10-06T11:33:23.615002+0000","last_peered":"2023-10-06T11:33:23.615002+0000","last_clean":"2023-10-06T11:33:23.615002+0000","last_became_active":"2023-10-06T11:33:23.612407+0000","last_became_peered":"2023-10-06T11:33:23.612407+0000","last_unstale":"2023-10-06T11:33:23.615002+0000","last_undegraded":"2023-10-06T11:33:23.615002+0000","last_fullsized":"2023-10-06T11:33:23.615002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623353+0000","last_change":"2023-10-06T11:33:23.614291+0000","last_active":"2023-10-06T11:33:23.623353+0000","last_peered":"2023-10-06T11:33:23.623353+0000","last_clean":"2023-10-06T11:33:23.623353+0000","last_became_active":"2023-10-06T11:33:23.613986+0000","last_became_peered":"2023-10-06T11:33:23.613986+0000","last_unstale":"2023-10-06T11:33:23.623353+0000","last_undegraded":"2023-10-06T11:33:23.623353+0000","last_fullsized":"2023-10-06T11:33:23.623353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:21.181531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633134+0000","last_change":"2023-10-06T11:33:23.625211+0000","last_active":"2023-10-06T11:33:23.633134+0000","last_peered":"2023-10-06T11:33:23.633134+0000","last_clean":"2023-10-06T11:33:23.633134+0000","last_became_active":"2023-10-06T11:33:23.624910+0000","last_became_peered":"2023-10-06T11:33:23.624910+0000","last_unstale":"2023-10-06T11:33:23.633134+0000","last_undegraded":"2023-10-06T11:33:23.633134+0000","last_fullsized":"2023-10-06T11:33:23.633134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:44.300280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633331+0000","last_change":"2023-10-06T11:33:23.626574+0000","last_active":"2023-10-06T11:33:23.633331+0000","last_peered":"2023-10-06T11:33:23.633331+0000","last_clean":"2023-10-06T11:33:23.633331+0000","last_became_active":"2023-10-06T11:33:23.626112+0000","last_became_peered":"2023-10-06T11:33:23.626112+0000","last_unstale":"2023-10-06T11:33:23.633331+0000","last_undegraded":"2023-10-06T11:33:23.633331+0000","last_fullsized":"2023-10-06T11:33:23.633331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:36.313062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615070+0000","last_change":"2023-10-06T11:33:23.611354+0000","last_active":"2023-10-06T11:33:23.615070+0000","last_peered":"2023-10-06T11:33:23.615070+0000","last_clean":"2023-10-06T11:33:23.615070+0000","last_became_active":"2023-10-06T11:33:23.610978+0000","last_became_peered":"2023-10-06T11:33:23.610978+0000","last_unstale":"2023-10-06T11:33:23.615070+0000","last_undegraded":"2023-10-06T11:33:23.615070+0000","last_fullsized":"2023-10-06T11:33:23.615070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615228+0000","last_change":"2023-10-06T11:33:23.612617+0000","last_active":"2023-10-06T11:33:23.615228+0000","last_peered":"2023-10-06T11:33:23.615228+0000","last_clean":"2023-10-06T11:33:23.615228+0000","last_became_active":"2023-10-06T11:33:23.612296+0000","last_became_peered":"2023-10-06T11:33:23.612296+0000","last_unstale":"2023-10-06T11:33:23.615228+0000","last_undegraded":"2023-10-06T11:33:23.615228+0000","last_fullsized":"2023-10-06T11:33:23.615228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:59.059580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622626+0000","last_change":"2023-10-06T11:33:23.618761+0000","last_active":"2023-10-06T11:33:23.622626+0000","last_peered":"2023-10-06T11:33:23.622626+0000","last_clean":"2023-10-06T11:33:23.622626+0000","last_became_active":"2023-10-06T11:33:23.618426+0000","last_became_peered":"2023-10-06T11:33:23.618426+0000","last_unstale":"2023-10-06T11:33:23.622626+0000","last_undegraded":"2023-10-06T11:33:23.622626+0000","last_fullsized":"2023-10-06T11:33:23.622626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:47.164365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.611072+0000","last_change":"2023-10-06T11:33:23.604575+0000","last_active":"2023-10-06T11:33:23.611072+0000","last_peered":"2023-10-06T11:33:23.611072+0000","last_clean":"2023-10-06T11:33:23.611072+0000","last_became_active":"2023-10-06T11:33:23.604024+0000","last_became_peered":"2023-10-06T11:33:23.604024+0000","last_unstale":"2023-10-06T11:33:23.611072+0000","last_undegraded":"2023-10-06T11:33:23.611072+0000","last_fullsized":"2023-10-06T11:33:23.611072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:59.816291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622772+0000","last_change":"2023-10-06T11:33:23.610239+0000","last_active":"2023-10-06T11:33:23.622772+0000","last_peered":"2023-10-06T11:33:23.622772+0000","last_clean":"2023-10-06T11:33:23.622772+0000","last_became_active":"2023-10-06T11:33:23.609919+0000","last_became_peered":"2023-10-06T11:33:23.609919+0000","last_unstale":"2023-10-06T11:33:23.622772+0000","last_undegraded":"2023-10-06T11:33:23.622772+0000","last_fullsized":"2023-10-06T11:33:23.622772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:59.293501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622990+0000","last_change":"2023-10-06T11:33:23.619699+0000","last_active":"2023-10-06T11:33:23.622990+0000","last_peered":"2023-10-06T11:33:23.622990+0000","last_clean":"2023-10-06T11:33:23.622990+0000","last_became_active":"2023-10-06T11:33:23.619385+0000","last_became_peered":"2023-10-06T11:33:23.619385+0000","last_unstale":"2023-10-06T11:33:23.622990+0000","last_undegraded":"2023-10-06T11:33:23.622990+0000","last_fullsized":"2023-10-06T11:33:23.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:57.076608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.610252+0000","last_change":"2023-10-06T11:33:23.602037+0000","last_active":"2023-10-06T11:33:23.610252+0000","last_peered":"2023-10-06T11:33:23.610252+0000","last_clean":"2023-10-06T11:33:23.610252+0000","last_became_active":"2023-10-06T11:33:23.601760+0000","last_became_peered":"2023-10-06T11:33:23.601760+0000","last_unstale":"2023-10-06T11:33:23.610252+0000","last_undegraded":"2023-10-06T11:33:23.610252+0000","last_fullsized":"2023-10-06T11:33:23.610252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.632591+0000","last_change":"2023-10-06T11:33:23.621794+0000","last_active":"2023-10-06T11:33:23.632591+0000","last_peered":"2023-10-06T11:33:23.632591+0000","last_clean":"2023-10-06T11:33:23.632591+0000","last_became_active":"2023-10-06T11:33:23.621468+0000","last_became_peered":"2023-10-06T11:33:23.621468+0000","last_unstale":"2023-10-06T11:33:23.632591+0000","last_undegraded":"2023-10-06T11:33:23.632591+0000","last_fullsized":"2023-10-06T11:33:23.632591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:59.059580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614285+0000","last_change":"2023-10-06T11:33:23.608839+0000","last_active":"2023-10-06T11:33:23.614285+0000","last_peered":"2023-10-06T11:33:23.614285+0000","last_clean":"2023-10-06T11:33:23.614285+0000","last_became_active":"2023-10-06T11:33:23.608523+0000","last_became_peered":"2023-10-06T11:33:23.608523+0000","last_unstale":"2023-10-06T11:33:23.614285+0000","last_undegraded":"2023-10-06T11:33:23.614285+0000","last_fullsized":"2023-10-06T11:33:23.614285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614185+0000","last_change":"2023-10-06T11:33:23.605409+0000","last_active":"2023-10-06T11:33:23.614185+0000","last_peered":"2023-10-06T11:33:23.614185+0000","last_clean":"2023-10-06T11:33:23.614185+0000","last_became_active":"2023-10-06T11:33:23.605076+0000","last_became_peered":"2023-10-06T11:33:23.605076+0000","last_unstale":"2023-10-06T11:33:23.614185+0000","last_undegraded":"2023-10-06T11:33:23.614185+0000","last_fullsized":"2023-10-06T11:33:23.614185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614484+0000","last_change":"2023-10-06T11:33:23.609258+0000","last_active":"2023-10-06T11:33:23.614484+0000","last_peered":"2023-10-06T11:33:23.614484+0000","last_clean":"2023-10-06T11:33:23.614484+0000","last_became_active":"2023-10-06T11:33:23.608978+0000","last_became_peered":"2023-10-06T11:33:23.608978+0000","last_unstale":"2023-10-06T11:33:23.614484+0000","last_undegraded":"2023-10-06T11:33:23.614484+0000","last_fullsized":"2023-10-06T11:33:23.614484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633389+0000","last_change":"2023-10-06T11:33:23.626717+0000","last_active":"2023-10-06T11:33:23.633389+0000","last_peered":"2023-10-06T11:33:23.633389+0000","last_clean":"2023-10-06T11:33:23.633389+0000","last_became_active":"2023-10-06T11:33:23.625611+0000","last_became_peered":"2023-10-06T11:33:23.625611+0000","last_unstale":"2023-10-06T11:33:23.633389+0000","last_undegraded":"2023-10-06T11:33:23.633389+0000","last_fullsized":"2023-10-06T11:33:23.633389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:21.680372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633361+0000","last_change":"2023-10-06T11:33:23.626845+0000","last_active":"2023-10-06T11:33:23.633361+0000","last_peered":"2023-10-06T11:33:23.633361+0000","last_clean":"2023-10-06T11:33:23.633361+0000","last_became_active":"2023-10-06T11:33:23.625171+0000","last_became_peered":"2023-10-06T11:33:23.625171+0000","last_unstale":"2023-10-06T11:33:23.633361+0000","last_undegraded":"2023-10-06T11:33:23.633361+0000","last_fullsized":"2023-10-06T11:33:23.633361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:41.014328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624905+0000","last_change":"2023-10-06T11:33:23.617605+0000","last_active":"2023-10-06T11:33:23.624905+0000","last_peered":"2023-10-06T11:33:23.624905+0000","last_clean":"2023-10-06T11:33:23.624905+0000","last_became_active":"2023-10-06T11:33:23.617276+0000","last_became_peered":"2023-10-06T11:33:23.617276+0000","last_unstale":"2023-10-06T11:33:23.624905+0000","last_undegraded":"2023-10-06T11:33:23.624905+0000","last_fullsized":"2023-10-06T11:33:23.624905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:13.594292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609373+0000","last_change":"2023-10-06T11:33:23.601978+0000","last_active":"2023-10-06T11:33:23.609373+0000","last_peered":"2023-10-06T11:33:23.609373+0000","last_clean":"2023-10-06T11:33:23.609373+0000","last_became_active":"2023-10-06T11:33:23.601685+0000","last_became_peered":"2023-10-06T11:33:23.601685+0000","last_unstale":"2023-10-06T11:33:23.609373+0000","last_undegraded":"2023-10-06T11:33:23.609373+0000","last_fullsized":"2023-10-06T11:33:23.609373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:01.232455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627616+0000","last_change":"2023-10-06T11:33:23.621354+0000","last_active":"2023-10-06T11:33:23.627616+0000","last_peered":"2023-10-06T11:33:23.627616+0000","last_clean":"2023-10-06T11:33:23.627616+0000","last_became_active":"2023-10-06T11:33:23.621030+0000","last_became_peered":"2023-10-06T11:33:23.621030+0000","last_unstale":"2023-10-06T11:33:23.627616+0000","last_undegraded":"2023-10-06T11:33:23.627616+0000","last_fullsized":"2023-10-06T11:33:23.627616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:36.313062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624520+0000","last_change":"2023-10-06T11:33:23.617812+0000","last_active":"2023-10-06T11:33:23.624520+0000","last_peered":"2023-10-06T11:33:23.624520+0000","last_clean":"2023-10-06T11:33:23.624520+0000","last_became_active":"2023-10-06T11:33:23.616486+0000","last_became_peered":"2023-10-06T11:33:23.616486+0000","last_unstale":"2023-10-06T11:33:23.624520+0000","last_undegraded":"2023-10-06T11:33:23.624520+0000","last_fullsized":"2023-10-06T11:33:23.624520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:19.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623447+0000","last_change":"2023-10-06T11:33:23.617522+0000","last_active":"2023-10-06T11:33:23.623447+0000","last_peered":"2023-10-06T11:33:23.623447+0000","last_clean":"2023-10-06T11:33:23.623447+0000","last_became_active":"2023-10-06T11:33:23.617189+0000","last_became_peered":"2023-10-06T11:33:23.617189+0000","last_unstale":"2023-10-06T11:33:23.623447+0000","last_undegraded":"2023-10-06T11:33:23.623447+0000","last_fullsized":"2023-10-06T11:33:23.623447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:35.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614930+0000","last_change":"2023-10-06T11:33:23.611258+0000","last_active":"2023-10-06T11:33:23.614930+0000","last_peered":"2023-10-06T11:33:23.614930+0000","last_clean":"2023-10-06T11:33:23.614930+0000","last_became_active":"2023-10-06T11:33:23.610939+0000","last_became_peered":"2023-10-06T11:33:23.610939+0000","last_unstale":"2023-10-06T11:33:23.614930+0000","last_undegraded":"2023-10-06T11:33:23.614930+0000","last_fullsized":"2023-10-06T11:33:23.614930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:46.356768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.630914+0000","last_change":"2023-10-06T11:33:23.625995+0000","last_active":"2023-10-06T11:33:23.630914+0000","last_peered":"2023-10-06T11:33:23.630914+0000","last_clean":"2023-10-06T11:33:23.630914+0000","last_became_active":"2023-10-06T11:33:23.625609+0000","last_became_peered":"2023-10-06T11:33:23.625609+0000","last_unstale":"2023-10-06T11:33:23.630914+0000","last_undegraded":"2023-10-06T11:33:23.630914+0000","last_fullsized":"2023-10-06T11:33:23.630914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:13.594292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"78'4","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.629758+0000","last_change":"2023-10-06T11:33:22.608312+0000","last_active":"2023-10-06T11:33:23.629758+0000","last_peered":"2023-10-06T11:33:23.629758+0000","last_clean":"2023-10-06T11:33:23.629758+0000","last_became_active":"2023-10-06T11:33:22.608007+0000","last_became_peered":"2023-10-06T11:33:22.608007+0000","last_unstale":"2023-10-06T11:33:23.629758+0000","last_undegraded":"2023-10-06T11:33:23.629758+0000","last_fullsized":"2023-10-06T11:33:23.629758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:25.173229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627812+0000","last_change":"2023-10-06T11:33:23.624778+0000","last_active":"2023-10-06T11:33:23.627812+0000","last_peered":"2023-10-06T11:33:23.627812+0000","last_clean":"2023-10-06T11:33:23.627812+0000","last_became_active":"2023-10-06T11:33:23.624445+0000","last_became_peered":"2023-10-06T11:33:23.624445+0000","last_unstale":"2023-10-06T11:33:23.627812+0000","last_undegraded":"2023-10-06T11:33:23.627812+0000","last_fullsized":"2023-10-06T11:33:23.627812+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:09:33.956419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.626683+0000","last_change":"2023-10-06T11:33:22.609795+0000","last_active":"2023-10-06T11:33:23.626683+0000","last_peered":"2023-10-06T11:33:23.626683+0000","last_clean":"2023-10-06T11:33:23.626683+0000","last_became_active":"2023-10-06T11:33:22.609281+0000","last_became_peered":"2023-10-06T11:33:22.609281+0000","last_unstale":"2023-10-06T11:33:23.626683+0000","last_undegraded":"2023-10-06T11:33:23.626683+0000","last_fullsized":"2023-10-06T11:33:23.626683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:35.299926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.610322+0000","last_change":"2023-10-06T11:33:23.601603+0000","last_active":"2023-10-06T11:33:23.610322+0000","last_peered":"2023-10-06T11:33:23.610322+0000","last_clean":"2023-10-06T11:33:23.610322+0000","last_became_active":"2023-10-06T11:33:23.601288+0000","last_became_peered":"2023-10-06T11:33:23.601288+0000","last_unstale":"2023-10-06T11:33:23.610322+0000","last_undegraded":"2023-10-06T11:33:23.610322+0000","last_fullsized":"2023-10-06T11:33:23.610322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:41:46.184752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627130+0000","last_change":"2023-10-06T11:33:22.606718+0000","last_active":"2023-10-06T11:33:23.627130+0000","last_peered":"2023-10-06T11:33:23.627130+0000","last_clean":"2023-10-06T11:33:23.627130+0000","last_became_active":"2023-10-06T11:33:22.606404+0000","last_became_peered":"2023-10-06T11:33:22.606404+0000","last_unstale":"2023-10-06T11:33:23.627130+0000","last_undegraded":"2023-10-06T11:33:23.627130+0000","last_fullsized":"2023-10-06T11:33:23.627130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:40.001192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622371+0000","last_change":"2023-10-06T11:33:23.613523+0000","last_active":"2023-10-06T11:33:23.622371+0000","last_peered":"2023-10-06T11:33:23.622371+0000","last_clean":"2023-10-06T11:33:23.622371+0000","last_became_active":"2023-10-06T11:33:23.613184+0000","last_became_peered":"2023-10-06T11:33:23.613184+0000","last_unstale":"2023-10-06T11:33:23.622371+0000","last_undegraded":"2023-10-06T11:33:23.622371+0000","last_fullsized":"2023-10-06T11:33:23.622371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:41.014328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627931+0000","last_change":"2023-10-06T11:33:22.599222+0000","last_active":"2023-10-06T11:33:23.627931+0000","last_peered":"2023-10-06T11:33:23.627931+0000","last_clean":"2023-10-06T11:33:23.627931+0000","last_became_active":"2023-10-06T11:33:22.598935+0000","last_became_peered":"2023-10-06T11:33:22.598935+0000","last_unstale":"2023-10-06T11:33:23.627931+0000","last_undegraded":"2023-10-06T11:33:23.627931+0000","last_fullsized":"2023-10-06T11:33:23.627931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:18.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614340+0000","last_change":"2023-10-06T11:33:23.607597+0000","last_active":"2023-10-06T11:33:23.614340+0000","last_peered":"2023-10-06T11:33:23.614340+0000","last_clean":"2023-10-06T11:33:23.614340+0000","last_became_active":"2023-10-06T11:33:23.606440+0000","last_became_peered":"2023-10-06T11:33:23.606440+0000","last_unstale":"2023-10-06T11:33:23.614340+0000","last_undegraded":"2023-10-06T11:33:23.614340+0000","last_fullsized":"2023-10-06T11:33:23.614340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:36.676636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624969+0000","last_change":"2023-10-06T11:33:22.596627+0000","last_active":"2023-10-06T11:33:23.624969+0000","last_peered":"2023-10-06T11:33:23.624969+0000","last_clean":"2023-10-06T11:33:23.624969+0000","last_became_active":"2023-10-06T11:33:22.596298+0000","last_became_peered":"2023-10-06T11:33:22.596298+0000","last_unstale":"2023-10-06T11:33:23.624969+0000","last_undegraded":"2023-10-06T11:33:23.624969+0000","last_fullsized":"2023-10-06T11:33:23.624969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:12.062125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.621864+0000","last_change":"2023-10-06T11:33:23.612470+0000","last_active":"2023-10-06T11:33:23.621864+0000","last_peered":"2023-10-06T11:33:23.621864+0000","last_clean":"2023-10-06T11:33:23.621864+0000","last_became_active":"2023-10-06T11:33:23.611949+0000","last_became_peered":"2023-10-06T11:33:23.611949+0000","last_unstale":"2023-10-06T11:33:23.621864+0000","last_undegraded":"2023-10-06T11:33:23.621864+0000","last_fullsized":"2023-10-06T11:33:23.621864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:52.311438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.625752+0000","last_change":"2023-10-06T11:33:22.597755+0000","last_active":"2023-10-06T11:33:23.625752+0000","last_peered":"2023-10-06T11:33:23.625752+0000","last_clean":"2023-10-06T11:33:23.625752+0000","last_became_active":"2023-10-06T11:33:22.597452+0000","last_became_peered":"2023-10-06T11:33:22.597452+0000","last_unstale":"2023-10-06T11:33:23.625752+0000","last_undegraded":"2023-10-06T11:33:23.625752+0000","last_fullsized":"2023-10-06T11:33:23.625752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:10.906697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.620811+0000","last_change":"2023-10-06T11:33:23.609749+0000","last_active":"2023-10-06T11:33:23.620811+0000","last_peered":"2023-10-06T11:33:23.620811+0000","last_clean":"2023-10-06T11:33:23.620811+0000","last_became_active":"2023-10-06T11:33:23.609434+0000","last_became_peered":"2023-10-06T11:33:23.609434+0000","last_unstale":"2023-10-06T11:33:23.620811+0000","last_undegraded":"2023-10-06T11:33:23.620811+0000","last_fullsized":"2023-10-06T11:33:23.620811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:14.694506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.599082+0000","last_change":"2023-10-06T11:33:22.603183+0000","last_active":"2023-10-06T11:33:23.599082+0000","last_peered":"2023-10-06T11:33:23.599082+0000","last_clean":"2023-10-06T11:33:23.599082+0000","last_became_active":"2023-10-06T11:33:22.602844+0000","last_became_peered":"2023-10-06T11:33:22.602844+0000","last_unstale":"2023-10-06T11:33:23.599082+0000","last_undegraded":"2023-10-06T11:33:23.599082+0000","last_fullsized":"2023-10-06T11:33:23.599082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.617646+0000","last_change":"2023-10-06T11:33:23.616110+0000","last_active":"2023-10-06T11:33:23.617646+0000","last_peered":"2023-10-06T11:33:23.617646+0000","last_clean":"2023-10-06T11:33:23.617646+0000","last_became_active":"2023-10-06T11:33:23.615713+0000","last_became_peered":"2023-10-06T11:33:23.615713+0000","last_unstale":"2023-10-06T11:33:23.617646+0000","last_undegraded":"2023-10-06T11:33:23.617646+0000","last_fullsized":"2023-10-06T11:33:23.617646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.597228+0000","last_change":"2023-10-06T11:33:22.614701+0000","last_active":"2023-10-06T11:33:23.597228+0000","last_peered":"2023-10-06T11:33:23.597228+0000","last_clean":"2023-10-06T11:33:23.597228+0000","last_became_active":"2023-10-06T11:33:22.614414+0000","last_became_peered":"2023-10-06T11:33:22.614414+0000","last_unstale":"2023-10-06T11:33:23.597228+0000","last_undegraded":"2023-10-06T11:33:23.597228+0000","last_fullsized":"2023-10-06T11:33:23.597228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:50.928558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609391+0000","last_change":"2023-10-06T11:33:23.604348+0000","last_active":"2023-10-06T11:33:23.609391+0000","last_peered":"2023-10-06T11:33:23.609391+0000","last_clean":"2023-10-06T11:33:23.609391+0000","last_became_active":"2023-10-06T11:33:23.603945+0000","last_became_peered":"2023-10-06T11:33:23.603945+0000","last_unstale":"2023-10-06T11:33:23.609391+0000","last_undegraded":"2023-10-06T11:33:23.609391+0000","last_fullsized":"2023-10-06T11:33:23.609391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:18.862380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.602893+0000","last_change":"2023-10-06T11:33:22.619367+0000","last_active":"2023-10-06T11:33:23.602893+0000","last_peered":"2023-10-06T11:33:23.602893+0000","last_clean":"2023-10-06T11:33:23.602893+0000","last_became_active":"2023-10-06T11:33:22.619037+0000","last_became_peered":"2023-10-06T11:33:22.619037+0000","last_unstale":"2023-10-06T11:33:23.602893+0000","last_undegraded":"2023-10-06T11:33:23.602893+0000","last_fullsized":"2023-10-06T11:33:23.602893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:20.984640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615541+0000","last_change":"2023-10-06T11:33:23.613227+0000","last_active":"2023-10-06T11:33:23.615541+0000","last_peered":"2023-10-06T11:33:23.615541+0000","last_clean":"2023-10-06T11:33:23.615541+0000","last_became_active":"2023-10-06T11:33:23.612927+0000","last_became_peered":"2023-10-06T11:33:23.612927+0000","last_unstale":"2023-10-06T11:33:23.615541+0000","last_undegraded":"2023-10-06T11:33:23.615541+0000","last_fullsized":"2023-10-06T11:33:23.615541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:24:36.214989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.605868+0000","last_change":"2023-10-06T11:33:22.605988+0000","last_active":"2023-10-06T11:33:23.605868+0000","last_peered":"2023-10-06T11:33:23.605868+0000","last_clean":"2023-10-06T11:33:23.605868+0000","last_became_active":"2023-10-06T11:33:22.605715+0000","last_became_peered":"2023-10-06T11:33:22.605715+0000","last_unstale":"2023-10-06T11:33:23.605868+0000","last_undegraded":"2023-10-06T11:33:23.605868+0000","last_fullsized":"2023-10-06T11:33:23.605868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:58.803156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623096+0000","last_change":"2023-10-06T11:33:23.620690+0000","last_active":"2023-10-06T11:33:23.623096+0000","last_peered":"2023-10-06T11:33:23.623096+0000","last_clean":"2023-10-06T11:33:23.623096+0000","last_became_active":"2023-10-06T11:33:23.620357+0000","last_became_peered":"2023-10-06T11:33:23.620357+0000","last_unstale":"2023-10-06T11:33:23.623096+0000","last_undegraded":"2023-10-06T11:33:23.623096+0000","last_fullsized":"2023-10-06T11:33:23.623096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:00.131085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"78'3","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637064+0000","last_change":"2023-10-06T11:33:22.617618+0000","last_active":"2023-10-06T11:33:23.637064+0000","last_peered":"2023-10-06T11:33:23.637064+0000","last_clean":"2023-10-06T11:33:23.637064+0000","last_became_active":"2023-10-06T11:33:22.617224+0000","last_became_peered":"2023-10-06T11:33:22.617224+0000","last_unstale":"2023-10-06T11:33:23.637064+0000","last_undegraded":"2023-10-06T11:33:23.637064+0000","last_fullsized":"2023-10-06T11:33:23.637064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622441+0000","last_change":"2023-10-06T11:33:23.615876+0000","last_active":"2023-10-06T11:33:23.622441+0000","last_peered":"2023-10-06T11:33:23.622441+0000","last_clean":"2023-10-06T11:33:23.622441+0000","last_became_active":"2023-10-06T11:33:23.615513+0000","last_became_peered":"2023-10-06T11:33:23.615513+0000","last_unstale":"2023-10-06T11:33:23.622441+0000","last_undegraded":"2023-10-06T11:33:23.622441+0000","last_fullsized":"2023-10-06T11:33:23.622441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:11.919833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"78'6","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.631164+0000","last_change":"2023-10-06T11:33:22.599253+0000","last_active":"2023-10-06T11:33:23.631164+0000","last_peered":"2023-10-06T11:33:23.631164+0000","last_clean":"2023-10-06T11:33:23.631164+0000","last_became_active":"2023-10-06T11:33:22.598955+0000","last_became_peered":"2023-10-06T11:33:22.598955+0000","last_unstale":"2023-10-06T11:33:23.631164+0000","last_undegraded":"2023-10-06T11:33:23.631164+0000","last_fullsized":"2023-10-06T11:33:23.631164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:29.655925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623534+0000","last_change":"2023-10-06T11:33:23.617679+0000","last_active":"2023-10-06T11:33:23.623534+0000","last_peered":"2023-10-06T11:33:23.623534+0000","last_clean":"2023-10-06T11:33:23.623534+0000","last_became_active":"2023-10-06T11:33:23.617382+0000","last_became_peered":"2023-10-06T11:33:23.617382+0000","last_unstale":"2023-10-06T11:33:23.623534+0000","last_undegraded":"2023-10-06T11:33:23.623534+0000","last_fullsized":"2023-10-06T11:33:23.623534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:51.813242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.639321+0000","last_change":"2023-10-06T11:33:22.619831+0000","last_active":"2023-10-06T11:33:23.639321+0000","last_peered":"2023-10-06T11:33:23.639321+0000","last_clean":"2023-10-06T11:33:23.639321+0000","last_became_active":"2023-10-06T11:33:22.619531+0000","last_became_peered":"2023-10-06T11:33:22.619531+0000","last_unstale":"2023-10-06T11:33:23.639321+0000","last_undegraded":"2023-10-06T11:33:23.639321+0000","last_fullsized":"2023-10-06T11:33:23.639321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:16.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624604+0000","last_change":"2023-10-06T11:33:23.618475+0000","last_active":"2023-10-06T11:33:23.624604+0000","last_peered":"2023-10-06T11:33:23.624604+0000","last_clean":"2023-10-06T11:33:23.624604+0000","last_became_active":"2023-10-06T11:33:23.618101+0000","last_became_peered":"2023-10-06T11:33:23.618101+0000","last_unstale":"2023-10-06T11:33:23.624604+0000","last_undegraded":"2023-10-06T11:33:23.624604+0000","last_fullsized":"2023-10-06T11:33:23.624604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.606958+0000","last_change":"2023-10-06T11:33:22.613776+0000","last_active":"2023-10-06T11:33:23.606958+0000","last_peered":"2023-10-06T11:33:23.606958+0000","last_clean":"2023-10-06T11:33:23.606958+0000","last_became_active":"2023-10-06T11:33:22.613485+0000","last_became_peered":"2023-10-06T11:33:22.613485+0000","last_unstale":"2023-10-06T11:33:23.606958+0000","last_undegraded":"2023-10-06T11:33:23.606958+0000","last_fullsized":"2023-10-06T11:33:23.606958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:56.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623091+0000","last_change":"2023-10-06T11:33:23.619494+0000","last_active":"2023-10-06T11:33:23.623091+0000","last_peered":"2023-10-06T11:33:23.623091+0000","last_clean":"2023-10-06T11:33:23.623091+0000","last_became_active":"2023-10-06T11:33:23.618664+0000","last_became_peered":"2023-10-06T11:33:23.618664+0000","last_unstale":"2023-10-06T11:33:23.623091+0000","last_undegraded":"2023-10-06T11:33:23.623091+0000","last_fullsized":"2023-10-06T11:33:23.623091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:45.182698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600724+0000","last_change":"2023-10-06T11:33:22.616267+0000","last_active":"2023-10-06T11:33:23.600724+0000","last_peered":"2023-10-06T11:33:23.600724+0000","last_clean":"2023-10-06T11:33:23.600724+0000","last_became_active":"2023-10-06T11:33:22.615855+0000","last_became_peered":"2023-10-06T11:33:22.615855+0000","last_unstale":"2023-10-06T11:33:23.600724+0000","last_undegraded":"2023-10-06T11:33:23.600724+0000","last_fullsized":"2023-10-06T11:33:23.600724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:12.581156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615110+0000","last_change":"2023-10-06T11:33:23.607922+0000","last_active":"2023-10-06T11:33:23.615110+0000","last_peered":"2023-10-06T11:33:23.615110+0000","last_clean":"2023-10-06T11:33:23.615110+0000","last_became_active":"2023-10-06T11:33:23.607639+0000","last_became_peered":"2023-10-06T11:33:23.607639+0000","last_unstale":"2023-10-06T11:33:23.615110+0000","last_undegraded":"2023-10-06T11:33:23.615110+0000","last_fullsized":"2023-10-06T11:33:23.615110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:19.061615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"78'8","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.639768+0000","last_change":"2023-10-06T11:33:22.614520+0000","last_active":"2023-10-06T11:33:23.639768+0000","last_peered":"2023-10-06T11:33:23.639768+0000","last_clean":"2023-10-06T11:33:23.639768+0000","last_became_active":"2023-10-06T11:33:22.614243+0000","last_became_peered":"2023-10-06T11:33:22.614243+0000","last_unstale":"2023-10-06T11:33:23.639768+0000","last_undegraded":"2023-10-06T11:33:23.639768+0000","last_fullsized":"2023-10-06T11:33:23.639768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:51.298302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.612830+0000","last_change":"2023-10-06T11:33:23.606980+0000","last_active":"2023-10-06T11:33:23.612830+0000","last_peered":"2023-10-06T11:33:23.612830+0000","last_clean":"2023-10-06T11:33:23.612830+0000","last_became_active":"2023-10-06T11:33:23.606671+0000","last_became_peered":"2023-10-06T11:33:23.606671+0000","last_unstale":"2023-10-06T11:33:23.612830+0000","last_undegraded":"2023-10-06T11:33:23.612830+0000","last_fullsized":"2023-10-06T11:33:23.612830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:18.554241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637513+0000","last_change":"2023-10-06T11:33:22.615405+0000","last_active":"2023-10-06T11:33:23.637513+0000","last_peered":"2023-10-06T11:33:23.637513+0000","last_clean":"2023-10-06T11:33:23.637513+0000","last_became_active":"2023-10-06T11:33:22.614765+0000","last_became_peered":"2023-10-06T11:33:22.614765+0000","last_unstale":"2023-10-06T11:33:23.637513+0000","last_undegraded":"2023-10-06T11:33:23.637513+0000","last_fullsized":"2023-10-06T11:33:23.637513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607958+0000","last_change":"2023-10-06T11:33:23.602976+0000","last_active":"2023-10-06T11:33:23.607958+0000","last_peered":"2023-10-06T11:33:23.607958+0000","last_clean":"2023-10-06T11:33:23.607958+0000","last_became_active":"2023-10-06T11:33:23.602529+0000","last_became_peered":"2023-10-06T11:33:23.602529+0000","last_unstale":"2023-10-06T11:33:23.607958+0000","last_undegraded":"2023-10-06T11:33:23.607958+0000","last_fullsized":"2023-10-06T11:33:23.607958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:55:57.225708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609578+0000","last_change":"2023-10-06T11:33:22.613321+0000","last_active":"2023-10-06T11:33:23.609578+0000","last_peered":"2023-10-06T11:33:23.609578+0000","last_clean":"2023-10-06T11:33:23.609578+0000","last_became_active":"2023-10-06T11:33:22.613013+0000","last_became_peered":"2023-10-06T11:33:22.613013+0000","last_unstale":"2023-10-06T11:33:23.609578+0000","last_undegraded":"2023-10-06T11:33:23.609578+0000","last_fullsized":"2023-10-06T11:33:23.609578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624618+0000","last_change":"2023-10-06T11:33:23.620035+0000","last_active":"2023-10-06T11:33:23.624618+0000","last_peered":"2023-10-06T11:33:23.624618+0000","last_clean":"2023-10-06T11:33:23.624618+0000","last_became_active":"2023-10-06T11:33:23.619662+0000","last_became_peered":"2023-10-06T11:33:23.619662+0000","last_unstale":"2023-10-06T11:33:23.624618+0000","last_undegraded":"2023-10-06T11:33:23.624618+0000","last_fullsized":"2023-10-06T11:33:23.624618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.592776+0000","last_change":"2023-10-06T11:33:22.606822+0000","last_active":"2023-10-06T11:33:23.592776+0000","last_peered":"2023-10-06T11:33:23.592776+0000","last_clean":"2023-10-06T11:33:23.592776+0000","last_became_active":"2023-10-06T11:33:22.606513+0000","last_became_peered":"2023-10-06T11:33:22.606513+0000","last_unstale":"2023-10-06T11:33:23.592776+0000","last_undegraded":"2023-10-06T11:33:23.592776+0000","last_fullsized":"2023-10-06T11:33:23.592776+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:58.046445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622939+0000","last_change":"2023-10-06T11:33:23.618918+0000","last_active":"2023-10-06T11:33:23.622939+0000","last_peered":"2023-10-06T11:33:23.622939+0000","last_clean":"2023-10-06T11:33:23.622939+0000","last_became_active":"2023-10-06T11:33:23.618587+0000","last_became_peered":"2023-10-06T11:33:23.618587+0000","last_unstale":"2023-10-06T11:33:23.622939+0000","last_undegraded":"2023-10-06T11:33:23.622939+0000","last_fullsized":"2023-10-06T11:33:23.622939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:21.997776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594647+0000","last_change":"2023-10-06T11:33:22.598153+0000","last_active":"2023-10-06T11:33:23.594647+0000","last_peered":"2023-10-06T11:33:23.594647+0000","last_clean":"2023-10-06T11:33:23.594647+0000","last_became_active":"2023-10-06T11:33:22.597683+0000","last_became_peered":"2023-10-06T11:33:22.597683+0000","last_unstale":"2023-10-06T11:33:23.594647+0000","last_undegraded":"2023-10-06T11:33:23.594647+0000","last_fullsized":"2023-10-06T11:33:23.594647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:43:57.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623109+0000","last_change":"2023-10-06T11:33:23.612023+0000","last_active":"2023-10-06T11:33:23.623109+0000","last_peered":"2023-10-06T11:33:23.623109+0000","last_clean":"2023-10-06T11:33:23.623109+0000","last_became_active":"2023-10-06T11:33:23.611687+0000","last_became_peered":"2023-10-06T11:33:23.611687+0000","last_unstale":"2023-10-06T11:33:23.623109+0000","last_undegraded":"2023-10-06T11:33:23.623109+0000","last_fullsized":"2023-10-06T11:33:23.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:35.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.601125+0000","last_change":"2023-10-06T11:33:22.616956+0000","last_active":"2023-10-06T11:33:23.601125+0000","last_peered":"2023-10-06T11:33:23.601125+0000","last_clean":"2023-10-06T11:33:23.601125+0000","last_became_active":"2023-10-06T11:33:22.616621+0000","last_became_peered":"2023-10-06T11:33:22.616621+0000","last_unstale":"2023-10-06T11:33:23.601125+0000","last_undegraded":"2023-10-06T11:33:23.601125+0000","last_fullsized":"2023-10-06T11:33:23.601125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622240+0000","last_change":"2023-10-06T11:33:23.612266+0000","last_active":"2023-10-06T11:33:23.622240+0000","last_peered":"2023-10-06T11:33:23.622240+0000","last_clean":"2023-10-06T11:33:23.622240+0000","last_became_active":"2023-10-06T11:33:23.611923+0000","last_became_peered":"2023-10-06T11:33:23.611923+0000","last_unstale":"2023-10-06T11:33:23.622240+0000","last_undegraded":"2023-10-06T11:33:23.622240+0000","last_fullsized":"2023-10-06T11:33:23.622240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:35:35.295378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598545+0000","last_change":"2023-10-06T11:33:22.598153+0000","last_active":"2023-10-06T11:33:23.598545+0000","last_peered":"2023-10-06T11:33:23.598545+0000","last_clean":"2023-10-06T11:33:23.598545+0000","last_became_active":"2023-10-06T11:33:22.596906+0000","last_became_peered":"2023-10-06T11:33:22.596906+0000","last_unstale":"2023-10-06T11:33:23.598545+0000","last_undegraded":"2023-10-06T11:33:23.598545+0000","last_fullsized":"2023-10-06T11:33:23.598545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:34.630065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.620187+0000","last_change":"2023-10-06T11:33:23.611164+0000","last_active":"2023-10-06T11:33:23.620187+0000","last_peered":"2023-10-06T11:33:23.620187+0000","last_clean":"2023-10-06T11:33:23.620187+0000","last_became_active":"2023-10-06T11:33:23.610350+0000","last_became_peered":"2023-10-06T11:33:23.610350+0000","last_unstale":"2023-10-06T11:33:23.620187+0000","last_undegraded":"2023-10-06T11:33:23.620187+0000","last_fullsized":"2023-10-06T11:33:23.620187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:08.369054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598956+0000","last_change":"2023-10-06T11:33:22.600332+0000","last_active":"2023-10-06T11:33:23.598956+0000","last_peered":"2023-10-06T11:33:23.598956+0000","last_clean":"2023-10-06T11:33:23.598956+0000","last_became_active":"2023-10-06T11:33:22.599975+0000","last_became_peered":"2023-10-06T11:33:22.599975+0000","last_unstale":"2023-10-06T11:33:23.598956+0000","last_undegraded":"2023-10-06T11:33:23.598956+0000","last_fullsized":"2023-10-06T11:33:23.598956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:58.280365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.619814+0000","last_change":"2023-10-06T11:33:23.618378+0000","last_active":"2023-10-06T11:33:23.619814+0000","last_peered":"2023-10-06T11:33:23.619814+0000","last_clean":"2023-10-06T11:33:23.619814+0000","last_became_active":"2023-10-06T11:33:23.617976+0000","last_became_peered":"2023-10-06T11:33:23.617976+0000","last_unstale":"2023-10-06T11:33:23.619814+0000","last_undegraded":"2023-10-06T11:33:23.619814+0000","last_fullsized":"2023-10-06T11:33:23.619814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:40.419613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.593962+0000","last_change":"2023-10-06T11:33:22.602448+0000","last_active":"2023-10-06T11:33:23.593962+0000","last_peered":"2023-10-06T11:33:23.593962+0000","last_clean":"2023-10-06T11:33:23.593962+0000","last_became_active":"2023-10-06T11:33:22.602104+0000","last_became_peered":"2023-10-06T11:33:22.602104+0000","last_unstale":"2023-10-06T11:33:23.593962+0000","last_undegraded":"2023-10-06T11:33:23.593962+0000","last_fullsized":"2023-10-06T11:33:23.593962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:07.248883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.621471+0000","last_change":"2023-10-06T11:33:23.611359+0000","last_active":"2023-10-06T11:33:23.621471+0000","last_peered":"2023-10-06T11:33:23.621471+0000","last_clean":"2023-10-06T11:33:23.621471+0000","last_became_active":"2023-10-06T11:33:23.611011+0000","last_became_peered":"2023-10-06T11:33:23.611011+0000","last_unstale":"2023-10-06T11:33:23.621471+0000","last_undegraded":"2023-10-06T11:33:23.621471+0000","last_fullsized":"2023-10-06T11:33:23.621471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:09.451685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598437+0000","last_change":"2023-10-06T11:33:22.597320+0000","last_active":"2023-10-06T11:33:23.598437+0000","last_peered":"2023-10-06T11:33:23.598437+0000","last_clean":"2023-10-06T11:33:23.598437+0000","last_became_active":"2023-10-06T11:33:22.596977+0000","last_became_peered":"2023-10-06T11:33:22.596977+0000","last_unstale":"2023-10-06T11:33:23.598437+0000","last_undegraded":"2023-10-06T11:33:23.598437+0000","last_fullsized":"2023-10-06T11:33:23.598437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:12.581156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":370,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600714+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:33:23.600714+0000","last_peered":"2023-10-06T11:33:23.600714+0000","last_clean":"2023-10-06T11:33:23.600714+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:33:23.600714+0000","last_undegraded":"2023-10-06T11:33:23.600714+0000","last_fullsized":"2023-10-06T11:33:23.600714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.608191+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:33:23.608191+0000","last_peered":"2023-10-06T11:33:23.608191+0000","last_clean":"2023-10-06T11:33:23.608191+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:33:23.608191+0000","last_undegraded":"2023-10-06T11:33:23.608191+0000","last_fullsized":"2023-10-06T11:33:23.608191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.612795+0000","last_change":"2023-10-06T11:33:23.610818+0000","last_active":"2023-10-06T11:33:23.612795+0000","last_peered":"2023-10-06T11:33:23.612795+0000","last_clean":"2023-10-06T11:33:23.612795+0000","last_became_active":"2023-10-06T11:33:23.610397+0000","last_became_peered":"2023-10-06T11:33:23.610397+0000","last_unstale":"2023-10-06T11:33:23.612795+0000","last_undegraded":"2023-10-06T11:33:23.612795+0000","last_fullsized":"2023-10-06T11:33:23.612795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:27.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.625468+0000","last_change":"2023-10-06T11:33:22.603754+0000","last_active":"2023-10-06T11:33:23.625468+0000","last_peered":"2023-10-06T11:33:23.625468+0000","last_clean":"2023-10-06T11:33:23.625468+0000","last_became_active":"2023-10-06T11:33:22.603428+0000","last_became_peered":"2023-10-06T11:33:22.603428+0000","last_unstale":"2023-10-06T11:33:23.625468+0000","last_undegraded":"2023-10-06T11:33:23.625468+0000","last_fullsized":"2023-10-06T11:33:23.625468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:20.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600797+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:33:23.600797+0000","last_peered":"2023-10-06T11:33:23.600797+0000","last_clean":"2023-10-06T11:33:23.600797+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:33:23.600797+0000","last_undegraded":"2023-10-06T11:33:23.600797+0000","last_fullsized":"2023-10-06T11:33:23.600797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624584+0000","last_change":"2023-10-06T11:33:23.615471+0000","last_active":"2023-10-06T11:33:23.624584+0000","last_peered":"2023-10-06T11:33:23.624584+0000","last_clean":"2023-10-06T11:33:23.624584+0000","last_became_active":"2023-10-06T11:33:23.615165+0000","last_became_peered":"2023-10-06T11:33:23.615165+0000","last_unstale":"2023-10-06T11:33:23.624584+0000","last_undegraded":"2023-10-06T11:33:23.624584+0000","last_fullsized":"2023-10-06T11:33:23.624584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607073+0000","last_change":"2023-10-06T11:33:22.609602+0000","last_active":"2023-10-06T11:33:23.607073+0000","last_peered":"2023-10-06T11:33:23.607073+0000","last_clean":"2023-10-06T11:33:23.607073+0000","last_became_active":"2023-10-06T11:33:22.609235+0000","last_became_peered":"2023-10-06T11:33:22.609235+0000","last_unstale":"2023-10-06T11:33:23.607073+0000","last_undegraded":"2023-10-06T11:33:23.607073+0000","last_fullsized":"2023-10-06T11:33:23.607073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:59.117949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.586319+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:33:23.586319+0000","last_peered":"2023-10-06T11:33:23.586319+0000","last_clean":"2023-10-06T11:33:23.586319+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:33:23.586319+0000","last_undegraded":"2023-10-06T11:33:23.586319+0000","last_fullsized":"2023-10-06T11:33:23.586319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.611813+0000","last_change":"2023-10-06T11:33:23.607398+0000","last_active":"2023-10-06T11:33:23.611813+0000","last_peered":"2023-10-06T11:33:23.611813+0000","last_clean":"2023-10-06T11:33:23.611813+0000","last_became_active":"2023-10-06T11:33:23.607003+0000","last_became_peered":"2023-10-06T11:33:23.607003+0000","last_unstale":"2023-10-06T11:33:23.611813+0000","last_undegraded":"2023-10-06T11:33:23.611813+0000","last_fullsized":"2023-10-06T11:33:23.611813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:17.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.591365+0000","last_change":"2023-10-06T11:33:22.607981+0000","last_active":"2023-10-06T11:33:23.591365+0000","last_peered":"2023-10-06T11:33:23.591365+0000","last_clean":"2023-10-06T11:33:23.591365+0000","last_became_active":"2023-10-06T11:33:22.607679+0000","last_became_peered":"2023-10-06T11:33:22.607679+0000","last_unstale":"2023-10-06T11:33:23.591365+0000","last_undegraded":"2023-10-06T11:33:23.591365+0000","last_fullsized":"2023-10-06T11:33:23.591365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:43.287144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607176+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:33:23.607176+0000","last_peered":"2023-10-06T11:33:23.607176+0000","last_clean":"2023-10-06T11:33:23.607176+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:33:23.607176+0000","last_undegraded":"2023-10-06T11:33:23.607176+0000","last_fullsized":"2023-10-06T11:33:23.607176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627322+0000","last_change":"2023-10-06T11:33:23.621327+0000","last_active":"2023-10-06T11:33:23.627322+0000","last_peered":"2023-10-06T11:33:23.627322+0000","last_clean":"2023-10-06T11:33:23.627322+0000","last_became_active":"2023-10-06T11:33:23.621001+0000","last_became_peered":"2023-10-06T11:33:23.621001+0000","last_unstale":"2023-10-06T11:33:23.627322+0000","last_undegraded":"2023-10-06T11:33:23.627322+0000","last_fullsized":"2023-10-06T11:33:23.627322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:21.680372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.635729+0000","last_change":"2023-10-06T11:33:22.610379+0000","last_active":"2023-10-06T11:33:23.635729+0000","last_peered":"2023-10-06T11:33:23.635729+0000","last_clean":"2023-10-06T11:33:23.635729+0000","last_became_active":"2023-10-06T11:33:22.610075+0000","last_became_peered":"2023-10-06T11:33:22.610075+0000","last_unstale":"2023-10-06T11:33:23.635729+0000","last_undegraded":"2023-10-06T11:33:23.635729+0000","last_fullsized":"2023-10-06T11:33:23.635729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:16.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594729+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:33:23.594729+0000","last_peered":"2023-10-06T11:33:23.594729+0000","last_clean":"2023-10-06T11:33:23.594729+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:33:23.594729+0000","last_undegraded":"2023-10-06T11:33:23.594729+0000","last_fullsized":"2023-10-06T11:33:23.594729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623321+0000","last_change":"2023-10-06T11:33:23.620650+0000","last_active":"2023-10-06T11:33:23.623321+0000","last_peered":"2023-10-06T11:33:23.623321+0000","last_clean":"2023-10-06T11:33:23.623321+0000","last_became_active":"2023-10-06T11:33:23.620346+0000","last_became_peered":"2023-10-06T11:33:23.620346+0000","last_unstale":"2023-10-06T11:33:23.623321+0000","last_undegraded":"2023-10-06T11:33:23.623321+0000","last_fullsized":"2023-10-06T11:33:23.623321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:03.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.636854+0000","last_change":"2023-10-06T11:33:22.619272+0000","last_active":"2023-10-06T11:33:23.636854+0000","last_peered":"2023-10-06T11:33:23.636854+0000","last_clean":"2023-10-06T11:33:23.636854+0000","last_became_active":"2023-10-06T11:33:22.618615+0000","last_became_peered":"2023-10-06T11:33:22.618615+0000","last_unstale":"2023-10-06T11:33:23.636854+0000","last_undegraded":"2023-10-06T11:33:23.636854+0000","last_fullsized":"2023-10-06T11:33:23.636854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:56.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.601968+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:33:23.601968+0000","last_peered":"2023-10-06T11:33:23.601968+0000","last_clean":"2023-10-06T11:33:23.601968+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:33:23.601968+0000","last_undegraded":"2023-10-06T11:33:23.601968+0000","last_fullsized":"2023-10-06T11:33:23.601968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627785+0000","last_change":"2023-10-06T11:33:23.624440+0000","last_active":"2023-10-06T11:33:23.627785+0000","last_peered":"2023-10-06T11:33:23.627785+0000","last_clean":"2023-10-06T11:33:23.627785+0000","last_became_active":"2023-10-06T11:33:23.623829+0000","last_became_peered":"2023-10-06T11:33:23.623829+0000","last_unstale":"2023-10-06T11:33:23.627785+0000","last_undegraded":"2023-10-06T11:33:23.627785+0000","last_fullsized":"2023-10-06T11:33:23.627785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:41.298737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637151+0000","last_change":"2023-10-06T11:33:22.619236+0000","last_active":"2023-10-06T11:33:23.637151+0000","last_peered":"2023-10-06T11:33:23.637151+0000","last_clean":"2023-10-06T11:33:23.637151+0000","last_became_active":"2023-10-06T11:33:22.618945+0000","last_became_peered":"2023-10-06T11:33:22.618945+0000","last_unstale":"2023-10-06T11:33:23.637151+0000","last_undegraded":"2023-10-06T11:33:23.637151+0000","last_fullsized":"2023-10-06T11:33:23.637151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:59.117949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594175+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:33:23.594175+0000","last_peered":"2023-10-06T11:33:23.594175+0000","last_clean":"2023-10-06T11:33:23.594175+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:33:23.594175+0000","last_undegraded":"2023-10-06T11:33:23.594175+0000","last_fullsized":"2023-10-06T11:33:23.594175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622956+0000","last_change":"2023-10-06T11:33:23.618963+0000","last_active":"2023-10-06T11:33:23.622956+0000","last_peered":"2023-10-06T11:33:23.622956+0000","last_clean":"2023-10-06T11:33:23.622956+0000","last_became_active":"2023-10-06T11:33:23.618643+0000","last_became_peered":"2023-10-06T11:33:23.618643+0000","last_unstale":"2023-10-06T11:33:23.622956+0000","last_undegraded":"2023-10-06T11:33:23.622956+0000","last_fullsized":"2023-10-06T11:33:23.622956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:19.363557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598712+0000","last_change":"2023-10-06T11:33:22.601550+0000","last_active":"2023-10-06T11:33:23.598712+0000","last_peered":"2023-10-06T11:33:23.598712+0000","last_clean":"2023-10-06T11:33:23.598712+0000","last_became_active":"2023-10-06T11:33:22.600465+0000","last_became_peered":"2023-10-06T11:33:22.600465+0000","last_unstale":"2023-10-06T11:33:23.598712+0000","last_undegraded":"2023-10-06T11:33:23.598712+0000","last_fullsized":"2023-10-06T11:33:23.598712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598251+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:33:23.598251+0000","last_peered":"2023-10-06T11:33:23.598251+0000","last_clean":"2023-10-06T11:33:23.598251+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:33:23.598251+0000","last_undegraded":"2023-10-06T11:33:23.598251+0000","last_fullsized":"2023-10-06T11:33:23.598251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311092,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060748,"statfs":{"total":96636764160,"available":96318205952,"internally_reserved":0,"allocated":1552384,"data_stored":1063542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1359999999999999,"15min":1.135},"min":{"1min":0.79100000000000004,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.423,"5min":1.571,"15min":1.827},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.649,"15min":2.0030000000000001},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":1.103,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.851,"5min":1.871,"15min":2.6059999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2350000000000001,"15min":1.25},"min":{"1min":1.117,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.829,"5min":1.829,"15min":1.931},"last":1.3979999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.234,"15min":1.2130000000000001},"min":{"1min":0.79400000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.554,"5min":1.827,"15min":2.3799999999999999},"last":1.335},{"interface":"front","average":{"1min":1.2,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.90300000000000002,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.5369999999999999,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.284,"15min":1.2849999999999999},"min":{"1min":1.042,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6970000000000001,"5min":1.9450000000000001,"15min":2.665},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.94899999999999995,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.754,"5min":1.796,"15min":2.8570000000000002},"last":1.1160000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.71399999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5329999999999999,"5min":2.149,"15min":2.7970000000000002},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.268,"15min":1.278},"min":{"1min":0.87,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5760000000000001,"5min":2.0499999999999998,"15min":2.5339999999999998},"last":1.046}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3240000000000001,"15min":1.3320000000000001},"min":{"1min":0.98399999999999999,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.29},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3149999999999999,"15min":1.2909999999999999},"min":{"1min":1.081,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.6259999999999999,"5min":2.2480000000000002,"15min":3.105},"last":1.304}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3700000000000001},"min":{"1min":1.01,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.742,"5min":2.4289999999999998,"15min":3.004},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4119999999999999},"min":{"1min":0.76000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.913,"5min":2.121,"15min":2.9119999999999999},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":479545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.28,"15min":1.2889999999999999},"min":{"1min":1.089,"5min":0.621,"15min":0.621},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.385},{"interface":"front","average":{"1min":1.454,"5min":1.3149999999999999,"15min":1.3220000000000001},"min":{"1min":1.0289999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.659,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.323}]},{"osd":1,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1850000000000001,"15min":1.163},"min":{"1min":0.91300000000000003,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.5980000000000001,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.226,"15min":1.212},"min":{"1min":0.88400000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.569,"5min":1.669,"15min":1.8480000000000001},"last":1.605}]},{"osd":2,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.1879999999999999,"15min":1.196},"min":{"1min":0.91000000000000003,"5min":0.78300000000000003,"15min":0.74299999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.6919999999999999},"last":1.083},{"interface":"front","average":{"1min":1.24,"5min":1.2170000000000001,"15min":1.2330000000000001},"min":{"1min":0.91300000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4930000000000001,"5min":1.615,"15min":2.7789999999999999},"last":1.026}]},{"osd":3,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2929999999999999,"15min":1.3140000000000001},"min":{"1min":1,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6990000000000001,"5min":1.764,"15min":1.9850000000000001},"last":1.53},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3029999999999999,"15min":1.3080000000000001},"min":{"1min":1.1459999999999999,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.7130000000000001,"5min":1.724,"15min":2.109},"last":1.194}]},{"osd":4,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2430000000000001,"15min":1.228},"min":{"1min":0.85899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.728,"5min":1.8,"15min":1.956},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.2929999999999999,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.8879999999999999},"last":1.274}]},{"osd":5,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.79800000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.552,"5min":1.72,"15min":2.198},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.7969999999999999,"5min":1.879,"15min":2.1509999999999998},"last":1.718}]},{"osd":7,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":1.1479999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.772,"5min":1.927,"15min":2.2919999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.444},"min":{"1min":0.95799999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6419999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.4410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":472448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.409,"15min":1.2669999999999999},"min":{"1min":0.876,"5min":0.5,"15min":0.5},"max":{"1min":1.573,"5min":17.626000000000001,"15min":17.626000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3069999999999999,"15min":1.3100000000000001},"min":{"1min":1.097,"5min":0.77600000000000002,"15min":0.64900000000000002},"max":{"1min":1.508,"5min":1.972,"15min":2.544},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1910000000000001,"15min":1.1830000000000001},"min":{"1min":0.81100000000000005,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.161},{"interface":"front","average":{"1min":1.119,"5min":1.278,"15min":1.2290000000000001},"min":{"1min":0.91800000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.359,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.3500000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.171,"15min":1.218},"min":{"1min":0.92800000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5960000000000001,"5min":1.6879999999999999,"15min":3.609},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2390000000000001,"15min":1.244},"min":{"1min":0.878,"5min":0.878,"15min":0.73299999999999998},"max":{"1min":1.6679999999999999,"5min":1.738,"15min":2.6949999999999998},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3899999999999999,"15min":1.3149999999999999},"min":{"1min":0.94299999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.643,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3089999999999999,"15min":1.359},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.6899999999999999,"5min":1.734,"15min":3.7770000000000001},"last":1.21}]},{"osd":4,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3200000000000001,"15min":1.274},"min":{"1min":0.80400000000000005,"5min":0.68100000000000005,"15min":0.40699999999999997},"max":{"1min":1.5569999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4850000000000001,"15min":1.4219999999999999},"min":{"1min":0.90800000000000003,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.7110000000000001,"5min":18.48,"15min":18.48},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.319,"15min":1.3340000000000001},"min":{"1min":0.85299999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.6020000000000001,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.387,"15min":1.3680000000000001},"min":{"1min":0.92600000000000005,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.885,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.385}]},{"osd":7,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.5129999999999999,"15min":1.4319999999999999},"min":{"1min":1.0600000000000001,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7769999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.415},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.4710000000000001},"min":{"1min":0.97099999999999997,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7829999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.1200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302328,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069512,"statfs":{"total":96636764160,"available":96327180288,"internally_reserved":0,"allocated":966656,"data_stored":481047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.137,"15min":1.145},"min":{"1min":0.97699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.119,"15min":1.1539999999999999},"min":{"1min":0.74299999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4530000000000001,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3180000000000001,"15min":1.339},"min":{"1min":0.85499999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.6830000000000001,"15min":2.1989999999999998},"last":1.135},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":1.1060000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":1.7450000000000001,"15min":1.851},"last":1.085}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1859999999999999,"15min":1.214},"min":{"1min":0.77600000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.4770000000000001,"5min":1.5449999999999999,"15min":2.2269999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.266,"15min":1.278},"min":{"1min":0.95399999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.617,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.0109999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.214,"15min":1.218},"min":{"1min":0.999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.4930000000000001,"5min":1.671,"15min":1.6839999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2929999999999999,"15min":1.3129999999999999},"min":{"1min":1.085,"5min":0.82999999999999996,"15min":0.77400000000000002},"max":{"1min":1.663,"5min":1.6639999999999999,"15min":2.258},"last":1.575}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.181,"15min":1.2250000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5549999999999999,"5min":1.7390000000000001,"15min":2.8250000000000002},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2150000000000001,"15min":1.204},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.40999999999999998},"max":{"1min":1.613,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.2210000000000001,"15min":1.2589999999999999},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.54900000000000004},"max":{"1min":1.518,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.421},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3560000000000001,"15min":1.377},"min":{"1min":0.93400000000000005,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.5609999999999999,"5min":1.8129999999999999,"15min":2.4329999999999998},"last":1.2350000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3620000000000001,"15min":1.361},"min":{"1min":1.054,"5min":0.95799999999999996,"15min":0.69399999999999995},"max":{"1min":1.579,"5min":1.841,"15min":2.653},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.401,"15min":1.4079999999999999},"min":{"1min":1.028,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":1.657,"5min":1.8480000000000001,"15min":2.464},"last":1.6739999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306412,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065428,"statfs":{"total":96636764160,"available":96322998272,"internally_reserved":0,"allocated":954368,"data_stored":479511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97299999999999998,"15min":0.97899999999999998},"min":{"1min":0.68600000000000005,"5min":0.54800000000000004,"15min":0.437},"max":{"1min":1.504,"5min":1.5129999999999999,"15min":1.627},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.91700000000000004,"15min":0.95699999999999996},"min":{"1min":0.68400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4079999999999999,"5min":1.448,"15min":1.516},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.075,"15min":1.075},"min":{"1min":0.77000000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.45,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.091,"15min":1.123},"min":{"1min":0.879,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4019999999999999,"5min":1.6830000000000001,"15min":1.887},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.147,"15min":1.202},"min":{"1min":0.86899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.556,"5min":1.5629999999999999,"15min":1.7030000000000001},"last":1.2270000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1859999999999999,"15min":1.2430000000000001},"min":{"1min":0.86599999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7110000000000001,"5min":1.768,"15min":2.6640000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3160000000000001,"15min":1.363},"min":{"1min":1.083,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.617,"5min":1.718,"15min":2.0840000000000001},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.3660000000000001,"15min":1.415},"min":{"1min":1.0620000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7470000000000001,"5min":1.883,"15min":2.1699999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.365,"15min":1.401},"min":{"1min":1.036,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.857,"5min":1.857,"15min":2.1179999999999999},"last":1.496}]},{"osd":6,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.415,"15min":1.45},"min":{"1min":0.94999999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.3440000000000001,"15min":1.3879999999999999},"min":{"1min":1.1020000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.7649999999999999,"5min":1.823,"15min":2.137},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4039999999999999,"15min":1.4490000000000001},"min":{"1min":1.0589999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6990000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.296},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4490000000000001,"15min":1.488},"min":{"1min":1.089,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.8080000000000001,"5min":2.266,"15min":2.266},"last":1.46}]}]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310508,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061332,"statfs":{"total":96636764160,"available":96318803968,"internally_reserved":0,"allocated":954368,"data_stored":479511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.95799999999999996,"15min":0.96499999999999997},"min":{"1min":0.69999999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4179999999999999,"5min":1.4930000000000001,"15min":1.8380000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0609999999999999,"15min":1.069},"min":{"1min":0.80600000000000005,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.415,"5min":1.5309999999999999,"15min":1.9650000000000001},"last":1.103}]},{"osd":1,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92800000000000005,"15min":0.92200000000000004},"min":{"1min":0.70399999999999996,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.2769999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0389999999999999,"15min":1.0249999999999999},"min":{"1min":0.78900000000000003,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.4339999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.181}]},{"osd":3,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1499999999999999,"15min":1.1830000000000001},"min":{"1min":0.81100000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.405,"5min":1.8240000000000001,"15min":1.917},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2869999999999999,"15min":1.302},"min":{"1min":0.90900000000000003,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.593,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.96499999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.222},"min":{"1min":0.88200000000000001,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.575,"5min":1.754,"15min":2.1899999999999999},"last":1.165},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.268,"15min":1.3049999999999999},"min":{"1min":0.93799999999999994,"5min":0.86199999999999999,"15min":0.84399999999999997},"max":{"1min":1.5800000000000001,"5min":2.024,"15min":2.6920000000000002},"last":1.149}]},{"osd":5,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.367,"15min":1.369},"min":{"1min":1.0720000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.742,"5min":2.0680000000000001,"15min":2.601},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.329,"15min":1.3400000000000001},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5489999999999999,"5min":1.778,"15min":2.6400000000000001},"last":1.228}]},{"osd":6,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":1.0489999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.6599999999999999,"5min":1.8779999999999999,"15min":2.8340000000000001},"last":1.208},{"interface":"front","average":{"1min":1.431,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":1.0229999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8160000000000001,"5min":2.617,"15min":2.617},"last":1.2450000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3660000000000001,"15min":1.3919999999999999},"min":{"1min":0.97199999999999998,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.667,"5min":1.863,"15min":2.9020000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4370000000000001,"15min":1.47},"min":{"1min":1.0740000000000001,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.73,"5min":2.2829999999999999,"15min":3.2240000000000002},"last":1.2609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306440,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065400,"statfs":{"total":96636764160,"available":96322969600,"internally_reserved":0,"allocated":983040,"data_stored":481783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.70499999999999996,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":1.3340000000000001,"5min":1.724,"15min":1.724},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":1.3100000000000001,"5min":1.4259999999999999,"15min":1.855},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.72699999999999998,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.2729999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":1.101},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1339999999999999,"15min":1.1100000000000001},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5269999999999999,"5min":2.137,"15min":2.137},"last":1.0289999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.266,"15min":1.2370000000000001},"min":{"1min":1.0069999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6339999999999999,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.111},"min":{"1min":0.79100000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.3959999999999999,"5min":1.7210000000000001,"15min":4.0830000000000002},"last":1.2929999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.216,"15min":1.274},"min":{"1min":1.022,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.4330000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":1.238,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.736,"5min":1.7589999999999999,"15min":3.5609999999999999},"last":1.337}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.325,"15min":1.345},"min":{"1min":1.119,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.361,"15min":1.3640000000000001},"min":{"1min":1.121,"5min":0.94699999999999995,"15min":0.85999999999999999},"max":{"1min":1.5489999999999999,"5min":2.1240000000000001,"15min":2.766},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4890000000000001,"15min":1.492},"min":{"1min":1.246,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8169999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.462},{"interface":"front","average":{"1min":1.353,"5min":1.3999999999999999,"15min":1.419},"min":{"1min":1.0589999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0600000000000001,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.671}]},{"osd":7,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4950000000000001,"15min":1.5269999999999999},"min":{"1min":1.1519999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.9390000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.474},"min":{"1min":1.3080000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6799999999999999,"5min":1.9099999999999999,"15min":2.8740000000000001},"last":1.577}]}]},{"osd":0,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306980,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064860,"statfs":{"total":96636764160,"available":96322416640,"internally_reserved":0,"allocated":1536000,"data_stored":1062794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.66700000000000004,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.286,"5min":1.3560000000000001,"15min":1.571},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.94599999999999995,"15min":0.96499999999999997},"min":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.49,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0129999999999999,"15min":1.0329999999999999},"min":{"1min":0.83499999999999996,"5min":0.58899999999999997,"15min":0.56699999999999995},"max":{"1min":1.379,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.054,"15min":1.085},"min":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.4239999999999999,"5min":1.6160000000000001,"15min":2.7719999999999998},"last":0.91500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0800000000000001,"15min":1.083},"min":{"1min":0.86499999999999999,"5min":0.70999999999999996,"15min":0.64000000000000001},"max":{"1min":1.3999999999999999,"5min":1.609,"15min":1.919},"last":1.079},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.82999999999999996,"5min":0.70399999999999996,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":1.121}]},{"osd":4,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3520000000000001,"15min":1.3500000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.68999999999999995},"max":{"1min":1.8440000000000001,"5min":2.294,"15min":2.294},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3149999999999999,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.641,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5469999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.3859999999999999,"15min":1.385},"min":{"1min":1.2370000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.819,"5min":2.0859999999999999,"15min":2.573},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3859999999999999,"15min":1.4179999999999999},"min":{"1min":1.115,"5min":1.0860000000000001,"15min":0.84599999999999997},"max":{"1min":1.6799999999999999,"5min":1.982,"15min":2.8650000000000002},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":1.0469999999999999,"5min":0.89300000000000002,"15min":0.88900000000000001},"max":{"1min":1.7949999999999999,"5min":2.2240000000000002,"15min":2.52},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4450000000000001,"15min":1.466},"min":{"1min":1.0660000000000001,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":2.0030000000000001,"5min":2.254,"15min":2.742},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.458,"15min":1.504},"min":{"1min":1.28,"5min":1.012,"15min":0.98199999999999998},"max":{"1min":1.7070000000000001,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4079999999999999,"15min":1.4339999999999999},"min":{"1min":1.181,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.847,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.6839999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:33:27.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:33:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 Processor -- start 2023-10-06T11:33:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 -- start start 2023-10-06T11:33:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc54c0c3c40 0x7fc54c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0ccfa0 con 0x7fc54c0c3160 2023-10-06T11:33:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0cb7b0 con 0x7fc54c0cd430 2023-10-06T11:33:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc550963700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0cc390 con 0x7fc54c0c3c40 2023-10-06T11:33:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc54b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc54c0c3160 0x7fc54c0cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33292/0 (socket says 172.21.15.46:33292) 2023-10-06T11:33:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc54b7fe700 1 -- 172.21.15.46:0/1287157948 learned_addr learned my addr 172.21.15.46:0/1287157948 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.084+0000 7fc54bfff700 1 --2- 172.21.15.46:0/1287157948 >> v2:172.21.15.132:3300/0 conn(0x7fc54c0c3c40 0x7fc54c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345934946 0 0) 0x7fc54c0cb7b0 con 0x7fc54c0cd430 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc538003570 con 0x7fc54c0cd430 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914966042 0 0) 0x7fc54c0ccfa0 con 0x7fc54c0c3160 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc54c0cb7b0 con 0x7fc54c0c3160 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54bfff700 1 -- 172.21.15.46:0/1287157948 >> v1:172.21.15.132:6789/0 conn(0x7fc54c0cd430 legacy=0x7fc54c0c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54bfff700 1 -- 172.21.15.46:0/1287157948 >> v1:172.21.15.46:6789/0 conn(0x7fc54c0c3160 legacy=0x7fc54c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54bfff700 1 -- 172.21.15.46:0/1287157948 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0cc670 con 0x7fc54c0c3c40 2023-10-06T11:33:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54bfff700 1 --2- 172.21.15.46:0/1287157948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c3c40 0x7fc54c0c84c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc544009e60 tx=0x7fc544009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae7706055688cb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400e070 con 0x7fc54c0c3c40 2023-10-06T11:33:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400bab0 con 0x7fc54c0c3c40 2023-10-06T11:33:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc54affd700 1 -- 172.21.15.46:0/1287157948 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400be70 con 0x7fc54c0c3c40 2023-10-06T11:33:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 -- 172.21.15.46:0/1287157948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c3c40 msgr2=0x7fc54c0c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 --2- 172.21.15.46:0/1287157948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c3c40 0x7fc54c0c84c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc544009e60 tx=0x7fc544009780 comp rx=0 tx=0).stop 2023-10-06T11:33:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 -- 172.21.15.46:0/1287157948 shutdown_connections 2023-10-06T11:33:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 --2- 172.21.15.46:0/1287157948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c3c40 0x7fc54c0c84c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 -- 172.21.15.46:0/1287157948 >> 172.21.15.46:0/1287157948 conn(0x7fc54c0970c0 msgr2=0x7fc54c067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 -- 172.21.15.46:0/1287157948 shutdown_connections 2023-10-06T11:33:28.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.088+0000 7fc550963700 1 -- 172.21.15.46:0/1287157948 wait complete. 2023-10-06T11:33:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 Processor -- start 2023-10-06T11:33:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 -- start start 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0c3c40 0x7fc54c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0cd430 0x7fc54c0b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0b3a90 con 0x7fc54c0c3160 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c16cb00 con 0x7fc54c0cd430 2023-10-06T11:33:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc550963700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0b3540 con 0x7fc54c0c3c40 2023-10-06T11:33:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40394/0 (socket says 172.21.15.46:40394) 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc54b7fe700 1 -- 172.21.15.46:0/7606593 learned_addr learned my addr 172.21.15.46:0/7606593 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc54bfff700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0cd430 0x7fc54c0b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.092+0000 7fc543fff700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0c3c40 0x7fc54c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 -- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0c3c40 msgr2=0x7fc54c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0c3c40 0x7fc54c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 -- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0cd430 msgr2=0x7fc54c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0cd430 0x7fc54c0b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 -- 172.21.15.46:0/7606593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544009a20 con 0x7fc54c0c3160 2023-10-06T11:33:28.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c7f0 tx=0x7fc53c007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0073b0 con 0x7fc54c0c3160 2023-10-06T11:33:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c007510 con 0x7fc54c0c3160 2023-10-06T11:33:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005a00 con 0x7fc54c0c3160 2023-10-06T11:33:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c0b37b0 con 0x7fc54c0c3160 2023-10-06T11:33:28.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c16a150 con 0x7fc54c0c3160 2023-10-06T11:33:28.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.096+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c064590 con 0x7fc54c0c3160 2023-10-06T11:33:28.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.100+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc53c01b3b0 con 0x7fc54c0c3160 2023-10-06T11:33:28.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.100+0000 7fc5437fe700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 0x7fc52c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.100+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fc53c07c090 con 0x7fc54c0c3160 2023-10-06T11:33:28.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.100+0000 7fc543fff700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 0x7fc52c062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.104+0000 7fc543fff700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 0x7fc52c062ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc54c06b430 tx=0x7fc534007d50 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.104+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc53c049be0 con 0x7fc54c0c3160 2023-10-06T11:33:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.272+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc54c16c8f0 con 0x7fc52c0606f0 2023-10-06T11:33:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.288+0000 7fc5437fe700 1 -- 172.21.15.46:0/7606593 <== mgr.6150 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308791 (secure 0 0 0) 0x7fc53400f620 con 0x7fc52c0606f0 2023-10-06T11:33:28.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:28.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 msgr2=0x7fc52c062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 0x7fc52c062ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc54c06b430 tx=0x7fc534007d50 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 msgr2=0x7fc54c152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c7f0 tx=0x7fc53c007c80 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 shutdown_connections 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc52c0606f0 0x7fc52c062ba0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0cd430 0x7fc54c0b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0c3c40 0x7fc54c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 --2- 172.21.15.46:0/7606593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c3160 0x7fc54c152c80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 >> 172.21.15.46:0/7606593 conn(0x7fc54c0970c0 msgr2=0x7fc54c1539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 shutdown_connections 2023-10-06T11:33:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.296+0000 7fc550963700 1 -- 172.21.15.46:0/7606593 wait complete. 2023-10-06T11:33:28.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:33:28.329 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:33:27.232519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448388,"kb_used_data":8644,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752526332,"statfs":{"total":773094113280,"available":770586963968,"internally_reserved":0,"allocated":8851456,"data_stored":5000181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.669656"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624008+0000","last_change":"2023-10-06T11:33:23.614474+0000","last_active":"2023-10-06T11:33:23.624008+0000","last_peered":"2023-10-06T11:33:23.624008+0000","last_clean":"2023-10-06T11:33:23.624008+0000","last_became_active":"2023-10-06T11:33:23.614195+0000","last_became_peered":"2023-10-06T11:33:23.614195+0000","last_unstale":"2023-10-06T11:33:23.624008+0000","last_undegraded":"2023-10-06T11:33:23.624008+0000","last_fullsized":"2023-10-06T11:33:23.624008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:45.182698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609398+0000","last_change":"2023-10-06T11:33:23.602424+0000","last_active":"2023-10-06T11:33:23.609398+0000","last_peered":"2023-10-06T11:33:23.609398+0000","last_clean":"2023-10-06T11:33:23.609398+0000","last_became_active":"2023-10-06T11:33:23.602127+0000","last_became_peered":"2023-10-06T11:33:23.602127+0000","last_unstale":"2023-10-06T11:33:23.609398+0000","last_undegraded":"2023-10-06T11:33:23.609398+0000","last_fullsized":"2023-10-06T11:33:23.609398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:40:17.879667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614335+0000","last_change":"2023-10-06T11:33:23.610017+0000","last_active":"2023-10-06T11:33:23.614335+0000","last_peered":"2023-10-06T11:33:23.614335+0000","last_clean":"2023-10-06T11:33:23.614335+0000","last_became_active":"2023-10-06T11:33:23.609714+0000","last_became_peered":"2023-10-06T11:33:23.609714+0000","last_unstale":"2023-10-06T11:33:23.614335+0000","last_undegraded":"2023-10-06T11:33:23.614335+0000","last_fullsized":"2023-10-06T11:33:23.614335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622836+0000","last_change":"2023-10-06T11:33:23.614075+0000","last_active":"2023-10-06T11:33:23.622836+0000","last_peered":"2023-10-06T11:33:23.622836+0000","last_clean":"2023-10-06T11:33:23.622836+0000","last_became_active":"2023-10-06T11:33:23.612877+0000","last_became_peered":"2023-10-06T11:33:23.612877+0000","last_unstale":"2023-10-06T11:33:23.622836+0000","last_undegraded":"2023-10-06T11:33:23.622836+0000","last_fullsized":"2023-10-06T11:33:23.622836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:13.075260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622900+0000","last_change":"2023-10-06T11:33:23.611515+0000","last_active":"2023-10-06T11:33:23.622900+0000","last_peered":"2023-10-06T11:33:23.622900+0000","last_clean":"2023-10-06T11:33:23.622900+0000","last_became_active":"2023-10-06T11:33:23.611214+0000","last_became_peered":"2023-10-06T11:33:23.611214+0000","last_unstale":"2023-10-06T11:33:23.622900+0000","last_undegraded":"2023-10-06T11:33:23.622900+0000","last_fullsized":"2023-10-06T11:33:23.622900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:19.061615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.616555+0000","last_change":"2023-10-06T11:33:23.614534+0000","last_active":"2023-10-06T11:33:23.616555+0000","last_peered":"2023-10-06T11:33:23.616555+0000","last_clean":"2023-10-06T11:33:23.616555+0000","last_became_active":"2023-10-06T11:33:23.612294+0000","last_became_peered":"2023-10-06T11:33:23.612294+0000","last_unstale":"2023-10-06T11:33:23.616555+0000","last_undegraded":"2023-10-06T11:33:23.616555+0000","last_fullsized":"2023-10-06T11:33:23.616555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:59.265142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.613609+0000","last_change":"2023-10-06T11:33:23.604365+0000","last_active":"2023-10-06T11:33:23.613609+0000","last_peered":"2023-10-06T11:33:23.613609+0000","last_clean":"2023-10-06T11:33:23.613609+0000","last_became_active":"2023-10-06T11:33:23.604055+0000","last_became_peered":"2023-10-06T11:33:23.604055+0000","last_unstale":"2023-10-06T11:33:23.613609+0000","last_undegraded":"2023-10-06T11:33:23.613609+0000","last_fullsized":"2023-10-06T11:33:23.613609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:22:23.482421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633210+0000","last_change":"2023-10-06T11:33:23.625528+0000","last_active":"2023-10-06T11:33:23.633210+0000","last_peered":"2023-10-06T11:33:23.633210+0000","last_clean":"2023-10-06T11:33:23.633210+0000","last_became_active":"2023-10-06T11:33:23.625193+0000","last_became_peered":"2023-10-06T11:33:23.625193+0000","last_unstale":"2023-10-06T11:33:23.633210+0000","last_undegraded":"2023-10-06T11:33:23.633210+0000","last_fullsized":"2023-10-06T11:33:23.633210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627814+0000","last_change":"2023-10-06T11:33:23.622261+0000","last_active":"2023-10-06T11:33:23.627814+0000","last_peered":"2023-10-06T11:33:23.627814+0000","last_clean":"2023-10-06T11:33:23.627814+0000","last_became_active":"2023-10-06T11:33:23.621933+0000","last_became_peered":"2023-10-06T11:33:23.621933+0000","last_unstale":"2023-10-06T11:33:23.627814+0000","last_undegraded":"2023-10-06T11:33:23.627814+0000","last_fullsized":"2023-10-06T11:33:23.627814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:19.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615002+0000","last_change":"2023-10-06T11:33:23.612701+0000","last_active":"2023-10-06T11:33:23.615002+0000","last_peered":"2023-10-06T11:33:23.615002+0000","last_clean":"2023-10-06T11:33:23.615002+0000","last_became_active":"2023-10-06T11:33:23.612407+0000","last_became_peered":"2023-10-06T11:33:23.612407+0000","last_unstale":"2023-10-06T11:33:23.615002+0000","last_undegraded":"2023-10-06T11:33:23.615002+0000","last_fullsized":"2023-10-06T11:33:23.615002+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623353+0000","last_change":"2023-10-06T11:33:23.614291+0000","last_active":"2023-10-06T11:33:23.623353+0000","last_peered":"2023-10-06T11:33:23.623353+0000","last_clean":"2023-10-06T11:33:23.623353+0000","last_became_active":"2023-10-06T11:33:23.613986+0000","last_became_peered":"2023-10-06T11:33:23.613986+0000","last_unstale":"2023-10-06T11:33:23.623353+0000","last_undegraded":"2023-10-06T11:33:23.623353+0000","last_fullsized":"2023-10-06T11:33:23.623353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:21.181531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633134+0000","last_change":"2023-10-06T11:33:23.625211+0000","last_active":"2023-10-06T11:33:23.633134+0000","last_peered":"2023-10-06T11:33:23.633134+0000","last_clean":"2023-10-06T11:33:23.633134+0000","last_became_active":"2023-10-06T11:33:23.624910+0000","last_became_peered":"2023-10-06T11:33:23.624910+0000","last_unstale":"2023-10-06T11:33:23.633134+0000","last_undegraded":"2023-10-06T11:33:23.633134+0000","last_fullsized":"2023-10-06T11:33:23.633134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:44.300280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633331+0000","last_change":"2023-10-06T11:33:23.626574+0000","last_active":"2023-10-06T11:33:23.633331+0000","last_peered":"2023-10-06T11:33:23.633331+0000","last_clean":"2023-10-06T11:33:23.633331+0000","last_became_active":"2023-10-06T11:33:23.626112+0000","last_became_peered":"2023-10-06T11:33:23.626112+0000","last_unstale":"2023-10-06T11:33:23.633331+0000","last_undegraded":"2023-10-06T11:33:23.633331+0000","last_fullsized":"2023-10-06T11:33:23.633331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:36.313062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615070+0000","last_change":"2023-10-06T11:33:23.611354+0000","last_active":"2023-10-06T11:33:23.615070+0000","last_peered":"2023-10-06T11:33:23.615070+0000","last_clean":"2023-10-06T11:33:23.615070+0000","last_became_active":"2023-10-06T11:33:23.610978+0000","last_became_peered":"2023-10-06T11:33:23.610978+0000","last_unstale":"2023-10-06T11:33:23.615070+0000","last_undegraded":"2023-10-06T11:33:23.615070+0000","last_fullsized":"2023-10-06T11:33:23.615070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615228+0000","last_change":"2023-10-06T11:33:23.612617+0000","last_active":"2023-10-06T11:33:23.615228+0000","last_peered":"2023-10-06T11:33:23.615228+0000","last_clean":"2023-10-06T11:33:23.615228+0000","last_became_active":"2023-10-06T11:33:23.612296+0000","last_became_peered":"2023-10-06T11:33:23.612296+0000","last_unstale":"2023-10-06T11:33:23.615228+0000","last_undegraded":"2023-10-06T11:33:23.615228+0000","last_fullsized":"2023-10-06T11:33:23.615228+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:59.059580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622626+0000","last_change":"2023-10-06T11:33:23.618761+0000","last_active":"2023-10-06T11:33:23.622626+0000","last_peered":"2023-10-06T11:33:23.622626+0000","last_clean":"2023-10-06T11:33:23.622626+0000","last_became_active":"2023-10-06T11:33:23.618426+0000","last_became_peered":"2023-10-06T11:33:23.618426+0000","last_unstale":"2023-10-06T11:33:23.622626+0000","last_undegraded":"2023-10-06T11:33:23.622626+0000","last_fullsized":"2023-10-06T11:33:23.622626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:47.164365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.611072+0000","last_change":"2023-10-06T11:33:23.604575+0000","last_active":"2023-10-06T11:33:23.611072+0000","last_peered":"2023-10-06T11:33:23.611072+0000","last_clean":"2023-10-06T11:33:23.611072+0000","last_became_active":"2023-10-06T11:33:23.604024+0000","last_became_peered":"2023-10-06T11:33:23.604024+0000","last_unstale":"2023-10-06T11:33:23.611072+0000","last_undegraded":"2023-10-06T11:33:23.611072+0000","last_fullsized":"2023-10-06T11:33:23.611072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:59.816291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622772+0000","last_change":"2023-10-06T11:33:23.610239+0000","last_active":"2023-10-06T11:33:23.622772+0000","last_peered":"2023-10-06T11:33:23.622772+0000","last_clean":"2023-10-06T11:33:23.622772+0000","last_became_active":"2023-10-06T11:33:23.609919+0000","last_became_peered":"2023-10-06T11:33:23.609919+0000","last_unstale":"2023-10-06T11:33:23.622772+0000","last_undegraded":"2023-10-06T11:33:23.622772+0000","last_fullsized":"2023-10-06T11:33:23.622772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:59.293501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622990+0000","last_change":"2023-10-06T11:33:23.619699+0000","last_active":"2023-10-06T11:33:23.622990+0000","last_peered":"2023-10-06T11:33:23.622990+0000","last_clean":"2023-10-06T11:33:23.622990+0000","last_became_active":"2023-10-06T11:33:23.619385+0000","last_became_peered":"2023-10-06T11:33:23.619385+0000","last_unstale":"2023-10-06T11:33:23.622990+0000","last_undegraded":"2023-10-06T11:33:23.622990+0000","last_fullsized":"2023-10-06T11:33:23.622990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:57.076608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.610252+0000","last_change":"2023-10-06T11:33:23.602037+0000","last_active":"2023-10-06T11:33:23.610252+0000","last_peered":"2023-10-06T11:33:23.610252+0000","last_clean":"2023-10-06T11:33:23.610252+0000","last_became_active":"2023-10-06T11:33:23.601760+0000","last_became_peered":"2023-10-06T11:33:23.601760+0000","last_unstale":"2023-10-06T11:33:23.610252+0000","last_undegraded":"2023-10-06T11:33:23.610252+0000","last_fullsized":"2023-10-06T11:33:23.610252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.632591+0000","last_change":"2023-10-06T11:33:23.621794+0000","last_active":"2023-10-06T11:33:23.632591+0000","last_peered":"2023-10-06T11:33:23.632591+0000","last_clean":"2023-10-06T11:33:23.632591+0000","last_became_active":"2023-10-06T11:33:23.621468+0000","last_became_peered":"2023-10-06T11:33:23.621468+0000","last_unstale":"2023-10-06T11:33:23.632591+0000","last_undegraded":"2023-10-06T11:33:23.632591+0000","last_fullsized":"2023-10-06T11:33:23.632591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:59.059580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614285+0000","last_change":"2023-10-06T11:33:23.608839+0000","last_active":"2023-10-06T11:33:23.614285+0000","last_peered":"2023-10-06T11:33:23.614285+0000","last_clean":"2023-10-06T11:33:23.614285+0000","last_became_active":"2023-10-06T11:33:23.608523+0000","last_became_peered":"2023-10-06T11:33:23.608523+0000","last_unstale":"2023-10-06T11:33:23.614285+0000","last_undegraded":"2023-10-06T11:33:23.614285+0000","last_fullsized":"2023-10-06T11:33:23.614285+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:51.931723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614185+0000","last_change":"2023-10-06T11:33:23.605409+0000","last_active":"2023-10-06T11:33:23.614185+0000","last_peered":"2023-10-06T11:33:23.614185+0000","last_clean":"2023-10-06T11:33:23.614185+0000","last_became_active":"2023-10-06T11:33:23.605076+0000","last_became_peered":"2023-10-06T11:33:23.605076+0000","last_unstale":"2023-10-06T11:33:23.614185+0000","last_undegraded":"2023-10-06T11:33:23.614185+0000","last_fullsized":"2023-10-06T11:33:23.614185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614484+0000","last_change":"2023-10-06T11:33:23.609258+0000","last_active":"2023-10-06T11:33:23.614484+0000","last_peered":"2023-10-06T11:33:23.614484+0000","last_clean":"2023-10-06T11:33:23.614484+0000","last_became_active":"2023-10-06T11:33:23.608978+0000","last_became_peered":"2023-10-06T11:33:23.608978+0000","last_unstale":"2023-10-06T11:33:23.614484+0000","last_undegraded":"2023-10-06T11:33:23.614484+0000","last_fullsized":"2023-10-06T11:33:23.614484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633389+0000","last_change":"2023-10-06T11:33:23.626717+0000","last_active":"2023-10-06T11:33:23.633389+0000","last_peered":"2023-10-06T11:33:23.633389+0000","last_clean":"2023-10-06T11:33:23.633389+0000","last_became_active":"2023-10-06T11:33:23.625611+0000","last_became_peered":"2023-10-06T11:33:23.625611+0000","last_unstale":"2023-10-06T11:33:23.633389+0000","last_undegraded":"2023-10-06T11:33:23.633389+0000","last_fullsized":"2023-10-06T11:33:23.633389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:21.680372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.633361+0000","last_change":"2023-10-06T11:33:23.626845+0000","last_active":"2023-10-06T11:33:23.633361+0000","last_peered":"2023-10-06T11:33:23.633361+0000","last_clean":"2023-10-06T11:33:23.633361+0000","last_became_active":"2023-10-06T11:33:23.625171+0000","last_became_peered":"2023-10-06T11:33:23.625171+0000","last_unstale":"2023-10-06T11:33:23.633361+0000","last_undegraded":"2023-10-06T11:33:23.633361+0000","last_fullsized":"2023-10-06T11:33:23.633361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:41.014328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624905+0000","last_change":"2023-10-06T11:33:23.617605+0000","last_active":"2023-10-06T11:33:23.624905+0000","last_peered":"2023-10-06T11:33:23.624905+0000","last_clean":"2023-10-06T11:33:23.624905+0000","last_became_active":"2023-10-06T11:33:23.617276+0000","last_became_peered":"2023-10-06T11:33:23.617276+0000","last_unstale":"2023-10-06T11:33:23.624905+0000","last_undegraded":"2023-10-06T11:33:23.624905+0000","last_fullsized":"2023-10-06T11:33:23.624905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:13.594292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609373+0000","last_change":"2023-10-06T11:33:23.601978+0000","last_active":"2023-10-06T11:33:23.609373+0000","last_peered":"2023-10-06T11:33:23.609373+0000","last_clean":"2023-10-06T11:33:23.609373+0000","last_became_active":"2023-10-06T11:33:23.601685+0000","last_became_peered":"2023-10-06T11:33:23.601685+0000","last_unstale":"2023-10-06T11:33:23.609373+0000","last_undegraded":"2023-10-06T11:33:23.609373+0000","last_fullsized":"2023-10-06T11:33:23.609373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:01.232455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627616+0000","last_change":"2023-10-06T11:33:23.621354+0000","last_active":"2023-10-06T11:33:23.627616+0000","last_peered":"2023-10-06T11:33:23.627616+0000","last_clean":"2023-10-06T11:33:23.627616+0000","last_became_active":"2023-10-06T11:33:23.621030+0000","last_became_peered":"2023-10-06T11:33:23.621030+0000","last_unstale":"2023-10-06T11:33:23.627616+0000","last_undegraded":"2023-10-06T11:33:23.627616+0000","last_fullsized":"2023-10-06T11:33:23.627616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:36.313062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624520+0000","last_change":"2023-10-06T11:33:23.617812+0000","last_active":"2023-10-06T11:33:23.624520+0000","last_peered":"2023-10-06T11:33:23.624520+0000","last_clean":"2023-10-06T11:33:23.624520+0000","last_became_active":"2023-10-06T11:33:23.616486+0000","last_became_peered":"2023-10-06T11:33:23.616486+0000","last_unstale":"2023-10-06T11:33:23.624520+0000","last_undegraded":"2023-10-06T11:33:23.624520+0000","last_fullsized":"2023-10-06T11:33:23.624520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:19.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623447+0000","last_change":"2023-10-06T11:33:23.617522+0000","last_active":"2023-10-06T11:33:23.623447+0000","last_peered":"2023-10-06T11:33:23.623447+0000","last_clean":"2023-10-06T11:33:23.623447+0000","last_became_active":"2023-10-06T11:33:23.617189+0000","last_became_peered":"2023-10-06T11:33:23.617189+0000","last_unstale":"2023-10-06T11:33:23.623447+0000","last_undegraded":"2023-10-06T11:33:23.623447+0000","last_fullsized":"2023-10-06T11:33:23.623447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:35.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614930+0000","last_change":"2023-10-06T11:33:23.611258+0000","last_active":"2023-10-06T11:33:23.614930+0000","last_peered":"2023-10-06T11:33:23.614930+0000","last_clean":"2023-10-06T11:33:23.614930+0000","last_became_active":"2023-10-06T11:33:23.610939+0000","last_became_peered":"2023-10-06T11:33:23.610939+0000","last_unstale":"2023-10-06T11:33:23.614930+0000","last_undegraded":"2023-10-06T11:33:23.614930+0000","last_fullsized":"2023-10-06T11:33:23.614930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:46.356768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.630914+0000","last_change":"2023-10-06T11:33:23.625995+0000","last_active":"2023-10-06T11:33:23.630914+0000","last_peered":"2023-10-06T11:33:23.630914+0000","last_clean":"2023-10-06T11:33:23.630914+0000","last_became_active":"2023-10-06T11:33:23.625609+0000","last_became_peered":"2023-10-06T11:33:23.625609+0000","last_unstale":"2023-10-06T11:33:23.630914+0000","last_undegraded":"2023-10-06T11:33:23.630914+0000","last_fullsized":"2023-10-06T11:33:23.630914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:13.594292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"78'4","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.629758+0000","last_change":"2023-10-06T11:33:22.608312+0000","last_active":"2023-10-06T11:33:23.629758+0000","last_peered":"2023-10-06T11:33:23.629758+0000","last_clean":"2023-10-06T11:33:23.629758+0000","last_became_active":"2023-10-06T11:33:22.608007+0000","last_became_peered":"2023-10-06T11:33:22.608007+0000","last_unstale":"2023-10-06T11:33:23.629758+0000","last_undegraded":"2023-10-06T11:33:23.629758+0000","last_fullsized":"2023-10-06T11:33:23.629758+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:25.173229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627812+0000","last_change":"2023-10-06T11:33:23.624778+0000","last_active":"2023-10-06T11:33:23.627812+0000","last_peered":"2023-10-06T11:33:23.627812+0000","last_clean":"2023-10-06T11:33:23.627812+0000","last_became_active":"2023-10-06T11:33:23.624445+0000","last_became_peered":"2023-10-06T11:33:23.624445+0000","last_unstale":"2023-10-06T11:33:23.627812+0000","last_undegraded":"2023-10-06T11:33:23.627812+0000","last_fullsized":"2023-10-06T11:33:23.627812+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:09:33.956419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.626683+0000","last_change":"2023-10-06T11:33:22.609795+0000","last_active":"2023-10-06T11:33:23.626683+0000","last_peered":"2023-10-06T11:33:23.626683+0000","last_clean":"2023-10-06T11:33:23.626683+0000","last_became_active":"2023-10-06T11:33:22.609281+0000","last_became_peered":"2023-10-06T11:33:22.609281+0000","last_unstale":"2023-10-06T11:33:23.626683+0000","last_undegraded":"2023-10-06T11:33:23.626683+0000","last_fullsized":"2023-10-06T11:33:23.626683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:35.299926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.610322+0000","last_change":"2023-10-06T11:33:23.601603+0000","last_active":"2023-10-06T11:33:23.610322+0000","last_peered":"2023-10-06T11:33:23.610322+0000","last_clean":"2023-10-06T11:33:23.610322+0000","last_became_active":"2023-10-06T11:33:23.601288+0000","last_became_peered":"2023-10-06T11:33:23.601288+0000","last_unstale":"2023-10-06T11:33:23.610322+0000","last_undegraded":"2023-10-06T11:33:23.610322+0000","last_fullsized":"2023-10-06T11:33:23.610322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:41:46.184752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627130+0000","last_change":"2023-10-06T11:33:22.606718+0000","last_active":"2023-10-06T11:33:23.627130+0000","last_peered":"2023-10-06T11:33:23.627130+0000","last_clean":"2023-10-06T11:33:23.627130+0000","last_became_active":"2023-10-06T11:33:22.606404+0000","last_became_peered":"2023-10-06T11:33:22.606404+0000","last_unstale":"2023-10-06T11:33:23.627130+0000","last_undegraded":"2023-10-06T11:33:23.627130+0000","last_fullsized":"2023-10-06T11:33:23.627130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:40.001192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622371+0000","last_change":"2023-10-06T11:33:23.613523+0000","last_active":"2023-10-06T11:33:23.622371+0000","last_peered":"2023-10-06T11:33:23.622371+0000","last_clean":"2023-10-06T11:33:23.622371+0000","last_became_active":"2023-10-06T11:33:23.613184+0000","last_became_peered":"2023-10-06T11:33:23.613184+0000","last_unstale":"2023-10-06T11:33:23.622371+0000","last_undegraded":"2023-10-06T11:33:23.622371+0000","last_fullsized":"2023-10-06T11:33:23.622371+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:58:41.014328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627931+0000","last_change":"2023-10-06T11:33:22.599222+0000","last_active":"2023-10-06T11:33:23.627931+0000","last_peered":"2023-10-06T11:33:23.627931+0000","last_clean":"2023-10-06T11:33:23.627931+0000","last_became_active":"2023-10-06T11:33:22.598935+0000","last_became_peered":"2023-10-06T11:33:22.598935+0000","last_unstale":"2023-10-06T11:33:23.627931+0000","last_undegraded":"2023-10-06T11:33:23.627931+0000","last_fullsized":"2023-10-06T11:33:23.627931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:18.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.614340+0000","last_change":"2023-10-06T11:33:23.607597+0000","last_active":"2023-10-06T11:33:23.614340+0000","last_peered":"2023-10-06T11:33:23.614340+0000","last_clean":"2023-10-06T11:33:23.614340+0000","last_became_active":"2023-10-06T11:33:23.606440+0000","last_became_peered":"2023-10-06T11:33:23.606440+0000","last_unstale":"2023-10-06T11:33:23.614340+0000","last_undegraded":"2023-10-06T11:33:23.614340+0000","last_fullsized":"2023-10-06T11:33:23.614340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:36.676636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624969+0000","last_change":"2023-10-06T11:33:22.596627+0000","last_active":"2023-10-06T11:33:23.624969+0000","last_peered":"2023-10-06T11:33:23.624969+0000","last_clean":"2023-10-06T11:33:23.624969+0000","last_became_active":"2023-10-06T11:33:22.596298+0000","last_became_peered":"2023-10-06T11:33:22.596298+0000","last_unstale":"2023-10-06T11:33:23.624969+0000","last_undegraded":"2023-10-06T11:33:23.624969+0000","last_fullsized":"2023-10-06T11:33:23.624969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:12.062125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.621864+0000","last_change":"2023-10-06T11:33:23.612470+0000","last_active":"2023-10-06T11:33:23.621864+0000","last_peered":"2023-10-06T11:33:23.621864+0000","last_clean":"2023-10-06T11:33:23.621864+0000","last_became_active":"2023-10-06T11:33:23.611949+0000","last_became_peered":"2023-10-06T11:33:23.611949+0000","last_unstale":"2023-10-06T11:33:23.621864+0000","last_undegraded":"2023-10-06T11:33:23.621864+0000","last_fullsized":"2023-10-06T11:33:23.621864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:52.311438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.625752+0000","last_change":"2023-10-06T11:33:22.597755+0000","last_active":"2023-10-06T11:33:23.625752+0000","last_peered":"2023-10-06T11:33:23.625752+0000","last_clean":"2023-10-06T11:33:23.625752+0000","last_became_active":"2023-10-06T11:33:22.597452+0000","last_became_peered":"2023-10-06T11:33:22.597452+0000","last_unstale":"2023-10-06T11:33:23.625752+0000","last_undegraded":"2023-10-06T11:33:23.625752+0000","last_fullsized":"2023-10-06T11:33:23.625752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:10.906697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.620811+0000","last_change":"2023-10-06T11:33:23.609749+0000","last_active":"2023-10-06T11:33:23.620811+0000","last_peered":"2023-10-06T11:33:23.620811+0000","last_clean":"2023-10-06T11:33:23.620811+0000","last_became_active":"2023-10-06T11:33:23.609434+0000","last_became_peered":"2023-10-06T11:33:23.609434+0000","last_unstale":"2023-10-06T11:33:23.620811+0000","last_undegraded":"2023-10-06T11:33:23.620811+0000","last_fullsized":"2023-10-06T11:33:23.620811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:14.694506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.599082+0000","last_change":"2023-10-06T11:33:22.603183+0000","last_active":"2023-10-06T11:33:23.599082+0000","last_peered":"2023-10-06T11:33:23.599082+0000","last_clean":"2023-10-06T11:33:23.599082+0000","last_became_active":"2023-10-06T11:33:22.602844+0000","last_became_peered":"2023-10-06T11:33:22.602844+0000","last_unstale":"2023-10-06T11:33:23.599082+0000","last_undegraded":"2023-10-06T11:33:23.599082+0000","last_fullsized":"2023-10-06T11:33:23.599082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.617646+0000","last_change":"2023-10-06T11:33:23.616110+0000","last_active":"2023-10-06T11:33:23.617646+0000","last_peered":"2023-10-06T11:33:23.617646+0000","last_clean":"2023-10-06T11:33:23.617646+0000","last_became_active":"2023-10-06T11:33:23.615713+0000","last_became_peered":"2023-10-06T11:33:23.615713+0000","last_unstale":"2023-10-06T11:33:23.617646+0000","last_undegraded":"2023-10-06T11:33:23.617646+0000","last_fullsized":"2023-10-06T11:33:23.617646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.597228+0000","last_change":"2023-10-06T11:33:22.614701+0000","last_active":"2023-10-06T11:33:23.597228+0000","last_peered":"2023-10-06T11:33:23.597228+0000","last_clean":"2023-10-06T11:33:23.597228+0000","last_became_active":"2023-10-06T11:33:22.614414+0000","last_became_peered":"2023-10-06T11:33:22.614414+0000","last_unstale":"2023-10-06T11:33:23.597228+0000","last_undegraded":"2023-10-06T11:33:23.597228+0000","last_fullsized":"2023-10-06T11:33:23.597228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:50.928558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609391+0000","last_change":"2023-10-06T11:33:23.604348+0000","last_active":"2023-10-06T11:33:23.609391+0000","last_peered":"2023-10-06T11:33:23.609391+0000","last_clean":"2023-10-06T11:33:23.609391+0000","last_became_active":"2023-10-06T11:33:23.603945+0000","last_became_peered":"2023-10-06T11:33:23.603945+0000","last_unstale":"2023-10-06T11:33:23.609391+0000","last_undegraded":"2023-10-06T11:33:23.609391+0000","last_fullsized":"2023-10-06T11:33:23.609391+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:18.862380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.602893+0000","last_change":"2023-10-06T11:33:22.619367+0000","last_active":"2023-10-06T11:33:23.602893+0000","last_peered":"2023-10-06T11:33:23.602893+0000","last_clean":"2023-10-06T11:33:23.602893+0000","last_became_active":"2023-10-06T11:33:22.619037+0000","last_became_peered":"2023-10-06T11:33:22.619037+0000","last_unstale":"2023-10-06T11:33:23.602893+0000","last_undegraded":"2023-10-06T11:33:23.602893+0000","last_fullsized":"2023-10-06T11:33:23.602893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:20.984640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615541+0000","last_change":"2023-10-06T11:33:23.613227+0000","last_active":"2023-10-06T11:33:23.615541+0000","last_peered":"2023-10-06T11:33:23.615541+0000","last_clean":"2023-10-06T11:33:23.615541+0000","last_became_active":"2023-10-06T11:33:23.612927+0000","last_became_peered":"2023-10-06T11:33:23.612927+0000","last_unstale":"2023-10-06T11:33:23.615541+0000","last_undegraded":"2023-10-06T11:33:23.615541+0000","last_fullsized":"2023-10-06T11:33:23.615541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:24:36.214989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.605868+0000","last_change":"2023-10-06T11:33:22.605988+0000","last_active":"2023-10-06T11:33:23.605868+0000","last_peered":"2023-10-06T11:33:23.605868+0000","last_clean":"2023-10-06T11:33:23.605868+0000","last_became_active":"2023-10-06T11:33:22.605715+0000","last_became_peered":"2023-10-06T11:33:22.605715+0000","last_unstale":"2023-10-06T11:33:23.605868+0000","last_undegraded":"2023-10-06T11:33:23.605868+0000","last_fullsized":"2023-10-06T11:33:23.605868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:47:58.803156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623096+0000","last_change":"2023-10-06T11:33:23.620690+0000","last_active":"2023-10-06T11:33:23.623096+0000","last_peered":"2023-10-06T11:33:23.623096+0000","last_clean":"2023-10-06T11:33:23.623096+0000","last_became_active":"2023-10-06T11:33:23.620357+0000","last_became_peered":"2023-10-06T11:33:23.620357+0000","last_unstale":"2023-10-06T11:33:23.623096+0000","last_undegraded":"2023-10-06T11:33:23.623096+0000","last_fullsized":"2023-10-06T11:33:23.623096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:00.131085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"78'3","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637064+0000","last_change":"2023-10-06T11:33:22.617618+0000","last_active":"2023-10-06T11:33:23.637064+0000","last_peered":"2023-10-06T11:33:23.637064+0000","last_clean":"2023-10-06T11:33:23.637064+0000","last_became_active":"2023-10-06T11:33:22.617224+0000","last_became_peered":"2023-10-06T11:33:22.617224+0000","last_unstale":"2023-10-06T11:33:23.637064+0000","last_undegraded":"2023-10-06T11:33:23.637064+0000","last_fullsized":"2023-10-06T11:33:23.637064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622441+0000","last_change":"2023-10-06T11:33:23.615876+0000","last_active":"2023-10-06T11:33:23.622441+0000","last_peered":"2023-10-06T11:33:23.622441+0000","last_clean":"2023-10-06T11:33:23.622441+0000","last_became_active":"2023-10-06T11:33:23.615513+0000","last_became_peered":"2023-10-06T11:33:23.615513+0000","last_unstale":"2023-10-06T11:33:23.622441+0000","last_undegraded":"2023-10-06T11:33:23.622441+0000","last_fullsized":"2023-10-06T11:33:23.622441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:11.919833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"78'6","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.631164+0000","last_change":"2023-10-06T11:33:22.599253+0000","last_active":"2023-10-06T11:33:23.631164+0000","last_peered":"2023-10-06T11:33:23.631164+0000","last_clean":"2023-10-06T11:33:23.631164+0000","last_became_active":"2023-10-06T11:33:22.598955+0000","last_became_peered":"2023-10-06T11:33:22.598955+0000","last_unstale":"2023-10-06T11:33:23.631164+0000","last_undegraded":"2023-10-06T11:33:23.631164+0000","last_fullsized":"2023-10-06T11:33:23.631164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:24:29.655925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623534+0000","last_change":"2023-10-06T11:33:23.617679+0000","last_active":"2023-10-06T11:33:23.623534+0000","last_peered":"2023-10-06T11:33:23.623534+0000","last_clean":"2023-10-06T11:33:23.623534+0000","last_became_active":"2023-10-06T11:33:23.617382+0000","last_became_peered":"2023-10-06T11:33:23.617382+0000","last_unstale":"2023-10-06T11:33:23.623534+0000","last_undegraded":"2023-10-06T11:33:23.623534+0000","last_fullsized":"2023-10-06T11:33:23.623534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:51.813242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.639321+0000","last_change":"2023-10-06T11:33:22.619831+0000","last_active":"2023-10-06T11:33:23.639321+0000","last_peered":"2023-10-06T11:33:23.639321+0000","last_clean":"2023-10-06T11:33:23.639321+0000","last_became_active":"2023-10-06T11:33:22.619531+0000","last_became_peered":"2023-10-06T11:33:22.619531+0000","last_unstale":"2023-10-06T11:33:23.639321+0000","last_undegraded":"2023-10-06T11:33:23.639321+0000","last_fullsized":"2023-10-06T11:33:23.639321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:16.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624604+0000","last_change":"2023-10-06T11:33:23.618475+0000","last_active":"2023-10-06T11:33:23.624604+0000","last_peered":"2023-10-06T11:33:23.624604+0000","last_clean":"2023-10-06T11:33:23.624604+0000","last_became_active":"2023-10-06T11:33:23.618101+0000","last_became_peered":"2023-10-06T11:33:23.618101+0000","last_unstale":"2023-10-06T11:33:23.624604+0000","last_undegraded":"2023-10-06T11:33:23.624604+0000","last_fullsized":"2023-10-06T11:33:23.624604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:44.714757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.606958+0000","last_change":"2023-10-06T11:33:22.613776+0000","last_active":"2023-10-06T11:33:23.606958+0000","last_peered":"2023-10-06T11:33:23.606958+0000","last_clean":"2023-10-06T11:33:23.606958+0000","last_became_active":"2023-10-06T11:33:22.613485+0000","last_became_peered":"2023-10-06T11:33:22.613485+0000","last_unstale":"2023-10-06T11:33:23.606958+0000","last_undegraded":"2023-10-06T11:33:23.606958+0000","last_fullsized":"2023-10-06T11:33:23.606958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:56.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623091+0000","last_change":"2023-10-06T11:33:23.619494+0000","last_active":"2023-10-06T11:33:23.623091+0000","last_peered":"2023-10-06T11:33:23.623091+0000","last_clean":"2023-10-06T11:33:23.623091+0000","last_became_active":"2023-10-06T11:33:23.618664+0000","last_became_peered":"2023-10-06T11:33:23.618664+0000","last_unstale":"2023-10-06T11:33:23.623091+0000","last_undegraded":"2023-10-06T11:33:23.623091+0000","last_fullsized":"2023-10-06T11:33:23.623091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:45.182698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600724+0000","last_change":"2023-10-06T11:33:22.616267+0000","last_active":"2023-10-06T11:33:23.600724+0000","last_peered":"2023-10-06T11:33:23.600724+0000","last_clean":"2023-10-06T11:33:23.600724+0000","last_became_active":"2023-10-06T11:33:22.615855+0000","last_became_peered":"2023-10-06T11:33:22.615855+0000","last_unstale":"2023-10-06T11:33:23.600724+0000","last_undegraded":"2023-10-06T11:33:23.600724+0000","last_fullsized":"2023-10-06T11:33:23.600724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:12.581156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.615110+0000","last_change":"2023-10-06T11:33:23.607922+0000","last_active":"2023-10-06T11:33:23.615110+0000","last_peered":"2023-10-06T11:33:23.615110+0000","last_clean":"2023-10-06T11:33:23.615110+0000","last_became_active":"2023-10-06T11:33:23.607639+0000","last_became_peered":"2023-10-06T11:33:23.607639+0000","last_unstale":"2023-10-06T11:33:23.615110+0000","last_undegraded":"2023-10-06T11:33:23.615110+0000","last_fullsized":"2023-10-06T11:33:23.615110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:19.061615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"78'8","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.639768+0000","last_change":"2023-10-06T11:33:22.614520+0000","last_active":"2023-10-06T11:33:23.639768+0000","last_peered":"2023-10-06T11:33:23.639768+0000","last_clean":"2023-10-06T11:33:23.639768+0000","last_became_active":"2023-10-06T11:33:22.614243+0000","last_became_peered":"2023-10-06T11:33:22.614243+0000","last_unstale":"2023-10-06T11:33:23.639768+0000","last_undegraded":"2023-10-06T11:33:23.639768+0000","last_fullsized":"2023-10-06T11:33:23.639768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:43:51.298302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.612830+0000","last_change":"2023-10-06T11:33:23.606980+0000","last_active":"2023-10-06T11:33:23.612830+0000","last_peered":"2023-10-06T11:33:23.612830+0000","last_clean":"2023-10-06T11:33:23.612830+0000","last_became_active":"2023-10-06T11:33:23.606671+0000","last_became_peered":"2023-10-06T11:33:23.606671+0000","last_unstale":"2023-10-06T11:33:23.612830+0000","last_undegraded":"2023-10-06T11:33:23.612830+0000","last_fullsized":"2023-10-06T11:33:23.612830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:18.554241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637513+0000","last_change":"2023-10-06T11:33:22.615405+0000","last_active":"2023-10-06T11:33:23.637513+0000","last_peered":"2023-10-06T11:33:23.637513+0000","last_clean":"2023-10-06T11:33:23.637513+0000","last_became_active":"2023-10-06T11:33:22.614765+0000","last_became_peered":"2023-10-06T11:33:22.614765+0000","last_unstale":"2023-10-06T11:33:23.637513+0000","last_undegraded":"2023-10-06T11:33:23.637513+0000","last_fullsized":"2023-10-06T11:33:23.637513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607958+0000","last_change":"2023-10-06T11:33:23.602976+0000","last_active":"2023-10-06T11:33:23.607958+0000","last_peered":"2023-10-06T11:33:23.607958+0000","last_clean":"2023-10-06T11:33:23.607958+0000","last_became_active":"2023-10-06T11:33:23.602529+0000","last_became_peered":"2023-10-06T11:33:23.602529+0000","last_unstale":"2023-10-06T11:33:23.607958+0000","last_undegraded":"2023-10-06T11:33:23.607958+0000","last_fullsized":"2023-10-06T11:33:23.607958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:55:57.225708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.609578+0000","last_change":"2023-10-06T11:33:22.613321+0000","last_active":"2023-10-06T11:33:23.609578+0000","last_peered":"2023-10-06T11:33:23.609578+0000","last_clean":"2023-10-06T11:33:23.609578+0000","last_became_active":"2023-10-06T11:33:22.613013+0000","last_became_peered":"2023-10-06T11:33:22.613013+0000","last_unstale":"2023-10-06T11:33:23.609578+0000","last_undegraded":"2023-10-06T11:33:23.609578+0000","last_fullsized":"2023-10-06T11:33:23.609578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624618+0000","last_change":"2023-10-06T11:33:23.620035+0000","last_active":"2023-10-06T11:33:23.624618+0000","last_peered":"2023-10-06T11:33:23.624618+0000","last_clean":"2023-10-06T11:33:23.624618+0000","last_became_active":"2023-10-06T11:33:23.619662+0000","last_became_peered":"2023-10-06T11:33:23.619662+0000","last_unstale":"2023-10-06T11:33:23.624618+0000","last_undegraded":"2023-10-06T11:33:23.624618+0000","last_fullsized":"2023-10-06T11:33:23.624618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:56.614867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.592776+0000","last_change":"2023-10-06T11:33:22.606822+0000","last_active":"2023-10-06T11:33:23.592776+0000","last_peered":"2023-10-06T11:33:23.592776+0000","last_clean":"2023-10-06T11:33:23.592776+0000","last_became_active":"2023-10-06T11:33:22.606513+0000","last_became_peered":"2023-10-06T11:33:22.606513+0000","last_unstale":"2023-10-06T11:33:23.592776+0000","last_undegraded":"2023-10-06T11:33:23.592776+0000","last_fullsized":"2023-10-06T11:33:23.592776+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:29:58.046445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622939+0000","last_change":"2023-10-06T11:33:23.618918+0000","last_active":"2023-10-06T11:33:23.622939+0000","last_peered":"2023-10-06T11:33:23.622939+0000","last_clean":"2023-10-06T11:33:23.622939+0000","last_became_active":"2023-10-06T11:33:23.618587+0000","last_became_peered":"2023-10-06T11:33:23.618587+0000","last_unstale":"2023-10-06T11:33:23.622939+0000","last_undegraded":"2023-10-06T11:33:23.622939+0000","last_fullsized":"2023-10-06T11:33:23.622939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:21.997776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594647+0000","last_change":"2023-10-06T11:33:22.598153+0000","last_active":"2023-10-06T11:33:23.594647+0000","last_peered":"2023-10-06T11:33:23.594647+0000","last_clean":"2023-10-06T11:33:23.594647+0000","last_became_active":"2023-10-06T11:33:22.597683+0000","last_became_peered":"2023-10-06T11:33:22.597683+0000","last_unstale":"2023-10-06T11:33:23.594647+0000","last_undegraded":"2023-10-06T11:33:23.594647+0000","last_fullsized":"2023-10-06T11:33:23.594647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:43:57.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623109+0000","last_change":"2023-10-06T11:33:23.612023+0000","last_active":"2023-10-06T11:33:23.623109+0000","last_peered":"2023-10-06T11:33:23.623109+0000","last_clean":"2023-10-06T11:33:23.623109+0000","last_became_active":"2023-10-06T11:33:23.611687+0000","last_became_peered":"2023-10-06T11:33:23.611687+0000","last_unstale":"2023-10-06T11:33:23.623109+0000","last_undegraded":"2023-10-06T11:33:23.623109+0000","last_fullsized":"2023-10-06T11:33:23.623109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:35.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.601125+0000","last_change":"2023-10-06T11:33:22.616956+0000","last_active":"2023-10-06T11:33:23.601125+0000","last_peered":"2023-10-06T11:33:23.601125+0000","last_clean":"2023-10-06T11:33:23.601125+0000","last_became_active":"2023-10-06T11:33:22.616621+0000","last_became_peered":"2023-10-06T11:33:22.616621+0000","last_unstale":"2023-10-06T11:33:23.601125+0000","last_undegraded":"2023-10-06T11:33:23.601125+0000","last_fullsized":"2023-10-06T11:33:23.601125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:08.438550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622240+0000","last_change":"2023-10-06T11:33:23.612266+0000","last_active":"2023-10-06T11:33:23.622240+0000","last_peered":"2023-10-06T11:33:23.622240+0000","last_clean":"2023-10-06T11:33:23.622240+0000","last_became_active":"2023-10-06T11:33:23.611923+0000","last_became_peered":"2023-10-06T11:33:23.611923+0000","last_unstale":"2023-10-06T11:33:23.622240+0000","last_undegraded":"2023-10-06T11:33:23.622240+0000","last_fullsized":"2023-10-06T11:33:23.622240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:35:35.295378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598545+0000","last_change":"2023-10-06T11:33:22.598153+0000","last_active":"2023-10-06T11:33:23.598545+0000","last_peered":"2023-10-06T11:33:23.598545+0000","last_clean":"2023-10-06T11:33:23.598545+0000","last_became_active":"2023-10-06T11:33:22.596906+0000","last_became_peered":"2023-10-06T11:33:22.596906+0000","last_unstale":"2023-10-06T11:33:23.598545+0000","last_undegraded":"2023-10-06T11:33:23.598545+0000","last_fullsized":"2023-10-06T11:33:23.598545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:24:34.630065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.620187+0000","last_change":"2023-10-06T11:33:23.611164+0000","last_active":"2023-10-06T11:33:23.620187+0000","last_peered":"2023-10-06T11:33:23.620187+0000","last_clean":"2023-10-06T11:33:23.620187+0000","last_became_active":"2023-10-06T11:33:23.610350+0000","last_became_peered":"2023-10-06T11:33:23.610350+0000","last_unstale":"2023-10-06T11:33:23.620187+0000","last_undegraded":"2023-10-06T11:33:23.620187+0000","last_fullsized":"2023-10-06T11:33:23.620187+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:08.369054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598956+0000","last_change":"2023-10-06T11:33:22.600332+0000","last_active":"2023-10-06T11:33:23.598956+0000","last_peered":"2023-10-06T11:33:23.598956+0000","last_clean":"2023-10-06T11:33:23.598956+0000","last_became_active":"2023-10-06T11:33:22.599975+0000","last_became_peered":"2023-10-06T11:33:22.599975+0000","last_unstale":"2023-10-06T11:33:23.598956+0000","last_undegraded":"2023-10-06T11:33:23.598956+0000","last_fullsized":"2023-10-06T11:33:23.598956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:58.280365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.619814+0000","last_change":"2023-10-06T11:33:23.618378+0000","last_active":"2023-10-06T11:33:23.619814+0000","last_peered":"2023-10-06T11:33:23.619814+0000","last_clean":"2023-10-06T11:33:23.619814+0000","last_became_active":"2023-10-06T11:33:23.617976+0000","last_became_peered":"2023-10-06T11:33:23.617976+0000","last_unstale":"2023-10-06T11:33:23.619814+0000","last_undegraded":"2023-10-06T11:33:23.619814+0000","last_fullsized":"2023-10-06T11:33:23.619814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:40.419613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.593962+0000","last_change":"2023-10-06T11:33:22.602448+0000","last_active":"2023-10-06T11:33:23.593962+0000","last_peered":"2023-10-06T11:33:23.593962+0000","last_clean":"2023-10-06T11:33:23.593962+0000","last_became_active":"2023-10-06T11:33:22.602104+0000","last_became_peered":"2023-10-06T11:33:22.602104+0000","last_unstale":"2023-10-06T11:33:23.593962+0000","last_undegraded":"2023-10-06T11:33:23.593962+0000","last_fullsized":"2023-10-06T11:33:23.593962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:07.248883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.621471+0000","last_change":"2023-10-06T11:33:23.611359+0000","last_active":"2023-10-06T11:33:23.621471+0000","last_peered":"2023-10-06T11:33:23.621471+0000","last_clean":"2023-10-06T11:33:23.621471+0000","last_became_active":"2023-10-06T11:33:23.611011+0000","last_became_peered":"2023-10-06T11:33:23.611011+0000","last_unstale":"2023-10-06T11:33:23.621471+0000","last_undegraded":"2023-10-06T11:33:23.621471+0000","last_fullsized":"2023-10-06T11:33:23.621471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:09.451685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598437+0000","last_change":"2023-10-06T11:33:22.597320+0000","last_active":"2023-10-06T11:33:23.598437+0000","last_peered":"2023-10-06T11:33:23.598437+0000","last_clean":"2023-10-06T11:33:23.598437+0000","last_became_active":"2023-10-06T11:33:22.596977+0000","last_became_peered":"2023-10-06T11:33:22.596977+0000","last_unstale":"2023-10-06T11:33:23.598437+0000","last_undegraded":"2023-10-06T11:33:23.598437+0000","last_fullsized":"2023-10-06T11:33:23.598437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:12.581156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":370,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600714+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:33:23.600714+0000","last_peered":"2023-10-06T11:33:23.600714+0000","last_clean":"2023-10-06T11:33:23.600714+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:33:23.600714+0000","last_undegraded":"2023-10-06T11:33:23.600714+0000","last_fullsized":"2023-10-06T11:33:23.600714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.608191+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:33:23.608191+0000","last_peered":"2023-10-06T11:33:23.608191+0000","last_clean":"2023-10-06T11:33:23.608191+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:33:23.608191+0000","last_undegraded":"2023-10-06T11:33:23.608191+0000","last_fullsized":"2023-10-06T11:33:23.608191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.612795+0000","last_change":"2023-10-06T11:33:23.610818+0000","last_active":"2023-10-06T11:33:23.612795+0000","last_peered":"2023-10-06T11:33:23.612795+0000","last_clean":"2023-10-06T11:33:23.612795+0000","last_became_active":"2023-10-06T11:33:23.610397+0000","last_became_peered":"2023-10-06T11:33:23.610397+0000","last_unstale":"2023-10-06T11:33:23.612795+0000","last_undegraded":"2023-10-06T11:33:23.612795+0000","last_fullsized":"2023-10-06T11:33:23.612795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:27.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.625468+0000","last_change":"2023-10-06T11:33:22.603754+0000","last_active":"2023-10-06T11:33:23.625468+0000","last_peered":"2023-10-06T11:33:23.625468+0000","last_clean":"2023-10-06T11:33:23.625468+0000","last_became_active":"2023-10-06T11:33:22.603428+0000","last_became_peered":"2023-10-06T11:33:22.603428+0000","last_unstale":"2023-10-06T11:33:23.625468+0000","last_undegraded":"2023-10-06T11:33:23.625468+0000","last_fullsized":"2023-10-06T11:33:23.625468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:20.667236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.600797+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:33:23.600797+0000","last_peered":"2023-10-06T11:33:23.600797+0000","last_clean":"2023-10-06T11:33:23.600797+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:33:23.600797+0000","last_undegraded":"2023-10-06T11:33:23.600797+0000","last_fullsized":"2023-10-06T11:33:23.600797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.624584+0000","last_change":"2023-10-06T11:33:23.615471+0000","last_active":"2023-10-06T11:33:23.624584+0000","last_peered":"2023-10-06T11:33:23.624584+0000","last_clean":"2023-10-06T11:33:23.624584+0000","last_became_active":"2023-10-06T11:33:23.615165+0000","last_became_peered":"2023-10-06T11:33:23.615165+0000","last_unstale":"2023-10-06T11:33:23.624584+0000","last_undegraded":"2023-10-06T11:33:23.624584+0000","last_fullsized":"2023-10-06T11:33:23.624584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:13.253316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607073+0000","last_change":"2023-10-06T11:33:22.609602+0000","last_active":"2023-10-06T11:33:23.607073+0000","last_peered":"2023-10-06T11:33:23.607073+0000","last_clean":"2023-10-06T11:33:23.607073+0000","last_became_active":"2023-10-06T11:33:22.609235+0000","last_became_peered":"2023-10-06T11:33:22.609235+0000","last_unstale":"2023-10-06T11:33:23.607073+0000","last_undegraded":"2023-10-06T11:33:23.607073+0000","last_fullsized":"2023-10-06T11:33:23.607073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:59.117949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.586319+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:33:23.586319+0000","last_peered":"2023-10-06T11:33:23.586319+0000","last_clean":"2023-10-06T11:33:23.586319+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:33:23.586319+0000","last_undegraded":"2023-10-06T11:33:23.586319+0000","last_fullsized":"2023-10-06T11:33:23.586319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.611813+0000","last_change":"2023-10-06T11:33:23.607398+0000","last_active":"2023-10-06T11:33:23.611813+0000","last_peered":"2023-10-06T11:33:23.611813+0000","last_clean":"2023-10-06T11:33:23.611813+0000","last_became_active":"2023-10-06T11:33:23.607003+0000","last_became_peered":"2023-10-06T11:33:23.607003+0000","last_unstale":"2023-10-06T11:33:23.611813+0000","last_undegraded":"2023-10-06T11:33:23.611813+0000","last_fullsized":"2023-10-06T11:33:23.611813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:17.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.591365+0000","last_change":"2023-10-06T11:33:22.607981+0000","last_active":"2023-10-06T11:33:23.591365+0000","last_peered":"2023-10-06T11:33:23.591365+0000","last_clean":"2023-10-06T11:33:23.591365+0000","last_became_active":"2023-10-06T11:33:22.607679+0000","last_became_peered":"2023-10-06T11:33:22.607679+0000","last_unstale":"2023-10-06T11:33:23.591365+0000","last_undegraded":"2023-10-06T11:33:23.591365+0000","last_fullsized":"2023-10-06T11:33:23.591365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:43.287144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.607176+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:33:23.607176+0000","last_peered":"2023-10-06T11:33:23.607176+0000","last_clean":"2023-10-06T11:33:23.607176+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:33:23.607176+0000","last_undegraded":"2023-10-06T11:33:23.607176+0000","last_fullsized":"2023-10-06T11:33:23.607176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627322+0000","last_change":"2023-10-06T11:33:23.621327+0000","last_active":"2023-10-06T11:33:23.627322+0000","last_peered":"2023-10-06T11:33:23.627322+0000","last_clean":"2023-10-06T11:33:23.627322+0000","last_became_active":"2023-10-06T11:33:23.621001+0000","last_became_peered":"2023-10-06T11:33:23.621001+0000","last_unstale":"2023-10-06T11:33:23.627322+0000","last_undegraded":"2023-10-06T11:33:23.627322+0000","last_fullsized":"2023-10-06T11:33:23.627322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:58:21.680372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.635729+0000","last_change":"2023-10-06T11:33:22.610379+0000","last_active":"2023-10-06T11:33:23.635729+0000","last_peered":"2023-10-06T11:33:23.635729+0000","last_clean":"2023-10-06T11:33:23.635729+0000","last_became_active":"2023-10-06T11:33:22.610075+0000","last_became_peered":"2023-10-06T11:33:22.610075+0000","last_unstale":"2023-10-06T11:33:23.635729+0000","last_undegraded":"2023-10-06T11:33:23.635729+0000","last_fullsized":"2023-10-06T11:33:23.635729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:16.825541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594729+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:33:23.594729+0000","last_peered":"2023-10-06T11:33:23.594729+0000","last_clean":"2023-10-06T11:33:23.594729+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:33:23.594729+0000","last_undegraded":"2023-10-06T11:33:23.594729+0000","last_fullsized":"2023-10-06T11:33:23.594729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.623321+0000","last_change":"2023-10-06T11:33:23.620650+0000","last_active":"2023-10-06T11:33:23.623321+0000","last_peered":"2023-10-06T11:33:23.623321+0000","last_clean":"2023-10-06T11:33:23.623321+0000","last_became_active":"2023-10-06T11:33:23.620346+0000","last_became_peered":"2023-10-06T11:33:23.620346+0000","last_unstale":"2023-10-06T11:33:23.623321+0000","last_undegraded":"2023-10-06T11:33:23.623321+0000","last_fullsized":"2023-10-06T11:33:23.623321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:03.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.636854+0000","last_change":"2023-10-06T11:33:22.619272+0000","last_active":"2023-10-06T11:33:23.636854+0000","last_peered":"2023-10-06T11:33:23.636854+0000","last_clean":"2023-10-06T11:33:23.636854+0000","last_became_active":"2023-10-06T11:33:22.618615+0000","last_became_peered":"2023-10-06T11:33:22.618615+0000","last_unstale":"2023-10-06T11:33:23.636854+0000","last_undegraded":"2023-10-06T11:33:23.636854+0000","last_fullsized":"2023-10-06T11:33:23.636854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:56.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.601968+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:33:23.601968+0000","last_peered":"2023-10-06T11:33:23.601968+0000","last_clean":"2023-10-06T11:33:23.601968+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:33:23.601968+0000","last_undegraded":"2023-10-06T11:33:23.601968+0000","last_fullsized":"2023-10-06T11:33:23.601968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.627785+0000","last_change":"2023-10-06T11:33:23.624440+0000","last_active":"2023-10-06T11:33:23.627785+0000","last_peered":"2023-10-06T11:33:23.627785+0000","last_clean":"2023-10-06T11:33:23.627785+0000","last_became_active":"2023-10-06T11:33:23.623829+0000","last_became_peered":"2023-10-06T11:33:23.623829+0000","last_unstale":"2023-10-06T11:33:23.627785+0000","last_undegraded":"2023-10-06T11:33:23.627785+0000","last_fullsized":"2023-10-06T11:33:23.627785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:45:41.298737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.637151+0000","last_change":"2023-10-06T11:33:22.619236+0000","last_active":"2023-10-06T11:33:23.637151+0000","last_peered":"2023-10-06T11:33:23.637151+0000","last_clean":"2023-10-06T11:33:23.637151+0000","last_became_active":"2023-10-06T11:33:22.618945+0000","last_became_peered":"2023-10-06T11:33:22.618945+0000","last_unstale":"2023-10-06T11:33:23.637151+0000","last_undegraded":"2023-10-06T11:33:23.637151+0000","last_fullsized":"2023-10-06T11:33:23.637151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:59.117949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.594175+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:33:23.594175+0000","last_peered":"2023-10-06T11:33:23.594175+0000","last_clean":"2023-10-06T11:33:23.594175+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:33:23.594175+0000","last_undegraded":"2023-10-06T11:33:23.594175+0000","last_fullsized":"2023-10-06T11:33:23.594175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.622956+0000","last_change":"2023-10-06T11:33:23.618963+0000","last_active":"2023-10-06T11:33:23.622956+0000","last_peered":"2023-10-06T11:33:23.622956+0000","last_clean":"2023-10-06T11:33:23.622956+0000","last_became_active":"2023-10-06T11:33:23.618643+0000","last_became_peered":"2023-10-06T11:33:23.618643+0000","last_unstale":"2023-10-06T11:33:23.622956+0000","last_undegraded":"2023-10-06T11:33:23.622956+0000","last_fullsized":"2023-10-06T11:33:23.622956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:22.562790+0000","last_clean_scrub_stamp":"2023-10-06T11:33:22.562790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:19.363557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598712+0000","last_change":"2023-10-06T11:33:22.601550+0000","last_active":"2023-10-06T11:33:23.598712+0000","last_peered":"2023-10-06T11:33:23.598712+0000","last_clean":"2023-10-06T11:33:23.598712+0000","last_became_active":"2023-10-06T11:33:22.600465+0000","last_became_peered":"2023-10-06T11:33:22.600465+0000","last_unstale":"2023-10-06T11:33:23.598712+0000","last_undegraded":"2023-10-06T11:33:23.598712+0000","last_fullsized":"2023-10-06T11:33:23.598712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:21.549654+0000","last_clean_scrub_stamp":"2023-10-06T11:33:21.549654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:19:13.681370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-06T11:33:23.598251+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:33:23.598251+0000","last_peered":"2023-10-06T11:33:23.598251+0000","last_clean":"2023-10-06T11:33:23.598251+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:33:23.598251+0000","last_undegraded":"2023-10-06T11:33:23.598251+0000","last_fullsized":"2023-10-06T11:33:23.598251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311092,"kb_used_data":1516,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060748,"statfs":{"total":96636764160,"available":96318205952,"internally_reserved":0,"allocated":1552384,"data_stored":1063542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1359999999999999,"15min":1.135},"min":{"1min":0.79100000000000004,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.423,"5min":1.571,"15min":1.827},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.115,"15min":1.1399999999999999},"min":{"1min":0.84599999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.649,"15min":2.0030000000000001},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3180000000000001,"15min":1.331},"min":{"1min":1.103,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.851,"5min":1.871,"15min":2.6059999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2350000000000001,"15min":1.25},"min":{"1min":1.117,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.829,"5min":1.829,"15min":1.931},"last":1.3979999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.234,"15min":1.2130000000000001},"min":{"1min":0.79400000000000004,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.554,"5min":1.827,"15min":2.3799999999999999},"last":1.335},{"interface":"front","average":{"1min":1.2,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.90300000000000002,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.5369999999999999,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.284,"15min":1.2849999999999999},"min":{"1min":1.042,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6970000000000001,"5min":1.9450000000000001,"15min":2.665},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3009999999999999,"15min":1.3340000000000001},"min":{"1min":0.94899999999999995,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.754,"5min":1.796,"15min":2.8570000000000002},"last":1.1160000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.71399999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5329999999999999,"5min":2.149,"15min":2.7970000000000002},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.268,"15min":1.278},"min":{"1min":0.87,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5760000000000001,"5min":2.0499999999999998,"15min":2.5339999999999998},"last":1.046}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3240000000000001,"15min":1.3320000000000001},"min":{"1min":0.98399999999999999,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.29},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3149999999999999,"15min":1.2909999999999999},"min":{"1min":1.081,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.6259999999999999,"5min":2.2480000000000002,"15min":3.105},"last":1.304}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3700000000000001},"min":{"1min":1.01,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.742,"5min":2.4289999999999998,"15min":3.004},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.4350000000000001,"15min":1.4119999999999999},"min":{"1min":0.76000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.913,"5min":2.121,"15min":2.9119999999999999},"last":1.133}]}]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":479545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.28,"15min":1.2889999999999999},"min":{"1min":1.089,"5min":0.621,"15min":0.621},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.385},{"interface":"front","average":{"1min":1.454,"5min":1.3149999999999999,"15min":1.3220000000000001},"min":{"1min":1.0289999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.659,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.323}]},{"osd":1,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1850000000000001,"15min":1.163},"min":{"1min":0.91300000000000003,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.5980000000000001,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.226,"15min":1.212},"min":{"1min":0.88400000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.569,"5min":1.669,"15min":1.8480000000000001},"last":1.605}]},{"osd":2,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.1879999999999999,"15min":1.196},"min":{"1min":0.91000000000000003,"5min":0.78300000000000003,"15min":0.74299999999999999},"max":{"1min":1.635,"5min":1.635,"15min":1.6919999999999999},"last":1.083},{"interface":"front","average":{"1min":1.24,"5min":1.2170000000000001,"15min":1.2330000000000001},"min":{"1min":0.91300000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4930000000000001,"5min":1.615,"15min":2.7789999999999999},"last":1.026}]},{"osd":3,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2929999999999999,"15min":1.3140000000000001},"min":{"1min":1,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6990000000000001,"5min":1.764,"15min":1.9850000000000001},"last":1.53},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3029999999999999,"15min":1.3080000000000001},"min":{"1min":1.1459999999999999,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.7130000000000001,"5min":1.724,"15min":2.109},"last":1.194}]},{"osd":4,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2430000000000001,"15min":1.228},"min":{"1min":0.85899999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.728,"5min":1.8,"15min":1.956},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.2929999999999999,"15min":1.2549999999999999},"min":{"1min":0.98099999999999998,"5min":0.76500000000000001,"15min":0.73899999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.8879999999999999},"last":1.274}]},{"osd":5,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.224,"15min":1.2430000000000001},"min":{"1min":0.79800000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.552,"5min":1.72,"15min":2.198},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3939999999999999,"15min":1.391},"min":{"1min":1.0600000000000001,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.7969999999999999,"5min":1.879,"15min":2.1509999999999998},"last":1.718}]},{"osd":7,"last update":"Fri Oct 6 11:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.4399999999999999,"15min":1.4570000000000001},"min":{"1min":1.1479999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.772,"5min":1.927,"15min":2.2919999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.444},"min":{"1min":0.95799999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6419999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.4410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":472448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.409,"15min":1.2669999999999999},"min":{"1min":0.876,"5min":0.5,"15min":0.5},"max":{"1min":1.573,"5min":17.626000000000001,"15min":17.626000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3069999999999999,"15min":1.3100000000000001},"min":{"1min":1.097,"5min":0.77600000000000002,"15min":0.64900000000000002},"max":{"1min":1.508,"5min":1.972,"15min":2.544},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1910000000000001,"15min":1.1830000000000001},"min":{"1min":0.81100000000000005,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.161},{"interface":"front","average":{"1min":1.119,"5min":1.278,"15min":1.2290000000000001},"min":{"1min":0.91800000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.359,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.3500000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.171,"15min":1.218},"min":{"1min":0.92800000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5960000000000001,"5min":1.6879999999999999,"15min":3.609},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2390000000000001,"15min":1.244},"min":{"1min":0.878,"5min":0.878,"15min":0.73299999999999998},"max":{"1min":1.6679999999999999,"5min":1.738,"15min":2.6949999999999998},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3899999999999999,"15min":1.3149999999999999},"min":{"1min":0.94299999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.643,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3089999999999999,"15min":1.359},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.6899999999999999,"5min":1.734,"15min":3.7770000000000001},"last":1.21}]},{"osd":4,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3200000000000001,"15min":1.274},"min":{"1min":0.80400000000000005,"5min":0.68100000000000005,"15min":0.40699999999999997},"max":{"1min":1.5569999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4850000000000001,"15min":1.4219999999999999},"min":{"1min":0.90800000000000003,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.7110000000000001,"5min":18.48,"15min":18.48},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.319,"15min":1.3340000000000001},"min":{"1min":0.85299999999999998,"5min":0.622,"15min":0.622},"max":{"1min":1.6020000000000001,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.387,"15min":1.3680000000000001},"min":{"1min":0.92600000000000005,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.885,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.385}]},{"osd":7,"last update":"Fri Oct 6 11:32:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.5129999999999999,"15min":1.4319999999999999},"min":{"1min":1.0600000000000001,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7769999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.415},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.4710000000000001},"min":{"1min":0.97099999999999997,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7829999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.1200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302328,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069512,"statfs":{"total":96636764160,"available":96327180288,"internally_reserved":0,"allocated":966656,"data_stored":481047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.137,"15min":1.145},"min":{"1min":0.97699999999999998,"5min":0.61699999999999999,"15min":0.61099999999999999},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.119,"15min":1.1539999999999999},"min":{"1min":0.74299999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4530000000000001,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3180000000000001,"15min":1.339},"min":{"1min":0.85499999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5609999999999999,"5min":1.6830000000000001,"15min":2.1989999999999998},"last":1.135},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":1.1060000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5089999999999999,"5min":1.7450000000000001,"15min":1.851},"last":1.085}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1859999999999999,"15min":1.214},"min":{"1min":0.77600000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.4770000000000001,"5min":1.5449999999999999,"15min":2.2269999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.266,"15min":1.278},"min":{"1min":0.95399999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.617,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.0109999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.214,"15min":1.218},"min":{"1min":0.999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.4930000000000001,"5min":1.671,"15min":1.6839999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2929999999999999,"15min":1.3129999999999999},"min":{"1min":1.085,"5min":0.82999999999999996,"15min":0.77400000000000002},"max":{"1min":1.663,"5min":1.6639999999999999,"15min":2.258},"last":1.575}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.181,"15min":1.2250000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5549999999999999,"5min":1.7390000000000001,"15min":2.8250000000000002},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2150000000000001,"15min":1.204},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.40999999999999998},"max":{"1min":1.613,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.2210000000000001,"15min":1.2589999999999999},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.54900000000000004},"max":{"1min":1.518,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.421},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3560000000000001,"15min":1.377},"min":{"1min":0.93400000000000005,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.5609999999999999,"5min":1.8129999999999999,"15min":2.4329999999999998},"last":1.2350000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3620000000000001,"15min":1.361},"min":{"1min":1.054,"5min":0.95799999999999996,"15min":0.69399999999999995},"max":{"1min":1.579,"5min":1.841,"15min":2.653},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.401,"15min":1.4079999999999999},"min":{"1min":1.028,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":1.657,"5min":1.8480000000000001,"15min":2.464},"last":1.6739999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306412,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065428,"statfs":{"total":96636764160,"available":96322998272,"internally_reserved":0,"allocated":954368,"data_stored":479511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.65600000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97299999999999998,"15min":0.97899999999999998},"min":{"1min":0.68600000000000005,"5min":0.54800000000000004,"15min":0.437},"max":{"1min":1.504,"5min":1.5129999999999999,"15min":1.627},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.91700000000000004,"15min":0.95699999999999996},"min":{"1min":0.68400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4079999999999999,"5min":1.448,"15min":1.516},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.075,"15min":1.075},"min":{"1min":0.77000000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.45,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.091,"15min":1.123},"min":{"1min":0.879,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4019999999999999,"5min":1.6830000000000001,"15min":1.887},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.147,"15min":1.202},"min":{"1min":0.86899999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.556,"5min":1.5629999999999999,"15min":1.7030000000000001},"last":1.2270000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1859999999999999,"15min":1.2430000000000001},"min":{"1min":0.86599999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7110000000000001,"5min":1.768,"15min":2.6640000000000001},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3160000000000001,"15min":1.363},"min":{"1min":1.083,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.617,"5min":1.718,"15min":2.0840000000000001},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.3660000000000001,"15min":1.415},"min":{"1min":1.0620000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7470000000000001,"5min":1.883,"15min":2.1699999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.365,"15min":1.401},"min":{"1min":1.036,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.857,"5min":1.857,"15min":2.1179999999999999},"last":1.496}]},{"osd":6,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.415,"15min":1.45},"min":{"1min":0.94999999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.3440000000000001,"15min":1.3879999999999999},"min":{"1min":1.1020000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.7649999999999999,"5min":1.823,"15min":2.137},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:32:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4039999999999999,"15min":1.4490000000000001},"min":{"1min":1.0589999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6990000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.296},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4490000000000001,"15min":1.488},"min":{"1min":1.089,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.8080000000000001,"5min":2.266,"15min":2.266},"last":1.46}]}]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310508,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061332,"statfs":{"total":96636764160,"available":96318803968,"internally_reserved":0,"allocated":954368,"data_stored":479511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.95799999999999996,"15min":0.96499999999999997},"min":{"1min":0.69999999999999996,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4179999999999999,"5min":1.4930000000000001,"15min":1.8380000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0609999999999999,"15min":1.069},"min":{"1min":0.80600000000000005,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.415,"5min":1.5309999999999999,"15min":1.9650000000000001},"last":1.103}]},{"osd":1,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92800000000000005,"15min":0.92200000000000004},"min":{"1min":0.70399999999999996,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.2769999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0389999999999999,"15min":1.0249999999999999},"min":{"1min":0.78900000000000003,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.4339999999999999,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.181}]},{"osd":3,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1499999999999999,"15min":1.1830000000000001},"min":{"1min":0.81100000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.405,"5min":1.8240000000000001,"15min":1.917},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2869999999999999,"15min":1.302},"min":{"1min":0.90900000000000003,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.593,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.96499999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.222},"min":{"1min":0.88200000000000001,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.575,"5min":1.754,"15min":2.1899999999999999},"last":1.165},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.268,"15min":1.3049999999999999},"min":{"1min":0.93799999999999994,"5min":0.86199999999999999,"15min":0.84399999999999997},"max":{"1min":1.5800000000000001,"5min":2.024,"15min":2.6920000000000002},"last":1.149}]},{"osd":5,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.367,"15min":1.369},"min":{"1min":1.0720000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.742,"5min":2.0680000000000001,"15min":2.601},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.329,"15min":1.3400000000000001},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5489999999999999,"5min":1.778,"15min":2.6400000000000001},"last":1.228}]},{"osd":6,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4019999999999999,"15min":1.421},"min":{"1min":1.0489999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.6599999999999999,"5min":1.8779999999999999,"15min":2.8340000000000001},"last":1.208},{"interface":"front","average":{"1min":1.431,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":1.0229999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8160000000000001,"5min":2.617,"15min":2.617},"last":1.2450000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3660000000000001,"15min":1.3919999999999999},"min":{"1min":0.97199999999999998,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.667,"5min":1.863,"15min":2.9020000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4370000000000001,"15min":1.47},"min":{"1min":1.0740000000000001,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.73,"5min":2.2829999999999999,"15min":3.2240000000000002},"last":1.2609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306440,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065400,"statfs":{"total":96636764160,"available":96322969600,"internally_reserved":0,"allocated":983040,"data_stored":481783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.93999999999999995,"15min":0.93700000000000006},"min":{"1min":0.70499999999999996,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":1.3340000000000001,"5min":1.724,"15min":1.724},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.52100000000000002,"15min":0.51700000000000002},"max":{"1min":1.3100000000000001,"5min":1.4259999999999999,"15min":1.855},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.72699999999999998,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.2729999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":1.101},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.1339999999999999,"15min":1.1100000000000001},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5269999999999999,"5min":2.137,"15min":2.137},"last":1.0289999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.266,"15min":1.2370000000000001},"min":{"1min":1.0069999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6339999999999999,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.111},"min":{"1min":0.79100000000000004,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.3959999999999999,"5min":1.7210000000000001,"15min":4.0830000000000002},"last":1.2929999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.216,"15min":1.274},"min":{"1min":1.022,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.4330000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":1.238,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.736,"5min":1.7589999999999999,"15min":3.5609999999999999},"last":1.337}]},{"osd":5,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.325,"15min":1.345},"min":{"1min":1.119,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.361,"15min":1.3640000000000001},"min":{"1min":1.121,"5min":0.94699999999999995,"15min":0.85999999999999999},"max":{"1min":1.5489999999999999,"5min":2.1240000000000001,"15min":2.766},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4890000000000001,"15min":1.492},"min":{"1min":1.246,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8169999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.462},{"interface":"front","average":{"1min":1.353,"5min":1.3999999999999999,"15min":1.419},"min":{"1min":1.0589999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.0600000000000001,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.671}]},{"osd":7,"last update":"Fri Oct 6 11:32:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4950000000000001,"15min":1.5269999999999999},"min":{"1min":1.1519999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.9390000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4419999999999999,"15min":1.474},"min":{"1min":1.3080000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6799999999999999,"5min":1.9099999999999999,"15min":2.8740000000000001},"last":1.577}]}]},{"osd":0,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306980,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064860,"statfs":{"total":96636764160,"available":96322416640,"internally_reserved":0,"allocated":1536000,"data_stored":1062794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.873,"15min":0.88800000000000001},"min":{"1min":0.66700000000000004,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.286,"5min":1.3560000000000001,"15min":1.571},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.94599999999999995,"15min":0.96499999999999997},"min":{"1min":0.63700000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.49,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0409999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0129999999999999,"15min":1.0329999999999999},"min":{"1min":0.83499999999999996,"5min":0.58899999999999997,"15min":0.56699999999999995},"max":{"1min":1.379,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.054,"15min":1.085},"min":{"1min":0.58399999999999996,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.4239999999999999,"5min":1.6160000000000001,"15min":2.7719999999999998},"last":0.91500000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.0800000000000001,"15min":1.083},"min":{"1min":0.86499999999999999,"5min":0.70999999999999996,"15min":0.64000000000000001},"max":{"1min":1.3999999999999999,"5min":1.609,"15min":1.919},"last":1.079},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.82999999999999996,"5min":0.70399999999999996,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":1.121}]},{"osd":4,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3520000000000001,"15min":1.3500000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.68999999999999995},"max":{"1min":1.8440000000000001,"5min":2.294,"15min":2.294},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3149999999999999,"15min":1.2989999999999999},"min":{"1min":1.101,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.641,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.5469999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.3859999999999999,"15min":1.385},"min":{"1min":1.2370000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.819,"5min":2.0859999999999999,"15min":2.573},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3859999999999999,"15min":1.4179999999999999},"min":{"1min":1.115,"5min":1.0860000000000001,"15min":0.84599999999999997},"max":{"1min":1.6799999999999999,"5min":1.982,"15min":2.8650000000000002},"last":1.4359999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.341,"15min":1.3440000000000001},"min":{"1min":1.0469999999999999,"5min":0.89300000000000002,"15min":0.88900000000000001},"max":{"1min":1.7949999999999999,"5min":2.2240000000000002,"15min":2.52},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4450000000000001,"15min":1.466},"min":{"1min":1.0660000000000001,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":2.0030000000000001,"5min":2.254,"15min":2.742},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:32:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.458,"15min":1.504},"min":{"1min":1.28,"5min":1.012,"15min":0.98199999999999998},"max":{"1min":1.7070000000000001,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4079999999999999,"15min":1.4339999999999999},"min":{"1min":1.181,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.847,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.6839999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:33:28.336 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:33:28.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:33:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.540+0000 7f65bbfff700 1 Processor -- start 2023-10-06T11:33:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.540+0000 7f65bbfff700 1 -- start start 2023-10-06T11:33:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bbfff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 0x7f65bc0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bbfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65bc0cf4e0 0x7f65bc158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bbfff700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc0cf050 con 0x7f65bc0cb020 2023-10-06T11:33:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bbfff700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0cf8f0 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bbfff700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0ce440 con 0x7f65bc0d0750 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65bc0cf4e0 0x7f65bc158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65ba7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 0x7f65bc0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65bc0cf4e0 0x7f65bc158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40410/0 (socket says 172.21.15.46:40410) 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65ba7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 0x7f65bc0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44244/0 (socket says 172.21.15.46:44244) 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 -- 172.21.15.46:0/1983582174 learned_addr learned my addr 172.21.15.46:0/1983582174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65baffd700 1 --1- 172.21.15.46:0/1983582174 >> v1:172.21.15.132:6789/0 conn(0x7f65bc0cb020 0x7f65bc0cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42346/0 (socket says 172.21.15.46:42346) 2023-10-06T11:33:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 -- 172.21.15.46:0/1983582174 >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 msgr2=0x7f65bc0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 --2- 172.21.15.46:0/1983582174 >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 0x7f65bc0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 -- 172.21.15.46:0/1983582174 >> v1:172.21.15.132:6789/0 conn(0x7f65bc0cb020 legacy=0x7f65bc0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.544+0000 7f65bb7fe700 1 -- 172.21.15.46:0/1983582174 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0ce720 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bb7fe700 1 --2- 172.21.15.46:0/1983582174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cf4e0 0x7f65bc158bd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f65bc088f20 tx=0x7f65b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a4796c59aea61f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65b9ffb700 1 -- 172.21.15.46:0/1983582174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b400e070 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65b9ffb700 1 -- 172.21.15.46:0/1983582174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b400ba20 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65b9ffb700 1 -- 172.21.15.46:0/1983582174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40092e0 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 -- 172.21.15.46:0/1983582174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cf4e0 msgr2=0x7f65bc158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 --2- 172.21.15.46:0/1983582174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cf4e0 0x7f65bc158bd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f65bc088f20 tx=0x7f65b4009840 comp rx=0 tx=0).stop 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 -- 172.21.15.46:0/1983582174 shutdown_connections 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 --2- 172.21.15.46:0/1983582174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cf4e0 0x7f65bc158bd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 --2- 172.21.15.46:0/1983582174 >> v2:172.21.15.132:3300/0 conn(0x7f65bc0d0750 0x7f65bc0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 -- 172.21.15.46:0/1983582174 >> 172.21.15.46:0/1983582174 conn(0x7f65bc0683d0 msgr2=0x7f65bc0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:28.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 -- 172.21.15.46:0/1983582174 shutdown_connections 2023-10-06T11:33:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.548+0000 7f65bbfff700 1 -- 172.21.15.46:0/1983582174 wait complete. 2023-10-06T11:33:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 Processor -- start 2023-10-06T11:33:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 -- start start 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65bc0cf4e0 0x7f65bc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65bc0d0750 0x7f65bc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15de70 con 0x7f65bc0cb020 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15e990 con 0x7f65bc0d0750 2023-10-06T11:33:28.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bbfff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0c8270 con 0x7f65bc0cf4e0 2023-10-06T11:33:28.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40416/0 (socket says 172.21.15.46:40416) 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 -- 172.21.15.46:0/3357082668 learned_addr learned my addr 172.21.15.46:0/3357082668 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65ba7fc700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65bc0cf4e0 0x7f65bc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65bb7fe700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65bc0d0750 0x7f65bc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 -- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65bc0cf4e0 msgr2=0x7f65bc0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65bc0cf4e0 0x7f65bc0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 -- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65bc0d0750 msgr2=0x7f65bc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65bc0d0750 0x7f65bc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.552+0000 7f65baffd700 1 -- 172.21.15.46:0/3357082668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4009ae0 con 0x7f65bc0cb020 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65baffd700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7c0 tx=0x7f65ac007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac005a70 con 0x7f65bc0cb020 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac005bd0 con 0x7f65bc0cb020 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc15e060 con 0x7f65bc0cb020 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0070d0 con 0x7f65bc0cb020 2023-10-06T11:33:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc15e6a0 con 0x7f65bc0cb020 2023-10-06T11:33:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc156b30 con 0x7f65bc0cb020 2023-10-06T11:33:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.556+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f65ac007480 con 0x7f65bc0cb020 2023-10-06T11:33:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.560+0000 7f65a2ffd700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 0x7f65a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:28.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.560+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f65ac07c050 con 0x7f65bc0cb020 2023-10-06T11:33:28.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.560+0000 7f65ba7fc700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 0x7f65a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.560+0000 7f65ba7fc700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 0x7f65a8062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65bc15b8d0 tx=0x7f65b00091a0 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.564+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f65ac04c210 con 0x7f65bc0cb020 2023-10-06T11:33:28.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.792+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f65bc064590 con 0x7f65bc0cb020 2023-10-06T11:33:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.804+0000 7f65a2ffd700 1 -- 172.21.15.46:0/3357082668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f65ac04f830 con 0x7f65bc0cb020 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 msgr2=0x7f65a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 0x7f65a8062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65bc15b8d0 tx=0x7f65b00091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 msgr2=0x7f65bc15cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c7c0 tx=0x7f65ac007f70 comp rx=0 tx=0).stop 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 shutdown_connections 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f65a8060670 0x7f65a8062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65bc0d0750 0x7f65bc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65bc0cf4e0 0x7f65bc0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 --2- 172.21.15.46:0/3357082668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65bc0cb020 0x7f65bc15cf10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 >> 172.21.15.46:0/3357082668 conn(0x7f65bc0683d0 msgr2=0x7f65bc157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 shutdown_connections 2023-10-06T11:33:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.808+0000 7f65bbfff700 1 -- 172.21.15.46:0/3357082668 wait complete. 2023-10-06T11:33:28.822 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:33:28.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.992+0000 7f47865c5700 1 Processor -- start 2023-10-06T11:33:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.992+0000 7f47865c5700 1 -- start start 2023-10-06T11:33:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f47865c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f478015ab60 0x7f478006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f47865c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478006d460 con 0x7f478015af30 2023-10-06T11:33:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f47865c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47800c7000 con 0x7f47800ad6c0 2023-10-06T11:33:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f47865c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800c72c0 con 0x7f478015ab60 2023-10-06T11:33:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f478015af30 0x7f478006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33304/0 (socket says 172.21.15.46:33304) 2023-10-06T11:33:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477f7fe700 1 -- 172.21.15.46:0/3314168024 learned_addr learned my addr 172.21.15.46:0/3314168024 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477ffff700 1 --2- 172.21.15.46:0/3314168024 >> v2:172.21.15.46:3300/0 conn(0x7f478015ab60 0x7f478006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477effd700 1 -- 172.21.15.46:0/3314168024 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704073607 0 0) 0x7f478006d460 con 0x7f478015af30 2023-10-06T11:33:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477ffff700 1 -- 172.21.15.46:0/3314168024 >> v1:172.21.15.132:6789/0 conn(0x7f47800ad6c0 legacy=0x7f47800adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477ffff700 1 -- 172.21.15.46:0/3314168024 >> v1:172.21.15.46:6789/0 conn(0x7f478015af30 legacy=0x7f478006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477ffff700 1 -- 172.21.15.46:0/3314168024 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800c75a0 con 0x7f478015ab60 2023-10-06T11:33:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:28.996+0000 7f477ffff700 1 --2- 172.21.15.46:0/3314168024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015ab60 0x7f478006e8a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f47800c71f0 tx=0x7f4778009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f662a8e2b4f4c97c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f477effd700 1 -- 172.21.15.46:0/3314168024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778016070 con 0x7f478015ab60 2023-10-06T11:33:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f477effd700 1 -- 172.21.15.46:0/3314168024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477800bc30 con 0x7f478015ab60 2023-10-06T11:33:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f477effd700 1 -- 172.21.15.46:0/3314168024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800e070 con 0x7f478015ab60 2023-10-06T11:33:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- 172.21.15.46:0/3314168024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015ab60 msgr2=0x7f478006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 --2- 172.21.15.46:0/3314168024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015ab60 0x7f478006e8a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f47800c71f0 tx=0x7f4778009780 comp rx=0 tx=0).stop 2023-10-06T11:33:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- 172.21.15.46:0/3314168024 shutdown_connections 2023-10-06T11:33:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 --2- 172.21.15.46:0/3314168024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f478015ab60 0x7f478006e8a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- 172.21.15.46:0/3314168024 >> 172.21.15.46:0/3314168024 conn(0x7f47800cbe30 msgr2=0x7f47800cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- 172.21.15.46:0/3314168024 shutdown_connections 2023-10-06T11:33:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- 172.21.15.46:0/3314168024 wait complete. 2023-10-06T11:33:29.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 Processor -- start 2023-10-06T11:33:29.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.000+0000 7f47865c5700 1 -- start start 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47800ad6c0 0x7f4780150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f478015af30 0x7f478014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c4920 con 0x7f47800ca2a0 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780150380 con 0x7f47800ad6c0 2023-10-06T11:33:29.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800af9a0 con 0x7f478015af30 2023-10-06T11:33:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40436/0 (socket says 172.21.15.46:40436) 2023-10-06T11:33:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 -- 172.21.15.46:0/2778327067 learned_addr learned my addr 172.21.15.46:0/2778327067 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f477ffff700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47800ad6c0 0x7f4780150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f477f7fe700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f478015af30 0x7f478014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 -- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f478015af30 msgr2=0x7f478014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f478015af30 0x7f478014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 -- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47800ad6c0 msgr2=0x7f4780150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47800ad6c0 0x7f4780150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 -- 172.21.15.46:0/2778327067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009a20 con 0x7f47800ca2a0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4784b62700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f477000c4f0 tx=0x7f4770007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000ce60 con 0x7f47800ca2a0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770011590 con 0x7f47800ca2a0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770011930 con 0x7f47800ca2a0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800cab00 con 0x7f47800ca2a0 2023-10-06T11:33:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.004+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47800cb0c0 con 0x7f47800ca2a0 2023-10-06T11:33:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.008+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47800cb2e0 con 0x7f47800ca2a0 2023-10-06T11:33:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.008+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4770011ca0 con 0x7f47800ca2a0 2023-10-06T11:33:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.012+0000 7f4767fff700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 0x7f476c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.012+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f477007bfe0 con 0x7f47800ca2a0 2023-10-06T11:33:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.012+0000 7f477ffff700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 0x7f476c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.012+0000 7f477ffff700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 0x7f476c062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47800c71f0 tx=0x7f4778018000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.016+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f477004c100 con 0x7f47800ca2a0 2023-10-06T11:33:29.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.224+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47800af790 con 0x7f47800ca2a0 2023-10-06T11:33:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.224+0000 7f4767fff700 1 -- 172.21.15.46:0/2778327067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f4770017090 con 0x7f47800ca2a0 2023-10-06T11:33:29.238 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:29.238 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 msgr2=0x7f476c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 0x7f476c062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47800c71f0 tx=0x7f4778018000 comp rx=0 tx=0).stop 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 msgr2=0x7f47800c3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f477000c4f0 tx=0x7f4770007f50 comp rx=0 tx=0).stop 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 shutdown_connections 2023-10-06T11:33:29.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f476c060670 0x7f476c062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47800ca2a0 0x7f47800c3420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f478015af30 0x7f478014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 --2- 172.21.15.46:0/2778327067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47800ad6c0 0x7f4780150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 >> 172.21.15.46:0/2778327067 conn(0x7f47800cbe30 msgr2=0x7f47800adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 shutdown_connections 2023-10-06T11:33:29.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.228+0000 7f47865c5700 1 -- 172.21.15.46:0/2778327067 wait complete. 2023-10-06T11:33:29.244 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:29.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:33:23.035201+0000', 'modified': '2023-10-06T11:33:26.627051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6357}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6357': {'gid': 6357, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/3200380112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3200380112}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 3200380112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:33:29.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:33:29.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:33:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.812+0000 7fac610f0700 1 Processor -- start 2023-10-06T11:33:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 -- start start 2023-10-06T11:33:29.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 0x7fac5c0c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac5c0c2a10 0x7fac5c0c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5c06dd30 con 0x7fac5c155850 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5c0caa10 con 0x7fac5c0c2a10 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac610f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c06ce80 con 0x7fac5c155480 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac53fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fac5c155850 0x7fac5c0c2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33320/0 (socket says 172.21.15.46:33320) 2023-10-06T11:33:29.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac53fff700 1 -- 172.21.15.46:0/1572711603 learned_addr learned my addr 172.21.15.46:0/1572711603 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/1572711603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 0x7fac5c0c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.816+0000 7fac5b59e700 1 --2- 172.21.15.46:0/1572711603 >> v2:172.21.15.132:3300/0 conn(0x7fac5c0c2a10 0x7fac5c0c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5ad9d700 1 -- 172.21.15.46:0/1572711603 >> v2:172.21.15.132:3300/0 conn(0x7fac5c0c2a10 msgr2=0x7fac5c0c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/1572711603 >> v2:172.21.15.132:3300/0 conn(0x7fac5c0c2a10 0x7fac5c0c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5ad9d700 1 -- 172.21.15.46:0/1572711603 >> v1:172.21.15.46:6789/0 conn(0x7fac5c155850 legacy=0x7fac5c0c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5ad9d700 1 -- 172.21.15.46:0/1572711603 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c06d120 con 0x7fac5c155480 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5a59c700 1 -- 172.21.15.46:0/1572711603 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873018268 0 0) 0x7fac5c06dd30 con 0x7fac5c155850 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/1572711603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 0x7fac5c0c1460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0098d0 tx=0x7fac4c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45bc2d2f73c065e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5a59c700 1 -- 172.21.15.46:0/1572711603 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c009e70 con 0x7fac5c155480 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5a59c700 1 -- 172.21.15.46:0/1572711603 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c005c10 con 0x7fac5c155480 2023-10-06T11:33:29.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac5a59c700 1 -- 172.21.15.46:0/1572711603 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00b650 con 0x7fac5c155480 2023-10-06T11:33:29.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 -- 172.21.15.46:0/1572711603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 msgr2=0x7fac5c0c1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 --2- 172.21.15.46:0/1572711603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 0x7fac5c0c1460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0098d0 tx=0x7fac4c009120 comp rx=0 tx=0).stop 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 -- 172.21.15.46:0/1572711603 shutdown_connections 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 --2- 172.21.15.46:0/1572711603 >> v2:172.21.15.132:3300/0 conn(0x7fac5c0c2a10 0x7fac5c0c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 --2- 172.21.15.46:0/1572711603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155480 0x7fac5c0c1460 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 -- 172.21.15.46:0/1572711603 >> 172.21.15.46:0/1572711603 conn(0x7fac5c0cbe30 msgr2=0x7fac5c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 -- 172.21.15.46:0/1572711603 shutdown_connections 2023-10-06T11:33:29.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.820+0000 7fac610f0700 1 -- 172.21.15.46:0/1572711603 wait complete. 2023-10-06T11:33:29.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.824+0000 7fac610f0700 1 Processor -- start 2023-10-06T11:33:29.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.824+0000 7fac610f0700 1 -- start start 2023-10-06T11:33:29.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.824+0000 7fac610f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac5c155480 0x7fac5c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155850 0x7fac5c0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c155ff0 con 0x7fac5c0c2a10 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c1562d0 con 0x7fac5c155480 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c1584f0 con 0x7fac5c155850 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40450/0 (socket says 172.21.15.46:40450) 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 -- 172.21.15.46:0/2810536639 learned_addr learned my addr 172.21.15.46:0/2810536639 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5b59e700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155850 0x7fac5c0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac53fff700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac5c155480 0x7fac5c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 -- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155850 msgr2=0x7fac5c0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155850 0x7fac5c0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 -- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac5c155480 msgr2=0x7fac5c157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac5c155480 0x7fac5c157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 -- 172.21.15.46:0/2810536639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0093c0 con 0x7fac5c0c2a10 2023-10-06T11:33:29.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009ef0 tx=0x7fac4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac52ffd700 1 -- 172.21.15.46:0/2810536639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c006310 con 0x7fac5c0c2a10 2023-10-06T11:33:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac52ffd700 1 -- 172.21.15.46:0/2810536639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c006490 con 0x7fac5c0c2a10 2023-10-06T11:33:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac52ffd700 1 -- 172.21.15.46:0/2810536639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c006850 con 0x7fac5c0c2a10 2023-10-06T11:33:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c1587d0 con 0x7fac5c0c2a10 2023-10-06T11:33:29.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.828+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c14feb0 con 0x7fac5c0c2a10 2023-10-06T11:33:29.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.832+0000 7fac52ffd700 1 -- 172.21.15.46:0/2810536639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fac4c01b690 con 0x7fac5c0c2a10 2023-10-06T11:33:29.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.832+0000 7fac52ffd700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 0x7fac3c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.832+0000 7fac52ffd700 1 -- 172.21.15.46:0/2810536639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fac4c07ee80 con 0x7fac5c0c2a10 2023-10-06T11:33:29.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.832+0000 7fac53fff700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 0x7fac3c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac53fff700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 0x7fac3c062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac5c169a00 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 Processor -- start 2023-10-06T11:33:29.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 -- start start 2023-10-06T11:33:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x1802c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 0x18036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x172cf30 con 0x1581710 2023-10-06T11:33:29.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1728ed0 con 0x15cc790 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x16ef0b0 con 0x1581ae0 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.836+0000 7fac53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40460/0 (socket says 172.21.15.46:40460) 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 -- 172.21.15.46:0/3466328045 learned_addr learned my addr 172.21.15.46:0/3466328045 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x1802c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac5b59e700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 0x18036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 -- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 msgr2=0x1802c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x1802c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 -- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 msgr2=0x18036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 0x18036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 -- 172.21.15.46:0/3466328045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15cd9e0 con 0x1581710 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x1802c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac5b59e700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 0x18036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac53fff700 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac5c158e50 tx=0x7fac44013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9721b075ed46843 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac3bfff700 1 -- 172.21.15.46:0/3466328045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44007770 con 0x1581710 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac3bfff700 1 -- 172.21.15.46:0/3466328045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44005e30 con 0x1581710 2023-10-06T11:33:29.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac3bfff700 1 -- 172.21.15.46:0/3466328045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4400e880 con 0x1581710 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 -- 172.21.15.46:0/3466328045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 msgr2=0x1803170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fac5c158e50 tx=0x7fac44013000 comp rx=0 tx=0).stop 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 -- 172.21.15.46:0/3466328045 shutdown_connections 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x15cc790 0x18036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x1803170 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 --2- 172.21.15.46:0/3466328045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x1802c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 -- 172.21.15.46:0/3466328045 >> 172.21.15.46:0/3466328045 conn(0x1581eb0 msgr2=0x17bb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 -- 172.21.15.46:0/3466328045 shutdown_connections 2023-10-06T11:33:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.840+0000 7fac633db740 1 -- 172.21.15.46:0/3466328045 wait complete. 2023-10-06T11:33:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 -- start start 2023-10-06T11:33:29.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 Processor -- start 2023-10-06T11:33:29.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x17b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x16eed50 con 0x1581ae0 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x16c7970 con 0x1581710 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x177f270 con 0x15cc790 2023-10-06T11:33:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac633db740 10 client.0 _ll_register_callbacks cb 0x16cbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:33:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40474/0 (socket says 172.21.15.46:40474) 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 -- 172.21.15.46:0/19517777 learned_addr learned my addr 172.21.15.46:0/19517777 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x17b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac5b59e700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 -- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 msgr2=0x17b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17b96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 -- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 msgr2=0x17b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x17b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac53fff700 1 -- 172.21.15.46:0/19517777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4400f040 con 0x1581ae0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.844+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x17b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac53fff700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac440073b0 tx=0x7fac44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9145ae6d26689310 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44028070 con 0x1581ae0 2023-10-06T11:33:29.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16c7cc0 con 0x1581ae0 2023-10-06T11:33:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fac4400e840 con 0x1581ae0 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac44015c20 con 0x1581ae0 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 10 client.6504 fetch_fsmap learned FSMap version 34 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x16f7370 con 0x1581ae0 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16c7cc0 con 0x1581ae0 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fac44015080 con 0x1581ae0 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 10 client.6504 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 10 client.6504 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 20 client.6504 populate_metadata read hostname 'smithi046' 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x180f6a0 0x1811a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 4 client.6504 mds_command: new command op to 6357 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:33:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 --> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16cb3e0 con 0x180f6a0 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x180f6a0 0x1811a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac397fa700 10 client.6504 ms_handle_connect on v2:172.21.15.46:6834/3200380112 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.848+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x180f6a0 0x1811a80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cafe35e8c32dcf5 server_cookie=2a864331ff32b15f in_seq=0 out_seq=0 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac397fa700 1 -- 172.21.15.46:0/19517777 <== mds.0 v2:172.21.15.46:6834/3200380112 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x16cb3e0 con 0x180f6a0 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac397fa700 10 client.6504 handle_command_reply: tid=0 2023-10-06T11:33:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 client.6504 shutdown 2023-10-06T11:33:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 msgr2=0x17b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac440073b0 tx=0x7fac44017000 comp rx=0 tx=0).stop 2023-10-06T11:33:29.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 shutdown_connections 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x180f6a0 0x1811a80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581ae0 0x17b91a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 --2- 172.21.15.46:0/19517777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x17b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 >> 172.21.15.46:0/19517777 conn(0x1581eb0 msgr2=0x17e01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac397fa700 0 client.6504 ms_handle_reset on v2:172.21.15.46:6834/3200380112 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 shutdown_connections 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 1 -- 172.21.15.46:0/19517777 wait complete. 2023-10-06T11:33:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.852+0000 7fac633db740 20 client.6504 trim_cache size 0 max 16384 2023-10-06T11:33:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 Processor -- start 2023-10-06T11:33:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 -- start start 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 0x17b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x16fc040 con 0x1581710 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1725150 con 0x1581ae0 2023-10-06T11:33:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x17249b0 con 0x15cc790 2023-10-06T11:33:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40478/0 (socket says 172.21.15.46:40478) 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.872+0000 7fac5ad9d700 1 -- 172.21.15.46:0/3484426629 learned_addr learned my addr 172.21.15.46:0/3484426629 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac53fff700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 0x17b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5b59e700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 -- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 msgr2=0x17de280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17de280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 -- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 msgr2=0x17b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 0x17b6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 -- 172.21.15.46:0/3484426629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c150520 con 0x1581710 2023-10-06T11:33:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac53fff700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 0x17b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac4c058170 tx=0x7fac4c066040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9d64de901f0d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac50ff9700 1 -- 172.21.15.46:0/3484426629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c055ef0 con 0x1581710 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac50ff9700 1 -- 172.21.15.46:0/3484426629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c059c40 con 0x1581710 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac50ff9700 1 -- 172.21.15.46:0/3484426629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c050da0 con 0x1581710 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 -- 172.21.15.46:0/3484426629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 msgr2=0x17b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fac4c058170 tx=0x7fac4c066040 comp rx=0 tx=0).stop 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 -- 172.21.15.46:0/3484426629 shutdown_connections 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x15cc790 0x17de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581ae0 0x17b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 --2- 172.21.15.46:0/3484426629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1581710 0x17b6a10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 -- 172.21.15.46:0/3484426629 >> 172.21.15.46:0/3484426629 conn(0x1581eb0 msgr2=0x17e2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 -- 172.21.15.46:0/3484426629 shutdown_connections 2023-10-06T11:33:29.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.876+0000 7fac633db740 1 -- 172.21.15.46:0/3484426629 wait complete. 2023-10-06T11:33:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 -- start start 2023-10-06T11:33:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 Processor -- start 2023-10-06T11:33:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x185c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x185c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1803b00 con 0x15cc790 2023-10-06T11:33:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x172f400 con 0x1581710 2023-10-06T11:33:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x16f1920 con 0x1581ae0 2023-10-06T11:33:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac633db740 10 client.0 _ll_register_callbacks cb 0x16f4110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:33:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac5b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac5ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x185c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac5b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40484/0 (socket says 172.21.15.46:40484) 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.880+0000 7fac5b59e700 1 -- 172.21.15.46:0/2966235964 learned_addr learned my addr 172.21.15.46:0/2966235964 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac53fff700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x185c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 -- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 msgr2=0x185c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x185c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 -- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 msgr2=0x185c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x185c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 -- 172.21.15.46:0/2966235964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c14feb0 con 0x15cc790 2023-10-06T11:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5b59e700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac54007df0 tx=0x7fac5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dd9290560ca446 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54007e20 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac44005e30 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fac54015cb0 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac54009cf0 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 10 client.6510 fetch_fsmap learned FSMap version 34 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x16ddd80 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44005e30 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fac54018660 con 0x15cc790 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 10 client.6510 fetch_fsmap finished waiting for FSMap version 34 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 10 client.6510 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 20 client.6510 populate_metadata read hostname 'smithi046' 2023-10-06T11:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x1863320 0x1865700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 4 client.6510 mds_command: new command op to 6357 tid=0[{"prefix": "status"}] 2023-10-06T11:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 --> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] -- command(tid 0: {"prefix": "status"}) v1 -- 0x16fd410 con 0x1863320 2023-10-06T11:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.884+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x1863320 0x1865700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac397fa700 10 client.6510 ms_handle_connect on v2:172.21.15.46:6834/3200380112 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac5ad9d700 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x1863320 0x1865700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985adf728a9e966b server_cookie=4a8bf1ffa94b056b in_seq=0 out_seq=0 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac397fa700 1 -- 172.21.15.46:0/2966235964 <== mds.0 v2:172.21.15.46:6834/3200380112 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x16fd410 con 0x1863320 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac397fa700 10 client.6510 handle_command_reply: tid=0 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 client.6510 shutdown 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 msgr2=0x1810a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fac54007df0 tx=0x7fac5400b040 comp rx=0 tx=0).stop 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 shutdown_connections 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:6834/3200380112,v1:172.21.15.46:6836/3200380112] conn(0x1863320 0x1865700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x15cc790 0x1810a40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1581ae0 0x185c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 --2- 172.21.15.46:0/2966235964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1581710 0x185c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac397fa700 0 client.6510 ms_handle_reset on v2:172.21.15.46:6834/3200380112 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 >> 172.21.15.46:0/2966235964 conn(0x1581eb0 msgr2=0x17ba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 shutdown_connections 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 1 -- 172.21.15.46:0/2966235964 wait complete. 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac633db740 20 client.6510 trim_cache size 0 max 16384 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6357, 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:33:29.902 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.2890838589999998, 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 34, 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 79, 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 79, 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 13.257845709 2023-10-06T11:33:29.903 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 msgr2=0x7fac3c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 0x7fac3c062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac5c169a00 tx=0x7fac44009040 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 msgr2=0x7fac5c14f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009ef0 tx=0x7fac4c016000 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 shutdown_connections 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac3c060670 0x7fac3c062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac5c155850 0x7fac5c0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac5c155480 0x7fac5c157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 --2- 172.21.15.46:0/2810536639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c0c2a10 0x7fac5c14f970 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 >> 172.21.15.46:0/2810536639 conn(0x7fac5c0cbe30 msgr2=0x7fac5c0c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 shutdown_connections 2023-10-06T11:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:29.888+0000 7fac610f0700 1 -- 172.21.15.46:0/2810536639 wait complete. 2023-10-06T11:33:29.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:33:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.076+0000 7f82570cd700 1 Processor -- start 2023-10-06T11:33:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.076+0000 7f82570cd700 1 -- start start 2023-10-06T11:33:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82570cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 0x7f825006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82570cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82500ad6c0 0x7f82500adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82570cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825006d460 con 0x7f825015af30 2023-10-06T11:33:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82570cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500c7000 con 0x7f82500ad6c0 2023-10-06T11:33:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82570cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82500c72c0 con 0x7f825015ab60 2023-10-06T11:33:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 0x7f825006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8247fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f825015af30 0x7f825006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33334/0 (socket says 172.21.15.46:33334) 2023-10-06T11:33:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8247fff700 1 -- 172.21.15.46:0/731416270 learned_addr learned my addr 172.21.15.46:0/731416270 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f825566a700 1 --2- 172.21.15.46:0/731416270 >> v2:172.21.15.46:3300/0 conn(0x7f82500ad6c0 0x7f82500adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 -- 172.21.15.46:0/731416270 >> v2:172.21.15.46:3300/0 conn(0x7f82500ad6c0 msgr2=0x7f82500adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 --2- 172.21.15.46:0/731416270 >> v2:172.21.15.46:3300/0 conn(0x7f82500ad6c0 0x7f82500adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 -- 172.21.15.46:0/731416270 >> v1:172.21.15.46:6789/0 conn(0x7f825015af30 legacy=0x7f825006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 -- 172.21.15.46:0/731416270 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500c75a0 con 0x7f825015ab60 2023-10-06T11:33:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f8254e69700 1 --2- 172.21.15.46:0/731416270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 0x7f825006e8a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f824c009eb0 tx=0x7f824c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eecacd7f4dcb6c6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82477fe700 1 -- 172.21.15.46:0/731416270 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00e070 con 0x7f825015ab60 2023-10-06T11:33:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82477fe700 1 -- 172.21.15.46:0/731416270 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00bc30 con 0x7f825015ab60 2023-10-06T11:33:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.080+0000 7f82477fe700 1 -- 172.21.15.46:0/731416270 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c012430 con 0x7f825015ab60 2023-10-06T11:33:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 -- 172.21.15.46:0/731416270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 msgr2=0x7f825006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 --2- 172.21.15.46:0/731416270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 0x7f825006e8a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f824c009eb0 tx=0x7f824c009780 comp rx=0 tx=0).stop 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 -- 172.21.15.46:0/731416270 shutdown_connections 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 --2- 172.21.15.46:0/731416270 >> v2:172.21.15.46:3300/0 conn(0x7f82500ad6c0 0x7f82500adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 --2- 172.21.15.46:0/731416270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825015ab60 0x7f825006e8a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 -- 172.21.15.46:0/731416270 >> 172.21.15.46:0/731416270 conn(0x7f82500cbe30 msgr2=0x7f82500cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 -- 172.21.15.46:0/731416270 shutdown_connections 2023-10-06T11:33:30.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.084+0000 7f82570cd700 1 -- 172.21.15.46:0/731416270 wait complete. 2023-10-06T11:33:30.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 Processor -- start 2023-10-06T11:33:30.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 -- start start 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82500ad6c0 0x7f82500c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f825015af30 0x7f82501569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82500c3e30 con 0x7f825015ab60 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250171760 con 0x7f825015af30 2023-10-06T11:33:30.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f82570cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8250171a20 con 0x7f82500ad6c0 2023-10-06T11:33:30.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40504/0 (socket says 172.21.15.46:40504) 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 -- 172.21.15.46:0/3481267984 learned_addr learned my addr 172.21.15.46:0/3481267984 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8254e69700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82500ad6c0 0x7f82500c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f825566a700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f825015af30 0x7f82501569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 -- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82500ad6c0 msgr2=0x7f82500c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82500ad6c0 0x7f82500c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 -- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f825015af30 msgr2=0x7f82501569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f825015af30 0x7f82501569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 -- 172.21.15.46:0/3481267984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c009a20 con 0x7f825015ab60 2023-10-06T11:33:30.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.088+0000 7f8247fff700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f824000c790 tx=0x7f8240007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240005af0 con 0x7f825015ab60 2023-10-06T11:33:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8240005c70 con 0x7f825015ab60 2023-10-06T11:33:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82400070d0 con 0x7f825015ab60 2023-10-06T11:33:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8250171d00 con 0x7f825015ab60 2023-10-06T11:33:30.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82501722c0 con 0x7f825015ab60 2023-10-06T11:33:30.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82501724e0 con 0x7f825015ab60 2023-10-06T11:33:30.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.092+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f82400074c0 con 0x7f825015ab60 2023-10-06T11:33:30.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.096+0000 7f8244ff9700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 0x7f823c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:30.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.096+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f824007bf30 con 0x7f825015ab60 2023-10-06T11:33:30.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.096+0000 7f8254e69700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 0x7f823c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:30.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.096+0000 7f8254e69700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 0x7f823c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f824c009eb0 tx=0x7f824c017000 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:30.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.100+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8240049be0 con 0x7f825015ab60 2023-10-06T11:33:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.308+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8250063f90 con 0x7f825015ab60 2023-10-06T11:33:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f8244ff9700 1 -- 172.21.15.46:0/3481267984 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8240017060 con 0x7f825015ab60 2023-10-06T11:33:30.324 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 msgr2=0x7f823c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 0x7f823c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f824c009eb0 tx=0x7f824c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 msgr2=0x7f8250155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f824000c790 tx=0x7f8240007f60 comp rx=0 tx=0).stop 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 shutdown_connections 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f823c060670 0x7f823c062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f825015af30 0x7f82501569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f825015ab60 0x7f8250155380 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 --2- 172.21.15.46:0/3481267984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82500ad6c0 0x7f82500c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.312+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 >> 172.21.15.46:0/3481267984 conn(0x7f82500cbe30 msgr2=0x7f82500ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.316+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 shutdown_connections 2023-10-06T11:33:30.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:30.316+0000 7f82570cd700 1 -- 172.21.15.46:0/3481267984 wait complete. 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:33:30.348 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:33:30.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.349 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:33:30.369 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:33:30.369 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:33:30.369 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:33:30.369 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.370 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41520sec preferred_lft 41520sec 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.371 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:33:30.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.373 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:33:30.373 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:33:30.373 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:33:30.373 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:33:30.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.374 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:33:30.453 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:33:30.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.457 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:33:30.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:33:30.520 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:33:30.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.660 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:33:30.677 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:33:30.677 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:33:30.678 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:33:30.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:30.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:33:30.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:33:30.760 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:30.825 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:33:30.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:33:30.825 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:30.884 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:33:30.884 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:33:30.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:33:31.012 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:33:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:33:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:33:31.053 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:33:31.058 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:33:31.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:31.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:31.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:33:31.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.284+0000 7f6a5f4ae700 1 Processor -- start 2023-10-06T11:33:31.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.284+0000 7f6a5f4ae700 1 -- start start 2023-10-06T11:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5f4ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 0x7f6a58161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5f4ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6a580c21b0 0x7f6a580caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5f4ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580c2630 con 0x7f6a580c1d80 2023-10-06T11:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5f4ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a580c18f0 con 0x7f6a580c21b0 2023-10-06T11:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5f4ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c0ce0 con 0x7f6a58161470 2023-10-06T11:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5ca49700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a580c1d80 0x7f6a5806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33342/0 (socket says 172.21.15.46:33342) 2023-10-06T11:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5ca49700 1 -- 172.21.15.46:0/2253483893 learned_addr learned my addr 172.21.15.46:0/2253483893 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5da4b700 1 --2- 172.21.15.46:0/2253483893 >> v2:172.21.15.132:3300/0 conn(0x7f6a580c21b0 0x7f6a580caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/2253483893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 0x7f6a58161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a4ffff700 1 -- 172.21.15.46:0/2253483893 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110650980 0 0) 0x7f6a580c2630 con 0x7f6a580c1d80 2023-10-06T11:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a4ffff700 1 -- 172.21.15.46:0/2253483893 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003570 con 0x7f6a580c1d80 2023-10-06T11:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5d24a700 1 -- 172.21.15.46:0/2253483893 >> v2:172.21.15.132:3300/0 conn(0x7f6a580c21b0 msgr2=0x7f6a580caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/2253483893 >> v2:172.21.15.132:3300/0 conn(0x7f6a580c21b0 0x7f6a580caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5d24a700 1 -- 172.21.15.46:0/2253483893 >> v1:172.21.15.46:6789/0 conn(0x7f6a580c1d80 legacy=0x7f6a5806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.288+0000 7f6a5d24a700 1 -- 172.21.15.46:0/2253483893 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580c0fc0 con 0x7f6a58161470 2023-10-06T11:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/2253483893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 0x7f6a58161840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a540098d0 tx=0x7f6a54009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e71a6bc355acf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a4ffff700 1 -- 172.21.15.46:0/2253483893 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54005c10 con 0x7f6a58161470 2023-10-06T11:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a4ffff700 1 -- 172.21.15.46:0/2253483893 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54005d70 con 0x7f6a58161470 2023-10-06T11:33:31.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a4ffff700 1 -- 172.21.15.46:0/2253483893 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54006260 con 0x7f6a58161470 2023-10-06T11:33:31.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/2253483893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 msgr2=0x7f6a58161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/2253483893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 0x7f6a58161840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a540098d0 tx=0x7f6a54009120 comp rx=0 tx=0).stop 2023-10-06T11:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/2253483893 shutdown_connections 2023-10-06T11:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/2253483893 >> v2:172.21.15.132:3300/0 conn(0x7f6a580c21b0 0x7f6a580caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/2253483893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a58161470 0x7f6a58161840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/2253483893 >> 172.21.15.46:0/2253483893 conn(0x7f6a580cbe30 msgr2=0x7f6a580cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/2253483893 shutdown_connections 2023-10-06T11:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.292+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/2253483893 wait complete. 2023-10-06T11:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 Processor -- start 2023-10-06T11:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 -- start start 2023-10-06T11:33:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a580c21b0 0x7f6a580c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 0x7f6a580c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58162810 con 0x7f6a58161470 2023-10-06T11:33:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58147bb0 con 0x7f6a580c21b0 2023-10-06T11:33:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.296+0000 7f6a5f4ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580cb320 con 0x7f6a580c1d80 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5ca49700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a580c21b0 0x7f6a580c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:59690/0 (socket says 172.21.15.46:59690) 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 -- 172.21.15.46:0/3168675615 learned_addr learned my addr 172.21.15.46:0/3168675615 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5da4b700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 0x7f6a580c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 -- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a580c21b0 msgr2=0x7f6a580c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a580c21b0 0x7f6a580c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 -- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 msgr2=0x7f6a580c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 0x7f6a580c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 -- 172.21.15.46:0/3168675615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a540093c0 con 0x7f6a580c1d80 2023-10-06T11:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5da4b700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 0x7f6a580c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5d24a700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009ec0 tx=0x7f6a54016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5401d7b0 con 0x7f6a580c1d80 2023-10-06T11:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5401d910 con 0x7f6a580c1d80 2023-10-06T11:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5401dcb0 con 0x7f6a580c1d80 2023-10-06T11:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580cb600 con 0x7f6a580c1d80 2023-10-06T11:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.300+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a58162d00 con 0x7f6a580c1d80 2023-10-06T11:33:31.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.304+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58063f90 con 0x7f6a580c1d80 2023-10-06T11:33:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.304+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6a5401a420 con 0x7f6a580c1d80 2023-10-06T11:33:31.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.308+0000 7f6a4d7fa700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 0x7f6a3c062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.308+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6a5407e2b0 con 0x7f6a580c1d80 2023-10-06T11:33:31.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.308+0000 7f6a5ca49700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 0x7f6a3c062c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.308+0000 7f6a5ca49700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 0x7f6a3c062c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ab9a0 tx=0x7f6a48009040 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.312+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6a5404e450 con 0x7f6a580c1d80 2023-10-06T11:33:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.504+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a580c12a0 con 0x7f6a3c0607c0 2023-10-06T11:33:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.508+0000 7f6a4d7fa700 1 -- 172.21.15.46:0/3168675615 <== mgr.6150 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6a580c12a0 con 0x7f6a3c0607c0 2023-10-06T11:33:31.522 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:33:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:33:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:31.523 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:33:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 msgr2=0x7f6a3c062c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 0x7f6a3c062c70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a580ab9a0 tx=0x7f6a48009040 comp rx=0 tx=0).stop 2023-10-06T11:33:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 msgr2=0x7f6a5816dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009ec0 tx=0x7f6a54016000 comp rx=0 tx=0).stop 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 shutdown_connections 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a3c0607c0 0x7f6a3c062c70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58161470 0x7f6a580c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a580c21b0 0x7f6a580c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 --2- 172.21.15.46:0/3168675615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a580c1d80 0x7f6a5816dd50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 >> 172.21.15.46:0/3168675615 conn(0x7f6a580cbe30 msgr2=0x7f6a580ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 shutdown_connections 2023-10-06T11:33:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.512+0000 7f6a5f4ae700 1 -- 172.21.15.46:0/3168675615 wait complete. 2023-10-06T11:33:31.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:33:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 Processor -- start 2023-10-06T11:33:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 -- start start 2023-10-06T11:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f123c1591b0 0x7f123c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f123c159850 0x7f123c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c0ce6e0 con 0x7f123c069270 2023-10-06T11:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c15a5c0 con 0x7f123c1591b0 2023-10-06T11:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f1240877700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0cddb0 con 0x7f123c159850 2023-10-06T11:33:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f123259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f123c1591b0 0x7f123c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f123259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f123c1591b0 0x7f123c069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40530/0 (socket says 172.21.15.46:40530) 2023-10-06T11:33:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f123259c700 1 -- 172.21.15.46:0/1425580528 learned_addr learned my addr 172.21.15.46:0/1425580528 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f123b59e700 1 --2- 172.21.15.46:0/1425580528 >> v2:172.21.15.132:3300/0 conn(0x7f123c159850 0x7f123c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.708+0000 7f123ad9d700 1 --1- 172.21.15.46:0/1425580528 >> v1:172.21.15.132:6789/0 conn(0x7f123c069270 0x7f123c158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42358/0 (socket says 172.21.15.46:42358) 2023-10-06T11:33:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123259c700 1 -- 172.21.15.46:0/1425580528 >> v2:172.21.15.132:3300/0 conn(0x7f123c159850 msgr2=0x7f123c159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123259c700 1 --2- 172.21.15.46:0/1425580528 >> v2:172.21.15.132:3300/0 conn(0x7f123c159850 0x7f123c159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123259c700 1 -- 172.21.15.46:0/1425580528 >> v1:172.21.15.132:6789/0 conn(0x7f123c069270 legacy=0x7f123c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123259c700 1 -- 172.21.15.46:0/1425580528 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0ce090 con 0x7f123c1591b0 2023-10-06T11:33:31.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123a59c700 1 -- 172.21.15.46:0/1425580528 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851944447 0 0) 0x7f123c0ce6e0 con 0x7f123c069270 2023-10-06T11:33:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123259c700 1 --2- 172.21.15.46:0/1425580528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c069640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f123c067710 tx=0x7f1224009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28ce271b49c4da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123a59c700 1 -- 172.21.15.46:0/1425580528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224009e70 con 0x7f123c1591b0 2023-10-06T11:33:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123a59c700 1 -- 172.21.15.46:0/1425580528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400b860 con 0x7f123c1591b0 2023-10-06T11:33:31.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f123a59c700 1 -- 172.21.15.46:0/1425580528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400bc00 con 0x7f123c1591b0 2023-10-06T11:33:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 -- 172.21.15.46:0/1425580528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 msgr2=0x7f123c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 --2- 172.21.15.46:0/1425580528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c069640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f123c067710 tx=0x7f1224009250 comp rx=0 tx=0).stop 2023-10-06T11:33:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 -- 172.21.15.46:0/1425580528 shutdown_connections 2023-10-06T11:33:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 --2- 172.21.15.46:0/1425580528 >> v2:172.21.15.132:3300/0 conn(0x7f123c159850 0x7f123c159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 --2- 172.21.15.46:0/1425580528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c069640 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 -- 172.21.15.46:0/1425580528 >> 172.21.15.46:0/1425580528 conn(0x7f123c068390 msgr2=0x7f123c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 -- 172.21.15.46:0/1425580528 shutdown_connections 2023-10-06T11:33:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.712+0000 7f1240877700 1 -- 172.21.15.46:0/1425580528 wait complete. 2023-10-06T11:33:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 Processor -- start 2023-10-06T11:33:31.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 -- start start 2023-10-06T11:33:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f123c069270 0x7f123c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 0x7f123c14bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c15b140 con 0x7f123c1591b0 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c15b420 con 0x7f123c069270 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f1240877700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c15b700 con 0x7f123c159850 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40542/0 (socket says 172.21.15.46:40542) 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123259c700 1 -- 172.21.15.46:0/1900884520 learned_addr learned my addr 172.21.15.46:0/1900884520 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123ad9d700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f123c069270 0x7f123c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123b59e700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 0x7f123c14bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123259c700 1 -- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 msgr2=0x7f123c14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.716+0000 7f123259c700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 0x7f123c14bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f123259c700 1 -- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f123c069270 msgr2=0x7f123c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f123259c700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f123c069270 0x7f123c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f123259c700 1 -- 172.21.15.46:0/1900884520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240094f0 con 0x7f123c1591b0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f123b59e700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 0x7f123c14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f123259c700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f12240094c0 tx=0x7f1224017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122400b9d0 con 0x7f123c1591b0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122400bb50 con 0x7f123c1591b0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224025670 con 0x7f123c1591b0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c15b9e0 con 0x7f123c1591b0 2023-10-06T11:33:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c15ae10 con 0x7f123c1591b0 2023-10-06T11:33:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.720+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c15bee0 con 0x7f123c1591b0 2023-10-06T11:33:31.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.724+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1224023070 con 0x7f123c1591b0 2023-10-06T11:33:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.724+0000 7f12337fe700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 0x7f121c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.724+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f122407e1b0 con 0x7f123c1591b0 2023-10-06T11:33:31.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.724+0000 7f123ad9d700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 0x7f121c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:31.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.724+0000 7f123ad9d700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 0x7f121c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f122c007fd0 tx=0x7f122c007d50 comp rx=0 tx=0).ready entity=mgr.6150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.728+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f122404e370 con 0x7f123c1591b0 2023-10-06T11:33:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:31.940+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f123c064590 con 0x7f123c1591b0 2023-10-06T11:33:32.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.840+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 28) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f122404e740 con 0x7f123c1591b0 2023-10-06T11:33:32.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f12337fe700 1 -- 172.21.15.46:0/1900884520 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v28) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1224051e00 con 0x7f123c1591b0 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 msgr2=0x7f121c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 0x7f121c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f122c007fd0 tx=0x7f122c007d50 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 msgr2=0x7f123c15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f12240094c0 tx=0x7f1224017000 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 shutdown_connections 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f121c060670 0x7f121c062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f123c159850 0x7f123c14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f123c1591b0 0x7f123c15c1e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 --2- 172.21.15.46:0/1900884520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f123c069270 0x7f123c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 >> 172.21.15.46:0/1900884520 conn(0x7f123c068390 msgr2=0x7f123c0c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 shutdown_connections 2023-10-06T11:33:32.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:32.844+0000 7f1240877700 1 -- 172.21.15.46:0/1900884520 wait complete. 2023-10-06T11:33:32.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:33:33.034 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:33.030+0000 7f5a6acdc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:33.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.032+0000 7f5466e8e700 1 Processor -- start 2023-10-06T11:33:33.045 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:33.032+0000 7f5399d63000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:33.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.032+0000 7f5466e8e700 1 -- start start 2023-10-06T11:33:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54600c97e0 0x7f54600c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f54600c6250 0x7f546006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54600c1d90 con 0x7f546006cf30 2023-10-06T11:33:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600c1460 con 0x7f54600c97e0 2023-10-06T11:33:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600c1740 con 0x7f54600c6250 2023-10-06T11:33:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f546668d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f546006cf30 0x7f54600c2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33356/0 (socket says 172.21.15.46:33356) 2023-10-06T11:33:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f546668d700 1 -- 172.21.15.46:0/3701791031 learned_addr learned my addr 172.21.15.46:0/3701791031 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3701791031 >> v2:172.21.15.46:3300/0 conn(0x7f54600c97e0 0x7f54600c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f546568b700 1 --2- 172.21.15.46:0/3701791031 >> v2:172.21.15.132:3300/0 conn(0x7f54600c6250 0x7f546006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 -- 172.21.15.46:0/3701791031 >> v2:172.21.15.132:3300/0 conn(0x7f54600c6250 msgr2=0x7f546006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3701791031 >> v2:172.21.15.132:3300/0 conn(0x7f54600c6250 0x7f546006c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 -- 172.21.15.46:0/3701791031 >> v1:172.21.15.46:6789/0 conn(0x7f546006cf30 legacy=0x7f54600c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 -- 172.21.15.46:0/3701791031 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600c1a20 con 0x7f54600c97e0 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5464e8a700 1 -- 172.21.15.46:0/3701791031 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305030817 0 0) 0x7f54600c1d90 con 0x7f546006cf30 2023-10-06T11:33:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3701791031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c97e0 0x7f54600c7d40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f54600c9690 tx=0x7f545c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57345016fa47e3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5464e8a700 1 -- 172.21.15.46:0/3701791031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c011630 con 0x7f54600c97e0 2023-10-06T11:33:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5464e8a700 1 -- 172.21.15.46:0/3701791031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c011790 con 0x7f54600c97e0 2023-10-06T11:33:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5464e8a700 1 -- 172.21.15.46:0/3701791031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c011b30 con 0x7f54600c97e0 2023-10-06T11:33:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- 172.21.15.46:0/3701791031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c97e0 msgr2=0x7f54600c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3701791031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c97e0 0x7f54600c7d40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f54600c9690 tx=0x7f545c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- 172.21.15.46:0/3701791031 shutdown_connections 2023-10-06T11:33:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3701791031 >> v2:172.21.15.132:3300/0 conn(0x7f54600c6250 0x7f546006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3701791031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c97e0 0x7f54600c7d40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.036+0000 7f5466e8e700 1 -- 172.21.15.46:0/3701791031 >> 172.21.15.46:0/3701791031 conn(0x7f54600cbc10 msgr2=0x7f54600cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.040+0000 7f5466e8e700 1 -- 172.21.15.46:0/3701791031 shutdown_connections 2023-10-06T11:33:33.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.040+0000 7f5466e8e700 1 -- 172.21.15.46:0/3701791031 wait complete. 2023-10-06T11:33:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.040+0000 7f5466e8e700 1 Processor -- start 2023-10-06T11:33:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.040+0000 7f5466e8e700 1 -- start start 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 0x7f54600c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54600c97e0 0x7f5460171b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600c48c0 con 0x7f54600c6250 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600c4b40 con 0x7f54600c97e0 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600c4e00 con 0x7f546006cf30 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40582/0 (socket says 172.21.15.46:40582) 2023-10-06T11:33:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 -- 172.21.15.46:0/3457825271 learned_addr learned my addr 172.21.15.46:0/3457825271 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546668d700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54600c97e0 0x7f5460171b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 0x7f54600c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 -- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 msgr2=0x7f54600c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 0x7f54600c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 -- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54600c97e0 msgr2=0x7f5460171b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54600c97e0 0x7f5460171b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 -- 172.21.15.46:0/3457825271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c009420 con 0x7f54600c6250 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 0x7f54600c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f546568b700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f545000e930 tx=0x7f54500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450013680 con 0x7f54600c6250 2023-10-06T11:33:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500137e0 con 0x7f54600c6250 2023-10-06T11:33:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54600c50e0 con 0x7f54600c6250 2023-10-06T11:33:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450013b80 con 0x7f54600c6250 2023-10-06T11:33:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.044+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600c56a0 con 0x7f54600c6250 2023-10-06T11:33:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5460064590 con 0x7f54600c6250 2023-10-06T11:33:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f545000c460 con 0x7f54600c6250 2023-10-06T11:33:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5455ffb700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 0x7f5444062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f545007cf30 con 0x7f54600c6250 2023-10-06T11:33:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5465e8c700 1 -- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 msgr2=0x7f5444062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.048+0000 7f5465e8c700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 0x7f5444062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.056+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f545004d050 con 0x7f54600c6250 2023-10-06T11:33:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.232+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f546015d840 con 0x7f54600c6250 2023-10-06T11:33:33.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.240+0000 7f5455ffb700 1 -- 172.21.15.46:0/3457825271 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5450050670 con 0x7f54600c6250 2023-10-06T11:33:33.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 msgr2=0x7f5444062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 0x7f5444062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 msgr2=0x7f5460171620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f545000e930 tx=0x7f54500091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 shutdown_connections 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f54440606c0 0x7f5444062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54600c97e0 0x7f5460171b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600c6250 0x7f5460171620 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 --2- 172.21.15.46:0/3457825271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f546006cf30 0x7f54600c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 >> 172.21.15.46:0/3457825271 conn(0x7f54600cbc10 msgr2=0x7f5460155fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 shutdown_connections 2023-10-06T11:33:33.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.244+0000 7f5466e8e700 1 -- 172.21.15.46:0/3457825271 wait complete. 2023-10-06T11:33:33.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:33:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.432+0000 7fd660f45700 1 Processor -- start 2023-10-06T11:33:33.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.432+0000 7fd660f45700 1 -- start start 2023-10-06T11:33:33.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd660f45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd65c0b23c0 0x7fd65c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd660f45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0cc510 con 0x7fd65c0cd250 2023-10-06T11:33:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd660f45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0cb900 con 0x7fd65c0cae40 2023-10-06T11:33:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd660f45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c0cbbe0 con 0x7fd65c0b23c0 2023-10-06T11:33:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd65c0b23c0 0x7fd65c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd65c0cd250 0x7fd65c0c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33288/0 (socket says 172.21.15.46:33288) 2023-10-06T11:33:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65ad9d700 1 -- 172.21.15.46:0/3626627911 learned_addr learned my addr 172.21.15.46:0/3626627911 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65a59c700 1 -- 172.21.15.46:0/3626627911 >> v1:172.21.15.132:6789/0 conn(0x7fd65c0cae40 legacy=0x7fd65c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65a59c700 1 -- 172.21.15.46:0/3626627911 >> v1:172.21.15.46:6789/0 conn(0x7fd65c0cd250 legacy=0x7fd65c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65a59c700 1 -- 172.21.15.46:0/3626627911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0cbec0 con 0x7fd65c0b23c0 2023-10-06T11:33:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65959a700 1 -- 172.21.15.46:0/3626627911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933748978 0 0) 0x7fd65c0cc510 con 0x7fd65c0cd250 2023-10-06T11:33:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.436+0000 7fd65a59c700 1 --2- 172.21.15.46:0/3626627911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0b23c0 0x7fd65c0b2790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098d0 tx=0x7fd64c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130920912c4df217 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd65959a700 1 -- 172.21.15.46:0/3626627911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c009e70 con 0x7fd65c0b23c0 2023-10-06T11:33:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd65959a700 1 -- 172.21.15.46:0/3626627911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c00b860 con 0x7fd65c0b23c0 2023-10-06T11:33:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd65959a700 1 -- 172.21.15.46:0/3626627911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00bc00 con 0x7fd65c0b23c0 2023-10-06T11:33:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 -- 172.21.15.46:0/3626627911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0b23c0 msgr2=0x7fd65c0b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 --2- 172.21.15.46:0/3626627911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0b23c0 0x7fd65c0b2790 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098d0 tx=0x7fd64c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 -- 172.21.15.46:0/3626627911 shutdown_connections 2023-10-06T11:33:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 --2- 172.21.15.46:0/3626627911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0b23c0 0x7fd65c0b2790 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 -- 172.21.15.46:0/3626627911 >> 172.21.15.46:0/3626627911 conn(0x7fd65c099480 msgr2=0x7fd65c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 -- 172.21.15.46:0/3626627911 shutdown_connections 2023-10-06T11:33:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.440+0000 7fd660f45700 1 -- 172.21.15.46:0/3626627911 wait complete. 2023-10-06T11:33:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 Processor -- start 2023-10-06T11:33:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 -- start start 2023-10-06T11:33:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 0x7fd65c09d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd65c0cae40 0x7fd65c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c8af0 con 0x7fd65c0cd250 2023-10-06T11:33:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c1560b0 con 0x7fd65c0b23c0 2023-10-06T11:33:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd660f45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c87a0 con 0x7fd65c0cae40 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 0x7fd65c09d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 0x7fd65c09d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:41884/0 (socket says 172.21.15.46:41884) 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45594/0 (socket says 172.21.15.46:45594) 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65a59c700 1 -- 172.21.15.46:0/2470108710 learned_addr learned my addr 172.21.15.46:0/2470108710 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd659d9b700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd65c0cae40 0x7fd65c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 -- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd65c0cae40 msgr2=0x7fd65c0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd65c0cae40 0x7fd65c0c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 -- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 msgr2=0x7fd65c09d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.444+0000 7fd65ad9d700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 0x7fd65c09d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd65ad9d700 1 -- 172.21.15.46:0/2470108710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0093c0 con 0x7fd65c0cd250 2023-10-06T11:33:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd65ad9d700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cd650 tx=0x7fd654001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400a7e0 con 0x7fd65c0cd250 2023-10-06T11:33:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65400a940 con 0x7fd65c0cd250 2023-10-06T11:33:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400ad00 con 0x7fd65c0cd250 2023-10-06T11:33:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c0b2fe0 con 0x7fd65c0cd250 2023-10-06T11:33:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c0b35a0 con 0x7fd65c0cd250 2023-10-06T11:33:33.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.448+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c0c8c60 con 0x7fd65c0cd250 2023-10-06T11:33:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.452+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fd654007300 con 0x7fd65c0cd250 2023-10-06T11:33:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.452+0000 7fd64a7fc700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 0x7fd63c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.452+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd65407cff0 con 0x7fd65c0cd250 2023-10-06T11:33:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.452+0000 7fd65a59c700 1 -- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 msgr2=0x7fd63c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:33.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.452+0000 7fd65a59c700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 0x7fd63c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:33.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.456+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd65404d0a0 con 0x7fd65c0cd250 2023-10-06T11:33:33.557 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:33.554+0000 7f5a6acdc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:33.625 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:33.622+0000 7f5a6acdc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:33.628 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:33.616+0000 7f5399d63000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:33.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.624+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd65c0660f0 con 0x7fd65c0cd250 2023-10-06T11:33:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd64a7fc700 1 -- 172.21.15.46:0/2470108710 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd65400ae60 con 0x7fd65c0cd250 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 msgr2=0x7fd63c062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 0x7fd63c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 msgr2=0x7fd65c0c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cd650 tx=0x7fd654001980 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 shutdown_connections 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd63c0606f0 0x7fd63c062ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd65c0cd250 0x7fd65c0c64f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd65c0cae40 0x7fd65c0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 --2- 172.21.15.46:0/2470108710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd65c0b23c0 0x7fd65c09d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 >> 172.21.15.46:0/2470108710 conn(0x7fd65c099480 msgr2=0x7fd65c158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 shutdown_connections 2023-10-06T11:33:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.632+0000 7fd660f45700 1 -- 172.21.15.46:0/2470108710 wait complete. 2023-10-06T11:33:33.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:33.666 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:33:33.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T11:33:33.709 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:33.696+0000 7f5399d63000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:33.760 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:33.758+0000 7f5a6acdc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:33.861 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:33.848+0000 7f5399d63000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:33.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.860+0000 7feaae416700 1 Processor -- start 2023-10-06T11:33:33.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 -- start start 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feaa81595c0 0x7feaa80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feaa80cc500 0x7feaa80afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa80cf360 con 0x7feaa81591f0 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa80cff00 con 0x7feaa80cc500 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaae416700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa80b03c0 con 0x7feaa81595c0 2023-10-06T11:33:33.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feaa80cc500 0x7feaa80afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feaa80cc500 0x7feaa80afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45608/0 (socket says 172.21.15.46:45608) 2023-10-06T11:33:33.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 -- 172.21.15.46:0/2029769323 learned_addr learned my addr 172.21.15.46:0/2029769323 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:33.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaacc13700 1 --2- 172.21.15.46:0/2029769323 >> v2:172.21.15.132:3300/0 conn(0x7feaa81595c0 0x7feaa80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 -- 172.21.15.46:0/2029769323 >> v2:172.21.15.132:3300/0 conn(0x7feaa81595c0 msgr2=0x7feaa80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 --2- 172.21.15.46:0/2029769323 >> v2:172.21.15.132:3300/0 conn(0x7feaa81595c0 0x7feaa80af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 -- 172.21.15.46:0/2029769323 >> v1:172.21.15.46:6789/0 conn(0x7feaa81591f0 legacy=0x7feaa80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.864+0000 7feaadc15700 1 -- 172.21.15.46:0/2029769323 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa815dd60 con 0x7feaa80cc500 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaadc15700 1 --2- 172.21.15.46:0/2029769323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa80cc500 0x7feaa80afe80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feaa8088580 tx=0x7feaa0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d0b1d1cd534dca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7fea9ffff700 1 -- 172.21.15.46:0/2029769323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000e070 con 0x7feaa80cc500 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7fea9ffff700 1 -- 172.21.15.46:0/2029769323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa000bab0 con 0x7feaa80cc500 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7fea9ffff700 1 -- 172.21.15.46:0/2029769323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa00092e0 con 0x7feaa80cc500 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 -- 172.21.15.46:0/2029769323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa80cc500 msgr2=0x7feaa80afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 --2- 172.21.15.46:0/2029769323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa80cc500 0x7feaa80afe80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feaa8088580 tx=0x7feaa0009840 comp rx=0 tx=0).stop 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 -- 172.21.15.46:0/2029769323 shutdown_connections 2023-10-06T11:33:33.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 --2- 172.21.15.46:0/2029769323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa80cc500 0x7feaa80afe80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 --2- 172.21.15.46:0/2029769323 >> v2:172.21.15.132:3300/0 conn(0x7feaa81595c0 0x7feaa80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 -- 172.21.15.46:0/2029769323 >> 172.21.15.46:0/2029769323 conn(0x7feaa806ef60 msgr2=0x7feaa80d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 -- 172.21.15.46:0/2029769323 shutdown_connections 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.868+0000 7feaae416700 1 -- 172.21.15.46:0/2029769323 wait complete. 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 Processor -- start 2023-10-06T11:33:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 -- start start 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feaa80cc500 0x7feaa815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feaa81591f0 0x7feaa816ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa8169ee0 0x7feaa816a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa8099dc0 con 0x7feaa8169ee0 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa816b570 con 0x7feaa81591f0 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa816a960 con 0x7feaa80cc500 2023-10-06T11:33:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa8169ee0 0x7feaa816a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa8169ee0 0x7feaa816a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45616/0 (socket says 172.21.15.46:45616) 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 -- 172.21.15.46:0/3669050891 learned_addr learned my addr 172.21.15.46:0/3669050891 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaacc13700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feaa81591f0 0x7feaa816ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feaa80cc500 0x7feaa815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feaa80cc500 msgr2=0x7feaa815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feaa80cc500 0x7feaa815c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feaa81591f0 msgr2=0x7feaa816ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feaa81591f0 0x7feaa816ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0009ae0 con 0x7feaa8169ee0 2023-10-06T11:33:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaadc15700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feaa8169ee0 0x7feaa816a380 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7feaa8088770 tx=0x7feaa0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa00032f0 con 0x7feaa8169ee0 2023-10-06T11:33:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa0008e40 con 0x7feaa8169ee0 2023-10-06T11:33:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa001c660 con 0x7feaa8169ee0 2023-10-06T11:33:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.872+0000 7feaae416700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa816ac40 con 0x7feaa8169ee0 2023-10-06T11:33:33.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.876+0000 7feaae416700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa816b200 con 0x7feaa8169ee0 2023-10-06T11:33:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.876+0000 7feaae416700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa8064590 con 0x7feaa8169ee0 2023-10-06T11:33:33.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.876+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) v1 ==== 80078+0+0 (secure 0 0 0) 0x7feaa0003040 con 0x7feaa8169ee0 2023-10-06T11:33:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.876+0000 7fea9d7fa700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.880+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6502+0+0 (secure 0 0 0) 0x7feaa007e650 con 0x7feaa8169ee0 2023-10-06T11:33:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.880+0000 7feaad414700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:33.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.880+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:33.884+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feaa004e660 con 0x7feaa8169ee0 2023-10-06T11:33:34.019 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.018+0000 7f5a6acdc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:34.090 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.086+0000 7f5a6acdc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:34.084+0000 7feaad414700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:34.084+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:34.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:34.124+0000 7feaae416700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7feaa816b6b0 con 0x7fea8c0606f0 2023-10-06T11:33:34.187 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:34.172+0000 7f5399d63000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:34.268 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:34.256+0000 7f5399d63000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:34.314 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.310+0000 7f5a6acdc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:34.399 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.398+0000 7f5a6acdc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:34.482 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.482+0000 7f5a6acdc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:34.493 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:34.480+0000 7f5399d63000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:34.488+0000 7feaad414700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:34.488+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:33:34.582 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:34.568+0000 7f5399d63000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:34.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:34.660+0000 7f5399d63000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:34.890 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:34.890+0000 7f5a6acdc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:35.058 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.054+0000 7f5a6acdc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:35.090 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:35.076+0000 7f5399d63000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:35.262 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:35.248+0000 7f5399d63000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:35.292+0000 7feaad414700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:35.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:35.292+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:33:35.432 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.430+0000 7f5a6acdc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:35.650 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:35.636+0000 7f5399d63000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:35.658 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.654+0000 7f5a6acdc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:35.751 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.750+0000 7f5a6acdc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:35.826 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.822+0000 7f5a6acdc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:35.879 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:35.864+0000 7f5399d63000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:35.908 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:35.906+0000 7f5a6acdc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:35.976 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:35.964+0000 7f5399d63000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:36.061 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.048+0000 7f5399d63000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:36.141 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:36.138+0000 7f5a6acdc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:36.143 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.132+0000 7f5399d63000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:36.222 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:36.218+0000 7f5a6acdc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:36.315 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:36.314+0000 7f5a6acdc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:36.384 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.372+0000 7f5399d63000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:36.470 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.456+0000 7f5399d63000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:36.513 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:36.510+0000 7f5a6acdc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:36.565 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.552+0000 7f5399d63000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:36.603 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:36.602+0000 7f5a6acdc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:36.777 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.764+0000 7f5399d63000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:36.869 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:36.856+0000 7f5399d63000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:36.896+0000 7feaad414700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:36.896+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:33:37.320 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:37.318+0000 7f5a6acdc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:37.376+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 29) v1 ==== 80078+0+0 (secure 0 0 0) 0x7feaa004ea30 con 0x7feaa8169ee0 2023-10-06T11:33:37.631 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.616+0000 7f5399d63000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:37.636+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 30) v1 ==== 79687+0+0 (secure 0 0 0) 0x7feaa0029d00 con 0x7feaa8169ee0 2023-10-06T11:33:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:37.636+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 msgr2=0x7fea8c062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:37.636+0000 7fea9d7fa700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:37.842 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.828+0000 7f535b90f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.842 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.828+0000 7f535b90f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.843 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.828+0000 7f535b90f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.843 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.828+0000 7f535b90f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.843 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.828+0000 7f535b90f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.844 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.832+0000 7f5358909700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.832+0000 7f5358909700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.832+0000 7f5358909700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.832+0000 7f5358909700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:37.845 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:37.832+0000 7f5358909700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:38.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.644+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 31) v1 ==== 79814+0+0 (secure 0 0 0) 0x7feaa004f300 con 0x7feaa8169ee0 2023-10-06T11:33:38.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.648+0000 7fea9d7fa700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.648+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7feaa816b6b0 con 0x7fea8c0606f0 2023-10-06T11:33:38.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.652+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.656+0000 7feaad414700 1 --2- 172.21.15.46:0/3669050891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fea8c0606f0 0x7fea8c062b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feaa8166900 tx=0x7feaa4003250 comp rx=0 tx=0).ready entity=mgr.6546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:38.678 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:38.664+0000 7f536291d700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T11:33:38.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.664+0000 7fea9d7fa700 1 -- 172.21.15.46:0/3669050891 <== mgr.6546 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7feaa816b6b0 con 0x7fea8c0606f0 2023-10-06T11:33:38.681 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T11:33:38.688 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T11:33:38.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:38.689 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T11:33:38.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:33:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.912+0000 7f6ba9585700 1 Processor -- start 2023-10-06T11:33:38.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.912+0000 7f6ba9585700 1 -- start start 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba9585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba40b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba9585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba40ccc10 con 0x7f6ba4156de0 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba9585700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba40cc000 con 0x7f6ba41571b0 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba9585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40cc2e0 con 0x7f6ba40b3400 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba2ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ba4156de0 0x7f6ba40cb7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33302/0 (socket says 172.21.15.46:33302) 2023-10-06T11:33:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba2ffd700 1 -- 172.21.15.46:0/4238561467 learned_addr learned my addr 172.21.15.46:0/4238561467 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba37fe700 1 --2- 172.21.15.46:0/4238561467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba40b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba27fc700 1 -- 172.21.15.46:0/4238561467 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091516077 0 0) 0x7f6ba40ccc10 con 0x7f6ba4156de0 2023-10-06T11:33:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba27fc700 1 -- 172.21.15.46:0/4238561467 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b90003570 con 0x7f6ba4156de0 2023-10-06T11:33:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba37fe700 1 -- 172.21.15.46:0/4238561467 >> v1:172.21.15.132:6789/0 conn(0x7f6ba41571b0 legacy=0x7f6ba40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba37fe700 1 -- 172.21.15.46:0/4238561467 >> v1:172.21.15.46:6789/0 conn(0x7f6ba4156de0 legacy=0x7f6ba40cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.916+0000 7f6ba37fe700 1 -- 172.21.15.46:0/4238561467 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40cc5c0 con 0x7f6ba40b3400 2023-10-06T11:33:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba37fe700 1 --2- 172.21.15.46:0/4238561467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba40b1cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e90 tx=0x7f6b9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4dc9ed30d25ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba27fc700 1 -- 172.21.15.46:0/4238561467 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00e070 con 0x7f6ba40b3400 2023-10-06T11:33:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba27fc700 1 -- 172.21.15.46:0/4238561467 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c00bc30 con 0x7f6ba40b3400 2023-10-06T11:33:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba27fc700 1 -- 172.21.15.46:0/4238561467 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c012430 con 0x7f6ba40b3400 2023-10-06T11:33:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- 172.21.15.46:0/4238561467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 msgr2=0x7f6ba40b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 --2- 172.21.15.46:0/4238561467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba40b1cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009e90 tx=0x7f6b9c009780 comp rx=0 tx=0).stop 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- 172.21.15.46:0/4238561467 shutdown_connections 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 --2- 172.21.15.46:0/4238561467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba40b1cb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- 172.21.15.46:0/4238561467 >> 172.21.15.46:0/4238561467 conn(0x7f6ba406af10 msgr2=0x7f6ba406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- 172.21.15.46:0/4238561467 shutdown_connections 2023-10-06T11:33:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- 172.21.15.46:0/4238561467 wait complete. 2023-10-06T11:33:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 Processor -- start 2023-10-06T11:33:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.920+0000 7f6ba9585700 1 -- start start 2023-10-06T11:33:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 0x7f6ba40af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4150f30 con 0x7f6ba4156de0 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40ca760 con 0x7f6ba41571b0 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4150320 con 0x7f6ba40b3400 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45736/0 (socket says 172.21.15.46:45736) 2023-10-06T11:33:38.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 -- 172.21.15.46:0/1249637634 learned_addr learned my addr 172.21.15.46:0/1249637634 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba37fe700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 0x7f6ba40af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba2ffd700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 -- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 msgr2=0x7f6ba4157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba4157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 -- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 msgr2=0x7f6ba40af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 0x7f6ba40af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 -- 172.21.15.46:0/1249637634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009a20 con 0x7f6ba4156de0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba2ffd700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba37fe700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 0x7f6ba40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9bfff700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c510 tx=0x7f6b8c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011590 con 0x7f6ba4156de0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c011710 con 0x7f6ba4156de0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba4150600 con 0x7f6ba4156de0 2023-10-06T11:33:38.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c011ad0 con 0x7f6ba4156de0 2023-10-06T11:33:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.924+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba4150bc0 con 0x7f6ba4156de0 2023-10-06T11:33:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.928+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba406bb80 con 0x7f6ba4156de0 2023-10-06T11:33:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.928+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f6b8c01b3b0 con 0x7f6ba4156de0 2023-10-06T11:33:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.932+0000 7f6b9affd700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 0x7f6b840628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.932+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f6b8c07bf00 con 0x7f6ba4156de0 2023-10-06T11:33:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.932+0000 7f6ba2ffd700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 0x7f6b840628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:38.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.932+0000 7f6ba2ffd700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 0x7f6b840628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b94007fd0 tx=0x7f6b94009040 comp rx=0 tx=0).ready entity=mgr.6546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:38.936+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ba406bb80 con 0x7f6ba4156de0 2023-10-06T11:33:39.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.144+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6ba406bb80 con 0x7f6ba4156de0 2023-10-06T11:33:39.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.664+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f6b8c04c700 con 0x7f6ba4156de0 2023-10-06T11:33:39.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.664+0000 7f6b9affd700 1 -- 172.21.15.46:0/1249637634 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v32) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6b8c046e00 con 0x7f6ba4156de0 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 msgr2=0x7f6b840628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 0x7f6b840628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b94007fd0 tx=0x7f6b94009040 comp rx=0 tx=0).stop 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 msgr2=0x7f6ba414c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00c510 tx=0x7f6b8c00c8d0 comp rx=0 tx=0).stop 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 shutdown_connections 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b840603f0 0x7f6b840628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:39.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba41571b0 0x7f6ba40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba4156de0 0x7f6ba414c060 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 --2- 172.21.15.46:0/1249637634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba40b3400 0x7f6ba4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 >> 172.21.15.46:0/1249637634 conn(0x7f6ba406af10 msgr2=0x7f6ba414dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 shutdown_connections 2023-10-06T11:33:39.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:39.668+0000 7f6ba9585700 1 -- 172.21.15.46:0/1249637634 wait complete. 2023-10-06T11:33:39.702 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:33:39.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:39.741 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:39.776 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:33:39.777 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:33:39.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:33:39.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:33:39.856 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:39.854+0000 7f2fd4764000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:39.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:33:39.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:33:39.876 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:33:39.883 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:39.864+0000 7fce8d523000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:33:39.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.084+0000 7fc6c7b85700 1 Processor -- start 2023-10-06T11:33:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.084+0000 7fc6c7b85700 1 -- start start 2023-10-06T11:33:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6c0161470 0x7fc6c0161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00c18f0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cb120 con 0x7fc6c00c1d80 2023-10-06T11:33:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00c2660 con 0x7fc6c0161470 2023-10-06T11:33:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c6122700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6c00c21e0 0x7fc6c00caa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33314/0 (socket says 172.21.15.46:33314) 2023-10-06T11:33:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c6122700 1 -- 172.21.15.46:0/604771064 learned_addr learned my addr 172.21.15.46:0/604771064 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c5921700 1 --2- 172.21.15.46:0/604771064 >> v2:172.21.15.132:3300/0 conn(0x7fc6c0161470 0x7fc6c0161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c491f700 1 -- 172.21.15.46:0/604771064 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766971028 0 0) 0x7fc6c00c18f0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c5921700 1 -- 172.21.15.46:0/604771064 >> v1:172.21.15.132:6789/0 conn(0x7fc6c00c1d80 legacy=0x7fc6c006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c5921700 1 -- 172.21.15.46:0/604771064 >> v1:172.21.15.46:6789/0 conn(0x7fc6c00c21e0 legacy=0x7fc6c00caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c5921700 1 -- 172.21.15.46:0/604771064 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00c0ce0 con 0x7fc6c0161470 2023-10-06T11:33:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c5921700 1 --2- 172.21.15.46:0/604771064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c0161470 0x7fc6c0161840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0098d0 tx=0x7fc6bc009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5f2f1b7ea2118c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c491f700 1 -- 172.21.15.46:0/604771064 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc009e70 con 0x7fc6c0161470 2023-10-06T11:33:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c491f700 1 -- 172.21.15.46:0/604771064 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc005c10 con 0x7fc6c0161470 2023-10-06T11:33:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c491f700 1 -- 172.21.15.46:0/604771064 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00b6e0 con 0x7fc6c0161470 2023-10-06T11:33:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- 172.21.15.46:0/604771064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c0161470 msgr2=0x7fc6c0161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/604771064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c0161470 0x7fc6c0161840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0098d0 tx=0x7fc6bc009120 comp rx=0 tx=0).stop 2023-10-06T11:33:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- 172.21.15.46:0/604771064 shutdown_connections 2023-10-06T11:33:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/604771064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c0161470 0x7fc6c0161840 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- 172.21.15.46:0/604771064 >> 172.21.15.46:0/604771064 conn(0x7fc6c00cbe30 msgr2=0x7fc6c00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.088+0000 7fc6c7b85700 1 -- 172.21.15.46:0/604771064 shutdown_connections 2023-10-06T11:33:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 -- 172.21.15.46:0/604771064 wait complete. 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 Processor -- start 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 -- start start 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c00c1d80 0x7fc6c016ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6c0161470 0x7fc6c016e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c016e870 con 0x7fc6c00c21e0 2023-10-06T11:33:40.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00afbc0 con 0x7fc6c00c1d80 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.092+0000 7fc6c7b85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00aefb0 con 0x7fc6c0161470 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45752/0 (socket says 172.21.15.46:45752) 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 -- 172.21.15.46:0/1814803802 learned_addr learned my addr 172.21.15.46:0/1814803802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5921700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c00c1d80 0x7fc6c016ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c6122700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6c0161470 0x7fc6c016e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6c0161470 msgr2=0x7fc6c016e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6c0161470 0x7fc6c016e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c00c1d80 msgr2=0x7fc6c016ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c00c1d80 0x7fc6c016ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 -- 172.21.15.46:0/1814803802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0093c0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c5120700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c590 tx=0x7fc6b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0011590 con 0x7fc6c00c21e0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0011710 con 0x7fc6c00c21e0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0011ad0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c00af290 con 0x7fc6c00c21e0 2023-10-06T11:33:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.096+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c00af850 con 0x7fc6c00c21e0 2023-10-06T11:33:40.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0063f90 con 0x7fc6c00c21e0 2023-10-06T11:33:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc6b001b3b0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6b5ffb700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 0x7fc6a4062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc6b007c460 con 0x7fc6c00c21e0 2023-10-06T11:33:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6c5921700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 msgr2=0x7fc6a4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.100+0000 7fc6c5921700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 0x7fc6a4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:40.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.108+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc6b004c2c0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.304+0000 7fc6c5921700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 msgr2=0x7fc6a4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.304+0000 7fc6c5921700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 0x7fc6a4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.312+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c00afd00 con 0x7fc6c00c21e0 2023-10-06T11:33:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.316+0000 7fc6b5ffb700 1 -- 172.21.15.46:0/1814803802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fc6b004f8e0 con 0x7fc6c00c21e0 2023-10-06T11:33:40.330 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:40.331 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 msgr2=0x7fc6a4062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 0x7fc6a4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 msgr2=0x7fc6c0170810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000c590 tx=0x7fc6b000c8a0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 shutdown_connections 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc6a4060700 0x7fc6a4062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6c0161470 0x7fc6c016e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c00c21e0 0x7fc6c0170810 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 --2- 172.21.15.46:0/1814803802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6c00c1d80 0x7fc6c016ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 >> 172.21.15.46:0/1814803802 conn(0x7fc6c00cbe30 msgr2=0x7fc6c006a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:40.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 shutdown_connections 2023-10-06T11:33:40.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.324+0000 7fc6c7b85700 1 -- 172.21.15.46:0/1814803802 wait complete. 2023-10-06T11:33:40.337 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:40.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:40.388 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:40.386+0000 7f2fd4764000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:40.442 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:40.428+0000 7fce8d523000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:33:40.457 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:40.454+0000 7f2fd4764000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.500+0000 7f82faa8e700 1 Processor -- start 2023-10-06T11:33:40.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.500+0000 7f82faa8e700 1 -- start start 2023-10-06T11:33:40.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82faa8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82faa8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82f40b1130 0x7f82f40ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82faa8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f40ae340 con 0x7f82f4159290 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82faa8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40b15b0 con 0x7f82f40b1130 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82faa8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40b09a0 con 0x7f82f40d07b0 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82f40b1130 0x7f82f40ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f82f40b1130 0x7f82f40ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45754/0 (socket says 172.21.15.46:45754) 2023-10-06T11:33:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 -- 172.21.15.46:0/2961399468 learned_addr learned my addr 172.21.15.46:0/2961399468 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:40.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82f928b700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 -- 172.21.15.46:0/2961399468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 msgr2=0x7f82f40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 -- 172.21.15.46:0/2961399468 >> v1:172.21.15.46:6789/0 conn(0x7f82f4159290 legacy=0x7f82f40b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 -- 172.21.15.46:0/2961399468 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4159660 con 0x7f82f40b1130 2023-10-06T11:33:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82fa28d700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f40ae7d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009f20 tx=0x7f82ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37453cb96817361d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.520 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:40.504+0000 7fce8d523000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82f8a8a700 1 -- 172.21.15.46:0/2961399468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00e070 con 0x7f82f40b1130 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82f8a8a700 1 -- 172.21.15.46:0/2961399468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec00bb10 con 0x7f82f40b1130 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.504+0000 7f82f8a8a700 1 -- 172.21.15.46:0/2961399468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0092e0 con 0x7f82f40b1130 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- 172.21.15.46:0/2961399468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 msgr2=0x7f82f40ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f40ae7d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009f20 tx=0x7f82ec009840 comp rx=0 tx=0).stop 2023-10-06T11:33:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- 172.21.15.46:0/2961399468 shutdown_connections 2023-10-06T11:33:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f40ae7d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2961399468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- 172.21.15.46:0/2961399468 >> 172.21.15.46:0/2961399468 conn(0x7f82f406ef60 msgr2=0x7f82f40d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- 172.21.15.46:0/2961399468 shutdown_connections 2023-10-06T11:33:40.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- 172.21.15.46:0/2961399468 wait complete. 2023-10-06T11:33:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 Processor -- start 2023-10-06T11:33:40.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.508+0000 7f82faa8e700 1 -- start start 2023-10-06T11:33:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f4149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 0x7f82f40ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f414ac60 con 0x7f82f40b1130 2023-10-06T11:33:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f414dac0 con 0x7f82f4159290 2023-10-06T11:33:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4156d80 con 0x7f82f40d07b0 2023-10-06T11:33:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45768/0 (socket says 172.21.15.46:45768) 2023-10-06T11:33:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 -- 172.21.15.46:0/2670662639 learned_addr learned my addr 172.21.15.46:0/2670662639 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f928b700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f4149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82fa28d700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 0x7f82f40ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 -- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 msgr2=0x7f82f4149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f4149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 -- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 msgr2=0x7f82f40ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 0x7f82f40ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 -- 172.21.15.46:0/2670662639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec009ae0 con 0x7f82f40b1130 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82fa28d700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 0x7f82f40ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82f9a8c700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c510 tx=0x7f82f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0011590 con 0x7f82f40b1130 2023-10-06T11:33:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0011710 con 0x7f82f40b1130 2023-10-06T11:33:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f414d500 con 0x7f82f40b1130 2023-10-06T11:33:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0011ad0 con 0x7f82f40b1130 2023-10-06T11:33:40.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.512+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f414a8f0 con 0x7f82f40b1130 2023-10-06T11:33:40.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.516+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f4064590 con 0x7f82f40b1130 2023-10-06T11:33:40.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.516+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f82f001b3b0 con 0x7f82f40b1130 2023-10-06T11:33:40.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.520+0000 7f82e9ffb700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 0x7f82d8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.520+0000 7f82f928b700 1 -- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 msgr2=0x7f82d8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:40.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.520+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f82f007c400 con 0x7f82f40b1130 2023-10-06T11:33:40.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.520+0000 7f82f928b700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 0x7f82d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:40.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.524+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f82f004c260 con 0x7f82f40b1130 2023-10-06T11:33:40.590 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:40.586+0000 7f2fd4764000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:40.667 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:40.652+0000 7fce8d523000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:40.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.704+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f40b06c0 con 0x7f82f40b1130 2023-10-06T11:33:40.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.704+0000 7f82e9ffb700 1 -- 172.21.15.46:0/2670662639 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18026 (secure 0 0 0) 0x7f82f004f880 con 0x7f82f40b1130 2023-10-06T11:33:40.718 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:40.719 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:37.626871+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/38534374":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/382630866":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6832/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/656656827":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6833/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2119803579":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2042618050":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/3295527092":"2023-10-07T11:33:37.626781+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 msgr2=0x7f82d8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 0x7f82d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 msgr2=0x7f82f4156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c510 tx=0x7f82f000c820 comp rx=0 tx=0).stop 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 shutdown_connections 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f82d8060700 0x7f82d8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4159290 0x7f82f40ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82f40d07b0 0x7f82f4149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 --2- 172.21.15.46:0/2670662639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f40b1130 0x7f82f4156840 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 >> 172.21.15.46:0/2670662639 conn(0x7f82f406ef60 msgr2=0x7f82f40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 shutdown_connections 2023-10-06T11:33:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.708+0000 7f82faa8e700 1 -- 172.21.15.46:0/2670662639 wait complete. 2023-10-06T11:33:40.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:33:20.937889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:33:22.026201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:33:40.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:33:40.841 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:40.838+0000 7f2fd4764000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.884+0000 7f148efcc700 1 Processor -- start 2023-10-06T11:33:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.884+0000 7f148efcc700 1 -- start start 2023-10-06T11:33:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148efcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14880ac2d0 0x7f14880ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148efcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 0x7f14880965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148efcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1488068270 con 0x7f14880a6e20 2023-10-06T11:33:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148efcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880cc7d0 con 0x7f14880ac2d0 2023-10-06T11:33:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148efcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880684c0 con 0x7f1488096140 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14880ac2d0 0x7f14880ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148e7cb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 0x7f14880965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148d7c9700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f14880a6e20 0x7f14880ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34770/0 (socket says 172.21.15.46:34770) 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148d7c9700 1 -- 172.21.15.46:0/2519476296 learned_addr learned my addr 172.21.15.46:0/2519476296 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148cfc8700 1 -- 172.21.15.46:0/2519476296 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773236011 0 0) 0x7f1488068270 con 0x7f14880a6e20 2023-10-06T11:33:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148cfc8700 1 -- 172.21.15.46:0/2519476296 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1474003570 con 0x7f14880a6e20 2023-10-06T11:33:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 -- 172.21.15.46:0/2519476296 >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 msgr2=0x7f14880965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 --2- 172.21.15.46:0/2519476296 >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 0x7f14880965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 -- 172.21.15.46:0/2519476296 >> v1:172.21.15.132:6789/0 conn(0x7f14880a6e20 legacy=0x7f14880ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 -- 172.21.15.46:0/2519476296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880687a0 con 0x7f14880ac2d0 2023-10-06T11:33:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148e7cb700 1 --2- 172.21.15.46:0/2519476296 >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 0x7f14880965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:40.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148dfca700 1 --2- 172.21.15.46:0/2519476296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880ac2d0 0x7f14880ac6a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f14840098e0 tx=0x7f14840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6986a099e51aef01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.888+0000 7f148cfc8700 1 -- 172.21.15.46:0/2519476296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484011630 con 0x7f14880ac2d0 2023-10-06T11:33:40.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148cfc8700 1 -- 172.21.15.46:0/2519476296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484011790 con 0x7f14880ac2d0 2023-10-06T11:33:40.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148cfc8700 1 -- 172.21.15.46:0/2519476296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484011b30 con 0x7f14880ac2d0 2023-10-06T11:33:40.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 -- 172.21.15.46:0/2519476296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880ac2d0 msgr2=0x7f14880ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 --2- 172.21.15.46:0/2519476296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880ac2d0 0x7f14880ac6a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f14840098e0 tx=0x7f14840091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 -- 172.21.15.46:0/2519476296 shutdown_connections 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 --2- 172.21.15.46:0/2519476296 >> v2:172.21.15.132:3300/0 conn(0x7f1488096140 0x7f14880965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 --2- 172.21.15.46:0/2519476296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880ac2d0 0x7f14880ac6a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 -- 172.21.15.46:0/2519476296 >> 172.21.15.46:0/2519476296 conn(0x7f14880afd10 msgr2=0x7f148806e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 -- 172.21.15.46:0/2519476296 shutdown_connections 2023-10-06T11:33:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.892+0000 7f148efcc700 1 -- 172.21.15.46:0/2519476296 wait complete. 2023-10-06T11:33:40.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 Processor -- start 2023-10-06T11:33:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 -- start start 2023-10-06T11:33:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1488096140 0x7f1488155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14880ac2d0 0x7f14880aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880a92b0 con 0x7f14880a6e20 2023-10-06T11:33:40.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880a9740 con 0x7f14880ac2d0 2023-10-06T11:33:40.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880a86e0 con 0x7f1488096140 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45782/0 (socket says 172.21.15.46:45782) 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 -- 172.21.15.46:0/2341123036 learned_addr learned my addr 172.21.15.46:0/2341123036 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148e7cb700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14880ac2d0 0x7f14880aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148dfca700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1488096140 0x7f1488155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:40.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 -- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1488096140 msgr2=0x7f1488155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1488096140 0x7f1488155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 -- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14880ac2d0 msgr2=0x7f14880aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14880ac2d0 0x7f14880aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 -- 172.21.15.46:0/2341123036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148000ab30 con 0x7f14880a6e20 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148d7c9700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f147800c7c0 tx=0x7f1478007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478005c00 con 0x7f14880a6e20 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478005d60 con 0x7f14880a6e20 2023-10-06T11:33:40.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1484009420 con 0x7f14880a6e20 2023-10-06T11:33:40.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.896+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478007320 con 0x7f14880a6e20 2023-10-06T11:33:40.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.900+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14880a8c60 con 0x7f14880a6e20 2023-10-06T11:33:40.915 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:40.910+0000 7f2fd4764000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:40.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.900+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14880689e0 con 0x7f14880a6e20 2023-10-06T11:33:40.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.900+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f147801b3b0 con 0x7f14880a6e20 2023-10-06T11:33:40.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.904+0000 7f147dffb700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 0x7f146c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:40.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.904+0000 7f148dfca700 1 -- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 msgr2=0x7f146c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.904+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f147807c4c0 con 0x7f14880a6e20 2023-10-06T11:33:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.904+0000 7f148dfca700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 0x7f146c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:40.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:40.908+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f147804c320 con 0x7f14880a6e20 2023-10-06T11:33:40.948 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:40.936+0000 7fce8d523000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:33:41.023 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:41.008+0000 7fce8d523000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.076+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1488064590 con 0x7f14880a6e20 2023-10-06T11:33:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.076+0000 7f147dffb700 1 -- 172.21.15.46:0/2341123036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f147804f940 con 0x7f14880a6e20 2023-10-06T11:33:41.091 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:33:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 msgr2=0x7f146c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 0x7f146c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 msgr2=0x7f1488099b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f147800c7c0 tx=0x7f1478007dd0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 shutdown_connections 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f146c060700 0x7f146c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14880ac2d0 0x7f14880aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14880a6e20 0x7f1488099b20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 --2- 172.21.15.46:0/2341123036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1488096140 0x7f1488155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 >> 172.21.15.46:0/2341123036 conn(0x7f14880afd10 msgr2=0x7f148806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 shutdown_connections 2023-10-06T11:33:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.080+0000 7f148efcc700 1 -- 172.21.15.46:0/2341123036 wait complete. 2023-10-06T11:33:41.110 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:41.106+0000 7f2fd4764000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:41.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:33:41.192 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:41.190+0000 7f2fd4764000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:41.237 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:41.224+0000 7fce8d523000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:33:41.271 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:41.270+0000 7f2fd4764000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.264+0000 7f2608985700 1 Processor -- start 2023-10-06T11:33:41.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.264+0000 7f2608985700 1 -- start start 2023-10-06T11:33:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2608985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26040b2790 0x7f26040ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2608985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 0x7f26040c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2608985700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26040cc510 con 0x7f26040b23c0 2023-10-06T11:33:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2608985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cd500 con 0x7f26040b2790 2023-10-06T11:33:41.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2608985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040b2be0 con 0x7f26040cd080 2023-10-06T11:33:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26040b23c0 0x7f26040cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33340/0 (socket says 172.21.15.46:33340) 2023-10-06T11:33:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260259c700 1 -- 172.21.15.46:0/1966108260 learned_addr learned my addr 172.21.15.46:0/1966108260 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1966108260 >> v2:172.21.15.132:3300/0 conn(0x7f26040b2790 0x7f26040ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2602d9d700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 0x7f26040c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260159a700 1 -- 172.21.15.46:0/1966108260 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847892643 0 0) 0x7f26040cc510 con 0x7f26040b23c0 2023-10-06T11:33:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260159a700 1 -- 172.21.15.46:0/1966108260 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec003570 con 0x7f26040b23c0 2023-10-06T11:33:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 -- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 msgr2=0x7f26040c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 0x7f26040c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 -- 172.21.15.46:0/1966108260 >> v1:172.21.15.46:6789/0 conn(0x7f26040b23c0 legacy=0x7f26040cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 -- 172.21.15.46:0/1966108260 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040cb900 con 0x7f26040b2790 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2602d9d700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 0x7f26040c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040b2790 0x7f26040ccb40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25f80098e0 tx=0x7f25f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b540e8409b9eeb41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260159a700 1 -- 172.21.15.46:0/1966108260 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8011630 con 0x7f26040b2790 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260159a700 1 -- 172.21.15.46:0/1966108260 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f8011790 con 0x7f26040b2790 2023-10-06T11:33:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.268+0000 7f260159a700 1 -- 172.21.15.46:0/1966108260 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8011b30 con 0x7f26040b2790 2023-10-06T11:33:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040b2790 msgr2=0x7f26040ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040b2790 0x7f26040ccb40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25f80098e0 tx=0x7f25f80091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- 172.21.15.46:0/1966108260 shutdown_connections 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040cd080 0x7f26040c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- 172.21.15.46:0/1966108260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040b2790 0x7f26040ccb40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- 172.21.15.46:0/1966108260 >> 172.21.15.46:0/1966108260 conn(0x7f2604099480 msgr2=0x7f260406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- 172.21.15.46:0/1966108260 shutdown_connections 2023-10-06T11:33:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- 172.21.15.46:0/1966108260 wait complete. 2023-10-06T11:33:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 Processor -- start 2023-10-06T11:33:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- start start 2023-10-06T11:33:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 0x7f260414ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040cd080 0x7f26040c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c8a00 con 0x7f26040b23c0 2023-10-06T11:33:41.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c8ce0 con 0x7f26040cd080 2023-10-06T11:33:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.272+0000 7f2608985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c8fc0 con 0x7f26040b2790 2023-10-06T11:33:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45788/0 (socket says 172.21.15.46:45788) 2023-10-06T11:33:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 -- 172.21.15.46:0/1624368479 learned_addr learned my addr 172.21.15.46:0/1624368479 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 0x7f260414ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2602d9d700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040cd080 0x7f26040c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 msgr2=0x7f260414ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 0x7f260414ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040cd080 msgr2=0x7f26040c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040cd080 0x7f26040c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 -- 172.21.15.46:0/1624368479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009a20 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 0x7f260414ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f260259c700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f25f4004b50 tx=0x7f25f4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4005c00 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4005d60 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f8009420 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40072e0 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040c9580 con 0x7f26040b23c0 2023-10-06T11:33:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.276+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26040c9770 con 0x7f26040b23c0 2023-10-06T11:33:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.280+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f25f4012400 con 0x7f26040b23c0 2023-10-06T11:33:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.280+0000 7f25f27fc700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 0x7f25e4062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.280+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f25f407c4a0 con 0x7f26040b23c0 2023-10-06T11:33:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.280+0000 7f2601d9b700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 msgr2=0x7f25e4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.280+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 0x7f25e4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.288+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f25f404c320 con 0x7f26040b23c0 2023-10-06T11:33:41.333 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:41.320+0000 7fce8d523000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:33:41.465 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:41.452+0000 7fce8d523000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:33:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.488+0000 7f2601d9b700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 msgr2=0x7f25e4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.488+0000 7f2601d9b700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 0x7f25e4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:41.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.604+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f26040660f0 con 0x7f26040b23c0 2023-10-06T11:33:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.604+0000 7f25f27fc700 1 -- 172.21.15.46:0/1624368479 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f25f404f940 con 0x7f26040b23c0 2023-10-06T11:33:41.617 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:33:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 msgr2=0x7f25e4062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 0x7f25e4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 msgr2=0x7f26040b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f25f4004b50 tx=0x7f25f4007dd0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 shutdown_connections 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f25e4060700 0x7f25e4062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040cd080 0x7f26040c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26040b2790 0x7f260414ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 --2- 172.21.15.46:0/1624368479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26040b23c0 0x7f26040b5a40 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 >> 172.21.15.46:0/1624368479 conn(0x7f2604099480 msgr2=0x7f2604156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 shutdown_connections 2023-10-06T11:33:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.608+0000 7f2608985700 1 -- 172.21.15.46:0/1624368479 wait complete. 2023-10-06T11:33:41.633 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:41.630+0000 7f2fd4764000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:41.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:33:41.784 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:41.782+0000 7f2fd4764000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.796+0000 7f9c4084c700 1 Processor -- start 2023-10-06T11:33:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.796+0000 7f9c4084c700 1 -- start start 2023-10-06T11:33:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c4084c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c4084c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0b3570 0x7f9c3c0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c4084c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0afd20 con 0x7f9c3c0adbe0 2023-10-06T11:33:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c4084c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c159740 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c4084c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b0000 con 0x7f9c3c0b3570 2023-10-06T11:33:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:42164/0 (socket says 172.21.15.46:42164) 2023-10-06T11:33:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 -- 172.21.15.46:0/375391010 learned_addr learned my addr 172.21.15.46:0/375391010 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3bfff700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0b3570 0x7f9c3c0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 -- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0b3570 msgr2=0x7f9c3c0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0b3570 0x7f9c3c0b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 -- 172.21.15.46:0/375391010 >> v1:172.21.15.132:6789/0 conn(0x7f9c3c0adbe0 legacy=0x7f9c3c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 -- 172.21.15.46:0/375391010 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0b02e0 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3affd700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c300099b0 tx=0x7f9c30009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e56dc4fd803e3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3a7fc700 1 -- 172.21.15.46:0/375391010 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000b860 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3a7fc700 1 -- 172.21.15.46:0/375391010 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000b9c0 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.800+0000 7f9c3a7fc700 1 -- 172.21.15.46:0/375391010 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000bd60 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 msgr2=0x7f9c3c0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9c300099b0 tx=0x7f9c30009250 comp rx=0 tx=0).stop 2023-10-06T11:33:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- 172.21.15.46:0/375391010 shutdown_connections 2023-10-06T11:33:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0b3570 0x7f9c3c0b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 --2- 172.21.15.46:0/375391010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c0b0dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- 172.21.15.46:0/375391010 >> 172.21.15.46:0/375391010 conn(0x7f9c3c06ef60 msgr2=0x7f9c3c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- 172.21.15.46:0/375391010 shutdown_connections 2023-10-06T11:33:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- 172.21.15.46:0/375391010 wait complete. 2023-10-06T11:33:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 Processor -- start 2023-10-06T11:33:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.804+0000 7f9c4084c700 1 -- start start 2023-10-06T11:33:41.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 0x7f9c3c099cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c09aec0 con 0x7f9c3c0b3570 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15bbb0 con 0x7f9c3c0b30f0 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c09a2b0 con 0x7f9c3c0adbe0 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45804/0 (socket says 172.21.15.46:45804) 2023-10-06T11:33:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 -- 172.21.15.46:0/1130729614 learned_addr learned my addr 172.21.15.46:0/1130729614 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:41.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3affd700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3b7fe700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 0x7f9c3c099cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:41.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 -- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 msgr2=0x7f9c3c099cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 0x7f9c3c099cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 -- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 msgr2=0x7f9c3c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 -- 172.21.15.46:0/1130729614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300094f0 con 0x7f9c3c0b3570 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3b7fe700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 0x7f9c3c099cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c3bfff700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009fa0 tx=0x7f9c34009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400ed20 con 0x7f9c3c0b3570 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400ee80 con 0x7f9c3c0b3570 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c09a590 con 0x7f9c3c0b3570 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34014670 con 0x7f9c3c0b3570 2023-10-06T11:33:41.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c09ab50 con 0x7f9c3c0b3570 2023-10-06T11:33:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.808+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c09b030 con 0x7f9c3c0b3570 2023-10-06T11:33:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.812+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f9c340149e0 con 0x7f9c3c0b3570 2023-10-06T11:33:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.812+0000 7f9c237fe700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 0x7f9c28062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.812+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f9c3407d7d0 con 0x7f9c3c0b3570 2023-10-06T11:33:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.812+0000 7f9c3b7fe700 1 -- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 msgr2=0x7f9c28062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.812+0000 7f9c3b7fe700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 0x7f9c28062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:41.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.820+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c3404d630 con 0x7f9c3c0b3570 2023-10-06T11:33:41.917 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:41.904+0000 7fce8d523000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:33:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.980+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c3c064590 con 0x7f9c3c0b3570 2023-10-06T11:33:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.984+0000 7f9c237fe700 1 -- 172.21.15.46:0/1130729614 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c34050c50 con 0x7f9c3c0b3570 2023-10-06T11:33:41.995 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 msgr2=0x7f9c28062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 0x7f9c28062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 msgr2=0x7f9c3c155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009fa0 tx=0x7f9c34009900 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 shutdown_connections 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c28060700 0x7f9c28062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b3570 0x7f9c3c155ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c3c0b30f0 0x7f9c3c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 --2- 172.21.15.46:0/1130729614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c3c0adbe0 0x7f9c3c099cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 >> 172.21.15.46:0/1130729614 conn(0x7f9c3c06ef60 msgr2=0x7f9c3c0aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 shutdown_connections 2023-10-06T11:33:42.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:41.988+0000 7f9c4084c700 1 -- 172.21.15.46:0/1130729614 wait complete. 2023-10-06T11:33:42.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:33:42.080 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.068+0000 7fce8d523000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:33:42.118 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.118+0000 7f2fd4764000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:42.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.164+0000 7f7f19a0c700 1 Processor -- start 2023-10-06T11:33:42.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.164+0000 7f7f19a0c700 1 -- start start 2023-10-06T11:33:42.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f19a0c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f140adbe0 0x7f7f140d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f19a0c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f140afd20 con 0x7f7f140d0d60 2023-10-06T11:33:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f19a0c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f140b0000 con 0x7f7f140b0dc0 2023-10-06T11:33:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f19a0c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f14159740 con 0x7f7f140adbe0 2023-10-06T11:33:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f13fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f140d0d60 0x7f7f140b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33348/0 (socket says 172.21.15.46:33348) 2023-10-06T11:33:42.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f13fff700 1 -- 172.21.15.46:0/1409045742 learned_addr learned my addr 172.21.15.46:0/1409045742 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:42.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f18a0a700 1 --2- 172.21.15.46:0/1409045742 >> v2:172.21.15.132:3300/0 conn(0x7f7f140adbe0 0x7f7f140d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786899734 0 0) 0x7f7f140b0000 con 0x7f7f140b0dc0 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f04003570 con 0x7f7f140b0dc0 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987343918 0 0) 0x7f7f140afd20 con 0x7f7f140d0d60 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f140b0000 con 0x7f7f140d0d60 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f18a0a700 1 -- 172.21.15.46:0/1409045742 >> v1:172.21.15.132:6789/0 conn(0x7f7f140b0dc0 legacy=0x7f7f140b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f18a0a700 1 -- 172.21.15.46:0/1409045742 >> v1:172.21.15.46:6789/0 conn(0x7f7f140d0d60 legacy=0x7f7f140b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3834269699 0 0) 0x7f7f140b0000 con 0x7f7f140d0d60 2023-10-06T11:33:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.168+0000 7f7f18a0a700 1 -- 172.21.15.46:0/1409045742 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140b02e0 con 0x7f7f140adbe0 2023-10-06T11:33:42.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f18a0a700 1 --2- 172.21.15.46:0/1409045742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140adbe0 0x7f7f140d0820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f080098e0 tx=0x7f7f080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628b03979f073d9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00001470 con 0x7f7f140adbe0 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08011420 con 0x7f7f140adbe0 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f137fe700 1 -- 172.21.15.46:0/1409045742 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080117c0 con 0x7f7f140adbe0 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 -- 172.21.15.46:0/1409045742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140adbe0 msgr2=0x7f7f140d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/1409045742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140adbe0 0x7f7f140d0820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7f080098e0 tx=0x7f7f080091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 -- 172.21.15.46:0/1409045742 shutdown_connections 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/1409045742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140adbe0 0x7f7f140d0820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 -- 172.21.15.46:0/1409045742 >> 172.21.15.46:0/1409045742 conn(0x7f7f1406ef60 msgr2=0x7f7f140d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 -- 172.21.15.46:0/1409045742 shutdown_connections 2023-10-06T11:33:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.172+0000 7f7f19a0c700 1 -- 172.21.15.46:0/1409045742 wait complete. 2023-10-06T11:33:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 Processor -- start 2023-10-06T11:33:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 -- start start 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 0x7f7f14158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 0x7f7f14158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141509f0 con 0x7f7f140adbe0 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1414fd70 con 0x7f7f140b0dc0 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14150050 con 0x7f7f140d0d60 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45814/0 (socket says 172.21.15.46:45814) 2023-10-06T11:33:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 -- 172.21.15.46:0/2872941702 learned_addr learned my addr 172.21.15.46:0/2872941702 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 0x7f7f14158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f1920b700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 0x7f7f14158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 -- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 msgr2=0x7f7f14158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 0x7f7f14158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 -- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 msgr2=0x7f7f14158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 0x7f7f14158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 -- 172.21.15.46:0/2872941702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08009420 con 0x7f7f140adbe0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 0x7f7f14158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f1920b700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 0x7f7f14158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f18a0a700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7f14088270 tx=0x7f7f08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0801cc90 con 0x7f7f140adbe0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0801cdf0 con 0x7f7f140adbe0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14150330 con 0x7f7f140adbe0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.176+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0801a6a0 con 0x7f7f140adbe0 2023-10-06T11:33:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.180+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14140f50 con 0x7f7f140adbe0 2023-10-06T11:33:42.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.180+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14064590 con 0x7f7f140adbe0 2023-10-06T11:33:42.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.180+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f7f0801aa50 con 0x7f7f140adbe0 2023-10-06T11:33:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.180+0000 7f7f10ff9700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 0x7f7ef8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.180+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f7f0807e980 con 0x7f7f140adbe0 2023-10-06T11:33:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.184+0000 7f7f13fff700 1 -- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 msgr2=0x7f7ef8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:42.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.184+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 0x7f7ef8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.188+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f0804e7e0 con 0x7f7f140adbe0 2023-10-06T11:33:42.326 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.322+0000 7f2fd4764000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f140af510 con 0x7f7f140adbe0 2023-10-06T11:33:42.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f10ff9700 1 -- 172.21.15.46:0/2872941702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f080290a0 con 0x7f7f140adbe0 2023-10-06T11:33:42.368 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 msgr2=0x7f7ef8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 0x7f7ef8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 msgr2=0x7f7f14140a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7f14088270 tx=0x7f7f08016000 comp rx=0 tx=0).stop 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 shutdown_connections 2023-10-06T11:33:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ef8060700 0x7f7ef8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f140d0d60 0x7f7f14158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140b0dc0 0x7f7f14158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 --2- 172.21.15.46:0/2872941702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140adbe0 0x7f7f14140a10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.356+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 >> 172.21.15.46:0/2872941702 conn(0x7f7f1406ef60 msgr2=0x7f7f140b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.360+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 shutdown_connections 2023-10-06T11:33:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.360+0000 7f7f19a0c700 1 -- 172.21.15.46:0/2872941702 wait complete. 2023-10-06T11:33:42.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:42.411 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.410+0000 7f2fd4764000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:42.440 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.428+0000 7fce8d523000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:33:42.484 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.482+0000 7f2fd4764000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:42.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.528+0000 7f56772ae700 1 Processor -- start 2023-10-06T11:33:42.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.528+0000 7f56772ae700 1 -- start start 2023-10-06T11:33:42.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f56772ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f567006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f56772ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567006d460 con 0x7f567015af30 2023-10-06T11:33:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f56772ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56700c7000 con 0x7f56700ad6c0 2023-10-06T11:33:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f56772ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c72c0 con 0x7f567015ab60 2023-10-06T11:33:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f5674849700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f567015af30 0x7f567006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33352/0 (socket says 172.21.15.46:33352) 2023-10-06T11:33:42.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f5674849700 1 -- 172.21.15.46:0/1748057291 learned_addr learned my addr 172.21.15.46:0/1748057291 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:42.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f567504a700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f567006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709542365 0 0) 0x7f567006d460 con 0x7f567015af30 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.532+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f565c003570 con 0x7f567015af30 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697524546 0 0) 0x7f56700c7000 con 0x7f56700ad6c0 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f567006d460 con 0x7f56700ad6c0 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2389576102 0 0) 0x7f565c003570 con 0x7f567015af30 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 msgr2=0x7f567006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f567006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 >> v1:172.21.15.132:6789/0 conn(0x7f56700ad6c0 legacy=0x7f56700adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700c75a0 con 0x7f567015af30 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f5660001490 con 0x7f567015af30 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 >> v1:172.21.15.46:6789/0 conn(0x7f567015af30 legacy=0x7f567006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c003c20 0x7f565c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f565c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565c004d50 0x7f565c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c72c0 con 0x7f565c004d50 2023-10-06T11:33:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c00bb40 con 0x7f565c003c20 2023-10-06T11:33:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c00bd00 con 0x7f567015ab60 2023-10-06T11:33:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565c004d50 0x7f565c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567504a700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c003c20 0x7f565c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5674849700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f565c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 -- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 msgr2=0x7f565c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567015ab60 0x7f565c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 -- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c003c20 msgr2=0x7f565c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c003c20 0x7f565c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 -- 172.21.15.46:0/1748057291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c003570 con 0x7f565c004d50 2023-10-06T11:33:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565c004d50 0x7f565c0051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5668002120 tx=0x7f56680017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52606639a42e5437 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668001ab0 con 0x7f565c004d50 2023-10-06T11:33:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566800c980 con 0x7f565c004d50 2023-10-06T11:33:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f5667fff700 1 -- 172.21.15.46:0/1748057291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566800cd20 con 0x7f565c004d50 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f56772ae700 1 -- 172.21.15.46:0/1748057291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565c004d50 msgr2=0x7f565c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f56772ae700 1 --2- 172.21.15.46:0/1748057291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f565c004d50 0x7f565c0051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5668002120 tx=0x7f56680017a0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f567584b700 1 -- 172.21.15.46:0/1748057291 reap_dead start 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f56772ae700 1 -- 172.21.15.46:0/1748057291 shutdown_connections 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.536+0000 7f56772ae700 1 -- 172.21.15.46:0/1748057291 >> 172.21.15.46:0/1748057291 conn(0x7f56700cbe30 msgr2=0x7f56700cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.540+0000 7f56772ae700 1 -- 172.21.15.46:0/1748057291 shutdown_connections 2023-10-06T11:33:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.540+0000 7f56772ae700 1 -- 172.21.15.46:0/1748057291 wait complete. 2023-10-06T11:33:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.540+0000 7f56772ae700 1 Processor -- start 2023-10-06T11:33:42.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.540+0000 7f56772ae700 1 -- start start 2023-10-06T11:33:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 0x7f56700c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 0x7f567016c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567016ceb0 0x7f5670169330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700b1ad0 con 0x7f56700cab80 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670166e30 con 0x7f565c004d50 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567016d320 con 0x7f567016ceb0 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f5674849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 0x7f567016c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f5674849700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 0x7f567016c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45822/0 (socket says 172.21.15.46:45822) 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f5674849700 1 -- 172.21.15.46:0/579333998 learned_addr learned my addr 172.21.15.46:0/579333998 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 0x7f56700c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567584b700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567016ceb0 0x7f5670169330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567016ceb0 msgr2=0x7f5670169330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567016ceb0 0x7f5670169330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 msgr2=0x7f567016c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 0x7f567016c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 -- 172.21.15.46:0/579333998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700b0680 con 0x7f565c004d50 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f567504a700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 0x7f56700c38f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f566c011b10 tx=0x7f566c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00f8e0 con 0x7f565c004d50 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00fa40 con 0x7f565c004d50 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56700c3e30 con 0x7f565c004d50 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00fde0 con 0x7f565c004d50 2023-10-06T11:33:42.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.544+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56700b1510 con 0x7f565c004d50 2023-10-06T11:33:42.560 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.558+0000 7f2fd4764000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670063f90 con 0x7f565c004d50 2023-10-06T11:33:42.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f566c014650 con 0x7f565c004d50 2023-10-06T11:33:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f56657fa700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 0x7f5654062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f5674849700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 msgr2=0x7f5654062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f566c018030 con 0x7f565c004d50 2023-10-06T11:33:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.548+0000 7f5674849700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 0x7f5654062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:42.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.556+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f566c04d7e0 con 0x7f565c004d50 2023-10-06T11:33:42.664 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.652+0000 7fce8d523000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:33:42.757 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.744+0000 7fce8d523000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:33:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.752+0000 7f5674849700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 msgr2=0x7f5654062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.752+0000 7f5674849700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 0x7f5654062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:42.775 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.774+0000 7f2fd4764000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:42.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.764+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f567004b5c0 con 0x7f565c004d50 2023-10-06T11:33:42.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56657fa700 1 -- 172.21.15.46:0/579333998 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f566c022090 con 0x7f565c004d50 2023-10-06T11:33:42.779 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:42.779 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 msgr2=0x7f5654062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 0x7f5654062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 msgr2=0x7f56700c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 0x7f56700c38f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f566c011b10 tx=0x7f566c009120 comp rx=0 tx=0).stop 2023-10-06T11:33:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 shutdown_connections 2023-10-06T11:33:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5654060700 0x7f5654062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567016ceb0 0x7f5670169330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56700cab80 0x7f567016c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 --2- 172.21.15.46:0/579333998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004d50 0x7f56700c38f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 >> 172.21.15.46:0/579333998 conn(0x7f56700cbe30 msgr2=0x7f56700c7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 shutdown_connections 2023-10-06T11:33:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.768+0000 7f56772ae700 1 -- 172.21.15.46:0/579333998 wait complete. 2023-10-06T11:33:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:42.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:42.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.820+0000 7fce8d523000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:33:42.848 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.846+0000 7f2fd4764000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:42.920 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:42.908+0000 7fce8d523000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:33:42.932 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:42.930+0000 7f2fd4764000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.972+0000 7f78fbd05700 1 Processor -- start 2023-10-06T11:33:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.972+0000 7f78fbd05700 1 -- start start 2023-10-06T11:33:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fbd05700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f40c7fc0 0x7f78f40c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fbd05700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f40c6470 con 0x7f78f4156c80 2023-10-06T11:33:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fbd05700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f40c7760 con 0x7f78f40c7b50 2023-10-06T11:33:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fbd05700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f40c66c0 con 0x7f78f40c7fc0 2023-10-06T11:33:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fa2a2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f40c7fc0 0x7f78f40c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f92a0700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f78f40c7b50 0x7f78f40c0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34790/0 (socket says 172.21.15.46:34790) 2023-10-06T11:33:42.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f92a0700 1 -- 172.21.15.46:0/1924402924 learned_addr learned my addr 172.21.15.46:0/1924402924 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249923580 0 0) 0x7f78f40c7760 con 0x7f78f40c7b50 2023-10-06T11:33:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e0003570 con 0x7f78f40c7b50 2023-10-06T11:33:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2019413777 0 0) 0x7f78e0003570 con 0x7f78f40c7b50 2023-10-06T11:33:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fa2a2700 1 -- 172.21.15.46:0/1924402924 >> v1:172.21.15.132:6789/0 conn(0x7f78f40c7b50 legacy=0x7f78f40c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fa2a2700 1 -- 172.21.15.46:0/1924402924 >> v1:172.21.15.46:6789/0 conn(0x7f78f4156c80 legacy=0x7f78f40ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fa2a2700 1 -- 172.21.15.46:0/1924402924 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40c69a0 con 0x7f78f40c7fc0 2023-10-06T11:33:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78fa2a2700 1 --2- 172.21.15.46:0/1924402924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7fc0 0x7f78f40c12e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009de0 tx=0x7f78ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d93774182e216f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00e070 con 0x7f78f40c7fc0 2023-10-06T11:33:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b8d0 con 0x7f78f40c7fc0 2023-10-06T11:33:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.976+0000 7f78f8a9f700 1 -- 172.21.15.46:0/1924402924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4001470 con 0x7f78f40c7fc0 2023-10-06T11:33:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 -- 172.21.15.46:0/1924402924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7fc0 msgr2=0x7f78f40c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 --2- 172.21.15.46:0/1924402924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7fc0 0x7f78f40c12e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009de0 tx=0x7f78ec009780 comp rx=0 tx=0).stop 2023-10-06T11:33:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 -- 172.21.15.46:0/1924402924 shutdown_connections 2023-10-06T11:33:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 --2- 172.21.15.46:0/1924402924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7fc0 0x7f78f40c12e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 -- 172.21.15.46:0/1924402924 >> 172.21.15.46:0/1924402924 conn(0x7f78f40cbe30 msgr2=0x7f78f40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 -- 172.21.15.46:0/1924402924 shutdown_connections 2023-10-06T11:33:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 -- 172.21.15.46:0/1924402924 wait complete. 2023-10-06T11:33:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.980+0000 7f78fbd05700 1 Processor -- start 2023-10-06T11:33:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 -- start start 2023-10-06T11:33:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78f40c7fc0 0x7f78f4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4156c80 0x7f78f4155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ae3d0 con 0x7f78f40c7b50 2023-10-06T11:33:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40af6a0 con 0x7f78f4156c80 2023-10-06T11:33:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f414ff20 con 0x7f78f40c7fc0 2023-10-06T11:33:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45826/0 (socket says 172.21.15.46:45826) 2023-10-06T11:33:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 -- 172.21.15.46:0/4103569385 learned_addr learned my addr 172.21.15.46:0/4103569385 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f92a0700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78f40c7fc0 0x7f78f4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fa2a2700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4156c80 0x7f78f4155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 -- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78f40c7fc0 msgr2=0x7f78f4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78f40c7fc0 0x7f78f4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 -- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4156c80 msgr2=0x7f78f4155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4156c80 0x7f78f4155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 -- 172.21.15.46:0/4103569385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009a20 con 0x7f78f40c7b50 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78f9aa1700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c540 tx=0x7f78f0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000f470 con 0x7f78f40c7b50 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000f5d0 con 0x7f78f40c7b50 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f40aed70 con 0x7f78f40c7b50 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.984+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000f990 con 0x7f78f40c7b50 2023-10-06T11:33:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.988+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f40af330 con 0x7f78f40c7b50 2023-10-06T11:33:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.988+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f4063f90 con 0x7f78f40c7b50 2023-10-06T11:33:43.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.988+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f78f000fd40 con 0x7f78f40c7b50 2023-10-06T11:33:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.988+0000 7f78e9ffb700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 0x7f78d8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.988+0000 7f78f92a0700 1 -- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 msgr2=0x7f78d8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.992+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f78f007c4f0 con 0x7f78f40c7b50 2023-10-06T11:33:43.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.992+0000 7f78f92a0700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 0x7f78d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:43.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:42.996+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f78f004c350 con 0x7f78f40c7b50 2023-10-06T11:33:43.113 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:43.110+0000 7f2fd4764000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:43.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:43.132+0000 7fce8d523000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:33:43.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.168+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78f41507c0 con 0x7f78f40c7b50 2023-10-06T11:33:43.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78e9ffb700 1 -- 172.21.15.46:0/4103569385 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18026 (secure 0 0 0) 0x7f78f004f970 con 0x7f78f40c7b50 2023-10-06T11:33:43.184 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:43.184 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:37.626871+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/38534374":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/382630866":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6832/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/656656827":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6833/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2119803579":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2042618050":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/3295527092":"2023-10-07T11:33:37.626781+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 msgr2=0x7f78d8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 0x7f78d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 msgr2=0x7f78f414f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c540 tx=0x7f78f0007f60 comp rx=0 tx=0).stop 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 shutdown_connections 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78d8060700 0x7f78d8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4156c80 0x7f78f4155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78f40c7fc0 0x7f78f4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 --2- 172.21.15.46:0/4103569385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f40c7b50 0x7f78f414f9e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 >> 172.21.15.46:0/4103569385 conn(0x7f78f40cbe30 msgr2=0x7f78f40c8770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:43.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.172+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 shutdown_connections 2023-10-06T11:33:43.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.176+0000 7f78fbd05700 1 -- 172.21.15.46:0/4103569385 wait complete. 2023-10-06T11:33:43.198 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:43.194+0000 7f2fd4764000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:43.208 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:33:43.208 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:43.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:43.208+0000 7fce8d523000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:33:43.312 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:43.300+0000 7fce8d523000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:33:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.348+0000 7f11d562e700 1 Processor -- start 2023-10-06T11:33:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 -- start start 2023-10-06T11:33:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155480 0x7f11d00c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11d0155850 0x7f11d00c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d006dd30 con 0x7f11d00c2300 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00caa10 con 0x7f11d0155850 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11d562e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d006ce80 con 0x7f11d0155480 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11cf7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11d00c2300 0x7f11d00c2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60758/0 (socket says 172.21.15.46:60758) 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11cf7fe700 1 -- 172.21.15.46:0/1794332849 learned_addr learned my addr 172.21.15.46:0/1794332849 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/1794332849 >> v2:172.21.15.46:3300/0 conn(0x7f11d0155850 0x7f11d00c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ceffd700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155480 0x7f11d00c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ce7fc700 1 -- 172.21.15.46:0/1794332849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155480 msgr2=0x7f11d00c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155480 0x7f11d00c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ce7fc700 1 -- 172.21.15.46:0/1794332849 >> v1:172.21.15.46:6789/0 conn(0x7f11d00c2300 legacy=0x7f11d00c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.352+0000 7f11ce7fc700 1 -- 172.21.15.46:0/1794332849 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d006d120 con 0x7f11d0155850 2023-10-06T11:33:43.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d0155850 0x7f11d00c19a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11c40098e0 tx=0x7f11c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de02ccab9b8e1eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11cdffb700 1 -- 172.21.15.46:0/1794332849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400b640 con 0x7f11d0155850 2023-10-06T11:33:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11cdffb700 1 -- 172.21.15.46:0/1794332849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400b7a0 con 0x7f11d0155850 2023-10-06T11:33:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11cdffb700 1 -- 172.21.15.46:0/1794332849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400bb40 con 0x7f11d0155850 2023-10-06T11:33:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- 172.21.15.46:0/1794332849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d0155850 msgr2=0x7f11d00c19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d0155850 0x7f11d00c19a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11c40098e0 tx=0x7f11c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- 172.21.15.46:0/1794332849 shutdown_connections 2023-10-06T11:33:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d0155850 0x7f11d00c19a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 --2- 172.21.15.46:0/1794332849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155480 0x7f11d00c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- 172.21.15.46:0/1794332849 >> 172.21.15.46:0/1794332849 conn(0x7f11d00cbe30 msgr2=0x7f11d00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:43.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- 172.21.15.46:0/1794332849 shutdown_connections 2023-10-06T11:33:43.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- 172.21.15.46:0/1794332849 wait complete. 2023-10-06T11:33:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 Processor -- start 2023-10-06T11:33:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.356+0000 7f11d562e700 1 -- start start 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11d0155480 0x7f11d0155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155850 0x7f11d0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00ac3e0 con 0x7f11d00c2300 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00ab7d0 con 0x7f11d0155480 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00abab0 con 0x7f11d0155850 2023-10-06T11:33:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40250/0 (socket says 172.21.15.46:40250) 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 -- 172.21.15.46:0/4226523621 learned_addr learned my addr 172.21.15.46:0/4226523621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11d0155480 0x7f11d0155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11cf7fe700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155850 0x7f11d0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155850 msgr2=0x7f11d0156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155850 0x7f11d0156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11d0155480 msgr2=0x7f11d0155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11d0155480 0x7f11d0155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 -- 172.21.15.46:0/4226523621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4009420 con 0x7f11d00c2300 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11ceffd700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ec80 tx=0x7f11c0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0009680 con 0x7f11d00c2300 2023-10-06T11:33:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c000c850 con 0x7f11d00c2300 2023-10-06T11:33:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d00abd90 con 0x7f11d00c2300 2023-10-06T11:33:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000cbd0 con 0x7f11d00c2300 2023-10-06T11:33:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.360+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d006a200 con 0x7f11d00c2300 2023-10-06T11:33:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.364+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d0063f90 con 0x7f11d00c2300 2023-10-06T11:33:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.368+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f11c0013470 con 0x7f11d00c2300 2023-10-06T11:33:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.368+0000 7f11b6ffd700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 0x7f11bc062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.368+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f11c007d430 con 0x7f11d00c2300 2023-10-06T11:33:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.368+0000 7f11ce7fc700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 msgr2=0x7f11bc062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.368+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 0x7f11bc062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.376+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11c004d330 con 0x7f11d00c2300 2023-10-06T11:33:43.508 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:43.496+0000 7fce8d523000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:33:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.572+0000 7f11ce7fc700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 msgr2=0x7f11bc062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.572+0000 7f11ce7fc700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 0x7f11bc062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d006d400 con 0x7f11d00c2300 2023-10-06T11:33:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11b6ffd700 1 -- 172.21.15.46:0/4226523621 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f11c0050950 con 0x7f11d00c2300 2023-10-06T11:33:43.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:43.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:43.593 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:43.580+0000 7fce8d523000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 msgr2=0x7f11bc062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 0x7f11bc062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 msgr2=0x7f11d014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f11c000ec80 tx=0x7f11c0009120 comp rx=0 tx=0).stop 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 shutdown_connections 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11bc060730 0x7f11bc062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11d0155850 0x7f11d0156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11d0155480 0x7f11d0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 --2- 172.21.15.46:0/4226523621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11d00c2300 0x7f11d014ea50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 >> 172.21.15.46:0/4226523621 conn(0x7f11d00cbe30 msgr2=0x7f11d00cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 shutdown_connections 2023-10-06T11:33:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.580+0000 7f11d562e700 1 -- 172.21.15.46:0/4226523621 wait complete. 2023-10-06T11:33:43.596 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:43.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.800+0000 7fc9964ae700 1 Processor -- start 2023-10-06T11:33:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.800+0000 7fc9964ae700 1 -- start start 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc99015ab60 0x7fc99006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc99015af30 0x7fc99006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99006d460 con 0x7fc9900ad6c0 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900c7000 con 0x7fc99015ab60 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900c72c0 con 0x7fc99015af30 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc994a4b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc9900ad6c0 0x7fc9900adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60766/0 (socket says 172.21.15.46:60766) 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc994a4b700 1 -- 172.21.15.46:0/3505398067 learned_addr learned my addr 172.21.15.46:0/3505398067 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3505398067 >> v2:172.21.15.46:3300/0 conn(0x7fc99015ab60 0x7fc99006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9877fe700 1 --2- 172.21.15.46:0/3505398067 >> v2:172.21.15.132:3300/0 conn(0x7fc99015af30 0x7fc99006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98f7fe700 1 -- 172.21.15.46:0/3505398067 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483446540 0 0) 0x7fc99006d460 con 0x7fc9900ad6c0 2023-10-06T11:33:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98f7fe700 1 -- 172.21.15.46:0/3505398067 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc97c003570 con 0x7fc9900ad6c0 2023-10-06T11:33:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 -- 172.21.15.46:0/3505398067 >> v2:172.21.15.132:3300/0 conn(0x7fc99015af30 msgr2=0x7fc99006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3505398067 >> v2:172.21.15.132:3300/0 conn(0x7fc99015af30 0x7fc99006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 -- 172.21.15.46:0/3505398067 >> v1:172.21.15.46:6789/0 conn(0x7fc9900ad6c0 legacy=0x7fc9900adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 -- 172.21.15.46:0/3505398067 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900c75a0 con 0x7fc99015ab60 2023-10-06T11:33:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3505398067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99015ab60 0x7fc99006e8a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc99006f0b0 tx=0x7fc9800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2402dcfa5fe87680 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98f7fe700 1 -- 172.21.15.46:0/3505398067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980011630 con 0x7fc99015ab60 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98f7fe700 1 -- 172.21.15.46:0/3505398067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980011790 con 0x7fc99015ab60 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc98f7fe700 1 -- 172.21.15.46:0/3505398067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980011b30 con 0x7fc99015ab60 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 -- 172.21.15.46:0/3505398067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99015ab60 msgr2=0x7fc99006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3505398067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99015ab60 0x7fc99006e8a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc99006f0b0 tx=0x7fc9800091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 -- 172.21.15.46:0/3505398067 shutdown_connections 2023-10-06T11:33:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.804+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3505398067 >> v2:172.21.15.132:3300/0 conn(0x7fc99015af30 0x7fc99006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3505398067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99015ab60 0x7fc99006e8a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- 172.21.15.46:0/3505398067 >> 172.21.15.46:0/3505398067 conn(0x7fc9900cbe30 msgr2=0x7fc9900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- 172.21.15.46:0/3505398067 shutdown_connections 2023-10-06T11:33:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- 172.21.15.46:0/3505398067 wait complete. 2023-10-06T11:33:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 Processor -- start 2023-10-06T11:33:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- start start 2023-10-06T11:33:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc99015ab60 0x7fc9900c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc99015af30 0x7fc9900afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc990141210 con 0x7fc9900ad6c0 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.808+0000 7fc9964ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901414f0 con 0x7fc99015ab60 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc9964ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901417d0 con 0x7fc99015af30 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40256/0 (socket says 172.21.15.46:40256) 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 -- 172.21.15.46:0/3897597745 learned_addr learned my addr 172.21.15.46:0/3897597745 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc9877fe700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc99015ab60 0x7fc9900c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc994a4b700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc99015af30 0x7fc9900afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:43.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc99015af30 msgr2=0x7fc9900afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc99015af30 0x7fc9900afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc99015ab60 msgr2=0x7fc9900c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc99015ab60 0x7fc9900c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 -- 172.21.15.46:0/3897597745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800093c0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98ffff700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc980009390 tx=0x7fc980017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980011b10 con 0x7fc9900ad6c0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980000cc0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800266a0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc990141ab0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9901422b0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.812+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc990141fb0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.816+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc9800303f0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.816+0000 7fc98cff9700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 0x7fc970062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.816+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc98007e980 con 0x7fc9900ad6c0 2023-10-06T11:33:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.816+0000 7fc9877fe700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 msgr2=0x7fc970062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.816+0000 7fc9877fe700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 0x7fc970062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:43.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:43.820+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc98004e7e0 con 0x7fc9900ad6c0 2023-10-06T11:33:43.905 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:33:43.902+0000 7f2fd4764000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.020+0000 7fc9877fe700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 msgr2=0x7fc970062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.020+0000 7fc9877fe700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 0x7fc970062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:33:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.212+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc990063f90 con 0x7fc9900ad6c0 2023-10-06T11:33:44.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.216+0000 7fc98cff9700 1 -- 172.21.15.46:0/3897597745 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fc980051e00 con 0x7fc9900ad6c0 2023-10-06T11:33:44.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:44.228 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:23.035201+0000","modified":"2023-10-06T11:33:26.627051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6357},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6357":{"gid":6357,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3200380112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3200380112},{"type":"v1","addr":"172.21.15.46:6836","nonce":3200380112}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 msgr2=0x7fc970062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 0x7fc970062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 msgr2=0x7fc9900c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc980009390 tx=0x7fc980017000 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 shutdown_connections 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc970060700 0x7fc970062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc99015af30 0x7fc9900afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc99015ab60 0x7fc9900c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 --2- 172.21.15.46:0/3897597745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9900ad6c0 0x7fc9900c0db0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 >> 172.21.15.46:0/3897597745 conn(0x7fc9900cbe30 msgr2=0x7fc99006a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:44.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 shutdown_connections 2023-10-06T11:33:44.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.220+0000 7fc9964ae700 1 -- 172.21.15.46:0/3897597745 wait complete. 2023-10-06T11:33:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2023-10-06T11:33:44.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.400+0000 7f2e3eaf0700 1 Processor -- start 2023-10-06T11:33:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.400+0000 7f2e3eaf0700 1 -- start start 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e38159050 0x7f2e38159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e380b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2e380b39b0 0x7f2e380b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e380b4f80 con 0x7f2e38159050 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e380b4370 con 0x7f2e380b39b0 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b4650 con 0x7f2e380d0aa0 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e38159050 0x7f2e38159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e38159050 0x7f2e38159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40264/0 (socket says 172.21.15.46:40264) 2023-10-06T11:33:44.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 -- 172.21.15.46:0/126741339 learned_addr learned my addr 172.21.15.46:0/126741339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3d08d700 1 --2- 172.21.15.46:0/126741339 >> v2:172.21.15.132:3300/0 conn(0x7f2e380b39b0 0x7f2e380b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e37fff700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e380b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 -- 172.21.15.46:0/126741339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 msgr2=0x7f2e380b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e380b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 -- 172.21.15.46:0/126741339 >> v2:172.21.15.132:3300/0 conn(0x7f2e380b39b0 msgr2=0x7f2e380b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/126741339 >> v2:172.21.15.132:3300/0 conn(0x7f2e380b39b0 0x7f2e380b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 -- 172.21.15.46:0/126741339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380b4930 con 0x7f2e38159050 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e38159420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2e380c3a10 tx=0x7f2e2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e504da73b24b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e377fe700 1 -- 172.21.15.46:0/126741339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00b860 con 0x7f2e38159050 2023-10-06T11:33:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e377fe700 1 -- 172.21.15.46:0/126741339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c00b9c0 con 0x7f2e38159050 2023-10-06T11:33:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e377fe700 1 -- 172.21.15.46:0/126741339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00bd60 con 0x7f2e38159050 2023-10-06T11:33:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/126741339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 msgr2=0x7f2e38159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.404+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e38159420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2e380c3a10 tx=0x7f2e2c009250 comp rx=0 tx=0).stop 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/126741339 shutdown_connections 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/126741339 >> v2:172.21.15.132:3300/0 conn(0x7f2e380b39b0 0x7f2e380b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e380b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/126741339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e38159420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/126741339 >> 172.21.15.46:0/126741339 conn(0x7f2e3806e780 msgr2=0x7f2e3806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/126741339 shutdown_connections 2023-10-06T11:33:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/126741339 wait complete. 2023-10-06T11:33:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 Processor -- start 2023-10-06T11:33:44.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.408+0000 7f2e3eaf0700 1 -- start start 2023-10-06T11:33:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 0x7f2e38157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e381582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c8820 con 0x7f2e38159050 2023-10-06T11:33:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c7780 con 0x7f2e380b39b0 2023-10-06T11:33:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c7a60 con 0x7f2e380d0aa0 2023-10-06T11:33:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40270/0 (socket says 172.21.15.46:40270) 2023-10-06T11:33:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 -- 172.21.15.46:0/3406355538 learned_addr learned my addr 172.21.15.46:0/3406355538 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e37fff700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e381582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 0x7f2e38157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 -- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 msgr2=0x7f2e381582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e381582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 -- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 msgr2=0x7f2e38157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 0x7f2e38157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 -- 172.21.15.46:0/3406355538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0094f0 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e37fff700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e381582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 0x7f2e38157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3d08d700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fa0 tx=0x7f2e30009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000ed20 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3000ee80 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30014670 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e380c7d40 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.412+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e380c8300 con 0x7f2e38159050 2023-10-06T11:33:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e380c8520 con 0x7f2e38159050 2023-10-06T11:33:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f2e300149e0 con 0x7f2e38159050 2023-10-06T11:33:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e34ff9700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 0x7f2e1c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f2e3007d700 con 0x7f2e38159050 2023-10-06T11:33:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e3c88c700 1 -- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 msgr2=0x7f2e1c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:33:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.416+0000 7f2e3c88c700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 0x7f2e1c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:33:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.424+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2e3004d600 con 0x7f2e38159050 2023-10-06T11:33:44.489 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.476+0000 7fce8d523000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:33:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.496+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 33) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f2e3004d9d0 con 0x7f2e38159050 2023-10-06T11:33:44.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.496+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 msgr2=0x7f2e1c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:33:44.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.496+0000 7f2e34ff9700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 0x7f2e1c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce57485700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce57485700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce57485700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce57485700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce57485700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.598 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce5ac8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.599 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce5ac8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.599 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce5ac8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.599 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce5ac8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.599 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:33:44.584+0000 7fce5ac8c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:33:44.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.624+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e380b4c10 con 0x7f2e38159050 2023-10-06T11:33:44.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.624+0000 7f2e34ff9700 1 -- 172.21.15.46:0/3406355538 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18458 (secure 0 0 0) 0x7f2e30067c20 con 0x7f2e38159050 2023-10-06T11:33:44.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:44.639 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":81,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:44.485292+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:20.937889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:22.026201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/4086238419":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/3217839558":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/1495113578":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/2355073621":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/2164225360":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/804885762":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/619749358":"2023-10-07T11:33:44.485237+0000","172.21.15.46:0/38534374":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/382630866":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6832/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/656656827":"2023-10-07T11:33:37.626781+0000","172.21.15.46:6833/32974":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2119803579":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/2042618050":"2023-10-07T11:33:37.626781+0000","172.21.15.46:0/3295527092":"2023-10-07T11:33:37.626781+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 msgr2=0x7f2e3815b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009fa0 tx=0x7f2e30009900 comp rx=0 tx=0).stop 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 shutdown_connections 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2e1c060700 0x7f2e1c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e38159050 0x7f2e3815b1b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e380d0aa0 0x7f2e381582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 --2- 172.21.15.46:0/3406355538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e380b39b0 0x7f2e38157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 >> 172.21.15.46:0/3406355538 conn(0x7f2e3806e780 msgr2=0x7f2e380cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 shutdown_connections 2023-10-06T11:33:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.628+0000 7f2e3eaf0700 1 -- 172.21.15.46:0/3406355538 wait complete. 2023-10-06T11:33:44.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:33:44.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.808+0000 7f1cb72f0700 1 Processor -- start 2023-10-06T11:33:44.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.808+0000 7f1cb72f0700 1 -- start start 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb72f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161840 0x7f1cb00caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb72f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1cb00caf50 0x7f1cb006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb72f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00c18f0 con 0x7f1cb0161470 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb72f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb00c0ce0 con 0x7f1cb00caf50 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb72f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00c0fc0 con 0x7f1cb0161840 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb508c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1cb0161470 0x7f1cb00c1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60774/0 (socket says 172.21.15.46:60774) 2023-10-06T11:33:44.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb508c700 1 -- 172.21.15.46:0/2701539987 learned_addr learned my addr 172.21.15.46:0/2701539987 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:44.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb588d700 1 --2- 172.21.15.46:0/2701539987 >> v2:172.21.15.46:3300/0 conn(0x7f1cb00caf50 0x7f1cb006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb488b700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161840 0x7f1cb00caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1ca7fff700 1 -- 172.21.15.46:0/2701539987 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007798372 0 0) 0x7f1cb00c18f0 con 0x7f1cb0161470 2023-10-06T11:33:44.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1ca7fff700 1 -- 172.21.15.46:0/2701539987 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c003570 con 0x7f1cb0161470 2023-10-06T11:33:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb588d700 1 -- 172.21.15.46:0/2701539987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161840 msgr2=0x7f1cb00caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb588d700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161840 0x7f1cb00caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb588d700 1 -- 172.21.15.46:0/2701539987 >> v1:172.21.15.46:6789/0 conn(0x7f1cb0161470 legacy=0x7f1cb00c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.812+0000 7f1cb588d700 1 -- 172.21.15.46:0/2701539987 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00c12a0 con 0x7f1cb00caf50 2023-10-06T11:33:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb588d700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00caf50 0x7f1cb006cc30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80098e0 tx=0x7f1ca80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24e49cdfa687339 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1ca7fff700 1 -- 172.21.15.46:0/2701539987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8011630 con 0x7f1cb00caf50 2023-10-06T11:33:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1ca7fff700 1 -- 172.21.15.46:0/2701539987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8011790 con 0x7f1cb00caf50 2023-10-06T11:33:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1ca7fff700 1 -- 172.21.15.46:0/2701539987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8011b30 con 0x7f1cb00caf50 2023-10-06T11:33:44.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 -- 172.21.15.46:0/2701539987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00caf50 msgr2=0x7f1cb006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00caf50 0x7f1cb006cc30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80098e0 tx=0x7f1ca80091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 -- 172.21.15.46:0/2701539987 shutdown_connections 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb00caf50 0x7f1cb006cc30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/2701539987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161840 0x7f1cb00caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 -- 172.21.15.46:0/2701539987 >> 172.21.15.46:0/2701539987 conn(0x7f1cb00cbe30 msgr2=0x7f1cb00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 -- 172.21.15.46:0/2701539987 shutdown_connections 2023-10-06T11:33:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.816+0000 7f1cb72f0700 1 -- 172.21.15.46:0/2701539987 wait complete. 2023-10-06T11:33:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.820+0000 7f1cb72f0700 1 Processor -- start 2023-10-06T11:33:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.820+0000 7f1cb72f0700 1 -- start start 2023-10-06T11:33:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cb00caf50 0x7f1cb016dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161470 0x7f1cb014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb016e510 con 0x7f1cb0161840 2023-10-06T11:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb014f6a0 con 0x7f1cb00caf50 2023-10-06T11:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00c2ee0 con 0x7f1cb0161470 2023-10-06T11:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40408/0 (socket says 172.21.15.46:40408) 2023-10-06T11:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 -- 172.21.15.46:0/4221568550 learned_addr learned my addr 172.21.15.46:0/4221568550 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb508c700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cb00caf50 0x7f1cb016dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb488b700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161470 0x7f1cb014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 -- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161470 msgr2=0x7f1cb014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161470 0x7f1cb014f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 -- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cb00caf50 msgr2=0x7f1cb016dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cb00caf50 0x7f1cb016dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 -- 172.21.15.46:0/4221568550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8009420 con 0x7f1cb0161840 2023-10-06T11:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb588d700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ed0 tx=0x7f1ca8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8011b10 con 0x7f1cb0161840 2023-10-06T11:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca80253f0 con 0x7f1cb0161840 2023-10-06T11:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00c34a0 con 0x7f1cb0161840 2023-10-06T11:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8025790 con 0x7f1cb0161840 2023-10-06T11:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.824+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb00b19c0 con 0x7f1cb0161840 2023-10-06T11:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.828+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb00c3630 con 0x7f1cb0161840 2023-10-06T11:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.828+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f1ca8025b80 con 0x7f1cb0161840 2023-10-06T11:33:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.828+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7147+0+0 (secure 0 0 0) 0x7f1ca807e1d0 con 0x7f1cb0161840 2023-10-06T11:33:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:44.836+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ca804e4f0 con 0x7f1cb0161840 2023-10-06T11:33:45.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.040+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1cb0063f90 con 0x7f1cb0161840 2023-10-06T11:33:45.055 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:33:45.044+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:33:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.048+0000 7f1ca57fa700 1 -- 172.21.15.46:0/4221568550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1ca8015090 con 0x7f1cb0161840 2023-10-06T11:33:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 msgr2=0x7f1cb00c3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ed0 tx=0x7f1ca8017000 comp rx=0 tx=0).stop 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 shutdown_connections 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cb0161840 0x7f1cb00c3930 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cb0161470 0x7f1cb014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 --2- 172.21.15.46:0/4221568550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cb00caf50 0x7f1cb016dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 >> 172.21.15.46:0/4221568550 conn(0x7f1cb00cbe30 msgr2=0x7f1cb015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 shutdown_connections 2023-10-06T11:33:45.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.052+0000 7f1cb72f0700 1 -- 172.21.15.46:0/4221568550 wait complete. 2023-10-06T11:33:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:33:45.071 INFO:tasks.ceph.mds.a.smithi046.stderr: -2379> 2023-10-06T11:33:27.532+0000 7f30e8288700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:33:45.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:33:45.115 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:33:45.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.228+0000 7fa11b216700 1 Processor -- start 2023-10-06T11:33:45.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.228+0000 7fa11b216700 1 -- start start 2023-10-06T11:33:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11b216700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1140adbe0 0x7fa1140d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11b216700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140afd20 con 0x7fa1140d0d60 2023-10-06T11:33:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11b216700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140b0000 con 0x7fa1140b0dc0 2023-10-06T11:33:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11b216700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa114159740 con 0x7fa1140adbe0 2023-10-06T11:33:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa119a13700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa1140d0d60 0x7fa1140b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60796/0 (socket says 172.21.15.46:60796) 2023-10-06T11:33:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa119a13700 1 -- 172.21.15.46:0/411866691 learned_addr learned my addr 172.21.15.46:0/411866691 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:45.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11a214700 1 --2- 172.21.15.46:0/411866691 >> v2:172.21.15.46:3300/0 conn(0x7fa1140adbe0 0x7fa1140d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa119212700 1 -- 172.21.15.46:0/411866691 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187946200 0 0) 0x7fa1140afd20 con 0x7fa1140d0d60 2023-10-06T11:33:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa119212700 1 -- 172.21.15.46:0/411866691 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa100003570 con 0x7fa1140d0d60 2023-10-06T11:33:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11a214700 1 -- 172.21.15.46:0/411866691 >> v1:172.21.15.132:6789/0 conn(0x7fa1140b0dc0 legacy=0x7fa1140b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11a214700 1 -- 172.21.15.46:0/411866691 >> v1:172.21.15.46:6789/0 conn(0x7fa1140d0d60 legacy=0x7fa1140b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11a214700 1 -- 172.21.15.46:0/411866691 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1140b02e0 con 0x7fa1140adbe0 2023-10-06T11:33:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.232+0000 7fa11a214700 1 --2- 172.21.15.46:0/411866691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140adbe0 0x7fa1140d0820 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa110009e60 tx=0x7fa110009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e96d9da491792c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa119212700 1 -- 172.21.15.46:0/411866691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11000e070 con 0x7fa1140adbe0 2023-10-06T11:33:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa119212700 1 -- 172.21.15.46:0/411866691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11000bb60 con 0x7fa1140adbe0 2023-10-06T11:33:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa119212700 1 -- 172.21.15.46:0/411866691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110012430 con 0x7fa1140adbe0 2023-10-06T11:33:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 -- 172.21.15.46:0/411866691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140adbe0 msgr2=0x7fa1140d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 --2- 172.21.15.46:0/411866691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140adbe0 0x7fa1140d0820 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa110009e60 tx=0x7fa110009780 comp rx=0 tx=0).stop 2023-10-06T11:33:45.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 -- 172.21.15.46:0/411866691 shutdown_connections 2023-10-06T11:33:45.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 --2- 172.21.15.46:0/411866691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140adbe0 0x7fa1140d0820 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 -- 172.21.15.46:0/411866691 >> 172.21.15.46:0/411866691 conn(0x7fa11406ef60 msgr2=0x7fa1140d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 -- 172.21.15.46:0/411866691 shutdown_connections 2023-10-06T11:33:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.236+0000 7fa11b216700 1 -- 172.21.15.46:0/411866691 wait complete. 2023-10-06T11:33:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 Processor -- start 2023-10-06T11:33:45.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 -- start start 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1140adbe0 0x7fa1141554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1140b0dc0 0x7fa11416d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11409d970 con 0x7fa1140d0d60 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11409bf50 con 0x7fa1140adbe0 2023-10-06T11:33:45.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.240+0000 7fa11b216700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11409d360 con 0x7fa1140b0dc0 2023-10-06T11:33:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40452/0 (socket says 172.21.15.46:40452) 2023-10-06T11:33:45.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 -- 172.21.15.46:0/2096868716 learned_addr learned my addr 172.21.15.46:0/2096868716 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11a214700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1140adbe0 0x7fa1141554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa119a13700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1140b0dc0 0x7fa11416d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 -- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1140b0dc0 msgr2=0x7fa11416d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1140b0dc0 0x7fa11416d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 -- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1140adbe0 msgr2=0x7fa1141554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1140adbe0 0x7fa1141554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 -- 172.21.15.46:0/2096868716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110009a20 con 0x7fa1140d0d60 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11aa15700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c590 tx=0x7fa10c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011590 con 0x7fa1140d0d60 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0116f0 con 0x7fa1140d0d60 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11416d970 con 0x7fa1140d0d60 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011ab0 con 0x7fa1140d0d60 2023-10-06T11:33:45.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11415b810 con 0x7fa1140d0d60 2023-10-06T11:33:45.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.244+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11409dab0 con 0x7fa1140d0d60 2023-10-06T11:33:45.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.248+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 33) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fa10c01b3b0 con 0x7fa1140d0d60 2023-10-06T11:33:45.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.248+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7233+0+0 (secure 0 0 0) 0x7fa10c07bc60 con 0x7fa1140d0d60 2023-10-06T11:33:45.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.256+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa11409dab0 con 0x7fa1140d0d60 2023-10-06T11:33:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.464+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa11416d970 con 0x7fa1140d0d60 2023-10-06T11:33:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.508+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 34) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fa10c04b9d0 con 0x7fa1140d0d60 2023-10-06T11:33:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.508+0000 7fa10a7fc700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 0x7fa0f8068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.508+0000 7fa11a214700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 0x7fa0f8068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:45.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:45.512+0000 7fa11a214700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 0x7fa0f8068d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa1100097c0 tx=0x7fa110019000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.056+0000 7fa10a7fc700 1 -- 172.21.15.46:0/2096868716 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa10c069830 con 0x7fa1140d0d60 2023-10-06T11:33:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.056+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 msgr2=0x7fa0f8068d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.056+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 0x7fa0f8068d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa1100097c0 tx=0x7fa110019000 comp rx=0 tx=0).stop 2023-10-06T11:33:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 msgr2=0x7fa114068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c590 tx=0x7fa10c007ef0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 shutdown_connections 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa0f8066980 0x7fa0f8068d60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1140d0d60 0x7fa114068a40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1140b0dc0 0x7fa11416d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 --2- 172.21.15.46:0/2096868716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1140adbe0 0x7fa1141554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 >> 172.21.15.46:0/2096868716 conn(0x7fa11406ef60 msgr2=0x7fa114158150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 shutdown_connections 2023-10-06T11:33:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.060+0000 7fa11b216700 1 -- 172.21.15.46:0/2096868716 wait complete. 2023-10-06T11:33:46.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:33:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.292+0000 7fbb39a4c700 1 Processor -- start 2023-10-06T11:33:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.296+0000 7fbb39a4c700 1 -- start start 2023-10-06T11:33:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.296+0000 7fbb39a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb34155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.296+0000 7fbb39a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb34156400 con 0x7fbb340a9040 2023-10-06T11:33:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.296+0000 7fbb39a4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340a9ea0 con 0x7fbb34067a10 2023-10-06T11:33:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.296+0000 7fbb39a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34155e70 con 0x7fbb34155560 2023-10-06T11:33:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb3924b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb340a9040 0x7fbb34156890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60800/0 (socket says 172.21.15.46:60800) 2023-10-06T11:33:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb3924b700 1 -- 172.21.15.46:0/1066181147 learned_addr learned my addr 172.21.15.46:0/1066181147 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb38a4a700 1 --2- 172.21.15.46:0/1066181147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb34155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409112818 0 0) 0x7fbb34156400 con 0x7fbb340a9040 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb24003570 con 0x7fbb340a9040 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175861557 0 0) 0x7fbb340a9ea0 con 0x7fbb34067a10 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb34156400 con 0x7fbb34067a10 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb38a4a700 1 -- 172.21.15.46:0/1066181147 >> v1:172.21.15.132:6789/0 conn(0x7fbb34067a10 legacy=0x7fbb34067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb38a4a700 1 -- 172.21.15.46:0/1066181147 >> v1:172.21.15.46:6789/0 conn(0x7fbb340a9040 legacy=0x7fbb34156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb38a4a700 1 -- 172.21.15.46:0/1066181147 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340685e0 con 0x7fbb34155560 2023-10-06T11:33:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb38a4a700 1 --2- 172.21.15.46:0/1066181147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb34155930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009900 tx=0x7fbb28009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3de0c597e8413a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28005c10 con 0x7fbb34155560 2023-10-06T11:33:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb28005d70 con 0x7fbb34155560 2023-10-06T11:33:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.300+0000 7fbb337fe700 1 -- 172.21.15.46:0/1066181147 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28006260 con 0x7fbb34155560 2023-10-06T11:33:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- 172.21.15.46:0/1066181147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 msgr2=0x7fbb34155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/1066181147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb34155930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009900 tx=0x7fbb28009120 comp rx=0 tx=0).stop 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- 172.21.15.46:0/1066181147 shutdown_connections 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/1066181147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb34155930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- 172.21.15.46:0/1066181147 >> 172.21.15.46:0/1066181147 conn(0x7fbb340afe40 msgr2=0x7fbb3406e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- 172.21.15.46:0/1066181147 shutdown_connections 2023-10-06T11:33:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- 172.21.15.46:0/1066181147 wait complete. 2023-10-06T11:33:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 Processor -- start 2023-10-06T11:33:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- start start 2023-10-06T11:33:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb34067a10 0x7fbb34145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb341483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34145d90 con 0x7fbb340a9040 2023-10-06T11:33:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340ae2f0 con 0x7fbb34067a10 2023-10-06T11:33:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.304+0000 7fbb39a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340ad920 con 0x7fbb34155560 2023-10-06T11:33:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40454/0 (socket says 172.21.15.46:40454) 2023-10-06T11:33:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 -- 172.21.15.46:0/733579915 learned_addr learned my addr 172.21.15.46:0/733579915 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb38a4a700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb34067a10 0x7fbb34145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb3924b700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb341483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 -- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 msgr2=0x7fbb341483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb341483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 -- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb34067a10 msgr2=0x7fbb34145850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb34067a10 0x7fbb34145850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 -- 172.21.15.46:0/733579915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280093c0 con 0x7fbb340a9040 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb33fff700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c5c0 tx=0x7fbb20007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20010510 con 0x7fbb340a9040 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20010690 con 0x7fbb340a9040 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20010a50 con 0x7fbb340a9040 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340acd10 con 0x7fbb340a9040 2023-10-06T11:33:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb340ad2d0 con 0x7fbb340a9040 2023-10-06T11:33:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.308+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb340adad0 con 0x7fbb340a9040 2023-10-06T11:33:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.312+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 34) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fbb2001b3b0 con 0x7fbb340a9040 2023-10-06T11:33:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.312+0000 7fbb30ff9700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 0x7fbb18062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.312+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7233+0+0 (secure 0 0 0) 0x7fbb2007c300 con 0x7fbb340a9040 2023-10-06T11:33:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.312+0000 7fbb38a4a700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 0x7fbb18062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.312+0000 7fbb38a4a700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 0x7fbb18062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009390 tx=0x7fbb28016000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.320+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbb2004c4d0 con 0x7fbb340a9040 2023-10-06T11:33:46.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:46.492+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbb34064590 con 0x7fbb340a9040 2023-10-06T11:33:47.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.060+0000 7fbb30ff9700 1 -- 172.21.15.46:0/733579915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbb20017020 con 0x7fbb340a9040 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 msgr2=0x7fbb18062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 0x7fbb18062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009390 tx=0x7fbb28016000 comp rx=0 tx=0).stop 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 msgr2=0x7fbb340addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c5c0 tx=0x7fbb20007de0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 shutdown_connections 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb180603a0 0x7fbb18062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb34155560 0x7fbb341483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340a9040 0x7fbb340addb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 --2- 172.21.15.46:0/733579915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb34067a10 0x7fbb34145850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 >> 172.21.15.46:0/733579915 conn(0x7fbb340afe40 msgr2=0x7fbb3406aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:47.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 shutdown_connections 2023-10-06T11:33:47.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.064+0000 7fbb39a4c700 1 -- 172.21.15.46:0/733579915 wait complete. 2023-10-06T11:33:47.081 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:33:47.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:33:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.256+0000 7f083359e700 1 Processor -- start 2023-10-06T11:33:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.256+0000 7f083359e700 1 -- start start 2023-10-06T11:33:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.256+0000 7f083359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0834158de0 0x7f08340b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08341591b0 0x7f0834069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 0x7f083415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0834069a20 con 0x7f0834158de0 2023-10-06T11:33:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08340b5eb0 con 0x7f08340695a0 2023-10-06T11:33:47.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08340cef30 con 0x7f08341591b0 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0834158de0 0x7f08340b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0834158de0 0x7f08340b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40462/0 (socket says 172.21.15.46:40462) 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 -- 172.21.15.46:0/2616808502 learned_addr learned my addr 172.21.15.46:0/2616808502 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f0831d9b700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08341591b0 0x7f0834069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f0832d9d700 1 --2- 172.21.15.46:0/2616808502 >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 0x7f083415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 -- 172.21.15.46:0/2616808502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08341591b0 msgr2=0x7f0834069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08341591b0 0x7f0834069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 -- 172.21.15.46:0/2616808502 >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 msgr2=0x7f083415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:47.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 --2- 172.21.15.46:0/2616808502 >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 0x7f083415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 -- 172.21.15.46:0/2616808502 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08340cf4f0 con 0x7f0834158de0 2023-10-06T11:33:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f0832d9d700 1 --2- 172.21.15.46:0/2616808502 >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 0x7f083415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083259c700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340b5970 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f08340b26f0 tx=0x7f0824009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64e4b2ec31e975f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:47.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083159a700 1 -- 172.21.15.46:0/2616808502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824009e70 con 0x7f0834158de0 2023-10-06T11:33:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083159a700 1 -- 172.21.15.46:0/2616808502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082400b860 con 0x7f0834158de0 2023-10-06T11:33:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083159a700 1 -- 172.21.15.46:0/2616808502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082400bc00 con 0x7f0834158de0 2023-10-06T11:33:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 -- 172.21.15.46:0/2616808502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 msgr2=0x7f08340b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.260+0000 7f083359e700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340b5970 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f08340b26f0 tx=0x7f0824009250 comp rx=0 tx=0).stop 2023-10-06T11:33:47.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- 172.21.15.46:0/2616808502 shutdown_connections 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- 172.21.15.46:0/2616808502 >> v2:172.21.15.132:3300/0 conn(0x7f08340695a0 0x7f083415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08341591b0 0x7f0834069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- 172.21.15.46:0/2616808502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340b5970 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- 172.21.15.46:0/2616808502 >> 172.21.15.46:0/2616808502 conn(0x7f0834068390 msgr2=0x7f08340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- 172.21.15.46:0/2616808502 shutdown_connections 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- 172.21.15.46:0/2616808502 wait complete. 2023-10-06T11:33:47.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 Processor -- start 2023-10-06T11:33:47.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- start start 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08340695a0 0x7f08340c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 0x7f083416cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08340c7c30 con 0x7f0834158de0 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08340c7020 con 0x7f08341591b0 2023-10-06T11:33:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f083359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08340c7300 con 0x7f08340695a0 2023-10-06T11:33:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f0831d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f0831d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40470/0 (socket says 172.21.15.46:40470) 2023-10-06T11:33:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f0831d9b700 1 -- 172.21.15.46:0/2644775741 learned_addr learned my addr 172.21.15.46:0/2644775741 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.264+0000 7f0832d9d700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 0x7f083416cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 -- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08340695a0 msgr2=0x7f08340c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08340695a0 0x7f08340c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 -- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 msgr2=0x7f083416cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 0x7f083416cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 -- 172.21.15.46:0/2644775741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240094f0 con 0x7f0834158de0 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0832d9d700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 0x7f083416cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:47.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f0831d9b700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f082800e930 tx=0x7f08280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800eed0 con 0x7f0834158de0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082800c850 con 0x7f0834158de0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08340c75e0 con 0x7f0834158de0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082800cbf0 con 0x7f0834158de0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083416d000 con 0x7f0834158de0 2023-10-06T11:33:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0834064590 con 0x7f0834158de0 2023-10-06T11:33:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.268+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0828013470 con 0x7f0834158de0 2023-10-06T11:33:47.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.272+0000 7f08227fc700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 0x7f0814062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:47.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.272+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6795+0+0 (secure 0 0 0) 0x7f082807cff0 con 0x7f0834158de0 2023-10-06T11:33:47.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.272+0000 7f083259c700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 0x7f0814062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.272+0000 7f083259c700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 0x7f0814062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f08340b26f0 tx=0x7f0824017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:47.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.280+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f082804cff0 con 0x7f0834158de0 2023-10-06T11:33:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:47.448+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f08340ccac0 con 0x7f0834158de0 2023-10-06T11:33:47.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:33:47.479 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:33:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.080+0000 7f08227fc700 1 -- 172.21.15.46:0/2644775741 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0828050610 con 0x7f0834158de0 2023-10-06T11:33:48.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 msgr2=0x7f0814062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 0x7f0814062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f08340b26f0 tx=0x7f0824017000 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 msgr2=0x7f08340c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f082800e930 tx=0x7f08280091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 shutdown_connections 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0814060670 0x7f0814062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08341591b0 0x7f083416cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0834158de0 0x7f08340c9ea0 secure :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f082800e930 tx=0x7f08280091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 --2- 172.21.15.46:0/2644775741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08340695a0 0x7f08340c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 >> 172.21.15.46:0/2644775741 conn(0x7f0834068390 msgr2=0x7f08340b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 shutdown_connections 2023-10-06T11:33:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.084+0000 7f083359e700 1 -- 172.21.15.46:0/2644775741 wait complete. 2023-10-06T11:33:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:33:48.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:33:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 Processor -- start 2023-10-06T11:33:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 -- start start 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98900ccc00 0x7f98900ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98900cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900cb4a0 con 0x7f98900b23f0 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98900cc780 con 0x7f98900ccc00 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.284+0000 7f9894d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900cb6e0 con 0x7f98900b27c0 2023-10-06T11:33:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988e59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98900b23f0 0x7f98900cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60806/0 (socket says 172.21.15.46:60806) 2023-10-06T11:33:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988e59c700 1 -- 172.21.15.46:0/2476455282 learned_addr learned my addr 172.21.15.46:0/2476455282 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988ed9d700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98900cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 --2- 172.21.15.46:0/2476455282 >> v2:172.21.15.132:3300/0 conn(0x7f98900ccc00 0x7f98900ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988d59a700 1 -- 172.21.15.46:0/2476455282 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076059360 0 0) 0x7f9880004b80 con 0x7f98900b23f0 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988d59a700 1 -- 172.21.15.46:0/2476455282 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9878003570 con 0x7f98900b23f0 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 -- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 msgr2=0x7f98900cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98900cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 -- 172.21.15.46:0/2476455282 >> v1:172.21.15.46:6789/0 conn(0x7f98900b23f0 legacy=0x7f98900cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 -- 172.21.15.46:0/2476455282 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900cb9c0 con 0x7f98900ccc00 2023-10-06T11:33:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988ed9d700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98900cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988dd9b700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900ccc00 0x7f98900ce4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f98840098e0 tx=0x7f98840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c55cd7a18f580d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988d59a700 1 -- 172.21.15.46:0/2476455282 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884011630 con 0x7f98900ccc00 2023-10-06T11:33:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988d59a700 1 -- 172.21.15.46:0/2476455282 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9884011790 con 0x7f98900ccc00 2023-10-06T11:33:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.288+0000 7f988d59a700 1 -- 172.21.15.46:0/2476455282 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884011b30 con 0x7f98900ccc00 2023-10-06T11:33:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900ccc00 msgr2=0x7f98900ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900ccc00 0x7f98900ce4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f98840098e0 tx=0x7f98840091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- 172.21.15.46:0/2476455282 shutdown_connections 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98900cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 --2- 172.21.15.46:0/2476455282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900ccc00 0x7f98900ce4d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- 172.21.15.46:0/2476455282 >> 172.21.15.46:0/2476455282 conn(0x7f9890099440 msgr2=0x7f989006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- 172.21.15.46:0/2476455282 shutdown_connections 2023-10-06T11:33:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- 172.21.15.46:0/2476455282 wait complete. 2023-10-06T11:33:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 Processor -- start 2023-10-06T11:33:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.292+0000 7f9894d45700 1 -- start start 2023-10-06T11:33:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900b23f0 0x7f98901562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98901695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901567f0 con 0x7f98900ccc00 2023-10-06T11:33:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900c8e90 con 0x7f98900b23f0 2023-10-06T11:33:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900c8280 con 0x7f98900b27c0 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40486/0 (socket says 172.21.15.46:40486) 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98901695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988ed9d700 1 -- 172.21.15.46:0/1800111219 learned_addr learned my addr 172.21.15.46:0/1800111219 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988e59c700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900b23f0 0x7f98901562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 -- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900b23f0 msgr2=0x7f98901562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900b23f0 0x7f98901562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 -- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 msgr2=0x7f989009a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 -- 172.21.15.46:0/1800111219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888009a20 con 0x7f98900b27c0 2023-10-06T11:33:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988ed9d700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f988dd9b700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98901695d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9884016000 tx=0x7f9884017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884000c40 con 0x7f98900b27c0 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9884011900 con 0x7f98900b27c0 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9884009420 con 0x7f98900b27c0 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884025420 con 0x7f98900b27c0 2023-10-06T11:33:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.296+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900c8840 con 0x7f98900b27c0 2023-10-06T11:33:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.300+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98900660f0 con 0x7f98900b27c0 2023-10-06T11:33:48.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.300+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9884023070 con 0x7f98900b27c0 2023-10-06T11:33:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.304+0000 7f987e7fc700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 0x7f9870062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.304+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f988407df00 con 0x7f98900b27c0 2023-10-06T11:33:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.304+0000 7f988e59c700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 0x7f9870062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.304+0000 7f988e59c700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 0x7f9870062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9880004ce0 tx=0x7f9880007b70 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.308+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f988404e120 con 0x7f98900b27c0 2023-10-06T11:33:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.484+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f989004b5c0 con 0x7f98900b27c0 2023-10-06T11:33:48.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.496+0000 7f987e7fc700 1 -- 172.21.15.46:0/1800111219 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9884051740 con 0x7f98900b27c0 2023-10-06T11:33:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 msgr2=0x7f9870062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 0x7f9870062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9880004ce0 tx=0x7f9880007b70 comp rx=0 tx=0).stop 2023-10-06T11:33:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 msgr2=0x7f98901695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98901695d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9884016000 tx=0x7f9884017000 comp rx=0 tx=0).stop 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 shutdown_connections 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9870060670 0x7f9870062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98900ccc00 0x7f989009a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98900b27c0 0x7f98901695d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 --2- 172.21.15.46:0/1800111219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98900b23f0 0x7f98901562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 >> 172.21.15.46:0/1800111219 conn(0x7f9890099440 msgr2=0x7f9890067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 shutdown_connections 2023-10-06T11:33:48.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.500+0000 7f9894d45700 1 -- 172.21.15.46:0/1800111219 wait complete. 2023-10-06T11:33:48.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:33:48.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.680+0000 7f4fb162e700 1 Processor -- start 2023-10-06T11:33:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.680+0000 7f4fb162e700 1 -- start start 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fb162e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac158da0 0x7f4fac15d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fb162e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4fac0cb5c0 0x7f4fac15a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fb162e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac15d3c0 con 0x7f4fac0d0af0 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fb162e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0d0f40 con 0x7f4fac0cb5c0 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fb162e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac15cdb0 con 0x7f4fac158da0 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4faa7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4fac0d0af0 0x7f4fac0caeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60814/0 (socket says 172.21.15.46:60814) 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4faa7fc700 1 -- 172.21.15.46:0/522240269 learned_addr learned my addr 172.21.15.46:0/522240269 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac158da0 0x7f4fac15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fab7fe700 1 --2- 172.21.15.46:0/522240269 >> v2:172.21.15.132:3300/0 conn(0x7f4fac0cb5c0 0x7f4fac15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443165428 0 0) 0x7f4fac15d3c0 con 0x7f4fac0d0af0 2023-10-06T11:33:48.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.684+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f94003570 con 0x7f4fac0d0af0 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3264607143 0 0) 0x7f4f94003570 con 0x7f4fac0d0af0 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac158da0 msgr2=0x7f4fac15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac158da0 0x7f4fac15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 >> v2:172.21.15.132:3300/0 conn(0x7f4fac0cb5c0 msgr2=0x7f4fac15a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 --2- 172.21.15.46:0/522240269 >> v2:172.21.15.132:3300/0 conn(0x7f4fac0cb5c0 0x7f4fac15a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac069800 con 0x7f4fac0d0af0 2023-10-06T11:33:48.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f4fa0001470 con 0x7f4fac0d0af0 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 >> v1:172.21.15.46:6789/0 conn(0x7f4fac0d0af0 legacy=0x7f4fac0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f94003a60 0x7f4f94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94004810 0x7f4f94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f940051c0 0x7f4f9400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0d0f40 con 0x7f4f94003a60 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac15cdb0 con 0x7f4f94004810 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9400bfa0 con 0x7f4f940051c0 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac158da0 0x7f4fac15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f94003a60 0x7f4f94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fab7fe700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f940051c0 0x7f4f9400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faa7fc700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94004810 0x7f4f94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 -- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f940051c0 msgr2=0x7f4f9400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f940051c0 0x7f4f9400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 -- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94004810 msgr2=0x7f4f94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94004810 0x7f4f94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 -- 172.21.15.46:0/522240269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94003570 con 0x7f4f94003a60 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4faaffd700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f94003a60 0x7f4f94003e70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cdaf0 tx=0x7f4f9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea67e018ec55ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00e710 con 0x7f4f94003a60 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00e870 con 0x7f4f94003a60 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.688+0000 7f4fa9ffb700 1 -- 172.21.15.46:0/522240269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00ec10 con 0x7f4f94003a60 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- 172.21.15.46:0/522240269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f94003a60 msgr2=0x7f4f94003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 --2- 172.21.15.46:0/522240269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f94003a60 0x7f4f94003e70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cdaf0 tx=0x7f4f9c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fab7fe700 1 -- 172.21.15.46:0/522240269 reap_dead start 2023-10-06T11:33:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- 172.21.15.46:0/522240269 shutdown_connections 2023-10-06T11:33:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- 172.21.15.46:0/522240269 >> 172.21.15.46:0/522240269 conn(0x7f4fac0683f0 msgr2=0x7f4fac0d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- 172.21.15.46:0/522240269 shutdown_connections 2023-10-06T11:33:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- 172.21.15.46:0/522240269 wait complete. 2023-10-06T11:33:48.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 Processor -- start 2023-10-06T11:33:48.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 -- start start 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94003a60 0x7f4fac0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.692+0000 7f4fb162e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 0x7f4fac0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac15c0c0 con 0x7f4fac15b710 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b3490 con 0x7f4f94003a60 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b3e60 con 0x7f4fac148b00 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40510/0 (socket says 172.21.15.46:40510) 2023-10-06T11:33:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fab7fe700 1 -- 172.21.15.46:0/2073095892 learned_addr learned my addr 172.21.15.46:0/2073095892 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 0x7f4fac0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faaffd700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94003a60 0x7f4fac0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 -- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94003a60 msgr2=0x7f4fac0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94003a60 0x7f4fac0b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 -- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 msgr2=0x7f4fac15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 -- 172.21.15.46:0/2073095892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0b2eb0 con 0x7f4fac148b00 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fab7fe700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4faa7fc700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 0x7f4fac0b41f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007390 tx=0x7f4fa0001e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0018030 con 0x7f4fac148b00 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0007770 con 0x7f4fac148b00 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac0c74e0 con 0x7f4fac148b00 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000fb00 con 0x7f4fac148b00 2023-10-06T11:33:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.696+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac0c77c0 con 0x7f4fac148b00 2023-10-06T11:33:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.700+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac0b3600 con 0x7f4fac148b00 2023-10-06T11:33:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.700+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4fa001b070 con 0x7f4fac148b00 2023-10-06T11:33:48.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.700+0000 7f4f92ffd700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 0x7f4f98062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.700+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f4fa0019070 con 0x7f4fac148b00 2023-10-06T11:33:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.704+0000 7f4faaffd700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 0x7f4f98062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.704+0000 7f4faaffd700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 0x7f4f98062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00a010 tx=0x7f4f9c019000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.708+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4fa004cbf0 con 0x7f4fac148b00 2023-10-06T11:33:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.888+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f4fac064590 con 0x7f4fac148b00 2023-10-06T11:33:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.900+0000 7f4f92ffd700 1 -- 172.21.15.46:0/2073095892 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4fa0050210 con 0x7f4fac148b00 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.904+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 msgr2=0x7f4f98062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.904+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 0x7f4f98062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00a010 tx=0x7f4f9c019000 comp rx=0 tx=0).stop 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.904+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 msgr2=0x7f4fac0b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.904+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 0x7f4fac0b41f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0007390 tx=0x7f4fa0001e90 comp rx=0 tx=0).stop 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 shutdown_connections 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4f98060670 0x7f4f98062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fac15b710 0x7f4fac15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fac148b00 0x7f4fac0b41f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 --2- 172.21.15.46:0/2073095892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f94003a60 0x7f4fac0b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 >> 172.21.15.46:0/2073095892 conn(0x7f4fac0683f0 msgr2=0x7f4fac0c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 shutdown_connections 2023-10-06T11:33:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:48.908+0000 7f4fb162e700 1 -- 172.21.15.46:0/2073095892 wait complete. 2023-10-06T11:33:48.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:33:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.092+0000 7f0434c8e700 1 Processor -- start 2023-10-06T11:33:49.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.092+0000 7f0434c8e700 1 -- start start 2023-10-06T11:33:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f0434c8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f04300d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f0434c8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 0x7f04300b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f0434c8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04300afd20 con 0x7f04300d0d60 2023-10-06T11:33:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f0434c8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0430159740 con 0x7f04300b0dc0 2023-10-06T11:33:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f0434c8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04300b1760 con 0x7f04300adbe0 2023-10-06T11:33:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042effd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f04300d0d60 0x7f04300b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60828/0 (socket says 172.21.15.46:60828) 2023-10-06T11:33:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042effd700 1 -- 172.21.15.46:0/2814396833 learned_addr learned my addr 172.21.15.46:0/2814396833 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 --2- 172.21.15.46:0/2814396833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f04300d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042ffff700 1 --2- 172.21.15.46:0/2814396833 >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 0x7f04300b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 -- 172.21.15.46:0/2814396833 >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 msgr2=0x7f04300b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 --2- 172.21.15.46:0/2814396833 >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 0x7f04300b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 -- 172.21.15.46:0/2814396833 >> v1:172.21.15.46:6789/0 conn(0x7f04300d0d60 legacy=0x7f04300b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 -- 172.21.15.46:0/2814396833 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300b0000 con 0x7f04300adbe0 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042ffff700 1 --2- 172.21.15.46:0/2814396833 >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 0x7f04300b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042e7fc700 1 -- 172.21.15.46:0/2814396833 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756252029 0 0) 0x7f04300afd20 con 0x7f04300d0d60 2023-10-06T11:33:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042f7fe700 1 --2- 172.21.15.46:0/2814396833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f04300d0820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f04200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c1bd4a6dddf58f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042e7fc700 1 -- 172.21.15.46:0/2814396833 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420009e80 con 0x7f04300adbe0 2023-10-06T11:33:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042e7fc700 1 -- 172.21.15.46:0/2814396833 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042000b860 con 0x7f04300adbe0 2023-10-06T11:33:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.096+0000 7f042e7fc700 1 -- 172.21.15.46:0/2814396833 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042000bbe0 con 0x7f04300adbe0 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- 172.21.15.46:0/2814396833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 msgr2=0x7f04300d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 --2- 172.21.15.46:0/2814396833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f04300d0820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f04200091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- 172.21.15.46:0/2814396833 shutdown_connections 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 --2- 172.21.15.46:0/2814396833 >> v2:172.21.15.132:3300/0 conn(0x7f04300b0dc0 0x7f04300b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 --2- 172.21.15.46:0/2814396833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f04300d0820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- 172.21.15.46:0/2814396833 >> 172.21.15.46:0/2814396833 conn(0x7f043006ef60 msgr2=0x7f04300d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- 172.21.15.46:0/2814396833 shutdown_connections 2023-10-06T11:33:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- 172.21.15.46:0/2814396833 wait complete. 2023-10-06T11:33:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 Processor -- start 2023-10-06T11:33:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.100+0000 7f0434c8e700 1 -- start start 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f0430159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 0x7f0430099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04300d0d60 0x7f0430155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043009c9e0 con 0x7f04300b0dc0 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043009cb70 con 0x7f04300d0d60 2023-10-06T11:33:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0430159e70 con 0x7f04300adbe0 2023-10-06T11:33:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 0x7f0430099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 0x7f0430099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40520/0 (socket says 172.21.15.46:40520) 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042effd700 1 -- 172.21.15.46:0/685508591 learned_addr learned my addr 172.21.15.46:0/685508591 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f0430159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042ffff700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04300d0d60 0x7f0430155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 -- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04300d0d60 msgr2=0x7f0430155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04300d0d60 0x7f0430155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 -- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 msgr2=0x7f0430099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 0x7f0430099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 -- 172.21.15.46:0/685508591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420009420 con 0x7f04300adbe0 2023-10-06T11:33:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f042f7fe700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f0430159890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f0420017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420000c90 con 0x7f04300adbe0 2023-10-06T11:33:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042000b9d0 con 0x7f04300adbe0 2023-10-06T11:33:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043015a150 con 0x7f04300adbe0 2023-10-06T11:33:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420026660 con 0x7f04300adbe0 2023-10-06T11:33:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.104+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043015a710 con 0x7f04300adbe0 2023-10-06T11:33:49.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.108+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430064590 con 0x7f04300adbe0 2023-10-06T11:33:49.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.108+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0420023030 con 0x7f04300adbe0 2023-10-06T11:33:49.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.108+0000 7f04177fe700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 0x7f041c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.108+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f042007e2b0 con 0x7f04300adbe0 2023-10-06T11:33:49.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.112+0000 7f042effd700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 0x7f041c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.112+0000 7f042effd700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 0x7f041c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0430158c80 tx=0x7f0424007df0 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.116+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f042004e4d0 con 0x7f04300adbe0 2023-10-06T11:33:49.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.292+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f04300675d0 con 0x7f04300adbe0 2023-10-06T11:33:49.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.304+0000 7f04177fe700 1 -- 172.21.15.46:0/685508591 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0420051af0 con 0x7f04300adbe0 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 msgr2=0x7f041c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 0x7f041c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0430158c80 tx=0x7f0424007df0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 msgr2=0x7f0430159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f0430159890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04200098e0 tx=0x7f0420017000 comp rx=0 tx=0).stop 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 shutdown_connections 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f041c060670 0x7f041c062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04300d0d60 0x7f0430155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04300b0dc0 0x7f0430099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 --2- 172.21.15.46:0/685508591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04300adbe0 0x7f0430159890 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 >> 172.21.15.46:0/685508591 conn(0x7f043006ef60 msgr2=0x7f04300b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 shutdown_connections 2023-10-06T11:33:49.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.308+0000 7f0434c8e700 1 -- 172.21.15.46:0/685508591 wait complete. 2023-10-06T11:33:49.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-06T11:33:49.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.492+0000 7fb25ec65700 1 Processor -- start 2023-10-06T11:33:49.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.492+0000 7fb25ec65700 1 -- start start 2023-10-06T11:33:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ec65700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb25815c770 0x7fb2580cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ec65700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2580cc150 con 0x7fb2580ccb50 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ec65700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2580cb280 con 0x7fb2580cec00 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ec65700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2580cfbb0 con 0x7fb25815c770 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb257fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb2580ccb50 0x7fb2580ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60832/0 (socket says 172.21.15.46:60832) 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb257fff700 1 -- 172.21.15.46:0/1044912963 learned_addr learned my addr 172.21.15.46:0/1044912963 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ca01700 1 --2- 172.21.15.46:0/1044912963 >> v2:172.21.15.132:3300/0 conn(0x7fb25815c770 0x7fb2580cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ca01700 1 -- 172.21.15.46:0/1044912963 >> v1:172.21.15.132:6789/0 conn(0x7fb2580cec00 legacy=0x7fb258162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ca01700 1 -- 172.21.15.46:0/1044912963 >> v1:172.21.15.46:6789/0 conn(0x7fb2580ccb50 legacy=0x7fb2580ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ca01700 1 -- 172.21.15.46:0/1044912963 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580cb540 con 0x7fb25815c770 2023-10-06T11:33:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb25ca01700 1 --2- 172.21.15.46:0/1044912963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb25815c770 0x7fb2580cad40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098d0 tx=0x7fb24c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03a961babc2dc87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb2577fe700 1 -- 172.21.15.46:0/1044912963 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c009e70 con 0x7fb25815c770 2023-10-06T11:33:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb2577fe700 1 -- 172.21.15.46:0/1044912963 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c005c10 con 0x7fb25815c770 2023-10-06T11:33:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.496+0000 7fb2577fe700 1 -- 172.21.15.46:0/1044912963 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00b650 con 0x7fb25815c770 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- 172.21.15.46:0/1044912963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb25815c770 msgr2=0x7fb2580cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 --2- 172.21.15.46:0/1044912963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb25815c770 0x7fb2580cad40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098d0 tx=0x7fb24c009120 comp rx=0 tx=0).stop 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- 172.21.15.46:0/1044912963 shutdown_connections 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 --2- 172.21.15.46:0/1044912963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb25815c770 0x7fb2580cad40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- 172.21.15.46:0/1044912963 >> 172.21.15.46:0/1044912963 conn(0x7fb258099480 msgr2=0x7fb25806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- 172.21.15.46:0/1044912963 shutdown_connections 2023-10-06T11:33:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- 172.21.15.46:0/1044912963 wait complete. 2023-10-06T11:33:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 Processor -- start 2023-10-06T11:33:49.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.500+0000 7fb25ec65700 1 -- start start 2023-10-06T11:33:49.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2580ccb50 0x7fb2580c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 0x7fb258151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258154380 con 0x7fb25815c770 2023-10-06T11:33:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258171ff0 con 0x7fb2580ccb50 2023-10-06T11:33:49.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25816daa0 con 0x7fb2580cec00 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25d202700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 0x7fb258151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:52856/0 (socket says 172.21.15.46:52856) 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 -- 172.21.15.46:0/2804617560 learned_addr learned my addr 172.21.15.46:0/2804617560 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25d202700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 0x7fb258151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40532/0 (socket says 172.21.15.46:40532) 2023-10-06T11:33:49.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ca01700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2580ccb50 0x7fb2580c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 -- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2580ccb50 msgr2=0x7fb2580c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2580ccb50 0x7fb2580c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 -- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 msgr2=0x7fb258151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 0x7fb258151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 -- 172.21.15.46:0/2804617560 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c0093c0 con 0x7fb2580cec00 2023-10-06T11:33:49.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb257fff700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c7c0 tx=0x7fb244007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244005b30 con 0x7fb2580cec00 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb244005c90 con 0x7fb2580cec00 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2580c9550 con 0x7fb2580cec00 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244004a80 con 0x7fb2580cec00 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.504+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2581538b0 con 0x7fb2580cec00 2023-10-06T11:33:49.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.508+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2580660f0 con 0x7fb2580cec00 2023-10-06T11:33:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.508+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb2440070d0 con 0x7fb2580cec00 2023-10-06T11:33:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.508+0000 7fb254ff9700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 0x7fb23c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.508+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7fb24407bf10 con 0x7fb2580cec00 2023-10-06T11:33:49.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.508+0000 7fb25ca01700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 0x7fb23c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.512+0000 7fb25ca01700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 0x7fb23c062b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009160 tx=0x7fb24c016000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:49.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.516+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb244049be0 con 0x7fb2580cec00 2023-10-06T11:33:49.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:49.744+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fb25804b5c0 con 0x7fb2580cec00 2023-10-06T11:33:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.084+0000 7fb254ff9700 1 -- 172.21.15.46:0/2804617560 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v573) v1 ==== 156+0+0 (secure 0 0 0) 0x7fb24404c0b0 con 0x7fb2580cec00 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 msgr2=0x7fb23c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 0x7fb23c062b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009160 tx=0x7fb24c016000 comp rx=0 tx=0).stop 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 msgr2=0x7fb258171ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb24400c7c0 tx=0x7fb244007f70 comp rx=0 tx=0).stop 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 shutdown_connections 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb23c0606a0 0x7fb23c062b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb25815c770 0x7fb258151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2580cec00 0x7fb258171ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 --2- 172.21.15.46:0/2804617560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2580ccb50 0x7fb2580c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.088+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 >> 172.21.15.46:0/2804617560 conn(0x7fb258099480 msgr2=0x7fb2580cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.092+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 shutdown_connections 2023-10-06T11:33:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.092+0000 7fb25ec65700 1 -- 172.21.15.46:0/2804617560 wait complete. 2023-10-06T11:33:50.125 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-06T11:33:50.125 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-06T11:33:50.125 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:33:50.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-06T11:33:50.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 Processor -- start 2023-10-06T11:33:50.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 -- start start 2023-10-06T11:33:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d380c5ff0 0x7f8d380ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d380cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d380cc150 con 0x7f8d3815c770 2023-10-06T11:33:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380cfbb0 con 0x7f8d380c5ff0 2023-10-06T11:33:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d3dc6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380cdbf0 con 0x7f8d380ccb50 2023-10-06T11:33:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d377fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d3815c770 0x7f8d380cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60838/0 (socket says 172.21.15.46:60838) 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d377fe700 1 -- 172.21.15.46:0/841582649 learned_addr learned my addr 172.21.15.46:0/841582649 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/841582649 >> v2:172.21.15.132:3300/0 conn(0x7f8d380c5ff0 0x7f8d380ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d37fff700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d380cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907045354 0 0) 0x7f8d380cc150 con 0x7f8d3815c770 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.276+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d20003570 con 0x7f8d3815c770 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3220795301 0 0) 0x7f8d20003570 con 0x7f8d3815c770 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 msgr2=0x7f8d380cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d380cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 >> v2:172.21.15.132:3300/0 conn(0x7f8d380c5ff0 msgr2=0x7f8d380ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 --2- 172.21.15.46:0/841582649 >> v2:172.21.15.132:3300/0 conn(0x7f8d380c5ff0 0x7f8d380ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380cb540 con 0x7f8d3815c770 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f8d280015f0 con 0x7f8d3815c770 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 >> v1:172.21.15.46:6789/0 conn(0x7f8d3815c770 legacy=0x7f8d380cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d20003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d20003fe0 0x7f8d20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d20004d50 0x7f8d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380cfbb0 con 0x7f8d20003fe0 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380cdbf0 con 0x7f8d20004d50 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2000ba90 con 0x7f8d380ccb50 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d20003fe0 0x7f8d20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d377fe700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d20003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 -- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 msgr2=0x7f8d20003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380ccb50 0x7f8d20003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 -- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d20004d50 msgr2=0x7f8d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d20004d50 0x7f8d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 -- 172.21.15.46:0/841582649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20003570 con 0x7f8d20003fe0 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d20003fe0 0x7f8d20004810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c00 tx=0x7f8d2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdf0f63b35e10ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00e6b0 con 0x7f8d20003fe0 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00e810 con 0x7f8d20003fe0 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d367fc700 1 -- 172.21.15.46:0/841582649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00ebd0 con 0x7f8d20003fe0 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/841582649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d20003fe0 msgr2=0x7f8d20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/841582649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d20003fe0 0x7f8d20004810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c00 tx=0x7f8d2c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d37fff700 1 -- 172.21.15.46:0/841582649 reap_dead start 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/841582649 shutdown_connections 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/841582649 >> 172.21.15.46:0/841582649 conn(0x7f8d38099480 msgr2=0x7f8d3806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/841582649 shutdown_connections 2023-10-06T11:33:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.280+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/841582649 wait complete. 2023-10-06T11:33:50.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 Processor -- start 2023-10-06T11:33:50.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- start start 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 0x7f8d3809ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380c9440 0x7f8d380c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3809ca10 con 0x7f8d3809bf70 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380c9df0 con 0x7f8d3815a8f0 2023-10-06T11:33:50.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3809a980 con 0x7f8d380c9440 2023-10-06T11:33:50.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40540/0 (socket says 172.21.15.46:40540) 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 -- 172.21.15.46:0/252893316 learned_addr learned my addr 172.21.15.46:0/252893316 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d377fe700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 0x7f8d3809ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d37fff700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380c9440 0x7f8d380c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 -- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380c9440 msgr2=0x7f8d380c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380c9440 0x7f8d380c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 -- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 msgr2=0x7f8d3809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 0x7f8d3809ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 -- 172.21.15.46:0/252893316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3809b350 con 0x7f8d3809bf70 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d377fe700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 0x7f8d3809ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d36ffd700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d20004730 tx=0x7f8d2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c022070 con 0x7f8d3809bf70 2023-10-06T11:33:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00e980 con 0x7f8d3809bf70 2023-10-06T11:33:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00e6b0 con 0x7f8d3809bf70 2023-10-06T11:33:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3809a090 con 0x7f8d3809bf70 2023-10-06T11:33:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3809a610 con 0x7f8d3809bf70 2023-10-06T11:33:50.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.288+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3809ab30 con 0x7f8d3809bf70 2023-10-06T11:33:50.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.296+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8d2c01c660 con 0x7f8d3809bf70 2023-10-06T11:33:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.296+0000 7f8d1f7fe700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 0x7f8d24062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.296+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f8d2c07e340 con 0x7f8d3809bf70 2023-10-06T11:33:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.296+0000 7f8d377fe700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 0x7f8d24062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.296+0000 7f8d377fe700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 0x7f8d24062b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d280017a0 tx=0x7f8d28009040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:50.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.300+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8d3809ab30 con 0x7f8d3809bf70 2023-10-06T11:33:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:50.528+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f8d3809aaf0 con 0x7f8d3809bf70 2023-10-06T11:33:51.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.092+0000 7f8d1f7fe700 1 -- 172.21.15.46:0/252893316 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v574) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d2c0570f0 con 0x7f8d3809bf70 2023-10-06T11:33:51.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 msgr2=0x7f8d24062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 0x7f8d24062b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d280017a0 tx=0x7f8d28009040 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 msgr2=0x7f8d3809c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d20004730 tx=0x7f8d2c01a000 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 shutdown_connections 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d240606a0 0x7f8d24062b50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d380c9440 0x7f8d380c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d3809bf70 0x7f8d3809c3e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 --2- 172.21.15.46:0/252893316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d3815a8f0 0x7f8d3809ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.096+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 >> 172.21.15.46:0/252893316 conn(0x7f8d38099480 msgr2=0x7f8d380cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.100+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 shutdown_connections 2023-10-06T11:33:51.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.100+0000 7f8d3dc6e700 1 -- 172.21.15.46:0/252893316 wait complete. 2023-10-06T11:33:51.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:33:51.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 Processor -- start 2023-10-06T11:33:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 -- start start 2023-10-06T11:33:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd038147280 0x7fd03806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0380c1980 con 0x7fd038161fc0 2023-10-06T11:33:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0380c2520 con 0x7fd038146eb0 2023-10-06T11:33:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03f0cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0380c0d70 con 0x7fd038147280 2023-10-06T11:33:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd037fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd038161fc0 0x7fd0380caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60854/0 (socket says 172.21.15.46:60854) 2023-10-06T11:33:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd037fff700 1 -- 172.21.15.46:0/2916408128 learned_addr learned my addr 172.21.15.46:0/2916408128 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.292+0000 7fd03d66a700 1 --2- 172.21.15.46:0/2916408128 >> v2:172.21.15.46:3300/0 conn(0x7fd038147280 0x7fd03806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631689049 0 0) 0x7fd0380c1980 con 0x7fd038161fc0 2023-10-06T11:33:51.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd028003570 con 0x7fd038161fc0 2023-10-06T11:33:51.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271655307 0 0) 0x7fd0380c2520 con 0x7fd038146eb0 2023-10-06T11:33:51.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0380c1980 con 0x7fd038146eb0 2023-10-06T11:33:51.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03d66a700 1 -- 172.21.15.46:0/2916408128 >> v1:172.21.15.132:6789/0 conn(0x7fd038146eb0 legacy=0x7fd0380c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03d66a700 1 -- 172.21.15.46:0/2916408128 >> v1:172.21.15.46:6789/0 conn(0x7fd038161fc0 legacy=0x7fd0380caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03d66a700 1 -- 172.21.15.46:0/2916408128 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380c1050 con 0x7fd038147280 2023-10-06T11:33:51.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03d66a700 1 --2- 172.21.15.46:0/2916408128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd03806cc30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd030009e60 tx=0x7fd030009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a47cdeb7749b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000e070 con 0x7fd038147280 2023-10-06T11:33:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03000bab0 con 0x7fd038147280 2023-10-06T11:33:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd0377fe700 1 -- 172.21.15.46:0/2916408128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000be70 con 0x7fd038147280 2023-10-06T11:33:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 -- 172.21.15.46:0/2916408128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 msgr2=0x7fd03806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/2916408128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd03806cc30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd030009e60 tx=0x7fd030009780 comp rx=0 tx=0).stop 2023-10-06T11:33:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 -- 172.21.15.46:0/2916408128 shutdown_connections 2023-10-06T11:33:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/2916408128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd03806cc30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 -- 172.21.15.46:0/2916408128 >> 172.21.15.46:0/2916408128 conn(0x7fd0380cbe30 msgr2=0x7fd0380cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 -- 172.21.15.46:0/2916408128 shutdown_connections 2023-10-06T11:33:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.296+0000 7fd03f0cd700 1 -- 172.21.15.46:0/2916408128 wait complete. 2023-10-06T11:33:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 Processor -- start 2023-10-06T11:33:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 -- start start 2023-10-06T11:33:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038146eb0 0x7fd03815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd038161fc0 0x7fd0380b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038160600 con 0x7fd038147280 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380c3060 con 0x7fd038146eb0 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.300+0000 7fd03f0cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380c3340 con 0x7fd038161fc0 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40560/0 (socket says 172.21.15.46:40560) 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 -- 172.21.15.46:0/1403596363 learned_addr learned my addr 172.21.15.46:0/1403596363 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd03ce69700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038146eb0 0x7fd03815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd03d66a700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd038161fc0 0x7fd0380b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 -- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd038161fc0 msgr2=0x7fd0380b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd038161fc0 0x7fd0380b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 -- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038146eb0 msgr2=0x7fd03815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038146eb0 0x7fd03815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 -- 172.21.15.46:0/1403596363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030009a20 con 0x7fd038147280 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd037fff700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c790 tx=0x7fd024007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024005b30 con 0x7fd038147280 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd024005cd0 con 0x7fd038147280 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0380c3620 con 0x7fd038147280 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024007320 con 0x7fd038147280 2023-10-06T11:33:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.304+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03815d650 con 0x7fd038147280 2023-10-06T11:33:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.308+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0380c3b20 con 0x7fd038147280 2023-10-06T11:33:51.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.308+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd024012400 con 0x7fd038147280 2023-10-06T11:33:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.308+0000 7fd034ff9700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 0x7fd01c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.308+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7fd02407bee0 con 0x7fd038147280 2023-10-06T11:33:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.308+0000 7fd03ce69700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 0x7fd01c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.312+0000 7fd03ce69700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 0x7fd01c062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c007d50 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.316+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd024049be0 con 0x7fd038147280 2023-10-06T11:33:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.488+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd0380660f0 con 0x7fd038147280 2023-10-06T11:33:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.496+0000 7fd034ff9700 1 -- 172.21.15.46:0/1403596363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0240170c0 con 0x7fd038147280 2023-10-06T11:33:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 msgr2=0x7fd01c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 0x7fd01c062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd02c007fd0 tx=0x7fd02c007d50 comp rx=0 tx=0).stop 2023-10-06T11:33:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 msgr2=0x7fd0380b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c790 tx=0x7fd024007e80 comp rx=0 tx=0).stop 2023-10-06T11:33:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 shutdown_connections 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd01c060670 0x7fd01c062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd038161fc0 0x7fd0380b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd038147280 0x7fd0380b04d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 --2- 172.21.15.46:0/1403596363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd038146eb0 0x7fd03815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 >> 172.21.15.46:0/1403596363 conn(0x7fd0380cbe30 msgr2=0x7fd0380cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 shutdown_connections 2023-10-06T11:33:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.500+0000 7fd03f0cd700 1 -- 172.21.15.46:0/1403596363 wait complete. 2023-10-06T11:33:51.530 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:33:51.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:51.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:33:51.614 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:33:51.615 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:33:51.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.776+0000 7fcd11b23700 1 Processor -- start 2023-10-06T11:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.776+0000 7fcd11b23700 1 -- start start 2023-10-06T11:33:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd11b23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd0c156e20 0x7fcd0c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd11b23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c06d790 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd11b23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0acd60 con 0x7fcd0c067760 2023-10-06T11:33:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd11b23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c0ad040 con 0x7fcd0c156e20 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd10b21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd0c156e20 0x7fcd0c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd03fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd0c0aaf30 0x7fcd0c067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60870/0 (socket says 172.21.15.46:60870) 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd03fff700 1 -- 172.21.15.46:0/672413314 learned_addr learned my addr 172.21.15.46:0/672413314 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd10b21700 1 -- 172.21.15.46:0/672413314 >> v1:172.21.15.132:6789/0 conn(0x7fcd0c067760 legacy=0x7fcd0c068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd10b21700 1 -- 172.21.15.46:0/672413314 >> v1:172.21.15.46:6789/0 conn(0x7fcd0c0aaf30 legacy=0x7fcd0c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd10b21700 1 -- 172.21.15.46:0/672413314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0ad320 con 0x7fcd0c156e20 2023-10-06T11:33:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.780+0000 7fcd037fe700 1 -- 172.21.15.46:0/672413314 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120652570 0 0) 0x7fcd0c06d790 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd10b21700 1 --2- 172.21.15.46:0/672413314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c156e20 0x7fcd0c06d250 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcd080098e0 tx=0x7fcd080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362929f2385df274 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd037fe700 1 -- 172.21.15.46:0/672413314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08009e80 con 0x7fcd0c156e20 2023-10-06T11:33:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd037fe700 1 -- 172.21.15.46:0/672413314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0800b860 con 0x7fcd0c156e20 2023-10-06T11:33:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd037fe700 1 -- 172.21.15.46:0/672413314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800bbe0 con 0x7fcd0c156e20 2023-10-06T11:33:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 -- 172.21.15.46:0/672413314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c156e20 msgr2=0x7fcd0c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 --2- 172.21.15.46:0/672413314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c156e20 0x7fcd0c06d250 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcd080098e0 tx=0x7fcd080091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 -- 172.21.15.46:0/672413314 shutdown_connections 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 --2- 172.21.15.46:0/672413314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c156e20 0x7fcd0c06d250 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 -- 172.21.15.46:0/672413314 >> 172.21.15.46:0/672413314 conn(0x7fcd0c062ef0 msgr2=0x7fcd0c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 -- 172.21.15.46:0/672413314 shutdown_connections 2023-10-06T11:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.784+0000 7fcd11b23700 1 -- 172.21.15.46:0/672413314 wait complete. 2023-10-06T11:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 Processor -- start 2023-10-06T11:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 -- start start 2023-10-06T11:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd0c067760 0x7fcd0c16da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd0c156e20 0x7fcd0c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0af930 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c16e300 con 0x7fcd0c156e20 2023-10-06T11:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.788+0000 7fcd11b23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c16e5e0 con 0x7fcd0c067760 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40590/0 (socket says 172.21.15.46:40590) 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 -- 172.21.15.46:0/2710101616 learned_addr learned my addr 172.21.15.46:0/2710101616 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd10b21700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd0c067760 0x7fcd0c16da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd11322700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd0c156e20 0x7fcd0c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 -- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd0c067760 msgr2=0x7fcd0c16da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd0c067760 0x7fcd0c16da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 -- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd0c156e20 msgr2=0x7fcd0c0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd0c156e20 0x7fcd0c0b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 -- 172.21.15.46:0/2710101616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08009420 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd03fff700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c810 tx=0x7fccfc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc005c00 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc005d60 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc007320 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c16e8c0 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.792+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c16dfc0 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c0c29b0 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fccfc01b3b0 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd00ff9700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 0x7fccf8062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7fccfc07c0e0 con 0x7fcd0c0aaf30 2023-10-06T11:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd10b21700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 0x7fccf8062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.796+0000 7fcd10b21700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 0x7fccf8062bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd080093f0 tx=0x7fcd08017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:51.804+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd0c0c29b0 con 0x7fcd0c0aaf30 2023-10-06T11:33:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.012+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd0c0c29b0 con 0x7fcd0c0aaf30 2023-10-06T11:33:52.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.012+0000 7fcd00ff9700 1 -- 172.21.15.46:0/2710101616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fccfc017de0 con 0x7fcd0c0aaf30 2023-10-06T11:33:52.022 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:52.022 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5699,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1551766312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1551766312},{"type":"v1","addr":"172.21.15.132:6835","nonce":1551766312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3432821350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3432821350},{"type":"v1","addr":"172.21.15.132:6834","nonce":3432821350}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/672337913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":672337913},{"type":"v1","addr":"172.21.15.46:6837","nonce":672337913}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6759,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3773498684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3773498684},{"type":"v1","addr":"172.21.15.46:6836","nonce":3773498684}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 msgr2=0x7fccf8062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 0x7fccf8062bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd080093f0 tx=0x7fcd08017000 comp rx=0 tx=0).stop 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 msgr2=0x7fcd0c157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c810 tx=0x7fccfc007d90 comp rx=0 tx=0).stop 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 shutdown_connections 2023-10-06T11:33:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fccf8060740 0x7fccf8062bf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd0c156e20 0x7fcd0c0b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd0c0aaf30 0x7fcd0c157880 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 --2- 172.21.15.46:0/2710101616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd0c067760 0x7fcd0c16da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 >> 172.21.15.46:0/2710101616 conn(0x7fcd0c062ef0 msgr2=0x7fcd0c06ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 shutdown_connections 2023-10-06T11:33:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.016+0000 7fcd11b23700 1 -- 172.21.15.46:0/2710101616 wait complete. 2023-10-06T11:33:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2023-10-06T11:33:52.048 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:33:52.048 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:33:52.048 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:33:52.048 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:52.049 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:33:52.049 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:33:52.049 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:33:52.049 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:52.049 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:33:52.049 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:33:52.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:33:52.050 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:52.050 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:33:52.050 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:33:52.050 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:33:52.050 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:33:52.068 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:33:52.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:33:52.069 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:33:52.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:33:52.079 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:33:52.079 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:33:52.082 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:33:52.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:33:52.115 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:33:52.116 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:52.118 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:33:52.118 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:52.137 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:33:52.137 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:52.141 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:33:52.141 DEBUG:teuthology.parallel:result is None 2023-10-06T11:33:52.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:33:52.229 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:33:52.230 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:33:52.245 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:33:52.245 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:33:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.296+0000 7f70aeef0700 1 Processor -- start 2023-10-06T11:33:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.296+0000 7f70aeef0700 1 -- start start 2023-10-06T11:33:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70aeef0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70a8159050 0x7f70a8159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70aeef0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f70a80d0aa0 0x7f70a80b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70aeef0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a80b4f80 con 0x7f70a80b39b0 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70aeef0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a80b4370 con 0x7f70a8159050 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70aeef0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a80b4650 con 0x7f70a80d0aa0 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70acc8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70a8159050 0x7f70a8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70ad48d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70a80b39b0 0x7f70a80b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60878/0 (socket says 172.21.15.46:60878) 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a7fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f70a80d0aa0 0x7f70a80b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70ad48d700 1 -- 172.21.15.46:0/2993797286 learned_addr learned my addr 172.21.15.46:0/2993797286 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a7fff700 1 -- 172.21.15.46:0/2993797286 >> v2:172.21.15.46:3300/0 conn(0x7f70a8159050 msgr2=0x7f70a8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a7fff700 1 --2- 172.21.15.46:0/2993797286 >> v2:172.21.15.46:3300/0 conn(0x7f70a8159050 0x7f70a8159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a7fff700 1 -- 172.21.15.46:0/2993797286 >> v1:172.21.15.46:6789/0 conn(0x7f70a80b39b0 legacy=0x7f70a80b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a77fe700 1 -- 172.21.15.46:0/2993797286 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110269604 0 0) 0x7f70a80b4f80 con 0x7f70a80b39b0 2023-10-06T11:33:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.300+0000 7f70a7fff700 1 -- 172.21.15.46:0/2993797286 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a80b4930 con 0x7f70a80d0aa0 2023-10-06T11:33:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70a7fff700 1 --2- 172.21.15.46:0/2993797286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80d0aa0 0x7f70a80b5410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70a80892d0 tx=0x7f70940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2a150b22f1c500 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70a77fe700 1 -- 172.21.15.46:0/2993797286 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094011630 con 0x7f70a80d0aa0 2023-10-06T11:33:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70a77fe700 1 -- 172.21.15.46:0/2993797286 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094011790 con 0x7f70a80d0aa0 2023-10-06T11:33:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70a77fe700 1 -- 172.21.15.46:0/2993797286 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094011b30 con 0x7f70a80d0aa0 2023-10-06T11:33:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 -- 172.21.15.46:0/2993797286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80d0aa0 msgr2=0x7f70a80b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 --2- 172.21.15.46:0/2993797286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80d0aa0 0x7f70a80b5410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70a80892d0 tx=0x7f70940091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 -- 172.21.15.46:0/2993797286 shutdown_connections 2023-10-06T11:33:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 --2- 172.21.15.46:0/2993797286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80d0aa0 0x7f70a80b5410 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 --2- 172.21.15.46:0/2993797286 >> v2:172.21.15.46:3300/0 conn(0x7f70a8159050 0x7f70a8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 -- 172.21.15.46:0/2993797286 >> 172.21.15.46:0/2993797286 conn(0x7f70a806e780 msgr2=0x7f70a806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 -- 172.21.15.46:0/2993797286 shutdown_connections 2023-10-06T11:33:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.304+0000 7f70aeef0700 1 -- 172.21.15.46:0/2993797286 wait complete. 2023-10-06T11:33:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 Processor -- start 2023-10-06T11:33:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 -- start start 2023-10-06T11:33:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80b39b0 0x7f70a815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70a80d0aa0 0x7f70a8169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a816aa10 con 0x7f70a8159050 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a816add0 con 0x7f70a80b39b0 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70aeef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a816a170 con 0x7f70a80d0aa0 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40640/0 (socket says 172.21.15.46:40640) 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 -- 172.21.15.46:0/1305305145 learned_addr learned my addr 172.21.15.46:0/1305305145 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70acc8c700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80b39b0 0x7f70a815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70a7fff700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70a80d0aa0 0x7f70a8169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 -- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70a80d0aa0 msgr2=0x7f70a8169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70a80d0aa0 0x7f70a8169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 -- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80b39b0 msgr2=0x7f70a815c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80b39b0 0x7f70a815c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.308+0000 7f70ad48d700 1 -- 172.21.15.46:0/1305305145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009420 con 0x7f70a8159050 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70ad48d700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f70a000cc90 tx=0x7f70a0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000cea0 con 0x7f70a8159050 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a000ab70 con 0x7f70a8159050 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a816a450 con 0x7f70a8159050 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a00070f0 con 0x7f70a8159050 2023-10-06T11:33:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a80c6880 con 0x7f70a8159050 2023-10-06T11:33:52.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.312+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a80b3810 con 0x7f70a8159050 2023-10-06T11:33:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.316+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f70a0007460 con 0x7f70a8159050 2023-10-06T11:33:52.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.316+0000 7f70a4ff9700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 0x7f708c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.316+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f70a007d090 con 0x7f70a8159050 2023-10-06T11:33:52.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.316+0000 7f70acc8c700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 0x7f708c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:52.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.316+0000 7f70acc8c700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 0x7f708c062b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f709c009ca0 tx=0x7f709c008040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.324+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f70a80b3810 con 0x7f70a8159050 2023-10-06T11:33:52.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:52.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f70a80b3810 con 0x7f70a8159050 2023-10-06T11:33:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.508+0000 7f70a4ff9700 1 -- 172.21.15.46:0/1305305145 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f70a80b3810 con 0x7f70a8159050 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 msgr2=0x7f708c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 0x7f708c062b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f709c009ca0 tx=0x7f709c008040 comp rx=0 tx=0).stop 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 msgr2=0x7f70a80c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f70a000cc90 tx=0x7f70a0001a80 comp rx=0 tx=0).stop 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 shutdown_connections 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f708c0606a0 0x7f708c062b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70a8159050 0x7f70a80c6340 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70a80d0aa0 0x7f70a8169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 --2- 172.21.15.46:0/1305305145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a80b39b0 0x7f70a815c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 >> 172.21.15.46:0/1305305145 conn(0x7f70a806e780 msgr2=0x7f70a8158500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 shutdown_connections 2023-10-06T11:33:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.512+0000 7f70aeef0700 1 -- 172.21.15.46:0/1305305145 wait complete. 2023-10-06T11:33:53.524 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:33:53.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:33:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 Processor -- start 2023-10-06T11:33:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 -- start start 2023-10-06T11:33:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe68806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68806d460 con 0x7fe6880ad6c0 2023-10-06T11:33:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880c7000 con 0x7fe68815ab60 2023-10-06T11:33:53.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68e345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68806ed10 con 0x7fe68806e8a0 2023-10-06T11:33:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68c8e2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe6880ad6c0 0x7fe6880adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52270/0 (socket says 172.21.15.46:52270) 2023-10-06T11:33:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.704+0000 7fe68c8e2700 1 -- 172.21.15.46:0/4103358883 learned_addr learned my addr 172.21.15.46:0/4103358883 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe6877fe700 1 --2- 172.21.15.46:0/4103358883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe68806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503741291 0 0) 0x7fe6880c7000 con 0x7fe68815ab60 2023-10-06T11:33:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe670003570 con 0x7fe68815ab60 2023-10-06T11:33:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861570962 0 0) 0x7fe68806d460 con 0x7fe6880ad6c0 2023-10-06T11:33:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6880c7000 con 0x7fe6880ad6c0 2023-10-06T11:33:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083979884 0 0) 0x7fe6880c7000 con 0x7fe6880ad6c0 2023-10-06T11:33:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe6877fe700 1 -- 172.21.15.46:0/4103358883 >> v1:172.21.15.132:6789/0 conn(0x7fe68815ab60 legacy=0x7fe68815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe6877fe700 1 -- 172.21.15.46:0/4103358883 >> v1:172.21.15.46:6789/0 conn(0x7fe6880ad6c0 legacy=0x7fe6880adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe6877fe700 1 -- 172.21.15.46:0/4103358883 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880c72c0 con 0x7fe68806e8a0 2023-10-06T11:33:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe6877fe700 1 --2- 172.21.15.46:0/4103358883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe68806cc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009890 tx=0x7fe67c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c06e1885f67d26c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680001810 con 0x7fe68806e8a0 2023-10-06T11:33:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c011420 con 0x7fe68806e8a0 2023-10-06T11:33:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe686ffd700 1 -- 172.21.15.46:0/4103358883 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6780014c0 con 0x7fe68806e8a0 2023-10-06T11:33:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe68e345700 1 -- 172.21.15.46:0/4103358883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 msgr2=0x7fe68806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.708+0000 7fe68e345700 1 --2- 172.21.15.46:0/4103358883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe68806cc30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009890 tx=0x7fe67c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 -- 172.21.15.46:0/4103358883 shutdown_connections 2023-10-06T11:33:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 --2- 172.21.15.46:0/4103358883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe68806cc30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 -- 172.21.15.46:0/4103358883 >> 172.21.15.46:0/4103358883 conn(0x7fe6880cbe30 msgr2=0x7fe6880cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:53.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 -- 172.21.15.46:0/4103358883 shutdown_connections 2023-10-06T11:33:53.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 -- 172.21.15.46:0/4103358883 wait complete. 2023-10-06T11:33:53.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 Processor -- start 2023-10-06T11:33:53.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.712+0000 7fe68e345700 1 -- start start 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe688141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68815ab60 0x7fe688158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881423c0 con 0x7fe6880ad6c0 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688159e10 con 0x7fe68815ab60 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880c54b0 con 0x7fe68806e8a0 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:53.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54082/0 (socket says 172.21.15.46:54082) 2023-10-06T11:33:53.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 -- 172.21.15.46:0/2609377206 learned_addr learned my addr 172.21.15.46:0/2609377206 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:53.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe687fff700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe688141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:53.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68c8e2700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68815ab60 0x7fe688158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 -- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 msgr2=0x7fe688141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe688141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 -- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68815ab60 msgr2=0x7fe688158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68815ab60 0x7fe688158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 -- 172.21.15.46:0/2609377206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c009420 con 0x7fe6880ad6c0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe6877fe700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0093f0 tx=0x7fe67c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c010ac0 con 0x7fe6880ad6c0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c010c20 con 0x7fe6880ad6c0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c01d430 con 0x7fe6880ad6c0 2023-10-06T11:33:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.716+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe688159200 con 0x7fe6880ad6c0 2023-10-06T11:33:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.720+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6881597c0 con 0x7fe6880ad6c0 2023-10-06T11:33:53.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.720+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe688159fa0 con 0x7fe6880ad6c0 2023-10-06T11:33:53.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.724+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe67c01d7a0 con 0x7fe6880ad6c0 2023-10-06T11:33:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.724+0000 7fe66ffff700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 0x7fe674062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.724+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fe67c07e510 con 0x7fe6880ad6c0 2023-10-06T11:33:53.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.724+0000 7fe687fff700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 0x7fe674062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:53.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.724+0000 7fe687fff700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 0x7fe674062af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe678001230 tx=0x7fe67800a580 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:53.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.728+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe67c04ea20 con 0x7fe6880ad6c0 2023-10-06T11:33:53.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.956+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe688063f90 con 0x7fe6880ad6c0 2023-10-06T11:33:53.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.960+0000 7fe66ffff700 1 -- 172.21.15.46:0/2609377206 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe67c052040 con 0x7fe6880ad6c0 2023-10-06T11:33:53.968 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:53.968 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.972 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.974 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.975 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.978 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:33:53.979 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.980 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:53.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:53.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:33:53.983 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:33:53.984 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:33:53.985 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:53.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:53.987 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:33:53.988 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:33:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 msgr2=0x7fe674062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 0x7fe674062af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe678001230 tx=0x7fe67800a580 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 msgr2=0x7fe688158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0093f0 tx=0x7fe67c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 shutdown_connections 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe674060640 0x7fe674062af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68815ab60 0x7fe688158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6880ad6c0 0x7fe688158690 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 --2- 172.21.15.46:0/2609377206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe68806e8a0 0x7fe688141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 >> 172.21.15.46:0/2609377206 conn(0x7fe6880cbe30 msgr2=0x7fe688068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 shutdown_connections 2023-10-06T11:33:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:53.964+0000 7fe68e345700 1 -- 172.21.15.46:0/2609377206 wait complete. 2023-10-06T11:33:53.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.144+0000 7fbbccc8e700 1 Processor -- start 2023-10-06T11:33:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.144+0000 7fbbccc8e700 1 -- start start 2023-10-06T11:33:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbccc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 0x7fbbc8067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbccc8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbbc81591a0 0x7fbbc8067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbccc8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc8069780 con 0x7fbbc8068bf0 2023-10-06T11:33:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbccc8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80699c0 con 0x7fbbc81591a0 2023-10-06T11:33:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbccc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815db50 con 0x7fbbc8158dd0 2023-10-06T11:33:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbbc8068bf0 0x7fbbc8069070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52282/0 (socket says 172.21.15.46:52282) 2023-10-06T11:33:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc7fff700 1 -- 172.21.15.46:0/394580155 learned_addr learned my addr 172.21.15.46:0/394580155 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 0x7fbbc8067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 --2- 172.21.15.46:0/394580155 >> v2:172.21.15.132:3300/0 conn(0x7fbbc81591a0 0x7fbbc8067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/394580155 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603175661 0 0) 0x7fbbc0001110 con 0x7fbbc8068bf0 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/394580155 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb4003570 con 0x7fbbc8068bf0 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 -- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 msgr2=0x7fbbc8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 0x7fbbc8067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 -- 172.21.15.46:0/394580155 >> v1:172.21.15.46:6789/0 conn(0x7fbbc8068bf0 legacy=0x7fbbc8069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 -- 172.21.15.46:0/394580155 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc815d6c0 con 0x7fbbc81591a0 2023-10-06T11:33:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 0x7fbbc8067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.148+0000 7fbbbeffd700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc81591a0 0x7fbbc8067590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00098e0 tx=0x7fbbb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a78a01140c9517e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/394580155 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0011630 con 0x7fbbc81591a0 2023-10-06T11:33:54.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/394580155 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0011790 con 0x7fbbc81591a0 2023-10-06T11:33:54.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbc6ffd700 1 -- 172.21.15.46:0/394580155 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0011b30 con 0x7fbbc81591a0 2023-10-06T11:33:54.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 -- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc81591a0 msgr2=0x7fbbc8067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc81591a0 0x7fbbc8067590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00098e0 tx=0x7fbbb00091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 -- 172.21.15.46:0/394580155 shutdown_connections 2023-10-06T11:33:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc81591a0 0x7fbbc8067590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/394580155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8158dd0 0x7fbbc8067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 -- 172.21.15.46:0/394580155 >> 172.21.15.46:0/394580155 conn(0x7fbbc8062ef0 msgr2=0x7fbbc80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 -- 172.21.15.46:0/394580155 shutdown_connections 2023-10-06T11:33:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.152+0000 7fbbccc8e700 1 -- 172.21.15.46:0/394580155 wait complete. 2023-10-06T11:33:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 Processor -- start 2023-10-06T11:33:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 -- start start 2023-10-06T11:33:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8068bf0 0x7fbbc8145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8158dd0 0x7fbbc815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80c5fa0 con 0x7fbbc81591a0 2023-10-06T11:33:54.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815b570 con 0x7fbbc8158dd0 2023-10-06T11:33:54.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbccc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815a960 con 0x7fbbc8068bf0 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54086/0 (socket says 172.21.15.46:54086) 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbbeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8158dd0 0x7fbbc815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 -- 172.21.15.46:0/1372951818 learned_addr learned my addr 172.21.15.46:0/1372951818 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8068bf0 0x7fbbc8145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 -- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8068bf0 msgr2=0x7fbbc8145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8068bf0 0x7fbbc8145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 -- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8158dd0 msgr2=0x7fbbc815ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8158dd0 0x7fbbc815ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.156+0000 7fbbc7fff700 1 -- 172.21.15.46:0/1372951818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009a20 con 0x7fbbc81591a0 2023-10-06T11:33:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbc7fff700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c790 tx=0x7fbbc0007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0005b30 con 0x7fbbc81591a0 2023-10-06T11:33:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0005c90 con 0x7fbbc81591a0 2023-10-06T11:33:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb0009420 con 0x7fbbc81591a0 2023-10-06T11:33:54.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00072e0 con 0x7fbbc81591a0 2023-10-06T11:33:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc815af20 con 0x7fbbc81591a0 2023-10-06T11:33:54.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc8064590 con 0x7fbbc81591a0 2023-10-06T11:33:54.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.160+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbbc001b3b0 con 0x7fbbc81591a0 2023-10-06T11:33:54.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.164+0000 7fbbbffff700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 0x7fbba8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.164+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fbbc007bae0 con 0x7fbbc81591a0 2023-10-06T11:33:54.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.164+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 0x7fbba8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.164+0000 7fbbc77fe700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 0x7fbba8062b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8157970 tx=0x7fbbb800b040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.168+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbbc0049be0 con 0x7fbbc81591a0 2023-10-06T11:33:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.336+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbc8069780 con 0x7fbbc81591a0 2023-10-06T11:33:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.336+0000 7fbbbffff700 1 -- 172.21.15.46:0/1372951818 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+14322 (secure 0 0 0) 0x7fbbc8069780 con 0x7fbbc81591a0 2023-10-06T11:33:54.347 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:54.347 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:53.494946+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 msgr2=0x7fbba8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 0x7fbba8062b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8157970 tx=0x7fbbb800b040 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 msgr2=0x7fbbc80bf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c790 tx=0x7fbbc0007c30 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 shutdown_connections 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbba80606a0 0x7fbba8062b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbc81591a0 0x7fbbc80bf1f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbc8158dd0 0x7fbbc815ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 --2- 172.21.15.46:0/1372951818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbbc8068bf0 0x7fbbc8145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 >> 172.21.15.46:0/1372951818 conn(0x7fbbc8062ef0 msgr2=0x7fbbc8158030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 shutdown_connections 2023-10-06T11:33:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.344+0000 7fbbccc8e700 1 -- 172.21.15.46:0/1372951818 wait complete. 2023-10-06T11:33:54.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:33:54.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:33:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.528+0000 7efcf56f0700 1 Processor -- start 2023-10-06T11:33:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.532+0000 7efcf56f0700 1 -- start start 2023-10-06T11:33:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.532+0000 7efcf56f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 0x7efcf00c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.532+0000 7efcf56f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf00c8940 con 0x7efcf0159460 2023-10-06T11:33:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf00c8330 con 0x7efcf0159090 2023-10-06T11:33:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcef7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efcf0159460 0x7efcf006d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52298/0 (socket says 172.21.15.46:52298) 2023-10-06T11:33:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcef7fe700 1 -- 172.21.15.46:0/1069269242 learned_addr learned my addr 172.21.15.46:0/1069269242 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00c8140 con 0x7efcf00678f0 2023-10-06T11:33:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcee7fc700 1 --2- 172.21.15.46:0/1069269242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 0x7efcf00c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490802307 0 0) 0x7efcf00c8940 con 0x7efcf0159460 2023-10-06T11:33:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003570 con 0x7efcf0159460 2023-10-06T11:33:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398656347 0 0) 0x7efcd8003570 con 0x7efcf0159460 2023-10-06T11:33:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcee7fc700 1 -- 172.21.15.46:0/1069269242 >> v1:172.21.15.132:6789/0 conn(0x7efcf0159090 legacy=0x7efcf0062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcee7fc700 1 -- 172.21.15.46:0/1069269242 >> v1:172.21.15.46:6789/0 conn(0x7efcf0159460 legacy=0x7efcf006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcee7fc700 1 -- 172.21.15.46:0/1069269242 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00680b0 con 0x7efcf00678f0 2023-10-06T11:33:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcee7fc700 1 --2- 172.21.15.46:0/1069269242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 0x7efcf00c5960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efce4009910 tx=0x7efce40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9dbd6c1d88bce0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce80017f0 con 0x7efcf00678f0 2023-10-06T11:33:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce4011420 con 0x7efcf00678f0 2023-10-06T11:33:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcedffb700 1 -- 172.21.15.46:0/1069269242 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40117e0 con 0x7efcf00678f0 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 msgr2=0x7efcf00c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 --2- 172.21.15.46:0/1069269242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 0x7efcf00c5960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efce4009910 tx=0x7efce40091a0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 shutdown_connections 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 --2- 172.21.15.46:0/1069269242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf00678f0 0x7efcf00c5960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 >> 172.21.15.46:0/1069269242 conn(0x7efcf00cd120 msgr2=0x7efcf00c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 shutdown_connections 2023-10-06T11:33:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.536+0000 7efcf56f0700 1 -- 172.21.15.46:0/1069269242 wait complete. 2023-10-06T11:33:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 Processor -- start 2023-10-06T11:33:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 -- start start 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf00678f0 0x7efcf00af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 0x7efcf00cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf00b2350 con 0x7efcf0159090 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf0143610 con 0x7efcf00678f0 2023-10-06T11:33:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcf56f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf015c380 con 0x7efcf0159460 2023-10-06T11:33:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcee7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcee7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54092/0 (socket says 172.21.15.46:54092) 2023-10-06T11:33:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcee7fc700 1 -- 172.21.15.46:0/771040340 learned_addr learned my addr 172.21.15.46:0/771040340 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efcef7fe700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 0x7efcf00cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.540+0000 7efceeffd700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf00678f0 0x7efcf00af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 -- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 msgr2=0x7efcf00cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 0x7efcf00cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 -- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf00678f0 msgr2=0x7efcf00af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf00678f0 0x7efcf00af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 -- 172.21.15.46:0/771040340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4009420 con 0x7efcf0159090 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcef7fe700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 0x7efcf00cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcee7fc700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efce4002000 tx=0x7efce4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce401caf0 con 0x7efcf0159090 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce401cc90 con 0x7efcf0159090 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce401a6a0 con 0x7efcf0159090 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf00b1d00 con 0x7efcf0159090 2023-10-06T11:33:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf00b19f0 con 0x7efcf0159090 2023-10-06T11:33:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.544+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf00b24c0 con 0x7efcf0159090 2023-10-06T11:33:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.548+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7efce401aa10 con 0x7efcf0159090 2023-10-06T11:33:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.548+0000 7efcd6ffd700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 0x7efcdc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.548+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7efce407de40 con 0x7efcf0159090 2023-10-06T11:33:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.548+0000 7efceeffd700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 0x7efcdc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.552+0000 7efceeffd700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 0x7efcdc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efce0003a80 tx=0x7efce0007ec0 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.552+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efce404e360 con 0x7efcf0159090 2023-10-06T11:33:54.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcf004b5c0 con 0x7efcf0159090 2023-10-06T11:33:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcd6ffd700 1 -- 172.21.15.46:0/771040340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7efce4051980 con 0x7efcf0159090 2023-10-06T11:33:54.732 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:33:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 msgr2=0x7efcdc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 0x7efcdc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efce0003a80 tx=0x7efce0007ec0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 msgr2=0x7efcf01430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7efce4002000 tx=0x7efce4016000 comp rx=0 tx=0).stop 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 shutdown_connections 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efcdc060670 0x7efcdc062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcf0159460 0x7efcf00cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efcf0159090 0x7efcf01430d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 --2- 172.21.15.46:0/771040340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcf00678f0 0x7efcf00af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.724+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 >> 172.21.15.46:0/771040340 conn(0x7efcf00cd120 msgr2=0x7efcf0062d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.728+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 shutdown_connections 2023-10-06T11:33:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.728+0000 7efcf56f0700 1 -- 172.21.15.46:0/771040340 wait complete. 2023-10-06T11:33:54.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:33:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 Processor -- start 2023-10-06T11:33:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 -- start start 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff19c0cfb80 0x7ff19c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0ce6e0 con 0x7ff19c069270 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0cddb0 con 0x7ff19c1591b0 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a1323700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c15a5c0 con 0x7ff19c0cfb80 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff19bfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff19c069270 0x7ff19c158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52312/0 (socket says 172.21.15.46:52312) 2023-10-06T11:33:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff19bfff700 1 -- 172.21.15.46:0/1258157884 learned_addr learned my addr 172.21.15.46:0/1258157884 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff1a0b22700 1 --2- 172.21.15.46:0/1258157884 >> v2:172.21.15.132:3300/0 conn(0x7ff19c0cfb80 0x7ff19c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770572065 0 0) 0x7ff19c0ce6e0 con 0x7ff19c069270 2023-10-06T11:33:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.916+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff188003570 con 0x7ff19c069270 2023-10-06T11:33:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a0b22700 1 -- 172.21.15.46:0/1258157884 >> v1:172.21.15.132:6789/0 conn(0x7ff19c1591b0 legacy=0x7ff19c159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a0b22700 1 -- 172.21.15.46:0/1258157884 >> v1:172.21.15.46:6789/0 conn(0x7ff19c069270 legacy=0x7ff19c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171831435 0 0) 0x7ff188003570 con 0x7ff19c069270 2023-10-06T11:33:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a0b22700 1 -- 172.21.15.46:0/1258157884 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0ce090 con 0x7ff19c0cfb80 2023-10-06T11:33:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a0b22700 1 --2- 172.21.15.46:0/1258157884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c069640 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff194009de0 tx=0x7ff194009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168ba99c7c2a7584 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19400e070 con 0x7ff19c0cfb80 2023-10-06T11:33:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19400b850 con 0x7ff19c0cfb80 2023-10-06T11:33:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff19b7fe700 1 -- 172.21.15.46:0/1258157884 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c001470 con 0x7ff19c0cfb80 2023-10-06T11:33:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 -- 172.21.15.46:0/1258157884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 msgr2=0x7ff19c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 --2- 172.21.15.46:0/1258157884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c069640 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff194009de0 tx=0x7ff194009780 comp rx=0 tx=0).stop 2023-10-06T11:33:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 -- 172.21.15.46:0/1258157884 shutdown_connections 2023-10-06T11:33:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 --2- 172.21.15.46:0/1258157884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c069640 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 -- 172.21.15.46:0/1258157884 >> 172.21.15.46:0/1258157884 conn(0x7ff19c068390 msgr2=0x7ff19c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 -- 172.21.15.46:0/1258157884 shutdown_connections 2023-10-06T11:33:54.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.920+0000 7ff1a1323700 1 -- 172.21.15.46:0/1258157884 wait complete. 2023-10-06T11:33:54.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 Processor -- start 2023-10-06T11:33:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 -- start start 2023-10-06T11:33:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff19c1591b0 0x7ff19c16aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c90f0 con 0x7ff19c069270 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c158540 con 0x7ff19c0cfb80 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a1323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c16a130 con 0x7ff19c1591b0 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54100/0 (socket says 172.21.15.46:54100) 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 -- 172.21.15.46:0/3819104916 learned_addr learned my addr 172.21.15.46:0/3819104916 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff1a0b22700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff19c1591b0 0x7ff19c16aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff193fff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 -- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff19c1591b0 msgr2=0x7ff19c16aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff19c1591b0 0x7ff19c16aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 -- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 msgr2=0x7ff19c169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff19bfff700 1 -- 172.21.15.46:0/3819104916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194009a20 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.924+0000 7ff193fff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff19bfff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00cb40 tx=0x7ff18c001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c007360 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c0074e0 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c16a410 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c004830 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c169db0 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.928+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c064590 con 0x7ff19c069270 2023-10-06T11:33:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.932+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff18c004bc0 con 0x7ff19c069270 2023-10-06T11:33:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.932+0000 7ff198ff9700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 0x7ff17c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.932+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7ff18c07bea0 con 0x7ff19c069270 2023-10-06T11:33:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.932+0000 7ff193fff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 0x7ff17c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.932+0000 7ff193fff700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 0x7ff17c062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0b5880 tx=0x7ff184009040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:54.940+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff18c04c390 con 0x7ff19c069270 2023-10-06T11:33:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.108+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff19c0ce370 con 0x7ff19c069270 2023-10-06T11:33:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.108+0000 7ff198ff9700 1 -- 172.21.15.46:0/3819104916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff18c018020 con 0x7ff19c069270 2023-10-06T11:33:55.117 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 msgr2=0x7ff17c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 0x7ff17c062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0b5880 tx=0x7ff184009040 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 msgr2=0x7ff19c158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00cb40 tx=0x7ff18c001bd0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 shutdown_connections 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff17c060670 0x7ff17c062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff19c1591b0 0x7ff19c16aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff19c0cfb80 0x7ff19c169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 --2- 172.21.15.46:0/3819104916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff19c069270 0x7ff19c158000 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 >> 172.21.15.46:0/3819104916 conn(0x7ff19c068390 msgr2=0x7ff19c155a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 shutdown_connections 2023-10-06T11:33:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.112+0000 7ff1a1323700 1 -- 172.21.15.46:0/3819104916 wait complete. 2023-10-06T11:33:55.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:33:55.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 Processor -- start 2023-10-06T11:33:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 -- start start 2023-10-06T11:33:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02a00bedd0 0x7f02a0067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 0x7f02a0067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a0069660 con 0x7f02a00bf1a0 2023-10-06T11:33:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a0068ac0 con 0x7f02a00bedd0 2023-10-06T11:33:55.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.291+0000 7f02a624c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a0068d30 con 0x7f02a00c8110 2023-10-06T11:33:55.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02a00bedd0 0x7f02a0067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02a00bedd0 0x7f02a0067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54110/0 (socket says 172.21.15.46:54110) 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 -- 172.21.15.46:0/898224444 learned_addr learned my addr 172.21.15.46:0/898224444 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 -- 172.21.15.46:0/898224444 >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 msgr2=0x7f02a0067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a5a4b700 1 --2- 172.21.15.46:0/898224444 >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 0x7f02a0067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 --2- 172.21.15.46:0/898224444 >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 0x7f02a0067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 -- 172.21.15.46:0/898224444 >> v1:172.21.15.132:6789/0 conn(0x7f02a00bf1a0 legacy=0x7f02a00c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 -- 172.21.15.46:0/898224444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0069010 con 0x7f02a00bedd0 2023-10-06T11:33:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a5a4b700 1 --2- 172.21.15.46:0/898224444 >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 0x7f02a0067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a524a700 1 --2- 172.21.15.46:0/898224444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00bedd0 0x7f02a0067050 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f02a00ae6e0 tx=0x7f029c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92cc650b07035ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f0297fff700 1 -- 172.21.15.46:0/898224444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e070 con 0x7f02a00bedd0 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f0297fff700 1 -- 172.21.15.46:0/898224444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00ba50 con 0x7f02a00bedd0 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f0297fff700 1 -- 172.21.15.46:0/898224444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0092e0 con 0x7f02a00bedd0 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 -- 172.21.15.46:0/898224444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00bedd0 msgr2=0x7f02a0067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 --2- 172.21.15.46:0/898224444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00bedd0 0x7f02a0067050 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f02a00ae6e0 tx=0x7f029c009840 comp rx=0 tx=0).stop 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 -- 172.21.15.46:0/898224444 shutdown_connections 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 --2- 172.21.15.46:0/898224444 >> v2:172.21.15.132:3300/0 conn(0x7f02a00c8110 0x7f02a0067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 --2- 172.21.15.46:0/898224444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00bedd0 0x7f02a0067050 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 -- 172.21.15.46:0/898224444 >> 172.21.15.46:0/898224444 conn(0x7f02a0062ef0 msgr2=0x7f02a00cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 -- 172.21.15.46:0/898224444 shutdown_connections 2023-10-06T11:33:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.295+0000 7f02a624c700 1 -- 172.21.15.46:0/898224444 wait complete. 2023-10-06T11:33:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 Processor -- start 2023-10-06T11:33:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 -- start start 2023-10-06T11:33:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 0x7f02a0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02a00bf1a0 0x7f02a0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a0156bc0 con 0x7f02a00c8110 2023-10-06T11:33:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01605f0 con 0x7f02a00bedd0 2023-10-06T11:33:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a624c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00ae960 con 0x7f02a00bf1a0 2023-10-06T11:33:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a5a4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.299+0000 7f02a524a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 0x7f02a0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a524a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 0x7f02a0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:46370/0 (socket says 172.21.15.46:46370) 2023-10-06T11:33:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54116/0 (socket says 172.21.15.46:54116) 2023-10-06T11:33:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 -- 172.21.15.46:0/3509091295 learned_addr learned my addr 172.21.15.46:0/3509091295 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a4a49700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02a00bf1a0 0x7f02a0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 -- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02a00bf1a0 msgr2=0x7f02a0155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02a00bf1a0 0x7f02a0155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 -- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 msgr2=0x7f02a0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 0x7f02a0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 -- 172.21.15.46:0/3509091295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009ae0 con 0x7f02a00c8110 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a5a4b700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f029800c880 tx=0x7f0298008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298011600 con 0x7f02a00c8110 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298011760 con 0x7f02a00c8110 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a00af1c0 con 0x7f02a00c8110 2023-10-06T11:33:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298011b20 con 0x7f02a00c8110 2023-10-06T11:33:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a00aeb70 con 0x7f02a00c8110 2023-10-06T11:33:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.303+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a00af350 con 0x7f02a00c8110 2023-10-06T11:33:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.307+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f02980193b0 con 0x7f02a00c8110 2023-10-06T11:33:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.307+0000 7f02957fa700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 0x7f0284062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.307+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f029807c670 con 0x7f02a00c8110 2023-10-06T11:33:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.307+0000 7f02a524a700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 0x7f0284062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.311+0000 7f02a524a700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 0x7f0284062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f029c009880 tx=0x7f029c018000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.315+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0298050510 con 0x7f02a00c8110 2023-10-06T11:33:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.487+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02a0064590 con 0x7f02a00c8110 2023-10-06T11:33:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02957fa700 1 -- 172.21.15.46:0/3509091295 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+126 (secure 0 0 0) 0x7f02980500a0 con 0x7f02a00c8110 2023-10-06T11:33:55.502 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:55.502 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:33:55.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:33:55.503 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 msgr2=0x7f0284062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 0x7f0284062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f029c009880 tx=0x7f029c018000 comp rx=0 tx=0).stop 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 msgr2=0x7f02a015f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f029800c880 tx=0x7f0298008040 comp rx=0 tx=0).stop 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 shutdown_connections 2023-10-06T11:33:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0284060670 0x7f0284062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02a00c8110 0x7f02a015f930 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02a00bf1a0 0x7f02a0155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 --2- 172.21.15.46:0/3509091295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02a00bedd0 0x7f02a0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.491+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 >> 172.21.15.46:0/3509091295 conn(0x7f02a0062ef0 msgr2=0x7f02a00cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.495+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 shutdown_connections 2023-10-06T11:33:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.495+0000 7f02a624c700 1 -- 172.21.15.46:0/3509091295 wait complete. 2023-10-06T11:33:55.524 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:33:55.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:33:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.687+0000 7f5a9a26e700 1 Processor -- start 2023-10-06T11:33:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.687+0000 7f5a9a26e700 1 -- start start 2023-10-06T11:33:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9a26e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b2600 0x7f5a940ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9a26e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a940cae50 0x7f5a940c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9a26e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940cbff0 con 0x7f5a940b29d0 2023-10-06T11:33:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9a26e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a941598f0 con 0x7f5a940cae50 2023-10-06T11:33:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9a26e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940cb3e0 con 0x7f5a940b2600 2023-10-06T11:33:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9880b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a940cae50 0x7f5a940c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a937fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5a940b29d0 0x7f5a940ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52314/0 (socket says 172.21.15.46:52314) 2023-10-06T11:33:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a937fe700 1 -- 172.21.15.46:0/3119050669 learned_addr learned my addr 172.21.15.46:0/3119050669 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a93fff700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b2600 0x7f5a940ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9880b700 1 -- 172.21.15.46:0/3119050669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b2600 msgr2=0x7f5a940ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9880b700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b2600 0x7f5a940ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9880b700 1 -- 172.21.15.46:0/3119050669 >> v1:172.21.15.46:6789/0 conn(0x7f5a940b29d0 legacy=0x7f5a940ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a9880b700 1 -- 172.21.15.46:0/3119050669 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940cb6c0 con 0x7f5a940cae50 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.691+0000 7f5a92ffd700 1 -- 172.21.15.46:0/3119050669 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636310437 0 0) 0x7f5a940cbff0 con 0x7f5a940b29d0 2023-10-06T11:33:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9880b700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940cae50 0x7f5a940c4ff0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009e60 tx=0x7f5a8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca262afdf7409ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a92ffd700 1 -- 172.21.15.46:0/3119050669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00e070 con 0x7f5a940cae50 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a92ffd700 1 -- 172.21.15.46:0/3119050669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00bb60 con 0x7f5a940cae50 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a92ffd700 1 -- 172.21.15.46:0/3119050669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c012430 con 0x7f5a940cae50 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 -- 172.21.15.46:0/3119050669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940cae50 msgr2=0x7f5a940c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940cae50 0x7f5a940c4ff0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009e60 tx=0x7f5a8c009780 comp rx=0 tx=0).stop 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 -- 172.21.15.46:0/3119050669 shutdown_connections 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940cae50 0x7f5a940c4ff0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/3119050669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b2600 0x7f5a940ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 -- 172.21.15.46:0/3119050669 >> 172.21.15.46:0/3119050669 conn(0x7f5a9406ef60 msgr2=0x7f5a940d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 -- 172.21.15.46:0/3119050669 shutdown_connections 2023-10-06T11:33:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 -- 172.21.15.46:0/3119050669 wait complete. 2023-10-06T11:33:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.695+0000 7f5a9a26e700 1 Processor -- start 2023-10-06T11:33:55.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 -- start start 2023-10-06T11:33:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b29d0 0x7f5a9414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a940cae50 0x7f5a9415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414c8a0 con 0x7f5a940b2600 2023-10-06T11:33:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940c7650 con 0x7f5a940cae50 2023-10-06T11:33:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9a26e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940c7930 con 0x7f5a940b29d0 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54124/0 (socket says 172.21.15.46:54124) 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 -- 172.21.15.46:0/769441466 learned_addr learned my addr 172.21.15.46:0/769441466 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a9880b700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a940cae50 0x7f5a9415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a937fe700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b29d0 0x7f5a9414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 -- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b29d0 msgr2=0x7f5a9414c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b29d0 0x7f5a9414c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 -- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a940cae50 msgr2=0x7f5a9415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a940cae50 0x7f5a9415f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:55.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.699+0000 7f5a93fff700 1 -- 172.21.15.46:0/769441466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c009a20 con 0x7f5a940b2600 2023-10-06T11:33:55.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a93fff700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400ea00 tx=0x7f5a84009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400c6a0 con 0x7f5a940b2600 2023-10-06T11:33:55.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8400c800 con 0x7f5a940b2600 2023-10-06T11:33:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400cb80 con 0x7f5a940b2600 2023-10-06T11:33:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a940c7c10 con 0x7f5a940b2600 2023-10-06T11:33:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a940c81d0 con 0x7f5a940b2600 2023-10-06T11:33:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.703+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a94069b50 con 0x7f5a940b2600 2023-10-06T11:33:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.707+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5a84013470 con 0x7f5a940b2600 2023-10-06T11:33:55.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.707+0000 7f5a7bfff700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 0x7f5a80062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.707+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f5a8407cdc0 con 0x7f5a940b2600 2023-10-06T11:33:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.707+0000 7f5a937fe700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 0x7f5a80062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:55.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.707+0000 7f5a937fe700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 0x7f5a80062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a9409e150 tx=0x7f5a88007d30 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.715+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5a8404d2d0 con 0x7f5a940b2600 2023-10-06T11:33:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:55.887+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5a940cb9a0 con 0x7f5a940b2600 2023-10-06T11:33:56.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.143+0000 7f5a7bfff700 1 -- 172.21.15.46:0/769441466 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5a840170b0 con 0x7f5a940b2600 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 msgr2=0x7f5a80062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 0x7f5a80062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a9409e150 tx=0x7f5a88007d30 comp rx=0 tx=0).stop 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 msgr2=0x7f5a940b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400ea00 tx=0x7f5a84009120 comp rx=0 tx=0).stop 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 shutdown_connections 2023-10-06T11:33:56.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a800606a0 0x7f5a80062b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a940cae50 0x7f5a9415f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a940b29d0 0x7f5a9414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 --2- 172.21.15.46:0/769441466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a940b2600 0x7f5a940b5430 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 >> 172.21.15.46:0/769441466 conn(0x7f5a9406ef60 msgr2=0x7f5a940c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 shutdown_connections 2023-10-06T11:33:56.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.147+0000 7f5a9a26e700 1 -- 172.21.15.46:0/769441466 wait complete. 2023-10-06T11:33:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:33:56.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:33:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.347+0000 7f1252923700 1 Processor -- start 2023-10-06T11:33:56.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.347+0000 7f1252923700 1 -- start start 2023-10-06T11:33:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f124c067760 0x7f124c1683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c161d40 con 0x7f124c0bedd0 2023-10-06T11:33:56.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c161130 con 0x7f124c0bf1a0 2023-10-06T11:33:56.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c15da30 con 0x7f124c067760 2023-10-06T11:33:56.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f124c067760 0x7f124c1683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:56.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f124c067760 0x7f124c1683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54136/0 (socket says 172.21.15.46:54136) 2023-10-06T11:33:56.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 -- 172.21.15.46:0/3120001878 learned_addr learned my addr 172.21.15.46:0/3120001878 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:56.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 -- 172.21.15.46:0/3120001878 >> v1:172.21.15.132:6789/0 conn(0x7f124c0bf1a0 legacy=0x7f124c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 -- 172.21.15.46:0/3120001878 >> v1:172.21.15.46:6789/0 conn(0x7f124c0bedd0 legacy=0x7f124c06e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 -- 172.21.15.46:0/3120001878 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c161410 con 0x7f124c067760 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252122700 1 --2- 172.21.15.46:0/3120001878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c067760 0x7f124c1683d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1244009f20 tx=0x7f1244009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1562866a1943703a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f125091f700 1 -- 172.21.15.46:0/3120001878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400e070 con 0x7f124c067760 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f125091f700 1 -- 172.21.15.46:0/3120001878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400ba40 con 0x7f124c067760 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f125091f700 1 -- 172.21.15.46:0/3120001878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440092e0 con 0x7f124c067760 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 -- 172.21.15.46:0/3120001878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c067760 msgr2=0x7f124c1683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.351+0000 7f1252923700 1 --2- 172.21.15.46:0/3120001878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c067760 0x7f124c1683d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1244009f20 tx=0x7f1244009840 comp rx=0 tx=0).stop 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- 172.21.15.46:0/3120001878 shutdown_connections 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 --2- 172.21.15.46:0/3120001878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c067760 0x7f124c1683d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- 172.21.15.46:0/3120001878 >> 172.21.15.46:0/3120001878 conn(0x7f124c0cbbb0 msgr2=0x7f124c0cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:56.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- 172.21.15.46:0/3120001878 shutdown_connections 2023-10-06T11:33:56.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- 172.21.15.46:0/3120001878 wait complete. 2023-10-06T11:33:56.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 Processor -- start 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- start start 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f124c067760 0x7f124c0be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f124c0bedd0 0x7f124c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0be600 con 0x7f124c0bf1a0 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c14c570 con 0x7f124c067760 2023-10-06T11:33:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15e9b0 con 0x7f124c0bedd0 2023-10-06T11:33:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.355+0000 7f1252122700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54152/0 (socket says 172.21.15.46:54152) 2023-10-06T11:33:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 -- 172.21.15.46:0/4216885249 learned_addr learned my addr 172.21.15.46:0/4216885249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1251921700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f124c067760 0x7f124c0be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:56.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1251120700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f124c0bedd0 0x7f124c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 -- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f124c0bedd0 msgr2=0x7f124c14c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f124c0bedd0 0x7f124c14c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 -- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f124c067760 msgr2=0x7f124c0be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f124c067760 0x7f124c0be0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 -- 172.21.15.46:0/4216885249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009ae0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252122700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1244003000 tx=0x7f1244017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:56.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440094e0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12440092e0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124401d420 con 0x7f124c0bf1a0 2023-10-06T11:33:56.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c0bd5e0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.359+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c0bdba0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.363+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c0bdd90 con 0x7f124c0bf1a0 2023-10-06T11:33:56.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.363+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f124401d810 con 0x7f124c0bf1a0 2023-10-06T11:33:56.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.367+0000 7f1241ffb700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 0x7f1230062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:56.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.367+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6020+0+0 (secure 0 0 0) 0x7f124407e7d0 con 0x7f124c0bf1a0 2023-10-06T11:33:56.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.367+0000 7f1251921700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 0x7f1230062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:56.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.367+0000 7f1251921700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 0x7f1230062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1248003a80 tx=0x7f1248007ec0 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:56.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.371+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f124404ea40 con 0x7f124c0bf1a0 2023-10-06T11:33:56.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:56.543+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f124c064590 con 0x7f124c0bf1a0 2023-10-06T11:33:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.163+0000 7f1241ffb700 1 -- 172.21.15.46:0/4216885249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1244052060 con 0x7f124c0bf1a0 2023-10-06T11:33:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.171+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 msgr2=0x7f1230062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.171+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 0x7f1230062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1248003a80 tx=0x7f1248007ec0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.171+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 msgr2=0x7f124c0bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.171+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1244003000 tx=0x7f1244017000 comp rx=0 tx=0).stop 2023-10-06T11:33:57.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 shutdown_connections 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1230060670 0x7f1230062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f124c0bf1a0 0x7f124c0bfe60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f124c0bedd0 0x7f124c14c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 --2- 172.21.15.46:0/4216885249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f124c067760 0x7f124c0be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 >> 172.21.15.46:0/4216885249 conn(0x7f124c0cbbb0 msgr2=0x7f124c15acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 shutdown_connections 2023-10-06T11:33:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.175+0000 7f1252923700 1 -- 172.21.15.46:0/4216885249 wait complete. 2023-10-06T11:33:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:33:57.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:33:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.363+0000 7fe2f0a6b700 1 Processor -- start 2023-10-06T11:33:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.363+0000 7fe2f0a6b700 1 -- start start 2023-10-06T11:33:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.363+0000 7fe2f0a6b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2ec159700 0x7fe2ec0ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.363+0000 7fe2f0a6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.363+0000 7fe2f0a6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec0cbbe0 con 0x7fe2ec156c90 2023-10-06T11:33:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2f0a6b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec06a230 con 0x7fe2ec159700 2023-10-06T11:33:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2f0a6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec0cb060 con 0x7fe2ec0cac20 2023-10-06T11:33:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2ec159700 0x7fe2ec0ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2ec159700 0x7fe2ec0ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:46404/0 (socket says 172.21.15.46:46404) 2023-10-06T11:33:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 -- 172.21.15.46:0/3184309719 learned_addr learned my addr 172.21.15.46:0/3184309719 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2ebfff700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 -- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 msgr2=0x7fe2ec0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:57.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 -- 172.21.15.46:0/3184309719 >> v1:172.21.15.132:6789/0 conn(0x7fe2ec156c90 legacy=0x7fe2ec0c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 -- 172.21.15.46:0/3184309719 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec157060 con 0x7fe2ec159700 2023-10-06T11:33:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2eaffd700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec159700 0x7fe2ec0ca6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00099b0 tx=0x7fe2e0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7995b9fec52b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2ea7fc700 1 -- 172.21.15.46:0/3184309719 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009ea0 con 0x7fe2ec159700 2023-10-06T11:33:57.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2ea7fc700 1 -- 172.21.15.46:0/3184309719 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e000b860 con 0x7fe2ec159700 2023-10-06T11:33:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2ea7fc700 1 -- 172.21.15.46:0/3184309719 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000bc00 con 0x7fe2ec159700 2023-10-06T11:33:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec159700 msgr2=0x7fe2ec0ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.367+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec159700 0x7fe2ec0ca6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00099b0 tx=0x7fe2e0009250 comp rx=0 tx=0).stop 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/3184309719 shutdown_connections 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/3184309719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec159700 0x7fe2ec0ca6e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/3184309719 >> 172.21.15.46:0/3184309719 conn(0x7fe2ec097060 msgr2=0x7fe2ec067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/3184309719 shutdown_connections 2023-10-06T11:33:57.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/3184309719 wait complete. 2023-10-06T11:33:57.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 Processor -- start 2023-10-06T11:33:57.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.371+0000 7fe2f0a6b700 1 -- start start 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 0x7fe2ec0c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec16cde0 con 0x7fe2ec159700 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec16d0c0 con 0x7fe2ec156c90 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec16d3a0 con 0x7fe2ec0cac20 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54154/0 (socket says 172.21.15.46:54154) 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 -- 172.21.15.46:0/1990620183 learned_addr learned my addr 172.21.15.46:0/1990620183 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:57.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2eaffd700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 0x7fe2ec0c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2eb7fe700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 -- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 msgr2=0x7fe2ec154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 -- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 msgr2=0x7fe2ec0c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 0x7fe2ec0c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 -- 172.21.15.46:0/1990620183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00094f0 con 0x7fe2ec159700 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2eaffd700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 0x7fe2ec0c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2ebfff700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009e90 tx=0x7fe2e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400cb70 con 0x7fe2ec159700 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e400f040 con 0x7fe2ec159700 2023-10-06T11:33:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec16d680 con 0x7fe2ec159700 2023-10-06T11:33:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4014480 con 0x7fe2ec159700 2023-10-06T11:33:57.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.375+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec0c1ef0 con 0x7fe2ec159700 2023-10-06T11:33:57.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.379+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec0b0c50 con 0x7fe2ec159700 2023-10-06T11:33:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.383+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe2e4014830 con 0x7fe2ec159700 2023-10-06T11:33:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.383+0000 7fe2d37fe700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 0x7fe2d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:57.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.383+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6402+0+0 (secure 0 0 0) 0x7fe2e407dcf0 con 0x7fe2ec159700 2023-10-06T11:33:57.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.383+0000 7fe2eb7fe700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 0x7fe2d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:57.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.387+0000 7fe2eb7fe700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 0x7fe2d8062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc007fd0 tx=0x7fe2dc007eb0 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:57.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.387+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe2e404dde0 con 0x7fe2ec159700 2023-10-06T11:33:57.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:57.599+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe2ec0b0c50 con 0x7fe2ec159700 2023-10-06T11:33:58.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.479+0000 7fe2d37fe700 1 -- 172.21.15.46:0/1990620183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe2ec0b0c50 con 0x7fe2ec159700 2023-10-06T11:33:58.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.483+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 msgr2=0x7fe2d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.483+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 0x7fe2d8062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc007fd0 tx=0x7fe2dc007eb0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.483+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 msgr2=0x7fe2ec0c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.483+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009e90 tx=0x7fe2e400b040 comp rx=0 tx=0).stop 2023-10-06T11:33:58.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 shutdown_connections 2023-10-06T11:33:58.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe2d8060670 0x7fe2d8062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2ec159700 0x7fe2ec0c6690 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec156c90 0x7fe2ec0c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 --2- 172.21.15.46:0/1990620183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec0cac20 0x7fe2ec154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 >> 172.21.15.46:0/1990620183 conn(0x7fe2ec097060 msgr2=0x7fe2ec156250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 shutdown_connections 2023-10-06T11:33:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.487+0000 7fe2f0a6b700 1 -- 172.21.15.46:0/1990620183 wait complete. 2023-10-06T11:33:58.503 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T11:33:58.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:58.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 Processor -- start 2023-10-06T11:33:58.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 -- start start 2023-10-06T11:33:58.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06fc159050 0x7f06fc159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f06fc0d0aa0 0x7f06fc0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0b4f80 con 0x7f06fc0b39b0 2023-10-06T11:33:58.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0b4370 con 0x7f06fc159050 2023-10-06T11:33:58.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0703945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0b4650 con 0x7f06fc0d0aa0 2023-10-06T11:33:58.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f07016e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06fc159050 0x7f06fc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f07016e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06fc159050 0x7f06fc159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54168/0 (socket says 172.21.15.46:54168) 2023-10-06T11:33:58.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f07016e1700 1 -- 172.21.15.46:0/1684052124 learned_addr learned my addr 172.21.15.46:0/1684052124 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:58.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f0700ee0700 1 --2- 172.21.15.46:0/1684052124 >> v2:172.21.15.132:3300/0 conn(0x7f06fc0d0aa0 0x7f06fc0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f07016e1700 1 -- 172.21.15.46:0/1684052124 >> v2:172.21.15.132:3300/0 conn(0x7f06fc0d0aa0 msgr2=0x7f06fc0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:58.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.675+0000 7f07016e1700 1 --2- 172.21.15.46:0/1684052124 >> v2:172.21.15.132:3300/0 conn(0x7f06fc0d0aa0 0x7f06fc0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f07016e1700 1 -- 172.21.15.46:0/1684052124 >> v1:172.21.15.132:6789/0 conn(0x7f06fc0b39b0 legacy=0x7f06fc0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f07016e1700 1 -- 172.21.15.46:0/1684052124 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0b4930 con 0x7f06fc159050 2023-10-06T11:33:58.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f07016e1700 1 --2- 172.21.15.46:0/1684052124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0892d0 tx=0x7f06f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0223ecf103778c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f06f3fff700 1 -- 172.21.15.46:0/1684052124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800b860 con 0x7f06fc159050 2023-10-06T11:33:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f06f3fff700 1 -- 172.21.15.46:0/1684052124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800b9c0 con 0x7f06fc159050 2023-10-06T11:33:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f06f3fff700 1 -- 172.21.15.46:0/1684052124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800bd60 con 0x7f06fc159050 2023-10-06T11:33:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 -- 172.21.15.46:0/1684052124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 msgr2=0x7f06fc159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 --2- 172.21.15.46:0/1684052124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0892d0 tx=0x7f06f8009250 comp rx=0 tx=0).stop 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 -- 172.21.15.46:0/1684052124 shutdown_connections 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 --2- 172.21.15.46:0/1684052124 >> v2:172.21.15.132:3300/0 conn(0x7f06fc0d0aa0 0x7f06fc0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 --2- 172.21.15.46:0/1684052124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 -- 172.21.15.46:0/1684052124 >> 172.21.15.46:0/1684052124 conn(0x7f06fc06e780 msgr2=0x7f06fc06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 -- 172.21.15.46:0/1684052124 shutdown_connections 2023-10-06T11:33:58.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.679+0000 7f0703945700 1 -- 172.21.15.46:0/1684052124 wait complete. 2023-10-06T11:33:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.683+0000 7f0703945700 1 Processor -- start 2023-10-06T11:33:58.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.683+0000 7f0703945700 1 -- start start 2023-10-06T11:33:58.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06fc0b39b0 0x7f06fc1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06fc0d0aa0 0x7f06fc1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc155800 con 0x7f06fc159050 2023-10-06T11:33:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc15a530 con 0x7f06fc0d0aa0 2023-10-06T11:33:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc15a810 con 0x7f06fc0b39b0 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54170/0 (socket says 172.21.15.46:54170) 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0700ee0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06fc0d0aa0 0x7f06fc1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 -- 172.21.15.46:0/3642221097 learned_addr learned my addr 172.21.15.46:0/3642221097 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f07016e1700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06fc0b39b0 0x7f06fc1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 -- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06fc0b39b0 msgr2=0x7f06fc1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06fc0b39b0 0x7f06fc1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 -- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06fc0d0aa0 msgr2=0x7f06fc1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06fc0d0aa0 0x7f06fc1599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 -- 172.21.15.46:0/3642221097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80094f0 con 0x7f06fc159050 2023-10-06T11:33:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0701ee2700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c8e0 tx=0x7f06f4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4012630 con 0x7f06fc159050 2023-10-06T11:33:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4012790 con 0x7f06fc159050 2023-10-06T11:33:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc15aaf0 con 0x7f06fc159050 2023-10-06T11:33:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4012b50 con 0x7f06fc159050 2023-10-06T11:33:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.687+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc15b0b0 con 0x7f06fc159050 2023-10-06T11:33:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.691+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc15b2d0 con 0x7f06fc159050 2023-10-06T11:33:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.691+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f06f401a3a0 con 0x7f06fc159050 2023-10-06T11:33:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.691+0000 7f06f17fa700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 0x7f06e0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.695+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f06f407ce70 con 0x7f06fc159050 2023-10-06T11:33:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.695+0000 7f07016e1700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 0x7f06e0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.695+0000 7f07016e1700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 0x7f06e0062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0892d0 tx=0x7f06f8017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.699+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f06f4050a20 con 0x7f06fc159050 2023-10-06T11:33:58.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.871+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06fc0b4c10 con 0x7f06fc159050 2023-10-06T11:33:58.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.871+0000 7f06f17fa700 1 -- 172.21.15.46:0/3642221097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17553 (secure 0 0 0) 0x7f06f40505b0 con 0x7f06fc159050 2023-10-06T11:33:58.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:58.884 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":92,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:57.602420+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:58.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.875+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 msgr2=0x7f06e0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 0x7f06e0062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0892d0 tx=0x7f06f8017000 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 msgr2=0x7f06fc159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c8e0 tx=0x7f06f4001710 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 shutdown_connections 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f06e0060670 0x7f06e0062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06fc159050 0x7f06fc159f00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06fc0d0aa0 0x7f06fc1599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 --2- 172.21.15.46:0/3642221097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06fc0b39b0 0x7f06fc1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:58.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 >> 172.21.15.46:0/3642221097 conn(0x7f06fc06e780 msgr2=0x7f06fc0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 shutdown_connections 2023-10-06T11:33:58.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:58.879+0000 7f0703945700 1 -- 172.21.15.46:0/3642221097 wait complete. 2023-10-06T11:33:58.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:33:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 Processor -- start 2023-10-06T11:33:59.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 -- start start 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5aa00af000 0x7f5aa00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 0x7f5aa00690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa00cf100 con 0x7f5aa00cb4c0 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa00ce300 con 0x7f5aa00af000 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa6ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ce4f0 con 0x7f5aa00af3d0 2023-10-06T11:33:59.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa5ce1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5aa00af000 0x7f5aa00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa5ce1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5aa00af000 0x7f5aa00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:46442/0 (socket says 172.21.15.46:46442) 2023-10-06T11:33:59.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa5ce1700 1 -- 172.21.15.46:0/2209711404 learned_addr learned my addr 172.21.15.46:0/2209711404 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:59.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.059+0000 7f5aa64e2700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 0x7f5aa00690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa5ce1700 1 -- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 msgr2=0x7f5aa00690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa5ce1700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 0x7f5aa00690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa5ce1700 1 -- 172.21.15.46:0/2209711404 >> v1:172.21.15.132:6789/0 conn(0x7f5aa00cb4c0 legacy=0x7f5aa00cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa5ce1700 1 -- 172.21.15.46:0/2209711404 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00ce7d0 con 0x7f5aa00af000 2023-10-06T11:33:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa64e2700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 0x7f5aa00690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa5ce1700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00cf590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099b0 tx=0x7f5a9c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbb70f9590112fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa4cdf700 1 -- 172.21.15.46:0/2209711404 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c009ea0 con 0x7f5aa00af000 2023-10-06T11:33:59.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa4cdf700 1 -- 172.21.15.46:0/2209711404 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00b860 con 0x7f5aa00af000 2023-10-06T11:33:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa4cdf700 1 -- 172.21.15.46:0/2209711404 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00bc00 con 0x7f5aa00af000 2023-10-06T11:33:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 msgr2=0x7f5aa00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00cf590 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099b0 tx=0x7f5a9c009250 comp rx=0 tx=0).stop 2023-10-06T11:33:59.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/2209711404 shutdown_connections 2023-10-06T11:33:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00af3d0 0x7f5aa00690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/2209711404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00cf590 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/2209711404 >> 172.21.15.46:0/2209711404 conn(0x7f5aa00683b0 msgr2=0x7f5aa00d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/2209711404 shutdown_connections 2023-10-06T11:33:59.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.063+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/2209711404 wait complete. 2023-10-06T11:33:59.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 Processor -- start 2023-10-06T11:33:59.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 -- start start 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 0x7f5aa00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b5620 con 0x7f5aa00af3d0 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b49a0 con 0x7f5aa00af000 2023-10-06T11:33:59.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa6ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00b4c80 con 0x7f5aa00cb4c0 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa54e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa54e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54176/0 (socket says 172.21.15.46:54176) 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa54e0700 1 -- 172.21.15.46:0/1264747747 learned_addr learned my addr 172.21.15.46:0/1264747747 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa64e2700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 0x7f5aa00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.067+0000 7f5aa5ce1700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 -- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 msgr2=0x7f5aa00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 0x7f5aa00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 -- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 msgr2=0x7f5aa00c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 -- 172.21.15.46:0/1264747747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009a20 con 0x7f5aa00af3d0 2023-10-06T11:33:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa64e2700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 0x7f5aa00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:59.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa54e0700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c7c0 tx=0x7f5a90007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90011590 con 0x7f5aa00af3d0 2023-10-06T11:33:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a90011710 con 0x7f5aa00af3d0 2023-10-06T11:33:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90011ad0 con 0x7f5aa00af3d0 2023-10-06T11:33:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c0094f0 con 0x7f5aa00af3d0 2023-10-06T11:33:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.071+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa00b5240 con 0x7f5aa00af3d0 2023-10-06T11:33:59.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.075+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa00b53d0 con 0x7f5aa00af3d0 2023-10-06T11:33:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.075+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5a9001a360 con 0x7f5aa00af3d0 2023-10-06T11:33:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.075+0000 7f5a95ffb700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 0x7f5a84062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.075+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5a9007bbb0 con 0x7f5aa00af3d0 2023-10-06T11:33:59.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.075+0000 7f5aa5ce1700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 0x7f5a84062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.079+0000 7f5aa5ce1700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 0x7f5a84062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099b0 tx=0x7f5a9c017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.083+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5a9004bd90 con 0x7f5aa00af3d0 2023-10-06T11:33:59.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.259+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aa00b5760 con 0x7f5aa00af3d0 2023-10-06T11:33:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.259+0000 7f5a95ffb700 1 -- 172.21.15.46:0/1264747747 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17540 (secure 0 0 0) 0x7f5aa00b5760 con 0x7f5aa00af3d0 2023-10-06T11:33:59.270 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:33:59.270 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:59.151843+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 msgr2=0x7f5a84062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 0x7f5a84062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099b0 tx=0x7f5a9c017000 comp rx=0 tx=0).stop 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 msgr2=0x7f5aa00b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c7c0 tx=0x7f5a90007f50 comp rx=0 tx=0).stop 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 shutdown_connections 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5a84060670 0x7f5a84062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5aa00cb4c0 0x7f5aa00b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5aa00af3d0 0x7f5aa00b3d90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 --2- 172.21.15.46:0/1264747747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5aa00af000 0x7f5aa00c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 >> 172.21.15.46:0/1264747747 conn(0x7f5aa00683b0 msgr2=0x7f5aa00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 shutdown_connections 2023-10-06T11:33:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.263+0000 7f5aa6ce3700 1 -- 172.21.15.46:0/1264747747 wait complete. 2023-10-06T11:33:59.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:33:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 Processor -- start 2023-10-06T11:33:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 -- start start 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52740c2220 0x7f52740c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740c1d90 con 0x7f52740c7d40 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740c1460 con 0x7f52740c97e0 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279f23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740c8190 con 0x7f52740c2220 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5273fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52740c7d40 0x7f527406c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52324/0 (socket says 172.21.15.46:52324) 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5273fff700 1 -- 172.21.15.46:0/3305155906 learned_addr learned my addr 172.21.15.46:0/3305155906 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:33:59.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.443+0000 7f5279722700 1 --2- 172.21.15.46:0/3305155906 >> v2:172.21.15.46:3300/0 conn(0x7f52740c2220 0x7f52740c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323155447 0 0) 0x7f52740c1d90 con 0x7f52740c7d40 2023-10-06T11:33:59.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5264003570 con 0x7f52740c7d40 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2917721961 0 0) 0x7f5264003570 con 0x7f52740c7d40 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 >> v2:172.21.15.46:3300/0 conn(0x7f52740c2220 msgr2=0x7f52740c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 --2- 172.21.15.46:0/3305155906 >> v2:172.21.15.46:3300/0 conn(0x7f52740c2220 0x7f52740c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 >> v1:172.21.15.132:6789/0 conn(0x7f52740c97e0 legacy=0x7f52740c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740c1740 con 0x7f52740c7d40 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f5260001470 con 0x7f52740c7d40 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 >> v1:172.21.15.46:6789/0 conn(0x7f52740c7d40 legacy=0x7f527406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 0x7f5264003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5264004810 0x7f5264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52640051c0 0x7f526400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c8190 con 0x7f5264003a60 2023-10-06T11:33:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c0b0 con 0x7f5264004810 2023-10-06T11:33:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c270 con 0x7f52640051c0 2023-10-06T11:33:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5279722700 1 --2- 172.21.15.46:0/3305155906 >> v2:172.21.15.46:3300/0 conn(0x7f52740c2220 0x7f52740c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:33:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 0x7f5264003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5279722700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52640051c0 0x7f526400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5273fff700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5264004810 0x7f5264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 -- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52640051c0 msgr2=0x7f526400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52640051c0 0x7f526400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 -- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5264004810 msgr2=0x7f5264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5264004810 0x7f5264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 -- 172.21.15.46:0/3305155906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264003570 con 0x7f5264003a60 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5279722700 1 -- 172.21.15.46:0/3305155906 reap_dead start 2023-10-06T11:33:59.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5273fff700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5264004810 0x7f5264004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f5278f21700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 0x7f5264003e30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f52680019a0 tx=0x7f5268007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b48b78e98f399fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526800c5d0 con 0x7f5264003a60 2023-10-06T11:33:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526800c730 con 0x7f5264003a60 2023-10-06T11:33:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.447+0000 7f52737fe700 1 -- 172.21.15.46:0/3305155906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526800cad0 con 0x7f5264003a60 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 -- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 msgr2=0x7f5264003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 0x7f5264003e30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f52680019a0 tx=0x7f5268007d10 comp rx=0 tx=0).stop 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 -- 172.21.15.46:0/3305155906 shutdown_connections 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 --2- 172.21.15.46:0/3305155906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5264003a60 0x7f5264003e30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 -- 172.21.15.46:0/3305155906 >> 172.21.15.46:0/3305155906 conn(0x7f52740cbc10 msgr2=0x7f52740cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 -- 172.21.15.46:0/3305155906 shutdown_connections 2023-10-06T11:33:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.451+0000 7f5279f23700 1 -- 172.21.15.46:0/3305155906 wait complete. 2023-10-06T11:33:59.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 Processor -- start 2023-10-06T11:33:59.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 -- start start 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5264003a60 0x7f52741618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 0x7f527406cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c3150 con 0x7f527406d0f0 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c3430 con 0x7f52740c2cc0 2023-10-06T11:33:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279f23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c3710 con 0x7f5264003a60 2023-10-06T11:33:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5278f21700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5264003a60 0x7f52741618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54194/0 (socket says 172.21.15.46:54194) 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 -- 172.21.15.46:0/2508823120 learned_addr learned my addr 172.21.15.46:0/2508823120 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5273fff700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 0x7f527406cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 -- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5264003a60 msgr2=0x7f52741618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5264003a60 0x7f52741618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 -- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 msgr2=0x7f527406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 0x7f527406cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 -- 172.21.15.46:0/2508823120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740c39f0 con 0x7f527406d0f0 2023-10-06T11:33:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5273fff700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 0x7f527406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.455+0000 7f5279722700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f527406a830 tx=0x7f526c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00e040 con 0x7f527406d0f0 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274162110 con 0x7f527406d0f0 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c00f040 con 0x7f527406d0f0 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c015430 con 0x7f527406d0f0 2023-10-06T11:33:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527415f010 con 0x7f527406d0f0 2023-10-06T11:33:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274064590 con 0x7f527406d0f0 2023-10-06T11:33:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.459+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f526c0157e0 con 0x7f527406d0f0 2023-10-06T11:33:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.463+0000 7f5270ff9700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 0x7f5258062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:33:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.463+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f526c07d850 con 0x7f527406d0f0 2023-10-06T11:33:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.463+0000 7f5278f21700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 0x7f5258062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:33:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.463+0000 7f5278f21700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 0x7f5258062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52740b1bb0 tx=0x7f5268015000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:33:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.467+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f526c04d860 con 0x7f527406d0f0 2023-10-06T11:33:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:33:59.679+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f52740c1a20 con 0x7f527406d0f0 2023-10-06T11:34:00.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.495+0000 7f5270ff9700 1 -- 172.21.15.46:0/2508823120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7f526c050e80 con 0x7f527406d0f0 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 msgr2=0x7f5258062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 0x7f5258062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52740b1bb0 tx=0x7f5268015000 comp rx=0 tx=0).stop 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 msgr2=0x7f527415e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f527406a830 tx=0x7f526c00b040 comp rx=0 tx=0).stop 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 shutdown_connections 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5258060670 0x7f5258062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f527406d0f0 0x7f527415e930 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52740c2cc0 0x7f527406cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 --2- 172.21.15.46:0/2508823120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5264003a60 0x7f52741618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.499+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 >> 172.21.15.46:0/2508823120 conn(0x7f52740cbc10 msgr2=0x7f5274158060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.503+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 shutdown_connections 2023-10-06T11:34:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.503+0000 7f5279f23700 1 -- 172.21.15.46:0/2508823120 wait complete. 2023-10-06T11:34:00.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:34:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.695+0000 7fd6c0d45700 1 Processor -- start 2023-10-06T11:34:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.695+0000 7fd6c0d45700 1 -- start start 2023-10-06T11:34:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6c0d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6bc15ce90 0x7fd6bc0697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6c0d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc0ce470 con 0x7fd6bc159420 2023-10-06T11:34:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6c0d45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6bc15d470 con 0x7fd6bc069410 2023-10-06T11:34:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6c0d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6bc15d750 con 0x7fd6bc15ce90 2023-10-06T11:34:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b9d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6bc159420 0x7fd6bc0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52334/0 (socket says 172.21.15.46:52334) 2023-10-06T11:34:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3508669979 learned_addr learned my addr 172.21.15.46:0/3508669979 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6bad9d700 1 --2- 172.21.15.46:0/3508669979 >> v2:172.21.15.132:3300/0 conn(0x7fd6bc15ce90 0x7fd6bc0697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312337377 0 0) 0x7fd6bc0ce470 con 0x7fd6bc159420 2023-10-06T11:34:00.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a4003570 con 0x7fd6bc159420 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370533068 0 0) 0x7fd6a4003570 con 0x7fd6bc159420 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 >> v2:172.21.15.132:3300/0 conn(0x7fd6bc15ce90 msgr2=0x7fd6bc0697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 --2- 172.21.15.46:0/3508669979 >> v2:172.21.15.132:3300/0 conn(0x7fd6bc15ce90 0x7fd6bc0697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 >> v1:172.21.15.132:6789/0 conn(0x7fd6bc069410 legacy=0x7fd6bc158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc15da30 con 0x7fd6bc159420 2023-10-06T11:34:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fd6b0001470 con 0x7fd6bc159420 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 >> v1:172.21.15.46:6789/0 conn(0x7fd6bc159420 legacy=0x7fd6bc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a4003a60 0x7fd6a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a40051c0 0x7fd6a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc15d750 con 0x7fd6a4003a60 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a400c160 con 0x7fd6a4004810 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a400c320 con 0x7fd6a40051c0 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6ba59c700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a4003a60 0x7fd6a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.699+0000 7fd6bad9d700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a40051c0 0x7fd6a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a40051c0 msgr2=0x7fd6a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a40051c0 0x7fd6a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3508669979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a4003a60 msgr2=0x7fd6a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6a4003a60 0x7fd6a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3508669979 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4003570 con 0x7fd6a4004810 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6bad9d700 1 -- 172.21.15.46:0/3508669979 reap_dead start 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c3c0 tx=0x7fd6b0001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5b3379f3b49c91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000cc90 con 0x7fd6a4004810 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000cdf0 con 0x7fd6a4004810 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6b959a700 1 -- 172.21.15.46:0/3508669979 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0005b50 con 0x7fd6a4004810 2023-10-06T11:34:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 msgr2=0x7fd6a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c3c0 tx=0x7fd6b0001e50 comp rx=0 tx=0).stop 2023-10-06T11:34:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3508669979 shutdown_connections 2023-10-06T11:34:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3508669979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3508669979 >> 172.21.15.46:0/3508669979 conn(0x7fd6bc0d0210 msgr2=0x7fd6bc0d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:00.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3508669979 shutdown_connections 2023-10-06T11:34:00.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.703+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3508669979 wait complete. 2023-10-06T11:34:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 Processor -- start 2023-10-06T11:34:00.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 -- start start 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 0x7fd6bc0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6bc0c9cd0 0x7fd6bc15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b0820 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b1630 con 0x7fd6bc0c9cd0 2023-10-06T11:34:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.707+0000 7fd6c0d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc0b5dc0 con 0x7fd6a4004810 2023-10-06T11:34:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54218/0 (socket says 172.21.15.46:54218) 2023-10-06T11:34:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3488062424 learned_addr learned my addr 172.21.15.46:0/3488062424 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6ba59c700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 0x7fd6bc0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6bad9d700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6bc0c9cd0 0x7fd6bc15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 msgr2=0x7fd6bc0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 0x7fd6bc0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6bc0c9cd0 msgr2=0x7fd6bc15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6bc0c9cd0 0x7fd6bc15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 -- 172.21.15.46:0/3488062424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0cf1a0 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6ba59c700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 0x7fd6bc0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6b9d9b700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c9b0 tx=0x7fd6b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0022070 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0000bc0 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000cc90 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc0b0a20 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc0b0fe0 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.711+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc088870 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.715+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd6b0007530 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.715+0000 7fd6aa7fc700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 0x7fd69c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.715+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd6b007de30 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.715+0000 7fd6ba59c700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 0x7fd69c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.719+0000 7fd6ba59c700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 0x7fd69c062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0cebc0 tx=0x7fd6ac007d30 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.723+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6b004df50 con 0x7fd6bc0b1ac0 2023-10-06T11:34:00.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:00.931+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd6bc15dd10 con 0x7fd6bc0b1ac0 2023-10-06T11:34:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.507+0000 7fd6aa7fc700 1 -- 172.21.15.46:0/3488062424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6b001be60 con 0x7fd6bc0b1ac0 2023-10-06T11:34:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 msgr2=0x7fd69c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 0x7fd69c062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0cebc0 tx=0x7fd6ac007d30 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 msgr2=0x7fd6bc0cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c9b0 tx=0x7fd6b0015000 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 shutdown_connections 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd69c060670 0x7fd69c062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6bc0c9cd0 0x7fd6bc15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6bc0b1ac0 0x7fd6bc0cec60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 --2- 172.21.15.46:0/3488062424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6a4004810 0x7fd6bc0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 >> 172.21.15.46:0/3488062424 conn(0x7fd6bc0d0210 msgr2=0x7fd6bc0c5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 shutdown_connections 2023-10-06T11:34:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.515+0000 7fd6c0d45700 1 -- 172.21.15.46:0/3488062424 wait complete. 2023-10-06T11:34:01.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 Processor -- start 2023-10-06T11:34:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 -- start start 2023-10-06T11:34:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d5415ab60 0x7f1d5406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d540ad6c0 0x7f1d540adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5406d460 con 0x7f1d5415af30 2023-10-06T11:34:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540c7000 con 0x7f1d5415ab60 2023-10-06T11:34:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d5a8a5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d540c72c0 con 0x7f1d540ad6c0 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.695+0000 7f1d53fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d5415ab60 0x7f1d5406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d5415ab60 0x7f1d5406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54228/0 (socket says 172.21.15.46:54228) 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 -- 172.21.15.46:0/2950148304 learned_addr learned my addr 172.21.15.46:0/2950148304 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d4b7fe700 1 --1- 172.21.15.46:0/2950148304 >> v1:172.21.15.46:6789/0 conn(0x7f1d5415af30 0x7f1d5406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52350/0 (socket says 172.21.15.46:52350) 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d58e42700 1 --2- 172.21.15.46:0/2950148304 >> v2:172.21.15.132:3300/0 conn(0x7f1d540ad6c0 0x7f1d540adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 -- 172.21.15.46:0/2950148304 >> v2:172.21.15.132:3300/0 conn(0x7f1d540ad6c0 msgr2=0x7f1d540adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 --2- 172.21.15.46:0/2950148304 >> v2:172.21.15.132:3300/0 conn(0x7f1d540ad6c0 0x7f1d540adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 -- 172.21.15.46:0/2950148304 >> v1:172.21.15.46:6789/0 conn(0x7f1d5415af30 legacy=0x7f1d5406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 -- 172.21.15.46:0/2950148304 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540c75a0 con 0x7f1d5415ab60 2023-10-06T11:34:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d53fff700 1 --2- 172.21.15.46:0/2950148304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415ab60 0x7f1d5406e8a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1d54151ba0 tx=0x7f1d44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1abe902f401a49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d537fe700 1 -- 172.21.15.46:0/2950148304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400b860 con 0x7f1d5415ab60 2023-10-06T11:34:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d537fe700 1 -- 172.21.15.46:0/2950148304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400b9c0 con 0x7f1d5415ab60 2023-10-06T11:34:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d537fe700 1 -- 172.21.15.46:0/2950148304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400bd60 con 0x7f1d5415ab60 2023-10-06T11:34:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/2950148304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415ab60 msgr2=0x7f1d5406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/2950148304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415ab60 0x7f1d5406e8a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1d54151ba0 tx=0x7f1d44009250 comp rx=0 tx=0).stop 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/2950148304 shutdown_connections 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/2950148304 >> v2:172.21.15.132:3300/0 conn(0x7f1d540ad6c0 0x7f1d540adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/2950148304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415ab60 0x7f1d5406e8a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/2950148304 >> 172.21.15.46:0/2950148304 conn(0x7f1d540cbe30 msgr2=0x7f1d540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/2950148304 shutdown_connections 2023-10-06T11:34:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.699+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/2950148304 wait complete. 2023-10-06T11:34:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 Processor -- start 2023-10-06T11:34:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 -- start start 2023-10-06T11:34:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d540ad6c0 0x7f1d54150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 0x7f1d540ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540af2d0 con 0x7f1d5415af30 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c5080 con 0x7f1d5415ab60 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.703+0000 7f1d5a8a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c47e0 con 0x7f1d540ad6c0 2023-10-06T11:34:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54242/0 (socket says 172.21.15.46:54242) 2023-10-06T11:34:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 -- 172.21.15.46:0/3650264723 learned_addr learned my addr 172.21.15.46:0/3650264723 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d4b7fe700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 0x7f1d540ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d53fff700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d540ad6c0 0x7f1d54150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 -- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d540ad6c0 msgr2=0x7f1d54150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d540ad6c0 0x7f1d54150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 -- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 msgr2=0x7f1d540ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 0x7f1d540ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 -- 172.21.15.46:0/3650264723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009490 con 0x7f1d5415af30 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d4b7fe700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 0x7f1d540ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d58e42700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c007f20 tx=0x7f1d4c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ebf0 con 0x7f1d5415af30 2023-10-06T11:34:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00ed70 con 0x7f1d5415af30 2023-10-06T11:34:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c014670 con 0x7f1d5415af30 2023-10-06T11:34:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540c4ac0 con 0x7f1d5415af30 2023-10-06T11:34:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540af580 con 0x7f1d5415af30 2023-10-06T11:34:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.707+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d540c4c50 con 0x7f1d5415af30 2023-10-06T11:34:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.711+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1d4c0149e0 con 0x7f1d5415af30 2023-10-06T11:34:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.711+0000 7f1d50ff9700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 0x7f1d34062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.711+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f1d4c07d340 con 0x7f1d5415af30 2023-10-06T11:34:01.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.711+0000 7f1d53fff700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 0x7f1d34062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:01.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.715+0000 7f1d53fff700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 0x7f1d34062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d54151ba0 tx=0x7f1d44017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:01.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.719+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1d4c04d460 con 0x7f1d5415af30 2023-10-06T11:34:01.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d540af710 con 0x7f1d5415af30 2023-10-06T11:34:01.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d50ff9700 1 -- 172.21.15.46:0/3650264723 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1d4c050a80 con 0x7f1d5415af30 2023-10-06T11:34:01.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:01.936 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 msgr2=0x7f1d34062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 0x7f1d34062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d54151ba0 tx=0x7f1d44017000 comp rx=0 tx=0).stop 2023-10-06T11:34:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 msgr2=0x7f1d540cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:01.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.927+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c007f20 tx=0x7f1d4c009860 comp rx=0 tx=0).stop 2023-10-06T11:34:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 shutdown_connections 2023-10-06T11:34:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1d34060670 0x7f1d34062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d5415af30 0x7f1d540cae00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d5415ab60 0x7f1d540ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 --2- 172.21.15.46:0/3650264723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d540ad6c0 0x7f1d54150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 >> 172.21.15.46:0/3650264723 conn(0x7f1d540cbe30 msgr2=0x7f1d540c6120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 shutdown_connections 2023-10-06T11:34:01.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:01.931+0000 7f1d5a8a5700 1 -- 172.21.15.46:0/3650264723 wait complete. 2023-10-06T11:34:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:01.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.107+0000 7f6b270ae700 1 Processor -- start 2023-10-06T11:34:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.107+0000 7f6b270ae700 1 -- start start 2023-10-06T11:34:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b270ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 0x7f6b2006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b270ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b200ad6c0 0x7f6b200adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b270ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2006d460 con 0x7f6b2015ab60 2023-10-06T11:34:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b270ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2006ed10 con 0x7f6b200ad6c0 2023-10-06T11:34:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b270ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200c7000 con 0x7f6b2006e8a0 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b24e4a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6b2015ab60 0x7f6b2015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52360/0 (socket says 172.21.15.46:52360) 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b24e4a700 1 -- 172.21.15.46:0/1152955229 learned_addr learned my addr 172.21.15.46:0/1152955229 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b2564b700 1 --2- 172.21.15.46:0/1152955229 >> v2:172.21.15.132:3300/0 conn(0x7f6b200ad6c0 0x7f6b200adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/1152955229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 0x7f6b2006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124457149 0 0) 0x7f6b2006d460 con 0x7f6b2015ab60 2023-10-06T11:34:02.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b10003570 con 0x7f6b2015ab60 2023-10-06T11:34:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2629840167 0 0) 0x7f6b10003570 con 0x7f6b2015ab60 2023-10-06T11:34:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 -- 172.21.15.46:0/1152955229 >> v2:172.21.15.132:3300/0 conn(0x7f6b200ad6c0 msgr2=0x7f6b200adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/1152955229 >> v2:172.21.15.132:3300/0 conn(0x7f6b200ad6c0 0x7f6b200adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 -- 172.21.15.46:0/1152955229 >> v1:172.21.15.46:6789/0 conn(0x7f6b2015ab60 legacy=0x7f6b2015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 -- 172.21.15.46:0/1152955229 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200c72c0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/1152955229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 0x7f6b2006cc30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00dd10 tx=0x7f6b0c00d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287d4094cc415541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.111+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c012070 con 0x7f6b2006e8a0 2023-10-06T11:34:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00de70 con 0x7f6b2006e8a0 2023-10-06T11:34:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b1f7fe700 1 -- 172.21.15.46:0/1152955229 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14004f00 con 0x7f6b2006e8a0 2023-10-06T11:34:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- 172.21.15.46:0/1152955229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 msgr2=0x7f6b2006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 --2- 172.21.15.46:0/1152955229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 0x7f6b2006cc30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00dd10 tx=0x7f6b0c00d630 comp rx=0 tx=0).stop 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- 172.21.15.46:0/1152955229 shutdown_connections 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 --2- 172.21.15.46:0/1152955229 >> v2:172.21.15.132:3300/0 conn(0x7f6b200ad6c0 0x7f6b200adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 --2- 172.21.15.46:0/1152955229 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2006e8a0 0x7f6b2006cc30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- 172.21.15.46:0/1152955229 >> 172.21.15.46:0/1152955229 conn(0x7f6b200cbe30 msgr2=0x7f6b200cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- 172.21.15.46:0/1152955229 shutdown_connections 2023-10-06T11:34:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- 172.21.15.46:0/1152955229 wait complete. 2023-10-06T11:34:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 Processor -- start 2023-10-06T11:34:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.115+0000 7f6b270ae700 1 -- start start 2023-10-06T11:34:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b200ad6c0 0x7f6b2014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2015ab60 0x7f6b2014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200c5d70 con 0x7f6b2006e8a0 2023-10-06T11:34:02.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200c3920 con 0x7f6b200ad6c0 2023-10-06T11:34:02.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b200b1a50 con 0x7f6b2015ab60 2023-10-06T11:34:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54254/0 (socket says 172.21.15.46:54254) 2023-10-06T11:34:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 -- 172.21.15.46:0/319353274 learned_addr learned my addr 172.21.15.46:0/319353274 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b2564b700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2015ab60 0x7f6b2014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b200ad6c0 0x7f6b2014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 -- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2015ab60 msgr2=0x7f6b2014d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2015ab60 0x7f6b2014d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 -- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b200ad6c0 msgr2=0x7f6b2014d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b200ad6c0 0x7f6b2014d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 -- 172.21.15.46:0/319353274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c00d8d0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b24e4a700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c8c0 tx=0x7f6b14005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14007300 con 0x7f6b2006e8a0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b140054a0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b20150860 con 0x7f6b2006e8a0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14005860 con 0x7f6b2006e8a0 2023-10-06T11:34:02.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.119+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b200c32d0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.123+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20063f90 con 0x7f6b2006e8a0 2023-10-06T11:34:02.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.123+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6b14005c10 con 0x7f6b2006e8a0 2023-10-06T11:34:02.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.127+0000 7f6b1cff9700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 0x7f6b04062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.127+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6b1407c980 con 0x7f6b2006e8a0 2023-10-06T11:34:02.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.127+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 0x7f6b04062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.127+0000 7f6b1ffff700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 0x7f6b04062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d670 tx=0x7f6b0c01b000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.131+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b1404caa0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.303+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b2004b5c0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.307+0000 7f6b1cff9700 1 -- 172.21.15.46:0/319353274 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17540 (secure 0 0 0) 0x7f6b140500c0 con 0x7f6b2006e8a0 2023-10-06T11:34:02.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:02.317 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:33:59.151843+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 msgr2=0x7f6b04062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 0x7f6b04062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d670 tx=0x7f6b0c01b000 comp rx=0 tx=0).stop 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 msgr2=0x7f6b200b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c8c0 tx=0x7f6b14005010 comp rx=0 tx=0).stop 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 shutdown_connections 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b040606a0 0x7f6b04062b50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2015ab60 0x7f6b2014d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b200ad6c0 0x7f6b2014d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 --2- 172.21.15.46:0/319353274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2006e8a0 0x7f6b200b1360 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 >> 172.21.15.46:0/319353274 conn(0x7f6b200cbe30 msgr2=0x7f6b200abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 shutdown_connections 2023-10-06T11:34:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.311+0000 7f6b270ae700 1 -- 172.21.15.46:0/319353274 wait complete. 2023-10-06T11:34:02.342 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:34:02.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:34:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.487+0000 7f2ed759e700 1 Processor -- start 2023-10-06T11:34:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.487+0000 7f2ed759e700 1 -- start start 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ed8067760 0x7f2ed8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed815d6c0 con 0x7f2ed8158dd0 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed815cd90 con 0x7f2ed81591a0 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80699c0 con 0x7f2ed8067760 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed659c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2ed8158dd0 0x7f2ed8067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52368/0 (socket says 172.21.15.46:52368) 2023-10-06T11:34:02.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed659c700 1 -- 172.21.15.46:0/2764551189 learned_addr learned my addr 172.21.15.46:0/2764551189 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed5d9b700 1 --2- 172.21.15.46:0/2764551189 >> v2:172.21.15.132:3300/0 conn(0x7f2ed8067760 0x7f2ed8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941070086 0 0) 0x7f2ed815d6c0 con 0x7f2ed8158dd0 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec0003570 con 0x7f2ed8158dd0 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540855777 0 0) 0x7f2ec0003570 con 0x7f2ed8158dd0 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 >> v2:172.21.15.132:3300/0 conn(0x7f2ed8067760 msgr2=0x7f2ed8068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 --2- 172.21.15.46:0/2764551189 >> v2:172.21.15.132:3300/0 conn(0x7f2ed8067760 0x7f2ed8068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 >> v1:172.21.15.132:6789/0 conn(0x7f2ed81591a0 legacy=0x7f2ed8069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed815d070 con 0x7f2ed8158dd0 2023-10-06T11:34:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f2ec8001470 con 0x7f2ed8158dd0 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 >> v1:172.21.15.46:6789/0 conn(0x7f2ed8158dd0 legacy=0x7f2ed8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 0x7f2ec0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec0004810 0x7f2ec0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec00051c0 0x7f2ec000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80699c0 con 0x7f2ec0003a60 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec000c050 con 0x7f2ec00051c0 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec000c210 con 0x7f2ec0004810 2023-10-06T11:34:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.491+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 0x7f2ec0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed5d9b700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec0004810 0x7f2ec0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed6d9d700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec00051c0 0x7f2ec000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 -- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec0004810 msgr2=0x7f2ec0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec0004810 0x7f2ec0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 -- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec00051c0 msgr2=0x7f2ec000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec00051c0 0x7f2ec000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 -- 172.21.15.46:0/2764551189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec0003570 con 0x7f2ec0003a60 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed6d9d700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec00051c0 0x7f2ec000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed6d9d700 1 -- 172.21.15.46:0/2764551189 reap_dead start 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 0x7f2ec0003e30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c2d0 tx=0x7f2ec8001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c5098c26dac5ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800cbc0 con 0x7f2ec0003a60 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800cd20 con 0x7f2ec0003a60 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed559a700 1 -- 172.21.15.46:0/2764551189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8005b70 con 0x7f2ec0003a60 2023-10-06T11:34:02.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 -- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 msgr2=0x7f2ec0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 0x7f2ec0003e30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c2d0 tx=0x7f2ec8001df0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 -- 172.21.15.46:0/2764551189 shutdown_connections 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2764551189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ec0003a60 0x7f2ec0003e30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 -- 172.21.15.46:0/2764551189 >> 172.21.15.46:0/2764551189 conn(0x7f2ed8062ef0 msgr2=0x7f2ed80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 -- 172.21.15.46:0/2764551189 shutdown_connections 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.495+0000 7f2ed759e700 1 -- 172.21.15.46:0/2764551189 wait complete. 2023-10-06T11:34:02.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 Processor -- start 2023-10-06T11:34:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 -- start start 2023-10-06T11:34:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 0x7f2ed815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ed806e150 0x7f2ed8169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed806e5c0 con 0x7f2ed81570c0 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81691a0 con 0x7f2ec0003a60 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed815a8a0 con 0x7f2ed806e150 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54280/0 (socket says 172.21.15.46:54280) 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed5d9b700 1 -- 172.21.15.46:0/2814347517 learned_addr learned my addr 172.21.15.46:0/2814347517 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed6d9d700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ed806e150 0x7f2ed8169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.499+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 0x7f2ed815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 -- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ed806e150 msgr2=0x7f2ed8169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ed806e150 0x7f2ed8169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 -- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 msgr2=0x7f2ed815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 0x7f2ed815a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 -- 172.21.15.46:0/2814347517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed815ab80 con 0x7f2ed81570c0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 0x7f2ed815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed5d9b700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc011710 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc015670 con 0x7f2ed81570c0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc0157d0 con 0x7f2ed81570c0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed815ae60 con 0x7f2ed81570c0 2023-10-06T11:34:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc015b70 con 0x7f2ed81570c0 2023-10-06T11:34:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed815a590 con 0x7f2ed81570c0 2023-10-06T11:34:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.503+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed815b340 con 0x7f2ed81570c0 2023-10-06T11:34:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.507+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2ecc013410 con 0x7f2ed81570c0 2023-10-06T11:34:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.507+0000 7f2ec67fc700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 0x7f2eb8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.507+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f2ecc07cf20 con 0x7f2ed81570c0 2023-10-06T11:34:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.507+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 0x7f2eb8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.507+0000 7f2ed659c700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 0x7f2eb8062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800d010 tx=0x7f2ec8015000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.511+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2ecc04d040 con 0x7f2ed81570c0 2023-10-06T11:34:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.683+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ed8064590 con 0x7f2eb80606a0 2023-10-06T11:34:02.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.703+0000 7f2ec67fc700 1 -- 172.21.15.46:0/2814347517 <== mgr.6502 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308438 (secure 0 0 0) 0x7f2ed8064590 con 0x7f2eb80606a0 2023-10-06T11:34:02.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:02.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 msgr2=0x7f2eb8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 0x7f2eb8062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800d010 tx=0x7f2ec8015000 comp rx=0 tx=0).stop 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 msgr2=0x7f2ed806dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc011710 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 shutdown_connections 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2eb80606a0 0x7f2eb8062b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ed806e150 0x7f2ed8169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ed81570c0 0x7f2ed806dc10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 --2- 172.21.15.46:0/2814347517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec0003a60 0x7f2ed815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 >> 172.21.15.46:0/2814347517 conn(0x7f2ed8062ef0 msgr2=0x7f2ed8157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 shutdown_connections 2023-10-06T11:34:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.707+0000 7f2ed759e700 1 -- 172.21.15.46:0/2814347517 wait complete. 2023-10-06T11:34:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:34:02.740 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:34:02.519186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449784,"kb_used_data":10040,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752524936,"statfs":{"total":773094113280,"available":770585534464,"internally_reserved":0,"allocated":10280960,"data_stored":5777760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002000"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181976+0000","last_change":"2023-10-06T11:33:58.181976+0000","last_active":"2023-10-06T11:33:58.181976+0000","last_peered":"2023-10-06T11:33:58.181976+0000","last_clean":"2023-10-06T11:33:58.181976+0000","last_became_active":"2023-10-06T11:33:58.181666+0000","last_became_peered":"2023-10-06T11:33:58.181666+0000","last_unstale":"2023-10-06T11:33:58.181976+0000","last_undegraded":"2023-10-06T11:33:58.181976+0000","last_fullsized":"2023-10-06T11:33:58.181976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:59.442916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192208+0000","last_change":"2023-10-06T11:33:58.190885+0000","last_active":"2023-10-06T11:33:58.192208+0000","last_peered":"2023-10-06T11:33:58.192208+0000","last_clean":"2023-10-06T11:33:58.192208+0000","last_became_active":"2023-10-06T11:33:58.190604+0000","last_became_peered":"2023-10-06T11:33:58.190604+0000","last_unstale":"2023-10-06T11:33:58.192208+0000","last_undegraded":"2023-10-06T11:33:58.192208+0000","last_fullsized":"2023-10-06T11:33:58.192208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:15.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.184117+0000","last_change":"2023-10-06T11:33:58.184117+0000","last_active":"2023-10-06T11:33:58.184117+0000","last_peered":"2023-10-06T11:33:58.184117+0000","last_clean":"2023-10-06T11:33:58.184117+0000","last_became_active":"2023-10-06T11:33:58.183494+0000","last_became_peered":"2023-10-06T11:33:58.183494+0000","last_unstale":"2023-10-06T11:33:58.184117+0000","last_undegraded":"2023-10-06T11:33:58.184117+0000","last_fullsized":"2023-10-06T11:33:58.184117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:16.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190305+0000","last_change":"2023-10-06T11:33:58.187409+0000","last_active":"2023-10-06T11:33:58.190305+0000","last_peered":"2023-10-06T11:33:58.190305+0000","last_clean":"2023-10-06T11:33:58.190305+0000","last_became_active":"2023-10-06T11:33:58.187101+0000","last_became_peered":"2023-10-06T11:33:58.187101+0000","last_unstale":"2023-10-06T11:33:58.190305+0000","last_undegraded":"2023-10-06T11:33:58.190305+0000","last_fullsized":"2023-10-06T11:33:58.190305+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:42.117096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.184602+0000","last_change":"2023-10-06T11:33:58.170978+0000","last_active":"2023-10-06T11:33:58.184602+0000","last_peered":"2023-10-06T11:33:58.184602+0000","last_clean":"2023-10-06T11:33:58.184602+0000","last_became_active":"2023-10-06T11:33:58.170593+0000","last_became_peered":"2023-10-06T11:33:58.170593+0000","last_unstale":"2023-10-06T11:33:58.184602+0000","last_undegraded":"2023-10-06T11:33:58.184602+0000","last_fullsized":"2023-10-06T11:33:58.184602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:06.214786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.168216+0000","last_change":"2023-10-06T11:33:58.168216+0000","last_active":"2023-10-06T11:33:58.168216+0000","last_peered":"2023-10-06T11:33:58.168216+0000","last_clean":"2023-10-06T11:33:58.168216+0000","last_became_active":"2023-10-06T11:33:58.167652+0000","last_became_peered":"2023-10-06T11:33:58.167652+0000","last_unstale":"2023-10-06T11:33:58.168216+0000","last_undegraded":"2023-10-06T11:33:58.168216+0000","last_fullsized":"2023-10-06T11:33:58.168216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:31.451651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199306+0000","last_change":"2023-10-06T11:33:58.198211+0000","last_active":"2023-10-06T11:33:58.199306+0000","last_peered":"2023-10-06T11:33:58.199306+0000","last_clean":"2023-10-06T11:33:58.199306+0000","last_became_active":"2023-10-06T11:33:58.197875+0000","last_became_peered":"2023-10-06T11:33:58.197875+0000","last_unstale":"2023-10-06T11:33:58.199306+0000","last_undegraded":"2023-10-06T11:33:58.199306+0000","last_fullsized":"2023-10-06T11:33:58.199306+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:05.108545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.180507+0000","last_change":"2023-10-06T11:33:58.171894+0000","last_active":"2023-10-06T11:33:58.180507+0000","last_peered":"2023-10-06T11:33:58.180507+0000","last_clean":"2023-10-06T11:33:58.180507+0000","last_became_active":"2023-10-06T11:33:58.171560+0000","last_became_peered":"2023-10-06T11:33:58.171560+0000","last_unstale":"2023-10-06T11:33:58.180507+0000","last_undegraded":"2023-10-06T11:33:58.180507+0000","last_fullsized":"2023-10-06T11:33:58.180507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:13.049151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190031+0000","last_change":"2023-10-06T11:33:58.176888+0000","last_active":"2023-10-06T11:33:58.190031+0000","last_peered":"2023-10-06T11:33:58.190031+0000","last_clean":"2023-10-06T11:33:58.190031+0000","last_became_active":"2023-10-06T11:33:58.176556+0000","last_became_peered":"2023-10-06T11:33:58.176556+0000","last_unstale":"2023-10-06T11:33:58.190031+0000","last_undegraded":"2023-10-06T11:33:58.190031+0000","last_fullsized":"2023-10-06T11:33:58.190031+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:30.766304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187512+0000","last_change":"2023-10-06T11:33:58.184415+0000","last_active":"2023-10-06T11:33:58.187512+0000","last_peered":"2023-10-06T11:33:58.187512+0000","last_clean":"2023-10-06T11:33:58.187512+0000","last_became_active":"2023-10-06T11:33:58.184078+0000","last_became_peered":"2023-10-06T11:33:58.184078+0000","last_unstale":"2023-10-06T11:33:58.187512+0000","last_undegraded":"2023-10-06T11:33:58.187512+0000","last_fullsized":"2023-10-06T11:33:58.187512+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:01.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.193626+0000","last_change":"2023-10-06T11:33:58.192689+0000","last_active":"2023-10-06T11:33:58.193626+0000","last_peered":"2023-10-06T11:33:58.193626+0000","last_clean":"2023-10-06T11:33:58.193626+0000","last_became_active":"2023-10-06T11:33:58.192366+0000","last_became_peered":"2023-10-06T11:33:58.192366+0000","last_unstale":"2023-10-06T11:33:58.193626+0000","last_undegraded":"2023-10-06T11:33:58.193626+0000","last_fullsized":"2023-10-06T11:33:58.193626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:11.429657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192839+0000","last_change":"2023-10-06T11:33:58.190623+0000","last_active":"2023-10-06T11:33:58.192839+0000","last_peered":"2023-10-06T11:33:58.192839+0000","last_clean":"2023-10-06T11:33:58.192839+0000","last_became_active":"2023-10-06T11:33:58.190112+0000","last_became_peered":"2023-10-06T11:33:58.190112+0000","last_unstale":"2023-10-06T11:33:58.192839+0000","last_undegraded":"2023-10-06T11:33:58.192839+0000","last_fullsized":"2023-10-06T11:33:58.192839+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:48.486298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.191889+0000","last_change":"2023-10-06T11:33:58.184927+0000","last_active":"2023-10-06T11:33:58.191889+0000","last_peered":"2023-10-06T11:33:58.191889+0000","last_clean":"2023-10-06T11:33:58.191889+0000","last_became_active":"2023-10-06T11:33:58.177334+0000","last_became_peered":"2023-10-06T11:33:58.177334+0000","last_unstale":"2023-10-06T11:33:58.191889+0000","last_undegraded":"2023-10-06T11:33:58.191889+0000","last_fullsized":"2023-10-06T11:33:58.191889+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:52.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.167961+0000","last_change":"2023-10-06T11:33:58.167961+0000","last_active":"2023-10-06T11:33:58.167961+0000","last_peered":"2023-10-06T11:33:58.167961+0000","last_clean":"2023-10-06T11:33:58.167961+0000","last_became_active":"2023-10-06T11:33:58.167249+0000","last_became_peered":"2023-10-06T11:33:58.167249+0000","last_unstale":"2023-10-06T11:33:58.167961+0000","last_undegraded":"2023-10-06T11:33:58.167961+0000","last_fullsized":"2023-10-06T11:33:58.167961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:01.509004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187298+0000","last_change":"2023-10-06T11:33:58.182499+0000","last_active":"2023-10-06T11:33:58.187298+0000","last_peered":"2023-10-06T11:33:58.187298+0000","last_clean":"2023-10-06T11:33:58.187298+0000","last_became_active":"2023-10-06T11:33:58.182163+0000","last_became_peered":"2023-10-06T11:33:58.182163+0000","last_unstale":"2023-10-06T11:33:58.187298+0000","last_undegraded":"2023-10-06T11:33:58.187298+0000","last_fullsized":"2023-10-06T11:33:58.187298+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:11.018913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.179861+0000","last_change":"2023-10-06T11:33:58.179861+0000","last_active":"2023-10-06T11:33:58.179861+0000","last_peered":"2023-10-06T11:33:58.179861+0000","last_clean":"2023-10-06T11:33:58.179861+0000","last_became_active":"2023-10-06T11:33:58.179542+0000","last_became_peered":"2023-10-06T11:33:58.179542+0000","last_unstale":"2023-10-06T11:33:58.179861+0000","last_undegraded":"2023-10-06T11:33:58.179861+0000","last_fullsized":"2023-10-06T11:33:58.179861+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:41.889801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190256+0000","last_change":"2023-10-06T11:33:58.183519+0000","last_active":"2023-10-06T11:33:58.190256+0000","last_peered":"2023-10-06T11:33:58.190256+0000","last_clean":"2023-10-06T11:33:58.190256+0000","last_became_active":"2023-10-06T11:33:58.183171+0000","last_became_peered":"2023-10-06T11:33:58.183171+0000","last_unstale":"2023-10-06T11:33:58.190256+0000","last_undegraded":"2023-10-06T11:33:58.190256+0000","last_fullsized":"2023-10-06T11:33:58.190256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:16.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182069+0000","last_change":"2023-10-06T11:33:58.182069+0000","last_active":"2023-10-06T11:33:58.182069+0000","last_peered":"2023-10-06T11:33:58.182069+0000","last_clean":"2023-10-06T11:33:58.182069+0000","last_became_active":"2023-10-06T11:33:58.181758+0000","last_became_peered":"2023-10-06T11:33:58.181758+0000","last_unstale":"2023-10-06T11:33:58.182069+0000","last_undegraded":"2023-10-06T11:33:58.182069+0000","last_fullsized":"2023-10-06T11:33:58.182069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:13.049151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175361+0000","last_change":"2023-10-06T11:33:58.175361+0000","last_active":"2023-10-06T11:33:58.175361+0000","last_peered":"2023-10-06T11:33:58.175361+0000","last_clean":"2023-10-06T11:33:58.175361+0000","last_became_active":"2023-10-06T11:33:58.174901+0000","last_became_peered":"2023-10-06T11:33:58.174901+0000","last_unstale":"2023-10-06T11:33:58.175361+0000","last_undegraded":"2023-10-06T11:33:58.175361+0000","last_fullsized":"2023-10-06T11:33:58.175361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:33.046664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.191808+0000","last_change":"2023-10-06T11:33:58.185048+0000","last_active":"2023-10-06T11:33:58.191808+0000","last_peered":"2023-10-06T11:33:58.191808+0000","last_clean":"2023-10-06T11:33:58.191808+0000","last_became_active":"2023-10-06T11:33:58.184708+0000","last_became_peered":"2023-10-06T11:33:58.184708+0000","last_unstale":"2023-10-06T11:33:58.191808+0000","last_undegraded":"2023-10-06T11:33:58.191808+0000","last_fullsized":"2023-10-06T11:33:58.191808+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.931569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192544+0000","last_change":"2023-10-06T11:33:58.187941+0000","last_active":"2023-10-06T11:33:58.192544+0000","last_peered":"2023-10-06T11:33:58.192544+0000","last_clean":"2023-10-06T11:33:58.192544+0000","last_became_active":"2023-10-06T11:33:58.187258+0000","last_became_peered":"2023-10-06T11:33:58.187258+0000","last_unstale":"2023-10-06T11:33:58.192544+0000","last_undegraded":"2023-10-06T11:33:58.192544+0000","last_fullsized":"2023-10-06T11:33:58.192544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.721768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190043+0000","last_change":"2023-10-06T11:33:58.177701+0000","last_active":"2023-10-06T11:33:58.190043+0000","last_peered":"2023-10-06T11:33:58.190043+0000","last_clean":"2023-10-06T11:33:58.190043+0000","last_became_active":"2023-10-06T11:33:58.177346+0000","last_became_peered":"2023-10-06T11:33:58.177346+0000","last_unstale":"2023-10-06T11:33:58.190043+0000","last_undegraded":"2023-10-06T11:33:58.190043+0000","last_fullsized":"2023-10-06T11:33:58.190043+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:17.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190376+0000","last_change":"2023-10-06T11:33:58.181156+0000","last_active":"2023-10-06T11:33:58.190376+0000","last_peered":"2023-10-06T11:33:58.190376+0000","last_clean":"2023-10-06T11:33:58.190376+0000","last_became_active":"2023-10-06T11:33:58.180838+0000","last_became_peered":"2023-10-06T11:33:58.180838+0000","last_unstale":"2023-10-06T11:33:58.190376+0000","last_undegraded":"2023-10-06T11:33:58.190376+0000","last_fullsized":"2023-10-06T11:33:58.190376+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:10.076262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185441+0000","last_change":"2023-10-06T11:33:58.180778+0000","last_active":"2023-10-06T11:33:58.185441+0000","last_peered":"2023-10-06T11:33:58.185441+0000","last_clean":"2023-10-06T11:33:58.185441+0000","last_became_active":"2023-10-06T11:33:58.180250+0000","last_became_peered":"2023-10-06T11:33:58.180250+0000","last_unstale":"2023-10-06T11:33:58.185441+0000","last_undegraded":"2023-10-06T11:33:58.185441+0000","last_fullsized":"2023-10-06T11:33:58.185441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:23.987851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192232+0000","last_change":"2023-10-06T11:33:58.190460+0000","last_active":"2023-10-06T11:33:58.192232+0000","last_peered":"2023-10-06T11:33:58.192232+0000","last_clean":"2023-10-06T11:33:58.192232+0000","last_became_active":"2023-10-06T11:33:58.190146+0000","last_became_peered":"2023-10-06T11:33:58.190146+0000","last_unstale":"2023-10-06T11:33:58.192232+0000","last_undegraded":"2023-10-06T11:33:58.192232+0000","last_fullsized":"2023-10-06T11:33:58.192232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:11.565286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195714+0000","last_change":"2023-10-06T11:33:58.194181+0000","last_active":"2023-10-06T11:33:58.195714+0000","last_peered":"2023-10-06T11:33:58.195714+0000","last_clean":"2023-10-06T11:33:58.195714+0000","last_became_active":"2023-10-06T11:33:58.193882+0000","last_became_peered":"2023-10-06T11:33:58.193882+0000","last_unstale":"2023-10-06T11:33:58.195714+0000","last_undegraded":"2023-10-06T11:33:58.195714+0000","last_fullsized":"2023-10-06T11:33:58.195714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:55.056089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.179816+0000","last_change":"2023-10-06T11:33:58.179816+0000","last_active":"2023-10-06T11:33:58.179816+0000","last_peered":"2023-10-06T11:33:58.179816+0000","last_clean":"2023-10-06T11:33:58.179816+0000","last_became_active":"2023-10-06T11:33:58.179502+0000","last_became_peered":"2023-10-06T11:33:58.179502+0000","last_unstale":"2023-10-06T11:33:58.179816+0000","last_undegraded":"2023-10-06T11:33:58.179816+0000","last_fullsized":"2023-10-06T11:33:58.179816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:02:52.028195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181170+0000","last_change":"2023-10-06T11:33:58.173468+0000","last_active":"2023-10-06T11:33:58.181170+0000","last_peered":"2023-10-06T11:33:58.181170+0000","last_clean":"2023-10-06T11:33:58.181170+0000","last_became_active":"2023-10-06T11:33:58.173133+0000","last_became_peered":"2023-10-06T11:33:58.173133+0000","last_unstale":"2023-10-06T11:33:58.181170+0000","last_undegraded":"2023-10-06T11:33:58.181170+0000","last_fullsized":"2023-10-06T11:33:58.181170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:08.100971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187162+0000","last_change":"2023-10-06T11:33:58.181435+0000","last_active":"2023-10-06T11:33:58.187162+0000","last_peered":"2023-10-06T11:33:58.187162+0000","last_clean":"2023-10-06T11:33:58.187162+0000","last_became_active":"2023-10-06T11:33:58.181119+0000","last_became_peered":"2023-10-06T11:33:58.181119+0000","last_unstale":"2023-10-06T11:33:58.187162+0000","last_undegraded":"2023-10-06T11:33:58.187162+0000","last_fullsized":"2023-10-06T11:33:58.187162+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:30.729845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.189840+0000","last_change":"2023-10-06T11:33:58.189840+0000","last_active":"2023-10-06T11:33:58.189840+0000","last_peered":"2023-10-06T11:33:58.189840+0000","last_clean":"2023-10-06T11:33:58.189840+0000","last_became_active":"2023-10-06T11:33:58.189529+0000","last_became_peered":"2023-10-06T11:33:58.189529+0000","last_unstale":"2023-10-06T11:33:58.189840+0000","last_undegraded":"2023-10-06T11:33:58.189840+0000","last_fullsized":"2023-10-06T11:33:58.189840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:30.440178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.178643+0000","last_change":"2023-10-06T11:33:58.178643+0000","last_active":"2023-10-06T11:33:58.178643+0000","last_peered":"2023-10-06T11:33:58.178643+0000","last_clean":"2023-10-06T11:33:58.178643+0000","last_became_active":"2023-10-06T11:33:58.178318+0000","last_became_peered":"2023-10-06T11:33:58.178318+0000","last_unstale":"2023-10-06T11:33:58.178643+0000","last_undegraded":"2023-10-06T11:33:58.178643+0000","last_fullsized":"2023-10-06T11:33:58.178643+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.931569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.176231+0000","last_change":"2023-10-06T11:33:58.176231+0000","last_active":"2023-10-06T11:33:58.176231+0000","last_peered":"2023-10-06T11:33:58.176231+0000","last_clean":"2023-10-06T11:33:58.176231+0000","last_became_active":"2023-10-06T11:33:58.174941+0000","last_became_peered":"2023-10-06T11:33:58.174941+0000","last_unstale":"2023-10-06T11:33:58.176231+0000","last_undegraded":"2023-10-06T11:33:58.176231+0000","last_fullsized":"2023-10-06T11:33:58.176231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:35.089621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182566+0000","last_change":"2023-10-06T11:33:58.182566+0000","last_active":"2023-10-06T11:33:58.182566+0000","last_peered":"2023-10-06T11:33:58.182566+0000","last_clean":"2023-10-06T11:33:58.182566+0000","last_became_active":"2023-10-06T11:33:58.182266+0000","last_became_peered":"2023-10-06T11:33:58.182266+0000","last_unstale":"2023-10-06T11:33:58.182566+0000","last_undegraded":"2023-10-06T11:33:58.182566+0000","last_fullsized":"2023-10-06T11:33:58.182566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:52.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'4","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.495610+0000","last_change":"2023-10-06T11:33:57.184229+0000","last_active":"2023-10-06T11:33:58.495610+0000","last_peered":"2023-10-06T11:33:58.495610+0000","last_clean":"2023-10-06T11:33:58.495610+0000","last_became_active":"2023-10-06T11:33:57.183942+0000","last_became_peered":"2023-10-06T11:33:57.183942+0000","last_unstale":"2023-10-06T11:33:58.495610+0000","last_undegraded":"2023-10-06T11:33:58.495610+0000","last_fullsized":"2023-10-06T11:33:58.495610+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:35:22.187051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185270+0000","last_change":"2023-10-06T11:33:58.180175+0000","last_active":"2023-10-06T11:33:58.185270+0000","last_peered":"2023-10-06T11:33:58.185270+0000","last_clean":"2023-10-06T11:33:58.185270+0000","last_became_active":"2023-10-06T11:33:58.179820+0000","last_became_peered":"2023-10-06T11:33:58.179820+0000","last_unstale":"2023-10-06T11:33:58.185270+0000","last_undegraded":"2023-10-06T11:33:58.185270+0000","last_fullsized":"2023-10-06T11:33:58.185270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:36.711635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169159+0000","last_change":"2023-10-06T11:33:57.192091+0000","last_active":"2023-10-06T11:33:58.169159+0000","last_peered":"2023-10-06T11:33:58.169159+0000","last_clean":"2023-10-06T11:33:58.169159+0000","last_became_active":"2023-10-06T11:33:57.191595+0000","last_became_peered":"2023-10-06T11:33:57.191595+0000","last_unstale":"2023-10-06T11:33:58.169159+0000","last_undegraded":"2023-10-06T11:33:58.169159+0000","last_fullsized":"2023-10-06T11:33:58.169159+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:43.705070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190090+0000","last_change":"2023-10-06T11:33:58.177028+0000","last_active":"2023-10-06T11:33:58.190090+0000","last_peered":"2023-10-06T11:33:58.190090+0000","last_clean":"2023-10-06T11:33:58.190090+0000","last_became_active":"2023-10-06T11:33:58.176509+0000","last_became_peered":"2023-10-06T11:33:58.176509+0000","last_unstale":"2023-10-06T11:33:58.190090+0000","last_undegraded":"2023-10-06T11:33:58.190090+0000","last_fullsized":"2023-10-06T11:33:58.190090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:18.903134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.500440+0000","last_change":"2023-10-06T11:33:57.197048+0000","last_active":"2023-10-06T11:33:58.500440+0000","last_peered":"2023-10-06T11:33:58.500440+0000","last_clean":"2023-10-06T11:33:58.500440+0000","last_became_active":"2023-10-06T11:33:57.196746+0000","last_became_peered":"2023-10-06T11:33:57.196746+0000","last_unstale":"2023-10-06T11:33:58.500440+0000","last_undegraded":"2023-10-06T11:33:58.500440+0000","last_fullsized":"2023-10-06T11:33:58.500440+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:00.492306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192564+0000","last_change":"2023-10-06T11:33:58.188131+0000","last_active":"2023-10-06T11:33:58.192564+0000","last_peered":"2023-10-06T11:33:58.192564+0000","last_clean":"2023-10-06T11:33:58.192564+0000","last_became_active":"2023-10-06T11:33:58.187649+0000","last_became_peered":"2023-10-06T11:33:58.187649+0000","last_unstale":"2023-10-06T11:33:58.192564+0000","last_undegraded":"2023-10-06T11:33:58.192564+0000","last_fullsized":"2023-10-06T11:33:58.192564+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:37.726524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504868+0000","last_change":"2023-10-06T11:33:57.194451+0000","last_active":"2023-10-06T11:33:58.504868+0000","last_peered":"2023-10-06T11:33:58.504868+0000","last_clean":"2023-10-06T11:33:58.504868+0000","last_became_active":"2023-10-06T11:33:57.194016+0000","last_became_peered":"2023-10-06T11:33:57.194016+0000","last_unstale":"2023-10-06T11:33:58.504868+0000","last_undegraded":"2023-10-06T11:33:58.504868+0000","last_fullsized":"2023-10-06T11:33:58.504868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:33.769943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192990+0000","last_change":"2023-10-06T11:33:58.191525+0000","last_active":"2023-10-06T11:33:58.192990+0000","last_peered":"2023-10-06T11:33:58.192990+0000","last_clean":"2023-10-06T11:33:58.192990+0000","last_became_active":"2023-10-06T11:33:58.191239+0000","last_became_peered":"2023-10-06T11:33:58.191239+0000","last_unstale":"2023-10-06T11:33:58.192990+0000","last_undegraded":"2023-10-06T11:33:58.192990+0000","last_fullsized":"2023-10-06T11:33:58.192990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.317821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.500190+0000","last_change":"2023-10-06T11:33:57.190426+0000","last_active":"2023-10-06T11:33:58.500190+0000","last_peered":"2023-10-06T11:33:58.500190+0000","last_clean":"2023-10-06T11:33:58.500190+0000","last_became_active":"2023-10-06T11:33:57.189292+0000","last_became_peered":"2023-10-06T11:33:57.189292+0000","last_unstale":"2023-10-06T11:33:58.500190+0000","last_undegraded":"2023-10-06T11:33:58.500190+0000","last_fullsized":"2023-10-06T11:33:58.500190+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:36.161187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188108+0000","last_change":"2023-10-06T11:33:58.186633+0000","last_active":"2023-10-06T11:33:58.188108+0000","last_peered":"2023-10-06T11:33:58.188108+0000","last_clean":"2023-10-06T11:33:58.188108+0000","last_became_active":"2023-10-06T11:33:58.186357+0000","last_became_peered":"2023-10-06T11:33:58.186357+0000","last_unstale":"2023-10-06T11:33:58.188108+0000","last_undegraded":"2023-10-06T11:33:58.188108+0000","last_fullsized":"2023-10-06T11:33:58.188108+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:54.945156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504368+0000","last_change":"2023-10-06T11:33:57.200315+0000","last_active":"2023-10-06T11:33:58.504368+0000","last_peered":"2023-10-06T11:33:58.504368+0000","last_clean":"2023-10-06T11:33:58.504368+0000","last_became_active":"2023-10-06T11:33:57.200040+0000","last_became_peered":"2023-10-06T11:33:57.200040+0000","last_unstale":"2023-10-06T11:33:58.504368+0000","last_undegraded":"2023-10-06T11:33:58.504368+0000","last_fullsized":"2023-10-06T11:33:58.504368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:42.920564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188658+0000","last_change":"2023-10-06T11:33:58.177367+0000","last_active":"2023-10-06T11:33:58.188658+0000","last_peered":"2023-10-06T11:33:58.188658+0000","last_clean":"2023-10-06T11:33:58.188658+0000","last_became_active":"2023-10-06T11:33:58.177049+0000","last_became_peered":"2023-10-06T11:33:58.177049+0000","last_unstale":"2023-10-06T11:33:58.188658+0000","last_undegraded":"2023-10-06T11:33:58.188658+0000","last_fullsized":"2023-10-06T11:33:58.188658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:43.937262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.168645+0000","last_change":"2023-10-06T11:33:57.188577+0000","last_active":"2023-10-06T11:33:58.168645+0000","last_peered":"2023-10-06T11:33:58.168645+0000","last_clean":"2023-10-06T11:33:58.168645+0000","last_became_active":"2023-10-06T11:33:57.186811+0000","last_became_peered":"2023-10-06T11:33:57.186811+0000","last_unstale":"2023-10-06T11:33:58.168645+0000","last_undegraded":"2023-10-06T11:33:58.168645+0000","last_fullsized":"2023-10-06T11:33:58.168645+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:47.469600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199320+0000","last_change":"2023-10-06T11:33:58.198279+0000","last_active":"2023-10-06T11:33:58.199320+0000","last_peered":"2023-10-06T11:33:58.199320+0000","last_clean":"2023-10-06T11:33:58.199320+0000","last_became_active":"2023-10-06T11:33:58.197978+0000","last_became_peered":"2023-10-06T11:33:58.197978+0000","last_unstale":"2023-10-06T11:33:58.199320+0000","last_undegraded":"2023-10-06T11:33:58.199320+0000","last_fullsized":"2023-10-06T11:33:58.199320+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:43.937262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.166998+0000","last_change":"2023-10-06T11:33:57.182971+0000","last_active":"2023-10-06T11:33:58.166998+0000","last_peered":"2023-10-06T11:33:58.166998+0000","last_clean":"2023-10-06T11:33:58.166998+0000","last_became_active":"2023-10-06T11:33:57.182671+0000","last_became_peered":"2023-10-06T11:33:57.182671+0000","last_unstale":"2023-10-06T11:33:58.166998+0000","last_undegraded":"2023-10-06T11:33:58.166998+0000","last_fullsized":"2023-10-06T11:33:58.166998+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:52.119107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185421+0000","last_change":"2023-10-06T11:33:58.180576+0000","last_active":"2023-10-06T11:33:58.185421+0000","last_peered":"2023-10-06T11:33:58.185421+0000","last_clean":"2023-10-06T11:33:58.185421+0000","last_became_active":"2023-10-06T11:33:58.179868+0000","last_became_peered":"2023-10-06T11:33:58.179868+0000","last_unstale":"2023-10-06T11:33:58.185421+0000","last_undegraded":"2023-10-06T11:33:58.185421+0000","last_fullsized":"2023-10-06T11:33:58.185421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:09.385892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195652+0000","last_change":"2023-10-06T11:33:57.191619+0000","last_active":"2023-10-06T11:33:57.195652+0000","last_peered":"2023-10-06T11:33:57.195652+0000","last_clean":"2023-10-06T11:33:57.195652+0000","last_became_active":"2023-10-06T11:33:57.191268+0000","last_became_peered":"2023-10-06T11:33:57.191268+0000","last_unstale":"2023-10-06T11:33:57.195652+0000","last_undegraded":"2023-10-06T11:33:57.195652+0000","last_fullsized":"2023-10-06T11:33:57.195652+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:36.709826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190990+0000","last_change":"2023-10-06T11:33:58.188862+0000","last_active":"2023-10-06T11:33:58.190990+0000","last_peered":"2023-10-06T11:33:58.190990+0000","last_clean":"2023-10-06T11:33:58.190990+0000","last_became_active":"2023-10-06T11:33:58.188510+0000","last_became_peered":"2023-10-06T11:33:58.188510+0000","last_unstale":"2023-10-06T11:33:58.190990+0000","last_undegraded":"2023-10-06T11:33:58.190990+0000","last_fullsized":"2023-10-06T11:33:58.190990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:17.413801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175992+0000","last_change":"2023-10-06T11:33:57.190428+0000","last_active":"2023-10-06T11:33:58.175992+0000","last_peered":"2023-10-06T11:33:58.175992+0000","last_clean":"2023-10-06T11:33:58.175992+0000","last_became_active":"2023-10-06T11:33:57.190161+0000","last_became_peered":"2023-10-06T11:33:57.190161+0000","last_unstale":"2023-10-06T11:33:58.175992+0000","last_undegraded":"2023-10-06T11:33:58.175992+0000","last_fullsized":"2023-10-06T11:33:58.175992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:56.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.180054+0000","last_change":"2023-10-06T11:33:58.180054+0000","last_active":"2023-10-06T11:33:58.180054+0000","last_peered":"2023-10-06T11:33:58.180054+0000","last_clean":"2023-10-06T11:33:58.180054+0000","last_became_active":"2023-10-06T11:33:58.179758+0000","last_became_peered":"2023-10-06T11:33:58.179758+0000","last_unstale":"2023-10-06T11:33:58.180054+0000","last_undegraded":"2023-10-06T11:33:58.180054+0000","last_fullsized":"2023-10-06T11:33:58.180054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:17.413801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.495042+0000","last_change":"2023-10-06T11:33:57.187332+0000","last_active":"2023-10-06T11:33:58.495042+0000","last_peered":"2023-10-06T11:33:58.495042+0000","last_clean":"2023-10-06T11:33:58.495042+0000","last_became_active":"2023-10-06T11:33:57.187041+0000","last_became_peered":"2023-10-06T11:33:57.187041+0000","last_unstale":"2023-10-06T11:33:58.495042+0000","last_undegraded":"2023-10-06T11:33:58.495042+0000","last_fullsized":"2023-10-06T11:33:58.495042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:28.849818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187184+0000","last_change":"2023-10-06T11:33:58.187184+0000","last_active":"2023-10-06T11:33:58.187184+0000","last_peered":"2023-10-06T11:33:58.187184+0000","last_clean":"2023-10-06T11:33:58.187184+0000","last_became_active":"2023-10-06T11:33:58.186914+0000","last_became_peered":"2023-10-06T11:33:58.186914+0000","last_unstale":"2023-10-06T11:33:58.187184+0000","last_undegraded":"2023-10-06T11:33:58.187184+0000","last_fullsized":"2023-10-06T11:33:58.187184+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.179000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.506724+0000","last_change":"2023-10-06T11:33:57.190206+0000","last_active":"2023-10-06T11:33:58.506724+0000","last_peered":"2023-10-06T11:33:58.506724+0000","last_clean":"2023-10-06T11:33:58.506724+0000","last_became_active":"2023-10-06T11:33:57.189908+0000","last_became_peered":"2023-10-06T11:33:57.189908+0000","last_unstale":"2023-10-06T11:33:58.506724+0000","last_undegraded":"2023-10-06T11:33:58.506724+0000","last_fullsized":"2023-10-06T11:33:58.506724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:05.198088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188157+0000","last_change":"2023-10-06T11:33:58.185076+0000","last_active":"2023-10-06T11:33:58.188157+0000","last_peered":"2023-10-06T11:33:58.188157+0000","last_clean":"2023-10-06T11:33:58.188157+0000","last_became_active":"2023-10-06T11:33:58.184760+0000","last_became_peered":"2023-10-06T11:33:58.184760+0000","last_unstale":"2023-10-06T11:33:58.188157+0000","last_undegraded":"2023-10-06T11:33:58.188157+0000","last_fullsized":"2023-10-06T11:33:58.188157+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:31.451651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.505026+0000","last_change":"2023-10-06T11:33:57.188872+0000","last_active":"2023-10-06T11:33:58.505026+0000","last_peered":"2023-10-06T11:33:58.505026+0000","last_clean":"2023-10-06T11:33:58.505026+0000","last_became_active":"2023-10-06T11:33:57.188578+0000","last_became_peered":"2023-10-06T11:33:57.188578+0000","last_unstale":"2023-10-06T11:33:58.505026+0000","last_undegraded":"2023-10-06T11:33:58.505026+0000","last_fullsized":"2023-10-06T11:33:58.505026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.753592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195589+0000","last_change":"2023-10-06T11:33:58.189505+0000","last_active":"2023-10-06T11:33:58.195589+0000","last_peered":"2023-10-06T11:33:58.195589+0000","last_clean":"2023-10-06T11:33:58.195589+0000","last_became_active":"2023-10-06T11:33:58.189215+0000","last_became_peered":"2023-10-06T11:33:58.189215+0000","last_unstale":"2023-10-06T11:33:58.195589+0000","last_undegraded":"2023-10-06T11:33:58.195589+0000","last_fullsized":"2023-10-06T11:33:58.195589+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:57.303486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192749+0000","last_change":"2023-10-06T11:33:57.189231+0000","last_active":"2023-10-06T11:33:57.192749+0000","last_peered":"2023-10-06T11:33:57.192749+0000","last_clean":"2023-10-06T11:33:57.192749+0000","last_became_active":"2023-10-06T11:33:57.188918+0000","last_became_peered":"2023-10-06T11:33:57.188918+0000","last_unstale":"2023-10-06T11:33:57.192749+0000","last_undegraded":"2023-10-06T11:33:57.192749+0000","last_fullsized":"2023-10-06T11:33:57.192749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:19:10.521490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195607+0000","last_change":"2023-10-06T11:33:58.189670+0000","last_active":"2023-10-06T11:33:58.195607+0000","last_peered":"2023-10-06T11:33:58.195607+0000","last_clean":"2023-10-06T11:33:58.195607+0000","last_became_active":"2023-10-06T11:33:58.189383+0000","last_became_peered":"2023-10-06T11:33:58.189383+0000","last_unstale":"2023-10-06T11:33:58.195607+0000","last_undegraded":"2023-10-06T11:33:58.195607+0000","last_fullsized":"2023-10-06T11:33:58.195607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:52.183355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.170439+0000","last_change":"2023-10-06T11:33:57.184682+0000","last_active":"2023-10-06T11:33:58.170439+0000","last_peered":"2023-10-06T11:33:58.170439+0000","last_clean":"2023-10-06T11:33:58.170439+0000","last_became_active":"2023-10-06T11:33:57.184401+0000","last_became_peered":"2023-10-06T11:33:57.184401+0000","last_unstale":"2023-10-06T11:33:58.170439+0000","last_undegraded":"2023-10-06T11:33:58.170439+0000","last_fullsized":"2023-10-06T11:33:58.170439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:32.830008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195642+0000","last_change":"2023-10-06T11:33:58.190331+0000","last_active":"2023-10-06T11:33:58.195642+0000","last_peered":"2023-10-06T11:33:58.195642+0000","last_clean":"2023-10-06T11:33:58.195642+0000","last_became_active":"2023-10-06T11:33:58.190056+0000","last_became_peered":"2023-10-06T11:33:58.190056+0000","last_unstale":"2023-10-06T11:33:58.195642+0000","last_undegraded":"2023-10-06T11:33:58.195642+0000","last_fullsized":"2023-10-06T11:33:58.195642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:05.974947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'8","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.497758+0000","last_change":"2023-10-06T11:33:57.185775+0000","last_active":"2023-10-06T11:33:58.497758+0000","last_peered":"2023-10-06T11:33:58.497758+0000","last_clean":"2023-10-06T11:33:58.497758+0000","last_became_active":"2023-10-06T11:33:57.185479+0000","last_became_peered":"2023-10-06T11:33:57.185479+0000","last_unstale":"2023-10-06T11:33:58.497758+0000","last_undegraded":"2023-10-06T11:33:58.497758+0000","last_fullsized":"2023-10-06T11:33:58.497758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:41.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192458+0000","last_change":"2023-10-06T11:33:58.186614+0000","last_active":"2023-10-06T11:33:58.192458+0000","last_peered":"2023-10-06T11:33:58.192458+0000","last_clean":"2023-10-06T11:33:58.192458+0000","last_became_active":"2023-10-06T11:33:58.185073+0000","last_became_peered":"2023-10-06T11:33:58.185073+0000","last_unstale":"2023-10-06T11:33:58.192458+0000","last_undegraded":"2023-10-06T11:33:58.192458+0000","last_fullsized":"2023-10-06T11:33:58.192458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504533+0000","last_change":"2023-10-06T11:33:57.199599+0000","last_active":"2023-10-06T11:33:58.504533+0000","last_peered":"2023-10-06T11:33:58.504533+0000","last_clean":"2023-10-06T11:33:58.504533+0000","last_became_active":"2023-10-06T11:33:57.199203+0000","last_became_peered":"2023-10-06T11:33:57.199203+0000","last_unstale":"2023-10-06T11:33:58.504533+0000","last_undegraded":"2023-10-06T11:33:58.504533+0000","last_fullsized":"2023-10-06T11:33:58.504533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:29.749606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195522+0000","last_change":"2023-10-06T11:33:58.188312+0000","last_active":"2023-10-06T11:33:58.195522+0000","last_peered":"2023-10-06T11:33:58.195522+0000","last_clean":"2023-10-06T11:33:58.195522+0000","last_became_active":"2023-10-06T11:33:58.188021+0000","last_became_peered":"2023-10-06T11:33:58.188021+0000","last_unstale":"2023-10-06T11:33:58.195522+0000","last_undegraded":"2023-10-06T11:33:58.195522+0000","last_fullsized":"2023-10-06T11:33:58.195522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:27.099347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.173488+0000","last_change":"2023-10-06T11:33:57.188316+0000","last_active":"2023-10-06T11:33:58.173488+0000","last_peered":"2023-10-06T11:33:58.173488+0000","last_clean":"2023-10-06T11:33:58.173488+0000","last_became_active":"2023-10-06T11:33:57.187978+0000","last_became_peered":"2023-10-06T11:33:57.187978+0000","last_unstale":"2023-10-06T11:33:58.173488+0000","last_undegraded":"2023-10-06T11:33:58.173488+0000","last_fullsized":"2023-10-06T11:33:58.173488+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:54.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187043+0000","last_change":"2023-10-06T11:33:58.176814+0000","last_active":"2023-10-06T11:33:58.187043+0000","last_peered":"2023-10-06T11:33:58.187043+0000","last_clean":"2023-10-06T11:33:58.187043+0000","last_became_active":"2023-10-06T11:33:58.175739+0000","last_became_peered":"2023-10-06T11:33:58.175739+0000","last_unstale":"2023-10-06T11:33:58.187043+0000","last_undegraded":"2023-10-06T11:33:58.187043+0000","last_fullsized":"2023-10-06T11:33:58.187043+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:57.303486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195581+0000","last_change":"2023-10-06T11:33:57.190718+0000","last_active":"2023-10-06T11:33:57.195581+0000","last_peered":"2023-10-06T11:33:57.195581+0000","last_clean":"2023-10-06T11:33:57.195581+0000","last_became_active":"2023-10-06T11:33:57.190424+0000","last_became_peered":"2023-10-06T11:33:57.190424+0000","last_unstale":"2023-10-06T11:33:57.195581+0000","last_undegraded":"2023-10-06T11:33:57.195581+0000","last_fullsized":"2023-10-06T11:33:57.195581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:56.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187185+0000","last_change":"2023-10-06T11:33:58.182064+0000","last_active":"2023-10-06T11:33:58.187185+0000","last_peered":"2023-10-06T11:33:58.187185+0000","last_clean":"2023-10-06T11:33:58.187185+0000","last_became_active":"2023-10-06T11:33:58.181755+0000","last_became_peered":"2023-10-06T11:33:58.181755+0000","last_unstale":"2023-10-06T11:33:58.187185+0000","last_undegraded":"2023-10-06T11:33:58.187185+0000","last_fullsized":"2023-10-06T11:33:58.187185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:10:27.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.197304+0000","last_change":"2023-10-06T11:33:57.194038+0000","last_active":"2023-10-06T11:33:57.197304+0000","last_peered":"2023-10-06T11:33:57.197304+0000","last_clean":"2023-10-06T11:33:57.197304+0000","last_became_active":"2023-10-06T11:33:57.193745+0000","last_became_peered":"2023-10-06T11:33:57.193745+0000","last_unstale":"2023-10-06T11:33:57.197304+0000","last_undegraded":"2023-10-06T11:33:57.197304+0000","last_fullsized":"2023-10-06T11:33:57.197304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:09.059564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.193413+0000","last_change":"2023-10-06T11:33:58.187653+0000","last_active":"2023-10-06T11:33:58.193413+0000","last_peered":"2023-10-06T11:33:58.193413+0000","last_clean":"2023-10-06T11:33:58.193413+0000","last_became_active":"2023-10-06T11:33:58.187364+0000","last_became_peered":"2023-10-06T11:33:58.187364+0000","last_unstale":"2023-10-06T11:33:58.193413+0000","last_undegraded":"2023-10-06T11:33:58.193413+0000","last_fullsized":"2023-10-06T11:33:58.193413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:59.821489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181338+0000","last_change":"2023-10-06T11:33:57.194566+0000","last_active":"2023-10-06T11:33:58.181338+0000","last_peered":"2023-10-06T11:33:58.181338+0000","last_clean":"2023-10-06T11:33:58.181338+0000","last_became_active":"2023-10-06T11:33:57.193886+0000","last_became_peered":"2023-10-06T11:33:57.193886+0000","last_unstale":"2023-10-06T11:33:58.181338+0000","last_undegraded":"2023-10-06T11:33:58.181338+0000","last_fullsized":"2023-10-06T11:33:58.181338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:13.162302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199336+0000","last_change":"2023-10-06T11:33:58.198321+0000","last_active":"2023-10-06T11:33:58.199336+0000","last_peered":"2023-10-06T11:33:58.199336+0000","last_clean":"2023-10-06T11:33:58.199336+0000","last_became_active":"2023-10-06T11:33:58.197987+0000","last_became_peered":"2023-10-06T11:33:58.197987+0000","last_unstale":"2023-10-06T11:33:58.199336+0000","last_undegraded":"2023-10-06T11:33:58.199336+0000","last_fullsized":"2023-10-06T11:33:58.199336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.167218+0000","last_change":"2023-10-06T11:33:57.190087+0000","last_active":"2023-10-06T11:33:58.167218+0000","last_peered":"2023-10-06T11:33:58.167218+0000","last_clean":"2023-10-06T11:33:58.167218+0000","last_became_active":"2023-10-06T11:33:57.189365+0000","last_became_peered":"2023-10-06T11:33:57.189365+0000","last_unstale":"2023-10-06T11:33:58.167218+0000","last_undegraded":"2023-10-06T11:33:58.167218+0000","last_fullsized":"2023-10-06T11:33:58.167218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.434104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.186604+0000","last_change":"2023-10-06T11:33:58.186604+0000","last_active":"2023-10-06T11:33:58.186604+0000","last_peered":"2023-10-06T11:33:58.186604+0000","last_clean":"2023-10-06T11:33:58.186604+0000","last_became_active":"2023-10-06T11:33:58.186324+0000","last_became_peered":"2023-10-06T11:33:58.186324+0000","last_unstale":"2023-10-06T11:33:58.186604+0000","last_undegraded":"2023-10-06T11:33:58.186604+0000","last_fullsized":"2023-10-06T11:33:58.186604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:06.214786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.183747+0000","last_change":"2023-10-06T11:33:57.196356+0000","last_active":"2023-10-06T11:33:58.183747+0000","last_peered":"2023-10-06T11:33:58.183747+0000","last_clean":"2023-10-06T11:33:58.183747+0000","last_became_active":"2023-10-06T11:33:57.196002+0000","last_became_peered":"2023-10-06T11:33:57.196002+0000","last_unstale":"2023-10-06T11:33:58.183747+0000","last_undegraded":"2023-10-06T11:33:58.183747+0000","last_fullsized":"2023-10-06T11:33:58.183747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:12.032453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.178573+0000","last_change":"2023-10-06T11:33:58.178573+0000","last_active":"2023-10-06T11:33:58.178573+0000","last_peered":"2023-10-06T11:33:58.178573+0000","last_clean":"2023-10-06T11:33:58.178573+0000","last_became_active":"2023-10-06T11:33:58.178218+0000","last_became_peered":"2023-10-06T11:33:58.178218+0000","last_unstale":"2023-10-06T11:33:58.178573+0000","last_undegraded":"2023-10-06T11:33:58.178573+0000","last_fullsized":"2023-10-06T11:33:58.178573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:34.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195128+0000","last_change":"2023-10-06T11:33:57.193721+0000","last_active":"2023-10-06T11:33:57.195128+0000","last_peered":"2023-10-06T11:33:57.195128+0000","last_clean":"2023-10-06T11:33:57.195128+0000","last_became_active":"2023-10-06T11:33:57.193245+0000","last_became_peered":"2023-10-06T11:33:57.193245+0000","last_unstale":"2023-10-06T11:33:57.195128+0000","last_undegraded":"2023-10-06T11:33:57.195128+0000","last_fullsized":"2023-10-06T11:33:57.195128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:40.243448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.189675+0000","last_change":"2023-10-06T11:33:58.189675+0000","last_active":"2023-10-06T11:33:58.189675+0000","last_peered":"2023-10-06T11:33:58.189675+0000","last_clean":"2023-10-06T11:33:58.189675+0000","last_became_active":"2023-10-06T11:33:58.189367+0000","last_became_peered":"2023-10-06T11:33:58.189367+0000","last_unstale":"2023-10-06T11:33:58.189675+0000","last_undegraded":"2023-10-06T11:33:58.189675+0000","last_fullsized":"2023-10-06T11:33:58.189675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:46:47.958647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192326+0000","last_change":"2023-10-06T11:33:57.187543+0000","last_active":"2023-10-06T11:33:57.192326+0000","last_peered":"2023-10-06T11:33:57.192326+0000","last_clean":"2023-10-06T11:33:57.192326+0000","last_became_active":"2023-10-06T11:33:57.187254+0000","last_became_peered":"2023-10-06T11:33:57.187254+0000","last_unstale":"2023-10-06T11:33:57.192326+0000","last_undegraded":"2023-10-06T11:33:57.192326+0000","last_fullsized":"2023-10-06T11:33:57.192326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:04.958249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"81'183","reported_seq":460,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.176732+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:33:57.176732+0000","last_peered":"2023-10-06T11:33:57.176732+0000","last_clean":"2023-10-06T11:33:57.176732+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:33:57.176732+0000","last_undegraded":"2023-10-06T11:33:57.176732+0000","last_fullsized":"2023-10-06T11:33:57.176732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175005+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:33:58.175005+0000","last_peered":"2023-10-06T11:33:58.175005+0000","last_clean":"2023-10-06T11:33:58.175005+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:33:58.175005+0000","last_undegraded":"2023-10-06T11:33:58.175005+0000","last_fullsized":"2023-10-06T11:33:58.175005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188178+0000","last_change":"2023-10-06T11:33:58.182836+0000","last_active":"2023-10-06T11:33:58.188178+0000","last_peered":"2023-10-06T11:33:58.188178+0000","last_clean":"2023-10-06T11:33:58.188178+0000","last_became_active":"2023-10-06T11:33:58.182554+0000","last_became_peered":"2023-10-06T11:33:58.182554+0000","last_unstale":"2023-10-06T11:33:58.188178+0000","last_undegraded":"2023-10-06T11:33:58.188178+0000","last_fullsized":"2023-10-06T11:33:58.188178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:33.186375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192280+0000","last_change":"2023-10-06T11:33:57.182559+0000","last_active":"2023-10-06T11:33:57.192280+0000","last_peered":"2023-10-06T11:33:57.192280+0000","last_clean":"2023-10-06T11:33:57.192280+0000","last_became_active":"2023-10-06T11:33:57.182246+0000","last_became_peered":"2023-10-06T11:33:57.182246+0000","last_unstale":"2023-10-06T11:33:57.192280+0000","last_undegraded":"2023-10-06T11:33:57.192280+0000","last_fullsized":"2023-10-06T11:33:57.192280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:04.091847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.176618+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:33:57.176618+0000","last_peered":"2023-10-06T11:33:57.176618+0000","last_clean":"2023-10-06T11:33:57.176618+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:33:57.176618+0000","last_undegraded":"2023-10-06T11:33:57.176618+0000","last_fullsized":"2023-10-06T11:33:57.176618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187865+0000","last_change":"2023-10-06T11:33:58.186017+0000","last_active":"2023-10-06T11:33:58.187865+0000","last_peered":"2023-10-06T11:33:58.187865+0000","last_clean":"2023-10-06T11:33:58.187865+0000","last_became_active":"2023-10-06T11:33:58.185532+0000","last_became_peered":"2023-10-06T11:33:58.185532+0000","last_unstale":"2023-10-06T11:33:58.187865+0000","last_undegraded":"2023-10-06T11:33:58.187865+0000","last_fullsized":"2023-10-06T11:33:58.187865+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:30.766304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.191833+0000","last_change":"2023-10-06T11:33:57.182254+0000","last_active":"2023-10-06T11:33:57.191833+0000","last_peered":"2023-10-06T11:33:57.191833+0000","last_clean":"2023-10-06T11:33:57.191833+0000","last_became_active":"2023-10-06T11:33:57.181930+0000","last_became_peered":"2023-10-06T11:33:57.181930+0000","last_unstale":"2023-10-06T11:33:57.191833+0000","last_undegraded":"2023-10-06T11:33:57.191833+0000","last_fullsized":"2023-10-06T11:33:57.191833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:58.804792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.170543+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:33:58.170543+0000","last_peered":"2023-10-06T11:33:58.170543+0000","last_clean":"2023-10-06T11:33:58.170543+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:33:58.170543+0000","last_undegraded":"2023-10-06T11:33:58.170543+0000","last_fullsized":"2023-10-06T11:33:58.170543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187273+0000","last_change":"2023-10-06T11:33:58.182030+0000","last_active":"2023-10-06T11:33:58.187273+0000","last_peered":"2023-10-06T11:33:58.187273+0000","last_clean":"2023-10-06T11:33:58.187273+0000","last_became_active":"2023-10-06T11:33:58.181703+0000","last_became_peered":"2023-10-06T11:33:58.181703+0000","last_unstale":"2023-10-06T11:33:58.187273+0000","last_undegraded":"2023-10-06T11:33:58.187273+0000","last_fullsized":"2023-10-06T11:33:58.187273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:35.527326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.494876+0000","last_change":"2023-10-06T11:33:57.191201+0000","last_active":"2023-10-06T11:33:58.494876+0000","last_peered":"2023-10-06T11:33:58.494876+0000","last_clean":"2023-10-06T11:33:58.494876+0000","last_became_active":"2023-10-06T11:33:57.190876+0000","last_became_peered":"2023-10-06T11:33:57.190876+0000","last_unstale":"2023-10-06T11:33:58.494876+0000","last_undegraded":"2023-10-06T11:33:58.494876+0000","last_fullsized":"2023-10-06T11:33:58.494876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:51.166657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.174939+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:33:58.174939+0000","last_peered":"2023-10-06T11:33:58.174939+0000","last_clean":"2023-10-06T11:33:58.174939+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:33:58.174939+0000","last_undegraded":"2023-10-06T11:33:58.174939+0000","last_fullsized":"2023-10-06T11:33:58.174939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190264+0000","last_change":"2023-10-06T11:33:58.183771+0000","last_active":"2023-10-06T11:33:58.190264+0000","last_peered":"2023-10-06T11:33:58.190264+0000","last_clean":"2023-10-06T11:33:58.190264+0000","last_became_active":"2023-10-06T11:33:58.183451+0000","last_became_peered":"2023-10-06T11:33:58.183451+0000","last_unstale":"2023-10-06T11:33:58.190264+0000","last_undegraded":"2023-10-06T11:33:58.190264+0000","last_fullsized":"2023-10-06T11:33:58.190264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.179000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.494270+0000","last_change":"2023-10-06T11:33:57.182916+0000","last_active":"2023-10-06T11:33:58.494270+0000","last_peered":"2023-10-06T11:33:58.494270+0000","last_clean":"2023-10-06T11:33:58.494270+0000","last_became_active":"2023-10-06T11:33:57.182602+0000","last_became_peered":"2023-10-06T11:33:57.182602+0000","last_unstale":"2023-10-06T11:33:58.494270+0000","last_undegraded":"2023-10-06T11:33:58.494270+0000","last_fullsized":"2023-10-06T11:33:58.494270+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:19.921634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.174836+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:33:57.174836+0000","last_peered":"2023-10-06T11:33:57.174836+0000","last_clean":"2023-10-06T11:33:57.174836+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:33:57.174836+0000","last_undegraded":"2023-10-06T11:33:57.174836+0000","last_fullsized":"2023-10-06T11:33:57.174836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175304+0000","last_change":"2023-10-06T11:33:58.175304+0000","last_active":"2023-10-06T11:33:58.175304+0000","last_peered":"2023-10-06T11:33:58.175304+0000","last_clean":"2023-10-06T11:33:58.175304+0000","last_became_active":"2023-10-06T11:33:58.174963+0000","last_became_peered":"2023-10-06T11:33:58.174963+0000","last_unstale":"2023-10-06T11:33:58.175304+0000","last_undegraded":"2023-10-06T11:33:58.175304+0000","last_fullsized":"2023-10-06T11:33:58.175304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:37.726524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.186628+0000","last_change":"2023-10-06T11:33:57.182518+0000","last_active":"2023-10-06T11:33:57.186628+0000","last_peered":"2023-10-06T11:33:57.186628+0000","last_clean":"2023-10-06T11:33:57.186628+0000","last_became_active":"2023-10-06T11:33:57.182210+0000","last_became_peered":"2023-10-06T11:33:57.182210+0000","last_unstale":"2023-10-06T11:33:57.186628+0000","last_undegraded":"2023-10-06T11:33:57.186628+0000","last_fullsized":"2023-10-06T11:33:57.186628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:40.922998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.178638+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:33:57.178638+0000","last_peered":"2023-10-06T11:33:57.178638+0000","last_clean":"2023-10-06T11:33:57.178638+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:33:57.178638+0000","last_undegraded":"2023-10-06T11:33:57.178638+0000","last_fullsized":"2023-10-06T11:33:57.178638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169937+0000","last_change":"2023-10-06T11:33:58.169937+0000","last_active":"2023-10-06T11:33:58.169937+0000","last_peered":"2023-10-06T11:33:58.169937+0000","last_clean":"2023-10-06T11:33:58.169937+0000","last_became_active":"2023-10-06T11:33:58.168881+0000","last_became_peered":"2023-10-06T11:33:58.168881+0000","last_unstale":"2023-10-06T11:33:58.169937+0000","last_undegraded":"2023-10-06T11:33:58.169937+0000","last_fullsized":"2023-10-06T11:33:58.169937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:19:11.538188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169819+0000","last_change":"2023-10-06T11:33:57.199227+0000","last_active":"2023-10-06T11:33:58.169819+0000","last_peered":"2023-10-06T11:33:58.169819+0000","last_clean":"2023-10-06T11:33:58.169819+0000","last_became_active":"2023-10-06T11:33:57.198885+0000","last_became_peered":"2023-10-06T11:33:57.198885+0000","last_unstale":"2023-10-06T11:33:58.169819+0000","last_undegraded":"2023-10-06T11:33:58.169819+0000","last_fullsized":"2023-10-06T11:33:58.169819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:17.886436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.185133+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:33:57.185133+0000","last_peered":"2023-10-06T11:33:57.185133+0000","last_clean":"2023-10-06T11:33:57.185133+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:33:57.185133+0000","last_undegraded":"2023-10-06T11:33:57.185133+0000","last_fullsized":"2023-10-06T11:33:57.185133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190152+0000","last_change":"2023-10-06T11:33:58.182715+0000","last_active":"2023-10-06T11:33:58.190152+0000","last_peered":"2023-10-06T11:33:58.190152+0000","last_clean":"2023-10-06T11:33:58.190152+0000","last_became_active":"2023-10-06T11:33:58.182414+0000","last_became_peered":"2023-10-06T11:33:58.182414+0000","last_unstale":"2023-10-06T11:33:58.190152+0000","last_undegraded":"2023-10-06T11:33:58.190152+0000","last_fullsized":"2023-10-06T11:33:58.190152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:34.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.503308+0000","last_change":"2023-10-06T11:33:57.194801+0000","last_active":"2023-10-06T11:33:58.503308+0000","last_peered":"2023-10-06T11:33:58.503308+0000","last_clean":"2023-10-06T11:33:58.503308+0000","last_became_active":"2023-10-06T11:33:57.194228+0000","last_became_peered":"2023-10-06T11:33:57.194228+0000","last_unstale":"2023-10-06T11:33:58.503308+0000","last_undegraded":"2023-10-06T11:33:58.503308+0000","last_fullsized":"2023-10-06T11:33:58.503308+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:25.131248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182304+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:33:58.182304+0000","last_peered":"2023-10-06T11:33:58.182304+0000","last_clean":"2023-10-06T11:33:58.182304+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:33:58.182304+0000","last_undegraded":"2023-10-06T11:33:58.182304+0000","last_fullsized":"2023-10-06T11:33:58.182304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311256,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060584,"statfs":{"total":96636764160,"available":96318038016,"internally_reserved":0,"allocated":1720320,"data_stored":1160899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.827},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.577,"5min":1.649,"15min":2.0030000000000001},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3169999999999999,"15min":1.3300000000000001},"min":{"1min":0.90000000000000002,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7649999999999999,"5min":1.871,"15min":2.6059999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.927,"5min":1.927,"15min":1.931},"last":1.1379999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.502,"5min":1.827,"15min":2.3799999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.258,"15min":1.2609999999999999},"min":{"1min":1.054,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.7,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.339}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.82099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8,"5min":1.9450000000000001,"15min":2.665},"last":1.405},{"interface":"front","average":{"1min":1.27,"5min":1.296,"15min":1.321},"min":{"1min":0.92900000000000005,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.796,"15min":2.8570000000000002},"last":1.554}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.77300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5569999999999999,"5min":2.149,"15min":2.7970000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.86599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.5339999999999998},"last":1.272}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.335,"15min":1.3160000000000001},"min":{"1min":0.77900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7410000000000001,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.28},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.71,"5min":2.2480000000000002,"15min":3.105},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.361},"min":{"1min":1.0369999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0190000000000001,"5min":2.4289999999999998,"15min":3.004},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.86299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.905,"5min":2.121,"15min":2.9119999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302504,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069336,"statfs":{"total":96636764160,"available":96327000064,"internally_reserved":0,"allocated":1146880,"data_stored":579130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.288},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5629999999999999,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.347},{"interface":"front","average":{"1min":1.379,"5min":1.333,"15min":1.325},"min":{"1min":0.83799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6830000000000001,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.4850000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.80500000000000005,"5min":0.76200000000000001,"15min":0.63900000000000001},"max":{"1min":1.5029999999999999,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":1.119},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.605,"5min":1.669,"15min":1.8480000000000001},"last":1.054}]},{"osd":2,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.2},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.74299999999999999},"max":{"1min":1.514,"5min":1.635,"15min":1.6919999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.194,"15min":1.222},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4450000000000001,"5min":1.587,"15min":2.7789999999999999},"last":1.018}]},{"osd":3,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.3100000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.534,"5min":1.764,"15min":1.9850000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3120000000000001,"15min":1.3049999999999999},"min":{"1min":1.077,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.5269999999999999,"5min":1.724,"15min":2.109},"last":1.1379999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.238,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.76500000000000001,"15min":0.61399999999999999},"max":{"1min":1.6639999999999999,"5min":1.8,"15min":1.956},"last":1.272},{"interface":"front","average":{"1min":1.226,"5min":1.292,"15min":1.2490000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.73899999999999999},"max":{"1min":1.645,"5min":1.827,"15min":1.8879999999999999},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6000000000000001,"5min":1.72,"15min":2.198},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.395,"15min":1.385},"min":{"1min":0.86499999999999999,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.718,"5min":1.879,"15min":2.1509999999999998},"last":1.5189999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4510000000000001,"15min":1.448},"min":{"1min":0.90100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.871,"5min":1.927,"15min":2.2919999999999998},"last":1.746},{"interface":"front","average":{"1min":1.381,"5min":1.4139999999999999,"15min":1.4299999999999999},"min":{"1min":1.004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6879999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.651}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302496,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069344,"statfs":{"total":96636764160,"available":96327008256,"internally_reserved":0,"allocated":1138688,"data_stored":577628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.3899999999999999,"15min":1.268},"min":{"1min":0.67400000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.772,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64900000000000002},"max":{"1min":1.679,"5min":1.972,"15min":2.544},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.194,"15min":1.175},"min":{"1min":0.79800000000000004,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.478,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.256},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.2190000000000001},"min":{"1min":0.79900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5189999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.159,"15min":1.1950000000000001},"min":{"1min":0.78600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.601,"5min":1.6879999999999999,"15min":3.609},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.73299999999999998},"max":{"1min":1.512,"5min":1.738,"15min":2.6949999999999998},"last":1.129}]},{"osd":3,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.393,"15min":1.296},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.639,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.448},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.302,"15min":1.3340000000000001},"min":{"1min":0.91800000000000004,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.5229999999999999,"5min":1.734,"15min":3.7770000000000001},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.40699999999999997},"max":{"1min":1.6060000000000001,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.393},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.516,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.724,"5min":18.48,"15min":18.48},"last":1.5249999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.3089999999999999},"min":{"1min":1.0129999999999999,"5min":0.85199999999999998,"15min":0.622},"max":{"1min":1.6799999999999999,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.411,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.6870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5249999999999999,"15min":1.4039999999999999},"min":{"1min":0.85499999999999998,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7909999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.494,"15min":1.444},"min":{"1min":0.998,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7709999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.4179999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302512,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069328,"statfs":{"total":96636764160,"available":96326991872,"internally_reserved":0,"allocated":1155072,"data_stored":579234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1479999999999999,"15min":1.143},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.554,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.111,"15min":1.143},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.478,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":1.3779999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2989999999999999,"15min":1.323},"min":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.546,"5min":1.645,"15min":2.1989999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.68300000000000005},"max":{"1min":1.508,"5min":1.7450000000000001,"15min":1.851},"last":1.4550000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.73499999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":2.2269999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.78500000000000003},"max":{"1min":1.583,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.476}]},{"osd":3,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.671,"15min":1.6839999999999999},"last":1.256},{"interface":"front","average":{"1min":1.288,"5min":1.3009999999999999,"15min":1.304},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.77400000000000002},"max":{"1min":1.575,"5min":1.6639999999999999,"15min":2.258},"last":1.498}]},{"osd":5,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.202,"15min":1.21},"min":{"1min":0.79600000000000004,"5min":0.621,"15min":0.621},"max":{"1min":1.653,"5min":1.653,"15min":2.8250000000000002},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2130000000000001,"15min":1.1970000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.40999999999999998},"max":{"1min":1.421,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.6950000000000001,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.099},{"interface":"front","average":{"1min":1.385,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":1.0860000000000001,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.845,"5min":1.845,"15min":2.4329999999999998},"last":1.319}]},{"osd":7,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.69399999999999995},"max":{"1min":1.5880000000000001,"5min":1.841,"15min":2.653},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.3919999999999999},"min":{"1min":0.96699999999999997,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.464},"last":1.2949999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306580,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065260,"statfs":{"total":96636764160,"available":96322826240,"internally_reserved":0,"allocated":1126400,"data_stored":571063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.61099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0960000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.437},"max":{"1min":1.427,"5min":1.504,"15min":1.627},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.95099999999999996},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.343,"5min":1.4079999999999999,"15min":1.516},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0660000000000001,"15min":1.077},"min":{"1min":0.753,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.093,"15min":1.121},"min":{"1min":0.95399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.68700000000000006,"5min":0.44,"15min":0.44},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.7030000000000001},"last":1.341}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.1919999999999999,"15min":1.236},"min":{"1min":0.93300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6890000000000001,"5min":1.768,"15min":2.6640000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3029999999999999,"15min":1.3560000000000001},"min":{"1min":1.1379999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0840000000000001},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3420000000000001,"15min":1.4019999999999999},"min":{"1min":0.92100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6080000000000001,"5min":1.883,"15min":2.1699999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.357,"15min":1.3979999999999999},"min":{"1min":1.0620000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.1179999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.411,"15min":1.45},"min":{"1min":0.98599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8520000000000001,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":1.49},{"interface":"front","average":{"1min":1.387,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.97499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.75,"5min":1.823,"15min":2.137},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4119999999999999,"15min":1.4419999999999999},"min":{"1min":1.163,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8540000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.04},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4319999999999999,"15min":1.482},"min":{"1min":1.0860000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.645,"5min":2.004,"15min":2.266},"last":1.381}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310684,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061156,"statfs":{"total":96636764160,"available":96318623744,"internally_reserved":0,"allocated":1134592,"data_stored":571223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1200000000000001,"5min":1.4670000000000001,"15min":1.8380000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.383,"5min":1.415,"15min":1.9650000000000001},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.3049999999999999,"5min":1.631,"15min":2.3940000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0229999999999999},"min":{"1min":0.68700000000000006,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.599,"5min":1.6779999999999999,"15min":2.5390000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"min":{"1min":0.90600000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.758,"5min":1.8,"15min":1.917},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.28,"15min":1.306},"min":{"1min":0.96499999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.948,"5min":1.948,"15min":2.4550000000000001},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.89700000000000002,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":2.1899999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":1.0589999999999999,"5min":0.88500000000000001,"15min":0.84399999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.6920000000000002},"last":1.087}]},{"osd":5,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":1.171,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8480000000000001,"5min":2.0680000000000001,"15min":2.601},"last":1.294},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.319,"15min":1.337},"min":{"1min":1.0089999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.6400000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4039999999999999,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.972,"5min":1.972,"15min":2.8340000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.427,"15min":1.444},"min":{"1min":1.1719999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8320000000000001,"5min":1.9910000000000001,"15min":2.617},"last":1.264}]},{"osd":7,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.371,"15min":1.391},"min":{"1min":0.94299999999999995,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.7549999999999999,"5min":1.8049999999999999,"15min":2.9020000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.464,"15min":1.4690000000000001},"min":{"1min":1.2609999999999999,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.978,"5min":1.978,"15min":3.2240000000000002},"last":1.4470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306584,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065256,"statfs":{"total":96636764160,"available":96322822144,"internally_reserved":0,"allocated":1130496,"data_stored":570621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.93000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.252,"5min":1.724,"15min":1.724},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.58399999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.4259999999999999,"15min":1.855},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.97199999999999998},"min":{"1min":0.67100000000000004,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.1859999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.319,"5min":2.137,"15min":2.137},"last":1.159}]},{"osd":3,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.272,"15min":1.228},"min":{"1min":0.879,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6120000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.073,"15min":1.0980000000000001},"min":{"1min":0.72199999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":4.0830000000000002},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.238,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7350000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.01},{"interface":"front","average":{"1min":1.419,"5min":1.387,"15min":1.397},"min":{"1min":1.056,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.788,"5min":1.788,"15min":3.5609999999999999},"last":1.266}]},{"osd":5,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.5720000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.119},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.365,"15min":1.3500000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.85999999999999999},"max":{"1min":1.849,"5min":2.1240000000000001,"15min":2.766},"last":1.206}]},{"osd":6,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4890000000000001,"15min":1.4830000000000001},"min":{"1min":1.1299999999999999,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8759999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.411,"15min":1.407},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.0449999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.88,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8580000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.4650000000000001,"15min":1.466},"min":{"1min":1.133,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.085,"5min":2.085,"15min":2.8740000000000001},"last":1.381}]}]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307168,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064672,"statfs":{"total":96636764160,"available":96322224128,"internally_reserved":0,"allocated":1728512,"data_stored":1167962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85899999999999999,"15min":0.88300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2290000000000001,"5min":1.3560000000000001,"15min":1.571},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95499999999999996,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2549999999999999,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.738,"5min":1.738,"15min":2.7719999999999998},"last":1.458}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0860000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.919},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.762,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":1.0069999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3280000000000001,"15min":1.343},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.294,"15min":2.294},"last":1.53},{"interface":"front","average":{"1min":1.169,"5min":1.276,"15min":1.2929999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.609,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.427}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.393,"15min":1.383},"min":{"1min":1.0800000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8520000000000001,"5min":2.0859999999999999,"15min":2.573},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":1.024,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.843,"5min":1.881,"15min":2.8650000000000002},"last":1.587}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.3420000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6160000000000001,"5min":2.2240000000000002,"15min":2.52},"last":1.212},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4410000000000001,"15min":1.4610000000000001},"min":{"1min":1.163,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.909,"5min":2.254,"15min":2.742},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4399999999999999,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7509999999999999,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.645},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4119999999999999,"15min":1.4319999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7090000000000001,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:34:02.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:34:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.939+0000 7fb9052e3700 1 Processor -- start 2023-10-06T11:34:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.939+0000 7fb9052e3700 1 -- start start 2023-10-06T11:34:02.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb9052e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb9000cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb9052e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9000cb4c0 0x7fb9000cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb9052e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000cf100 con 0x7fb9000af3d0 2023-10-06T11:34:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb9052e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9000ce4f0 con 0x7fb9000cb4c0 2023-10-06T11:34:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb9052e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000ce7d0 con 0x7fb9000af000 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb904ae2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb9000af3d0 0x7fb9000690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52370/0 (socket says 172.21.15.46:52370) 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb904ae2700 1 -- 172.21.15.46:0/2260999833 learned_addr learned my addr 172.21.15.46:0/2260999833 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8ff7fe700 1 --2- 172.21.15.46:0/2260999833 >> v2:172.21.15.132:3300/0 conn(0x7fb9000cb4c0 0x7fb9000cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789061332 0 0) 0x7fb9000cf100 con 0x7fb9000af3d0 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8fffff700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb9000cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e8003570 con 0x7fb9000af3d0 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2388977091 0 0) 0x7fb8e8003570 con 0x7fb9000af3d0 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 msgr2=0x7fb9000cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb9000cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 >> v2:172.21.15.132:3300/0 conn(0x7fb9000cb4c0 msgr2=0x7fb9000cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 --2- 172.21.15.46:0/2260999833 >> v2:172.21.15.132:3300/0 conn(0x7fb9000cb4c0 0x7fb9000cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000ceab0 con 0x7fb9000af3d0 2023-10-06T11:34:02.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fb8f80017f0 con 0x7fb9000af3d0 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 >> v1:172.21.15.46:6789/0 conn(0x7fb9000af3d0 legacy=0x7fb9000690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb8e8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e8003fe0 0x7fb8e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8e8004d50 0x7fb8e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000ce4f0 con 0x7fb8e8004d50 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000ce7d0 con 0x7fb8e8003fe0 2023-10-06T11:34:02.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.943+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800ba90 con 0x7fb9000af000 2023-10-06T11:34:02.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8e8004d50 0x7fb8e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb8ff7fe700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e8003fe0 0x7fb8e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb8fffff700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb8e8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 -- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 msgr2=0x7fb8e8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9000af000 0x7fb8e8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 -- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e8003fe0 msgr2=0x7fb8e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:02.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e8003fe0 0x7fb8e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 -- 172.21.15.46:0/2260999833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8003570 con 0x7fb8e8004d50 2023-10-06T11:34:02.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8e8004d50 0x7fb8e80051c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800a970 tx=0x7fb8f8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30140a8da7322a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80070d0 con 0x7fb8e8004d50 2023-10-06T11:34:02.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f8007270 con 0x7fb8e8004d50 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb8feffd700 1 -- 172.21.15.46:0/2260999833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8007630 con 0x7fb8e8004d50 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 -- 172.21.15.46:0/2260999833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8e8004d50 msgr2=0x7fb8e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 --2- 172.21.15.46:0/2260999833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8e8004d50 0x7fb8e80051c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800a970 tx=0x7fb8f8002210 comp rx=0 tx=0).stop 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb904ae2700 1 -- 172.21.15.46:0/2260999833 reap_dead start 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 -- 172.21.15.46:0/2260999833 shutdown_connections 2023-10-06T11:34:02.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 -- 172.21.15.46:0/2260999833 >> 172.21.15.46:0/2260999833 conn(0x7fb9000683b0 msgr2=0x7fb9000d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 -- 172.21.15.46:0/2260999833 shutdown_connections 2023-10-06T11:34:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.947+0000 7fb9052e3700 1 -- 172.21.15.46:0/2260999833 wait complete. 2023-10-06T11:34:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 Processor -- start 2023-10-06T11:34:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 -- start start 2023-10-06T11:34:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8e8004d50 0x7fb9000b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9000b1de0 0x7fb90015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90016a210 con 0x7fb90015c290 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900169590 con 0x7fb9000b1de0 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb9052e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb900169870 con 0x7fb8e8004d50 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.951+0000 7fb904ae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54298/0 (socket says 172.21.15.46:54298) 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 -- 172.21.15.46:0/1590916237 learned_addr learned my addr 172.21.15.46:0/1590916237 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb8fffff700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8e8004d50 0x7fb9000b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb8ff7fe700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9000b1de0 0x7fb90015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 -- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8e8004d50 msgr2=0x7fb9000b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8e8004d50 0x7fb9000b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 -- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9000b1de0 msgr2=0x7fb90015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9000b1de0 0x7fb90015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 -- 172.21.15.46:0/1590916237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900169b50 con 0x7fb90015c290 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb904ae2700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8002250 tx=0x7fb8f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8023070 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f80017b0 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb900169e30 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f801e870 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000c88f0 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.955+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90016a350 con 0x7fb90015c290 2023-10-06T11:34:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.959+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb8f801ebe0 con 0x7fb90015c290 2023-10-06T11:34:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.959+0000 7fb8e7fff700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 0x7fb8ec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.959+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fb8f807e6b0 con 0x7fb90015c290 2023-10-06T11:34:02.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.959+0000 7fb8fffff700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 0x7fb8ec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.959+0000 7fb8fffff700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 0x7fb8ec062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009d20 tx=0x7fb8f0008040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:02.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:02.963+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb90016a350 con 0x7fb90015c290 2023-10-06T11:34:03.134 INFO:tasks.ceph.mds.c.smithi046.stderr:2023-10-06T11:34:03.123+0000 7f12afbb7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:34:03.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.131+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb90015b730 con 0x7fb8ec060670 2023-10-06T11:34:03.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.147+0000 7fb8e7fff700 1 -- 172.21.15.46:0/1590916237 <== mgr.6502 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308438 (secure 0 0 0) 0x7fb8f0014ae0 con 0x7fb8ec060670 2023-10-06T11:34:03.170 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:03.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 msgr2=0x7fb8ec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 0x7fb8ec062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009d20 tx=0x7fb8f0008040 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 msgr2=0x7fb9001565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8002250 tx=0x7fb8f8015000 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 shutdown_connections 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8ec060670 0x7fb8ec062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb90015c290 0x7fb9001565a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9000b1de0 0x7fb90015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 --2- 172.21.15.46:0/1590916237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8e8004d50 0x7fb9000b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 >> 172.21.15.46:0/1590916237 conn(0x7fb9000683b0 msgr2=0x7fb9000c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 shutdown_connections 2023-10-06T11:34:03.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.155+0000 7fb9052e3700 1 -- 172.21.15.46:0/1590916237 wait complete. 2023-10-06T11:34:03.174 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:34:03.185 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:34:02.519186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449784,"kb_used_data":10040,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752524936,"statfs":{"total":773094113280,"available":770585534464,"internally_reserved":0,"allocated":10280960,"data_stored":5777760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002000"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181976+0000","last_change":"2023-10-06T11:33:58.181976+0000","last_active":"2023-10-06T11:33:58.181976+0000","last_peered":"2023-10-06T11:33:58.181976+0000","last_clean":"2023-10-06T11:33:58.181976+0000","last_became_active":"2023-10-06T11:33:58.181666+0000","last_became_peered":"2023-10-06T11:33:58.181666+0000","last_unstale":"2023-10-06T11:33:58.181976+0000","last_undegraded":"2023-10-06T11:33:58.181976+0000","last_fullsized":"2023-10-06T11:33:58.181976+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:59.442916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192208+0000","last_change":"2023-10-06T11:33:58.190885+0000","last_active":"2023-10-06T11:33:58.192208+0000","last_peered":"2023-10-06T11:33:58.192208+0000","last_clean":"2023-10-06T11:33:58.192208+0000","last_became_active":"2023-10-06T11:33:58.190604+0000","last_became_peered":"2023-10-06T11:33:58.190604+0000","last_unstale":"2023-10-06T11:33:58.192208+0000","last_undegraded":"2023-10-06T11:33:58.192208+0000","last_fullsized":"2023-10-06T11:33:58.192208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:15.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.184117+0000","last_change":"2023-10-06T11:33:58.184117+0000","last_active":"2023-10-06T11:33:58.184117+0000","last_peered":"2023-10-06T11:33:58.184117+0000","last_clean":"2023-10-06T11:33:58.184117+0000","last_became_active":"2023-10-06T11:33:58.183494+0000","last_became_peered":"2023-10-06T11:33:58.183494+0000","last_unstale":"2023-10-06T11:33:58.184117+0000","last_undegraded":"2023-10-06T11:33:58.184117+0000","last_fullsized":"2023-10-06T11:33:58.184117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:16.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190305+0000","last_change":"2023-10-06T11:33:58.187409+0000","last_active":"2023-10-06T11:33:58.190305+0000","last_peered":"2023-10-06T11:33:58.190305+0000","last_clean":"2023-10-06T11:33:58.190305+0000","last_became_active":"2023-10-06T11:33:58.187101+0000","last_became_peered":"2023-10-06T11:33:58.187101+0000","last_unstale":"2023-10-06T11:33:58.190305+0000","last_undegraded":"2023-10-06T11:33:58.190305+0000","last_fullsized":"2023-10-06T11:33:58.190305+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:42.117096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.184602+0000","last_change":"2023-10-06T11:33:58.170978+0000","last_active":"2023-10-06T11:33:58.184602+0000","last_peered":"2023-10-06T11:33:58.184602+0000","last_clean":"2023-10-06T11:33:58.184602+0000","last_became_active":"2023-10-06T11:33:58.170593+0000","last_became_peered":"2023-10-06T11:33:58.170593+0000","last_unstale":"2023-10-06T11:33:58.184602+0000","last_undegraded":"2023-10-06T11:33:58.184602+0000","last_fullsized":"2023-10-06T11:33:58.184602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:06.214786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.168216+0000","last_change":"2023-10-06T11:33:58.168216+0000","last_active":"2023-10-06T11:33:58.168216+0000","last_peered":"2023-10-06T11:33:58.168216+0000","last_clean":"2023-10-06T11:33:58.168216+0000","last_became_active":"2023-10-06T11:33:58.167652+0000","last_became_peered":"2023-10-06T11:33:58.167652+0000","last_unstale":"2023-10-06T11:33:58.168216+0000","last_undegraded":"2023-10-06T11:33:58.168216+0000","last_fullsized":"2023-10-06T11:33:58.168216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:31.451651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199306+0000","last_change":"2023-10-06T11:33:58.198211+0000","last_active":"2023-10-06T11:33:58.199306+0000","last_peered":"2023-10-06T11:33:58.199306+0000","last_clean":"2023-10-06T11:33:58.199306+0000","last_became_active":"2023-10-06T11:33:58.197875+0000","last_became_peered":"2023-10-06T11:33:58.197875+0000","last_unstale":"2023-10-06T11:33:58.199306+0000","last_undegraded":"2023-10-06T11:33:58.199306+0000","last_fullsized":"2023-10-06T11:33:58.199306+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:05.108545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.180507+0000","last_change":"2023-10-06T11:33:58.171894+0000","last_active":"2023-10-06T11:33:58.180507+0000","last_peered":"2023-10-06T11:33:58.180507+0000","last_clean":"2023-10-06T11:33:58.180507+0000","last_became_active":"2023-10-06T11:33:58.171560+0000","last_became_peered":"2023-10-06T11:33:58.171560+0000","last_unstale":"2023-10-06T11:33:58.180507+0000","last_undegraded":"2023-10-06T11:33:58.180507+0000","last_fullsized":"2023-10-06T11:33:58.180507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:13.049151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190031+0000","last_change":"2023-10-06T11:33:58.176888+0000","last_active":"2023-10-06T11:33:58.190031+0000","last_peered":"2023-10-06T11:33:58.190031+0000","last_clean":"2023-10-06T11:33:58.190031+0000","last_became_active":"2023-10-06T11:33:58.176556+0000","last_became_peered":"2023-10-06T11:33:58.176556+0000","last_unstale":"2023-10-06T11:33:58.190031+0000","last_undegraded":"2023-10-06T11:33:58.190031+0000","last_fullsized":"2023-10-06T11:33:58.190031+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:30.766304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187512+0000","last_change":"2023-10-06T11:33:58.184415+0000","last_active":"2023-10-06T11:33:58.187512+0000","last_peered":"2023-10-06T11:33:58.187512+0000","last_clean":"2023-10-06T11:33:58.187512+0000","last_became_active":"2023-10-06T11:33:58.184078+0000","last_became_peered":"2023-10-06T11:33:58.184078+0000","last_unstale":"2023-10-06T11:33:58.187512+0000","last_undegraded":"2023-10-06T11:33:58.187512+0000","last_fullsized":"2023-10-06T11:33:58.187512+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:01.436585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.193626+0000","last_change":"2023-10-06T11:33:58.192689+0000","last_active":"2023-10-06T11:33:58.193626+0000","last_peered":"2023-10-06T11:33:58.193626+0000","last_clean":"2023-10-06T11:33:58.193626+0000","last_became_active":"2023-10-06T11:33:58.192366+0000","last_became_peered":"2023-10-06T11:33:58.192366+0000","last_unstale":"2023-10-06T11:33:58.193626+0000","last_undegraded":"2023-10-06T11:33:58.193626+0000","last_fullsized":"2023-10-06T11:33:58.193626+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:13:11.429657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192839+0000","last_change":"2023-10-06T11:33:58.190623+0000","last_active":"2023-10-06T11:33:58.192839+0000","last_peered":"2023-10-06T11:33:58.192839+0000","last_clean":"2023-10-06T11:33:58.192839+0000","last_became_active":"2023-10-06T11:33:58.190112+0000","last_became_peered":"2023-10-06T11:33:58.190112+0000","last_unstale":"2023-10-06T11:33:58.192839+0000","last_undegraded":"2023-10-06T11:33:58.192839+0000","last_fullsized":"2023-10-06T11:33:58.192839+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:48.486298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.191889+0000","last_change":"2023-10-06T11:33:58.184927+0000","last_active":"2023-10-06T11:33:58.191889+0000","last_peered":"2023-10-06T11:33:58.191889+0000","last_clean":"2023-10-06T11:33:58.191889+0000","last_became_active":"2023-10-06T11:33:58.177334+0000","last_became_peered":"2023-10-06T11:33:58.177334+0000","last_unstale":"2023-10-06T11:33:58.191889+0000","last_undegraded":"2023-10-06T11:33:58.191889+0000","last_fullsized":"2023-10-06T11:33:58.191889+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:52.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.167961+0000","last_change":"2023-10-06T11:33:58.167961+0000","last_active":"2023-10-06T11:33:58.167961+0000","last_peered":"2023-10-06T11:33:58.167961+0000","last_clean":"2023-10-06T11:33:58.167961+0000","last_became_active":"2023-10-06T11:33:58.167249+0000","last_became_peered":"2023-10-06T11:33:58.167249+0000","last_unstale":"2023-10-06T11:33:58.167961+0000","last_undegraded":"2023-10-06T11:33:58.167961+0000","last_fullsized":"2023-10-06T11:33:58.167961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:01.509004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187298+0000","last_change":"2023-10-06T11:33:58.182499+0000","last_active":"2023-10-06T11:33:58.187298+0000","last_peered":"2023-10-06T11:33:58.187298+0000","last_clean":"2023-10-06T11:33:58.187298+0000","last_became_active":"2023-10-06T11:33:58.182163+0000","last_became_peered":"2023-10-06T11:33:58.182163+0000","last_unstale":"2023-10-06T11:33:58.187298+0000","last_undegraded":"2023-10-06T11:33:58.187298+0000","last_fullsized":"2023-10-06T11:33:58.187298+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:11.018913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.179861+0000","last_change":"2023-10-06T11:33:58.179861+0000","last_active":"2023-10-06T11:33:58.179861+0000","last_peered":"2023-10-06T11:33:58.179861+0000","last_clean":"2023-10-06T11:33:58.179861+0000","last_became_active":"2023-10-06T11:33:58.179542+0000","last_became_peered":"2023-10-06T11:33:58.179542+0000","last_unstale":"2023-10-06T11:33:58.179861+0000","last_undegraded":"2023-10-06T11:33:58.179861+0000","last_fullsized":"2023-10-06T11:33:58.179861+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:41.889801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190256+0000","last_change":"2023-10-06T11:33:58.183519+0000","last_active":"2023-10-06T11:33:58.190256+0000","last_peered":"2023-10-06T11:33:58.190256+0000","last_clean":"2023-10-06T11:33:58.190256+0000","last_became_active":"2023-10-06T11:33:58.183171+0000","last_became_peered":"2023-10-06T11:33:58.183171+0000","last_unstale":"2023-10-06T11:33:58.190256+0000","last_undegraded":"2023-10-06T11:33:58.190256+0000","last_fullsized":"2023-10-06T11:33:58.190256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:07:16.463863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182069+0000","last_change":"2023-10-06T11:33:58.182069+0000","last_active":"2023-10-06T11:33:58.182069+0000","last_peered":"2023-10-06T11:33:58.182069+0000","last_clean":"2023-10-06T11:33:58.182069+0000","last_became_active":"2023-10-06T11:33:58.181758+0000","last_became_peered":"2023-10-06T11:33:58.181758+0000","last_unstale":"2023-10-06T11:33:58.182069+0000","last_undegraded":"2023-10-06T11:33:58.182069+0000","last_fullsized":"2023-10-06T11:33:58.182069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:13.049151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175361+0000","last_change":"2023-10-06T11:33:58.175361+0000","last_active":"2023-10-06T11:33:58.175361+0000","last_peered":"2023-10-06T11:33:58.175361+0000","last_clean":"2023-10-06T11:33:58.175361+0000","last_became_active":"2023-10-06T11:33:58.174901+0000","last_became_peered":"2023-10-06T11:33:58.174901+0000","last_unstale":"2023-10-06T11:33:58.175361+0000","last_undegraded":"2023-10-06T11:33:58.175361+0000","last_fullsized":"2023-10-06T11:33:58.175361+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:33.046664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.191808+0000","last_change":"2023-10-06T11:33:58.185048+0000","last_active":"2023-10-06T11:33:58.191808+0000","last_peered":"2023-10-06T11:33:58.191808+0000","last_clean":"2023-10-06T11:33:58.191808+0000","last_became_active":"2023-10-06T11:33:58.184708+0000","last_became_peered":"2023-10-06T11:33:58.184708+0000","last_unstale":"2023-10-06T11:33:58.191808+0000","last_undegraded":"2023-10-06T11:33:58.191808+0000","last_fullsized":"2023-10-06T11:33:58.191808+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.931569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192544+0000","last_change":"2023-10-06T11:33:58.187941+0000","last_active":"2023-10-06T11:33:58.192544+0000","last_peered":"2023-10-06T11:33:58.192544+0000","last_clean":"2023-10-06T11:33:58.192544+0000","last_became_active":"2023-10-06T11:33:58.187258+0000","last_became_peered":"2023-10-06T11:33:58.187258+0000","last_unstale":"2023-10-06T11:33:58.192544+0000","last_undegraded":"2023-10-06T11:33:58.192544+0000","last_fullsized":"2023-10-06T11:33:58.192544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.721768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190043+0000","last_change":"2023-10-06T11:33:58.177701+0000","last_active":"2023-10-06T11:33:58.190043+0000","last_peered":"2023-10-06T11:33:58.190043+0000","last_clean":"2023-10-06T11:33:58.190043+0000","last_became_active":"2023-10-06T11:33:58.177346+0000","last_became_peered":"2023-10-06T11:33:58.177346+0000","last_unstale":"2023-10-06T11:33:58.190043+0000","last_undegraded":"2023-10-06T11:33:58.190043+0000","last_fullsized":"2023-10-06T11:33:58.190043+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:17.057817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190376+0000","last_change":"2023-10-06T11:33:58.181156+0000","last_active":"2023-10-06T11:33:58.190376+0000","last_peered":"2023-10-06T11:33:58.190376+0000","last_clean":"2023-10-06T11:33:58.190376+0000","last_became_active":"2023-10-06T11:33:58.180838+0000","last_became_peered":"2023-10-06T11:33:58.180838+0000","last_unstale":"2023-10-06T11:33:58.190376+0000","last_undegraded":"2023-10-06T11:33:58.190376+0000","last_fullsized":"2023-10-06T11:33:58.190376+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:10.076262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185441+0000","last_change":"2023-10-06T11:33:58.180778+0000","last_active":"2023-10-06T11:33:58.185441+0000","last_peered":"2023-10-06T11:33:58.185441+0000","last_clean":"2023-10-06T11:33:58.185441+0000","last_became_active":"2023-10-06T11:33:58.180250+0000","last_became_peered":"2023-10-06T11:33:58.180250+0000","last_unstale":"2023-10-06T11:33:58.185441+0000","last_undegraded":"2023-10-06T11:33:58.185441+0000","last_fullsized":"2023-10-06T11:33:58.185441+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:23.987851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192232+0000","last_change":"2023-10-06T11:33:58.190460+0000","last_active":"2023-10-06T11:33:58.192232+0000","last_peered":"2023-10-06T11:33:58.192232+0000","last_clean":"2023-10-06T11:33:58.192232+0000","last_became_active":"2023-10-06T11:33:58.190146+0000","last_became_peered":"2023-10-06T11:33:58.190146+0000","last_unstale":"2023-10-06T11:33:58.192232+0000","last_undegraded":"2023-10-06T11:33:58.192232+0000","last_fullsized":"2023-10-06T11:33:58.192232+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:11.565286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195714+0000","last_change":"2023-10-06T11:33:58.194181+0000","last_active":"2023-10-06T11:33:58.195714+0000","last_peered":"2023-10-06T11:33:58.195714+0000","last_clean":"2023-10-06T11:33:58.195714+0000","last_became_active":"2023-10-06T11:33:58.193882+0000","last_became_peered":"2023-10-06T11:33:58.193882+0000","last_unstale":"2023-10-06T11:33:58.195714+0000","last_undegraded":"2023-10-06T11:33:58.195714+0000","last_fullsized":"2023-10-06T11:33:58.195714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:55.056089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.179816+0000","last_change":"2023-10-06T11:33:58.179816+0000","last_active":"2023-10-06T11:33:58.179816+0000","last_peered":"2023-10-06T11:33:58.179816+0000","last_clean":"2023-10-06T11:33:58.179816+0000","last_became_active":"2023-10-06T11:33:58.179502+0000","last_became_peered":"2023-10-06T11:33:58.179502+0000","last_unstale":"2023-10-06T11:33:58.179816+0000","last_undegraded":"2023-10-06T11:33:58.179816+0000","last_fullsized":"2023-10-06T11:33:58.179816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:02:52.028195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181170+0000","last_change":"2023-10-06T11:33:58.173468+0000","last_active":"2023-10-06T11:33:58.181170+0000","last_peered":"2023-10-06T11:33:58.181170+0000","last_clean":"2023-10-06T11:33:58.181170+0000","last_became_active":"2023-10-06T11:33:58.173133+0000","last_became_peered":"2023-10-06T11:33:58.173133+0000","last_unstale":"2023-10-06T11:33:58.181170+0000","last_undegraded":"2023-10-06T11:33:58.181170+0000","last_fullsized":"2023-10-06T11:33:58.181170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:06:08.100971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187162+0000","last_change":"2023-10-06T11:33:58.181435+0000","last_active":"2023-10-06T11:33:58.187162+0000","last_peered":"2023-10-06T11:33:58.187162+0000","last_clean":"2023-10-06T11:33:58.187162+0000","last_became_active":"2023-10-06T11:33:58.181119+0000","last_became_peered":"2023-10-06T11:33:58.181119+0000","last_unstale":"2023-10-06T11:33:58.187162+0000","last_undegraded":"2023-10-06T11:33:58.187162+0000","last_fullsized":"2023-10-06T11:33:58.187162+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:30.729845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.189840+0000","last_change":"2023-10-06T11:33:58.189840+0000","last_active":"2023-10-06T11:33:58.189840+0000","last_peered":"2023-10-06T11:33:58.189840+0000","last_clean":"2023-10-06T11:33:58.189840+0000","last_became_active":"2023-10-06T11:33:58.189529+0000","last_became_peered":"2023-10-06T11:33:58.189529+0000","last_unstale":"2023-10-06T11:33:58.189840+0000","last_undegraded":"2023-10-06T11:33:58.189840+0000","last_fullsized":"2023-10-06T11:33:58.189840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:30.440178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.178643+0000","last_change":"2023-10-06T11:33:58.178643+0000","last_active":"2023-10-06T11:33:58.178643+0000","last_peered":"2023-10-06T11:33:58.178643+0000","last_clean":"2023-10-06T11:33:58.178643+0000","last_became_active":"2023-10-06T11:33:58.178318+0000","last_became_peered":"2023-10-06T11:33:58.178318+0000","last_unstale":"2023-10-06T11:33:58.178643+0000","last_undegraded":"2023-10-06T11:33:58.178643+0000","last_fullsized":"2023-10-06T11:33:58.178643+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:13.931569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.176231+0000","last_change":"2023-10-06T11:33:58.176231+0000","last_active":"2023-10-06T11:33:58.176231+0000","last_peered":"2023-10-06T11:33:58.176231+0000","last_clean":"2023-10-06T11:33:58.176231+0000","last_became_active":"2023-10-06T11:33:58.174941+0000","last_became_peered":"2023-10-06T11:33:58.174941+0000","last_unstale":"2023-10-06T11:33:58.176231+0000","last_undegraded":"2023-10-06T11:33:58.176231+0000","last_fullsized":"2023-10-06T11:33:58.176231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:35.089621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182566+0000","last_change":"2023-10-06T11:33:58.182566+0000","last_active":"2023-10-06T11:33:58.182566+0000","last_peered":"2023-10-06T11:33:58.182566+0000","last_clean":"2023-10-06T11:33:58.182566+0000","last_became_active":"2023-10-06T11:33:58.182266+0000","last_became_peered":"2023-10-06T11:33:58.182266+0000","last_unstale":"2023-10-06T11:33:58.182566+0000","last_undegraded":"2023-10-06T11:33:58.182566+0000","last_fullsized":"2023-10-06T11:33:58.182566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:52.464514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'4","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.495610+0000","last_change":"2023-10-06T11:33:57.184229+0000","last_active":"2023-10-06T11:33:58.495610+0000","last_peered":"2023-10-06T11:33:58.495610+0000","last_clean":"2023-10-06T11:33:58.495610+0000","last_became_active":"2023-10-06T11:33:57.183942+0000","last_became_peered":"2023-10-06T11:33:57.183942+0000","last_unstale":"2023-10-06T11:33:58.495610+0000","last_undegraded":"2023-10-06T11:33:58.495610+0000","last_fullsized":"2023-10-06T11:33:58.495610+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:35:22.187051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185270+0000","last_change":"2023-10-06T11:33:58.180175+0000","last_active":"2023-10-06T11:33:58.185270+0000","last_peered":"2023-10-06T11:33:58.185270+0000","last_clean":"2023-10-06T11:33:58.185270+0000","last_became_active":"2023-10-06T11:33:58.179820+0000","last_became_peered":"2023-10-06T11:33:58.179820+0000","last_unstale":"2023-10-06T11:33:58.185270+0000","last_undegraded":"2023-10-06T11:33:58.185270+0000","last_fullsized":"2023-10-06T11:33:58.185270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:36.711635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169159+0000","last_change":"2023-10-06T11:33:57.192091+0000","last_active":"2023-10-06T11:33:58.169159+0000","last_peered":"2023-10-06T11:33:58.169159+0000","last_clean":"2023-10-06T11:33:58.169159+0000","last_became_active":"2023-10-06T11:33:57.191595+0000","last_became_peered":"2023-10-06T11:33:57.191595+0000","last_unstale":"2023-10-06T11:33:58.169159+0000","last_undegraded":"2023-10-06T11:33:58.169159+0000","last_fullsized":"2023-10-06T11:33:58.169159+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:43.705070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190090+0000","last_change":"2023-10-06T11:33:58.177028+0000","last_active":"2023-10-06T11:33:58.190090+0000","last_peered":"2023-10-06T11:33:58.190090+0000","last_clean":"2023-10-06T11:33:58.190090+0000","last_became_active":"2023-10-06T11:33:58.176509+0000","last_became_peered":"2023-10-06T11:33:58.176509+0000","last_unstale":"2023-10-06T11:33:58.190090+0000","last_undegraded":"2023-10-06T11:33:58.190090+0000","last_fullsized":"2023-10-06T11:33:58.190090+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:18.903134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.500440+0000","last_change":"2023-10-06T11:33:57.197048+0000","last_active":"2023-10-06T11:33:58.500440+0000","last_peered":"2023-10-06T11:33:58.500440+0000","last_clean":"2023-10-06T11:33:58.500440+0000","last_became_active":"2023-10-06T11:33:57.196746+0000","last_became_peered":"2023-10-06T11:33:57.196746+0000","last_unstale":"2023-10-06T11:33:58.500440+0000","last_undegraded":"2023-10-06T11:33:58.500440+0000","last_fullsized":"2023-10-06T11:33:58.500440+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:00.492306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192564+0000","last_change":"2023-10-06T11:33:58.188131+0000","last_active":"2023-10-06T11:33:58.192564+0000","last_peered":"2023-10-06T11:33:58.192564+0000","last_clean":"2023-10-06T11:33:58.192564+0000","last_became_active":"2023-10-06T11:33:58.187649+0000","last_became_peered":"2023-10-06T11:33:58.187649+0000","last_unstale":"2023-10-06T11:33:58.192564+0000","last_undegraded":"2023-10-06T11:33:58.192564+0000","last_fullsized":"2023-10-06T11:33:58.192564+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:37.726524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504868+0000","last_change":"2023-10-06T11:33:57.194451+0000","last_active":"2023-10-06T11:33:58.504868+0000","last_peered":"2023-10-06T11:33:58.504868+0000","last_clean":"2023-10-06T11:33:58.504868+0000","last_became_active":"2023-10-06T11:33:57.194016+0000","last_became_peered":"2023-10-06T11:33:57.194016+0000","last_unstale":"2023-10-06T11:33:58.504868+0000","last_undegraded":"2023-10-06T11:33:58.504868+0000","last_fullsized":"2023-10-06T11:33:58.504868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:33.769943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192990+0000","last_change":"2023-10-06T11:33:58.191525+0000","last_active":"2023-10-06T11:33:58.192990+0000","last_peered":"2023-10-06T11:33:58.192990+0000","last_clean":"2023-10-06T11:33:58.192990+0000","last_became_active":"2023-10-06T11:33:58.191239+0000","last_became_peered":"2023-10-06T11:33:58.191239+0000","last_unstale":"2023-10-06T11:33:58.192990+0000","last_undegraded":"2023-10-06T11:33:58.192990+0000","last_fullsized":"2023-10-06T11:33:58.192990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.317821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.500190+0000","last_change":"2023-10-06T11:33:57.190426+0000","last_active":"2023-10-06T11:33:58.500190+0000","last_peered":"2023-10-06T11:33:58.500190+0000","last_clean":"2023-10-06T11:33:58.500190+0000","last_became_active":"2023-10-06T11:33:57.189292+0000","last_became_peered":"2023-10-06T11:33:57.189292+0000","last_unstale":"2023-10-06T11:33:58.500190+0000","last_undegraded":"2023-10-06T11:33:58.500190+0000","last_fullsized":"2023-10-06T11:33:58.500190+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:36.161187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188108+0000","last_change":"2023-10-06T11:33:58.186633+0000","last_active":"2023-10-06T11:33:58.188108+0000","last_peered":"2023-10-06T11:33:58.188108+0000","last_clean":"2023-10-06T11:33:58.188108+0000","last_became_active":"2023-10-06T11:33:58.186357+0000","last_became_peered":"2023-10-06T11:33:58.186357+0000","last_unstale":"2023-10-06T11:33:58.188108+0000","last_undegraded":"2023-10-06T11:33:58.188108+0000","last_fullsized":"2023-10-06T11:33:58.188108+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:54.945156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504368+0000","last_change":"2023-10-06T11:33:57.200315+0000","last_active":"2023-10-06T11:33:58.504368+0000","last_peered":"2023-10-06T11:33:58.504368+0000","last_clean":"2023-10-06T11:33:58.504368+0000","last_became_active":"2023-10-06T11:33:57.200040+0000","last_became_peered":"2023-10-06T11:33:57.200040+0000","last_unstale":"2023-10-06T11:33:58.504368+0000","last_undegraded":"2023-10-06T11:33:58.504368+0000","last_fullsized":"2023-10-06T11:33:58.504368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:42.920564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188658+0000","last_change":"2023-10-06T11:33:58.177367+0000","last_active":"2023-10-06T11:33:58.188658+0000","last_peered":"2023-10-06T11:33:58.188658+0000","last_clean":"2023-10-06T11:33:58.188658+0000","last_became_active":"2023-10-06T11:33:58.177049+0000","last_became_peered":"2023-10-06T11:33:58.177049+0000","last_unstale":"2023-10-06T11:33:58.188658+0000","last_undegraded":"2023-10-06T11:33:58.188658+0000","last_fullsized":"2023-10-06T11:33:58.188658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:43.937262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.168645+0000","last_change":"2023-10-06T11:33:57.188577+0000","last_active":"2023-10-06T11:33:58.168645+0000","last_peered":"2023-10-06T11:33:58.168645+0000","last_clean":"2023-10-06T11:33:58.168645+0000","last_became_active":"2023-10-06T11:33:57.186811+0000","last_became_peered":"2023-10-06T11:33:57.186811+0000","last_unstale":"2023-10-06T11:33:58.168645+0000","last_undegraded":"2023-10-06T11:33:58.168645+0000","last_fullsized":"2023-10-06T11:33:58.168645+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:47.469600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199320+0000","last_change":"2023-10-06T11:33:58.198279+0000","last_active":"2023-10-06T11:33:58.199320+0000","last_peered":"2023-10-06T11:33:58.199320+0000","last_clean":"2023-10-06T11:33:58.199320+0000","last_became_active":"2023-10-06T11:33:58.197978+0000","last_became_peered":"2023-10-06T11:33:58.197978+0000","last_unstale":"2023-10-06T11:33:58.199320+0000","last_undegraded":"2023-10-06T11:33:58.199320+0000","last_fullsized":"2023-10-06T11:33:58.199320+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:43.937262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.166998+0000","last_change":"2023-10-06T11:33:57.182971+0000","last_active":"2023-10-06T11:33:58.166998+0000","last_peered":"2023-10-06T11:33:58.166998+0000","last_clean":"2023-10-06T11:33:58.166998+0000","last_became_active":"2023-10-06T11:33:57.182671+0000","last_became_peered":"2023-10-06T11:33:57.182671+0000","last_unstale":"2023-10-06T11:33:58.166998+0000","last_undegraded":"2023-10-06T11:33:58.166998+0000","last_fullsized":"2023-10-06T11:33:58.166998+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:52.119107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.185421+0000","last_change":"2023-10-06T11:33:58.180576+0000","last_active":"2023-10-06T11:33:58.185421+0000","last_peered":"2023-10-06T11:33:58.185421+0000","last_clean":"2023-10-06T11:33:58.185421+0000","last_became_active":"2023-10-06T11:33:58.179868+0000","last_became_peered":"2023-10-06T11:33:58.179868+0000","last_unstale":"2023-10-06T11:33:58.185421+0000","last_undegraded":"2023-10-06T11:33:58.185421+0000","last_fullsized":"2023-10-06T11:33:58.185421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:09.385892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195652+0000","last_change":"2023-10-06T11:33:57.191619+0000","last_active":"2023-10-06T11:33:57.195652+0000","last_peered":"2023-10-06T11:33:57.195652+0000","last_clean":"2023-10-06T11:33:57.195652+0000","last_became_active":"2023-10-06T11:33:57.191268+0000","last_became_peered":"2023-10-06T11:33:57.191268+0000","last_unstale":"2023-10-06T11:33:57.195652+0000","last_undegraded":"2023-10-06T11:33:57.195652+0000","last_fullsized":"2023-10-06T11:33:57.195652+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:36.709826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190990+0000","last_change":"2023-10-06T11:33:58.188862+0000","last_active":"2023-10-06T11:33:58.190990+0000","last_peered":"2023-10-06T11:33:58.190990+0000","last_clean":"2023-10-06T11:33:58.190990+0000","last_became_active":"2023-10-06T11:33:58.188510+0000","last_became_peered":"2023-10-06T11:33:58.188510+0000","last_unstale":"2023-10-06T11:33:58.190990+0000","last_undegraded":"2023-10-06T11:33:58.190990+0000","last_fullsized":"2023-10-06T11:33:58.190990+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:17.413801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175992+0000","last_change":"2023-10-06T11:33:57.190428+0000","last_active":"2023-10-06T11:33:58.175992+0000","last_peered":"2023-10-06T11:33:58.175992+0000","last_clean":"2023-10-06T11:33:58.175992+0000","last_became_active":"2023-10-06T11:33:57.190161+0000","last_became_peered":"2023-10-06T11:33:57.190161+0000","last_unstale":"2023-10-06T11:33:58.175992+0000","last_undegraded":"2023-10-06T11:33:58.175992+0000","last_fullsized":"2023-10-06T11:33:58.175992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:56.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.180054+0000","last_change":"2023-10-06T11:33:58.180054+0000","last_active":"2023-10-06T11:33:58.180054+0000","last_peered":"2023-10-06T11:33:58.180054+0000","last_clean":"2023-10-06T11:33:58.180054+0000","last_became_active":"2023-10-06T11:33:58.179758+0000","last_became_peered":"2023-10-06T11:33:58.179758+0000","last_unstale":"2023-10-06T11:33:58.180054+0000","last_undegraded":"2023-10-06T11:33:58.180054+0000","last_fullsized":"2023-10-06T11:33:58.180054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:17.413801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.495042+0000","last_change":"2023-10-06T11:33:57.187332+0000","last_active":"2023-10-06T11:33:58.495042+0000","last_peered":"2023-10-06T11:33:58.495042+0000","last_clean":"2023-10-06T11:33:58.495042+0000","last_became_active":"2023-10-06T11:33:57.187041+0000","last_became_peered":"2023-10-06T11:33:57.187041+0000","last_unstale":"2023-10-06T11:33:58.495042+0000","last_undegraded":"2023-10-06T11:33:58.495042+0000","last_fullsized":"2023-10-06T11:33:58.495042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:40:28.849818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187184+0000","last_change":"2023-10-06T11:33:58.187184+0000","last_active":"2023-10-06T11:33:58.187184+0000","last_peered":"2023-10-06T11:33:58.187184+0000","last_clean":"2023-10-06T11:33:58.187184+0000","last_became_active":"2023-10-06T11:33:58.186914+0000","last_became_peered":"2023-10-06T11:33:58.186914+0000","last_unstale":"2023-10-06T11:33:58.187184+0000","last_undegraded":"2023-10-06T11:33:58.187184+0000","last_fullsized":"2023-10-06T11:33:58.187184+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.179000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.506724+0000","last_change":"2023-10-06T11:33:57.190206+0000","last_active":"2023-10-06T11:33:58.506724+0000","last_peered":"2023-10-06T11:33:58.506724+0000","last_clean":"2023-10-06T11:33:58.506724+0000","last_became_active":"2023-10-06T11:33:57.189908+0000","last_became_peered":"2023-10-06T11:33:57.189908+0000","last_unstale":"2023-10-06T11:33:58.506724+0000","last_undegraded":"2023-10-06T11:33:58.506724+0000","last_fullsized":"2023-10-06T11:33:58.506724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:05.198088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188157+0000","last_change":"2023-10-06T11:33:58.185076+0000","last_active":"2023-10-06T11:33:58.188157+0000","last_peered":"2023-10-06T11:33:58.188157+0000","last_clean":"2023-10-06T11:33:58.188157+0000","last_became_active":"2023-10-06T11:33:58.184760+0000","last_became_peered":"2023-10-06T11:33:58.184760+0000","last_unstale":"2023-10-06T11:33:58.188157+0000","last_undegraded":"2023-10-06T11:33:58.188157+0000","last_fullsized":"2023-10-06T11:33:58.188157+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:56:31.451651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.505026+0000","last_change":"2023-10-06T11:33:57.188872+0000","last_active":"2023-10-06T11:33:58.505026+0000","last_peered":"2023-10-06T11:33:58.505026+0000","last_clean":"2023-10-06T11:33:58.505026+0000","last_became_active":"2023-10-06T11:33:57.188578+0000","last_became_peered":"2023-10-06T11:33:57.188578+0000","last_unstale":"2023-10-06T11:33:58.505026+0000","last_undegraded":"2023-10-06T11:33:58.505026+0000","last_fullsized":"2023-10-06T11:33:58.505026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:49.753592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195589+0000","last_change":"2023-10-06T11:33:58.189505+0000","last_active":"2023-10-06T11:33:58.195589+0000","last_peered":"2023-10-06T11:33:58.195589+0000","last_clean":"2023-10-06T11:33:58.195589+0000","last_became_active":"2023-10-06T11:33:58.189215+0000","last_became_peered":"2023-10-06T11:33:58.189215+0000","last_unstale":"2023-10-06T11:33:58.195589+0000","last_undegraded":"2023-10-06T11:33:58.195589+0000","last_fullsized":"2023-10-06T11:33:58.195589+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:57.303486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192749+0000","last_change":"2023-10-06T11:33:57.189231+0000","last_active":"2023-10-06T11:33:57.192749+0000","last_peered":"2023-10-06T11:33:57.192749+0000","last_clean":"2023-10-06T11:33:57.192749+0000","last_became_active":"2023-10-06T11:33:57.188918+0000","last_became_peered":"2023-10-06T11:33:57.188918+0000","last_unstale":"2023-10-06T11:33:57.192749+0000","last_undegraded":"2023-10-06T11:33:57.192749+0000","last_fullsized":"2023-10-06T11:33:57.192749+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:19:10.521490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195607+0000","last_change":"2023-10-06T11:33:58.189670+0000","last_active":"2023-10-06T11:33:58.195607+0000","last_peered":"2023-10-06T11:33:58.195607+0000","last_clean":"2023-10-06T11:33:58.195607+0000","last_became_active":"2023-10-06T11:33:58.189383+0000","last_became_peered":"2023-10-06T11:33:58.189383+0000","last_unstale":"2023-10-06T11:33:58.195607+0000","last_undegraded":"2023-10-06T11:33:58.195607+0000","last_fullsized":"2023-10-06T11:33:58.195607+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:52.183355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.170439+0000","last_change":"2023-10-06T11:33:57.184682+0000","last_active":"2023-10-06T11:33:58.170439+0000","last_peered":"2023-10-06T11:33:58.170439+0000","last_clean":"2023-10-06T11:33:58.170439+0000","last_became_active":"2023-10-06T11:33:57.184401+0000","last_became_peered":"2023-10-06T11:33:57.184401+0000","last_unstale":"2023-10-06T11:33:58.170439+0000","last_undegraded":"2023-10-06T11:33:58.170439+0000","last_fullsized":"2023-10-06T11:33:58.170439+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:32.830008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195642+0000","last_change":"2023-10-06T11:33:58.190331+0000","last_active":"2023-10-06T11:33:58.195642+0000","last_peered":"2023-10-06T11:33:58.195642+0000","last_clean":"2023-10-06T11:33:58.195642+0000","last_became_active":"2023-10-06T11:33:58.190056+0000","last_became_peered":"2023-10-06T11:33:58.190056+0000","last_unstale":"2023-10-06T11:33:58.195642+0000","last_undegraded":"2023-10-06T11:33:58.195642+0000","last_fullsized":"2023-10-06T11:33:58.195642+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:05.974947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'8","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.497758+0000","last_change":"2023-10-06T11:33:57.185775+0000","last_active":"2023-10-06T11:33:58.497758+0000","last_peered":"2023-10-06T11:33:58.497758+0000","last_clean":"2023-10-06T11:33:58.497758+0000","last_became_active":"2023-10-06T11:33:57.185479+0000","last_became_peered":"2023-10-06T11:33:57.185479+0000","last_unstale":"2023-10-06T11:33:58.497758+0000","last_undegraded":"2023-10-06T11:33:58.497758+0000","last_fullsized":"2023-10-06T11:33:58.497758+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:16:41.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.192458+0000","last_change":"2023-10-06T11:33:58.186614+0000","last_active":"2023-10-06T11:33:58.192458+0000","last_peered":"2023-10-06T11:33:58.192458+0000","last_clean":"2023-10-06T11:33:58.192458+0000","last_became_active":"2023-10-06T11:33:58.185073+0000","last_became_peered":"2023-10-06T11:33:58.185073+0000","last_unstale":"2023-10-06T11:33:58.192458+0000","last_undegraded":"2023-10-06T11:33:58.192458+0000","last_fullsized":"2023-10-06T11:33:58.192458+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.504533+0000","last_change":"2023-10-06T11:33:57.199599+0000","last_active":"2023-10-06T11:33:58.504533+0000","last_peered":"2023-10-06T11:33:58.504533+0000","last_clean":"2023-10-06T11:33:58.504533+0000","last_became_active":"2023-10-06T11:33:57.199203+0000","last_became_peered":"2023-10-06T11:33:57.199203+0000","last_unstale":"2023-10-06T11:33:58.504533+0000","last_undegraded":"2023-10-06T11:33:58.504533+0000","last_fullsized":"2023-10-06T11:33:58.504533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:29.749606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.195522+0000","last_change":"2023-10-06T11:33:58.188312+0000","last_active":"2023-10-06T11:33:58.195522+0000","last_peered":"2023-10-06T11:33:58.195522+0000","last_clean":"2023-10-06T11:33:58.195522+0000","last_became_active":"2023-10-06T11:33:58.188021+0000","last_became_peered":"2023-10-06T11:33:58.188021+0000","last_unstale":"2023-10-06T11:33:58.195522+0000","last_undegraded":"2023-10-06T11:33:58.195522+0000","last_fullsized":"2023-10-06T11:33:58.195522+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:27.099347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.173488+0000","last_change":"2023-10-06T11:33:57.188316+0000","last_active":"2023-10-06T11:33:58.173488+0000","last_peered":"2023-10-06T11:33:58.173488+0000","last_clean":"2023-10-06T11:33:58.173488+0000","last_became_active":"2023-10-06T11:33:57.187978+0000","last_became_peered":"2023-10-06T11:33:57.187978+0000","last_unstale":"2023-10-06T11:33:58.173488+0000","last_undegraded":"2023-10-06T11:33:58.173488+0000","last_fullsized":"2023-10-06T11:33:58.173488+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:54.039391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187043+0000","last_change":"2023-10-06T11:33:58.176814+0000","last_active":"2023-10-06T11:33:58.187043+0000","last_peered":"2023-10-06T11:33:58.187043+0000","last_clean":"2023-10-06T11:33:58.187043+0000","last_became_active":"2023-10-06T11:33:58.175739+0000","last_became_peered":"2023-10-06T11:33:58.175739+0000","last_unstale":"2023-10-06T11:33:58.187043+0000","last_undegraded":"2023-10-06T11:33:58.187043+0000","last_fullsized":"2023-10-06T11:33:58.187043+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:57.303486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195581+0000","last_change":"2023-10-06T11:33:57.190718+0000","last_active":"2023-10-06T11:33:57.195581+0000","last_peered":"2023-10-06T11:33:57.195581+0000","last_clean":"2023-10-06T11:33:57.195581+0000","last_became_active":"2023-10-06T11:33:57.190424+0000","last_became_peered":"2023-10-06T11:33:57.190424+0000","last_unstale":"2023-10-06T11:33:57.195581+0000","last_undegraded":"2023-10-06T11:33:57.195581+0000","last_fullsized":"2023-10-06T11:33:57.195581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:02:56.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187185+0000","last_change":"2023-10-06T11:33:58.182064+0000","last_active":"2023-10-06T11:33:58.187185+0000","last_peered":"2023-10-06T11:33:58.187185+0000","last_clean":"2023-10-06T11:33:58.187185+0000","last_became_active":"2023-10-06T11:33:58.181755+0000","last_became_peered":"2023-10-06T11:33:58.181755+0000","last_unstale":"2023-10-06T11:33:58.187185+0000","last_undegraded":"2023-10-06T11:33:58.187185+0000","last_fullsized":"2023-10-06T11:33:58.187185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:10:27.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.197304+0000","last_change":"2023-10-06T11:33:57.194038+0000","last_active":"2023-10-06T11:33:57.197304+0000","last_peered":"2023-10-06T11:33:57.197304+0000","last_clean":"2023-10-06T11:33:57.197304+0000","last_became_active":"2023-10-06T11:33:57.193745+0000","last_became_peered":"2023-10-06T11:33:57.193745+0000","last_unstale":"2023-10-06T11:33:57.197304+0000","last_undegraded":"2023-10-06T11:33:57.197304+0000","last_fullsized":"2023-10-06T11:33:57.197304+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:09.059564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.193413+0000","last_change":"2023-10-06T11:33:58.187653+0000","last_active":"2023-10-06T11:33:58.193413+0000","last_peered":"2023-10-06T11:33:58.193413+0000","last_clean":"2023-10-06T11:33:58.193413+0000","last_became_active":"2023-10-06T11:33:58.187364+0000","last_became_peered":"2023-10-06T11:33:58.187364+0000","last_unstale":"2023-10-06T11:33:58.193413+0000","last_undegraded":"2023-10-06T11:33:58.193413+0000","last_fullsized":"2023-10-06T11:33:58.193413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:59.821489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.181338+0000","last_change":"2023-10-06T11:33:57.194566+0000","last_active":"2023-10-06T11:33:58.181338+0000","last_peered":"2023-10-06T11:33:58.181338+0000","last_clean":"2023-10-06T11:33:58.181338+0000","last_became_active":"2023-10-06T11:33:57.193886+0000","last_became_peered":"2023-10-06T11:33:57.193886+0000","last_unstale":"2023-10-06T11:33:58.181338+0000","last_undegraded":"2023-10-06T11:33:58.181338+0000","last_fullsized":"2023-10-06T11:33:58.181338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:13.162302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.199336+0000","last_change":"2023-10-06T11:33:58.198321+0000","last_active":"2023-10-06T11:33:58.199336+0000","last_peered":"2023-10-06T11:33:58.199336+0000","last_clean":"2023-10-06T11:33:58.199336+0000","last_became_active":"2023-10-06T11:33:58.197987+0000","last_became_peered":"2023-10-06T11:33:58.197987+0000","last_unstale":"2023-10-06T11:33:58.199336+0000","last_undegraded":"2023-10-06T11:33:58.199336+0000","last_fullsized":"2023-10-06T11:33:58.199336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:50.770290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.167218+0000","last_change":"2023-10-06T11:33:57.190087+0000","last_active":"2023-10-06T11:33:58.167218+0000","last_peered":"2023-10-06T11:33:58.167218+0000","last_clean":"2023-10-06T11:33:58.167218+0000","last_became_active":"2023-10-06T11:33:57.189365+0000","last_became_peered":"2023-10-06T11:33:57.189365+0000","last_unstale":"2023-10-06T11:33:58.167218+0000","last_undegraded":"2023-10-06T11:33:58.167218+0000","last_fullsized":"2023-10-06T11:33:58.167218+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:10.434104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.186604+0000","last_change":"2023-10-06T11:33:58.186604+0000","last_active":"2023-10-06T11:33:58.186604+0000","last_peered":"2023-10-06T11:33:58.186604+0000","last_clean":"2023-10-06T11:33:58.186604+0000","last_became_active":"2023-10-06T11:33:58.186324+0000","last_became_peered":"2023-10-06T11:33:58.186324+0000","last_unstale":"2023-10-06T11:33:58.186604+0000","last_undegraded":"2023-10-06T11:33:58.186604+0000","last_fullsized":"2023-10-06T11:33:58.186604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:06.214786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.183747+0000","last_change":"2023-10-06T11:33:57.196356+0000","last_active":"2023-10-06T11:33:58.183747+0000","last_peered":"2023-10-06T11:33:58.183747+0000","last_clean":"2023-10-06T11:33:58.183747+0000","last_became_active":"2023-10-06T11:33:57.196002+0000","last_became_peered":"2023-10-06T11:33:57.196002+0000","last_unstale":"2023-10-06T11:33:58.183747+0000","last_undegraded":"2023-10-06T11:33:58.183747+0000","last_fullsized":"2023-10-06T11:33:58.183747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:12.032453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.178573+0000","last_change":"2023-10-06T11:33:58.178573+0000","last_active":"2023-10-06T11:33:58.178573+0000","last_peered":"2023-10-06T11:33:58.178573+0000","last_clean":"2023-10-06T11:33:58.178573+0000","last_became_active":"2023-10-06T11:33:58.178218+0000","last_became_peered":"2023-10-06T11:33:58.178218+0000","last_unstale":"2023-10-06T11:33:58.178573+0000","last_undegraded":"2023-10-06T11:33:58.178573+0000","last_fullsized":"2023-10-06T11:33:58.178573+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:34.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.195128+0000","last_change":"2023-10-06T11:33:57.193721+0000","last_active":"2023-10-06T11:33:57.195128+0000","last_peered":"2023-10-06T11:33:57.195128+0000","last_clean":"2023-10-06T11:33:57.195128+0000","last_became_active":"2023-10-06T11:33:57.193245+0000","last_became_peered":"2023-10-06T11:33:57.193245+0000","last_unstale":"2023-10-06T11:33:57.195128+0000","last_undegraded":"2023-10-06T11:33:57.195128+0000","last_fullsized":"2023-10-06T11:33:57.195128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:40.243448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.189675+0000","last_change":"2023-10-06T11:33:58.189675+0000","last_active":"2023-10-06T11:33:58.189675+0000","last_peered":"2023-10-06T11:33:58.189675+0000","last_clean":"2023-10-06T11:33:58.189675+0000","last_became_active":"2023-10-06T11:33:58.189367+0000","last_became_peered":"2023-10-06T11:33:58.189367+0000","last_unstale":"2023-10-06T11:33:58.189675+0000","last_undegraded":"2023-10-06T11:33:58.189675+0000","last_fullsized":"2023-10-06T11:33:58.189675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:46:47.958647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192326+0000","last_change":"2023-10-06T11:33:57.187543+0000","last_active":"2023-10-06T11:33:57.192326+0000","last_peered":"2023-10-06T11:33:57.192326+0000","last_clean":"2023-10-06T11:33:57.192326+0000","last_became_active":"2023-10-06T11:33:57.187254+0000","last_became_peered":"2023-10-06T11:33:57.187254+0000","last_unstale":"2023-10-06T11:33:57.192326+0000","last_undegraded":"2023-10-06T11:33:57.192326+0000","last_fullsized":"2023-10-06T11:33:57.192326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:04.958249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"81'183","reported_seq":460,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.176732+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:33:57.176732+0000","last_peered":"2023-10-06T11:33:57.176732+0000","last_clean":"2023-10-06T11:33:57.176732+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:33:57.176732+0000","last_undegraded":"2023-10-06T11:33:57.176732+0000","last_fullsized":"2023-10-06T11:33:57.176732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175005+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:33:58.175005+0000","last_peered":"2023-10-06T11:33:58.175005+0000","last_clean":"2023-10-06T11:33:58.175005+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:33:58.175005+0000","last_undegraded":"2023-10-06T11:33:58.175005+0000","last_fullsized":"2023-10-06T11:33:58.175005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.188178+0000","last_change":"2023-10-06T11:33:58.182836+0000","last_active":"2023-10-06T11:33:58.188178+0000","last_peered":"2023-10-06T11:33:58.188178+0000","last_clean":"2023-10-06T11:33:58.188178+0000","last_became_active":"2023-10-06T11:33:58.182554+0000","last_became_peered":"2023-10-06T11:33:58.182554+0000","last_unstale":"2023-10-06T11:33:58.188178+0000","last_undegraded":"2023-10-06T11:33:58.188178+0000","last_fullsized":"2023-10-06T11:33:58.188178+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:11:33.186375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.192280+0000","last_change":"2023-10-06T11:33:57.182559+0000","last_active":"2023-10-06T11:33:57.192280+0000","last_peered":"2023-10-06T11:33:57.192280+0000","last_clean":"2023-10-06T11:33:57.192280+0000","last_became_active":"2023-10-06T11:33:57.182246+0000","last_became_peered":"2023-10-06T11:33:57.182246+0000","last_unstale":"2023-10-06T11:33:57.192280+0000","last_undegraded":"2023-10-06T11:33:57.192280+0000","last_fullsized":"2023-10-06T11:33:57.192280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:04.091847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.176618+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:33:57.176618+0000","last_peered":"2023-10-06T11:33:57.176618+0000","last_clean":"2023-10-06T11:33:57.176618+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:33:57.176618+0000","last_undegraded":"2023-10-06T11:33:57.176618+0000","last_fullsized":"2023-10-06T11:33:57.176618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187865+0000","last_change":"2023-10-06T11:33:58.186017+0000","last_active":"2023-10-06T11:33:58.187865+0000","last_peered":"2023-10-06T11:33:58.187865+0000","last_clean":"2023-10-06T11:33:58.187865+0000","last_became_active":"2023-10-06T11:33:58.185532+0000","last_became_peered":"2023-10-06T11:33:58.185532+0000","last_unstale":"2023-10-06T11:33:58.187865+0000","last_undegraded":"2023-10-06T11:33:58.187865+0000","last_fullsized":"2023-10-06T11:33:58.187865+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:30.766304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.191833+0000","last_change":"2023-10-06T11:33:57.182254+0000","last_active":"2023-10-06T11:33:57.191833+0000","last_peered":"2023-10-06T11:33:57.191833+0000","last_clean":"2023-10-06T11:33:57.191833+0000","last_became_active":"2023-10-06T11:33:57.181930+0000","last_became_peered":"2023-10-06T11:33:57.181930+0000","last_unstale":"2023-10-06T11:33:57.191833+0000","last_undegraded":"2023-10-06T11:33:57.191833+0000","last_fullsized":"2023-10-06T11:33:57.191833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:58.804792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.170543+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:33:58.170543+0000","last_peered":"2023-10-06T11:33:58.170543+0000","last_clean":"2023-10-06T11:33:58.170543+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:33:58.170543+0000","last_undegraded":"2023-10-06T11:33:58.170543+0000","last_fullsized":"2023-10-06T11:33:58.170543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.187273+0000","last_change":"2023-10-06T11:33:58.182030+0000","last_active":"2023-10-06T11:33:58.187273+0000","last_peered":"2023-10-06T11:33:58.187273+0000","last_clean":"2023-10-06T11:33:58.187273+0000","last_became_active":"2023-10-06T11:33:58.181703+0000","last_became_peered":"2023-10-06T11:33:58.181703+0000","last_unstale":"2023-10-06T11:33:58.187273+0000","last_undegraded":"2023-10-06T11:33:58.187273+0000","last_fullsized":"2023-10-06T11:33:58.187273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:16:35.527326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'2","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.494876+0000","last_change":"2023-10-06T11:33:57.191201+0000","last_active":"2023-10-06T11:33:58.494876+0000","last_peered":"2023-10-06T11:33:58.494876+0000","last_clean":"2023-10-06T11:33:58.494876+0000","last_became_active":"2023-10-06T11:33:57.190876+0000","last_became_peered":"2023-10-06T11:33:57.190876+0000","last_unstale":"2023-10-06T11:33:58.494876+0000","last_undegraded":"2023-10-06T11:33:58.494876+0000","last_fullsized":"2023-10-06T11:33:58.494876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:51.166657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":80,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.174939+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:33:58.174939+0000","last_peered":"2023-10-06T11:33:58.174939+0000","last_clean":"2023-10-06T11:33:58.174939+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:33:58.174939+0000","last_undegraded":"2023-10-06T11:33:58.174939+0000","last_fullsized":"2023-10-06T11:33:58.174939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190264+0000","last_change":"2023-10-06T11:33:58.183771+0000","last_active":"2023-10-06T11:33:58.190264+0000","last_peered":"2023-10-06T11:33:58.190264+0000","last_clean":"2023-10-06T11:33:58.190264+0000","last_became_active":"2023-10-06T11:33:58.183451+0000","last_became_peered":"2023-10-06T11:33:58.183451+0000","last_unstale":"2023-10-06T11:33:58.190264+0000","last_undegraded":"2023-10-06T11:33:58.190264+0000","last_fullsized":"2023-10-06T11:33:58.190264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.179000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.494270+0000","last_change":"2023-10-06T11:33:57.182916+0000","last_active":"2023-10-06T11:33:58.494270+0000","last_peered":"2023-10-06T11:33:58.494270+0000","last_clean":"2023-10-06T11:33:58.494270+0000","last_became_active":"2023-10-06T11:33:57.182602+0000","last_became_peered":"2023-10-06T11:33:57.182602+0000","last_unstale":"2023-10-06T11:33:58.494270+0000","last_undegraded":"2023-10-06T11:33:58.494270+0000","last_fullsized":"2023-10-06T11:33:58.494270+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:19.921634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.174836+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:33:57.174836+0000","last_peered":"2023-10-06T11:33:57.174836+0000","last_clean":"2023-10-06T11:33:57.174836+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:33:57.174836+0000","last_undegraded":"2023-10-06T11:33:57.174836+0000","last_fullsized":"2023-10-06T11:33:57.174836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.175304+0000","last_change":"2023-10-06T11:33:58.175304+0000","last_active":"2023-10-06T11:33:58.175304+0000","last_peered":"2023-10-06T11:33:58.175304+0000","last_clean":"2023-10-06T11:33:58.175304+0000","last_became_active":"2023-10-06T11:33:58.174963+0000","last_became_peered":"2023-10-06T11:33:58.174963+0000","last_unstale":"2023-10-06T11:33:58.175304+0000","last_undegraded":"2023-10-06T11:33:58.175304+0000","last_fullsized":"2023-10-06T11:33:58.175304+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:37.726524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.186628+0000","last_change":"2023-10-06T11:33:57.182518+0000","last_active":"2023-10-06T11:33:57.186628+0000","last_peered":"2023-10-06T11:33:57.186628+0000","last_clean":"2023-10-06T11:33:57.186628+0000","last_became_active":"2023-10-06T11:33:57.182210+0000","last_became_peered":"2023-10-06T11:33:57.182210+0000","last_unstale":"2023-10-06T11:33:57.186628+0000","last_undegraded":"2023-10-06T11:33:57.186628+0000","last_fullsized":"2023-10-06T11:33:57.186628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:40.922998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.178638+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:33:57.178638+0000","last_peered":"2023-10-06T11:33:57.178638+0000","last_clean":"2023-10-06T11:33:57.178638+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:33:57.178638+0000","last_undegraded":"2023-10-06T11:33:57.178638+0000","last_fullsized":"2023-10-06T11:33:57.178638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169937+0000","last_change":"2023-10-06T11:33:58.169937+0000","last_active":"2023-10-06T11:33:58.169937+0000","last_peered":"2023-10-06T11:33:58.169937+0000","last_clean":"2023-10-06T11:33:58.169937+0000","last_became_active":"2023-10-06T11:33:58.168881+0000","last_became_peered":"2023-10-06T11:33:58.168881+0000","last_unstale":"2023-10-06T11:33:58.169937+0000","last_undegraded":"2023-10-06T11:33:58.169937+0000","last_fullsized":"2023-10-06T11:33:58.169937+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:19:11.538188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.169819+0000","last_change":"2023-10-06T11:33:57.199227+0000","last_active":"2023-10-06T11:33:58.169819+0000","last_peered":"2023-10-06T11:33:58.169819+0000","last_clean":"2023-10-06T11:33:58.169819+0000","last_became_active":"2023-10-06T11:33:57.198885+0000","last_became_peered":"2023-10-06T11:33:57.198885+0000","last_unstale":"2023-10-06T11:33:58.169819+0000","last_undegraded":"2023-10-06T11:33:58.169819+0000","last_fullsized":"2023-10-06T11:33:58.169819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:17.886436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-06T11:33:57.185133+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:33:57.185133+0000","last_peered":"2023-10-06T11:33:57.185133+0000","last_clean":"2023-10-06T11:33:57.185133+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:33:57.185133+0000","last_undegraded":"2023-10-06T11:33:57.185133+0000","last_fullsized":"2023-10-06T11:33:57.185133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.190152+0000","last_change":"2023-10-06T11:33:58.182715+0000","last_active":"2023-10-06T11:33:58.190152+0000","last_peered":"2023-10-06T11:33:58.190152+0000","last_clean":"2023-10-06T11:33:58.190152+0000","last_became_active":"2023-10-06T11:33:58.182414+0000","last_became_peered":"2023-10-06T11:33:58.182414+0000","last_unstale":"2023-10-06T11:33:58.190152+0000","last_undegraded":"2023-10-06T11:33:58.190152+0000","last_fullsized":"2023-10-06T11:33:58.190152+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:57.144353+0000","last_clean_scrub_stamp":"2023-10-06T11:33:57.144353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:34.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.503308+0000","last_change":"2023-10-06T11:33:57.194801+0000","last_active":"2023-10-06T11:33:58.503308+0000","last_peered":"2023-10-06T11:33:58.503308+0000","last_clean":"2023-10-06T11:33:58.503308+0000","last_became_active":"2023-10-06T11:33:57.194228+0000","last_became_peered":"2023-10-06T11:33:57.194228+0000","last_unstale":"2023-10-06T11:33:58.503308+0000","last_undegraded":"2023-10-06T11:33:58.503308+0000","last_fullsized":"2023-10-06T11:33:58.503308+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:33:56.127655+0000","last_clean_scrub_stamp":"2023-10-06T11:33:56.127655+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:02:25.131248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T11:33:58.182304+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:33:58.182304+0000","last_peered":"2023-10-06T11:33:58.182304+0000","last_clean":"2023-10-06T11:33:58.182304+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:33:58.182304+0000","last_undegraded":"2023-10-06T11:33:58.182304+0000","last_fullsized":"2023-10-06T11:33:58.182304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311256,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060584,"statfs":{"total":96636764160,"available":96318038016,"internally_reserved":0,"allocated":1720320,"data_stored":1160899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.827},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.577,"5min":1.649,"15min":2.0030000000000001},"last":1.2270000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3169999999999999,"15min":1.3300000000000001},"min":{"1min":0.90000000000000002,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7649999999999999,"5min":1.871,"15min":2.6059999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.927,"5min":1.927,"15min":1.931},"last":1.1379999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.502,"5min":1.827,"15min":2.3799999999999999},"last":1.095},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.258,"15min":1.2609999999999999},"min":{"1min":1.054,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.7,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.339}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.82099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8,"5min":1.9450000000000001,"15min":2.665},"last":1.405},{"interface":"front","average":{"1min":1.27,"5min":1.296,"15min":1.321},"min":{"1min":0.92900000000000005,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.796,"15min":2.8570000000000002},"last":1.554}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.77300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5569999999999999,"5min":2.149,"15min":2.7970000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.86599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.5339999999999998},"last":1.272}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.335,"15min":1.3160000000000001},"min":{"1min":0.77900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7410000000000001,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.28},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.71,"5min":2.2480000000000002,"15min":3.105},"last":1.4370000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.361},"min":{"1min":1.0369999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0190000000000001,"5min":2.4289999999999998,"15min":3.004},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.86299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.905,"5min":2.121,"15min":2.9119999999999999},"last":1.1890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575036,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302504,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069336,"statfs":{"total":96636764160,"available":96327000064,"internally_reserved":0,"allocated":1146880,"data_stored":579130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.288},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5629999999999999,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.347},{"interface":"front","average":{"1min":1.379,"5min":1.333,"15min":1.325},"min":{"1min":0.83799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6830000000000001,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.4850000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.80500000000000005,"5min":0.76200000000000001,"15min":0.63900000000000001},"max":{"1min":1.5029999999999999,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":1.119},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.605,"5min":1.669,"15min":1.8480000000000001},"last":1.054}]},{"osd":2,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.2},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.74299999999999999},"max":{"1min":1.514,"5min":1.635,"15min":1.6919999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.194,"15min":1.222},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4450000000000001,"5min":1.587,"15min":2.7789999999999999},"last":1.018}]},{"osd":3,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.3100000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.534,"5min":1.764,"15min":1.9850000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3120000000000001,"15min":1.3049999999999999},"min":{"1min":1.077,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.5269999999999999,"5min":1.724,"15min":2.109},"last":1.1379999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.238,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.76500000000000001,"15min":0.61399999999999999},"max":{"1min":1.6639999999999999,"5min":1.8,"15min":1.956},"last":1.272},{"interface":"front","average":{"1min":1.226,"5min":1.292,"15min":1.2490000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.73899999999999999},"max":{"1min":1.645,"5min":1.827,"15min":1.8879999999999999},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6000000000000001,"5min":1.72,"15min":2.198},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.395,"15min":1.385},"min":{"1min":0.86499999999999999,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.718,"5min":1.879,"15min":2.1509999999999998},"last":1.5189999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4510000000000001,"15min":1.448},"min":{"1min":0.90100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.871,"5min":1.927,"15min":2.2919999999999998},"last":1.746},{"interface":"front","average":{"1min":1.381,"5min":1.4139999999999999,"15min":1.4299999999999999},"min":{"1min":1.004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6879999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.651}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302496,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069344,"statfs":{"total":96636764160,"available":96327008256,"internally_reserved":0,"allocated":1138688,"data_stored":577628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.3899999999999999,"15min":1.268},"min":{"1min":0.67400000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.772,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64900000000000002},"max":{"1min":1.679,"5min":1.972,"15min":2.544},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.194,"15min":1.175},"min":{"1min":0.79800000000000004,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.478,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.256},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.2190000000000001},"min":{"1min":0.79900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5189999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.159,"15min":1.1950000000000001},"min":{"1min":0.78600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.601,"5min":1.6879999999999999,"15min":3.609},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.73299999999999998},"max":{"1min":1.512,"5min":1.738,"15min":2.6949999999999998},"last":1.129}]},{"osd":3,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.393,"15min":1.296},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.639,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.448},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.302,"15min":1.3340000000000001},"min":{"1min":0.91800000000000004,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.5229999999999999,"5min":1.734,"15min":3.7770000000000001},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.40699999999999997},"max":{"1min":1.6060000000000001,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.393},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.516,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.724,"5min":18.48,"15min":18.48},"last":1.5249999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.3089999999999999},"min":{"1min":1.0129999999999999,"5min":0.85199999999999998,"15min":0.622},"max":{"1min":1.6799999999999999,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.411,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.6870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5249999999999999,"15min":1.4039999999999999},"min":{"1min":0.85499999999999998,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7909999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.494,"15min":1.444},"min":{"1min":0.998,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7709999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.4179999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302512,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069328,"statfs":{"total":96636764160,"available":96326991872,"internally_reserved":0,"allocated":1155072,"data_stored":579234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1479999999999999,"15min":1.143},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.554,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.111,"15min":1.143},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.478,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":1.3779999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2989999999999999,"15min":1.323},"min":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.546,"5min":1.645,"15min":2.1989999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.68300000000000005},"max":{"1min":1.508,"5min":1.7450000000000001,"15min":1.851},"last":1.4550000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.73499999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":2.2269999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.78500000000000003},"max":{"1min":1.583,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.476}]},{"osd":3,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.671,"15min":1.6839999999999999},"last":1.256},{"interface":"front","average":{"1min":1.288,"5min":1.3009999999999999,"15min":1.304},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.77400000000000002},"max":{"1min":1.575,"5min":1.6639999999999999,"15min":2.258},"last":1.498}]},{"osd":5,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.202,"15min":1.21},"min":{"1min":0.79600000000000004,"5min":0.621,"15min":0.621},"max":{"1min":1.653,"5min":1.653,"15min":2.8250000000000002},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2130000000000001,"15min":1.1970000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.40999999999999998},"max":{"1min":1.421,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3480000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.6950000000000001,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.099},{"interface":"front","average":{"1min":1.385,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":1.0860000000000001,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.845,"5min":1.845,"15min":2.4329999999999998},"last":1.319}]},{"osd":7,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.69399999999999995},"max":{"1min":1.5880000000000001,"5min":1.841,"15min":2.653},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.3919999999999999},"min":{"1min":0.96699999999999997,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.464},"last":1.2949999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306580,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065260,"statfs":{"total":96636764160,"available":96322826240,"internally_reserved":0,"allocated":1126400,"data_stored":571063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.61099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0960000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.437},"max":{"1min":1.427,"5min":1.504,"15min":1.627},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.95099999999999996},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.343,"5min":1.4079999999999999,"15min":1.516},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0660000000000001,"15min":1.077},"min":{"1min":0.753,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.093,"15min":1.121},"min":{"1min":0.95399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.68700000000000006,"5min":0.44,"15min":0.44},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.7030000000000001},"last":1.341}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.1919999999999999,"15min":1.236},"min":{"1min":0.93300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6890000000000001,"5min":1.768,"15min":2.6640000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3029999999999999,"15min":1.3560000000000001},"min":{"1min":1.1379999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0840000000000001},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3420000000000001,"15min":1.4019999999999999},"min":{"1min":0.92100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6080000000000001,"5min":1.883,"15min":2.1699999999999999},"last":1.274},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.357,"15min":1.3979999999999999},"min":{"1min":1.0620000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.1179999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.411,"15min":1.45},"min":{"1min":0.98599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8520000000000001,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":1.49},{"interface":"front","average":{"1min":1.387,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.97499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.75,"5min":1.823,"15min":2.137},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4119999999999999,"15min":1.4419999999999999},"min":{"1min":1.163,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8540000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":1.04},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4319999999999999,"15min":1.482},"min":{"1min":1.0860000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.645,"5min":2.004,"15min":2.266},"last":1.381}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310684,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061156,"statfs":{"total":96636764160,"available":96318623744,"internally_reserved":0,"allocated":1134592,"data_stored":571223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1200000000000001,"5min":1.4670000000000001,"15min":1.8380000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.383,"5min":1.415,"15min":1.9650000000000001},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.3049999999999999,"5min":1.631,"15min":2.3940000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0229999999999999},"min":{"1min":0.68700000000000006,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.599,"5min":1.6779999999999999,"15min":2.5390000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"min":{"1min":0.90600000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.758,"5min":1.8,"15min":1.917},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.28,"15min":1.306},"min":{"1min":0.96499999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.948,"5min":1.948,"15min":2.4550000000000001},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.89700000000000002,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":2.1899999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":1.0589999999999999,"5min":0.88500000000000001,"15min":0.84399999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.6920000000000002},"last":1.087}]},{"osd":5,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":1.171,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8480000000000001,"5min":2.0680000000000001,"15min":2.601},"last":1.294},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.319,"15min":1.337},"min":{"1min":1.0089999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.6400000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4039999999999999,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.972,"5min":1.972,"15min":2.8340000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.427,"15min":1.444},"min":{"1min":1.1719999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8320000000000001,"5min":1.9910000000000001,"15min":2.617},"last":1.264}]},{"osd":7,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.371,"15min":1.391},"min":{"1min":0.94299999999999995,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.7549999999999999,"5min":1.8049999999999999,"15min":2.9020000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.464,"15min":1.4690000000000001},"min":{"1min":1.2609999999999999,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.978,"5min":1.978,"15min":3.2240000000000002},"last":1.4470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306584,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065256,"statfs":{"total":96636764160,"available":96322822144,"internally_reserved":0,"allocated":1130496,"data_stored":570621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.93000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.252,"5min":1.724,"15min":1.724},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.58399999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.4259999999999999,"15min":1.855},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.97199999999999998},"min":{"1min":0.67100000000000004,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.1859999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.319,"5min":2.137,"15min":2.137},"last":1.159}]},{"osd":3,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.272,"15min":1.228},"min":{"1min":0.879,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6120000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.071},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.073,"15min":1.0980000000000001},"min":{"1min":0.72199999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":4.0830000000000002},"last":0.76800000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.238,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7350000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.01},{"interface":"front","average":{"1min":1.419,"5min":1.387,"15min":1.397},"min":{"1min":1.056,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.788,"5min":1.788,"15min":3.5609999999999999},"last":1.266}]},{"osd":5,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.5720000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.119},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.365,"15min":1.3500000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.85999999999999999},"max":{"1min":1.849,"5min":2.1240000000000001,"15min":2.766},"last":1.206}]},{"osd":6,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4890000000000001,"15min":1.4830000000000001},"min":{"1min":1.1299999999999999,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8759999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.411,"15min":1.407},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.0449999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.88,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8580000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.4650000000000001,"15min":1.466},"min":{"1min":1.133,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.085,"5min":2.085,"15min":2.8740000000000001},"last":1.381}]}]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307168,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064672,"statfs":{"total":96636764160,"available":96322224128,"internally_reserved":0,"allocated":1728512,"data_stored":1167962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85899999999999999,"15min":0.88300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2290000000000001,"5min":1.3560000000000001,"15min":1.571},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95499999999999996,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2549999999999999,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.738,"5min":1.738,"15min":2.7719999999999998},"last":1.458}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0860000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.919},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.762,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":1.0069999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3280000000000001,"15min":1.343},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.294,"15min":2.294},"last":1.53},{"interface":"front","average":{"1min":1.169,"5min":1.276,"15min":1.2929999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.609,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.427}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.393,"15min":1.383},"min":{"1min":1.0800000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8520000000000001,"5min":2.0859999999999999,"15min":2.573},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":1.024,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.843,"5min":1.881,"15min":2.8650000000000002},"last":1.587}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.3420000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6160000000000001,"5min":2.2240000000000002,"15min":2.52},"last":1.212},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4410000000000001,"15min":1.4610000000000001},"min":{"1min":1.163,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.909,"5min":2.254,"15min":2.742},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4399999999999999,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7509999999999999,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.645},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4119999999999999,"15min":1.4319999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7090000000000001,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:34:03.193 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:34:03.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:34:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.399+0000 7fca09ee3700 1 Processor -- start 2023-10-06T11:34:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.399+0000 7fca09ee3700 1 -- start start 2023-10-06T11:34:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca09ee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca040cb020 0x7fca040cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca09ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca040cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca09ee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040cf050 con 0x7fca040cdc10 2023-10-06T11:34:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca09ee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040ce440 con 0x7fca040cb020 2023-10-06T11:34:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca09ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040ce720 con 0x7fca040d0750 2023-10-06T11:34:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca040cb020 0x7fca040cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca040cb020 0x7fca040cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58244/0 (socket says 172.21.15.46:58244) 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 -- 172.21.15.46:0/4006016757 learned_addr learned my addr 172.21.15.46:0/4006016757 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca03fff700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca040cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 -- 172.21.15.46:0/4006016757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 msgr2=0x7fca040cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca040cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 -- 172.21.15.46:0/4006016757 >> v1:172.21.15.132:6789/0 conn(0x7fca040cdc10 legacy=0x7fca04158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.403+0000 7fca08ee1700 1 -- 172.21.15.46:0/4006016757 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040cea00 con 0x7fca040cb020 2023-10-06T11:34:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca08ee1700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cb020 0x7fca040cb3f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80099b0 tx=0x7fc9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7aef1803e5c72c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca037fe700 1 -- 172.21.15.46:0/4006016757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8009ea0 con 0x7fca040cb020 2023-10-06T11:34:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca037fe700 1 -- 172.21.15.46:0/4006016757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f800b860 con 0x7fca040cb020 2023-10-06T11:34:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca037fe700 1 -- 172.21.15.46:0/4006016757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800bc00 con 0x7fca040cb020 2023-10-06T11:34:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 -- 172.21.15.46:0/4006016757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cb020 msgr2=0x7fca040cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cb020 0x7fca040cb3f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80099b0 tx=0x7fc9f8009250 comp rx=0 tx=0).stop 2023-10-06T11:34:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 -- 172.21.15.46:0/4006016757 shutdown_connections 2023-10-06T11:34:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca040cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 --2- 172.21.15.46:0/4006016757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cb020 0x7fca040cb3f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 -- 172.21.15.46:0/4006016757 >> 172.21.15.46:0/4006016757 conn(0x7fca040683d0 msgr2=0x7fca040d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 -- 172.21.15.46:0/4006016757 shutdown_connections 2023-10-06T11:34:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.407+0000 7fca09ee3700 1 -- 172.21.15.46:0/4006016757 wait complete. 2023-10-06T11:34:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 Processor -- start 2023-10-06T11:34:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 -- start start 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 0x7fca041552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca041589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04154940 con 0x7fca040cdc10 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0416d700 con 0x7fca040cb020 2023-10-06T11:34:03.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca09ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0416d9e0 con 0x7fca040d0750 2023-10-06T11:34:03.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca03fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca03fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58252/0 (socket says 172.21.15.46:58252) 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca03fff700 1 -- 172.21.15.46:0/1985083539 learned_addr learned my addr 172.21.15.46:0/1985083539 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca08ee1700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 0x7fca041552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.411+0000 7fca096e2700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca041589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 -- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 msgr2=0x7fca041589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca041589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 -- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 msgr2=0x7fca041552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 0x7fca041552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 -- 172.21.15.46:0/1985083539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80094f0 con 0x7fca040cdc10 2023-10-06T11:34:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca08ee1700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 0x7fca041552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca03fff700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e930 tx=0x7fc9f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0013680 con 0x7fca040cdc10 2023-10-06T11:34:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f00137e0 con 0x7fca040cdc10 2023-10-06T11:34:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0013b80 con 0x7fca040cdc10 2023-10-06T11:34:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0416dcc0 con 0x7fca040cdc10 2023-10-06T11:34:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca04155810 con 0x7fca040cdc10 2023-10-06T11:34:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.415+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0416af40 con 0x7fca040cdc10 2023-10-06T11:34:03.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.419+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc9f000c3e0 con 0x7fca040cdc10 2023-10-06T11:34:03.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.419+0000 7fca00ff9700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 0x7fc9e8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.419+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fc9f007cd90 con 0x7fca040cdc10 2023-10-06T11:34:03.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.419+0000 7fca08ee1700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 0x7fc9e8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.419+0000 7fca08ee1700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 0x7fc9e8062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80099b0 tx=0x7fc9f8017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.423+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fca0416af40 con 0x7fca040cdc10 2023-10-06T11:34:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.655+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fca0416af40 con 0x7fca040cdc10 2023-10-06T11:34:03.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.667+0000 7fca00ff9700 1 -- 172.21.15.46:0/1985083539 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fca0416af40 con 0x7fca040cdc10 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 msgr2=0x7fc9e8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 0x7fc9e8062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80099b0 tx=0x7fc9f8017000 comp rx=0 tx=0).stop 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 msgr2=0x7fca0416a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e930 tx=0x7fc9f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 shutdown_connections 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc9e8060670 0x7fc9e8062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca040d0750 0x7fca041589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca040cdc10 0x7fca0416a6b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 --2- 172.21.15.46:0/1985083539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca040cb020 0x7fca041552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 >> 172.21.15.46:0/1985083539 conn(0x7fca040683d0 msgr2=0x7fca040b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 shutdown_connections 2023-10-06T11:34:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.671+0000 7fca09ee3700 1 -- 172.21.15.46:0/1985083539 wait complete. 2023-10-06T11:34:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:34:03.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:03.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.855+0000 7f38fcc8e700 1 Processor -- start 2023-10-06T11:34:03.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.855+0000 7f38fcc8e700 1 -- start start 2023-10-06T11:34:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38fcc8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38f8067760 0x7f38f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38fcc8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f815d6c0 con 0x7f38f81591a0 2023-10-06T11:34:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38fcc8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f815cd90 con 0x7f38f8158dd0 2023-10-06T11:34:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38fcc8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80699c0 con 0x7f38f8067760 2023-10-06T11:34:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38f81591a0 0x7f38f8069130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51518/0 (socket says 172.21.15.46:51518) 2023-10-06T11:34:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f7fff700 1 -- 172.21.15.46:0/3723079884 learned_addr learned my addr 172.21.15.46:0/3723079884 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:03.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38eeffd700 1 --2- 172.21.15.46:0/3723079884 >> v2:172.21.15.132:3300/0 conn(0x7f38f8067760 0x7f38f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f6ffd700 1 -- 172.21.15.46:0/3723079884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598658975 0 0) 0x7f38f815d6c0 con 0x7f38f81591a0 2023-10-06T11:34:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f6ffd700 1 -- 172.21.15.46:0/3723079884 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003570 con 0x7f38f81591a0 2023-10-06T11:34:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38eeffd700 1 -- 172.21.15.46:0/3723079884 >> v1:172.21.15.132:6789/0 conn(0x7f38f8158dd0 legacy=0x7f38f8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38eeffd700 1 -- 172.21.15.46:0/3723079884 >> v1:172.21.15.46:6789/0 conn(0x7f38f81591a0 legacy=0x7f38f8069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38eeffd700 1 -- 172.21.15.46:0/3723079884 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f815d070 con 0x7f38f8067760 2023-10-06T11:34:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38eeffd700 1 --2- 172.21.15.46:0/3723079884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8067760 0x7f38f8068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009e60 tx=0x7f38e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede359ef1ac1123c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f6ffd700 1 -- 172.21.15.46:0/3723079884 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000e070 con 0x7f38f8067760 2023-10-06T11:34:03.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f6ffd700 1 -- 172.21.15.46:0/3723079884 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e000bb60 con 0x7f38f8067760 2023-10-06T11:34:03.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.859+0000 7f38f6ffd700 1 -- 172.21.15.46:0/3723079884 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0012430 con 0x7f38f8067760 2023-10-06T11:34:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 -- 172.21.15.46:0/3723079884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8067760 msgr2=0x7f38f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/3723079884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8067760 0x7f38f8068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009e60 tx=0x7f38e0009780 comp rx=0 tx=0).stop 2023-10-06T11:34:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 -- 172.21.15.46:0/3723079884 shutdown_connections 2023-10-06T11:34:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/3723079884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8067760 0x7f38f8068bf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 -- 172.21.15.46:0/3723079884 >> 172.21.15.46:0/3723079884 conn(0x7f38f8062ef0 msgr2=0x7f38f80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 -- 172.21.15.46:0/3723079884 shutdown_connections 2023-10-06T11:34:03.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.863+0000 7f38fcc8e700 1 -- 172.21.15.46:0/3723079884 wait complete. 2023-10-06T11:34:03.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 Processor -- start 2023-10-06T11:34:03.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 -- start start 2023-10-06T11:34:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38f8067760 0x7f38f80adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8158dd0 0x7f38f814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f814d5f0 con 0x7f38f81591a0 2023-10-06T11:34:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80ad7a0 con 0x7f38f8158dd0 2023-10-06T11:34:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38fcc8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80ace70 con 0x7f38f8067760 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58258/0 (socket says 172.21.15.46:58258) 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 -- 172.21.15.46:0/1614078166 learned_addr learned my addr 172.21.15.46:0/1614078166 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f77fe700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38f8067760 0x7f38f80adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38eeffd700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8158dd0 0x7f38f814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 -- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38f8067760 msgr2=0x7f38f80adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38f8067760 0x7f38f80adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 -- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8158dd0 msgr2=0x7f38f814d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8158dd0 0x7f38f814d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.867+0000 7f38f7fff700 1 -- 172.21.15.46:0/1614078166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e0009a20 con 0x7f38f81591a0 2023-10-06T11:34:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38f7fff700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38f00010e0 tx=0x7f38f0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0005c00 con 0x7f38f81591a0 2023-10-06T11:34:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f0005d60 con 0x7f38f81591a0 2023-10-06T11:34:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f80ad150 con 0x7f38f81591a0 2023-10-06T11:34:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00072e0 con 0x7f38f81591a0 2023-10-06T11:34:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80acb20 con 0x7f38f81591a0 2023-10-06T11:34:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.871+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f806d2b0 con 0x7f38f81591a0 2023-10-06T11:34:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.875+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f38f0012400 con 0x7f38f81591a0 2023-10-06T11:34:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.875+0000 7f38effff700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 0x7f38d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.875+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f38f007bff0 con 0x7f38f81591a0 2023-10-06T11:34:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.875+0000 7f38f77fe700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 0x7f38d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:03.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.875+0000 7f38f77fe700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 0x7f38d8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38f80ca5a0 tx=0x7f38e8009040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:03.883+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f38f0049be0 con 0x7f38f81591a0 2023-10-06T11:34:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.083+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38f80ad930 con 0x7f38f81591a0 2023-10-06T11:34:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.083+0000 7f38effff700 1 -- 172.21.15.46:0/1614078166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f38f806d2b0 con 0x7f38f81591a0 2023-10-06T11:34:04.095 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:04.095 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 msgr2=0x7f38d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 0x7f38d8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38f80ca5a0 tx=0x7f38e8009040 comp rx=0 tx=0).stop 2023-10-06T11:34:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 msgr2=0x7f38f80c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38f00010e0 tx=0x7f38f0007dd0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 shutdown_connections 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38d8060670 0x7f38d8062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f81591a0 0x7f38f80c9770 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38f8158dd0 0x7f38f814d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 --2- 172.21.15.46:0/1614078166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38f8067760 0x7f38f80adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 >> 172.21.15.46:0/1614078166 conn(0x7f38f8062ef0 msgr2=0x7f38f806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 shutdown_connections 2023-10-06T11:34:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.087+0000 7f38fcc8e700 1 -- 172.21.15.46:0/1614078166 wait complete. 2023-10-06T11:34:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:04.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:33:57.603583+0000', 'modified': '2023-10-06T11:34:01.495264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6831}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6831': {'gid': 6831, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/3896471769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3896471769}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3896471769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:34:04.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:34:04.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T11:34:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.683+0000 7f1708c2e700 1 Processor -- start 2023-10-06T11:34:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.683+0000 7f1708c2e700 1 -- start start 2023-10-06T11:34:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f170415ab60 0x7f170406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 0x7f170406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 0x7f17040adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170406d460 con 0x7f170415ab60 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17040c7000 con 0x7f17040ad6c0 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1708c2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17040c72c0 con 0x7f170415af30 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f170415ab60 0x7f170406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f170415ab60 0x7f170406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58270/0 (socket says 172.21.15.46:58270) 2023-10-06T11:34:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 -- 172.21.15.46:0/2316781904 learned_addr learned my addr 172.21.15.46:0/2316781904 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1701d9b700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 0x7f170406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1702d9d700 1 --2- 172.21.15.46:0/2316781904 >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 0x7f17040adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 -- 172.21.15.46:0/2316781904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 msgr2=0x7f170406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 0x7f170406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 -- 172.21.15.46:0/2316781904 >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 msgr2=0x7f17040adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 --2- 172.21.15.46:0/2316781904 >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 0x7f17040adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f170259c700 1 -- 172.21.15.46:0/2316781904 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040c75a0 con 0x7f170415ab60 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1701d9b700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 0x7f170406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.687+0000 7f1702d9d700 1 --2- 172.21.15.46:0/2316781904 >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 0x7f17040adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f170259c700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415ab60 0x7f170406e8a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f17040ae950 tx=0x7f16f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530b49bb47ff9e7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f170159a700 1 -- 172.21.15.46:0/2316781904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400b860 con 0x7f170415ab60 2023-10-06T11:34:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f170159a700 1 -- 172.21.15.46:0/2316781904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f400b9c0 con 0x7f170415ab60 2023-10-06T11:34:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f170159a700 1 -- 172.21.15.46:0/2316781904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400bd60 con 0x7f170415ab60 2023-10-06T11:34:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 -- 172.21.15.46:0/2316781904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415ab60 msgr2=0x7f170406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415ab60 0x7f170406e8a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f17040ae950 tx=0x7f16f4009250 comp rx=0 tx=0).stop 2023-10-06T11:34:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 -- 172.21.15.46:0/2316781904 shutdown_connections 2023-10-06T11:34:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 --2- 172.21.15.46:0/2316781904 >> v2:172.21.15.132:3300/0 conn(0x7f17040ad6c0 0x7f17040adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415af30 0x7f170406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 --2- 172.21.15.46:0/2316781904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415ab60 0x7f170406e8a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 -- 172.21.15.46:0/2316781904 >> 172.21.15.46:0/2316781904 conn(0x7f17040cbe30 msgr2=0x7f17040cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 -- 172.21.15.46:0/2316781904 shutdown_connections 2023-10-06T11:34:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.691+0000 7f1708c2e700 1 -- 172.21.15.46:0/2316781904 wait complete. 2023-10-06T11:34:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 Processor -- start 2023-10-06T11:34:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 -- start start 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17040ad6c0 0x7f17040a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 0x7f1704169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170416a130 con 0x7f170415af30 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17040aaa30 con 0x7f17040ad6c0 2023-10-06T11:34:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1708c2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17040a9e20 con 0x7f170415ab60 2023-10-06T11:34:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1702d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1702d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58286/0 (socket says 172.21.15.46:58286) 2023-10-06T11:34:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1702d9d700 1 -- 172.21.15.46:0/3256297962 learned_addr learned my addr 172.21.15.46:0/3256297962 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f170259c700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17040ad6c0 0x7f17040a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.695+0000 7f1701d9b700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 0x7f1704169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 -- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 msgr2=0x7f1704169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 0x7f1704169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 -- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17040ad6c0 msgr2=0x7f17040a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17040ad6c0 0x7f17040a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 -- 172.21.15.46:0/3256297962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4009490 con 0x7f170415af30 2023-10-06T11:34:04.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1701d9b700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 0x7f1704169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1702d9d700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fc0 tx=0x7f16fc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f16f27fc700 1 -- 172.21.15.46:0/3256297962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00dc90 con 0x7f170415af30 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f16f27fc700 1 -- 172.21.15.46:0/3256297962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00f040 con 0x7f170415af30 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17040aa100 con 0x7f170415af30 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f16f27fc700 1 -- 172.21.15.46:0/3256297962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc015690 con 0x7f170415af30 2023-10-06T11:34:04.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.699+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17040aa6c0 con 0x7f170415af30 2023-10-06T11:34:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f16f27fc700 1 -- 172.21.15.46:0/3256297962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f16fc01e3b0 con 0x7f170415af30 2023-10-06T11:34:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f16f27fc700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 0x7f16e4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f16f27fc700 1 -- 172.21.15.46:0/3256297962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f16fc07d2a0 con 0x7f170415af30 2023-10-06T11:34:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f170259c700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 0x7f16e4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f170259c700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 0x7f16e4062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17040ae950 tx=0x7f16f4017000 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.703+0000 7f170aece740 1 Processor -- start 2023-10-06T11:34:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 -- start start 2023-10-06T11:34:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x181aae0 0x1a97630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1865790 0x1a980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c5f30 con 0x181a710 2023-10-06T11:34:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c1ed0 con 0x1865790 2023-10-06T11:34:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x19880b0 con 0x181aae0 2023-10-06T11:34:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58298/0 (socket says 172.21.15.46:58298) 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 -- 172.21.15.46:0/295876949 learned_addr learned my addr 172.21.15.46:0/295876949 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f170259c700 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x181aae0 0x1a97630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1702d9d700 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1865790 0x1a980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 -- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x181aae0 msgr2=0x1a97630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x181aae0 0x1a97630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 -- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1865790 msgr2=0x1a980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1865790 0x1a980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 -- 172.21.15.46:0/295876949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18669e0 con 0x181a710 2023-10-06T11:34:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f1701d9b700 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009bb0 tx=0x7f16f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc3a9ffce909a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f16e37fe700 1 -- 172.21.15.46:0/295876949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f80109c0 con 0x181a710 2023-10-06T11:34:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f16e37fe700 1 -- 172.21.15.46:0/295876949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f800c050 con 0x181a710 2023-10-06T11:34:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.707+0000 7f16e37fe700 1 -- 172.21.15.46:0/295876949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8010d30 con 0x181a710 2023-10-06T11:34:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- 172.21.15.46:0/295876949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 msgr2=0x1a97b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009bb0 tx=0x7f16f8008040 comp rx=0 tx=0).stop 2023-10-06T11:34:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- 172.21.15.46:0/295876949 shutdown_connections 2023-10-06T11:34:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1865790 0x1a980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a97b70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 --2- 172.21.15.46:0/295876949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x181aae0 0x1a97630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- 172.21.15.46:0/295876949 >> 172.21.15.46:0/295876949 conn(0x181aeb0 msgr2=0x1a54060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- 172.21.15.46:0/295876949 shutdown_connections 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- 172.21.15.46:0/295876949 wait complete. 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 -- start start 2023-10-06T11:34:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.711+0000 7f170aece740 1 Processor -- start 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181a710 0x1a9bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a51c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1987d50 con 0x181aae0 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1960970 con 0x181a710 2023-10-06T11:34:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x17f1060 con 0x1865790 2023-10-06T11:34:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 10 client.0 _ll_register_callbacks cb 0x1964b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:34:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181a710 0x1a9bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58312/0 (socket says 172.21.15.46:58312) 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 -- 172.21.15.46:0/3867089567 learned_addr learned my addr 172.21.15.46:0/3867089567 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1702d9d700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a51c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 -- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 msgr2=0x1a51c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a51c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 -- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181a710 msgr2=0x1a9bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181a710 0x1a9bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 -- 172.21.15.46:0/3867089567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f8009760 con 0x181aae0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f1701d9b700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16f80096c0 tx=0x7f16f801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c1deb8fd04c02d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f16e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f8010e60 con 0x181aae0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f16f8015680 con 0x181aae0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.715+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1960cc0 con 0x181aae0 2023-10-06T11:34:04.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f8015a50 con 0x181aae0 2023-10-06T11:34:04.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 10 client.6978 fetch_fsmap learned FSMap version 42 2023-10-06T11:34:04.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x1990370 con 0x181aae0 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1960cc0 con 0x181aae0 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f16f8015bb0 con 0x181aae0 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 10 client.6978 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 10 client.6978 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 20 client.6978 populate_metadata read hostname 'smithi046' 2023-10-06T11:34:04.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1a53180 0x1aa6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 4 client.6978 mds_command: new command op to 6831 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 --> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x19643e0 con 0x1a53180 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170259c700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1a53180 0x1aa6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170259c700 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1a53180 0x1aa6de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4635b696c7f7199e server_cookie=a3748a2be266e7ed in_seq=0 out_seq=0 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 10 client.6978 ms_handle_connect on v2:172.21.15.46:6834/3896471769 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 1 -- 172.21.15.46:0/3867089567 <== mds.0 v2:172.21.15.46:6834/3896471769 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x19643e0 con 0x1a53180 2023-10-06T11:34:04.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f16e0ff9700 10 client.6978 handle_command_reply: tid=0 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.719+0000 7f170aece740 1 client.6978 shutdown 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 msgr2=0x1a9c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16f80096c0 tx=0x7f16f801d000 comp rx=0 tx=0).stop 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 shutdown_connections 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1a53180 0x1aa6de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a51c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181aae0 0x1a9c180 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 --2- 172.21.15.46:0/3867089567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181a710 0x1a9bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 >> 172.21.15.46:0/3867089567 conn(0x181aeb0 msgr2=0x1a7b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f16e0ff9700 0 client.6978 ms_handle_reset on v2:172.21.15.46:6834/3896471769 2023-10-06T11:34:04.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 shutdown_connections 2023-10-06T11:34:04.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 1 -- 172.21.15.46:0/3867089567 wait complete. 2023-10-06T11:34:04.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.723+0000 7f170aece740 20 client.6978 trim_cache size 0 max 16384 2023-10-06T11:34:04.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 Processor -- start 2023-10-06T11:34:04.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 -- start start 2023-10-06T11:34:04.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1a4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a76e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c4590 con 0x181a710 2023-10-06T11:34:04.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x19e9110 con 0x181aae0 2023-10-06T11:34:04.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.743+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1970c90 con 0x1865790 2023-10-06T11:34:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58318/0 (socket says 172.21.15.46:58318) 2023-10-06T11:34:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 -- 172.21.15.46:0/2020458443 learned_addr learned my addr 172.21.15.46:0/2020458443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f1702d9d700 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a76e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f1701d9b700 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1a4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 -- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 msgr2=0x1a76e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a76e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 -- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 msgr2=0x1a4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1a4cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 -- 172.21.15.46:0/2020458443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040ab200 con 0x181a710 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170259c700 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f170406bf10 tx=0x7f16f4029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcac00b3ce487c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f16e3fff700 1 -- 172.21.15.46:0/2020458443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f402c040 con 0x181a710 2023-10-06T11:34:04.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f16e3fff700 1 -- 172.21.15.46:0/2020458443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f402d040 con 0x181a710 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f16e3fff700 1 -- 172.21.15.46:0/2020458443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4020070 con 0x181a710 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 -- 172.21.15.46:0/2020458443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 msgr2=0x1a4c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f170406bf10 tx=0x7f16f4029040 comp rx=0 tx=0).stop 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 -- 172.21.15.46:0/2020458443 shutdown_connections 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1a76e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1a4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 --2- 172.21.15.46:0/2020458443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1a4c6b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.747+0000 7f170aece740 1 -- 172.21.15.46:0/2020458443 >> 172.21.15.46:0/2020458443 conn(0x181aeb0 msgr2=0x1a9a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.751+0000 7f170aece740 1 -- 172.21.15.46:0/2020458443 shutdown_connections 2023-10-06T11:34:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.751+0000 7f170aece740 1 -- 172.21.15.46:0/2020458443 wait complete. 2023-10-06T11:34:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.751+0000 7f170aece740 1 -- start start 2023-10-06T11:34:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.751+0000 7f170aece740 1 Processor -- start 2023-10-06T11:34:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1af1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1aa5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c55d0 con 0x181a710 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1997550 con 0x181aae0 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1995f40 con 0x1865790 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 10 client.0 _ll_register_callbacks cb 0x198d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58332/0 (socket says 172.21.15.46:58332) 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 -- 172.21.15.46:0/1106926729 learned_addr learned my addr 172.21.15.46:0/1106926729 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f1702d9d700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1aa5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f1701d9b700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1af1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 -- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 msgr2=0x1aa5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1aa5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 -- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 msgr2=0x1af1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1af1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 -- 172.21.15.46:0/1106926729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17040c75a0 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f1701d9b700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1af1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170259c700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f16f40090b0 tx=0x7f16f4034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b630bbe99a68d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f402c040 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16f800c050 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f16f40377a0 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f4020040 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 10 client.6984 fetch_fsmap learned FSMap version 42 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x1976d80 con 0x181a710 2023-10-06T11:34:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f800c050 con 0x181a710 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f16f4037b70 con 0x181a710 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 10 client.6984 fetch_fsmap finished waiting for FSMap version 42 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 10 client.6984 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 20 client.6984 populate_metadata read hostname 'smithi046' 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1aa6760 0x1aa6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 4 client.6984 mds_command: new command op to 6831 tid=0[{"prefix": "status"}] 2023-10-06T11:34:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.755+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 --> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1996410 con 0x1aa6760 2023-10-06T11:34:04.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f1701d9b700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1aa6760 0x1aa6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f1701d9b700 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1aa6760 0x1aa6b30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c076b2535f935ae6 server_cookie=6e17813bd9eec884 in_seq=0 out_seq=0 2023-10-06T11:34:04.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f16e0ff9700 10 client.6984 ms_handle_connect on v2:172.21.15.46:6834/3896471769 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f16e0ff9700 1 -- 172.21.15.46:0/1106926729 <== mds.0 v2:172.21.15.46:6834/3896471769 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1996410 con 0x1aa6760 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f16e0ff9700 10 client.6984 handle_command_reply: tid=0 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 client.6984 shutdown 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 msgr2=0x1af19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f16f40090b0 tx=0x7f16f4034000 comp rx=0 tx=0).stop 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 shutdown_connections 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:6834/3896471769,v1:172.21.15.46:6835/3896471769] conn(0x1aa6760 0x1aa6b30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1865790 0x1aa5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x181aae0 0x1af1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 --2- 172.21.15.46:0/1106926729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x181a710 0x1af19d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 >> 172.21.15.46:0/1106926729 conn(0x181aeb0 msgr2=0x1a53550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f16e0ff9700 0 client.6984 ms_handle_reset on v2:172.21.15.46:6834/3896471769 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 shutdown_connections 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 1 -- 172.21.15.46:0/1106926729 wait complete. 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.759+0000 7f170aece740 20 client.6984 trim_cache size 0 max 16384 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6831, 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:34:04.774 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.2827445879999999, 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 42, 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 93, 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.535112591000001 2023-10-06T11:34:04.775 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 msgr2=0x7f16e4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 0x7f16e4062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17040ae950 tx=0x7f16f4017000 comp rx=0 tx=0).stop 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 msgr2=0x7f17040cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fc0 tx=0x7f16fc0099b0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 shutdown_connections 2023-10-06T11:34:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f16e4060670 0x7f16e4062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f170415af30 0x7f17040cb3a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f170415ab60 0x7f1704169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 --2- 172.21.15.46:0/3256297962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17040ad6c0 0x7f17040a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 >> 172.21.15.46:0/3256297962 conn(0x7f17040cbe30 msgr2=0x7f1704068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 shutdown_connections 2023-10-06T11:34:04.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.763+0000 7f1708c2e700 1 -- 172.21.15.46:0/3256297962 wait complete. 2023-10-06T11:34:04.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:34:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.943+0000 7f163a9cc700 1 Processor -- start 2023-10-06T11:34:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.943+0000 7f163a9cc700 1 -- start start 2023-10-06T11:34:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f163a9cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1634159290 0x7f16340b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f163a9cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f16340b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f163a9cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340ae340 con 0x7f16340b1130 2023-10-06T11:34:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f163a9cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340b15b0 con 0x7f1634159290 2023-10-06T11:34:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f163a9cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340b09a0 con 0x7f16340d07b0 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1634159290 0x7f16340b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1634159290 0x7f16340b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58344/0 (socket says 172.21.15.46:58344) 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 -- 172.21.15.46:0/1769867596 learned_addr learned my addr 172.21.15.46:0/1769867596 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16391c9700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f16340b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 -- 172.21.15.46:0/1769867596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 msgr2=0x7f16340b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f16340b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 -- 172.21.15.46:0/1769867596 >> v1:172.21.15.132:6789/0 conn(0x7f16340b1130 legacy=0x7f16340ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 -- 172.21.15.46:0/1769867596 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634159660 con 0x7f1634159290 2023-10-06T11:34:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16399ca700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1634159290 0x7f16340b2f10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f16300099b0 tx=0x7f1630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b189d11457d6b0ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16389c8700 1 -- 172.21.15.46:0/1769867596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000b8f0 con 0x7f1634159290 2023-10-06T11:34:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16389c8700 1 -- 172.21.15.46:0/1769867596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163000ba50 con 0x7f1634159290 2023-10-06T11:34:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.947+0000 7f16389c8700 1 -- 172.21.15.46:0/1769867596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000bdf0 con 0x7f1634159290 2023-10-06T11:34:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 -- 172.21.15.46:0/1769867596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1634159290 msgr2=0x7f16340b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1634159290 0x7f16340b2f10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f16300099b0 tx=0x7f1630009250 comp rx=0 tx=0).stop 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 -- 172.21.15.46:0/1769867596 shutdown_connections 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f16340b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1769867596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1634159290 0x7f16340b2f10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 -- 172.21.15.46:0/1769867596 >> 172.21.15.46:0/1769867596 conn(0x7f163406ef60 msgr2=0x7f16340d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 -- 172.21.15.46:0/1769867596 shutdown_connections 2023-10-06T11:34:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 -- 172.21.15.46:0/1769867596 wait complete. 2023-10-06T11:34:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.951+0000 7f163a9cc700 1 Processor -- start 2023-10-06T11:34:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 -- start start 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f1634158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1634159290 0x7f1634159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163409c0d0 con 0x7f16340b1130 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340ce450 con 0x7f1634159290 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340cd7d0 con 0x7f16340d07b0 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58360/0 (socket says 172.21.15.46:58360) 2023-10-06T11:34:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 -- 172.21.15.46:0/1731881918 learned_addr learned my addr 172.21.15.46:0/1731881918 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16391c9700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f1634158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a1cb700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1634159290 0x7f1634159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 -- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 msgr2=0x7f1634158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f1634158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 -- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1634159290 msgr2=0x7f1634159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1634159290 0x7f1634159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 -- 172.21.15.46:0/1731881918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16300094f0 con 0x7f16340b1130 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f16399ca700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1630009290 tx=0x7f1630017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163001a420 con 0x7f16340b1130 2023-10-06T11:34:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163001a5a0 con 0x7f16340b1130 2023-10-06T11:34:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340cdab0 con 0x7f16340b1130 2023-10-06T11:34:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.955+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163001a960 con 0x7f16340b1130 2023-10-06T11:34:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.959+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16340ce070 con 0x7f16340b1130 2023-10-06T11:34:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.959+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1634064590 con 0x7f16340b1130 2023-10-06T11:34:04.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.959+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f163001acd0 con 0x7f16340b1130 2023-10-06T11:34:04.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.963+0000 7f1629ffb700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 0x7f1618062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:04.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.963+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f163007e770 con 0x7f16340b1130 2023-10-06T11:34:04.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.963+0000 7f16391c9700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 0x7f1618062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:04.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.963+0000 7f16391c9700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 0x7f1618062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f163409b220 tx=0x7f1624008040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:04.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:04.967+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f163004e890 con 0x7f16340b1130 2023-10-06T11:34:05.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.171+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f16340675d0 con 0x7f16340b1130 2023-10-06T11:34:05.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.171+0000 7f1629ffb700 1 -- 172.21.15.46:0/1731881918 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1630051eb0 con 0x7f16340b1130 2023-10-06T11:34:05.182 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 msgr2=0x7f1618062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 0x7f1618062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f163409b220 tx=0x7f1624008040 comp rx=0 tx=0).stop 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 msgr2=0x7f16340ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1630009290 tx=0x7f1630017000 comp rx=0 tx=0).stop 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 shutdown_connections 2023-10-06T11:34:05.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1618060670 0x7f1618062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1634159290 0x7f1634159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16340d07b0 0x7f1634158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 --2- 172.21.15.46:0/1731881918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16340b1130 0x7f16340ceda0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.175+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 >> 172.21.15.46:0/1731881918 conn(0x7f163406ef60 msgr2=0x7f1634067bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.179+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 shutdown_connections 2023-10-06T11:34:05.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:05.179+0000 7f163a9cc700 1 -- 172.21.15.46:0/1731881918 wait complete. 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:34:05.207 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:34:05.208 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:05.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.208 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:34:05.222 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:34:05.222 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41485sec preferred_lft 41485sec 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:34:05.224 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:34:05.225 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.226 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:34:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:34:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:34:05.226 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:05.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.227 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:34:05.312 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:34:05.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.315 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:34:05.379 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:34:05.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.380 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:34:05.380 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:34:05.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:34:05.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:34:05.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:34:05.381 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:34:05.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.523 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:34:05.537 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:34:05.537 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:34:05.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:05.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:34:05.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:34:05.618 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:05.678 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:05.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:34:05.678 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:05.737 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:34:05.737 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:34:05.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:05.894 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:05.894 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:05.894 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:05.894 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:34:05.894 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:34:05.907 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:34:05.908 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:34:05.914 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:34:05.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:05.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:05.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:34:06.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.155+0000 7fd8aed05700 1 Processor -- start 2023-10-06T11:34:06.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.155+0000 7fd8aed05700 1 -- start start 2023-10-06T11:34:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8aed05700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a80ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8aed05700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a806d460 con 0x7fd8a815ab60 2023-10-06T11:34:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8aed05700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80c7000 con 0x7fd8a806e8a0 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8aed05700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a806ed40 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8acaa1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8a815ab60 0x7fd8a815af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51532/0 (socket says 172.21.15.46:51532) 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8ad2a2700 1 --2- 172.21.15.46:0/3606883176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a80ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8acaa1700 1 -- 172.21.15.46:0/3606883176 learned_addr learned my addr 172.21.15.46:0/3606883176 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873383062 0 0) 0x7fd8a80c7000 con 0x7fd8a806e8a0 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd898003570 con 0x7fd8a806e8a0 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295391775 0 0) 0x7fd8a806d460 con 0x7fd8a815ab60 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a80c7000 con 0x7fd8a815ab60 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8ad2a2700 1 -- 172.21.15.46:0/3606883176 >> v1:172.21.15.132:6789/0 conn(0x7fd8a806e8a0 legacy=0x7fd8a806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8ad2a2700 1 -- 172.21.15.46:0/3606883176 >> v1:172.21.15.46:6789/0 conn(0x7fd8a815ab60 legacy=0x7fd8a815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8ad2a2700 1 -- 172.21.15.46:0/3606883176 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80c72c0 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8ad2a2700 1 --2- 172.21.15.46:0/3606883176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a80ad6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009e90 tx=0x7fd8a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da9cb98de444bdc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000e070 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000bb80 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.159+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3606883176 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0012430 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- 172.21.15.46:0/3606883176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 msgr2=0x7fd8a80ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 --2- 172.21.15.46:0/3606883176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a80ad6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009e90 tx=0x7fd8a0009780 comp rx=0 tx=0).stop 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- 172.21.15.46:0/3606883176 shutdown_connections 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 --2- 172.21.15.46:0/3606883176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a80ad6c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- 172.21.15.46:0/3606883176 >> 172.21.15.46:0/3606883176 conn(0x7fd8a80cbe30 msgr2=0x7fd8a80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- 172.21.15.46:0/3606883176 shutdown_connections 2023-10-06T11:34:06.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- 172.21.15.46:0/3606883176 wait complete. 2023-10-06T11:34:06.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 Processor -- start 2023-10-06T11:34:06.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.163+0000 7fd8aed05700 1 -- start start 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a8169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8a815ab60 0x7fd8a8069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80ca770 con 0x7fd8a806e8a0 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c9ba0 con 0x7fd8a815ab60 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8aed05700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c9e40 con 0x7fd8a80c1ec0 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58376/0 (socket says 172.21.15.46:58376) 2023-10-06T11:34:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 -- 172.21.15.46:0/2046002364 learned_addr learned my addr 172.21.15.46:0/2046002364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8a7fff700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a8169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8ad2a2700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8a815ab60 0x7fd8a8069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 -- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 msgr2=0x7fd8a8169640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a8169640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 -- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8a815ab60 msgr2=0x7fd8a8069f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8a815ab60 0x7fd8a8069f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8acaa1700 1 -- 172.21.15.46:0/2046002364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0009a20 con 0x7fd8a806e8a0 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.167+0000 7fd8a7fff700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a8169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8acaa1700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00c790 tx=0x7fd89c007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0072e0 con 0x7fd8a806e8a0 2023-10-06T11:34:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c007440 con 0x7fd8a806e8a0 2023-10-06T11:34:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0057f0 con 0x7fd8a806e8a0 2023-10-06T11:34:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a80ca120 con 0x7fd8a806e8a0 2023-10-06T11:34:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a816a4b0 con 0x7fd8a806e8a0 2023-10-06T11:34:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.171+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8063f90 con 0x7fd8a806e8a0 2023-10-06T11:34:06.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.175+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd89c005be0 con 0x7fd8a806e8a0 2023-10-06T11:34:06.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.175+0000 7fd8a4ff9700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 0x7fd88c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.175+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd89c07c260 con 0x7fd8a806e8a0 2023-10-06T11:34:06.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.175+0000 7fd8a7fff700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 0x7fd88c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.179+0000 7fd8a7fff700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 0x7fd88c062b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80ae540 tx=0x7fd894009040 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.183+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd89c04c380 con 0x7fd8a806e8a0 2023-10-06T11:34:06.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.375+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8a80ca8b0 con 0x7fd88c0606a0 2023-10-06T11:34:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.379+0000 7fd8a4ff9700 1 -- 172.21.15.46:0/2046002364 <== mgr.6502 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd894003d30 con 0x7fd88c0606a0 2023-10-06T11:34:06.390 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:34:06.390 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:06.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:34:06.390 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:06.390 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:34:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 msgr2=0x7fd88c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 0x7fd88c062b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80ae540 tx=0x7fd894009040 comp rx=0 tx=0).stop 2023-10-06T11:34:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 msgr2=0x7fd8a8169f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00c790 tx=0x7fd89c007c60 comp rx=0 tx=0).stop 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 shutdown_connections 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd88c0606a0 0x7fd88c062b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8a815ab60 0x7fd8a8069f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8a80c1ec0 0x7fd8a8169640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 --2- 172.21.15.46:0/2046002364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8a806e8a0 0x7fd8a8169f70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 >> 172.21.15.46:0/2046002364 conn(0x7fd8a80cbe30 msgr2=0x7fd8a80cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 shutdown_connections 2023-10-06T11:34:06.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.383+0000 7fd8aed05700 1 -- 172.21.15.46:0/2046002364 wait complete. 2023-10-06T11:34:06.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:34:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.563+0000 7f2e13985700 1 Processor -- start 2023-10-06T11:34:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.563+0000 7f2e13985700 1 -- start start 2023-10-06T11:34:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e13985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2e0c068f20 0x7f2e0c069370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e13985700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c068a90 con 0x7f2e0c0b0010 2023-10-06T11:34:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e13985700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c0698b0 con 0x7f2e0c0afc40 2023-10-06T11:34:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e13985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0b18c0 con 0x7f2e0c068f20 2023-10-06T11:34:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e10f20700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e0c0b0010 0x7f2e0c0b0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51546/0 (socket says 172.21.15.46:51546) 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e10f20700 1 -- 172.21.15.46:0/2076983715 learned_addr learned my addr 172.21.15.46:0/2076983715 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e11f22700 1 --2- 172.21.15.46:0/2076983715 >> v2:172.21.15.132:3300/0 conn(0x7f2e0c068f20 0x7f2e0c069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e11f22700 1 -- 172.21.15.46:0/2076983715 >> v1:172.21.15.132:6789/0 conn(0x7f2e0c0afc40 legacy=0x7f2e0c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e11f22700 1 -- 172.21.15.46:0/2076983715 >> v1:172.21.15.46:6789/0 conn(0x7f2e0c0b0010 legacy=0x7f2e0c0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e03fff700 1 -- 172.21.15.46:0/2076983715 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417507156 0 0) 0x7f2e0c068a90 con 0x7f2e0c0b0010 2023-10-06T11:34:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.567+0000 7f2e11f22700 1 -- 172.21.15.46:0/2076983715 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c068160 con 0x7f2e0c068f20 2023-10-06T11:34:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e11f22700 1 --2- 172.21.15.46:0/2076983715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c069370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009e60 tx=0x7f2e04009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4aa5f156826035 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e03fff700 1 -- 172.21.15.46:0/2076983715 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400e070 con 0x7f2e0c068f20 2023-10-06T11:34:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e03fff700 1 -- 172.21.15.46:0/2076983715 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0400bc30 con 0x7f2e0c068f20 2023-10-06T11:34:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e03fff700 1 -- 172.21.15.46:0/2076983715 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04012430 con 0x7f2e0c068f20 2023-10-06T11:34:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 -- 172.21.15.46:0/2076983715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 msgr2=0x7f2e0c069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 --2- 172.21.15.46:0/2076983715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c069370 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009e60 tx=0x7f2e04009780 comp rx=0 tx=0).stop 2023-10-06T11:34:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 -- 172.21.15.46:0/2076983715 shutdown_connections 2023-10-06T11:34:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 --2- 172.21.15.46:0/2076983715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c069370 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 -- 172.21.15.46:0/2076983715 >> 172.21.15.46:0/2076983715 conn(0x7f2e0c06aea0 msgr2=0x7f2e0c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 -- 172.21.15.46:0/2076983715 shutdown_connections 2023-10-06T11:34:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.571+0000 7f2e13985700 1 -- 172.21.15.46:0/2076983715 wait complete. 2023-10-06T11:34:06.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 Processor -- start 2023-10-06T11:34:06.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 -- start start 2023-10-06T11:34:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e0c0b0010 0x7f2e0c13c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c16d490 con 0x7f2e0c0afc40 2023-10-06T11:34:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c16a3d0 con 0x7f2e0c068f20 2023-10-06T11:34:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e13985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c16a6b0 con 0x7f2e0c0b0010 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58380/0 (socket says 172.21.15.46:58380) 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 -- 172.21.15.46:0/3934903520 learned_addr learned my addr 172.21.15.46:0/3934903520 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e11f22700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e0c0b0010 0x7f2e0c13c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e11721700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 -- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e0c0b0010 msgr2=0x7f2e0c13c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e0c0b0010 0x7f2e0c13c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 -- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 msgr2=0x7f2e0c158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:06.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:06.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.575+0000 7f2e10f20700 1 -- 172.21.15.46:0/3934903520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e04009a20 con 0x7f2e0c0afc40 2023-10-06T11:34:06.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e10f20700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00c5b0 tx=0x7f2dfc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc005b30 con 0x7f2e0c0afc40 2023-10-06T11:34:06.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc005c90 con 0x7f2e0c0afc40 2023-10-06T11:34:06.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c16d120 con 0x7f2e0c0afc40 2023-10-06T11:34:06.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc007300 con 0x7f2e0c0afc40 2023-10-06T11:34:06.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1590b0 con 0x7f2e0c0afc40 2023-10-06T11:34:06.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c159270 con 0x7f2e0c0afc40 2023-10-06T11:34:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.579+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 35) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2dfc01b3b0 con 0x7f2e0c0afc40 2023-10-06T11:34:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.583+0000 7f2e017fa700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 0x7f2df0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.583+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f2dfc07c110 con 0x7f2e0c0afc40 2023-10-06T11:34:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.583+0000 7f2e11721700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 0x7f2df0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.583+0000 7f2e11721700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 0x7f2df0062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e08003a80 tx=0x7f2e08007ec0 comp rx=0 tx=0).ready entity=mgr.6502 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:06.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.587+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2dfc04c230 con 0x7f2e0c0afc40 2023-10-06T11:34:06.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:06.799+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2e0c16cf80 con 0x7f2e0c0afc40 2023-10-06T11:34:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.695+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 36) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f2dfc04c600 con 0x7f2e0c0afc40 2023-10-06T11:34:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.695+0000 7f2e017fa700 1 -- 172.21.15.46:0/3934903520 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v36) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2dfc0466a0 con 0x7f2e0c0afc40 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 msgr2=0x7f2df0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 0x7f2df0062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e08003a80 tx=0x7f2e08007ec0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 msgr2=0x7f2e0c169940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00c5b0 tx=0x7f2dfc001070 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 shutdown_connections 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2df0060670 0x7f2df0062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e0c0b0010 0x7f2e0c13c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0afc40 0x7f2e0c169940 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 --2- 172.21.15.46:0/3934903520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e0c068f20 0x7f2e0c158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 >> 172.21.15.46:0/3934903520 conn(0x7f2e0c06aea0 msgr2=0x7f2e0c1418d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 shutdown_connections 2023-10-06T11:34:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.699+0000 7f2e13985700 1 -- 172.21.15.46:0/3934903520 wait complete. 2023-10-06T11:34:07.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:34:07.888 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:07.885+0000 7fa606566000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.883+0000 7f750aaae700 1 Processor -- start 2023-10-06T11:34:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.883+0000 7f750aaae700 1 -- start start 2023-10-06T11:34:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f750aaae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75040b27c0 0x7f7504162010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f750aaae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750415b880 con 0x7f7504157210 2023-10-06T11:34:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f750aaae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750415ac70 con 0x7f7504156e40 2023-10-06T11:34:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f750aaae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040cab50 con 0x7f75040b27c0 2023-10-06T11:34:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f7503fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7504157210 0x7f75040b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51566/0 (socket says 172.21.15.46:51566) 2023-10-06T11:34:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f7503fff700 1 -- 172.21.15.46:0/3599171269 learned_addr learned my addr 172.21.15.46:0/3599171269 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f750904b700 1 --2- 172.21.15.46:0/3599171269 >> v2:172.21.15.132:3300/0 conn(0x7f75040b27c0 0x7f7504162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256316201 0 0) 0x7f750415b880 con 0x7f7504157210 2023-10-06T11:34:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec003570 con 0x7f7504157210 2023-10-06T11:34:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226915322 0 0) 0x7f74ec003570 con 0x7f7504157210 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 >> v2:172.21.15.132:3300/0 conn(0x7f75040b27c0 msgr2=0x7f7504162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 --2- 172.21.15.46:0/3599171269 >> v2:172.21.15.132:3300/0 conn(0x7f75040b27c0 0x7f7504162010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 >> v1:172.21.15.132:6789/0 conn(0x7f7504156e40 legacy=0x7f7504069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750415af50 con 0x7f7504157210 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f74f4001470 con 0x7f7504157210 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 >> v1:172.21.15.46:6789/0 conn(0x7f7504157210 legacy=0x7f75040b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec003a60 0x7f74ec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74ec004810 0x7f74ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 0x7f74ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040cab50 con 0x7f74ec0051c0 2023-10-06T11:34:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00c0e0 con 0x7f74ec003a60 2023-10-06T11:34:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.887+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec00c2a0 con 0x7f74ec004810 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f7503fff700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74ec004810 0x7f74ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 0x7f74ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750884a700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec003a60 0x7f74ec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 -- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74ec004810 msgr2=0x7f74ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74ec004810 0x7f74ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 -- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec003a60 msgr2=0x7f74ec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec003a60 0x7f74ec003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 -- 172.21.15.46:0/3599171269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec003570 con 0x7f74ec0051c0 2023-10-06T11:34:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 -- 172.21.15.46:0/3599171269 reap_dead start 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750904b700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 0x7f74ec00ba60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f75040b2bc0 tx=0x7f74fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f340a5ce658c64d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00ea90 con 0x7f74ec0051c0 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc00f040 con 0x7f74ec0051c0 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f75037fe700 1 -- 172.21.15.46:0/3599171269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00ee40 con 0x7f74ec0051c0 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 -- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 msgr2=0x7f74ec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 0x7f74ec00ba60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f75040b2bc0 tx=0x7f74fc00b040 comp rx=0 tx=0).stop 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 -- 172.21.15.46:0/3599171269 shutdown_connections 2023-10-06T11:34:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 --2- 172.21.15.46:0/3599171269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec0051c0 0x7f74ec00ba60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 -- 172.21.15.46:0/3599171269 >> 172.21.15.46:0/3599171269 conn(0x7f7504068c30 msgr2=0x7f75040aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 -- 172.21.15.46:0/3599171269 shutdown_connections 2023-10-06T11:34:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.891+0000 7f750aaae700 1 -- 172.21.15.46:0/3599171269 wait complete. 2023-10-06T11:34:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.895+0000 7f750aaae700 1 Processor -- start 2023-10-06T11:34:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.895+0000 7f750aaae700 1 -- start start 2023-10-06T11:34:07.906 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:07.895+0000 7fcaa135a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec0051c0 0x7f7504140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7504150cf0 0x7f75040abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040ac2f0 con 0x7f75041576c0 2023-10-06T11:34:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040aa930 con 0x7f74ec0051c0 2023-10-06T11:34:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7504151180 con 0x7f7504150cf0 2023-10-06T11:34:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58402/0 (socket says 172.21.15.46:58402) 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 -- 172.21.15.46:0/602557596 learned_addr learned my addr 172.21.15.46:0/602557596 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750884a700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec0051c0 0x7f7504140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f7503fff700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7504150cf0 0x7f75040abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 -- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7504150cf0 msgr2=0x7f75040abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7504150cf0 0x7f75040abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 -- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec0051c0 msgr2=0x7f7504140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec0051c0 0x7f7504140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 -- 172.21.15.46:0/602557596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040aa2e0 con 0x7f75041576c0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750904b700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f75040b3070 tx=0x7f74fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc01d660 con 0x7f75041576c0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc022070 con 0x7f75041576c0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc01da30 con 0x7f75041576c0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75040aa5c0 con 0x7f75041576c0 2023-10-06T11:34:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.899+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750414f000 con 0x7f75041576c0 2023-10-06T11:34:07.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.903+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7504063820 con 0x7f75041576c0 2023-10-06T11:34:07.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.903+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 36) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f74fc003020 con 0x7f75041576c0 2023-10-06T11:34:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.903+0000 7f7500ff9700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 0x7f74f0062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.903+0000 7f750884a700 1 -- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 msgr2=0x7f74f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.903+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f74fc024070 con 0x7f75041576c0 2023-10-06T11:34:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.907+0000 7f750884a700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 0x7f74f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:07.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:07.911+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74fc04ef60 con 0x7f75041576c0 2023-10-06T11:34:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.095+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7504152120 con 0x7f75041576c0 2023-10-06T11:34:08.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.103+0000 7f7500ff9700 1 -- 172.21.15.46:0/602557596 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f74fc0180d0 con 0x7f75041576c0 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 msgr2=0x7f74f0062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 0x7f74f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 msgr2=0x7f750415d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f75040b3070 tx=0x7f74fc019000 comp rx=0 tx=0).stop 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 shutdown_connections 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74f00606c0 0x7f74f0062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75041576c0 0x7f750415d350 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7504150cf0 0x7f75040abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 --2- 172.21.15.46:0/602557596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74ec0051c0 0x7f7504140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 >> 172.21.15.46:0/602557596 conn(0x7f7504068c30 msgr2=0x7f75040b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:08.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 shutdown_connections 2023-10-06T11:34:08.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.107+0000 7f750aaae700 1 -- 172.21.15.46:0/602557596 wait complete. 2023-10-06T11:34:08.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:34:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.303+0000 7fcd57fff700 1 Processor -- start 2023-10-06T11:34:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.303+0000 7fcd57fff700 1 -- start start 2023-10-06T11:34:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 0x7fcd580b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcd580b0dc0 0x7fcd580b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580afd20 con 0x7fcd580adbe0 2023-10-06T11:34:08.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd58159740 con 0x7fcd580b0dc0 2023-10-06T11:34:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580b1760 con 0x7fcd580b30f0 2023-10-06T11:34:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd56ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd580adbe0 0x7fcd580d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51580/0 (socket says 172.21.15.46:51580) 2023-10-06T11:34:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd56ffd700 1 -- 172.21.15.46:0/244447980 learned_addr learned my addr 172.21.15.46:0/244447980 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 --2- 172.21.15.46:0/244447980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 0x7fcd580b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd577fe700 1 --2- 172.21.15.46:0/244447980 >> v2:172.21.15.132:3300/0 conn(0x7fcd580b0dc0 0x7fcd580b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd55ffb700 1 -- 172.21.15.46:0/244447980 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646676000 0 0) 0x7fcd580afd20 con 0x7fcd580adbe0 2023-10-06T11:34:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd55ffb700 1 -- 172.21.15.46:0/244447980 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003570 con 0x7fcd580adbe0 2023-10-06T11:34:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 -- 172.21.15.46:0/244447980 >> v2:172.21.15.132:3300/0 conn(0x7fcd580b0dc0 msgr2=0x7fcd580b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 --2- 172.21.15.46:0/244447980 >> v2:172.21.15.132:3300/0 conn(0x7fcd580b0dc0 0x7fcd580b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 -- 172.21.15.46:0/244447980 >> v1:172.21.15.46:6789/0 conn(0x7fcd580adbe0 legacy=0x7fcd580d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 -- 172.21.15.46:0/244447980 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580b0000 con 0x7fcd580b30f0 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd567fc700 1 --2- 172.21.15.46:0/244447980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 0x7fcd580b3540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0098e0 tx=0x7fcd4c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76e1c299e40cdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd55ffb700 1 -- 172.21.15.46:0/244447980 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c011630 con 0x7fcd580b30f0 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd55ffb700 1 -- 172.21.15.46:0/244447980 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c011790 con 0x7fcd580b30f0 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd55ffb700 1 -- 172.21.15.46:0/244447980 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c011b30 con 0x7fcd580b30f0 2023-10-06T11:34:08.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 -- 172.21.15.46:0/244447980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 msgr2=0x7fcd580b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.307+0000 7fcd57fff700 1 --2- 172.21.15.46:0/244447980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 0x7fcd580b3540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0098e0 tx=0x7fcd4c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- 172.21.15.46:0/244447980 shutdown_connections 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 --2- 172.21.15.46:0/244447980 >> v2:172.21.15.132:3300/0 conn(0x7fcd580b0dc0 0x7fcd580b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 --2- 172.21.15.46:0/244447980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580b30f0 0x7fcd580b3540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- 172.21.15.46:0/244447980 >> 172.21.15.46:0/244447980 conn(0x7fcd5806ef60 msgr2=0x7fcd580d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- 172.21.15.46:0/244447980 shutdown_connections 2023-10-06T11:34:08.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- 172.21.15.46:0/244447980 wait complete. 2023-10-06T11:34:08.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 Processor -- start 2023-10-06T11:34:08.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- start start 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 0x7fcd58169a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd580b30f0 0x7fcd58155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5816a090 con 0x7fcd580b0dc0 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5816a370 con 0x7fcd580b30f0 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd57fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5816a650 con 0x7fcd580adbe0 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd56ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 0x7fcd58169a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58406/0 (socket says 172.21.15.46:58406) 2023-10-06T11:34:08.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd567fc700 1 -- 172.21.15.46:0/2430461474 learned_addr learned my addr 172.21.15.46:0/2430461474 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:08.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.311+0000 7fcd577fe700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd580b30f0 0x7fcd58155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 -- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 msgr2=0x7fcd58169a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 0x7fcd58169a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 -- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd580b30f0 msgr2=0x7fcd58155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd580b30f0 0x7fcd58155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 -- 172.21.15.46:0/2430461474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009420 con 0x7fcd580b0dc0 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd56ffd700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 0x7fcd58169a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd567fc700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ed0 tx=0x7fcd4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c000d10 con 0x7fcd580b0dc0 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c011b10 con 0x7fcd580b0dc0 2023-10-06T11:34:08.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5816a930 con 0x7fcd580b0dc0 2023-10-06T11:34:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c025660 con 0x7fcd580b0dc0 2023-10-06T11:34:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5809dac0 con 0x7fcd580b0dc0 2023-10-06T11:34:08.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.315+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58099f90 con 0x7fcd580b0dc0 2023-10-06T11:34:08.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.319+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 36) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fcd4c025a50 con 0x7fcd580b0dc0 2023-10-06T11:34:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.319+0000 7fcd3effd700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 0x7fcd44062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.319+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fcd4c07e500 con 0x7fcd580b0dc0 2023-10-06T11:34:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.319+0000 7fcd56ffd700 1 -- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 msgr2=0x7fcd44062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:08.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.319+0000 7fcd56ffd700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 0x7fcd44062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:08.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.323+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd4c04e620 con 0x7fcd580b0dc0 2023-10-06T11:34:08.425 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:08.421+0000 7fa606566000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:08.494 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:08.489+0000 7fa606566000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:08.508 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:08.499+0000 7fcaa135a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:08.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.499+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fcd58064590 con 0x7fcd580b0dc0 2023-10-06T11:34:08.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.507+0000 7fcd3effd700 1 -- 172.21.15.46:0/2430461474 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcd4c051c40 con 0x7fcd580b0dc0 2023-10-06T11:34:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 msgr2=0x7fcd44062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 0x7fcd44062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 msgr2=0x7fcd5809d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ed0 tx=0x7fcd4c017000 comp rx=0 tx=0).stop 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 shutdown_connections 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcd440606c0 0x7fcd44062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd580b30f0 0x7fcd58155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd580b0dc0 0x7fcd5809d580 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 --2- 172.21.15.46:0/2430461474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd580adbe0 0x7fcd58169a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 >> 172.21.15.46:0/2430461474 conn(0x7fcd5806ef60 msgr2=0x7fcd58158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 shutdown_connections 2023-10-06T11:34:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.511+0000 7fcd57fff700 1 -- 172.21.15.46:0/2430461474 wait complete. 2023-10-06T11:34:08.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T11:34:08.589 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:08.579+0000 7fcaa135a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:08.630 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:08.625+0000 7fa606566000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.691+0000 7ff50eb45700 1 Processor -- start 2023-10-06T11:34:08.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.691+0000 7ff50eb45700 1 -- start start 2023-10-06T11:34:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50eb45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508159130 0x7ff5080d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50eb45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff508159500 0x7ff5080aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50eb45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080ce2c0 con 0x7ff508069190 2023-10-06T11:34:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50eb45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5080afb30 con 0x7ff508159500 2023-10-06T11:34:08.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50eb45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cc090 con 0x7ff508159130 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff508159500 0x7ff5080aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff508159500 0x7ff5080aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58410/0 (socket says 172.21.15.46:58410) 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 -- 172.21.15.46:0/43438777 learned_addr learned my addr 172.21.15.46:0/43438777 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508159130 0x7ff5080d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 -- 172.21.15.46:0/43438777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508159130 msgr2=0x7ff5080d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508159130 0x7ff5080d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 -- 172.21.15.46:0/43438777 >> v1:172.21.15.46:6789/0 conn(0x7ff508069190 legacy=0x7ff50815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 -- 172.21.15.46:0/43438777 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080af570 con 0x7ff508159500 2023-10-06T11:34:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff507fff700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159500 0x7ff5080aed40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40099b0 tx=0x7ff4f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82212d0a97a5b58f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff5077fe700 1 -- 172.21.15.46:0/43438777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4009ea0 con 0x7ff508159500 2023-10-06T11:34:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff5077fe700 1 -- 172.21.15.46:0/43438777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f400b860 con 0x7ff508159500 2023-10-06T11:34:08.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.695+0000 7ff5077fe700 1 -- 172.21.15.46:0/43438777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400bc00 con 0x7ff508159500 2023-10-06T11:34:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 -- 172.21.15.46:0/43438777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159500 msgr2=0x7ff5080aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159500 0x7ff5080aed40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40099b0 tx=0x7ff4f4009250 comp rx=0 tx=0).stop 2023-10-06T11:34:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 -- 172.21.15.46:0/43438777 shutdown_connections 2023-10-06T11:34:08.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159500 0x7ff5080aed40 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 --2- 172.21.15.46:0/43438777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508159130 0x7ff5080d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 -- 172.21.15.46:0/43438777 >> 172.21.15.46:0/43438777 conn(0x7ff508068510 msgr2=0x7ff5080d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 -- 172.21.15.46:0/43438777 shutdown_connections 2023-10-06T11:34:08.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.699+0000 7ff50eb45700 1 -- 172.21.15.46:0/43438777 wait complete. 2023-10-06T11:34:08.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 Processor -- start 2023-10-06T11:34:08.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 -- start start 2023-10-06T11:34:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508069190 0x7ff5080c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159130 0x7ff508159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff508159500 0x7ff5080c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080ca180 con 0x7ff508159130 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b4150 con 0x7ff508159500 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.703+0000 7ff50eb45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b4430 con 0x7ff508069190 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159130 0x7ff508159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159130 0x7ff508159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58420/0 (socket says 172.21.15.46:58420) 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 -- 172.21.15.46:0/3620842203 learned_addr learned my addr 172.21.15.46:0/3620842203 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508069190 0x7ff5080c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50d0e2700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff508159500 0x7ff5080c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:08.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508069190 msgr2=0x7ff5080c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff508069190 0x7ff5080c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff508159500 msgr2=0x7ff5080c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff508159500 0x7ff5080c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f40094f0 con 0x7ff508159130 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50d0e2700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff508159500 0x7ff5080c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff507fff700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff508159130 0x7ff508159a10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff508064f70 tx=0x7ff4f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4000d10 con 0x7ff508159130 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f400bbe0 con 0x7ff508159130 2023-10-06T11:34:08.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50eb45700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5080b4710 con 0x7ff508159130 2023-10-06T11:34:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4011670 con 0x7ff508159130 2023-10-06T11:34:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50eb45700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff508159f50 con 0x7ff508159130 2023-10-06T11:34:08.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.707+0000 7ff50eb45700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff508088540 con 0x7ff508159130 2023-10-06T11:34:08.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.711+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 36) v1 ==== 80078+0+0 (secure 0 0 0) 0x7ff4f4011aa0 con 0x7ff508159130 2023-10-06T11:34:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.711+0000 7ff504ff9700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.711+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ff4f407e530 con 0x7ff508159130 2023-10-06T11:34:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.711+0000 7ff50c8e1700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:08.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.711+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:08.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.715+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff4f404e650 con 0x7ff508159130 2023-10-06T11:34:08.741 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:08.731+0000 7fcaa135a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:08.881 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:08.877+0000 7fa606566000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:08.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.907+0000 7ff50eb45700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5080af850 con 0x7ff4ec0606c0 2023-10-06T11:34:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.915+0000 7ff50c8e1700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:08.915+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:08.948 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:08.945+0000 7fa606566000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:09.023 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.011+0000 7fcaa135a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:09.098 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.087+0000 7fcaa135a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:09.157 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:09.153+0000 7fa606566000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:09.239 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:09.237+0000 7fa606566000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:09.323 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:09.321+0000 7fa606566000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:09.326 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.315+0000 7fcaa135a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:09.319+0000 7ff50c8e1700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:09.319+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:34:09.410 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.399+0000 7fcaa135a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:09.500 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.491+0000 7fcaa135a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:09.690 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:09.685+0000 7fa606566000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:09.846 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:09.841+0000 7fa606566000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:09.910 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:09.899+0000 7fcaa135a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:10.085 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.075+0000 7fcaa135a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:10.123+0000 7ff50c8e1700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:10.123+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:34:10.214 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.209+0000 7fa606566000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:10.423 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.421+0000 7fa606566000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:10.464 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.455+0000 7fcaa135a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:10.513 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.509+0000 7fa606566000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:10.585 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.581+0000 7fa606566000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:10.659 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.653+0000 7fa606566000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:10.681 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.667+0000 7fcaa135a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:10.766 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.755+0000 7fcaa135a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:10.841 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.831+0000 7fcaa135a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:10.879 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.877+0000 7fa606566000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:10.924 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:10.915+0000 7fcaa135a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:10.955 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:10.953+0000 7fa606566000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:11.042 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:11.037+0000 7fa606566000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:11.143 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:11.131+0000 7fcaa135a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:11.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:11.211+0000 7fcaa135a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:11.228 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:11.225+0000 7fa606566000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:11.309 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:11.305+0000 7fa606566000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:11.311 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:11.299+0000 7fcaa135a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:11.502 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:11.491+0000 7fcaa135a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:11.590 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:11.579+0000 7fcaa135a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:11.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:11.727+0000 7ff50c8e1700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:11.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:11.727+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:34:12.011 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:12.009+0000 7fa606566000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:12.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:12.067+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 37) v1 ==== 80078+0+0 (secure 0 0 0) 0x7ff4f4011d50 con 0x7ff508159130 2023-10-06T11:34:12.324 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.315+0000 7fcaa135a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:12.331+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 38) v1 ==== 79687+0+0 (secure 0 0 0) 0x7ff4f404ea20 con 0x7ff508159130 2023-10-06T11:34:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:12.331+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 msgr2=0x7ff4ec062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:12.331+0000 7ff504ff9700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:12.513 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca5fde6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.514 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca5fde6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.514 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca5fde6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.514 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca5fde6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.514 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca5fde6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.515 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca61dea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.515 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca61dea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.515 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca61dea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.515 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca61dea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:12.515 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:12.503+0000 7fca61dea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.343+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 39) v1 ==== 79814+0+0 (secure 0 0 0) 0x7ff4f4008400 con 0x7ff508159130 2023-10-06T11:34:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.343+0000 7ff504ff9700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.343+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5080af850 con 0x7ff4ec0606c0 2023-10-06T11:34:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.347+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.351+0000 7ff50c8e1700 1 --2- 172.21.15.46:0/3620842203 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff4ec0606c0 0x7ff4ec062af0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff50815cd10 tx=0x7ff4fc0091a0 comp rx=0 tx=0).ready entity=mgr.7020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:13.370 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:13.359+0000 7fca69dfa700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T11:34:13.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.359+0000 7ff504ff9700 1 -- 172.21.15.46:0/3620842203 <== mgr.7020 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7ff5080af850 con 0x7ff4ec0606c0 2023-10-06T11:34:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T11:34:13.380 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T11:34:13.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:34:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 Processor -- start 2023-10-06T11:34:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 -- start start 2023-10-06T11:34:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f46680ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466806ed10 0x7f46680c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46680c1bf0 con 0x7f466806e8a0 2023-10-06T11:34:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46680abbe0 con 0x7f466806ed10 2023-10-06T11:34:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.587+0000 7f466e0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466806cf80 con 0x7f46680c6470 2023-10-06T11:34:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466806ed10 0x7f46680c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466806ed10 0x7f46680c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59348/0 (socket says 172.21.15.46:59348) 2023-10-06T11:34:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 -- 172.21.15.46:0/1355502295 learned_addr learned my addr 172.21.15.46:0/1355502295 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f46677fe700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f46680ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 -- 172.21.15.46:0/1355502295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 msgr2=0x7f46680ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f46680ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 -- 172.21.15.46:0/1355502295 >> v1:172.21.15.132:6789/0 conn(0x7f466806e8a0 legacy=0x7f46680c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 -- 172.21.15.46:0/1355502295 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466806d260 con 0x7f466806ed10 2023-10-06T11:34:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f46677fe700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f46680ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f4667fff700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f46680c2080 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4660009f20 tx=0x7f4660009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f77aea5d6f9dd1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:13.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f46667fc700 1 -- 172.21.15.46:0/1355502295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000e070 con 0x7f466806ed10 2023-10-06T11:34:13.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f46667fc700 1 -- 172.21.15.46:0/1355502295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000ba80 con 0x7f466806ed10 2023-10-06T11:34:13.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f46667fc700 1 -- 172.21.15.46:0/1355502295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46600092e0 con 0x7f466806ed10 2023-10-06T11:34:13.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 -- 172.21.15.46:0/1355502295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 msgr2=0x7f46680c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:13.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f46680c2080 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4660009f20 tx=0x7f4660009840 comp rx=0 tx=0).stop 2023-10-06T11:34:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 -- 172.21.15.46:0/1355502295 shutdown_connections 2023-10-06T11:34:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f46680c2080 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 --2- 172.21.15.46:0/1355502295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f46680ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 -- 172.21.15.46:0/1355502295 >> 172.21.15.46:0/1355502295 conn(0x7f46680cbe30 msgr2=0x7f46680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 -- 172.21.15.46:0/1355502295 shutdown_connections 2023-10-06T11:34:13.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.591+0000 7f466e0f0700 1 -- 172.21.15.46:0/1355502295 wait complete. 2023-10-06T11:34:13.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.595+0000 7f466e0f0700 1 Processor -- start 2023-10-06T11:34:13.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.595+0000 7f466e0f0700 1 -- start start 2023-10-06T11:34:13.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 0x7f466813d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f4668157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680a9d70 con 0x7f466806ed10 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668140340 con 0x7f466806e8a0 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680c9ba0 con 0x7f46680c6470 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59358/0 (socket says 172.21.15.46:59358) 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 -- 172.21.15.46:0/2277072961 learned_addr learned my addr 172.21.15.46:0/2277072961 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f46677fe700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 0x7f466813d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4667fff700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f4668157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 -- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 msgr2=0x7f4668157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:13.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f4668157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 -- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 msgr2=0x7f466813d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 0x7f466813d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 -- 172.21.15.46:0/2277072961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660009ae0 con 0x7f466806ed10 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f46677fe700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 0x7f466813d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f4666ffd700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c4e0 tx=0x7f465c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011590 con 0x7f466806ed10 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c011710 con 0x7f466806ed10 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c011ad0 con 0x7f466806ed10 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46680c9ee0 con 0x7f466806ed10 2023-10-06T11:34:13.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.599+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680a9a00 con 0x7f466806ed10 2023-10-06T11:34:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.603+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46680ca5e0 con 0x7f466806ed10 2023-10-06T11:34:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.603+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 39) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f465c01b3b0 con 0x7f466806ed10 2023-10-06T11:34:13.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.603+0000 7f464f7fe700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 0x7f46540628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:13.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.607+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f465c07bfb0 con 0x7f466806ed10 2023-10-06T11:34:13.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.607+0000 7f46677fe700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 0x7f46540628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:13.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.607+0000 7f46677fe700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 0x7f46540628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4658006f00 tx=0x7f4658008040 comp rx=0 tx=0).ready entity=mgr.7020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.611+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f465c04c310 con 0x7f466806ed10 2023-10-06T11:34:13.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:13.823+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f46680660f0 con 0x7f466806ed10 2023-10-06T11:34:14.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.359+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f465c04c6e0 con 0x7f466806ed10 2023-10-06T11:34:14.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.363+0000 7f464f7fe700 1 -- 172.21.15.46:0/2277072961 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v40) v1 ==== 93+0+0 (secure 0 0 0) 0x7f465c04fda0 con 0x7f466806ed10 2023-10-06T11:34:14.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 msgr2=0x7f46540628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 0x7f46540628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4658006f00 tx=0x7f4658008040 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 msgr2=0x7f4668157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c4e0 tx=0x7f465c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 shutdown_connections 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f46540603f0 0x7f46540628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46680c6470 0x7f4668157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466806ed10 0x7f4668157540 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 --2- 172.21.15.46:0/2277072961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466806e8a0 0x7f466813d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 >> 172.21.15.46:0/2277072961 conn(0x7f46680cbe30 msgr2=0x7f4668158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 shutdown_connections 2023-10-06T11:34:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.367+0000 7f466e0f0700 1 -- 172.21.15.46:0/2277072961 wait complete. 2023-10-06T11:34:14.396 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:34:14.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:34:14.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:34:14.473 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:34:14.473 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:34:14.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:34:14.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:14.551 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:14.549+0000 7f771b15f000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:14.574 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:34:14.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:14.574 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:34:14.575 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:14.563+0000 7f5f36b68000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:14.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 Processor -- start 2023-10-06T11:34:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 -- start start 2023-10-06T11:34:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f805c0d0aa0 0x7f805c0b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805c0b4f80 con 0x7f805c0b39b0 2023-10-06T11:34:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c0b4370 con 0x7f805c0d0aa0 2023-10-06T11:34:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8062745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0b4650 con 0x7f805c159050 2023-10-06T11:34:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8060ce2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f805c0b39b0 0x7f805c0b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51552/0 (socket says 172.21.15.46:51552) 2023-10-06T11:34:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f8060ce2700 1 -- 172.21.15.46:0/4244153518 learned_addr learned my addr 172.21.15.46:0/4244153518 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f805bfff700 1 --2- 172.21.15.46:0/4244153518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f805b7fe700 1 --2- 172.21.15.46:0/4244153518 >> v2:172.21.15.132:3300/0 conn(0x7f805c0d0aa0 0x7f805c0b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.783+0000 7f805affd700 1 -- 172.21.15.46:0/4244153518 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043423379 0 0) 0x7f805c0b4f80 con 0x7f805c0b39b0 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805affd700 1 -- 172.21.15.46:0/4244153518 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8044003570 con 0x7f805c0b39b0 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805bfff700 1 -- 172.21.15.46:0/4244153518 >> v2:172.21.15.132:3300/0 conn(0x7f805c0d0aa0 msgr2=0x7f805c0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805bfff700 1 --2- 172.21.15.46:0/4244153518 >> v2:172.21.15.132:3300/0 conn(0x7f805c0d0aa0 0x7f805c0b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805bfff700 1 -- 172.21.15.46:0/4244153518 >> v1:172.21.15.46:6789/0 conn(0x7f805c0b39b0 legacy=0x7f805c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805bfff700 1 -- 172.21.15.46:0/4244153518 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0b4930 con 0x7f805c159050 2023-10-06T11:34:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805bfff700 1 --2- 172.21.15.46:0/4244153518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c159420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098e0 tx=0x7f804c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff67f54e06cc3ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805affd700 1 -- 172.21.15.46:0/4244153518 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c011630 con 0x7f805c159050 2023-10-06T11:34:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805affd700 1 -- 172.21.15.46:0/4244153518 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c011790 con 0x7f805c159050 2023-10-06T11:34:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f805affd700 1 -- 172.21.15.46:0/4244153518 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c011b30 con 0x7f805c159050 2023-10-06T11:34:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 -- 172.21.15.46:0/4244153518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 msgr2=0x7f805c159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 --2- 172.21.15.46:0/4244153518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c159420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098e0 tx=0x7f804c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 -- 172.21.15.46:0/4244153518 shutdown_connections 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 --2- 172.21.15.46:0/4244153518 >> v2:172.21.15.132:3300/0 conn(0x7f805c0d0aa0 0x7f805c0b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 --2- 172.21.15.46:0/4244153518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c159420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 -- 172.21.15.46:0/4244153518 >> 172.21.15.46:0/4244153518 conn(0x7f805c06e780 msgr2=0x7f805c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 -- 172.21.15.46:0/4244153518 shutdown_connections 2023-10-06T11:34:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.787+0000 7f8062745700 1 -- 172.21.15.46:0/4244153518 wait complete. 2023-10-06T11:34:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 Processor -- start 2023-10-06T11:34:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 -- start start 2023-10-06T11:34:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f805c0d0aa0 0x7f805c0c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c156840 con 0x7f805c0b39b0 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c15a2f0 con 0x7f805c0d0aa0 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8062745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0c5450 con 0x7f805c159050 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59386/0 (socket says 172.21.15.46:59386) 2023-10-06T11:34:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 -- 172.21.15.46:0/2414494258 learned_addr learned my addr 172.21.15.46:0/2414494258 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805b7fe700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f805c0d0aa0 0x7f805c0c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f8060ce2700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 msgr2=0x7f805c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f805c0d0aa0 msgr2=0x7f805c0c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f805c0d0aa0 0x7f805c0c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.791+0000 7f805bfff700 1 -- 172.21.15.46:0/2414494258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c009420 con 0x7f805c0b39b0 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f805bfff700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f805c0b4790 tx=0x7f804c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c000d10 con 0x7f805c0b39b0 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c011b10 con 0x7f805c0b39b0 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c1599c0 con 0x7f805c0b39b0 2023-10-06T11:34:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c025660 con 0x7f805c0b39b0 2023-10-06T11:34:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c159f80 con 0x7f805c0b39b0 2023-10-06T11:34:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.795+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c088820 con 0x7f805c0b39b0 2023-10-06T11:34:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.799+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f804c025a10 con 0x7f805c0b39b0 2023-10-06T11:34:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.799+0000 7f8043fff700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 0x7f8048062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.799+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f804c07e680 con 0x7f805c0b39b0 2023-10-06T11:34:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.799+0000 7f805b7fe700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 msgr2=0x7f8048062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.799+0000 7f805b7fe700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 0x7f8048062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:14.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:14.807+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f804c04e580 con 0x7f805c0b39b0 2023-10-06T11:34:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.003+0000 7f805b7fe700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 msgr2=0x7f8048062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.003+0000 7f805b7fe700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 0x7f8048062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.015+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f805c0b4c10 con 0x7f805c0b39b0 2023-10-06T11:34:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.015+0000 7f8043fff700 1 -- 172.21.15.46:0/2414494258 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f804c051ba0 con 0x7f805c0b39b0 2023-10-06T11:34:15.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:15.028 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 msgr2=0x7f8048062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 0x7f8048062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 msgr2=0x7f805c156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f805c0b4790 tx=0x7f804c017000 comp rx=0 tx=0).stop 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 shutdown_connections 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8048060700 0x7f8048062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f805c159050 0x7f805c0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f805c0d0aa0 0x7f805c0c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 --2- 172.21.15.46:0/2414494258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c0b39b0 0x7f805c156300 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 >> 172.21.15.46:0/2414494258 conn(0x7f805c06e780 msgr2=0x7f805c158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 shutdown_connections 2023-10-06T11:34:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.023+0000 7f8062745700 1 -- 172.21.15.46:0/2414494258 wait complete. 2023-10-06T11:34:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:15.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:15.089 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.085+0000 7f771b15f000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:15.146 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.135+0000 7f5f36b68000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:15.159 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.157+0000 7f771b15f000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 Processor -- start 2023-10-06T11:34:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 -- start start 2023-10-06T11:34:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 0x7fe9f4159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9f40b5410 0x7fe9f40b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f40b4f80 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f40b40c0 con 0x7fe9f40b5410 2023-10-06T11:34:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.207+0000 7fe9fc678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40b4370 con 0x7fe9f4159050 2023-10-06T11:34:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9c13700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe9f40d0aa0 0x7fe9f40b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42398/0 (socket says 172.21.15.46:42398) 2023-10-06T11:34:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9c13700 1 -- 172.21.15.46:0/3441518905 learned_addr learned my addr 172.21.15.46:0/3441518905 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 --2- 172.21.15.46:0/3441518905 >> v2:172.21.15.132:3300/0 conn(0x7fe9f40b5410 0x7fe9f40b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fa414700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 0x7fe9f4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9412700 1 -- 172.21.15.46:0/3441518905 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902325187 0 0) 0x7fe9f40b4f80 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9412700 1 -- 172.21.15.46:0/3441518905 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e0003570 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 -- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 msgr2=0x7fe9f4159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 0x7fe9f4159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 -- 172.21.15.46:0/3441518905 >> v1:172.21.15.132:6789/0 conn(0x7fe9f40d0aa0 legacy=0x7fe9f40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 -- 172.21.15.46:0/3441518905 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40b4650 con 0x7fe9f40b5410 2023-10-06T11:34:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fa414700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 0x7fe9f4159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fac15700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f40b5410 0x7fe9f40b1fc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009e60 tx=0x7fe9ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9cce2ba2d6996 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9412700 1 -- 172.21.15.46:0/3441518905 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00e070 con 0x7fe9f40b5410 2023-10-06T11:34:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9412700 1 -- 172.21.15.46:0/3441518905 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00bab0 con 0x7fe9f40b5410 2023-10-06T11:34:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9f9412700 1 -- 172.21.15.46:0/3441518905 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00be70 con 0x7fe9f40b5410 2023-10-06T11:34:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 -- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f40b5410 msgr2=0x7fe9f40b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f40b5410 0x7fe9f40b1fc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009e60 tx=0x7fe9ec009780 comp rx=0 tx=0).stop 2023-10-06T11:34:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 -- 172.21.15.46:0/3441518905 shutdown_connections 2023-10-06T11:34:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f40b5410 0x7fe9f40b1fc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 --2- 172.21.15.46:0/3441518905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f4159050 0x7fe9f4159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 -- 172.21.15.46:0/3441518905 >> 172.21.15.46:0/3441518905 conn(0x7fe9f406e780 msgr2=0x7fe9f406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 -- 172.21.15.46:0/3441518905 shutdown_connections 2023-10-06T11:34:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.211+0000 7fe9fc678700 1 -- 172.21.15.46:0/3441518905 wait complete. 2023-10-06T11:34:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.215+0000 7fe9fc678700 1 Processor -- start 2023-10-06T11:34:15.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.215+0000 7fe9fc678700 1 -- start start 2023-10-06T11:34:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.215+0000 7fe9fc678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f40b5410 0x7fe9f41599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.215+0000 7fe9fc678700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f4159050 0x7fe9f40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c7380 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c7660 con 0x7fe9f4159050 2023-10-06T11:34:15.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c7940 con 0x7fe9f40b5410 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59400/0 (socket says 172.21.15.46:59400) 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fa414700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f40b5410 0x7fe9f41599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 -- 172.21.15.46:0/1687937155 learned_addr learned my addr 172.21.15.46:0/1687937155 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fac15700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f4159050 0x7fe9f40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 -- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f40b5410 msgr2=0x7fe9f41599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f40b5410 0x7fe9f41599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 -- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f4159050 msgr2=0x7fe9f40c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f4159050 0x7fe9f40c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 -- 172.21.15.46:0/1687937155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009420 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9f9c13700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c7c0 tx=0x7fe9e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:15.230 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.219+0000 7f5f36b68000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4005cd0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4005e30 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4007320 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec009a20 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.219+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f41554d0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f4088820 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fe9e401b3b0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9ea7fc700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 0x7fe9d8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fe9e407c640 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9fa414700 1 -- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 msgr2=0x7fe9d8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.223+0000 7fe9fa414700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 0x7fe9d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:15.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.231+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe9e404c4a0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.290 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.285+0000 7f771b15f000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:15.378 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.367+0000 7f5f36b68000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:15.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.403+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9f40c74c0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.403+0000 7fe9ea7fc700 1 -- 172.21.15.46:0/1687937155 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18029 (secure 0 0 0) 0x7fe9f40c74c0 con 0x7fe9f40d0aa0 2023-10-06T11:34:15.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:15.415 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:12.318870+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/730464998":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6832/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/921761638":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1451694921":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1040714298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3304341298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3114837258":"2023-10-07T11:34:12.318801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.407+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 msgr2=0x7fe9d8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.407+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 0x7fe9d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.407+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 msgr2=0x7fe9f4154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.407+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400c7c0 tx=0x7fe9e4007e00 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 shutdown_connections 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe9d8060700 0x7fe9d8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9f4159050 0x7fe9f40c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f40d0aa0 0x7fe9f4154f90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 --2- 172.21.15.46:0/1687937155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9f40b5410 0x7fe9f41599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 >> 172.21.15.46:0/1687937155 conn(0x7fe9f406e780 msgr2=0x7fe9f40cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 shutdown_connections 2023-10-06T11:34:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.411+0000 7fe9fc678700 1 -- 172.21.15.46:0/1687937155 wait complete. 2023-10-06T11:34:15.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:33:55.950284+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:33:56.602195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:34:15.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:34:15.548 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.545+0000 7f771b15f000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 Processor -- start 2023-10-06T11:34:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 -- start start 2023-10-06T11:34:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc81c0b23c0 0x7fc81c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 0x7fc81c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0cc510 con 0x7fc81c0cae40 2023-10-06T11:34:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0cb900 con 0x7fc81c0b23c0 2023-10-06T11:34:15.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc8235c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0cbbe0 con 0x7fc81c0cd250 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc81c0b23c0 0x7fc81c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc81c0b23c0 0x7fc81c0b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59410/0 (socket says 172.21.15.46:59410) 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 -- 172.21.15.46:0/1104584443 learned_addr learned my addr 172.21.15.46:0/1104584443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821b62700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 0x7fc81c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 -- 172.21.15.46:0/1104584443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 msgr2=0x7fc81c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 0x7fc81c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 -- 172.21.15.46:0/1104584443 >> v1:172.21.15.132:6789/0 conn(0x7fc81c0cae40 legacy=0x7fc81c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 -- 172.21.15.46:0/1104584443 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0cbec0 con 0x7fc81c0b23c0 2023-10-06T11:34:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821b62700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 0x7fc81c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.599+0000 7fc821361700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0b23c0 0x7fc81c0b2790 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc818009990 tx=0x7fc818009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997331fb4de28e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc813fff700 1 -- 172.21.15.46:0/1104584443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818005c10 con 0x7fc81c0b23c0 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc813fff700 1 -- 172.21.15.46:0/1104584443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818005d70 con 0x7fc81c0b23c0 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc813fff700 1 -- 172.21.15.46:0/1104584443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800b840 con 0x7fc81c0b23c0 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 -- 172.21.15.46:0/1104584443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0b23c0 msgr2=0x7fc81c0b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0b23c0 0x7fc81c0b2790 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc818009990 tx=0x7fc818009220 comp rx=0 tx=0).stop 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 -- 172.21.15.46:0/1104584443 shutdown_connections 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0cd250 0x7fc81c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 --2- 172.21.15.46:0/1104584443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0b23c0 0x7fc81c0b2790 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 -- 172.21.15.46:0/1104584443 >> 172.21.15.46:0/1104584443 conn(0x7fc81c099480 msgr2=0x7fc81c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 -- 172.21.15.46:0/1104584443 shutdown_connections 2023-10-06T11:34:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.603+0000 7fc8235c5700 1 -- 172.21.15.46:0/1104584443 wait complete. 2023-10-06T11:34:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 Processor -- start 2023-10-06T11:34:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 -- start start 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0b23c0 0x7fc81c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81c0cae40 0x7fc81c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c156010 con 0x7fc81c0cd250 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c159d20 con 0x7fc81c0cae40 2023-10-06T11:34:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc8235c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1599f0 con 0x7fc81c0b23c0 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59418/0 (socket says 172.21.15.46:59418) 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 -- 172.21.15.46:0/2564226775 learned_addr learned my addr 172.21.15.46:0/2564226775 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821361700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0b23c0 0x7fc81c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc820b60700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81c0cae40 0x7fc81c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 -- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0b23c0 msgr2=0x7fc81c0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0b23c0 0x7fc81c0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 -- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81c0cae40 msgr2=0x7fc81c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81c0cae40 0x7fc81c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.607+0000 7fc821b62700 1 -- 172.21.15.46:0/2564226775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009480 con 0x7fc81c0cd250 2023-10-06T11:34:15.621 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.617+0000 7f771b15f000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:15.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc821b62700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cfc0 tx=0x7fc8140078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814015440 con 0x7fc81c0cd250 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140155a0 con 0x7fc81c0cd250 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c16d4a0 con 0x7fc81c0cd250 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814015960 con 0x7fc81c0cd250 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c16da20 con 0x7fc81c0cd250 2023-10-06T11:34:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c0660f0 con 0x7fc81c0cd250 2023-10-06T11:34:15.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.611+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc81400d440 con 0x7fc81c0cd250 2023-10-06T11:34:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.615+0000 7fc8117fa700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 0x7fc800062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.615+0000 7fc821361700 1 -- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 msgr2=0x7fc800062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:15.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.615+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc81407db70 con 0x7fc81c0cd250 2023-10-06T11:34:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.615+0000 7fc821361700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 0x7fc800062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:15.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.623+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc81404d9d0 con 0x7fc81c0cd250 2023-10-06T11:34:15.671 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.663+0000 7f5f36b68000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:15.746 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.735+0000 7f5f36b68000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:15.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.795+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc81c159e60 con 0x7fc81c0cd250 2023-10-06T11:34:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.795+0000 7fc8117fa700 1 -- 172.21.15.46:0/2564226775 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc81401a0a0 con 0x7fc81c0cd250 2023-10-06T11:34:15.806 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:34:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 msgr2=0x7fc800062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 0x7fc800062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 msgr2=0x7fc81c16cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cfc0 tx=0x7fc8140078d0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 shutdown_connections 2023-10-06T11:34:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc800060700 0x7fc800062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc81c0cd250 0x7fc81c16cf60 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc81c0cae40 0x7fc81c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 --2- 172.21.15.46:0/2564226775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc81c0b23c0 0x7fc81c0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:15.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 >> 172.21.15.46:0/2564226775 conn(0x7fc81c099480 msgr2=0x7fc81c157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:15.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 shutdown_connections 2023-10-06T11:34:15.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.799+0000 7fc8235c5700 1 -- 172.21.15.46:0/2564226775 wait complete. 2023-10-06T11:34:15.827 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.825+0000 7f771b15f000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:15.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:34:15.907 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.905+0000 7f771b15f000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:15.977 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:15.967+0000 7f5f36b68000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:15.988 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:15.985+0000 7f771b15f000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:15.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.987+0000 7fb3f484c700 1 Processor -- start 2023-10-06T11:34:15.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.987+0000 7fb3f484c700 1 -- start start 2023-10-06T11:34:15.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.987+0000 7fb3f484c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:15.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3f00c9fc0 0x7fb3f00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 0x7fb3f00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00c0d60 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f00cadb0 con 0x7fb3f00ca950 2023-10-06T11:34:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f006e0d0 con 0x7fb3f00c6250 2023-10-06T11:34:16.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3f00c9fc0 0x7fb3f00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3f00c9fc0 0x7fb3f00ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59430/0 (socket says 172.21.15.46:59430) 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 -- 172.21.15.46:0/3384039439 learned_addr learned my addr 172.21.15.46:0/3384039439 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3effff700 1 --2- 172.21.15.46:0/3384039439 >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 0x7fb3f00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3ef7fe700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 -- 172.21.15.46:0/3384039439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 msgr2=0x7fb3f00c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f00c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 -- 172.21.15.46:0/3384039439 >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 msgr2=0x7fb3f00c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/3384039439 >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 0x7fb3f00c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 -- 172.21.15.46:0/3384039439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f006e3b0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3ef7fe700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3effff700 1 --2- 172.21.15.46:0/3384039439 >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 0x7fb3f00c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f00ca410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00c1030 tx=0x7fb3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8df9f4cbf2aec06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3ee7fc700 1 -- 172.21.15.46:0/3384039439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400b860 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3ee7fc700 1 -- 172.21.15.46:0/3384039439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e400b9c0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3ee7fc700 1 -- 172.21.15.46:0/3384039439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400bd60 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- 172.21.15.46:0/3384039439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 msgr2=0x7fb3f00ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f00ca410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00c1030 tx=0x7fb3e4009250 comp rx=0 tx=0).stop 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- 172.21.15.46:0/3384039439 shutdown_connections 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- 172.21.15.46:0/3384039439 >> v2:172.21.15.132:3300/0 conn(0x7fb3f00ca950 0x7fb3f00c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f00ca410 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 --2- 172.21.15.46:0/3384039439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- 172.21.15.46:0/3384039439 >> 172.21.15.46:0/3384039439 conn(0x7fb3f00cbc10 msgr2=0x7fb3f00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- 172.21.15.46:0/3384039439 shutdown_connections 2023-10-06T11:34:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.991+0000 7fb3f484c700 1 -- 172.21.15.46:0/3384039439 wait complete. 2023-10-06T11:34:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.995+0000 7fb3f484c700 1 Processor -- start 2023-10-06T11:34:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.995+0000 7fb3f484c700 1 -- start start 2023-10-06T11:34:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f0167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3f00ca950 0x7fb3f014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00aed90 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00af070 con 0x7fb3f00ca950 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f00af350 con 0x7fb3f00c6250 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59438/0 (socket says 172.21.15.46:59438) 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 -- 172.21.15.46:0/627526210 learned_addr learned my addr 172.21.15.46:0/627526210 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3effff700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3f00ca950 0x7fb3f014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3ef7fe700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f0167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 -- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 msgr2=0x7fb3f0167d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f0167d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 -- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3f00ca950 msgr2=0x7fb3f014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3f00ca950 0x7fb3f014f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 -- 172.21.15.46:0/627526210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40094f0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3ef7fe700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f0167d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3eeffd700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009290 tx=0x7fb3e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4026480 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e401f070 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f00af630 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4026830 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:15.999+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f00ae9b0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0064590 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fb3e4026ba0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3d77fe700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 0x7fb3dc062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fb3e4021070 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3ef7fe700 1 -- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 msgr2=0x7fb3dc062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.003+0000 7fb3ef7fe700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 0x7fb3dc062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.011+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb3e404ed10 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.070 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:16.059+0000 7f5f36b68000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:16.153 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:16.143+0000 7f5f36b68000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:16.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.187+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb3f006e690 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.187+0000 7fb3d77fe700 1 -- 172.21.15.46:0/627526210 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb3e40150a0 con 0x7fb3f00c9fc0 2023-10-06T11:34:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 msgr2=0x7fb3dc062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 0x7fb3dc062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 msgr2=0x7fb3f01499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009290 tx=0x7fb3e4017000 comp rx=0 tx=0).stop 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 shutdown_connections 2023-10-06T11:34:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb3dc060700 0x7fb3dc062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3f00ca950 0x7fb3f014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3f00c9fc0 0x7fb3f01499e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 --2- 172.21.15.46:0/627526210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3f00c6250 0x7fb3f0167d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 >> 172.21.15.46:0/627526210 conn(0x7fb3f00cbc10 msgr2=0x7fb3f00c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 shutdown_connections 2023-10-06T11:34:16.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.191+0000 7fb3f484c700 1 -- 172.21.15.46:0/627526210 wait complete. 2023-10-06T11:34:16.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:34:16.357 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:16.353+0000 7f771b15f000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 Processor -- start 2023-10-06T11:34:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 -- start start 2023-10-06T11:34:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 0x7fdfb40ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb40a6e20 0x7fdfb40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfb4096140 0x7fdfb4096590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb4068270 con 0x7fdfb4096140 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb40cc7d0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.371+0000 7fdfb928e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb40684c0 con 0x7fdfb40a6e20 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfb4096140 0x7fdfb4096590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfb4096140 0x7fdfb4096590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59446/0 (socket says 172.21.15.46:59446) 2023-10-06T11:34:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3753358483 learned_addr learned my addr 172.21.15.46:0/3753358483 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb37fe700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb40a6e20 0x7fdfb40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb3fff700 1 --2- 172.21.15.46:0/3753358483 >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 0x7fdfb40ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3753358483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb40a6e20 msgr2=0x7fdfb40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb40a6e20 0x7fdfb40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3753358483 >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 msgr2=0x7fdfb40ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3753358483 >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 0x7fdfb40ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3753358483 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb40687a0 con 0x7fdfb4096140 2023-10-06T11:34:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb3fff700 1 --2- 172.21.15.46:0/3753358483 >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 0x7fdfb40ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb4096140 0x7fdfb4096590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4069060 tx=0x7fdfac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374c08c46477d252 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb2ffd700 1 -- 172.21.15.46:0/3753358483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac010070 con 0x7fdfb4096140 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb2ffd700 1 -- 172.21.15.46:0/3753358483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac00dde0 con 0x7fdfb4096140 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb2ffd700 1 -- 172.21.15.46:0/3753358483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00b320 con 0x7fdfb4096140 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 -- 172.21.15.46:0/3753358483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb4096140 msgr2=0x7fdfb4096590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb4096140 0x7fdfb4096590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4069060 tx=0x7fdfac00ba50 comp rx=0 tx=0).stop 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 -- 172.21.15.46:0/3753358483 shutdown_connections 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb4096140 0x7fdfb4096590 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3753358483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb40a6e20 0x7fdfb40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3753358483 >> v2:172.21.15.132:3300/0 conn(0x7fdfb40ac2d0 0x7fdfb40ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 -- 172.21.15.46:0/3753358483 >> 172.21.15.46:0/3753358483 conn(0x7fdfb40afd10 msgr2=0x7fdfb406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 -- 172.21.15.46:0/3753358483 shutdown_connections 2023-10-06T11:34:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.375+0000 7fdfb928e700 1 -- 172.21.15.46:0/3753358483 wait complete. 2023-10-06T11:34:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 Processor -- start 2023-10-06T11:34:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 -- start start 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 0x7fdfb4157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 0x7fdfb40957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb4157f00 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb41581e0 con 0x7fdfb40a6e20 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb928e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb41584c0 con 0x7fdfb4096140 2023-10-06T11:34:16.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59458/0 (socket says 172.21.15.46:59458) 2023-10-06T11:34:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3326489970 learned_addr learned my addr 172.21.15.46:0/3326489970 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb3fff700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 0x7fdfb4157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb37fe700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 0x7fdfb40957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 msgr2=0x7fdfb4157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 0x7fdfb4157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 msgr2=0x7fdfb40957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 0x7fdfb40957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 -- 172.21.15.46:0/3326489970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac00bcf0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb3fff700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 0x7fdfb4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:16.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb37fe700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 0x7fdfb40957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.379+0000 7fdfb8a8d700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4070bf0 tx=0x7fdfac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00ab90 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac00ad30 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb41587a0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb4157bd0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00aea0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb40689e0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.383+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fdfac003040 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.387+0000 7fdf9bfff700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 0x7fdfa0062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.387+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fdfac07ece0 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.387+0000 7fdfb3fff700 1 -- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 msgr2=0x7fdfa0062b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.387+0000 7fdfb3fff700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 0x7fdfa0062b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:16.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.395+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdfac04ea30 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.525 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:16.513+0000 7f771b15f000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:16.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.571+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfb4158c80 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.571+0000 7fdf9bfff700 1 -- 172.21.15.46:0/3326489970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfac052050 con 0x7fdfb40ac2d0 2023-10-06T11:34:16.583 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:34:16.585 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:16.575+0000 7f5f36b68000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 msgr2=0x7fdfa0062b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 0x7fdfa0062b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 msgr2=0x7fdfb4158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4070bf0 tx=0x7fdfac01a000 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 shutdown_connections 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdfa00606d0 0x7fdfa0062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfb40ac2d0 0x7fdfb4158fa0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb40a6e20 0x7fdfb40957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 --2- 172.21.15.46:0/3326489970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb4096140 0x7fdfb4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.575+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 >> 172.21.15.46:0/3326489970 conn(0x7fdfb40afd10 msgr2=0x7fdfb40ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.579+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 shutdown_connections 2023-10-06T11:34:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.579+0000 7fdfb928e700 1 -- 172.21.15.46:0/3326489970 wait complete. 2023-10-06T11:34:16.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:34:16.755 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:16.743+0000 7f5f36b68000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.771+0000 7f1fc346b700 1 Processor -- start 2023-10-06T11:34:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.771+0000 7f1fc346b700 1 -- start start 2023-10-06T11:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc346b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1fbc0b30f0 0x7f1fbc0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc346b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc0afd20 con 0x7f1fbc0adbe0 2023-10-06T11:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc346b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc0b0000 con 0x7f1fbc0b3570 2023-10-06T11:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc346b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc159740 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc2469700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1fbc0adbe0 0x7f1fbc0d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2023-10-06T11:34:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc2469700 1 -- 172.21.15.46:0/376404712 learned_addr learned my addr 172.21.15.46:0/376404712 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1c68700 1 --2- 172.21.15.46:0/376404712 >> v2:172.21.15.132:3300/0 conn(0x7f1fbc0b30f0 0x7f1fbc0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508113998 0 0) 0x7f1fbc0afd20 con 0x7f1fbc0adbe0 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa8003570 con 0x7f1fbc0adbe0 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1c68700 1 -- 172.21.15.46:0/376404712 >> v1:172.21.15.132:6789/0 conn(0x7f1fbc0b3570 legacy=0x7f1fbc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272839110 0 0) 0x7f1fa8003570 con 0x7f1fbc0adbe0 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1c68700 1 -- 172.21.15.46:0/376404712 >> v1:172.21.15.46:6789/0 conn(0x7f1fbc0adbe0 legacy=0x7f1fbc0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1c68700 1 -- 172.21.15.46:0/376404712 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc0b02e0 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1c68700 1 --2- 172.21.15.46:0/376404712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc0b0dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0098e0 tx=0x7f1fac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbc9a53897d90d5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8001470 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac011420 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.775+0000 7f1fc1467700 1 -- 172.21.15.46:0/376404712 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0117c0 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- 172.21.15.46:0/376404712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0b30f0 msgr2=0x7f1fbc0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 --2- 172.21.15.46:0/376404712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc0b0dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0098e0 tx=0x7f1fac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- 172.21.15.46:0/376404712 shutdown_connections 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 --2- 172.21.15.46:0/376404712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc0b0dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- 172.21.15.46:0/376404712 >> 172.21.15.46:0/376404712 conn(0x7f1fbc06ef60 msgr2=0x7f1fbc0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- 172.21.15.46:0/376404712 shutdown_connections 2023-10-06T11:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- 172.21.15.46:0/376404712 wait complete. 2023-10-06T11:34:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 Processor -- start 2023-10-06T11:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.779+0000 7f1fc346b700 1 -- start start 2023-10-06T11:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0adbe0 0x7f1fbc154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc0b3570 0x7f1fbc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc15b030 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc09b120 con 0x7f1fbc0adbe0 2023-10-06T11:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc09adf0 con 0x7f1fbc0b3570 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59460/0 (socket says 172.21.15.46:59460) 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 -- 172.21.15.46:0/280108044 learned_addr learned my addr 172.21.15.46:0/280108044 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc2c6a700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc0b3570 0x7f1fbc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc2469700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0adbe0 0x7f1fbc154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 -- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc0b3570 msgr2=0x7f1fbc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc0b3570 0x7f1fbc0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 -- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0adbe0 msgr2=0x7f1fbc154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0adbe0 0x7f1fbc154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 -- 172.21.15.46:0/280108044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009420 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc1c68700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0091e0 tx=0x7f1fac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac01cb30 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac01cc90 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac01a6a0 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc15a790 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.783+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc15ad50 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.787+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc064590 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.787+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f1fac01aa90 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.791+0000 7f1fb27fc700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 0x7f1fa0062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.791+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f1fac07e8f0 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.791+0000 7f1fc2469700 1 -- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 msgr2=0x7f1fa0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.791+0000 7f1fc2469700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 0x7f1fa0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.795+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1fac04e750 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.857 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:16.853+0000 7f771b15f000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.959+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fbc0675d0 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fb27fc700 1 -- 172.21.15.46:0/280108044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fac051d70 con 0x7f1fbc0b30f0 2023-10-06T11:34:16.972 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:34:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 msgr2=0x7f1fa0062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 0x7f1fa0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 msgr2=0x7f1fbc159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0091e0 tx=0x7f1fac016000 comp rx=0 tx=0).stop 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 shutdown_connections 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1fa0060700 0x7f1fa0062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fbc0b3570 0x7f1fbc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.963+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc0b30f0 0x7f1fbc159d80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.967+0000 7f1fc346b700 1 --2- 172.21.15.46:0/280108044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fbc0adbe0 0x7f1fbc154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.967+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 >> 172.21.15.46:0/280108044 conn(0x7f1fbc06ef60 msgr2=0x7f1fbc157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.967+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 shutdown_connections 2023-10-06T11:34:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:16.967+0000 7f1fc346b700 1 -- 172.21.15.46:0/280108044 wait complete. 2023-10-06T11:34:16.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:17.069 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.065+0000 7f771b15f000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:17.150 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.139+0000 7f5f36b68000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.143+0000 7fc677fff700 1 Processor -- start 2023-10-06T11:34:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.143+0000 7fc677fff700 1 -- start start 2023-10-06T11:34:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc677fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc678159290 0x7fc6780b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc677fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc6780b1300 0x7fc6780ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc677fff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6780ae340 con 0x7fc6780d07b0 2023-10-06T11:34:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc677fff700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780b09a0 con 0x7fc678159290 2023-10-06T11:34:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc677fff700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc678159660 con 0x7fc6780b1300 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc6767fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6780d07b0 0x7fc6780b0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51564/0 (socket says 172.21.15.46:51564) 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc6767fc700 1 -- 172.21.15.46:0/1889784895 learned_addr learned my addr 172.21.15.46:0/1889784895 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 --2- 172.21.15.46:0/1889784895 >> v2:172.21.15.46:3300/0 conn(0x7fc678159290 0x7fc6780b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc6777fe700 1 --2- 172.21.15.46:0/1889784895 >> v2:172.21.15.132:3300/0 conn(0x7fc6780b1300 0x7fc6780ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 -- 172.21.15.46:0/1889784895 >> v2:172.21.15.132:3300/0 conn(0x7fc6780b1300 msgr2=0x7fc6780ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 --2- 172.21.15.46:0/1889784895 >> v2:172.21.15.132:3300/0 conn(0x7fc6780b1300 0x7fc6780ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 -- 172.21.15.46:0/1889784895 >> v1:172.21.15.46:6789/0 conn(0x7fc6780d07b0 legacy=0x7fc6780b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 -- 172.21.15.46:0/1889784895 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780b3450 con 0x7fc678159290 2023-10-06T11:34:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc675ffb700 1 -- 172.21.15.46:0/1889784895 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46548799 0 0) 0x7fc6780ae340 con 0x7fc6780d07b0 2023-10-06T11:34:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc676ffd700 1 --2- 172.21.15.46:0/1889784895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc678159290 0x7fc6780b2f10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc678088f90 tx=0x7fc6680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b02cb261ce0309 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc675ffb700 1 -- 172.21.15.46:0/1889784895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800b690 con 0x7fc678159290 2023-10-06T11:34:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc675ffb700 1 -- 172.21.15.46:0/1889784895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66800b7f0 con 0x7fc678159290 2023-10-06T11:34:17.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.147+0000 7fc675ffb700 1 -- 172.21.15.46:0/1889784895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800bb90 con 0x7fc678159290 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- 172.21.15.46:0/1889784895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc678159290 msgr2=0x7fc6780b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 --2- 172.21.15.46:0/1889784895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc678159290 0x7fc6780b2f10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc678088f90 tx=0x7fc6680091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- 172.21.15.46:0/1889784895 shutdown_connections 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 --2- 172.21.15.46:0/1889784895 >> v2:172.21.15.132:3300/0 conn(0x7fc6780b1300 0x7fc6780ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 --2- 172.21.15.46:0/1889784895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc678159290 0x7fc6780b2f10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- 172.21.15.46:0/1889784895 >> 172.21.15.46:0/1889784895 conn(0x7fc67806ef60 msgr2=0x7fc6780d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- 172.21.15.46:0/1889784895 shutdown_connections 2023-10-06T11:34:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- 172.21.15.46:0/1889784895 wait complete. 2023-10-06T11:34:17.161 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.157+0000 7f771b15f000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 Processor -- start 2023-10-06T11:34:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.151+0000 7fc677fff700 1 -- start start 2023-10-06T11:34:17.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6780b1300 0x7fc678169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc678159290 0x7fc678068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678151d80 con 0x7fc6780d07b0 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc678153690 con 0x7fc6780b1300 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67816cf50 con 0x7fc678159290 2023-10-06T11:34:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59482/0 (socket says 172.21.15.46:59482) 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 -- 172.21.15.46:0/3931914287 learned_addr learned my addr 172.21.15.46:0/3931914287 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc676ffd700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6780b1300 0x7fc678169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6777fe700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc678159290 0x7fc678068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc678159290 msgr2=0x7fc678068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc678159290 0x7fc678068fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6780b1300 msgr2=0x7fc678169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6780b1300 0x7fc678169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 -- 172.21.15.46:0/3931914287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009420 con 0x7fc6780d07b0 2023-10-06T11:34:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc6767fc700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c7e0 tx=0x7fc66c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c005b30 con 0x7fc6780d07b0 2023-10-06T11:34:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c005c90 con 0x7fc6780d07b0 2023-10-06T11:34:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67816d230 con 0x7fc6780d07b0 2023-10-06T11:34:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c007300 con 0x7fc6780d07b0 2023-10-06T11:34:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.155+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67816d7f0 con 0x7fc6780d07b0 2023-10-06T11:34:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc678064590 con 0x7fc6780d07b0 2023-10-06T11:34:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc66c01b3b0 con 0x7fc6780d07b0 2023-10-06T11:34:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc65effd700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 0x7fc664062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc66c07c4c0 con 0x7fc6780d07b0 2023-10-06T11:34:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc676ffd700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 msgr2=0x7fc664062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.159+0000 7fc676ffd700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 0x7fc664062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.167+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc66c04c320 con 0x7fc6780d07b0 2023-10-06T11:34:17.232 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.229+0000 7f771b15f000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:17.304 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.301+0000 7f771b15f000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.363+0000 7fc676ffd700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 msgr2=0x7fc664062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.363+0000 7fc676ffd700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 0x7fc664062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:17.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.375+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6780b06c0 con 0x7fc6780d07b0 2023-10-06T11:34:17.384 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.375+0000 7f5f36b68000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:17.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.375+0000 7fc65effd700 1 -- 172.21.15.46:0/3931914287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fc66c001d60 con 0x7fc6780d07b0 2023-10-06T11:34:17.385 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:17.385 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 msgr2=0x7fc664062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 0x7fc664062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 msgr2=0x7fc67814a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00c7e0 tx=0x7fc66c007df0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 shutdown_connections 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc664060700 0x7fc664062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc678159290 0x7fc678068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780d07b0 0x7fc67814a280 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 --2- 172.21.15.46:0/3931914287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc6780b1300 0x7fc678169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 >> 172.21.15.46:0/3931914287 conn(0x7fc67806ef60 msgr2=0x7fc6780b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:17.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 shutdown_connections 2023-10-06T11:34:17.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.379+0000 7fc677fff700 1 -- 172.21.15.46:0/3931914287 wait complete. 2023-10-06T11:34:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:17.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:17.481 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.471+0000 7f5f36b68000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:17.515 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.513+0000 7f771b15f000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:17.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:34:17.552 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:34:17.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.551+0000 7f0cea2e3700 1 Processor -- start 2023-10-06T11:34:17.562 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.551+0000 7f5f36b68000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:17.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.551+0000 7f0cea2e3700 1 -- start start 2023-10-06T11:34:17.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cea2e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 0x7f0ce40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cea2e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ce40af3d0 0x7f0ce40690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cea2e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40cf100 con 0x7f0ce40af000 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cea2e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40ce300 con 0x7f0ce40af3d0 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cea2e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40ce4f0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ce40af3d0 0x7f0ce40690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ce40af3d0 0x7f0ce40690b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59490/0 (socket says 172.21.15.46:59490) 2023-10-06T11:34:17.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/864832690 learned_addr learned my addr 172.21.15.46:0/864832690 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce8ae0700 1 --2- 172.21.15.46:0/864832690 >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 0x7f0ce40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/864832690 >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 msgr2=0x7f0ce40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 --2- 172.21.15.46:0/864832690 >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 0x7f0ce40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/864832690 >> v1:172.21.15.132:6789/0 conn(0x7f0ce40af000 legacy=0x7f0ce40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/864832690 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40ce7d0 con 0x7f0ce40af3d0 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce8ae0700 1 --2- 172.21.15.46:0/864832690 >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 0x7f0ce40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0ce9ae2700 1 --2- 172.21.15.46:0/864832690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40690b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4087fd0 tx=0x7f0cdc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22f9ee760182ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cdbfff700 1 -- 172.21.15.46:0/864832690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00e070 con 0x7f0ce40af3d0 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cdbfff700 1 -- 172.21.15.46:0/864832690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00ba10 con 0x7f0ce40af3d0 2023-10-06T11:34:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.555+0000 7f0cdbfff700 1 -- 172.21.15.46:0/864832690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0092e0 con 0x7f0ce40af3d0 2023-10-06T11:34:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- 172.21.15.46:0/864832690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40af3d0 msgr2=0x7f0ce40690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/864832690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40690b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4087fd0 tx=0x7f0cdc009840 comp rx=0 tx=0).stop 2023-10-06T11:34:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- 172.21.15.46:0/864832690 shutdown_connections 2023-10-06T11:34:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/864832690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40690b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/864832690 >> v2:172.21.15.132:3300/0 conn(0x7f0ce40cb4c0 0x7f0ce40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- 172.21.15.46:0/864832690 >> 172.21.15.46:0/864832690 conn(0x7f0ce40683b0 msgr2=0x7f0ce40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:17.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- 172.21.15.46:0/864832690 shutdown_connections 2023-10-06T11:34:17.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- 172.21.15.46:0/864832690 wait complete. 2023-10-06T11:34:17.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 Processor -- start 2023-10-06T11:34:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.559+0000 7f0cea2e3700 1 -- start start 2023-10-06T11:34:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 0x7f0ce40c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4148050 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce416aa80 con 0x7f0ce40af3d0 2023-10-06T11:34:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40c9dd0 con 0x7f0ce40af000 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce92e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce8ae0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 0x7f0ce40c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce92e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:38254/0 (socket says 172.21.15.46:38254) 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce92e1700 1 -- 172.21.15.46:0/1752763421 learned_addr learned my addr 172.21.15.46:0/1752763421 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 msgr2=0x7f0ce4147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 msgr2=0x7f0ce40c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 -- 172.21.15.46:0/1752763421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc009ae0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce92e1700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce8ae0700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0ce9ae2700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 0x7f0ce40c7610 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc014000 tx=0x7f0cdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0032f0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc012d00 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc01d670 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce40ca0b0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.563+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40b37d0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce40b39c0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f0cdc01daa0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0cd97fa700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 0x7f0cc8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0ce92e1700 1 -- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 msgr2=0x7f0cc8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f0cdc07e860 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.567+0000 7f0ce92e1700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 0x7f0cc8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.579+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0cdc04e5b0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.593 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.589+0000 7f771b15f000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:17.653 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.643+0000 7f5f36b68000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:17.681 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.677+0000 7f771b15f000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:17.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.751+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ce4064590 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.751+0000 7f0cd97fa700 1 -- 172.21.15.46:0/1752763421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18029 (secure 0 0 0) 0x7f0cdc051bd0 con 0x7f0ce40cb4c0 2023-10-06T11:34:17.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:12.318870+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/730464998":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6832/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/921761638":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1451694921":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1040714298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3304341298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3114837258":"2023-10-07T11:34:12.318801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 msgr2=0x7f0cc8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 0x7f0cc8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 msgr2=0x7f0ce40c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 0x7f0ce40c7610 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc014000 tx=0x7f0cdc017000 comp rx=0 tx=0).stop 2023-10-06T11:34:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 shutdown_connections 2023-10-06T11:34:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc8060700 0x7f0cc8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ce40cb4c0 0x7f0ce40c7610 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ce40af3d0 0x7f0ce40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 --2- 172.21.15.46:0/1752763421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ce40af000 0x7f0ce4147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 >> 172.21.15.46:0/1752763421 conn(0x7f0ce40683b0 msgr2=0x7f0ce41578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 shutdown_connections 2023-10-06T11:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.759+0000 7f0cea2e3700 1 -- 172.21.15.46:0/1752763421 wait complete. 2023-10-06T11:34:17.789 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:34:17.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:17.870 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.865+0000 7f771b15f000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:17.892 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.883+0000 7f5f36b68000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:17.953 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:17.949+0000 7f771b15f000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:17.977 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:17.967+0000 7f5f36b68000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.979+0000 7f650566e700 1 Processor -- start 2023-10-06T11:34:17.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.979+0000 7f650566e700 1 -- start start 2023-10-06T11:34:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.979+0000 7f650566e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 0x7f65000b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f650566e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65000cdc80 0x7f65000ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f650566e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65000adbc0 con 0x7f65000b0330 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f650566e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65000b11b0 con 0x7f65000cdc80 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f650566e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65000acfb0 con 0x7f65000b2a30 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f65000b0330 0x7f6500159ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fffff700 1 -- 172.21.15.46:0/2555240908 learned_addr learned my addr 172.21.15.46:0/2555240908 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:17.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64feffd700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 0x7f65000b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 --2- 172.21.15.46:0/2555240908 >> v2:172.21.15.132:3300/0 conn(0x7f65000cdc80 0x7f65000ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:17.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fe7fc700 1 -- 172.21.15.46:0/2555240908 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307138990 0 0) 0x7f65000adbc0 con 0x7f65000b0330 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fe7fc700 1 -- 172.21.15.46:0/2555240908 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64ec003570 con 0x7f65000b0330 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 -- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 msgr2=0x7f65000b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 0x7f65000b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 -- 172.21.15.46:0/2555240908 >> v1:172.21.15.132:6789/0 conn(0x7f65000b0330 legacy=0x7f6500159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 -- 172.21.15.46:0/2555240908 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000ad290 con 0x7f65000cdc80 2023-10-06T11:34:17.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64feffd700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 0x7f65000b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:17.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64f67fc700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000cdc80 0x7f65000ce0b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009e60 tx=0x7f64e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab969491a7a6f68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:17.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fe7fc700 1 -- 172.21.15.46:0/2555240908 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800e070 con 0x7f65000cdc80 2023-10-06T11:34:17.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fe7fc700 1 -- 172.21.15.46:0/2555240908 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e800bab0 con 0x7f65000cdc80 2023-10-06T11:34:17.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.983+0000 7f64fe7fc700 1 -- 172.21.15.46:0/2555240908 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800be70 con 0x7f65000cdc80 2023-10-06T11:34:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 -- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000cdc80 msgr2=0x7f65000ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:17.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000cdc80 0x7f65000ce0b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009e60 tx=0x7f64e8009780 comp rx=0 tx=0).stop 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 -- 172.21.15.46:0/2555240908 shutdown_connections 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000cdc80 0x7f65000ce0b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 --2- 172.21.15.46:0/2555240908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b2a30 0x7f65000b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 -- 172.21.15.46:0/2555240908 >> 172.21.15.46:0/2555240908 conn(0x7f6500069250 msgr2=0x7f65000cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 -- 172.21.15.46:0/2555240908 shutdown_connections 2023-10-06T11:34:17.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 -- 172.21.15.46:0/2555240908 wait complete. 2023-10-06T11:34:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.987+0000 7f650566e700 1 Processor -- start 2023-10-06T11:34:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- start start 2023-10-06T11:34:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b0330 0x7f6500152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 0x7f650014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650014d110 con 0x7f65000cdc80 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65001516f0 con 0x7f65000b2a30 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650016a090 con 0x7f65000b0330 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59502/0 (socket says 172.21.15.46:59502) 2023-10-06T11:34:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 -- 172.21.15.46:0/2751183022 learned_addr learned my addr 172.21.15.46:0/2751183022 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:18.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64feffd700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b0330 0x7f6500152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64f67fc700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 0x7f650014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b0330 msgr2=0x7f6500152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b0330 0x7f6500152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 msgr2=0x7f650014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 0x7f650014f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 -- 172.21.15.46:0/2751183022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f0009420 con 0x7f65000cdc80 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64f67fc700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 0x7f650014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64fffff700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f64f80010e0 tx=0x7f64f8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8005c00 con 0x7f65000cdc80 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8005d60 con 0x7f65000cdc80 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e8009a20 con 0x7f65000cdc80 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80072e0 con 0x7f65000cdc80 2023-10-06T11:34:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.991+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6500151c70 con 0x7f65000cdc80 2023-10-06T11:34:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.995+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65000acc00 con 0x7f65000cdc80 2023-10-06T11:34:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.995+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f64f8012400 con 0x7f65000cdc80 2023-10-06T11:34:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.999+0000 7f64f77fe700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 0x7f64e0062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.999+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f64f807c3e0 con 0x7f65000cdc80 2023-10-06T11:34:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.999+0000 7f64feffd700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 msgr2=0x7f64e0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:17.999+0000 7f64feffd700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 0x7f64e0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:18.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.003+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64f8049be0 con 0x7f65000cdc80 2023-10-06T11:34:18.072 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:18.063+0000 7f5f36b68000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:18.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.203+0000 7f64feffd700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 msgr2=0x7f64e0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:18.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.203+0000 7f64feffd700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 0x7f64e0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:18.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.215+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65000ad570 con 0x7f65000cdc80 2023-10-06T11:34:18.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f64f77fe700 1 -- 172.21.15.46:0/2751183022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f64f804c260 con 0x7f65000cdc80 2023-10-06T11:34:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:18.227 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 msgr2=0x7f64e0062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 0x7f64e0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 msgr2=0x7f650006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f64f80010e0 tx=0x7f64f8007dd0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 shutdown_connections 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e0060700 0x7f64e0062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000cdc80 0x7f650006b570 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000b2a30 0x7f650014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 --2- 172.21.15.46:0/2751183022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65000b0330 0x7f6500152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 >> 172.21.15.46:0/2751183022 conn(0x7f6500069250 msgr2=0x7f65000af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 shutdown_connections 2023-10-06T11:34:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.219+0000 7f650566e700 1 -- 172.21.15.46:0/2751183022 wait complete. 2023-10-06T11:34:18.234 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:18.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:18.285 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:18.275+0000 7f5f36b68000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:18.381 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:18.371+0000 7f5f36b68000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.399+0000 7fc43d80c700 1 Processor -- start 2023-10-06T11:34:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.399+0000 7fc43d80c700 1 -- start start 2023-10-06T11:34:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc438156e20 0x7fc43806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc438067760 0x7fc438068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43806d790 con 0x7fc4380aaf30 2023-10-06T11:34:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4380acd60 con 0x7fc438067760 2023-10-06T11:34:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380ad040 con 0x7fc438156e20 2023-10-06T11:34:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc4380aaf30 0x7fc438067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51574/0 (socket says 172.21.15.46:51574) 2023-10-06T11:34:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42ffff700 1 -- 172.21.15.46:0/2197507701 learned_addr learned my addr 172.21.15.46:0/2197507701 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 --2- 172.21.15.46:0/2197507701 >> v2:172.21.15.132:3300/0 conn(0x7fc438067760 0x7fc438068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43c80a700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc438156e20 0x7fc43806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42f7fe700 1 -- 172.21.15.46:0/2197507701 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251202428 0 0) 0x7fc43806d790 con 0x7fc4380aaf30 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42f7fe700 1 -- 172.21.15.46:0/2197507701 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc420003570 con 0x7fc4380aaf30 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 -- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc438156e20 msgr2=0x7fc43806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc438156e20 0x7fc43806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 -- 172.21.15.46:0/2197507701 >> v1:172.21.15.46:6789/0 conn(0x7fc4380aaf30 legacy=0x7fc438067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 -- 172.21.15.46:0/2197507701 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380ad320 con 0x7fc438067760 2023-10-06T11:34:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d00b700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc438068a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc430009e60 tx=0x7fc430009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59f6688d18f2a37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42f7fe700 1 -- 172.21.15.46:0/2197507701 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000e070 con 0x7fc438067760 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42f7fe700 1 -- 172.21.15.46:0/2197507701 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000bab0 con 0x7fc438067760 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc42f7fe700 1 -- 172.21.15.46:0/2197507701 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000be70 con 0x7fc438067760 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 -- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 msgr2=0x7fc438068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.403+0000 7fc43d80c700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc438068a20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc430009e60 tx=0x7fc430009780 comp rx=0 tx=0).stop 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 -- 172.21.15.46:0/2197507701 shutdown_connections 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc438068a20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 --2- 172.21.15.46:0/2197507701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc438156e20 0x7fc43806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 -- 172.21.15.46:0/2197507701 >> 172.21.15.46:0/2197507701 conn(0x7fc438062ef0 msgr2=0x7fc4380cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 -- 172.21.15.46:0/2197507701 shutdown_connections 2023-10-06T11:34:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 -- 172.21.15.46:0/2197507701 wait complete. 2023-10-06T11:34:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 Processor -- start 2023-10-06T11:34:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.407+0000 7fc43d80c700 1 -- start start 2023-10-06T11:34:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4380aaf30 0x7fc4381531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc438156e20 0x7fc4380c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815bd70 con 0x7fc438156e20 2023-10-06T11:34:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815b4b0 con 0x7fc438067760 2023-10-06T11:34:18.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815b720 con 0x7fc4380aaf30 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d00b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc438156e20 0x7fc4380c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:35114/0 (socket says 172.21.15.46:35114) 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 -- 172.21.15.46:0/372278696 learned_addr learned my addr 172.21.15.46:0/372278696 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc42ffff700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4380aaf30 0x7fc4381531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4380aaf30 msgr2=0x7fc4381531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4380aaf30 0x7fc4381531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc438156e20 msgr2=0x7fc4380c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc438156e20 0x7fc4380c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 -- 172.21.15.46:0/372278696 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430009a20 con 0x7fc438067760 2023-10-06T11:34:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43c80a700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e930 tx=0x7fc4340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434013680 con 0x7fc438067760 2023-10-06T11:34:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4340137e0 con 0x7fc438067760 2023-10-06T11:34:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43815ba00 con 0x7fc438067760 2023-10-06T11:34:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434013b80 con 0x7fc438067760 2023-10-06T11:34:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.411+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380c7830 con 0x7fc438067760 2023-10-06T11:34:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.415+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc438064590 con 0x7fc438067760 2023-10-06T11:34:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.415+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc43401c450 con 0x7fc438067760 2023-10-06T11:34:18.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.419+0000 7fc42cff9700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 0x7fc424062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.419+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc43407d050 con 0x7fc438067760 2023-10-06T11:34:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.419+0000 7fc42ffff700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 msgr2=0x7fc424062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.419+0000 7fc42ffff700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 0x7fc424062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.427+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc4340acbe0 con 0x7fc438067760 2023-10-06T11:34:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.623+0000 7fc42ffff700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 msgr2=0x7fc424062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.623+0000 7fc42ffff700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 0x7fc424062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:18.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.635+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4380ad600 con 0x7fc438067760 2023-10-06T11:34:18.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.635+0000 7fc42cff9700 1 -- 172.21.15.46:0/372278696 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fc434017090 con 0x7fc438067760 2023-10-06T11:34:18.647 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:18.648 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:33:57.603583+0000","modified":"2023-10-06T11:34:01.495264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6831},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6831":{"gid":6831,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3896471769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3896471769},{"type":"v1","addr":"172.21.15.46:6835","nonce":3896471769}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T11:34:18.649 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:18.645+0000 7f771b15f000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.639+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 msgr2=0x7fc424062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 0x7fc424062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 msgr2=0x7fc43815c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e930 tx=0x7fc4340091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 shutdown_connections 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc424060730 0x7fc424062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc438156e20 0x7fc4380c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4380aaf30 0x7fc4381531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 --2- 172.21.15.46:0/372278696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc438067760 0x7fc43815c200 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 >> 172.21.15.46:0/372278696 conn(0x7fc438062ef0 msgr2=0x7fc438156010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 shutdown_connections 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.643+0000 7fc43d80c700 1 -- 172.21.15.46:0/372278696 wait complete. 2023-10-06T11:34:18.654 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2023-10-06T11:34:18.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 Processor -- start 2023-10-06T11:34:18.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 -- start start 2023-10-06T11:34:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fabe80c5ff0 0x7fabe80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fabe80ccb50 0x7fabe80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe80cc150 con 0x7fabe815c770 2023-10-06T11:34:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe80cfbb0 con 0x7fabe80c5ff0 2023-10-06T11:34:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabee385700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe80cdbf0 con 0x7fabe80ccb50 2023-10-06T11:34:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabe7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fabe815c770 0x7fabe80cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51578/0 (socket says 172.21.15.46:51578) 2023-10-06T11:34:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabe7fff700 1 -- 172.21.15.46:0/315964220 learned_addr learned my addr 172.21.15.46:0/315964220 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabdffff700 1 --2- 172.21.15.46:0/315964220 >> v2:172.21.15.46:3300/0 conn(0x7fabe80c5ff0 0x7fabe80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.879+0000 7fabec922700 1 --2- 172.21.15.46:0/315964220 >> v2:172.21.15.132:3300/0 conn(0x7fabe80ccb50 0x7fabe80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabe77fe700 1 -- 172.21.15.46:0/315964220 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151957981 0 0) 0x7fabe80cc150 con 0x7fabe815c770 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabe77fe700 1 -- 172.21.15.46:0/315964220 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd4003570 con 0x7fabe815c770 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabdffff700 1 -- 172.21.15.46:0/315964220 >> v2:172.21.15.132:3300/0 conn(0x7fabe80ccb50 msgr2=0x7fabe80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabdffff700 1 --2- 172.21.15.46:0/315964220 >> v2:172.21.15.132:3300/0 conn(0x7fabe80ccb50 0x7fabe80cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabdffff700 1 -- 172.21.15.46:0/315964220 >> v1:172.21.15.46:6789/0 conn(0x7fabe815c770 legacy=0x7fabe80cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabdffff700 1 -- 172.21.15.46:0/315964220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80cb540 con 0x7fabe80c5ff0 2023-10-06T11:34:18.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabdffff700 1 --2- 172.21.15.46:0/315964220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80c5ff0 0x7fabe80ce4f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fabe80cb450 tx=0x7fabd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f163850d33dde67f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabe77fe700 1 -- 172.21.15.46:0/315964220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011630 con 0x7fabe80c5ff0 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabe77fe700 1 -- 172.21.15.46:0/315964220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00117b0 con 0x7fabe80c5ff0 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabe77fe700 1 -- 172.21.15.46:0/315964220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011b70 con 0x7fabe80c5ff0 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 -- 172.21.15.46:0/315964220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80c5ff0 msgr2=0x7fabe80ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 --2- 172.21.15.46:0/315964220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80c5ff0 0x7fabe80ce4f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fabe80cb450 tx=0x7fabd00091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 -- 172.21.15.46:0/315964220 shutdown_connections 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 --2- 172.21.15.46:0/315964220 >> v2:172.21.15.132:3300/0 conn(0x7fabe80ccb50 0x7fabe80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 --2- 172.21.15.46:0/315964220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80c5ff0 0x7fabe80ce4f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 -- 172.21.15.46:0/315964220 >> 172.21.15.46:0/315964220 conn(0x7fabe8099480 msgr2=0x7fabe806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:18.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 -- 172.21.15.46:0/315964220 shutdown_connections 2023-10-06T11:34:18.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.883+0000 7fabee385700 1 -- 172.21.15.46:0/315964220 wait complete. 2023-10-06T11:34:18.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 Processor -- start 2023-10-06T11:34:18.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- start start 2023-10-06T11:34:18.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabe80c5ff0 0x7fabe809d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabe815c770 0x7fabe815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe809b040 con 0x7fabe80ccb50 2023-10-06T11:34:18.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe809cfc0 con 0x7fabe80c5ff0 2023-10-06T11:34:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe8159f70 con 0x7fabe815c770 2023-10-06T11:34:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59530/0 (socket says 172.21.15.46:59530) 2023-10-06T11:34:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 -- 172.21.15.46:0/3600745644 learned_addr learned my addr 172.21.15.46:0/3600745644 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:18.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabe7fff700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabe80c5ff0 0x7fabe809d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabec922700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabe815c770 0x7fabe815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:18.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 -- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabe815c770 msgr2=0x7fabe815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabe815c770 0x7fabe815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 -- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabe80c5ff0 msgr2=0x7fabe809d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabe80c5ff0 0x7fabe809d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 -- 172.21.15.46:0/3600745644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009420 con 0x7fabe80ccb50 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabdffff700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fabd0013000 tx=0x7fabd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd001f070 con 0x7fabe80ccb50 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd0000bc0 con 0x7fabe80ccb50 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0011b70 con 0x7fabe80ccb50 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe809d520 con 0x7fabe80ccb50 2023-10-06T11:34:18.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.887+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe809aa80 con 0x7fabe80ccb50 2023-10-06T11:34:18.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe809d6b0 con 0x7fabe80ccb50 2023-10-06T11:34:18.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 40) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fabd0023070 con 0x7fabe80ccb50 2023-10-06T11:34:18.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabe4ff9700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 0x7fabc8062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:18.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fabd007e2e0 con 0x7fabe80ccb50 2023-10-06T11:34:18.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabe7fff700 1 -- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 msgr2=0x7fabc8062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:18.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.891+0000 7fabe7fff700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 0x7fabc8062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:18.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:18.899+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fabd004e1e0 con 0x7fabe80ccb50 2023-10-06T11:34:19.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.071+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabe80660f0 con 0x7fabe80ccb50 2023-10-06T11:34:19.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabe4ff9700 1 -- 172.21.15.46:0/3600745644 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18029 (secure 0 0 0) 0x7fabd0051800 con 0x7fabe80ccb50 2023-10-06T11:34:19.084 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:19.084 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:12.318870+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:33:55.950284+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T11:33:56.602195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/730464998":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:6832/32974":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/921761638":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1451694921":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/1040714298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3304341298":"2023-10-07T11:34:12.318801+0000","172.21.15.46:0/3114837258":"2023-10-07T11:34:12.318801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:19.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 msgr2=0x7fabc8062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 0x7fabc8062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 msgr2=0x7fabe815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fabd0013000 tx=0x7fabd0017000 comp rx=0 tx=0).stop 2023-10-06T11:34:19.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 shutdown_connections 2023-10-06T11:34:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fabc8060730 0x7fabc8062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabe815c770 0x7fabe815d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabe80ccb50 0x7fabe815cf20 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 --2- 172.21.15.46:0/3600745644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fabe80c5ff0 0x7fabe809d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 >> 172.21.15.46:0/3600745644 conn(0x7fabe8099480 msgr2=0x7fabe815baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 shutdown_connections 2023-10-06T11:34:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.075+0000 7fabee385700 1 -- 172.21.15.46:0/3600745644 wait complete. 2023-10-06T11:34:19.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:34:19.204 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.195+0000 7f5f36b68000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 Processor -- start 2023-10-06T11:34:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 -- start start 2023-10-06T11:34:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8dc806da60 0x7f8dc80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc80c6690 con 0x7f8dc8067050 2023-10-06T11:34:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc80c74d0 con 0x7f8dc8155c70 2023-10-06T11:34:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.259+0000 7f8dc759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc806dee0 con 0x7f8dc806da60 2023-10-06T11:34:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc6d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8dc8067050 0x7f8dc80abad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51594/0 (socket says 172.21.15.46:51594) 2023-10-06T11:34:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc6d9d700 1 -- 172.21.15.46:0/1122985427 learned_addr learned my addr 172.21.15.46:0/1122985427 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dbffff700 1 --2- 172.21.15.46:0/1122985427 >> v2:172.21.15.132:3300/0 conn(0x7f8dc806da60 0x7f8dc80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc5d9b700 1 -- 172.21.15.46:0/1122985427 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636310448 0 0) 0x7f8dc80c6690 con 0x7f8dc8067050 2023-10-06T11:34:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc5d9b700 1 -- 172.21.15.46:0/1122985427 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db4003570 con 0x7f8dc8067050 2023-10-06T11:34:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dbffff700 1 -- 172.21.15.46:0/1122985427 >> v1:172.21.15.132:6789/0 conn(0x7f8dc8155c70 legacy=0x7f8dc80c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dbffff700 1 -- 172.21.15.46:0/1122985427 >> v1:172.21.15.46:6789/0 conn(0x7f8dc8067050 legacy=0x7f8dc80abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dbffff700 1 -- 172.21.15.46:0/1122985427 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80c68c0 con 0x7f8dc806da60 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dbffff700 1 --2- 172.21.15.46:0/1122985427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc806da60 0x7f8dc80c6150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098e0 tx=0x7f8db00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6b677437005b42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc5d9b700 1 -- 172.21.15.46:0/1122985427 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000b640 con 0x7f8dc806da60 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc5d9b700 1 -- 172.21.15.46:0/1122985427 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db000b7a0 con 0x7f8dc806da60 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc5d9b700 1 -- 172.21.15.46:0/1122985427 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000bb20 con 0x7f8dc806da60 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc759e700 1 -- 172.21.15.46:0/1122985427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc806da60 msgr2=0x7f8dc80c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1122985427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc806da60 0x7f8dc80c6150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098e0 tx=0x7f8db00091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc759e700 1 -- 172.21.15.46:0/1122985427 shutdown_connections 2023-10-06T11:34:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1122985427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc806da60 0x7f8dc80c6150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.263+0000 7f8dc759e700 1 -- 172.21.15.46:0/1122985427 >> 172.21.15.46:0/1122985427 conn(0x7f8dc8062ef0 msgr2=0x7f8dc80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.267+0000 7f8dc759e700 1 -- 172.21.15.46:0/1122985427 shutdown_connections 2023-10-06T11:34:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.267+0000 7f8dc759e700 1 -- 172.21.15.46:0/1122985427 wait complete. 2023-10-06T11:34:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.267+0000 7f8dc759e700 1 Processor -- start 2023-10-06T11:34:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.267+0000 7f8dc759e700 1 -- start start 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8dc806da60 0x7f8dc80c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc8155c70 0x7f8dc80c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc81693f0 con 0x7f8dc8067050 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80c0870 con 0x7f8dc8155c70 2023-10-06T11:34:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80c0b50 con 0x7f8dc806da60 2023-10-06T11:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dbffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8dc806da60 0x7f8dc80c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59576/0 (socket says 172.21.15.46:59576) 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 -- 172.21.15.46:0/1836163023 learned_addr learned my addr 172.21.15.46:0/1836163023 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc6d9d700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc8155c70 0x7f8dc80c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 -- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8dc806da60 msgr2=0x7f8dc80c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8dc806da60 0x7f8dc80c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 -- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc8155c70 msgr2=0x7f8dc80c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc8155c70 0x7f8dc80c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 -- 172.21.15.46:0/1836163023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0009420 con 0x7f8dc8067050 2023-10-06T11:34:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc659c700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c4e0 tx=0x7f8db800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8011590 con 0x7f8dc8067050 2023-10-06T11:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8011710 con 0x7f8dc8067050 2023-10-06T11:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8011ad0 con 0x7f8dc8067050 2023-10-06T11:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc80c0e30 con 0x7f8dc8067050 2023-10-06T11:34:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.271+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc80c0560 con 0x7f8dc8067050 2023-10-06T11:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.275+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc8169610 con 0x7f8dc8067050 2023-10-06T11:34:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.275+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 41) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f8db801b3b0 con 0x7f8dc8067050 2023-10-06T11:34:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.275+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7147+0+0 (secure 0 0 0) 0x7f8db807bd00 con 0x7f8dc8067050 2023-10-06T11:34:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.279+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8dc8169610 con 0x7f8dc8067050 2023-10-06T11:34:19.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5f027ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5f027ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5f027ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5f027ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5f027ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5eff7a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5eff7a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5eff7a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5eff7a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:19.307+0000 7f5eff7a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.495+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8dc8064590 con 0x7f8dc8067050 2023-10-06T11:34:19.508 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:34:19.499+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:34:19.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.507+0000 7f8dbe7fc700 1 -- 172.21.15.46:0/1836163023 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8db804c020 con 0x7f8dc8067050 2023-10-06T11:34:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 msgr2=0x7f8dc80c0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c4e0 tx=0x7f8db800c7f0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 shutdown_connections 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8dc8155c70 0x7f8dc80c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8dc806da60 0x7f8dc80c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 --2- 172.21.15.46:0/1836163023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc8067050 0x7f8dc80c0020 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 >> 172.21.15.46:0/1836163023 conn(0x7f8dc8062ef0 msgr2=0x7f8dc80c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 shutdown_connections 2023-10-06T11:34:19.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.511+0000 7f8dc759e700 1 -- 172.21.15.46:0/1836163023 wait complete. 2023-10-06T11:34:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:34:19.526 INFO:tasks.ceph.mds.c.smithi046.stderr: -1383> 2023-10-06T11:34:03.123+0000 7f12afbb7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:34:19.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:34:19.569 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:34:19.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.687+0000 7f522dd23700 1 Processor -- start 2023-10-06T11:34:19.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.687+0000 7f522dd23700 1 -- start start 2023-10-06T11:34:19.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522dd23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5228159240 0x7f52280a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522dd23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280c10a0 con 0x7f52280c72d0 2023-10-06T11:34:19.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522dd23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280ca240 con 0x7f5228158e70 2023-10-06T11:34:19.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522dd23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280ca520 con 0x7f5228159240 2023-10-06T11:34:19.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522d522700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52280c72d0 0x7f52280c7a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51606/0 (socket says 172.21.15.46:51606) 2023-10-06T11:34:19.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f522d522700 1 -- 172.21.15.46:0/663099997 learned_addr learned my addr 172.21.15.46:0/663099997 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:19.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f5227fff700 1 --2- 172.21.15.46:0/663099997 >> v2:172.21.15.46:3300/0 conn(0x7f5228159240 0x7f52280a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353561080 0 0) 0x7f52280c10a0 con 0x7f52280c72d0 2023-10-06T11:34:19.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5218003570 con 0x7f52280c72d0 2023-10-06T11:34:19.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088588070 0 0) 0x7f52280ca240 con 0x7f5228158e70 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f5227fff700 1 -- 172.21.15.46:0/663099997 >> v1:172.21.15.132:6789/0 conn(0x7f5228158e70 legacy=0x7f52280c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f5227fff700 1 -- 172.21.15.46:0/663099997 >> v1:172.21.15.46:6789/0 conn(0x7f52280c72d0 legacy=0x7f52280c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.691+0000 7f5227fff700 1 -- 172.21.15.46:0/663099997 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280c0a50 con 0x7f5228159240 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f5227fff700 1 --2- 172.21.15.46:0/663099997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5228159240 0x7f52280a9840 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f52140098e0 tx=0x7f52140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158091fd01df01d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220001470 con 0x7f5228159240 2023-10-06T11:34:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214011420 con 0x7f5228159240 2023-10-06T11:34:19.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f52277fe700 1 -- 172.21.15.46:0/663099997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140117c0 con 0x7f5228159240 2023-10-06T11:34:19.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 -- 172.21.15.46:0/663099997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5228159240 msgr2=0x7f52280a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 --2- 172.21.15.46:0/663099997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5228159240 0x7f52280a9840 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f52140098e0 tx=0x7f52140091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 -- 172.21.15.46:0/663099997 shutdown_connections 2023-10-06T11:34:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 --2- 172.21.15.46:0/663099997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5228159240 0x7f52280a9840 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 -- 172.21.15.46:0/663099997 >> 172.21.15.46:0/663099997 conn(0x7f52280cbc10 msgr2=0x7f52280cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 -- 172.21.15.46:0/663099997 shutdown_connections 2023-10-06T11:34:19.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.695+0000 7f522dd23700 1 -- 172.21.15.46:0/663099997 wait complete. 2023-10-06T11:34:19.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 Processor -- start 2023-10-06T11:34:19.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 -- start start 2023-10-06T11:34:19.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5228158e70 0x7f52280b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5228159240 0x7f52280b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5228153120 con 0x7f52280c72d0 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52281563c0 con 0x7f5228159240 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522dd23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52281557b0 con 0x7f5228158e70 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59682/0 (socket says 172.21.15.46:59682) 2023-10-06T11:34:19.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 -- 172.21.15.46:0/2892829346 learned_addr learned my addr 172.21.15.46:0/2892829346 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:19.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f5227fff700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5228158e70 0x7f52280b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522d522700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5228159240 0x7f52280b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:19.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 -- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5228158e70 msgr2=0x7f52280b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5228158e70 0x7f52280b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 -- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5228159240 msgr2=0x7f52280b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5228159240 0x7f52280b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.699+0000 7f522cd21700 1 -- 172.21.15.46:0/2892829346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009420 con 0x7f52280c72d0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f522cd21700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f521c00cb60 tx=0x7f521c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00a7e0 con 0x7f52280c72d0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c00a960 con 0x7f52280c72d0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00ad20 con 0x7f52280c72d0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5228155a90 con 0x7f52280c72d0 2023-10-06T11:34:19.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5228156050 con 0x7f52280c72d0 2023-10-06T11:34:19.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.703+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52280cb3a0 con 0x7f52280c72d0 2023-10-06T11:34:19.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.707+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 41) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f521c003030 con 0x7f52280c72d0 2023-10-06T11:34:19.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.707+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7233+0+0 (secure 0 0 0) 0x7f521c07cae0 con 0x7f52280c72d0 2023-10-06T11:34:19.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.711+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52280cb3a0 con 0x7f52280c72d0 2023-10-06T11:34:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:19.919+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f52280cb3a0 con 0x7f52280c72d0 2023-10-06T11:34:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.227+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 42) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f521c04c850 con 0x7f52280c72d0 2023-10-06T11:34:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.231+0000 7f5224ff9700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 0x7f520c068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.231+0000 7f5227fff700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 0x7f520c068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.231+0000 7f5227fff700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 0x7f520c068d60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f52140030c0 tx=0x7f5214017000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:20.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.511+0000 7f5224ff9700 1 -- 172.21.15.46:0/2892829346 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f521c069f70 con 0x7f52280c72d0 2023-10-06T11:34:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 msgr2=0x7f520c068d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 0x7f520c068d60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f52140030c0 tx=0x7f5214017000 comp rx=0 tx=0).stop 2023-10-06T11:34:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 msgr2=0x7f5228152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f521c00cb60 tx=0x7f521c0017a0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 shutdown_connections 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f520c066980 0x7f520c068d60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5228159240 0x7f52280b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5228158e70 0x7f52280b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 --2- 172.21.15.46:0/2892829346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52280c72d0 0x7f5228152be0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 >> 172.21.15.46:0/2892829346 conn(0x7f52280cbc10 msgr2=0x7f5228068380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:20.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 shutdown_connections 2023-10-06T11:34:20.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.519+0000 7f522dd23700 1 -- 172.21.15.46:0/2892829346 wait complete. 2023-10-06T11:34:20.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:34:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.751+0000 7f788a26e700 1 Processor -- start 2023-10-06T11:34:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.751+0000 7f788a26e700 1 -- start start 2023-10-06T11:34:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788a26e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78840c6b70 0x7f788406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788a26e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840c25a0 con 0x7f7884140bd0 2023-10-06T11:34:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788a26e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840c1c70 con 0x7f7884140800 2023-10-06T11:34:20.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788a26e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840c1f50 con 0x7f78840c6b70 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78840c6b70 0x7f788406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78840c6b70 0x7f788406cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59694/0 (socket says 172.21.15.46:59694) 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 -- 172.21.15.46:0/3279820005 learned_addr learned my addr 172.21.15.46:0/3279820005 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 -- 172.21.15.46:0/3279820005 >> v1:172.21.15.132:6789/0 conn(0x7f7884140800 legacy=0x7f78840c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 -- 172.21.15.46:0/3279820005 >> v1:172.21.15.46:6789/0 conn(0x7f7884140bd0 legacy=0x7f78840ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 -- 172.21.15.46:0/3279820005 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840c2230 con 0x7f78840c6b70 2023-10-06T11:34:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f788880b700 1 --2- 172.21.15.46:0/3279820005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f788406cc10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f787c009f20 tx=0x7f787c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aca2e0693fff4b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.755+0000 7f7882ffd700 1 -- 172.21.15.46:0/3279820005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00e070 con 0x7f78840c6b70 2023-10-06T11:34:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f7882ffd700 1 -- 172.21.15.46:0/3279820005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00ba80 con 0x7f78840c6b70 2023-10-06T11:34:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f7882ffd700 1 -- 172.21.15.46:0/3279820005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c0092e0 con 0x7f78840c6b70 2023-10-06T11:34:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 -- 172.21.15.46:0/3279820005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 msgr2=0x7f788406cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 --2- 172.21.15.46:0/3279820005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f788406cc10 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f787c009f20 tx=0x7f787c009840 comp rx=0 tx=0).stop 2023-10-06T11:34:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 -- 172.21.15.46:0/3279820005 shutdown_connections 2023-10-06T11:34:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 --2- 172.21.15.46:0/3279820005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f788406cc10 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 -- 172.21.15.46:0/3279820005 >> 172.21.15.46:0/3279820005 conn(0x7f78840cbe20 msgr2=0x7f78840cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 -- 172.21.15.46:0/3279820005 shutdown_connections 2023-10-06T11:34:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.759+0000 7f788a26e700 1 -- 172.21.15.46:0/3279820005 wait complete. 2023-10-06T11:34:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 Processor -- start 2023-10-06T11:34:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 -- start start 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7884140800 0x7f78840b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7884140bd0 0x7f788413ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788416a4d0 con 0x7f78840c6b70 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c3270 con 0x7f7884140bd0 2023-10-06T11:34:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788a26e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c3550 con 0x7f7884140800 2023-10-06T11:34:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f7883fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f7883fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59698/0 (socket says 172.21.15.46:59698) 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f7883fff700 1 -- 172.21.15.46:0/2332400368 learned_addr learned my addr 172.21.15.46:0/2332400368 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f78837fe700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7884140800 0x7f78840b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.763+0000 7f788880b700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7884140bd0 0x7f788413ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 -- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7884140800 msgr2=0x7f78840b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7884140800 0x7f78840b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 -- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7884140bd0 msgr2=0x7f788413ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7884140bd0 0x7f788413ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 -- 172.21.15.46:0/2332400368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c009ae0 con 0x7f78840c6b70 2023-10-06T11:34:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f7883fff700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f787400c590 tx=0x7f787400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011590 con 0x7f78840c6b70 2023-10-06T11:34:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874011710 con 0x7f78840c6b70 2023-10-06T11:34:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011ad0 con 0x7f78840c6b70 2023-10-06T11:34:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840c3830 con 0x7f78840c6b70 2023-10-06T11:34:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840c3df0 con 0x7f78840c6b70 2023-10-06T11:34:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840c4040 con 0x7f78840c6b70 2023-10-06T11:34:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.767+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 42) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f787401b3b0 con 0x7f78840c6b70 2023-10-06T11:34:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.771+0000 7f786bfff700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 0x7f7870062920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.771+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7233+0+0 (secure 0 0 0) 0x7f787407bf50 con 0x7f78840c6b70 2023-10-06T11:34:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.771+0000 7f78837fe700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 0x7f7870062920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:20.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.771+0000 7f78837fe700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 0x7f7870062920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78840cafe0 tx=0x7f7878007ec0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.775+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f787404c1a0 con 0x7f78840c6b70 2023-10-06T11:34:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:20.943+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f788406d2b0 con 0x7f78840c6b70 2023-10-06T11:34:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.523+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7874055080 con 0x7f78840c6b70 2023-10-06T11:34:21.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.531+0000 7f786bfff700 1 -- 172.21.15.46:0/2332400368 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f787404fc30 con 0x7f78840c6b70 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 msgr2=0x7f7870062920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 0x7f7870062920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78840cafe0 tx=0x7f7878007ec0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 msgr2=0x7f7884169f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f787400c590 tx=0x7f787400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 shutdown_connections 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7870060470 0x7f7870062920 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7884140bd0 0x7f788413ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7884140800 0x7f78840b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 --2- 172.21.15.46:0/2332400368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78840c6b70 0x7f7884169f90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 >> 172.21.15.46:0/2332400368 conn(0x7f78840cbe20 msgr2=0x7f7884068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 shutdown_connections 2023-10-06T11:34:21.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.535+0000 7f788a26e700 1 -- 172.21.15.46:0/2332400368 wait complete. 2023-10-06T11:34:21.548 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:34:21.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:34:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 Processor -- start 2023-10-06T11:34:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 -- start start 2023-10-06T11:34:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc0c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0c1800 con 0x7f8bcc161e50 2023-10-06T11:34:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0c6ce0 con 0x7f8bcc146ea0 2023-10-06T11:34:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.727+0000 7f8bd3678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c1580 con 0x7f8bcc147270 2023-10-06T11:34:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd0c13700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8bcc161e50 0x7f8bcc0caa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51626/0 (socket says 172.21.15.46:51626) 2023-10-06T11:34:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd0c13700 1 -- 172.21.15.46:0/1698597590 learned_addr learned my addr 172.21.15.46:0/1698597590 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd1c15700 1 --2- 172.21.15.46:0/1698597590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc0c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212999115 0 0) 0x7f8bcc0c1800 con 0x7f8bcc161e50 2023-10-06T11:34:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb8003570 con 0x7f8bcc161e50 2023-10-06T11:34:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd1c15700 1 -- 172.21.15.46:0/1698597590 >> v1:172.21.15.132:6789/0 conn(0x7f8bcc146ea0 legacy=0x7f8bcc06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd1c15700 1 -- 172.21.15.46:0/1698597590 >> v1:172.21.15.46:6789/0 conn(0x7f8bcc161e50 legacy=0x7f8bcc0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075568896 0 0) 0x7f8bb8003570 con 0x7f8bcc161e50 2023-10-06T11:34:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd1c15700 1 -- 172.21.15.46:0/1698597590 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0c6460 con 0x7f8bcc147270 2023-10-06T11:34:21.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bd1c15700 1 --2- 172.21.15.46:0/1698597590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc0c1c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009910 tx=0x7f8bc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a693bf89c375ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:21.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc001470 con 0x7f8bcc147270 2023-10-06T11:34:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4011420 con 0x7f8bcc147270 2023-10-06T11:34:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.731+0000 7f8bc3fff700 1 -- 172.21.15.46:0/1698597590 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40117e0 con 0x7f8bcc147270 2023-10-06T11:34:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- 172.21.15.46:0/1698597590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 msgr2=0x7f8bcc0c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 --2- 172.21.15.46:0/1698597590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc0c1c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009910 tx=0x7f8bc40091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- 172.21.15.46:0/1698597590 shutdown_connections 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 --2- 172.21.15.46:0/1698597590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc0c1c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- 172.21.15.46:0/1698597590 >> 172.21.15.46:0/1698597590 conn(0x7f8bcc0cbe20 msgr2=0x7f8bcc0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- 172.21.15.46:0/1698597590 shutdown_connections 2023-10-06T11:34:21.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- 172.21.15.46:0/1698597590 wait complete. 2023-10-06T11:34:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 Processor -- start 2023-10-06T11:34:21.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.735+0000 7f8bd3678700 1 -- start start 2023-10-06T11:34:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc146ea0 0x7f8bcc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc161010 con 0x7f8bcc161e50 2023-10-06T11:34:21.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15d700 con 0x7f8bcc146ea0 2023-10-06T11:34:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1606e0 con 0x7f8bcc147270 2023-10-06T11:34:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59714/0 (socket says 172.21.15.46:59714) 2023-10-06T11:34:21.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 -- 172.21.15.46:0/87031400 learned_addr learned my addr 172.21.15.46:0/87031400 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1414700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc146ea0 0x7f8bcc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd0c13700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:21.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 -- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 msgr2=0x7f8bcc15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc15dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 -- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc146ea0 msgr2=0x7f8bcc15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc146ea0 0x7f8bcc15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 -- 172.21.15.46:0/87031400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009420 con 0x7f8bcc161e50 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd0c13700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd1c15700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06f470 tx=0x7f8bc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc401cda0 con 0x7f8bcc161e50 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc401a450 con 0x7f8bcc161e50 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc401a810 con 0x7f8bcc161e50 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc1609c0 con 0x7f8bcc161e50 2023-10-06T11:34:21.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.739+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1603d0 con 0x7f8bcc161e50 2023-10-06T11:34:21.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.743+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc1611c0 con 0x7f8bcc161e50 2023-10-06T11:34:21.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.743+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8bc401abc0 con 0x7f8bcc161e50 2023-10-06T11:34:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.743+0000 7f8bc17fa700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 0x7f8bb0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.743+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6795+0+0 (secure 0 0 0) 0x7f8bc407ea50 con 0x7f8bcc161e50 2023-10-06T11:34:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.747+0000 7f8bd1414700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 0x7f8bb0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:21.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.747+0000 7f8bd1414700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 0x7f8bb0062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8007fd0 tx=0x7f8bc8009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:21.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.751+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8bcc1611c0 con 0x7f8bcc161e50 2023-10-06T11:34:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:21.927+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bcc161150 con 0x7f8bcc161e50 2023-10-06T11:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.539+0000 7f8bc17fa700 1 -- 172.21.15.46:0/87031400 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8bc404ea50 con 0x7f8bcc161e50 2023-10-06T11:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 msgr2=0x7f8bb0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 0x7f8bb0062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8007fd0 tx=0x7f8bc8009040 comp rx=0 tx=0).stop 2023-10-06T11:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 msgr2=0x7f8bcc0c9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06f470 tx=0x7f8bc4016000 comp rx=0 tx=0).stop 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 shutdown_connections 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8bb0060670 0x7f8bb0062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bcc161e50 0x7f8bcc0c9990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bcc147270 0x7f8bcc15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 --2- 172.21.15.46:0/87031400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bcc146ea0 0x7f8bcc15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 >> 172.21.15.46:0/87031400 conn(0x7f8bcc0cbe20 msgr2=0x7f8bcc144a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 shutdown_connections 2023-10-06T11:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.543+0000 7f8bd3678700 1 -- 172.21.15.46:0/87031400 wait complete. 2023-10-06T11:34:22.556 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:34:22.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:34:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.735+0000 7f293e145700 1 Processor -- start 2023-10-06T11:34:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.735+0000 7f293e145700 1 -- start start 2023-10-06T11:34:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f293e145700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29380ccc00 0x7f29380ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f293e145700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 0x7f29380cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f293e145700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380cb4a0 con 0x7f29380b23f0 2023-10-06T11:34:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f293e145700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380cc780 con 0x7f29380ccc00 2023-10-06T11:34:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f293e145700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380cb6e0 con 0x7f29380b27c0 2023-10-06T11:34:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29377fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29380b23f0 0x7f29380cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51636/0 (socket says 172.21.15.46:51636) 2023-10-06T11:34:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29377fe700 1 -- 172.21.15.46:0/3959357706 learned_addr learned my addr 172.21.15.46:0/3959357706 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3959357706 >> v2:172.21.15.46:3300/0 conn(0x7f29380ccc00 0x7f29380ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2937fff700 1 --2- 172.21.15.46:0/3959357706 >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 0x7f29380cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29367fc700 1 -- 172.21.15.46:0/3959357706 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278181915 0 0) 0x7f2928001610 con 0x7f29380b23f0 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29367fc700 1 -- 172.21.15.46:0/3959357706 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2920003570 con 0x7f29380b23f0 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 -- 172.21.15.46:0/3959357706 >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 msgr2=0x7f29380cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3959357706 >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 0x7f29380cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 -- 172.21.15.46:0/3959357706 >> v1:172.21.15.46:6789/0 conn(0x7f29380b23f0 legacy=0x7f29380cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 -- 172.21.15.46:0/3959357706 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380cb9c0 con 0x7f29380ccc00 2023-10-06T11:34:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2937fff700 1 --2- 172.21.15.46:0/3959357706 >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 0x7f29380cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3959357706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380ccc00 0x7f29380ce4d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f293806f750 tx=0x7f292c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61807b97ee63ec8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29367fc700 1 -- 172.21.15.46:0/3959357706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c011630 con 0x7f29380ccc00 2023-10-06T11:34:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29367fc700 1 -- 172.21.15.46:0/3959357706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c011790 con 0x7f29380ccc00 2023-10-06T11:34:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.739+0000 7f29367fc700 1 -- 172.21.15.46:0/3959357706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c011b30 con 0x7f29380ccc00 2023-10-06T11:34:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- 172.21.15.46:0/3959357706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380ccc00 msgr2=0x7f29380ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 --2- 172.21.15.46:0/3959357706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380ccc00 0x7f29380ce4d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f293806f750 tx=0x7f292c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- 172.21.15.46:0/3959357706 shutdown_connections 2023-10-06T11:34:22.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 --2- 172.21.15.46:0/3959357706 >> v2:172.21.15.132:3300/0 conn(0x7f29380b27c0 0x7f29380cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 --2- 172.21.15.46:0/3959357706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380ccc00 0x7f29380ce4d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- 172.21.15.46:0/3959357706 >> 172.21.15.46:0/3959357706 conn(0x7f2938099440 msgr2=0x7f293806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- 172.21.15.46:0/3959357706 shutdown_connections 2023-10-06T11:34:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- 172.21.15.46:0/3959357706 wait complete. 2023-10-06T11:34:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 Processor -- start 2023-10-06T11:34:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.743+0000 7f293e145700 1 -- start start 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29380b23f0 0x7f293814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 0x7f293809c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293814d7f0 con 0x7f29380b27c0 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293809d670 con 0x7f29380ccc00 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293809ca60 con 0x7f29380b23f0 2023-10-06T11:34:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59724/0 (socket says 172.21.15.46:59724) 2023-10-06T11:34:22.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 -- 172.21.15.46:0/3899527727 learned_addr learned my addr 172.21.15.46:0/3899527727 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f29377fe700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29380b23f0 0x7f293814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2937fff700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 0x7f293809c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 -- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29380b23f0 msgr2=0x7f293814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29380b23f0 0x7f293814d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 -- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 msgr2=0x7f293809c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 0x7f293809c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 -- 172.21.15.46:0/3899527727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c009420 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2937fff700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 0x7f293809c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f2936ffd700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2938069e30 tx=0x7f292c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c011b10 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c0233f0 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293809cd40 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c023790 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.747+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293809d300 con 0x7f29380b27c0 2023-10-06T11:34:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.751+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293809d7e0 con 0x7f29380b27c0 2023-10-06T11:34:22.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.751+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f292c0193b0 con 0x7f29380b27c0 2023-10-06T11:34:22.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.755+0000 7f291f7fe700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 0x7f2924062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:22.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.755+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f292c07e120 con 0x7f29380b27c0 2023-10-06T11:34:22.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.755+0000 7f29377fe700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 0x7f2924062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:22.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.755+0000 7f29377fe700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 0x7f2924062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2928001770 tx=0x7f2928009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:22.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.759+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f292c04e2c0 con 0x7f29380b27c0 2023-10-06T11:34:22.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.931+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f29380660f0 con 0x7f29380b27c0 2023-10-06T11:34:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.939+0000 7f291f7fe700 1 -- 172.21.15.46:0/3899527727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f292c0518e0 con 0x7f29380b27c0 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 msgr2=0x7f2924062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 0x7f2924062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2928001770 tx=0x7f2928009040 comp rx=0 tx=0).stop 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 msgr2=0x7f293809bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2938069e30 tx=0x7f292c017000 comp rx=0 tx=0).stop 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 shutdown_connections 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2924060670 0x7f2924062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29380ccc00 0x7f293809c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29380b27c0 0x7f293809bee0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 --2- 172.21.15.46:0/3899527727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29380b23f0 0x7f293814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 >> 172.21.15.46:0/3899527727 conn(0x7f2938099440 msgr2=0x7f29380b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 shutdown_connections 2023-10-06T11:34:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:22.943+0000 7f293e145700 1 -- 172.21.15.46:0/3899527727 wait complete. 2023-10-06T11:34:22.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:34:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 Processor -- start 2023-10-06T11:34:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 -- start start 2023-10-06T11:34:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ce0154c50 0x7f1ce00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce00c6cf0 con 0x7f1ce00c7f20 2023-10-06T11:34:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce00c2a30 con 0x7f1ce0154880 2023-10-06T11:34:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1ce4a4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce00c83e0 con 0x7f1ce0154c50 2023-10-06T11:34:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ce0154c50 0x7f1ce00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ce0154c50 0x7f1ce00c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34420/0 (socket says 172.21.15.46:34420) 2023-10-06T11:34:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 -- 172.21.15.46:0/4146893052 learned_addr learned my addr 172.21.15.46:0/4146893052 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 -- 172.21.15.46:0/4146893052 >> v1:172.21.15.132:6789/0 conn(0x7f1ce0154880 legacy=0x7f1ce006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 -- 172.21.15.46:0/4146893052 >> v1:172.21.15.46:6789/0 conn(0x7f1ce00c7f20 legacy=0x7f1ce015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 -- 172.21.15.46:0/4146893052 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce006e5d0 con 0x7f1ce0154c50 2023-10-06T11:34:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cd7fff700 1 --2- 172.21.15.46:0/4146893052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce0154c50 0x7f1ce00c7180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80099b0 tx=0x7f1cc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d24cf7842fa9a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.135+0000 7f1cdeffd700 1 -- 172.21.15.46:0/4146893052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800b820 con 0x7f1ce0154c50 2023-10-06T11:34:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1cdeffd700 1 -- 172.21.15.46:0/4146893052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800b980 con 0x7f1ce0154c50 2023-10-06T11:34:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1cdeffd700 1 -- 172.21.15.46:0/4146893052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800bd20 con 0x7f1ce0154c50 2023-10-06T11:34:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/4146893052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce0154c50 msgr2=0x7f1ce00c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/4146893052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce0154c50 0x7f1ce00c7180 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80099b0 tx=0x7f1cc8009250 comp rx=0 tx=0).stop 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/4146893052 shutdown_connections 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/4146893052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce0154c50 0x7f1ce00c7180 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/4146893052 >> 172.21.15.46:0/4146893052 conn(0x7f1ce00cbc10 msgr2=0x7f1ce00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/4146893052 shutdown_connections 2023-10-06T11:34:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/4146893052 wait complete. 2023-10-06T11:34:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 Processor -- start 2023-10-06T11:34:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.139+0000 7f1ce4a4c700 1 -- start start 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ce0154880 0x7f1ce0069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 0x7f1ce014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0150700 con 0x7f1ce00c7f20 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00c2d60 con 0x7f1ce0154880 2023-10-06T11:34:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00c3040 con 0x7f1ce0154c50 2023-10-06T11:34:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34432/0 (socket says 172.21.15.46:34432) 2023-10-06T11:34:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 -- 172.21.15.46:0/3002157909 learned_addr learned my addr 172.21.15.46:0/3002157909 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdffff700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 0x7f1ce014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cd7fff700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ce0154880 0x7f1ce0069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 -- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 msgr2=0x7f1ce014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 0x7f1ce014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 -- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ce0154880 msgr2=0x7f1ce0069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ce0154880 0x7f1ce0069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 -- 172.21.15.46:0/3002157909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80094f0 con 0x7f1ce00c7f20 2023-10-06T11:34:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdffff700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 0x7f1ce014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cdf7fe700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c5d0 tx=0x7f1cd0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011590 con 0x7f1ce00c7f20 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd0011710 con 0x7f1ce00c7f20 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0011ad0 con 0x7f1ce00c7f20 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.143+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce00c3320 con 0x7f1ce00c7f20 2023-10-06T11:34:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.147+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce0150360 con 0x7f1ce00c7f20 2023-10-06T11:34:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.147+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce004be00 con 0x7f1ce00c7f20 2023-10-06T11:34:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.147+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1cd001c460 con 0x7f1ce00c7f20 2023-10-06T11:34:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.147+0000 7f1cd77fe700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.147+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f1cd007c990 con 0x7f1ce00c7f20 2023-10-06T11:34:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.151+0000 7f1cd7fff700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.151+0000 7f1cd7fff700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 0x7f1cc0062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80099b0 tx=0x7f1cc8016000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.155+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ce004be00 con 0x7f1ce00c7f20 2023-10-06T11:34:23.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.327+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f1ce004be00 con 0x7f1ce00c7f20 2023-10-06T11:34:23.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.339+0000 7f1cd77fe700 1 -- 172.21.15.46:0/3002157909 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1ce004be00 con 0x7f1ce00c7f20 2023-10-06T11:34:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 msgr2=0x7f1cc0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 0x7f1cc0062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80099b0 tx=0x7f1cc8016000 comp rx=0 tx=0).stop 2023-10-06T11:34:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 msgr2=0x7f1ce00c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c5d0 tx=0x7f1cd0007f60 comp rx=0 tx=0).stop 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 shutdown_connections 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc0060670 0x7f1cc0062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ce0154c50 0x7f1ce014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ce0154880 0x7f1ce0069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 --2- 172.21.15.46:0/3002157909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ce00c7f20 0x7f1ce00c6350 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 >> 172.21.15.46:0/3002157909 conn(0x7f1ce00cbc10 msgr2=0x7f1ce0068380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 shutdown_connections 2023-10-06T11:34:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.343+0000 7f1ce4a4c700 1 -- 172.21.15.46:0/3002157909 wait complete. 2023-10-06T11:34:23.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:34:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.535+0000 7fd92e0e3700 1 Processor -- start 2023-10-06T11:34:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.535+0000 7fd92e0e3700 1 -- start start 2023-10-06T11:34:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92e0e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd92813e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92e0e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd92813e8c0 0x7fd9280c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92e0e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92813ed40 con 0x7fd9280c6480 2023-10-06T11:34:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92e0e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280c1ff0 con 0x7fd92813e8c0 2023-10-06T11:34:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92e0e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280c13e0 con 0x7fd9280c6850 2023-10-06T11:34:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d0e1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd9280c6480 0x7fd9280c09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40478/0 (socket says 172.21.15.46:40478) 2023-10-06T11:34:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d0e1700 1 -- 172.21.15.46:0/1905640327 learned_addr learned my addr 172.21.15.46:0/1905640327 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92c8e0700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd92813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 --2- 172.21.15.46:0/1905640327 >> v2:172.21.15.46:3300/0 conn(0x7fd92813e8c0 0x7fd9280c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 -- 172.21.15.46:0/1905640327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 msgr2=0x7fd92813e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:23.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd92813e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 -- 172.21.15.46:0/1905640327 >> v1:172.21.15.46:6789/0 conn(0x7fd9280c6480 legacy=0x7fd9280c09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 -- 172.21.15.46:0/1905640327 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280c16c0 con 0x7fd92813e8c0 2023-10-06T11:34:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.539+0000 7fd92d8e2700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92813e8c0 0x7fd9280c97e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd92000f300 tx=0x7fd92000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce15a9c7a8278e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.547+0000 7fd91ffff700 1 -- 172.21.15.46:0/1905640327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920012070 con 0x7fd92813e8c0 2023-10-06T11:34:23.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.547+0000 7fd91ffff700 1 -- 172.21.15.46:0/1905640327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92000d0d0 con 0x7fd92813e8c0 2023-10-06T11:34:23.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.547+0000 7fd91ffff700 1 -- 172.21.15.46:0/1905640327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000d490 con 0x7fd92813e8c0 2023-10-06T11:34:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1905640327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92813e8c0 msgr2=0x7fd9280c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92813e8c0 0x7fd9280c97e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd92000f300 tx=0x7fd92000db90 comp rx=0 tx=0).stop 2023-10-06T11:34:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1905640327 shutdown_connections 2023-10-06T11:34:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92813e8c0 0x7fd9280c97e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1905640327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd92813e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1905640327 >> 172.21.15.46:0/1905640327 conn(0x7fd9280cbc10 msgr2=0x7fd9280cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:23.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1905640327 shutdown_connections 2023-10-06T11:34:23.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1905640327 wait complete. 2023-10-06T11:34:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 Processor -- start 2023-10-06T11:34:23.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.551+0000 7fd92e0e3700 1 -- start start 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd9280ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd92813e8c0 0x7fd928157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280aff30 con 0x7fd9280c6480 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928155ed0 con 0x7fd92813e8c0 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280b03c0 con 0x7fd9280c6850 2023-10-06T11:34:23.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34454/0 (socket says 172.21.15.46:34454) 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 -- 172.21.15.46:0/1409980173 learned_addr learned my addr 172.21.15.46:0/1409980173 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92c8e0700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd9280ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d8e2700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd92813e8c0 0x7fd928157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 -- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 msgr2=0x7fd9280ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd9280ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 -- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd92813e8c0 msgr2=0x7fd928157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd92813e8c0 0x7fd928157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 -- 172.21.15.46:0/1409980173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92000de30 con 0x7fd9280c6480 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92d0e1700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd92400d990 tx=0x7fd9240060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400de00 con 0x7fd9280c6480 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924008950 con 0x7fd9280c6480 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280af320 con 0x7fd9280c6480 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924008d10 con 0x7fd9280c6480 2023-10-06T11:34:23.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9280af8e0 con 0x7fd9280c6480 2023-10-06T11:34:23.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.555+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928064590 con 0x7fd9280c6480 2023-10-06T11:34:23.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.559+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd924010670 con 0x7fd9280c6480 2023-10-06T11:34:23.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.559+0000 7fd91d7fa700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 0x7fd90c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.559+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7fd92407d330 con 0x7fd9280c6480 2023-10-06T11:34:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.559+0000 7fd92c8e0700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 0x7fd90c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.559+0000 7fd92c8e0700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 0x7fd90c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd918005fd0 tx=0x7fd91800a0f0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.563+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd92404d3c0 con 0x7fd9280c6480 2023-10-06T11:34:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.731+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd92813ed40 con 0x7fd9280c6480 2023-10-06T11:34:23.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.743+0000 7fd91d7fa700 1 -- 172.21.15.46:0/1409980173 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd92813ed40 con 0x7fd9280c6480 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 msgr2=0x7fd90c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 0x7fd90c062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd918005fd0 tx=0x7fd91800a0f0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 msgr2=0x7fd928155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd92400d990 tx=0x7fd9240060d0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 shutdown_connections 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd90c060670 0x7fd90c062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd92813e8c0 0x7fd928157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9280c6850 0x7fd9280ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 --2- 172.21.15.46:0/1409980173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9280c6480 0x7fd928155990 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 >> 172.21.15.46:0/1409980173 conn(0x7fd9280cbc10 msgr2=0x7fd9280c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:23.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 shutdown_connections 2023-10-06T11:34:23.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.747+0000 7fd92e0e3700 1 -- 172.21.15.46:0/1409980173 wait complete. 2023-10-06T11:34:23.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-06T11:34:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.939+0000 7f523e865700 1 Processor -- start 2023-10-06T11:34:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.939+0000 7f523e865700 1 -- start start 2023-10-06T11:34:23.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523e865700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52380d0aa0 0x7f5238159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523e865700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52380b5410 0x7f52380b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523e865700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52380b5830 con 0x7f5238159050 2023-10-06T11:34:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523e865700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380b4f80 con 0x7f52380b5410 2023-10-06T11:34:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523e865700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380b40c0 con 0x7f52380d0aa0 2023-10-06T11:34:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52380d0aa0 0x7f5238159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f52380d0aa0 0x7f5238159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44888/0 (socket says 172.21.15.46:44888) 2023-10-06T11:34:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 -- 172.21.15.46:0/4097254228 learned_addr learned my addr 172.21.15.46:0/4097254228 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f523ce02700 1 --2- 172.21.15.46:0/4097254228 >> v2:172.21.15.46:3300/0 conn(0x7f52380b5410 0x7f52380b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 -- 172.21.15.46:0/4097254228 >> v2:172.21.15.46:3300/0 conn(0x7f52380b5410 msgr2=0x7f52380b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:23.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 --2- 172.21.15.46:0/4097254228 >> v2:172.21.15.46:3300/0 conn(0x7f52380b5410 0x7f52380b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 -- 172.21.15.46:0/4097254228 >> v1:172.21.15.132:6789/0 conn(0x7f5238159050 legacy=0x7f52380b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.943+0000 7f52377fe700 1 -- 172.21.15.46:0/4097254228 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380b4370 con 0x7f52380d0aa0 2023-10-06T11:34:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f52377fe700 1 --2- 172.21.15.46:0/4097254228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52380d0aa0 0x7f5238159420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f52380892d0 tx=0x7f522c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b7ca54ebabb876 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f5236ffd700 1 -- 172.21.15.46:0/4097254228 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c009e70 con 0x7f52380d0aa0 2023-10-06T11:34:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f5236ffd700 1 -- 172.21.15.46:0/4097254228 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00b860 con 0x7f52380d0aa0 2023-10-06T11:34:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f5236ffd700 1 -- 172.21.15.46:0/4097254228 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00bc00 con 0x7f52380d0aa0 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 -- 172.21.15.46:0/4097254228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52380d0aa0 msgr2=0x7f5238159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 --2- 172.21.15.46:0/4097254228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52380d0aa0 0x7f5238159420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f52380892d0 tx=0x7f522c009250 comp rx=0 tx=0).stop 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 -- 172.21.15.46:0/4097254228 shutdown_connections 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 --2- 172.21.15.46:0/4097254228 >> v2:172.21.15.46:3300/0 conn(0x7f52380b5410 0x7f52380b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 --2- 172.21.15.46:0/4097254228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52380d0aa0 0x7f5238159420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 -- 172.21.15.46:0/4097254228 >> 172.21.15.46:0/4097254228 conn(0x7f523806e780 msgr2=0x7f523806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 -- 172.21.15.46:0/4097254228 shutdown_connections 2023-10-06T11:34:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.947+0000 7f523e865700 1 -- 172.21.15.46:0/4097254228 wait complete. 2023-10-06T11:34:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 Processor -- start 2023-10-06T11:34:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 -- start start 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52380b5410 0x7f5238155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5238159050 0x7f52380c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238155790 con 0x7f52380d0aa0 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380c96a0 con 0x7f5238159050 2023-10-06T11:34:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.951+0000 7f523e865700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52380c9210 con 0x7f52380b5410 2023-10-06T11:34:23.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34470/0 (socket says 172.21.15.46:34470) 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 -- 172.21.15.46:0/4237451191 learned_addr learned my addr 172.21.15.46:0/4237451191 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f5237fff700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52380b5410 0x7f5238155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f523ce02700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5238159050 0x7f52380c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 -- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52380b5410 msgr2=0x7f5238155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52380b5410 0x7f5238155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 -- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5238159050 msgr2=0x7f52380c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5238159050 0x7f52380c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 -- 172.21.15.46:0/4237451191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0094f0 con 0x7f52380d0aa0 2023-10-06T11:34:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f52377fe700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f522c006fd0 tx=0x7f522c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00b9d0 con 0x7f52380d0aa0 2023-10-06T11:34:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c00bb30 con 0x7f52380d0aa0 2023-10-06T11:34:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c011670 con 0x7f52380d0aa0 2023-10-06T11:34:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523815c920 con 0x7f52380d0aa0 2023-10-06T11:34:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.955+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52380c88e0 con 0x7f52380d0aa0 2023-10-06T11:34:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.959+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238088820 con 0x7f52380d0aa0 2023-10-06T11:34:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.959+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f522c011a60 con 0x7f52380d0aa0 2023-10-06T11:34:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.959+0000 7f521ffff700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 0x7f5224062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.959+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f522c07e450 con 0x7f52380d0aa0 2023-10-06T11:34:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.959+0000 7f5237fff700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 0x7f5224062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:23.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.963+0000 7f5237fff700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 0x7f5224062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5228000c00 tx=0x7f5228007ea0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:23.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:23.967+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f522c04e5f0 con 0x7f52380d0aa0 2023-10-06T11:34:24.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.195+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f52380b46b0 con 0x7f52380d0aa0 2023-10-06T11:34:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.547+0000 7f521ffff700 1 -- 172.21.15.46:0/4237451191 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v603) v1 ==== 166+0+0 (secure 0 0 0) 0x7f522c051c10 con 0x7f52380d0aa0 2023-10-06T11:34:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.551+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 msgr2=0x7f5224062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.551+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 0x7f5224062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5228000c00 tx=0x7f5228007ea0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.551+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 msgr2=0x7f523815c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.551+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f522c006fd0 tx=0x7f522c017000 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 shutdown_connections 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5224060670 0x7f5224062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5238159050 0x7f52380c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52380d0aa0 0x7f523815c3e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 --2- 172.21.15.46:0/4237451191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52380b5410 0x7f5238155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 >> 172.21.15.46:0/4237451191 conn(0x7f523806e780 msgr2=0x7f5238157510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:24.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 shutdown_connections 2023-10-06T11:34:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.555+0000 7f523e865700 1 -- 172.21.15.46:0/4237451191 wait complete. 2023-10-06T11:34:24.586 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-06T11:34:24.586 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-06T11:34:24.586 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:34:24.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-06T11:34:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.739+0000 7feae9478700 1 Processor -- start 2023-10-06T11:34:24.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.739+0000 7feae9478700 1 -- start start 2023-10-06T11:34:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae9478700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 0x7feae4159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae9478700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae415a5d0 con 0x7feae4158ea0 2023-10-06T11:34:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae9478700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae40ce950 con 0x7feae4069770 2023-10-06T11:34:24.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae9478700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40cfdf0 con 0x7feae40b5d00 2023-10-06T11:34:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae2ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feae4158ea0 0x7feae4069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40484/0 (socket says 172.21.15.46:40484) 2023-10-06T11:34:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae2ffd700 1 -- 172.21.15.46:0/1511563867 learned_addr learned my addr 172.21.15.46:0/1511563867 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.743+0000 7feae37fe700 1 --2- 172.21.15.46:0/1511563867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 0x7feae4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:24.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae37fe700 1 -- 172.21.15.46:0/1511563867 >> v1:172.21.15.132:6789/0 conn(0x7feae4069770 legacy=0x7feae4159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae1ffb700 1 -- 172.21.15.46:0/1511563867 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959434187 0 0) 0x7feae40ce950 con 0x7feae4069770 2023-10-06T11:34:24.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae37fe700 1 -- 172.21.15.46:0/1511563867 >> v1:172.21.15.46:6789/0 conn(0x7feae4158ea0 legacy=0x7feae4069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae37fe700 1 -- 172.21.15.46:0/1511563867 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae415a010 con 0x7feae40b5d00 2023-10-06T11:34:24.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae37fe700 1 --2- 172.21.15.46:0/1511563867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 0x7feae4159980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feadc009e90 tx=0x7feadc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aae8ec8e987744e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae1ffb700 1 -- 172.21.15.46:0/1511563867 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00e070 con 0x7feae40b5d00 2023-10-06T11:34:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae1ffb700 1 -- 172.21.15.46:0/1511563867 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00bb30 con 0x7feae40b5d00 2023-10-06T11:34:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae1ffb700 1 -- 172.21.15.46:0/1511563867 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc012430 con 0x7feae40b5d00 2023-10-06T11:34:24.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 -- 172.21.15.46:0/1511563867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 msgr2=0x7feae4159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 --2- 172.21.15.46:0/1511563867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 0x7feae4159980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feadc009e90 tx=0x7feadc009780 comp rx=0 tx=0).stop 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 -- 172.21.15.46:0/1511563867 shutdown_connections 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 --2- 172.21.15.46:0/1511563867 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae40b5d00 0x7feae4159980 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 -- 172.21.15.46:0/1511563867 >> 172.21.15.46:0/1511563867 conn(0x7feae4068390 msgr2=0x7feae40b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 -- 172.21.15.46:0/1511563867 shutdown_connections 2023-10-06T11:34:24.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.747+0000 7feae9478700 1 -- 172.21.15.46:0/1511563867 wait complete. 2023-10-06T11:34:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 Processor -- start 2023-10-06T11:34:24.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 -- start start 2023-10-06T11:34:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae4069770 0x7feae414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40b5d00 0x7feae4153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:24.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4151b00 con 0x7feae4158ea0 2023-10-06T11:34:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4150ef0 con 0x7feae40b5d00 2023-10-06T11:34:24.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae9478700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae41511d0 con 0x7feae4069770 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34472/0 (socket says 172.21.15.46:34472) 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae37fe700 1 -- 172.21.15.46:0/497560600 learned_addr learned my addr 172.21.15.46:0/497560600 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae27fc700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40b5d00 0x7feae4153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.751+0000 7feae2ffd700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae4069770 0x7feae414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 -- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae4069770 msgr2=0x7feae414e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae4069770 0x7feae414e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 -- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40b5d00 msgr2=0x7feae4153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40b5d00 0x7feae4153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:24.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 -- 172.21.15.46:0/497560600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009a20 con 0x7feae4158ea0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae37fe700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feadc003000 tx=0x7feadc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc027470 con 0x7feae4158ea0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc0275f0 con 0x7feae4158ea0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41514b0 con 0x7feae4158ea0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0279b0 con 0x7feae4158ea0 2023-10-06T11:34:24.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae4151f90 con 0x7feae4158ea0 2023-10-06T11:34:24.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.755+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4064590 con 0x7feae4158ea0 2023-10-06T11:34:24.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.759+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7feadc024070 con 0x7feae4158ea0 2023-10-06T11:34:24.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.763+0000 7feacaffd700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 0x7fead0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:24.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.763+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7feadc07f810 con 0x7feae4158ea0 2023-10-06T11:34:24.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.763+0000 7feae2ffd700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 0x7fead0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:24.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.763+0000 7feae2ffd700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 0x7fead0062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fead40058d0 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.767+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feadc04f9b0 con 0x7feae4158ea0 2023-10-06T11:34:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:24.995+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7feae4151c40 con 0x7feae4158ea0 2023-10-06T11:34:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.559+0000 7feacaffd700 1 -- 172.21.15.46:0/497560600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v604) v1 ==== 186+0+0 (secure 0 0 0) 0x7feadc052fd0 con 0x7feae4158ea0 2023-10-06T11:34:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 msgr2=0x7fead0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 0x7fead0062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fead40058d0 tx=0x7fead4009040 comp rx=0 tx=0).stop 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 msgr2=0x7feae41575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7feadc003000 tx=0x7feadc018000 comp rx=0 tx=0).stop 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 shutdown_connections 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fead0060670 0x7fead0062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feae4158ea0 0x7feae41575c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40b5d00 0x7feae4153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 --2- 172.21.15.46:0/497560600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feae4069770 0x7feae414e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 >> 172.21.15.46:0/497560600 conn(0x7feae4068390 msgr2=0x7feae414ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 shutdown_connections 2023-10-06T11:34:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.563+0000 7feae9478700 1 -- 172.21.15.46:0/497560600 wait complete. 2023-10-06T11:34:25.598 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:34:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.751+0000 7feb3359e700 1 Processor -- start 2023-10-06T11:34:25.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.751+0000 7feb3359e700 1 -- start start 2023-10-06T11:34:25.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.751+0000 7feb3359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feb34158e70 0x7feb340a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.751+0000 7feb3359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34159240 0x7feb340c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3359e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb340c10a0 con 0x7feb340c72d0 2023-10-06T11:34:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3359e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb340ca240 con 0x7feb34158e70 2023-10-06T11:34:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340ca520 con 0x7feb34159240 2023-10-06T11:34:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb32d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feb340c72d0 0x7feb340c7a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40490/0 (socket says 172.21.15.46:40490) 2023-10-06T11:34:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb32d9d700 1 -- 172.21.15.46:0/3885062315 learned_addr learned my addr 172.21.15.46:0/3885062315 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:25.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 --2- 172.21.15.46:0/3885062315 >> v2:172.21.15.46:3300/0 conn(0x7feb34158e70 0x7feb340a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb31d9b700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34159240 0x7feb340c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3159a700 1 -- 172.21.15.46:0/3885062315 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855093963 0 0) 0x7feb340c10a0 con 0x7feb340c72d0 2023-10-06T11:34:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3159a700 1 -- 172.21.15.46:0/3885062315 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb1c003570 con 0x7feb340c72d0 2023-10-06T11:34:25.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 -- 172.21.15.46:0/3885062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34159240 msgr2=0x7feb340c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34159240 0x7feb340c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 -- 172.21.15.46:0/3885062315 >> v1:172.21.15.46:6789/0 conn(0x7feb340c72d0 legacy=0x7feb340c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 -- 172.21.15.46:0/3885062315 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340c0a50 con 0x7feb34158e70 2023-10-06T11:34:25.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3259c700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb34158e70 0x7feb340a9840 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb240098e0 tx=0x7feb240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478533dfd612092 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3159a700 1 -- 172.21.15.46:0/3885062315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011630 con 0x7feb34158e70 2023-10-06T11:34:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3159a700 1 -- 172.21.15.46:0/3885062315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24011790 con 0x7feb34158e70 2023-10-06T11:34:25.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3159a700 1 -- 172.21.15.46:0/3885062315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011b30 con 0x7feb34158e70 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3359e700 1 -- 172.21.15.46:0/3885062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb34158e70 msgr2=0x7feb340a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.755+0000 7feb3359e700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb34158e70 0x7feb340a9840 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb240098e0 tx=0x7feb240091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 -- 172.21.15.46:0/3885062315 shutdown_connections 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34159240 0x7feb340c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 --2- 172.21.15.46:0/3885062315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb34158e70 0x7feb340a9840 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 -- 172.21.15.46:0/3885062315 >> 172.21.15.46:0/3885062315 conn(0x7feb340cbc10 msgr2=0x7feb340cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 -- 172.21.15.46:0/3885062315 shutdown_connections 2023-10-06T11:34:25.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 -- 172.21.15.46:0/3885062315 wait complete. 2023-10-06T11:34:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 Processor -- start 2023-10-06T11:34:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.759+0000 7feb3359e700 1 -- start start 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34158e70 0x7feb341535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 0x7feb341572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3416a310 con 0x7feb340c72d0 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340b0aa0 con 0x7feb34159240 2023-10-06T11:34:25.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340afe90 con 0x7feb34158e70 2023-10-06T11:34:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34478/0 (socket says 172.21.15.46:34478) 2023-10-06T11:34:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 -- 172.21.15.46:0/2017081656 learned_addr learned my addr 172.21.15.46:0/2017081656 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb31d9b700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34158e70 0x7feb341535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb32d9d700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 0x7feb341572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 -- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34158e70 msgr2=0x7feb341535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34158e70 0x7feb341535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 -- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 msgr2=0x7feb341572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 0x7feb341572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 -- 172.21.15.46:0/2017081656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24009420 con 0x7feb340c72d0 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb32d9d700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 0x7feb341572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3259c700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7feb340c9790 tx=0x7feb24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011b10 con 0x7feb340c72d0 2023-10-06T11:34:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb240253f0 con 0x7feb340c72d0 2023-10-06T11:34:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb340b0170 con 0x7feb340c72d0 2023-10-06T11:34:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24025790 con 0x7feb340c72d0 2023-10-06T11:34:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.763+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb340b0730 con 0x7feb340c72d0 2023-10-06T11:34:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.767+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb34064590 con 0x7feb340c72d0 2023-10-06T11:34:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.767+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7feb24025b80 con 0x7feb340c72d0 2023-10-06T11:34:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.767+0000 7feb227fc700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 0x7feb14062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.771+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7feb2407e490 con 0x7feb340c72d0 2023-10-06T11:34:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.771+0000 7feb31d9b700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 0x7feb14062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.771+0000 7feb31d9b700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 0x7feb14062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb341542e0 tx=0x7feb28008040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.775+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feb2404e630 con 0x7feb340c72d0 2023-10-06T11:34:25.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.947+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feb3404b5c0 con 0x7feb340c72d0 2023-10-06T11:34:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.955+0000 7feb227fc700 1 -- 172.21.15.46:0/2017081656 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb24051c50 con 0x7feb340c72d0 2023-10-06T11:34:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 msgr2=0x7feb14062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 0x7feb14062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb341542e0 tx=0x7feb28008040 comp rx=0 tx=0).stop 2023-10-06T11:34:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 msgr2=0x7feb34169dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7feb340c9790 tx=0x7feb24017000 comp rx=0 tx=0).stop 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 shutdown_connections 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7feb140606a0 0x7feb14062b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34159240 0x7feb341572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb34158e70 0x7feb341535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 --2- 172.21.15.46:0/2017081656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb340c72d0 0x7feb34169dd0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 >> 172.21.15.46:0/2017081656 conn(0x7feb340cbc10 msgr2=0x7feb340c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 shutdown_connections 2023-10-06T11:34:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:25.959+0000 7feb3359e700 1 -- 172.21.15.46:0/2017081656 wait complete. 2023-10-06T11:34:25.987 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:34:25.987 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:34:26.029 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:34:26.073 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:34:26.073 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:34:26.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:26.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.239+0000 7f875b06e700 1 Processor -- start 2023-10-06T11:34:26.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.239+0000 7f875b06e700 1 -- start start 2023-10-06T11:34:26.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875b06e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87540cec00 0x7f8754163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875b06e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540cc150 con 0x7f875415c770 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875b06e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87540cb540 con 0x7f87540c5ff0 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875b06e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87540cfbb0 con 0x7f87540cec00 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f8758e0a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f875415c770 0x7f87540cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40500/0 (socket says 172.21.15.46:40500) 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f8758e0a700 1 -- 172.21.15.46:0/343858089 learned_addr learned my addr 172.21.15.46:0/343858089 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:26.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875960b700 1 --2- 172.21.15.46:0/343858089 >> v2:172.21.15.132:3300/0 conn(0x7f87540cec00 0x7f8754163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235262648 0 0) 0x7f87540cc150 con 0x7f875415c770 2023-10-06T11:34:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8744003570 con 0x7f875415c770 2023-10-06T11:34:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361900403 0 0) 0x7f87540cb540 con 0x7f87540c5ff0 2023-10-06T11:34:26.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87540cc150 con 0x7f87540c5ff0 2023-10-06T11:34:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875960b700 1 -- 172.21.15.46:0/343858089 >> v1:172.21.15.132:6789/0 conn(0x7f87540c5ff0 legacy=0x7f87540ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875960b700 1 -- 172.21.15.46:0/343858089 >> v1:172.21.15.46:6789/0 conn(0x7f875415c770 legacy=0x7f87540cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875960b700 1 -- 172.21.15.46:0/343858089 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540cb820 con 0x7f87540cec00 2023-10-06T11:34:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f875960b700 1 --2- 172.21.15.46:0/343858089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540cec00 0x7f8754163e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f874c009e60 tx=0x7f874c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a435c7cb7ccb98f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00e070 con 0x7f87540cec00 2023-10-06T11:34:26.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c00bab0 con 0x7f87540cec00 2023-10-06T11:34:26.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.243+0000 7f87537fe700 1 -- 172.21.15.46:0/343858089 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00be70 con 0x7f87540cec00 2023-10-06T11:34:26.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 -- 172.21.15.46:0/343858089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540cec00 msgr2=0x7f8754163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 --2- 172.21.15.46:0/343858089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540cec00 0x7f8754163e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f874c009e60 tx=0x7f874c009780 comp rx=0 tx=0).stop 2023-10-06T11:34:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 -- 172.21.15.46:0/343858089 shutdown_connections 2023-10-06T11:34:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 --2- 172.21.15.46:0/343858089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540cec00 0x7f8754163e60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 -- 172.21.15.46:0/343858089 >> 172.21.15.46:0/343858089 conn(0x7f8754099480 msgr2=0x7f875406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 -- 172.21.15.46:0/343858089 shutdown_connections 2023-10-06T11:34:26.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.247+0000 7f875b06e700 1 -- 172.21.15.46:0/343858089 wait complete. 2023-10-06T11:34:26.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 Processor -- start 2023-10-06T11:34:26.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 -- start start 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 0x7f875414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cec00 0x7f875415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754159df0 con 0x7f875415c770 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875414f370 con 0x7f87540c5ff0 2023-10-06T11:34:26.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875415a040 con 0x7f87540cec00 2023-10-06T11:34:26.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34484/0 (socket says 172.21.15.46:34484) 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 -- 172.21.15.46:0/2603574230 learned_addr learned my addr 172.21.15.46:0/2603574230 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8758e0a700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 0x7f875414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8753fff700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cec00 0x7f875415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 -- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cec00 msgr2=0x7f875415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cec00 0x7f875415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 -- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 msgr2=0x7f875414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 0x7f875414ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 -- 172.21.15.46:0/2603574230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c009a20 con 0x7f875415c770 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8758e0a700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 0x7f875414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:26.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875960b700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f874c013000 tx=0x7f874c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c026480 con 0x7f875415c770 2023-10-06T11:34:26.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c026600 con 0x7f875415c770 2023-10-06T11:34:26.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875415a320 con 0x7f875415c770 2023-10-06T11:34:26.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.251+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c0269c0 con 0x7f875415c770 2023-10-06T11:34:26.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.255+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875415a8e0 con 0x7f875415c770 2023-10-06T11:34:26.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.255+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87540660f0 con 0x7f875415c770 2023-10-06T11:34:26.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.255+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f874c01c400 con 0x7f875415c770 2023-10-06T11:34:26.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.259+0000 7f8750ff9700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 0x7f8738062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.259+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7f874c07eff0 con 0x7f875415c770 2023-10-06T11:34:26.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.259+0000 7f8758e0a700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 0x7f8738062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.259+0000 7f8758e0a700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 0x7f8738062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87480079e0 tx=0x7f8748009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.263+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f874c04f0e0 con 0x7f875415c770 2023-10-06T11:34:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.463+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f875406f090 con 0x7f875415c770 2023-10-06T11:34:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f8750ff9700 1 -- 172.21.15.46:0/2603574230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f874c052700 con 0x7f875415c770 2023-10-06T11:34:26.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:26.476 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4189338183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4189338183},{"type":"v1","addr":"172.21.15.46:6837","nonce":4189338183}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1425073997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1425073997},{"type":"v1","addr":"172.21.15.132:6834","nonce":1425073997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2981847066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2981847066},{"type":"v1","addr":"172.21.15.132:6835","nonce":2981847066}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":7215,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3509314746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3509314746},{"type":"v1","addr":"172.21.15.46:6835","nonce":3509314746}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[]} 2023-10-06T11:34:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 msgr2=0x7f8738062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 0x7f8738062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87480079e0 tx=0x7f8748009040 comp rx=0 tx=0).stop 2023-10-06T11:34:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 msgr2=0x7f875415ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f874c013000 tx=0x7f874c019000 comp rx=0 tx=0).stop 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 shutdown_connections 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8738060670 0x7f8738062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f875415c770 0x7f875415ed40 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87540cec00 0x7f875415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 --2- 172.21.15.46:0/2603574230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87540c5ff0 0x7f875414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 >> 172.21.15.46:0/2603574230 conn(0x7f8754099480 msgr2=0x7f87540cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 shutdown_connections 2023-10-06T11:34:26.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.467+0000 7f875b06e700 1 -- 172.21.15.46:0/2603574230 wait complete. 2023-10-06T11:34:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 44 2023-10-06T11:34:26.501 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:34:26.501 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:34:26.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:34:26.502 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:34:26.502 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:34:26.502 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:34:26.503 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:34:26.503 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:34:26.503 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:34:26.503 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:34:26.503 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:34:26.503 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:34:26.503 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:34:26.503 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:34:26.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:34:26.504 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:34:26.518 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:34:26.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:34:26.519 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:34:26.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:34:26.527 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:34:26.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:34:26.530 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:34:26.530 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:34:26.569 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:34:26.570 DEBUG:teuthology.parallel:result is None 2023-10-06T11:34:26.575 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:34:26.575 DEBUG:teuthology.parallel:result is None 2023-10-06T11:34:26.576 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:34:26.576 DEBUG:teuthology.parallel:result is None 2023-10-06T11:34:26.580 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:34:26.580 DEBUG:teuthology.parallel:result is None 2023-10-06T11:34:26.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:34:26.673 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:34:26.676 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:34:26.680 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:34:26.680 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:34:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.731+0000 7fe8dec8e700 1 Processor -- start 2023-10-06T11:34:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.731+0000 7fe8dec8e700 1 -- start start 2023-10-06T11:34:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dec8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8d80bedd0 0x7fe8d8067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dec8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80bf1a0 0x7fe8d8067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dec8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d8069660 con 0x7fe8d80c7a00 2023-10-06T11:34:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dec8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d8068ac0 con 0x7fe8d80bedd0 2023-10-06T11:34:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dec8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8068d30 con 0x7fe8d80bf1a0 2023-10-06T11:34:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8d80bedd0 0x7fe8d8067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8de48d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe8d80c7a00 0x7fe8d80c7ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40524/0 (socket says 172.21.15.46:40524) 2023-10-06T11:34:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8de48d700 1 -- 172.21.15.46:0/161719020 learned_addr learned my addr 172.21.15.46:0/161719020 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dd48b700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80bf1a0 0x7fe8d8067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dcc8a700 1 -- 172.21.15.46:0/161719020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526080173 0 0) 0x7fe8d8069660 con 0x7fe8d80c7a00 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dcc8a700 1 -- 172.21.15.46:0/161719020 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8c4003570 con 0x7fe8d80c7a00 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/161719020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80bf1a0 msgr2=0x7fe8d8067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80bf1a0 0x7fe8d8067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/161719020 >> v1:172.21.15.46:6789/0 conn(0x7fe8d80c7a00 legacy=0x7fe8d80c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/161719020 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8069010 con 0x7fe8d80bedd0 2023-10-06T11:34:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8ddc8c700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d8067050 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40098e0 tx=0x7fe8d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116da8b12705a9b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dcc8a700 1 -- 172.21.15.46:0/161719020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011630 con 0x7fe8d80bedd0 2023-10-06T11:34:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dcc8a700 1 -- 172.21.15.46:0/161719020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4011790 con 0x7fe8d80bedd0 2023-10-06T11:34:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.735+0000 7fe8dcc8a700 1 -- 172.21.15.46:0/161719020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011b30 con 0x7fe8d80bedd0 2023-10-06T11:34:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 -- 172.21.15.46:0/161719020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 msgr2=0x7fe8d8067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d8067050 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40098e0 tx=0x7fe8d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 -- 172.21.15.46:0/161719020 shutdown_connections 2023-10-06T11:34:26.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80bf1a0 0x7fe8d8067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/161719020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d8067050 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 -- 172.21.15.46:0/161719020 >> 172.21.15.46:0/161719020 conn(0x7fe8d8062ef0 msgr2=0x7fe8d80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 -- 172.21.15.46:0/161719020 shutdown_connections 2023-10-06T11:34:26.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.739+0000 7fe8dec8e700 1 -- 172.21.15.46:0/161719020 wait complete. 2023-10-06T11:34:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 Processor -- start 2023-10-06T11:34:26.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 -- start start 2023-10-06T11:34:26.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 0x7fe8d80c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80c7a00 0x7fe8d8161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d815f7c0 con 0x7fe8d80bedd0 2023-10-06T11:34:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c6710 con 0x7fe8d80bf1a0 2023-10-06T11:34:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dec8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c69f0 con 0x7fe8d80c7a00 2023-10-06T11:34:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34530/0 (socket says 172.21.15.46:34530) 2023-10-06T11:34:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/2565665224 learned_addr learned my addr 172.21.15.46:0/2565665224 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dd48b700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 0x7fe8d80c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8de48d700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80c7a00 0x7fe8d8161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80c7a00 msgr2=0x7fe8d8161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80c7a00 0x7fe8d8161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 msgr2=0x7fe8d80c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 0x7fe8d80c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8ddc8c700 1 -- 172.21.15.46:0/2565665224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d4009420 con 0x7fe8d80bedd0 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.743+0000 7fe8dd48b700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 0x7fe8d80c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:26.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8ddc8c700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe8d806ab70 tx=0x7fe8d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4011b10 con 0x7fe8d80bedd0 2023-10-06T11:34:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4000cc0 con 0x7fe8d80bedd0 2023-10-06T11:34:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d401a660 con 0x7fe8d80bedd0 2023-10-06T11:34:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d815f170 con 0x7fe8d80bedd0 2023-10-06T11:34:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d80c3b80 con 0x7fe8d80bedd0 2023-10-06T11:34:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.747+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d8064590 con 0x7fe8d80bedd0 2023-10-06T11:34:26.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.751+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe8d401aa90 con 0x7fe8d80bedd0 2023-10-06T11:34:26.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.751+0000 7fe8cdffb700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 0x7fe8bc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:26.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.751+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6381+0+0 (secure 0 0 0) 0x7fe8d407e830 con 0x7fe8d80bedd0 2023-10-06T11:34:26.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.751+0000 7fe8dd48b700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 0x7fe8bc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:26.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.751+0000 7fe8dd48b700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 0x7fe8bc062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80ca860 tx=0x7fe8c8008040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.759+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe8d404ea10 con 0x7fe8d80bedd0 2023-10-06T11:34:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:26.943+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8d804b5c0 con 0x7fe8d80bedd0 2023-10-06T11:34:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.523+0000 7fe8cdffb700 1 -- 172.21.15.46:0/2565665224 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8d4051ff0 con 0x7fe8d80bedd0 2023-10-06T11:34:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 msgr2=0x7fe8bc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 0x7fe8bc062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80ca860 tx=0x7fe8c8008040 comp rx=0 tx=0).stop 2023-10-06T11:34:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 msgr2=0x7fe8d815b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe8d806ab70 tx=0x7fe8d4017000 comp rx=0 tx=0).stop 2023-10-06T11:34:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 shutdown_connections 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe8bc060670 0x7fe8bc062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8d80c7a00 0x7fe8d8161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8d80bf1a0 0x7fe8d80c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 --2- 172.21.15.46:0/2565665224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8d80bedd0 0x7fe8d815b2d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 >> 172.21.15.46:0/2565665224 conn(0x7fe8d8062ef0 msgr2=0x7fe8d815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 shutdown_connections 2023-10-06T11:34:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.527+0000 7fe8dec8e700 1 -- 172.21.15.46:0/2565665224 wait complete. 2023-10-06T11:34:28.538 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:34:28.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:34:28.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.707+0000 7f5a1d4cd700 1 Processor -- start 2023-10-06T11:34:28.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 -- start start 2023-10-06T11:34:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5a1815ab60 0x7f5a1806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a1815af30 0x7f5a1806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 0x7f5a180adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1806d460 con 0x7f5a1815af30 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a180c7000 con 0x7f5a1815ab60 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a1d4cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180c72c0 con 0x7f5a180ad6c0 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a0ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a1815af30 0x7f5a1806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a0ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5a1815af30 0x7f5a1806cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34540/0 (socket says 172.21.15.46:34540) 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.711+0000 7f5a0ffff700 1 -- 172.21.15.46:0/290226379 learned_addr learned my addr 172.21.15.46:0/290226379 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:28.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a16ffd700 1 --2- 172.21.15.46:0/290226379 >> v2:172.21.15.132:3300/0 conn(0x7f5a1815ab60 0x7f5a1806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a177fe700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 0x7f5a180adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 -- 172.21.15.46:0/290226379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 msgr2=0x7f5a180adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 0x7f5a180adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 -- 172.21.15.46:0/290226379 >> v2:172.21.15.132:3300/0 conn(0x7f5a1815ab60 msgr2=0x7f5a1806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/290226379 >> v2:172.21.15.132:3300/0 conn(0x7f5a1815ab60 0x7f5a1806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 -- 172.21.15.46:0/290226379 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180c75a0 con 0x7f5a1815af30 2023-10-06T11:34:28.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a177fe700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 0x7f5a180adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815af30 0x7f5a1806cc30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5a18063be0 tx=0x7f5a00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b517b76f7d7531 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a167fc700 1 -- 172.21.15.46:0/290226379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000b860 con 0x7f5a1815af30 2023-10-06T11:34:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a167fc700 1 -- 172.21.15.46:0/290226379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000b9c0 con 0x7f5a1815af30 2023-10-06T11:34:28.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a167fc700 1 -- 172.21.15.46:0/290226379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000bd60 con 0x7f5a1815af30 2023-10-06T11:34:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/290226379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815af30 msgr2=0x7f5a1806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815af30 0x7f5a1806cc30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5a18063be0 tx=0x7f5a00009250 comp rx=0 tx=0).stop 2023-10-06T11:34:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/290226379 shutdown_connections 2023-10-06T11:34:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a180ad6c0 0x7f5a180adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/290226379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815af30 0x7f5a1806cc30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/290226379 >> v2:172.21.15.132:3300/0 conn(0x7f5a1815ab60 0x7f5a1806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/290226379 >> 172.21.15.46:0/290226379 conn(0x7f5a180cbe30 msgr2=0x7f5a180cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/290226379 shutdown_connections 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.715+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/290226379 wait complete. 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 Processor -- start 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 -- start start 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a180ad6c0 0x7f5a180cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 0x7f5a1816c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180cb910 con 0x7f5a1815ab60 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180aa300 con 0x7f5a180ad6c0 2023-10-06T11:34:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a1d4cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a180aa5e0 con 0x7f5a1815af30 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34550/0 (socket says 172.21.15.46:34550) 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 -- 172.21.15.46:0/2796312086 learned_addr learned my addr 172.21.15.46:0/2796312086 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a16ffd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a180ad6c0 0x7f5a180cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a177fe700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 0x7f5a1816c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 -- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 msgr2=0x7f5a1816c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 0x7f5a1816c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 -- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a180ad6c0 msgr2=0x7f5a180cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a180ad6c0 0x7f5a180cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 -- 172.21.15.46:0/2796312086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000094f0 con 0x7f5a1815ab60 2023-10-06T11:34:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a177fe700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 0x7f5a1816c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.719+0000 7f5a0ffff700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5a1806bf10 tx=0x7f5a00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001a450 con 0x7f5a1815ab60 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0001a5b0 con 0x7f5a1815ab60 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001a970 con 0x7f5a1815ab60 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a180aa8c0 con 0x7f5a1815ab60 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1816d460 con 0x7f5a1815ab60 2023-10-06T11:34:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a180ab0c0 con 0x7f5a1815ab60 2023-10-06T11:34:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.723+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5a0001ad20 con 0x7f5a1815ab60 2023-10-06T11:34:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.727+0000 7f5a0effd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 0x7f59f8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.727+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f5a0007e4e0 con 0x7f5a1815ab60 2023-10-06T11:34:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.727+0000 7f5a16ffd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 0x7f59f8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:28.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.727+0000 7f5a16ffd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 0x7f59f8062b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a080061f0 tx=0x7f5a08008040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.731+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5a180ab0c0 con 0x7f5a1815ab60 2023-10-06T11:34:28.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.963+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5a180ab0c0 con 0x7f5a1815ab60 2023-10-06T11:34:28.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.963+0000 7f5a0effd700 1 -- 172.21.15.46:0/2796312086 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5a180ab0c0 con 0x7f5a1815ab60 2023-10-06T11:34:28.974 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:28.974 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:34:28.975 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:34:28.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:34:28.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:34:28.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.980 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:34:28.982 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:34:28.985 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:34:28.987 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:34:28.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.989 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:34:28.989 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.990 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.991 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.991 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:34:28.991 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:34:28.991 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:34:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:34:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:34:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:34:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:34:28.994 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:34:28.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.994 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:28.995 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:34:28.995 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 msgr2=0x7f59f8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 0x7f59f8062b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a080061f0 tx=0x7f5a08008040 comp rx=0 tx=0).stop 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 msgr2=0x7f5a180a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5a1806bf10 tx=0x7f5a00017000 comp rx=0 tx=0).stop 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 shutdown_connections 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f59f80606a0 0x7f59f8062b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5a1815af30 0x7f5a1816c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a1815ab60 0x7f5a180a9840 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 --2- 172.21.15.46:0/2796312086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a180ad6c0 0x7f5a180cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 >> 172.21.15.46:0/2796312086 conn(0x7f5a180cbe30 msgr2=0x7f5a180ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 shutdown_connections 2023-10-06T11:34:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:28.967+0000 7f5a1d4cd700 1 -- 172.21.15.46:0/2796312086 wait complete. 2023-10-06T11:34:29.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:29.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 Processor -- start 2023-10-06T11:34:29.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 -- start start 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd3600c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd36015a1a0 0x7fd3600c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3600c67e0 con 0x7fd3601506c0 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36006e810 con 0x7fd36015a1a0 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd365878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006eaf0 con 0x7fd3601502f0 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd35e7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3601506c0 0x7fd360159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40528/0 (socket says 172.21.15.46:40528) 2023-10-06T11:34:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.159+0000 7fd35e7fc700 1 -- 172.21.15.46:0/3318801298 learned_addr learned my addr 172.21.15.46:0/3318801298 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35effd700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd3600c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3318801298 >> v2:172.21.15.132:3300/0 conn(0x7fd36015a1a0 0x7fd3600c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107676775 0 0) 0x7fd3600c67e0 con 0x7fd3601506c0 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd348003570 con 0x7fd3601506c0 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2095642443 0 0) 0x7fd348003570 con 0x7fd3601506c0 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 msgr2=0x7fd3600c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd3600c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 >> v2:172.21.15.132:3300/0 conn(0x7fd36015a1a0 msgr2=0x7fd3600c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 --2- 172.21.15.46:0/3318801298 >> v2:172.21.15.132:3300/0 conn(0x7fd36015a1a0 0x7fd3600c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36006edd0 con 0x7fd3601506c0 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fd354001470 con 0x7fd3601506c0 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 >> v1:172.21.15.46:6789/0 conn(0x7fd3601506c0 legacy=0x7fd360159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd348003a60 0x7fd348003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd348004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd3480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006e810 con 0x7fd348004d50 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006eaf0 con 0x7fd348003a60 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34800ba60 con 0x7fd3601502f0 2023-10-06T11:34:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd3480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35effd700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd348003a60 0x7fd348003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35e7fc700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd348004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 -- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 msgr2=0x7fd348004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601502f0 0x7fd348004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 -- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd348003a60 msgr2=0x7fd348003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd348003a60 0x7fd348003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 -- 172.21.15.46:0/3318801298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348003570 con 0x7fd348004d50 2023-10-06T11:34:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35effd700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd348003a60 0x7fd348003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd3480051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd3600c6420 tx=0x7fd35800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bf77a485c117 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ea00 con 0x7fd348004d50 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800f040 con 0x7fd348004d50 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35dffb700 1 -- 172.21.15.46:0/3318801298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800edb0 con 0x7fd348004d50 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd365878700 1 -- 172.21.15.46:0/3318801298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 msgr2=0x7fd3480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd365878700 1 --2- 172.21.15.46:0/3318801298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd3480051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd3600c6420 tx=0x7fd35800b040 comp rx=0 tx=0).stop 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.163+0000 7fd35f7fe700 1 -- 172.21.15.46:0/3318801298 reap_dead start 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- 172.21.15.46:0/3318801298 shutdown_connections 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- 172.21.15.46:0/3318801298 >> 172.21.15.46:0/3318801298 conn(0x7fd3600cbe30 msgr2=0x7fd3600cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- 172.21.15.46:0/3318801298 shutdown_connections 2023-10-06T11:34:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- 172.21.15.46:0/3318801298 wait complete. 2023-10-06T11:34:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 Processor -- start 2023-10-06T11:34:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- start start 2023-10-06T11:34:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601683a0 0x7fd3600ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3600abbe0 0x7fd3600ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600ac050 con 0x7fd348004d50 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600ae5e0 con 0x7fd3600abbe0 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.167+0000 7fd365878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3601577a0 con 0x7fd3601683a0 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34568/0 (socket says 172.21.15.46:34568) 2023-10-06T11:34:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 -- 172.21.15.46:0/3850961035 learned_addr learned my addr 172.21.15.46:0/3850961035 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35f7fe700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3600abbe0 0x7fd3600ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35e7fc700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601683a0 0x7fd3600ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 -- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601683a0 msgr2=0x7fd3600ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601683a0 0x7fd3600ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 -- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3600abbe0 msgr2=0x7fd3600ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3600abbe0 0x7fd3600ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 -- 172.21.15.46:0/3850961035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd360166d50 con 0x7fd348004d50 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd35effd700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd350011960 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350011ea0 con 0x7fd348004d50 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd360157a40 con 0x7fd348004d50 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35000f8e0 con 0x7fd348004d50 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000fca0 con 0x7fd348004d50 2023-10-06T11:34:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd360158000 con 0x7fd348004d50 2023-10-06T11:34:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.171+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360063f90 con 0x7fd348004d50 2023-10-06T11:34:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.175+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd350015670 con 0x7fd348004d50 2023-10-06T11:34:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.175+0000 7fd346ffd700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 0x7fd34c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.175+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fd35007cd50 con 0x7fd348004d50 2023-10-06T11:34:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.175+0000 7fd35e7fc700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 0x7fd34c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.179+0000 7fd35e7fc700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 0x7fd34c062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd354001050 tx=0x7fd354009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.183+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd35004d1c0 con 0x7fd348004d50 2023-10-06T11:34:29.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.367+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3600c6470 con 0x7fd348004d50 2023-10-06T11:34:29.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.371+0000 7fd346ffd700 1 -- 172.21.15.46:0/3850961035 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14323 (secure 0 0 0) 0x7fd3500507e0 con 0x7fd348004d50 2023-10-06T11:34:29.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:29.382 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:28.513732+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 msgr2=0x7fd34c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 0x7fd34c062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd354001050 tx=0x7fd354009040 comp rx=0 tx=0).stop 2023-10-06T11:34:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 msgr2=0x7fd360166810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd350011960 tx=0x7fd350008040 comp rx=0 tx=0).stop 2023-10-06T11:34:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 shutdown_connections 2023-10-06T11:34:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd34c060670 0x7fd34c062b20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3600abbe0 0x7fd3600ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3601683a0 0x7fd3600ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 --2- 172.21.15.46:0/3850961035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd348004d50 0x7fd360166810 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 >> 172.21.15.46:0/3850961035 conn(0x7fd3600cbe30 msgr2=0x7fd360068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 shutdown_connections 2023-10-06T11:34:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.375+0000 7fd365878700 1 -- 172.21.15.46:0/3850961035 wait complete. 2023-10-06T11:34:29.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:34:29.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:34:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.563+0000 7fe802985700 1 Processor -- start 2023-10-06T11:34:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.563+0000 7fe802985700 1 -- start start 2023-10-06T11:34:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe802985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7fc161fc0 0x7fe7fc0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe802985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc147280 0x7fe7fc06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe802985700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc0c1980 con 0x7fe7fc146eb0 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe802985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0c2520 con 0x7fe7fc161fc0 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe802985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cb1e0 con 0x7fe7fc147280 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7fc161fc0 0x7fe7fc0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7fc161fc0 0x7fe7fc0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45012/0 (socket says 172.21.15.46:45012) 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 -- 172.21.15.46:0/2808956319 learned_addr learned my addr 172.21.15.46:0/2808956319 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe800f22700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc147280 0x7fe7fc06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 -- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc147280 msgr2=0x7fe7fc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc147280 0x7fe7fc06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 -- 172.21.15.46:0/2808956319 >> v1:172.21.15.132:6789/0 conn(0x7fe7fc146eb0 legacy=0x7fe7fc0c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 -- 172.21.15.46:0/2808956319 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0c0d70 con 0x7fe7fc161fc0 2023-10-06T11:34:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.567+0000 7fe7f37fe700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0caca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40099b0 tx=0x7fe7e4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd566377f00c067 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe7fb7fe700 1 -- 172.21.15.46:0/2808956319 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4009ea0 con 0x7fe7fc161fc0 2023-10-06T11:34:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe7fb7fe700 1 -- 172.21.15.46:0/2808956319 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e400b860 con 0x7fe7fc161fc0 2023-10-06T11:34:29.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe7fb7fe700 1 -- 172.21.15.46:0/2808956319 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400bc00 con 0x7fe7fc161fc0 2023-10-06T11:34:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 -- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc161fc0 msgr2=0x7fe7fc0caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0caca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40099b0 tx=0x7fe7e4009250 comp rx=0 tx=0).stop 2023-10-06T11:34:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 -- 172.21.15.46:0/2808956319 shutdown_connections 2023-10-06T11:34:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc147280 0x7fe7fc06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 --2- 172.21.15.46:0/2808956319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0caca0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 -- 172.21.15.46:0/2808956319 >> 172.21.15.46:0/2808956319 conn(0x7fe7fc0cbe30 msgr2=0x7fe7fc0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:29.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 -- 172.21.15.46:0/2808956319 shutdown_connections 2023-10-06T11:34:29.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 -- 172.21.15.46:0/2808956319 wait complete. 2023-10-06T11:34:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.571+0000 7fe802985700 1 Processor -- start 2023-10-06T11:34:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- start start 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc146eb0 0x7fe7fc06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc147280 0x7fe7fc142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc06da30 con 0x7fe7fc161fc0 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc06e860 con 0x7fe7fc147280 2023-10-06T11:34:29.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1604b0 con 0x7fe7fc146eb0 2023-10-06T11:34:29.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34572/0 (socket says 172.21.15.46:34572) 2023-10-06T11:34:29.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 -- 172.21.15.46:0/392889805 learned_addr learned my addr 172.21.15.46:0/392889805 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe7f37fe700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc147280 0x7fe7fc142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe7fbfff700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc146eb0 0x7fe7fc06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 -- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc146eb0 msgr2=0x7fe7fc06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc146eb0 0x7fe7fc06d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 -- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc147280 msgr2=0x7fe7fc142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc147280 0x7fe7fc142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 -- 172.21.15.46:0/392889805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e40094f0 con 0x7fe7fc161fc0 2023-10-06T11:34:29.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe800f22700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009ec0 tx=0x7fe7f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400cc20 con 0x7fe7fc161fc0 2023-10-06T11:34:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400f040 con 0x7fe7fc161fc0 2023-10-06T11:34:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc15f8a0 con 0x7fe7fc161fc0 2023-10-06T11:34:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4014480 con 0x7fe7fc161fc0 2023-10-06T11:34:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.575+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc15fe60 con 0x7fe7fc161fc0 2023-10-06T11:34:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.579+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc160640 con 0x7fe7fc161fc0 2023-10-06T11:34:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.583+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe7f40147f0 con 0x7fe7fc161fc0 2023-10-06T11:34:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.583+0000 7fe7f8ff9700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 0x7fe7dc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.583+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fe7f407d700 con 0x7fe7fc161fc0 2023-10-06T11:34:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.583+0000 7fe7fbfff700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 0x7fe7dc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.583+0000 7fe7fbfff700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 0x7fe7dc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0037c0 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.587+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe7f404da60 con 0x7fe7fc161fc0 2023-10-06T11:34:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.759+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7fc0660f0 con 0x7fe7fc161fc0 2023-10-06T11:34:29.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.759+0000 7fe7f8ff9700 1 -- 172.21.15.46:0/392889805 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7f4051080 con 0x7fe7fc161fc0 2023-10-06T11:34:29.770 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:34:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 msgr2=0x7fe7dc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 0x7fe7dc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0037c0 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2023-10-06T11:34:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 msgr2=0x7fe7fc0ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009ec0 tx=0x7fe7f400b040 comp rx=0 tx=0).stop 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 shutdown_connections 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7dc060670 0x7fe7dc062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7fc161fc0 0x7fe7fc0ca310 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7fc147280 0x7fe7fc142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 --2- 172.21.15.46:0/392889805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7fc146eb0 0x7fe7fc06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 >> 172.21.15.46:0/392889805 conn(0x7fe7fc0cbe30 msgr2=0x7fe7fc143f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 shutdown_connections 2023-10-06T11:34:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.763+0000 7fe802985700 1 -- 172.21.15.46:0/392889805 wait complete. 2023-10-06T11:34:29.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:34:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.951+0000 7f56b2ecd700 1 Processor -- start 2023-10-06T11:34:29.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.951+0000 7f56b2ecd700 1 -- start start 2023-10-06T11:34:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56ac0c7d80 0x7f56ac0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ac0ca430 0x7f56ac0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56ac0c1630 con 0x7f56ac0a9840 2023-10-06T11:34:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0c0d00 con 0x7f56ac0ca430 2023-10-06T11:34:29.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56ac0c0fe0 con 0x7f56ac0c7d80 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ac0ca430 0x7f56ac0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56ac0ca430 0x7f56ac0c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34584/0 (socket says 172.21.15.46:34584) 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 -- 172.21.15.46:0/3987882339 learned_addr learned my addr 172.21.15.46:0/3987882339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b0c69700 1 --2- 172.21.15.46:0/3987882339 >> v2:172.21.15.132:3300/0 conn(0x7f56ac0c7d80 0x7f56ac0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 -- 172.21.15.46:0/3987882339 >> v2:172.21.15.132:3300/0 conn(0x7f56ac0c7d80 msgr2=0x7f56ac0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:29.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 --2- 172.21.15.46:0/3987882339 >> v2:172.21.15.132:3300/0 conn(0x7f56ac0c7d80 0x7f56ac0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 -- 172.21.15.46:0/3987882339 >> v1:172.21.15.46:6789/0 conn(0x7f56ac0a9840 legacy=0x7f56ac0c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 -- 172.21.15.46:0/3987882339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0c12c0 con 0x7f56ac0ca430 2023-10-06T11:34:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56abfff700 1 --2- 172.21.15.46:0/3987882339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0c6e30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f56ac06c640 tx=0x7f569c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ce6adfda5bdab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56ab7fe700 1 -- 172.21.15.46:0/3987882339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00b860 con 0x7f56ac0ca430 2023-10-06T11:34:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56ab7fe700 1 -- 172.21.15.46:0/3987882339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c00b9c0 con 0x7f56ac0ca430 2023-10-06T11:34:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56ab7fe700 1 -- 172.21.15.46:0/3987882339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00bd60 con 0x7f56ac0ca430 2023-10-06T11:34:29.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 -- 172.21.15.46:0/3987882339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 msgr2=0x7f56ac0c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.955+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/3987882339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0c6e30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f56ac06c640 tx=0x7f569c009250 comp rx=0 tx=0).stop 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 -- 172.21.15.46:0/3987882339 shutdown_connections 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/3987882339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0c6e30 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/3987882339 >> v2:172.21.15.132:3300/0 conn(0x7f56ac0c7d80 0x7f56ac0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 -- 172.21.15.46:0/3987882339 >> 172.21.15.46:0/3987882339 conn(0x7f56ac0cbe30 msgr2=0x7f56ac0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 -- 172.21.15.46:0/3987882339 shutdown_connections 2023-10-06T11:34:29.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 -- 172.21.15.46:0/3987882339 wait complete. 2023-10-06T11:34:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.959+0000 7f56b2ecd700 1 Processor -- start 2023-10-06T11:34:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- start start 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56ac0a9840 0x7f56ac14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 0x7f56ac157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac155030 con 0x7f56ac0ca430 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac155e50 con 0x7f56ac0c7d80 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac14f300 con 0x7f56ac0a9840 2023-10-06T11:34:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34598/0 (socket says 172.21.15.46:34598) 2023-10-06T11:34:29.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 -- 172.21.15.46:0/2843953781 learned_addr learned my addr 172.21.15.46:0/2843953781 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b0c69700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56ac0a9840 0x7f56ac14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56abfff700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 0x7f56ac157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 -- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56ac0a9840 msgr2=0x7f56ac14edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56ac0a9840 0x7f56ac14edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 -- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 msgr2=0x7f56ac157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 0x7f56ac157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:29.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 -- 172.21.15.46:0/2843953781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0094f0 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56abfff700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 0x7f56ac157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b146a700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f56a400c8e0 tx=0x7f56a4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400ce80 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a400a7e0 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac155240 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400ab80 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.963+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac155800 con 0x7f56ac0ca430 2023-10-06T11:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.967+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac155fe0 con 0x7f56ac0ca430 2023-10-06T11:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.967+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f56a4012420 con 0x7f56ac0ca430 2023-10-06T11:34:29.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.967+0000 7f56a8ff9700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 0x7f5698062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.971+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f56a407cd00 con 0x7f56ac0ca430 2023-10-06T11:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.971+0000 7f56b0c69700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 0x7f5698062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.971+0000 7f56b0c69700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 0x7f5698062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0008040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:29.975+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f56a404d170 con 0x7f56ac0ca430 2023-10-06T11:34:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.147+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f56ac0660f0 con 0x7f56ac0ca430 2023-10-06T11:34:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.147+0000 7f56a8ff9700 1 -- 172.21.15.46:0/2843953781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f56a4050790 con 0x7f56ac0ca430 2023-10-06T11:34:30.158 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 msgr2=0x7f5698062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 0x7f5698062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0008040 comp rx=0 tx=0).stop 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 msgr2=0x7f56ac0adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f56a400c8e0 tx=0x7f56a4001710 comp rx=0 tx=0).stop 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 shutdown_connections 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5698060670 0x7f5698062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ac0ca430 0x7f56ac0adb00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ac0c7d80 0x7f56ac157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 --2- 172.21.15.46:0/2843953781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56ac0a9840 0x7f56ac14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 >> 172.21.15.46:0/2843953781 conn(0x7f56ac0cbe30 msgr2=0x7f56ac0c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 shutdown_connections 2023-10-06T11:34:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.151+0000 7f56b2ecd700 1 -- 172.21.15.46:0/2843953781 wait complete. 2023-10-06T11:34:30.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:34:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.331+0000 7f11eb00c700 1 Processor -- start 2023-10-06T11:34:30.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.331+0000 7f11eb00c700 1 -- start start 2023-10-06T11:34:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11eb00c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e4156e20 0x7f11e406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11eb00c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4067760 0x7f11e4068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11eb00c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e406d790 con 0x7f11e40aaf30 2023-10-06T11:34:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11eb00c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e40acd60 con 0x7f11e4156e20 2023-10-06T11:34:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11eb00c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40ad040 con 0x7f11e4067760 2023-10-06T11:34:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e4156e20 0x7f11e406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e4156e20 0x7f11e406d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34604/0 (socket says 172.21.15.46:34604) 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 -- 172.21.15.46:0/1467742087 learned_addr learned my addr 172.21.15.46:0/1467742087 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea80b700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4067760 0x7f11e4068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 -- 172.21.15.46:0/1467742087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4067760 msgr2=0x7f11e4068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4067760 0x7f11e4068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 -- 172.21.15.46:0/1467742087 >> v1:172.21.15.132:6789/0 conn(0x7f11e40aaf30 legacy=0x7f11e4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 -- 172.21.15.46:0/1467742087 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40ad320 con 0x7f11e4156e20 2023-10-06T11:34:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11ea00a700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4156e20 0x7f11e406d250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f11e00099b0 tx=0x7f11e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb89f92fd07be2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11e9008700 1 -- 172.21.15.46:0/1467742087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000b860 con 0x7f11e4156e20 2023-10-06T11:34:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11e9008700 1 -- 172.21.15.46:0/1467742087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e000b9c0 con 0x7f11e4156e20 2023-10-06T11:34:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.335+0000 7f11e9008700 1 -- 172.21.15.46:0/1467742087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000bd60 con 0x7f11e4156e20 2023-10-06T11:34:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- 172.21.15.46:0/1467742087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4156e20 msgr2=0x7f11e406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4156e20 0x7f11e406d250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f11e00099b0 tx=0x7f11e0009250 comp rx=0 tx=0).stop 2023-10-06T11:34:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- 172.21.15.46:0/1467742087 shutdown_connections 2023-10-06T11:34:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4067760 0x7f11e4068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 --2- 172.21.15.46:0/1467742087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e4156e20 0x7f11e406d250 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- 172.21.15.46:0/1467742087 >> 172.21.15.46:0/1467742087 conn(0x7f11e4062ef0 msgr2=0x7f11e40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- 172.21.15.46:0/1467742087 shutdown_connections 2023-10-06T11:34:30.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- 172.21.15.46:0/1467742087 wait complete. 2023-10-06T11:34:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 Processor -- start 2023-10-06T11:34:30.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.339+0000 7f11eb00c700 1 -- start start 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4067760 0x7f11e4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4156e20 0x7f11e4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41554b0 con 0x7f11e40aaf30 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e41677e0 con 0x7f11e4067760 2023-10-06T11:34:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11eb00c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4167aa0 con 0x7f11e4156e20 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34608/0 (socket says 172.21.15.46:34608) 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 -- 172.21.15.46:0/2079560853 learned_addr learned my addr 172.21.15.46:0/2079560853 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11ea00a700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4067760 0x7f11e4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11ea80b700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4156e20 0x7f11e4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 -- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4156e20 msgr2=0x7f11e4154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4156e20 0x7f11e4154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 -- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4067760 msgr2=0x7f11e4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4067760 0x7f11e4159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 -- 172.21.15.46:0/2079560853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00094f0 con 0x7f11e40aaf30 2023-10-06T11:34:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.343+0000 7f11e9809700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f11d400c590 tx=0x7f11d400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4011590 con 0x7f11e40aaf30 2023-10-06T11:34:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e4167d80 con 0x7f11e40aaf30 2023-10-06T11:34:30.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d4011710 con 0x7f11e40aaf30 2023-10-06T11:34:30.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d4011ad0 con 0x7f11e40aaf30 2023-10-06T11:34:30.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e4168340 con 0x7f11e40aaf30 2023-10-06T11:34:30.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e406ad30 con 0x7f11e40aaf30 2023-10-06T11:34:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.347+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f11d401b3b0 con 0x7f11e40aaf30 2023-10-06T11:34:30.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.351+0000 7f11da7fc700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 0x7f11c8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.351+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f11d407be70 con 0x7f11e40aaf30 2023-10-06T11:34:30.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.351+0000 7f11ea00a700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 0x7f11c8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.351+0000 7f11ea00a700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 0x7f11c8062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11e00099b0 tx=0x7f11e0017000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.359+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11e406ad30 con 0x7f11e40aaf30 2023-10-06T11:34:30.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.535+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f11e406ad30 con 0x7f11e40aaf30 2023-10-06T11:34:30.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.535+0000 7f11da7fc700 1 -- 172.21.15.46:0/2079560853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+126 (secure 0 0 0) 0x7f11e406ad30 con 0x7f11e40aaf30 2023-10-06T11:34:30.547 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:30.548 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:34:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:34:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:34:30.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:34:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:34:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:34:30.549 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:30.549 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:34:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.539+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 msgr2=0x7f11c8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.539+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 0x7f11c8062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f11e00099b0 tx=0x7f11e0017000 comp rx=0 tx=0).stop 2023-10-06T11:34:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.539+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 msgr2=0x7f11e4154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.539+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f11d400c590 tx=0x7f11d400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 shutdown_connections 2023-10-06T11:34:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11c8060670 0x7f11c8062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e4156e20 0x7f11e4154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e40aaf30 0x7f11e4154a30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 --2- 172.21.15.46:0/2079560853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e4067760 0x7f11e4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 >> 172.21.15.46:0/2079560853 conn(0x7f11e4062ef0 msgr2=0x7f11e40c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 shutdown_connections 2023-10-06T11:34:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.543+0000 7f11eb00c700 1 -- 172.21.15.46:0/2079560853 wait complete. 2023-10-06T11:34:30.573 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:34:30.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:34:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.735+0000 7f672e5cc700 1 Processor -- start 2023-10-06T11:34:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.735+0000 7f672e5cc700 1 -- start start 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672e5cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 0x7f67280b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672e5cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f67280b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672e5cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67280b5eb0 con 0x7f67281591b0 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672e5cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67280cef30 con 0x7f6728158de0 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672e5cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67280cf4f0 con 0x7f6728069770 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672cdc9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f67281591b0 0x7f6728069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40548/0 (socket says 172.21.15.46:40548) 2023-10-06T11:34:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672cdc9700 1 -- 172.21.15.46:0/3886656702 learned_addr learned my addr 172.21.15.46:0/3886656702 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672d5ca700 1 --2- 172.21.15.46:0/3886656702 >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 0x7f67280b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 --2- 172.21.15.46:0/3886656702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f67280b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f671ffff700 1 -- 172.21.15.46:0/3886656702 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772922172 0 0) 0x7f6718001110 con 0x7f67281591b0 2023-10-06T11:34:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f671ffff700 1 -- 172.21.15.46:0/3886656702 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6714003570 con 0x7f67281591b0 2023-10-06T11:34:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 -- 172.21.15.46:0/3886656702 >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 msgr2=0x7f67280b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 --2- 172.21.15.46:0/3886656702 >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 0x7f67280b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 -- 172.21.15.46:0/3886656702 >> v1:172.21.15.46:6789/0 conn(0x7f67281591b0 legacy=0x7f6728069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 -- 172.21.15.46:0/3886656702 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280ccac0 con 0x7f6728069770 2023-10-06T11:34:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672d5ca700 1 --2- 172.21.15.46:0/3886656702 >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 0x7f67280b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f672ddcb700 1 --2- 172.21.15.46:0/3886656702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f67280b2e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e60 tx=0x7f6720009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819fb36b30fbee6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f671ffff700 1 -- 172.21.15.46:0/3886656702 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672000e070 con 0x7f6728069770 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f671ffff700 1 -- 172.21.15.46:0/3886656702 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672000bab0 con 0x7f6728069770 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.739+0000 7f671ffff700 1 -- 172.21.15.46:0/3886656702 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672000be70 con 0x7f6728069770 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- 172.21.15.46:0/3886656702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 msgr2=0x7f67280b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 --2- 172.21.15.46:0/3886656702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f67280b2e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e60 tx=0x7f6720009780 comp rx=0 tx=0).stop 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- 172.21.15.46:0/3886656702 shutdown_connections 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 --2- 172.21.15.46:0/3886656702 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f67280b2e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 --2- 172.21.15.46:0/3886656702 >> v2:172.21.15.132:3300/0 conn(0x7f6728158de0 0x7f67280b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- 172.21.15.46:0/3886656702 >> 172.21.15.46:0/3886656702 conn(0x7f6728068390 msgr2=0x7f67280b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- 172.21.15.46:0/3886656702 shutdown_connections 2023-10-06T11:34:30.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- 172.21.15.46:0/3886656702 wait complete. 2023-10-06T11:34:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 Processor -- start 2023-10-06T11:34:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.743+0000 7f672e5cc700 1 -- start start 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f672814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6728158de0 0x7f67280c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728152020 con 0x7f67281591b0 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672814fb60 con 0x7f6728158de0 2023-10-06T11:34:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6728169670 con 0x7f6728069770 2023-10-06T11:34:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34612/0 (socket says 172.21.15.46:34612) 2023-10-06T11:34:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 -- 172.21.15.46:0/568088246 learned_addr learned my addr 172.21.15.46:0/568088246 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672d5ca700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f672814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672cdc9700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6728158de0 0x7f67280c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 -- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 msgr2=0x7f672814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f672814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 -- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6728158de0 msgr2=0x7f67280c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6728158de0 0x7f67280c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 -- 172.21.15.46:0/568088246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724009420 con 0x7f67281591b0 2023-10-06T11:34:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672d5ca700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f672814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672ddcb700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f67200099f0 tx=0x7f6720019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720012e50 con 0x7f67281591b0 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6720027470 con 0x7f67281591b0 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6720009a20 con 0x7f67281591b0 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720027830 con 0x7f67281591b0 2023-10-06T11:34:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.747+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67280c5da0 con 0x7f67281591b0 2023-10-06T11:34:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.751+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67280b24f0 con 0x7f67281591b0 2023-10-06T11:34:30.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.751+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6720024070 con 0x7f67281591b0 2023-10-06T11:34:30.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.755+0000 7f671d7fa700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 0x7f670c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:30.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.755+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f672007efa0 con 0x7f67281591b0 2023-10-06T11:34:30.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.755+0000 7f672d5ca700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 0x7f670c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:30.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.755+0000 7f672d5ca700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 0x7f670c062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6724011600 tx=0x7f67240091a0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:30.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.759+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f67280b24f0 con 0x7f67281591b0 2023-10-06T11:34:30.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:30.931+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f672814fd20 con 0x7f67281591b0 2023-10-06T11:34:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.611+0000 7f671d7fa700 1 -- 172.21.15.46:0/568088246 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7f672002c020 con 0x7f67281591b0 2023-10-06T11:34:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 msgr2=0x7f670c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 0x7f670c062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6724011600 tx=0x7f67240091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 msgr2=0x7f67280c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f67200099f0 tx=0x7f6720019000 comp rx=0 tx=0).stop 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 shutdown_connections 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f670c060670 0x7f670c062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67281591b0 0x7f67280c6c50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6728158de0 0x7f67280c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 --2- 172.21.15.46:0/568088246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6728069770 0x7f672814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.615+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 >> 172.21.15.46:0/568088246 conn(0x7f6728068390 msgr2=0x7f67280cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:31.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.619+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 shutdown_connections 2023-10-06T11:34:31.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.619+0000 7f672e5cc700 1 -- 172.21.15.46:0/568088246 wait complete. 2023-10-06T11:34:31.632 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:34:31.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:34:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.815+0000 7f129ffff700 1 Processor -- start 2023-10-06T11:34:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.815+0000 7f129ffff700 1 -- start start 2023-10-06T11:34:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a00cb020 0x7f12a00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cf4e0 0x7f12a0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129ffff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a00cf050 con 0x7f12a00d0750 2023-10-06T11:34:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129ffff700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00cf8f0 con 0x7f12a00cb020 2023-10-06T11:34:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129ffff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00ce440 con 0x7f12a00cf4e0 2023-10-06T11:34:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.819+0000 7f129effd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a00cb020 0x7f12a00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a00cb020 0x7f12a00cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34620/0 (socket says 172.21.15.46:34620) 2023-10-06T11:34:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 -- 172.21.15.46:0/4186780108 learned_addr learned my addr 172.21.15.46:0/4186780108 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129f7fe700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cf4e0 0x7f12a0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 -- 172.21.15.46:0/4186780108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cf4e0 msgr2=0x7f12a0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cf4e0 0x7f12a0158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 -- 172.21.15.46:0/4186780108 >> v1:172.21.15.46:6789/0 conn(0x7f12a00d0750 legacy=0x7f12a00cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129dffb700 1 -- 172.21.15.46:0/4186780108 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997444447 0 0) 0x7f12a00cf050 con 0x7f12a00d0750 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 -- 172.21.15.46:0/4186780108 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00ce720 con 0x7f12a00cb020 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129effd700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00cb020 0x7f12a00cb3f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f12900099b0 tx=0x7f1290009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620300cf5e49d59d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129dffb700 1 -- 172.21.15.46:0/4186780108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290009ea0 con 0x7f12a00cb020 2023-10-06T11:34:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129dffb700 1 -- 172.21.15.46:0/4186780108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129000b860 con 0x7f12a00cb020 2023-10-06T11:34:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129dffb700 1 -- 172.21.15.46:0/4186780108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000bc00 con 0x7f12a00cb020 2023-10-06T11:34:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 -- 172.21.15.46:0/4186780108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00cb020 msgr2=0x7f12a00cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00cb020 0x7f12a00cb3f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f12900099b0 tx=0x7f1290009250 comp rx=0 tx=0).stop 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 -- 172.21.15.46:0/4186780108 shutdown_connections 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cf4e0 0x7f12a0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 --2- 172.21.15.46:0/4186780108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00cb020 0x7f12a00cb3f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 -- 172.21.15.46:0/4186780108 >> 172.21.15.46:0/4186780108 conn(0x7f12a00683d0 msgr2=0x7f12a00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 -- 172.21.15.46:0/4186780108 shutdown_connections 2023-10-06T11:34:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.823+0000 7f129ffff700 1 -- 172.21.15.46:0/4186780108 wait complete. 2023-10-06T11:34:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 Processor -- start 2023-10-06T11:34:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 -- start start 2023-10-06T11:34:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cb020 0x7f12a01521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a00cf4e0 0x7f12a00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0159750 con 0x7f12a00d0750 2023-10-06T11:34:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0158b40 con 0x7f12a00cf4e0 2023-10-06T11:34:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129ffff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0158e20 con 0x7f12a00cb020 2023-10-06T11:34:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34628/0 (socket says 172.21.15.46:34628) 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129f7fe700 1 -- 172.21.15.46:0/2953672330 learned_addr learned my addr 172.21.15.46:0/2953672330 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129effd700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cb020 0x7f12a01521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.827+0000 7f129e7fc700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a00cf4e0 0x7f12a00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 -- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cb020 msgr2=0x7f12a01521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cb020 0x7f12a01521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 -- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a00cf4e0 msgr2=0x7f12a00b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a00cf4e0 0x7f12a00b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 -- 172.21.15.46:0/2953672330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900094f0 con 0x7f12a00d0750 2023-10-06T11:34:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129f7fe700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1298009e90 tx=0x7f129800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800cc20 con 0x7f12a00d0750 2023-10-06T11:34:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800f040 con 0x7f12a00d0750 2023-10-06T11:34:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0159100 con 0x7f12a00d0750 2023-10-06T11:34:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298014480 con 0x7f12a00d0750 2023-10-06T11:34:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a0159be0 con 0x7f12a00d0750 2023-10-06T11:34:31.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.831+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a0159900 con 0x7f12a00d0750 2023-10-06T11:34:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.835+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1298014830 con 0x7f12a00d0750 2023-10-06T11:34:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.835+0000 7f1286ffd700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 0x7f128c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.835+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6020+0+0 (secure 0 0 0) 0x7f129807d940 con 0x7f12a00d0750 2023-10-06T11:34:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.835+0000 7f129effd700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 0x7f128c062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:31.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.839+0000 7f129effd700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 0x7f128c062ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12900099b0 tx=0x7f1290017000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:31.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:31.843+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12a0159900 con 0x7f12a00d0750 2023-10-06T11:34:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.015+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f12a0064590 con 0x7f12a00d0750 2023-10-06T11:34:32.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.619+0000 7f1286ffd700 1 -- 172.21.15.46:0/2953672330 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f12980180d0 con 0x7f12a00d0750 2023-10-06T11:34:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 msgr2=0x7f128c062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 0x7f128c062ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12900099b0 tx=0x7f1290017000 comp rx=0 tx=0).stop 2023-10-06T11:34:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 msgr2=0x7f12a00b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1298009e90 tx=0x7f129800b040 comp rx=0 tx=0).stop 2023-10-06T11:34:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 shutdown_connections 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f128c0606f0 0x7f128c062ba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a00d0750 0x7f12a00b08f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a00cf4e0 0x7f12a00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 --2- 172.21.15.46:0/2953672330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a00cb020 0x7f12a01521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.631+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 >> 172.21.15.46:0/2953672330 conn(0x7f12a00683d0 msgr2=0x7f12a00c4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.635+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 shutdown_connections 2023-10-06T11:34:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.635+0000 7f129ffff700 1 -- 172.21.15.46:0/2953672330 wait complete. 2023-10-06T11:34:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:34:32.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:34:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 Processor -- start 2023-10-06T11:34:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 -- start start 2023-10-06T11:34:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 0x7ff85c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff85c159850 0x7ff85c159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0ce6e0 con 0x7ff85c069270 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c15a5c0 con 0x7ff85c1591b0 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862d23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0cddb0 con 0x7ff85c159850 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff861520700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 0x7ff85c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff861520700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 0x7ff85c069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34638/0 (socket says 172.21.15.46:34638) 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff861520700 1 -- 172.21.15.46:0/3951395993 learned_addr learned my addr 172.21.15.46:0/3951395993 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.823+0000 7ff862522700 1 --2- 172.21.15.46:0/3951395993 >> v2:172.21.15.132:3300/0 conn(0x7ff85c159850 0x7ff85c159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862522700 1 -- 172.21.15.46:0/3951395993 >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 msgr2=0x7ff85c069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862522700 1 --2- 172.21.15.46:0/3951395993 >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 0x7ff85c069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862522700 1 -- 172.21.15.46:0/3951395993 >> v1:172.21.15.132:6789/0 conn(0x7ff85c069270 legacy=0x7ff85c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862522700 1 -- 172.21.15.46:0/3951395993 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0ce090 con 0x7ff85c159850 2023-10-06T11:34:32.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862522700 1 --2- 172.21.15.46:0/3951395993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c159850 0x7ff85c159cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff85c067710 tx=0x7ff854009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37015d21993798a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:32.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff860d1f700 1 -- 172.21.15.46:0/3951395993 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400e070 con 0x7ff85c159850 2023-10-06T11:34:32.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff860d1f700 1 -- 172.21.15.46:0/3951395993 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85400bb60 con 0x7ff85c159850 2023-10-06T11:34:32.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff860d1f700 1 -- 172.21.15.46:0/3951395993 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854012430 con 0x7ff85c159850 2023-10-06T11:34:32.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 -- 172.21.15.46:0/3951395993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c159850 msgr2=0x7ff85c159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 --2- 172.21.15.46:0/3951395993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c159850 0x7ff85c159cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff85c067710 tx=0x7ff854009780 comp rx=0 tx=0).stop 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 -- 172.21.15.46:0/3951395993 shutdown_connections 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 --2- 172.21.15.46:0/3951395993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c159850 0x7ff85c159cd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 --2- 172.21.15.46:0/3951395993 >> v2:172.21.15.46:3300/0 conn(0x7ff85c1591b0 0x7ff85c069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 -- 172.21.15.46:0/3951395993 >> 172.21.15.46:0/3951395993 conn(0x7ff85c068390 msgr2=0x7ff85c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 -- 172.21.15.46:0/3951395993 shutdown_connections 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.827+0000 7ff862d23700 1 -- 172.21.15.46:0/3951395993 wait complete. 2023-10-06T11:34:32.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 Processor -- start 2023-10-06T11:34:32.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 -- start start 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85c069270 0x7ff85c14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 0x7ff85c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 0x7ff85c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c15bee0 con 0x7ff85c159850 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b5220 con 0x7ff85c1591b0 2023-10-06T11:34:32.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862d23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0b4610 con 0x7ff85c069270 2023-10-06T11:34:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 0x7ff85c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 0x7ff85c0cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34642/0 (socket says 172.21.15.46:34642) 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff862522700 1 -- 172.21.15.46:0/2933646667 learned_addr learned my addr 172.21.15.46:0/2933646667 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff861520700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 0x7ff85c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.831+0000 7ff861d21700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85c069270 0x7ff85c14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 -- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85c069270 msgr2=0x7ff85c14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85c069270 0x7ff85c14ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 -- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 msgr2=0x7ff85c0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 0x7ff85c0cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 -- 172.21.15.46:0/2933646667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854009a20 con 0x7ff85c1591b0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff861520700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 0x7ff85c0b16e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006d40 tx=0x7ff84c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c013680 con 0x7ff85c1591b0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c0137e0 con 0x7ff85c1591b0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c0b48f0 con 0x7ff85c1591b0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c013b60 con 0x7ff85c1591b0 2023-10-06T11:34:32.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c0b4eb0 con 0x7ff85c1591b0 2023-10-06T11:34:32.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.835+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c064590 con 0x7ff85c1591b0 2023-10-06T11:34:32.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.839+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff84c00c3e0 con 0x7ff85c1591b0 2023-10-06T11:34:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.839+0000 7ff851ffb700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 0x7ff840062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.839+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6402+0+0 (secure 0 0 0) 0x7ff84c07d070 con 0x7ff85c1591b0 2023-10-06T11:34:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.839+0000 7ff861d21700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 0x7ff840062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.843+0000 7ff861d21700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 0x7ff840062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff858003a80 tx=0x7ff858007ec0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:32.847+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff84c0acbe0 con 0x7ff85c1591b0 2023-10-06T11:34:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.059+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff85c0b5360 con 0x7ff85c1591b0 2023-10-06T11:34:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.651+0000 7ff851ffb700 1 -- 172.21.15.46:0/2933646667 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff84c04d1f0 con 0x7ff85c1591b0 2023-10-06T11:34:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 msgr2=0x7ff840062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 0x7ff840062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff858003a80 tx=0x7ff858007ec0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 msgr2=0x7ff85c0b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 0x7ff85c0b16e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff84c006d40 tx=0x7ff84c009250 comp rx=0 tx=0).stop 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 shutdown_connections 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff8400606a0 0x7ff840062b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff85c159850 0x7ff85c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff85c1591b0 0x7ff85c0b16e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 --2- 172.21.15.46:0/2933646667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff85c069270 0x7ff85c14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 >> 172.21.15.46:0/2933646667 conn(0x7ff85c068390 msgr2=0x7ff85c0c4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 shutdown_connections 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:33.671+0000 7ff862d23700 1 -- 172.21.15.46:0/2933646667 wait complete. 2023-10-06T11:34:34.217 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T11:34:34.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.379+0000 7eff10c2e700 1 Processor -- start 2023-10-06T11:34:34.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.383+0000 7eff10c2e700 1 -- start start 2023-10-06T11:34:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.383+0000 7eff10c2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff0c1502f0 0x7eff0c0c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.383+0000 7eff10c2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c0c67e0 con 0x7eff0c1506c0 2023-10-06T11:34:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.383+0000 7eff10c2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c06e810 con 0x7eff0c15a1a0 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.383+0000 7eff10c2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c06eaf0 con 0x7eff0c1502f0 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff0c1502f0 0x7eff0c0c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff01d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7eff0c1506c0 0x7eff0c159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50864/0 (socket says 172.21.15.46:50864) 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff0c1502f0 0x7eff0c0c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38640/0 (socket says 172.21.15.46:38640) 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff01d9b700 1 -- 172.21.15.46:0/1181684923 learned_addr learned my addr 172.21.15.46:0/1181684923 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 -- 172.21.15.46:0/1181684923 >> v1:172.21.15.132:6789/0 conn(0x7eff0c15a1a0 legacy=0x7eff0c0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 -- 172.21.15.46:0/1181684923 >> v1:172.21.15.46:6789/0 conn(0x7eff0c1506c0 legacy=0x7eff0c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff09d9b700 1 -- 172.21.15.46:0/1181684923 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743332422 0 0) 0x7eff0c0c67e0 con 0x7eff0c1506c0 2023-10-06T11:34:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 -- 172.21.15.46:0/1181684923 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c06edd0 con 0x7eff0c1502f0 2023-10-06T11:34:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff0a59c700 1 --2- 172.21.15.46:0/1181684923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c0c0bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efefc00bb80 tx=0x7efefc00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f843cf28f69fada5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff09d9b700 1 -- 172.21.15.46:0/1181684923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc00dba0 con 0x7eff0c1502f0 2023-10-06T11:34:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff09d9b700 1 -- 172.21.15.46:0/1181684923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc00dd00 con 0x7eff0c1502f0 2023-10-06T11:34:34.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff09d9b700 1 -- 172.21.15.46:0/1181684923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc0196b0 con 0x7eff0c1502f0 2023-10-06T11:34:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff10c2e700 1 -- 172.21.15.46:0/1181684923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 msgr2=0x7eff0c0c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.387+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1181684923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c0c0bd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efefc00bb80 tx=0x7efefc00b430 comp rx=0 tx=0).stop 2023-10-06T11:34:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 -- 172.21.15.46:0/1181684923 shutdown_connections 2023-10-06T11:34:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1181684923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c0c0bd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 -- 172.21.15.46:0/1181684923 >> 172.21.15.46:0/1181684923 conn(0x7eff0c0cbe30 msgr2=0x7eff0c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 -- 172.21.15.46:0/1181684923 shutdown_connections 2023-10-06T11:34:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 -- 172.21.15.46:0/1181684923 wait complete. 2023-10-06T11:34:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 Processor -- start 2023-10-06T11:34:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.391+0000 7eff10c2e700 1 -- start start 2023-10-06T11:34:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 0x7eff0c169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 0x7eff0c16a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0afba0 con 0x7eff0c1502f0 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1546f0 con 0x7eff0c15a1a0 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1567f0 con 0x7eff0c1506c0 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38652/0 (socket says 172.21.15.46:38652) 2023-10-06T11:34:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 -- 172.21.15.46:0/1549878644 learned_addr learned my addr 172.21.15.46:0/1549878644 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff01d9b700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 0x7eff0c169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 -- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 msgr2=0x7eff0c169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0ad9d700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 0x7eff0c16a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 0x7eff0c169d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 -- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 msgr2=0x7eff0c16a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 0x7eff0c16a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 -- 172.21.15.46:0/1549878644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc00b670 con 0x7eff0c1502f0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff01d9b700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 0x7eff0c169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0ad9d700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 0x7eff0c16a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff0a59c700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7eff0c06c150 tx=0x7efefc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc0106c0 con 0x7eff0c1502f0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc010860 con 0x7eff0c1502f0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc010c20 con 0x7eff0c1502f0 2023-10-06T11:34:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c156ad0 con 0x7eff0c1502f0 2023-10-06T11:34:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.395+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c156230 con 0x7eff0c1502f0 2023-10-06T11:34:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.399+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c063f90 con 0x7eff0c1502f0 2023-10-06T11:34:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.399+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7efefc024420 con 0x7eff0c1502f0 2023-10-06T11:34:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.403+0000 7eff02ffd700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 0x7efeec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.403+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6502+0+0 (secure 0 0 0) 0x7efefc020070 con 0x7eff0c1502f0 2023-10-06T11:34:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.403+0000 7eff01d9b700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 0x7efeec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.403+0000 7eff01d9b700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 0x7efeec062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0aaf10 tx=0x7efef4009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.407+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efefc04ee70 con 0x7eff0c1502f0 2023-10-06T11:34:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.583+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff0c0c6470 con 0x7eff0c1502f0 2023-10-06T11:34:34.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.583+0000 7eff02ffd700 1 -- 172.21.15.46:0/1549878644 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17564 (secure 0 0 0) 0x7efefc052450 con 0x7eff0c1502f0 2023-10-06T11:34:34.595 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:34.595 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:33.064358+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 msgr2=0x7efeec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 0x7efeec062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0aaf10 tx=0x7efef4009040 comp rx=0 tx=0).stop 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 msgr2=0x7eff0c153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7eff0c06c150 tx=0x7efefc01b000 comp rx=0 tx=0).stop 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 shutdown_connections 2023-10-06T11:34:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efeec060670 0x7efeec062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff0c15a1a0 0x7eff0c16a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7eff0c1506c0 0x7eff0c169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 --2- 172.21.15.46:0/1549878644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff0c1502f0 0x7eff0c153ff0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 >> 172.21.15.46:0/1549878644 conn(0x7eff0c0cbe30 msgr2=0x7eff0c1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 shutdown_connections 2023-10-06T11:34:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.587+0000 7eff10c2e700 1 -- 172.21.15.46:0/1549878644 wait complete. 2023-10-06T11:34:34.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.779+0000 7f6baf523700 1 Processor -- start 2023-10-06T11:34:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.779+0000 7f6baf523700 1 -- start start 2023-10-06T11:34:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baf523700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ba8156e20 0x7f6ba806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baf523700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba806d790 con 0x7f6ba8067760 2023-10-06T11:34:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baf523700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80acd60 con 0x7f6ba80aaf30 2023-10-06T11:34:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baf523700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba80ad040 con 0x7f6ba8156e20 2023-10-06T11:34:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baed22700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ba8067760 0x7f6ba8068a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50870/0 (socket says 172.21.15.46:50870) 2023-10-06T11:34:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6baed22700 1 -- 172.21.15.46:0/1371226602 learned_addr learned my addr 172.21.15.46:0/1371226602 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bae521700 1 --2- 172.21.15.46:0/1371226602 >> v2:172.21.15.46:3300/0 conn(0x7f6ba8156e20 0x7f6ba806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655498248 0 0) 0x7f6ba806d790 con 0x7f6ba8067760 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003570 con 0x7f6ba8067760 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603695645 0 0) 0x7f6ba80acd60 con 0x7f6ba80aaf30 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba806d790 con 0x7f6ba80aaf30 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bae521700 1 -- 172.21.15.46:0/1371226602 >> v1:172.21.15.132:6789/0 conn(0x7f6ba80aaf30 legacy=0x7f6ba8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bae521700 1 -- 172.21.15.46:0/1371226602 >> v1:172.21.15.46:6789/0 conn(0x7f6ba8067760 legacy=0x7f6ba8068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.783+0000 7f6bae521700 1 -- 172.21.15.46:0/1371226602 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80ad320 con 0x7f6ba8156e20 2023-10-06T11:34:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6bae521700 1 --2- 172.21.15.46:0/1371226602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba8156e20 0x7f6ba806d250 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098e0 tx=0x7f6ba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a015a87f5b7e631b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011630 con 0x7f6ba8156e20 2023-10-06T11:34:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4011790 con 0x7f6ba8156e20 2023-10-06T11:34:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6bad51f700 1 -- 172.21.15.46:0/1371226602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011b30 con 0x7f6ba8156e20 2023-10-06T11:34:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 -- 172.21.15.46:0/1371226602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba8156e20 msgr2=0x7f6ba806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 --2- 172.21.15.46:0/1371226602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba8156e20 0x7f6ba806d250 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40098e0 tx=0x7f6ba40091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 -- 172.21.15.46:0/1371226602 shutdown_connections 2023-10-06T11:34:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 --2- 172.21.15.46:0/1371226602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba8156e20 0x7f6ba806d250 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 -- 172.21.15.46:0/1371226602 >> 172.21.15.46:0/1371226602 conn(0x7f6ba8062ef0 msgr2=0x7f6ba80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 -- 172.21.15.46:0/1371226602 shutdown_connections 2023-10-06T11:34:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.787+0000 7f6baf523700 1 -- 172.21.15.46:0/1371226602 wait complete. 2023-10-06T11:34:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 Processor -- start 2023-10-06T11:34:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 -- start start 2023-10-06T11:34:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba8067760 0x7f6ba80c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba8156e20 0x7f6ba80ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c5b90 con 0x7f6ba80aaf30 2023-10-06T11:34:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c5960 con 0x7f6ba8156e20 2023-10-06T11:34:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baf523700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81585d0 con 0x7f6ba8067760 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38668/0 (socket says 172.21.15.46:38668) 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 -- 172.21.15.46:0/3497357481 learned_addr learned my addr 172.21.15.46:0/3497357481 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6baed22700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba8156e20 0x7f6ba80ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6bae521700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba8067760 0x7f6ba80c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 -- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba8067760 msgr2=0x7f6ba80c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba8067760 0x7f6ba80c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 -- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba8156e20 msgr2=0x7f6ba80ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba8156e20 0x7f6ba80ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 -- 172.21.15.46:0/3497357481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4009420 con 0x7f6ba80aaf30 2023-10-06T11:34:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.791+0000 7f6badd20700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c790 tx=0x7f6b98007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98010510 con 0x7f6ba80aaf30 2023-10-06T11:34:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b98010670 con 0x7f6ba80aaf30 2023-10-06T11:34:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba8157ca0 con 0x7f6ba80aaf30 2023-10-06T11:34:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98010a10 con 0x7f6ba80aaf30 2023-10-06T11:34:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8158260 con 0x7f6ba80aaf30 2023-10-06T11:34:34.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba8158710 con 0x7f6ba80aaf30 2023-10-06T11:34:34.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.795+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6b9801b3b0 con 0x7f6ba80aaf30 2023-10-06T11:34:34.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.799+0000 7f6b9e7fc700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 0x7f6b8c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:34.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.799+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6b9807c490 con 0x7f6ba80aaf30 2023-10-06T11:34:34.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.799+0000 7f6bae521700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 0x7f6b8c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:34.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.799+0000 7f6bae521700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 0x7f6b8c062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40093f0 tx=0x7f6ba4017000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:34.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.807+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ba8158710 con 0x7f6ba80aaf30 2023-10-06T11:34:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.983+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba8158710 con 0x7f6ba80aaf30 2023-10-06T11:34:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.983+0000 7f6b9e7fc700 1 -- 172.21.15.46:0/3497357481 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17551 (secure 0 0 0) 0x7f6ba8158710 con 0x7f6ba80aaf30 2023-10-06T11:34:34.994 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:34.995 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:34.637275+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 msgr2=0x7f6b8c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 0x7f6b8c062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40093f0 tx=0x7f6ba4017000 comp rx=0 tx=0).stop 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 msgr2=0x7f6ba81554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800c790 tx=0x7f6b98007e10 comp rx=0 tx=0).stop 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 shutdown_connections 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b8c060670 0x7f6b8c062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ba8156e20 0x7f6ba80ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ba80aaf30 0x7f6ba81554f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 --2- 172.21.15.46:0/3497357481 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ba8067760 0x7f6ba80c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 >> 172.21.15.46:0/3497357481 conn(0x7f6ba8062ef0 msgr2=0x7f6ba8156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 shutdown_connections 2023-10-06T11:34:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:34.991+0000 7f6baf523700 1 -- 172.21.15.46:0/3497357481 wait complete. 2023-10-06T11:34:35.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:34:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.175+0000 7f79808ef700 1 Processor -- start 2023-10-06T11:34:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.175+0000 7f79808ef700 1 -- start start 2023-10-06T11:34:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f79808ef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 0x7f797c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f79808ef700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f797c0cab20 0x7f797c0c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f79808ef700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06e0d0 con 0x7f797c0c9fc0 2023-10-06T11:34:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f79808ef700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c0c0d60 con 0x7f797c0cab20 2023-10-06T11:34:35.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f79808ef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c06e3b0 con 0x7f797c0c6250 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f797c0c9fc0 0x7f797c0ca410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50882/0 (socket says 172.21.15.46:50882) 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797a59c700 1 -- 172.21.15.46:0/2786015236 learned_addr learned my addr 172.21.15.46:0/2786015236 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797ad9d700 1 --2- 172.21.15.46:0/2786015236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 0x7f797c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797b59e700 1 --2- 172.21.15.46:0/2786015236 >> v2:172.21.15.132:3300/0 conn(0x7f797c0cab20 0x7f797c0c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f7979d9b700 1 -- 172.21.15.46:0/2786015236 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069761184 0 0) 0x7f797c06e0d0 con 0x7f797c0c9fc0 2023-10-06T11:34:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f7979d9b700 1 -- 172.21.15.46:0/2786015236 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7964003570 con 0x7f797c0c9fc0 2023-10-06T11:34:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797ad9d700 1 -- 172.21.15.46:0/2786015236 >> v2:172.21.15.132:3300/0 conn(0x7f797c0cab20 msgr2=0x7f797c0c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797ad9d700 1 --2- 172.21.15.46:0/2786015236 >> v2:172.21.15.132:3300/0 conn(0x7f797c0cab20 0x7f797c0c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797ad9d700 1 -- 172.21.15.46:0/2786015236 >> v1:172.21.15.46:6789/0 conn(0x7f797c0c9fc0 legacy=0x7f797c0ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.179+0000 7f797ad9d700 1 -- 172.21.15.46:0/2786015236 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c06e690 con 0x7f797c0c6250 2023-10-06T11:34:35.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f797ad9d700 1 --2- 172.21.15.46:0/2786015236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 0x7f797c0c9a80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098e0 tx=0x7f796c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c35b5b9a3b610 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f7979d9b700 1 -- 172.21.15.46:0/2786015236 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c011630 con 0x7f797c0c6250 2023-10-06T11:34:35.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f7979d9b700 1 -- 172.21.15.46:0/2786015236 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c011790 con 0x7f797c0c6250 2023-10-06T11:34:35.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f7979d9b700 1 -- 172.21.15.46:0/2786015236 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c011b30 con 0x7f797c0c6250 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 -- 172.21.15.46:0/2786015236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 msgr2=0x7f797c0c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 --2- 172.21.15.46:0/2786015236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 0x7f797c0c9a80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098e0 tx=0x7f796c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 -- 172.21.15.46:0/2786015236 shutdown_connections 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 --2- 172.21.15.46:0/2786015236 >> v2:172.21.15.132:3300/0 conn(0x7f797c0cab20 0x7f797c0c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 --2- 172.21.15.46:0/2786015236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0c6250 0x7f797c0c9a80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 -- 172.21.15.46:0/2786015236 >> 172.21.15.46:0/2786015236 conn(0x7f797c0cbc10 msgr2=0x7f797c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 -- 172.21.15.46:0/2786015236 shutdown_connections 2023-10-06T11:34:35.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.183+0000 7f79808ef700 1 -- 172.21.15.46:0/2786015236 wait complete. 2023-10-06T11:34:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 Processor -- start 2023-10-06T11:34:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 -- start start 2023-10-06T11:34:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 0x7f797c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0cab20 0x7f797c158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c1560b0 con 0x7f797c0c6250 2023-10-06T11:34:35.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c158fc0 con 0x7f797c0c9fc0 2023-10-06T11:34:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.187+0000 7f79808ef700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c16b1e0 con 0x7f797c0cab20 2023-10-06T11:34:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38680/0 (socket says 172.21.15.46:38680) 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 -- 172.21.15.46:0/1083232859 learned_addr learned my addr 172.21.15.46:0/1083232859 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797b59e700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0cab20 0x7f797c158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797a59c700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 0x7f797c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 -- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0cab20 msgr2=0x7f797c158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0cab20 0x7f797c158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 -- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 msgr2=0x7f797c158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 0x7f797c158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 -- 172.21.15.46:0/1083232859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c009420 con 0x7f797c0c6250 2023-10-06T11:34:35.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797a59c700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 0x7f797c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f797ad9d700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f797c06e4f0 tx=0x7f796c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c011b10 con 0x7f797c0c6250 2023-10-06T11:34:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c000cc0 con 0x7f797c0c6250 2023-10-06T11:34:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c1562c0 con 0x7f797c0c6250 2023-10-06T11:34:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c023600 con 0x7f797c0c6250 2023-10-06T11:34:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.191+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c156880 con 0x7f797c0c6250 2023-10-06T11:34:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.195+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c156a70 con 0x7f797c0c6250 2023-10-06T11:34:35.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.195+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f796c02c3b0 con 0x7f797c0c6250 2023-10-06T11:34:35.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.195+0000 7f7962ffd700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 0x7f7968062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.195+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f796c07e5b0 con 0x7f797c0c6250 2023-10-06T11:34:35.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.199+0000 7f797a59c700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 0x7f7968062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.199+0000 7f797a59c700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 0x7f7968062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f797c149a00 tx=0x7f7970009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.203+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f796c04e7d0 con 0x7f797c0c6250 2023-10-06T11:34:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.411+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f797c064590 con 0x7f797c0c6250 2023-10-06T11:34:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.695+0000 7f7962ffd700 1 -- 172.21.15.46:0/1083232859 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f796c0150b0 con 0x7f797c0c6250 2023-10-06T11:34:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 msgr2=0x7f7968062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 0x7f7968062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f797c149a00 tx=0x7f7970009040 comp rx=0 tx=0).stop 2023-10-06T11:34:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 msgr2=0x7f797c13ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f797c06e4f0 tx=0x7f796c017000 comp rx=0 tx=0).stop 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 shutdown_connections 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7968060670 0x7f7968062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f797c0cab20 0x7f797c158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f797c0c9fc0 0x7f797c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 --2- 172.21.15.46:0/1083232859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f797c0c6250 0x7f797c13ed90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 >> 172.21.15.46:0/1083232859 conn(0x7f797c0cbc10 msgr2=0x7f797c153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.699+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 shutdown_connections 2023-10-06T11:34:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.703+0000 7f79808ef700 1 -- 172.21.15.46:0/1083232859 wait complete. 2023-10-06T11:34:35.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:34:35.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.895+0000 7fa258a6e700 1 Processor -- start 2023-10-06T11:34:35.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.895+0000 7fa258a6e700 1 -- start start 2023-10-06T11:34:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa258a6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa25406e8a0 0x7fa2540c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa258a6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa25406ed10 0x7fa2540c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa258a6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2540c1bf0 con 0x7fa2540c6470 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa258a6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2540c72c0 con 0x7fa25406e8a0 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa258a6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25406cf80 con 0x7fa25406ed10 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2540c6470 0x7fa2540ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50894/0 (socket says 172.21.15.46:50894) 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25259c700 1 -- 172.21.15.46:0/1025980693 learned_addr learned my addr 172.21.15.46:0/1025980693 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 --2- 172.21.15.46:0/1025980693 >> v2:172.21.15.46:3300/0 conn(0x7fa25406e8a0 0x7fa2540c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa252d9d700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa25406ed10 0x7fa2540c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25159a700 1 -- 172.21.15.46:0/1025980693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458718837 0 0) 0x7fa2540c1bf0 con 0x7fa2540c6470 2023-10-06T11:34:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25159a700 1 -- 172.21.15.46:0/1025980693 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa23c003570 con 0x7fa2540c6470 2023-10-06T11:34:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 -- 172.21.15.46:0/1025980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa25406ed10 msgr2=0x7fa2540c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa25406ed10 0x7fa2540c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 -- 172.21.15.46:0/1025980693 >> v1:172.21.15.46:6789/0 conn(0x7fa2540c6470 legacy=0x7fa2540ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 -- 172.21.15.46:0/1025980693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25406d260 con 0x7fa25406e8a0 2023-10-06T11:34:35.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa251d9b700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406e8a0 0x7fa2540c6840 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa2480098e0 tx=0x7fa2480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c8a5d6cedf69b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25159a700 1 -- 172.21.15.46:0/1025980693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248011630 con 0x7fa25406e8a0 2023-10-06T11:34:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25159a700 1 -- 172.21.15.46:0/1025980693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248011790 con 0x7fa25406e8a0 2023-10-06T11:34:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.899+0000 7fa25159a700 1 -- 172.21.15.46:0/1025980693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248011b30 con 0x7fa25406e8a0 2023-10-06T11:34:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- 172.21.15.46:0/1025980693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406e8a0 msgr2=0x7fa2540c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406e8a0 0x7fa2540c6840 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa2480098e0 tx=0x7fa2480091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- 172.21.15.46:0/1025980693 shutdown_connections 2023-10-06T11:34:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa25406ed10 0x7fa2540c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- 172.21.15.46:0/1025980693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406e8a0 0x7fa2540c6840 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- 172.21.15.46:0/1025980693 >> 172.21.15.46:0/1025980693 conn(0x7fa2540cbe30 msgr2=0x7fa2540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- 172.21.15.46:0/1025980693 shutdown_connections 2023-10-06T11:34:35.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- 172.21.15.46:0/1025980693 wait complete. 2023-10-06T11:34:35.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 Processor -- start 2023-10-06T11:34:35.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- start start 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa25406e8a0 0x7fa254159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2540c6470 0x7fa254169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540acad0 con 0x7fa25406ed10 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2540aa200 con 0x7fa25406e8a0 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.903+0000 7fa258a6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25415a1a0 con 0x7fa2540c6470 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38690/0 (socket says 172.21.15.46:38690) 2023-10-06T11:34:35.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 -- 172.21.15.46:0/643602087 learned_addr learned my addr 172.21.15.46:0/643602087 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa25259c700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa25406e8a0 0x7fa254159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa252d9d700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2540c6470 0x7fa254169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 -- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2540c6470 msgr2=0x7fa254169dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2540c6470 0x7fa254169dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 -- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa25406e8a0 msgr2=0x7fa254159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa25406e8a0 0x7fa254159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 -- 172.21.15.46:0/643602087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248009420 con 0x7fa25406ed10 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa251d9b700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ed0 tx=0x7fa248017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248011900 con 0x7fa25406ed10 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248011a60 con 0x7fa25406ed10 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2540aaa40 con 0x7fa25406ed10 2023-10-06T11:34:35.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248025640 con 0x7fa25406ed10 2023-10-06T11:34:35.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.907+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2540ac8a0 con 0x7fa25406ed10 2023-10-06T11:34:35.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.911+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2540aa820 con 0x7fa25406ed10 2023-10-06T11:34:35.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.911+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa248025a30 con 0x7fa25406ed10 2023-10-06T11:34:35.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.911+0000 7fa2427fc700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 0x7fa234062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:35.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.911+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa24807e3d0 con 0x7fa25406ed10 2023-10-06T11:34:35.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.911+0000 7fa25259c700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 0x7fa234062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:35.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.915+0000 7fa25259c700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 0x7fa234062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa244007fd0 tx=0x7fa244009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:35.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:35.919+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa24804e4f0 con 0x7fa25406ed10 2023-10-06T11:34:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.127+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2540aabd0 con 0x7fa25406ed10 2023-10-06T11:34:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.699+0000 7fa2427fc700 1 -- 172.21.15.46:0/643602087 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa248051b10 con 0x7fa25406ed10 2023-10-06T11:34:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 msgr2=0x7fa234062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 0x7fa234062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa244007fd0 tx=0x7fa244009040 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 msgr2=0x7fa254169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ed0 tx=0x7fa248017000 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 shutdown_connections 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa234060670 0x7fa234062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2540c6470 0x7fa254169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa25406ed10 0x7fa254169880 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 --2- 172.21.15.46:0/643602087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa25406e8a0 0x7fa254159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 >> 172.21.15.46:0/643602087 conn(0x7fa2540cbe30 msgr2=0x7fa2540c2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 shutdown_connections 2023-10-06T11:34:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.703+0000 7fa258a6e700 1 -- 172.21.15.46:0/643602087 wait complete. 2023-10-06T11:34:36.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:36.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.891+0000 7f4bd41c5700 1 Processor -- start 2023-10-06T11:34:36.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 -- start start 2023-10-06T11:34:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc067050 0x7f4bcc067470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc0679b0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc0abe20 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd41c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc15a780 con 0x7f4bcc067050 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38704/0 (socket says 172.21.15.46:38704) 2023-10-06T11:34:36.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 -- 172.21.15.46:0/3073373206 learned_addr learned my addr 172.21.15.46:0/3073373206 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd2762700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc067050 0x7f4bcc067470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 -- 172.21.15.46:0/3073373206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc067050 msgr2=0x7f4bcc067470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc067050 0x7f4bcc067470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 -- 172.21.15.46:0/3073373206 >> v1:172.21.15.46:6789/0 conn(0x7f4bcc0af0b0 legacy=0x7f4bcc15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 -- 172.21.15.46:0/3073373206 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0ac0c0 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd1f61700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009f20 tx=0x7f4bc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397737729dd69e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.895+0000 7f4bd0f5f700 1 -- 172.21.15.46:0/3073373206 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800e070 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd0f5f700 1 -- 172.21.15.46:0/3073373206 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc800ba80 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd0f5f700 1 -- 172.21.15.46:0/3073373206 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80092e0 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- 172.21.15.46:0/3073373206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0cc2d0 msgr2=0x7f4bcc0ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009f20 tx=0x7f4bc8009840 comp rx=0 tx=0).stop 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- 172.21.15.46:0/3073373206 shutdown_connections 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc067050 0x7f4bcc067470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/3073373206 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0cc2d0 0x7f4bcc0ab8e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- 172.21.15.46:0/3073373206 >> 172.21.15.46:0/3073373206 conn(0x7f4bcc068bf0 msgr2=0x7f4bcc0ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- 172.21.15.46:0/3073373206 shutdown_connections 2023-10-06T11:34:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- 172.21.15.46:0/3073373206 wait complete. 2023-10-06T11:34:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 Processor -- start 2023-10-06T11:34:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.899+0000 7f4bd41c5700 1 -- start start 2023-10-06T11:34:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc067050 0x7f4bcc15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc0cc2d0 0x7f4bcc156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc15aad0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0c2b50 con 0x7f4bcc067050 2023-10-06T11:34:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0c2e30 con 0x7f4bcc0cc2d0 2023-10-06T11:34:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38706/0 (socket says 172.21.15.46:38706) 2023-10-06T11:34:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 -- 172.21.15.46:0/2172352152 learned_addr learned my addr 172.21.15.46:0/2172352152 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1f61700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc067050 0x7f4bcc15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd2762700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc0cc2d0 0x7f4bcc156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 -- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc0cc2d0 msgr2=0x7f4bcc156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc0cc2d0 0x7f4bcc156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 -- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc067050 msgr2=0x7f4bcc15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc067050 0x7f4bcc15d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 -- 172.21.15.46:0/2172352152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009ae0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd1760700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c7c0 tx=0x7f4bbc007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc011590 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc0116f0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc0c3110 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc011ab0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.903+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc15d880 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.907+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc15ac80 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.907+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4bbc01c460 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.907+0000 7f4bc1ffb700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 0x7f4bb0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.907+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f4bbc07c9f0 con 0x7f4bcc0af0b0 2023-10-06T11:34:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.911+0000 7f4bd1f61700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 0x7f4bb0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.911+0000 7f4bd1f61700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 0x7f4bb0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009880 tx=0x7f4bc8018000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:36.911+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4bcc15ac80 con 0x7f4bcc0af0b0 2023-10-06T11:34:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.123+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bcc15ac10 con 0x7f4bcc0af0b0 2023-10-06T11:34:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.123+0000 7f4bc1ffb700 1 -- 172.21.15.46:0/2172352152 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f4bbc04cb10 con 0x7f4bcc0af0b0 2023-10-06T11:34:37.134 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:37.134 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 msgr2=0x7f4bb0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 0x7f4bb0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009880 tx=0x7f4bc8018000 comp rx=0 tx=0).stop 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 msgr2=0x7f4bcc155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c7c0 tx=0x7f4bbc007f10 comp rx=0 tx=0).stop 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 shutdown_connections 2023-10-06T11:34:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bb0060670 0x7f4bb0062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bcc0cc2d0 0x7f4bcc156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bcc0af0b0 0x7f4bcc155b50 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 --2- 172.21.15.46:0/2172352152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc067050 0x7f4bcc15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 >> 172.21.15.46:0/2172352152 conn(0x7f4bcc068bf0 msgr2=0x7f4bcc158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 shutdown_connections 2023-10-06T11:34:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.127+0000 7f4bd41c5700 1 -- 172.21.15.46:0/2172352152 wait complete. 2023-10-06T11:34:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:34:37.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:34:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 Processor -- start 2023-10-06T11:34:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 -- start start 2023-10-06T11:34:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48780d07b0 0x7f48780b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48780b1130 0x7f48780ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780ae340 con 0x7f4878159290 2023-10-06T11:34:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780b15b0 con 0x7f48780d07b0 2023-10-06T11:34:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487ca2d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780b09a0 con 0x7f48780b1130 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48780d07b0 0x7f48780b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48780d07b0 0x7f48780b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38720/0 (socket says 172.21.15.46:38720) 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.311+0000 7f487659c700 1 -- 172.21.15.46:0/769544875 learned_addr learned my addr 172.21.15.46:0/769544875 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487759e700 1 --2- 172.21.15.46:0/769544875 >> v2:172.21.15.132:3300/0 conn(0x7f48780b1130 0x7f48780ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487659c700 1 -- 172.21.15.46:0/769544875 >> v2:172.21.15.132:3300/0 conn(0x7f48780b1130 msgr2=0x7f48780ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487659c700 1 --2- 172.21.15.46:0/769544875 >> v2:172.21.15.132:3300/0 conn(0x7f48780b1130 0x7f48780ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487659c700 1 -- 172.21.15.46:0/769544875 >> v1:172.21.15.132:6789/0 conn(0x7f4878159290 legacy=0x7f48780b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487659c700 1 -- 172.21.15.46:0/769544875 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878159660 con 0x7f48780d07b0 2023-10-06T11:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487659c700 1 --2- 172.21.15.46:0/769544875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48780d07b0 0x7f48780b0bf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4878089490 tx=0x7f486c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe02729aad9a8be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f4875d9b700 1 -- 172.21.15.46:0/769544875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c009e70 con 0x7f48780d07b0 2023-10-06T11:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f4875d9b700 1 -- 172.21.15.46:0/769544875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c00b8f0 con 0x7f48780d07b0 2023-10-06T11:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f4875d9b700 1 -- 172.21.15.46:0/769544875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00bc90 con 0x7f48780d07b0 2023-10-06T11:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 -- 172.21.15.46:0/769544875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48780d07b0 msgr2=0x7f48780b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 --2- 172.21.15.46:0/769544875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48780d07b0 0x7f48780b0bf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4878089490 tx=0x7f486c009250 comp rx=0 tx=0).stop 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 -- 172.21.15.46:0/769544875 shutdown_connections 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 --2- 172.21.15.46:0/769544875 >> v2:172.21.15.132:3300/0 conn(0x7f48780b1130 0x7f48780ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 --2- 172.21.15.46:0/769544875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48780d07b0 0x7f48780b0bf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 -- 172.21.15.46:0/769544875 >> 172.21.15.46:0/769544875 conn(0x7f487806ef60 msgr2=0x7f48780d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 -- 172.21.15.46:0/769544875 shutdown_connections 2023-10-06T11:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.315+0000 7f487ca2d700 1 -- 172.21.15.46:0/769544875 wait complete. 2023-10-06T11:34:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.319+0000 7f487ca2d700 1 Processor -- start 2023-10-06T11:34:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.319+0000 7f487ca2d700 1 -- start start 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48780b1130 0x7f487809b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48780d07b0 0x7f487816a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4878142430 con 0x7f4878159290 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487809d410 con 0x7f48780b1130 2023-10-06T11:34:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487815e370 con 0x7f48780d07b0 2023-10-06T11:34:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38728/0 (socket says 172.21.15.46:38728) 2023-10-06T11:34:37.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 -- 172.21.15.46:0/3520664288 learned_addr learned my addr 172.21.15.46:0/3520664288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487659c700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48780d07b0 0x7f487816a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f4876d9d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48780b1130 0x7f487809b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 -- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48780d07b0 msgr2=0x7f487816a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48780d07b0 0x7f487816a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 -- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48780b1130 msgr2=0x7f487809b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48780b1130 0x7f487809b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 -- 172.21.15.46:0/3520664288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0094f0 con 0x7f4878159290 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487759e700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e90 tx=0x7f487000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000cb70 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000f040 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487815e800 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870014480 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.323+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487815da40 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.327+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487815ec20 con 0x7f4878159290 2023-10-06T11:34:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.327+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4870014830 con 0x7f4878159290 2023-10-06T11:34:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.327+0000 7f485effd700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 0x7f4864062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.327+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f487007d740 con 0x7f4878159290 2023-10-06T11:34:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.331+0000 7f4876d9d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 0x7f4864062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.331+0000 7f4876d9d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 0x7f4864062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48680039e0 tx=0x7f4868007ec0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.331+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f487815ec20 con 0x7f4878159290 2023-10-06T11:34:37.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.503+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f487815ec20 con 0x7f4878159290 2023-10-06T11:34:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.507+0000 7f485effd700 1 -- 172.21.15.46:0/3520664288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17551 (secure 0 0 0) 0x7f487815ec20 con 0x7f4878159290 2023-10-06T11:34:37.517 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:37.517 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:34:34.637275+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 msgr2=0x7f4864062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 0x7f4864062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f48680039e0 tx=0x7f4868007ec0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 msgr2=0x7f487815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4870009e90 tx=0x7f487000b040 comp rx=0 tx=0).stop 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 shutdown_connections 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4864060670 0x7f4864062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878159290 0x7f487815a9d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48780d07b0 0x7f487816a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 --2- 172.21.15.46:0/3520664288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48780b1130 0x7f487809b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 >> 172.21.15.46:0/3520664288 conn(0x7f487806ef60 msgr2=0x7f48781585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 shutdown_connections 2023-10-06T11:34:37.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.511+0000 7f487ca2d700 1 -- 172.21.15.46:0/3520664288 wait complete. 2023-10-06T11:34:37.540 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:34:37.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:34:37.596 INFO:tasks.ceph.mds.d.smithi132.stderr:2023-10-06T11:34:37.592+0000 7f0c99fbc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:34:37.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.695+0000 7fcdeeea5700 1 Processor -- start 2023-10-06T11:34:37.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.695+0000 7fcdeeea5700 1 -- start start 2023-10-06T11:34:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdeeea5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcde81502f0 0x7fcde80c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdeeea5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81506c0 0x7fcde8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdeeea5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80c67e0 con 0x7fcde806e2a0 2023-10-06T11:34:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdeeea5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde806e810 con 0x7fcde81502f0 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdeeea5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806eaf0 con 0x7fcde81506c0 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcde81502f0 0x7fcde80c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcde81502f0 0x7fcde80c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:51414/0 (socket says 172.21.15.46:51414) 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 -- 172.21.15.46:0/411456797 learned_addr learned my addr 172.21.15.46:0/411456797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcde7fff700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81506c0 0x7fcde8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 -- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81506c0 msgr2=0x7fcde8159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81506c0 0x7fcde8159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 -- 172.21.15.46:0/411456797 >> v1:172.21.15.132:6789/0 conn(0x7fcde806e2a0 legacy=0x7fcde8156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 -- 172.21.15.46:0/411456797 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde806edd0 con 0x7fcde81502f0 2023-10-06T11:34:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.699+0000 7fcdecc41700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81502f0 0x7fcde80c0bd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1872a39fbb1aca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcde77fe700 1 -- 172.21.15.46:0/411456797 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc009e80 con 0x7fcde81502f0 2023-10-06T11:34:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcde77fe700 1 -- 172.21.15.46:0/411456797 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc005c10 con 0x7fcde81502f0 2023-10-06T11:34:37.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcde77fe700 1 -- 172.21.15.46:0/411456797 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00b650 con 0x7fcde81502f0 2023-10-06T11:34:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 -- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81502f0 msgr2=0x7fcde80c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81502f0 0x7fcde80c0bd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009220 comp rx=0 tx=0).stop 2023-10-06T11:34:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 -- 172.21.15.46:0/411456797 shutdown_connections 2023-10-06T11:34:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81506c0 0x7fcde8159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/411456797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81502f0 0x7fcde80c0bd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 -- 172.21.15.46:0/411456797 >> 172.21.15.46:0/411456797 conn(0x7fcde80cbe30 msgr2=0x7fcde80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 -- 172.21.15.46:0/411456797 shutdown_connections 2023-10-06T11:34:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.707+0000 7fcdeeea5700 1 -- 172.21.15.46:0/411456797 wait complete. 2023-10-06T11:34:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.711+0000 7fcdeeea5700 1 Processor -- start 2023-10-06T11:34:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.711+0000 7fcdeeea5700 1 -- start start 2023-10-06T11:34:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 0x7fcde81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c56a0 con 0x7fcde806e2a0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c4a90 con 0x7fcde81506c0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c4d70 con 0x7fcde81502f0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcded442700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 0x7fcde81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:43612/0 (socket says 172.21.15.46:43612) 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde7fff700 1 -- 172.21.15.46:0/1764026897 learned_addr learned my addr 172.21.15.46:0/1764026897 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38736/0 (socket says 172.21.15.46:38736) 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 -- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 msgr2=0x7fcde80c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 -- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 msgr2=0x7fcde81434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 0x7fcde81434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 -- 172.21.15.46:0/1764026897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009480 con 0x7fcde806e2a0 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcded442700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 0x7fcde81434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdecc41700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcddc015000 tx=0x7fcddc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc011c30 con 0x7fcde806e2a0 2023-10-06T11:34:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc011d90 con 0x7fcde806e2a0 2023-10-06T11:34:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80c5050 con 0x7fcde806e2a0 2023-10-06T11:34:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc01b660 con 0x7fcde806e2a0 2023-10-06T11:34:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.715+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde814ce70 con 0x7fcde806e2a0 2023-10-06T11:34:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.719+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8063f90 con 0x7fcde806e2a0 2023-10-06T11:34:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.719+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fcddc01ba50 con 0x7fcde806e2a0 2023-10-06T11:34:37.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.723+0000 7fcde4ff9700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 0x7fcdcc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:37.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.723+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fcddc07e920 con 0x7fcde806e2a0 2023-10-06T11:34:37.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.723+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 0x7fcdcc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:37.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.723+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 0x7fcdcc062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcde8147430 tx=0x7fcdd40091a0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.727+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcddc04ea40 con 0x7fcde806e2a0 2023-10-06T11:34:37.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.891+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcde80c57e0 con 0x7fcdcc0606a0 2023-10-06T11:34:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.911+0000 7fcde4ff9700 1 -- 172.21.15.46:0/1764026897 <== mgr.6955 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308865 (secure 0 0 0) 0x7fcdd4011cd0 con 0x7fcdcc0606a0 2023-10-06T11:34:37.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:37.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 msgr2=0x7fcdcc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 0x7fcdcc062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcde8147430 tx=0x7fcdd40091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 msgr2=0x7fcde814c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcddc015000 tx=0x7fcddc016000 comp rx=0 tx=0).stop 2023-10-06T11:34:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 shutdown_connections 2023-10-06T11:34:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdcc0606a0 0x7fcdcc062b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcde81506c0 0x7fcde81434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcde81502f0 0x7fcde80c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 --2- 172.21.15.46:0/1764026897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcde806e2a0 0x7fcde814c930 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:37.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 >> 172.21.15.46:0/1764026897 conn(0x7fcde80cbe30 msgr2=0x7fcde806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:37.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 shutdown_connections 2023-10-06T11:34:37.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:37.919+0000 7fcdeeea5700 1 -- 172.21.15.46:0/1764026897 wait complete. 2023-10-06T11:34:37.932 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:34:37.951 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:34:37.239301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459336,"kb_used_data":11400,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752515384,"statfs":{"total":773094113280,"available":770575753216,"internally_reserved":0,"allocated":11673600,"data_stored":6534084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628741"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.658057+0000","last_change":"2023-10-06T11:34:33.658057+0000","last_active":"2023-10-06T11:34:33.658057+0000","last_peered":"2023-10-06T11:34:33.658057+0000","last_clean":"2023-10-06T11:34:33.658057+0000","last_became_active":"2023-10-06T11:34:33.657747+0000","last_became_peered":"2023-10-06T11:34:33.657747+0000","last_unstale":"2023-10-06T11:34:33.658057+0000","last_undegraded":"2023-10-06T11:34:33.658057+0000","last_fullsized":"2023-10-06T11:34:33.658057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:58.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679488+0000","last_change":"2023-10-06T11:34:33.673036+0000","last_active":"2023-10-06T11:34:33.679488+0000","last_peered":"2023-10-06T11:34:33.679488+0000","last_clean":"2023-10-06T11:34:33.679488+0000","last_became_active":"2023-10-06T11:34:33.672717+0000","last_became_peered":"2023-10-06T11:34:33.672717+0000","last_unstale":"2023-10-06T11:34:33.679488+0000","last_undegraded":"2023-10-06T11:34:33.679488+0000","last_fullsized":"2023-10-06T11:34:33.679488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:03.145906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.654916+0000","last_change":"2023-10-06T11:34:33.654916+0000","last_active":"2023-10-06T11:34:33.654916+0000","last_peered":"2023-10-06T11:34:33.654916+0000","last_clean":"2023-10-06T11:34:33.654916+0000","last_became_active":"2023-10-06T11:34:33.654378+0000","last_became_peered":"2023-10-06T11:34:33.654378+0000","last_unstale":"2023-10-06T11:34:33.654916+0000","last_undegraded":"2023-10-06T11:34:33.654916+0000","last_fullsized":"2023-10-06T11:34:33.654916+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:44.279258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676941+0000","last_change":"2023-10-06T11:34:33.676941+0000","last_active":"2023-10-06T11:34:33.676941+0000","last_peered":"2023-10-06T11:34:33.676941+0000","last_clean":"2023-10-06T11:34:33.676941+0000","last_became_active":"2023-10-06T11:34:33.676605+0000","last_became_peered":"2023-10-06T11:34:33.676605+0000","last_unstale":"2023-10-06T11:34:33.676941+0000","last_undegraded":"2023-10-06T11:34:33.676941+0000","last_fullsized":"2023-10-06T11:34:33.676941+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:33.820941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678848+0000","last_change":"2023-10-06T11:34:33.671777+0000","last_active":"2023-10-06T11:34:33.678848+0000","last_peered":"2023-10-06T11:34:33.678848+0000","last_clean":"2023-10-06T11:34:33.678848+0000","last_became_active":"2023-10-06T11:34:33.671363+0000","last_became_peered":"2023-10-06T11:34:33.671363+0000","last_unstale":"2023-10-06T11:34:33.678848+0000","last_undegraded":"2023-10-06T11:34:33.678848+0000","last_fullsized":"2023-10-06T11:34:33.678848+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:41.284844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684918+0000","last_change":"2023-10-06T11:34:33.683251+0000","last_active":"2023-10-06T11:34:33.684918+0000","last_peered":"2023-10-06T11:34:33.684918+0000","last_clean":"2023-10-06T11:34:33.684918+0000","last_became_active":"2023-10-06T11:34:33.682958+0000","last_became_peered":"2023-10-06T11:34:33.682958+0000","last_unstale":"2023-10-06T11:34:33.684918+0000","last_undegraded":"2023-10-06T11:34:33.684918+0000","last_fullsized":"2023-10-06T11:34:33.684918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:45.381724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.688029+0000","last_change":"2023-10-06T11:34:33.687131+0000","last_active":"2023-10-06T11:34:33.688029+0000","last_peered":"2023-10-06T11:34:33.688029+0000","last_clean":"2023-10-06T11:34:33.688029+0000","last_became_active":"2023-10-06T11:34:33.686775+0000","last_became_peered":"2023-10-06T11:34:33.686775+0000","last_unstale":"2023-10-06T11:34:33.688029+0000","last_undegraded":"2023-10-06T11:34:33.688029+0000","last_fullsized":"2023-10-06T11:34:33.688029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:42.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676379+0000","last_change":"2023-10-06T11:34:33.676379+0000","last_active":"2023-10-06T11:34:33.676379+0000","last_peered":"2023-10-06T11:34:33.676379+0000","last_clean":"2023-10-06T11:34:33.676379+0000","last_became_active":"2023-10-06T11:34:33.675284+0000","last_became_peered":"2023-10-06T11:34:33.675284+0000","last_unstale":"2023-10-06T11:34:33.676379+0000","last_undegraded":"2023-10-06T11:34:33.676379+0000","last_fullsized":"2023-10-06T11:34:33.676379+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:16.715165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680598+0000","last_change":"2023-10-06T11:34:33.677278+0000","last_active":"2023-10-06T11:34:33.680598+0000","last_peered":"2023-10-06T11:34:33.680598+0000","last_clean":"2023-10-06T11:34:33.680598+0000","last_became_active":"2023-10-06T11:34:33.676979+0000","last_became_peered":"2023-10-06T11:34:33.676979+0000","last_unstale":"2023-10-06T11:34:33.680598+0000","last_undegraded":"2023-10-06T11:34:33.680598+0000","last_fullsized":"2023-10-06T11:34:33.680598+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:31.489008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680146+0000","last_change":"2023-10-06T11:34:33.680146+0000","last_active":"2023-10-06T11:34:33.680146+0000","last_peered":"2023-10-06T11:34:33.680146+0000","last_clean":"2023-10-06T11:34:33.680146+0000","last_became_active":"2023-10-06T11:34:33.679772+0000","last_became_peered":"2023-10-06T11:34:33.679772+0000","last_unstale":"2023-10-06T11:34:33.680146+0000","last_undegraded":"2023-10-06T11:34:33.680146+0000","last_fullsized":"2023-10-06T11:34:33.680146+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:09.576482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.669872+0000","last_change":"2023-10-06T11:34:33.664254+0000","last_active":"2023-10-06T11:34:33.669872+0000","last_peered":"2023-10-06T11:34:33.669872+0000","last_clean":"2023-10-06T11:34:33.669872+0000","last_became_active":"2023-10-06T11:34:33.663847+0000","last_became_peered":"2023-10-06T11:34:33.663847+0000","last_unstale":"2023-10-06T11:34:33.669872+0000","last_undegraded":"2023-10-06T11:34:33.669872+0000","last_fullsized":"2023-10-06T11:34:33.669872+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:44.487912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.660252+0000","last_change":"2023-10-06T11:34:33.660252+0000","last_active":"2023-10-06T11:34:33.660252+0000","last_peered":"2023-10-06T11:34:33.660252+0000","last_clean":"2023-10-06T11:34:33.660252+0000","last_became_active":"2023-10-06T11:34:33.659887+0000","last_became_peered":"2023-10-06T11:34:33.659887+0000","last_unstale":"2023-10-06T11:34:33.660252+0000","last_undegraded":"2023-10-06T11:34:33.660252+0000","last_fullsized":"2023-10-06T11:34:33.660252+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:56:31.867762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.675401+0000","last_change":"2023-10-06T11:34:33.675401+0000","last_active":"2023-10-06T11:34:33.675401+0000","last_peered":"2023-10-06T11:34:33.675401+0000","last_clean":"2023-10-06T11:34:33.675401+0000","last_became_active":"2023-10-06T11:34:33.675096+0000","last_became_peered":"2023-10-06T11:34:33.675096+0000","last_unstale":"2023-10-06T11:34:33.675401+0000","last_undegraded":"2023-10-06T11:34:33.675401+0000","last_fullsized":"2023-10-06T11:34:33.675401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:24.870251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694908+0000","last_change":"2023-10-06T11:34:33.693665+0000","last_active":"2023-10-06T11:34:33.694908+0000","last_peered":"2023-10-06T11:34:33.694908+0000","last_clean":"2023-10-06T11:34:33.694908+0000","last_became_active":"2023-10-06T11:34:33.693388+0000","last_became_peered":"2023-10-06T11:34:33.693388+0000","last_unstale":"2023-10-06T11:34:33.694908+0000","last_undegraded":"2023-10-06T11:34:33.694908+0000","last_fullsized":"2023-10-06T11:34:33.694908+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:02.880995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685231+0000","last_change":"2023-10-06T11:34:33.685231+0000","last_active":"2023-10-06T11:34:33.685231+0000","last_peered":"2023-10-06T11:34:33.685231+0000","last_clean":"2023-10-06T11:34:33.685231+0000","last_became_active":"2023-10-06T11:34:33.684907+0000","last_became_peered":"2023-10-06T11:34:33.684907+0000","last_unstale":"2023-10-06T11:34:33.685231+0000","last_undegraded":"2023-10-06T11:34:33.685231+0000","last_fullsized":"2023-10-06T11:34:33.685231+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:53.929485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665740+0000","last_change":"2023-10-06T11:34:33.662241+0000","last_active":"2023-10-06T11:34:33.665740+0000","last_peered":"2023-10-06T11:34:33.665740+0000","last_clean":"2023-10-06T11:34:33.665740+0000","last_became_active":"2023-10-06T11:34:33.661744+0000","last_became_peered":"2023-10-06T11:34:33.661744+0000","last_unstale":"2023-10-06T11:34:33.665740+0000","last_undegraded":"2023-10-06T11:34:33.665740+0000","last_fullsized":"2023-10-06T11:34:33.665740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:57.259900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665453+0000","last_change":"2023-10-06T11:34:33.658292+0000","last_active":"2023-10-06T11:34:33.665453+0000","last_peered":"2023-10-06T11:34:33.665453+0000","last_clean":"2023-10-06T11:34:33.665453+0000","last_became_active":"2023-10-06T11:34:33.657990+0000","last_became_peered":"2023-10-06T11:34:33.657990+0000","last_unstale":"2023-10-06T11:34:33.665453+0000","last_undegraded":"2023-10-06T11:34:33.665453+0000","last_fullsized":"2023-10-06T11:34:33.665453+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:46.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670401+0000","last_change":"2023-10-06T11:34:33.670401+0000","last_active":"2023-10-06T11:34:33.670401+0000","last_peered":"2023-10-06T11:34:33.670401+0000","last_clean":"2023-10-06T11:34:33.670401+0000","last_became_active":"2023-10-06T11:34:33.670112+0000","last_became_peered":"2023-10-06T11:34:33.670112+0000","last_unstale":"2023-10-06T11:34:33.670401+0000","last_undegraded":"2023-10-06T11:34:33.670401+0000","last_fullsized":"2023-10-06T11:34:33.670401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.555298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.687178+0000","last_change":"2023-10-06T11:34:33.684575+0000","last_active":"2023-10-06T11:34:33.687178+0000","last_peered":"2023-10-06T11:34:33.687178+0000","last_clean":"2023-10-06T11:34:33.687178+0000","last_became_active":"2023-10-06T11:34:33.684281+0000","last_became_peered":"2023-10-06T11:34:33.684281+0000","last_unstale":"2023-10-06T11:34:33.687178+0000","last_undegraded":"2023-10-06T11:34:33.687178+0000","last_fullsized":"2023-10-06T11:34:33.687178+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:26.697053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676893+0000","last_change":"2023-10-06T11:34:33.676893+0000","last_active":"2023-10-06T11:34:33.676893+0000","last_peered":"2023-10-06T11:34:33.676893+0000","last_clean":"2023-10-06T11:34:33.676893+0000","last_became_active":"2023-10-06T11:34:33.676216+0000","last_became_peered":"2023-10-06T11:34:33.676216+0000","last_unstale":"2023-10-06T11:34:33.676893+0000","last_undegraded":"2023-10-06T11:34:33.676893+0000","last_fullsized":"2023-10-06T11:34:33.676893+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:49.270036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678869+0000","last_change":"2023-10-06T11:34:33.673680+0000","last_active":"2023-10-06T11:34:33.678869+0000","last_peered":"2023-10-06T11:34:33.678869+0000","last_clean":"2023-10-06T11:34:33.678869+0000","last_became_active":"2023-10-06T11:34:33.673361+0000","last_became_peered":"2023-10-06T11:34:33.673361+0000","last_unstale":"2023-10-06T11:34:33.678869+0000","last_undegraded":"2023-10-06T11:34:33.678869+0000","last_fullsized":"2023-10-06T11:34:33.678869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:37.851550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694891+0000","last_change":"2023-10-06T11:34:33.693460+0000","last_active":"2023-10-06T11:34:33.694891+0000","last_peered":"2023-10-06T11:34:33.694891+0000","last_clean":"2023-10-06T11:34:33.694891+0000","last_became_active":"2023-10-06T11:34:33.693141+0000","last_became_peered":"2023-10-06T11:34:33.693141+0000","last_unstale":"2023-10-06T11:34:33.694891+0000","last_undegraded":"2023-10-06T11:34:33.694891+0000","last_fullsized":"2023-10-06T11:34:33.694891+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:43.276708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.693615+0000","last_change":"2023-10-06T11:34:33.693615+0000","last_active":"2023-10-06T11:34:33.693615+0000","last_peered":"2023-10-06T11:34:33.693615+0000","last_clean":"2023-10-06T11:34:33.693615+0000","last_became_active":"2023-10-06T11:34:33.693240+0000","last_became_peered":"2023-10-06T11:34:33.693240+0000","last_unstale":"2023-10-06T11:34:33.693615+0000","last_undegraded":"2023-10-06T11:34:33.693615+0000","last_fullsized":"2023-10-06T11:34:33.693615+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:41.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.654820+0000","last_change":"2023-10-06T11:34:33.654820+0000","last_active":"2023-10-06T11:34:33.654820+0000","last_peered":"2023-10-06T11:34:33.654820+0000","last_clean":"2023-10-06T11:34:33.654820+0000","last_became_active":"2023-10-06T11:34:33.654478+0000","last_became_peered":"2023-10-06T11:34:33.654478+0000","last_unstale":"2023-10-06T11:34:33.654820+0000","last_undegraded":"2023-10-06T11:34:33.654820+0000","last_fullsized":"2023-10-06T11:34:33.654820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:43.911900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679868+0000","last_change":"2023-10-06T11:34:33.674067+0000","last_active":"2023-10-06T11:34:33.679868+0000","last_peered":"2023-10-06T11:34:33.679868+0000","last_clean":"2023-10-06T11:34:33.679868+0000","last_became_active":"2023-10-06T11:34:33.673182+0000","last_became_peered":"2023-10-06T11:34:33.673182+0000","last_unstale":"2023-10-06T11:34:33.679868+0000","last_undegraded":"2023-10-06T11:34:33.679868+0000","last_fullsized":"2023-10-06T11:34:33.679868+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:31.489008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.687463+0000","last_change":"2023-10-06T11:34:33.685847+0000","last_active":"2023-10-06T11:34:33.687463+0000","last_peered":"2023-10-06T11:34:33.687463+0000","last_clean":"2023-10-06T11:34:33.687463+0000","last_became_active":"2023-10-06T11:34:33.685560+0000","last_became_peered":"2023-10-06T11:34:33.685560+0000","last_unstale":"2023-10-06T11:34:33.687463+0000","last_undegraded":"2023-10-06T11:34:33.687463+0000","last_fullsized":"2023-10-06T11:34:33.687463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:42.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.664841+0000","last_change":"2023-10-06T11:34:33.664841+0000","last_active":"2023-10-06T11:34:33.664841+0000","last_peered":"2023-10-06T11:34:33.664841+0000","last_clean":"2023-10-06T11:34:33.664841+0000","last_became_active":"2023-10-06T11:34:33.664526+0000","last_became_peered":"2023-10-06T11:34:33.664526+0000","last_unstale":"2023-10-06T11:34:33.664841+0000","last_undegraded":"2023-10-06T11:34:33.664841+0000","last_fullsized":"2023-10-06T11:34:33.664841+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:37.826261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676661+0000","last_change":"2023-10-06T11:34:33.676661+0000","last_active":"2023-10-06T11:34:33.676661+0000","last_peered":"2023-10-06T11:34:33.676661+0000","last_clean":"2023-10-06T11:34:33.676661+0000","last_became_active":"2023-10-06T11:34:33.675273+0000","last_became_peered":"2023-10-06T11:34:33.675273+0000","last_unstale":"2023-10-06T11:34:33.676661+0000","last_undegraded":"2023-10-06T11:34:33.676661+0000","last_fullsized":"2023-10-06T11:34:33.676661+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:15.684545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680444+0000","last_change":"2023-10-06T11:34:33.678142+0000","last_active":"2023-10-06T11:34:33.680444+0000","last_peered":"2023-10-06T11:34:33.680444+0000","last_clean":"2023-10-06T11:34:33.680444+0000","last_became_active":"2023-10-06T11:34:33.677836+0000","last_became_peered":"2023-10-06T11:34:33.677836+0000","last_unstale":"2023-10-06T11:34:33.680444+0000","last_undegraded":"2023-10-06T11:34:33.680444+0000","last_fullsized":"2023-10-06T11:34:33.680444+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:02.880995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680638+0000","last_change":"2023-10-06T11:34:33.677324+0000","last_active":"2023-10-06T11:34:33.680638+0000","last_peered":"2023-10-06T11:34:33.680638+0000","last_clean":"2023-10-06T11:34:33.680638+0000","last_became_active":"2023-10-06T11:34:33.677034+0000","last_became_peered":"2023-10-06T11:34:33.677034+0000","last_unstale":"2023-10-06T11:34:33.680638+0000","last_undegraded":"2023-10-06T11:34:33.680638+0000","last_fullsized":"2023-10-06T11:34:33.680638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:10.885411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694965+0000","last_change":"2023-10-06T11:34:33.692650+0000","last_active":"2023-10-06T11:34:33.694965+0000","last_peered":"2023-10-06T11:34:33.694965+0000","last_clean":"2023-10-06T11:34:33.694965+0000","last_became_active":"2023-10-06T11:34:33.692216+0000","last_became_peered":"2023-10-06T11:34:33.692216+0000","last_unstale":"2023-10-06T11:34:33.694965+0000","last_undegraded":"2023-10-06T11:34:33.694965+0000","last_fullsized":"2023-10-06T11:34:33.694965+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:41.284844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665560+0000","last_change":"2023-10-06T11:34:33.662040+0000","last_active":"2023-10-06T11:34:33.665560+0000","last_peered":"2023-10-06T11:34:33.665560+0000","last_clean":"2023-10-06T11:34:33.665560+0000","last_became_active":"2023-10-06T11:34:33.661363+0000","last_became_peered":"2023-10-06T11:34:33.661363+0000","last_unstale":"2023-10-06T11:34:33.665560+0000","last_undegraded":"2023-10-06T11:34:33.665560+0000","last_fullsized":"2023-10-06T11:34:33.665560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:46.992557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661008+0000","last_change":"2023-10-06T11:34:33.657115+0000","last_active":"2023-10-06T11:34:33.661008+0000","last_peered":"2023-10-06T11:34:33.661008+0000","last_clean":"2023-10-06T11:34:33.661008+0000","last_became_active":"2023-10-06T11:34:33.656821+0000","last_became_peered":"2023-10-06T11:34:33.656821+0000","last_unstale":"2023-10-06T11:34:33.661008+0000","last_undegraded":"2023-10-06T11:34:33.661008+0000","last_fullsized":"2023-10-06T11:34:33.661008+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680610+0000","last_change":"2023-10-06T11:34:32.657300+0000","last_active":"2023-10-06T11:34:33.680610+0000","last_peered":"2023-10-06T11:34:33.680610+0000","last_clean":"2023-10-06T11:34:33.680610+0000","last_became_active":"2023-10-06T11:34:32.656968+0000","last_became_peered":"2023-10-06T11:34:32.656968+0000","last_unstale":"2023-10-06T11:34:33.680610+0000","last_undegraded":"2023-10-06T11:34:33.680610+0000","last_fullsized":"2023-10-06T11:34:33.680610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:14.671378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.669154+0000","last_change":"2023-10-06T11:34:33.663688+0000","last_active":"2023-10-06T11:34:33.669154+0000","last_peered":"2023-10-06T11:34:33.669154+0000","last_clean":"2023-10-06T11:34:33.669154+0000","last_became_active":"2023-10-06T11:34:33.663027+0000","last_became_peered":"2023-10-06T11:34:33.663027+0000","last_unstale":"2023-10-06T11:34:33.669154+0000","last_undegraded":"2023-10-06T11:34:33.669154+0000","last_fullsized":"2023-10-06T11:34:33.669154+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:17.151153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.674242+0000","last_change":"2023-10-06T11:34:32.647592+0000","last_active":"2023-10-06T11:34:33.674242+0000","last_peered":"2023-10-06T11:34:33.674242+0000","last_clean":"2023-10-06T11:34:33.674242+0000","last_became_active":"2023-10-06T11:34:32.647230+0000","last_became_peered":"2023-10-06T11:34:32.647230+0000","last_unstale":"2023-10-06T11:34:33.674242+0000","last_undegraded":"2023-10-06T11:34:33.674242+0000","last_fullsized":"2023-10-06T11:34:33.674242+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694453+0000","last_change":"2023-10-06T11:34:33.692671+0000","last_active":"2023-10-06T11:34:33.694453+0000","last_peered":"2023-10-06T11:34:33.694453+0000","last_clean":"2023-10-06T11:34:33.694453+0000","last_became_active":"2023-10-06T11:34:33.692313+0000","last_became_peered":"2023-10-06T11:34:33.692313+0000","last_unstale":"2023-10-06T11:34:33.694453+0000","last_undegraded":"2023-10-06T11:34:33.694453+0000","last_fullsized":"2023-10-06T11:34:33.694453+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:44.487912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679866+0000","last_change":"2023-10-06T11:34:32.649377+0000","last_active":"2023-10-06T11:34:33.679866+0000","last_peered":"2023-10-06T11:34:33.679866+0000","last_clean":"2023-10-06T11:34:33.679866+0000","last_became_active":"2023-10-06T11:34:32.649105+0000","last_became_peered":"2023-10-06T11:34:32.649105+0000","last_unstale":"2023-10-06T11:34:33.679866+0000","last_undegraded":"2023-10-06T11:34:33.679866+0000","last_fullsized":"2023-10-06T11:34:33.679866+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:02.357176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670760+0000","last_change":"2023-10-06T11:34:33.667041+0000","last_active":"2023-10-06T11:34:33.670760+0000","last_peered":"2023-10-06T11:34:33.670760+0000","last_clean":"2023-10-06T11:34:33.670760+0000","last_became_active":"2023-10-06T11:34:33.666730+0000","last_became_peered":"2023-10-06T11:34:33.666730+0000","last_unstale":"2023-10-06T11:34:33.670760+0000","last_undegraded":"2023-10-06T11:34:33.670760+0000","last_fullsized":"2023-10-06T11:34:33.670760+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:36.837595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.692907+0000","last_change":"2023-10-06T11:34:32.655485+0000","last_active":"2023-10-06T11:34:33.692907+0000","last_peered":"2023-10-06T11:34:33.692907+0000","last_clean":"2023-10-06T11:34:33.692907+0000","last_became_active":"2023-10-06T11:34:32.655097+0000","last_became_peered":"2023-10-06T11:34:32.655097+0000","last_unstale":"2023-10-06T11:34:33.692907+0000","last_undegraded":"2023-10-06T11:34:33.692907+0000","last_fullsized":"2023-10-06T11:34:33.692907+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:47.664215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685926+0000","last_change":"2023-10-06T11:34:33.685926+0000","last_active":"2023-10-06T11:34:33.685926+0000","last_peered":"2023-10-06T11:34:33.685926+0000","last_clean":"2023-10-06T11:34:33.685926+0000","last_became_active":"2023-10-06T11:34:33.685622+0000","last_became_peered":"2023-10-06T11:34:33.685622+0000","last_unstale":"2023-10-06T11:34:33.685926+0000","last_undegraded":"2023-10-06T11:34:33.685926+0000","last_fullsized":"2023-10-06T11:34:33.685926+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:48.677381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679642+0000","last_change":"2023-10-06T11:34:32.649956+0000","last_active":"2023-10-06T11:34:33.679642+0000","last_peered":"2023-10-06T11:34:33.679642+0000","last_clean":"2023-10-06T11:34:33.679642+0000","last_became_active":"2023-10-06T11:34:32.649694+0000","last_became_peered":"2023-10-06T11:34:32.649694+0000","last_unstale":"2023-10-06T11:34:33.679642+0000","last_undegraded":"2023-10-06T11:34:33.679642+0000","last_fullsized":"2023-10-06T11:34:33.679642+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.672492+0000","last_change":"2023-10-06T11:34:33.672492+0000","last_active":"2023-10-06T11:34:33.672492+0000","last_peered":"2023-10-06T11:34:33.672492+0000","last_clean":"2023-10-06T11:34:33.672492+0000","last_became_active":"2023-10-06T11:34:33.672202+0000","last_became_peered":"2023-10-06T11:34:33.672202+0000","last_unstale":"2023-10-06T11:34:33.672492+0000","last_undegraded":"2023-10-06T11:34:33.672492+0000","last_fullsized":"2023-10-06T11:34:33.672492+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:43.911900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.668851+0000","last_change":"2023-10-06T11:34:32.645279+0000","last_active":"2023-10-06T11:34:33.668851+0000","last_peered":"2023-10-06T11:34:33.668851+0000","last_clean":"2023-10-06T11:34:33.668851+0000","last_became_active":"2023-10-06T11:34:32.644888+0000","last_became_peered":"2023-10-06T11:34:32.644888+0000","last_unstale":"2023-10-06T11:34:33.668851+0000","last_undegraded":"2023-10-06T11:34:33.668851+0000","last_fullsized":"2023-10-06T11:34:33.668851+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:56.770037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684942+0000","last_change":"2023-10-06T11:34:33.683623+0000","last_active":"2023-10-06T11:34:33.684942+0000","last_peered":"2023-10-06T11:34:33.684942+0000","last_clean":"2023-10-06T11:34:33.684942+0000","last_became_active":"2023-10-06T11:34:33.683305+0000","last_became_peered":"2023-10-06T11:34:33.683305+0000","last_unstale":"2023-10-06T11:34:33.684942+0000","last_undegraded":"2023-10-06T11:34:33.684942+0000","last_fullsized":"2023-10-06T11:34:33.684942+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:46.879894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.647822+0000","last_change":"2023-10-06T11:34:32.645455+0000","last_active":"2023-10-06T11:34:32.647822+0000","last_peered":"2023-10-06T11:34:32.647822+0000","last_clean":"2023-10-06T11:34:32.647822+0000","last_became_active":"2023-10-06T11:34:32.645124+0000","last_became_peered":"2023-10-06T11:34:32.645124+0000","last_unstale":"2023-10-06T11:34:32.647822+0000","last_undegraded":"2023-10-06T11:34:32.647822+0000","last_fullsized":"2023-10-06T11:34:32.647822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:30.672956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683162+0000","last_change":"2023-10-06T11:34:33.683162+0000","last_active":"2023-10-06T11:34:33.683162+0000","last_peered":"2023-10-06T11:34:33.683162+0000","last_clean":"2023-10-06T11:34:33.683162+0000","last_became_active":"2023-10-06T11:34:33.682638+0000","last_became_peered":"2023-10-06T11:34:33.682638+0000","last_unstale":"2023-10-06T11:34:33.683162+0000","last_undegraded":"2023-10-06T11:34:33.683162+0000","last_fullsized":"2023-10-06T11:34:33.683162+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:18.021965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665586+0000","last_change":"2023-10-06T11:34:32.654888+0000","last_active":"2023-10-06T11:34:33.665586+0000","last_peered":"2023-10-06T11:34:33.665586+0000","last_clean":"2023-10-06T11:34:33.665586+0000","last_became_active":"2023-10-06T11:34:32.654616+0000","last_became_peered":"2023-10-06T11:34:32.654616+0000","last_unstale":"2023-10-06T11:34:33.665586+0000","last_undegraded":"2023-10-06T11:34:33.665586+0000","last_fullsized":"2023-10-06T11:34:33.665586+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:59.992301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667495+0000","last_change":"2023-10-06T11:34:33.661439+0000","last_active":"2023-10-06T11:34:33.667495+0000","last_peered":"2023-10-06T11:34:33.667495+0000","last_clean":"2023-10-06T11:34:33.667495+0000","last_became_active":"2023-10-06T11:34:33.661081+0000","last_became_peered":"2023-10-06T11:34:33.661081+0000","last_unstale":"2023-10-06T11:34:33.667495+0000","last_undegraded":"2023-10-06T11:34:33.667495+0000","last_fullsized":"2023-10-06T11:34:33.667495+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:21.956959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661600+0000","last_change":"2023-10-06T11:34:32.648019+0000","last_active":"2023-10-06T11:34:33.661600+0000","last_peered":"2023-10-06T11:34:33.661600+0000","last_clean":"2023-10-06T11:34:33.661600+0000","last_became_active":"2023-10-06T11:34:32.647715+0000","last_became_peered":"2023-10-06T11:34:32.647715+0000","last_unstale":"2023-10-06T11:34:33.661600+0000","last_undegraded":"2023-10-06T11:34:33.661600+0000","last_fullsized":"2023-10-06T11:34:33.661600+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:59.542131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680555+0000","last_change":"2023-10-06T11:34:33.675319+0000","last_active":"2023-10-06T11:34:33.680555+0000","last_peered":"2023-10-06T11:34:33.680555+0000","last_clean":"2023-10-06T11:34:33.680555+0000","last_became_active":"2023-10-06T11:34:33.675029+0000","last_became_peered":"2023-10-06T11:34:33.675029+0000","last_unstale":"2023-10-06T11:34:33.680555+0000","last_undegraded":"2023-10-06T11:34:33.680555+0000","last_fullsized":"2023-10-06T11:34:33.680555+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:43.276708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.648133+0000","last_change":"2023-10-06T11:34:32.645376+0000","last_active":"2023-10-06T11:34:32.648133+0000","last_peered":"2023-10-06T11:34:32.648133+0000","last_clean":"2023-10-06T11:34:32.648133+0000","last_became_active":"2023-10-06T11:34:32.645035+0000","last_became_peered":"2023-10-06T11:34:32.645035+0000","last_unstale":"2023-10-06T11:34:32.648133+0000","last_undegraded":"2023-10-06T11:34:32.648133+0000","last_fullsized":"2023-10-06T11:34:32.648133+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:32.807774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.657062+0000","last_change":"2023-10-06T11:34:33.657062+0000","last_active":"2023-10-06T11:34:33.657062+0000","last_peered":"2023-10-06T11:34:33.657062+0000","last_clean":"2023-10-06T11:34:33.657062+0000","last_became_active":"2023-10-06T11:34:33.656765+0000","last_became_peered":"2023-10-06T11:34:33.656765+0000","last_unstale":"2023-10-06T11:34:33.657062+0000","last_undegraded":"2023-10-06T11:34:33.657062+0000","last_fullsized":"2023-10-06T11:34:33.657062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:39.610289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679228+0000","last_change":"2023-10-06T11:34:32.655778+0000","last_active":"2023-10-06T11:34:33.679228+0000","last_peered":"2023-10-06T11:34:33.679228+0000","last_clean":"2023-10-06T11:34:33.679228+0000","last_became_active":"2023-10-06T11:34:32.655472+0000","last_became_peered":"2023-10-06T11:34:32.655472+0000","last_unstale":"2023-10-06T11:34:33.679228+0000","last_undegraded":"2023-10-06T11:34:33.679228+0000","last_fullsized":"2023-10-06T11:34:33.679228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:26.335636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683558+0000","last_change":"2023-10-06T11:34:33.683558+0000","last_active":"2023-10-06T11:34:33.683558+0000","last_peered":"2023-10-06T11:34:33.683558+0000","last_clean":"2023-10-06T11:34:33.683558+0000","last_became_active":"2023-10-06T11:34:33.682593+0000","last_became_peered":"2023-10-06T11:34:33.682593+0000","last_unstale":"2023-10-06T11:34:33.683558+0000","last_undegraded":"2023-10-06T11:34:33.683558+0000","last_fullsized":"2023-10-06T11:34:33.683558+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:42.405551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683118+0000","last_change":"2023-10-06T11:34:32.644341+0000","last_active":"2023-10-06T11:34:33.683118+0000","last_peered":"2023-10-06T11:34:33.683118+0000","last_clean":"2023-10-06T11:34:33.683118+0000","last_became_active":"2023-10-06T11:34:32.643800+0000","last_became_peered":"2023-10-06T11:34:32.643800+0000","last_unstale":"2023-10-06T11:34:33.683118+0000","last_undegraded":"2023-10-06T11:34:33.683118+0000","last_fullsized":"2023-10-06T11:34:33.683118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:36.438764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676840+0000","last_change":"2023-10-06T11:34:33.676840+0000","last_active":"2023-10-06T11:34:33.676840+0000","last_peered":"2023-10-06T11:34:33.676840+0000","last_clean":"2023-10-06T11:34:33.676840+0000","last_became_active":"2023-10-06T11:34:33.676372+0000","last_became_peered":"2023-10-06T11:34:33.676372+0000","last_unstale":"2023-10-06T11:34:33.676840+0000","last_undegraded":"2023-10-06T11:34:33.676840+0000","last_fullsized":"2023-10-06T11:34:33.676840+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:09.455819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679268+0000","last_change":"2023-10-06T11:34:32.657277+0000","last_active":"2023-10-06T11:34:33.679268+0000","last_peered":"2023-10-06T11:34:33.679268+0000","last_clean":"2023-10-06T11:34:33.679268+0000","last_became_active":"2023-10-06T11:34:32.656940+0000","last_became_peered":"2023-10-06T11:34:32.656940+0000","last_unstale":"2023-10-06T11:34:33.679268+0000","last_undegraded":"2023-10-06T11:34:33.679268+0000","last_fullsized":"2023-10-06T11:34:33.679268+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:30.475842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678816+0000","last_change":"2023-10-06T11:34:33.671809+0000","last_active":"2023-10-06T11:34:33.678816+0000","last_peered":"2023-10-06T11:34:33.678816+0000","last_clean":"2023-10-06T11:34:33.678816+0000","last_became_active":"2023-10-06T11:34:33.671508+0000","last_became_peered":"2023-10-06T11:34:33.671508+0000","last_unstale":"2023-10-06T11:34:33.678816+0000","last_undegraded":"2023-10-06T11:34:33.678816+0000","last_fullsized":"2023-10-06T11:34:33.678816+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:46.220694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.650445+0000","last_change":"2023-10-06T11:34:32.643183+0000","last_active":"2023-10-06T11:34:33.650445+0000","last_peered":"2023-10-06T11:34:33.650445+0000","last_clean":"2023-10-06T11:34:33.650445+0000","last_became_active":"2023-10-06T11:34:32.642817+0000","last_became_peered":"2023-10-06T11:34:32.642817+0000","last_unstale":"2023-10-06T11:34:33.650445+0000","last_undegraded":"2023-10-06T11:34:33.650445+0000","last_fullsized":"2023-10-06T11:34:33.650445+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:00.944215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683254+0000","last_change":"2023-10-06T11:34:33.683254+0000","last_active":"2023-10-06T11:34:33.683254+0000","last_peered":"2023-10-06T11:34:33.683254+0000","last_clean":"2023-10-06T11:34:33.683254+0000","last_became_active":"2023-10-06T11:34:33.682947+0000","last_became_peered":"2023-10-06T11:34:33.682947+0000","last_unstale":"2023-10-06T11:34:33.683254+0000","last_undegraded":"2023-10-06T11:34:33.683254+0000","last_fullsized":"2023-10-06T11:34:33.683254+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:36.159994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643450+0000","last_change":"2023-10-06T11:34:32.641151+0000","last_active":"2023-10-06T11:34:32.643450+0000","last_peered":"2023-10-06T11:34:32.643450+0000","last_clean":"2023-10-06T11:34:32.643450+0000","last_became_active":"2023-10-06T11:34:32.640828+0000","last_became_peered":"2023-10-06T11:34:32.640828+0000","last_unstale":"2023-10-06T11:34:32.643450+0000","last_undegraded":"2023-10-06T11:34:32.643450+0000","last_fullsized":"2023-10-06T11:34:32.643450+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:46.017502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659861+0000","last_change":"2023-10-06T11:34:33.659861+0000","last_active":"2023-10-06T11:34:33.659861+0000","last_peered":"2023-10-06T11:34:33.659861+0000","last_clean":"2023-10-06T11:34:33.659861+0000","last_became_active":"2023-10-06T11:34:33.659544+0000","last_became_peered":"2023-10-06T11:34:33.659544+0000","last_unstale":"2023-10-06T11:34:33.659861+0000","last_undegraded":"2023-10-06T11:34:33.659861+0000","last_fullsized":"2023-10-06T11:34:33.659861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:40.124804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'8","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683428+0000","last_change":"2023-10-06T11:34:32.649537+0000","last_active":"2023-10-06T11:34:33.683428+0000","last_peered":"2023-10-06T11:34:33.683428+0000","last_clean":"2023-10-06T11:34:33.683428+0000","last_became_active":"2023-10-06T11:34:32.648576+0000","last_became_peered":"2023-10-06T11:34:32.648576+0000","last_unstale":"2023-10-06T11:34:33.683428+0000","last_undegraded":"2023-10-06T11:34:33.683428+0000","last_fullsized":"2023-10-06T11:34:33.683428+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:58.440067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676004+0000","last_change":"2023-10-06T11:34:33.676004+0000","last_active":"2023-10-06T11:34:33.676004+0000","last_peered":"2023-10-06T11:34:33.676004+0000","last_clean":"2023-10-06T11:34:33.676004+0000","last_became_active":"2023-10-06T11:34:33.675711+0000","last_became_peered":"2023-10-06T11:34:33.675711+0000","last_unstale":"2023-10-06T11:34:33.676004+0000","last_undegraded":"2023-10-06T11:34:33.676004+0000","last_fullsized":"2023-10-06T11:34:33.676004+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:05.528927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.677458+0000","last_change":"2023-10-06T11:34:32.648689+0000","last_active":"2023-10-06T11:34:33.677458+0000","last_peered":"2023-10-06T11:34:33.677458+0000","last_clean":"2023-10-06T11:34:33.677458+0000","last_became_active":"2023-10-06T11:34:32.648413+0000","last_became_peered":"2023-10-06T11:34:32.648413+0000","last_unstale":"2023-10-06T11:34:33.677458+0000","last_undegraded":"2023-10-06T11:34:33.677458+0000","last_fullsized":"2023-10-06T11:34:33.677458+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:56.442043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.663901+0000","last_change":"2023-10-06T11:34:33.660286+0000","last_active":"2023-10-06T11:34:33.663901+0000","last_peered":"2023-10-06T11:34:33.663901+0000","last_clean":"2023-10-06T11:34:33.663901+0000","last_became_active":"2023-10-06T11:34:33.659955+0000","last_became_peered":"2023-10-06T11:34:33.659955+0000","last_unstale":"2023-10-06T11:34:33.663901+0000","last_undegraded":"2023-10-06T11:34:33.663901+0000","last_fullsized":"2023-10-06T11:34:33.663901+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:44.228067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.656246+0000","last_change":"2023-10-06T11:34:32.642265+0000","last_active":"2023-10-06T11:34:33.656246+0000","last_peered":"2023-10-06T11:34:33.656246+0000","last_clean":"2023-10-06T11:34:33.656246+0000","last_became_active":"2023-10-06T11:34:32.641949+0000","last_became_peered":"2023-10-06T11:34:32.641949+0000","last_unstale":"2023-10-06T11:34:33.656246+0000","last_undegraded":"2023-10-06T11:34:33.656246+0000","last_fullsized":"2023-10-06T11:34:33.656246+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:26.283803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670001+0000","last_change":"2023-10-06T11:34:33.663534+0000","last_active":"2023-10-06T11:34:33.670001+0000","last_peered":"2023-10-06T11:34:33.670001+0000","last_clean":"2023-10-06T11:34:33.670001+0000","last_became_active":"2023-10-06T11:34:33.663226+0000","last_became_peered":"2023-10-06T11:34:33.663226+0000","last_unstale":"2023-10-06T11:34:33.670001+0000","last_undegraded":"2023-10-06T11:34:33.670001+0000","last_fullsized":"2023-10-06T11:34:33.670001+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:34.075211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.656391+0000","last_change":"2023-10-06T11:34:32.655780+0000","last_active":"2023-10-06T11:34:33.656391+0000","last_peered":"2023-10-06T11:34:33.656391+0000","last_clean":"2023-10-06T11:34:33.656391+0000","last_became_active":"2023-10-06T11:34:32.655466+0000","last_became_peered":"2023-10-06T11:34:32.655466+0000","last_unstale":"2023-10-06T11:34:33.656391+0000","last_undegraded":"2023-10-06T11:34:33.656391+0000","last_fullsized":"2023-10-06T11:34:33.656391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:06.714958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679835+0000","last_change":"2023-10-06T11:34:33.672940+0000","last_active":"2023-10-06T11:34:33.679835+0000","last_peered":"2023-10-06T11:34:33.679835+0000","last_clean":"2023-10-06T11:34:33.679835+0000","last_became_active":"2023-10-06T11:34:33.672642+0000","last_became_peered":"2023-10-06T11:34:33.672642+0000","last_unstale":"2023-10-06T11:34:33.679835+0000","last_undegraded":"2023-10-06T11:34:33.679835+0000","last_fullsized":"2023-10-06T11:34:33.679835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:10.885411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.651864+0000","last_change":"2023-10-06T11:34:32.649796+0000","last_active":"2023-10-06T11:34:32.651864+0000","last_peered":"2023-10-06T11:34:32.651864+0000","last_clean":"2023-10-06T11:34:32.651864+0000","last_became_active":"2023-10-06T11:34:32.649436+0000","last_became_peered":"2023-10-06T11:34:32.649436+0000","last_unstale":"2023-10-06T11:34:32.651864+0000","last_undegraded":"2023-10-06T11:34:32.651864+0000","last_fullsized":"2023-10-06T11:34:32.651864+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:52.916318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667135+0000","last_change":"2023-10-06T11:34:33.667135+0000","last_active":"2023-10-06T11:34:33.667135+0000","last_peered":"2023-10-06T11:34:33.667135+0000","last_clean":"2023-10-06T11:34:33.667135+0000","last_became_active":"2023-10-06T11:34:33.666576+0000","last_became_peered":"2023-10-06T11:34:33.666576+0000","last_unstale":"2023-10-06T11:34:33.667135+0000","last_undegraded":"2023-10-06T11:34:33.667135+0000","last_fullsized":"2023-10-06T11:34:33.667135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:47.521730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.651819+0000","last_change":"2023-10-06T11:34:32.643385+0000","last_active":"2023-10-06T11:34:33.651819+0000","last_peered":"2023-10-06T11:34:33.651819+0000","last_clean":"2023-10-06T11:34:33.651819+0000","last_became_active":"2023-10-06T11:34:32.642888+0000","last_became_peered":"2023-10-06T11:34:32.642888+0000","last_unstale":"2023-10-06T11:34:33.651819+0000","last_undegraded":"2023-10-06T11:34:33.651819+0000","last_fullsized":"2023-10-06T11:34:33.651819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:11.370315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678222+0000","last_change":"2023-10-06T11:34:33.678222+0000","last_active":"2023-10-06T11:34:33.678222+0000","last_peered":"2023-10-06T11:34:33.678222+0000","last_clean":"2023-10-06T11:34:33.678222+0000","last_became_active":"2023-10-06T11:34:33.677851+0000","last_became_peered":"2023-10-06T11:34:33.677851+0000","last_unstale":"2023-10-06T11:34:33.678222+0000","last_undegraded":"2023-10-06T11:34:33.678222+0000","last_fullsized":"2023-10-06T11:34:33.678222+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:58.197183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661069+0000","last_change":"2023-10-06T11:34:32.654697+0000","last_active":"2023-10-06T11:34:33.661069+0000","last_peered":"2023-10-06T11:34:33.661069+0000","last_clean":"2023-10-06T11:34:33.661069+0000","last_became_active":"2023-10-06T11:34:32.654203+0000","last_became_peered":"2023-10-06T11:34:32.654203+0000","last_unstale":"2023-10-06T11:34:33.661069+0000","last_undegraded":"2023-10-06T11:34:33.661069+0000","last_fullsized":"2023-10-06T11:34:33.661069+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:43.266091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.693540+0000","last_change":"2023-10-06T11:34:33.693540+0000","last_active":"2023-10-06T11:34:33.693540+0000","last_peered":"2023-10-06T11:34:33.693540+0000","last_clean":"2023-10-06T11:34:33.693540+0000","last_became_active":"2023-10-06T11:34:33.693158+0000","last_became_peered":"2023-10-06T11:34:33.693158+0000","last_unstale":"2023-10-06T11:34:33.693540+0000","last_undegraded":"2023-10-06T11:34:33.693540+0000","last_fullsized":"2023-10-06T11:34:33.693540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:38:04.711975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.651958+0000","last_change":"2023-10-06T11:34:32.650023+0000","last_active":"2023-10-06T11:34:32.651958+0000","last_peered":"2023-10-06T11:34:32.651958+0000","last_clean":"2023-10-06T11:34:32.651958+0000","last_became_active":"2023-10-06T11:34:32.649734+0000","last_became_peered":"2023-10-06T11:34:32.649734+0000","last_unstale":"2023-10-06T11:34:32.651958+0000","last_undegraded":"2023-10-06T11:34:32.651958+0000","last_fullsized":"2023-10-06T11:34:32.651958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:17.008799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661047+0000","last_change":"2023-10-06T11:34:33.658055+0000","last_active":"2023-10-06T11:34:33.661047+0000","last_peered":"2023-10-06T11:34:33.661047+0000","last_clean":"2023-10-06T11:34:33.661047+0000","last_became_active":"2023-10-06T11:34:33.657743+0000","last_became_peered":"2023-10-06T11:34:33.657743+0000","last_unstale":"2023-10-06T11:34:33.661047+0000","last_undegraded":"2023-10-06T11:34:33.661047+0000","last_fullsized":"2023-10-06T11:34:33.661047+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.268916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665476+0000","last_change":"2023-10-06T11:34:32.649029+0000","last_active":"2023-10-06T11:34:33.665476+0000","last_peered":"2023-10-06T11:34:33.665476+0000","last_clean":"2023-10-06T11:34:33.665476+0000","last_became_active":"2023-10-06T11:34:32.648670+0000","last_became_peered":"2023-10-06T11:34:32.648670+0000","last_unstale":"2023-10-06T11:34:33.665476+0000","last_undegraded":"2023-10-06T11:34:33.665476+0000","last_fullsized":"2023-10-06T11:34:33.665476+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:09.872244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.686921+0000","last_change":"2023-10-06T11:34:33.685128+0000","last_active":"2023-10-06T11:34:33.686921+0000","last_peered":"2023-10-06T11:34:33.686921+0000","last_clean":"2023-10-06T11:34:33.686921+0000","last_became_active":"2023-10-06T11:34:33.684834+0000","last_became_peered":"2023-10-06T11:34:33.684834+0000","last_unstale":"2023-10-06T11:34:33.686921+0000","last_undegraded":"2023-10-06T11:34:33.686921+0000","last_fullsized":"2023-10-06T11:34:33.686921+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:01.957381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.648663+0000","last_change":"2023-10-06T11:34:32.645362+0000","last_active":"2023-10-06T11:34:32.648663+0000","last_peered":"2023-10-06T11:34:32.648663+0000","last_clean":"2023-10-06T11:34:32.648663+0000","last_became_active":"2023-10-06T11:34:32.645029+0000","last_became_peered":"2023-10-06T11:34:32.645029+0000","last_unstale":"2023-10-06T11:34:32.648663+0000","last_undegraded":"2023-10-06T11:34:32.648663+0000","last_fullsized":"2023-10-06T11:34:32.648663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:29.397372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"95'205","reported_seq":551,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667599+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:34:33.667599+0000","last_peered":"2023-10-06T11:34:33.667599+0000","last_clean":"2023-10-06T11:34:33.667599+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:34:33.667599+0000","last_undegraded":"2023-10-06T11:34:33.667599+0000","last_fullsized":"2023-10-06T11:34:33.667599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.660947+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:34:33.660947+0000","last_peered":"2023-10-06T11:34:33.660947+0000","last_clean":"2023-10-06T11:34:33.660947+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:34:33.660947+0000","last_undegraded":"2023-10-06T11:34:33.660947+0000","last_fullsized":"2023-10-06T11:34:33.660947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659762+0000","last_change":"2023-10-06T11:34:33.656565+0000","last_active":"2023-10-06T11:34:33.659762+0000","last_peered":"2023-10-06T11:34:33.659762+0000","last_clean":"2023-10-06T11:34:33.659762+0000","last_became_active":"2023-10-06T11:34:33.656236+0000","last_became_peered":"2023-10-06T11:34:33.656236+0000","last_unstale":"2023-10-06T11:34:33.659762+0000","last_undegraded":"2023-10-06T11:34:33.659762+0000","last_fullsized":"2023-10-06T11:34:33.659762+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:42.750299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665522+0000","last_change":"2023-10-06T11:34:32.646863+0000","last_active":"2023-10-06T11:34:33.665522+0000","last_peered":"2023-10-06T11:34:33.665522+0000","last_clean":"2023-10-06T11:34:33.665522+0000","last_became_active":"2023-10-06T11:34:32.646465+0000","last_became_peered":"2023-10-06T11:34:32.646465+0000","last_unstale":"2023-10-06T11:34:33.665522+0000","last_undegraded":"2023-10-06T11:34:33.665522+0000","last_fullsized":"2023-10-06T11:34:33.665522+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665674+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:34:33.665674+0000","last_peered":"2023-10-06T11:34:33.665674+0000","last_clean":"2023-10-06T11:34:33.665674+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:34:33.665674+0000","last_undegraded":"2023-10-06T11:34:33.665674+0000","last_fullsized":"2023-10-06T11:34:33.665674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678339+0000","last_change":"2023-10-06T11:34:33.678339+0000","last_active":"2023-10-06T11:34:33.678339+0000","last_peered":"2023-10-06T11:34:33.678339+0000","last_clean":"2023-10-06T11:34:33.678339+0000","last_became_active":"2023-10-06T11:34:33.678043+0000","last_became_peered":"2023-10-06T11:34:33.678043+0000","last_unstale":"2023-10-06T11:34:33.678339+0000","last_undegraded":"2023-10-06T11:34:33.678339+0000","last_fullsized":"2023-10-06T11:34:33.678339+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:27.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.677092+0000","last_change":"2023-10-06T11:34:32.647129+0000","last_active":"2023-10-06T11:34:33.677092+0000","last_peered":"2023-10-06T11:34:33.677092+0000","last_clean":"2023-10-06T11:34:33.677092+0000","last_became_active":"2023-10-06T11:34:32.646792+0000","last_became_peered":"2023-10-06T11:34:32.646792+0000","last_unstale":"2023-10-06T11:34:33.677092+0000","last_undegraded":"2023-10-06T11:34:33.677092+0000","last_fullsized":"2023-10-06T11:34:33.677092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:15.563869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.629582+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:34:32.629582+0000","last_peered":"2023-10-06T11:34:32.629582+0000","last_clean":"2023-10-06T11:34:32.629582+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:34:32.629582+0000","last_undegraded":"2023-10-06T11:34:32.629582+0000","last_fullsized":"2023-10-06T11:34:32.629582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.671777+0000","last_change":"2023-10-06T11:34:33.668139+0000","last_active":"2023-10-06T11:34:33.671777+0000","last_peered":"2023-10-06T11:34:33.671777+0000","last_clean":"2023-10-06T11:34:33.671777+0000","last_became_active":"2023-10-06T11:34:33.667026+0000","last_became_peered":"2023-10-06T11:34:33.667026+0000","last_unstale":"2023-10-06T11:34:33.671777+0000","last_undegraded":"2023-10-06T11:34:33.671777+0000","last_fullsized":"2023-10-06T11:34:33.671777+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:03.145906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694854+0000","last_change":"2023-10-06T11:34:32.649272+0000","last_active":"2023-10-06T11:34:33.694854+0000","last_peered":"2023-10-06T11:34:33.694854+0000","last_clean":"2023-10-06T11:34:33.694854+0000","last_became_active":"2023-10-06T11:34:32.648974+0000","last_became_peered":"2023-10-06T11:34:32.648974+0000","last_unstale":"2023-10-06T11:34:33.694854+0000","last_undegraded":"2023-10-06T11:34:33.694854+0000","last_fullsized":"2023-10-06T11:34:33.694854+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:38.525361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661168+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:34:33.661168+0000","last_peered":"2023-10-06T11:34:33.661168+0000","last_clean":"2023-10-06T11:34:33.661168+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:34:33.661168+0000","last_undegraded":"2023-10-06T11:34:33.661168+0000","last_fullsized":"2023-10-06T11:34:33.661168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685088+0000","last_change":"2023-10-06T11:34:33.680636+0000","last_active":"2023-10-06T11:34:33.685088+0000","last_peered":"2023-10-06T11:34:33.685088+0000","last_clean":"2023-10-06T11:34:33.685088+0000","last_became_active":"2023-10-06T11:34:33.680098+0000","last_became_peered":"2023-10-06T11:34:33.680098+0000","last_unstale":"2023-10-06T11:34:33.685088+0000","last_undegraded":"2023-10-06T11:34:33.685088+0000","last_fullsized":"2023-10-06T11:34:33.685088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:07:28.152443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684902+0000","last_change":"2023-10-06T11:34:32.654547+0000","last_active":"2023-10-06T11:34:33.684902+0000","last_peered":"2023-10-06T11:34:33.684902+0000","last_clean":"2023-10-06T11:34:33.684902+0000","last_became_active":"2023-10-06T11:34:32.654132+0000","last_became_peered":"2023-10-06T11:34:32.654132+0000","last_unstale":"2023-10-06T11:34:33.684902+0000","last_undegraded":"2023-10-06T11:34:33.684902+0000","last_fullsized":"2023-10-06T11:34:33.684902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:42.898733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.637934+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:34:32.637934+0000","last_peered":"2023-10-06T11:34:32.637934+0000","last_clean":"2023-10-06T11:34:32.637934+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:34:32.637934+0000","last_undegraded":"2023-10-06T11:34:32.637934+0000","last_fullsized":"2023-10-06T11:34:32.637934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680598+0000","last_change":"2023-10-06T11:34:33.676211+0000","last_active":"2023-10-06T11:34:33.680598+0000","last_peered":"2023-10-06T11:34:33.680598+0000","last_clean":"2023-10-06T11:34:33.680598+0000","last_became_active":"2023-10-06T11:34:33.675930+0000","last_became_peered":"2023-10-06T11:34:33.675930+0000","last_unstale":"2023-10-06T11:34:33.680598+0000","last_undegraded":"2023-10-06T11:34:33.680598+0000","last_fullsized":"2023-10-06T11:34:33.680598+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:37.851550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.644287+0000","last_change":"2023-10-06T11:34:32.642162+0000","last_active":"2023-10-06T11:34:32.644287+0000","last_peered":"2023-10-06T11:34:32.644287+0000","last_clean":"2023-10-06T11:34:32.644287+0000","last_became_active":"2023-10-06T11:34:32.641874+0000","last_became_peered":"2023-10-06T11:34:32.641874+0000","last_unstale":"2023-10-06T11:34:32.644287+0000","last_undegraded":"2023-10-06T11:34:32.644287+0000","last_fullsized":"2023-10-06T11:34:32.644287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:29.397372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.666544+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:34:33.666544+0000","last_peered":"2023-10-06T11:34:33.666544+0000","last_clean":"2023-10-06T11:34:33.666544+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:34:33.666544+0000","last_undegraded":"2023-10-06T11:34:33.666544+0000","last_fullsized":"2023-10-06T11:34:33.666544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694973+0000","last_change":"2023-10-06T11:34:33.692682+0000","last_active":"2023-10-06T11:34:33.694973+0000","last_peered":"2023-10-06T11:34:33.694973+0000","last_clean":"2023-10-06T11:34:33.694973+0000","last_became_active":"2023-10-06T11:34:33.692195+0000","last_became_peered":"2023-10-06T11:34:33.692195+0000","last_unstale":"2023-10-06T11:34:33.694973+0000","last_undegraded":"2023-10-06T11:34:33.694973+0000","last_fullsized":"2023-10-06T11:34:33.694973+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:21.956959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659610+0000","last_change":"2023-10-06T11:34:32.651500+0000","last_active":"2023-10-06T11:34:33.659610+0000","last_peered":"2023-10-06T11:34:33.659610+0000","last_clean":"2023-10-06T11:34:33.659610+0000","last_became_active":"2023-10-06T11:34:32.651202+0000","last_became_peered":"2023-10-06T11:34:32.651202+0000","last_unstale":"2023-10-06T11:34:33.659610+0000","last_undegraded":"2023-10-06T11:34:33.659610+0000","last_fullsized":"2023-10-06T11:34:33.659610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:40.756548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.637407+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:34:32.637407+0000","last_peered":"2023-10-06T11:34:32.637407+0000","last_clean":"2023-10-06T11:34:32.637407+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:34:32.637407+0000","last_undegraded":"2023-10-06T11:34:32.637407+0000","last_fullsized":"2023-10-06T11:34:32.637407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665770+0000","last_change":"2023-10-06T11:34:33.662908+0000","last_active":"2023-10-06T11:34:33.665770+0000","last_peered":"2023-10-06T11:34:33.665770+0000","last_clean":"2023-10-06T11:34:33.665770+0000","last_became_active":"2023-10-06T11:34:33.662590+0000","last_became_peered":"2023-10-06T11:34:33.662590+0000","last_unstale":"2023-10-06T11:34:33.665770+0000","last_undegraded":"2023-10-06T11:34:33.665770+0000","last_fullsized":"2023-10-06T11:34:33.665770+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:08:45.367549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643003+0000","last_change":"2023-10-06T11:34:32.636772+0000","last_active":"2023-10-06T11:34:32.643003+0000","last_peered":"2023-10-06T11:34:32.643003+0000","last_clean":"2023-10-06T11:34:32.643003+0000","last_became_active":"2023-10-06T11:34:32.636407+0000","last_became_peered":"2023-10-06T11:34:32.636407+0000","last_unstale":"2023-10-06T11:34:32.643003+0000","last_undegraded":"2023-10-06T11:34:32.643003+0000","last_fullsized":"2023-10-06T11:34:32.643003+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:58.440067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643721+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:34:32.643721+0000","last_peered":"2023-10-06T11:34:32.643721+0000","last_clean":"2023-10-06T11:34:32.643721+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:34:32.643721+0000","last_undegraded":"2023-10-06T11:34:32.643721+0000","last_fullsized":"2023-10-06T11:34:32.643721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311436,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060404,"statfs":{"total":96636764160,"available":96317853696,"internally_reserved":0,"allocated":1904640,"data_stored":1259016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.827},"last":1.526},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.577,"5min":1.649,"15min":2.0030000000000001},"last":1.1599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3169999999999999,"15min":1.3300000000000001},"min":{"1min":0.90000000000000002,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7649999999999999,"5min":1.871,"15min":2.6059999999999999},"last":1.26},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.927,"5min":1.927,"15min":1.931},"last":1.212}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.502,"5min":1.827,"15min":2.3799999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.258,"15min":1.2609999999999999},"min":{"1min":1.054,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.7,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.82099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8,"5min":1.9450000000000001,"15min":2.665},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.296,"15min":1.321},"min":{"1min":0.92900000000000005,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.796,"15min":2.8570000000000002},"last":1.4259999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.77300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5569999999999999,"5min":2.149,"15min":2.7970000000000002},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.86599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.5339999999999998},"last":1.7609999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.335,"15min":1.3160000000000001},"min":{"1min":0.77900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7410000000000001,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.28},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.71,"5min":2.2480000000000002,"15min":3.105},"last":1.677}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.361},"min":{"1min":1.0369999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0190000000000001,"5min":2.4289999999999998,"15min":3.004},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.86299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.905,"5min":2.121,"15min":2.9119999999999999},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":670704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.288},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5629999999999999,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.333,"15min":1.325},"min":{"1min":0.83799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6830000000000001,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.02}]},{"osd":1,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.80500000000000005,"5min":0.76200000000000001,"15min":0.63900000000000001},"max":{"1min":1.5029999999999999,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.605,"5min":1.669,"15min":1.8480000000000001},"last":1.3580000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.2},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.74299999999999999},"max":{"1min":1.514,"5min":1.635,"15min":1.6919999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.194,"15min":1.222},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4450000000000001,"5min":1.587,"15min":2.7789999999999999},"last":1.2030000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.3100000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.534,"5min":1.764,"15min":1.9850000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3120000000000001,"15min":1.3049999999999999},"min":{"1min":1.077,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.5269999999999999,"5min":1.724,"15min":2.109},"last":1.585}]},{"osd":4,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.238,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.76500000000000001,"15min":0.61399999999999999},"max":{"1min":1.6639999999999999,"5min":1.8,"15min":1.956},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.292,"15min":1.2490000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.73899999999999999},"max":{"1min":1.645,"5min":1.827,"15min":1.8879999999999999},"last":1.494}]},{"osd":5,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6000000000000001,"5min":1.72,"15min":2.198},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.395,"15min":1.385},"min":{"1min":0.86499999999999999,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.718,"5min":1.879,"15min":2.1509999999999998},"last":1.524}]},{"osd":7,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4510000000000001,"15min":1.448},"min":{"1min":0.90100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.871,"5min":1.927,"15min":2.2919999999999998},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4139999999999999,"15min":1.4299999999999999},"min":{"1min":1.004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6879999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.5589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575042,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.3899999999999999,"15min":1.268},"min":{"1min":0.67400000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.772,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64900000000000002},"max":{"1min":1.679,"5min":1.972,"15min":2.544},"last":1.196}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.194,"15min":1.175},"min":{"1min":0.79800000000000004,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.478,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.262},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.2190000000000001},"min":{"1min":0.79900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5189999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.4590000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.159,"15min":1.1950000000000001},"min":{"1min":0.78600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.601,"5min":1.6879999999999999,"15min":3.609},"last":1.091},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.73299999999999998},"max":{"1min":1.512,"5min":1.738,"15min":2.6949999999999998},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.393,"15min":1.296},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.639,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.302,"15min":1.3340000000000001},"min":{"1min":0.91800000000000004,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.5229999999999999,"5min":1.734,"15min":3.7770000000000001},"last":1.361}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.40699999999999997},"max":{"1min":1.6060000000000001,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.516,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.724,"5min":18.48,"15min":18.48},"last":1.222}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.3089999999999999},"min":{"1min":1.0129999999999999,"5min":0.85199999999999998,"15min":0.622},"max":{"1min":1.6799999999999999,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.411,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.6870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.1479999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5249999999999999,"15min":1.4039999999999999},"min":{"1min":0.85499999999999998,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7909999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.494,"15min":1.444},"min":{"1min":0.998,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7709999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":668752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1479999999999999,"15min":1.143},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.554,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.111,"15min":1.143},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.478,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2989999999999999,"15min":1.323},"min":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.546,"5min":1.645,"15min":2.1989999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.68300000000000005},"max":{"1min":1.508,"5min":1.7450000000000001,"15min":1.851},"last":1.552}]},{"osd":2,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.73499999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":2.2269999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.78500000000000003},"max":{"1min":1.583,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.671,"15min":1.6839999999999999},"last":1.212},{"interface":"front","average":{"1min":1.288,"5min":1.3009999999999999,"15min":1.304},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.77400000000000002},"max":{"1min":1.575,"5min":1.6639999999999999,"15min":2.258},"last":1.2709999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.202,"15min":1.21},"min":{"1min":0.79600000000000004,"5min":0.621,"15min":0.621},"max":{"1min":1.653,"5min":1.653,"15min":2.8250000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2130000000000001,"15min":1.1970000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.40999999999999998},"max":{"1min":1.421,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3919999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.6950000000000001,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":1.0860000000000001,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.845,"5min":1.845,"15min":2.4329999999999998},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.69399999999999995},"max":{"1min":1.5880000000000001,"5min":1.841,"15min":2.653},"last":1.645},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.3919999999999999},"min":{"1min":0.96699999999999997,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.464},"last":1.536}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310848,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060992,"statfs":{"total":96636764160,"available":96318455808,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.61099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0960000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":14.49},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.437},"max":{"1min":1.427,"5min":1.504,"15min":1.627},"last":19.800999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.95099999999999996},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.343,"5min":1.4079999999999999,"15min":1.516},"last":6.6790000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0660000000000001,"15min":1.077},"min":{"1min":0.753,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":11.513999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.093,"15min":1.121},"min":{"1min":0.95399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":12.446},{"interface":"front","average":{"1min":1.228,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.68700000000000006,"5min":0.44,"15min":0.44},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.7030000000000001},"last":7.4550000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.1919999999999999,"15min":1.236},"min":{"1min":0.93300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6890000000000001,"5min":1.768,"15min":2.6640000000000001},"last":10.460000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3029999999999999,"15min":1.3560000000000001},"min":{"1min":1.1379999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0840000000000001},"last":6.7249999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3420000000000001,"15min":1.4019999999999999},"min":{"1min":0.92100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6080000000000001,"5min":1.883,"15min":2.1699999999999999},"last":6.8520000000000003},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.357,"15min":1.3979999999999999},"min":{"1min":1.0620000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.1179999999999999},"last":7.1180000000000003}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.411,"15min":1.45},"min":{"1min":0.98599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8520000000000001,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":11.362},{"interface":"front","average":{"1min":1.387,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.97499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.75,"5min":1.823,"15min":2.137},"last":10.515000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4119999999999999,"15min":1.4419999999999999},"min":{"1min":1.163,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8540000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":10.561},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4319999999999999,"15min":1.482},"min":{"1min":1.0860000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.645,"5min":2.004,"15min":2.266},"last":6.7629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310848,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060992,"statfs":{"total":96636764160,"available":96318455808,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1200000000000001,"5min":1.4670000000000001,"15min":1.8380000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.383,"5min":1.415,"15min":1.9650000000000001},"last":1.127}]},{"osd":1,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.3049999999999999,"5min":1.631,"15min":2.3940000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0229999999999999},"min":{"1min":0.68700000000000006,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.599,"5min":1.6779999999999999,"15min":2.5390000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"min":{"1min":0.90600000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.758,"5min":1.8,"15min":1.917},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.28,"15min":1.306},"min":{"1min":0.96499999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.948,"5min":1.948,"15min":2.4550000000000001},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.89700000000000002,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":2.1899999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":1.0589999999999999,"5min":0.88500000000000001,"15min":0.84399999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.6920000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":1.171,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8480000000000001,"5min":2.0680000000000001,"15min":2.601},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.319,"15min":1.337},"min":{"1min":1.0089999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.6400000000000001},"last":1.288}]},{"osd":6,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4039999999999999,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.972,"5min":1.972,"15min":2.8340000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.427,"15min":1.444},"min":{"1min":1.1719999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8320000000000001,"5min":1.9910000000000001,"15min":2.617},"last":1.147}]},{"osd":7,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.371,"15min":1.391},"min":{"1min":0.94299999999999995,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.7549999999999999,"5min":1.8049999999999999,"15min":2.9020000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.464,"15min":1.4690000000000001},"min":{"1min":1.2609999999999999,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.978,"5min":1.978,"15min":3.2240000000000002},"last":1.1719999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306756,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065084,"statfs":{"total":96636764160,"available":96322646016,"internally_reserved":0,"allocated":1306624,"data_stored":668662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.93000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.252,"5min":1.724,"15min":1.724},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.58399999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.4259999999999999,"15min":1.855},"last":1.288}]},{"osd":2,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.97199999999999998},"min":{"1min":0.67100000000000004,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.1859999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":1.208},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.319,"5min":2.137,"15min":2.137},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.272,"15min":1.228},"min":{"1min":0.879,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6120000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.04},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.073,"15min":1.0980000000000001},"min":{"1min":0.72199999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":4.0830000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.238,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7350000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.341},{"interface":"front","average":{"1min":1.419,"5min":1.387,"15min":1.397},"min":{"1min":1.056,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.788,"5min":1.788,"15min":3.5609999999999999},"last":1.3839999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.5720000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.45},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.365,"15min":1.3500000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.85999999999999999},"max":{"1min":1.849,"5min":2.1240000000000001,"15min":2.766},"last":1.923}]},{"osd":6,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4890000000000001,"15min":1.4830000000000001},"min":{"1min":1.1299999999999999,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8759999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.48},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.411,"15min":1.407},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.88,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8580000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.573},{"interface":"front","average":{"1min":1.575,"5min":1.4650000000000001,"15min":1.466},"min":{"1min":1.133,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.085,"5min":2.085,"15min":2.8740000000000001},"last":1.7989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307356,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064484,"statfs":{"total":96636764160,"available":96322031616,"internally_reserved":0,"allocated":1921024,"data_stored":1261174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85899999999999999,"15min":0.88300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2290000000000001,"5min":1.3560000000000001,"15min":1.571},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95499999999999996,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.395}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2549999999999999,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.738,"5min":1.738,"15min":2.7719999999999998},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0860000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.919},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.762,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":0.877}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3280000000000001,"15min":1.343},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.294,"15min":2.294},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.276,"15min":1.2929999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.609,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.143}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.393,"15min":1.383},"min":{"1min":1.0800000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8520000000000001,"5min":2.0859999999999999,"15min":2.573},"last":1.246},{"interface":"front","average":{"1min":1.373,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":1.024,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.843,"5min":1.881,"15min":2.8650000000000002},"last":1.218}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.3420000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6160000000000001,"5min":2.2240000000000002,"15min":2.52},"last":1.264},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4410000000000001,"15min":1.4610000000000001},"min":{"1min":1.163,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.909,"5min":2.254,"15min":2.742},"last":1.284}]},{"osd":7,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4399999999999999,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7509999999999999,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4119999999999999,"15min":1.4319999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7090000000000001,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.3480000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:34:37.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:34:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.147+0000 7f9b1226e700 1 Processor -- start 2023-10-06T11:34:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.147+0000 7f9b1226e700 1 -- start start 2023-10-06T11:34:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b1226e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b1226e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b1226e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c06d460 con 0x7f9b0c0ad6c0 2023-10-06T11:34:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b1226e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c7000 con 0x7f9b0c15ab60 2023-10-06T11:34:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b1226e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c72c0 con 0x7f9b0c15af30 2023-10-06T11:34:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b0bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38738/0 (socket says 172.21.15.46:38738) 2023-10-06T11:34:38.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b0bfff700 1 -- 172.21.15.46:0/1639456039 learned_addr learned my addr 172.21.15.46:0/1639456039 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.151+0000 7f9b037fe700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0bfff700 1 -- 172.21.15.46:0/1639456039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 msgr2=0x7f9b0c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0bfff700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0bfff700 1 -- 172.21.15.46:0/1639456039 >> v1:172.21.15.132:6789/0 conn(0x7f9b0c0ad6c0 legacy=0x7f9b0c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0bfff700 1 -- 172.21.15.46:0/1639456039 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0c75a0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0bfff700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0099a0 tx=0x7f9afc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a16af96666a8a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0b7fe700 1 -- 172.21.15.46:0/1639456039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc009e90 con 0x7f9b0c15ab60 2023-10-06T11:34:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0b7fe700 1 -- 172.21.15.46:0/1639456039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc00b860 con 0x7f9b0c15ab60 2023-10-06T11:34:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b0b7fe700 1 -- 172.21.15.46:0/1639456039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00bc20 con 0x7f9b0c15ab60 2023-10-06T11:34:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 -- 172.21.15.46:0/1639456039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 msgr2=0x7f9b0c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0099a0 tx=0x7f9afc009250 comp rx=0 tx=0).stop 2023-10-06T11:34:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 -- 172.21.15.46:0/1639456039 shutdown_connections 2023-10-06T11:34:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 --2- 172.21.15.46:0/1639456039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c06e8a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 -- 172.21.15.46:0/1639456039 >> 172.21.15.46:0/1639456039 conn(0x7f9b0c0cbe30 msgr2=0x7f9b0c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 -- 172.21.15.46:0/1639456039 shutdown_connections 2023-10-06T11:34:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.155+0000 7f9b1226e700 1 -- 172.21.15.46:0/1639456039 wait complete. 2023-10-06T11:34:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 Processor -- start 2023-10-06T11:34:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 -- start start 2023-10-06T11:34:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b0c0ad6c0 0x7f9b0c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0afc70 con 0x7f9b0c15ab60 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c16e130 con 0x7f9b0c0ad6c0 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1226e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0c3610 con 0x7f9b0c15af30 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38746/0 (socket says 172.21.15.46:38746) 2023-10-06T11:34:38.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 -- 172.21.15.46:0/2280578921 learned_addr learned my addr 172.21.15.46:0/2280578921 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b0bfff700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b0c0ad6c0 0x7f9b0c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1080b700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 -- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 msgr2=0x7f9b0c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 -- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b0c0ad6c0 msgr2=0x7f9b0c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b0c0ad6c0 0x7f9b0c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b037fe700 1 -- 172.21.15.46:0/2280578921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009490 con 0x7f9b0c15ab60 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.159+0000 7f9b1080b700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b037fe700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9af400ec80 tx=0x7f9af40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af400c9d0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af400cb30 con 0x7f9b0c15ab60 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4013470 con 0x7f9b0c15ab60 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c0c38f0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c0c3eb0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c0afde0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.163+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9af4013820 con 0x7f9b0c15ab60 2023-10-06T11:34:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.167+0000 7f9b08ff9700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 0x7f9aec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.167+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f9af407cfe0 con 0x7f9b0c15ab60 2023-10-06T11:34:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.167+0000 7f9b0bfff700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 0x7f9aec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.167+0000 7f9b0bfff700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 0x7f9aec062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0099a0 tx=0x7f9afc017000 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.171+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9af404d160 con 0x7f9b0c15ab60 2023-10-06T11:34:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.343+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b0c063f90 con 0x7f9aec060670 2023-10-06T11:34:38.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.359+0000 7f9b08ff9700 1 -- 172.21.15.46:0/2280578921 <== mgr.6955 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308865 (secure 0 0 0) 0x7f9b0c063f90 con 0x7f9aec060670 2023-10-06T11:34:38.373 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 msgr2=0x7f9aec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 0x7f9aec062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0099a0 tx=0x7f9afc017000 comp rx=0 tx=0).stop 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 msgr2=0x7f9b0c0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9af400ec80 tx=0x7f9af40091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 shutdown_connections 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9aec060670 0x7f9aec062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b0c15af30 0x7f9b0c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b0c15ab60 0x7f9b0c0ca2b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 --2- 172.21.15.46:0/2280578921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b0c0ad6c0 0x7f9b0c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 >> 172.21.15.46:0/2280578921 conn(0x7f9b0c0cbe30 msgr2=0x7f9b0c0acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 shutdown_connections 2023-10-06T11:34:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.371+0000 7f9b1226e700 1 -- 172.21.15.46:0/2280578921 wait complete. 2023-10-06T11:34:38.382 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:34:38.400 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T11:34:37.239301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459336,"kb_used_data":11400,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752515384,"statfs":{"total":773094113280,"available":770575753216,"internally_reserved":0,"allocated":11673600,"data_stored":6534084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628741"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.658057+0000","last_change":"2023-10-06T11:34:33.658057+0000","last_active":"2023-10-06T11:34:33.658057+0000","last_peered":"2023-10-06T11:34:33.658057+0000","last_clean":"2023-10-06T11:34:33.658057+0000","last_became_active":"2023-10-06T11:34:33.657747+0000","last_became_peered":"2023-10-06T11:34:33.657747+0000","last_unstale":"2023-10-06T11:34:33.658057+0000","last_undegraded":"2023-10-06T11:34:33.658057+0000","last_fullsized":"2023-10-06T11:34:33.658057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:58.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679488+0000","last_change":"2023-10-06T11:34:33.673036+0000","last_active":"2023-10-06T11:34:33.679488+0000","last_peered":"2023-10-06T11:34:33.679488+0000","last_clean":"2023-10-06T11:34:33.679488+0000","last_became_active":"2023-10-06T11:34:33.672717+0000","last_became_peered":"2023-10-06T11:34:33.672717+0000","last_unstale":"2023-10-06T11:34:33.679488+0000","last_undegraded":"2023-10-06T11:34:33.679488+0000","last_fullsized":"2023-10-06T11:34:33.679488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:03.145906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.654916+0000","last_change":"2023-10-06T11:34:33.654916+0000","last_active":"2023-10-06T11:34:33.654916+0000","last_peered":"2023-10-06T11:34:33.654916+0000","last_clean":"2023-10-06T11:34:33.654916+0000","last_became_active":"2023-10-06T11:34:33.654378+0000","last_became_peered":"2023-10-06T11:34:33.654378+0000","last_unstale":"2023-10-06T11:34:33.654916+0000","last_undegraded":"2023-10-06T11:34:33.654916+0000","last_fullsized":"2023-10-06T11:34:33.654916+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:44.279258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676941+0000","last_change":"2023-10-06T11:34:33.676941+0000","last_active":"2023-10-06T11:34:33.676941+0000","last_peered":"2023-10-06T11:34:33.676941+0000","last_clean":"2023-10-06T11:34:33.676941+0000","last_became_active":"2023-10-06T11:34:33.676605+0000","last_became_peered":"2023-10-06T11:34:33.676605+0000","last_unstale":"2023-10-06T11:34:33.676941+0000","last_undegraded":"2023-10-06T11:34:33.676941+0000","last_fullsized":"2023-10-06T11:34:33.676941+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:33.820941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678848+0000","last_change":"2023-10-06T11:34:33.671777+0000","last_active":"2023-10-06T11:34:33.678848+0000","last_peered":"2023-10-06T11:34:33.678848+0000","last_clean":"2023-10-06T11:34:33.678848+0000","last_became_active":"2023-10-06T11:34:33.671363+0000","last_became_peered":"2023-10-06T11:34:33.671363+0000","last_unstale":"2023-10-06T11:34:33.678848+0000","last_undegraded":"2023-10-06T11:34:33.678848+0000","last_fullsized":"2023-10-06T11:34:33.678848+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:41.284844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684918+0000","last_change":"2023-10-06T11:34:33.683251+0000","last_active":"2023-10-06T11:34:33.684918+0000","last_peered":"2023-10-06T11:34:33.684918+0000","last_clean":"2023-10-06T11:34:33.684918+0000","last_became_active":"2023-10-06T11:34:33.682958+0000","last_became_peered":"2023-10-06T11:34:33.682958+0000","last_unstale":"2023-10-06T11:34:33.684918+0000","last_undegraded":"2023-10-06T11:34:33.684918+0000","last_fullsized":"2023-10-06T11:34:33.684918+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:45.381724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.688029+0000","last_change":"2023-10-06T11:34:33.687131+0000","last_active":"2023-10-06T11:34:33.688029+0000","last_peered":"2023-10-06T11:34:33.688029+0000","last_clean":"2023-10-06T11:34:33.688029+0000","last_became_active":"2023-10-06T11:34:33.686775+0000","last_became_peered":"2023-10-06T11:34:33.686775+0000","last_unstale":"2023-10-06T11:34:33.688029+0000","last_undegraded":"2023-10-06T11:34:33.688029+0000","last_fullsized":"2023-10-06T11:34:33.688029+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:42.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676379+0000","last_change":"2023-10-06T11:34:33.676379+0000","last_active":"2023-10-06T11:34:33.676379+0000","last_peered":"2023-10-06T11:34:33.676379+0000","last_clean":"2023-10-06T11:34:33.676379+0000","last_became_active":"2023-10-06T11:34:33.675284+0000","last_became_peered":"2023-10-06T11:34:33.675284+0000","last_unstale":"2023-10-06T11:34:33.676379+0000","last_undegraded":"2023-10-06T11:34:33.676379+0000","last_fullsized":"2023-10-06T11:34:33.676379+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:16.715165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680598+0000","last_change":"2023-10-06T11:34:33.677278+0000","last_active":"2023-10-06T11:34:33.680598+0000","last_peered":"2023-10-06T11:34:33.680598+0000","last_clean":"2023-10-06T11:34:33.680598+0000","last_became_active":"2023-10-06T11:34:33.676979+0000","last_became_peered":"2023-10-06T11:34:33.676979+0000","last_unstale":"2023-10-06T11:34:33.680598+0000","last_undegraded":"2023-10-06T11:34:33.680598+0000","last_fullsized":"2023-10-06T11:34:33.680598+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:31.489008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680146+0000","last_change":"2023-10-06T11:34:33.680146+0000","last_active":"2023-10-06T11:34:33.680146+0000","last_peered":"2023-10-06T11:34:33.680146+0000","last_clean":"2023-10-06T11:34:33.680146+0000","last_became_active":"2023-10-06T11:34:33.679772+0000","last_became_peered":"2023-10-06T11:34:33.679772+0000","last_unstale":"2023-10-06T11:34:33.680146+0000","last_undegraded":"2023-10-06T11:34:33.680146+0000","last_fullsized":"2023-10-06T11:34:33.680146+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:25:09.576482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.669872+0000","last_change":"2023-10-06T11:34:33.664254+0000","last_active":"2023-10-06T11:34:33.669872+0000","last_peered":"2023-10-06T11:34:33.669872+0000","last_clean":"2023-10-06T11:34:33.669872+0000","last_became_active":"2023-10-06T11:34:33.663847+0000","last_became_peered":"2023-10-06T11:34:33.663847+0000","last_unstale":"2023-10-06T11:34:33.669872+0000","last_undegraded":"2023-10-06T11:34:33.669872+0000","last_fullsized":"2023-10-06T11:34:33.669872+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:44.487912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.660252+0000","last_change":"2023-10-06T11:34:33.660252+0000","last_active":"2023-10-06T11:34:33.660252+0000","last_peered":"2023-10-06T11:34:33.660252+0000","last_clean":"2023-10-06T11:34:33.660252+0000","last_became_active":"2023-10-06T11:34:33.659887+0000","last_became_peered":"2023-10-06T11:34:33.659887+0000","last_unstale":"2023-10-06T11:34:33.660252+0000","last_undegraded":"2023-10-06T11:34:33.660252+0000","last_fullsized":"2023-10-06T11:34:33.660252+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:56:31.867762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.675401+0000","last_change":"2023-10-06T11:34:33.675401+0000","last_active":"2023-10-06T11:34:33.675401+0000","last_peered":"2023-10-06T11:34:33.675401+0000","last_clean":"2023-10-06T11:34:33.675401+0000","last_became_active":"2023-10-06T11:34:33.675096+0000","last_became_peered":"2023-10-06T11:34:33.675096+0000","last_unstale":"2023-10-06T11:34:33.675401+0000","last_undegraded":"2023-10-06T11:34:33.675401+0000","last_fullsized":"2023-10-06T11:34:33.675401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:24.870251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694908+0000","last_change":"2023-10-06T11:34:33.693665+0000","last_active":"2023-10-06T11:34:33.694908+0000","last_peered":"2023-10-06T11:34:33.694908+0000","last_clean":"2023-10-06T11:34:33.694908+0000","last_became_active":"2023-10-06T11:34:33.693388+0000","last_became_peered":"2023-10-06T11:34:33.693388+0000","last_unstale":"2023-10-06T11:34:33.694908+0000","last_undegraded":"2023-10-06T11:34:33.694908+0000","last_fullsized":"2023-10-06T11:34:33.694908+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:02.880995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685231+0000","last_change":"2023-10-06T11:34:33.685231+0000","last_active":"2023-10-06T11:34:33.685231+0000","last_peered":"2023-10-06T11:34:33.685231+0000","last_clean":"2023-10-06T11:34:33.685231+0000","last_became_active":"2023-10-06T11:34:33.684907+0000","last_became_peered":"2023-10-06T11:34:33.684907+0000","last_unstale":"2023-10-06T11:34:33.685231+0000","last_undegraded":"2023-10-06T11:34:33.685231+0000","last_fullsized":"2023-10-06T11:34:33.685231+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:53.929485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665740+0000","last_change":"2023-10-06T11:34:33.662241+0000","last_active":"2023-10-06T11:34:33.665740+0000","last_peered":"2023-10-06T11:34:33.665740+0000","last_clean":"2023-10-06T11:34:33.665740+0000","last_became_active":"2023-10-06T11:34:33.661744+0000","last_became_peered":"2023-10-06T11:34:33.661744+0000","last_unstale":"2023-10-06T11:34:33.665740+0000","last_undegraded":"2023-10-06T11:34:33.665740+0000","last_fullsized":"2023-10-06T11:34:33.665740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:05:57.259900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665453+0000","last_change":"2023-10-06T11:34:33.658292+0000","last_active":"2023-10-06T11:34:33.665453+0000","last_peered":"2023-10-06T11:34:33.665453+0000","last_clean":"2023-10-06T11:34:33.665453+0000","last_became_active":"2023-10-06T11:34:33.657990+0000","last_became_peered":"2023-10-06T11:34:33.657990+0000","last_unstale":"2023-10-06T11:34:33.665453+0000","last_undegraded":"2023-10-06T11:34:33.665453+0000","last_fullsized":"2023-10-06T11:34:33.665453+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:46.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670401+0000","last_change":"2023-10-06T11:34:33.670401+0000","last_active":"2023-10-06T11:34:33.670401+0000","last_peered":"2023-10-06T11:34:33.670401+0000","last_clean":"2023-10-06T11:34:33.670401+0000","last_became_active":"2023-10-06T11:34:33.670112+0000","last_became_peered":"2023-10-06T11:34:33.670112+0000","last_unstale":"2023-10-06T11:34:33.670401+0000","last_undegraded":"2023-10-06T11:34:33.670401+0000","last_fullsized":"2023-10-06T11:34:33.670401+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:45:00.555298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.687178+0000","last_change":"2023-10-06T11:34:33.684575+0000","last_active":"2023-10-06T11:34:33.687178+0000","last_peered":"2023-10-06T11:34:33.687178+0000","last_clean":"2023-10-06T11:34:33.687178+0000","last_became_active":"2023-10-06T11:34:33.684281+0000","last_became_peered":"2023-10-06T11:34:33.684281+0000","last_unstale":"2023-10-06T11:34:33.687178+0000","last_undegraded":"2023-10-06T11:34:33.687178+0000","last_fullsized":"2023-10-06T11:34:33.687178+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:26.697053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676893+0000","last_change":"2023-10-06T11:34:33.676893+0000","last_active":"2023-10-06T11:34:33.676893+0000","last_peered":"2023-10-06T11:34:33.676893+0000","last_clean":"2023-10-06T11:34:33.676893+0000","last_became_active":"2023-10-06T11:34:33.676216+0000","last_became_peered":"2023-10-06T11:34:33.676216+0000","last_unstale":"2023-10-06T11:34:33.676893+0000","last_undegraded":"2023-10-06T11:34:33.676893+0000","last_fullsized":"2023-10-06T11:34:33.676893+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:49.270036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678869+0000","last_change":"2023-10-06T11:34:33.673680+0000","last_active":"2023-10-06T11:34:33.678869+0000","last_peered":"2023-10-06T11:34:33.678869+0000","last_clean":"2023-10-06T11:34:33.678869+0000","last_became_active":"2023-10-06T11:34:33.673361+0000","last_became_peered":"2023-10-06T11:34:33.673361+0000","last_unstale":"2023-10-06T11:34:33.678869+0000","last_undegraded":"2023-10-06T11:34:33.678869+0000","last_fullsized":"2023-10-06T11:34:33.678869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:37.851550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694891+0000","last_change":"2023-10-06T11:34:33.693460+0000","last_active":"2023-10-06T11:34:33.694891+0000","last_peered":"2023-10-06T11:34:33.694891+0000","last_clean":"2023-10-06T11:34:33.694891+0000","last_became_active":"2023-10-06T11:34:33.693141+0000","last_became_peered":"2023-10-06T11:34:33.693141+0000","last_unstale":"2023-10-06T11:34:33.694891+0000","last_undegraded":"2023-10-06T11:34:33.694891+0000","last_fullsized":"2023-10-06T11:34:33.694891+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:43.276708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.693615+0000","last_change":"2023-10-06T11:34:33.693615+0000","last_active":"2023-10-06T11:34:33.693615+0000","last_peered":"2023-10-06T11:34:33.693615+0000","last_clean":"2023-10-06T11:34:33.693615+0000","last_became_active":"2023-10-06T11:34:33.693240+0000","last_became_peered":"2023-10-06T11:34:33.693240+0000","last_unstale":"2023-10-06T11:34:33.693615+0000","last_undegraded":"2023-10-06T11:34:33.693615+0000","last_fullsized":"2023-10-06T11:34:33.693615+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:41.769715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.654820+0000","last_change":"2023-10-06T11:34:33.654820+0000","last_active":"2023-10-06T11:34:33.654820+0000","last_peered":"2023-10-06T11:34:33.654820+0000","last_clean":"2023-10-06T11:34:33.654820+0000","last_became_active":"2023-10-06T11:34:33.654478+0000","last_became_peered":"2023-10-06T11:34:33.654478+0000","last_unstale":"2023-10-06T11:34:33.654820+0000","last_undegraded":"2023-10-06T11:34:33.654820+0000","last_fullsized":"2023-10-06T11:34:33.654820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:43.911900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679868+0000","last_change":"2023-10-06T11:34:33.674067+0000","last_active":"2023-10-06T11:34:33.679868+0000","last_peered":"2023-10-06T11:34:33.679868+0000","last_clean":"2023-10-06T11:34:33.679868+0000","last_became_active":"2023-10-06T11:34:33.673182+0000","last_became_peered":"2023-10-06T11:34:33.673182+0000","last_unstale":"2023-10-06T11:34:33.679868+0000","last_undegraded":"2023-10-06T11:34:33.679868+0000","last_fullsized":"2023-10-06T11:34:33.679868+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:31.489008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.687463+0000","last_change":"2023-10-06T11:34:33.685847+0000","last_active":"2023-10-06T11:34:33.687463+0000","last_peered":"2023-10-06T11:34:33.687463+0000","last_clean":"2023-10-06T11:34:33.687463+0000","last_became_active":"2023-10-06T11:34:33.685560+0000","last_became_peered":"2023-10-06T11:34:33.685560+0000","last_unstale":"2023-10-06T11:34:33.687463+0000","last_undegraded":"2023-10-06T11:34:33.687463+0000","last_fullsized":"2023-10-06T11:34:33.687463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:42.764678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.664841+0000","last_change":"2023-10-06T11:34:33.664841+0000","last_active":"2023-10-06T11:34:33.664841+0000","last_peered":"2023-10-06T11:34:33.664841+0000","last_clean":"2023-10-06T11:34:33.664841+0000","last_became_active":"2023-10-06T11:34:33.664526+0000","last_became_peered":"2023-10-06T11:34:33.664526+0000","last_unstale":"2023-10-06T11:34:33.664841+0000","last_undegraded":"2023-10-06T11:34:33.664841+0000","last_fullsized":"2023-10-06T11:34:33.664841+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:30:37.826261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676661+0000","last_change":"2023-10-06T11:34:33.676661+0000","last_active":"2023-10-06T11:34:33.676661+0000","last_peered":"2023-10-06T11:34:33.676661+0000","last_clean":"2023-10-06T11:34:33.676661+0000","last_became_active":"2023-10-06T11:34:33.675273+0000","last_became_peered":"2023-10-06T11:34:33.675273+0000","last_unstale":"2023-10-06T11:34:33.676661+0000","last_undegraded":"2023-10-06T11:34:33.676661+0000","last_fullsized":"2023-10-06T11:34:33.676661+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:15.684545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680444+0000","last_change":"2023-10-06T11:34:33.678142+0000","last_active":"2023-10-06T11:34:33.680444+0000","last_peered":"2023-10-06T11:34:33.680444+0000","last_clean":"2023-10-06T11:34:33.680444+0000","last_became_active":"2023-10-06T11:34:33.677836+0000","last_became_peered":"2023-10-06T11:34:33.677836+0000","last_unstale":"2023-10-06T11:34:33.680444+0000","last_undegraded":"2023-10-06T11:34:33.680444+0000","last_fullsized":"2023-10-06T11:34:33.680444+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:33:02.880995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680638+0000","last_change":"2023-10-06T11:34:33.677324+0000","last_active":"2023-10-06T11:34:33.680638+0000","last_peered":"2023-10-06T11:34:33.680638+0000","last_clean":"2023-10-06T11:34:33.680638+0000","last_became_active":"2023-10-06T11:34:33.677034+0000","last_became_peered":"2023-10-06T11:34:33.677034+0000","last_unstale":"2023-10-06T11:34:33.680638+0000","last_undegraded":"2023-10-06T11:34:33.680638+0000","last_fullsized":"2023-10-06T11:34:33.680638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:10.885411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694965+0000","last_change":"2023-10-06T11:34:33.692650+0000","last_active":"2023-10-06T11:34:33.694965+0000","last_peered":"2023-10-06T11:34:33.694965+0000","last_clean":"2023-10-06T11:34:33.694965+0000","last_became_active":"2023-10-06T11:34:33.692216+0000","last_became_peered":"2023-10-06T11:34:33.692216+0000","last_unstale":"2023-10-06T11:34:33.694965+0000","last_undegraded":"2023-10-06T11:34:33.694965+0000","last_fullsized":"2023-10-06T11:34:33.694965+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:41.284844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665560+0000","last_change":"2023-10-06T11:34:33.662040+0000","last_active":"2023-10-06T11:34:33.665560+0000","last_peered":"2023-10-06T11:34:33.665560+0000","last_clean":"2023-10-06T11:34:33.665560+0000","last_became_active":"2023-10-06T11:34:33.661363+0000","last_became_peered":"2023-10-06T11:34:33.661363+0000","last_unstale":"2023-10-06T11:34:33.665560+0000","last_undegraded":"2023-10-06T11:34:33.665560+0000","last_fullsized":"2023-10-06T11:34:33.665560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:46.992557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661008+0000","last_change":"2023-10-06T11:34:33.657115+0000","last_active":"2023-10-06T11:34:33.661008+0000","last_peered":"2023-10-06T11:34:33.661008+0000","last_clean":"2023-10-06T11:34:33.661008+0000","last_became_active":"2023-10-06T11:34:33.656821+0000","last_became_peered":"2023-10-06T11:34:33.656821+0000","last_unstale":"2023-10-06T11:34:33.661008+0000","last_undegraded":"2023-10-06T11:34:33.661008+0000","last_fullsized":"2023-10-06T11:34:33.661008+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:34.714772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'4","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680610+0000","last_change":"2023-10-06T11:34:32.657300+0000","last_active":"2023-10-06T11:34:33.680610+0000","last_peered":"2023-10-06T11:34:33.680610+0000","last_clean":"2023-10-06T11:34:33.680610+0000","last_became_active":"2023-10-06T11:34:32.656968+0000","last_became_peered":"2023-10-06T11:34:32.656968+0000","last_unstale":"2023-10-06T11:34:33.680610+0000","last_undegraded":"2023-10-06T11:34:33.680610+0000","last_fullsized":"2023-10-06T11:34:33.680610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:14.671378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.669154+0000","last_change":"2023-10-06T11:34:33.663688+0000","last_active":"2023-10-06T11:34:33.669154+0000","last_peered":"2023-10-06T11:34:33.669154+0000","last_clean":"2023-10-06T11:34:33.669154+0000","last_became_active":"2023-10-06T11:34:33.663027+0000","last_became_peered":"2023-10-06T11:34:33.663027+0000","last_unstale":"2023-10-06T11:34:33.669154+0000","last_undegraded":"2023-10-06T11:34:33.669154+0000","last_fullsized":"2023-10-06T11:34:33.669154+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:17.151153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.674242+0000","last_change":"2023-10-06T11:34:32.647592+0000","last_active":"2023-10-06T11:34:33.674242+0000","last_peered":"2023-10-06T11:34:33.674242+0000","last_clean":"2023-10-06T11:34:33.674242+0000","last_became_active":"2023-10-06T11:34:32.647230+0000","last_became_peered":"2023-10-06T11:34:32.647230+0000","last_unstale":"2023-10-06T11:34:33.674242+0000","last_undegraded":"2023-10-06T11:34:33.674242+0000","last_fullsized":"2023-10-06T11:34:33.674242+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.492138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694453+0000","last_change":"2023-10-06T11:34:33.692671+0000","last_active":"2023-10-06T11:34:33.694453+0000","last_peered":"2023-10-06T11:34:33.694453+0000","last_clean":"2023-10-06T11:34:33.694453+0000","last_became_active":"2023-10-06T11:34:33.692313+0000","last_became_peered":"2023-10-06T11:34:33.692313+0000","last_unstale":"2023-10-06T11:34:33.694453+0000","last_undegraded":"2023-10-06T11:34:33.694453+0000","last_fullsized":"2023-10-06T11:34:33.694453+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:44.487912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679866+0000","last_change":"2023-10-06T11:34:32.649377+0000","last_active":"2023-10-06T11:34:33.679866+0000","last_peered":"2023-10-06T11:34:33.679866+0000","last_clean":"2023-10-06T11:34:33.679866+0000","last_became_active":"2023-10-06T11:34:32.649105+0000","last_became_peered":"2023-10-06T11:34:32.649105+0000","last_unstale":"2023-10-06T11:34:33.679866+0000","last_undegraded":"2023-10-06T11:34:33.679866+0000","last_fullsized":"2023-10-06T11:34:33.679866+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:02.357176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670760+0000","last_change":"2023-10-06T11:34:33.667041+0000","last_active":"2023-10-06T11:34:33.670760+0000","last_peered":"2023-10-06T11:34:33.670760+0000","last_clean":"2023-10-06T11:34:33.670760+0000","last_became_active":"2023-10-06T11:34:33.666730+0000","last_became_peered":"2023-10-06T11:34:33.666730+0000","last_unstale":"2023-10-06T11:34:33.670760+0000","last_undegraded":"2023-10-06T11:34:33.670760+0000","last_fullsized":"2023-10-06T11:34:33.670760+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:36.837595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"106'2","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.692907+0000","last_change":"2023-10-06T11:34:32.655485+0000","last_active":"2023-10-06T11:34:33.692907+0000","last_peered":"2023-10-06T11:34:33.692907+0000","last_clean":"2023-10-06T11:34:33.692907+0000","last_became_active":"2023-10-06T11:34:32.655097+0000","last_became_peered":"2023-10-06T11:34:32.655097+0000","last_unstale":"2023-10-06T11:34:33.692907+0000","last_undegraded":"2023-10-06T11:34:33.692907+0000","last_fullsized":"2023-10-06T11:34:33.692907+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:47.664215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685926+0000","last_change":"2023-10-06T11:34:33.685926+0000","last_active":"2023-10-06T11:34:33.685926+0000","last_peered":"2023-10-06T11:34:33.685926+0000","last_clean":"2023-10-06T11:34:33.685926+0000","last_became_active":"2023-10-06T11:34:33.685622+0000","last_became_peered":"2023-10-06T11:34:33.685622+0000","last_unstale":"2023-10-06T11:34:33.685926+0000","last_undegraded":"2023-10-06T11:34:33.685926+0000","last_fullsized":"2023-10-06T11:34:33.685926+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:20:48.677381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679642+0000","last_change":"2023-10-06T11:34:32.649956+0000","last_active":"2023-10-06T11:34:33.679642+0000","last_peered":"2023-10-06T11:34:33.679642+0000","last_clean":"2023-10-06T11:34:33.679642+0000","last_became_active":"2023-10-06T11:34:32.649694+0000","last_became_peered":"2023-10-06T11:34:32.649694+0000","last_unstale":"2023-10-06T11:34:33.679642+0000","last_undegraded":"2023-10-06T11:34:33.679642+0000","last_fullsized":"2023-10-06T11:34:33.679642+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.672492+0000","last_change":"2023-10-06T11:34:33.672492+0000","last_active":"2023-10-06T11:34:33.672492+0000","last_peered":"2023-10-06T11:34:33.672492+0000","last_clean":"2023-10-06T11:34:33.672492+0000","last_became_active":"2023-10-06T11:34:33.672202+0000","last_became_peered":"2023-10-06T11:34:33.672202+0000","last_unstale":"2023-10-06T11:34:33.672492+0000","last_undegraded":"2023-10-06T11:34:33.672492+0000","last_fullsized":"2023-10-06T11:34:33.672492+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:43.911900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.668851+0000","last_change":"2023-10-06T11:34:32.645279+0000","last_active":"2023-10-06T11:34:33.668851+0000","last_peered":"2023-10-06T11:34:33.668851+0000","last_clean":"2023-10-06T11:34:33.668851+0000","last_became_active":"2023-10-06T11:34:32.644888+0000","last_became_peered":"2023-10-06T11:34:32.644888+0000","last_unstale":"2023-10-06T11:34:33.668851+0000","last_undegraded":"2023-10-06T11:34:33.668851+0000","last_fullsized":"2023-10-06T11:34:33.668851+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:56.770037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684942+0000","last_change":"2023-10-06T11:34:33.683623+0000","last_active":"2023-10-06T11:34:33.684942+0000","last_peered":"2023-10-06T11:34:33.684942+0000","last_clean":"2023-10-06T11:34:33.684942+0000","last_became_active":"2023-10-06T11:34:33.683305+0000","last_became_peered":"2023-10-06T11:34:33.683305+0000","last_unstale":"2023-10-06T11:34:33.684942+0000","last_undegraded":"2023-10-06T11:34:33.684942+0000","last_fullsized":"2023-10-06T11:34:33.684942+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:46.879894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.647822+0000","last_change":"2023-10-06T11:34:32.645455+0000","last_active":"2023-10-06T11:34:32.647822+0000","last_peered":"2023-10-06T11:34:32.647822+0000","last_clean":"2023-10-06T11:34:32.647822+0000","last_became_active":"2023-10-06T11:34:32.645124+0000","last_became_peered":"2023-10-06T11:34:32.645124+0000","last_unstale":"2023-10-06T11:34:32.647822+0000","last_undegraded":"2023-10-06T11:34:32.647822+0000","last_fullsized":"2023-10-06T11:34:32.647822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:43:30.672956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683162+0000","last_change":"2023-10-06T11:34:33.683162+0000","last_active":"2023-10-06T11:34:33.683162+0000","last_peered":"2023-10-06T11:34:33.683162+0000","last_clean":"2023-10-06T11:34:33.683162+0000","last_became_active":"2023-10-06T11:34:33.682638+0000","last_became_peered":"2023-10-06T11:34:33.682638+0000","last_unstale":"2023-10-06T11:34:33.683162+0000","last_undegraded":"2023-10-06T11:34:33.683162+0000","last_fullsized":"2023-10-06T11:34:33.683162+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:18.021965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665586+0000","last_change":"2023-10-06T11:34:32.654888+0000","last_active":"2023-10-06T11:34:33.665586+0000","last_peered":"2023-10-06T11:34:33.665586+0000","last_clean":"2023-10-06T11:34:33.665586+0000","last_became_active":"2023-10-06T11:34:32.654616+0000","last_became_peered":"2023-10-06T11:34:32.654616+0000","last_unstale":"2023-10-06T11:34:33.665586+0000","last_undegraded":"2023-10-06T11:34:33.665586+0000","last_fullsized":"2023-10-06T11:34:33.665586+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:03:59.992301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667495+0000","last_change":"2023-10-06T11:34:33.661439+0000","last_active":"2023-10-06T11:34:33.667495+0000","last_peered":"2023-10-06T11:34:33.667495+0000","last_clean":"2023-10-06T11:34:33.667495+0000","last_became_active":"2023-10-06T11:34:33.661081+0000","last_became_peered":"2023-10-06T11:34:33.661081+0000","last_unstale":"2023-10-06T11:34:33.667495+0000","last_undegraded":"2023-10-06T11:34:33.667495+0000","last_fullsized":"2023-10-06T11:34:33.667495+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:21.956959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661600+0000","last_change":"2023-10-06T11:34:32.648019+0000","last_active":"2023-10-06T11:34:33.661600+0000","last_peered":"2023-10-06T11:34:33.661600+0000","last_clean":"2023-10-06T11:34:33.661600+0000","last_became_active":"2023-10-06T11:34:32.647715+0000","last_became_peered":"2023-10-06T11:34:32.647715+0000","last_unstale":"2023-10-06T11:34:33.661600+0000","last_undegraded":"2023-10-06T11:34:33.661600+0000","last_fullsized":"2023-10-06T11:34:33.661600+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:44:59.542131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680555+0000","last_change":"2023-10-06T11:34:33.675319+0000","last_active":"2023-10-06T11:34:33.680555+0000","last_peered":"2023-10-06T11:34:33.680555+0000","last_clean":"2023-10-06T11:34:33.680555+0000","last_became_active":"2023-10-06T11:34:33.675029+0000","last_became_peered":"2023-10-06T11:34:33.675029+0000","last_unstale":"2023-10-06T11:34:33.680555+0000","last_undegraded":"2023-10-06T11:34:33.680555+0000","last_fullsized":"2023-10-06T11:34:33.680555+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:43.276708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.648133+0000","last_change":"2023-10-06T11:34:32.645376+0000","last_active":"2023-10-06T11:34:32.648133+0000","last_peered":"2023-10-06T11:34:32.648133+0000","last_clean":"2023-10-06T11:34:32.648133+0000","last_became_active":"2023-10-06T11:34:32.645035+0000","last_became_peered":"2023-10-06T11:34:32.645035+0000","last_unstale":"2023-10-06T11:34:32.648133+0000","last_undegraded":"2023-10-06T11:34:32.648133+0000","last_fullsized":"2023-10-06T11:34:32.648133+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:32.807774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.657062+0000","last_change":"2023-10-06T11:34:33.657062+0000","last_active":"2023-10-06T11:34:33.657062+0000","last_peered":"2023-10-06T11:34:33.657062+0000","last_clean":"2023-10-06T11:34:33.657062+0000","last_became_active":"2023-10-06T11:34:33.656765+0000","last_became_peered":"2023-10-06T11:34:33.656765+0000","last_unstale":"2023-10-06T11:34:33.657062+0000","last_undegraded":"2023-10-06T11:34:33.657062+0000","last_fullsized":"2023-10-06T11:34:33.657062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:39.610289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679228+0000","last_change":"2023-10-06T11:34:32.655778+0000","last_active":"2023-10-06T11:34:33.679228+0000","last_peered":"2023-10-06T11:34:33.679228+0000","last_clean":"2023-10-06T11:34:33.679228+0000","last_became_active":"2023-10-06T11:34:32.655472+0000","last_became_peered":"2023-10-06T11:34:32.655472+0000","last_unstale":"2023-10-06T11:34:33.679228+0000","last_undegraded":"2023-10-06T11:34:33.679228+0000","last_fullsized":"2023-10-06T11:34:33.679228+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:26.335636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683558+0000","last_change":"2023-10-06T11:34:33.683558+0000","last_active":"2023-10-06T11:34:33.683558+0000","last_peered":"2023-10-06T11:34:33.683558+0000","last_clean":"2023-10-06T11:34:33.683558+0000","last_became_active":"2023-10-06T11:34:33.682593+0000","last_became_peered":"2023-10-06T11:34:33.682593+0000","last_unstale":"2023-10-06T11:34:33.683558+0000","last_undegraded":"2023-10-06T11:34:33.683558+0000","last_fullsized":"2023-10-06T11:34:33.683558+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:42.405551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683118+0000","last_change":"2023-10-06T11:34:32.644341+0000","last_active":"2023-10-06T11:34:33.683118+0000","last_peered":"2023-10-06T11:34:33.683118+0000","last_clean":"2023-10-06T11:34:33.683118+0000","last_became_active":"2023-10-06T11:34:32.643800+0000","last_became_peered":"2023-10-06T11:34:32.643800+0000","last_unstale":"2023-10-06T11:34:33.683118+0000","last_undegraded":"2023-10-06T11:34:33.683118+0000","last_fullsized":"2023-10-06T11:34:33.683118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:11:36.438764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676840+0000","last_change":"2023-10-06T11:34:33.676840+0000","last_active":"2023-10-06T11:34:33.676840+0000","last_peered":"2023-10-06T11:34:33.676840+0000","last_clean":"2023-10-06T11:34:33.676840+0000","last_became_active":"2023-10-06T11:34:33.676372+0000","last_became_peered":"2023-10-06T11:34:33.676372+0000","last_unstale":"2023-10-06T11:34:33.676840+0000","last_undegraded":"2023-10-06T11:34:33.676840+0000","last_fullsized":"2023-10-06T11:34:33.676840+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:57:09.455819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679268+0000","last_change":"2023-10-06T11:34:32.657277+0000","last_active":"2023-10-06T11:34:33.679268+0000","last_peered":"2023-10-06T11:34:33.679268+0000","last_clean":"2023-10-06T11:34:33.679268+0000","last_became_active":"2023-10-06T11:34:32.656940+0000","last_became_peered":"2023-10-06T11:34:32.656940+0000","last_unstale":"2023-10-06T11:34:33.679268+0000","last_undegraded":"2023-10-06T11:34:33.679268+0000","last_fullsized":"2023-10-06T11:34:33.679268+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:30.475842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678816+0000","last_change":"2023-10-06T11:34:33.671809+0000","last_active":"2023-10-06T11:34:33.678816+0000","last_peered":"2023-10-06T11:34:33.678816+0000","last_clean":"2023-10-06T11:34:33.678816+0000","last_became_active":"2023-10-06T11:34:33.671508+0000","last_became_peered":"2023-10-06T11:34:33.671508+0000","last_unstale":"2023-10-06T11:34:33.678816+0000","last_undegraded":"2023-10-06T11:34:33.678816+0000","last_fullsized":"2023-10-06T11:34:33.678816+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:46.220694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.650445+0000","last_change":"2023-10-06T11:34:32.643183+0000","last_active":"2023-10-06T11:34:33.650445+0000","last_peered":"2023-10-06T11:34:33.650445+0000","last_clean":"2023-10-06T11:34:33.650445+0000","last_became_active":"2023-10-06T11:34:32.642817+0000","last_became_peered":"2023-10-06T11:34:32.642817+0000","last_unstale":"2023-10-06T11:34:33.650445+0000","last_undegraded":"2023-10-06T11:34:33.650445+0000","last_fullsized":"2023-10-06T11:34:33.650445+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:00.944215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683254+0000","last_change":"2023-10-06T11:34:33.683254+0000","last_active":"2023-10-06T11:34:33.683254+0000","last_peered":"2023-10-06T11:34:33.683254+0000","last_clean":"2023-10-06T11:34:33.683254+0000","last_became_active":"2023-10-06T11:34:33.682947+0000","last_became_peered":"2023-10-06T11:34:33.682947+0000","last_unstale":"2023-10-06T11:34:33.683254+0000","last_undegraded":"2023-10-06T11:34:33.683254+0000","last_fullsized":"2023-10-06T11:34:33.683254+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:36.159994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643450+0000","last_change":"2023-10-06T11:34:32.641151+0000","last_active":"2023-10-06T11:34:32.643450+0000","last_peered":"2023-10-06T11:34:32.643450+0000","last_clean":"2023-10-06T11:34:32.643450+0000","last_became_active":"2023-10-06T11:34:32.640828+0000","last_became_peered":"2023-10-06T11:34:32.640828+0000","last_unstale":"2023-10-06T11:34:32.643450+0000","last_undegraded":"2023-10-06T11:34:32.643450+0000","last_fullsized":"2023-10-06T11:34:32.643450+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:23:46.017502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659861+0000","last_change":"2023-10-06T11:34:33.659861+0000","last_active":"2023-10-06T11:34:33.659861+0000","last_peered":"2023-10-06T11:34:33.659861+0000","last_clean":"2023-10-06T11:34:33.659861+0000","last_became_active":"2023-10-06T11:34:33.659544+0000","last_became_peered":"2023-10-06T11:34:33.659544+0000","last_unstale":"2023-10-06T11:34:33.659861+0000","last_undegraded":"2023-10-06T11:34:33.659861+0000","last_fullsized":"2023-10-06T11:34:33.659861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:40.124804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'8","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.683428+0000","last_change":"2023-10-06T11:34:32.649537+0000","last_active":"2023-10-06T11:34:33.683428+0000","last_peered":"2023-10-06T11:34:33.683428+0000","last_clean":"2023-10-06T11:34:33.683428+0000","last_became_active":"2023-10-06T11:34:32.648576+0000","last_became_peered":"2023-10-06T11:34:32.648576+0000","last_unstale":"2023-10-06T11:34:33.683428+0000","last_undegraded":"2023-10-06T11:34:33.683428+0000","last_fullsized":"2023-10-06T11:34:33.683428+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:58.440067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.676004+0000","last_change":"2023-10-06T11:34:33.676004+0000","last_active":"2023-10-06T11:34:33.676004+0000","last_peered":"2023-10-06T11:34:33.676004+0000","last_clean":"2023-10-06T11:34:33.676004+0000","last_became_active":"2023-10-06T11:34:33.675711+0000","last_became_peered":"2023-10-06T11:34:33.675711+0000","last_unstale":"2023-10-06T11:34:33.676004+0000","last_undegraded":"2023-10-06T11:34:33.676004+0000","last_fullsized":"2023-10-06T11:34:33.676004+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:05.528927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.677458+0000","last_change":"2023-10-06T11:34:32.648689+0000","last_active":"2023-10-06T11:34:33.677458+0000","last_peered":"2023-10-06T11:34:33.677458+0000","last_clean":"2023-10-06T11:34:33.677458+0000","last_became_active":"2023-10-06T11:34:32.648413+0000","last_became_peered":"2023-10-06T11:34:32.648413+0000","last_unstale":"2023-10-06T11:34:33.677458+0000","last_undegraded":"2023-10-06T11:34:33.677458+0000","last_fullsized":"2023-10-06T11:34:33.677458+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:25:56.442043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.663901+0000","last_change":"2023-10-06T11:34:33.660286+0000","last_active":"2023-10-06T11:34:33.663901+0000","last_peered":"2023-10-06T11:34:33.663901+0000","last_clean":"2023-10-06T11:34:33.663901+0000","last_became_active":"2023-10-06T11:34:33.659955+0000","last_became_peered":"2023-10-06T11:34:33.659955+0000","last_unstale":"2023-10-06T11:34:33.663901+0000","last_undegraded":"2023-10-06T11:34:33.663901+0000","last_fullsized":"2023-10-06T11:34:33.663901+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:44.228067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.656246+0000","last_change":"2023-10-06T11:34:32.642265+0000","last_active":"2023-10-06T11:34:33.656246+0000","last_peered":"2023-10-06T11:34:33.656246+0000","last_clean":"2023-10-06T11:34:33.656246+0000","last_became_active":"2023-10-06T11:34:32.641949+0000","last_became_peered":"2023-10-06T11:34:32.641949+0000","last_unstale":"2023-10-06T11:34:33.656246+0000","last_undegraded":"2023-10-06T11:34:33.656246+0000","last_fullsized":"2023-10-06T11:34:33.656246+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:26.283803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.670001+0000","last_change":"2023-10-06T11:34:33.663534+0000","last_active":"2023-10-06T11:34:33.670001+0000","last_peered":"2023-10-06T11:34:33.670001+0000","last_clean":"2023-10-06T11:34:33.670001+0000","last_became_active":"2023-10-06T11:34:33.663226+0000","last_became_peered":"2023-10-06T11:34:33.663226+0000","last_unstale":"2023-10-06T11:34:33.670001+0000","last_undegraded":"2023-10-06T11:34:33.670001+0000","last_fullsized":"2023-10-06T11:34:33.670001+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:08:34.075211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.656391+0000","last_change":"2023-10-06T11:34:32.655780+0000","last_active":"2023-10-06T11:34:33.656391+0000","last_peered":"2023-10-06T11:34:33.656391+0000","last_clean":"2023-10-06T11:34:33.656391+0000","last_became_active":"2023-10-06T11:34:32.655466+0000","last_became_peered":"2023-10-06T11:34:32.655466+0000","last_unstale":"2023-10-06T11:34:33.656391+0000","last_undegraded":"2023-10-06T11:34:33.656391+0000","last_fullsized":"2023-10-06T11:34:33.656391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:21:06.714958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.679835+0000","last_change":"2023-10-06T11:34:33.672940+0000","last_active":"2023-10-06T11:34:33.679835+0000","last_peered":"2023-10-06T11:34:33.679835+0000","last_clean":"2023-10-06T11:34:33.679835+0000","last_became_active":"2023-10-06T11:34:33.672642+0000","last_became_peered":"2023-10-06T11:34:33.672642+0000","last_unstale":"2023-10-06T11:34:33.679835+0000","last_undegraded":"2023-10-06T11:34:33.679835+0000","last_fullsized":"2023-10-06T11:34:33.679835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:10.885411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.651864+0000","last_change":"2023-10-06T11:34:32.649796+0000","last_active":"2023-10-06T11:34:32.651864+0000","last_peered":"2023-10-06T11:34:32.651864+0000","last_clean":"2023-10-06T11:34:32.651864+0000","last_became_active":"2023-10-06T11:34:32.649436+0000","last_became_peered":"2023-10-06T11:34:32.649436+0000","last_unstale":"2023-10-06T11:34:32.651864+0000","last_undegraded":"2023-10-06T11:34:32.651864+0000","last_fullsized":"2023-10-06T11:34:32.651864+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:52.916318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667135+0000","last_change":"2023-10-06T11:34:33.667135+0000","last_active":"2023-10-06T11:34:33.667135+0000","last_peered":"2023-10-06T11:34:33.667135+0000","last_clean":"2023-10-06T11:34:33.667135+0000","last_became_active":"2023-10-06T11:34:33.666576+0000","last_became_peered":"2023-10-06T11:34:33.666576+0000","last_unstale":"2023-10-06T11:34:33.667135+0000","last_undegraded":"2023-10-06T11:34:33.667135+0000","last_fullsized":"2023-10-06T11:34:33.667135+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:47.521730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.651819+0000","last_change":"2023-10-06T11:34:32.643385+0000","last_active":"2023-10-06T11:34:33.651819+0000","last_peered":"2023-10-06T11:34:33.651819+0000","last_clean":"2023-10-06T11:34:33.651819+0000","last_became_active":"2023-10-06T11:34:32.642888+0000","last_became_peered":"2023-10-06T11:34:32.642888+0000","last_unstale":"2023-10-06T11:34:33.651819+0000","last_undegraded":"2023-10-06T11:34:33.651819+0000","last_fullsized":"2023-10-06T11:34:33.651819+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:52:11.370315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678222+0000","last_change":"2023-10-06T11:34:33.678222+0000","last_active":"2023-10-06T11:34:33.678222+0000","last_peered":"2023-10-06T11:34:33.678222+0000","last_clean":"2023-10-06T11:34:33.678222+0000","last_became_active":"2023-10-06T11:34:33.677851+0000","last_became_peered":"2023-10-06T11:34:33.677851+0000","last_unstale":"2023-10-06T11:34:33.678222+0000","last_undegraded":"2023-10-06T11:34:33.678222+0000","last_fullsized":"2023-10-06T11:34:33.678222+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:58.197183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661069+0000","last_change":"2023-10-06T11:34:32.654697+0000","last_active":"2023-10-06T11:34:33.661069+0000","last_peered":"2023-10-06T11:34:33.661069+0000","last_clean":"2023-10-06T11:34:33.661069+0000","last_became_active":"2023-10-06T11:34:32.654203+0000","last_became_peered":"2023-10-06T11:34:32.654203+0000","last_unstale":"2023-10-06T11:34:33.661069+0000","last_undegraded":"2023-10-06T11:34:33.661069+0000","last_fullsized":"2023-10-06T11:34:33.661069+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:43.266091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.693540+0000","last_change":"2023-10-06T11:34:33.693540+0000","last_active":"2023-10-06T11:34:33.693540+0000","last_peered":"2023-10-06T11:34:33.693540+0000","last_clean":"2023-10-06T11:34:33.693540+0000","last_became_active":"2023-10-06T11:34:33.693158+0000","last_became_peered":"2023-10-06T11:34:33.693158+0000","last_unstale":"2023-10-06T11:34:33.693540+0000","last_undegraded":"2023-10-06T11:34:33.693540+0000","last_fullsized":"2023-10-06T11:34:33.693540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:38:04.711975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.651958+0000","last_change":"2023-10-06T11:34:32.650023+0000","last_active":"2023-10-06T11:34:32.651958+0000","last_peered":"2023-10-06T11:34:32.651958+0000","last_clean":"2023-10-06T11:34:32.651958+0000","last_became_active":"2023-10-06T11:34:32.649734+0000","last_became_peered":"2023-10-06T11:34:32.649734+0000","last_unstale":"2023-10-06T11:34:32.651958+0000","last_undegraded":"2023-10-06T11:34:32.651958+0000","last_fullsized":"2023-10-06T11:34:32.651958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:17.008799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661047+0000","last_change":"2023-10-06T11:34:33.658055+0000","last_active":"2023-10-06T11:34:33.661047+0000","last_peered":"2023-10-06T11:34:33.661047+0000","last_clean":"2023-10-06T11:34:33.661047+0000","last_became_active":"2023-10-06T11:34:33.657743+0000","last_became_peered":"2023-10-06T11:34:33.657743+0000","last_unstale":"2023-10-06T11:34:33.661047+0000","last_undegraded":"2023-10-06T11:34:33.661047+0000","last_fullsized":"2023-10-06T11:34:33.661047+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.268916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665476+0000","last_change":"2023-10-06T11:34:32.649029+0000","last_active":"2023-10-06T11:34:33.665476+0000","last_peered":"2023-10-06T11:34:33.665476+0000","last_clean":"2023-10-06T11:34:33.665476+0000","last_became_active":"2023-10-06T11:34:32.648670+0000","last_became_peered":"2023-10-06T11:34:32.648670+0000","last_unstale":"2023-10-06T11:34:33.665476+0000","last_undegraded":"2023-10-06T11:34:33.665476+0000","last_fullsized":"2023-10-06T11:34:33.665476+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:49:09.872244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.686921+0000","last_change":"2023-10-06T11:34:33.685128+0000","last_active":"2023-10-06T11:34:33.686921+0000","last_peered":"2023-10-06T11:34:33.686921+0000","last_clean":"2023-10-06T11:34:33.686921+0000","last_became_active":"2023-10-06T11:34:33.684834+0000","last_became_peered":"2023-10-06T11:34:33.684834+0000","last_unstale":"2023-10-06T11:34:33.686921+0000","last_undegraded":"2023-10-06T11:34:33.686921+0000","last_fullsized":"2023-10-06T11:34:33.686921+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:01.957381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.648663+0000","last_change":"2023-10-06T11:34:32.645362+0000","last_active":"2023-10-06T11:34:32.648663+0000","last_peered":"2023-10-06T11:34:32.648663+0000","last_clean":"2023-10-06T11:34:32.648663+0000","last_became_active":"2023-10-06T11:34:32.645029+0000","last_became_peered":"2023-10-06T11:34:32.645029+0000","last_unstale":"2023-10-06T11:34:32.648663+0000","last_undegraded":"2023-10-06T11:34:32.648663+0000","last_fullsized":"2023-10-06T11:34:32.648663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:29.397372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"95'205","reported_seq":551,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.667599+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:34:33.667599+0000","last_peered":"2023-10-06T11:34:33.667599+0000","last_clean":"2023-10-06T11:34:33.667599+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:34:33.667599+0000","last_undegraded":"2023-10-06T11:34:33.667599+0000","last_fullsized":"2023-10-06T11:34:33.667599+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.660947+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:34:33.660947+0000","last_peered":"2023-10-06T11:34:33.660947+0000","last_clean":"2023-10-06T11:34:33.660947+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:34:33.660947+0000","last_undegraded":"2023-10-06T11:34:33.660947+0000","last_fullsized":"2023-10-06T11:34:33.660947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659762+0000","last_change":"2023-10-06T11:34:33.656565+0000","last_active":"2023-10-06T11:34:33.659762+0000","last_peered":"2023-10-06T11:34:33.659762+0000","last_clean":"2023-10-06T11:34:33.659762+0000","last_became_active":"2023-10-06T11:34:33.656236+0000","last_became_peered":"2023-10-06T11:34:33.656236+0000","last_unstale":"2023-10-06T11:34:33.659762+0000","last_undegraded":"2023-10-06T11:34:33.659762+0000","last_fullsized":"2023-10-06T11:34:33.659762+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:23:42.750299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665522+0000","last_change":"2023-10-06T11:34:32.646863+0000","last_active":"2023-10-06T11:34:33.665522+0000","last_peered":"2023-10-06T11:34:33.665522+0000","last_clean":"2023-10-06T11:34:33.665522+0000","last_became_active":"2023-10-06T11:34:32.646465+0000","last_became_peered":"2023-10-06T11:34:32.646465+0000","last_unstale":"2023-10-06T11:34:33.665522+0000","last_undegraded":"2023-10-06T11:34:33.665522+0000","last_fullsized":"2023-10-06T11:34:33.665522+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:48.256870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665674+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:34:33.665674+0000","last_peered":"2023-10-06T11:34:33.665674+0000","last_clean":"2023-10-06T11:34:33.665674+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:34:33.665674+0000","last_undegraded":"2023-10-06T11:34:33.665674+0000","last_fullsized":"2023-10-06T11:34:33.665674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.678339+0000","last_change":"2023-10-06T11:34:33.678339+0000","last_active":"2023-10-06T11:34:33.678339+0000","last_peered":"2023-10-06T11:34:33.678339+0000","last_clean":"2023-10-06T11:34:33.678339+0000","last_became_active":"2023-10-06T11:34:33.678043+0000","last_became_peered":"2023-10-06T11:34:33.678043+0000","last_unstale":"2023-10-06T11:34:33.678339+0000","last_undegraded":"2023-10-06T11:34:33.678339+0000","last_fullsized":"2023-10-06T11:34:33.678339+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:27.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.677092+0000","last_change":"2023-10-06T11:34:32.647129+0000","last_active":"2023-10-06T11:34:33.677092+0000","last_peered":"2023-10-06T11:34:33.677092+0000","last_clean":"2023-10-06T11:34:33.677092+0000","last_became_active":"2023-10-06T11:34:32.646792+0000","last_became_peered":"2023-10-06T11:34:32.646792+0000","last_unstale":"2023-10-06T11:34:33.677092+0000","last_undegraded":"2023-10-06T11:34:33.677092+0000","last_fullsized":"2023-10-06T11:34:33.677092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:40:15.563869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.629582+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:34:32.629582+0000","last_peered":"2023-10-06T11:34:32.629582+0000","last_clean":"2023-10-06T11:34:32.629582+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:34:32.629582+0000","last_undegraded":"2023-10-06T11:34:32.629582+0000","last_fullsized":"2023-10-06T11:34:32.629582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.671777+0000","last_change":"2023-10-06T11:34:33.668139+0000","last_active":"2023-10-06T11:34:33.671777+0000","last_peered":"2023-10-06T11:34:33.671777+0000","last_clean":"2023-10-06T11:34:33.671777+0000","last_became_active":"2023-10-06T11:34:33.667026+0000","last_became_peered":"2023-10-06T11:34:33.667026+0000","last_unstale":"2023-10-06T11:34:33.671777+0000","last_undegraded":"2023-10-06T11:34:33.671777+0000","last_fullsized":"2023-10-06T11:34:33.671777+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:03.145906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694854+0000","last_change":"2023-10-06T11:34:32.649272+0000","last_active":"2023-10-06T11:34:33.694854+0000","last_peered":"2023-10-06T11:34:33.694854+0000","last_clean":"2023-10-06T11:34:33.694854+0000","last_became_active":"2023-10-06T11:34:32.648974+0000","last_became_peered":"2023-10-06T11:34:32.648974+0000","last_unstale":"2023-10-06T11:34:33.694854+0000","last_undegraded":"2023-10-06T11:34:33.694854+0000","last_fullsized":"2023-10-06T11:34:33.694854+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:38.525361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.661168+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:34:33.661168+0000","last_peered":"2023-10-06T11:34:33.661168+0000","last_clean":"2023-10-06T11:34:33.661168+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:34:33.661168+0000","last_undegraded":"2023-10-06T11:34:33.661168+0000","last_fullsized":"2023-10-06T11:34:33.661168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.685088+0000","last_change":"2023-10-06T11:34:33.680636+0000","last_active":"2023-10-06T11:34:33.685088+0000","last_peered":"2023-10-06T11:34:33.685088+0000","last_clean":"2023-10-06T11:34:33.685088+0000","last_became_active":"2023-10-06T11:34:33.680098+0000","last_became_peered":"2023-10-06T11:34:33.680098+0000","last_unstale":"2023-10-06T11:34:33.685088+0000","last_undegraded":"2023-10-06T11:34:33.685088+0000","last_fullsized":"2023-10-06T11:34:33.685088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:07:28.152443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.684902+0000","last_change":"2023-10-06T11:34:32.654547+0000","last_active":"2023-10-06T11:34:33.684902+0000","last_peered":"2023-10-06T11:34:33.684902+0000","last_clean":"2023-10-06T11:34:33.684902+0000","last_became_active":"2023-10-06T11:34:32.654132+0000","last_became_peered":"2023-10-06T11:34:32.654132+0000","last_unstale":"2023-10-06T11:34:33.684902+0000","last_undegraded":"2023-10-06T11:34:33.684902+0000","last_fullsized":"2023-10-06T11:34:33.684902+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:42.898733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.637934+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:34:32.637934+0000","last_peered":"2023-10-06T11:34:32.637934+0000","last_clean":"2023-10-06T11:34:32.637934+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:34:32.637934+0000","last_undegraded":"2023-10-06T11:34:32.637934+0000","last_fullsized":"2023-10-06T11:34:32.637934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.680598+0000","last_change":"2023-10-06T11:34:33.676211+0000","last_active":"2023-10-06T11:34:33.680598+0000","last_peered":"2023-10-06T11:34:33.680598+0000","last_clean":"2023-10-06T11:34:33.680598+0000","last_became_active":"2023-10-06T11:34:33.675930+0000","last_became_peered":"2023-10-06T11:34:33.675930+0000","last_unstale":"2023-10-06T11:34:33.680598+0000","last_undegraded":"2023-10-06T11:34:33.680598+0000","last_fullsized":"2023-10-06T11:34:33.680598+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:26:37.851550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.644287+0000","last_change":"2023-10-06T11:34:32.642162+0000","last_active":"2023-10-06T11:34:32.644287+0000","last_peered":"2023-10-06T11:34:32.644287+0000","last_clean":"2023-10-06T11:34:32.644287+0000","last_became_active":"2023-10-06T11:34:32.641874+0000","last_became_peered":"2023-10-06T11:34:32.641874+0000","last_unstale":"2023-10-06T11:34:32.644287+0000","last_undegraded":"2023-10-06T11:34:32.644287+0000","last_fullsized":"2023-10-06T11:34:32.644287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:29.397372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":86,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.666544+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:34:33.666544+0000","last_peered":"2023-10-06T11:34:33.666544+0000","last_clean":"2023-10-06T11:34:33.666544+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:34:33.666544+0000","last_undegraded":"2023-10-06T11:34:33.666544+0000","last_fullsized":"2023-10-06T11:34:33.666544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.694973+0000","last_change":"2023-10-06T11:34:33.692682+0000","last_active":"2023-10-06T11:34:33.694973+0000","last_peered":"2023-10-06T11:34:33.694973+0000","last_clean":"2023-10-06T11:34:33.694973+0000","last_became_active":"2023-10-06T11:34:33.692195+0000","last_became_peered":"2023-10-06T11:34:33.692195+0000","last_unstale":"2023-10-06T11:34:33.694973+0000","last_undegraded":"2023-10-06T11:34:33.694973+0000","last_fullsized":"2023-10-06T11:34:33.694973+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:21.956959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.659610+0000","last_change":"2023-10-06T11:34:32.651500+0000","last_active":"2023-10-06T11:34:33.659610+0000","last_peered":"2023-10-06T11:34:33.659610+0000","last_clean":"2023-10-06T11:34:33.659610+0000","last_became_active":"2023-10-06T11:34:32.651202+0000","last_became_peered":"2023-10-06T11:34:32.651202+0000","last_unstale":"2023-10-06T11:34:33.659610+0000","last_undegraded":"2023-10-06T11:34:33.659610+0000","last_fullsized":"2023-10-06T11:34:33.659610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:40.756548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":84,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.637407+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:34:32.637407+0000","last_peered":"2023-10-06T11:34:32.637407+0000","last_clean":"2023-10-06T11:34:32.637407+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:34:32.637407+0000","last_undegraded":"2023-10-06T11:34:32.637407+0000","last_fullsized":"2023-10-06T11:34:32.637407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T11:34:33.665770+0000","last_change":"2023-10-06T11:34:33.662908+0000","last_active":"2023-10-06T11:34:33.665770+0000","last_peered":"2023-10-06T11:34:33.665770+0000","last_clean":"2023-10-06T11:34:33.665770+0000","last_became_active":"2023-10-06T11:34:33.662590+0000","last_became_peered":"2023-10-06T11:34:33.662590+0000","last_unstale":"2023-10-06T11:34:33.665770+0000","last_undegraded":"2023-10-06T11:34:33.665770+0000","last_fullsized":"2023-10-06T11:34:33.665770+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:32.609736+0000","last_clean_scrub_stamp":"2023-10-06T11:34:32.609736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:08:45.367549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643003+0000","last_change":"2023-10-06T11:34:32.636772+0000","last_active":"2023-10-06T11:34:32.643003+0000","last_peered":"2023-10-06T11:34:32.643003+0000","last_clean":"2023-10-06T11:34:32.643003+0000","last_became_active":"2023-10-06T11:34:32.636407+0000","last_became_peered":"2023-10-06T11:34:32.636407+0000","last_unstale":"2023-10-06T11:34:32.643003+0000","last_undegraded":"2023-10-06T11:34:32.643003+0000","last_fullsized":"2023-10-06T11:34:32.643003+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:34:31.596570+0000","last_clean_scrub_stamp":"2023-10-06T11:34:31.596570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:58.440067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-06T11:34:32.643721+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:34:32.643721+0000","last_peered":"2023-10-06T11:34:32.643721+0000","last_clean":"2023-10-06T11:34:32.643721+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:34:32.643721+0000","last_undegraded":"2023-10-06T11:34:32.643721+0000","last_fullsized":"2023-10-06T11:34:32.643721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575042,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311436,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060404,"statfs":{"total":96636764160,"available":96317853696,"internally_reserved":0,"allocated":1904640,"data_stored":1259016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1200000000000001,"15min":1.1319999999999999},"min":{"1min":0.73599999999999999,"5min":0.71199999999999997,"15min":0.495},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.827},"last":1.526},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1279999999999999,"15min":1.135},"min":{"1min":0.71299999999999997,"5min":0.67500000000000004,"15min":0.56399999999999995},"max":{"1min":1.577,"5min":1.649,"15min":2.0030000000000001},"last":1.1599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3169999999999999,"15min":1.3300000000000001},"min":{"1min":0.90000000000000002,"5min":0.65900000000000003,"15min":0.59499999999999997},"max":{"1min":1.7649999999999999,"5min":1.871,"15min":2.6059999999999999},"last":1.26},{"interface":"front","average":{"1min":1.244,"5min":1.236,"15min":1.2470000000000001},"min":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.69799999999999995},"max":{"1min":1.927,"5min":1.927,"15min":1.931},"last":1.212}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.502,"5min":1.827,"15min":2.3799999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.258,"15min":1.2609999999999999},"min":{"1min":1.054,"5min":0.69899999999999995,"15min":0.65000000000000002},"max":{"1min":1.7,"5min":1.9830000000000001,"15min":2.2850000000000001},"last":1.0129999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.28},"min":{"1min":0.82099999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8,"5min":1.9450000000000001,"15min":2.665},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.296,"15min":1.321},"min":{"1min":0.92900000000000005,"5min":0.75600000000000001,"15min":0.71699999999999997},"max":{"1min":1.7509999999999999,"5min":1.796,"15min":2.8570000000000002},"last":1.4259999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.77300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5569999999999999,"5min":2.149,"15min":2.7970000000000002},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.264,"15min":1.2669999999999999},"min":{"1min":0.86599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.5339999999999998},"last":1.7609999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.335,"15min":1.3160000000000001},"min":{"1min":0.77900000000000003,"5min":0.58199999999999996,"15min":0.54000000000000004},"max":{"1min":1.7410000000000001,"5min":2.0840000000000001,"15min":2.9529999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.28},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.71,"5min":2.2480000000000002,"15min":3.105},"last":1.677}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.365,"15min":1.361},"min":{"1min":1.0369999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0190000000000001,"5min":2.4289999999999998,"15min":3.004},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.86299999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.905,"5min":2.121,"15min":2.9119999999999999},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575042,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":670704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.276,"15min":1.288},"min":{"1min":0.68400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5629999999999999,"5min":1.7490000000000001,"15min":2.1379999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.333,"15min":1.325},"min":{"1min":0.83799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6830000000000001,"5min":1.6899999999999999,"15min":2.1920000000000002},"last":1.02}]},{"osd":1,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.80500000000000005,"5min":0.76200000000000001,"15min":0.63900000000000001},"max":{"1min":1.5029999999999999,"5min":1.6379999999999999,"15min":2.0179999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.605,"5min":1.669,"15min":1.8480000000000001},"last":1.3580000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1970000000000001,"15min":1.2},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.74299999999999999},"max":{"1min":1.514,"5min":1.635,"15min":1.6919999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.194,"15min":1.222},"min":{"1min":0.79000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.4450000000000001,"5min":1.587,"15min":2.7789999999999999},"last":1.2030000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2749999999999999,"15min":1.3100000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.534,"5min":1.764,"15min":1.9850000000000001},"last":1.147},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3120000000000001,"15min":1.3049999999999999},"min":{"1min":1.077,"5min":0.95599999999999996,"15min":0.83199999999999996},"max":{"1min":1.5269999999999999,"5min":1.724,"15min":2.109},"last":1.585}]},{"osd":4,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.238,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.76500000000000001,"15min":0.61399999999999999},"max":{"1min":1.6639999999999999,"5min":1.8,"15min":1.956},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.292,"15min":1.2490000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.73899999999999999},"max":{"1min":1.645,"5min":1.827,"15min":1.8879999999999999},"last":1.494}]},{"osd":5,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.90500000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6000000000000001,"5min":1.72,"15min":2.198},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.395,"15min":1.385},"min":{"1min":0.86499999999999999,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.718,"5min":1.879,"15min":2.1509999999999998},"last":1.524}]},{"osd":7,"last update":"Fri Oct 6 11:33:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4510000000000001,"15min":1.448},"min":{"1min":0.90100000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.871,"5min":1.927,"15min":2.2919999999999998},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.4139999999999999,"15min":1.4299999999999999},"min":{"1min":1.004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6879999999999999,"5min":1.8260000000000001,"15min":3.891},"last":1.5589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575042,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302656,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069184,"statfs":{"total":96636764160,"available":96326844416,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.3899999999999999,"15min":1.268},"min":{"1min":0.67400000000000004,"5min":0.5,"15min":0.5},"max":{"1min":1.772,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.64900000000000002},"max":{"1min":1.679,"5min":1.972,"15min":2.544},"last":1.196}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.194,"15min":1.175},"min":{"1min":0.79800000000000004,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.478,"5min":1.8169999999999999,"15min":2.4209999999999998},"last":1.262},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.2190000000000001},"min":{"1min":0.79900000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5189999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.4590000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.159,"15min":1.1950000000000001},"min":{"1min":0.78600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.601,"5min":1.6879999999999999,"15min":3.609},"last":1.091},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.73299999999999998},"max":{"1min":1.512,"5min":1.738,"15min":2.6949999999999998},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.393,"15min":1.296},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.639,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.302,"15min":1.3340000000000001},"min":{"1min":0.91800000000000004,"5min":0.91300000000000003,"15min":0.77700000000000002},"max":{"1min":1.5229999999999999,"5min":1.734,"15min":3.7770000000000001},"last":1.361}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.40699999999999997},"max":{"1min":1.6060000000000001,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.516,"15min":1.405},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59799999999999998},"max":{"1min":1.724,"5min":18.48,"15min":18.48},"last":1.222}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.3089999999999999},"min":{"1min":1.0129999999999999,"5min":0.85199999999999998,"15min":0.622},"max":{"1min":1.6799999999999999,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.411,"15min":1.351},"min":{"1min":0.93799999999999994,"5min":0.81999999999999995,"15min":0.46300000000000002},"max":{"1min":1.6870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.1479999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.5249999999999999,"15min":1.4039999999999999},"min":{"1min":0.85499999999999998,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.7909999999999999,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.494,"15min":1.444},"min":{"1min":0.998,"5min":0.879,"15min":0.65400000000000003},"max":{"1min":1.7709999999999999,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":668752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1479999999999999,"15min":1.143},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.554,"5min":1.5740000000000001,"15min":2.1789999999999998},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.111,"15min":1.143},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.478,"5min":1.6299999999999999,"15min":1.8959999999999999},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2989999999999999,"15min":1.323},"min":{"1min":0.88900000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.546,"5min":1.645,"15min":2.1989999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.68300000000000005},"max":{"1min":1.508,"5min":1.7450000000000001,"15min":1.851},"last":1.552}]},{"osd":2,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.73499999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":2.2269999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.78500000000000003},"max":{"1min":1.583,"5min":1.6779999999999999,"15min":2.1150000000000002},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.202,"15min":1.2110000000000001},"min":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.671,"15min":1.6839999999999999},"last":1.212},{"interface":"front","average":{"1min":1.288,"5min":1.3009999999999999,"15min":1.304},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.77400000000000002},"max":{"1min":1.575,"5min":1.6639999999999999,"15min":2.258},"last":1.2709999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.202,"15min":1.21},"min":{"1min":0.79600000000000004,"5min":0.621,"15min":0.621},"max":{"1min":1.653,"5min":1.653,"15min":2.8250000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2130000000000001,"15min":1.1970000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.40999999999999998},"max":{"1min":1.421,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.3919999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2230000000000001,"15min":1.242},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.54900000000000004},"max":{"1min":1.6950000000000001,"5min":1.8759999999999999,"15min":2.3809999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":1.0860000000000001,"5min":0.84599999999999997,"15min":0.77000000000000002},"max":{"1min":1.845,"5min":1.845,"15min":2.4329999999999998},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.69399999999999995},"max":{"1min":1.5880000000000001,"5min":1.841,"15min":2.653},"last":1.645},{"interface":"front","average":{"1min":1.423,"5min":1.423,"15min":1.3919999999999999},"min":{"1min":0.96699999999999997,"5min":0.88500000000000001,"15min":0.70499999999999996},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.464},"last":1.536}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310848,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060992,"statfs":{"total":96636764160,"available":96318455808,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.61099999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0960000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":14.49},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.437},"max":{"1min":1.427,"5min":1.504,"15min":1.627},"last":19.800999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.95099999999999996},"min":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.343,"5min":1.4079999999999999,"15min":1.516},"last":6.6790000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0660000000000001,"15min":1.077},"min":{"1min":0.753,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.5489999999999999,"15min":1.6619999999999999},"last":11.513999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.093,"15min":1.121},"min":{"1min":0.95399999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4830000000000001,"5min":1.6830000000000001,"15min":1.887},"last":12.446},{"interface":"front","average":{"1min":1.228,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.68700000000000006,"5min":0.44,"15min":0.44},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.7030000000000001},"last":7.4550000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.1919999999999999,"15min":1.236},"min":{"1min":0.93300000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6890000000000001,"5min":1.768,"15min":2.6640000000000001},"last":10.460000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3029999999999999,"15min":1.3560000000000001},"min":{"1min":1.1379999999999999,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":2.0840000000000001},"last":6.7249999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3420000000000001,"15min":1.4019999999999999},"min":{"1min":0.92100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6080000000000001,"5min":1.883,"15min":2.1699999999999999},"last":6.8520000000000003},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.357,"15min":1.3979999999999999},"min":{"1min":1.0620000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.1179999999999999},"last":7.1180000000000003}]},{"osd":6,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.411,"15min":1.45},"min":{"1min":0.98599999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":1.8520000000000001,"5min":2.0510000000000002,"15min":2.2010000000000001},"last":11.362},{"interface":"front","average":{"1min":1.387,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.97499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.75,"5min":1.823,"15min":2.137},"last":10.515000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.4119999999999999,"15min":1.4419999999999999},"min":{"1min":1.163,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8540000000000001,"5min":1.8560000000000001,"15min":2.2240000000000002},"last":10.561},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4319999999999999,"15min":1.482},"min":{"1min":1.0860000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.645,"5min":2.004,"15min":2.266},"last":6.7629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575042,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310848,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060992,"statfs":{"total":96636764160,"available":96318455808,"internally_reserved":0,"allocated":1302528,"data_stored":668592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93899999999999995,"15min":0.95999999999999996},"min":{"1min":0.65400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1200000000000001,"5min":1.4670000000000001,"15min":1.8380000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.46400000000000002},"max":{"1min":1.383,"5min":1.415,"15min":1.9650000000000001},"last":1.127}]},{"osd":1,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.91500000000000004,"15min":0.92200000000000004},"min":{"1min":0.71799999999999997,"5min":0.57199999999999995,"15min":0.54900000000000004},"max":{"1min":1.3049999999999999,"5min":1.631,"15min":2.3940000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0229999999999999},"min":{"1min":0.68700000000000006,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.599,"5min":1.6779999999999999,"15min":2.5390000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1639999999999999,"15min":1.1890000000000001},"min":{"1min":0.90600000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.758,"5min":1.8,"15min":1.917},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.28,"15min":1.306},"min":{"1min":0.96499999999999997,"5min":0.62,"15min":0.60699999999999998},"max":{"1min":1.948,"5min":1.948,"15min":2.4550000000000001},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.89700000000000002,"5min":0.73199999999999998,"15min":0.61299999999999999},"max":{"1min":1.625,"5min":1.625,"15min":2.1899999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":1.0589999999999999,"5min":0.88500000000000001,"15min":0.84399999999999997},"max":{"1min":1.782,"5min":1.782,"15min":2.6920000000000002},"last":1.248}]},{"osd":5,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":1.171,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8480000000000001,"5min":2.0680000000000001,"15min":2.601},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.319,"15min":1.337},"min":{"1min":1.0089999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.6400000000000001},"last":1.288}]},{"osd":6,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4039999999999999,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.972,"5min":1.972,"15min":2.8340000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.427,"15min":1.444},"min":{"1min":1.1719999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.8320000000000001,"5min":1.9910000000000001,"15min":2.617},"last":1.147}]},{"osd":7,"last update":"Fri Oct 6 11:33:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.371,"15min":1.391},"min":{"1min":0.94299999999999995,"5min":0.85799999999999998,"15min":0.82699999999999996},"max":{"1min":1.7549999999999999,"5min":1.8049999999999999,"15min":2.9020000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.464,"15min":1.4690000000000001},"min":{"1min":1.2609999999999999,"5min":0.96799999999999997,"15min":0.85199999999999998},"max":{"1min":1.978,"5min":1.978,"15min":3.2240000000000002},"last":1.1719999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306756,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065084,"statfs":{"total":96636764160,"available":96322646016,"internally_reserved":0,"allocated":1306624,"data_stored":668662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.93300000000000005,"15min":0.93000000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.252,"5min":1.724,"15min":1.724},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95699999999999996,"15min":0.95799999999999996},"min":{"1min":0.58399999999999996,"5min":0.53600000000000003,"15min":0.51700000000000002},"max":{"1min":1.3109999999999999,"5min":1.4259999999999999,"15min":1.855},"last":1.288}]},{"osd":2,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.97199999999999998},"min":{"1min":0.67100000000000004,"5min":0.624,"15min":0.58399999999999996},"max":{"1min":1.1859999999999999,"5min":1.8540000000000001,"15min":1.9299999999999999},"last":1.208},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.1160000000000001,"15min":1.1000000000000001},"min":{"1min":0.66800000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.319,"5min":2.137,"15min":2.137},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.272,"15min":1.228},"min":{"1min":0.879,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6120000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"last":1.04},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.073,"15min":1.0980000000000001},"min":{"1min":0.72199999999999998,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":4.0830000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.238,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7350000000000001,"5min":2.2290000000000001,"15min":4.0099999999999998},"last":1.341},{"interface":"front","average":{"1min":1.419,"5min":1.387,"15min":1.397},"min":{"1min":1.056,"5min":0.88100000000000001,"15min":0.78000000000000003},"max":{"1min":1.788,"5min":1.788,"15min":3.5609999999999999},"last":1.3839999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3380000000000001,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.82599999999999996,"15min":0.78400000000000003},"max":{"1min":1.5720000000000001,"5min":1.9990000000000001,"15min":3.2559999999999998},"last":1.45},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.365,"15min":1.3500000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.85999999999999999},"max":{"1min":1.849,"5min":2.1240000000000001,"15min":2.766},"last":1.923}]},{"osd":6,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4890000000000001,"15min":1.4830000000000001},"min":{"1min":1.1299999999999999,"5min":0.96199999999999997,"15min":0.94299999999999995},"max":{"1min":1.8759999999999999,"5min":2.0310000000000001,"15min":3.0630000000000002},"last":1.48},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.411,"15min":1.407},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":2.8719999999999999,"15min":3.8500000000000001},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:33:46 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.492,"15min":1.5109999999999999},"min":{"1min":0.88,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8580000000000001,"5min":2.0390000000000001,"15min":3.9620000000000002},"last":1.573},{"interface":"front","average":{"1min":1.575,"5min":1.4650000000000001,"15min":1.466},"min":{"1min":1.133,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.085,"5min":2.085,"15min":2.8740000000000001},"last":1.7989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307356,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064484,"statfs":{"total":96636764160,"available":96322031616,"internally_reserved":0,"allocated":1921024,"data_stored":1261174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.85899999999999999,"15min":0.88300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2290000000000001,"5min":1.3560000000000001,"15min":1.571},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95499999999999996,"15min":0.96199999999999997},"min":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.395}]},{"osd":2,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2549999999999999,"5min":1.6699999999999999,"15min":2.1989999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.042,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.55800000000000005,"15min":0.52600000000000002},"max":{"1min":1.738,"5min":1.738,"15min":2.7719999999999998},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.069,"15min":1.0860000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.919},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.762,"5min":2.0830000000000002,"15min":2.3530000000000002},"last":0.877}]},{"osd":4,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3280000000000001,"15min":1.343},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.68999999999999995},"max":{"1min":1.696,"5min":2.294,"15min":2.294},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.276,"15min":1.2929999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.609,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.143}]},{"osd":5,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.393,"15min":1.383},"min":{"1min":1.0800000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8520000000000001,"5min":2.0859999999999999,"15min":2.573},"last":1.246},{"interface":"front","average":{"1min":1.373,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":1.024,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.843,"5min":1.881,"15min":2.8650000000000002},"last":1.218}]},{"osd":6,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.345,"15min":1.3420000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6160000000000001,"5min":2.2240000000000002,"15min":2.52},"last":1.264},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4410000000000001,"15min":1.4610000000000001},"min":{"1min":1.163,"5min":1.0229999999999999,"15min":0.98499999999999999},"max":{"1min":1.909,"5min":2.254,"15min":2.742},"last":1.284}]},{"osd":7,"last update":"Fri Oct 6 11:33:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4399999999999999,"15min":1.486},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7509999999999999,"5min":2.0219999999999998,"15min":4.1159999999999997},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4119999999999999,"15min":1.4319999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7090000000000001,"5min":1.9830000000000001,"15min":2.3980000000000001},"last":1.3480000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:34:38.408 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:34:38.408 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:34:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.615+0000 7f726c0f0700 1 Processor -- start 2023-10-06T11:34:38.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 -- start start 2023-10-06T11:34:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72640b23c0 0x7f72640b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640cae40 0x7f72640cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640cc510 con 0x7f72640ccb40 2023-10-06T11:34:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72640cb900 con 0x7f72640b23c0 2023-10-06T11:34:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.619+0000 7f726c0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72640cbbe0 con 0x7f72640cae40 2023-10-06T11:34:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72640b23c0 0x7f72640b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72640b23c0 0x7f72640b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:51428/0 (socket says 172.21.15.46:51428) 2023-10-06T11:34:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 -- 172.21.15.46:0/2413117414 learned_addr learned my addr 172.21.15.46:0/2413117414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:38.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726968b700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640cae40 0x7f72640cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 -- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640cae40 msgr2=0x7f72640cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640cae40 0x7f72640cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 -- 172.21.15.46:0/2413117414 >> v1:172.21.15.132:6789/0 conn(0x7f72640ccb40 legacy=0x7f726415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 -- 172.21.15.46:0/2413117414 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640cbec0 con 0x7f72640b23c0 2023-10-06T11:34:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7269e8c700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640b23c0 0x7f72640b2790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7260009990 tx=0x7f7260009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574fcea893d3565d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7268e8a700 1 -- 172.21.15.46:0/2413117414 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260005c10 con 0x7f72640b23c0 2023-10-06T11:34:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7268e8a700 1 -- 172.21.15.46:0/2413117414 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260005d70 con 0x7f72640b23c0 2023-10-06T11:34:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f7268e8a700 1 -- 172.21.15.46:0/2413117414 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000b880 con 0x7f72640b23c0 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 -- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640b23c0 msgr2=0x7f72640b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640b23c0 0x7f72640b2790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7260009990 tx=0x7f7260009220 comp rx=0 tx=0).stop 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 -- 172.21.15.46:0/2413117414 shutdown_connections 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640cae40 0x7f72640cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 --2- 172.21.15.46:0/2413117414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640b23c0 0x7f72640b2790 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 -- 172.21.15.46:0/2413117414 >> 172.21.15.46:0/2413117414 conn(0x7f7264099480 msgr2=0x7f726406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.623+0000 7f726c0f0700 1 -- 172.21.15.46:0/2413117414 shutdown_connections 2023-10-06T11:34:38.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 -- 172.21.15.46:0/2413117414 wait complete. 2023-10-06T11:34:38.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 Processor -- start 2023-10-06T11:34:38.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 -- start start 2023-10-06T11:34:38.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 0x7f72640c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640ccb40 0x7f72640c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640b5460 con 0x7f72640b23c0 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72640c5670 con 0x7f72640cae40 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.627+0000 7f726c0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72640c5950 con 0x7f72640ccb40 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38762/0 (socket says 172.21.15.46:38762) 2023-10-06T11:34:38.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 -- 172.21.15.46:0/1644619326 learned_addr learned my addr 172.21.15.46:0/1644619326 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f726968b700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 0x7f72640c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f726a68d700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640ccb40 0x7f72640c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 -- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640ccb40 msgr2=0x7f72640c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640ccb40 0x7f72640c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 -- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 msgr2=0x7f72640c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 0x7f72640c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 -- 172.21.15.46:0/1644619326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260009480 con 0x7f72640b23c0 2023-10-06T11:34:38.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f726968b700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 0x7f72640c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:34:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7269e8c700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7264064ad0 tx=0x7f7260016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600115a0 con 0x7f72640b23c0 2023-10-06T11:34:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260020070 con 0x7f72640b23c0 2023-10-06T11:34:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72640c5c30 con 0x7f72640b23c0 2023-10-06T11:34:38.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011950 con 0x7f72640b23c0 2023-10-06T11:34:38.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.631+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72640c94d0 con 0x7f72640b23c0 2023-10-06T11:34:38.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.635+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726416cf40 con 0x7f72640b23c0 2023-10-06T11:34:38.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.635+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7260022030 con 0x7f72640b23c0 2023-10-06T11:34:38.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.635+0000 7f7259ffb700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 0x7f7248062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:38.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.635+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f726007e820 con 0x7f72640b23c0 2023-10-06T11:34:38.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.639+0000 7f726968b700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 0x7f7248062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:38.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.639+0000 7f726968b700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 0x7f7248062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f726409e040 tx=0x7f7254008040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:38.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.647+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f726416cf40 con 0x7f72640b23c0 2023-10-06T11:34:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.875+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f726416cf40 con 0x7f72640b23c0 2023-10-06T11:34:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.883+0000 7f7259ffb700 1 -- 172.21.15.46:0/1644619326 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f726416cf40 con 0x7f72640b23c0 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.887+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 msgr2=0x7f7248062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 0x7f7248062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f726409e040 tx=0x7f7254008040 comp rx=0 tx=0).stop 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 msgr2=0x7f72640b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7264064ad0 tx=0x7f7260016000 comp rx=0 tx=0).stop 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 shutdown_connections 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7248060670 0x7f7248062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72640ccb40 0x7f72640c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72640cae40 0x7f72640c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 --2- 172.21.15.46:0/1644619326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72640b23c0 0x7f72640b57a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 >> 172.21.15.46:0/1644619326 conn(0x7f7264099480 msgr2=0x7f7264067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 shutdown_connections 2023-10-06T11:34:38.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:38.891+0000 7f726c0f0700 1 -- 172.21.15.46:0/1644619326 wait complete. 2023-10-06T11:34:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:34:38.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:34:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.071+0000 7f7824a2e700 1 Processor -- start 2023-10-06T11:34:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.071+0000 7f7824a2e700 1 -- start start 2023-10-06T11:34:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f7824a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f782015ab60 0x7f782006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f7824a2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782006d460 con 0x7f78200ad6c0 2023-10-06T11:34:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f7824a2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78200c7000 con 0x7f782015af30 2023-10-06T11:34:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f7824a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200c72c0 con 0x7f782015ab60 2023-10-06T11:34:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781ed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78200ad6c0 0x7f78200adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50898/0 (socket says 172.21.15.46:50898) 2023-10-06T11:34:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781ed9d700 1 -- 172.21.15.46:0/657549897 learned_addr learned my addr 172.21.15.46:0/657549897 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781e59c700 1 --2- 172.21.15.46:0/657549897 >> v2:172.21.15.46:3300/0 conn(0x7f782015ab60 0x7f782006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151605211 0 0) 0x7f782006d460 con 0x7f78200ad6c0 2023-10-06T11:34:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7808003570 con 0x7f78200ad6c0 2023-10-06T11:34:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699035603 0 0) 0x7f78200c7000 con 0x7f782015af30 2023-10-06T11:34:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782006d460 con 0x7f782015af30 2023-10-06T11:34:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781e59c700 1 -- 172.21.15.46:0/657549897 >> v1:172.21.15.132:6789/0 conn(0x7f782015af30 legacy=0x7f782006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781e59c700 1 -- 172.21.15.46:0/657549897 >> v1:172.21.15.46:6789/0 conn(0x7f78200ad6c0 legacy=0x7f78200adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781e59c700 1 -- 172.21.15.46:0/657549897 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200c75a0 con 0x7f782015ab60 2023-10-06T11:34:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781e59c700 1 --2- 172.21.15.46:0/657549897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f782015ab60 0x7f782006e8a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f78200c71f0 tx=0x7f7810009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5867e32e3749d839 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.075+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000e070 con 0x7f782015ab60 2023-10-06T11:34:39.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781000bab0 con 0x7f782015ab60 2023-10-06T11:34:39.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f781d59a700 1 -- 172.21.15.46:0/657549897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000be70 con 0x7f782015ab60 2023-10-06T11:34:39.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 -- 172.21.15.46:0/657549897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f782015ab60 msgr2=0x7f782006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 --2- 172.21.15.46:0/657549897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f782015ab60 0x7f782006e8a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f78200c71f0 tx=0x7f7810009780 comp rx=0 tx=0).stop 2023-10-06T11:34:39.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 -- 172.21.15.46:0/657549897 shutdown_connections 2023-10-06T11:34:39.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 --2- 172.21.15.46:0/657549897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f782015ab60 0x7f782006e8a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 -- 172.21.15.46:0/657549897 >> 172.21.15.46:0/657549897 conn(0x7f78200cbe30 msgr2=0x7f78200cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 -- 172.21.15.46:0/657549897 shutdown_connections 2023-10-06T11:34:39.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.079+0000 7f7824a2e700 1 -- 172.21.15.46:0/657549897 wait complete. 2023-10-06T11:34:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 Processor -- start 2023-10-06T11:34:39.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 -- start start 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f782015ab60 0x7f78200b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f782015af30 0x7f78200c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c4de0 con 0x7f78200ad6c0 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200afef0 con 0x7f782015af30 2023-10-06T11:34:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f7824a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b18f0 con 0x7f782015ab60 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38788/0 (socket says 172.21.15.46:38788) 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 -- 172.21.15.46:0/56673362 learned_addr learned my addr 172.21.15.46:0/56673362 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781ed9d700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f782015af30 0x7f78200c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781dd9b700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f782015ab60 0x7f78200b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 -- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f782015ab60 msgr2=0x7f78200b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f782015ab60 0x7f78200b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 -- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f782015af30 msgr2=0x7f78200c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f782015af30 0x7f78200c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 -- 172.21.15.46:0/56673362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7810009a20 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.083+0000 7f781e59c700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f78100099f0 tx=0x7f7810018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810001010 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781001e430 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78200c4790 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781001e7f0 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78200c9c50 con 0x7f78200ad6c0 2023-10-06T11:34:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7820063f90 con 0x7f78200ad6c0 2023-10-06T11:34:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.087+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f781001ebe0 con 0x7f78200ad6c0 2023-10-06T11:34:39.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.091+0000 7f780e7fc700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 0x7f7800062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.091+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f781007f3a0 con 0x7f78200ad6c0 2023-10-06T11:34:39.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.091+0000 7f781dd9b700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 0x7f7800062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.091+0000 7f781dd9b700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 0x7f7800062b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78200c1480 tx=0x7f7814009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.095+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f781004f3b0 con 0x7f78200ad6c0 2023-10-06T11:34:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.303+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f782004b5c0 con 0x7f78200ad6c0 2023-10-06T11:34:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.303+0000 7f780e7fc700 1 -- 172.21.15.46:0/56673362 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f78100529d0 con 0x7f78200ad6c0 2023-10-06T11:34:39.314 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:39.314 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 msgr2=0x7f7800062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 0x7f7800062b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78200c1480 tx=0x7f7814009040 comp rx=0 tx=0).stop 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 msgr2=0x7f78200cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f78100099f0 tx=0x7f7810018000 comp rx=0 tx=0).stop 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 shutdown_connections 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78000606a0 0x7f7800062b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f782015af30 0x7f78200c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f782015ab60 0x7f78200b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 --2- 172.21.15.46:0/56673362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78200ad6c0 0x7f78200cb6c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 >> 172.21.15.46:0/56673362 conn(0x7f78200cbe30 msgr2=0x7f78200c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 shutdown_connections 2023-10-06T11:34:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.307+0000 7f7824a2e700 1 -- 172.21.15.46:0/56673362 wait complete. 2023-10-06T11:34:39.320 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:34:39.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:34:33.064692+0000', 'modified': '2023-10-06T11:34:36.695988+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7147': {'gid': 7147, 'name': 'd', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6834/1726709153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 1726709153}, {'type': 'v1', 'addr': '172.21.15.132:6834', 'nonce': 1726709153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:34:39.739 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:34:39.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T11:34:39.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 Processor -- start 2023-10-06T11:34:39.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 -- start start 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38cc0c8650 0x7f38cc0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 0x7f38cc0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc15dd50 con 0x7f38cc158e70 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0ca5e0 con 0x7f38cc0c8650 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.903+0000 7f38d2af0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc15d530 con 0x7f38cc159240 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38cc0c8650 0x7f38cc0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38cc0c8650 0x7f38cc0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38800/0 (socket says 172.21.15.46:38800) 2023-10-06T11:34:39.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 -- 172.21.15.46:0/3384798132 learned_addr learned my addr 172.21.15.46:0/3384798132 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d108d700 1 --2- 172.21.15.46:0/3384798132 >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 0x7f38cc0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 -- 172.21.15.46:0/3384798132 >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 msgr2=0x7f38cc0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 --2- 172.21.15.46:0/3384798132 >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 0x7f38cc0cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 -- 172.21.15.46:0/3384798132 >> v1:172.21.15.132:6789/0 conn(0x7f38cc158e70 legacy=0x7f38cc15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 -- 172.21.15.46:0/3384798132 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc159670 con 0x7f38cc0c8650 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d108d700 1 --2- 172.21.15.46:0/3384798132 >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 0x7f38cc0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cbfff700 1 --2- 172.21.15.46:0/3384798132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc0c8650 0x7f38cc0b0da0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0b0af0 tx=0x7f38bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf0d6f887a406b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cb7fe700 1 -- 172.21.15.46:0/3384798132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00b860 con 0x7f38cc0c8650 2023-10-06T11:34:39.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cb7fe700 1 -- 172.21.15.46:0/3384798132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc00b9c0 con 0x7f38cc0c8650 2023-10-06T11:34:39.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38cb7fe700 1 -- 172.21.15.46:0/3384798132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00bd60 con 0x7f38cc0c8650 2023-10-06T11:34:39.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d2af0700 1 -- 172.21.15.46:0/3384798132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc0c8650 msgr2=0x7f38cc0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d2af0700 1 --2- 172.21.15.46:0/3384798132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc0c8650 0x7f38cc0b0da0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0b0af0 tx=0x7f38bc009250 comp rx=0 tx=0).stop 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d2af0700 1 -- 172.21.15.46:0/3384798132 shutdown_connections 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d2af0700 1 --2- 172.21.15.46:0/3384798132 >> v2:172.21.15.132:3300/0 conn(0x7f38cc159240 0x7f38cc0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.907+0000 7f38d2af0700 1 --2- 172.21.15.46:0/3384798132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc0c8650 0x7f38cc0b0da0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.911+0000 7f38d2af0700 1 -- 172.21.15.46:0/3384798132 >> 172.21.15.46:0/3384798132 conn(0x7f38cc062d50 msgr2=0x7f38cc063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.911+0000 7f38d2af0700 1 -- 172.21.15.46:0/3384798132 shutdown_connections 2023-10-06T11:34:39.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.911+0000 7f38d2af0700 1 -- 172.21.15.46:0/3384798132 wait complete. 2023-10-06T11:34:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.911+0000 7f38d2af0700 1 Processor -- start 2023-10-06T11:34:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 -- start start 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38cc0c8650 0x7f38cc16b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38cc158e70 0x7f38cc0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc155dd0 con 0x7f38cc159240 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0c6420 con 0x7f38cc0c8650 2023-10-06T11:34:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d2af0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0c5810 con 0x7f38cc158e70 2023-10-06T11:34:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38808/0 (socket says 172.21.15.46:38808) 2023-10-06T11:34:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 -- 172.21.15.46:0/107726435 learned_addr learned my addr 172.21.15.46:0/107726435 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38cbfff700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38cc158e70 0x7f38cc0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d088c700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38cc0c8650 0x7f38cc16b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 -- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38cc158e70 msgr2=0x7f38cc0c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38cc158e70 0x7f38cc0c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 -- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38cc0c8650 msgr2=0x7f38cc16b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38cc0c8650 0x7f38cc16b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 -- 172.21.15.46:0/107726435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0094f0 con 0x7f38cc159240 2023-10-06T11:34:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.915+0000 7f38d108d700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f38c400d180 tx=0x7f38c4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.919+0000 7f38c8ff9700 1 -- 172.21.15.46:0/107726435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400dcc0 con 0x7f38cc159240 2023-10-06T11:34:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.919+0000 7f38c8ff9700 1 -- 172.21.15.46:0/107726435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c400f040 con 0x7f38cc159240 2023-10-06T11:34:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.919+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc0c5af0 con 0x7f38cc159240 2023-10-06T11:34:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.919+0000 7f38c8ff9700 1 -- 172.21.15.46:0/107726435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4015690 con 0x7f38cc159240 2023-10-06T11:34:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.919+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc0c60b0 con 0x7f38cc159240 2023-10-06T11:34:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38c8ff9700 1 -- 172.21.15.46:0/107726435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f38c401e3b0 con 0x7f38cc159240 2023-10-06T11:34:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38c8ff9700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 0x7f38b8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38c8ff9700 1 -- 172.21.15.46:0/107726435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f38c407d2a0 con 0x7f38cc159240 2023-10-06T11:34:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38d088c700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 0x7f38b8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38d4dc8740 1 Processor -- start 2023-10-06T11:34:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38d088c700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 0x7f38b8062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38c0003a80 tx=0x7f38c0007ec0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.923+0000 7f38d4dc8740 1 -- start start 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0ae0 0xf4d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xd1b790 0xf4e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7bf30 con 0xcd0710 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0xe77ed0 con 0xd1b790 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0xe3e0b0 con 0xcd0ae0 2023-10-06T11:34:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38822/0 (socket says 172.21.15.46:38822) 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 -- 172.21.15.46:0/3903648141 learned_addr learned my addr 172.21.15.46:0/3903648141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d088c700 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0ae0 0xf4d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d108d700 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xd1b790 0xf4e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 -- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0ae0 msgr2=0xf4d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0ae0 0xf4d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 -- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xd1b790 msgr2=0xf4e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xd1b790 0xf4e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 -- 172.21.15.46:0/3903648141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd1c9e0 con 0xcd0710 2023-10-06T11:34:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38cbfff700 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3b0b1193f6108 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38b1ffb700 1 -- 172.21.15.46:0/3903648141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc020070 con 0xcd0710 2023-10-06T11:34:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38b1ffb700 1 -- 172.21.15.46:0/3903648141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc000c90 con 0xcd0710 2023-10-06T11:34:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38b1ffb700 1 -- 172.21.15.46:0/3903648141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00bb30 con 0xcd0710 2023-10-06T11:34:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- 172.21.15.46:0/3903648141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 msgr2=0xf4db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc017000 comp rx=0 tx=0).stop 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- 172.21.15.46:0/3903648141 shutdown_connections 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xd1b790 0xf4e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xf4db70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/3903648141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0ae0 0xf4d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- 172.21.15.46:0/3903648141 >> 172.21.15.46:0/3903648141 conn(0xcd0eb0 msgr2=0xf0a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- 172.21.15.46:0/3903648141 shutdown_connections 2023-10-06T11:34:39.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.927+0000 7f38d4dc8740 1 -- 172.21.15.46:0/3903648141 wait complete. 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 -- start start 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 Processor -- start 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 0xf51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0xe3dd50 con 0xcd0ae0 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0xe16970 con 0xcd0710 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0xe40920 con 0xd1b790 2023-10-06T11:34:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d4dc8740 10 client.0 _ll_register_callbacks cb 0xe1ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38836/0 (socket says 172.21.15.46:38836) 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 -- 172.21.15.46:0/583052383 learned_addr learned my addr 172.21.15.46:0/583052383 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d108d700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d088c700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 0xf51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 -- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 msgr2=0xf07c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf07c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 -- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 msgr2=0xf51c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 0xf51c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38cbfff700 1 -- 172.21.15.46:0/583052383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc159670 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.931+0000 7f38d088c700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 0xf51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38cbfff700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc01d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32526cb5397503e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc020040 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe16cc0 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f38bc011840 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38bc00bb30 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 10 client.7458 fetch_fsmap learned FSMap version 50 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0xe46370 con 0xcd0ae0 2023-10-06T11:34:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe16cc0 con 0xcd0ae0 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f38bc011c10 con 0xcd0ae0 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 10 client.7458 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 10 client.7458 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 20 client.7458 populate_metadata read hostname 'smithi046' 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf09300 0xf5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 4 client.7458 mds_command: new command op to 7147 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 --> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xe1a3e0 con 0xf09300 2023-10-06T11:34:39.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d088c700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf09300 0xf5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38aaffd700 10 client.7458 ms_handle_connect on v2:172.21.15.132:6832/1726709153 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.935+0000 7f38d088c700 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf09300 0xf5cde0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5193c9c534f97c38 server_cookie=e3d28d44992a9f20 in_seq=0 out_seq=0 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38aaffd700 1 -- 172.21.15.46:0/583052383 <== mds.0 v2:172.21.15.132:6832/1726709153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0xe1a3e0 con 0xf09300 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38aaffd700 10 client.7458 handle_command_reply: tid=0 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 client.7458 shutdown 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 msgr2=0xf52180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc01d050 comp rx=0 tx=0).stop 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 shutdown_connections 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf07c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0ae0 0xf52180 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf09300 0xf5cde0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/583052383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0710 0xf51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 >> 172.21.15.46:0/583052383 conn(0xcd0eb0 msgr2=0xf311e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38aaffd700 0 client.7458 ms_handle_reset on v2:172.21.15.132:6832/1726709153 2023-10-06T11:34:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 shutdown_connections 2023-10-06T11:34:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 1 -- 172.21.15.46:0/583052383 wait complete. 2023-10-06T11:34:39.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.939+0000 7f38d4dc8740 20 client.7458 trim_cache size 0 max 16384 2023-10-06T11:34:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 Processor -- start 2023-10-06T11:34:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 -- start start 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0710 0xf02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xf2ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc0b1dc0 con 0xcd0eb0 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0xd4d650 con 0xcd0ae0 2023-10-06T11:34:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.959+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0xe4d550 con 0xcd0710 2023-10-06T11:34:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0710 0xf02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d108d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xf2ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38850/0 (socket says 172.21.15.46:38850) 2023-10-06T11:34:39.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 -- 172.21.15.46:0/4107811993 learned_addr learned my addr 172.21.15.46:0/4107811993 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 -- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0710 msgr2=0xf02bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0710 0xf02bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 -- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 msgr2=0xf2ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xf2ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 -- 172.21.15.46:0/4107811993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc16fb00 con 0xcd0eb0 2023-10-06T11:34:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d088c700 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0xdbb100 tx=0x7f38c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e76c1dd976598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38b27fc700 1 -- 172.21.15.46:0/4107811993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0002cf0 con 0xcd0eb0 2023-10-06T11:34:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38b27fc700 1 -- 172.21.15.46:0/4107811993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0002e50 con 0xcd0eb0 2023-10-06T11:34:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38b27fc700 1 -- 172.21.15.46:0/4107811993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0027870 con 0xcd0eb0 2023-10-06T11:34:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d4dc8740 1 -- 172.21.15.46:0/4107811993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 msgr2=0xf026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.963+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0xdbb100 tx=0x7f38c0014000 comp rx=0 tx=0).stop 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 -- 172.21.15.46:0/4107811993 shutdown_connections 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xf2ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xcd0710 0xf02bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/4107811993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0eb0 0xf026b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 -- 172.21.15.46:0/4107811993 >> 172.21.15.46:0/4107811993 conn(0xf0a510 msgr2=0xf311e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 -- 172.21.15.46:0/4107811993 shutdown_connections 2023-10-06T11:34:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 -- 172.21.15.46:0/4107811993 wait complete. 2023-10-06T11:34:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 -- start start 2023-10-06T11:34:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.967+0000 7f38d4dc8740 1 Processor -- start 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xfa7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf2be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7b5d0 con 0xcd0710 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0xe75890 con 0xcd0ae0 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0xe76d60 con 0xd1b790 2023-10-06T11:34:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 10 client.0 _ll_register_callbacks cb 0xe43110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:34:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38860/0 (socket says 172.21.15.46:38860) 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 -- 172.21.15.46:0/1707058834 learned_addr learned my addr 172.21.15.46:0/1707058834 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38cbfff700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xfa7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d108d700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf2be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 -- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 msgr2=0xf2be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf2be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 -- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 msgr2=0xfa7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xfa7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 -- 172.21.15.46:0/1707058834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c0019040 con 0xcd0710 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d088c700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f38c0003f10 tx=0x7f38c0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e7dcb7f0f9b559 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0023660 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38bc000c90 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f38c001c040 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c00239d0 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 10 client.7464 fetch_fsmap learned FSMap version 50 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0xe2cd80 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc000c90 con 0xcd0710 2023-10-06T11:34:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f38c0012070 con 0xcd0710 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 10 client.7464 fetch_fsmap finished waiting for FSMap version 50 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 10 client.7464 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 20 client.7464 populate_metadata read hostname 'smithi046' 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf2c7b0 0xf2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 4 client.7464 mds_command: new command op to 7147 tid=0[{"prefix": "status"}] 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.971+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 --> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] -- command(tid 0: {"prefix": "status"}) v1 -- 0xe4c410 con 0xf2c7b0 2023-10-06T11:34:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38cbfff700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf2c7b0 0xf2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38cbfff700 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf2c7b0 0xf2cb80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff2715bec178d13 server_cookie=72505da02b55a3 in_seq=0 out_seq=0 2023-10-06T11:34:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38aaffd700 10 client.7464 ms_handle_connect on v2:172.21.15.132:6832/1726709153 2023-10-06T11:34:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38aaffd700 1 -- 172.21.15.46:0/1707058834 <== mds.0 v2:172.21.15.132:6832/1726709153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0xe4c410 con 0xf2c7b0 2023-10-06T11:34:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38aaffd700 10 client.7464 handle_command_reply: tid=0 2023-10-06T11:34:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 client.7464 shutdown 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 msgr2=0xfa7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f38c0003f10 tx=0x7f38c0015000 comp rx=0 tx=0).stop 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 shutdown_connections 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xd1b790 0xf2be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:6832/1726709153,v1:172.21.15.132:6834/1726709153] conn(0xf2c7b0 0xf2cb80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xcd0ae0 0xfa7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 --2- 172.21.15.46:0/1707058834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xcd0710 0xfa7440 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38aaffd700 0 client.7464 ms_handle_reset on v2:172.21.15.132:6832/1726709153 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 >> 172.21.15.46:0/1707058834 conn(0xf0a510 msgr2=0xf58140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 shutdown_connections 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 1 -- 172.21.15.46:0/1707058834 wait complete. 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.975+0000 7f38d4dc8740 20 client.7464 trim_cache size 0 max 16384 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7147, 2023-10-06T11:34:39.988 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.3365022890000002, 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 50, 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 107, 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 107, 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 13.301457857999999 2023-10-06T11:34:39.989 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 msgr2=0x7f38b8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 0x7f38b8062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38c0003a80 tx=0x7f38c0007ec0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 msgr2=0x7f38cc16f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f38c400d180 tx=0x7f38c4009a50 comp rx=0 tx=0).stop 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 shutdown_connections 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f38b8060670 0x7f38b8062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38cc159240 0x7f38cc16f260 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38cc158e70 0x7f38cc0c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 --2- 172.21.15.46:0/107726435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38cc0c8650 0x7f38cc16b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 >> 172.21.15.46:0/107726435 conn(0x7f38cc062d50 msgr2=0x7f38cc156660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 shutdown_connections 2023-10-06T11:34:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:39.979+0000 7f38d2af0700 1 -- 172.21.15.46:0/107726435 wait complete. 2023-10-06T11:34:40.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:34:40.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.167+0000 7f74f0345700 1 Processor -- start 2023-10-06T11:34:40.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.167+0000 7f74f0345700 1 -- start start 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f74e8156160 0x7f74e80c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74e80c8f30 0x7f74e806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80c7fc0 con 0x7f74e80c8f30 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e80cba20 con 0x7f74e8156160 2023-10-06T11:34:40.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74f0345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80c9590 con 0x7f74e8155d90 2023-10-06T11:34:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74e80c8f30 0x7f74e806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f74e80c8f30 0x7f74e806dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38866/0 (socket says 172.21.15.46:38866) 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 -- 172.21.15.46:0/308778414 learned_addr learned my addr 172.21.15.46:0/308778414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee0e1700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ed8e0700 1 --2- 172.21.15.46:0/308778414 >> v2:172.21.15.132:3300/0 conn(0x7f74e8156160 0x7f74e80c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 -- 172.21.15.46:0/308778414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 msgr2=0x7f74e80c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 -- 172.21.15.46:0/308778414 >> v2:172.21.15.132:3300/0 conn(0x7f74e8156160 msgr2=0x7f74e80c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 --2- 172.21.15.46:0/308778414 >> v2:172.21.15.132:3300/0 conn(0x7f74e8156160 0x7f74e80c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 -- 172.21.15.46:0/308778414 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e80c7690 con 0x7f74e80c8f30 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee0e1700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:40.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ee8e2700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e806dd00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f74e80caa00 tx=0x7f74e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f008a323ce02b8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ed0df700 1 -- 172.21.15.46:0/308778414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e000e070 con 0x7f74e80c8f30 2023-10-06T11:34:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ed0df700 1 -- 172.21.15.46:0/308778414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e000ba20 con 0x7f74e80c8f30 2023-10-06T11:34:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.171+0000 7f74ed0df700 1 -- 172.21.15.46:0/308778414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e00092e0 con 0x7f74e80c8f30 2023-10-06T11:34:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 -- 172.21.15.46:0/308778414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 msgr2=0x7f74e806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e806dd00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f74e80caa00 tx=0x7f74e0009840 comp rx=0 tx=0).stop 2023-10-06T11:34:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 -- 172.21.15.46:0/308778414 shutdown_connections 2023-10-06T11:34:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 --2- 172.21.15.46:0/308778414 >> v2:172.21.15.132:3300/0 conn(0x7f74e8156160 0x7f74e80c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e806dd00 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 --2- 172.21.15.46:0/308778414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 -- 172.21.15.46:0/308778414 >> 172.21.15.46:0/308778414 conn(0x7f74e80cced0 msgr2=0x7f74e80cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 -- 172.21.15.46:0/308778414 shutdown_connections 2023-10-06T11:34:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.175+0000 7f74f0345700 1 -- 172.21.15.46:0/308778414 wait complete. 2023-10-06T11:34:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 Processor -- start 2023-10-06T11:34:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 -- start start 2023-10-06T11:34:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 0x7f74e80c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e814aff0 con 0x7f74e80c8f30 2023-10-06T11:34:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80ae1d0 con 0x7f74e80ab750 2023-10-06T11:34:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74f0345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80ad5c0 con 0x7f74e8155d90 2023-10-06T11:34:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38876/0 (socket says 172.21.15.46:38876) 2023-10-06T11:34:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 -- 172.21.15.46:0/2618458483 learned_addr learned my addr 172.21.15.46:0/2618458483 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ed8e0700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee8e2700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 0x7f74e80c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 -- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 msgr2=0x7f74e80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 -- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 msgr2=0x7f74e80c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 0x7f74e80c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee0e1700 1 -- 172.21.15.46:0/2618458483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0009ae0 con 0x7f74e80c8f30 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.179+0000 7f74ee8e2700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 0x7f74e80c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74ee0e1700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e8d0 tx=0x7f74e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4013410 con 0x7f74e80c8f30 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4013570 con 0x7f74e80c8f30 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4013910 con 0x7f74e80c8f30 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e80ad8a0 con 0x7f74e80c8f30 2023-10-06T11:34:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e80ade60 con 0x7f74e80c8f30 2023-10-06T11:34:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.183+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e806d220 con 0x7f74e80c8f30 2023-10-06T11:34:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.187+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f74e4013d00 con 0x7f74e80c8f30 2023-10-06T11:34:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.187+0000 7f74de7fc700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 0x7f74cc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.187+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f74e407cb50 con 0x7f74e80c8f30 2023-10-06T11:34:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.187+0000 7f74ed8e0700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 0x7f74cc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:40.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.191+0000 7f74ed8e0700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 0x7f74cc062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74e8171060 tx=0x7f74d80091a0 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.195+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74e404cb60 con 0x7f74e80c8f30 2023-10-06T11:34:40.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.403+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f74e804b5c0 con 0x7f74e80c8f30 2023-10-06T11:34:40.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.403+0000 7f74de7fc700 1 -- 172.21.15.46:0/2618458483 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f74e4020070 con 0x7f74e80c8f30 2023-10-06T11:34:40.415 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:34:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 msgr2=0x7f74cc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 0x7f74cc062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f74e8171060 tx=0x7f74d80091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 msgr2=0x7f74e814aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:40.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f74e400e8d0 tx=0x7f74e4008040 comp rx=0 tx=0).stop 2023-10-06T11:34:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 shutdown_connections 2023-10-06T11:34:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f74cc060670 0x7f74cc062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74e80ab750 0x7f74e80c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74e8155d90 0x7f74e80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 --2- 172.21.15.46:0/2618458483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74e80c8f30 0x7f74e814aab0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 >> 172.21.15.46:0/2618458483 conn(0x7f74e80cced0 msgr2=0x7f74e806ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 shutdown_connections 2023-10-06T11:34:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:40.407+0000 7f74f0345700 1 -- 172.21.15.46:0/2618458483 wait complete. 2023-10-06T11:34:40.437 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:34:40.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:34:40.438 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:40.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.438 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:34:40.456 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:34:40.456 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:34:40.457 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41450sec preferred_lft 41450sec 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.458 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:34:40.459 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.460 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:34:40.460 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:34:40.460 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:34:40.460 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:34:40.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.461 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:34:40.541 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:34:40.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.545 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:34:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:34:40.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:34:40.609 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:34:40.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.747 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:34:40.766 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:34:40.766 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:34:40.767 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:40.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:40.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:34:40.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:34:40.843 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:40.907 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:34:40.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:34:40.908 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:40.969 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:34:40.970 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:34:40.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:41.089 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:41.089 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:41.089 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:34:41.114 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:34:41.129 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:34:41.130 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:34:41.130 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:34:41.134 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:34:41.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:34:41.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:34:41.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:34:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 Processor -- start 2023-10-06T11:34:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 -- start start 2023-10-06T11:34:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac800c7b50 0x7fac800c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac800c0bd0 0x7fac800c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac800c8500 con 0x7fac80156c80 2023-10-06T11:34:41.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800c6470 con 0x7fac800c0bd0 2023-10-06T11:34:41.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac8682e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac800c7760 con 0x7fac800c7b50 2023-10-06T11:34:41.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac800c0bd0 0x7fac800c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac77fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac800c7b50 0x7fac800c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac800c0bd0 0x7fac800c1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38884/0 (socket says 172.21.15.46:38884) 2023-10-06T11:34:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 -- 172.21.15.46:0/559021482 learned_addr learned my addr 172.21.15.46:0/559021482 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 -- 172.21.15.46:0/559021482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac800c7b50 msgr2=0x7fac800c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 --2- 172.21.15.46:0/559021482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac800c7b50 0x7fac800c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 -- 172.21.15.46:0/559021482 >> v1:172.21.15.132:6789/0 conn(0x7fac80156c80 legacy=0x7fac800ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac84dcb700 1 -- 172.21.15.46:0/559021482 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800c66c0 con 0x7fac800c0bd0 2023-10-06T11:34:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.367+0000 7fac7f7fe700 1 -- 172.21.15.46:0/559021482 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059397978 0 0) 0x7fac800c8500 con 0x7fac80156c80 2023-10-06T11:34:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac84dcb700 1 --2- 172.21.15.46:0/559021482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c0bd0 0x7fac800c1050 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fac78009f20 tx=0x7fac78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40b625fdea4559f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac7f7fe700 1 -- 172.21.15.46:0/559021482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7800e070 con 0x7fac800c0bd0 2023-10-06T11:34:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac7f7fe700 1 -- 172.21.15.46:0/559021482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7800b9d0 con 0x7fac800c0bd0 2023-10-06T11:34:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac7f7fe700 1 -- 172.21.15.46:0/559021482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac780092e0 con 0x7fac800c0bd0 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 -- 172.21.15.46:0/559021482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c0bd0 msgr2=0x7fac800c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 --2- 172.21.15.46:0/559021482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c0bd0 0x7fac800c1050 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fac78009f20 tx=0x7fac78009840 comp rx=0 tx=0).stop 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 -- 172.21.15.46:0/559021482 shutdown_connections 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 --2- 172.21.15.46:0/559021482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c0bd0 0x7fac800c1050 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 --2- 172.21.15.46:0/559021482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac800c7b50 0x7fac800c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 -- 172.21.15.46:0/559021482 >> 172.21.15.46:0/559021482 conn(0x7fac800cbe30 msgr2=0x7fac800cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 -- 172.21.15.46:0/559021482 shutdown_connections 2023-10-06T11:34:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.371+0000 7fac8682e700 1 -- 172.21.15.46:0/559021482 wait complete. 2023-10-06T11:34:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 Processor -- start 2023-10-06T11:34:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 -- start start 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac800c0bd0 0x7fac800ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac80156c80 0x7fac800b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800ac5f0 con 0x7fac800c7b50 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800ac870 con 0x7fac800c0bd0 2023-10-06T11:34:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac8682e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac800acb50 con 0x7fac80156c80 2023-10-06T11:34:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38892/0 (socket says 172.21.15.46:38892) 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac77fff700 1 -- 172.21.15.46:0/1188980547 learned_addr learned my addr 172.21.15.46:0/1188980547 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac84dcb700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac80156c80 0x7fac800b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac7ffff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac800c0bd0 0x7fac800ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac77fff700 1 -- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac80156c80 msgr2=0x7fac800b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.375+0000 7fac77fff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac80156c80 0x7fac800b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac77fff700 1 -- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac800c0bd0 msgr2=0x7fac800ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac77fff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac800c0bd0 0x7fac800ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac77fff700 1 -- 172.21.15.46:0/1188980547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78009ae0 con 0x7fac800c7b50 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac77fff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fac6800ec80 tx=0x7fac680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800c850 con 0x7fac800c7b50 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6800c9b0 con 0x7fac800c7b50 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac800ace30 con 0x7fac800c7b50 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800cd50 con 0x7fac800c7b50 2023-10-06T11:34:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac800c3c40 con 0x7fac800c7b50 2023-10-06T11:34:41.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.379+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac80063f90 con 0x7fac800c7b50 2023-10-06T11:34:41.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.383+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fac68013680 con 0x7fac800c7b50 2023-10-06T11:34:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.383+0000 7fac7cff9700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 0x7fac60062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.383+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fac6807d290 con 0x7fac800c7b50 2023-10-06T11:34:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.383+0000 7fac7ffff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 0x7fac60062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.387+0000 7fac7ffff700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 0x7fac60062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac70007fd0 tx=0x7fac70007d10 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.391+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fac6804d3b0 con 0x7fac800c7b50 2023-10-06T11:34:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.595+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fac8004b5c0 con 0x7fac60060670 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.595+0000 7fac7cff9700 1 -- 172.21.15.46:0/1188980547 <== mgr.6955 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fac8004b5c0 con 0x7fac60060670 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:34:41.607 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 msgr2=0x7fac60062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 0x7fac60062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac70007fd0 tx=0x7fac70007d10 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 msgr2=0x7fac800ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fac6800ec80 tx=0x7fac680091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 shutdown_connections 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fac60060670 0x7fac60062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac80156c80 0x7fac800b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac800c7b50 0x7fac800ac0b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 --2- 172.21.15.46:0/1188980547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac800c0bd0 0x7fac800ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 >> 172.21.15.46:0/1188980547 conn(0x7fac800cbe30 msgr2=0x7fac800c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 shutdown_connections 2023-10-06T11:34:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.599+0000 7fac8682e700 1 -- 172.21.15.46:0/1188980547 wait complete. 2023-10-06T11:34:41.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:34:41.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.783+0000 7f86aca4c700 1 Processor -- start 2023-10-06T11:34:41.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.783+0000 7f86aca4c700 1 -- start start 2023-10-06T11:34:41.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86aca4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a80c7d40 0x7f86a806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86aca4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86a806cf30 0x7f86a80c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86aca4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a80c1d90 con 0x7f86a80c97e0 2023-10-06T11:34:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86aca4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a80c8190 con 0x7f86a806cf30 2023-10-06T11:34:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86aca4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80c1460 con 0x7f86a80c7d40 2023-10-06T11:34:41.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a77fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f86a80c97e0 0x7f86a80c6250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50914/0 (socket says 172.21.15.46:50914) 2023-10-06T11:34:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a77fe700 1 -- 172.21.15.46:0/157795292 learned_addr learned my addr 172.21.15.46:0/157795292 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 --2- 172.21.15.46:0/157795292 >> v2:172.21.15.46:3300/0 conn(0x7f86a806cf30 0x7f86a80c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f869ffff700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a80c7d40 0x7f86a806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a6ffd700 1 -- 172.21.15.46:0/157795292 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280700489 0 0) 0x7f86a80c1d90 con 0x7f86a80c97e0 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a6ffd700 1 -- 172.21.15.46:0/157795292 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8694003570 con 0x7f86a80c97e0 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 -- 172.21.15.46:0/157795292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a80c7d40 msgr2=0x7f86a806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a80c7d40 0x7f86a806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 -- 172.21.15.46:0/157795292 >> v1:172.21.15.46:6789/0 conn(0x7f86a80c97e0 legacy=0x7f86a80c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 -- 172.21.15.46:0/157795292 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a80c1740 con 0x7f86a806cf30 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a7fff700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a806cf30 0x7f86a80c2220 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009e60 tx=0x7f86a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9739563a114b51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a6ffd700 1 -- 172.21.15.46:0/157795292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a000e070 con 0x7f86a806cf30 2023-10-06T11:34:41.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a6ffd700 1 -- 172.21.15.46:0/157795292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a000bab0 con 0x7f86a806cf30 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.787+0000 7f86a6ffd700 1 -- 172.21.15.46:0/157795292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a000be70 con 0x7f86a806cf30 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- 172.21.15.46:0/157795292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a806cf30 msgr2=0x7f86a80c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a806cf30 0x7f86a80c2220 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009e60 tx=0x7f86a0009780 comp rx=0 tx=0).stop 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- 172.21.15.46:0/157795292 shutdown_connections 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a806cf30 0x7f86a80c2220 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 --2- 172.21.15.46:0/157795292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a80c7d40 0x7f86a806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- 172.21.15.46:0/157795292 >> 172.21.15.46:0/157795292 conn(0x7f86a80cbc10 msgr2=0x7f86a80cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- 172.21.15.46:0/157795292 shutdown_connections 2023-10-06T11:34:41.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- 172.21.15.46:0/157795292 wait complete. 2023-10-06T11:34:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 Processor -- start 2023-10-06T11:34:41.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.791+0000 7f86aca4c700 1 -- start start 2023-10-06T11:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a806cf30 0x7f86a80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86a80c97e0 0x7f86a816d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80b0fb0 con 0x7f86a80c7d40 2023-10-06T11:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80af090 con 0x7f86a80c97e0 2023-10-06T11:34:41.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a80b11a0 con 0x7f86a806cf30 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38914/0 (socket says 172.21.15.46:38914) 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 -- 172.21.15.46:0/2430200691 learned_addr learned my addr 172.21.15.46:0/2430200691 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86a7fff700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86a80c97e0 0x7f86a816d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86a77fe700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a806cf30 0x7f86a80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 -- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a806cf30 msgr2=0x7f86a80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a806cf30 0x7f86a80ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 -- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86a80c97e0 msgr2=0x7f86a816d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86a80c97e0 0x7f86a816d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 -- 172.21.15.46:0/2430200691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0009a20 con 0x7f86a80c7d40 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869ffff700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f869000ec80 tx=0x7f86900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000c9d0 con 0x7f86a80c7d40 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869000cb30 con 0x7f86a80c7d40 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a80b1480 con 0x7f86a80c7d40 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690013470 con 0x7f86a80c7d40 2023-10-06T11:34:41.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.795+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a816dda0 con 0x7f86a80c7d40 2023-10-06T11:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.799+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a816e7f0 con 0x7f86a80c7d40 2023-10-06T11:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.799+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 43) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8690013820 con 0x7f86a80c7d40 2023-10-06T11:34:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.799+0000 7f869f7fe700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 0x7f8688062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:41.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.799+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f869007cfa0 con 0x7f86a80c7d40 2023-10-06T11:34:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.799+0000 7f86a77fe700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 0x7f8688062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.803+0000 7f86a77fe700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 0x7f8688062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86980058d0 tx=0x7f8698009040 comp rx=0 tx=0).ready entity=mgr.6955 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:41.807+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f869004d0c0 con 0x7f86a80c7d40 2023-10-06T11:34:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.015+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f86a8064590 con 0x7f86a80c7d40 2023-10-06T11:34:42.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.923+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 44) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f8690054d50 con 0x7f86a80c7d40 2023-10-06T11:34:42.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.923+0000 7f869f7fe700 1 -- 172.21.15.46:0/2430200691 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v44) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8690050b50 con 0x7f86a80c7d40 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 msgr2=0x7f8688062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 0x7f8688062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86980058d0 tx=0x7f8698009040 comp rx=0 tx=0).stop 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 msgr2=0x7f86a8159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f869000ec80 tx=0x7f86900091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 shutdown_connections 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f86880606a0 0x7f8688062b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86a80c97e0 0x7f86a816d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a80c7d40 0x7f86a8159190 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 --2- 172.21.15.46:0/2430200691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86a806cf30 0x7f86a80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 >> 172.21.15.46:0/2430200691 conn(0x7f86a80cbc10 msgr2=0x7f86a8068380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 shutdown_connections 2023-10-06T11:34:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:42.927+0000 7f86aca4c700 1 -- 172.21.15.46:0/2430200691 wait complete. 2023-10-06T11:34:42.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:34:43.115 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:43.112+0000 7ffae19c5000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.115+0000 7fda906f0700 1 Processor -- start 2023-10-06T11:34:43.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.115+0000 7fda906f0700 1 -- start start 2023-10-06T11:34:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda906f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda880b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda906f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda880ab390 con 0x7fda880af270 2023-10-06T11:34:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda906f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda880ae710 con 0x7fda8815a8a0 2023-10-06T11:34:43.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda906f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880b0190 con 0x7fda8815a4d0 2023-10-06T11:34:43.130 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:43.119+0000 7f3a825ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:34:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8ec8d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda880af270 0x7fda880ad9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55952/0 (socket says 172.21.15.46:55952) 2023-10-06T11:34:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8ec8d700 1 -- 172.21.15.46:0/336857968 learned_addr learned my addr 172.21.15.46:0/336857968 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:43.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8e48c700 1 --2- 172.21.15.46:0/336857968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda880b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884944191 0 0) 0x7fda880ae710 con 0x7fda8815a8a0 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda74003570 con 0x7fda8815a8a0 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258628666 0 0) 0x7fda880ab390 con 0x7fda880af270 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda880ae710 con 0x7fda880af270 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.119+0000 7fda8e48c700 1 -- 172.21.15.46:0/336857968 >> v1:172.21.15.132:6789/0 conn(0x7fda8815a8a0 legacy=0x7fda880aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8e48c700 1 -- 172.21.15.46:0/336857968 >> v1:172.21.15.46:6789/0 conn(0x7fda880af270 legacy=0x7fda880ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8e48c700 1 -- 172.21.15.46:0/336857968 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda880aad40 con 0x7fda8815a4d0 2023-10-06T11:34:43.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8e48c700 1 --2- 172.21.15.46:0/336857968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda880b10e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda84009910 tx=0x7fda840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5219cf597092172f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84011630 con 0x7fda8815a4d0 2023-10-06T11:34:43.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84011790 con 0x7fda8815a4d0 2023-10-06T11:34:43.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda8d48a700 1 -- 172.21.15.46:0/336857968 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84011b30 con 0x7fda8815a4d0 2023-10-06T11:34:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 -- 172.21.15.46:0/336857968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 msgr2=0x7fda880b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 --2- 172.21.15.46:0/336857968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda880b10e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda84009910 tx=0x7fda840091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 -- 172.21.15.46:0/336857968 shutdown_connections 2023-10-06T11:34:43.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 --2- 172.21.15.46:0/336857968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda880b10e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 -- 172.21.15.46:0/336857968 >> 172.21.15.46:0/336857968 conn(0x7fda88097330 msgr2=0x7fda88067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 -- 172.21.15.46:0/336857968 shutdown_connections 2023-10-06T11:34:43.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.123+0000 7fda906f0700 1 -- 172.21.15.46:0/336857968 wait complete. 2023-10-06T11:34:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 Processor -- start 2023-10-06T11:34:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 -- start start 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda88146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8815a8a0 0x7fda881675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880b1620 con 0x7fda880af270 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880ae9f0 con 0x7fda8815a8a0 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.127+0000 7fda906f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda88155560 con 0x7fda8815a4d0 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8dc8b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda88146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38246/0 (socket says 172.21.15.46:38246) 2023-10-06T11:34:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 -- 172.21.15.46:0/4007482983 learned_addr learned my addr 172.21.15.46:0/4007482983 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8ec8d700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8815a8a0 0x7fda881675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 -- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 msgr2=0x7fda88146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda88146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 -- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8815a8a0 msgr2=0x7fda881675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8815a8a0 0x7fda881675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 -- 172.21.15.46:0/4007482983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda84009420 con 0x7fda880af270 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda8e48c700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fda840091e0 tx=0x7fda84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8401f070 con 0x7fda880af270 2023-10-06T11:34:43.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda84000bc0 con 0x7fda880af270 2023-10-06T11:34:43.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84011b30 con 0x7fda880af270 2023-10-06T11:34:43.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda881594d0 con 0x7fda880af270 2023-10-06T11:34:43.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88158ba0 con 0x7fda880af270 2023-10-06T11:34:43.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.131+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8815ba50 con 0x7fda880af270 2023-10-06T11:34:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.135+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 44) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fda84023070 con 0x7fda880af270 2023-10-06T11:34:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.135+0000 7fda7e7fc700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 0x7fda6c062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.135+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fda8407dfe0 con 0x7fda880af270 2023-10-06T11:34:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.135+0000 7fda8dc8b700 1 -- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 msgr2=0x7fda6c062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.135+0000 7fda8dc8b700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 0x7fda6c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.143+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fda8815ba50 con 0x7fda880af270 2023-10-06T11:34:43.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.315+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fda8815ba50 con 0x7fda880af270 2023-10-06T11:34:43.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.323+0000 7fda7e7fc700 1 -- 172.21.15.46:0/4007482983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fda8815ba50 con 0x7fda880af270 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 msgr2=0x7fda6c062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 0x7fda6c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 msgr2=0x7fda880c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fda840091e0 tx=0x7fda84017000 comp rx=0 tx=0).stop 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 shutdown_connections 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fda6c0606c0 0x7fda6c062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda8815a8a0 0x7fda881675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda8815a4d0 0x7fda88146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 --2- 172.21.15.46:0/4007482983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda880af270 0x7fda880c2e30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 >> 172.21.15.46:0/4007482983 conn(0x7fda88097330 msgr2=0x7fda880cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:43.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 shutdown_connections 2023-10-06T11:34:43.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.327+0000 7fda906f0700 1 -- 172.21.15.46:0/4007482983 wait complete. 2023-10-06T11:34:43.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:34:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 Processor -- start 2023-10-06T11:34:43.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 -- start start 2023-10-06T11:34:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fee780c1a60 0x7fee78155350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7806de80 con 0x7fee78155890 2023-10-06T11:34:43.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780c6d90 con 0x7fee780c1690 2023-10-06T11:34:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.515+0000 7fee7f185700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780c67a0 con 0x7fee780c1a60 2023-10-06T11:34:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee7d722700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fee78155890 0x7fee780c9a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55958/0 (socket says 172.21.15.46:55958) 2023-10-06T11:34:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee7d722700 1 -- 172.21.15.46:0/2152116961 learned_addr learned my addr 172.21.15.46:0/2152116961 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:43.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee77fff700 1 --2- 172.21.15.46:0/2152116961 >> v2:172.21.15.132:3300/0 conn(0x7fee780c1a60 0x7fee78155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178818779 0 0) 0x7fee7806de80 con 0x7fee78155890 2023-10-06T11:34:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee68003570 con 0x7fee78155890 2023-10-06T11:34:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850138599 0 0) 0x7fee780c6d90 con 0x7fee780c1690 2023-10-06T11:34:43.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7806de80 con 0x7fee780c1690 2023-10-06T11:34:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee77fff700 1 -- 172.21.15.46:0/2152116961 >> v1:172.21.15.132:6789/0 conn(0x7fee780c1690 legacy=0x7fee780c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375278989 0 0) 0x7fee68003570 con 0x7fee78155890 2023-10-06T11:34:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee77fff700 1 -- 172.21.15.46:0/2152116961 >> v1:172.21.15.46:6789/0 conn(0x7fee78155890 legacy=0x7fee780c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee77fff700 1 -- 172.21.15.46:0/2152116961 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780c7070 con 0x7fee780c1a60 2023-10-06T11:34:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee77fff700 1 --2- 172.21.15.46:0/2152116961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1a60 0x7fee78155350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c955bd5492ae56ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700017f0 con 0x7fee780c1a60 2023-10-06T11:34:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64011420 con 0x7fee780c1a60 2023-10-06T11:34:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.519+0000 7fee777fe700 1 -- 172.21.15.46:0/2152116961 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee640117c0 con 0x7fee780c1a60 2023-10-06T11:34:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- 172.21.15.46:0/2152116961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1a60 msgr2=0x7fee78155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 --2- 172.21.15.46:0/2152116961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1a60 0x7fee78155350 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fee640098e0 tx=0x7fee640091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- 172.21.15.46:0/2152116961 shutdown_connections 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 --2- 172.21.15.46:0/2152116961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1a60 0x7fee78155350 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- 172.21.15.46:0/2152116961 >> 172.21.15.46:0/2152116961 conn(0x7fee780cbe30 msgr2=0x7fee780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- 172.21.15.46:0/2152116961 shutdown_connections 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- 172.21.15.46:0/2152116961 wait complete. 2023-10-06T11:34:43.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 Processor -- start 2023-10-06T11:34:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.523+0000 7fee7f185700 1 -- start start 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1690 0x7fee781505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee78155890 0x7fee780cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780cb8f0 con 0x7fee780c1a60 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78152b80 con 0x7fee780c1690 2023-10-06T11:34:43.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee78152e60 con 0x7fee78155890 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38256/0 (socket says 172.21.15.46:38256) 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 -- 172.21.15.46:0/2971518063 learned_addr learned my addr 172.21.15.46:0/2971518063 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7d722700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee78155890 0x7fee780cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7cf21700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1690 0x7fee781505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 -- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee78155890 msgr2=0x7fee780cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee78155890 0x7fee780cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 -- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1690 msgr2=0x7fee781505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1690 0x7fee781505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 -- 172.21.15.46:0/2971518063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009420 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee77fff700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fee7806c150 tx=0x7fee64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6401cc90 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6401cdf0 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78153140 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6401a6a0 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.527+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee780c3420 con 0x7fee780c1a60 2023-10-06T11:34:43.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.531+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee780660f0 con 0x7fee780c1a60 2023-10-06T11:34:43.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.531+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 44) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fee6401aa50 con 0x7fee780c1a60 2023-10-06T11:34:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.535+0000 7fee74ff9700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 0x7fee5c062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.535+0000 7fee7cf21700 1 -- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 msgr2=0x7fee5c062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.535+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fee6407e7a0 con 0x7fee780c1a60 2023-10-06T11:34:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.535+0000 7fee7cf21700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 0x7fee5c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.539+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fee6404e8c0 con 0x7fee780c1a60 2023-10-06T11:34:43.672 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:43.668+0000 7ffae19c5000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:43.709 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:43.699+0000 7f3a825ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:34:43.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.707+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fee780cbc10 con 0x7fee780c1a60 2023-10-06T11:34:43.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.715+0000 7fee74ff9700 1 -- 172.21.15.46:0/2971518063 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7fee64051ee0 con 0x7fee780c1a60 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.715+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 msgr2=0x7fee5c062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.715+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 0x7fee5c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 msgr2=0x7fee780c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fee7806c150 tx=0x7fee64016000 comp rx=0 tx=0).stop 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 shutdown_connections 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fee5c0606c0 0x7fee5c062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee78155890 0x7fee780cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee780c1a60 0x7fee780c2ee0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 --2- 172.21.15.46:0/2971518063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780c1690 0x7fee781505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 >> 172.21.15.46:0/2971518063 conn(0x7fee780cbe30 msgr2=0x7fee7806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 shutdown_connections 2023-10-06T11:34:43.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.719+0000 7fee7f185700 1 -- 172.21.15.46:0/2971518063 wait complete. 2023-10-06T11:34:43.742 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:43.736+0000 7ffae19c5000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:43.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-10-06T11:34:43.787 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:43.775+0000 7f3a825ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:34:43.883 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:43.880+0000 7ffae19c5000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.907+0000 7ffa7cdcc700 1 Processor -- start 2023-10-06T11:34:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.907+0000 7ffa7cdcc700 1 -- start start 2023-10-06T11:34:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa7cdcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 0x7ffa780cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa7cdcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780cf360 con 0x7ffa781595c0 2023-10-06T11:34:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa7cdcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7815dd60 con 0x7ffa780cc500 2023-10-06T11:34:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa7cdcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780cea30 con 0x7ffa781591f0 2023-10-06T11:34:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa76ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa781595c0 0x7ffa780af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55970/0 (socket says 172.21.15.46:55970) 2023-10-06T11:34:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.911+0000 7ffa76ffd700 1 -- 172.21.15.46:0/979335796 learned_addr learned my addr 172.21.15.46:0/979335796 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:34:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa777fe700 1 --2- 172.21.15.46:0/979335796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 0x7ffa780cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa777fe700 1 -- 172.21.15.46:0/979335796 >> v1:172.21.15.132:6789/0 conn(0x7ffa780cc500 legacy=0x7ffa7815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa777fe700 1 -- 172.21.15.46:0/979335796 >> v1:172.21.15.46:6789/0 conn(0x7ffa781595c0 legacy=0x7ffa780af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa777fe700 1 -- 172.21.15.46:0/979335796 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780ced10 con 0x7ffa781591f0 2023-10-06T11:34:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa767fc700 1 -- 172.21.15.46:0/979335796 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159222502 0 0) 0x7ffa780cf360 con 0x7ffa781595c0 2023-10-06T11:34:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa777fe700 1 --2- 172.21.15.46:0/979335796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 0x7ffa780cf7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009910 tx=0x7ffa680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baeebb45b2d9c12c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa767fc700 1 -- 172.21.15.46:0/979335796 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800b8f0 con 0x7ffa781591f0 2023-10-06T11:34:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa767fc700 1 -- 172.21.15.46:0/979335796 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800ba50 con 0x7ffa781591f0 2023-10-06T11:34:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa767fc700 1 -- 172.21.15.46:0/979335796 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800bdf0 con 0x7ffa781591f0 2023-10-06T11:34:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/979335796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 msgr2=0x7ffa780cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/979335796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 0x7ffa780cf7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009910 tx=0x7ffa680091a0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/979335796 shutdown_connections 2023-10-06T11:34:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/979335796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa781591f0 0x7ffa780cf7f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/979335796 >> 172.21.15.46:0/979335796 conn(0x7ffa7806ef60 msgr2=0x7ffa780d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.915+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/979335796 shutdown_connections 2023-10-06T11:34:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.919+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/979335796 wait complete. 2023-10-06T11:34:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.919+0000 7ffa7cdcc700 1 Processor -- start 2023-10-06T11:34:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.919+0000 7ffa7cdcc700 1 -- start start 2023-10-06T11:34:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa780cc500 0x7ffa780b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa78152540 0x7ffa781599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7816e160 con 0x7ffa781591f0 2023-10-06T11:34:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b1dc0 con 0x7ffa78152540 2023-10-06T11:34:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780b1490 con 0x7ffa780cc500 2023-10-06T11:34:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38264/0 (socket says 172.21.15.46:38264) 2023-10-06T11:34:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 -- 172.21.15.46:0/2400148228 learned_addr learned my addr 172.21.15.46:0/2400148228 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa780cc500 0x7ffa780b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa77fff700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa78152540 0x7ffa781599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa780cc500 msgr2=0x7ffa780b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa780cc500 0x7ffa780b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa78152540 msgr2=0x7ffa781599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa78152540 0x7ffa781599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa68009420 con 0x7ffa781591f0 2023-10-06T11:34:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa76ffd700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c840 tx=0x7ffa6c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c005c00 con 0x7ffa781591f0 2023-10-06T11:34:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6c005d60 con 0x7ffa781591f0 2023-10-06T11:34:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c007320 con 0x7ffa781591f0 2023-10-06T11:34:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa780b1770 con 0x7ffa781591f0 2023-10-06T11:34:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.923+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa780b2790 con 0x7ffa781591f0 2023-10-06T11:34:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.927+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa780b29b0 con 0x7ffa781591f0 2023-10-06T11:34:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.927+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 44) v1 ==== 80078+0+0 (secure 0 0 0) 0x7ffa6c01b3b0 con 0x7ffa781591f0 2023-10-06T11:34:43.939 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:43.927+0000 7f3a825ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:43.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.927+0000 7ffa5f7fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:43.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.931+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ffa6c07c1d0 con 0x7ffa781591f0 2023-10-06T11:34:43.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.931+0000 7ffa777fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:43.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.931+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:34:43.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:43.935+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffa6c04c2f0 con 0x7ffa781591f0 2023-10-06T11:34:44.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:44.131+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa78064590 con 0x7ffa640606c0 2023-10-06T11:34:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:44.135+0000 7ffa777fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:44.135+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:34:44.152 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:44.148+0000 7ffae19c5000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:44.227 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:44.224+0000 7ffae19c5000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:44.237 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:44.227+0000 7f3a825ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:34:44.315 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:44.303+0000 7f3a825ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:44.455 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:44.452+0000 7ffae19c5000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:44.537 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:44.532+0000 7ffae19c5000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:44.540 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:44.531+0000 7f3a825ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:34:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:44.539+0000 7ffa777fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:44.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:44.539+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:34:44.622 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:44.616+0000 7ffae19c5000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:44.626 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:44.615+0000 7f3a825ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:34:44.713 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:44.703+0000 7f3a825ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:34:45.022 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.016+0000 7ffae19c5000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:45.126 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:45.115+0000 7f3a825ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:34:45.190 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.184+0000 7ffae19c5000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:45.297 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:45.287+0000 7f3a825ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:34:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:45.343+0000 7ffa777fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:45.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:45.343+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:34:45.565 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.560+0000 7ffae19c5000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:45.681 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:45.671+0000 7f3a825ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:34:45.788 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.784+0000 7ffae19c5000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:45.879 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.876+0000 7ffae19c5000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:45.902 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:45.891+0000 7f3a825ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:34:45.959 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:45.956+0000 7ffae19c5000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:45.991 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:45.979+0000 7f3a825ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:34:46.037 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.032+0000 7ffae19c5000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:46.068 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.059+0000 7f3a825ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:34:46.146 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.135+0000 7f3a825ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:34:46.269 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.264+0000 7ffae19c5000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:46.349 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.344+0000 7ffae19c5000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:46.362 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.351+0000 7f3a825ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:34:46.435 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.432+0000 7ffae19c5000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:46.440 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.431+0000 7f3a825ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:34:46.529 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.519+0000 7f3a825ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:34:46.643 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.636+0000 7ffae19c5000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:46.728 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.719+0000 7f3a825ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:34:46.729 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:46.724+0000 7ffae19c5000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:46.815 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:46.803+0000 7f3a825ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:34:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:46.947+0000 7ffa777fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:34:46.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:46.947+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:34:47.463 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:34:47.460+0000 7ffae19c5000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:47.519+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 45) v1 ==== 80078+0+0 (secure 0 0 0) 0x7ffa6c04c6c0 con 0x7ffa781591f0 2023-10-06T11:34:47.553 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.543+0000 7f3a825ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:34:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:47.563+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 46) v1 ==== 79687+0+0 (secure 0 0 0) 0x7ffa6c07a320 con 0x7ffa781591f0 2023-10-06T11:34:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:47.563+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:34:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:47.563+0000 7ffa5f7fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:47.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:34:47.624 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:34:47.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.747+0000 7f3a42c9e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.759 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.747+0000 7f3a42c9e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.747+0000 7f3a42c9e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.747+0000 7f3a42c9e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.747+0000 7f3a42c9e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.751+0000 7f3a454a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.761 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.751+0000 7f3a454a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.761 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.751+0000 7f3a454a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.761 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.751+0000 7f3a454a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:47.761 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:34:47.751+0000 7f3a454a3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:34:48.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.575+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 47) v1 ==== 79814+0+0 (secure 0 0 0) 0x7ffa6c04cf90 con 0x7ffa781591f0 2023-10-06T11:34:48.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.575+0000 7ffa5f7fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.575+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa78064590 con 0x7ffa640606c0 2023-10-06T11:34:48.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.579+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.582+0000 7ffa777fe700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa680093f0 tx=0x7ffa68017000 comp rx=0 tx=0).ready entity=mgr.7303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.686+0000 7ffa5f7fe700 1 -- 172.21.15.46:0/2400148228 <== mgr.7303 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ffa78064590 con 0x7ffa640606c0 2023-10-06T11:34:48.699 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /bad-path 2023-10-06T11:34:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 msgr2=0x7ffa64062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa680093f0 tx=0x7ffa68017000 comp rx=0 tx=0).stop 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 msgr2=0x7ffa78152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00c840 tx=0x7ffa6c007d90 comp rx=0 tx=0).stop 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 shutdown_connections 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa640606c0 0x7ffa64062af0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa78152540 0x7ffa781599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa781591f0 0x7ffa78152000 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 --2- 172.21.15.46:0/2400148228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa780cc500 0x7ffa780b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.690+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 >> 172.21.15.46:0/2400148228 conn(0x7ffa7806ef60 msgr2=0x7ffa78069100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.694+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 shutdown_connections 2023-10-06T11:34:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.694+0000 7ffa7cdcc700 1 -- 172.21.15.46:0/2400148228 wait complete. 2023-10-06T11:34:48.728 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-06T11:34:48.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-06T11:34:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.898+0000 7fbefa7cc700 1 Processor -- start 2023-10-06T11:34:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.898+0000 7fbefa7cc700 1 -- start start 2023-10-06T11:34:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbefa7cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef40cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbefa7cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbef415a040 0x7fbef4068160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbefa7cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef415a570 con 0x7fbef41591e0 2023-10-06T11:34:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbefa7cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef415cc80 con 0x7fbef415a040 2023-10-06T11:34:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbefa7cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4069fc0 con 0x7fbef4158e10 2023-10-06T11:34:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbef415a040 0x7fbef4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbef415a040 0x7fbef4068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38426/0 (socket says 172.21.15.46:38426) 2023-10-06T11:34:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 -- 172.21.15.46:0/2766867113 learned_addr learned my addr 172.21.15.46:0/2766867113 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef9fcb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbef41591e0 0x7fbef40ca830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55972/0 (socket says 172.21.15.46:55972) 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef40cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 -- 172.21.15.46:0/2766867113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 msgr2=0x7fbef40cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef40cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 -- 172.21.15.46:0/2766867113 >> v1:172.21.15.46:6789/0 conn(0x7fbef41591e0 legacy=0x7fbef40ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 -- 172.21.15.46:0/2766867113 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40cbf40 con 0x7fbef415a040 2023-10-06T11:34:48.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef40cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbef8fc9700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef4068160 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbee40099b0 tx=0x7fbee4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55b1ab50b031e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.902+0000 7fbeebfff700 1 -- 172.21.15.46:0/2766867113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee400b860 con 0x7fbef415a040 2023-10-06T11:34:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbeebfff700 1 -- 172.21.15.46:0/2766867113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee400b9c0 con 0x7fbef415a040 2023-10-06T11:34:48.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbeebfff700 1 -- 172.21.15.46:0/2766867113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee400bd60 con 0x7fbef415a040 2023-10-06T11:34:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2766867113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 msgr2=0x7fbef4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef4068160 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbee40099b0 tx=0x7fbee4009250 comp rx=0 tx=0).stop 2023-10-06T11:34:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2766867113 shutdown_connections 2023-10-06T11:34:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef4068160 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2766867113 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef40cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2766867113 >> 172.21.15.46:0/2766867113 conn(0x7fbef406ad70 msgr2=0x7fbef406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2766867113 shutdown_connections 2023-10-06T11:34:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.906+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2766867113 wait complete. 2023-10-06T11:34:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 Processor -- start 2023-10-06T11:34:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 -- start start 2023-10-06T11:34:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef41511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 0x7fbef40b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40b2230 con 0x7fbef415a040 2023-10-06T11:34:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40b1620 con 0x7fbef41591e0 2023-10-06T11:34:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbefa7cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40b1900 con 0x7fbef4158e10 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef9fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef9fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38436/0 (socket says 172.21.15.46:38436) 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef9fcb700 1 -- 172.21.15.46:0/2336646213 learned_addr learned my addr 172.21.15.46:0/2336646213 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef41511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef8fc9700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 0x7fbef40b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef9fcb700 1 -- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 msgr2=0x7fbef41511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.910+0000 7fbef9fcb700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef41511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef9fcb700 1 -- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 msgr2=0x7fbef40b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef9fcb700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 0x7fbef40b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef9fcb700 1 -- 172.21.15.46:0/2336646213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee40094f0 con 0x7fbef415a040 2023-10-06T11:34:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef41511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef8fc9700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 0x7fbef40b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:34:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbef9fcb700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00cd00 tx=0x7fbeec001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec005df0 con 0x7fbef415a040 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00a960 con 0x7fbef415a040 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40b1be0 con 0x7fbef415a040 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00ad00 con 0x7fbef415a040 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4151730 con 0x7fbef415a040 2023-10-06T11:34:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.914+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef40cc990 con 0x7fbef415a040 2023-10-06T11:34:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.918+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 47) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fbeec007300 con 0x7fbef415a040 2023-10-06T11:34:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.918+0000 7fbee97fa700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 0x7fbed80628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:34:48.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.918+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fbeec07cac0 con 0x7fbef415a040 2023-10-06T11:34:48.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.918+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 0x7fbed80628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:34:48.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.918+0000 7fbef97ca700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 0x7fbed80628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006f00 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.7303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:34:48.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:48.926+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbef40cc990 con 0x7fbef415a040 2023-10-06T11:34:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.118+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbef40cc990 con 0x7fbed80603f0 2023-10-06T11:34:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.134+0000 7fbee97fa700 1 -- 172.21.15.46:0/2336646213 <== mgr.7303 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fbef40cc990 con 0x7fbed80603f0 2023-10-06T11:34:49.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:34:49.144 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:34:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-06T11:34:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.134+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 msgr2=0x7fbed80628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.134+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 0x7fbed80628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006f00 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2023-10-06T11:34:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.134+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 msgr2=0x7fbef415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:34:49.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.134+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00cd00 tx=0x7fbeec001b10 comp rx=0 tx=0).stop 2023-10-06T11:34:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 shutdown_connections 2023-10-06T11:34:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbed80603f0 0x7fbed80628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbef415a040 0x7fbef415e4c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef41591e0 0x7fbef40b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:49.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 --2- 172.21.15.46:0/2336646213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef4158e10 0x7fbef41511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:34:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 >> 172.21.15.46:0/2336646213 conn(0x7fbef406ad70 msgr2=0x7fbef40c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:34:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 shutdown_connections 2023-10-06T11:34:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:34:49.138+0000 7fbefa7cc700 1 -- 172.21.15.46:0/2336646213 wait complete. 2023-10-06T11:34:49.170 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:34:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-06T11:35:00.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:00.710+0000 7f3a35483700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:00.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:00.710+0000 7f3a35483700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:00.721 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:00.710+0000 7f3a35483700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:00.721 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:00.710+0000 7f3a35483700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:00.721 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:00.710+0000 7f3a35483700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:17.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:35:17.702 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:35:47.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:35:47.811 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:35:49.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-06T11:35:49.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.329+0000 7f466b59e700 1 Processor -- start 2023-10-06T11:35:49.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.329+0000 7f466b59e700 1 -- start start 2023-10-06T11:35:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466b59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f466c069770 0x7f466c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0b5eb0 con 0x7f466c158de0 2023-10-06T11:35:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0cef30 con 0x7f466c1591b0 2023-10-06T11:35:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466b59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c0cf4f0 con 0x7f466c069770 2023-10-06T11:35:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f466c158de0 0x7f466c069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53388/0 (socket says 172.21.15.46:53388) 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466a59c700 1 -- 172.21.15.46:0/1705770063 learned_addr learned my addr 172.21.15.46:0/1705770063 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816919109 0 0) 0x7f465c001110 con 0x7f466c158de0 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4654003570 con 0x7f466c158de0 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f4669d9b700 1 --2- 172.21.15.46:0/1705770063 >> v2:172.21.15.132:3300/0 conn(0x7f466c069770 0x7f466c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226695078 0 0) 0x7f4654003570 con 0x7f466c158de0 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 >> v2:172.21.15.132:3300/0 conn(0x7f466c069770 msgr2=0x7f466c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 --2- 172.21.15.46:0/1705770063 >> v2:172.21.15.132:3300/0 conn(0x7f466c069770 0x7f466c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 >> v1:172.21.15.132:6789/0 conn(0x7f466c1591b0 legacy=0x7f466c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0ccac0 con 0x7f466c158de0 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f465c001600 con 0x7f466c158de0 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 >> v1:172.21.15.46:6789/0 conn(0x7f466c158de0 legacy=0x7f466c069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4654003a60 0x7f4654003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4654004810 0x7f4654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 0x7f465400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0cf4f0 con 0x7f46540052e0 2023-10-06T11:35:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465400c190 con 0x7f4654004810 2023-10-06T11:35:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465400c350 con 0x7f4654003a60 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 0x7f465400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f4669d9b700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4654004810 0x7f4654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466a59c700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4654003a60 0x7f4654003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466ad9d700 1 -- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4654003a60 msgr2=0x7f4654003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.333+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4654003a60 0x7f4654003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466ad9d700 1 -- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4654004810 msgr2=0x7f4654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4654004810 0x7f4654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466ad9d700 1 -- 172.21.15.46:0/1705770063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4654003570 con 0x7f46540052e0 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466ad9d700 1 -- 172.21.15.46:0/1705770063 reap_dead start 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 0x7f465400ba60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4664001fc0 tx=0x7f4664001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8463fb5d6e89d4df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664001950 con 0x7f46540052e0 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466400c8b0 con 0x7f46540052e0 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466959a700 1 -- 172.21.15.46:0/1705770063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466400cc50 con 0x7f46540052e0 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 -- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 msgr2=0x7f465400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 0x7f465400ba60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4664001fc0 tx=0x7f4664001640 comp rx=0 tx=0).stop 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 -- 172.21.15.46:0/1705770063 shutdown_connections 2023-10-06T11:35:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 --2- 172.21.15.46:0/1705770063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46540052e0 0x7f465400ba60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 -- 172.21.15.46:0/1705770063 >> 172.21.15.46:0/1705770063 conn(0x7f466c068390 msgr2=0x7f466c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 -- 172.21.15.46:0/1705770063 shutdown_connections 2023-10-06T11:35:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.337+0000 7f466b59e700 1 -- 172.21.15.46:0/1705770063 wait complete. 2023-10-06T11:35:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 Processor -- start 2023-10-06T11:35:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 -- start start 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46540052e0 0x7f466c0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 0x7f466c16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c16a700 con 0x7f466c0c7490 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c154530 con 0x7f466c169d70 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0c7a10 con 0x7f46540052e0 2023-10-06T11:35:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f4669d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f4669d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34700/0 (socket says 172.21.15.46:34700) 2023-10-06T11:35:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f4669d9b700 1 -- 172.21.15.46:0/1095485520 learned_addr learned my addr 172.21.15.46:0/1095485520 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 0x7f466c16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.341+0000 7f466a59c700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46540052e0 0x7f466c0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 -- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46540052e0 msgr2=0x7f466c0c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46540052e0 0x7f466c0c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 -- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 msgr2=0x7f466c16a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 0x7f466c16a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 -- 172.21.15.46:0/1095485520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0c7cf0 con 0x7f466c0c7490 2023-10-06T11:35:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f466ad9d700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 0x7f466c16a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f4669d9b700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f46600109f0 tx=0x7f466000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660010ee0 con 0x7f466c0c7490 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000e8d0 con 0x7f466c0c7490 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000ecd0 con 0x7f466c0c7490 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c0c7fd0 con 0x7f466c0c7490 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c159b60 con 0x7f466c0c7490 2023-10-06T11:35:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.345+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c064590 con 0x7f466c0c7490 2023-10-06T11:35:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.349+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) v1 ==== 80192+0+0 (secure 0 0 0) 0x7f4660014640 con 0x7f466c0c7490 2023-10-06T11:35:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.349+0000 7f465a7fc700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 0x7f464c062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.349+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f466007d7d0 con 0x7f466c0c7490 2023-10-06T11:35:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.349+0000 7f466a59c700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 0x7f464c062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.353+0000 7f466a59c700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 0x7f464c062c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f465c001050 tx=0x7f466c0cf460 comp rx=0 tx=0).ready entity=mgr.7303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.357+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f466004d4d0 con 0x7f466c0c7490 2023-10-06T11:35:49.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.549+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f466c0ccda0 con 0x7f464c0607e0 2023-10-06T11:35:49.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f465a7fc700 1 -- 172.21.15.46:0/1095485520 <== mgr.7303 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f466c0ccda0 con 0x7f464c0607e0 2023-10-06T11:35:49.581 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:49.582 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:34:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-06T11:35:49.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 msgr2=0x7f464c062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 0x7f464c062c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f465c001050 tx=0x7f466c0cf460 comp rx=0 tx=0).stop 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 msgr2=0x7f466c169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f46600109f0 tx=0x7f466000b040 comp rx=0 tx=0).stop 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 shutdown_connections 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f464c0607e0 0x7f464c062c90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f466c169d70 0x7f466c16a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466c0c7490 0x7f466c169830 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 --2- 172.21.15.46:0/1095485520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46540052e0 0x7f466c0c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.569+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 >> 172.21.15.46:0/1095485520 conn(0x7f466c068390 msgr2=0x7f466c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:49.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.573+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 shutdown_connections 2023-10-06T11:35:49.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.573+0000 7f466b59e700 1 -- 172.21.15.46:0/1095485520 wait complete. 2023-10-06T11:35:49.603 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:34:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-06T11:35:49.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-10-06T11:35:49.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.761+0000 7f6507278700 1 Processor -- start 2023-10-06T11:35:49.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.761+0000 7f6507278700 1 -- start start 2023-10-06T11:35:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6507278700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65000abac0 0x7f65000abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6507278700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65000af0b0 0x7f650015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6507278700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6500067bd0 con 0x7f650015d340 2023-10-06T11:35:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6507278700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65000aa500 con 0x7f65000abac0 2023-10-06T11:35:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6507278700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6500159800 con 0x7f65000af0b0 2023-10-06T11:35:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65000abac0 0x7f65000abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505815700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f650015d340 0x7f6500159f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53390/0 (socket says 172.21.15.46:53390) 2023-10-06T11:35:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505815700 1 -- 172.21.15.46:0/3583034276 learned_addr learned my addr 172.21.15.46:0/3583034276 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65000abac0 0x7f65000abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34714/0 (socket says 172.21.15.46:34714) 2023-10-06T11:35:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6504813700 1 --2- 172.21.15.46:0/3583034276 >> v2:172.21.15.132:3300/0 conn(0x7f65000af0b0 0x7f650015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f64f7fff700 1 -- 172.21.15.46:0/3583034276 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231392531 0 0) 0x7f6500067bd0 con 0x7f650015d340 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f64f7fff700 1 -- 172.21.15.46:0/3583034276 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64ec003570 con 0x7f650015d340 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 -- 172.21.15.46:0/3583034276 >> v2:172.21.15.132:3300/0 conn(0x7f65000af0b0 msgr2=0x7f650015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 --2- 172.21.15.46:0/3583034276 >> v2:172.21.15.132:3300/0 conn(0x7f65000af0b0 0x7f650015ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 -- 172.21.15.46:0/3583034276 >> v1:172.21.15.46:6789/0 conn(0x7f650015d340 legacy=0x7f6500159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 -- 172.21.15.46:0/3583034276 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000ac3d0 con 0x7f65000abac0 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.765+0000 7f6505014700 1 --2- 172.21.15.46:0/3583034276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000abac0 0x7f65000abe90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f650006f8d0 tx=0x7f64fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe14124db7315a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f64f7fff700 1 -- 172.21.15.46:0/3583034276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011630 con 0x7f65000abac0 2023-10-06T11:35:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f64f7fff700 1 -- 172.21.15.46:0/3583034276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc0117b0 con 0x7f65000abac0 2023-10-06T11:35:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f64f7fff700 1 -- 172.21.15.46:0/3583034276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011b70 con 0x7f65000abac0 2023-10-06T11:35:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 -- 172.21.15.46:0/3583034276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000abac0 msgr2=0x7f65000abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 --2- 172.21.15.46:0/3583034276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000abac0 0x7f65000abe90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f650006f8d0 tx=0x7f64fc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 -- 172.21.15.46:0/3583034276 shutdown_connections 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 --2- 172.21.15.46:0/3583034276 >> v2:172.21.15.132:3300/0 conn(0x7f65000af0b0 0x7f650015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 --2- 172.21.15.46:0/3583034276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000abac0 0x7f65000abe90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 -- 172.21.15.46:0/3583034276 >> 172.21.15.46:0/3583034276 conn(0x7f6500068c00 msgr2=0x7f65000ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 -- 172.21.15.46:0/3583034276 shutdown_connections 2023-10-06T11:35:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.769+0000 7f6507278700 1 -- 172.21.15.46:0/3583034276 wait complete. 2023-10-06T11:35:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 Processor -- start 2023-10-06T11:35:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 -- start start 2023-10-06T11:35:49.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 0x7f65000c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f650015d340 0x7f650015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65000c4630 con 0x7f65000af0b0 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6500159ed0 con 0x7f65000abac0 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6507278700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650015a1b0 con 0x7f650015d340 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34730/0 (socket says 172.21.15.46:34730) 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 -- 172.21.15.46:0/2713539626 learned_addr learned my addr 172.21.15.46:0/2713539626 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6505815700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f650015d340 0x7f650015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6505014700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 0x7f65000c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 -- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f650015d340 msgr2=0x7f650015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f650015d340 0x7f650015c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 -- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 msgr2=0x7f65000c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 0x7f65000c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 -- 172.21.15.46:0/2713539626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009420 con 0x7f65000af0b0 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6505014700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 0x7f65000c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.773+0000 7f6504813700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ec80 tx=0x7f64f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000c850 con 0x7f65000af0b0 2023-10-06T11:35:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f000c9b0 con 0x7f65000af0b0 2023-10-06T11:35:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650015a490 con 0x7f65000af0b0 2023-10-06T11:35:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000cd50 con 0x7f65000af0b0 2023-10-06T11:35:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650015aa50 con 0x7f65000af0b0 2023-10-06T11:35:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65000660f0 con 0x7f65000af0b0 2023-10-06T11:35:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.777+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) v1 ==== 80192+0+0 (secure 0 0 0) 0x7f64f0013680 con 0x7f65000af0b0 2023-10-06T11:35:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.781+0000 7f64f57fa700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 0x7f64e4062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.781+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f64f007d780 con 0x7f65000af0b0 2023-10-06T11:35:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.781+0000 7f6505014700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 0x7f64e4062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.781+0000 7f6505014700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 0x7f64e4062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f650006f8d0 tx=0x7f64fc017000 comp rx=0 tx=0).ready entity=mgr.7303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.785+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64f004d550 con 0x7f65000af0b0 2023-10-06T11:35:49.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:49.981+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f65000a9c60 con 0x7f64e40607e0 2023-10-06T11:35:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.093+0000 7f64f57fa700 1 -- 172.21.15.46:0/2713539626 <== mgr.7303 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f65000a9c60 con 0x7f64e40607e0 2023-10-06T11:35:50.105 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /bad-path 2023-10-06T11:35:50.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 msgr2=0x7f64e4062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 0x7f64e4062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f650006f8d0 tx=0x7f64fc017000 comp rx=0 tx=0).stop 2023-10-06T11:35:50.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 msgr2=0x7f6500169a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ec80 tx=0x7f64f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 shutdown_connections 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f64e40607e0 0x7f64e4062c90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f650015d340 0x7f650015c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65000af0b0 0x7f6500169a30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 --2- 172.21.15.46:0/2713539626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65000abac0 0x7f65000c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 >> 172.21.15.46:0/2713539626 conn(0x7f6500068c00 msgr2=0x7f65000ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 shutdown_connections 2023-10-06T11:35:50.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.097+0000 7f6507278700 1 -- 172.21.15.46:0/2713539626 wait complete. 2023-10-06T11:35:50.131 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-06T11:35:50.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:35:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 Processor -- start 2023-10-06T11:35:50.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 -- start start 2023-10-06T11:35:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806e8a0 0x7fb8d80c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8d806ed10 0x7fb8d80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d80c1bf0 con 0x7fb8d80c6470 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80c72c0 con 0x7fb8d806ed10 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8e0585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d806cf80 con 0x7fb8d806e8a0 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8deb22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8d806ed10 0x7fb8d80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8deb22700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8d806ed10 0x7fb8d80c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34738/0 (socket says 172.21.15.46:34738) 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8deb22700 1 -- 172.21.15.46:0/1152209296 learned_addr learned my addr 172.21.15.46:0/1152209296 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:50.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.285+0000 7fb8ddb20700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806e8a0 0x7fb8d80c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8deb22700 1 -- 172.21.15.46:0/1152209296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806e8a0 msgr2=0x7fb8d80c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8deb22700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806e8a0 0x7fb8d80c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8deb22700 1 -- 172.21.15.46:0/1152209296 >> v1:172.21.15.46:6789/0 conn(0x7fb8d80c6470 legacy=0x7fb8d80ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8deb22700 1 -- 172.21.15.46:0/1152209296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d806d260 con 0x7fb8d806ed10 2023-10-06T11:35:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8deb22700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806ed10 0x7fb8d80c6d80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f20 tx=0x7fb8d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6c1edf751110c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8dd31f700 1 -- 172.21.15.46:0/1152209296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000e070 con 0x7fb8d806ed10 2023-10-06T11:35:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8dd31f700 1 -- 172.21.15.46:0/1152209296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000b9d0 con 0x7fb8d806ed10 2023-10-06T11:35:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8dd31f700 1 -- 172.21.15.46:0/1152209296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00092e0 con 0x7fb8d806ed10 2023-10-06T11:35:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 -- 172.21.15.46:0/1152209296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806ed10 msgr2=0x7fb8d80c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806ed10 0x7fb8d80c6d80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f20 tx=0x7fb8d0009840 comp rx=0 tx=0).stop 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 -- 172.21.15.46:0/1152209296 shutdown_connections 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806ed10 0x7fb8d80c6d80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 --2- 172.21.15.46:0/1152209296 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806e8a0 0x7fb8d80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 -- 172.21.15.46:0/1152209296 >> 172.21.15.46:0/1152209296 conn(0x7fb8d80cbe30 msgr2=0x7fb8d80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 -- 172.21.15.46:0/1152209296 shutdown_connections 2023-10-06T11:35:50.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.289+0000 7fb8e0585700 1 -- 172.21.15.46:0/1152209296 wait complete. 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 Processor -- start 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 -- start start 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 0x7fb8d80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 0x7fb8d8169220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80c9090 con 0x7fb8d806e8a0 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80cac20 con 0x7fb8d80c6470 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8e0585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80c9d10 con 0x7fb8d806ed10 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8de321700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8de321700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34748/0 (socket says 172.21.15.46:34748) 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8de321700 1 -- 172.21.15.46:0/478605355 learned_addr learned my addr 172.21.15.46:0/478605355 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8ddb20700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 0x7fb8d80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.293+0000 7fb8deb22700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 0x7fb8d8169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 -- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 msgr2=0x7fb8d80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 0x7fb8d80c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 -- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 msgr2=0x7fb8d8169220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 0x7fb8d8169220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 -- 172.21.15.46:0/478605355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009ae0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8ddb20700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 0x7fb8d80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:50.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8deb22700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 0x7fb8d8169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8de321700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400c5f0 tx=0x7fb8d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011590 con 0x7fb8d806e8a0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d4011710 con 0x7fb8d806e8a0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d80aee00 con 0x7fb8d806e8a0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4011ad0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d80c8ad0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.297+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d80aebd0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.301+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 49) v1 ==== 80192+0+0 (secure 0 0 0) 0x7fb8d401b3b0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.301+0000 7fb8ce7fc700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 0x7fb8bc062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:50.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.301+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb8d407c730 con 0x7fb8d806e8a0 2023-10-06T11:35:50.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.301+0000 7fb8ddb20700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 0x7fb8bc062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:50.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.301+0000 7fb8ddb20700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 0x7fb8bc062c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80c12e0 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.7303 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:50.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.309+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb8d404c500 con 0x7fb8d806e8a0 2023-10-06T11:35:50.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.517+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb8d80660f0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.589+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fb8d404c8d0 con 0x7fb8d806e8a0 2023-10-06T11:35:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.589+0000 7fb8ce7fc700 1 -- 172.21.15.46:0/478605355 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v50) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb8d404e300 con 0x7fb8d806e8a0 2023-10-06T11:35:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 msgr2=0x7fb8bc062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 0x7fb8bc062c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80c12e0 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2023-10-06T11:35:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 msgr2=0x7fb8d80c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400c5f0 tx=0x7fb8d400c9b0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 shutdown_connections 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8bc0607e0 0x7fb8bc062c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8d80c6470 0x7fb8d8169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8d806ed10 0x7fb8d80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 --2- 172.21.15.46:0/478605355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8d806e8a0 0x7fb8d80c7be0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 >> 172.21.15.46:0/478605355 conn(0x7fb8d80cbe30 msgr2=0x7fb8d806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 shutdown_connections 2023-10-06T11:35:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:50.597+0000 7fb8e0585700 1 -- 172.21.15.46:0/478605355 wait complete. 2023-10-06T11:35:50.635 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:35:50.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:35:50.678 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:35:50.714 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:35:50.714 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:35:50.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:35:50.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:35:50.781 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:50.778+0000 7f486a149000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:35:50.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:35:50.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:35:50.810 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:35:50.817 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:50.801+0000 7f7ede6e2000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:35:50.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:35:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.021+0000 7fef2759e700 1 Processor -- start 2023-10-06T11:35:51.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.021+0000 7fef2759e700 1 -- start start 2023-10-06T11:35:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef28156e20 0x7fef2806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fef28067760 0x7fef28068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2806d790 con 0x7fef280aaf30 2023-10-06T11:35:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef280acd60 con 0x7fef28156e20 2023-10-06T11:35:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef280ad040 con 0x7fef28067760 2023-10-06T11:35:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef1ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fef280aaf30 0x7fef28067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53422/0 (socket says 172.21.15.46:53422) 2023-10-06T11:35:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef1ffff700 1 -- 172.21.15.46:0/2086083356 learned_addr learned my addr 172.21.15.46:0/2086083356 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2659c700 1 --2- 172.21.15.46:0/2086083356 >> v2:172.21.15.46:3300/0 conn(0x7fef28156e20 0x7fef2806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef26d9d700 1 --2- 172.21.15.46:0/2086083356 >> v2:172.21.15.132:3300/0 conn(0x7fef28067760 0x7fef28068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773137487 0 0) 0x7fef2806d790 con 0x7fef280aaf30 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef14003570 con 0x7fef280aaf30 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154205155 0 0) 0x7fef14003570 con 0x7fef280aaf30 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 >> v2:172.21.15.132:3300/0 conn(0x7fef28067760 msgr2=0x7fef28068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 --2- 172.21.15.46:0/2086083356 >> v2:172.21.15.132:3300/0 conn(0x7fef28067760 0x7fef28068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 >> v2:172.21.15.46:3300/0 conn(0x7fef28156e20 msgr2=0x7fef2806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 --2- 172.21.15.46:0/2086083356 >> v2:172.21.15.46:3300/0 conn(0x7fef28156e20 0x7fef2806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280ad320 con 0x7fef280aaf30 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fef10001470 con 0x7fef280aaf30 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 >> v1:172.21.15.46:6789/0 conn(0x7fef280aaf30 legacy=0x7fef28067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef14003a60 0x7fef14003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 0x7fef14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef140051c0 0x7fef1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280acd60 con 0x7fef14004810 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280ad040 con 0x7fef14003a60 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1400bfd0 con 0x7fef140051c0 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef1ffff700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 0x7fef14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef2659c700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef14003a60 0x7fef14003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.025+0000 7fef26d9d700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef140051c0 0x7fef1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 -- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef140051c0 msgr2=0x7fef1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef140051c0 0x7fef1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 -- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef14003a60 msgr2=0x7fef14003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef14003a60 0x7fef14003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 -- 172.21.15.46:0/2086083356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14003570 con 0x7fef14004810 2023-10-06T11:35:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef26d9d700 1 -- 172.21.15.46:0/2086083356 reap_dead start 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef1ffff700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 0x7fef14004c80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c390 tx=0x7fef10001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff3a75a8b126425 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000cb80 con 0x7fef14004810 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1000cd20 con 0x7fef14004810 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef25d9b700 1 -- 172.21.15.46:0/2086083356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10005b70 con 0x7fef14004810 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 -- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 msgr2=0x7fef14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 0x7fef14004c80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c390 tx=0x7fef10001e90 comp rx=0 tx=0).stop 2023-10-06T11:35:51.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 -- 172.21.15.46:0/2086083356 shutdown_connections 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 --2- 172.21.15.46:0/2086083356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef14004810 0x7fef14004c80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 -- 172.21.15.46:0/2086083356 >> 172.21.15.46:0/2086083356 conn(0x7fef28062ef0 msgr2=0x7fef280cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 -- 172.21.15.46:0/2086083356 shutdown_connections 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.029+0000 7fef2759e700 1 -- 172.21.15.46:0/2086083356 wait complete. 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 Processor -- start 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 -- start start 2023-10-06T11:35:51.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 0x7fef280ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef280ca810 0x7fef28152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2806ea30 con 0x7fef2806e080 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280af1b0 con 0x7fef280ca810 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280b0b60 con 0x7fef14004810 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2659c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 0x7fef280ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34808/0 (socket says 172.21.15.46:34808) 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 -- 172.21.15.46:0/2296218882 learned_addr learned my addr 172.21.15.46:0/2296218882 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef1ffff700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef280ca810 0x7fef28152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 msgr2=0x7fef280ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 0x7fef280ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef280ca810 msgr2=0x7fef28152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef280ca810 0x7fef28152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef26d9d700 1 -- 172.21.15.46:0/2296218882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280b0e40 con 0x7fef2806e080 2023-10-06T11:35:51.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.033+0000 7fef2659c700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 0x7fef280ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef26d9d700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fef280c2b50 tx=0x7fef2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2000e070 con 0x7fef2806e080 2023-10-06T11:35:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef20011b80 con 0x7fef2806e080 2023-10-06T11:35:51.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20016470 con 0x7fef2806e080 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef280b1120 con 0x7fef2806e080 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef280b0830 con 0x7fef2806e080 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef280abe30 con 0x7fef2806e080 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.037+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fef20016860 con 0x7fef2806e080 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.041+0000 7fef1e7fc700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 0x7fef08062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.041+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fef2007da90 con 0x7fef2806e080 2023-10-06T11:35:51.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.041+0000 7fef2659c700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 msgr2=0x7fef08062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:51.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.041+0000 7fef2659c700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 0x7fef08062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:51.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.045+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fef280abe30 con 0x7fef2806e080 2023-10-06T11:35:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.245+0000 7fef2659c700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 msgr2=0x7fef08062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.245+0000 7fef2659c700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 0x7fef08062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:35:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.253+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef280abe30 con 0x7fef2806e080 2023-10-06T11:35:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.253+0000 7fef1e7fc700 1 -- 172.21.15.46:0/2296218882 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7fef2004d750 con 0x7fef2806e080 2023-10-06T11:35:51.266 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:51.267 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:35:51.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 msgr2=0x7fef08062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:51.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 0x7fef08062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 msgr2=0x7fef2806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fef280c2b50 tx=0x7fef2000b040 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 shutdown_connections 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fef080607c0 0x7fef08062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef2806e080 0x7fef2806e4f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef280ca810 0x7fef28152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 --2- 172.21.15.46:0/2296218882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef14004810 0x7fef280ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 >> 172.21.15.46:0/2296218882 conn(0x7fef28062ef0 msgr2=0x7fef28155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 shutdown_connections 2023-10-06T11:35:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.257+0000 7fef2759e700 1 -- 172.21.15.46:0/2296218882 wait complete. 2023-10-06T11:35:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:35:51.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:35:51.322 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:51.318+0000 7f486a149000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:35:51.379 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:51.365+0000 7f7ede6e2000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:35:51.399 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:51.398+0000 7f486a149000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:35:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 Processor -- start 2023-10-06T11:35:51.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 -- start start 2023-10-06T11:35:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ae006e8a0 0x7f8ae006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ae00adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae006d460 con 0x7f8ae015ab60 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae006ed40 con 0x7f8ae006e8a0 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae8385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00c7000 con 0x7f8ae00ad6c0 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae6121700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ae015ab60 0x7f8ae015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53438/0 (socket says 172.21.15.46:53438) 2023-10-06T11:35:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.429+0000 7f8ae6121700 1 -- 172.21.15.46:0/3222820269 learned_addr learned my addr 172.21.15.46:0/3222820269 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100056934 0 0) 0x7f8ae006d460 con 0x7f8ae015ab60 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac8003570 con 0x7f8ae015ab60 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6922700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ae00adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> v2:172.21.15.46:3300/0 conn(0x7f8ae006e8a0 0x7f8ae006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170541344 0 0) 0x7f8ac8003570 con 0x7f8ae015ab60 2023-10-06T11:35:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 msgr2=0x7f8ae00adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ae00adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 >> v2:172.21.15.46:3300/0 conn(0x7f8ae006e8a0 msgr2=0x7f8ae006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 --2- 172.21.15.46:0/3222820269 >> v2:172.21.15.46:3300/0 conn(0x7f8ae006e8a0 0x7f8ae006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00c72c0 con 0x7f8ae015ab60 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f8ad0001990 con 0x7f8ae015ab60 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> v2:172.21.15.46:3300/0 conn(0x7f8ae006e8a0 0x7f8ae006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 >> v1:172.21.15.46:6789/0 conn(0x7f8ae015ab60 legacy=0x7f8ae015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac8003a60 0x7f8ac8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ac8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ac80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae006ed40 con 0x7f8ac8004810 2023-10-06T11:35:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00c7000 con 0x7f8ac8003a60 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac800bb10 con 0x7f8ae00ad6c0 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ac8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6922700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ac80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6121700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac8003a60 0x7f8ac8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 -- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 msgr2=0x7f8ac80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ac80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 -- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac8003a60 msgr2=0x7f8ac8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac8003a60 0x7f8ac8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 -- 172.21.15.46:0/3222820269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8003570 con 0x7f8ac8004810 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6922700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae00ad6c0 0x7f8ac80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6121700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ac8003a60 0x7f8ac8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:51.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ac8004c80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ba0 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd8c5cff9e2fc57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400c070 con 0x7f8ac8004810 2023-10-06T11:35:51.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400e5e0 con 0x7f8ac8004810 2023-10-06T11:35:51.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae511f700 1 -- 172.21.15.46:0/3222820269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009510 con 0x7f8ac8004810 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae8385700 1 -- 172.21.15.46:0/3222820269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 msgr2=0x7f8ac8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3222820269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ac8004c80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ba0 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.433+0000 7f8ae6922700 1 -- 172.21.15.46:0/3222820269 reap_dead start 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 -- 172.21.15.46:0/3222820269 shutdown_connections 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 -- 172.21.15.46:0/3222820269 >> 172.21.15.46:0/3222820269 conn(0x7f8ae00cbe30 msgr2=0x7f8ae00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 -- 172.21.15.46:0/3222820269 shutdown_connections 2023-10-06T11:35:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 -- 172.21.15.46:0/3222820269 wait complete. 2023-10-06T11:35:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 Processor -- start 2023-10-06T11:35:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.437+0000 7f8ae8385700 1 -- start start 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae015ce70 0x7f8ae014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ae0150410 0x7f8ae015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0150880 con 0x7f8ac8004810 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae015c9e0 con 0x7f8ae0150410 2023-10-06T11:35:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae015bdd0 con 0x7f8ae015ce70 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34846/0 (socket says 172.21.15.46:34846) 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 -- 172.21.15.46:0/3580209129 learned_addr learned my addr 172.21.15.46:0/3580209129 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6922700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ae0150410 0x7f8ae015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae015ce70 0x7f8ae014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 -- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae015ce70 msgr2=0x7f8ae014fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae015ce70 0x7f8ae014fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 -- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ae0150410 msgr2=0x7f8ae015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ae0150410 0x7f8ae015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 -- 172.21.15.46:0/3580209129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae015c0b0 con 0x7f8ac8004810 2023-10-06T11:35:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae6121700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0007fd0 tx=0x7f8ad0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000de60 con 0x7f8ac8004810 2023-10-06T11:35:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0005d80 con 0x7f8ac8004810 2023-10-06T11:35:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae015c390 con 0x7f8ac8004810 2023-10-06T11:35:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000bb20 con 0x7f8ac8004810 2023-10-06T11:35:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae014b120 con 0x7f8ac8004810 2023-10-06T11:35:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.441+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae015c810 con 0x7f8ac8004810 2023-10-06T11:35:51.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.445+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f8ad00070f0 con 0x7f8ac8004810 2023-10-06T11:35:51.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.445+0000 7f8ade7fc700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 0x7f8acc062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.445+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f8ad007d480 con 0x7f8ac8004810 2023-10-06T11:35:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.445+0000 7f8ae5920700 1 -- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 msgr2=0x7f8acc062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.445+0000 7f8ae5920700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 0x7f8acc062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:51.462 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:51.449+0000 7f7ede6e2000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:35:51.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.453+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ad004d1c0 con 0x7f8ac8004810 2023-10-06T11:35:51.539 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:51.538+0000 7f486a149000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:35:51.613 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:51.601+0000 7f7ede6e2000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:35:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.625+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ae0063f90 con 0x7f8ac8004810 2023-10-06T11:35:51.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.625+0000 7f8ade7fc700 1 -- 172.21.15.46:0/3580209129 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f8ad00507e0 con 0x7f8ac8004810 2023-10-06T11:35:51.640 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:51.640 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:35:49.555711+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/606963126":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/321844900":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/1090757770":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/3400023669":"2023-10-07T11:34:47.548766+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/194565737":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/2862702378":"2023-10-07T11:34:47.548766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 msgr2=0x7f8acc062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 0x7f8acc062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 msgr2=0x7f8ae014abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0007fd0 tx=0x7f8ad0009040 comp rx=0 tx=0).stop 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 shutdown_connections 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8acc0607c0 0x7f8acc062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ae0150410 0x7f8ae015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ae015ce70 0x7f8ae014fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 --2- 172.21.15.46:0/3580209129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ac8004810 0x7f8ae014abe0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 >> 172.21.15.46:0/3580209129 conn(0x7f8ae00cbe30 msgr2=0x7f8ae00ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 shutdown_connections 2023-10-06T11:35:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.629+0000 7f8ae8385700 1 -- 172.21.15.46:0/3580209129 wait complete. 2023-10-06T11:35:51.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:34:30.994864+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:34:32.078476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:35:51.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:35:51.812 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:51.810+0000 7f486a149000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:35:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.821+0000 7fd67e0f0700 1 Processor -- start 2023-10-06T11:35:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.821+0000 7fd67e0f0700 1 -- start start 2023-10-06T11:35:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd67e0f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd678156dc0 0x7fd678157190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd67e0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 0x7fd6780ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd67e0f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6780ac8e0 con 0x7fd6780676a0 2023-10-06T11:35:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd67e0f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780ac320 con 0x7fd678156dc0 2023-10-06T11:35:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd67e0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780ac600 con 0x7fd67806d440 2023-10-06T11:35:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd66ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 0x7fd6780ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd6777fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd678156dc0 0x7fd678157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd677fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd6780676a0 0x7fd678067b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:60360/0 (socket says 172.21.15.46:60360) 2023-10-06T11:35:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd677fff700 1 -- 172.21.15.46:0/3350052613 learned_addr learned my addr 172.21.15.46:0/3350052613 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:51.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd6777fe700 1 -- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 msgr2=0x7fd6780ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd6777fe700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 0x7fd6780ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd6777fe700 1 -- 172.21.15.46:0/3350052613 >> v1:172.21.15.132:6789/0 conn(0x7fd6780676a0 legacy=0x7fd678067b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd6777fe700 1 -- 172.21.15.46:0/3350052613 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678067180 con 0x7fd678156dc0 2023-10-06T11:35:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.825+0000 7fd66ffff700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 0x7fd6780ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd6777fe700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd678156dc0 0x7fd678157190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd6680098e0 tx=0x7fd6680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c678ac1b9ef72e85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd676ffd700 1 -- 172.21.15.46:0/3350052613 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800b640 con 0x7fd678156dc0 2023-10-06T11:35:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd676ffd700 1 -- 172.21.15.46:0/3350052613 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800b7a0 con 0x7fd678156dc0 2023-10-06T11:35:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd676ffd700 1 -- 172.21.15.46:0/3350052613 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800bb40 con 0x7fd678156dc0 2023-10-06T11:35:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 -- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd678156dc0 msgr2=0x7fd678157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd678156dc0 0x7fd678157190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd6680098e0 tx=0x7fd6680091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 -- 172.21.15.46:0/3350052613 shutdown_connections 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd67806d440 0x7fd6780ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/3350052613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd678156dc0 0x7fd678157190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 -- 172.21.15.46:0/3350052613 >> 172.21.15.46:0/3350052613 conn(0x7fd6780cd120 msgr2=0x7fd6780c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 -- 172.21.15.46:0/3350052613 shutdown_connections 2023-10-06T11:35:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.829+0000 7fd67e0f0700 1 -- 172.21.15.46:0/3350052613 wait complete. 2023-10-06T11:35:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 Processor -- start 2023-10-06T11:35:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 -- start start 2023-10-06T11:35:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd67806d440 0x7fd6781549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 0x7fd6780c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678154520 con 0x7fd678156dc0 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd678153950 con 0x7fd67806d440 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd67e0f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd678153bf0 con 0x7fd6780676a0 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd677fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 0x7fd6780c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd6777fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd66ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd67806d440 0x7fd6781549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd6777fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:49970/0 (socket says 172.21.15.46:49970) 2023-10-06T11:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd6777fe700 1 -- 172.21.15.46:0/939961005 learned_addr learned my addr 172.21.15.46:0/939961005 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.833+0000 7fd677fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 0x7fd6780c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34848/0 (socket says 172.21.15.46:34848) 2023-10-06T11:35:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 -- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd67806d440 msgr2=0x7fd6781549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd67806d440 0x7fd6781549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 -- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 msgr2=0x7fd6780c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 0x7fd6780c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 -- 172.21.15.46:0/939961005 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009420 con 0x7fd6780676a0 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd6777fe700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd6680093f0 tx=0x7fd668017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66801b960 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66801bae0 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66801bea0 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668009420 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6781541b0 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.837+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67806c680 con 0x7fd6780676a0 2023-10-06T11:35:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.841+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd668023070 con 0x7fd6780676a0 2023-10-06T11:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.841+0000 7fd66f7fe700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 0x7fd658062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.841+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd66807e300 con 0x7fd6780676a0 2023-10-06T11:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.841+0000 7fd66ffff700 1 -- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 msgr2=0x7fd658062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.841+0000 7fd66ffff700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 0x7fd658062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:51.849+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd66804e0d0 con 0x7fd6780676a0 2023-10-06T11:35:51.885 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:51.882+0000 7f486a149000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:35:51.893 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:51.881+0000 7f7ede6e2000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:35:51.972 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:51.961+0000 7f7ede6e2000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:35:52.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.021+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd678153d30 con 0x7fd6780676a0 2023-10-06T11:35:52.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.021+0000 7fd66f7fe700 1 -- 172.21.15.46:0/939961005 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd678153d30 con 0x7fd6780676a0 2023-10-06T11:35:52.034 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:35:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 msgr2=0x7fd658062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 0x7fd658062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 msgr2=0x7fd67814f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd6680093f0 tx=0x7fd668017000 comp rx=0 tx=0).stop 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 shutdown_connections 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd658060790 0x7fd658062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd678156dc0 0x7fd6780c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd67806d440 0x7fd6781549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 --2- 172.21.15.46:0/939961005 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6780676a0 0x7fd67814f200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 >> 172.21.15.46:0/939961005 conn(0x7fd6780cd120 msgr2=0x7fd6780adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 shutdown_connections 2023-10-06T11:35:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.025+0000 7fd67e0f0700 1 -- 172.21.15.46:0/939961005 wait complete. 2023-10-06T11:35:52.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:35:52.102 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:52.102+0000 7f486a149000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:35:52.188 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:52.186+0000 7f486a149000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:35:52.202 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:52.189+0000 7f7ede6e2000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:35:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.213+0000 7f386cecd700 1 Processor -- start 2023-10-06T11:35:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.213+0000 7f386cecd700 1 -- start start 2023-10-06T11:35:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386cecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 0x7f38680ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386cecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38680cd080 0x7f38680c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386cecd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38680cc510 con 0x7f38680b23c0 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386cecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38680cd500 con 0x7f38680cd080 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386cecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680b2be0 con 0x7f38680b2790 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386659c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38680b23c0 0x7f38680cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53450/0 (socket says 172.21.15.46:53450) 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386659c700 1 -- 172.21.15.46:0/1424007594 learned_addr learned my addr 172.21.15.46:0/1424007594 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 --2- 172.21.15.46:0/1424007594 >> v2:172.21.15.132:3300/0 conn(0x7f38680cd080 0x7f38680c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3865d9b700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 0x7f38680ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386559a700 1 -- 172.21.15.46:0/1424007594 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743289642 0 0) 0x7f38680cc510 con 0x7f38680b23c0 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386559a700 1 -- 172.21.15.46:0/1424007594 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3850003570 con 0x7f38680b23c0 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 -- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 msgr2=0x7f38680ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 0x7f38680ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 -- 172.21.15.46:0/1424007594 >> v1:172.21.15.46:6789/0 conn(0x7f38680b23c0 legacy=0x7f38680cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 -- 172.21.15.46:0/1424007594 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680cb900 con 0x7f38680cd080 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3865d9b700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 0x7f38680ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:52.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f3866d9d700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680cd080 0x7f38680c4fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3860009e60 tx=0x7f3860009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8049caa05806c162 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386559a700 1 -- 172.21.15.46:0/1424007594 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386000e070 con 0x7f38680cd080 2023-10-06T11:35:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386559a700 1 -- 172.21.15.46:0/1424007594 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386000bab0 con 0x7f38680cd080 2023-10-06T11:35:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.217+0000 7f386559a700 1 -- 172.21.15.46:0/1424007594 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386000be70 con 0x7f38680cd080 2023-10-06T11:35:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680cd080 msgr2=0x7f38680c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680cd080 0x7f38680c4fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3860009e60 tx=0x7f3860009780 comp rx=0 tx=0).stop 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- 172.21.15.46:0/1424007594 shutdown_connections 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680cd080 0x7f38680c4fe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 --2- 172.21.15.46:0/1424007594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b2790 0x7f38680ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- 172.21.15.46:0/1424007594 >> 172.21.15.46:0/1424007594 conn(0x7f3868099480 msgr2=0x7f386806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- 172.21.15.46:0/1424007594 shutdown_connections 2023-10-06T11:35:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- 172.21.15.46:0/1424007594 wait complete. 2023-10-06T11:35:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 Processor -- start 2023-10-06T11:35:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.221+0000 7f386cecd700 1 -- start start 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b23c0 0x7f38680c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680b2790 0x7f38681575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386815a6a0 con 0x7f38680cd080 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386815ab30 con 0x7f38680b2790 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3868159a90 con 0x7f38680b23c0 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34860/0 (socket says 172.21.15.46:34860) 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 -- 172.21.15.46:0/2778993221 learned_addr learned my addr 172.21.15.46:0/2778993221 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:52.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3865d9b700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680b2790 0x7f38681575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386659c700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b23c0 0x7f38680c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 -- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b23c0 msgr2=0x7f38680c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b23c0 0x7f38680c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 -- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680b2790 msgr2=0x7f38681575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680b2790 0x7f38681575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 -- 172.21.15.46:0/2778993221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c009420 con 0x7f38680cd080 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f3866d9d700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f38600099f0 tx=0x7f3860019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860012e50 con 0x7f38680cd080 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860027470 con 0x7f38680cd080 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860009a20 con 0x7f38680cd080 2023-10-06T11:35:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860027830 con 0x7f38680cd080 2023-10-06T11:35:52.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.225+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386815a050 con 0x7f38680cd080 2023-10-06T11:35:52.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386815a810 con 0x7f38680cd080 2023-10-06T11:35:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f3860024070 con 0x7f38680cd080 2023-10-06T11:35:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f38567fc700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 0x7f3848062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f386007f6c0 con 0x7f38680cd080 2023-10-06T11:35:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f386659c700 1 -- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 msgr2=0x7f3848062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.229+0000 7f386659c700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 0x7f3848062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.237+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f386004f490 con 0x7f38680cd080 2023-10-06T11:35:52.274 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:52.270+0000 7f486a149000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:35:52.290 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:52.277+0000 7f7ede6e2000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:35:52.384 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:52.369+0000 7f7ede6e2000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:35:52.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.409+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38680660f0 con 0x7f38680cd080 2023-10-06T11:35:52.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.413+0000 7f38567fc700 1 -- 172.21.15.46:0/2778993221 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v110) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3860052ab0 con 0x7f38680cd080 2023-10-06T11:35:52.425 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:35:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.413+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 msgr2=0x7f3848062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.413+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 0x7f3848062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.413+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 msgr2=0x7f3868157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.413+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f38600099f0 tx=0x7f3860019000 comp rx=0 tx=0).stop 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 shutdown_connections 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3848060790 0x7f3848062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38680cd080 0x7f3868157b00 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38680b2790 0x7f38681575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 --2- 172.21.15.46:0/2778993221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38680b23c0 0x7f38680c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 >> 172.21.15.46:0/2778993221 conn(0x7f3868099480 msgr2=0x7f3868067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 shutdown_connections 2023-10-06T11:35:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.417+0000 7f386cecd700 1 -- 172.21.15.46:0/2778993221 wait complete. 2023-10-06T11:35:52.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:35:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 Processor -- start 2023-10-06T11:35:52.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 -- start start 2023-10-06T11:35:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffab4069c10 0x7ffab40af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 0x7ffab40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40b1b60 con 0x7ffab4068160 2023-10-06T11:35:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab41597e0 con 0x7ffab4069c10 2023-10-06T11:35:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.601+0000 7ffabb1c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab415a5c0 con 0x7ffab40b1710 2023-10-06T11:35:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffab4069c10 0x7ffab40af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab3fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ffab4068160 0x7ffab40b1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:60364/0 (socket says 172.21.15.46:60364) 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab3fff700 1 -- 172.21.15.46:0/3396069650 learned_addr learned my addr 172.21.15.46:0/3396069650 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab9762700 1 --2- 172.21.15.46:0/3396069650 >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 0x7ffab40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 -- 172.21.15.46:0/3396069650 >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 msgr2=0x7ffab40aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab37fe700 1 -- 172.21.15.46:0/3396069650 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319986419 0 0) 0x7ffab40b1b60 con 0x7ffab4068160 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 --2- 172.21.15.46:0/3396069650 >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 0x7ffab40aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 -- 172.21.15.46:0/3396069650 >> v1:172.21.15.132:6789/0 conn(0x7ffab4068160 legacy=0x7ffab40b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 -- 172.21.15.46:0/3396069650 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab41599f0 con 0x7ffab4069c10 2023-10-06T11:35:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab9762700 1 --2- 172.21.15.46:0/3396069650 >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 0x7ffab40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab8f61700 1 --2- 172.21.15.46:0/3396069650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4069c10 0x7ffab40af9e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ffab4159920 tx=0x7ffaa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea420abb8298cf20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab37fe700 1 -- 172.21.15.46:0/3396069650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8011630 con 0x7ffab4069c10 2023-10-06T11:35:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab37fe700 1 -- 172.21.15.46:0/3396069650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8011790 con 0x7ffab4069c10 2023-10-06T11:35:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffab37fe700 1 -- 172.21.15.46:0/3396069650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8011b50 con 0x7ffab4069c10 2023-10-06T11:35:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 -- 172.21.15.46:0/3396069650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4069c10 msgr2=0x7ffab40af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/3396069650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4069c10 0x7ffab40af9e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ffab4159920 tx=0x7ffaa80091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 -- 172.21.15.46:0/3396069650 shutdown_connections 2023-10-06T11:35:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/3396069650 >> v2:172.21.15.132:3300/0 conn(0x7ffab40b1710 0x7ffab40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/3396069650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4069c10 0x7ffab40af9e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 -- 172.21.15.46:0/3396069650 >> 172.21.15.46:0/3396069650 conn(0x7ffab406ae60 msgr2=0x7ffab406b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 -- 172.21.15.46:0/3396069650 shutdown_connections 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.605+0000 7ffabb1c5700 1 -- 172.21.15.46:0/3396069650 wait complete. 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 Processor -- start 2023-10-06T11:35:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 -- start start 2023-10-06T11:35:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffab4069c10 0x7ffab413b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 0x7ffab40ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab413ced0 con 0x7ffab4068160 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab413d1b0 con 0x7ffab40b1710 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.609+0000 7ffabb1c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab413d490 con 0x7ffab4069c10 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34874/0 (socket says 172.21.15.46:34874) 2023-10-06T11:35:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 -- 172.21.15.46:0/134437715 learned_addr learned my addr 172.21.15.46:0/134437715 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab9762700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 0x7ffab40ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab3fff700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffab4069c10 0x7ffab413b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 -- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffab4069c10 msgr2=0x7ffab413b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffab4069c10 0x7ffab413b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 -- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 msgr2=0x7ffab40ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 0x7ffab40ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 -- 172.21.15.46:0/134437715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009a20 con 0x7ffab4068160 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab9762700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 0x7ffab40ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab8f61700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8013000 tx=0x7ffaa8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa801adb0 con 0x7ffab4068160 2023-10-06T11:35:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa801f070 con 0x7ffab4068160 2023-10-06T11:35:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa801b6a0 con 0x7ffab4068160 2023-10-06T11:35:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa8009420 con 0x7ffab4068160 2023-10-06T11:35:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.613+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab413f680 con 0x7ffab4068160 2023-10-06T11:35:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab406da40 con 0x7ffab4068160 2023-10-06T11:35:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7ffaa801ba50 con 0x7ffab4068160 2023-10-06T11:35:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffab0ff9700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 0x7ffa98062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7ffaa807e680 con 0x7ffab4068160 2023-10-06T11:35:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffab3fff700 1 -- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 msgr2=0x7ffa98062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.617+0000 7ffab3fff700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 0x7ffa98062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.625+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffaa804e510 con 0x7ffab4068160 2023-10-06T11:35:52.657 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:52.654+0000 7f486a149000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:35:52.804 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:52.789+0000 7f7ede6e2000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:35:52.810 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:52.810+0000 7f486a149000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:35:52.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.801+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffab4154e60 con 0x7ffab4068160 2023-10-06T11:35:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.805+0000 7ffab0ff9700 1 -- 172.21.15.46:0/134437715 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffaa8051b30 con 0x7ffab4068160 2023-10-06T11:35:52.816 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:35:52.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 msgr2=0x7ffa98062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:52.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 0x7ffa98062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 msgr2=0x7ffab413f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8013000 tx=0x7ffaa8016000 comp rx=0 tx=0).stop 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 shutdown_connections 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa98060790 0x7ffa98062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffab40b1710 0x7ffab40ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffab4069c10 0x7ffab413b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 --2- 172.21.15.46:0/134437715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffab4068160 0x7ffab413f140 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 >> 172.21.15.46:0/134437715 conn(0x7ffab406ae60 msgr2=0x7ffab406d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 shutdown_connections 2023-10-06T11:35:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.809+0000 7ffabb1c5700 1 -- 172.21.15.46:0/134437715 wait complete. 2023-10-06T11:35:52.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:35:52.980 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:52.965+0000 7f7ede6e2000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:35:52.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.985+0000 7fc5540ae700 1 Processor -- start 2023-10-06T11:35:52.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.985+0000 7fc5540ae700 1 -- start start 2023-10-06T11:35:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc5540ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc54c159090 0x7fc54c062d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc5540ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc54c0c6070 0x7fc54c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc5540ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0c8940 con 0x7fc54c0678f0 2023-10-06T11:35:53.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc5540ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0c8140 con 0x7fc54c159090 2023-10-06T11:35:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc5540ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0c8330 con 0x7fc54c0c6070 2023-10-06T11:35:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc551e4a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc54c159090 0x7fc54c062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc54c0c6070 0x7fc54c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc551649700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc54c0678f0 0x7fc54c0c5960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:60378/0 (socket says 172.21.15.46:60378) 2023-10-06T11:35:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc551649700 1 -- 172.21.15.46:0/2353881259 learned_addr learned my addr 172.21.15.46:0/2353881259 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 -- 172.21.15.46:0/2353881259 >> v2:172.21.15.46:3300/0 conn(0x7fc54c159090 msgr2=0x7fc54c062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 --2- 172.21.15.46:0/2353881259 >> v2:172.21.15.46:3300/0 conn(0x7fc54c159090 0x7fc54c062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 -- 172.21.15.46:0/2353881259 >> v1:172.21.15.132:6789/0 conn(0x7fc54c0678f0 legacy=0x7fc54c0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc550e48700 1 -- 172.21.15.46:0/2353881259 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490210744 0 0) 0x7fc54c0c8940 con 0x7fc54c0678f0 2023-10-06T11:35:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 -- 172.21.15.46:0/2353881259 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0680b0 con 0x7fc54c0c6070 2023-10-06T11:35:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.989+0000 7fc55264b700 1 --2- 172.21.15.46:0/2353881259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c6070 0x7fc54c06d440 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc54c06a9d0 tx=0x7fc544009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89653ae68469529b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc550e48700 1 -- 172.21.15.46:0/2353881259 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400e070 con 0x7fc54c0c6070 2023-10-06T11:35:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc550e48700 1 -- 172.21.15.46:0/2353881259 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400bb60 con 0x7fc54c0c6070 2023-10-06T11:35:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc550e48700 1 -- 172.21.15.46:0/2353881259 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544012430 con 0x7fc54c0c6070 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 -- 172.21.15.46:0/2353881259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c6070 msgr2=0x7fc54c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 --2- 172.21.15.46:0/2353881259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c6070 0x7fc54c06d440 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc54c06a9d0 tx=0x7fc544009780 comp rx=0 tx=0).stop 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 -- 172.21.15.46:0/2353881259 shutdown_connections 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 --2- 172.21.15.46:0/2353881259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c0c6070 0x7fc54c06d440 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 --2- 172.21.15.46:0/2353881259 >> v2:172.21.15.46:3300/0 conn(0x7fc54c159090 0x7fc54c062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 -- 172.21.15.46:0/2353881259 >> 172.21.15.46:0/2353881259 conn(0x7fc54c0cd120 msgr2=0x7fc54c0c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 -- 172.21.15.46:0/2353881259 shutdown_connections 2023-10-06T11:35:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.993+0000 7fc5540ae700 1 -- 172.21.15.46:0/2353881259 wait complete. 2023-10-06T11:35:53.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 Processor -- start 2023-10-06T11:35:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 -- start start 2023-10-06T11:35:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0678f0 0x7fc54c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c159090 0x7fc54c0a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0af0e0 con 0x7fc54c0c6070 2023-10-06T11:35:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c169380 con 0x7fc54c159090 2023-10-06T11:35:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc5540ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c15b940 con 0x7fc54c0678f0 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34878/0 (socket says 172.21.15.46:34878) 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 -- 172.21.15.46:0/3397133426 learned_addr learned my addr 172.21.15.46:0/3397133426 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551e4a700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0678f0 0x7fc54c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc55264b700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c159090 0x7fc54c0a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 -- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0678f0 msgr2=0x7fc54c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0678f0 0x7fc54c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 -- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c159090 msgr2=0x7fc54c0a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c159090 0x7fc54c0a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 -- 172.21.15.46:0/3397133426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544009a20 con 0x7fc54c0c6070 2023-10-06T11:35:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:52.997+0000 7fc551649700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c810 tx=0x7fc53c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005c00 con 0x7fc54c0c6070 2023-10-06T11:35:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c005d60 con 0x7fc54c0c6070 2023-10-06T11:35:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c007320 con 0x7fc54c0c6070 2023-10-06T11:35:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c0aebb0 con 0x7fc54c0c6070 2023-10-06T11:35:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c0a65e0 con 0x7fc54c0c6070 2023-10-06T11:35:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c15bab0 con 0x7fc54c0c6070 2023-10-06T11:35:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.001+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fc53c01b400 con 0x7fc54c0c6070 2023-10-06T11:35:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.005+0000 7fc541ffb700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 0x7fc530062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.005+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc53c07cb50 con 0x7fc54c0c6070 2023-10-06T11:35:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.005+0000 7fc551e4a700 1 -- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 msgr2=0x7fc530062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:53.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.005+0000 7fc551e4a700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 0x7fc530062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.013+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc54c15bab0 con 0x7fc54c0c6070 2023-10-06T11:35:53.149 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.146+0000 7f486a149000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:35:53.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.189+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc54c15bab0 con 0x7fc54c0c6070 2023-10-06T11:35:53.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc541ffb700 1 -- 172.21.15.46:0/3397133426 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc54c15bab0 con 0x7fc54c0c6070 2023-10-06T11:35:53.205 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:35:53.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 msgr2=0x7fc530062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 0x7fc530062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 msgr2=0x7fc54c1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c810 tx=0x7fc53c007db0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 shutdown_connections 2023-10-06T11:35:53.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc530060790 0x7fc530062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc54c159090 0x7fc54c0a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c0c6070 0x7fc54c1574c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 --2- 172.21.15.46:0/3397133426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc54c0678f0 0x7fc54c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.193+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 >> 172.21.15.46:0/3397133426 conn(0x7fc54c0cd120 msgr2=0x7fc54c0c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.197+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 shutdown_connections 2023-10-06T11:35:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.197+0000 7fc5540ae700 1 -- 172.21.15.46:0/3397133426 wait complete. 2023-10-06T11:35:53.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:35:53.363 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.362+0000 7f486a149000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:35:53.380 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:53.365+0000 7f7ede6e2000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:35:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.381+0000 7fe45f42e700 1 Processor -- start 2023-10-06T11:35:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.381+0000 7fe45f42e700 1 -- start start 2023-10-06T11:35:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45f42e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe45806cc30 0x7fe45806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45f42e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580c18f0 con 0x7fe458161840 2023-10-06T11:35:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45f42e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580cb120 con 0x7fe458161470 2023-10-06T11:35:53.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45f42e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580c0ce0 con 0x7fe45806cc30 2023-10-06T11:35:53.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45c9c9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe458161840 0x7fe4580caa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58048/0 (socket says 172.21.15.46:58048) 2023-10-06T11:35:53.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45c9c9700 1 -- 172.21.15.46:0/1015978414 learned_addr learned my addr 172.21.15.46:0/1015978414 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:53.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe45d9cb700 1 --2- 172.21.15.46:0/1015978414 >> v2:172.21.15.132:3300/0 conn(0x7fe45806cc30 0x7fe45806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641301962 0 0) 0x7fe4580c18f0 con 0x7fe458161840 2023-10-06T11:35:53.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe444003570 con 0x7fe458161840 2023-10-06T11:35:53.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 683342985 0 0) 0x7fe444003570 con 0x7fe458161840 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 >> v2:172.21.15.132:3300/0 conn(0x7fe45806cc30 msgr2=0x7fe45806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 --2- 172.21.15.46:0/1015978414 >> v2:172.21.15.132:3300/0 conn(0x7fe45806cc30 0x7fe45806d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 >> v1:172.21.15.132:6789/0 conn(0x7fe458161470 legacy=0x7fe4580c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580c0fc0 con 0x7fe458161840 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.385+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fe448001470 con 0x7fe458161840 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 >> v1:172.21.15.46:6789/0 conn(0x7fe458161840 legacy=0x7fe4580caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 0x7fe444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4440051c0 0x7fe44400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580c0ce0 con 0x7fe4440051c0 2023-10-06T11:35:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44400c0d0 con 0x7fe444004810 2023-10-06T11:35:53.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44400c290 con 0x7fe444003a60 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 0x7fe444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45d1ca700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45d9cb700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4440051c0 0x7fe44400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 -- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe444003a60 msgr2=0x7fe444003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe444003a60 0x7fe444003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 -- 172.21.15.46:0/1015978414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4440051c0 msgr2=0x7fe44400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4440051c0 0x7fe44400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 -- 172.21.15.46:0/1015978414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe444003570 con 0x7fe444004810 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45d9cb700 1 -- 172.21.15.46:0/1015978414 reap_dead start 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 0x7fe444004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c3c0 tx=0x7fe448001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f47c65e8542b8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800cc10 con 0x7fe444004810 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44800cd70 con 0x7fe444004810 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe44ffff700 1 -- 172.21.15.46:0/1015978414 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448005b50 con 0x7fe444004810 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 -- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 msgr2=0x7fe444004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 0x7fe444004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c3c0 tx=0x7fe448001e50 comp rx=0 tx=0).stop 2023-10-06T11:35:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 -- 172.21.15.46:0/1015978414 shutdown_connections 2023-10-06T11:35:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 --2- 172.21.15.46:0/1015978414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe444004810 0x7fe444004c80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 -- 172.21.15.46:0/1015978414 >> 172.21.15.46:0/1015978414 conn(0x7fe4580cbe30 msgr2=0x7fe4580cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 -- 172.21.15.46:0/1015978414 shutdown_connections 2023-10-06T11:35:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.389+0000 7fe45f42e700 1 -- 172.21.15.46:0/1015978414 wait complete. 2023-10-06T11:35:53.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 Processor -- start 2023-10-06T11:35:53.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 -- start start 2023-10-06T11:35:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe458171600 0x7fe45816d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe45816dbf0 0x7fe4580c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815a660 con 0x7fe444004810 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815ddf0 con 0x7fe458171600 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45f42e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815d1e0 con 0x7fe45816dbf0 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45d1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45d1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48530/0 (socket says 172.21.15.46:48530) 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45d1ca700 1 -- 172.21.15.46:0/162440496 learned_addr learned my addr 172.21.15.46:0/162440496 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe458171600 0x7fe45816d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.393+0000 7fe45d9cb700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe45816dbf0 0x7fe4580c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe45816dbf0 msgr2=0x7fe4580c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe45816dbf0 0x7fe4580c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe458171600 msgr2=0x7fe45816d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe458171600 0x7fe45816d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 -- 172.21.15.46:0/162440496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45815d4c0 con 0x7fe444004810 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45d1ca700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe4540029f0 tx=0x7fe454007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45400f670 con 0x7fe444004810 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45400f7d0 con 0x7fe444004810 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45815d7a0 con 0x7fe444004810 2023-10-06T11:35:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45400fb50 con 0x7fe444004810 2023-10-06T11:35:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45815ce70 con 0x7fe444004810 2023-10-06T11:35:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.397+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458063f90 con 0x7fe444004810 2023-10-06T11:35:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.401+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fe45401c400 con 0x7fe444004810 2023-10-06T11:35:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.401+0000 7fe44d7fa700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 0x7fe43c062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.401+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe45407d420 con 0x7fe444004810 2023-10-06T11:35:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.401+0000 7fe45c9c9700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 msgr2=0x7fe43c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.401+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 0x7fe43c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:53.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.405+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe45404d340 con 0x7fe444004810 2023-10-06T11:35:53.450 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.446+0000 7f486a149000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:35:53.521 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.518+0000 7f486a149000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:35:53.597 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.594+0000 7f486a149000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:35:53.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.605+0000 7fe45c9c9700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 msgr2=0x7fe43c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:53.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.605+0000 7fe45c9c9700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 0x7fe43c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:35:53.628 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:53.613+0000 7f7ede6e2000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:35:53.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.617+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe45815dc20 con 0x7fe444004810 2023-10-06T11:35:53.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe44d7fa700 1 -- 172.21.15.46:0/162440496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7fe454050960 con 0x7fe444004810 2023-10-06T11:35:53.632 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:53.633 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:35:53.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 msgr2=0x7fe43c062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:53.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 0x7fe43c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 msgr2=0x7fe45815a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe4540029f0 tx=0x7fe454007d80 comp rx=0 tx=0).stop 2023-10-06T11:35:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 shutdown_connections 2023-10-06T11:35:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe43c060790 0x7fe43c062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe45816dbf0 0x7fe4580c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe458171600 0x7fe45816d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 --2- 172.21.15.46:0/162440496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe444004810 0x7fe45815a120 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 >> 172.21.15.46:0/162440496 conn(0x7fe4580cbe30 msgr2=0x7fe45815b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:53.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 shutdown_connections 2023-10-06T11:35:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.621+0000 7fe45f42e700 1 -- 172.21.15.46:0/162440496 wait complete. 2023-10-06T11:35:53.639 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:35:53.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:35:53.727 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:53.713+0000 7f7ede6e2000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:35:53.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.797+0000 7efc55ee3700 1 Processor -- start 2023-10-06T11:35:53.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.797+0000 7efc55ee3700 1 -- start start 2023-10-06T11:35:53.810 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:53.797+0000 7f7ede6e2000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:35:53.812 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.810+0000 7f486a149000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:35:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc55ee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc50159290 0x7efc500b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc55ee3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc500ae340 con 0x7efc500b1300 2023-10-06T11:35:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc55ee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc500b09a0 con 0x7efc500d07b0 2023-10-06T11:35:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc55ee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50159660 con 0x7efc50159290 2023-10-06T11:35:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc50159290 0x7efc500b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc50159290 0x7efc500b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48536/0 (socket says 172.21.15.46:48536) 2023-10-06T11:35:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 -- 172.21.15.46:0/2866711592 learned_addr learned my addr 172.21.15.46:0/2866711592 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 -- 172.21.15.46:0/2866711592 >> v1:172.21.15.132:6789/0 conn(0x7efc500d07b0 legacy=0x7efc500b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 -- 172.21.15.46:0/2866711592 >> v1:172.21.15.46:6789/0 conn(0x7efc500b1300 legacy=0x7efc500ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 -- 172.21.15.46:0/2866711592 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500b3450 con 0x7efc50159290 2023-10-06T11:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc4f7fe700 1 -- 172.21.15.46:0/2866711592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021718834 0 0) 0x7efc500ae340 con 0x7efc500b1300 2023-10-06T11:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.801+0000 7efc54ee1700 1 --2- 172.21.15.46:0/2866711592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc500b2f10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efc440099b0 tx=0x7efc44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67ace957738efac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc4f7fe700 1 -- 172.21.15.46:0/2866711592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400b8f0 con 0x7efc50159290 2023-10-06T11:35:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc4f7fe700 1 -- 172.21.15.46:0/2866711592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4400ba50 con 0x7efc50159290 2023-10-06T11:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc4f7fe700 1 -- 172.21.15.46:0/2866711592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400bdf0 con 0x7efc50159290 2023-10-06T11:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 -- 172.21.15.46:0/2866711592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 msgr2=0x7efc500b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 --2- 172.21.15.46:0/2866711592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc500b2f10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7efc440099b0 tx=0x7efc44009250 comp rx=0 tx=0).stop 2023-10-06T11:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 -- 172.21.15.46:0/2866711592 shutdown_connections 2023-10-06T11:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 --2- 172.21.15.46:0/2866711592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc500b2f10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 -- 172.21.15.46:0/2866711592 >> 172.21.15.46:0/2866711592 conn(0x7efc5006ef60 msgr2=0x7efc500d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 -- 172.21.15.46:0/2866711592 shutdown_connections 2023-10-06T11:35:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 -- 172.21.15.46:0/2866711592 wait complete. 2023-10-06T11:35:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.805+0000 7efc55ee3700 1 Processor -- start 2023-10-06T11:35:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 -- start start 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc500b1300 0x7efc5014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc500d07b0 0x7efc5014f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50169f40 con 0x7efc50159290 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501518d0 con 0x7efc500d07b0 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50169650 con 0x7efc500b1300 2023-10-06T11:35:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48550/0 (socket says 172.21.15.46:48550) 2023-10-06T11:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 -- 172.21.15.46:0/3296220177 learned_addr learned my addr 172.21.15.46:0/3296220177 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc54ee1700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc500b1300 0x7efc5014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc4ffff700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc500d07b0 0x7efc5014f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 -- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc500b1300 msgr2=0x7efc5014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc500b1300 0x7efc5014f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 -- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc500d07b0 msgr2=0x7efc5014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc500d07b0 0x7efc5014f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 -- 172.21.15.46:0/3296220177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440094f0 con 0x7efc50159290 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc556e2700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efc48001560 tx=0x7efc480016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48012630 con 0x7efc50159290 2023-10-06T11:35:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc48012790 con 0x7efc50159290 2023-10-06T11:35:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5016a3d0 con 0x7efc50159290 2023-10-06T11:35:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.809+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48012b50 con 0x7efc50159290 2023-10-06T11:35:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.813+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50169bd0 con 0x7efc50159290 2023-10-06T11:35:53.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.813+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5016a0b0 con 0x7efc50159290 2023-10-06T11:35:53.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.813+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7efc48003030 con 0x7efc50159290 2023-10-06T11:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.817+0000 7efc4cff9700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 0x7efc34062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.817+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7efc4807cff0 con 0x7efc50159290 2023-10-06T11:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.817+0000 7efc54ee1700 1 -- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 msgr2=0x7efc34062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:53.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.817+0000 7efc54ee1700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 0x7efc34062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.821+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efc4804cdc0 con 0x7efc50159290 2023-10-06T11:35:53.887 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.886+0000 7f486a149000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:35:53.901 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:53.889+0000 7f7ede6e2000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:35:53.974 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:53.970+0000 7f486a149000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:35:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.997+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc50064590 con 0x7efc50159290 2023-10-06T11:35:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:53.997+0000 7efc4cff9700 1 -- 172.21.15.46:0/3296220177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18233 (secure 0 0 0) 0x7efc480503e0 con 0x7efc50159290 2023-10-06T11:35:54.012 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:54.012 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:35:49.555711+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/606963126":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/321844900":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/1090757770":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/3400023669":"2023-10-07T11:34:47.548766+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/194565737":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/2862702378":"2023-10-07T11:34:47.548766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:35:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.001+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 msgr2=0x7efc34062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.001+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 0x7efc34062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.001+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 msgr2=0x7efc5015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.001+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7efc48001560 tx=0x7efc480016c0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 shutdown_connections 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efc34060790 0x7efc34062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc50159290 0x7efc5015b8c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc500d07b0 0x7efc5014f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 --2- 172.21.15.46:0/3296220177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc500b1300 0x7efc5014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 >> 172.21.15.46:0/3296220177 conn(0x7efc5006ef60 msgr2=0x7efc5014dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 shutdown_connections 2023-10-06T11:35:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.005+0000 7efc55ee3700 1 -- 172.21.15.46:0/3296220177 wait complete. 2023-10-06T11:35:54.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:35:54.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:35:54.142 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:54.129+0000 7f7ede6e2000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:35:54.157 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:54.154+0000 7f486a149000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:35:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.189+0000 7fd8c582e700 1 Processor -- start 2023-10-06T11:35:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.189+0000 7fd8c582e700 1 -- start start 2023-10-06T11:35:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8c582e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c015ab60 0x7fd8c006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8c582e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c006d460 con 0x7fd8c015af30 2023-10-06T11:35:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8c582e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c00c7000 con 0x7fd8c00ad6c0 2023-10-06T11:35:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8c582e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00c72c0 con 0x7fd8c015ab60 2023-10-06T11:35:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8b67fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8c015af30 0x7fd8c006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58076/0 (socket says 172.21.15.46:58076) 2023-10-06T11:35:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8b67fc700 1 -- 172.21.15.46:0/1129178431 learned_addr learned my addr 172.21.15.46:0/1129178431 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8beffd700 1 --2- 172.21.15.46:0/1129178431 >> v2:172.21.15.46:3300/0 conn(0x7fd8c015ab60 0x7fd8c006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8be7fc700 1 -- 172.21.15.46:0/1129178431 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216973783 0 0) 0x7fd8c006d460 con 0x7fd8c015af30 2023-10-06T11:35:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8beffd700 1 -- 172.21.15.46:0/1129178431 >> v1:172.21.15.132:6789/0 conn(0x7fd8c00ad6c0 legacy=0x7fd8c00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8beffd700 1 -- 172.21.15.46:0/1129178431 >> v1:172.21.15.46:6789/0 conn(0x7fd8c015af30 legacy=0x7fd8c006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8beffd700 1 -- 172.21.15.46:0/1129178431 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00c75a0 con 0x7fd8c015ab60 2023-10-06T11:35:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8beffd700 1 --2- 172.21.15.46:0/1129178431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c006e8a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00c71f0 tx=0x7fd8b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eafd0a5b6995e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8be7fc700 1 -- 172.21.15.46:0/1129178431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000b6b0 con 0x7fd8c015ab60 2023-10-06T11:35:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8be7fc700 1 -- 172.21.15.46:0/1129178431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b000b810 con 0x7fd8c015ab60 2023-10-06T11:35:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.193+0000 7fd8be7fc700 1 -- 172.21.15.46:0/1129178431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000bbd0 con 0x7fd8c015ab60 2023-10-06T11:35:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- 172.21.15.46:0/1129178431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 msgr2=0x7fd8c006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 --2- 172.21.15.46:0/1129178431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c006e8a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00c71f0 tx=0x7fd8b00091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- 172.21.15.46:0/1129178431 shutdown_connections 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 --2- 172.21.15.46:0/1129178431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c006e8a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- 172.21.15.46:0/1129178431 >> 172.21.15.46:0/1129178431 conn(0x7fd8c00cbe30 msgr2=0x7fd8c00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- 172.21.15.46:0/1129178431 shutdown_connections 2023-10-06T11:35:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- 172.21.15.46:0/1129178431 wait complete. 2023-10-06T11:35:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 Processor -- start 2023-10-06T11:35:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.197+0000 7fd8c582e700 1 -- start start 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00ad6c0 0x7fd8c00aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 0x7fd8c0069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00afe50 con 0x7fd8c015ab60 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00af240 con 0x7fd8c015af30 2023-10-06T11:35:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00af520 con 0x7fd8c00ad6c0 2023-10-06T11:35:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48572/0 (socket says 172.21.15.46:48572) 2023-10-06T11:35:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 -- 172.21.15.46:0/792073865 learned_addr learned my addr 172.21.15.46:0/792073865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8beffd700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00ad6c0 0x7fd8c00aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8bf7fe700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 0x7fd8c0069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00ad6c0 msgr2=0x7fd8c00aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00ad6c0 0x7fd8c00aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 msgr2=0x7fd8c0069bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 0x7fd8c0069bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 -- 172.21.15.46:0/792073865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00093c0 con 0x7fd8c015ab60 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8bf7fe700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 0x7fd8c0069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b67fc700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7e0 tx=0x7fd8a8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005b30 con 0x7fd8c015ab60 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8005c90 con 0x7fd8c015ab60 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c00af800 con 0x7fd8c015ab60 2023-10-06T11:35:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007300 con 0x7fd8c015ab60 2023-10-06T11:35:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.201+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00b19d0 con 0x7fd8c015ab60 2023-10-06T11:35:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.205+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c00b0000 con 0x7fd8c015ab60 2023-10-06T11:35:54.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.205+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd8a80076b0 con 0x7fd8c015ab60 2023-10-06T11:35:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.209+0000 7fd8b77fe700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 0x7fd8a0062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.209+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd8a807c6f0 con 0x7fd8c015ab60 2023-10-06T11:35:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.209+0000 7fd8beffd700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 msgr2=0x7fd8a0062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:54.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.209+0000 7fd8beffd700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 0x7fd8a0062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:54.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.209+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8c00b0000 con 0x7fd8c015ab60 2023-10-06T11:35:54.233 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:54.221+0000 7f7ede6e2000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:35:54.238 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:54.234+0000 7f486a149000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:35:54.359 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:54.345+0000 7f7ede6e2000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:35:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.413+0000 7fd8beffd700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 msgr2=0x7fd8a0062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.413+0000 7fd8beffd700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 0x7fd8a0062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:35:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.453+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c00b0000 con 0x7fd8c015ab60 2023-10-06T11:35:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.453+0000 7fd8b77fe700 1 -- 172.21.15.46:0/792073865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7fd8c00b0000 con 0x7fd8c015ab60 2023-10-06T11:35:54.467 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:54.467 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:35:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 msgr2=0x7fd8a0062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 0x7fd8a0062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 msgr2=0x7fd8c01508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7e0 tx=0x7fd8a8007df0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 shutdown_connections 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8a0060790 0x7fd8a0062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c015af30 0x7fd8c0069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c015ab60 0x7fd8c01508f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 --2- 172.21.15.46:0/792073865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8c00ad6c0 0x7fd8c00aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 >> 172.21.15.46:0/792073865 conn(0x7fd8c00cbe30 msgr2=0x7fd8c00c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 shutdown_connections 2023-10-06T11:35:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.457+0000 7fd8c582e700 1 -- 172.21.15.46:0/792073865 wait complete. 2023-10-06T11:35:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:35:54.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:35:54.573 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:54.561+0000 7f7ede6e2000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:35:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.633+0000 7f1588cf0700 1 Processor -- start 2023-10-06T11:35:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.633+0000 7f1588cf0700 1 -- start start 2023-10-06T11:35:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 0x7f15840c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f15840c0bd0 0x7f15840c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840c8500 con 0x7f1584156c80 2023-10-06T11:35:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840c6470 con 0x7f15840c0bd0 2023-10-06T11:35:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840c7760 con 0x7f15840c7b50 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 0x7f15840c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f158259c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1584156c80 0x7f15840ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:58444/0 (socket says 172.21.15.46:58444) 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f158259c700 1 -- 172.21.15.46:0/885582478 learned_addr learned my addr 172.21.15.46:0/885582478 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1582d9d700 1 --2- 172.21.15.46:0/885582478 >> v2:172.21.15.46:3300/0 conn(0x7f15840c0bd0 0x7f15840c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1581d9b700 1 -- 172.21.15.46:0/885582478 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604940905 0 0) 0x7f15840c8500 con 0x7f1584156c80 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1581d9b700 1 -- 172.21.15.46:0/885582478 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1570003570 con 0x7f1584156c80 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 -- 172.21.15.46:0/885582478 >> v2:172.21.15.46:3300/0 conn(0x7f15840c0bd0 msgr2=0x7f15840c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 --2- 172.21.15.46:0/885582478 >> v2:172.21.15.46:3300/0 conn(0x7f15840c0bd0 0x7f15840c1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 -- 172.21.15.46:0/885582478 >> v1:172.21.15.132:6789/0 conn(0x7f1584156c80 legacy=0x7f15840ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 -- 172.21.15.46:0/885582478 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840c66c0 con 0x7f15840c7b50 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1579d9b700 1 --2- 172.21.15.46:0/885582478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 0x7f15840c7fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098e0 tx=0x7f156c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e11d0a75eec2df82 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1581d9b700 1 -- 172.21.15.46:0/885582478 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011630 con 0x7f15840c7b50 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1581d9b700 1 -- 172.21.15.46:0/885582478 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c011790 con 0x7f15840c7b50 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1581d9b700 1 -- 172.21.15.46:0/885582478 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011b30 con 0x7f15840c7b50 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- 172.21.15.46:0/885582478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 msgr2=0x7f15840c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 --2- 172.21.15.46:0/885582478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 0x7f15840c7fc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098e0 tx=0x7f156c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- 172.21.15.46:0/885582478 shutdown_connections 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 --2- 172.21.15.46:0/885582478 >> v2:172.21.15.46:3300/0 conn(0x7f15840c0bd0 0x7f15840c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 --2- 172.21.15.46:0/885582478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c7b50 0x7f15840c7fc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- 172.21.15.46:0/885582478 >> 172.21.15.46:0/885582478 conn(0x7f15840cbe30 msgr2=0x7f15840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- 172.21.15.46:0/885582478 shutdown_connections 2023-10-06T11:35:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.637+0000 7f1588cf0700 1 -- 172.21.15.46:0/885582478 wait complete. 2023-10-06T11:35:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 Processor -- start 2023-10-06T11:35:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 -- start start 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c0bd0 0x7f15840c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 0x7f15840afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b1000 con 0x7f1584156c80 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b03f0 con 0x7f15840c7b50 2023-10-06T11:35:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1588cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b06d0 con 0x7f15840c0bd0 2023-10-06T11:35:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f158259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c0bd0 0x7f15840c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1579d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1582d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 0x7f15840afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1579d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:51314/0 (socket says 172.21.15.46:51314) 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.641+0000 7f1579d9b700 1 -- 172.21.15.46:0/4245598271 learned_addr learned my addr 172.21.15.46:0/4245598271 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c0bd0 msgr2=0x7f15840c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c0bd0 0x7f15840c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 msgr2=0x7f15840afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 0x7f15840afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 -- 172.21.15.46:0/4245598271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c009420 con 0x7f15840c7b50 2023-10-06T11:35:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1582d9d700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 0x7f15840afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1579d9b700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f15840c78a0 tx=0x7f156c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c000d10 con 0x7f15840c7b50 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c011b10 con 0x7f15840c7b50 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840b09b0 con 0x7f15840c7b50 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c025660 con 0x7f15840c7b50 2023-10-06T11:35:54.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15840b1490 con 0x7f15840c7b50 2023-10-06T11:35:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.645+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15840b0e30 con 0x7f15840c7b50 2023-10-06T11:35:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.649+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f156c025a50 con 0x7f15840c7b50 2023-10-06T11:35:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.649+0000 7f157affd700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 0x7f1564062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.649+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f156c07e870 con 0x7f15840c7b50 2023-10-06T11:35:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.649+0000 7f158259c700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 msgr2=0x7f1564062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:54.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.649+0000 7f158259c700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 0x7f1564062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.657+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f156c04e640 con 0x7f15840c7b50 2023-10-06T11:35:54.671 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:54.657+0000 7f7ede6e2000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:35:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.853+0000 7f158259c700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 msgr2=0x7f1564062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.853+0000 7f158259c700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 0x7f1564062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:35:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.861+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15840b1170 con 0x7f15840c7b50 2023-10-06T11:35:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f157affd700 1 -- 172.21.15.46:0/4245598271 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4593 (secure 0 0 0) 0x7f156c051c60 con 0x7f15840c7b50 2023-10-06T11:35:54.877 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:54.877 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:34:33.064692+0000","modified":"2023-10-06T11:34:36.695988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7147":{"gid":7147,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6834/1726709153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1726709153},{"type":"v1","addr":"172.21.15.132:6834","nonce":1726709153}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T11:35:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 msgr2=0x7f1564062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 0x7f1564062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 msgr2=0x7f158414f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f15840c78a0 tx=0x7f156c017000 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.865+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 shutdown_connections 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1564060790 0x7f1564062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1584156c80 0x7f15840afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15840c7b50 0x7f158414f6b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 --2- 172.21.15.46:0/4245598271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15840c0bd0 0x7f15840c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 >> 172.21.15.46:0/4245598271 conn(0x7f15840cbe30 msgr2=0x7f1584150370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 shutdown_connections 2023-10-06T11:35:54.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:54.869+0000 7f1588cf0700 1 -- 172.21.15.46:0/4245598271 wait complete. 2023-10-06T11:35:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2023-10-06T11:35:54.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:35:54.939 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:35:54.938+0000 7f486a149000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:35:55.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 Processor -- start 2023-10-06T11:35:55.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 -- start start 2023-10-06T11:35:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f137c0d07b0 0x7f137c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 0x7f137c0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0b15b0 con 0x7f137c0d07b0 2023-10-06T11:35:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0ae340 con 0x7f137c0b1130 2023-10-06T11:35:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1382163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b09a0 con 0x7f137c159290 2023-10-06T11:35:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f137c0d07b0 0x7f137c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f137c0d07b0 0x7f137c0b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48604/0 (socket says 172.21.15.46:48604) 2023-10-06T11:35:55.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 -- 172.21.15.46:0/2004205276 learned_addr learned my addr 172.21.15.46:0/2004205276 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 -- 172.21.15.46:0/2004205276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 msgr2=0x7f137c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1381962700 1 --2- 172.21.15.46:0/2004205276 >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 0x7f137c0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1381161700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 -- 172.21.15.46:0/2004205276 >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 msgr2=0x7f137c0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:55.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 --2- 172.21.15.46:0/2004205276 >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 0x7f137c0ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 -- 172.21.15.46:0/2004205276 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c159660 con 0x7f137c0d07b0 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1381962700 1 --2- 172.21.15.46:0/2004205276 >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 0x7f137c0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1381161700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1380960700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c0b0bf0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f137c0cffd0 tx=0x7f136c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bb3d9d60ab29ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1373fff700 1 -- 172.21.15.46:0/2004205276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00b8f0 con 0x7f137c0d07b0 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1373fff700 1 -- 172.21.15.46:0/2004205276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00ba50 con 0x7f137c0d07b0 2023-10-06T11:35:55.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.049+0000 7f1373fff700 1 -- 172.21.15.46:0/2004205276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00bdf0 con 0x7f137c0d07b0 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 -- 172.21.15.46:0/2004205276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 msgr2=0x7f137c0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c0b0bf0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f137c0cffd0 tx=0x7f136c009250 comp rx=0 tx=0).stop 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 -- 172.21.15.46:0/2004205276 shutdown_connections 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 --2- 172.21.15.46:0/2004205276 >> v2:172.21.15.132:3300/0 conn(0x7f137c0b1130 0x7f137c0ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c0b0bf0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 --2- 172.21.15.46:0/2004205276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 -- 172.21.15.46:0/2004205276 >> 172.21.15.46:0/2004205276 conn(0x7f137c06ef60 msgr2=0x7f137c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 -- 172.21.15.46:0/2004205276 shutdown_connections 2023-10-06T11:35:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 -- 172.21.15.46:0/2004205276 wait complete. 2023-10-06T11:35:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.053+0000 7f1382163700 1 Processor -- start 2023-10-06T11:35:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- start start 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f137c0b1130 0x7f137c14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c14b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c159ce0 con 0x7f137c0d07b0 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c159fc0 con 0x7f137c0b1130 2023-10-06T11:35:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c15a2a0 con 0x7f137c159290 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48612/0 (socket says 172.21.15.46:48612) 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 -- 172.21.15.46:0/1109213100 learned_addr learned my addr 172.21.15.46:0/1109213100 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1381962700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c14b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1381161700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f137c0b1130 0x7f137c14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 -- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 msgr2=0x7f137c14b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c14b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 -- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f137c0b1130 msgr2=0x7f137c14abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f137c0b1130 0x7f137c14abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 -- 172.21.15.46:0/1109213100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0094f0 con 0x7f137c0d07b0 2023-10-06T11:35:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1380960700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f137c088920 tx=0x7f136c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011e60 con 0x7f137c0d07b0 2023-10-06T11:35:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c01a630 con 0x7f137c0d07b0 2023-10-06T11:35:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c15a580 con 0x7f137c0d07b0 2023-10-06T11:35:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c01a9d0 con 0x7f137c0d07b0 2023-10-06T11:35:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.057+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c159850 con 0x7f137c0d07b0 2023-10-06T11:35:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.061+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c159aa0 con 0x7f137c0d07b0 2023-10-06T11:35:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.061+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f136c0253f0 con 0x7f137c0d07b0 2023-10-06T11:35:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.061+0000 7f13717fa700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 0x7f1360062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.065+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f136c07ed00 con 0x7f137c0d07b0 2023-10-06T11:35:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.065+0000 7f1381161700 1 -- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 msgr2=0x7f1360062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.065+0000 7f1381161700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 0x7f1360062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:55.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.069+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f137c159aa0 con 0x7f137c0d07b0 2023-10-06T11:35:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.241+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f137c159aa0 con 0x7f137c0d07b0 2023-10-06T11:35:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.241+0000 7f13717fa700 1 -- 172.21.15.46:0/1109213100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18233 (secure 0 0 0) 0x7f137c159aa0 con 0x7f137c0d07b0 2023-10-06T11:35:55.255 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:35:55.255 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:35:49.555711+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:34:30.994864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T11:34:32.078476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/606963126":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/321844900":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/1090757770":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/3400023669":"2023-10-07T11:34:47.548766+0000","172.21.15.46:6833/32974":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/194565737":"2023-10-07T11:34:47.548766+0000","172.21.15.46:0/2862702378":"2023-10-07T11:34:47.548766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 msgr2=0x7f1360062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 0x7f1360062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 msgr2=0x7f137c14f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f137c088920 tx=0x7f136c017000 comp rx=0 tx=0).stop 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 shutdown_connections 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1360060790 0x7f1360062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137c159290 0x7f137c14b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137c0d07b0 0x7f137c14f0c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 --2- 172.21.15.46:0/1109213100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f137c0b1130 0x7f137c14abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 >> 172.21.15.46:0/1109213100 conn(0x7f137c06ef60 msgr2=0x7f137c069370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 shutdown_connections 2023-10-06T11:35:55.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.245+0000 7f1382163700 1 -- 172.21.15.46:0/1109213100 wait complete. 2023-10-06T11:35:55.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:35:55.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.425+0000 7fe04af23700 1 Processor -- start 2023-10-06T11:35:55.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.425+0000 7fe04af23700 1 -- start start 2023-10-06T11:35:55.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe04af23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0440d07b0 0x7fe0440b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe04af23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440ae340 con 0x7fe0440b1130 2023-10-06T11:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe04af23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0440b09a0 con 0x7fe044159290 2023-10-06T11:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe04af23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0440b15b0 con 0x7fe0440d07b0 2023-10-06T11:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0440d07b0 0x7fe0440b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0440d07b0 0x7fe0440b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48622/0 (socket says 172.21.15.46:48622) 2023-10-06T11:35:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 -- 172.21.15.46:0/3022953226 learned_addr learned my addr 172.21.15.46:0/3022953226 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe04a722700 1 --1- 172.21.15.46:0/3022953226 >> v1:172.21.15.46:6789/0 conn(0x7fe0440b1130 0x7fe0440ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58080/0 (socket says 172.21.15.46:58080) 2023-10-06T11:35:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 -- 172.21.15.46:0/3022953226 >> v1:172.21.15.132:6789/0 conn(0x7fe044159290 legacy=0x7fe0440b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 -- 172.21.15.46:0/3022953226 >> v1:172.21.15.46:6789/0 conn(0x7fe0440b1130 legacy=0x7fe0440ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 -- 172.21.15.46:0/3022953226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044159660 con 0x7fe0440d07b0 2023-10-06T11:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe049720700 1 --2- 172.21.15.46:0/3022953226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0440d07b0 0x7fe0440b0bf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099b0 tx=0x7fe034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5256d1bc3a6d710c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe048f1f700 1 -- 172.21.15.46:0/3022953226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400b8f0 con 0x7fe0440d07b0 2023-10-06T11:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe048f1f700 1 -- 172.21.15.46:0/3022953226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400ba50 con 0x7fe0440d07b0 2023-10-06T11:35:55.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.429+0000 7fe048f1f700 1 -- 172.21.15.46:0/3022953226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400bdf0 con 0x7fe0440d07b0 2023-10-06T11:35:55.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- 172.21.15.46:0/3022953226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0440d07b0 msgr2=0x7fe0440b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 --2- 172.21.15.46:0/3022953226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0440d07b0 0x7fe0440b0bf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099b0 tx=0x7fe034009250 comp rx=0 tx=0).stop 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- 172.21.15.46:0/3022953226 shutdown_connections 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 --2- 172.21.15.46:0/3022953226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0440d07b0 0x7fe0440b0bf0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- 172.21.15.46:0/3022953226 >> 172.21.15.46:0/3022953226 conn(0x7fe04406ef60 msgr2=0x7fe0440d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- 172.21.15.46:0/3022953226 shutdown_connections 2023-10-06T11:35:55.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- 172.21.15.46:0/3022953226 wait complete. 2023-10-06T11:35:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 Processor -- start 2023-10-06T11:35:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.433+0000 7fe04af23700 1 -- start start 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 0x7fe04414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0440d07b0 0x7fe044067f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe044151ba0 con 0x7fe044159290 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04414dda0 con 0x7fe0440b1130 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0441527d0 con 0x7fe0440d07b0 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48632/0 (socket says 172.21.15.46:48632) 2023-10-06T11:35:55.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 -- 172.21.15.46:0/1259614 learned_addr learned my addr 172.21.15.46:0/1259614 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe049720700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0440d07b0 0x7fe044067f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe049f21700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 0x7fe04414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 -- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0440d07b0 msgr2=0x7fe044067f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0440d07b0 0x7fe044067f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 -- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 msgr2=0x7fe04414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 0x7fe04414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 -- 172.21.15.46:0/1259614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340094f0 con 0x7fe044159290 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe049f21700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 0x7fe04414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:55.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04a722700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00cd00 tx=0x7fe03c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c005df0 con 0x7fe044159290 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c00a960 con 0x7fe044159290 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe044152ab0 con 0x7fe044159290 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c00ad00 con 0x7fe044159290 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.437+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0441516c0 con 0x7fe044159290 2023-10-06T11:35:55.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044150090 con 0x7fe044159290 2023-10-06T11:35:55.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 50) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fe03c007300 con 0x7fe044159290 2023-10-06T11:35:55.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe039ffb700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 0x7fe028062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fe03c07d190 con 0x7fe044159290 2023-10-06T11:35:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe049f21700 1 -- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 msgr2=0x7fe028062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:35:55.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.441+0000 7fe049f21700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 0x7fe028062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:35:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.449+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe03c04d080 con 0x7fe044159290 2023-10-06T11:35:55.493 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.481+0000 7f7ede6e2000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:35:55.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.497+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 51) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fe03c04d450 con 0x7fe044159290 2023-10-06T11:35:55.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.497+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 msgr2=0x7fe028062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:35:55.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.497+0000 7fe039ffb700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 0x7fe028062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.616 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.601+0000 7f7eaae4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.616 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.601+0000 7f7eaae4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.616 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.601+0000 7f7eaae4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.616 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.601+0000 7f7eaae4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.616 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.601+0000 7f7eaae4a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.620 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.605+0000 7f7ea7e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.620 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.605+0000 7f7ea7e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.620 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.605+0000 7f7ea7e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.620 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.605+0000 7f7ea7e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.620 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:35:55.605+0000 7f7ea7e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:35:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.689+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe044150470 con 0x7fe044159290 2023-10-06T11:35:55.705 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:35:55.693+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:35:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.701+0000 7fe039ffb700 1 -- 172.21.15.46:0/1259614 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe03c078530 con 0x7fe044159290 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 msgr2=0x7fe0441405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00cd00 tx=0x7fe03c001b10 comp rx=0 tx=0).stop 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 shutdown_connections 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe028060790 0x7fe028062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044159290 0x7fe0441405b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0440d07b0 0x7fe044067f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 --2- 172.21.15.46:0/1259614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0440b1130 0x7fe04414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 >> 172.21.15.46:0/1259614 conn(0x7fe04406ef60 msgr2=0x7fe0440b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 shutdown_connections 2023-10-06T11:35:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.705+0000 7fe04af23700 1 -- 172.21.15.46:0/1259614 wait complete. 2023-10-06T11:35:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:35:55.728 INFO:tasks.ceph.mds.d.smithi132.stderr: -6094> 2023-10-06T11:34:37.592+0000 7f0c99fbc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:35:55.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:35:55.776 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:35:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.889+0000 7f4474c6e700 1 Processor -- start 2023-10-06T11:35:55.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.889+0000 7f4474c6e700 1 -- start start 2023-10-06T11:35:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f4474c6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447006e290 0x7f447015a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f4474c6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44700c6470 con 0x7f4470153c60 2023-10-06T11:35:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f4474c6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44700c6750 con 0x7f4470159d90 2023-10-06T11:35:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f4474c6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447015a9c0 con 0x7f447006e290 2023-10-06T11:35:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446dd9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447006e290 0x7f447015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446ed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4470153c60 0x7f44700c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58096/0 (socket says 172.21.15.46:58096) 2023-10-06T11:35:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446dd9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447006e290 0x7f447015a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48744/0 (socket says 172.21.15.46:48744) 2023-10-06T11:35:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446ed9d700 1 -- 172.21.15.46:0/225510447 learned_addr learned my addr 172.21.15.46:0/225510447 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446dd9b700 1 -- 172.21.15.46:0/225510447 >> v1:172.21.15.132:6789/0 conn(0x7f4470159d90 legacy=0x7f4470153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446dd9b700 1 -- 172.21.15.46:0/225510447 >> v1:172.21.15.46:6789/0 conn(0x7f4470153c60 legacy=0x7f44700c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446d59a700 1 -- 172.21.15.46:0/225510447 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210469217 0 0) 0x7f44700c6750 con 0x7f4470159d90 2023-10-06T11:35:55.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.893+0000 7f446dd9b700 1 -- 172.21.15.46:0/225510447 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447006e7f0 con 0x7f447006e290 2023-10-06T11:35:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f446dd9b700 1 --2- 172.21.15.46:0/225510447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447006e290 0x7f447015a160 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f44640098e0 tx=0x7f44640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b4417ecb9f5824 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f446d59a700 1 -- 172.21.15.46:0/225510447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464009e80 con 0x7f447006e290 2023-10-06T11:35:55.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f446d59a700 1 -- 172.21.15.46:0/225510447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400b760 con 0x7f447006e290 2023-10-06T11:35:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f446d59a700 1 -- 172.21.15.46:0/225510447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400bae0 con 0x7f447006e290 2023-10-06T11:35:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 -- 172.21.15.46:0/225510447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447006e290 msgr2=0x7f447015a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 --2- 172.21.15.46:0/225510447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447006e290 0x7f447015a160 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f44640098e0 tx=0x7f44640091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 -- 172.21.15.46:0/225510447 shutdown_connections 2023-10-06T11:35:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 --2- 172.21.15.46:0/225510447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447006e290 0x7f447015a160 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 -- 172.21.15.46:0/225510447 >> 172.21.15.46:0/225510447 conn(0x7f44700cbe30 msgr2=0x7f44700cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 -- 172.21.15.46:0/225510447 shutdown_connections 2023-10-06T11:35:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 -- 172.21.15.46:0/225510447 wait complete. 2023-10-06T11:35:55.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.897+0000 7f4474c6e700 1 Processor -- start 2023-10-06T11:35:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- start start 2023-10-06T11:35:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447006e290 0x7f44700ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 0x7f44700af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44700abef0 con 0x7f4470159d90 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44700ac1d0 con 0x7f4470153c60 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44700ac4b0 con 0x7f447006e290 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48746/0 (socket says 172.21.15.46:48746) 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 -- 172.21.15.46:0/2137463050 learned_addr learned my addr 172.21.15.46:0/2137463050 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446dd9b700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 0x7f44700af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 -- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447006e290 msgr2=0x7f44700ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447006e290 0x7f44700ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 -- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 msgr2=0x7f44700af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 0x7f44700af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 -- 172.21.15.46:0/2137463050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464009420 con 0x7f4470159d90 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446dd9b700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 0x7f44700af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f446ed9d700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f446800c7e0 tx=0x7f4468007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468005b30 con 0x7f4470159d90 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468005c90 con 0x7f4470159d90 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44700c3080 con 0x7f4470159d90 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468007340 con 0x7f4470159d90 2023-10-06T11:35:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.901+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44700af5d0 con 0x7f4470159d90 2023-10-06T11:35:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.905+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44700c2ee0 con 0x7f4470159d90 2023-10-06T11:35:55.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.905+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 51) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f446801b400 con 0x7f4470159d90 2023-10-06T11:35:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.909+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f446807c5c0 con 0x7f4470159d90 2023-10-06T11:35:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:55.917+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f44700c2ee0 con 0x7f4470159d90 2023-10-06T11:35:56.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.121+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f44700c2ee0 con 0x7f4470159d90 2023-10-06T11:35:56.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.513+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 52) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f446804c3f0 con 0x7f4470159d90 2023-10-06T11:35:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.513+0000 7f445e7fc700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 0x7f4450068f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.513+0000 7f446e59c700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 0x7f4450068f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.517+0000 7f446e59c700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 0x7f4450068f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4460001ce0 tx=0x7f4460007ea0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:56.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.705+0000 7f445e7fc700 1 -- 172.21.15.46:0/2137463050 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4468078390 con 0x7f4470159d90 2023-10-06T11:35:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 msgr2=0x7f4450068f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 0x7f4450068f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4460001ce0 tx=0x7f4460007ea0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 msgr2=0x7f44700c2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f446800c7e0 tx=0x7f4468007ea0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 shutdown_connections 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4450066b90 0x7f4450068f70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4470159d90 0x7f44700c2030 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4470153c60 0x7f44700af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 --2- 172.21.15.46:0/2137463050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447006e290 0x7f44700ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 >> 172.21.15.46:0/2137463050 conn(0x7f44700cbe30 msgr2=0x7f44701589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 shutdown_connections 2023-10-06T11:35:56.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.709+0000 7f4474c6e700 1 -- 172.21.15.46:0/2137463050 wait complete. 2023-10-06T11:35:56.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:35:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.909+0000 7f45c5fcc700 1 Processor -- start 2023-10-06T11:35:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.909+0000 7f45c5fcc700 1 -- start start 2023-10-06T11:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c5fcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f45c01591f0 0x7f45c00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c5fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c5fcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00cf360 con 0x7f45c01595c0 2023-10-06T11:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c5fcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c015dd60 con 0x7f45c01591f0 2023-10-06T11:35:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c5fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00cea30 con 0x7f45c00cc500 2023-10-06T11:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c57cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c4fca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f45c01591f0 0x7f45c00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c57cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:59534/0 (socket says 172.21.15.46:59534) 2023-10-06T11:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45c57cb700 1 -- 172.21.15.46:0/3383733148 learned_addr learned my addr 172.21.15.46:0/3383733148 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.913+0000 7f45bffff700 1 --1- 172.21.15.46:0/3383733148 >> v1:172.21.15.132:6789/0 conn(0x7f45c01595c0 0x7f45c00af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:58468/0 (socket says 172.21.15.46:58468) 2023-10-06T11:35:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c57cb700 1 -- 172.21.15.46:0/3383733148 >> v2:172.21.15.132:3300/0 conn(0x7f45c01591f0 msgr2=0x7f45c00cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c57cb700 1 --2- 172.21.15.46:0/3383733148 >> v2:172.21.15.132:3300/0 conn(0x7f45c01591f0 0x7f45c00cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c57cb700 1 -- 172.21.15.46:0/3383733148 >> v1:172.21.15.132:6789/0 conn(0x7f45c01595c0 legacy=0x7f45c00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c57cb700 1 -- 172.21.15.46:0/3383733148 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00ced10 con 0x7f45c00cc500 2023-10-06T11:35:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c57cb700 1 --2- 172.21.15.46:0/3383733148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f20 tx=0x7f45b8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe7340528755379 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45bf7fe700 1 -- 172.21.15.46:0/3383733148 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b800e070 con 0x7f45c00cc500 2023-10-06T11:35:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45bf7fe700 1 -- 172.21.15.46:0/3383733148 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b800ba60 con 0x7f45c00cc500 2023-10-06T11:35:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45bf7fe700 1 -- 172.21.15.46:0/3383733148 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b80092e0 con 0x7f45c00cc500 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 -- 172.21.15.46:0/3383733148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 msgr2=0x7f45c015ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/3383733148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f20 tx=0x7f45b8009840 comp rx=0 tx=0).stop 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 -- 172.21.15.46:0/3383733148 shutdown_connections 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/3383733148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c00cc500 0x7f45c015ceb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/3383733148 >> v2:172.21.15.132:3300/0 conn(0x7f45c01591f0 0x7f45c00cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 -- 172.21.15.46:0/3383733148 >> 172.21.15.46:0/3383733148 conn(0x7f45c006ef60 msgr2=0x7f45c00d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 -- 172.21.15.46:0/3383733148 shutdown_connections 2023-10-06T11:35:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.917+0000 7f45c5fcc700 1 -- 172.21.15.46:0/3383733148 wait complete. 2023-10-06T11:35:56.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.921+0000 7f45c5fcc700 1 Processor -- start 2023-10-06T11:35:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.921+0000 7f45c5fcc700 1 -- start start 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45c00cc500 0x7f45c01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c01591f0 0x7f45c00cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0150e00 con 0x7f45c009a160 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c009b2e0 con 0x7f45c00cc500 2023-10-06T11:35:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c5fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c009a6d0 con 0x7f45c01591f0 2023-10-06T11:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45bffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c01591f0 0x7f45c00cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c4fca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45c00cc500 0x7f45c01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48762/0 (socket says 172.21.15.46:48762) 2023-10-06T11:35:56.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 -- 172.21.15.46:0/488159109 learned_addr learned my addr 172.21.15.46:0/488159109 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 -- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c01591f0 msgr2=0x7f45c00cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c01591f0 0x7f45c00cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 -- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45c00cc500 msgr2=0x7f45c01508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45c00cc500 0x7f45c01508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.925+0000 7f45c57cb700 1 -- 172.21.15.46:0/488159109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b8009ae0 con 0x7f45c009a160 2023-10-06T11:35:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45c57cb700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009430 tx=0x7f45b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b80032f0 con 0x7f45c009a160 2023-10-06T11:35:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8008d10 con 0x7f45c009a160 2023-10-06T11:35:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c009a9b0 con 0x7f45c009a160 2023-10-06T11:35:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b801c620 con 0x7f45c009a160 2023-10-06T11:35:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c009af70 con 0x7f45c009a160 2023-10-06T11:35:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.929+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c01693e0 con 0x7f45c009a160 2023-10-06T11:35:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.933+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 52) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f45b8003040 con 0x7f45c009a160 2023-10-06T11:35:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.933+0000 7f45bcff9700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 0x7f45a4062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.933+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f45b807e180 con 0x7f45c009a160 2023-10-06T11:35:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.933+0000 7f45c4fca700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 0x7f45a4062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.937+0000 7f45c4fca700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 0x7f45a4062880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d10 tx=0x7f45b40091a0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:56.937+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f45b804e2e0 con 0x7f45c009a160 2023-10-06T11:35:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.109+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f45c0064590 con 0x7f45c009a160 2023-10-06T11:35:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.709+0000 7f45bcff9700 1 -- 172.21.15.46:0/488159109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 181+0+0 (secure 0 0 0) 0x7f45b8051900 con 0x7f45c009a160 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 msgr2=0x7f45a4062880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 0x7f45a4062880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009d10 tx=0x7f45b40091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 msgr2=0x7f45c00ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009430 tx=0x7f45b8018000 comp rx=0 tx=0).stop 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 shutdown_connections 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f45a40603d0 0x7f45a4062880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45c009a160 0x7f45c00ccf30 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45c01591f0 0x7f45c00cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 --2- 172.21.15.46:0/488159109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45c00cc500 0x7f45c01508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 >> 172.21.15.46:0/488159109 conn(0x7f45c006ef60 msgr2=0x7f45c0068e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 shutdown_connections 2023-10-06T11:35:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.717+0000 7f45c5fcc700 1 -- 172.21.15.46:0/488159109 wait complete. 2023-10-06T11:35:57.731 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:35:57.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:35:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.909+0000 7fb0f1985700 1 Processor -- start 2023-10-06T11:35:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.909+0000 7fb0f1985700 1 -- start start 2023-10-06T11:35:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0f1985700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0ec0c6b70 0x7fb0ec06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0f1985700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0c25a0 con 0x7fb0ec140bd0 2023-10-06T11:35:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0f1985700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0c1c70 con 0x7fb0ec140800 2023-10-06T11:35:57.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0f1985700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c1f50 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0ea7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb0ec140bd0 0x7fb0ec0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58110/0 (socket says 172.21.15.46:58110) 2023-10-06T11:35:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0ea7fc700 1 -- 172.21.15.46:0/448884447 learned_addr learned my addr 172.21.15.46:0/448884447 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.913+0000 7fb0eb7fe700 1 --2- 172.21.15.46:0/448884447 >> v2:172.21.15.46:3300/0 conn(0x7fb0ec0c6b70 0x7fb0ec06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0eb7fe700 1 -- 172.21.15.46:0/448884447 >> v1:172.21.15.132:6789/0 conn(0x7fb0ec140800 legacy=0x7fb0ec0c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0eb7fe700 1 -- 172.21.15.46:0/448884447 >> v1:172.21.15.46:6789/0 conn(0x7fb0ec140bd0 legacy=0x7fb0ec0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0eb7fe700 1 -- 172.21.15.46:0/448884447 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0c2230 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0eb7fe700 1 --2- 172.21.15.46:0/448884447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec06cc10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e60 tx=0x7fb0e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2120ee756ee521 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0e9ffb700 1 -- 172.21.15.46:0/448884447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400e070 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0e9ffb700 1 -- 172.21.15.46:0/448884447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400bc30 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0e9ffb700 1 -- 172.21.15.46:0/448884447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4012430 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 -- 172.21.15.46:0/448884447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 msgr2=0x7fb0ec06cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 --2- 172.21.15.46:0/448884447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec06cc10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009e60 tx=0x7fb0e4009780 comp rx=0 tx=0).stop 2023-10-06T11:35:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 -- 172.21.15.46:0/448884447 shutdown_connections 2023-10-06T11:35:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 --2- 172.21.15.46:0/448884447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec06cc10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 -- 172.21.15.46:0/448884447 >> 172.21.15.46:0/448884447 conn(0x7fb0ec0cbe20 msgr2=0x7fb0ec0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 -- 172.21.15.46:0/448884447 shutdown_connections 2023-10-06T11:35:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.917+0000 7fb0f1985700 1 -- 172.21.15.46:0/448884447 wait complete. 2023-10-06T11:35:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 Processor -- start 2023-10-06T11:35:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 -- start start 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0ec140800 0x7fb0ec0ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 0x7fb0ec0acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0b00e0 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec13ee10 con 0x7fb0ec140bd0 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0f1985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0af7b0 con 0x7fb0ec140800 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48768/0 (socket says 172.21.15.46:48768) 2023-10-06T11:35:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 -- 172.21.15.46:0/1390970472 learned_addr learned my addr 172.21.15.46:0/1390970472 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eb7fe700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 0x7fb0ec0acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 -- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0ec140800 msgr2=0x7fb0ec0ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0ec140800 0x7fb0ec0ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 -- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 msgr2=0x7fb0ec0acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 0x7fb0ec0acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 -- 172.21.15.46:0/1390970472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009a20 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eb7fe700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 0x7fb0ec0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.921+0000 7fb0eaffd700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c590 tx=0x7fb0dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011590 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc0116f0 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec0afa90 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011ab0 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec0af460 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec06b890 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.925+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb0dc01b3b0 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.929+0000 7fb0d2ffd700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 0x7fb0d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.929+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6954+0+0 (secure 0 0 0) 0x7fb0dc07c480 con 0x7fb0ec0c6b70 2023-10-06T11:35:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.929+0000 7fb0ea7fc700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 0x7fb0d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.933+0000 7fb0ea7fc700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 0x7fb0d8062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007b20 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:57.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:57.937+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb0dc04c3e0 con 0x7fb0ec0c6b70 2023-10-06T11:35:58.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.113+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0ec06d2b0 con 0x7fb0ec0c6b70 2023-10-06T11:35:58.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.729+0000 7fb0d2ffd700 1 -- 172.21.15.46:0/1390970472 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb0dc0170d0 con 0x7fb0ec0c6b70 2023-10-06T11:35:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 msgr2=0x7fb0d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 0x7fb0d8062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007b20 comp rx=0 tx=0).stop 2023-10-06T11:35:58.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 msgr2=0x7fb0ec13e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c590 tx=0x7fb0dc00c950 comp rx=0 tx=0).stop 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 shutdown_connections 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb0d8060670 0x7fb0d8062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0ec140bd0 0x7fb0ec0acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0ec140800 0x7fb0ec0ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 --2- 172.21.15.46:0/1390970472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0ec0c6b70 0x7fb0ec13e8d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 >> 172.21.15.46:0/1390970472 conn(0x7fb0ec0cbe20 msgr2=0x7fb0ec069950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 shutdown_connections 2023-10-06T11:35:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.733+0000 7fb0f1985700 1 -- 172.21.15.46:0/1390970472 wait complete. 2023-10-06T11:35:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:35:58.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:35:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.925+0000 7fb919003700 1 Processor -- start 2023-10-06T11:35:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.925+0000 7fb919003700 1 -- start start 2023-10-06T11:35:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb919003700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9140b0970 0x7fb9140cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb919003700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140cbfd0 con 0x7fb9140c8c80 2023-10-06T11:35:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb919003700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140cb120 con 0x7fb9140c91f0 2023-10-06T11:35:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb919003700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9140cb3c0 con 0x7fb9140b0970 2023-10-06T11:35:58.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb913fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9140b0970 0x7fb9140cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:58.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb913fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9140b0970 0x7fb9140cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48778/0 (socket says 172.21.15.46:48778) 2023-10-06T11:35:58.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb913fff700 1 -- 172.21.15.46:0/3956696433 learned_addr learned my addr 172.21.15.46:0/3956696433 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:58.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.929+0000 7fb90b7fe700 1 --1- 172.21.15.46:0/3956696433 >> v1:172.21.15.46:6789/0 conn(0x7fb9140c8c80 0x7fb9140b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58122/0 (socket says 172.21.15.46:58122) 2023-10-06T11:35:58.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb913fff700 1 -- 172.21.15.46:0/3956696433 >> v1:172.21.15.132:6789/0 conn(0x7fb9140c91f0 legacy=0x7fb9140ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb913fff700 1 -- 172.21.15.46:0/3956696433 >> v1:172.21.15.46:6789/0 conn(0x7fb9140c8c80 legacy=0x7fb9140b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb913fff700 1 -- 172.21.15.46:0/3956696433 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140cb6a0 con 0x7fb9140b0970 2023-10-06T11:35:58.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb9137fe700 1 -- 172.21.15.46:0/3956696433 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921867710 0 0) 0x7fb9140cbfd0 con 0x7fb9140c8c80 2023-10-06T11:35:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb913fff700 1 --2- 172.21.15.46:0/3956696433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9140cc460 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb904009f20 tx=0x7fb904009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffeb3f7d5b816b6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb9137fe700 1 -- 172.21.15.46:0/3956696433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400e070 con 0x7fb9140b0970 2023-10-06T11:35:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb9137fe700 1 -- 172.21.15.46:0/3956696433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90400ba80 con 0x7fb9140b0970 2023-10-06T11:35:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb9137fe700 1 -- 172.21.15.46:0/3956696433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9040092e0 con 0x7fb9140b0970 2023-10-06T11:35:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 -- 172.21.15.46:0/3956696433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 msgr2=0x7fb9140cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 --2- 172.21.15.46:0/3956696433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9140cc460 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb904009f20 tx=0x7fb904009840 comp rx=0 tx=0).stop 2023-10-06T11:35:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 -- 172.21.15.46:0/3956696433 shutdown_connections 2023-10-06T11:35:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 --2- 172.21.15.46:0/3956696433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9140cc460 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 -- 172.21.15.46:0/3956696433 >> 172.21.15.46:0/3956696433 conn(0x7fb9140ccd50 msgr2=0x7fb9140cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 -- 172.21.15.46:0/3956696433 shutdown_connections 2023-10-06T11:35:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.933+0000 7fb919003700 1 -- 172.21.15.46:0/3956696433 wait complete. 2023-10-06T11:35:58.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 Processor -- start 2023-10-06T11:35:58.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 -- start start 2023-10-06T11:35:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9140c8c80 0x7fb914156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9140c91f0 0x7fb914158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91415e8b0 con 0x7fb9140b0970 2023-10-06T11:35:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91415dca0 con 0x7fb9140c8c80 2023-10-06T11:35:58.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb919003700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91415df80 con 0x7fb9140c91f0 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48788/0 (socket says 172.21.15.46:48788) 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 -- 172.21.15.46:0/1344015979 learned_addr learned my addr 172.21.15.46:0/1344015979 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb90b7fe700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9140c8c80 0x7fb914156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb918802700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9140c91f0 0x7fb914158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 -- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9140c91f0 msgr2=0x7fb914158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9140c91f0 0x7fb914158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 -- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9140c8c80 msgr2=0x7fb914156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:58.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9140c8c80 0x7fb914156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.937+0000 7fb913fff700 1 -- 172.21.15.46:0/1344015979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904009ae0 con 0x7fb9140b0970 2023-10-06T11:35:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb913fff700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb91406bc50 tx=0x7fb904018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904007b80 con 0x7fb9140b0970 2023-10-06T11:35:58.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb904007d00 con 0x7fb9140b0970 2023-10-06T11:35:58.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91415e260 con 0x7fb9140b0970 2023-10-06T11:35:58.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9040266b0 con 0x7fb9140b0970 2023-10-06T11:35:58.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb914153720 con 0x7fb9140b0970 2023-10-06T11:35:58.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.941+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb914064590 con 0x7fb9140b0970 2023-10-06T11:35:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.945+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb90401d420 con 0x7fb9140b0970 2023-10-06T11:35:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.945+0000 7fb910ff9700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 0x7fb8f4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.945+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb90407ed80 con 0x7fb9140b0970 2023-10-06T11:35:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.945+0000 7fb90b7fe700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 0x7fb8f4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:58.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.945+0000 7fb90b7fe700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 0x7fb8f4062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb914158600 tx=0x7fb8fc007bd0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:58.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:58.949+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb90404ef30 con 0x7fb9140b0970 2023-10-06T11:35:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.121+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb914158810 con 0x7fb9140b0970 2023-10-06T11:35:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.133+0000 7fb910ff9700 1 -- 172.21.15.46:0/1344015979 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb904052550 con 0x7fb9140b0970 2023-10-06T11:35:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 msgr2=0x7fb8f4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 0x7fb8f4062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb914158600 tx=0x7fb8fc007bd0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 msgr2=0x7fb9141531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb91406bc50 tx=0x7fb904018000 comp rx=0 tx=0).stop 2023-10-06T11:35:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 shutdown_connections 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8f4060670 0x7fb8f4062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9140c91f0 0x7fb914158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9140c8c80 0x7fb914156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 --2- 172.21.15.46:0/1344015979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9140b0970 0x7fb9141531e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 >> 172.21.15.46:0/1344015979 conn(0x7fb9140ccd50 msgr2=0x7fb9140c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 shutdown_connections 2023-10-06T11:35:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.137+0000 7fb919003700 1 -- 172.21.15.46:0/1344015979 wait complete. 2023-10-06T11:35:59.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:35:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.325+0000 7fe734025700 1 Processor -- start 2023-10-06T11:35:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.325+0000 7fe734025700 1 -- start start 2023-10-06T11:35:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe734025700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe72c0ccc00 0x7fe72c0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe734025700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c0cb4a0 con 0x7fe72c0b27c0 2023-10-06T11:35:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe734025700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c0cb6e0 con 0x7fe72c0b23f0 2023-10-06T11:35:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe734025700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c0cc780 con 0x7fe72c0ccc00 2023-10-06T11:35:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe731dc1700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe72c0b23f0 0x7fe72c0cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:58494/0 (socket says 172.21.15.46:58494) 2023-10-06T11:35:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe731dc1700 1 -- 172.21.15.46:0/4093389780 learned_addr learned my addr 172.21.15.46:0/4093389780 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:35:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe7315c0700 1 --2- 172.21.15.46:0/4093389780 >> v2:172.21.15.132:3300/0 conn(0x7fe72c0ccc00 0x7fe72c0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471388326 0 0) 0x7fe72c0cb6e0 con 0x7fe72c0b23f0 2023-10-06T11:35:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe718003570 con 0x7fe72c0b23f0 2023-10-06T11:35:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473598413 0 0) 0x7fe724001490 con 0x7fe72c0b27c0 2023-10-06T11:35:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe72c0cb6e0 con 0x7fe72c0b27c0 2023-10-06T11:35:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe7315c0700 1 -- 172.21.15.46:0/4093389780 >> v1:172.21.15.132:6789/0 conn(0x7fe72c0b23f0 legacy=0x7fe72c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe7315c0700 1 -- 172.21.15.46:0/4093389780 >> v1:172.21.15.46:6789/0 conn(0x7fe72c0b27c0 legacy=0x7fe72c0c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe7315c0700 1 -- 172.21.15.46:0/4093389780 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0cb9c0 con 0x7fe72c0ccc00 2023-10-06T11:35:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe7315c0700 1 --2- 172.21.15.46:0/4093389780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c0ce4d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0098e0 tx=0x7fe71c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f9fa583b5e60b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.329+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c011630 con 0x7fe72c0ccc00 2023-10-06T11:35:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c011790 con 0x7fe72c0ccc00 2023-10-06T11:35:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe730dbf700 1 -- 172.21.15.46:0/4093389780 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c011b30 con 0x7fe72c0ccc00 2023-10-06T11:35:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- 172.21.15.46:0/4093389780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 msgr2=0x7fe72c0ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 --2- 172.21.15.46:0/4093389780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c0ce4d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0098e0 tx=0x7fe71c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- 172.21.15.46:0/4093389780 shutdown_connections 2023-10-06T11:35:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 --2- 172.21.15.46:0/4093389780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c0ce4d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- 172.21.15.46:0/4093389780 >> 172.21.15.46:0/4093389780 conn(0x7fe72c099440 msgr2=0x7fe72c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- 172.21.15.46:0/4093389780 shutdown_connections 2023-10-06T11:35:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- 172.21.15.46:0/4093389780 wait complete. 2023-10-06T11:35:59.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 Processor -- start 2023-10-06T11:35:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.333+0000 7fe734025700 1 -- start start 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe72c0b23f0 0x7fe72c09b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c16dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c1598e0 con 0x7fe72c0b27c0 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c16e2b0 con 0x7fe72c0ccc00 2023-10-06T11:35:59.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c15bb70 con 0x7fe72c0b23f0 2023-10-06T11:35:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48796/0 (socket says 172.21.15.46:48796) 2023-10-06T11:35:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 -- 172.21.15.46:0/2738251326 learned_addr learned my addr 172.21.15.46:0/2738251326 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe731dc1700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe72c0b23f0 0x7fe72c09b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7325c2700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c16dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 -- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe72c0b23f0 msgr2=0x7fe72c09b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe72c0b23f0 0x7fe72c09b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 -- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 msgr2=0x7fe72c16dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c16dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 -- 172.21.15.46:0/2738251326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009420 con 0x7fe72c0b27c0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe7315c0700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ed0 tx=0x7fe71c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c01f070 con 0x7fe72c0b27c0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c011900 con 0x7fe72c0b27c0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c15be50 con 0x7fe72c0b27c0 2023-10-06T11:35:59.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.337+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c023410 con 0x7fe72c0b27c0 2023-10-06T11:35:59.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.341+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c15c410 con 0x7fe72c0b27c0 2023-10-06T11:35:59.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.341+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c159a90 con 0x7fe72c0b27c0 2023-10-06T11:35:59.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.341+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe71c02c3b0 con 0x7fe72c0b27c0 2023-10-06T11:35:59.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.345+0000 7fe721ffb700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 0x7fe710062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.345+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe71c07ddc0 con 0x7fe72c0b27c0 2023-10-06T11:35:59.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.345+0000 7fe731dc1700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 0x7fe710062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.345+0000 7fe731dc1700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 0x7fe710062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe728005920 tx=0x7fe728009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.349+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe72c159a90 con 0x7fe72c0b27c0 2023-10-06T11:35:59.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.525+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe72c159a90 con 0x7fe72c0b27c0 2023-10-06T11:35:59.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.533+0000 7fe721ffb700 1 -- 172.21.15.46:0/2738251326 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe72c159a90 con 0x7fe72c0b27c0 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 msgr2=0x7fe710062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 0x7fe710062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe728005920 tx=0x7fe728009040 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 msgr2=0x7fe72c15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ed0 tx=0x7fe71c017000 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 shutdown_connections 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe710060670 0x7fe710062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe72c0ccc00 0x7fe72c16dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b27c0 0x7fe72c15b500 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 --2- 172.21.15.46:0/2738251326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe72c0b23f0 0x7fe72c09b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 >> 172.21.15.46:0/2738251326 conn(0x7fe72c099440 msgr2=0x7fe72c067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:59.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 shutdown_connections 2023-10-06T11:35:59.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.541+0000 7fe734025700 1 -- 172.21.15.46:0/2738251326 wait complete. 2023-10-06T11:35:59.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:35:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.725+0000 7f7b0ae25700 1 Processor -- start 2023-10-06T11:35:59.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.725+0000 7f7b0ae25700 1 -- start start 2023-10-06T11:35:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b0ae25700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b0ae25700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b04159550 0x7f7b040afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b0ae25700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040af6f0 con 0x7f7b040aee00 2023-10-06T11:35:59.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b0ae25700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040b43e0 con 0x7f7b04159550 2023-10-06T11:35:59.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b0ae25700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040b04b0 con 0x7f7b040b2360 2023-10-06T11:35:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b08bc1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b04159550 0x7f7b040afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b08bc1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:59596/0 (socket says 172.21.15.46:59596) 2023-10-06T11:35:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b08bc1700 1 -- 172.21.15.46:0/2302643500 learned_addr learned my addr 172.21.15.46:0/2302643500 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:59.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b03fff700 1 --1- 172.21.15.46:0/2302643500 >> v1:172.21.15.132:6789/0 conn(0x7f7b040aee00 0x7f7b04158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:58502/0 (socket says 172.21.15.46:58502) 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 -- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 msgr2=0x7f7b040b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 -- 172.21.15.46:0/2302643500 >> v1:172.21.15.132:6789/0 conn(0x7f7b040aee00 legacy=0x7f7b04158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 -- 172.21.15.46:0/2302643500 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040b0790 con 0x7f7b04159550 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b08bc1700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.729+0000 7f7b093c2700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b04159550 0x7f7b040afdd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009e90 tx=0x7f7afc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cee5dfe6f009527d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b037fe700 1 -- 172.21.15.46:0/2302643500 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00e070 con 0x7f7b04159550 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b037fe700 1 -- 172.21.15.46:0/2302643500 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00bbf0 con 0x7f7b04159550 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b037fe700 1 -- 172.21.15.46:0/2302643500 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc012430 con 0x7f7b04159550 2023-10-06T11:35:59.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 -- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b04159550 msgr2=0x7f7b040afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b04159550 0x7f7b040afdd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009e90 tx=0x7f7afc009780 comp rx=0 tx=0).stop 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 -- 172.21.15.46:0/2302643500 shutdown_connections 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b04159550 0x7f7b040afdd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/2302643500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040b2360 0x7f7b040b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 -- 172.21.15.46:0/2302643500 >> 172.21.15.46:0/2302643500 conn(0x7f7b0406e780 msgr2=0x7f7b0406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 -- 172.21.15.46:0/2302643500 shutdown_connections 2023-10-06T11:35:59.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.733+0000 7f7b0ae25700 1 -- 172.21.15.46:0/2302643500 wait complete. 2023-10-06T11:35:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 Processor -- start 2023-10-06T11:35:59.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 -- start start 2023-10-06T11:35:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040aee00 0x7f7b0415a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 0x7f7b040cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040c80f0 con 0x7f7b04159550 2023-10-06T11:35:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040c90e0 con 0x7f7b040b2360 2023-10-06T11:35:59.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b0ae25700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040c7200 con 0x7f7b040aee00 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48808/0 (socket says 172.21.15.46:48808) 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 -- 172.21.15.46:0/3115962324 learned_addr learned my addr 172.21.15.46:0/3115962324 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b08bc1700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040aee00 0x7f7b0415a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b03fff700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 0x7f7b040cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 -- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040aee00 msgr2=0x7f7b0415a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040aee00 0x7f7b0415a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 -- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 msgr2=0x7f7b040cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 0x7f7b040cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 -- 172.21.15.46:0/3115962324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80094f0 con 0x7f7b04159550 2023-10-06T11:35:59.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b03fff700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 0x7f7b040cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:35:59.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.737+0000 7f7b093c2700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b6c0 tx=0x7f7afc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc027470 con 0x7f7b04159550 2023-10-06T11:35:59.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc0275d0 con 0x7f7b04159550 2023-10-06T11:35:59.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc027990 con 0x7f7b04159550 2023-10-06T11:35:59.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc009a20 con 0x7f7b04159550 2023-10-06T11:35:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040c77c0 con 0x7f7b04159550 2023-10-06T11:35:59.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b040c82a0 con 0x7f7b04159550 2023-10-06T11:35:59.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.741+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7afc024070 con 0x7f7b04159550 2023-10-06T11:35:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.745+0000 7f7b00ff9700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 0x7f7ae8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:35:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.745+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f7afc07f730 con 0x7f7b04159550 2023-10-06T11:35:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.745+0000 7f7b08bc1700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 0x7f7ae8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:35:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.745+0000 7f7b08bc1700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 0x7f7ae8062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006d20 tx=0x7f7af8009250 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:35:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.749+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b040c82a0 con 0x7f7b04159550 2023-10-06T11:35:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.921+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f7b04088840 con 0x7f7b04159550 2023-10-06T11:35:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.929+0000 7f7b00ff9700 1 -- 172.21.15.46:0/3115962324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7afc04f8a0 con 0x7f7b04159550 2023-10-06T11:35:59.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.929+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 msgr2=0x7f7ae8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.929+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 0x7f7ae8062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006d20 tx=0x7f7af8009250 comp rx=0 tx=0).stop 2023-10-06T11:35:59.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.929+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 msgr2=0x7f7b040cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:35:59.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.929+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00b6c0 tx=0x7f7afc019000 comp rx=0 tx=0).stop 2023-10-06T11:35:59.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 shutdown_connections 2023-10-06T11:35:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7ae8060670 0x7f7ae8062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b04159550 0x7f7b040cca30 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b040b2360 0x7f7b040cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 --2- 172.21.15.46:0/3115962324 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b040aee00 0x7f7b0415a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:35:59.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 >> 172.21.15.46:0/3115962324 conn(0x7f7b0406e780 msgr2=0x7f7b04157420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:35:59.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 shutdown_connections 2023-10-06T11:35:59.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:35:59.933+0000 7f7b0ae25700 1 -- 172.21.15.46:0/3115962324 wait complete. 2023-10-06T11:35:59.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-06T11:36:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.117+0000 7f18c4878700 1 Processor -- start 2023-10-06T11:36:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.117+0000 7f18c4878700 1 -- start start 2023-10-06T11:36:00.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18c4878700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f18c0154b50 0x7f18c0067a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18c4878700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c0068260 con 0x7f18c0063180 2023-10-06T11:36:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18c4878700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c0157760 con 0x7f18c00cb9e0 2023-10-06T11:36:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18c4878700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c0157560 con 0x7f18c0154b50 2023-10-06T11:36:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18bdd9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f18c0063180 0x7f18c00cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58128/0 (socket says 172.21.15.46:58128) 2023-10-06T11:36:00.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.121+0000 7f18bdd9b700 1 -- 172.21.15.46:0/618797725 learned_addr learned my addr 172.21.15.46:0/618797725 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 --2- 172.21.15.46:0/618797725 >> v2:172.21.15.132:3300/0 conn(0x7f18c0154b50 0x7f18c0067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456669585 0 0) 0x7f18c0068260 con 0x7f18c0063180 2023-10-06T11:36:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18a8003570 con 0x7f18c0063180 2023-10-06T11:36:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3141104037 0 0) 0x7f18a8003570 con 0x7f18c0063180 2023-10-06T11:36:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 >> v2:172.21.15.132:3300/0 conn(0x7f18c0154b50 msgr2=0x7f18c0067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 --2- 172.21.15.46:0/618797725 >> v2:172.21.15.132:3300/0 conn(0x7f18c0154b50 0x7f18c0067a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 >> v1:172.21.15.132:6789/0 conn(0x7f18c00cb9e0 legacy=0x7f18c0069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0157a40 con 0x7f18c0063180 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f18b4001470 con 0x7f18c0063180 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 >> v1:172.21.15.46:6789/0 conn(0x7f18c0063180 legacy=0x7f18c00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004810 0x7f18a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a80051c0 0x7f18a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0157560 con 0x7f18a8003a60 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a800c0e0 con 0x7f18a80051c0 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a800c2a0 con 0x7f18a8004810 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bdd9b700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004810 0x7f18a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bed9d700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a80051c0 0x7f18a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 -- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004810 msgr2=0x7f18a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18a8004810 0x7f18a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 -- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a80051c0 msgr2=0x7f18a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a80051c0 0x7f18a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 -- 172.21.15.46:0/618797725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8003570 con 0x7f18a8003a60 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bed9d700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18a80051c0 0x7f18a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bed9d700 1 -- 172.21.15.46:0/618797725 reap_dead start 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18be59c700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18a8003e30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f18b000bb90 tx=0x7f18b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7c06f8116c95e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.125+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000d5d0 con 0x7f18a8003a60 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b000d730 con 0x7f18a8003a60 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18bd59a700 1 -- 172.21.15.46:0/618797725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000daf0 con 0x7f18a8003a60 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 -- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 msgr2=0x7f18a8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18a8003e30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f18b000bb90 tx=0x7f18b0008040 comp rx=0 tx=0).stop 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 -- 172.21.15.46:0/618797725 shutdown_connections 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 --2- 172.21.15.46:0/618797725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18a8003e30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 -- 172.21.15.46:0/618797725 >> 172.21.15.46:0/618797725 conn(0x7f18c00cd120 msgr2=0x7f18c00b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 -- 172.21.15.46:0/618797725 shutdown_connections 2023-10-06T11:36:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.129+0000 7f18c4878700 1 -- 172.21.15.46:0/618797725 wait complete. 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 Processor -- start 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 -- start start 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18c015b9b0 0x7f18c0151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18c0069490 0x7f18c00698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00c9980 con 0x7f18a8003a60 2023-10-06T11:36:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0069e00 con 0x7f18c015b9b0 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18c4878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c015c150 con 0x7f18c0069490 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18be59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18be59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48828/0 (socket says 172.21.15.46:48828) 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18be59c700 1 -- 172.21.15.46:0/3900099482 learned_addr learned my addr 172.21.15.46:0/3900099482 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18bdd9b700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18c015b9b0 0x7f18c0151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.133+0000 7f18bed9d700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18c0069490 0x7f18c00698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 -- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18c0069490 msgr2=0x7f18c00698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18c0069490 0x7f18c00698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 -- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18c015b9b0 msgr2=0x7f18c0151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18c015b9b0 0x7f18c0151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 -- 172.21.15.46:0/3900099482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c015c430 con 0x7f18a8003a60 2023-10-06T11:36:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18be59c700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c0b0 tx=0x7f18b001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0014640 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b00147c0 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c015c710 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0014b80 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c01524d0 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.137+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c00c9750 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.141+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f18b001e400 con 0x7f18a8003a60 2023-10-06T11:36:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.141+0000 7f18ae7fc700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 0x7f18a0062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.141+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f18b007e480 con 0x7f18a8003a60 2023-10-06T11:36:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.141+0000 7f18bdd9b700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 0x7f18a0062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.141+0000 7f18bdd9b700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 0x7f18a0062bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18c0156540 tx=0x7f18b4004880 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.145+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f18c00c9750 con 0x7f18a8003a60 2023-10-06T11:36:00.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.373+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f18c00c9750 con 0x7f18a8003a60 2023-10-06T11:36:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.737+0000 7f18ae7fc700 1 -- 172.21.15.46:0/3900099482 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v667) v1 ==== 183+0+0 (secure 0 0 0) 0x7f18b004e670 con 0x7f18a8003a60 2023-10-06T11:36:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 msgr2=0x7f18a0062bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 0x7f18a0062bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18c0156540 tx=0x7f18b4004880 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 msgr2=0x7f18c015d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c0b0 tx=0x7f18b001b000 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 shutdown_connections 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f18a0060720 0x7f18a0062bd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f18c0069490 0x7f18c00698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18c015b9b0 0x7f18c0151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 --2- 172.21.15.46:0/3900099482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18a8003a60 0x7f18c015d9c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 >> 172.21.15.46:0/3900099482 conn(0x7f18c00cd120 msgr2=0x7f18c0153df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 shutdown_connections 2023-10-06T11:36:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.741+0000 7f18c4878700 1 -- 172.21.15.46:0/3900099482 wait complete. 2023-10-06T11:36:00.778 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-06T11:36:00.779 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-06T11:36:00.779 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:36:00.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-06T11:36:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.929+0000 7fe7cec2e700 1 Processor -- start 2023-10-06T11:36:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.929+0000 7fe7cec2e700 1 -- start start 2023-10-06T11:36:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cec2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7c80abf60 0x7fe7c8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cec2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c815a780 con 0x7fe7c80aecb0 2023-10-06T11:36:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cec2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c815ae50 con 0x7fe7c80ac330 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cec2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c815a1c0 con 0x7fe7c80abf60 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cd1cb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe7c80aecb0 0x7fe7c80b14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58140/0 (socket says 172.21.15.46:58140) 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cd1cb700 1 -- 172.21.15.46:0/596935166 learned_addr learned my addr 172.21.15.46:0/596935166 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cc9ca700 1 --2- 172.21.15.46:0/596935166 >> v2:172.21.15.46:3300/0 conn(0x7fe7c80abf60 0x7fe7c8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/596935166 >> v1:172.21.15.132:6789/0 conn(0x7fe7c80ac330 legacy=0x7fe7c8068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/596935166 >> v1:172.21.15.46:6789/0 conn(0x7fe7c80aecb0 legacy=0x7fe7c80b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/596935166 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c815a4a0 con 0x7fe7c80abf60 2023-10-06T11:36:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.933+0000 7fe7bf7fe700 1 -- 172.21.15.46:0/596935166 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309022831 0 0) 0x7fe7c815a780 con 0x7fe7c80aecb0 2023-10-06T11:36:00.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cc9ca700 1 --2- 172.21.15.46:0/596935166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8159780 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009e60 tx=0x7fe7c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4737ac3e133cd3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7bf7fe700 1 -- 172.21.15.46:0/596935166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c400e070 con 0x7fe7c80abf60 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7bf7fe700 1 -- 172.21.15.46:0/596935166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c400bc30 con 0x7fe7c80abf60 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7bf7fe700 1 -- 172.21.15.46:0/596935166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4012430 con 0x7fe7c80abf60 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 -- 172.21.15.46:0/596935166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 msgr2=0x7fe7c8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/596935166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8159780 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009e60 tx=0x7fe7c4009780 comp rx=0 tx=0).stop 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 -- 172.21.15.46:0/596935166 shutdown_connections 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/596935166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8159780 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 -- 172.21.15.46:0/596935166 >> 172.21.15.46:0/596935166 conn(0x7fe7c806d490 msgr2=0x7fe7c806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 -- 172.21.15.46:0/596935166 shutdown_connections 2023-10-06T11:36:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.937+0000 7fe7cec2e700 1 -- 172.21.15.46:0/596935166 wait complete. 2023-10-06T11:36:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 Processor -- start 2023-10-06T11:36:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 -- start start 2023-10-06T11:36:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7c80ac330 0x7fe7c81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7c80aecb0 0x7fe7c8155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80c80f0 con 0x7fe7c80abf60 2023-10-06T11:36:00.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81579a0 con 0x7fe7c80aecb0 2023-10-06T11:36:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.941+0000 7fe7cec2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81584e0 con 0x7fe7c80ac330 2023-10-06T11:36:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48842/0 (socket says 172.21.15.46:48842) 2023-10-06T11:36:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/3236543195 learned_addr learned my addr 172.21.15.46:0/3236543195 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7bffff700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7c80ac330 0x7fe7c81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cd1cb700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7c80aecb0 0x7fe7c8155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7c80ac330 msgr2=0x7fe7c81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7c80ac330 0x7fe7c81558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7c80aecb0 msgr2=0x7fe7c8155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7c80aecb0 0x7fe7c8155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 -- 172.21.15.46:0/3236543195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4009a20 con 0x7fe7c80abf60 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cc9ca700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806ca10 tx=0x7fe7c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4012e20 con 0x7fe7c80abf60 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c401b630 con 0x7fe7c80abf60 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c8157bb0 con 0x7fe7c80abf60 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c401b9f0 con 0x7fe7c80abf60 2023-10-06T11:36:00.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.945+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c8157f40 con 0x7fe7c80abf60 2023-10-06T11:36:00.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.949+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8156570 con 0x7fe7c80abf60 2023-10-06T11:36:00.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.949+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe7c401c400 con 0x7fe7c80abf60 2023-10-06T11:36:00.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.949+0000 7fe7bcff9700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 0x7fe7b4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.949+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe7c407ec60 con 0x7fe7c80abf60 2023-10-06T11:36:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.949+0000 7fe7bffff700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 0x7fe7b4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.953+0000 7fe7bffff700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 0x7fe7b4062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7c815cce0 tx=0x7fe7b8009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:00.957+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe7c404ecc0 con 0x7fe7c80abf60 2023-10-06T11:36:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.185+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fe7c815aae0 con 0x7fe7c80abf60 2023-10-06T11:36:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.741+0000 7fe7bcff9700 1 -- 172.21.15.46:0/3236543195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v668) v1 ==== 151+0+0 (secure 0 0 0) 0x7fe7c40522e0 con 0x7fe7c80abf60 2023-10-06T11:36:01.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 msgr2=0x7fe7b4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 0x7fe7b4062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7c815cce0 tx=0x7fe7b8009040 comp rx=0 tx=0).stop 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 msgr2=0x7fe7c8152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806ca10 tx=0x7fe7c4019000 comp rx=0 tx=0).stop 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 shutdown_connections 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe7b4060670 0x7fe7b4062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7c80aecb0 0x7fe7c8155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7c80ac330 0x7fe7c81558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 --2- 172.21.15.46:0/3236543195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7c80abf60 0x7fe7c8152460 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 >> 172.21.15.46:0/3236543195 conn(0x7fe7c806d490 msgr2=0x7fe7c8067050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 shutdown_connections 2023-10-06T11:36:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.749+0000 7fe7cec2e700 1 -- 172.21.15.46:0/3236543195 wait complete. 2023-10-06T11:36:01.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:36:01.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.949+0000 7f4b7e043700 1 Processor -- start 2023-10-06T11:36:01.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.949+0000 7f4b7e043700 1 -- start start 2023-10-06T11:36:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.949+0000 7f4b7e043700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b78154880 0x7f4b780c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.949+0000 7f4b7e043700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b780c7f20 0x7f4b7815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7e043700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b780c6cf0 con 0x7f4b78154c50 2023-10-06T11:36:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7e043700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780c2a30 con 0x7f4b78154880 2023-10-06T11:36:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7e043700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780c83e0 con 0x7f4b780c7f20 2023-10-06T11:36:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b78154880 0x7f4b780c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7c840700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4b78154c50 0x7f4b7806c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:58536/0 (socket says 172.21.15.46:58536) 2023-10-06T11:36:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7c840700 1 -- 172.21.15.46:0/3687386716 learned_addr learned my addr 172.21.15.46:0/3687386716 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d842700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b780c7f20 0x7f4b7815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 -- 172.21.15.46:0/3687386716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b780c7f20 msgr2=0x7f4b7815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:01.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b780c7f20 0x7f4b7815aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 -- 172.21.15.46:0/3687386716 >> v1:172.21.15.132:6789/0 conn(0x7f4b78154c50 legacy=0x7f4b7806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 -- 172.21.15.46:0/3687386716 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7806e5d0 con 0x7f4b78154880 2023-10-06T11:36:01.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7d041700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154880 0x7f4b780c7180 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4b740098e0 tx=0x7f4b740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a71f76c168f7ea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:01.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b6ffff700 1 -- 172.21.15.46:0/3687386716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400b640 con 0x7f4b78154880 2023-10-06T11:36:01.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b6ffff700 1 -- 172.21.15.46:0/3687386716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7400b7a0 con 0x7f4b78154880 2023-10-06T11:36:01.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b6ffff700 1 -- 172.21.15.46:0/3687386716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400bb40 con 0x7f4b78154880 2023-10-06T11:36:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7e043700 1 -- 172.21.15.46:0/3687386716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154880 msgr2=0x7f4b780c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.953+0000 7f4b7e043700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154880 0x7f4b780c7180 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4b740098e0 tx=0x7f4b740091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 -- 172.21.15.46:0/3687386716 shutdown_connections 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b780c7f20 0x7f4b7815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 --2- 172.21.15.46:0/3687386716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154880 0x7f4b780c7180 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 -- 172.21.15.46:0/3687386716 >> 172.21.15.46:0/3687386716 conn(0x7f4b780cbc10 msgr2=0x7f4b780cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 -- 172.21.15.46:0/3687386716 shutdown_connections 2023-10-06T11:36:01.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 -- 172.21.15.46:0/3687386716 wait complete. 2023-10-06T11:36:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 Processor -- start 2023-10-06T11:36:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.957+0000 7f4b7e043700 1 -- start start 2023-10-06T11:36:01.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b780c7f20 0x7f4b781529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b78154880 0x7f4b78159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780c3900 con 0x7f4b78154c50 2023-10-06T11:36:01.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78155320 con 0x7f4b780c7f20 2023-10-06T11:36:01.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b781581c0 con 0x7f4b78154880 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48862/0 (socket says 172.21.15.46:48862) 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 -- 172.21.15.46:0/669216525 learned_addr learned my addr 172.21.15.46:0/669216525 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d041700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b780c7f20 0x7f4b781529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7c840700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b78154880 0x7f4b78159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 -- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b78154880 msgr2=0x7f4b78159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b78154880 0x7f4b78159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 -- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b780c7f20 msgr2=0x7f4b781529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b780c7f20 0x7f4b781529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 -- 172.21.15.46:0/669216525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009420 con 0x7f4b78154c50 2023-10-06T11:36:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7d842700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009e90 tx=0x7f4b7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cb70 con 0x7f4b78154c50 2023-10-06T11:36:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7000f040 con 0x7f4b78154c50 2023-10-06T11:36:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b780c2fd0 con 0x7f4b78154c50 2023-10-06T11:36:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70014480 con 0x7f4b78154c50 2023-10-06T11:36:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.961+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b780c3590 con 0x7f4b78154c50 2023-10-06T11:36:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.965+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b780c3ab0 con 0x7f4b78154c50 2023-10-06T11:36:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.965+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4b70014830 con 0x7f4b78154c50 2023-10-06T11:36:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.969+0000 7f4b6d7fa700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 0x7f4b5c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:01.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.969+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f4b7007d9b0 con 0x7f4b78154c50 2023-10-06T11:36:01.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.969+0000 7f4b7d041700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 0x7f4b5c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.969+0000 7f4b7d041700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 0x7f4b5c062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b740093f0 tx=0x7f4b74017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:01.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:01.973+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b780c3ab0 con 0x7f4b78154c50 2023-10-06T11:36:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.145+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4b78064590 con 0x7f4b78154c50 2023-10-06T11:36:02.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.153+0000 7f4b6d7fa700 1 -- 172.21.15.46:0/669216525 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4b700180c0 con 0x7f4b78154c50 2023-10-06T11:36:02.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 msgr2=0x7f4b5c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 0x7f4b5c062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4b740093f0 tx=0x7f4b74017000 comp rx=0 tx=0).stop 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 msgr2=0x7f4b78157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009e90 tx=0x7f4b7000b040 comp rx=0 tx=0).stop 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 shutdown_connections 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4b5c0606a0 0x7f4b5c062b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b78154c50 0x7f4b78157600 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b78154880 0x7f4b78159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 --2- 172.21.15.46:0/669216525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b780c7f20 0x7f4b781529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 >> 172.21.15.46:0/669216525 conn(0x7f4b780cbc10 msgr2=0x7f4b780c09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 shutdown_connections 2023-10-06T11:36:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.161+0000 7f4b7e043700 1 -- 172.21.15.46:0/669216525 wait complete. 2023-10-06T11:36:02.194 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:36:02.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:36:02.237 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:36:02.281 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:36:02.281 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:36:02.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:36:02.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.433+0000 7f974aae3700 1 Processor -- start 2023-10-06T11:36:02.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.433+0000 7f974aae3700 1 -- start start 2023-10-06T11:36:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f974aae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 0x7f97440c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f974aae3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9744069660 con 0x7f97440bedd0 2023-10-06T11:36:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f974aae3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9744068ac0 con 0x7f97440bf1a0 2023-10-06T11:36:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f974aae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9744068d30 con 0x7f9744067760 2023-10-06T11:36:02.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9749ae1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97440bedd0 0x7f9744067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58150/0 (socket says 172.21.15.46:58150) 2023-10-06T11:36:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9749ae1700 1 -- 172.21.15.46:0/1287846823 learned_addr learned my addr 172.21.15.46:0/1287846823 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f97492e0700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 0x7f97440c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127364253 0 0) 0x7f9744069660 con 0x7f97440bedd0 2023-10-06T11:36:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9730003570 con 0x7f97440bedd0 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540349157 0 0) 0x7f9730003570 con 0x7f97440bedd0 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 msgr2=0x7f97440c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 0x7f97440c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 >> v1:172.21.15.132:6789/0 conn(0x7f97440bf1a0 legacy=0x7f9744161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744069010 con 0x7f97440bedd0 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f97400017f0 con 0x7f97440bedd0 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 >> v1:172.21.15.46:6789/0 conn(0x7f97440bedd0 legacy=0x7f9744067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730003a90 0x7f9730003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9730004810 0x7f9730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 0x7f9730005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9744068d30 con 0x7f9730003a90 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973000bc20 con 0x7f9730004810 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973000bde0 con 0x7f9744067760 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f97492e0700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9730004810 0x7f9730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.437+0000 7f9749ae1700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730003a90 0x7f9730003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 -- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 msgr2=0x7f9730005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9744067760 0x7f9730005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 -- 172.21.15.46:0/1287846823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730003a90 msgr2=0x7f9730003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730003a90 0x7f9730003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 -- 172.21.15.46:0/1287846823 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730003570 con 0x7f9730004810 2023-10-06T11:36:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f97492e0700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9730004810 0x7f9730004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f973400bb60 tx=0x7f9734008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a396a8f1621af8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400d5d0 con 0x7f9730004810 2023-10-06T11:36:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400d730 con 0x7f9730004810 2023-10-06T11:36:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f9748adf700 1 -- 172.21.15.46:0/1287846823 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400daf0 con 0x7f9730004810 2023-10-06T11:36:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974aae3700 1 -- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9730004810 msgr2=0x7f9730004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974aae3700 1 --2- 172.21.15.46:0/1287846823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9730004810 0x7f9730004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f973400bb60 tx=0x7f9734008040 comp rx=0 tx=0).stop 2023-10-06T11:36:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974a2e2700 1 -- 172.21.15.46:0/1287846823 reap_dead start 2023-10-06T11:36:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974aae3700 1 -- 172.21.15.46:0/1287846823 shutdown_connections 2023-10-06T11:36:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974aae3700 1 -- 172.21.15.46:0/1287846823 >> 172.21.15.46:0/1287846823 conn(0x7f9744062ef0 msgr2=0x7f97440cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.441+0000 7f974aae3700 1 -- 172.21.15.46:0/1287846823 shutdown_connections 2023-10-06T11:36:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.445+0000 7f974aae3700 1 -- 172.21.15.46:0/1287846823 wait complete. 2023-10-06T11:36:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.445+0000 7f974aae3700 1 Processor -- start 2023-10-06T11:36:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.445+0000 7f974aae3700 1 -- start start 2023-10-06T11:36:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 0x7f974415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f974416e4c0 0x7f974415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974415fac0 con 0x7f9730004810 2023-10-06T11:36:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974415eeb0 con 0x7f97440b1160 2023-10-06T11:36:02.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974415f190 con 0x7f974416e4c0 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48890/0 (socket says 172.21.15.46:48890) 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 -- 172.21.15.46:0/256740883 learned_addr learned my addr 172.21.15.46:0/256740883 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974a2e2700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f974416e4c0 0x7f974415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f97492e0700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 0x7f974415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 -- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f974416e4c0 msgr2=0x7f974415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f974416e4c0 0x7f974415db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 -- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 msgr2=0x7f974415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 0x7f974415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 -- 172.21.15.46:0/256740883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974415f470 con 0x7f9730004810 2023-10-06T11:36:02.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f97492e0700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 0x7f974415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9749ae1700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9740007c80 tx=0x7f97400021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97400133f0 con 0x7f9730004810 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740013590 con 0x7f9730004810 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740013950 con 0x7f9730004810 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974415f750 con 0x7f9730004810 2023-10-06T11:36:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.449+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974415e0a0 con 0x7f9730004810 2023-10-06T11:36:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.453+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744064590 con 0x7f9730004810 2023-10-06T11:36:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.453+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9740013d00 con 0x7f9730004810 2023-10-06T11:36:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.453+0000 7f9739ffb700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 0x7f9728062bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.453+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f974007dbf0 con 0x7f9730004810 2023-10-06T11:36:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.457+0000 7f97492e0700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 0x7f9728062bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.457+0000 7f97492e0700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 0x7f9728062bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f973400bb60 tx=0x7f973401b000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.461+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f974004dd60 con 0x7f9730004810 2023-10-06T11:36:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.673+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f974404b5c0 con 0x7f9730004810 2023-10-06T11:36:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.673+0000 7f9739ffb700 1 -- 172.21.15.46:0/256740883 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f9740051380 con 0x7f9730004810 2023-10-06T11:36:02.687 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:02.687 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/915846116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":915846116},{"type":"v1","addr":"172.21.15.46:6835","nonce":915846116}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6368,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1791421425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":1791421425},{"type":"v1","addr":"172.21.15.46:6837","nonce":1791421425}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6374,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/9733027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":9733027},{"type":"v1","addr":"172.21.15.132:6835","nonce":9733027}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":7522,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4229846943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4229846943},{"type":"v1","addr":"172.21.15.132:6834","nonce":4229846943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[]} 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 msgr2=0x7f9728062bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 0x7f9728062bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f973400bb60 tx=0x7f973401b000 comp rx=0 tx=0).stop 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 msgr2=0x7f97441551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9740007c80 tx=0x7f97400021d0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 shutdown_connections 2023-10-06T11:36:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9728060720 0x7f9728062bd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f974416e4c0 0x7f974415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97440b1160 0x7f974415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 --2- 172.21.15.46:0/256740883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9730004810 0x7f97441551e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 >> 172.21.15.46:0/256740883 conn(0x7f9744062ef0 msgr2=0x7f97440c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 shutdown_connections 2023-10-06T11:36:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.677+0000 7f974aae3700 1 -- 172.21.15.46:0/256740883 wait complete. 2023-10-06T11:36:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 52 2023-10-06T11:36:02.713 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:36:02.713 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:36:02.714 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:36:02.714 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:36:02.714 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:36:02.714 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:36:02.714 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:36:02.714 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:36:02.714 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:36:02.715 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:36:02.715 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:36:02.715 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:36:02.715 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:36:02.715 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:36:02.715 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:36:02.715 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:36:02.736 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:36:02.736 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:36:02.737 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:36:02.737 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:36:02.739 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:36:02.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:36:02.740 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:36:02.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:36:02.784 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:36:02.784 DEBUG:teuthology.parallel:result is None 2023-10-06T11:36:02.787 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:36:02.787 DEBUG:teuthology.parallel:result is None 2023-10-06T11:36:02.793 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:36:02.794 DEBUG:teuthology.parallel:result is None 2023-10-06T11:36:02.797 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:36:02.797 DEBUG:teuthology.parallel:result is None 2023-10-06T11:36:02.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:36:02.892 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:36:02.893 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:36:02.899 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:36:02.902 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:36:02.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.945+0000 7f5dcb345700 1 Processor -- start 2023-10-06T11:36:02.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.945+0000 7f5dcb345700 1 -- start start 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dcb345700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 0x7f5dc40cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dcb345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 0x7f5dc4162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dcb345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cc150 con 0x7f5dc40ccb50 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dcb345700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40cfbb0 con 0x7f5dc415c770 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dcb345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40cdbf0 con 0x7f5dc40cec00 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dc88e0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5dc40ccb50 0x7f5dc40ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58166/0 (socket says 172.21.15.46:58166) 2023-10-06T11:36:02.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dc88e0700 1 -- 172.21.15.46:0/4116113045 learned_addr learned my addr 172.21.15.46:0/4116113045 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:02.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dc98e2700 1 --2- 172.21.15.46:0/4116113045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 0x7f5dc4162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/4116113045 >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 0x7f5dc40cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.949+0000 7f5dbbfff700 1 -- 172.21.15.46:0/4116113045 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400092073 0 0) 0x7f5dc40cc150 con 0x7f5dc40ccb50 2023-10-06T11:36:02.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dbbfff700 1 -- 172.21.15.46:0/4116113045 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db0003570 con 0x7f5dc40ccb50 2023-10-06T11:36:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc98e2700 1 -- 172.21.15.46:0/4116113045 >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 msgr2=0x7f5dc40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc98e2700 1 --2- 172.21.15.46:0/4116113045 >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 0x7f5dc40cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc98e2700 1 -- 172.21.15.46:0/4116113045 >> v1:172.21.15.46:6789/0 conn(0x7f5dc40ccb50 legacy=0x7f5dc40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc98e2700 1 -- 172.21.15.46:0/4116113045 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40cb280 con 0x7f5dc40cec00 2023-10-06T11:36:02.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/4116113045 >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 0x7f5dc40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dc98e2700 1 --2- 172.21.15.46:0/4116113045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 0x7f5dc4162e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cfcf0 tx=0x7f5dbc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab0957f338ce964 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dbbfff700 1 -- 172.21.15.46:0/4116113045 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc016070 con 0x7f5dc40cec00 2023-10-06T11:36:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dbbfff700 1 -- 172.21.15.46:0/4116113045 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00bab0 con 0x7f5dc40cec00 2023-10-06T11:36:02.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dbbfff700 1 -- 172.21.15.46:0/4116113045 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00e070 con 0x7f5dc40cec00 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 -- 172.21.15.46:0/4116113045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 msgr2=0x7f5dc4162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 --2- 172.21.15.46:0/4116113045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 0x7f5dc4162e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40cfcf0 tx=0x7f5dbc009780 comp rx=0 tx=0).stop 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 -- 172.21.15.46:0/4116113045 shutdown_connections 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 --2- 172.21.15.46:0/4116113045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc40cec00 0x7f5dc4162e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 --2- 172.21.15.46:0/4116113045 >> v2:172.21.15.132:3300/0 conn(0x7f5dc415c770 0x7f5dc40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 -- 172.21.15.46:0/4116113045 >> 172.21.15.46:0/4116113045 conn(0x7f5dc4099480 msgr2=0x7f5dc406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 -- 172.21.15.46:0/4116113045 shutdown_connections 2023-10-06T11:36:02.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.953+0000 7f5dcb345700 1 -- 172.21.15.46:0/4116113045 wait complete. 2023-10-06T11:36:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 Processor -- start 2023-10-06T11:36:02.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 -- start start 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 0x7f5dc40c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc415c770 0x7f5dc4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc409aff0 con 0x7f5dc40cec00 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c8610 con 0x7f5dc40ccb50 2023-10-06T11:36:02.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c7830 con 0x7f5dc415c770 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48914/0 (socket says 172.21.15.46:48914) 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 -- 172.21.15.46:0/3250349055 learned_addr learned my addr 172.21.15.46:0/3250349055 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc98e2700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc415c770 0x7f5dc4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 0x7f5dc40c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 -- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc415c770 msgr2=0x7f5dc4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc415c770 0x7f5dc4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 -- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 msgr2=0x7f5dc40c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 0x7f5dc40c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 -- 172.21.15.46:0/3250349055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00093c0 con 0x7f5dc40cec00 2023-10-06T11:36:02.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 0x7f5dc40c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dc88e0700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5db400c7c0 tx=0x7f5db4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4005b30 con 0x7f5dc40cec00 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4005c90 con 0x7f5dc40cec00 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4007340 con 0x7f5dc40cec00 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.957+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc009a20 con 0x7f5dc40cec00 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.961+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40c7b10 con 0x7f5dc40cec00 2023-10-06T11:36:02.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.961+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc40660f0 con 0x7f5dc40cec00 2023-10-06T11:36:02.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.961+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5db401b400 con 0x7f5dc40cec00 2023-10-06T11:36:02.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.965+0000 7f5db97fa700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 0x7f5da8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.965+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5db407c790 con 0x7f5dc40cec00 2023-10-06T11:36:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.965+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 0x7f5da8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.965+0000 7f5dc90e1700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 0x7f5da8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009390 tx=0x7f5dc0008040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:02.969+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5db404c980 con 0x7f5dc40cec00 2023-10-06T11:36:03.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.149+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5dc404b5c0 con 0x7f5dc40cec00 2023-10-06T11:36:03.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.769+0000 7f5db97fa700 1 -- 172.21.15.46:0/3250349055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5db404ffa0 con 0x7f5dc40cec00 2023-10-06T11:36:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 msgr2=0x7f5da8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 0x7f5da8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009390 tx=0x7f5dc0008040 comp rx=0 tx=0).stop 2023-10-06T11:36:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 msgr2=0x7f5dc40c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f5db400c7c0 tx=0x7f5db4007e00 comp rx=0 tx=0).stop 2023-10-06T11:36:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 shutdown_connections 2023-10-06T11:36:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5da8060670 0x7f5da8062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dc415c770 0x7f5dc4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc40cec00 0x7f5dc40c72f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 --2- 172.21.15.46:0/3250349055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dc40ccb50 0x7f5dc40c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 >> 172.21.15.46:0/3250349055 conn(0x7f5dc4099480 msgr2=0x7f5dc4068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:03.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 shutdown_connections 2023-10-06T11:36:03.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.773+0000 7f5dcb345700 1 -- 172.21.15.46:0/3250349055 wait complete. 2023-10-06T11:36:03.791 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:36:03.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:36:03.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.961+0000 7fdf98185700 1 Processor -- start 2023-10-06T11:36:03.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 -- start start 2023-10-06T11:36:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdf9006e8a0 0x7fdf9006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf900ad6c0 0x7fdf900adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9006d460 con 0x7fdf9015ab60 2023-10-06T11:36:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9006ed40 con 0x7fdf900ad6c0 2023-10-06T11:36:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf98185700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900c7000 con 0x7fdf9006e8a0 2023-10-06T11:36:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf96722700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf900ad6c0 0x7fdf900adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf95720700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdf9006e8a0 0x7fdf9006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf96722700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf900ad6c0 0x7fdf900adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60430/0 (socket says 172.21.15.46:60430) 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.965+0000 7fdf96722700 1 -- 172.21.15.46:0/418126908 learned_addr learned my addr 172.21.15.46:0/418126908 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf96722700 1 -- 172.21.15.46:0/418126908 >> v2:172.21.15.132:3300/0 conn(0x7fdf9006e8a0 msgr2=0x7fdf9006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf96722700 1 --2- 172.21.15.46:0/418126908 >> v2:172.21.15.132:3300/0 conn(0x7fdf9006e8a0 0x7fdf9006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf96722700 1 -- 172.21.15.46:0/418126908 >> v1:172.21.15.132:6789/0 conn(0x7fdf9015ab60 legacy=0x7fdf9015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf94f1f700 1 -- 172.21.15.46:0/418126908 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426520184 0 0) 0x7fdf9006d460 con 0x7fdf9015ab60 2023-10-06T11:36:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf96722700 1 -- 172.21.15.46:0/418126908 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf900c72c0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf96722700 1 --2- 172.21.15.46:0/418126908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900adaf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006f0b0 tx=0x7fdf88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291088d5d4cd7ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf94f1f700 1 -- 172.21.15.46:0/418126908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88009e70 con 0x7fdf900ad6c0 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf94f1f700 1 -- 172.21.15.46:0/418126908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8800b860 con 0x7fdf900ad6c0 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf94f1f700 1 -- 172.21.15.46:0/418126908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800bc00 con 0x7fdf900ad6c0 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 -- 172.21.15.46:0/418126908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 msgr2=0x7fdf900adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 --2- 172.21.15.46:0/418126908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900adaf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006f0b0 tx=0x7fdf88009250 comp rx=0 tx=0).stop 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 -- 172.21.15.46:0/418126908 shutdown_connections 2023-10-06T11:36:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 --2- 172.21.15.46:0/418126908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900adaf0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 --2- 172.21.15.46:0/418126908 >> v2:172.21.15.132:3300/0 conn(0x7fdf9006e8a0 0x7fdf9006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 -- 172.21.15.46:0/418126908 >> 172.21.15.46:0/418126908 conn(0x7fdf900cbe30 msgr2=0x7fdf900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 -- 172.21.15.46:0/418126908 shutdown_connections 2023-10-06T11:36:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.969+0000 7fdf98185700 1 -- 172.21.15.46:0/418126908 wait complete. 2023-10-06T11:36:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.973+0000 7fdf98185700 1 Processor -- start 2023-10-06T11:36:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.973+0000 7fdf98185700 1 -- start start 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9006e8a0 0x7fdf900aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015ab60 0x7fdf900c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900b01c0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900af5b0 con 0x7fdf9006e8a0 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900af890 con 0x7fdf9015ab60 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60436/0 (socket says 172.21.15.46:60436) 2023-10-06T11:36:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 -- 172.21.15.46:0/1511126664 learned_addr learned my addr 172.21.15.46:0/1511126664 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95f21700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9006e8a0 0x7fdf900aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf96722700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015ab60 0x7fdf900c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 -- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015ab60 msgr2=0x7fdf900c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015ab60 0x7fdf900c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 -- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9006e8a0 msgr2=0x7fdf900aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9006e8a0 0x7fdf900aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 -- 172.21.15.46:0/1511126664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880094f0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf95720700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e930 tx=0x7fdf800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80013680 con 0x7fdf900ad6c0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf800137e0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf900afb70 con 0x7fdf900ad6c0 2023-10-06T11:36:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80013b80 con 0x7fdf900ad6c0 2023-10-06T11:36:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.977+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf900b0650 con 0x7fdf900ad6c0 2023-10-06T11:36:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.981+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf900b0370 con 0x7fdf900ad6c0 2023-10-06T11:36:03.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.981+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fdf8000c3e0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.985+0000 7fdf85ffb700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 0x7fdf74062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.985+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fdf8007c9a0 con 0x7fdf900ad6c0 2023-10-06T11:36:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.985+0000 7fdf95f21700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 0x7fdf74062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.985+0000 7fdf95f21700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 0x7fdf74062b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c5f0 tx=0x7fdf8c001b10 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:03.989+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdf900b0370 con 0x7fdf900ad6c0 2023-10-06T11:36:04.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.217+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdf90063f90 con 0x7fdf900ad6c0 2023-10-06T11:36:04.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.217+0000 7fdf85ffb700 1 -- 172.21.15.46:0/1511126664 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fdf80017090 con 0x7fdf900ad6c0 2023-10-06T11:36:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:04.232 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:36:04.232 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.233 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:36:04.234 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.235 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:36:04.238 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.239 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.241 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:36:04.242 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.243 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:36:04.244 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:04.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:36:04.249 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:36:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 msgr2=0x7fdf74062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 0x7fdf74062b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00c5f0 tx=0x7fdf8c001b10 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 msgr2=0x7fdf900c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e930 tx=0x7fdf800091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 shutdown_connections 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdf740606a0 0x7fdf74062b50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf9015ab60 0x7fdf900c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf900ad6c0 0x7fdf900c2ee0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 --2- 172.21.15.46:0/1511126664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf9006e8a0 0x7fdf900aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 >> 172.21.15.46:0/1511126664 conn(0x7fdf900cbe30 msgr2=0x7fdf90150130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 shutdown_connections 2023-10-06T11:36:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.225+0000 7fdf98185700 1 -- 172.21.15.46:0/1511126664 wait complete. 2023-10-06T11:36:04.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:36:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.409+0000 7fbfc606e700 1 Processor -- start 2023-10-06T11:36:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.409+0000 7fbfc606e700 1 -- start start 2023-10-06T11:36:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfc606e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfc015ab60 0x7fbfc006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfc606e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbfc015af30 0x7fbfc006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfc606e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc006d460 con 0x7fbfc00ad6c0 2023-10-06T11:36:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfc606e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00c7000 con 0x7fbfc015ab60 2023-10-06T11:36:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfc606e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00c72c0 con 0x7fbfc015af30 2023-10-06T11:36:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfc015ab60 0x7fbfc006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfc015ab60 0x7fbfc006e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60452/0 (socket says 172.21.15.46:60452) 2023-10-06T11:36:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 -- 172.21.15.46:0/97621912 learned_addr learned my addr 172.21.15.46:0/97621912 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbeffd700 1 --2- 172.21.15.46:0/97621912 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015af30 0x7fbfc006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 -- 172.21.15.46:0/97621912 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015af30 msgr2=0x7fbfc006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 --2- 172.21.15.46:0/97621912 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015af30 0x7fbfc006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 -- 172.21.15.46:0/97621912 >> v1:172.21.15.132:6789/0 conn(0x7fbfc00ad6c0 legacy=0x7fbfc00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 -- 172.21.15.46:0/97621912 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00c75a0 con 0x7fbfc015ab60 2023-10-06T11:36:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.413+0000 7fbfbf7fe700 1 --2- 172.21.15.46:0/97621912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015ab60 0x7fbfc006e8a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbfc006f0b0 tx=0x7fbfb0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d8f8d6ac0eafbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfbe7fc700 1 -- 172.21.15.46:0/97621912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0005c10 con 0x7fbfc015ab60 2023-10-06T11:36:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfbe7fc700 1 -- 172.21.15.46:0/97621912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0005d70 con 0x7fbfc015ab60 2023-10-06T11:36:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfbe7fc700 1 -- 172.21.15.46:0/97621912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000b840 con 0x7fbfc015ab60 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 -- 172.21.15.46:0/97621912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015ab60 msgr2=0x7fbfc006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 --2- 172.21.15.46:0/97621912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015ab60 0x7fbfc006e8a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbfc006f0b0 tx=0x7fbfb0009220 comp rx=0 tx=0).stop 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 -- 172.21.15.46:0/97621912 shutdown_connections 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 --2- 172.21.15.46:0/97621912 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015af30 0x7fbfc006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 --2- 172.21.15.46:0/97621912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015ab60 0x7fbfc006e8a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 -- 172.21.15.46:0/97621912 >> 172.21.15.46:0/97621912 conn(0x7fbfc00cbe30 msgr2=0x7fbfc00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 -- 172.21.15.46:0/97621912 shutdown_connections 2023-10-06T11:36:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.417+0000 7fbfc606e700 1 -- 172.21.15.46:0/97621912 wait complete. 2023-10-06T11:36:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 Processor -- start 2023-10-06T11:36:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 -- start start 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc00ad6c0 0x7fbfc00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 0x7fbfc00a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0168340 con 0x7fbfc015af30 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c3e00 con 0x7fbfc015ab60 2023-10-06T11:36:04.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfc606e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c40e0 con 0x7fbfc00ad6c0 2023-10-06T11:36:04.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60458/0 (socket says 172.21.15.46:60458) 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 -- 172.21.15.46:0/1581456884 learned_addr learned my addr 172.21.15.46:0/1581456884 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbf7fe700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc00ad6c0 0x7fbfc00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbeffd700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 0x7fbfc00a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 -- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc00ad6c0 msgr2=0x7fbfc00c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc00ad6c0 0x7fbfc00c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 -- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 msgr2=0x7fbfc00a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 0x7fbfc00a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 -- 172.21.15.46:0/1581456884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0009480 con 0x7fbfc015af30 2023-10-06T11:36:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbeffd700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 0x7fbfc00a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.421+0000 7fbfbffff700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800bc30 tx=0x7fbfb8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800c900 con 0x7fbfc015af30 2023-10-06T11:36:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800caa0 con 0x7fbfc015af30 2023-10-06T11:36:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00c43c0 con 0x7fbfc015af30 2023-10-06T11:36:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800ce60 con 0x7fbfc015af30 2023-10-06T11:36:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc00c4980 con 0x7fbfc015af30 2023-10-06T11:36:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.425+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc006e1a0 con 0x7fbfc015af30 2023-10-06T11:36:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.429+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbfb8014620 con 0x7fbfc015af30 2023-10-06T11:36:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.429+0000 7fbfa77fe700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 0x7fbfac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.429+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7fbfb807ca40 con 0x7fbfc015af30 2023-10-06T11:36:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.429+0000 7fbfbf7fe700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 0x7fbfac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.429+0000 7fbfbf7fe700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 0x7fbfac062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0003080 tx=0x7fbfb0015000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.437+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbfb804abe0 con 0x7fbfc015af30 2023-10-06T11:36:04.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.609+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfc00c4ba0 con 0x7fbfc015af30 2023-10-06T11:36:04.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.609+0000 7fbfa77fe700 1 -- 172.21.15.46:0/1581456884 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+14331 (secure 0 0 0) 0x7fbfc006e1a0 con 0x7fbfc015af30 2023-10-06T11:36:04.624 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:04.624 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":118,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:03.786733+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 msgr2=0x7fbfac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 0x7fbfac062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0003080 tx=0x7fbfb0015000 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 msgr2=0x7fbfc00c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800bc30 tx=0x7fbfb8009820 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 shutdown_connections 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfac060670 0x7fbfac062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc015af30 0x7fbfc00c5980 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc015ab60 0x7fbfc00a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 --2- 172.21.15.46:0/1581456884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc00ad6c0 0x7fbfc00c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 >> 172.21.15.46:0/1581456884 conn(0x7fbfc00cbe30 msgr2=0x7fbfc006a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 shutdown_connections 2023-10-06T11:36:04.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.613+0000 7fbfc606e700 1 -- 172.21.15.46:0/1581456884 wait complete. 2023-10-06T11:36:04.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:36:04.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:36:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.801+0000 7f8d316e3700 1 Processor -- start 2023-10-06T11:36:04.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.801+0000 7f8d316e3700 1 -- start start 2023-10-06T11:36:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d316e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 0x7f8d2c067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d316e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0ab550 con 0x7f8d2c159340 2023-10-06T11:36:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d316e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0aa980 con 0x7f8d2c067590 2023-10-06T11:36:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d316e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c067a30 con 0x7f8d2c158f70 2023-10-06T11:36:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d2c159340 0x7f8d2c0ac990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56908/0 (socket says 172.21.15.46:56908) 2023-10-06T11:36:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2b7fe700 1 -- 172.21.15.46:0/418174490 learned_addr learned my addr 172.21.15.46:0/418174490 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2bfff700 1 --2- 172.21.15.46:0/418174490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 0x7f8d2c067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346482453 0 0) 0x7f8d2c0ab550 con 0x7f8d2c159340 2023-10-06T11:36:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d14003570 con 0x7f8d2c159340 2023-10-06T11:36:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2bfff700 1 -- 172.21.15.46:0/418174490 >> v1:172.21.15.132:6789/0 conn(0x7f8d2c067590 legacy=0x7f8d2c0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717883014 0 0) 0x7f8d2c0aa980 con 0x7f8d2c067590 2023-10-06T11:36:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2bfff700 1 -- 172.21.15.46:0/418174490 >> v1:172.21.15.46:6789/0 conn(0x7f8d2c159340 legacy=0x7f8d2c0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2bfff700 1 -- 172.21.15.46:0/418174490 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0aac20 con 0x7f8d2c158f70 2023-10-06T11:36:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.805+0000 7f8d2bfff700 1 --2- 172.21.15.46:0/418174490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 0x7f8d2c067050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009910 tx=0x7f8d1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c41d9a59d915ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20001470 con 0x7f8d2c158f70 2023-10-06T11:36:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c011420 con 0x7f8d2c158f70 2023-10-06T11:36:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d2affd700 1 -- 172.21.15.46:0/418174490 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0117c0 con 0x7f8d2c158f70 2023-10-06T11:36:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 -- 172.21.15.46:0/418174490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 msgr2=0x7f8d2c067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 --2- 172.21.15.46:0/418174490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 0x7f8d2c067050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009910 tx=0x7f8d1c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 -- 172.21.15.46:0/418174490 shutdown_connections 2023-10-06T11:36:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 --2- 172.21.15.46:0/418174490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c158f70 0x7f8d2c067050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 -- 172.21.15.46:0/418174490 >> 172.21.15.46:0/418174490 conn(0x7f8d2c0ccd50 msgr2=0x7f8d2c0cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 -- 172.21.15.46:0/418174490 shutdown_connections 2023-10-06T11:36:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.809+0000 7f8d316e3700 1 -- 172.21.15.46:0/418174490 wait complete. 2023-10-06T11:36:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 Processor -- start 2023-10-06T11:36:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 -- start start 2023-10-06T11:36:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d2c158f70 0x7f8d2c16a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 0x7f8d2c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c169f70 con 0x7f8d2c067590 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1693f0 con 0x7f8d2c158f70 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.813+0000 7f8d316e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1696d0 con 0x7f8d2c159340 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60468/0 (socket says 172.21.15.46:60468) 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 -- 172.21.15.46:0/4151307465 learned_addr learned my addr 172.21.15.46:0/4151307465 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2b7fe700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d2c158f70 0x7f8d2c16a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d30ee2700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 0x7f8d2c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 -- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 msgr2=0x7f8d2c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 0x7f8d2c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 -- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d2c158f70 msgr2=0x7f8d2c16a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d2c158f70 0x7f8d2c16a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 -- 172.21.15.46:0/4151307465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c009420 con 0x7f8d2c067590 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d30ee2700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 0x7f8d2c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d2bfff700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c15cc30 tx=0x7f8d1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c010b70 con 0x7f8d2c067590 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c010cd0 con 0x7f8d2c067590 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c1699b0 con 0x7f8d2c067590 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c01d680 con 0x7f8d2c067590 2023-10-06T11:36:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c169010 con 0x7f8d2c067590 2023-10-06T11:36:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.817+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c064590 con 0x7f8d2c067590 2023-10-06T11:36:04.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.821+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8d1c01d9f0 con 0x7f8d2c067590 2023-10-06T11:36:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.821+0000 7f8d13fff700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 0x7f8d18062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.821+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f8d1c07e280 con 0x7f8d2c067590 2023-10-06T11:36:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.821+0000 7f8d2b7fe700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 0x7f8d18062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.825+0000 7f8d2b7fe700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 0x7f8d18062af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0aa2b0 tx=0x7f8d20000bc0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:04.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:04.829+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8d1c04e6f0 con 0x7f8d2c067590 2023-10-06T11:36:05.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.001+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d2c04b5c0 con 0x7f8d2c067590 2023-10-06T11:36:05.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.001+0000 7f8d13fff700 1 -- 172.21.15.46:0/4151307465 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d1c051d10 con 0x7f8d2c067590 2023-10-06T11:36:05.015 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 msgr2=0x7f8d18062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 0x7f8d18062af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0aa2b0 tx=0x7f8d20000bc0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 msgr2=0x7f8d2c0aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c15cc30 tx=0x7f8d1c017000 comp rx=0 tx=0).stop 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 shutdown_connections 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8d18060640 0x7f8d18062af0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d2c159340 0x7f8d2c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d2c158f70 0x7f8d2c16a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 --2- 172.21.15.46:0/4151307465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d2c067590 0x7f8d2c0aff50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 >> 172.21.15.46:0/4151307465 conn(0x7f8d2c0ccd50 msgr2=0x7f8d2c155480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 shutdown_connections 2023-10-06T11:36:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.005+0000 7f8d316e3700 1 -- 172.21.15.46:0/4151307465 wait complete. 2023-10-06T11:36:05.040 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:36:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 Processor -- start 2023-10-06T11:36:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 -- start start 2023-10-06T11:36:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f343c06ed10 0x7f343c0c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0c1bf0 con 0x7f343c0c6470 2023-10-06T11:36:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c06cf80 con 0x7f343c06e8a0 2023-10-06T11:36:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.197+0000 7f3441145700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c06d260 con 0x7f343c06ed10 2023-10-06T11:36:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f343c0c6470 0x7f343c0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56916/0 (socket says 172.21.15.46:56916) 2023-10-06T11:36:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343ad9d700 1 -- 172.21.15.46:0/229285845 learned_addr learned my addr 172.21.15.46:0/229285845 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343b59e700 1 --2- 172.21.15.46:0/229285845 >> v2:172.21.15.132:3300/0 conn(0x7f343c06ed10 0x7f343c0c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822764078 0 0) 0x7f343c0c1bf0 con 0x7f343c0c6470 2023-10-06T11:36:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424003570 con 0x7f343c0c6470 2023-10-06T11:36:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343b59e700 1 -- 172.21.15.46:0/229285845 >> v1:172.21.15.132:6789/0 conn(0x7f343c06e8a0 legacy=0x7f343c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530411023 0 0) 0x7f343c06cf80 con 0x7f343c06e8a0 2023-10-06T11:36:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343b59e700 1 -- 172.21.15.46:0/229285845 >> v1:172.21.15.46:6789/0 conn(0x7f343c0c6470 legacy=0x7f343c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343b59e700 1 -- 172.21.15.46:0/229285845 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0c1880 con 0x7f343c06ed10 2023-10-06T11:36:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f343b59e700 1 --2- 172.21.15.46:0/229285845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c06ed10 0x7f343c0c2080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3434009e60 tx=0x7f3434009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d0e50e899c5e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400e070 con 0x7f343c06ed10 2023-10-06T11:36:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343400bb60 con 0x7f343c06ed10 2023-10-06T11:36:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3439d9b700 1 -- 172.21.15.46:0/229285845 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434012430 con 0x7f343c06ed10 2023-10-06T11:36:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3441145700 1 -- 172.21.15.46:0/229285845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c06ed10 msgr2=0x7f343c0c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.201+0000 7f3441145700 1 --2- 172.21.15.46:0/229285845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c06ed10 0x7f343c0c2080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3434009e60 tx=0x7f3434009780 comp rx=0 tx=0).stop 2023-10-06T11:36:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 -- 172.21.15.46:0/229285845 shutdown_connections 2023-10-06T11:36:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 --2- 172.21.15.46:0/229285845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c06ed10 0x7f343c0c2080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 -- 172.21.15.46:0/229285845 >> 172.21.15.46:0/229285845 conn(0x7f343c0cbe30 msgr2=0x7f343c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 -- 172.21.15.46:0/229285845 shutdown_connections 2023-10-06T11:36:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 -- 172.21.15.46:0/229285845 wait complete. 2023-10-06T11:36:05.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 Processor -- start 2023-10-06T11:36:05.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.205+0000 7f3441145700 1 -- start start 2023-10-06T11:36:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 0x7f343c13d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c0c6470 0x7f343c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0aebd0 con 0x7f343c06ed10 2023-10-06T11:36:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0aee20 con 0x7f343c0c6470 2023-10-06T11:36:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0af100 con 0x7f343c06e8a0 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60472/0 (socket says 172.21.15.46:60472) 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 -- 172.21.15.46:0/2068276428 learned_addr learned my addr 172.21.15.46:0/2068276428 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343ad9d700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 0x7f343c13d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343b59e700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c0c6470 0x7f343c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 -- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 msgr2=0x7f343c13d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 0x7f343c13d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 -- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c0c6470 msgr2=0x7f343c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c0c6470 0x7f343c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 -- 172.21.15.46:0/2068276428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009a20 con 0x7f343c06ed10 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343ad9d700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 0x7f343c13d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f343a59c700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f343000c7e0 tx=0x7f3430007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430011590 con 0x7f343c06ed10 2023-10-06T11:36:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3430011710 con 0x7f343c06ed10 2023-10-06T11:36:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430011ad0 con 0x7f343c06ed10 2023-10-06T11:36:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.209+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0af3e0 con 0x7f343c06ed10 2023-10-06T11:36:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.213+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0af9a0 con 0x7f343c06ed10 2023-10-06T11:36:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.213+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0660f0 con 0x7f343c06ed10 2023-10-06T11:36:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.213+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f343001b3b0 con 0x7f343c06ed10 2023-10-06T11:36:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.213+0000 7f3422ffd700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 0x7f3428062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.213+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f343007bee0 con 0x7f343c06ed10 2023-10-06T11:36:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.217+0000 7f343ad9d700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 0x7f3428062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.217+0000 7f343ad9d700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 0x7f3428062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f342c0058d0 tx=0x7f342c009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.221+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f343004c350 con 0x7f343c06ed10 2023-10-06T11:36:05.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.393+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f343c13dd80 con 0x7f343c06ed10 2023-10-06T11:36:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.393+0000 7f3422ffd700 1 -- 172.21.15.46:0/2068276428 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3430017020 con 0x7f343c06ed10 2023-10-06T11:36:05.407 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 msgr2=0x7f3428062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 0x7f3428062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f342c0058d0 tx=0x7f342c009040 comp rx=0 tx=0).stop 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 msgr2=0x7f343c0ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f343000c7e0 tx=0x7f3430007df0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 shutdown_connections 2023-10-06T11:36:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3428060670 0x7f3428062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f343c0c6470 0x7f343c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f343c06ed10 0x7f343c0ca660 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 --2- 172.21.15.46:0/2068276428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f343c06e8a0 0x7f343c13d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 >> 172.21.15.46:0/2068276428 conn(0x7f343c0cbe30 msgr2=0x7f343c0aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 shutdown_connections 2023-10-06T11:36:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.397+0000 7f3441145700 1 -- 172.21.15.46:0/2068276428 wait complete. 2023-10-06T11:36:05.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:36:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.585+0000 7f99602ae700 1 Processor -- start 2023-10-06T11:36:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 -- start start 2023-10-06T11:36:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f995815c770 0x7f99580cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f99580cec00 0x7f9958162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99580cc150 con 0x7f99580ccb50 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580cfbb0 con 0x7f995815c770 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.589+0000 7f99602ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99580cdbf0 con 0x7f99580cec00 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f995815c770 0x7f99580cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f995815c770 0x7f99580cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60478/0 (socket says 172.21.15.46:60478) 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 -- 172.21.15.46:0/2373048211 learned_addr learned my addr 172.21.15.46:0/2373048211 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e84b700 1 --2- 172.21.15.46:0/2373048211 >> v2:172.21.15.132:3300/0 conn(0x7f99580cec00 0x7f9958162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 -- 172.21.15.46:0/2373048211 >> v2:172.21.15.132:3300/0 conn(0x7f99580cec00 msgr2=0x7f9958162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 --2- 172.21.15.46:0/2373048211 >> v2:172.21.15.132:3300/0 conn(0x7f99580cec00 0x7f9958162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 -- 172.21.15.46:0/2373048211 >> v1:172.21.15.46:6789/0 conn(0x7f99580ccb50 legacy=0x7f99580ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 -- 172.21.15.46:0/2373048211 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580cb280 con 0x7f995815c770 2023-10-06T11:36:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995e04a700 1 --2- 172.21.15.46:0/2373048211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f99580cad40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9958064360 tx=0x7f9954009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70caedbc7251e9ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995d048700 1 -- 172.21.15.46:0/2373048211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954005c10 con 0x7f995815c770 2023-10-06T11:36:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995d048700 1 -- 172.21.15.46:0/2373048211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954005d70 con 0x7f995815c770 2023-10-06T11:36:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f995d048700 1 -- 172.21.15.46:0/2373048211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995400b880 con 0x7f995815c770 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 -- 172.21.15.46:0/2373048211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 msgr2=0x7f99580cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 --2- 172.21.15.46:0/2373048211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f99580cad40 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9958064360 tx=0x7f9954009220 comp rx=0 tx=0).stop 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 -- 172.21.15.46:0/2373048211 shutdown_connections 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 --2- 172.21.15.46:0/2373048211 >> v2:172.21.15.132:3300/0 conn(0x7f99580cec00 0x7f9958162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 --2- 172.21.15.46:0/2373048211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f99580cad40 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 -- 172.21.15.46:0/2373048211 >> 172.21.15.46:0/2373048211 conn(0x7f9958099480 msgr2=0x7f995806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 -- 172.21.15.46:0/2373048211 shutdown_connections 2023-10-06T11:36:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.593+0000 7f99602ae700 1 -- 172.21.15.46:0/2373048211 wait complete. 2023-10-06T11:36:05.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.597+0000 7f99602ae700 1 Processor -- start 2023-10-06T11:36:05.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.597+0000 7f99602ae700 1 -- start start 2023-10-06T11:36:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 0x7f9958151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99580cec00 0x7f99580c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99580c7700 con 0x7f995815c770 2023-10-06T11:36:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9958156c20 con 0x7f99580ccb50 2023-10-06T11:36:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f99602ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9958156f00 con 0x7f99580cec00 2023-10-06T11:36:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60480/0 (socket says 172.21.15.46:60480) 2023-10-06T11:36:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 -- 172.21.15.46:0/3796246297 learned_addr learned my addr 172.21.15.46:0/3796246297 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e04a700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 0x7f9958151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995d849700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99580cec00 0x7f99580c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 -- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99580cec00 msgr2=0x7f99580c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99580cec00 0x7f99580c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 -- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 msgr2=0x7f9958151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 0x7f9958151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 -- 172.21.15.46:0/3796246297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954009480 con 0x7f995815c770 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e04a700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 0x7f9958151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f995e84b700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9950007f30 tx=0x7f99500098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.601+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000eb90 con 0x7f995815c770 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000ecf0 con 0x7f995815c770 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99581571e0 con 0x7f995815c770 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950014670 con 0x7f995815c770 2023-10-06T11:36:05.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9958151780 con 0x7f995815c770 2023-10-06T11:36:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99580c78b0 con 0x7f995815c770 2023-10-06T11:36:05.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.605+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9950014a20 con 0x7f995815c770 2023-10-06T11:36:05.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.609+0000 7f994e7fc700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 0x7f993c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:05.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.609+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f995007cfd0 con 0x7f995815c770 2023-10-06T11:36:05.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.609+0000 7f995e04a700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 0x7f993c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:05.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.609+0000 7f995e04a700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 0x7f993c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9954009260 tx=0x7f9954016000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:05.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.617+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99580c78b0 con 0x7f995815c770 2023-10-06T11:36:05.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.797+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f99580660f0 con 0x7f995815c770 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.797+0000 7f994e7fc700 1 -- 172.21.15.46:0/3796246297 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9950050ed0 con 0x7f995815c770 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:36:05.812 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:05.813 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:36:05.813 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:36:05.813 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 msgr2=0x7f993c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 0x7f993c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9954009260 tx=0x7f9954016000 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 msgr2=0x7f9958156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9950007f30 tx=0x7f99500098e0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 shutdown_connections 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f993c060670 0x7f993c062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f995815c770 0x7f9958156240 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99580cec00 0x7f99580c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 --2- 172.21.15.46:0/3796246297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99580ccb50 0x7f9958151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 >> 172.21.15.46:0/3796246297 conn(0x7f9958099480 msgr2=0x7f9958068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:05.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 shutdown_connections 2023-10-06T11:36:05.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.801+0000 7f99602ae700 1 -- 172.21.15.46:0/3796246297 wait complete. 2023-10-06T11:36:05.835 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:36:05.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:36:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.989+0000 7f961886e700 1 Processor -- start 2023-10-06T11:36:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.989+0000 7f961886e700 1 -- start start 2023-10-06T11:36:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961886e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9614146ea0 0x7f961406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961886e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9614161e50 0x7f96140c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961886e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140c1800 con 0x7f961406d150 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961886e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96140c1580 con 0x7f9614161e50 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961886e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96140c6ce0 con 0x7f9614146ea0 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9612d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f961406d150 0x7f96141611c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56934/0 (socket says 172.21.15.46:56934) 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9612d9d700 1 -- 172.21.15.46:0/3583054057 learned_addr learned my addr 172.21.15.46:0/3583054057 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 --2- 172.21.15.46:0/3583054057 >> v2:172.21.15.46:3300/0 conn(0x7f9614161e50 0x7f96140c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961259c700 1 --2- 172.21.15.46:0/3583054057 >> v2:172.21.15.132:3300/0 conn(0x7f9614146ea0 0x7f961406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961159a700 1 -- 172.21.15.46:0/3583054057 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778134553 0 0) 0x7f96140c1800 con 0x7f961406d150 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961159a700 1 -- 172.21.15.46:0/3583054057 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc003570 con 0x7f961406d150 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 -- 172.21.15.46:0/3583054057 >> v2:172.21.15.132:3300/0 conn(0x7f9614146ea0 msgr2=0x7f961406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 --2- 172.21.15.46:0/3583054057 >> v2:172.21.15.132:3300/0 conn(0x7f9614146ea0 0x7f961406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 -- 172.21.15.46:0/3583054057 >> v1:172.21.15.46:6789/0 conn(0x7f961406d150 legacy=0x7f96141611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 -- 172.21.15.46:0/3583054057 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140c6460 con 0x7f9614161e50 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f9611d9b700 1 --2- 172.21.15.46:0/3583054057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1c90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f961406c340 tx=0x7f96080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f106acba16606f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961159a700 1 -- 172.21.15.46:0/3583054057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011630 con 0x7f9614161e50 2023-10-06T11:36:06.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961159a700 1 -- 172.21.15.46:0/3583054057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96080117b0 con 0x7f9614161e50 2023-10-06T11:36:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.993+0000 7f961159a700 1 -- 172.21.15.46:0/3583054057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608011b70 con 0x7f9614161e50 2023-10-06T11:36:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- 172.21.15.46:0/3583054057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 msgr2=0x7f96140c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- 172.21.15.46:0/3583054057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1c90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f961406c340 tx=0x7f96080091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- 172.21.15.46:0/3583054057 shutdown_connections 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- 172.21.15.46:0/3583054057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1c90 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- 172.21.15.46:0/3583054057 >> v2:172.21.15.132:3300/0 conn(0x7f9614146ea0 0x7f961406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- 172.21.15.46:0/3583054057 >> 172.21.15.46:0/3583054057 conn(0x7f96140cbe20 msgr2=0x7f96140cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- 172.21.15.46:0/3583054057 shutdown_connections 2023-10-06T11:36:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- 172.21.15.46:0/3583054057 wait complete. 2023-10-06T11:36:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 Processor -- start 2023-10-06T11:36:06.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- start start 2023-10-06T11:36:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f961406d150 0x7f96140c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 0x7f9614159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140ca5b0 con 0x7f9614161e50 2023-10-06T11:36:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614170c90 con 0x7f9614146ea0 2023-10-06T11:36:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:05.997+0000 7f961886e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140ca310 con 0x7f961406d150 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60498/0 (socket says 172.21.15.46:60498) 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 -- 172.21.15.46:0/4127633887 learned_addr learned my addr 172.21.15.46:0/4127633887 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f961259c700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f961406d150 0x7f96140c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9611d9b700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 0x7f9614159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 -- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f961406d150 msgr2=0x7f96140c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f961406d150 0x7f96140c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 -- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 msgr2=0x7f9614159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 0x7f9614159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 -- 172.21.15.46:0/4127633887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009420 con 0x7f9614161e50 2023-10-06T11:36:06.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9611d9b700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 0x7f9614159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f9612d9d700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f960c001460 tx=0x7f960c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00a890 con 0x7f9614161e50 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c00a9f0 con 0x7f9614161e50 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c00ad70 con 0x7f9614161e50 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9614170f30 con 0x7f9614161e50 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.001+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96141714f0 con 0x7f9614161e50 2023-10-06T11:36:06.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.005+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96141716e0 con 0x7f9614161e50 2023-10-06T11:36:06.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.005+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f960c007300 con 0x7f9614161e50 2023-10-06T11:36:06.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.005+0000 7f96027fc700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 0x7f95f4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.005+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5650+0+0 (secure 0 0 0) 0x7f960c07cd60 con 0x7f9614161e50 2023-10-06T11:36:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.005+0000 7f961259c700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 0x7f95f4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.009+0000 7f961259c700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 0x7f95f4062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9604009db0 tx=0x7f96040091a0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:06.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.013+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f960c04d250 con 0x7f9614161e50 2023-10-06T11:36:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.185+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96140c6720 con 0x7f9614161e50 2023-10-06T11:36:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.793+0000 7f96027fc700 1 -- 172.21.15.46:0/4127633887 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) v1 ==== 138+0+0 (secure 0 0 0) 0x7f960c050870 con 0x7f9614161e50 2023-10-06T11:36:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.797+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 msgr2=0x7f95f4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.797+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 0x7f95f4062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9604009db0 tx=0x7f96040091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.797+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 msgr2=0x7f96140c1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.797+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f960c001460 tx=0x7f960c001ae0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 shutdown_connections 2023-10-06T11:36:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f95f4060670 0x7f95f4062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9614161e50 0x7f96140c1b30 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9614146ea0 0x7f9614159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 --2- 172.21.15.46:0/4127633887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f961406d150 0x7f96140c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 >> 172.21.15.46:0/4127633887 conn(0x7f96140cbe20 msgr2=0x7f9614162b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 shutdown_connections 2023-10-06T11:36:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.801+0000 7f961886e700 1 -- 172.21.15.46:0/4127633887 wait complete. 2023-10-06T11:36:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:36:06.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:36:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.997+0000 7f96d14f0700 1 Processor -- start 2023-10-06T11:36:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:06.997+0000 7f96d14f0700 1 -- start start 2023-10-06T11:36:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96cc15aba0 0x7f96cc0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15af70 0x7f96cc0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96cc0adf30 0x7f96cc156880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0685e0 con 0x7f96cc0adf30 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc068e80 con 0x7f96cc15aba0 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96d14f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0ae410 con 0x7f96cc15af70 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96cc0adf30 0x7f96cc156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96cc0adf30 0x7f96cc156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60502/0 (socket says 172.21.15.46:60502) 2023-10-06T11:36:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 -- 172.21.15.46:0/3945550367 learned_addr learned my addr 172.21.15.46:0/3945550367 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96c27fc700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15af70 0x7f96cc0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96caffd700 1 --2- 172.21.15.46:0/3945550367 >> v2:172.21.15.132:3300/0 conn(0x7f96cc15aba0 0x7f96cc0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 -- 172.21.15.46:0/3945550367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15af70 msgr2=0x7f96cc0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15af70 0x7f96cc0ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 -- 172.21.15.46:0/3945550367 >> v2:172.21.15.132:3300/0 conn(0x7f96cc15aba0 msgr2=0x7f96cc0ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 --2- 172.21.15.46:0/3945550367 >> v2:172.21.15.132:3300/0 conn(0x7f96cc15aba0 0x7f96cc0ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 -- 172.21.15.46:0/3945550367 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc068830 con 0x7f96cc0adf30 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96cb7fe700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc156880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06c120 tx=0x7f96c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04057027f22c3ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96ca7fc700 1 -- 172.21.15.46:0/3945550367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400b640 con 0x7f96cc0adf30 2023-10-06T11:36:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96ca7fc700 1 -- 172.21.15.46:0/3945550367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c400b7a0 con 0x7f96cc0adf30 2023-10-06T11:36:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.001+0000 7f96ca7fc700 1 -- 172.21.15.46:0/3945550367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400bb60 con 0x7f96cc0adf30 2023-10-06T11:36:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 -- 172.21.15.46:0/3945550367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 msgr2=0x7f96cc156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:07.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc156880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06c120 tx=0x7f96c4009250 comp rx=0 tx=0).stop 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 -- 172.21.15.46:0/3945550367 shutdown_connections 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc156880 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 --2- 172.21.15.46:0/3945550367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15af70 0x7f96cc0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 --2- 172.21.15.46:0/3945550367 >> v2:172.21.15.132:3300/0 conn(0x7f96cc15aba0 0x7f96cc0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 -- 172.21.15.46:0/3945550367 >> 172.21.15.46:0/3945550367 conn(0x7f96cc0cc690 msgr2=0x7f96cc0c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 -- 172.21.15.46:0/3945550367 shutdown_connections 2023-10-06T11:36:07.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 -- 172.21.15.46:0/3945550367 wait complete. 2023-10-06T11:36:07.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.005+0000 7f96d14f0700 1 Processor -- start 2023-10-06T11:36:07.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 -- start start 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15aba0 0x7f96cc0c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 0x7f96cc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0a9600 con 0x7f96cc0adf30 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0c42f0 con 0x7f96cc15af70 2023-10-06T11:36:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96d14f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0c45d0 con 0x7f96cc15aba0 2023-10-06T11:36:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60504/0 (socket says 172.21.15.46:60504) 2023-10-06T11:36:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 -- 172.21.15.46:0/1632376758 learned_addr learned my addr 172.21.15.46:0/1632376758 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96c27fc700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15aba0 0x7f96cc0c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96cb7fe700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 0x7f96cc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 -- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15aba0 msgr2=0x7f96cc0c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15aba0 0x7f96cc0c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 -- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 msgr2=0x7f96cc06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 0x7f96cc06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96caffd700 1 -- 172.21.15.46:0/1632376758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40094f0 con 0x7f96cc0adf30 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.009+0000 7f96cb7fe700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 0x7f96cc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96caffd700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00ec80 tx=0x7f96bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00c850 con 0x7f96cc0adf30 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00c9b0 con 0x7f96cc0adf30 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc0c48b0 con 0x7f96cc0adf30 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00cd50 con 0x7f96cc0adf30 2023-10-06T11:36:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc06e320 con 0x7f96cc0adf30 2023-10-06T11:36:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.013+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc064590 con 0x7f96cc0adf30 2023-10-06T11:36:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.017+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f96bc013680 con 0x7f96cc0adf30 2023-10-06T11:36:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.017+0000 7f96c37fe700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 0x7f96ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.017+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6020+0+0 (secure 0 0 0) 0x7f96bc07d0d0 con 0x7f96cc0adf30 2023-10-06T11:36:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.017+0000 7f96c27fc700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 0x7f96ac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.017+0000 7f96c27fc700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 0x7f96ac062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009fd0 tx=0x7f96b400b040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.025+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f96bc04d3d0 con 0x7f96cc0adf30 2023-10-06T11:36:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.193+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f96cc0a9740 con 0x7f96cc0adf30 2023-10-06T11:36:07.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.809+0000 7f96c37fe700 1 -- 172.21.15.46:0/1632376758 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) v1 ==== 170+0+0 (secure 0 0 0) 0x7f96bc0509f0 con 0x7f96cc0adf30 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.821+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 msgr2=0x7f96ac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.821+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 0x7f96ac062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009fd0 tx=0x7f96b400b040 comp rx=0 tx=0).stop 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.821+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 msgr2=0x7f96cc0a7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.821+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00ec80 tx=0x7f96bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.825+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 shutdown_connections 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.825+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f96ac060670 0x7f96ac062b20 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009fd0 tx=0x7f96b400b040 comp rx=0 tx=0).stop 2023-10-06T11:36:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.825+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96cc15af70 0x7f96cc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.825+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96cc15aba0 0x7f96cc0c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.829+0000 7f96d14f0700 1 --2- 172.21.15.46:0/1632376758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96cc0adf30 0x7f96cc0a7c30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.829+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 >> 172.21.15.46:0/1632376758 conn(0x7f96cc0cc690 msgr2=0x7f96cc1592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.829+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 shutdown_connections 2023-10-06T11:36:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:07.833+0000 7f96d14f0700 1 -- 172.21.15.46:0/1632376758 wait complete. 2023-10-06T11:36:07.851 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:36:07.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:36:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.025+0000 7fb988f85700 1 Processor -- start 2023-10-06T11:36:08.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.025+0000 7fb988f85700 1 -- start start 2023-10-06T11:36:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb988f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 0x7fb9840ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb988f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb988f85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9840cc510 con 0x7fb9840b23c0 2023-10-06T11:36:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb988f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840cd500 con 0x7fb9840cd080 2023-10-06T11:36:08.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb988f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840b2be0 con 0x7fb9840b2790 2023-10-06T11:36:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb982d9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 0x7fb9840ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb982d9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45648/0 (socket says 172.21.15.46:45648) 2023-10-06T11:36:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb982d9d700 1 -- 172.21.15.46:0/2646361512 learned_addr learned my addr 172.21.15.46:0/2646361512 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb98259c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb9840b23c0 0x7fb9840cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:54336/0 (socket says 172.21.15.46:54336) 2023-10-06T11:36:08.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 -- 172.21.15.46:0/2646361512 >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 msgr2=0x7fb9840c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:08.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 --2- 172.21.15.46:0/2646361512 >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 -- 172.21.15.46:0/2646361512 >> v1:172.21.15.132:6789/0 conn(0x7fb9840b23c0 legacy=0x7fb9840cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 -- 172.21.15.46:0/2646361512 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840cb900 con 0x7fb9840b2790 2023-10-06T11:36:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb98159a700 1 -- 172.21.15.46:0/2646361512 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417747106 0 0) 0x7fb9840cc510 con 0x7fb9840b23c0 2023-10-06T11:36:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb982d9d700 1 --2- 172.21.15.46:0/2646361512 >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.029+0000 7fb981d9b700 1 --2- 172.21.15.46:0/2646361512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 0x7fb9840ccb40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb9780098e0 tx=0x7fb9780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc17d9893356171 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb98159a700 1 -- 172.21.15.46:0/2646361512 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800b640 con 0x7fb9840b2790 2023-10-06T11:36:08.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb98159a700 1 -- 172.21.15.46:0/2646361512 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97800b7a0 con 0x7fb9840b2790 2023-10-06T11:36:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb98159a700 1 -- 172.21.15.46:0/2646361512 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800bb40 con 0x7fb9840b2790 2023-10-06T11:36:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 -- 172.21.15.46:0/2646361512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 msgr2=0x7fb9840ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 --2- 172.21.15.46:0/2646361512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 0x7fb9840ccb40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb9780098e0 tx=0x7fb9780091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 -- 172.21.15.46:0/2646361512 shutdown_connections 2023-10-06T11:36:08.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 --2- 172.21.15.46:0/2646361512 >> v2:172.21.15.132:3300/0 conn(0x7fb9840cd080 0x7fb9840c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 --2- 172.21.15.46:0/2646361512 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840b2790 0x7fb9840ccb40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 -- 172.21.15.46:0/2646361512 >> 172.21.15.46:0/2646361512 conn(0x7fb984099480 msgr2=0x7fb98406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:08.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 -- 172.21.15.46:0/2646361512 shutdown_connections 2023-10-06T11:36:08.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.033+0000 7fb988f85700 1 -- 172.21.15.46:0/2646361512 wait complete. 2023-10-06T11:36:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 Processor -- start 2023-10-06T11:36:08.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 -- start start 2023-10-06T11:36:08.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 0x7fb9841599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840cd080 0x7fb98414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98416a7b0 con 0x7fb9840b2790 2023-10-06T11:36:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984159f30 con 0x7fb9840b23c0 2023-10-06T11:36:08.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb988f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb984169ba0 con 0x7fb9840cd080 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb981d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 0x7fb9841599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb98259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb98259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45662/0 (socket says 172.21.15.46:45662) 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.037+0000 7fb98259c700 1 -- 172.21.15.46:0/2879390712 learned_addr learned my addr 172.21.15.46:0/2879390712 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb982d9d700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840cd080 0x7fb98414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 -- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840cd080 msgr2=0x7fb98414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840cd080 0x7fb98414abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 -- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 msgr2=0x7fb9841599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 0x7fb9841599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 -- 172.21.15.46:0/2879390712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009ae0 con 0x7fb9840b23c0 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb981d9b700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 0x7fb9841599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:08.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb98259c700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c5e0 tx=0x7fb974001030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:08.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974005ac0 con 0x7fb9840b23c0 2023-10-06T11:36:08.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb974005c40 con 0x7fb9840b23c0 2023-10-06T11:36:08.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb978009420 con 0x7fb9840b23c0 2023-10-06T11:36:08.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740070d0 con 0x7fb9840b23c0 2023-10-06T11:36:08.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.041+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98416a160 con 0x7fb9840b23c0 2023-10-06T11:36:08.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.045+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9840660f0 con 0x7fb9840b23c0 2023-10-06T11:36:08.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.045+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb974007480 con 0x7fb9840b23c0 2023-10-06T11:36:08.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.045+0000 7fb9727fc700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 0x7fb964062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:08.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.049+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6402+0+0 (secure 0 0 0) 0x7fb97407c0b0 con 0x7fb9840b23c0 2023-10-06T11:36:08.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.049+0000 7fb981d9b700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 0x7fb964062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.049+0000 7fb981d9b700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 0x7fb964062bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9780093f0 tx=0x7fb978017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.053+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb97404c2b0 con 0x7fb9840b23c0 2023-10-06T11:36:08.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.269+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb9840cbbe0 con 0x7fb9840b23c0 2023-10-06T11:36:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.837+0000 7fb9727fc700 1 -- 172.21.15.46:0/2879390712 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb97404f8d0 con 0x7fb9840b23c0 2023-10-06T11:36:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 msgr2=0x7fb964062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 0x7fb964062bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9780093f0 tx=0x7fb978017000 comp rx=0 tx=0).stop 2023-10-06T11:36:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 msgr2=0x7fb9840b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c5e0 tx=0x7fb974001030 comp rx=0 tx=0).stop 2023-10-06T11:36:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 shutdown_connections 2023-10-06T11:36:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb964060740 0x7fb964062bf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9840cd080 0x7fb98414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9840b2790 0x7fb9841599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 --2- 172.21.15.46:0/2879390712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9840b23c0 0x7fb9840b4650 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 >> 172.21.15.46:0/2879390712 conn(0x7fb984099480 msgr2=0x7fb984158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 shutdown_connections 2023-10-06T11:36:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:08.845+0000 7fb988f85700 1 -- 172.21.15.46:0/2879390712 wait complete. 2023-10-06T11:36:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 15 and data pool 16 2023-10-06T11:36:08.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:36:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 Processor -- start 2023-10-06T11:36:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 -- start start 2023-10-06T11:36:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 0x7fc7c8159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b4f80 con 0x7fc7c8159050 2023-10-06T11:36:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b40c0 con 0x7fc7c80b5410 2023-10-06T11:36:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7ce02e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b4370 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7c77fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc7c8159050 0x7fc7c80b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56950/0 (socket says 172.21.15.46:56950) 2023-10-06T11:36:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7c77fe700 1 -- 172.21.15.46:0/1761388157 learned_addr learned my addr 172.21.15.46:0/1761388157 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/1761388157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 0x7fc7c8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.037+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328378390 0 0) 0x7fc7c80b4f80 con 0x7fc7c8159050 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b0003570 con 0x7fc7c8159050 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/1761388157 >> v1:172.21.15.132:6789/0 conn(0x7fc7c80b5410 legacy=0x7fc7c80b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144620263 0 0) 0x7fc7c80b40c0 con 0x7fc7c80b5410 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/1761388157 >> v1:172.21.15.46:6789/0 conn(0x7fc7c8159050 legacy=0x7fc7c80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/1761388157 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80b4650 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/1761388157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 0x7fc7c8159420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009910 tx=0x7fc7bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9bbfaf941c6b245 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00b680 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc00b7e0 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7c67fc700 1 -- 172.21.15.46:0/1761388157 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00bba0 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 -- 172.21.15.46:0/1761388157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 msgr2=0x7fc7c8159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/1761388157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 0x7fc7c8159420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009910 tx=0x7fc7bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 -- 172.21.15.46:0/1761388157 shutdown_connections 2023-10-06T11:36:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/1761388157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80d0aa0 0x7fc7c8159420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 -- 172.21.15.46:0/1761388157 >> 172.21.15.46:0/1761388157 conn(0x7fc7c806e780 msgr2=0x7fc7c806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 -- 172.21.15.46:0/1761388157 shutdown_connections 2023-10-06T11:36:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.041+0000 7fc7ce02e700 1 -- 172.21.15.46:0/1761388157 wait complete. 2023-10-06T11:36:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.045+0000 7fc7ce02e700 1 Processor -- start 2023-10-06T11:36:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.045+0000 7fc7ce02e700 1 -- start start 2023-10-06T11:36:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80b5410 0x7fc7c8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 0x7fc7c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815a740 con 0x7fc7c8159050 2023-10-06T11:36:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8159b30 con 0x7fc7c80d0aa0 2023-10-06T11:36:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8159e10 con 0x7fc7c80b5410 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 0x7fc7c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80b5410 0x7fc7c8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60518/0 (socket says 172.21.15.46:60518) 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 -- 172.21.15.46:0/3320913683 learned_addr learned my addr 172.21.15.46:0/3320913683 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 0x7fc7c815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45672/0 (socket says 172.21.15.46:45672) 2023-10-06T11:36:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 -- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80b5410 msgr2=0x7fc7c8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80b5410 0x7fc7c8157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 -- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 msgr2=0x7fc7c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 0x7fc7c815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 -- 172.21.15.46:0/3320913683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc009420 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c890 tx=0x7fc7c0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0012630 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0012790 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c815a0f0 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0012b50 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.049+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81578c0 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.053+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c815a8b0 con 0x7fc7c8159050 2023-10-06T11:36:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.053+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc7c001a3a0 con 0x7fc7c8159050 2023-10-06T11:36:09.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.057+0000 7fc7af7fe700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 0x7fc7b4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.057+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fc7c007ce80 con 0x7fc7c8159050 2023-10-06T11:36:09.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.057+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 0x7fc7b4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.057+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 0x7fc7b4062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007fd0 tx=0x7fc7b8009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.061+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7c004cfa0 con 0x7fc7c8159050 2023-10-06T11:36:09.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.233+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c8157b10 con 0x7fc7c8159050 2023-10-06T11:36:09.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.233+0000 7fc7af7fe700 1 -- 172.21.15.46:0/3320913683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17564 (secure 0 0 0) 0x7fc7c00505c0 con 0x7fc7c8159050 2023-10-06T11:36:09.248 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:09.248 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:08.274977+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:36:09.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 msgr2=0x7fc7b4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 0x7fc7b4062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007fd0 tx=0x7fc7b8009040 comp rx=0 tx=0).stop 2023-10-06T11:36:09.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 msgr2=0x7fc7c815b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c890 tx=0x7fc7c0001670 comp rx=0 tx=0).stop 2023-10-06T11:36:09.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 shutdown_connections 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7b4060670 0x7fc7b4062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8159050 0x7fc7c815b110 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7c80d0aa0 0x7fc7c815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 --2- 172.21.15.46:0/3320913683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7c80b5410 0x7fc7c8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 >> 172.21.15.46:0/3320913683 conn(0x7fc7c806e780 msgr2=0x7fc7c80cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 shutdown_connections 2023-10-06T11:36:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.237+0000 7fc7ce02e700 1 -- 172.21.15.46:0/3320913683 wait complete. 2023-10-06T11:36:09.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:36:09.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.421+0000 7fc636cae700 1 Processor -- start 2023-10-06T11:36:09.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.421+0000 7fc636cae700 1 -- start start 2023-10-06T11:36:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc636cae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc63015ab60 0x7fc63006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc636cae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63006d460 con 0x7fc63015af30 2023-10-06T11:36:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc636cae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6300c7000 con 0x7fc6300ad6c0 2023-10-06T11:36:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc636cae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6300c72c0 con 0x7fc63015ab60 2023-10-06T11:36:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc634a4a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc63015ab60 0x7fc63006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc62ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc63015af30 0x7fc63006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56966/0 (socket says 172.21.15.46:56966) 2023-10-06T11:36:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc62ffff700 1 -- 172.21.15.46:0/4105206189 learned_addr learned my addr 172.21.15.46:0/4105206189 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc634a4a700 1 -- 172.21.15.46:0/4105206189 >> v1:172.21.15.132:6789/0 conn(0x7fc6300ad6c0 legacy=0x7fc6300adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc634a4a700 1 -- 172.21.15.46:0/4105206189 >> v1:172.21.15.46:6789/0 conn(0x7fc63015af30 legacy=0x7fc63006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc634a4a700 1 -- 172.21.15.46:0/4105206189 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300c75a0 con 0x7fc63015ab60 2023-10-06T11:36:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc634a4a700 1 --2- 172.21.15.46:0/4105206189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc63006e8a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc6300c71f0 tx=0x7fc624009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe08d21b7250654 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc62f7fe700 1 -- 172.21.15.46:0/4105206189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624015070 con 0x7fc63015ab60 2023-10-06T11:36:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc62f7fe700 1 -- 172.21.15.46:0/4105206189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc624005c10 con 0x7fc63015ab60 2023-10-06T11:36:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.425+0000 7fc62f7fe700 1 -- 172.21.15.46:0/4105206189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624006030 con 0x7fc63015ab60 2023-10-06T11:36:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- 172.21.15.46:0/4105206189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 msgr2=0x7fc63006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 --2- 172.21.15.46:0/4105206189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc63006e8a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc6300c71f0 tx=0x7fc624009120 comp rx=0 tx=0).stop 2023-10-06T11:36:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- 172.21.15.46:0/4105206189 shutdown_connections 2023-10-06T11:36:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 --2- 172.21.15.46:0/4105206189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc63006e8a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- 172.21.15.46:0/4105206189 >> 172.21.15.46:0/4105206189 conn(0x7fc6300cbe30 msgr2=0x7fc6300cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- 172.21.15.46:0/4105206189 shutdown_connections 2023-10-06T11:36:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- 172.21.15.46:0/4105206189 wait complete. 2023-10-06T11:36:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 Processor -- start 2023-10-06T11:36:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.429+0000 7fc636cae700 1 -- start start 2023-10-06T11:36:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6300ad6c0 0x7fc630150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 0x7fc6300ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300b06b0 con 0x7fc63015ab60 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300c6800 con 0x7fc63015af30 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc630150f40 con 0x7fc6300ad6c0 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60524/0 (socket says 172.21.15.46:60524) 2023-10-06T11:36:09.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 -- 172.21.15.46:0/2854780733 learned_addr learned my addr 172.21.15.46:0/2854780733 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc634a4a700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6300ad6c0 0x7fc630150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc63524b700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 0x7fc6300ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 -- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6300ad6c0 msgr2=0x7fc630150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6300ad6c0 0x7fc630150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 -- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 msgr2=0x7fc6300ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 0x7fc6300ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 -- 172.21.15.46:0/2854780733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6240093c0 con 0x7fc63015ab60 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc63524b700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 0x7fc6300ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62ffff700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c7f0 tx=0x7fc61c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c005b30 con 0x7fc63015ab60 2023-10-06T11:36:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c005cb0 con 0x7fc63015ab60 2023-10-06T11:36:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6300c6520 con 0x7fc63015ab60 2023-10-06T11:36:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004a80 con 0x7fc63015ab60 2023-10-06T11:36:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.433+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6300b0340 con 0x7fc63015ab60 2023-10-06T11:36:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.437+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6300ab2f0 con 0x7fc63015ab60 2023-10-06T11:36:09.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.437+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc61c0070f0 con 0x7fc63015ab60 2023-10-06T11:36:09.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.437+0000 7fc62cff9700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 0x7fc614062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.437+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fc61c07c3c0 con 0x7fc63015ab60 2023-10-06T11:36:09.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.441+0000 7fc634a4a700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 0x7fc614062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.441+0000 7fc634a4a700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 0x7fc614062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc624009390 tx=0x7fc624015000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.445+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc61c04c5e0 con 0x7fc63015ab60 2023-10-06T11:36:09.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.617+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6300b0860 con 0x7fc63015ab60 2023-10-06T11:36:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.617+0000 7fc62cff9700 1 -- 172.21.15.46:0/2854780733 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17564 (secure 0 0 0) 0x7fc6300b0860 con 0x7fc63015ab60 2023-10-06T11:36:09.631 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:09.631 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:08.274977+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:36:09.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 msgr2=0x7fc614062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 0x7fc614062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc624009390 tx=0x7fc624015000 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 msgr2=0x7fc6300ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc61c00c7f0 tx=0x7fc61c007f70 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 shutdown_connections 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc614060670 0x7fc614062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63015af30 0x7fc6300ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc63015ab60 0x7fc6300ac940 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 --2- 172.21.15.46:0/2854780733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6300ad6c0 0x7fc630150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 >> 172.21.15.46:0/2854780733 conn(0x7fc6300cbe30 msgr2=0x7fc6300c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:09.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 shutdown_connections 2023-10-06T11:36:09.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.621+0000 7fc636cae700 1 -- 172.21.15.46:0/2854780733 wait complete. 2023-10-06T11:36:09.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:36:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.817+0000 7fa87f42e700 1 Processor -- start 2023-10-06T11:36:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.817+0000 7fa87f42e700 1 -- start start 2023-10-06T11:36:09.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa878069c10 0x7fa8780af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878068160 0x7fa8780aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8780b1000 0x7fa8780b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8781597e0 con 0x7fa8780b1000 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87815a5c0 con 0x7fa878069c10 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781599f0 con 0x7fa878068160 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8780b1000 0x7fa8780b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d1ca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa878069c10 0x7fa8780af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87c9c9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878068160 0x7fa8780aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8780b1000 0x7fa8780b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60528/0 (socket says 172.21.15.46:60528) 2023-10-06T11:36:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 -- 172.21.15.46:0/2014586351 learned_addr learned my addr 172.21.15.46:0/2014586351 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 -- 172.21.15.46:0/2014586351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878068160 msgr2=0x7fa8780aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 --2- 172.21.15.46:0/2014586351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878068160 0x7fa8780aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 -- 172.21.15.46:0/2014586351 >> v2:172.21.15.132:3300/0 conn(0x7fa878069c10 msgr2=0x7fa8780af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 --2- 172.21.15.46:0/2014586351 >> v2:172.21.15.132:3300/0 conn(0x7fa878069c10 0x7fa8780af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 -- 172.21.15.46:0/2014586351 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878159c90 con 0x7fa8780b1000 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87d9cb700 1 --2- 172.21.15.46:0/2014586351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780b1410 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa87806a6b0 tx=0x7fa870009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c104dcedc878bba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa86ffff700 1 -- 172.21.15.46:0/2014586351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e070 con 0x7fa8780b1000 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa86ffff700 1 -- 172.21.15.46:0/2014586351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000b970 con 0x7fa8780b1000 2023-10-06T11:36:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa86ffff700 1 -- 172.21.15.46:0/2014586351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8700092e0 con 0x7fa8780b1000 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- 172.21.15.46:0/2014586351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 msgr2=0x7fa8780b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- 172.21.15.46:0/2014586351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780b1410 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa87806a6b0 tx=0x7fa870009840 comp rx=0 tx=0).stop 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- 172.21.15.46:0/2014586351 shutdown_connections 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- 172.21.15.46:0/2014586351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780b1410 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- 172.21.15.46:0/2014586351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878068160 0x7fa8780aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 --2- 172.21.15.46:0/2014586351 >> v2:172.21.15.132:3300/0 conn(0x7fa878069c10 0x7fa8780af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.821+0000 7fa87f42e700 1 -- 172.21.15.46:0/2014586351 >> 172.21.15.46:0/2014586351 conn(0x7fa87806ae60 msgr2=0x7fa87806b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.825+0000 7fa87f42e700 1 -- 172.21.15.46:0/2014586351 shutdown_connections 2023-10-06T11:36:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.825+0000 7fa87f42e700 1 -- 172.21.15.46:0/2014586351 wait complete. 2023-10-06T11:36:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.825+0000 7fa87f42e700 1 Processor -- start 2023-10-06T11:36:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.825+0000 7fa87f42e700 1 -- start start 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 0x7fa87815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878154270 con 0x7fa8780b1000 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878154550 con 0x7fa878068160 2023-10-06T11:36:09.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87f42e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878154830 con 0x7fa878069c10 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d9cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87c9c9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 0x7fa87815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87c9c9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 0x7fa87815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:49668/0 (socket says 172.21.15.46:49668) 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45706/0 (socket says 172.21.15.46:45706) 2023-10-06T11:36:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87c9c9700 1 -- 172.21.15.46:0/3372097664 learned_addr learned my addr 172.21.15.46:0/3372097664 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 -- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 msgr2=0x7fa87815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 0x7fa87815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 -- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 msgr2=0x7fa8780aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 -- 172.21.15.46:0/3372097664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009ae0 con 0x7fa878068160 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d9cb700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:09.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.829+0000 7fa87d1ca700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa87400ea50 tx=0x7fa8740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400ee90 con 0x7fa878068160 2023-10-06T11:36:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87400c7f0 con 0x7fa878068160 2023-10-06T11:36:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878154b10 con 0x7fa878068160 2023-10-06T11:36:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400cb90 con 0x7fa878068160 2023-10-06T11:36:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878153e70 con 0x7fa878068160 2023-10-06T11:36:09.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.833+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878154f90 con 0x7fa878068160 2023-10-06T11:36:09.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.837+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa874013470 con 0x7fa878068160 2023-10-06T11:36:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.837+0000 7fa86d7fa700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 0x7fa85c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.837+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fa87407cea0 con 0x7fa878068160 2023-10-06T11:36:09.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.837+0000 7fa87c9c9700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 0x7fa85c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:09.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.841+0000 7fa87c9c9700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 0x7fa85c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa87815c460 tx=0x7fa8680091a0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:09.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:09.845+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa87404cfc0 con 0x7fa878068160 2023-10-06T11:36:10.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.049+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa878159f70 con 0x7fa878068160 2023-10-06T11:36:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.845+0000 7fa86d7fa700 1 -- 172.21.15.46:0/3372097664 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa8740505e0 con 0x7fa878068160 2023-10-06T11:36:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 msgr2=0x7fa85c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 0x7fa85c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa87815c460 tx=0x7fa8680091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 msgr2=0x7fa878155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa87400ea50 tx=0x7fa8740091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 shutdown_connections 2023-10-06T11:36:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa85c060670 0x7fa85c062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8780b1000 0x7fa8780aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa878069c10 0x7fa87815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 --2- 172.21.15.46:0/3372097664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa878068160 0x7fa878155310 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 >> 172.21.15.46:0/3372097664 conn(0x7fa87806ae60 msgr2=0x7fa8780b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 shutdown_connections 2023-10-06T11:36:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:10.849+0000 7fa87f42e700 1 -- 172.21.15.46:0/3372097664 wait complete. 2023-10-06T11:36:10.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:36:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.045+0000 7f5ccf59e700 1 Processor -- start 2023-10-06T11:36:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.045+0000 7f5ccf59e700 1 -- start start 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccf59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5cd00cfb80 0x7f5cd0069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccf59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd00ce6e0 con 0x7f5cd0069270 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccf59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd00cddb0 con 0x7f5cd01591b0 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccf59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd015a5c0 con 0x7f5cd00cfb80 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cce59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5cd0069270 0x7f5cd0158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56972/0 (socket says 172.21.15.46:56972) 2023-10-06T11:36:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cce59c700 1 -- 172.21.15.46:0/2456523970 learned_addr learned my addr 172.21.15.46:0/2456523970 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:11.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cced9d700 1 --2- 172.21.15.46:0/2456523970 >> v2:172.21.15.132:3300/0 conn(0x7f5cd00cfb80 0x7f5cd0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:11.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccd59a700 1 -- 172.21.15.46:0/2456523970 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571152998 0 0) 0x7f5cd00ce6e0 con 0x7f5cd0069270 2023-10-06T11:36:11.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5ccd59a700 1 -- 172.21.15.46:0/2456523970 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb8003570 con 0x7f5cd0069270 2023-10-06T11:36:11.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cced9d700 1 -- 172.21.15.46:0/2456523970 >> v1:172.21.15.132:6789/0 conn(0x7f5cd01591b0 legacy=0x7f5cd0159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cced9d700 1 -- 172.21.15.46:0/2456523970 >> v1:172.21.15.46:6789/0 conn(0x7f5cd0069270 legacy=0x7f5cd0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.049+0000 7f5cced9d700 1 -- 172.21.15.46:0/2456523970 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd00ce090 con 0x7f5cd00cfb80 2023-10-06T11:36:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5cced9d700 1 --2- 172.21.15.46:0/2456523970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd00cfb80 0x7f5cd0069640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009e60 tx=0x7f5cc8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bec3c7e153fad5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccd59a700 1 -- 172.21.15.46:0/2456523970 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800e070 con 0x7f5cd00cfb80 2023-10-06T11:36:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccd59a700 1 -- 172.21.15.46:0/2456523970 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc800bb60 con 0x7f5cd00cfb80 2023-10-06T11:36:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccd59a700 1 -- 172.21.15.46:0/2456523970 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8012430 con 0x7f5cd00cfb80 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 -- 172.21.15.46:0/2456523970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd00cfb80 msgr2=0x7f5cd0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/2456523970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd00cfb80 0x7f5cd0069640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009e60 tx=0x7f5cc8009780 comp rx=0 tx=0).stop 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 -- 172.21.15.46:0/2456523970 shutdown_connections 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/2456523970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd00cfb80 0x7f5cd0069640 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 -- 172.21.15.46:0/2456523970 >> 172.21.15.46:0/2456523970 conn(0x7f5cd0068390 msgr2=0x7f5cd00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:11.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 -- 172.21.15.46:0/2456523970 shutdown_connections 2023-10-06T11:36:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.053+0000 7f5ccf59e700 1 -- 172.21.15.46:0/2456523970 wait complete. 2023-10-06T11:36:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.057+0000 7f5ccf59e700 1 Processor -- start 2023-10-06T11:36:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.057+0000 7f5ccf59e700 1 -- start start 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cd00cfb80 0x7f5cd00c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 0x7f5cd014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00b1660 con 0x7f5cd0069270 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00b0a50 con 0x7f5cd01591b0 2023-10-06T11:36:11.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00b0d30 con 0x7f5cd00cfb80 2023-10-06T11:36:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60554/0 (socket says 172.21.15.46:60554) 2023-10-06T11:36:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 -- 172.21.15.46:0/640036467 learned_addr learned my addr 172.21.15.46:0/640036467 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cced9d700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 0x7f5cd014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccdd9b700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cd00cfb80 0x7f5cd00c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 -- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cd00cfb80 msgr2=0x7f5cd00c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cd00cfb80 0x7f5cd00c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 -- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 msgr2=0x7f5cd014fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 0x7f5cd014fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 -- 172.21.15.46:0/640036467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8009a20 con 0x7f5cd0069270 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cced9d700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 0x7f5cd014fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cce59c700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c7c0 tx=0x7f5cc0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0005b30 con 0x7f5cd0069270 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc0005c90 con 0x7f5cd0069270 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0007340 con 0x7f5cd0069270 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd00b1010 con 0x7f5cd0069270 2023-10-06T11:36:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.061+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd00c9e60 con 0x7f5cd0069270 2023-10-06T11:36:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.065+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd0158f60 con 0x7f5cd0069270 2023-10-06T11:36:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.065+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5cc001b400 con 0x7f5cd0069270 2023-10-06T11:36:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.065+0000 7f5cbe7fc700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 0x7f5cb0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.065+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f5cc007c770 con 0x7f5cd0069270 2023-10-06T11:36:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.065+0000 7f5ccdd9b700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 0x7f5cb0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.069+0000 7f5ccdd9b700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 0x7f5cb0062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00c7580 tx=0x7f5cc4009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.073+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5cd0158f60 con 0x7f5cd0069270 2023-10-06T11:36:11.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.281+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5cd0158f60 con 0x7f5cd0069270 2023-10-06T11:36:11.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.849+0000 7f5cbe7fc700 1 -- 172.21.15.46:0/640036467 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5cd0158f60 con 0x7f5cd0069270 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 msgr2=0x7f5cb0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 0x7f5cb0062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00c7580 tx=0x7f5cc4009040 comp rx=0 tx=0).stop 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 msgr2=0x7f5cd00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c7c0 tx=0x7f5cc0007e00 comp rx=0 tx=0).stop 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 shutdown_connections 2023-10-06T11:36:11.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5cb0060670 0x7f5cb0062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cd01591b0 0x7f5cd014fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cd00cfb80 0x7f5cd00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 --2- 172.21.15.46:0/640036467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cd0069270 0x7f5cd00b1af0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 >> 172.21.15.46:0/640036467 conn(0x7f5cd0068390 msgr2=0x7f5cd00c50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 shutdown_connections 2023-10-06T11:36:11.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:11.853+0000 7f5ccf59e700 1 -- 172.21.15.46:0/640036467 wait complete. 2023-10-06T11:36:11.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:36:12.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.045+0000 7f1ce14cd700 1 Processor -- start 2023-10-06T11:36:12.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.045+0000 7f1ce14cd700 1 -- start start 2023-10-06T11:36:12.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1ce14cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1cdc161840 0x7f1cdc06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1ce14cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc06c8d0 con 0x7f1cdc0adcf0 2023-10-06T11:36:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1ce14cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc06eb90 con 0x7f1cdc0ad920 2023-10-06T11:36:12.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1ce14cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc06d1c0 con 0x7f1cdc161840 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cdb7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1cdc0adcf0 0x7f1cdc15a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56974/0 (socket says 172.21.15.46:56974) 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cdb7fe700 1 -- 172.21.15.46:0/2948307601 learned_addr learned my addr 172.21.15.46:0/2948307601 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/2948307601 >> v2:172.21.15.132:3300/0 conn(0x7f1cdc161840 0x7f1cdc06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cda7fc700 1 -- 172.21.15.46:0/2948307601 >> v1:172.21.15.132:6789/0 conn(0x7f1cdc0ad920 legacy=0x7f1cdc161130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cda7fc700 1 -- 172.21.15.46:0/2948307601 >> v1:172.21.15.46:6789/0 conn(0x7f1cdc0adcf0 legacy=0x7f1cdc15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cda7fc700 1 -- 172.21.15.46:0/2948307601 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc06cb70 con 0x7f1cdc161840 2023-10-06T11:36:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.049+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/2948307601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc161840 0x7f1cdc06e5b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00098e0 tx=0x7f1cd00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4dba213eb8587d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1cd9ffb700 1 -- 172.21.15.46:0/2948307601 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0009e80 con 0x7f1cdc161840 2023-10-06T11:36:12.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1cd9ffb700 1 -- 172.21.15.46:0/2948307601 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000b860 con 0x7f1cdc161840 2023-10-06T11:36:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1cd9ffb700 1 -- 172.21.15.46:0/2948307601 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000bbe0 con 0x7f1cdc161840 2023-10-06T11:36:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- 172.21.15.46:0/2948307601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc161840 msgr2=0x7f1cdc06e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/2948307601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc161840 0x7f1cdc06e5b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00098e0 tx=0x7f1cd00091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- 172.21.15.46:0/2948307601 shutdown_connections 2023-10-06T11:36:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/2948307601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc161840 0x7f1cdc06e5b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- 172.21.15.46:0/2948307601 >> 172.21.15.46:0/2948307601 conn(0x7f1cdc0cc6b0 msgr2=0x7f1cdc0c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- 172.21.15.46:0/2948307601 shutdown_connections 2023-10-06T11:36:12.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- 172.21.15.46:0/2948307601 wait complete. 2023-10-06T11:36:12.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 Processor -- start 2023-10-06T11:36:12.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.053+0000 7f1ce14cd700 1 -- start start 2023-10-06T11:36:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 0x7f1cdc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cdc161840 0x7f1cdc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0c35a0 con 0x7f1cdc0ad920 2023-10-06T11:36:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0c0620 con 0x7f1cdc0adcf0 2023-10-06T11:36:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0bfc50 con 0x7f1cdc161840 2023-10-06T11:36:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60558/0 (socket says 172.21.15.46:60558) 2023-10-06T11:36:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 -- 172.21.15.46:0/3791662574 learned_addr learned my addr 172.21.15.46:0/3791662574 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 0x7f1cdc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdb7fe700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cdc161840 0x7f1cdc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 -- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cdc161840 msgr2=0x7f1cdc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cdc161840 0x7f1cdc15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 -- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 msgr2=0x7f1cdc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 0x7f1cdc15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 -- 172.21.15.46:0/3791662574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009420 con 0x7f1cdc0ad920 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 0x7f1cdc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cdaffd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c5c0 tx=0x7f1ccc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011590 con 0x7f1cdc0ad920 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc011710 con 0x7f1cdc0ad920 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0c2f10 con 0x7f1cdc0ad920 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011ad0 con 0x7f1cdc0ad920 2023-10-06T11:36:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.057+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0c3780 con 0x7f1cdc0ad920 2023-10-06T11:36:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.061+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc064590 con 0x7f1cdc0ad920 2023-10-06T11:36:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.061+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1ccc01b3b0 con 0x7f1cdc0ad920 2023-10-06T11:36:12.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.061+0000 7f1cc2ffd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 0x7f1cc8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.061+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f1ccc07c280 con 0x7f1cdc0ad920 2023-10-06T11:36:12.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.065+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 0x7f1cc8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.065+0000 7f1cda7fc700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 0x7f1cc8062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc161c40 tx=0x7f1cd0017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.069+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ccc04c3a0 con 0x7f1cdc0ad920 2023-10-06T11:36:12.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.277+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cdc0c3b20 con 0x7f1cdc0ad920 2023-10-06T11:36:12.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.277+0000 7f1cc2ffd700 1 -- 172.21.15.46:0/3791662574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1ccc017090 con 0x7f1cdc0ad920 2023-10-06T11:36:12.292 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:12.292 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 msgr2=0x7f1cc8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 0x7f1cc8062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc161c40 tx=0x7f1cd0017000 comp rx=0 tx=0).stop 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 msgr2=0x7f1cdc0c00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c5c0 tx=0x7f1ccc00c980 comp rx=0 tx=0).stop 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 shutdown_connections 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1cc8060670 0x7f1cc8062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cdc161840 0x7f1cdc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cdc0adcf0 0x7f1cdc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 --2- 172.21.15.46:0/3791662574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cdc0ad920 0x7f1cdc0c00e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 >> 172.21.15.46:0/3791662574 conn(0x7f1cdc0cc6b0 msgr2=0x7f1cdc15d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 shutdown_connections 2023-10-06T11:36:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.281+0000 7f1ce14cd700 1 -- 172.21.15.46:0/3791662574 wait complete. 2023-10-06T11:36:12.297 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:36:12.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:36:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 Processor -- start 2023-10-06T11:36:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 -- start start 2023-10-06T11:36:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff54c15ab60 0x7ff54c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff54c15af30 0x7ff54c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c06d460 con 0x7ff54c0ad6c0 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0c7000 con 0x7ff54c15af30 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff5545c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0c72c0 con 0x7ff54c15ab60 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff551b60700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff54c15af30 0x7ff54c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff552b62700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff54c0ad6c0 0x7ff54c0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56980/0 (socket says 172.21.15.46:56980) 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff552b62700 1 -- 172.21.15.46:0/3706725390 learned_addr learned my addr 172.21.15.46:0/3706725390 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.469+0000 7ff552361700 1 --2- 172.21.15.46:0/3706725390 >> v2:172.21.15.132:3300/0 conn(0x7ff54c15ab60 0x7ff54c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff551b60700 1 -- 172.21.15.46:0/3706725390 >> v2:172.21.15.132:3300/0 conn(0x7ff54c15ab60 msgr2=0x7ff54c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff551b60700 1 --2- 172.21.15.46:0/3706725390 >> v2:172.21.15.132:3300/0 conn(0x7ff54c15ab60 0x7ff54c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff551b60700 1 -- 172.21.15.46:0/3706725390 >> v1:172.21.15.46:6789/0 conn(0x7ff54c0ad6c0 legacy=0x7ff54c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff551b60700 1 -- 172.21.15.46:0/3706725390 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0c75a0 con 0x7ff54c15af30 2023-10-06T11:36:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff55135f700 1 -- 172.21.15.46:0/3706725390 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272639389 0 0) 0x7ff54c06d460 con 0x7ff54c0ad6c0 2023-10-06T11:36:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff551b60700 1 --2- 172.21.15.46:0/3706725390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15af30 0x7ff54c06cc30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff54c063be0 tx=0x7ff53c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c462753326787b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff55135f700 1 -- 172.21.15.46:0/3706725390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011630 con 0x7ff54c15af30 2023-10-06T11:36:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff55135f700 1 -- 172.21.15.46:0/3706725390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c011790 con 0x7ff54c15af30 2023-10-06T11:36:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff55135f700 1 -- 172.21.15.46:0/3706725390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011b30 con 0x7ff54c15af30 2023-10-06T11:36:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 -- 172.21.15.46:0/3706725390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15af30 msgr2=0x7ff54c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3706725390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15af30 0x7ff54c06cc30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff54c063be0 tx=0x7ff53c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 -- 172.21.15.46:0/3706725390 shutdown_connections 2023-10-06T11:36:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3706725390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15af30 0x7ff54c06cc30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3706725390 >> v2:172.21.15.132:3300/0 conn(0x7ff54c15ab60 0x7ff54c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 -- 172.21.15.46:0/3706725390 >> 172.21.15.46:0/3706725390 conn(0x7ff54c0cbe30 msgr2=0x7ff54c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 -- 172.21.15.46:0/3706725390 shutdown_connections 2023-10-06T11:36:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.473+0000 7ff5545c5700 1 -- 172.21.15.46:0/3706725390 wait complete. 2023-10-06T11:36:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 Processor -- start 2023-10-06T11:36:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 -- start start 2023-10-06T11:36:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff54c0ad6c0 0x7ff54c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff54c15af30 0x7ff54c144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c16c7c0 con 0x7ff54c15ab60 2023-10-06T11:36:12.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c16caa0 con 0x7ff54c0ad6c0 2023-10-06T11:36:12.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff5545c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c16cd80 con 0x7ff54c15af30 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60566/0 (socket says 172.21.15.46:60566) 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 -- 172.21.15.46:0/3338515926 learned_addr learned my addr 172.21.15.46:0/3338515926 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff552b62700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff54c15af30 0x7ff54c144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff552361700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff54c0ad6c0 0x7ff54c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 -- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff54c15af30 msgr2=0x7ff54c144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff54c15af30 0x7ff54c144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 -- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff54c0ad6c0 msgr2=0x7ff54c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff54c0ad6c0 0x7ff54c0b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 -- 172.21.15.46:0/3338515926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c009420 con 0x7ff54c15ab60 2023-10-06T11:36:12.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.477+0000 7ff551b60700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff54c06d5f0 tx=0x7ff53c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c000d10 con 0x7ff54c15ab60 2023-10-06T11:36:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c011b10 con 0x7ff54c15ab60 2023-10-06T11:36:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c16d060 con 0x7ff54c15ab60 2023-10-06T11:36:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c023660 con 0x7ff54c15ab60 2023-10-06T11:36:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c0b19e0 con 0x7ff54c15ab60 2023-10-06T11:36:12.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.481+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c063f90 con 0x7ff54c15ab60 2023-10-06T11:36:12.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.485+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff53c0193b0 con 0x7ff54c15ab60 2023-10-06T11:36:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.485+0000 7ff5427fc700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 0x7ff530062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.485+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ff53c07e100 con 0x7ff54c15ab60 2023-10-06T11:36:12.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.485+0000 7ff552361700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 0x7ff530062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.485+0000 7ff552361700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 0x7ff530062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d10 tx=0x7ff548009120 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.489+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff53c04e2a0 con 0x7ff54c15ab60 2023-10-06T11:36:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.665+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff54c0c6c60 con 0x7ff54c15ab60 2023-10-06T11:36:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.669+0000 7ff5427fc700 1 -- 172.21.15.46:0/3338515926 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17551 (secure 0 0 0) 0x7ff53c0518c0 con 0x7ff54c15ab60 2023-10-06T11:36:12.683 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:12.683 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:09.804313+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:36:12.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 msgr2=0x7ff530062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 0x7ff530062b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d10 tx=0x7ff548009120 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 msgr2=0x7ff54c0aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff54c06d5f0 tx=0x7ff53c017000 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 shutdown_connections 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff5300606a0 0x7ff530062b50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff54c15af30 0x7ff54c144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff54c15ab60 0x7ff54c0aaa40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 --2- 172.21.15.46:0/3338515926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff54c0ad6c0 0x7ff54c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 >> 172.21.15.46:0/3338515926 conn(0x7ff54c0cbe30 msgr2=0x7ff54c0c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 shutdown_connections 2023-10-06T11:36:12.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.673+0000 7ff5545c5700 1 -- 172.21.15.46:0/3338515926 wait complete. 2023-10-06T11:36:12.706 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:36:12.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:36:12.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 Processor -- start 2023-10-06T11:36:12.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 -- start start 2023-10-06T11:36:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a6c0b1300 0x7f9a6c0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c0ae340 con 0x7f9a6c159290 2023-10-06T11:36:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c0b09a0 con 0x7f9a6c0d07b0 2023-10-06T11:36:12.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a70e8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c159660 con 0x7f9a6c0b1300 2023-10-06T11:36:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a6b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a6c159290 0x7f9a6c0b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56996/0 (socket says 172.21.15.46:56996) 2023-10-06T11:36:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a6b7fe700 1 -- 172.21.15.46:0/2489041316 learned_addr learned my addr 172.21.15.46:0/2489041316 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:12.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.857+0000 7f9a6bfff700 1 --2- 172.21.15.46:0/2489041316 >> v2:172.21.15.132:3300/0 conn(0x7f9a6c0b1300 0x7f9a6c0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6bfff700 1 -- 172.21.15.46:0/2489041316 >> v1:172.21.15.132:6789/0 conn(0x7f9a6c0d07b0 legacy=0x7f9a6c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6bfff700 1 -- 172.21.15.46:0/2489041316 >> v1:172.21.15.46:6789/0 conn(0x7f9a6c159290 legacy=0x7f9a6c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6bfff700 1 -- 172.21.15.46:0/2489041316 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0b06c0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6bfff700 1 --2- 172.21.15.46:0/2489041316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c0ae7d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f921dc3e585e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6affd700 1 -- 172.21.15.46:0/2489041316 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400e070 con 0x7f9a6c0b1300 2023-10-06T11:36:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6affd700 1 -- 172.21.15.46:0/2489041316 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400bcc0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.861+0000 7f9a6affd700 1 -- 172.21.15.46:0/2489041316 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64012680 con 0x7f9a6c0b1300 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2489041316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0b1300 msgr2=0x7f9a6c0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2489041316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c0ae7d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009e60 tx=0x7f9a64009780 comp rx=0 tx=0).stop 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2489041316 shutdown_connections 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2489041316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c0ae7d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2489041316 >> 172.21.15.46:0/2489041316 conn(0x7f9a6c06ef60 msgr2=0x7f9a6c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2489041316 shutdown_connections 2023-10-06T11:36:12.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2489041316 wait complete. 2023-10-06T11:36:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 Processor -- start 2023-10-06T11:36:12.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.865+0000 7f9a70e8e700 1 -- start start 2023-10-06T11:36:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0d07b0 0x7f9a6c14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a6c159290 0x7f9a6c150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c1550c0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c15c380 con 0x7f9a6c0d07b0 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c1545c0 con 0x7f9a6c159290 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60578/0 (socket says 172.21.15.46:60578) 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 -- 172.21.15.46:0/2670808787 learned_addr learned my addr 172.21.15.46:0/2670808787 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6bfff700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a6c159290 0x7f9a6c150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a63fff700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0d07b0 0x7f9a6c14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 -- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a6c159290 msgr2=0x7f9a6c150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a6c159290 0x7f9a6c150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 -- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0d07b0 msgr2=0x7f9a6c14c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0d07b0 0x7f9a6c14c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 -- 172.21.15.46:0/2670808787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64009a20 con 0x7f9a6c0b1300 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a6b7fe700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c590 tx=0x7f9a5c001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c005b30 con 0x7f9a6c0b1300 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c005cb0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c007320 con 0x7f9a6c0b1300 2023-10-06T11:36:12.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c154820 con 0x7f9a6c0b1300 2023-10-06T11:36:12.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.869+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c154de0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.873+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c064590 con 0x7f9a6c0b1300 2023-10-06T11:36:12.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.873+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9a5c01b3b0 con 0x7f9a6c0b1300 2023-10-06T11:36:12.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.873+0000 7f9a637fe700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 0x7f9a4c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:12.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.873+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f9a5c07c100 con 0x7f9a6c0b1300 2023-10-06T11:36:12.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.873+0000 7f9a63fff700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 0x7f9a4c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:12.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.877+0000 7f9a63fff700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 0x7f9a4c062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b0280 tx=0x7f9a54009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:12.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:12.881+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9a5c04c220 con 0x7f9a6c0b1300 2023-10-06T11:36:13.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.049+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a6c0adcf0 con 0x7f9a4c0606a0 2023-10-06T11:36:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.065+0000 7f9a637fe700 1 -- 172.21.15.46:0/2670808787 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300529 (secure 0 0 0) 0x7f9a6c0adcf0 con 0x7f9a4c0606a0 2023-10-06T11:36:13.085 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 msgr2=0x7f9a4c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 0x7f9a4c062b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b0280 tx=0x7f9a54009040 comp rx=0 tx=0).stop 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 msgr2=0x7f9a6c154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c590 tx=0x7f9a5c001020 comp rx=0 tx=0).stop 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 shutdown_connections 2023-10-06T11:36:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9a4c0606a0 0x7f9a4c062b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a6c159290 0x7f9a6c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a6c0d07b0 0x7f9a6c14c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 --2- 172.21.15.46:0/2670808787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c0b1300 0x7f9a6c154080 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 >> 172.21.15.46:0/2670808787 conn(0x7f9a6c06ef60 msgr2=0x7f9a6c14e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.073+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 shutdown_connections 2023-10-06T11:36:13.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.077+0000 7f9a70e8e700 1 -- 172.21.15.46:0/2670808787 wait complete. 2023-10-06T11:36:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:36:13.109 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:36:11.524906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":367,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460656,"kb_used_data":12720,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752514064,"statfs":{"total":773094113280,"available":770574401536,"internally_reserved":0,"allocated":13025280,"data_stored":7288535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.728246"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839807+0000","last_change":"2023-10-06T11:36:08.834959+0000","last_active":"2023-10-06T11:36:08.839807+0000","last_peered":"2023-10-06T11:36:08.839807+0000","last_clean":"2023-10-06T11:36:08.839807+0000","last_became_active":"2023-10-06T11:36:08.834673+0000","last_became_peered":"2023-10-06T11:36:08.834673+0000","last_unstale":"2023-10-06T11:36:08.839807+0000","last_undegraded":"2023-10-06T11:36:08.839807+0000","last_fullsized":"2023-10-06T11:36:08.839807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:45.178435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.826608+0000","last_change":"2023-10-06T11:36:08.822154+0000","last_active":"2023-10-06T11:36:08.826608+0000","last_peered":"2023-10-06T11:36:08.826608+0000","last_clean":"2023-10-06T11:36:08.826608+0000","last_became_active":"2023-10-06T11:36:08.821837+0000","last_became_peered":"2023-10-06T11:36:08.821837+0000","last_unstale":"2023-10-06T11:36:08.826608+0000","last_undegraded":"2023-10-06T11:36:08.826608+0000","last_fullsized":"2023-10-06T11:36:08.826608+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.339584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841877+0000","last_change":"2023-10-06T11:36:08.835584+0000","last_active":"2023-10-06T11:36:08.841877+0000","last_peered":"2023-10-06T11:36:08.841877+0000","last_clean":"2023-10-06T11:36:08.841877+0000","last_became_active":"2023-10-06T11:36:08.835284+0000","last_became_peered":"2023-10-06T11:36:08.835284+0000","last_unstale":"2023-10-06T11:36:08.841877+0000","last_undegraded":"2023-10-06T11:36:08.841877+0000","last_fullsized":"2023-10-06T11:36:08.841877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850850+0000","last_change":"2023-10-06T11:36:08.846553+0000","last_active":"2023-10-06T11:36:08.850850+0000","last_peered":"2023-10-06T11:36:08.850850+0000","last_clean":"2023-10-06T11:36:08.850850+0000","last_became_active":"2023-10-06T11:36:08.846228+0000","last_became_peered":"2023-10-06T11:36:08.846228+0000","last_unstale":"2023-10-06T11:36:08.850850+0000","last_undegraded":"2023-10-06T11:36:08.850850+0000","last_fullsized":"2023-10-06T11:36:08.850850+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834928+0000","last_change":"2023-10-06T11:36:08.829614+0000","last_active":"2023-10-06T11:36:08.834928+0000","last_peered":"2023-10-06T11:36:08.834928+0000","last_clean":"2023-10-06T11:36:08.834928+0000","last_became_active":"2023-10-06T11:36:08.829269+0000","last_became_peered":"2023-10-06T11:36:08.829269+0000","last_unstale":"2023-10-06T11:36:08.834928+0000","last_undegraded":"2023-10-06T11:36:08.834928+0000","last_fullsized":"2023-10-06T11:36:08.834928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.821913+0000","last_change":"2023-10-06T11:36:08.820676+0000","last_active":"2023-10-06T11:36:08.821913+0000","last_peered":"2023-10-06T11:36:08.821913+0000","last_clean":"2023-10-06T11:36:08.821913+0000","last_became_active":"2023-10-06T11:36:08.820319+0000","last_became_peered":"2023-10-06T11:36:08.820319+0000","last_unstale":"2023-10-06T11:36:08.821913+0000","last_undegraded":"2023-10-06T11:36:08.821913+0000","last_fullsized":"2023-10-06T11:36:08.821913+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:53:05.714850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835909+0000","last_change":"2023-10-06T11:36:08.833419+0000","last_active":"2023-10-06T11:36:08.835909+0000","last_peered":"2023-10-06T11:36:08.835909+0000","last_clean":"2023-10-06T11:36:08.835909+0000","last_became_active":"2023-10-06T11:36:08.832970+0000","last_became_peered":"2023-10-06T11:36:08.832970+0000","last_unstale":"2023-10-06T11:36:08.835909+0000","last_undegraded":"2023-10-06T11:36:08.835909+0000","last_fullsized":"2023-10-06T11:36:08.835909+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:28:24.073017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847734+0000","last_change":"2023-10-06T11:36:08.834706+0000","last_active":"2023-10-06T11:36:08.847734+0000","last_peered":"2023-10-06T11:36:08.847734+0000","last_clean":"2023-10-06T11:36:08.847734+0000","last_became_active":"2023-10-06T11:36:08.834392+0000","last_became_peered":"2023-10-06T11:36:08.834392+0000","last_unstale":"2023-10-06T11:36:08.847734+0000","last_undegraded":"2023-10-06T11:36:08.847734+0000","last_fullsized":"2023-10-06T11:36:08.847734+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832710+0000","last_change":"2023-10-06T11:36:08.825112+0000","last_active":"2023-10-06T11:36:08.832710+0000","last_peered":"2023-10-06T11:36:08.832710+0000","last_clean":"2023-10-06T11:36:08.832710+0000","last_became_active":"2023-10-06T11:36:08.824817+0000","last_became_peered":"2023-10-06T11:36:08.824817+0000","last_unstale":"2023-10-06T11:36:08.832710+0000","last_undegraded":"2023-10-06T11:36:08.832710+0000","last_fullsized":"2023-10-06T11:36:08.832710+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:47.661809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847711+0000","last_change":"2023-10-06T11:36:08.830867+0000","last_active":"2023-10-06T11:36:08.847711+0000","last_peered":"2023-10-06T11:36:08.847711+0000","last_clean":"2023-10-06T11:36:08.847711+0000","last_became_active":"2023-10-06T11:36:08.830226+0000","last_became_peered":"2023-10-06T11:36:08.830226+0000","last_unstale":"2023-10-06T11:36:08.847711+0000","last_undegraded":"2023-10-06T11:36:08.847711+0000","last_fullsized":"2023-10-06T11:36:08.847711+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827000+0000","last_change":"2023-10-06T11:36:08.825532+0000","last_active":"2023-10-06T11:36:08.827000+0000","last_peered":"2023-10-06T11:36:08.827000+0000","last_clean":"2023-10-06T11:36:08.827000+0000","last_became_active":"2023-10-06T11:36:08.825246+0000","last_became_peered":"2023-10-06T11:36:08.825246+0000","last_unstale":"2023-10-06T11:36:08.827000+0000","last_undegraded":"2023-10-06T11:36:08.827000+0000","last_fullsized":"2023-10-06T11:36:08.827000+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837616+0000","last_change":"2023-10-06T11:36:08.832215+0000","last_active":"2023-10-06T11:36:08.837616+0000","last_peered":"2023-10-06T11:36:08.837616+0000","last_clean":"2023-10-06T11:36:08.837616+0000","last_became_active":"2023-10-06T11:36:08.831906+0000","last_became_peered":"2023-10-06T11:36:08.831906+0000","last_unstale":"2023-10-06T11:36:08.837616+0000","last_undegraded":"2023-10-06T11:36:08.837616+0000","last_fullsized":"2023-10-06T11:36:08.837616+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:21.848572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"121'4","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.854694+0000","last_change":"2023-10-06T11:36:07.836398+0000","last_active":"2023-10-06T11:36:08.854694+0000","last_peered":"2023-10-06T11:36:08.854694+0000","last_clean":"2023-10-06T11:36:08.854694+0000","last_became_active":"2023-10-06T11:36:07.836063+0000","last_became_peered":"2023-10-06T11:36:07.836063+0000","last_unstale":"2023-10-06T11:36:08.854694+0000","last_undegraded":"2023-10-06T11:36:08.854694+0000","last_fullsized":"2023-10-06T11:36:08.854694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850887+0000","last_change":"2023-10-06T11:36:08.845254+0000","last_active":"2023-10-06T11:36:08.850887+0000","last_peered":"2023-10-06T11:36:08.850887+0000","last_clean":"2023-10-06T11:36:08.850887+0000","last_became_active":"2023-10-06T11:36:08.844729+0000","last_became_peered":"2023-10-06T11:36:08.844729+0000","last_unstale":"2023-10-06T11:36:08.850887+0000","last_undegraded":"2023-10-06T11:36:08.850887+0000","last_fullsized":"2023-10-06T11:36:08.850887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:46.014384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857089+0000","last_change":"2023-10-06T11:36:07.830544+0000","last_active":"2023-10-06T11:36:08.857089+0000","last_peered":"2023-10-06T11:36:08.857089+0000","last_clean":"2023-10-06T11:36:08.857089+0000","last_became_active":"2023-10-06T11:36:07.830229+0000","last_became_peered":"2023-10-06T11:36:07.830229+0000","last_unstale":"2023-10-06T11:36:08.857089+0000","last_undegraded":"2023-10-06T11:36:08.857089+0000","last_fullsized":"2023-10-06T11:36:08.857089+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841854+0000","last_change":"2023-10-06T11:36:08.834817+0000","last_active":"2023-10-06T11:36:08.841854+0000","last_peered":"2023-10-06T11:36:08.841854+0000","last_clean":"2023-10-06T11:36:08.841854+0000","last_became_active":"2023-10-06T11:36:08.834510+0000","last_became_peered":"2023-10-06T11:36:08.834510+0000","last_unstale":"2023-10-06T11:36:08.841854+0000","last_undegraded":"2023-10-06T11:36:08.841854+0000","last_fullsized":"2023-10-06T11:36:08.841854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835929+0000","last_change":"2023-10-06T11:36:08.833625+0000","last_active":"2023-10-06T11:36:08.835929+0000","last_peered":"2023-10-06T11:36:08.835929+0000","last_clean":"2023-10-06T11:36:08.835929+0000","last_became_active":"2023-10-06T11:36:08.833132+0000","last_became_peered":"2023-10-06T11:36:08.833132+0000","last_unstale":"2023-10-06T11:36:08.835929+0000","last_undegraded":"2023-10-06T11:36:08.835929+0000","last_fullsized":"2023-10-06T11:36:08.835929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.375194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833946+0000","last_change":"2023-10-06T11:36:08.822864+0000","last_active":"2023-10-06T11:36:08.833946+0000","last_peered":"2023-10-06T11:36:08.833946+0000","last_clean":"2023-10-06T11:36:08.833946+0000","last_became_active":"2023-10-06T11:36:08.822567+0000","last_became_peered":"2023-10-06T11:36:08.822567+0000","last_unstale":"2023-10-06T11:36:08.833946+0000","last_undegraded":"2023-10-06T11:36:08.833946+0000","last_fullsized":"2023-10-06T11:36:08.833946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827606+0000","last_change":"2023-10-06T11:36:07.838208+0000","last_active":"2023-10-06T11:36:08.827606+0000","last_peered":"2023-10-06T11:36:08.827606+0000","last_clean":"2023-10-06T11:36:08.827606+0000","last_became_active":"2023-10-06T11:36:07.837843+0000","last_became_peered":"2023-10-06T11:36:07.837843+0000","last_unstale":"2023-10-06T11:36:08.827606+0000","last_undegraded":"2023-10-06T11:36:08.827606+0000","last_fullsized":"2023-10-06T11:36:08.827606+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837595+0000","last_change":"2023-10-06T11:36:08.831541+0000","last_active":"2023-10-06T11:36:08.837595+0000","last_peered":"2023-10-06T11:36:08.837595+0000","last_clean":"2023-10-06T11:36:08.837595+0000","last_became_active":"2023-10-06T11:36:08.831234+0000","last_became_peered":"2023-10-06T11:36:08.831234+0000","last_unstale":"2023-10-06T11:36:08.837595+0000","last_undegraded":"2023-10-06T11:36:08.837595+0000","last_fullsized":"2023-10-06T11:36:08.837595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:26.203324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810127+0000","last_change":"2023-10-06T11:36:07.831652+0000","last_active":"2023-10-06T11:36:08.810127+0000","last_peered":"2023-10-06T11:36:08.810127+0000","last_clean":"2023-10-06T11:36:08.810127+0000","last_became_active":"2023-10-06T11:36:07.831331+0000","last_became_peered":"2023-10-06T11:36:07.831331+0000","last_unstale":"2023-10-06T11:36:08.810127+0000","last_undegraded":"2023-10-06T11:36:08.810127+0000","last_fullsized":"2023-10-06T11:36:08.810127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:30:03.936003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850794+0000","last_change":"2023-10-06T11:36:08.843325+0000","last_active":"2023-10-06T11:36:08.850794+0000","last_peered":"2023-10-06T11:36:08.850794+0000","last_clean":"2023-10-06T11:36:08.850794+0000","last_became_active":"2023-10-06T11:36:08.843032+0000","last_became_peered":"2023-10-06T11:36:08.843032+0000","last_unstale":"2023-10-06T11:36:08.850794+0000","last_undegraded":"2023-10-06T11:36:08.850794+0000","last_fullsized":"2023-10-06T11:36:08.850794+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:50:13.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.830870+0000","last_change":"2023-10-06T11:36:07.833180+0000","last_active":"2023-10-06T11:36:08.830870+0000","last_peered":"2023-10-06T11:36:08.830870+0000","last_clean":"2023-10-06T11:36:08.830870+0000","last_became_active":"2023-10-06T11:36:07.832883+0000","last_became_peered":"2023-10-06T11:36:07.832883+0000","last_unstale":"2023-10-06T11:36:08.830870+0000","last_undegraded":"2023-10-06T11:36:08.830870+0000","last_fullsized":"2023-10-06T11:36:08.830870+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827807+0000","last_change":"2023-10-06T11:36:08.826278+0000","last_active":"2023-10-06T11:36:08.827807+0000","last_peered":"2023-10-06T11:36:08.827807+0000","last_clean":"2023-10-06T11:36:08.827807+0000","last_became_active":"2023-10-06T11:36:08.825985+0000","last_became_peered":"2023-10-06T11:36:08.825985+0000","last_unstale":"2023-10-06T11:36:08.827807+0000","last_undegraded":"2023-10-06T11:36:08.827807+0000","last_fullsized":"2023-10-06T11:36:08.827807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820190+0000","last_change":"2023-10-06T11:36:07.840409+0000","last_active":"2023-10-06T11:36:08.820190+0000","last_peered":"2023-10-06T11:36:08.820190+0000","last_clean":"2023-10-06T11:36:08.820190+0000","last_became_active":"2023-10-06T11:36:07.840059+0000","last_became_peered":"2023-10-06T11:36:07.840059+0000","last_unstale":"2023-10-06T11:36:08.820190+0000","last_undegraded":"2023-10-06T11:36:08.820190+0000","last_fullsized":"2023-10-06T11:36:08.820190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:54.481491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832875+0000","last_change":"2023-10-06T11:36:08.826025+0000","last_active":"2023-10-06T11:36:08.832875+0000","last_peered":"2023-10-06T11:36:08.832875+0000","last_clean":"2023-10-06T11:36:08.832875+0000","last_became_active":"2023-10-06T11:36:08.825431+0000","last_became_peered":"2023-10-06T11:36:08.825431+0000","last_unstale":"2023-10-06T11:36:08.832875+0000","last_undegraded":"2023-10-06T11:36:08.832875+0000","last_fullsized":"2023-10-06T11:36:08.832875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:56.221716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839926+0000","last_change":"2023-10-06T11:36:08.837343+0000","last_active":"2023-10-06T11:36:08.839926+0000","last_peered":"2023-10-06T11:36:08.839926+0000","last_clean":"2023-10-06T11:36:08.839926+0000","last_became_active":"2023-10-06T11:36:08.835511+0000","last_became_peered":"2023-10-06T11:36:08.835511+0000","last_unstale":"2023-10-06T11:36:08.839926+0000","last_undegraded":"2023-10-06T11:36:08.839926+0000","last_fullsized":"2023-10-06T11:36:08.839926+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:20.565177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810349+0000","last_change":"2023-10-06T11:36:07.833889+0000","last_active":"2023-10-06T11:36:08.810349+0000","last_peered":"2023-10-06T11:36:08.810349+0000","last_clean":"2023-10-06T11:36:08.810349+0000","last_became_active":"2023-10-06T11:36:07.833564+0000","last_became_peered":"2023-10-06T11:36:07.833564+0000","last_unstale":"2023-10-06T11:36:08.810349+0000","last_undegraded":"2023-10-06T11:36:08.810349+0000","last_fullsized":"2023-10-06T11:36:08.810349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:43.919700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835073+0000","last_change":"2023-10-06T11:36:08.832266+0000","last_active":"2023-10-06T11:36:08.835073+0000","last_peered":"2023-10-06T11:36:08.835073+0000","last_clean":"2023-10-06T11:36:08.835073+0000","last_became_active":"2023-10-06T11:36:08.830568+0000","last_became_peered":"2023-10-06T11:36:08.830568+0000","last_unstale":"2023-10-06T11:36:08.835073+0000","last_undegraded":"2023-10-06T11:36:08.835073+0000","last_fullsized":"2023-10-06T11:36:08.835073+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850140+0000","last_change":"2023-10-06T11:36:08.836975+0000","last_active":"2023-10-06T11:36:08.850140+0000","last_peered":"2023-10-06T11:36:08.850140+0000","last_clean":"2023-10-06T11:36:08.850140+0000","last_became_active":"2023-10-06T11:36:08.836692+0000","last_became_peered":"2023-10-06T11:36:08.836692+0000","last_unstale":"2023-10-06T11:36:08.850140+0000","last_undegraded":"2023-10-06T11:36:08.850140+0000","last_fullsized":"2023-10-06T11:36:08.850140+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:19.467727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850838+0000","last_change":"2023-10-06T11:36:08.843940+0000","last_active":"2023-10-06T11:36:08.850838+0000","last_peered":"2023-10-06T11:36:08.850838+0000","last_clean":"2023-10-06T11:36:08.850838+0000","last_became_active":"2023-10-06T11:36:08.843666+0000","last_became_peered":"2023-10-06T11:36:08.843666+0000","last_unstale":"2023-10-06T11:36:08.850838+0000","last_undegraded":"2023-10-06T11:36:08.850838+0000","last_fullsized":"2023-10-06T11:36:08.850838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822603+0000","last_change":"2023-10-06T11:36:07.840413+0000","last_active":"2023-10-06T11:36:08.822603+0000","last_peered":"2023-10-06T11:36:08.822603+0000","last_clean":"2023-10-06T11:36:08.822603+0000","last_became_active":"2023-10-06T11:36:07.840091+0000","last_became_peered":"2023-10-06T11:36:07.840091+0000","last_unstale":"2023-10-06T11:36:08.822603+0000","last_undegraded":"2023-10-06T11:36:08.822603+0000","last_fullsized":"2023-10-06T11:36:08.822603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:25.925632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819931+0000","last_change":"2023-10-06T11:36:07.841033+0000","last_active":"2023-10-06T11:36:08.819931+0000","last_peered":"2023-10-06T11:36:08.819931+0000","last_clean":"2023-10-06T11:36:08.819931+0000","last_became_active":"2023-10-06T11:36:07.840720+0000","last_became_peered":"2023-10-06T11:36:07.840720+0000","last_unstale":"2023-10-06T11:36:08.819931+0000","last_undegraded":"2023-10-06T11:36:08.819931+0000","last_fullsized":"2023-10-06T11:36:08.819931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857037+0000","last_change":"2023-10-06T11:36:07.838666+0000","last_active":"2023-10-06T11:36:08.857037+0000","last_peered":"2023-10-06T11:36:08.857037+0000","last_clean":"2023-10-06T11:36:08.857037+0000","last_became_active":"2023-10-06T11:36:07.837999+0000","last_became_peered":"2023-10-06T11:36:07.837999+0000","last_unstale":"2023-10-06T11:36:08.857037+0000","last_undegraded":"2023-10-06T11:36:08.857037+0000","last_fullsized":"2023-10-06T11:36:08.857037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"111'227","reported_seq":637,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.666094+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:35:59.666094+0000","last_peered":"2023-10-06T11:35:59.666094+0000","last_clean":"2023-10-06T11:35:59.666094+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:35:59.666094+0000","last_undegraded":"2023-10-06T11:35:59.666094+0000","last_fullsized":"2023-10-06T11:35:59.666094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.795932+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:36:03.795932+0000","last_peered":"2023-10-06T11:36:03.795932+0000","last_clean":"2023-10-06T11:36:03.795932+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:36:03.795932+0000","last_undegraded":"2023-10-06T11:36:03.795932+0000","last_fullsized":"2023-10-06T11:36:03.795932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.665926+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:35:59.665926+0000","last_peered":"2023-10-06T11:35:59.665926+0000","last_clean":"2023-10-06T11:35:59.665926+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:35:59.665926+0000","last_undegraded":"2023-10-06T11:35:59.665926+0000","last_fullsized":"2023-10-06T11:35:59.665926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820278+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:36:08.820278+0000","last_peered":"2023-10-06T11:36:08.820278+0000","last_clean":"2023-10-06T11:36:08.820278+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:36:08.820278+0000","last_undegraded":"2023-10-06T11:36:08.820278+0000","last_fullsized":"2023-10-06T11:36:08.820278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847753+0000","last_change":"2023-10-06T11:36:08.834917+0000","last_active":"2023-10-06T11:36:08.847753+0000","last_peered":"2023-10-06T11:36:08.847753+0000","last_clean":"2023-10-06T11:36:08.847753+0000","last_became_active":"2023-10-06T11:36:08.834522+0000","last_became_peered":"2023-10-06T11:36:08.834522+0000","last_unstale":"2023-10-06T11:36:08.847753+0000","last_undegraded":"2023-10-06T11:36:08.847753+0000","last_fullsized":"2023-10-06T11:36:08.847753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.795885+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:36:03.795885+0000","last_peered":"2023-10-06T11:36:03.795885+0000","last_clean":"2023-10-06T11:36:03.795885+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:36:03.795885+0000","last_undegraded":"2023-10-06T11:36:03.795885+0000","last_fullsized":"2023-10-06T11:36:03.795885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834204+0000","last_change":"2023-10-06T11:36:08.831735+0000","last_active":"2023-10-06T11:36:08.834204+0000","last_peered":"2023-10-06T11:36:08.834204+0000","last_clean":"2023-10-06T11:36:08.834204+0000","last_became_active":"2023-10-06T11:36:08.831450+0000","last_became_peered":"2023-10-06T11:36:08.831450+0000","last_unstale":"2023-10-06T11:36:08.834204+0000","last_undegraded":"2023-10-06T11:36:08.834204+0000","last_fullsized":"2023-10-06T11:36:08.834204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.792576+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:36:03.792576+0000","last_peered":"2023-10-06T11:36:03.792576+0000","last_clean":"2023-10-06T11:36:03.792576+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:36:03.792576+0000","last_undegraded":"2023-10-06T11:36:03.792576+0000","last_fullsized":"2023-10-06T11:36:03.792576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848126+0000","last_change":"2023-10-06T11:36:08.836395+0000","last_active":"2023-10-06T11:36:08.848126+0000","last_peered":"2023-10-06T11:36:08.848126+0000","last_clean":"2023-10-06T11:36:08.848126+0000","last_became_active":"2023-10-06T11:36:08.836112+0000","last_became_peered":"2023-10-06T11:36:08.836112+0000","last_unstale":"2023-10-06T11:36:08.848126+0000","last_undegraded":"2023-10-06T11:36:08.848126+0000","last_fullsized":"2023-10-06T11:36:08.848126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819833+0000","last_change":"2023-10-06T11:36:07.840551+0000","last_active":"2023-10-06T11:36:08.819833+0000","last_peered":"2023-10-06T11:36:08.819833+0000","last_clean":"2023-10-06T11:36:08.819833+0000","last_became_active":"2023-10-06T11:36:07.840210+0000","last_became_peered":"2023-10-06T11:36:07.840210+0000","last_unstale":"2023-10-06T11:36:08.819833+0000","last_undegraded":"2023-10-06T11:36:08.819833+0000","last_fullsized":"2023-10-06T11:36:08.819833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.664611+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:35:59.664611+0000","last_peered":"2023-10-06T11:35:59.664611+0000","last_clean":"2023-10-06T11:35:59.664611+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:35:59.664611+0000","last_undegraded":"2023-10-06T11:35:59.664611+0000","last_fullsized":"2023-10-06T11:35:59.664611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839738+0000","last_change":"2023-10-06T11:36:08.836510+0000","last_active":"2023-10-06T11:36:08.839738+0000","last_peered":"2023-10-06T11:36:08.839738+0000","last_clean":"2023-10-06T11:36:08.839738+0000","last_became_active":"2023-10-06T11:36:08.836215+0000","last_became_peered":"2023-10-06T11:36:08.836215+0000","last_unstale":"2023-10-06T11:36:08.839738+0000","last_undegraded":"2023-10-06T11:36:08.839738+0000","last_fullsized":"2023-10-06T11:36:08.839738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:32.797025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857016+0000","last_change":"2023-10-06T11:36:07.841111+0000","last_active":"2023-10-06T11:36:08.857016+0000","last_peered":"2023-10-06T11:36:08.857016+0000","last_clean":"2023-10-06T11:36:08.857016+0000","last_became_active":"2023-10-06T11:36:07.840810+0000","last_became_peered":"2023-10-06T11:36:07.840810+0000","last_unstale":"2023-10-06T11:36:08.857016+0000","last_undegraded":"2023-10-06T11:36:08.857016+0000","last_fullsized":"2023-10-06T11:36:08.857016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:03.725992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.792625+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:36:03.792625+0000","last_peered":"2023-10-06T11:36:03.792625+0000","last_clean":"2023-10-06T11:36:03.792625+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:36:03.792625+0000","last_undegraded":"2023-10-06T11:36:03.792625+0000","last_fullsized":"2023-10-06T11:36:03.792625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837031+0000","last_change":"2023-10-06T11:36:08.833641+0000","last_active":"2023-10-06T11:36:08.837031+0000","last_peered":"2023-10-06T11:36:08.837031+0000","last_clean":"2023-10-06T11:36:08.837031+0000","last_became_active":"2023-10-06T11:36:08.833372+0000","last_became_peered":"2023-10-06T11:36:08.833372+0000","last_unstale":"2023-10-06T11:36:08.837031+0000","last_undegraded":"2023-10-06T11:36:08.837031+0000","last_fullsized":"2023-10-06T11:36:08.837031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:45.164451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822466+0000","last_change":"2023-10-06T11:36:07.841750+0000","last_active":"2023-10-06T11:36:08.822466+0000","last_peered":"2023-10-06T11:36:08.822466+0000","last_clean":"2023-10-06T11:36:08.822466+0000","last_became_active":"2023-10-06T11:36:07.841432+0000","last_became_peered":"2023-10-06T11:36:07.841432+0000","last_unstale":"2023-10-06T11:36:08.822466+0000","last_undegraded":"2023-10-06T11:36:08.822466+0000","last_fullsized":"2023-10-06T11:36:08.822466+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:27:06.372507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.800706+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:36:03.800706+0000","last_peered":"2023-10-06T11:36:03.800706+0000","last_clean":"2023-10-06T11:36:03.800706+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:36:03.800706+0000","last_undegraded":"2023-10-06T11:36:03.800706+0000","last_fullsized":"2023-10-06T11:36:03.800706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":3},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":26,"acting":26,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575059,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311580,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060260,"statfs":{"total":96636764160,"available":96317706240,"internally_reserved":0,"allocated":2052096,"data_stored":1342951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1020000000000001,"15min":1.1339999999999999},"min":{"1min":0.84499999999999997,"5min":0.59599999999999997,"15min":0.495},"max":{"1min":1.5249999999999999,"5min":1.6579999999999999,"15min":1.724},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.159,"5min":1.143,"15min":1.139},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.56399999999999995},"max":{"1min":1.5229999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.359,"15min":1.333},"min":{"1min":0.78100000000000003,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.651,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.448},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.232},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5349999999999999,"5min":1.927,"15min":1.927},"last":1.256}]},{"osd":2,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2,"15min":1.214},"min":{"1min":1.006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6040000000000001,"5min":1.827,"15min":1.827},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.379}]},{"osd":3,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.256,"15min":1.264},"min":{"1min":0.92900000000000005,"5min":0.79300000000000004,"15min":0.68500000000000005},"max":{"1min":1.7210000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.048},{"interface":"front","average":{"1min":1.454,"5min":1.3380000000000001,"15min":1.323},"min":{"1min":1.2829999999999999,"5min":0.89000000000000001,"15min":0.75600000000000001},"max":{"1min":1.8029999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.208}]},{"osd":4,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1120000000000001},"min":{"1min":0.72799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.601,"5min":1.663,"15min":2.149},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.25},"min":{"1min":0.85199999999999998,"5min":0.55300000000000005,"15min":0.41699999999999998},"max":{"1min":1.7849999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3700000000000001,"15min":1.3109999999999999},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.58199999999999996},"max":{"1min":1.881,"5min":2.488,"15min":2.488},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":1.903,"15min":2.2480000000000002},"last":1.077}]},{"osd":6,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.347,"15min":1.349},"min":{"1min":0.93600000000000005,"5min":0.67000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7250000000000001,"5min":2.0219999999999998,"15min":2.4289999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.60099999999999998,"15min":0.54200000000000004},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306904,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064936,"statfs":{"total":96636764160,"available":96322494464,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2869999999999999,"15min":1.2889999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.891,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.3180000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.546,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.6319999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1439999999999999,"15min":1.1579999999999999},"min":{"1min":1.024,"5min":0.76200000000000001,"15min":0.64000000000000001},"max":{"1min":1.48,"5min":1.5980000000000001,"15min":1.8169999999999999},"last":1.01},{"interface":"front","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.214},"min":{"1min":0.78600000000000003,"5min":0.66200000000000003,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.202,"15min":1.206},"min":{"1min":0.85399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.476,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.214},"min":{"1min":0.79300000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5009999999999999,"5min":1.736,"15min":1.736},"last":1.675}]},{"osd":3,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.306},"min":{"1min":0.89500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.625,"5min":2.121,"15min":2.121},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3240000000000001,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.597,"5min":1.7430000000000001,"15min":1.7509999999999999},"last":1.7050000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.90700000000000003,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.304,"15min":1.248},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":1.649,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.196,"15min":1.2270000000000001},"min":{"1min":0.89200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5529999999999999,"5min":1.879,"15min":1.879},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.92100000000000004,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.9359999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4390000000000001},"min":{"1min":0.84199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.77,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.256},{"interface":"front","average":{"1min":1.363,"5min":1.389,"15min":1.415},"min":{"1min":1.0129999999999999,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5620000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3979999999999999,"15min":1.284},"min":{"1min":0.89900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.615,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.173},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.296},"min":{"1min":1.095,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":1.972,"15min":2.3959999999999999},"last":1.6140000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1479999999999999,"15min":1.1639999999999999},"min":{"1min":0.82999999999999996,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.3999999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.212},"min":{"1min":0.84799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5529999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":0.874}]},{"osd":2,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.88800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5149999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.554},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.6990000000000001,"15min":1.738},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.397,"15min":1.28},"min":{"1min":1.0109999999999999,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.454,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.2989999999999999},"min":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.581,"5min":1.8400000000000001,"15min":1.871},"last":1.2330000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3460000000000001,"15min":1.2609999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.5369999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5309999999999999,"15min":1.383},"min":{"1min":0.83899999999999997,"5min":0.72199999999999998,"15min":0.63500000000000001},"max":{"1min":1.6739999999999999,"5min":18.48,"15min":18.48},"last":1.5840000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.472},{"interface":"front","average":{"1min":1.252,"5min":1.3999999999999999,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.4870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.488,"15min":1.3660000000000001},"min":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.609,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.522},{"interface":"front","average":{"1min":1.268,"5min":1.4550000000000001,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.534,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.8360000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.77400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.669,"5min":1.72,"15min":1.72},"last":1.375},{"interface":"front","average":{"1min":1.175,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.84499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.48,"5min":1.5669999999999999,"15min":1.6910000000000001},"last":1.127}]},{"osd":1,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":0.94399999999999995,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.26,"15min":1.272},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2010000000000001,"15min":1.196},"min":{"1min":0.874,"5min":0.628,"15min":0.628},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.105},{"interface":"front","average":{"1min":1.302,"5min":1.274,"15min":1.2669999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6619999999999999,"5min":1.6779999999999999,"15min":1.78},"last":1.2849999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1839999999999999,"15min":1.204},"min":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6040000000000001,"5min":1.617,"15min":1.6839999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.3180000000000001,"15min":1.3049999999999999},"min":{"1min":1.0369999999999999,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.77,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.82699999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.706,"5min":1.706,"15min":1.7390000000000001},"last":1.218},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.571,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.155}]},{"osd":6,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2390000000000001,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.995},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.3480000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.831,"5min":1.845,"15min":1.8660000000000001},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.349,"15min":1.329},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4359999999999999,"15min":1.3879999999999999},"min":{"1min":0.96299999999999997,"5min":0.88700000000000001,"15min":0.73799999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.494}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":781444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.052,"15min":0.93500000000000005},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":14.49,"15min":14.49},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.998,"5min":1.175,"15min":1.05},"min":{"1min":0.81200000000000006,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.98599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98099999999999998,"15min":0.96699999999999997},"min":{"1min":0.73899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1890000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.1579999999999999,"15min":1.1140000000000001},"min":{"1min":0.72899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.698,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.23,"15min":1.1599999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.6080000000000001,"5min":12.446,"15min":12.446},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.76100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.5009999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3169999999999999,"15min":1.2589999999999999},"min":{"1min":0.89200000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":10.460000000000001,"15min":10.460000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.343,"15min":1.3560000000000001},"min":{"1min":1.0289999999999999,"5min":0.80800000000000005,"15min":0.78400000000000003},"max":{"1min":1.5569999999999999,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.407,"15min":1.411},"min":{"1min":1.0860000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.73,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.111},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.874,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7509999999999999,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.3009999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.8080000000000001,"5min":11.362,"15min":11.362},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.4419999999999999,"15min":1.399},"min":{"1min":0.98499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6679999999999999,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.52,"15min":1.4610000000000001},"min":{"1min":1.1759999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7689999999999999,"5min":10.561,"15min":10.561},"last":1.607},{"interface":"front","average":{"1min":1.48,"5min":1.4890000000000001,"15min":1.4870000000000001},"min":{"1min":1.1679999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.792,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.3320000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.56999999999999995,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4530000000000001,"5min":1.524,"15min":1.8380000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.52,"5min":1.5409999999999999,"15min":1.9650000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.018},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.5760000000000001,"5min":1.599,"15min":2.5390000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.84599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.546,"5min":1.758,"15min":1.8240000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.288,"5min":1.256,"15min":1.2989999999999999},"min":{"1min":0.85499999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"max":{"1min":1.6419999999999999,"5min":1.948,"15min":2.4550000000000001},"last":1.016}]},{"osd":4,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.96999999999999997,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.569,"5min":1.625,"15min":1.907},"last":1.093},{"interface":"front","average":{"1min":1.373,"5min":1.3,"15min":1.29},"min":{"1min":1.071,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3420000000000001,"15min":1.3580000000000001},"min":{"1min":0.94099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8049999999999999,"5min":1.8480000000000001,"15min":2.0680000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2909999999999999,"15min":1.3300000000000001},"min":{"1min":1.0089999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.8049999999999999,"15min":2.0630000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.411,"15min":1.403},"min":{"1min":1.121,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0150000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.76900000000000002},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":2.617},"last":1.369}]},{"osd":7,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.375,"15min":1.387},"min":{"1min":1.1160000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.05},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.468,"15min":1.4650000000000001},"min":{"1min":1.238,"5min":0.753,"15min":0.753},"max":{"1min":2.0830000000000002,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.399}]}]},{"osd":1,"up_from":13,"seq":55834575059,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306904,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064936,"statfs":{"total":96636764160,"available":96322494464,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.874,"15min":0.92100000000000004},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.724},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.59199999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.90000000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.337,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.077,"15min":1.093},"min":{"1min":0.79100000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":2.137},"last":1.028}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1799999999999999,"15min":1.214},"min":{"1min":0.82999999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0640000000000001,"15min":1.0920000000000001},"min":{"1min":0.71299999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5549999999999999,"15min":1.756},"last":0.97199999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2270000000000001,"15min":1.2430000000000001},"min":{"1min":1.0149999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5529999999999999,"5min":1.891,"15min":2.2290000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3560000000000001,"15min":1.375},"min":{"1min":0.90900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.944},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.94799999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.6970000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3460000000000001,"15min":1.3440000000000001},"min":{"1min":1.173,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.871,"5min":1.923,"15min":2.1240000000000001},"last":1.129}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4350000000000001,"15min":1.46},"min":{"1min":1.0289999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8979999999999999,"5min":1.966,"15min":2.0310000000000001},"last":1.159},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.0369999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.901,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.169}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4810000000000001,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0209999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.4870000000000001,"15min":1.454},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.8100000000000001,"5min":2.085,"15min":2.085},"last":1.349}]}]},{"osd":0,"up_from":13,"seq":55834575061,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2117632,"data_stored":1372884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.571},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.54200000000000004},"max":{"1min":1.5069999999999999,"5min":1.609,"15min":1.609},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.026,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.6699999999999999,"15min":1.6950000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.0640000000000001,"15min":1.0760000000000001},"min":{"1min":0.83199999999999996,"5min":0.58399999999999996,"15min":0.55800000000000005},"max":{"1min":1.3720000000000001,"5min":1.738,"15min":1.738},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0800000000000001},"min":{"1min":0.80100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.615,"5min":1.6639999999999999,"15min":1.919},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2749999999999999,"15min":1.3169999999999999},"min":{"1min":0.90600000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.673,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.286}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3140000000000001,"15min":1.339},"min":{"1min":1.0309999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.7749999999999999,"5min":2.1720000000000002,"15min":2.294},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2609999999999999,"15min":1.2909999999999999},"min":{"1min":0.93300000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.454}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.409,"15min":1.377},"min":{"1min":1.0609999999999999,"5min":0.90100000000000002,"15min":0.65900000000000003},"max":{"1min":1.9350000000000001,"5min":2.0859999999999999,"15min":2.25},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3979999999999999,"15min":1.3959999999999999},"min":{"1min":1.103,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.893,"5min":1.893,"15min":2.5510000000000002},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.22,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.647,"5min":2.2240000000000002,"15min":2.52},"last":1.129},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.835,"5min":2.391,"15min":2.4750000000000001},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.4650000000000001},"min":{"1min":1.129,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.728,"5min":2.0219999999999998,"15min":2.4260000000000002},"last":1.236},{"interface":"front","average":{"1min":1.413,"5min":1.431,"15min":1.429},"min":{"1min":1.0820000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7589999999999999,"5min":2.1459999999999999,"15min":2.3809999999999998},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:36:13.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:36:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 Processor -- start 2023-10-06T11:36:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 -- start start 2023-10-06T11:36:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c0c7b50 0x7fd48c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd48c0c0bd0 0x7fd48c0c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0c8500 con 0x7fd48c156c80 2023-10-06T11:36:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0c6470 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.309+0000 7fd49422e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0c7760 con 0x7fd48c0c7b50 2023-10-06T11:36:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd491fca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd48c156c80 0x7fd48c0ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56450/0 (socket says 172.21.15.46:56450) 2023-10-06T11:36:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd491fca700 1 -- 172.21.15.46:0/3745719012 learned_addr learned my addr 172.21.15.46:0/3745719012 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 --2- 172.21.15.46:0/3745719012 >> v2:172.21.15.46:3300/0 conn(0x7fd48c0c0bd0 0x7fd48c0c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4917c9700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c0c7b50 0x7fd48c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204887139 0 0) 0x7fd48c0c8500 con 0x7fd48c156c80 2023-10-06T11:36:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd478003570 con 0x7fd48c156c80 2023-10-06T11:36:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 -- 172.21.15.46:0/3745719012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c0c7b50 msgr2=0x7fd48c0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c0c7b50 0x7fd48c0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 -- 172.21.15.46:0/3745719012 >> v1:172.21.15.46:6789/0 conn(0x7fd48c156c80 legacy=0x7fd48c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406838417 0 0) 0x7fd478003570 con 0x7fd48c156c80 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 -- 172.21.15.46:0/3745719012 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0c66c0 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd4927cb700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c0c1050 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd4840098e0 tx=0x7fd4840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6392286992ef5960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488001970 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd484011420 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd490fc8700 1 -- 172.21.15.46:0/3745719012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840117a0 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.313+0000 7fd49422e700 1 -- 172.21.15.46:0/3745719012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c0bd0 msgr2=0x7fd48c0c1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c0c1050 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fd4840098e0 tx=0x7fd4840091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 -- 172.21.15.46:0/3745719012 shutdown_connections 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c0c1050 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 --2- 172.21.15.46:0/3745719012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c0c7b50 0x7fd48c0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 -- 172.21.15.46:0/3745719012 >> 172.21.15.46:0/3745719012 conn(0x7fd48c0cbe30 msgr2=0x7fd48c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 -- 172.21.15.46:0/3745719012 shutdown_connections 2023-10-06T11:36:13.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 -- 172.21.15.46:0/3745719012 wait complete. 2023-10-06T11:36:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 Processor -- start 2023-10-06T11:36:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.317+0000 7fd49422e700 1 -- start start 2023-10-06T11:36:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c156c80 0x7fd48c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c06d020 con 0x7fd48c0c7b50 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0c2ee0 con 0x7fd48c0c0bd0 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0b1970 con 0x7fd48c156c80 2023-10-06T11:36:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34896/0 (socket says 172.21.15.46:34896) 2023-10-06T11:36:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 -- 172.21.15.46:0/3966526141 learned_addr learned my addr 172.21.15.46:0/3966526141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4927cb700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c156c80 0x7fd48c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd491fca700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 -- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c156c80 msgr2=0x7fd48c0b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c156c80 0x7fd48c0b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 -- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48c0c0bd0 msgr2=0x7fd48c1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 -- 172.21.15.46:0/3966526141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009420 con 0x7fd48c0c7b50 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd4917c9700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e930 tx=0x7fd47c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c013680 con 0x7fd48c0c7b50 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c0137e0 con 0x7fd48c0c7b50 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c013b80 con 0x7fd48c0c7b50 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.321+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c0b13b0 con 0x7fd48c0c7b50 2023-10-06T11:36:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.325+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c06ccb0 con 0x7fd48c0c7b50 2023-10-06T11:36:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.325+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c06d190 con 0x7fd48c0c7b50 2023-10-06T11:36:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.325+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd47c00c3e0 con 0x7fd48c0c7b50 2023-10-06T11:36:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.329+0000 7fd481ffb700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 0x7fd470062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.329+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd47c07cd20 con 0x7fd48c0c7b50 2023-10-06T11:36:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.329+0000 7fd491fca700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 0x7fd470062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.329+0000 7fd491fca700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 0x7fd470062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd488001270 tx=0x7fd488001a80 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.333+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd47c04ce40 con 0x7fd48c0c7b50 2023-10-06T11:36:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.505+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd48c063f90 con 0x7fd470060670 2023-10-06T11:36:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.521+0000 7fd481ffb700 1 -- 172.21.15.46:0/3966526141 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300529 (secure 0 0 0) 0x7fd48c063f90 con 0x7fd470060670 2023-10-06T11:36:13.540 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 msgr2=0x7fd470062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 0x7fd470062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd488001270 tx=0x7fd488001a80 comp rx=0 tx=0).stop 2023-10-06T11:36:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 msgr2=0x7fd48c0b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e930 tx=0x7fd47c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 shutdown_connections 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd470060670 0x7fd470062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd48c156c80 0x7fd48c0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48c0c7b50 0x7fd48c0b0270 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 --2- 172.21.15.46:0/3966526141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48c0c0bd0 0x7fd48c1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 >> 172.21.15.46:0/3966526141 conn(0x7fd48c0cbe30 msgr2=0x7fd48c0c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 shutdown_connections 2023-10-06T11:36:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.533+0000 7fd49422e700 1 -- 172.21.15.46:0/3966526141 wait complete. 2023-10-06T11:36:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:36:13.565 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:36:11.524906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":367,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460656,"kb_used_data":12720,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752514064,"statfs":{"total":773094113280,"available":770574401536,"internally_reserved":0,"allocated":13025280,"data_stored":7288535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.728246"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839807+0000","last_change":"2023-10-06T11:36:08.834959+0000","last_active":"2023-10-06T11:36:08.839807+0000","last_peered":"2023-10-06T11:36:08.839807+0000","last_clean":"2023-10-06T11:36:08.839807+0000","last_became_active":"2023-10-06T11:36:08.834673+0000","last_became_peered":"2023-10-06T11:36:08.834673+0000","last_unstale":"2023-10-06T11:36:08.839807+0000","last_undegraded":"2023-10-06T11:36:08.839807+0000","last_fullsized":"2023-10-06T11:36:08.839807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:45.178435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.826608+0000","last_change":"2023-10-06T11:36:08.822154+0000","last_active":"2023-10-06T11:36:08.826608+0000","last_peered":"2023-10-06T11:36:08.826608+0000","last_clean":"2023-10-06T11:36:08.826608+0000","last_became_active":"2023-10-06T11:36:08.821837+0000","last_became_peered":"2023-10-06T11:36:08.821837+0000","last_unstale":"2023-10-06T11:36:08.826608+0000","last_undegraded":"2023-10-06T11:36:08.826608+0000","last_fullsized":"2023-10-06T11:36:08.826608+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.339584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841877+0000","last_change":"2023-10-06T11:36:08.835584+0000","last_active":"2023-10-06T11:36:08.841877+0000","last_peered":"2023-10-06T11:36:08.841877+0000","last_clean":"2023-10-06T11:36:08.841877+0000","last_became_active":"2023-10-06T11:36:08.835284+0000","last_became_peered":"2023-10-06T11:36:08.835284+0000","last_unstale":"2023-10-06T11:36:08.841877+0000","last_undegraded":"2023-10-06T11:36:08.841877+0000","last_fullsized":"2023-10-06T11:36:08.841877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850850+0000","last_change":"2023-10-06T11:36:08.846553+0000","last_active":"2023-10-06T11:36:08.850850+0000","last_peered":"2023-10-06T11:36:08.850850+0000","last_clean":"2023-10-06T11:36:08.850850+0000","last_became_active":"2023-10-06T11:36:08.846228+0000","last_became_peered":"2023-10-06T11:36:08.846228+0000","last_unstale":"2023-10-06T11:36:08.850850+0000","last_undegraded":"2023-10-06T11:36:08.850850+0000","last_fullsized":"2023-10-06T11:36:08.850850+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834928+0000","last_change":"2023-10-06T11:36:08.829614+0000","last_active":"2023-10-06T11:36:08.834928+0000","last_peered":"2023-10-06T11:36:08.834928+0000","last_clean":"2023-10-06T11:36:08.834928+0000","last_became_active":"2023-10-06T11:36:08.829269+0000","last_became_peered":"2023-10-06T11:36:08.829269+0000","last_unstale":"2023-10-06T11:36:08.834928+0000","last_undegraded":"2023-10-06T11:36:08.834928+0000","last_fullsized":"2023-10-06T11:36:08.834928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.821913+0000","last_change":"2023-10-06T11:36:08.820676+0000","last_active":"2023-10-06T11:36:08.821913+0000","last_peered":"2023-10-06T11:36:08.821913+0000","last_clean":"2023-10-06T11:36:08.821913+0000","last_became_active":"2023-10-06T11:36:08.820319+0000","last_became_peered":"2023-10-06T11:36:08.820319+0000","last_unstale":"2023-10-06T11:36:08.821913+0000","last_undegraded":"2023-10-06T11:36:08.821913+0000","last_fullsized":"2023-10-06T11:36:08.821913+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:53:05.714850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835909+0000","last_change":"2023-10-06T11:36:08.833419+0000","last_active":"2023-10-06T11:36:08.835909+0000","last_peered":"2023-10-06T11:36:08.835909+0000","last_clean":"2023-10-06T11:36:08.835909+0000","last_became_active":"2023-10-06T11:36:08.832970+0000","last_became_peered":"2023-10-06T11:36:08.832970+0000","last_unstale":"2023-10-06T11:36:08.835909+0000","last_undegraded":"2023-10-06T11:36:08.835909+0000","last_fullsized":"2023-10-06T11:36:08.835909+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:28:24.073017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847734+0000","last_change":"2023-10-06T11:36:08.834706+0000","last_active":"2023-10-06T11:36:08.847734+0000","last_peered":"2023-10-06T11:36:08.847734+0000","last_clean":"2023-10-06T11:36:08.847734+0000","last_became_active":"2023-10-06T11:36:08.834392+0000","last_became_peered":"2023-10-06T11:36:08.834392+0000","last_unstale":"2023-10-06T11:36:08.847734+0000","last_undegraded":"2023-10-06T11:36:08.847734+0000","last_fullsized":"2023-10-06T11:36:08.847734+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832710+0000","last_change":"2023-10-06T11:36:08.825112+0000","last_active":"2023-10-06T11:36:08.832710+0000","last_peered":"2023-10-06T11:36:08.832710+0000","last_clean":"2023-10-06T11:36:08.832710+0000","last_became_active":"2023-10-06T11:36:08.824817+0000","last_became_peered":"2023-10-06T11:36:08.824817+0000","last_unstale":"2023-10-06T11:36:08.832710+0000","last_undegraded":"2023-10-06T11:36:08.832710+0000","last_fullsized":"2023-10-06T11:36:08.832710+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:47.661809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847711+0000","last_change":"2023-10-06T11:36:08.830867+0000","last_active":"2023-10-06T11:36:08.847711+0000","last_peered":"2023-10-06T11:36:08.847711+0000","last_clean":"2023-10-06T11:36:08.847711+0000","last_became_active":"2023-10-06T11:36:08.830226+0000","last_became_peered":"2023-10-06T11:36:08.830226+0000","last_unstale":"2023-10-06T11:36:08.847711+0000","last_undegraded":"2023-10-06T11:36:08.847711+0000","last_fullsized":"2023-10-06T11:36:08.847711+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827000+0000","last_change":"2023-10-06T11:36:08.825532+0000","last_active":"2023-10-06T11:36:08.827000+0000","last_peered":"2023-10-06T11:36:08.827000+0000","last_clean":"2023-10-06T11:36:08.827000+0000","last_became_active":"2023-10-06T11:36:08.825246+0000","last_became_peered":"2023-10-06T11:36:08.825246+0000","last_unstale":"2023-10-06T11:36:08.827000+0000","last_undegraded":"2023-10-06T11:36:08.827000+0000","last_fullsized":"2023-10-06T11:36:08.827000+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837616+0000","last_change":"2023-10-06T11:36:08.832215+0000","last_active":"2023-10-06T11:36:08.837616+0000","last_peered":"2023-10-06T11:36:08.837616+0000","last_clean":"2023-10-06T11:36:08.837616+0000","last_became_active":"2023-10-06T11:36:08.831906+0000","last_became_peered":"2023-10-06T11:36:08.831906+0000","last_unstale":"2023-10-06T11:36:08.837616+0000","last_undegraded":"2023-10-06T11:36:08.837616+0000","last_fullsized":"2023-10-06T11:36:08.837616+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:21.848572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"121'4","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.854694+0000","last_change":"2023-10-06T11:36:07.836398+0000","last_active":"2023-10-06T11:36:08.854694+0000","last_peered":"2023-10-06T11:36:08.854694+0000","last_clean":"2023-10-06T11:36:08.854694+0000","last_became_active":"2023-10-06T11:36:07.836063+0000","last_became_peered":"2023-10-06T11:36:07.836063+0000","last_unstale":"2023-10-06T11:36:08.854694+0000","last_undegraded":"2023-10-06T11:36:08.854694+0000","last_fullsized":"2023-10-06T11:36:08.854694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850887+0000","last_change":"2023-10-06T11:36:08.845254+0000","last_active":"2023-10-06T11:36:08.850887+0000","last_peered":"2023-10-06T11:36:08.850887+0000","last_clean":"2023-10-06T11:36:08.850887+0000","last_became_active":"2023-10-06T11:36:08.844729+0000","last_became_peered":"2023-10-06T11:36:08.844729+0000","last_unstale":"2023-10-06T11:36:08.850887+0000","last_undegraded":"2023-10-06T11:36:08.850887+0000","last_fullsized":"2023-10-06T11:36:08.850887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:46.014384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857089+0000","last_change":"2023-10-06T11:36:07.830544+0000","last_active":"2023-10-06T11:36:08.857089+0000","last_peered":"2023-10-06T11:36:08.857089+0000","last_clean":"2023-10-06T11:36:08.857089+0000","last_became_active":"2023-10-06T11:36:07.830229+0000","last_became_peered":"2023-10-06T11:36:07.830229+0000","last_unstale":"2023-10-06T11:36:08.857089+0000","last_undegraded":"2023-10-06T11:36:08.857089+0000","last_fullsized":"2023-10-06T11:36:08.857089+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841854+0000","last_change":"2023-10-06T11:36:08.834817+0000","last_active":"2023-10-06T11:36:08.841854+0000","last_peered":"2023-10-06T11:36:08.841854+0000","last_clean":"2023-10-06T11:36:08.841854+0000","last_became_active":"2023-10-06T11:36:08.834510+0000","last_became_peered":"2023-10-06T11:36:08.834510+0000","last_unstale":"2023-10-06T11:36:08.841854+0000","last_undegraded":"2023-10-06T11:36:08.841854+0000","last_fullsized":"2023-10-06T11:36:08.841854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835929+0000","last_change":"2023-10-06T11:36:08.833625+0000","last_active":"2023-10-06T11:36:08.835929+0000","last_peered":"2023-10-06T11:36:08.835929+0000","last_clean":"2023-10-06T11:36:08.835929+0000","last_became_active":"2023-10-06T11:36:08.833132+0000","last_became_peered":"2023-10-06T11:36:08.833132+0000","last_unstale":"2023-10-06T11:36:08.835929+0000","last_undegraded":"2023-10-06T11:36:08.835929+0000","last_fullsized":"2023-10-06T11:36:08.835929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.375194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833946+0000","last_change":"2023-10-06T11:36:08.822864+0000","last_active":"2023-10-06T11:36:08.833946+0000","last_peered":"2023-10-06T11:36:08.833946+0000","last_clean":"2023-10-06T11:36:08.833946+0000","last_became_active":"2023-10-06T11:36:08.822567+0000","last_became_peered":"2023-10-06T11:36:08.822567+0000","last_unstale":"2023-10-06T11:36:08.833946+0000","last_undegraded":"2023-10-06T11:36:08.833946+0000","last_fullsized":"2023-10-06T11:36:08.833946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827606+0000","last_change":"2023-10-06T11:36:07.838208+0000","last_active":"2023-10-06T11:36:08.827606+0000","last_peered":"2023-10-06T11:36:08.827606+0000","last_clean":"2023-10-06T11:36:08.827606+0000","last_became_active":"2023-10-06T11:36:07.837843+0000","last_became_peered":"2023-10-06T11:36:07.837843+0000","last_unstale":"2023-10-06T11:36:08.827606+0000","last_undegraded":"2023-10-06T11:36:08.827606+0000","last_fullsized":"2023-10-06T11:36:08.827606+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837595+0000","last_change":"2023-10-06T11:36:08.831541+0000","last_active":"2023-10-06T11:36:08.837595+0000","last_peered":"2023-10-06T11:36:08.837595+0000","last_clean":"2023-10-06T11:36:08.837595+0000","last_became_active":"2023-10-06T11:36:08.831234+0000","last_became_peered":"2023-10-06T11:36:08.831234+0000","last_unstale":"2023-10-06T11:36:08.837595+0000","last_undegraded":"2023-10-06T11:36:08.837595+0000","last_fullsized":"2023-10-06T11:36:08.837595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:26.203324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810127+0000","last_change":"2023-10-06T11:36:07.831652+0000","last_active":"2023-10-06T11:36:08.810127+0000","last_peered":"2023-10-06T11:36:08.810127+0000","last_clean":"2023-10-06T11:36:08.810127+0000","last_became_active":"2023-10-06T11:36:07.831331+0000","last_became_peered":"2023-10-06T11:36:07.831331+0000","last_unstale":"2023-10-06T11:36:08.810127+0000","last_undegraded":"2023-10-06T11:36:08.810127+0000","last_fullsized":"2023-10-06T11:36:08.810127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:30:03.936003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850794+0000","last_change":"2023-10-06T11:36:08.843325+0000","last_active":"2023-10-06T11:36:08.850794+0000","last_peered":"2023-10-06T11:36:08.850794+0000","last_clean":"2023-10-06T11:36:08.850794+0000","last_became_active":"2023-10-06T11:36:08.843032+0000","last_became_peered":"2023-10-06T11:36:08.843032+0000","last_unstale":"2023-10-06T11:36:08.850794+0000","last_undegraded":"2023-10-06T11:36:08.850794+0000","last_fullsized":"2023-10-06T11:36:08.850794+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:50:13.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.830870+0000","last_change":"2023-10-06T11:36:07.833180+0000","last_active":"2023-10-06T11:36:08.830870+0000","last_peered":"2023-10-06T11:36:08.830870+0000","last_clean":"2023-10-06T11:36:08.830870+0000","last_became_active":"2023-10-06T11:36:07.832883+0000","last_became_peered":"2023-10-06T11:36:07.832883+0000","last_unstale":"2023-10-06T11:36:08.830870+0000","last_undegraded":"2023-10-06T11:36:08.830870+0000","last_fullsized":"2023-10-06T11:36:08.830870+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827807+0000","last_change":"2023-10-06T11:36:08.826278+0000","last_active":"2023-10-06T11:36:08.827807+0000","last_peered":"2023-10-06T11:36:08.827807+0000","last_clean":"2023-10-06T11:36:08.827807+0000","last_became_active":"2023-10-06T11:36:08.825985+0000","last_became_peered":"2023-10-06T11:36:08.825985+0000","last_unstale":"2023-10-06T11:36:08.827807+0000","last_undegraded":"2023-10-06T11:36:08.827807+0000","last_fullsized":"2023-10-06T11:36:08.827807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820190+0000","last_change":"2023-10-06T11:36:07.840409+0000","last_active":"2023-10-06T11:36:08.820190+0000","last_peered":"2023-10-06T11:36:08.820190+0000","last_clean":"2023-10-06T11:36:08.820190+0000","last_became_active":"2023-10-06T11:36:07.840059+0000","last_became_peered":"2023-10-06T11:36:07.840059+0000","last_unstale":"2023-10-06T11:36:08.820190+0000","last_undegraded":"2023-10-06T11:36:08.820190+0000","last_fullsized":"2023-10-06T11:36:08.820190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:54.481491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832875+0000","last_change":"2023-10-06T11:36:08.826025+0000","last_active":"2023-10-06T11:36:08.832875+0000","last_peered":"2023-10-06T11:36:08.832875+0000","last_clean":"2023-10-06T11:36:08.832875+0000","last_became_active":"2023-10-06T11:36:08.825431+0000","last_became_peered":"2023-10-06T11:36:08.825431+0000","last_unstale":"2023-10-06T11:36:08.832875+0000","last_undegraded":"2023-10-06T11:36:08.832875+0000","last_fullsized":"2023-10-06T11:36:08.832875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:56.221716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839926+0000","last_change":"2023-10-06T11:36:08.837343+0000","last_active":"2023-10-06T11:36:08.839926+0000","last_peered":"2023-10-06T11:36:08.839926+0000","last_clean":"2023-10-06T11:36:08.839926+0000","last_became_active":"2023-10-06T11:36:08.835511+0000","last_became_peered":"2023-10-06T11:36:08.835511+0000","last_unstale":"2023-10-06T11:36:08.839926+0000","last_undegraded":"2023-10-06T11:36:08.839926+0000","last_fullsized":"2023-10-06T11:36:08.839926+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:20.565177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810349+0000","last_change":"2023-10-06T11:36:07.833889+0000","last_active":"2023-10-06T11:36:08.810349+0000","last_peered":"2023-10-06T11:36:08.810349+0000","last_clean":"2023-10-06T11:36:08.810349+0000","last_became_active":"2023-10-06T11:36:07.833564+0000","last_became_peered":"2023-10-06T11:36:07.833564+0000","last_unstale":"2023-10-06T11:36:08.810349+0000","last_undegraded":"2023-10-06T11:36:08.810349+0000","last_fullsized":"2023-10-06T11:36:08.810349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:43.919700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835073+0000","last_change":"2023-10-06T11:36:08.832266+0000","last_active":"2023-10-06T11:36:08.835073+0000","last_peered":"2023-10-06T11:36:08.835073+0000","last_clean":"2023-10-06T11:36:08.835073+0000","last_became_active":"2023-10-06T11:36:08.830568+0000","last_became_peered":"2023-10-06T11:36:08.830568+0000","last_unstale":"2023-10-06T11:36:08.835073+0000","last_undegraded":"2023-10-06T11:36:08.835073+0000","last_fullsized":"2023-10-06T11:36:08.835073+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850140+0000","last_change":"2023-10-06T11:36:08.836975+0000","last_active":"2023-10-06T11:36:08.850140+0000","last_peered":"2023-10-06T11:36:08.850140+0000","last_clean":"2023-10-06T11:36:08.850140+0000","last_became_active":"2023-10-06T11:36:08.836692+0000","last_became_peered":"2023-10-06T11:36:08.836692+0000","last_unstale":"2023-10-06T11:36:08.850140+0000","last_undegraded":"2023-10-06T11:36:08.850140+0000","last_fullsized":"2023-10-06T11:36:08.850140+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:19.467727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850838+0000","last_change":"2023-10-06T11:36:08.843940+0000","last_active":"2023-10-06T11:36:08.850838+0000","last_peered":"2023-10-06T11:36:08.850838+0000","last_clean":"2023-10-06T11:36:08.850838+0000","last_became_active":"2023-10-06T11:36:08.843666+0000","last_became_peered":"2023-10-06T11:36:08.843666+0000","last_unstale":"2023-10-06T11:36:08.850838+0000","last_undegraded":"2023-10-06T11:36:08.850838+0000","last_fullsized":"2023-10-06T11:36:08.850838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822603+0000","last_change":"2023-10-06T11:36:07.840413+0000","last_active":"2023-10-06T11:36:08.822603+0000","last_peered":"2023-10-06T11:36:08.822603+0000","last_clean":"2023-10-06T11:36:08.822603+0000","last_became_active":"2023-10-06T11:36:07.840091+0000","last_became_peered":"2023-10-06T11:36:07.840091+0000","last_unstale":"2023-10-06T11:36:08.822603+0000","last_undegraded":"2023-10-06T11:36:08.822603+0000","last_fullsized":"2023-10-06T11:36:08.822603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:25.925632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819931+0000","last_change":"2023-10-06T11:36:07.841033+0000","last_active":"2023-10-06T11:36:08.819931+0000","last_peered":"2023-10-06T11:36:08.819931+0000","last_clean":"2023-10-06T11:36:08.819931+0000","last_became_active":"2023-10-06T11:36:07.840720+0000","last_became_peered":"2023-10-06T11:36:07.840720+0000","last_unstale":"2023-10-06T11:36:08.819931+0000","last_undegraded":"2023-10-06T11:36:08.819931+0000","last_fullsized":"2023-10-06T11:36:08.819931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857037+0000","last_change":"2023-10-06T11:36:07.838666+0000","last_active":"2023-10-06T11:36:08.857037+0000","last_peered":"2023-10-06T11:36:08.857037+0000","last_clean":"2023-10-06T11:36:08.857037+0000","last_became_active":"2023-10-06T11:36:07.837999+0000","last_became_peered":"2023-10-06T11:36:07.837999+0000","last_unstale":"2023-10-06T11:36:08.857037+0000","last_undegraded":"2023-10-06T11:36:08.857037+0000","last_fullsized":"2023-10-06T11:36:08.857037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"111'227","reported_seq":637,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.666094+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:35:59.666094+0000","last_peered":"2023-10-06T11:35:59.666094+0000","last_clean":"2023-10-06T11:35:59.666094+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:35:59.666094+0000","last_undegraded":"2023-10-06T11:35:59.666094+0000","last_fullsized":"2023-10-06T11:35:59.666094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.795932+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:36:03.795932+0000","last_peered":"2023-10-06T11:36:03.795932+0000","last_clean":"2023-10-06T11:36:03.795932+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:36:03.795932+0000","last_undegraded":"2023-10-06T11:36:03.795932+0000","last_fullsized":"2023-10-06T11:36:03.795932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.665926+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:35:59.665926+0000","last_peered":"2023-10-06T11:35:59.665926+0000","last_clean":"2023-10-06T11:35:59.665926+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:35:59.665926+0000","last_undegraded":"2023-10-06T11:35:59.665926+0000","last_fullsized":"2023-10-06T11:35:59.665926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:07.796936+0000","last_change":"2023-10-06T11:36:07.796936+0000","last_active":"2023-10-06T11:36:07.796936+0000","last_peered":"2023-10-06T11:36:07.796936+0000","last_clean":"2023-10-06T11:36:07.796936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:07.796936+0000","last_undegraded":"2023-10-06T11:36:07.796936+0000","last_fullsized":"2023-10-06T11:36:07.796936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820278+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:36:08.820278+0000","last_peered":"2023-10-06T11:36:08.820278+0000","last_clean":"2023-10-06T11:36:08.820278+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:36:08.820278+0000","last_undegraded":"2023-10-06T11:36:08.820278+0000","last_fullsized":"2023-10-06T11:36:08.820278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847753+0000","last_change":"2023-10-06T11:36:08.834917+0000","last_active":"2023-10-06T11:36:08.847753+0000","last_peered":"2023-10-06T11:36:08.847753+0000","last_clean":"2023-10-06T11:36:08.847753+0000","last_became_active":"2023-10-06T11:36:08.834522+0000","last_became_peered":"2023-10-06T11:36:08.834522+0000","last_unstale":"2023-10-06T11:36:08.847753+0000","last_undegraded":"2023-10-06T11:36:08.847753+0000","last_fullsized":"2023-10-06T11:36:08.847753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.795885+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:36:03.795885+0000","last_peered":"2023-10-06T11:36:03.795885+0000","last_clean":"2023-10-06T11:36:03.795885+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:36:03.795885+0000","last_undegraded":"2023-10-06T11:36:03.795885+0000","last_fullsized":"2023-10-06T11:36:03.795885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834204+0000","last_change":"2023-10-06T11:36:08.831735+0000","last_active":"2023-10-06T11:36:08.834204+0000","last_peered":"2023-10-06T11:36:08.834204+0000","last_clean":"2023-10-06T11:36:08.834204+0000","last_became_active":"2023-10-06T11:36:08.831450+0000","last_became_peered":"2023-10-06T11:36:08.831450+0000","last_unstale":"2023-10-06T11:36:08.834204+0000","last_undegraded":"2023-10-06T11:36:08.834204+0000","last_fullsized":"2023-10-06T11:36:08.834204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:36:06.782014+0000","last_change":"2023-10-06T11:36:06.782014+0000","last_active":"2023-10-06T11:36:06.782014+0000","last_peered":"2023-10-06T11:36:06.782014+0000","last_clean":"2023-10-06T11:36:06.782014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:36:06.782014+0000","last_undegraded":"2023-10-06T11:36:06.782014+0000","last_fullsized":"2023-10-06T11:36:06.782014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.792576+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:36:03.792576+0000","last_peered":"2023-10-06T11:36:03.792576+0000","last_clean":"2023-10-06T11:36:03.792576+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:36:03.792576+0000","last_undegraded":"2023-10-06T11:36:03.792576+0000","last_fullsized":"2023-10-06T11:36:03.792576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848126+0000","last_change":"2023-10-06T11:36:08.836395+0000","last_active":"2023-10-06T11:36:08.848126+0000","last_peered":"2023-10-06T11:36:08.848126+0000","last_clean":"2023-10-06T11:36:08.848126+0000","last_became_active":"2023-10-06T11:36:08.836112+0000","last_became_peered":"2023-10-06T11:36:08.836112+0000","last_unstale":"2023-10-06T11:36:08.848126+0000","last_undegraded":"2023-10-06T11:36:08.848126+0000","last_fullsized":"2023-10-06T11:36:08.848126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819833+0000","last_change":"2023-10-06T11:36:07.840551+0000","last_active":"2023-10-06T11:36:08.819833+0000","last_peered":"2023-10-06T11:36:08.819833+0000","last_clean":"2023-10-06T11:36:08.819833+0000","last_became_active":"2023-10-06T11:36:07.840210+0000","last_became_peered":"2023-10-06T11:36:07.840210+0000","last_unstale":"2023-10-06T11:36:08.819833+0000","last_undegraded":"2023-10-06T11:36:08.819833+0000","last_fullsized":"2023-10-06T11:36:08.819833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T11:35:59.664611+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:35:59.664611+0000","last_peered":"2023-10-06T11:35:59.664611+0000","last_clean":"2023-10-06T11:35:59.664611+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:35:59.664611+0000","last_undegraded":"2023-10-06T11:35:59.664611+0000","last_fullsized":"2023-10-06T11:35:59.664611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839738+0000","last_change":"2023-10-06T11:36:08.836510+0000","last_active":"2023-10-06T11:36:08.839738+0000","last_peered":"2023-10-06T11:36:08.839738+0000","last_clean":"2023-10-06T11:36:08.839738+0000","last_became_active":"2023-10-06T11:36:08.836215+0000","last_became_peered":"2023-10-06T11:36:08.836215+0000","last_unstale":"2023-10-06T11:36:08.839738+0000","last_undegraded":"2023-10-06T11:36:08.839738+0000","last_fullsized":"2023-10-06T11:36:08.839738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:32.797025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857016+0000","last_change":"2023-10-06T11:36:07.841111+0000","last_active":"2023-10-06T11:36:08.857016+0000","last_peered":"2023-10-06T11:36:08.857016+0000","last_clean":"2023-10-06T11:36:08.857016+0000","last_became_active":"2023-10-06T11:36:07.840810+0000","last_became_peered":"2023-10-06T11:36:07.840810+0000","last_unstale":"2023-10-06T11:36:08.857016+0000","last_undegraded":"2023-10-06T11:36:08.857016+0000","last_fullsized":"2023-10-06T11:36:08.857016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:03.725992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.792625+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:36:03.792625+0000","last_peered":"2023-10-06T11:36:03.792625+0000","last_clean":"2023-10-06T11:36:03.792625+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:36:03.792625+0000","last_undegraded":"2023-10-06T11:36:03.792625+0000","last_fullsized":"2023-10-06T11:36:03.792625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837031+0000","last_change":"2023-10-06T11:36:08.833641+0000","last_active":"2023-10-06T11:36:08.837031+0000","last_peered":"2023-10-06T11:36:08.837031+0000","last_clean":"2023-10-06T11:36:08.837031+0000","last_became_active":"2023-10-06T11:36:08.833372+0000","last_became_peered":"2023-10-06T11:36:08.833372+0000","last_unstale":"2023-10-06T11:36:08.837031+0000","last_undegraded":"2023-10-06T11:36:08.837031+0000","last_fullsized":"2023-10-06T11:36:08.837031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:45.164451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822466+0000","last_change":"2023-10-06T11:36:07.841750+0000","last_active":"2023-10-06T11:36:08.822466+0000","last_peered":"2023-10-06T11:36:08.822466+0000","last_clean":"2023-10-06T11:36:08.822466+0000","last_became_active":"2023-10-06T11:36:07.841432+0000","last_became_peered":"2023-10-06T11:36:07.841432+0000","last_unstale":"2023-10-06T11:36:08.822466+0000","last_undegraded":"2023-10-06T11:36:08.822466+0000","last_fullsized":"2023-10-06T11:36:08.822466+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:27:06.372507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-06T11:36:03.800706+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:36:03.800706+0000","last_peered":"2023-10-06T11:36:03.800706+0000","last_clean":"2023-10-06T11:36:03.800706+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:36:03.800706+0000","last_undegraded":"2023-10-06T11:36:03.800706+0000","last_fullsized":"2023-10-06T11:36:03.800706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":3},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":554,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":26,"acting":26,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575059,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311580,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060260,"statfs":{"total":96636764160,"available":96317706240,"internally_reserved":0,"allocated":2052096,"data_stored":1342951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1020000000000001,"15min":1.1339999999999999},"min":{"1min":0.84499999999999997,"5min":0.59599999999999997,"15min":0.495},"max":{"1min":1.5249999999999999,"5min":1.6579999999999999,"15min":1.724},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.159,"5min":1.143,"15min":1.139},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.56399999999999995},"max":{"1min":1.5229999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.359,"15min":1.333},"min":{"1min":0.78100000000000003,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.651,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.448},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.232},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5349999999999999,"5min":1.927,"15min":1.927},"last":1.256}]},{"osd":2,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2,"15min":1.214},"min":{"1min":1.006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6040000000000001,"5min":1.827,"15min":1.827},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.379}]},{"osd":3,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.256,"15min":1.264},"min":{"1min":0.92900000000000005,"5min":0.79300000000000004,"15min":0.68500000000000005},"max":{"1min":1.7210000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.048},{"interface":"front","average":{"1min":1.454,"5min":1.3380000000000001,"15min":1.323},"min":{"1min":1.2829999999999999,"5min":0.89000000000000001,"15min":0.75600000000000001},"max":{"1min":1.8029999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.208}]},{"osd":4,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1120000000000001},"min":{"1min":0.72799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.601,"5min":1.663,"15min":2.149},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.25},"min":{"1min":0.85199999999999998,"5min":0.55300000000000005,"15min":0.41699999999999998},"max":{"1min":1.7849999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3700000000000001,"15min":1.3109999999999999},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.58199999999999996},"max":{"1min":1.881,"5min":2.488,"15min":2.488},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":1.903,"15min":2.2480000000000002},"last":1.077}]},{"osd":6,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.347,"15min":1.349},"min":{"1min":0.93600000000000005,"5min":0.67000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7250000000000001,"5min":2.0219999999999998,"15min":2.4289999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.60099999999999998,"15min":0.54200000000000004},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.1919999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306904,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064936,"statfs":{"total":96636764160,"available":96322494464,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2869999999999999,"15min":1.2889999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.891,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.3180000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.546,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.6319999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1439999999999999,"15min":1.1579999999999999},"min":{"1min":1.024,"5min":0.76200000000000001,"15min":0.64000000000000001},"max":{"1min":1.48,"5min":1.5980000000000001,"15min":1.8169999999999999},"last":1.01},{"interface":"front","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.214},"min":{"1min":0.78600000000000003,"5min":0.66200000000000003,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.202,"15min":1.206},"min":{"1min":0.85399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.476,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.214},"min":{"1min":0.79300000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5009999999999999,"5min":1.736,"15min":1.736},"last":1.675}]},{"osd":3,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.306},"min":{"1min":0.89500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.625,"5min":2.121,"15min":2.121},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3240000000000001,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.597,"5min":1.7430000000000001,"15min":1.7509999999999999},"last":1.7050000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.90700000000000003,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.304,"15min":1.248},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":1.649,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.875}]},{"osd":5,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.196,"15min":1.2270000000000001},"min":{"1min":0.89200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5529999999999999,"5min":1.879,"15min":1.879},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.92100000000000004,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.9359999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4390000000000001},"min":{"1min":0.84199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.77,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.256},{"interface":"front","average":{"1min":1.363,"5min":1.389,"15min":1.415},"min":{"1min":1.0129999999999999,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5620000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3979999999999999,"15min":1.284},"min":{"1min":0.89900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.615,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.173},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.296},"min":{"1min":1.095,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":1.972,"15min":2.3959999999999999},"last":1.6140000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1479999999999999,"15min":1.1639999999999999},"min":{"1min":0.82999999999999996,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.3999999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.212},"min":{"1min":0.84799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5529999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":0.874}]},{"osd":2,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.88800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5149999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.554},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.6990000000000001,"15min":1.738},"last":1.3460000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.397,"15min":1.28},"min":{"1min":1.0109999999999999,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.454,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.2989999999999999},"min":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.581,"5min":1.8400000000000001,"15min":1.871},"last":1.2330000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3460000000000001,"15min":1.2609999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.5369999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5309999999999999,"15min":1.383},"min":{"1min":0.83899999999999997,"5min":0.72199999999999998,"15min":0.63500000000000001},"max":{"1min":1.6739999999999999,"5min":18.48,"15min":18.48},"last":1.5840000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.472},{"interface":"front","average":{"1min":1.252,"5min":1.3999999999999999,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.4870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.6759999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.488,"15min":1.3660000000000001},"min":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.609,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.522},{"interface":"front","average":{"1min":1.268,"5min":1.4550000000000001,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.534,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.8360000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302808,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.77400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.669,"5min":1.72,"15min":1.72},"last":1.375},{"interface":"front","average":{"1min":1.175,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.84499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.48,"5min":1.5669999999999999,"15min":1.6910000000000001},"last":1.127}]},{"osd":1,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":0.94399999999999995,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.26,"15min":1.272},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2010000000000001,"15min":1.196},"min":{"1min":0.874,"5min":0.628,"15min":0.628},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.105},{"interface":"front","average":{"1min":1.302,"5min":1.274,"15min":1.2669999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6619999999999999,"5min":1.6779999999999999,"15min":1.78},"last":1.2849999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1839999999999999,"15min":1.204},"min":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6040000000000001,"5min":1.617,"15min":1.6839999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.3180000000000001,"15min":1.3049999999999999},"min":{"1min":1.0369999999999999,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.77,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.82699999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.706,"5min":1.706,"15min":1.7390000000000001},"last":1.218},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.571,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.155}]},{"osd":6,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2390000000000001,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.995},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.3480000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.831,"5min":1.845,"15min":1.8660000000000001},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.349,"15min":1.329},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4359999999999999,"15min":1.3879999999999999},"min":{"1min":0.96299999999999997,"5min":0.88700000000000001,"15min":0.73799999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.494}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":781444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.052,"15min":0.93500000000000005},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":14.49,"15min":14.49},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.998,"5min":1.175,"15min":1.05},"min":{"1min":0.81200000000000006,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.98599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98099999999999998,"15min":0.96699999999999997},"min":{"1min":0.73899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1890000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.1579999999999999,"15min":1.1140000000000001},"min":{"1min":0.72899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.698,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.23,"15min":1.1599999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.6080000000000001,"5min":12.446,"15min":12.446},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.76100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.5009999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3169999999999999,"15min":1.2589999999999999},"min":{"1min":0.89200000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":10.460000000000001,"15min":10.460000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.343,"15min":1.3560000000000001},"min":{"1min":1.0289999999999999,"5min":0.80800000000000005,"15min":0.78400000000000003},"max":{"1min":1.5569999999999999,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.407,"15min":1.411},"min":{"1min":1.0860000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.73,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.111},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.874,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7509999999999999,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.3009999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.8080000000000001,"5min":11.362,"15min":11.362},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.4419999999999999,"15min":1.399},"min":{"1min":0.98499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6679999999999999,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.52,"15min":1.4610000000000001},"min":{"1min":1.1759999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7689999999999999,"5min":10.561,"15min":10.561},"last":1.607},{"interface":"front","average":{"1min":1.48,"5min":1.4890000000000001,"15min":1.4870000000000001},"min":{"1min":1.1679999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.792,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.3320000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.56999999999999995,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4530000000000001,"5min":1.524,"15min":1.8380000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.52,"5min":1.5409999999999999,"15min":1.9650000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.018},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.5760000000000001,"5min":1.599,"15min":2.5390000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.84599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.546,"5min":1.758,"15min":1.8240000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.288,"5min":1.256,"15min":1.2989999999999999},"min":{"1min":0.85499999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"max":{"1min":1.6419999999999999,"5min":1.948,"15min":2.4550000000000001},"last":1.016}]},{"osd":4,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.96999999999999997,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.569,"5min":1.625,"15min":1.907},"last":1.093},{"interface":"front","average":{"1min":1.373,"5min":1.3,"15min":1.29},"min":{"1min":1.071,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3420000000000001,"15min":1.3580000000000001},"min":{"1min":0.94099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8049999999999999,"5min":1.8480000000000001,"15min":2.0680000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2909999999999999,"15min":1.3300000000000001},"min":{"1min":1.0089999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.8049999999999999,"15min":2.0630000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.411,"15min":1.403},"min":{"1min":1.121,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0150000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.76900000000000002},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":2.617},"last":1.369}]},{"osd":7,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.375,"15min":1.387},"min":{"1min":1.1160000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.05},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.468,"15min":1.4650000000000001},"min":{"1min":1.238,"5min":0.753,"15min":0.753},"max":{"1min":2.0830000000000002,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.399}]}]},{"osd":1,"up_from":13,"seq":55834575059,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306904,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064936,"statfs":{"total":96636764160,"available":96322494464,"internally_reserved":0,"allocated":1458176,"data_stored":752583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.874,"15min":0.92100000000000004},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.724},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.59199999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.90000000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.337,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.077,"15min":1.093},"min":{"1min":0.79100000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":2.137},"last":1.028}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1799999999999999,"15min":1.214},"min":{"1min":0.82999999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0640000000000001,"15min":1.0920000000000001},"min":{"1min":0.71299999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5549999999999999,"15min":1.756},"last":0.97199999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2270000000000001,"15min":1.2430000000000001},"min":{"1min":1.0149999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5529999999999999,"5min":1.891,"15min":2.2290000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3560000000000001,"15min":1.375},"min":{"1min":0.90900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.944},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.94799999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.6970000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3460000000000001,"15min":1.3440000000000001},"min":{"1min":1.173,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.871,"5min":1.923,"15min":2.1240000000000001},"last":1.129}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4350000000000001,"15min":1.46},"min":{"1min":1.0289999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8979999999999999,"5min":1.966,"15min":2.0310000000000001},"last":1.159},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.0369999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.901,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.169}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4810000000000001,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0209999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.4870000000000001,"15min":1.454},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.8100000000000001,"5min":2.085,"15min":2.085},"last":1.349}]}]},{"osd":0,"up_from":13,"seq":55834575061,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2117632,"data_stored":1372884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.571},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.54200000000000004},"max":{"1min":1.5069999999999999,"5min":1.609,"15min":1.609},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.026,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.6699999999999999,"15min":1.6950000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.0640000000000001,"15min":1.0760000000000001},"min":{"1min":0.83199999999999996,"5min":0.58399999999999996,"15min":0.55800000000000005},"max":{"1min":1.3720000000000001,"5min":1.738,"15min":1.738},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0800000000000001},"min":{"1min":0.80100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.615,"5min":1.6639999999999999,"15min":1.919},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2749999999999999,"15min":1.3169999999999999},"min":{"1min":0.90600000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.673,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.286}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3140000000000001,"15min":1.339},"min":{"1min":1.0309999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.7749999999999999,"5min":2.1720000000000002,"15min":2.294},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2609999999999999,"15min":1.2909999999999999},"min":{"1min":0.93300000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.454}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.409,"15min":1.377},"min":{"1min":1.0609999999999999,"5min":0.90100000000000002,"15min":0.65900000000000003},"max":{"1min":1.9350000000000001,"5min":2.0859999999999999,"15min":2.25},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3979999999999999,"15min":1.3959999999999999},"min":{"1min":1.103,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.893,"5min":1.893,"15min":2.5510000000000002},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.22,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.647,"5min":2.2240000000000002,"15min":2.52},"last":1.129},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.835,"5min":2.391,"15min":2.4750000000000001},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.4650000000000001},"min":{"1min":1.129,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.728,"5min":2.0219999999999998,"15min":2.4260000000000002},"last":1.236},{"interface":"front","average":{"1min":1.413,"5min":1.431,"15min":1.429},"min":{"1min":1.0820000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7589999999999999,"5min":2.1459999999999999,"15min":2.3809999999999998},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:36:13.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:36:13.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.757+0000 7f491f945700 1 Processor -- start 2023-10-06T11:36:13.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 -- start start 2023-10-06T11:36:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4918161470 0x7f4918161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49180c1d80 0x7f49180c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f491806cc30 0x7f491806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180c18f0 con 0x7f49180c1d80 2023-10-06T11:36:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49180c0ce0 con 0x7f491806cc30 2023-10-06T11:36:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491f945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c0fc0 con 0x7f4918161470 2023-10-06T11:36:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49180c1d80 0x7f49180c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49180c1d80 0x7f49180c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34900/0 (socket says 172.21.15.46:34900) 2023-10-06T11:36:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 -- 172.21.15.46:0/2807885794 learned_addr learned my addr 172.21.15.46:0/2807885794 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491dee2700 1 --2- 172.21.15.46:0/2807885794 >> v2:172.21.15.132:3300/0 conn(0x7f491806cc30 0x7f491806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491d6e1700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4918161470 0x7f4918161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 -- 172.21.15.46:0/2807885794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4918161470 msgr2=0x7f4918161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4918161470 0x7f4918161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 -- 172.21.15.46:0/2807885794 >> v2:172.21.15.132:3300/0 conn(0x7f491806cc30 msgr2=0x7f491806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 --2- 172.21.15.46:0/2807885794 >> v2:172.21.15.132:3300/0 conn(0x7f491806cc30 0x7f491806d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.761+0000 7f491cee0700 1 -- 172.21.15.46:0/2807885794 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180c12a0 con 0x7f49180c1d80 2023-10-06T11:36:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491cee0700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49180c1d80 0x7f49180c21b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f49180c0bd0 tx=0x7f4908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4177e934c900906d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f490ffff700 1 -- 172.21.15.46:0/2807885794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490800b6d0 con 0x7f49180c1d80 2023-10-06T11:36:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f490ffff700 1 -- 172.21.15.46:0/2807885794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490800b830 con 0x7f49180c1d80 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f490ffff700 1 -- 172.21.15.46:0/2807885794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490800bbf0 con 0x7f49180c1d80 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 -- 172.21.15.46:0/2807885794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49180c1d80 msgr2=0x7f49180c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49180c1d80 0x7f49180c21b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f49180c0bd0 tx=0x7f4908009250 comp rx=0 tx=0).stop 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 -- 172.21.15.46:0/2807885794 shutdown_connections 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 --2- 172.21.15.46:0/2807885794 >> v2:172.21.15.132:3300/0 conn(0x7f491806cc30 0x7f491806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49180c1d80 0x7f49180c21b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 --2- 172.21.15.46:0/2807885794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4918161470 0x7f4918161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 -- 172.21.15.46:0/2807885794 >> 172.21.15.46:0/2807885794 conn(0x7f49180cbe30 msgr2=0x7f49180cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 -- 172.21.15.46:0/2807885794 shutdown_connections 2023-10-06T11:36:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.765+0000 7f491f945700 1 -- 172.21.15.46:0/2807885794 wait complete. 2023-10-06T11:36:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 Processor -- start 2023-10-06T11:36:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 -- start start 2023-10-06T11:36:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49180c1d80 0x7f491815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4918161470 0x7f4918151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491815b7d0 con 0x7f491806cc30 2023-10-06T11:36:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c3cb0 con 0x7f4918161470 2023-10-06T11:36:13.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491f945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c3f90 con 0x7f49180c1d80 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34902/0 (socket says 172.21.15.46:34902) 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 -- 172.21.15.46:0/1708131541 learned_addr learned my addr 172.21.15.46:0/1708131541 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491cee0700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49180c1d80 0x7f491815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491dee2700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4918161470 0x7f4918151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 -- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49180c1d80 msgr2=0x7f491815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49180c1d80 0x7f491815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 -- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4918161470 msgr2=0x7f4918151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4918161470 0x7f4918151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 -- 172.21.15.46:0/1708131541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49080094f0 con 0x7f491806cc30 2023-10-06T11:36:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.769+0000 7f491d6e1700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f491400ea20 tx=0x7f4914009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400c850 con 0x7f491806cc30 2023-10-06T11:36:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491400c9b0 con 0x7f491806cc30 2023-10-06T11:36:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400cd70 con 0x7f491806cc30 2023-10-06T11:36:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491815f6c0 con 0x7f491806cc30 2023-10-06T11:36:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491815dce0 con 0x7f491806cc30 2023-10-06T11:36:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4918063f90 con 0x7f491806cc30 2023-10-06T11:36:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.773+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4914013680 con 0x7f491806cc30 2023-10-06T11:36:13.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.777+0000 7f490d7fa700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 0x7f48fc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:13.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.777+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f491407d2d0 con 0x7f491806cc30 2023-10-06T11:36:13.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.777+0000 7f491cee0700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 0x7f48fc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:13.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.777+0000 7f491cee0700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 0x7f48fc062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49180c0bd0 tx=0x7f4908017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:13.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.781+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f491404d3f0 con 0x7f491806cc30 2023-10-06T11:36:13.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.949+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49180c1580 con 0x7f48fc0606a0 2023-10-06T11:36:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.965+0000 7f490d7fa700 1 -- 172.21.15.46:0/1708131541 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309227 (secure 0 0 0) 0x7f49180c1580 con 0x7f48fc0606a0 2023-10-06T11:36:13.984 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:13.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 msgr2=0x7f48fc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 0x7f48fc062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f49180c0bd0 tx=0x7f4908017000 comp rx=0 tx=0).stop 2023-10-06T11:36:13.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 msgr2=0x7f49180c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:13.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f491400ea20 tx=0x7f4914009120 comp rx=0 tx=0).stop 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 shutdown_connections 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f48fc0606a0 0x7f48fc062b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4918161470 0x7f4918151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49180c1d80 0x7f491815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 --2- 172.21.15.46:0/1708131541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491806cc30 0x7f49180c4310 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 >> 172.21.15.46:0/1708131541 conn(0x7f49180cbe30 msgr2=0x7f491815c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 shutdown_connections 2023-10-06T11:36:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:13.973+0000 7f491f945700 1 -- 172.21.15.46:0/1708131541 wait complete. 2023-10-06T11:36:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:36:14.007 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T11:36:13.525973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460936,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752513784,"statfs":{"total":773094113280,"available":770574114816,"internally_reserved":0,"allocated":13312000,"data_stored":7432468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002280"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839807+0000","last_change":"2023-10-06T11:36:08.834959+0000","last_active":"2023-10-06T11:36:08.839807+0000","last_peered":"2023-10-06T11:36:08.839807+0000","last_clean":"2023-10-06T11:36:08.839807+0000","last_became_active":"2023-10-06T11:36:08.834673+0000","last_became_peered":"2023-10-06T11:36:08.834673+0000","last_unstale":"2023-10-06T11:36:08.839807+0000","last_undegraded":"2023-10-06T11:36:08.839807+0000","last_fullsized":"2023-10-06T11:36:08.839807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:45.178435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843699+0000","last_change":"2023-10-06T11:36:08.839427+0000","last_active":"2023-10-06T11:36:08.843699+0000","last_peered":"2023-10-06T11:36:08.843699+0000","last_clean":"2023-10-06T11:36:08.843699+0000","last_became_active":"2023-10-06T11:36:08.839074+0000","last_became_peered":"2023-10-06T11:36:08.839074+0000","last_unstale":"2023-10-06T11:36:08.843699+0000","last_undegraded":"2023-10-06T11:36:08.843699+0000","last_fullsized":"2023-10-06T11:36:08.843699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:44.389800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.826608+0000","last_change":"2023-10-06T11:36:08.822154+0000","last_active":"2023-10-06T11:36:08.826608+0000","last_peered":"2023-10-06T11:36:08.826608+0000","last_clean":"2023-10-06T11:36:08.826608+0000","last_became_active":"2023-10-06T11:36:08.821837+0000","last_became_peered":"2023-10-06T11:36:08.821837+0000","last_unstale":"2023-10-06T11:36:08.826608+0000","last_undegraded":"2023-10-06T11:36:08.826608+0000","last_fullsized":"2023-10-06T11:36:08.826608+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.339584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841877+0000","last_change":"2023-10-06T11:36:08.835584+0000","last_active":"2023-10-06T11:36:08.841877+0000","last_peered":"2023-10-06T11:36:08.841877+0000","last_clean":"2023-10-06T11:36:08.841877+0000","last_became_active":"2023-10-06T11:36:08.835284+0000","last_became_peered":"2023-10-06T11:36:08.835284+0000","last_unstale":"2023-10-06T11:36:08.841877+0000","last_undegraded":"2023-10-06T11:36:08.841877+0000","last_fullsized":"2023-10-06T11:36:08.841877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842628+0000","last_change":"2023-10-06T11:36:08.837618+0000","last_active":"2023-10-06T11:36:08.842628+0000","last_peered":"2023-10-06T11:36:08.842628+0000","last_clean":"2023-10-06T11:36:08.842628+0000","last_became_active":"2023-10-06T11:36:08.837297+0000","last_became_peered":"2023-10-06T11:36:08.837297+0000","last_unstale":"2023-10-06T11:36:08.842628+0000","last_undegraded":"2023-10-06T11:36:08.842628+0000","last_fullsized":"2023-10-06T11:36:08.842628+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850512+0000","last_change":"2023-10-06T11:36:08.847872+0000","last_active":"2023-10-06T11:36:08.850512+0000","last_peered":"2023-10-06T11:36:08.850512+0000","last_clean":"2023-10-06T11:36:08.850512+0000","last_became_active":"2023-10-06T11:36:08.847547+0000","last_became_peered":"2023-10-06T11:36:08.847547+0000","last_unstale":"2023-10-06T11:36:08.850512+0000","last_undegraded":"2023-10-06T11:36:08.850512+0000","last_fullsized":"2023-10-06T11:36:08.850512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:43.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849958+0000","last_change":"2023-10-06T11:36:08.846412+0000","last_active":"2023-10-06T11:36:08.849958+0000","last_peered":"2023-10-06T11:36:08.849958+0000","last_clean":"2023-10-06T11:36:08.849958+0000","last_became_active":"2023-10-06T11:36:08.843620+0000","last_became_peered":"2023-10-06T11:36:08.843620+0000","last_unstale":"2023-10-06T11:36:08.849958+0000","last_undegraded":"2023-10-06T11:36:08.849958+0000","last_fullsized":"2023-10-06T11:36:08.849958+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:23.081039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850850+0000","last_change":"2023-10-06T11:36:08.846553+0000","last_active":"2023-10-06T11:36:08.850850+0000","last_peered":"2023-10-06T11:36:08.850850+0000","last_clean":"2023-10-06T11:36:08.850850+0000","last_became_active":"2023-10-06T11:36:08.846228+0000","last_became_peered":"2023-10-06T11:36:08.846228+0000","last_unstale":"2023-10-06T11:36:08.850850+0000","last_undegraded":"2023-10-06T11:36:08.850850+0000","last_fullsized":"2023-10-06T11:36:08.850850+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849890+0000","last_change":"2023-10-06T11:36:08.846081+0000","last_active":"2023-10-06T11:36:08.849890+0000","last_peered":"2023-10-06T11:36:08.849890+0000","last_clean":"2023-10-06T11:36:08.849890+0000","last_became_active":"2023-10-06T11:36:08.844160+0000","last_became_peered":"2023-10-06T11:36:08.844160+0000","last_unstale":"2023-10-06T11:36:08.849890+0000","last_undegraded":"2023-10-06T11:36:08.849890+0000","last_fullsized":"2023-10-06T11:36:08.849890+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:32.586598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835394+0000","last_change":"2023-10-06T11:36:08.832132+0000","last_active":"2023-10-06T11:36:08.835394+0000","last_peered":"2023-10-06T11:36:08.835394+0000","last_clean":"2023-10-06T11:36:08.835394+0000","last_became_active":"2023-10-06T11:36:08.831813+0000","last_became_peered":"2023-10-06T11:36:08.831813+0000","last_unstale":"2023-10-06T11:36:08.835394+0000","last_undegraded":"2023-10-06T11:36:08.835394+0000","last_fullsized":"2023-10-06T11:36:08.835394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847126+0000","last_change":"2023-10-06T11:36:08.839463+0000","last_active":"2023-10-06T11:36:08.847126+0000","last_peered":"2023-10-06T11:36:08.847126+0000","last_clean":"2023-10-06T11:36:08.847126+0000","last_became_active":"2023-10-06T11:36:08.839166+0000","last_became_peered":"2023-10-06T11:36:08.839166+0000","last_unstale":"2023-10-06T11:36:08.847126+0000","last_undegraded":"2023-10-06T11:36:08.847126+0000","last_fullsized":"2023-10-06T11:36:08.847126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:41.417191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848083+0000","last_change":"2023-10-06T11:36:08.845554+0000","last_active":"2023-10-06T11:36:08.848083+0000","last_peered":"2023-10-06T11:36:08.848083+0000","last_clean":"2023-10-06T11:36:08.848083+0000","last_became_active":"2023-10-06T11:36:08.844765+0000","last_became_peered":"2023-10-06T11:36:08.844765+0000","last_unstale":"2023-10-06T11:36:08.848083+0000","last_undegraded":"2023-10-06T11:36:08.848083+0000","last_fullsized":"2023-10-06T11:36:08.848083+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:56.739651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834928+0000","last_change":"2023-10-06T11:36:08.829614+0000","last_active":"2023-10-06T11:36:08.834928+0000","last_peered":"2023-10-06T11:36:08.834928+0000","last_clean":"2023-10-06T11:36:08.834928+0000","last_became_active":"2023-10-06T11:36:08.829269+0000","last_became_peered":"2023-10-06T11:36:08.829269+0000","last_unstale":"2023-10-06T11:36:08.834928+0000","last_undegraded":"2023-10-06T11:36:08.834928+0000","last_fullsized":"2023-10-06T11:36:08.834928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847934+0000","last_change":"2023-10-06T11:36:08.845000+0000","last_active":"2023-10-06T11:36:08.847934+0000","last_peered":"2023-10-06T11:36:08.847934+0000","last_clean":"2023-10-06T11:36:08.847934+0000","last_became_active":"2023-10-06T11:36:08.844706+0000","last_became_peered":"2023-10-06T11:36:08.844706+0000","last_unstale":"2023-10-06T11:36:08.847934+0000","last_undegraded":"2023-10-06T11:36:08.847934+0000","last_fullsized":"2023-10-06T11:36:08.847934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849167+0000","last_change":"2023-10-06T11:36:08.847706+0000","last_active":"2023-10-06T11:36:08.849167+0000","last_peered":"2023-10-06T11:36:08.849167+0000","last_clean":"2023-10-06T11:36:08.849167+0000","last_became_active":"2023-10-06T11:36:08.847386+0000","last_became_peered":"2023-10-06T11:36:08.847386+0000","last_unstale":"2023-10-06T11:36:08.849167+0000","last_undegraded":"2023-10-06T11:36:08.849167+0000","last_fullsized":"2023-10-06T11:36:08.849167+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.844371+0000","last_change":"2023-10-06T11:36:08.840984+0000","last_active":"2023-10-06T11:36:08.844371+0000","last_peered":"2023-10-06T11:36:08.844371+0000","last_clean":"2023-10-06T11:36:08.844371+0000","last_became_active":"2023-10-06T11:36:08.840669+0000","last_became_peered":"2023-10-06T11:36:08.840669+0000","last_unstale":"2023-10-06T11:36:08.844371+0000","last_undegraded":"2023-10-06T11:36:08.844371+0000","last_fullsized":"2023-10-06T11:36:08.844371+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.821913+0000","last_change":"2023-10-06T11:36:08.820676+0000","last_active":"2023-10-06T11:36:08.821913+0000","last_peered":"2023-10-06T11:36:08.821913+0000","last_clean":"2023-10-06T11:36:08.821913+0000","last_became_active":"2023-10-06T11:36:08.820319+0000","last_became_peered":"2023-10-06T11:36:08.820319+0000","last_unstale":"2023-10-06T11:36:08.821913+0000","last_undegraded":"2023-10-06T11:36:08.821913+0000","last_fullsized":"2023-10-06T11:36:08.821913+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:53:05.714850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850475+0000","last_change":"2023-10-06T11:36:08.847440+0000","last_active":"2023-10-06T11:36:08.850475+0000","last_peered":"2023-10-06T11:36:08.850475+0000","last_clean":"2023-10-06T11:36:08.850475+0000","last_became_active":"2023-10-06T11:36:08.844060+0000","last_became_peered":"2023-10-06T11:36:08.844060+0000","last_unstale":"2023-10-06T11:36:08.850475+0000","last_undegraded":"2023-10-06T11:36:08.850475+0000","last_fullsized":"2023-10-06T11:36:08.850475+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:10.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835909+0000","last_change":"2023-10-06T11:36:08.833419+0000","last_active":"2023-10-06T11:36:08.835909+0000","last_peered":"2023-10-06T11:36:08.835909+0000","last_clean":"2023-10-06T11:36:08.835909+0000","last_became_active":"2023-10-06T11:36:08.832970+0000","last_became_peered":"2023-10-06T11:36:08.832970+0000","last_unstale":"2023-10-06T11:36:08.835909+0000","last_undegraded":"2023-10-06T11:36:08.835909+0000","last_fullsized":"2023-10-06T11:36:08.835909+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:28:24.073017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843554+0000","last_change":"2023-10-06T11:36:08.839073+0000","last_active":"2023-10-06T11:36:08.843554+0000","last_peered":"2023-10-06T11:36:08.843554+0000","last_clean":"2023-10-06T11:36:08.843554+0000","last_became_active":"2023-10-06T11:36:08.838729+0000","last_became_peered":"2023-10-06T11:36:08.838729+0000","last_unstale":"2023-10-06T11:36:08.843554+0000","last_undegraded":"2023-10-06T11:36:08.843554+0000","last_fullsized":"2023-10-06T11:36:08.843554+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:43.453476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847734+0000","last_change":"2023-10-06T11:36:08.834706+0000","last_active":"2023-10-06T11:36:08.847734+0000","last_peered":"2023-10-06T11:36:08.847734+0000","last_clean":"2023-10-06T11:36:08.847734+0000","last_became_active":"2023-10-06T11:36:08.834392+0000","last_became_peered":"2023-10-06T11:36:08.834392+0000","last_unstale":"2023-10-06T11:36:08.847734+0000","last_undegraded":"2023-10-06T11:36:08.847734+0000","last_fullsized":"2023-10-06T11:36:08.847734+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835267+0000","last_change":"2023-10-06T11:36:08.827263+0000","last_active":"2023-10-06T11:36:08.835267+0000","last_peered":"2023-10-06T11:36:08.835267+0000","last_clean":"2023-10-06T11:36:08.835267+0000","last_became_active":"2023-10-06T11:36:08.826927+0000","last_became_peered":"2023-10-06T11:36:08.826927+0000","last_unstale":"2023-10-06T11:36:08.835267+0000","last_undegraded":"2023-10-06T11:36:08.835267+0000","last_fullsized":"2023-10-06T11:36:08.835267+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:09.477110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832710+0000","last_change":"2023-10-06T11:36:08.825112+0000","last_active":"2023-10-06T11:36:08.832710+0000","last_peered":"2023-10-06T11:36:08.832710+0000","last_clean":"2023-10-06T11:36:08.832710+0000","last_became_active":"2023-10-06T11:36:08.824817+0000","last_became_peered":"2023-10-06T11:36:08.824817+0000","last_unstale":"2023-10-06T11:36:08.832710+0000","last_undegraded":"2023-10-06T11:36:08.832710+0000","last_fullsized":"2023-10-06T11:36:08.832710+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:47.661809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840092+0000","last_change":"2023-10-06T11:36:08.833482+0000","last_active":"2023-10-06T11:36:08.840092+0000","last_peered":"2023-10-06T11:36:08.840092+0000","last_clean":"2023-10-06T11:36:08.840092+0000","last_became_active":"2023-10-06T11:36:08.833172+0000","last_became_peered":"2023-10-06T11:36:08.833172+0000","last_unstale":"2023-10-06T11:36:08.840092+0000","last_undegraded":"2023-10-06T11:36:08.840092+0000","last_fullsized":"2023-10-06T11:36:08.840092+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837751+0000","last_change":"2023-10-06T11:36:08.834331+0000","last_active":"2023-10-06T11:36:08.837751+0000","last_peered":"2023-10-06T11:36:08.837751+0000","last_clean":"2023-10-06T11:36:08.837751+0000","last_became_active":"2023-10-06T11:36:08.834010+0000","last_became_peered":"2023-10-06T11:36:08.834010+0000","last_unstale":"2023-10-06T11:36:08.837751+0000","last_undegraded":"2023-10-06T11:36:08.837751+0000","last_fullsized":"2023-10-06T11:36:08.837751+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:37.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843481+0000","last_change":"2023-10-06T11:36:08.839346+0000","last_active":"2023-10-06T11:36:08.843481+0000","last_peered":"2023-10-06T11:36:08.843481+0000","last_clean":"2023-10-06T11:36:08.843481+0000","last_became_active":"2023-10-06T11:36:08.839069+0000","last_became_peered":"2023-10-06T11:36:08.839069+0000","last_unstale":"2023-10-06T11:36:08.843481+0000","last_undegraded":"2023-10-06T11:36:08.843481+0000","last_fullsized":"2023-10-06T11:36:08.843481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:43.156987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847105+0000","last_change":"2023-10-06T11:36:08.838625+0000","last_active":"2023-10-06T11:36:08.847105+0000","last_peered":"2023-10-06T11:36:08.847105+0000","last_clean":"2023-10-06T11:36:08.847105+0000","last_became_active":"2023-10-06T11:36:08.838342+0000","last_became_peered":"2023-10-06T11:36:08.838342+0000","last_unstale":"2023-10-06T11:36:08.847105+0000","last_undegraded":"2023-10-06T11:36:08.847105+0000","last_fullsized":"2023-10-06T11:36:08.847105+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849748+0000","last_change":"2023-10-06T11:36:08.843721+0000","last_active":"2023-10-06T11:36:08.849748+0000","last_peered":"2023-10-06T11:36:08.849748+0000","last_clean":"2023-10-06T11:36:08.849748+0000","last_became_active":"2023-10-06T11:36:08.843407+0000","last_became_peered":"2023-10-06T11:36:08.843407+0000","last_unstale":"2023-10-06T11:36:08.849748+0000","last_undegraded":"2023-10-06T11:36:08.849748+0000","last_fullsized":"2023-10-06T11:36:08.849748+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847711+0000","last_change":"2023-10-06T11:36:08.830867+0000","last_active":"2023-10-06T11:36:08.847711+0000","last_peered":"2023-10-06T11:36:08.847711+0000","last_clean":"2023-10-06T11:36:08.847711+0000","last_became_active":"2023-10-06T11:36:08.830226+0000","last_became_peered":"2023-10-06T11:36:08.830226+0000","last_unstale":"2023-10-06T11:36:08.847711+0000","last_undegraded":"2023-10-06T11:36:08.847711+0000","last_fullsized":"2023-10-06T11:36:08.847711+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835740+0000","last_change":"2023-10-06T11:36:08.833495+0000","last_active":"2023-10-06T11:36:08.835740+0000","last_peered":"2023-10-06T11:36:08.835740+0000","last_clean":"2023-10-06T11:36:08.835740+0000","last_became_active":"2023-10-06T11:36:08.833119+0000","last_became_peered":"2023-10-06T11:36:08.833119+0000","last_unstale":"2023-10-06T11:36:08.835740+0000","last_undegraded":"2023-10-06T11:36:08.835740+0000","last_fullsized":"2023-10-06T11:36:08.835740+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:49.438750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849754+0000","last_change":"2023-10-06T11:36:08.843345+0000","last_active":"2023-10-06T11:36:08.849754+0000","last_peered":"2023-10-06T11:36:08.849754+0000","last_clean":"2023-10-06T11:36:08.849754+0000","last_became_active":"2023-10-06T11:36:08.842930+0000","last_became_peered":"2023-10-06T11:36:08.842930+0000","last_unstale":"2023-10-06T11:36:08.849754+0000","last_undegraded":"2023-10-06T11:36:08.849754+0000","last_fullsized":"2023-10-06T11:36:08.849754+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:25.950732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827000+0000","last_change":"2023-10-06T11:36:08.825532+0000","last_active":"2023-10-06T11:36:08.827000+0000","last_peered":"2023-10-06T11:36:08.827000+0000","last_clean":"2023-10-06T11:36:08.827000+0000","last_became_active":"2023-10-06T11:36:08.825246+0000","last_became_peered":"2023-10-06T11:36:08.825246+0000","last_unstale":"2023-10-06T11:36:08.827000+0000","last_undegraded":"2023-10-06T11:36:08.827000+0000","last_fullsized":"2023-10-06T11:36:08.827000+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837616+0000","last_change":"2023-10-06T11:36:08.832215+0000","last_active":"2023-10-06T11:36:08.837616+0000","last_peered":"2023-10-06T11:36:08.837616+0000","last_clean":"2023-10-06T11:36:08.837616+0000","last_became_active":"2023-10-06T11:36:08.831906+0000","last_became_peered":"2023-10-06T11:36:08.831906+0000","last_unstale":"2023-10-06T11:36:08.837616+0000","last_undegraded":"2023-10-06T11:36:08.837616+0000","last_fullsized":"2023-10-06T11:36:08.837616+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:21.848572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"121'4","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.854694+0000","last_change":"2023-10-06T11:36:07.836398+0000","last_active":"2023-10-06T11:36:08.854694+0000","last_peered":"2023-10-06T11:36:08.854694+0000","last_clean":"2023-10-06T11:36:08.854694+0000","last_became_active":"2023-10-06T11:36:07.836063+0000","last_became_peered":"2023-10-06T11:36:07.836063+0000","last_unstale":"2023-10-06T11:36:08.854694+0000","last_undegraded":"2023-10-06T11:36:08.854694+0000","last_fullsized":"2023-10-06T11:36:08.854694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850887+0000","last_change":"2023-10-06T11:36:08.845254+0000","last_active":"2023-10-06T11:36:08.850887+0000","last_peered":"2023-10-06T11:36:08.850887+0000","last_clean":"2023-10-06T11:36:08.850887+0000","last_became_active":"2023-10-06T11:36:08.844729+0000","last_became_peered":"2023-10-06T11:36:08.844729+0000","last_unstale":"2023-10-06T11:36:08.850887+0000","last_undegraded":"2023-10-06T11:36:08.850887+0000","last_fullsized":"2023-10-06T11:36:08.850887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:46.014384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857089+0000","last_change":"2023-10-06T11:36:07.830544+0000","last_active":"2023-10-06T11:36:08.857089+0000","last_peered":"2023-10-06T11:36:08.857089+0000","last_clean":"2023-10-06T11:36:08.857089+0000","last_became_active":"2023-10-06T11:36:07.830229+0000","last_became_peered":"2023-10-06T11:36:07.830229+0000","last_unstale":"2023-10-06T11:36:08.857089+0000","last_undegraded":"2023-10-06T11:36:08.857089+0000","last_fullsized":"2023-10-06T11:36:08.857089+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842017+0000","last_change":"2023-10-06T11:36:08.837123+0000","last_active":"2023-10-06T11:36:08.842017+0000","last_peered":"2023-10-06T11:36:08.842017+0000","last_clean":"2023-10-06T11:36:08.842017+0000","last_became_active":"2023-10-06T11:36:08.836794+0000","last_became_peered":"2023-10-06T11:36:08.836794+0000","last_unstale":"2023-10-06T11:36:08.842017+0000","last_undegraded":"2023-10-06T11:36:08.842017+0000","last_fullsized":"2023-10-06T11:36:08.842017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:34.957119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864720+0000","last_change":"2023-10-06T11:36:07.843540+0000","last_active":"2023-10-06T11:36:08.864720+0000","last_peered":"2023-10-06T11:36:08.864720+0000","last_clean":"2023-10-06T11:36:08.864720+0000","last_became_active":"2023-10-06T11:36:07.843215+0000","last_became_peered":"2023-10-06T11:36:07.843215+0000","last_unstale":"2023-10-06T11:36:08.864720+0000","last_undegraded":"2023-10-06T11:36:08.864720+0000","last_fullsized":"2023-10-06T11:36:08.864720+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:22.066116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842320+0000","last_change":"2023-10-06T11:36:08.828471+0000","last_active":"2023-10-06T11:36:08.842320+0000","last_peered":"2023-10-06T11:36:08.842320+0000","last_clean":"2023-10-06T11:36:08.842320+0000","last_became_active":"2023-10-06T11:36:08.828169+0000","last_became_peered":"2023-10-06T11:36:08.828169+0000","last_unstale":"2023-10-06T11:36:08.842320+0000","last_undegraded":"2023-10-06T11:36:08.842320+0000","last_fullsized":"2023-10-06T11:36:08.842320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:32.232594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.855976+0000","last_change":"2023-10-06T11:36:07.834816+0000","last_active":"2023-10-06T11:36:08.855976+0000","last_peered":"2023-10-06T11:36:08.855976+0000","last_clean":"2023-10-06T11:36:08.855976+0000","last_became_active":"2023-10-06T11:36:07.834503+0000","last_became_peered":"2023-10-06T11:36:07.834503+0000","last_unstale":"2023-10-06T11:36:08.855976+0000","last_undegraded":"2023-10-06T11:36:08.855976+0000","last_fullsized":"2023-10-06T11:36:08.855976+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841854+0000","last_change":"2023-10-06T11:36:08.834817+0000","last_active":"2023-10-06T11:36:08.841854+0000","last_peered":"2023-10-06T11:36:08.841854+0000","last_clean":"2023-10-06T11:36:08.841854+0000","last_became_active":"2023-10-06T11:36:08.834510+0000","last_became_peered":"2023-10-06T11:36:08.834510+0000","last_unstale":"2023-10-06T11:36:08.841854+0000","last_undegraded":"2023-10-06T11:36:08.841854+0000","last_fullsized":"2023-10-06T11:36:08.841854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861794+0000","last_change":"2023-10-06T11:36:07.842608+0000","last_active":"2023-10-06T11:36:08.861794+0000","last_peered":"2023-10-06T11:36:08.861794+0000","last_clean":"2023-10-06T11:36:08.861794+0000","last_became_active":"2023-10-06T11:36:07.842325+0000","last_became_peered":"2023-10-06T11:36:07.842325+0000","last_unstale":"2023-10-06T11:36:08.861794+0000","last_undegraded":"2023-10-06T11:36:08.861794+0000","last_fullsized":"2023-10-06T11:36:08.861794+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835929+0000","last_change":"2023-10-06T11:36:08.833625+0000","last_active":"2023-10-06T11:36:08.835929+0000","last_peered":"2023-10-06T11:36:08.835929+0000","last_clean":"2023-10-06T11:36:08.835929+0000","last_became_active":"2023-10-06T11:36:08.833132+0000","last_became_peered":"2023-10-06T11:36:08.833132+0000","last_unstale":"2023-10-06T11:36:08.835929+0000","last_undegraded":"2023-10-06T11:36:08.835929+0000","last_fullsized":"2023-10-06T11:36:08.835929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.375194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861213+0000","last_change":"2023-10-06T11:36:07.836739+0000","last_active":"2023-10-06T11:36:08.861213+0000","last_peered":"2023-10-06T11:36:08.861213+0000","last_clean":"2023-10-06T11:36:08.861213+0000","last_became_active":"2023-10-06T11:36:07.836402+0000","last_became_peered":"2023-10-06T11:36:07.836402+0000","last_unstale":"2023-10-06T11:36:08.861213+0000","last_undegraded":"2023-10-06T11:36:08.861213+0000","last_fullsized":"2023-10-06T11:36:08.861213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:13.703496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849866+0000","last_change":"2023-10-06T11:36:08.845779+0000","last_active":"2023-10-06T11:36:08.849866+0000","last_peered":"2023-10-06T11:36:08.849866+0000","last_clean":"2023-10-06T11:36:08.849866+0000","last_became_active":"2023-10-06T11:36:08.843903+0000","last_became_peered":"2023-10-06T11:36:08.843903+0000","last_unstale":"2023-10-06T11:36:08.849866+0000","last_undegraded":"2023-10-06T11:36:08.849866+0000","last_fullsized":"2023-10-06T11:36:08.849866+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:49.631159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824932+0000","last_change":"2023-10-06T11:36:07.842155+0000","last_active":"2023-10-06T11:36:08.824932+0000","last_peered":"2023-10-06T11:36:08.824932+0000","last_clean":"2023-10-06T11:36:08.824932+0000","last_became_active":"2023-10-06T11:36:07.841850+0000","last_became_peered":"2023-10-06T11:36:07.841850+0000","last_unstale":"2023-10-06T11:36:08.824932+0000","last_undegraded":"2023-10-06T11:36:08.824932+0000","last_fullsized":"2023-10-06T11:36:08.824932+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:09.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833946+0000","last_change":"2023-10-06T11:36:08.822864+0000","last_active":"2023-10-06T11:36:08.833946+0000","last_peered":"2023-10-06T11:36:08.833946+0000","last_clean":"2023-10-06T11:36:08.833946+0000","last_became_active":"2023-10-06T11:36:08.822567+0000","last_became_peered":"2023-10-06T11:36:08.822567+0000","last_unstale":"2023-10-06T11:36:08.833946+0000","last_undegraded":"2023-10-06T11:36:08.833946+0000","last_fullsized":"2023-10-06T11:36:08.833946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827606+0000","last_change":"2023-10-06T11:36:07.838208+0000","last_active":"2023-10-06T11:36:08.827606+0000","last_peered":"2023-10-06T11:36:08.827606+0000","last_clean":"2023-10-06T11:36:08.827606+0000","last_became_active":"2023-10-06T11:36:07.837843+0000","last_became_peered":"2023-10-06T11:36:07.837843+0000","last_unstale":"2023-10-06T11:36:08.827606+0000","last_undegraded":"2023-10-06T11:36:08.827606+0000","last_fullsized":"2023-10-06T11:36:08.827606+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837595+0000","last_change":"2023-10-06T11:36:08.831541+0000","last_active":"2023-10-06T11:36:08.837595+0000","last_peered":"2023-10-06T11:36:08.837595+0000","last_clean":"2023-10-06T11:36:08.837595+0000","last_became_active":"2023-10-06T11:36:08.831234+0000","last_became_peered":"2023-10-06T11:36:08.831234+0000","last_unstale":"2023-10-06T11:36:08.837595+0000","last_undegraded":"2023-10-06T11:36:08.837595+0000","last_fullsized":"2023-10-06T11:36:08.837595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:26.203324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825150+0000","last_change":"2023-10-06T11:36:07.844770+0000","last_active":"2023-10-06T11:36:08.825150+0000","last_peered":"2023-10-06T11:36:08.825150+0000","last_clean":"2023-10-06T11:36:08.825150+0000","last_became_active":"2023-10-06T11:36:07.843489+0000","last_became_peered":"2023-10-06T11:36:07.843489+0000","last_unstale":"2023-10-06T11:36:08.825150+0000","last_undegraded":"2023-10-06T11:36:08.825150+0000","last_fullsized":"2023-10-06T11:36:08.825150+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:55.205873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835302+0000","last_change":"2023-10-06T11:36:08.828494+0000","last_active":"2023-10-06T11:36:08.835302+0000","last_peered":"2023-10-06T11:36:08.835302+0000","last_clean":"2023-10-06T11:36:08.835302+0000","last_became_active":"2023-10-06T11:36:08.828149+0000","last_became_peered":"2023-10-06T11:36:08.828149+0000","last_unstale":"2023-10-06T11:36:08.835302+0000","last_undegraded":"2023-10-06T11:36:08.835302+0000","last_fullsized":"2023-10-06T11:36:08.835302+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:25.498441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810127+0000","last_change":"2023-10-06T11:36:07.831652+0000","last_active":"2023-10-06T11:36:08.810127+0000","last_peered":"2023-10-06T11:36:08.810127+0000","last_clean":"2023-10-06T11:36:08.810127+0000","last_became_active":"2023-10-06T11:36:07.831331+0000","last_became_peered":"2023-10-06T11:36:07.831331+0000","last_unstale":"2023-10-06T11:36:08.810127+0000","last_undegraded":"2023-10-06T11:36:08.810127+0000","last_fullsized":"2023-10-06T11:36:08.810127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:30:03.936003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850794+0000","last_change":"2023-10-06T11:36:08.843325+0000","last_active":"2023-10-06T11:36:08.850794+0000","last_peered":"2023-10-06T11:36:08.850794+0000","last_clean":"2023-10-06T11:36:08.850794+0000","last_became_active":"2023-10-06T11:36:08.843032+0000","last_became_peered":"2023-10-06T11:36:08.843032+0000","last_unstale":"2023-10-06T11:36:08.850794+0000","last_undegraded":"2023-10-06T11:36:08.850794+0000","last_fullsized":"2023-10-06T11:36:08.850794+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:50:13.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"121'3","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861425+0000","last_change":"2023-10-06T11:36:07.845696+0000","last_active":"2023-10-06T11:36:08.861425+0000","last_peered":"2023-10-06T11:36:08.861425+0000","last_clean":"2023-10-06T11:36:08.861425+0000","last_became_active":"2023-10-06T11:36:07.845420+0000","last_became_peered":"2023-10-06T11:36:07.845420+0000","last_unstale":"2023-10-06T11:36:08.861425+0000","last_undegraded":"2023-10-06T11:36:08.861425+0000","last_fullsized":"2023-10-06T11:36:08.861425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837117+0000","last_change":"2023-10-06T11:36:08.827973+0000","last_active":"2023-10-06T11:36:08.837117+0000","last_peered":"2023-10-06T11:36:08.837117+0000","last_clean":"2023-10-06T11:36:08.837117+0000","last_became_active":"2023-10-06T11:36:08.827650+0000","last_became_peered":"2023-10-06T11:36:08.827650+0000","last_unstale":"2023-10-06T11:36:08.837117+0000","last_undegraded":"2023-10-06T11:36:08.837117+0000","last_fullsized":"2023-10-06T11:36:08.837117+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:48.124515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"121'6","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.865336+0000","last_change":"2023-10-06T11:36:07.845721+0000","last_active":"2023-10-06T11:36:08.865336+0000","last_peered":"2023-10-06T11:36:08.865336+0000","last_clean":"2023-10-06T11:36:08.865336+0000","last_became_active":"2023-10-06T11:36:07.845389+0000","last_became_peered":"2023-10-06T11:36:07.845389+0000","last_unstale":"2023-10-06T11:36:08.865336+0000","last_undegraded":"2023-10-06T11:36:08.865336+0000","last_fullsized":"2023-10-06T11:36:08.865336+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842565+0000","last_change":"2023-10-06T11:36:08.831561+0000","last_active":"2023-10-06T11:36:08.842565+0000","last_peered":"2023-10-06T11:36:08.842565+0000","last_clean":"2023-10-06T11:36:08.842565+0000","last_became_active":"2023-10-06T11:36:08.831286+0000","last_became_peered":"2023-10-06T11:36:08.831286+0000","last_unstale":"2023-10-06T11:36:08.842565+0000","last_undegraded":"2023-10-06T11:36:08.842565+0000","last_fullsized":"2023-10-06T11:36:08.842565+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857133+0000","last_change":"2023-10-06T11:36:07.836509+0000","last_active":"2023-10-06T11:36:08.857133+0000","last_peered":"2023-10-06T11:36:08.857133+0000","last_clean":"2023-10-06T11:36:08.857133+0000","last_became_active":"2023-10-06T11:36:07.835626+0000","last_became_peered":"2023-10-06T11:36:07.835626+0000","last_unstale":"2023-10-06T11:36:08.857133+0000","last_undegraded":"2023-10-06T11:36:08.857133+0000","last_fullsized":"2023-10-06T11:36:08.857133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849972+0000","last_change":"2023-10-06T11:36:08.846593+0000","last_active":"2023-10-06T11:36:08.849972+0000","last_peered":"2023-10-06T11:36:08.849972+0000","last_clean":"2023-10-06T11:36:08.849972+0000","last_became_active":"2023-10-06T11:36:08.846204+0000","last_became_peered":"2023-10-06T11:36:08.846204+0000","last_unstale":"2023-10-06T11:36:08.849972+0000","last_undegraded":"2023-10-06T11:36:08.849972+0000","last_fullsized":"2023-10-06T11:36:08.849972+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:46.893794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.830870+0000","last_change":"2023-10-06T11:36:07.833180+0000","last_active":"2023-10-06T11:36:08.830870+0000","last_peered":"2023-10-06T11:36:08.830870+0000","last_clean":"2023-10-06T11:36:08.830870+0000","last_became_active":"2023-10-06T11:36:07.832883+0000","last_became_peered":"2023-10-06T11:36:07.832883+0000","last_unstale":"2023-10-06T11:36:08.830870+0000","last_undegraded":"2023-10-06T11:36:08.830870+0000","last_fullsized":"2023-10-06T11:36:08.830870+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827807+0000","last_change":"2023-10-06T11:36:08.826278+0000","last_active":"2023-10-06T11:36:08.827807+0000","last_peered":"2023-10-06T11:36:08.827807+0000","last_clean":"2023-10-06T11:36:08.827807+0000","last_became_active":"2023-10-06T11:36:08.825985+0000","last_became_peered":"2023-10-06T11:36:08.825985+0000","last_unstale":"2023-10-06T11:36:08.827807+0000","last_undegraded":"2023-10-06T11:36:08.827807+0000","last_fullsized":"2023-10-06T11:36:08.827807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820190+0000","last_change":"2023-10-06T11:36:07.840409+0000","last_active":"2023-10-06T11:36:08.820190+0000","last_peered":"2023-10-06T11:36:08.820190+0000","last_clean":"2023-10-06T11:36:08.820190+0000","last_became_active":"2023-10-06T11:36:07.840059+0000","last_became_peered":"2023-10-06T11:36:07.840059+0000","last_unstale":"2023-10-06T11:36:08.820190+0000","last_undegraded":"2023-10-06T11:36:08.820190+0000","last_fullsized":"2023-10-06T11:36:08.820190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:54.481491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832875+0000","last_change":"2023-10-06T11:36:08.826025+0000","last_active":"2023-10-06T11:36:08.832875+0000","last_peered":"2023-10-06T11:36:08.832875+0000","last_clean":"2023-10-06T11:36:08.832875+0000","last_became_active":"2023-10-06T11:36:08.825431+0000","last_became_peered":"2023-10-06T11:36:08.825431+0000","last_unstale":"2023-10-06T11:36:08.832875+0000","last_undegraded":"2023-10-06T11:36:08.832875+0000","last_fullsized":"2023-10-06T11:36:08.832875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:56.221716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"121'8","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864905+0000","last_change":"2023-10-06T11:36:07.844146+0000","last_active":"2023-10-06T11:36:08.864905+0000","last_peered":"2023-10-06T11:36:08.864905+0000","last_clean":"2023-10-06T11:36:08.864905+0000","last_became_active":"2023-10-06T11:36:07.843809+0000","last_became_peered":"2023-10-06T11:36:07.843809+0000","last_unstale":"2023-10-06T11:36:08.864905+0000","last_undegraded":"2023-10-06T11:36:08.864905+0000","last_fullsized":"2023-10-06T11:36:08.864905+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839926+0000","last_change":"2023-10-06T11:36:08.837343+0000","last_active":"2023-10-06T11:36:08.839926+0000","last_peered":"2023-10-06T11:36:08.839926+0000","last_clean":"2023-10-06T11:36:08.839926+0000","last_became_active":"2023-10-06T11:36:08.835511+0000","last_became_peered":"2023-10-06T11:36:08.835511+0000","last_unstale":"2023-10-06T11:36:08.839926+0000","last_undegraded":"2023-10-06T11:36:08.839926+0000","last_fullsized":"2023-10-06T11:36:08.839926+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:20.565177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857120+0000","last_change":"2023-10-06T11:36:07.832997+0000","last_active":"2023-10-06T11:36:08.857120+0000","last_peered":"2023-10-06T11:36:08.857120+0000","last_clean":"2023-10-06T11:36:08.857120+0000","last_became_active":"2023-10-06T11:36:07.832693+0000","last_became_peered":"2023-10-06T11:36:07.832693+0000","last_unstale":"2023-10-06T11:36:08.857120+0000","last_undegraded":"2023-10-06T11:36:08.857120+0000","last_fullsized":"2023-10-06T11:36:08.857120+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833519+0000","last_change":"2023-10-06T11:36:08.831230+0000","last_active":"2023-10-06T11:36:08.833519+0000","last_peered":"2023-10-06T11:36:08.833519+0000","last_clean":"2023-10-06T11:36:08.833519+0000","last_became_active":"2023-10-06T11:36:08.830920+0000","last_became_peered":"2023-10-06T11:36:08.830920+0000","last_unstale":"2023-10-06T11:36:08.833519+0000","last_undegraded":"2023-10-06T11:36:08.833519+0000","last_fullsized":"2023-10-06T11:36:08.833519+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:01.109540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810349+0000","last_change":"2023-10-06T11:36:07.833889+0000","last_active":"2023-10-06T11:36:08.810349+0000","last_peered":"2023-10-06T11:36:08.810349+0000","last_clean":"2023-10-06T11:36:08.810349+0000","last_became_active":"2023-10-06T11:36:07.833564+0000","last_became_peered":"2023-10-06T11:36:07.833564+0000","last_unstale":"2023-10-06T11:36:08.810349+0000","last_undegraded":"2023-10-06T11:36:08.810349+0000","last_fullsized":"2023-10-06T11:36:08.810349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:43.919700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835073+0000","last_change":"2023-10-06T11:36:08.832266+0000","last_active":"2023-10-06T11:36:08.835073+0000","last_peered":"2023-10-06T11:36:08.835073+0000","last_clean":"2023-10-06T11:36:08.835073+0000","last_became_active":"2023-10-06T11:36:08.830568+0000","last_became_peered":"2023-10-06T11:36:08.830568+0000","last_unstale":"2023-10-06T11:36:08.835073+0000","last_undegraded":"2023-10-06T11:36:08.835073+0000","last_fullsized":"2023-10-06T11:36:08.835073+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836702+0000","last_change":"2023-10-06T11:36:07.847402+0000","last_active":"2023-10-06T11:36:08.836702+0000","last_peered":"2023-10-06T11:36:08.836702+0000","last_clean":"2023-10-06T11:36:08.836702+0000","last_became_active":"2023-10-06T11:36:07.847014+0000","last_became_peered":"2023-10-06T11:36:07.847014+0000","last_unstale":"2023-10-06T11:36:08.836702+0000","last_undegraded":"2023-10-06T11:36:08.836702+0000","last_fullsized":"2023-10-06T11:36:08.836702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847206+0000","last_change":"2023-10-06T11:36:08.837814+0000","last_active":"2023-10-06T11:36:08.847206+0000","last_peered":"2023-10-06T11:36:08.847206+0000","last_clean":"2023-10-06T11:36:08.847206+0000","last_became_active":"2023-10-06T11:36:08.837531+0000","last_became_peered":"2023-10-06T11:36:08.837531+0000","last_unstale":"2023-10-06T11:36:08.847206+0000","last_undegraded":"2023-10-06T11:36:08.847206+0000","last_fullsized":"2023-10-06T11:36:08.847206+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:56.220796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816582+0000","last_change":"2023-10-06T11:36:07.834704+0000","last_active":"2023-10-06T11:36:08.816582+0000","last_peered":"2023-10-06T11:36:08.816582+0000","last_clean":"2023-10-06T11:36:08.816582+0000","last_became_active":"2023-10-06T11:36:07.834381+0000","last_became_peered":"2023-10-06T11:36:07.834381+0000","last_unstale":"2023-10-06T11:36:08.816582+0000","last_undegraded":"2023-10-06T11:36:08.816582+0000","last_fullsized":"2023-10-06T11:36:08.816582+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850140+0000","last_change":"2023-10-06T11:36:08.836975+0000","last_active":"2023-10-06T11:36:08.850140+0000","last_peered":"2023-10-06T11:36:08.850140+0000","last_clean":"2023-10-06T11:36:08.850140+0000","last_became_active":"2023-10-06T11:36:08.836692+0000","last_became_peered":"2023-10-06T11:36:08.836692+0000","last_unstale":"2023-10-06T11:36:08.850140+0000","last_undegraded":"2023-10-06T11:36:08.850140+0000","last_fullsized":"2023-10-06T11:36:08.850140+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:19.467727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825222+0000","last_change":"2023-10-06T11:36:07.843092+0000","last_active":"2023-10-06T11:36:08.825222+0000","last_peered":"2023-10-06T11:36:08.825222+0000","last_clean":"2023-10-06T11:36:08.825222+0000","last_became_active":"2023-10-06T11:36:07.842729+0000","last_became_peered":"2023-10-06T11:36:07.842729+0000","last_unstale":"2023-10-06T11:36:08.825222+0000","last_undegraded":"2023-10-06T11:36:08.825222+0000","last_fullsized":"2023-10-06T11:36:08.825222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850838+0000","last_change":"2023-10-06T11:36:08.843940+0000","last_active":"2023-10-06T11:36:08.850838+0000","last_peered":"2023-10-06T11:36:08.850838+0000","last_clean":"2023-10-06T11:36:08.850838+0000","last_became_active":"2023-10-06T11:36:08.843666+0000","last_became_peered":"2023-10-06T11:36:08.843666+0000","last_unstale":"2023-10-06T11:36:08.850838+0000","last_undegraded":"2023-10-06T11:36:08.850838+0000","last_fullsized":"2023-10-06T11:36:08.850838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822603+0000","last_change":"2023-10-06T11:36:07.840413+0000","last_active":"2023-10-06T11:36:08.822603+0000","last_peered":"2023-10-06T11:36:08.822603+0000","last_clean":"2023-10-06T11:36:08.822603+0000","last_became_active":"2023-10-06T11:36:07.840091+0000","last_became_peered":"2023-10-06T11:36:07.840091+0000","last_unstale":"2023-10-06T11:36:08.822603+0000","last_undegraded":"2023-10-06T11:36:08.822603+0000","last_fullsized":"2023-10-06T11:36:08.822603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:25.925632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849923+0000","last_change":"2023-10-06T11:36:08.848796+0000","last_active":"2023-10-06T11:36:08.849923+0000","last_peered":"2023-10-06T11:36:08.849923+0000","last_clean":"2023-10-06T11:36:08.849923+0000","last_became_active":"2023-10-06T11:36:08.848494+0000","last_became_peered":"2023-10-06T11:36:08.848494+0000","last_unstale":"2023-10-06T11:36:08.849923+0000","last_undegraded":"2023-10-06T11:36:08.849923+0000","last_fullsized":"2023-10-06T11:36:08.849923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835941+0000","last_change":"2023-10-06T11:36:07.847520+0000","last_active":"2023-10-06T11:36:08.835941+0000","last_peered":"2023-10-06T11:36:08.835941+0000","last_clean":"2023-10-06T11:36:08.835941+0000","last_became_active":"2023-10-06T11:36:07.847113+0000","last_became_peered":"2023-10-06T11:36:07.847113+0000","last_unstale":"2023-10-06T11:36:08.835941+0000","last_undegraded":"2023-10-06T11:36:08.835941+0000","last_fullsized":"2023-10-06T11:36:08.835941+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:44.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847166+0000","last_change":"2023-10-06T11:36:08.839414+0000","last_active":"2023-10-06T11:36:08.847166+0000","last_peered":"2023-10-06T11:36:08.847166+0000","last_clean":"2023-10-06T11:36:08.847166+0000","last_became_active":"2023-10-06T11:36:08.839089+0000","last_became_peered":"2023-10-06T11:36:08.839089+0000","last_unstale":"2023-10-06T11:36:08.847166+0000","last_undegraded":"2023-10-06T11:36:08.847166+0000","last_fullsized":"2023-10-06T11:36:08.847166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819931+0000","last_change":"2023-10-06T11:36:07.841033+0000","last_active":"2023-10-06T11:36:08.819931+0000","last_peered":"2023-10-06T11:36:08.819931+0000","last_clean":"2023-10-06T11:36:08.819931+0000","last_became_active":"2023-10-06T11:36:07.840720+0000","last_became_peered":"2023-10-06T11:36:07.840720+0000","last_unstale":"2023-10-06T11:36:08.819931+0000","last_undegraded":"2023-10-06T11:36:08.819931+0000","last_fullsized":"2023-10-06T11:36:08.819931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842397+0000","last_change":"2023-10-06T11:36:08.830558+0000","last_active":"2023-10-06T11:36:08.842397+0000","last_peered":"2023-10-06T11:36:08.842397+0000","last_clean":"2023-10-06T11:36:08.842397+0000","last_became_active":"2023-10-06T11:36:08.830244+0000","last_became_peered":"2023-10-06T11:36:08.830244+0000","last_unstale":"2023-10-06T11:36:08.842397+0000","last_undegraded":"2023-10-06T11:36:08.842397+0000","last_fullsized":"2023-10-06T11:36:08.842397+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857037+0000","last_change":"2023-10-06T11:36:07.838666+0000","last_active":"2023-10-06T11:36:08.857037+0000","last_peered":"2023-10-06T11:36:08.857037+0000","last_clean":"2023-10-06T11:36:08.857037+0000","last_became_active":"2023-10-06T11:36:07.837999+0000","last_became_peered":"2023-10-06T11:36:07.837999+0000","last_unstale":"2023-10-06T11:36:08.857037+0000","last_undegraded":"2023-10-06T11:36:08.857037+0000","last_fullsized":"2023-10-06T11:36:08.857037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"111'227","reported_seq":641,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833386+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:36:08.833386+0000","last_peered":"2023-10-06T11:36:08.833386+0000","last_clean":"2023-10-06T11:36:08.833386+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:36:08.833386+0000","last_undegraded":"2023-10-06T11:36:08.833386+0000","last_fullsized":"2023-10-06T11:36:08.833386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824390+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:36:08.824390+0000","last_peered":"2023-10-06T11:36:08.824390+0000","last_clean":"2023-10-06T11:36:08.824390+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:36:08.824390+0000","last_undegraded":"2023-10-06T11:36:08.824390+0000","last_fullsized":"2023-10-06T11:36:08.824390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840213+0000","last_change":"2023-10-06T11:36:08.836795+0000","last_active":"2023-10-06T11:36:08.840213+0000","last_peered":"2023-10-06T11:36:08.840213+0000","last_clean":"2023-10-06T11:36:08.840213+0000","last_became_active":"2023-10-06T11:36:08.836163+0000","last_became_peered":"2023-10-06T11:36:08.836163+0000","last_unstale":"2023-10-06T11:36:08.840213+0000","last_undegraded":"2023-10-06T11:36:08.840213+0000","last_fullsized":"2023-10-06T11:36:08.840213+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:55.806624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.863443+0000","last_change":"2023-10-06T11:36:07.839247+0000","last_active":"2023-10-06T11:36:08.863443+0000","last_peered":"2023-10-06T11:36:08.863443+0000","last_clean":"2023-10-06T11:36:08.863443+0000","last_became_active":"2023-10-06T11:36:07.838584+0000","last_became_peered":"2023-10-06T11:36:07.838584+0000","last_unstale":"2023-10-06T11:36:08.863443+0000","last_undegraded":"2023-10-06T11:36:08.863443+0000","last_fullsized":"2023-10-06T11:36:08.863443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:48.616236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833289+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:36:08.833289+0000","last_peered":"2023-10-06T11:36:08.833289+0000","last_clean":"2023-10-06T11:36:08.833289+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:36:08.833289+0000","last_undegraded":"2023-10-06T11:36:08.833289+0000","last_fullsized":"2023-10-06T11:36:08.833289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837137+0000","last_change":"2023-10-06T11:36:08.831245+0000","last_active":"2023-10-06T11:36:08.837137+0000","last_peered":"2023-10-06T11:36:08.837137+0000","last_clean":"2023-10-06T11:36:08.837137+0000","last_became_active":"2023-10-06T11:36:08.830977+0000","last_became_peered":"2023-10-06T11:36:08.830977+0000","last_unstale":"2023-10-06T11:36:08.837137+0000","last_undegraded":"2023-10-06T11:36:08.837137+0000","last_fullsized":"2023-10-06T11:36:08.837137+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:49.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824163+0000","last_change":"2023-10-06T11:36:07.841765+0000","last_active":"2023-10-06T11:36:08.824163+0000","last_peered":"2023-10-06T11:36:08.824163+0000","last_clean":"2023-10-06T11:36:08.824163+0000","last_became_active":"2023-10-06T11:36:07.841405+0000","last_became_peered":"2023-10-06T11:36:07.841405+0000","last_unstale":"2023-10-06T11:36:08.824163+0000","last_undegraded":"2023-10-06T11:36:08.824163+0000","last_fullsized":"2023-10-06T11:36:08.824163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:26.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820278+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:36:08.820278+0000","last_peered":"2023-10-06T11:36:08.820278+0000","last_clean":"2023-10-06T11:36:08.820278+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:36:08.820278+0000","last_undegraded":"2023-10-06T11:36:08.820278+0000","last_fullsized":"2023-10-06T11:36:08.820278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847753+0000","last_change":"2023-10-06T11:36:08.834917+0000","last_active":"2023-10-06T11:36:08.847753+0000","last_peered":"2023-10-06T11:36:08.847753+0000","last_clean":"2023-10-06T11:36:08.847753+0000","last_became_active":"2023-10-06T11:36:08.834522+0000","last_became_peered":"2023-10-06T11:36:08.834522+0000","last_unstale":"2023-10-06T11:36:08.847753+0000","last_undegraded":"2023-10-06T11:36:08.847753+0000","last_fullsized":"2023-10-06T11:36:08.847753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.860358+0000","last_change":"2023-10-06T11:36:07.840773+0000","last_active":"2023-10-06T11:36:08.860358+0000","last_peered":"2023-10-06T11:36:08.860358+0000","last_clean":"2023-10-06T11:36:08.860358+0000","last_became_active":"2023-10-06T11:36:07.840415+0000","last_became_peered":"2023-10-06T11:36:07.840415+0000","last_unstale":"2023-10-06T11:36:08.860358+0000","last_undegraded":"2023-10-06T11:36:08.860358+0000","last_fullsized":"2023-10-06T11:36:08.860358+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824689+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:36:08.824689+0000","last_peered":"2023-10-06T11:36:08.824689+0000","last_clean":"2023-10-06T11:36:08.824689+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:36:08.824689+0000","last_undegraded":"2023-10-06T11:36:08.824689+0000","last_fullsized":"2023-10-06T11:36:08.824689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834204+0000","last_change":"2023-10-06T11:36:08.831735+0000","last_active":"2023-10-06T11:36:08.834204+0000","last_peered":"2023-10-06T11:36:08.834204+0000","last_clean":"2023-10-06T11:36:08.834204+0000","last_became_active":"2023-10-06T11:36:08.831450+0000","last_became_peered":"2023-10-06T11:36:08.831450+0000","last_unstale":"2023-10-06T11:36:08.834204+0000","last_undegraded":"2023-10-06T11:36:08.834204+0000","last_fullsized":"2023-10-06T11:36:08.834204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835829+0000","last_change":"2023-10-06T11:36:07.847157+0000","last_active":"2023-10-06T11:36:08.835829+0000","last_peered":"2023-10-06T11:36:08.835829+0000","last_clean":"2023-10-06T11:36:08.835829+0000","last_became_active":"2023-10-06T11:36:07.846801+0000","last_became_peered":"2023-10-06T11:36:07.846801+0000","last_unstale":"2023-10-06T11:36:08.835829+0000","last_undegraded":"2023-10-06T11:36:08.835829+0000","last_fullsized":"2023-10-06T11:36:08.835829+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:08.462187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816701+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:36:08.816701+0000","last_peered":"2023-10-06T11:36:08.816701+0000","last_clean":"2023-10-06T11:36:08.816701+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:36:08.816701+0000","last_undegraded":"2023-10-06T11:36:08.816701+0000","last_fullsized":"2023-10-06T11:36:08.816701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848126+0000","last_change":"2023-10-06T11:36:08.836395+0000","last_active":"2023-10-06T11:36:08.848126+0000","last_peered":"2023-10-06T11:36:08.848126+0000","last_clean":"2023-10-06T11:36:08.848126+0000","last_became_active":"2023-10-06T11:36:08.836112+0000","last_became_peered":"2023-10-06T11:36:08.836112+0000","last_unstale":"2023-10-06T11:36:08.848126+0000","last_undegraded":"2023-10-06T11:36:08.848126+0000","last_fullsized":"2023-10-06T11:36:08.848126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819833+0000","last_change":"2023-10-06T11:36:07.840551+0000","last_active":"2023-10-06T11:36:08.819833+0000","last_peered":"2023-10-06T11:36:08.819833+0000","last_clean":"2023-10-06T11:36:08.819833+0000","last_became_active":"2023-10-06T11:36:07.840210+0000","last_became_peered":"2023-10-06T11:36:07.840210+0000","last_unstale":"2023-10-06T11:36:08.819833+0000","last_undegraded":"2023-10-06T11:36:08.819833+0000","last_fullsized":"2023-10-06T11:36:08.819833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833204+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:36:08.833204+0000","last_peered":"2023-10-06T11:36:08.833204+0000","last_clean":"2023-10-06T11:36:08.833204+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:36:08.833204+0000","last_undegraded":"2023-10-06T11:36:08.833204+0000","last_fullsized":"2023-10-06T11:36:08.833204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839738+0000","last_change":"2023-10-06T11:36:08.836510+0000","last_active":"2023-10-06T11:36:08.839738+0000","last_peered":"2023-10-06T11:36:08.839738+0000","last_clean":"2023-10-06T11:36:08.839738+0000","last_became_active":"2023-10-06T11:36:08.836215+0000","last_became_peered":"2023-10-06T11:36:08.836215+0000","last_unstale":"2023-10-06T11:36:08.839738+0000","last_undegraded":"2023-10-06T11:36:08.839738+0000","last_fullsized":"2023-10-06T11:36:08.839738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:32.797025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857016+0000","last_change":"2023-10-06T11:36:07.841111+0000","last_active":"2023-10-06T11:36:08.857016+0000","last_peered":"2023-10-06T11:36:08.857016+0000","last_clean":"2023-10-06T11:36:08.857016+0000","last_became_active":"2023-10-06T11:36:07.840810+0000","last_became_peered":"2023-10-06T11:36:07.840810+0000","last_unstale":"2023-10-06T11:36:08.857016+0000","last_undegraded":"2023-10-06T11:36:08.857016+0000","last_fullsized":"2023-10-06T11:36:08.857016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:03.725992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.818400+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:36:08.818400+0000","last_peered":"2023-10-06T11:36:08.818400+0000","last_clean":"2023-10-06T11:36:08.818400+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:36:08.818400+0000","last_undegraded":"2023-10-06T11:36:08.818400+0000","last_fullsized":"2023-10-06T11:36:08.818400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837031+0000","last_change":"2023-10-06T11:36:08.833641+0000","last_active":"2023-10-06T11:36:08.837031+0000","last_peered":"2023-10-06T11:36:08.837031+0000","last_clean":"2023-10-06T11:36:08.837031+0000","last_became_active":"2023-10-06T11:36:08.833372+0000","last_became_peered":"2023-10-06T11:36:08.833372+0000","last_unstale":"2023-10-06T11:36:08.837031+0000","last_undegraded":"2023-10-06T11:36:08.837031+0000","last_fullsized":"2023-10-06T11:36:08.837031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:45.164451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822466+0000","last_change":"2023-10-06T11:36:07.841750+0000","last_active":"2023-10-06T11:36:08.822466+0000","last_peered":"2023-10-06T11:36:08.822466+0000","last_clean":"2023-10-06T11:36:08.822466+0000","last_became_active":"2023-10-06T11:36:07.841432+0000","last_became_peered":"2023-10-06T11:36:07.841432+0000","last_unstale":"2023-10-06T11:36:08.822466+0000","last_undegraded":"2023-10-06T11:36:08.822466+0000","last_fullsized":"2023-10-06T11:36:08.822466+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:27:06.372507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836500+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:36:08.836500+0000","last_peered":"2023-10-06T11:36:08.836500+0000","last_clean":"2023-10-06T11:36:08.836500+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:36:08.836500+0000","last_undegraded":"2023-10-06T11:36:08.836500+0000","last_fullsized":"2023-10-06T11:36:08.836500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575060,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":2109440,"data_stored":1371418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1020000000000001,"15min":1.1339999999999999},"min":{"1min":0.84499999999999997,"5min":0.59599999999999997,"15min":0.495},"max":{"1min":1.5249999999999999,"5min":1.6579999999999999,"15min":1.724},"last":1.224},{"interface":"front","average":{"1min":1.159,"5min":1.143,"15min":1.139},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.56399999999999995},"max":{"1min":1.5229999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":1.4410000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.359,"15min":1.333},"min":{"1min":0.78100000000000003,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.651,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.232},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5349999999999999,"5min":1.927,"15min":1.927},"last":1.1679999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2,"15min":1.214},"min":{"1min":1.006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6040000000000001,"5min":1.827,"15min":1.827},"last":1.335},{"interface":"front","average":{"1min":1.353,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.294}]},{"osd":3,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.256,"15min":1.264},"min":{"1min":0.92900000000000005,"5min":0.79300000000000004,"15min":0.68500000000000005},"max":{"1min":1.7210000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.486},{"interface":"front","average":{"1min":1.454,"5min":1.3380000000000001,"15min":1.323},"min":{"1min":1.2829999999999999,"5min":0.89000000000000001,"15min":0.75600000000000001},"max":{"1min":1.8029999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1120000000000001},"min":{"1min":0.72799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.601,"5min":1.663,"15min":2.149},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.25},"min":{"1min":0.85199999999999998,"5min":0.55300000000000005,"15min":0.41699999999999998},"max":{"1min":1.7849999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3700000000000001,"15min":1.3109999999999999},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.58199999999999996},"max":{"1min":1.881,"5min":2.488,"15min":2.488},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":1.903,"15min":2.2480000000000002},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.347,"15min":1.349},"min":{"1min":0.93600000000000005,"5min":0.67000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7250000000000001,"5min":2.0219999999999998,"15min":2.4289999999999998},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.60099999999999998,"15min":0.54200000000000004},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575060,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":783002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2869999999999999,"15min":1.2889999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.891,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.3180000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.546,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1439999999999999,"15min":1.1579999999999999},"min":{"1min":1.024,"5min":0.76200000000000001,"15min":0.64000000000000001},"max":{"1min":1.48,"5min":1.5980000000000001,"15min":1.8169999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.214},"min":{"1min":0.78600000000000003,"5min":0.66200000000000003,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.046}]},{"osd":2,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.202,"15min":1.206},"min":{"1min":0.85399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.476,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.214},"min":{"1min":0.79300000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5009999999999999,"5min":1.736,"15min":1.736},"last":1.222}]},{"osd":3,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.306},"min":{"1min":0.89500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.625,"5min":2.121,"15min":2.121},"last":1.302},{"interface":"front","average":{"1min":1.321,"5min":1.3240000000000001,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.597,"5min":1.7430000000000001,"15min":1.7509999999999999},"last":1.345}]},{"osd":4,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.90700000000000003,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.159},{"interface":"front","average":{"1min":1.198,"5min":1.304,"15min":1.248},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":1.649,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.25}]},{"osd":5,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.196,"15min":1.2270000000000001},"min":{"1min":0.89200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5529999999999999,"5min":1.879,"15min":1.879},"last":1.369},{"interface":"front","average":{"1min":1.282,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.92100000000000004,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.9359999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.399}]},{"osd":7,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4390000000000001},"min":{"1min":0.84199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.77,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.581},{"interface":"front","average":{"1min":1.363,"5min":1.389,"15min":1.415},"min":{"1min":1.0129999999999999,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.508}]}]},{"osd":5,"up_from":13,"seq":55834575060,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":780890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3979999999999999,"15min":1.284},"min":{"1min":0.89900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.615,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.296},"min":{"1min":1.095,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":1.972,"15min":2.3959999999999999},"last":1.117}]},{"osd":1,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1479999999999999,"15min":1.1639999999999999},"min":{"1min":0.82999999999999996,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.3999999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.212},"min":{"1min":0.84799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5529999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.044}]},{"osd":2,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.88800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5149999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.181},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.6990000000000001,"15min":1.738},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.397,"15min":1.28},"min":{"1min":1.0109999999999999,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.454,"5min":18.074999999999999,"15min":18.074999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.2989999999999999},"min":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.581,"5min":1.8400000000000001,"15min":1.871},"last":1}]},{"osd":4,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3460000000000001,"15min":1.2609999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.5369999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5309999999999999,"15min":1.383},"min":{"1min":0.83899999999999997,"5min":0.72199999999999998,"15min":0.63500000000000001},"max":{"1min":1.6739999999999999,"5min":18.48,"15min":18.48},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3999999999999999,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.4870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.488,"15min":1.3660000000000001},"min":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.609,"5min":18.379999999999999,"15min":18.379999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.268,"5min":1.4550000000000001,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.534,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.0620000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575061,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1515520,"data_stored":780982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.77400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.669,"5min":1.72,"15min":1.72},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.175,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.84499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.48,"5min":1.5669999999999999,"15min":1.6910000000000001},"last":0.998}]},{"osd":1,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":0.94399999999999995,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7709999999999999},"last":1.23},{"interface":"front","average":{"1min":1.331,"5min":1.26,"15min":1.272},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.0620000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2010000000000001,"15min":1.196},"min":{"1min":0.874,"5min":0.628,"15min":0.628},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.274,"15min":1.2669999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6619999999999999,"5min":1.6779999999999999,"15min":1.78},"last":1.353}]},{"osd":3,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1839999999999999,"15min":1.204},"min":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6040000000000001,"5min":1.617,"15min":1.6839999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.3180000000000001,"15min":1.3049999999999999},"min":{"1min":1.0369999999999999,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.77,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.82699999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.706,"5min":1.706,"15min":1.7390000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.571,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.2669999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2390000000000001,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.389},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.3480000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.831,"5min":1.845,"15min":1.8660000000000001},"last":1.522}]},{"osd":7,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.349,"15min":1.329},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4359999999999999,"15min":1.3879999999999999},"min":{"1min":0.96299999999999997,"5min":0.88700000000000001,"15min":0.73799999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.482}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":781444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.052,"15min":0.93500000000000005},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":14.49,"15min":14.49},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.998,"5min":1.175,"15min":1.05},"min":{"1min":0.81200000000000006,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.98599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98099999999999998,"15min":0.96699999999999997},"min":{"1min":0.73899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1890000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.1579999999999999,"15min":1.1140000000000001},"min":{"1min":0.72899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.698,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.23,"15min":1.1599999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.6080000000000001,"5min":12.446,"15min":12.446},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.76100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.5009999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3169999999999999,"15min":1.2589999999999999},"min":{"1min":0.89200000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":10.460000000000001,"15min":10.460000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.343,"15min":1.3560000000000001},"min":{"1min":1.0289999999999999,"5min":0.80800000000000005,"15min":0.78400000000000003},"max":{"1min":1.5569999999999999,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.407,"15min":1.411},"min":{"1min":1.0860000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.73,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.111},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.874,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7509999999999999,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.3009999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.8080000000000001,"5min":11.362,"15min":11.362},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.4419999999999999,"15min":1.399},"min":{"1min":0.98499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6679999999999999,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.52,"15min":1.4610000000000001},"min":{"1min":1.1759999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7689999999999999,"5min":10.561,"15min":10.561},"last":1.607},{"interface":"front","average":{"1min":1.48,"5min":1.4890000000000001,"15min":1.4870000000000001},"min":{"1min":1.1679999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.792,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.3320000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.56999999999999995,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4530000000000001,"5min":1.524,"15min":1.8380000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.52,"5min":1.5409999999999999,"15min":1.9650000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.018},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.5760000000000001,"5min":1.599,"15min":2.5390000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.84599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.546,"5min":1.758,"15min":1.8240000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.288,"5min":1.256,"15min":1.2989999999999999},"min":{"1min":0.85499999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"max":{"1min":1.6419999999999999,"5min":1.948,"15min":2.4550000000000001},"last":1.016}]},{"osd":4,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.96999999999999997,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.569,"5min":1.625,"15min":1.907},"last":1.093},{"interface":"front","average":{"1min":1.373,"5min":1.3,"15min":1.29},"min":{"1min":1.071,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3420000000000001,"15min":1.3580000000000001},"min":{"1min":0.94099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8049999999999999,"5min":1.8480000000000001,"15min":2.0680000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2909999999999999,"15min":1.3300000000000001},"min":{"1min":1.0089999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.8049999999999999,"15min":2.0630000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.411,"15min":1.403},"min":{"1min":1.121,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0150000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.76900000000000002},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":2.617},"last":1.369}]},{"osd":7,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.375,"15min":1.387},"min":{"1min":1.1160000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.05},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.468,"15min":1.4650000000000001},"min":{"1min":1.238,"5min":0.753,"15min":0.753},"max":{"1min":2.0830000000000002,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.399}]}]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.874,"15min":0.92100000000000004},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.724},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.59199999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.337,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":1.216},{"interface":"front","average":{"1min":1.107,"5min":1.077,"15min":1.093},"min":{"1min":0.79100000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":2.137},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1799999999999999,"15min":1.214},"min":{"1min":0.82999999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0640000000000001,"15min":1.0920000000000001},"min":{"1min":0.71299999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5549999999999999,"15min":1.756},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2270000000000001,"15min":1.2430000000000001},"min":{"1min":1.0149999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5529999999999999,"5min":1.891,"15min":2.2290000000000001},"last":1.071},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3560000000000001,"15min":1.375},"min":{"1min":0.90900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.944},"last":1.1100000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.94799999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.6970000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3460000000000001,"15min":1.3440000000000001},"min":{"1min":1.173,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.871,"5min":1.923,"15min":2.1240000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4350000000000001,"15min":1.46},"min":{"1min":1.0289999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8979999999999999,"5min":1.966,"15min":2.0310000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.0369999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.901,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.3779999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4810000000000001,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0209999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.458},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.4870000000000001,"15min":1.454},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.8100000000000001,"5min":2.085,"15min":2.085},"last":1.286}]}]},{"osd":0,"up_from":13,"seq":55834575061,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2117632,"data_stored":1372884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.571},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.54200000000000004},"max":{"1min":1.5069999999999999,"5min":1.609,"15min":1.609},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.026,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.6699999999999999,"15min":1.6950000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.0640000000000001,"15min":1.0760000000000001},"min":{"1min":0.83199999999999996,"5min":0.58399999999999996,"15min":0.55800000000000005},"max":{"1min":1.3720000000000001,"5min":1.738,"15min":1.738},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0800000000000001},"min":{"1min":0.80100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.615,"5min":1.6639999999999999,"15min":1.919},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2749999999999999,"15min":1.3169999999999999},"min":{"1min":0.90600000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.673,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.286}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3140000000000001,"15min":1.339},"min":{"1min":1.0309999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.7749999999999999,"5min":2.1720000000000002,"15min":2.294},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2609999999999999,"15min":1.2909999999999999},"min":{"1min":0.93300000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.454}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.409,"15min":1.377},"min":{"1min":1.0609999999999999,"5min":0.90100000000000002,"15min":0.65900000000000003},"max":{"1min":1.9350000000000001,"5min":2.0859999999999999,"15min":2.25},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3979999999999999,"15min":1.3959999999999999},"min":{"1min":1.103,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.893,"5min":1.893,"15min":2.5510000000000002},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.22,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.647,"5min":2.2240000000000002,"15min":2.52},"last":1.129},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.835,"5min":2.391,"15min":2.4750000000000001},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.4650000000000001},"min":{"1min":1.129,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.728,"5min":2.0219999999999998,"15min":2.4260000000000002},"last":1.236},{"interface":"front","average":{"1min":1.413,"5min":1.431,"15min":1.429},"min":{"1min":1.0820000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7589999999999999,"5min":2.1459999999999999,"15min":2.3809999999999998},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:36:14.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T11:36:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.201+0000 7ffab0b45700 1 Processor -- start 2023-10-06T11:36:14.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.201+0000 7ffab0b45700 1 -- start start 2023-10-06T11:36:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffab0b45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffab0b45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffab0b45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac0c67e0 con 0x7ffaac1502f0 2023-10-06T11:36:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffab0b45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac06e810 con 0x7ffaac15a1a0 2023-10-06T11:36:14.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffab0b45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac06eaf0 con 0x7ffaac1506c0 2023-10-06T11:36:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaaad9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43110/0 (socket says 172.21.15.46:43110) 2023-10-06T11:36:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaaa59c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ffaac1502f0 0x7ffaac159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:54438/0 (socket says 172.21.15.46:54438) 2023-10-06T11:36:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 -- 172.21.15.46:0/3987203700 learned_addr learned my addr 172.21.15.46:0/3987203700 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 -- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 msgr2=0x7ffaac0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac0c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 -- 172.21.15.46:0/3987203700 >> v1:172.21.15.132:6789/0 conn(0x7ffaac1502f0 legacy=0x7ffaac159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa3fff700 1 -- 172.21.15.46:0/3987203700 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac06edd0 con 0x7ffaac15a1a0 2023-10-06T11:36:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaa9d9b700 1 -- 172.21.15.46:0/3987203700 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768648051 0 0) 0x7ffaac0c67e0 con 0x7ffaac1502f0 2023-10-06T11:36:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.205+0000 7ffaaad9d700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:36:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffaa3fff700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffa940099b0 tx=0x7ffa94009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a6dd49e5798fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffaa9d9b700 1 -- 172.21.15.46:0/3987203700 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94009ea0 con 0x7ffaac15a1a0 2023-10-06T11:36:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffaa9d9b700 1 -- 172.21.15.46:0/3987203700 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9400b860 con 0x7ffaac15a1a0 2023-10-06T11:36:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffaa9d9b700 1 -- 172.21.15.46:0/3987203700 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400bc00 con 0x7ffaac15a1a0 2023-10-06T11:36:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 -- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 msgr2=0x7ffaac0c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffa940099b0 tx=0x7ffa94009250 comp rx=0 tx=0).stop 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 -- 172.21.15.46:0/3987203700 shutdown_connections 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 --2- 172.21.15.46:0/3987203700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac0c0bd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 -- 172.21.15.46:0/3987203700 >> 172.21.15.46:0/3987203700 conn(0x7ffaac0cbe30 msgr2=0x7ffaac0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 -- 172.21.15.46:0/3987203700 shutdown_connections 2023-10-06T11:36:14.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.209+0000 7ffab0b45700 1 -- 172.21.15.46:0/3987203700 wait complete. 2023-10-06T11:36:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 Processor -- start 2023-10-06T11:36:14.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 -- start start 2023-10-06T11:36:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac14f230 con 0x7ffaac1502f0 2023-10-06T11:36:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0c39f0 con 0x7ffaac15a1a0 2023-10-06T11:36:14.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffab0b45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac14f430 con 0x7ffaac1506c0 2023-10-06T11:36:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffaaa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34916/0 (socket says 172.21.15.46:34916) 2023-10-06T11:36:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffaaa59c700 1 -- 172.21.15.46:0/2699539388 learned_addr learned my addr 172.21.15.46:0/2699539388 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffaa3fff700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.213+0000 7ffaaad9d700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 -- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 msgr2=0x7ffaac14e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac14e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 -- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 msgr2=0x7ffaac14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac14ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 -- 172.21.15.46:0/2699539388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4009a20 con 0x7ffaac1502f0 2023-10-06T11:36:14.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaaa59c700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c810 tx=0x7ffa9c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c005b90 con 0x7ffaac1502f0 2023-10-06T11:36:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c005cf0 con 0x7ffaac1502f0 2023-10-06T11:36:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c007300 con 0x7ffaac1502f0 2023-10-06T11:36:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa940094f0 con 0x7ffaac1502f0 2023-10-06T11:36:14.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac14f7c0 con 0x7ffaac1502f0 2023-10-06T11:36:14.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.217+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac0c3bb0 con 0x7ffaac1502f0 2023-10-06T11:36:14.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.221+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ffa9c01b3b0 con 0x7ffaac1502f0 2023-10-06T11:36:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.221+0000 7ffaa27fc700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 0x7ffa8c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.221+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7ffa9c07c130 con 0x7ffaac1502f0 2023-10-06T11:36:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.225+0000 7ffaa3fff700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 0x7ffa8c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.225+0000 7ffaa3fff700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 0x7ffa8c062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa940099b0 tx=0x7ffa94017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.229+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffa9c04c310 con 0x7ffaac1502f0 2023-10-06T11:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.465+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ffaac063f90 con 0x7ffaac1502f0 2023-10-06T11:36:14.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.465+0000 7ffaa27fc700 1 -- 172.21.15.46:0/2699539388 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1019 (secure 0 0 0) 0x7ffa9c005e60 con 0x7ffaac1502f0 2023-10-06T11:36:14.480 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:14.480 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"36b5f543-5934-4e63-898c-253575113a16","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1097,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":122,"num_osds":8,"num_up_osds":8,"osd_up_since":1696591089,"num_in_osds":8,"osd_in_since":1696591080,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2519998464,"bytes_avail":770574114816,"bytes_total":773094113280,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":59,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7824}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-10-06T11:35:49.603933+0000","services":{}},"progress_events":{}} 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 msgr2=0x7ffa8c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 0x7ffa8c062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa940099b0 tx=0x7ffa94017000 comp rx=0 tx=0).stop 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 msgr2=0x7ffaac0c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00c810 tx=0x7ffa9c007d90 comp rx=0 tx=0).stop 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 shutdown_connections 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ffa8c060670 0x7ffa8c062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaac15a1a0 0x7ffaac14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaac1506c0 0x7ffaac14e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 --2- 172.21.15.46:0/2699539388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaac1502f0 0x7ffaac0c9d10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 >> 172.21.15.46:0/2699539388 conn(0x7ffaac0cbe30 msgr2=0x7ffaac0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 shutdown_connections 2023-10-06T11:36:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.469+0000 7ffab0b45700 1 -- 172.21.15.46:0/2699539388 wait complete. 2023-10-06T11:36:14.506 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T11:36:14.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:36:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.653+0000 7f4bf3e78700 1 Processor -- start 2023-10-06T11:36:14.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.653+0000 7f4bf3e78700 1 -- start start 2023-10-06T11:36:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf3e78700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 0x7f4bec161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf3e78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bec0c1d80 0x7f4bec0c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf3e78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec0c18f0 con 0x7f4bec06cc30 2023-10-06T11:36:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf3e78700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0c0ce0 con 0x7f4bec161470 2023-10-06T11:36:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf3e78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0c0fc0 con 0x7f4bec0c1d80 2023-10-06T11:36:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1c14700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 0x7f4bec161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bec0c1d80 0x7f4bec0c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1c14700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 0x7f4bec161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34924/0 (socket says 172.21.15.46:34924) 2023-10-06T11:36:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1c14700 1 -- 172.21.15.46:0/2301238668 learned_addr learned my addr 172.21.15.46:0/2301238668 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 -- 172.21.15.46:0/2301238668 >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 msgr2=0x7f4bec161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 --2- 172.21.15.46:0/2301238668 >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 0x7f4bec161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 -- 172.21.15.46:0/2301238668 >> v1:172.21.15.132:6789/0 conn(0x7f4bec06cc30 legacy=0x7f4bec15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 -- 172.21.15.46:0/2301238668 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0c12a0 con 0x7f4bec0c1d80 2023-10-06T11:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.657+0000 7f4bf1413700 1 --2- 172.21.15.46:0/2301238668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0c1d80 0x7f4bec0c21e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06ba90 tx=0x7f4bdc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89ce681c7524433 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf0c12700 1 -- 172.21.15.46:0/2301238668 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00b690 con 0x7f4bec0c1d80 2023-10-06T11:36:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf0c12700 1 -- 172.21.15.46:0/2301238668 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00b7f0 con 0x7f4bec0c1d80 2023-10-06T11:36:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf0c12700 1 -- 172.21.15.46:0/2301238668 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00bb90 con 0x7f4bec0c1d80 2023-10-06T11:36:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 -- 172.21.15.46:0/2301238668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0c1d80 msgr2=0x7f4bec0c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/2301238668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0c1d80 0x7f4bec0c21e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4bec06ba90 tx=0x7f4bdc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 -- 172.21.15.46:0/2301238668 shutdown_connections 2023-10-06T11:36:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/2301238668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0c1d80 0x7f4bec0c21e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/2301238668 >> v2:172.21.15.46:3300/0 conn(0x7f4bec161470 0x7f4bec161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 -- 172.21.15.46:0/2301238668 >> 172.21.15.46:0/2301238668 conn(0x7f4bec0cbe30 msgr2=0x7f4bec0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 -- 172.21.15.46:0/2301238668 shutdown_connections 2023-10-06T11:36:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.661+0000 7f4bf3e78700 1 -- 172.21.15.46:0/2301238668 wait complete. 2023-10-06T11:36:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 Processor -- start 2023-10-06T11:36:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 -- start start 2023-10-06T11:36:14.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec06cc30 0x7f4bec163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec161470 0x7f4bec14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec172150 con 0x7f4bec0c1d80 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec15d940 con 0x7f4bec161470 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf3e78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec14b900 con 0x7f4bec06cc30 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34932/0 (socket says 172.21.15.46:34932) 2023-10-06T11:36:14.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 -- 172.21.15.46:0/4064267098 learned_addr learned my addr 172.21.15.46:0/4064267098 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1c14700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec06cc30 0x7f4bec163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf2415700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec161470 0x7f4bec14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 -- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec06cc30 msgr2=0x7f4bec163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec06cc30 0x7f4bec163bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 -- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec161470 msgr2=0x7f4bec14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec161470 0x7f4bec14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 -- 172.21.15.46:0/4064267098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009420 con 0x7f4bec0c1d80 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.665+0000 7f4bf1413700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0c1100 tx=0x7f4bdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011420 con 0x7f4bec0c1d80 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc011580 con 0x7f4bec0c1d80 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011940 con 0x7f4bec0c1d80 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec15d010 con 0x7f4bec0c1d80 2023-10-06T11:36:14.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec15d5d0 con 0x7f4bec0c1d80 2023-10-06T11:36:14.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.669+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec063f90 con 0x7f4bec0c1d80 2023-10-06T11:36:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.673+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4bdc011d30 con 0x7f4bec0c1d80 2023-10-06T11:36:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.673+0000 7f4be1ffb700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 0x7f4bd0062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.673+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f4bdc07e690 con 0x7f4bec0c1d80 2023-10-06T11:36:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.673+0000 7f4bf1c14700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 0x7f4bd0062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.673+0000 7f4bf1c14700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 0x7f4bd0062b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006fd0 tx=0x7f4be8008040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.681+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4bdc04e8d0 con 0x7f4bec0c1d80 2023-10-06T11:36:14.787 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-10-06T11:36:14.773+0000 7f5190ed2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:36:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.849+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bec15cab0 con 0x7f4bd00606a0 2023-10-06T11:36:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.865+0000 7f4be1ffb700 1 -- 172.21.15.46:0/4064267098 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309227 (secure 0 0 0) 0x7f4bec15cab0 con 0x7f4bd00606a0 2023-10-06T11:36:14.884 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:14.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.873+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 msgr2=0x7f4bd0062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.873+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 0x7f4bd0062b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4be8006fd0 tx=0x7f4be8008040 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.873+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 msgr2=0x7f4bec14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.873+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0c1100 tx=0x7f4bdc017000 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 shutdown_connections 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bd00606a0 0x7f4bd0062b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec161470 0x7f4bec14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bec0c1d80 0x7f4bec14ae80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 --2- 172.21.15.46:0/4064267098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec06cc30 0x7f4bec163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 >> 172.21.15.46:0/4064267098 conn(0x7f4bec0cbe30 msgr2=0x7f4bec0acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 shutdown_connections 2023-10-06T11:36:14.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:14.877+0000 7f4bf3e78700 1 -- 172.21.15.46:0/4064267098 wait complete. 2023-10-06T11:36:14.890 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:36:14.908 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T11:36:13.525973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460936,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752513784,"statfs":{"total":773094113280,"available":770574114816,"internally_reserved":0,"allocated":13312000,"data_stored":7432468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002280"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839807+0000","last_change":"2023-10-06T11:36:08.834959+0000","last_active":"2023-10-06T11:36:08.839807+0000","last_peered":"2023-10-06T11:36:08.839807+0000","last_clean":"2023-10-06T11:36:08.839807+0000","last_became_active":"2023-10-06T11:36:08.834673+0000","last_became_peered":"2023-10-06T11:36:08.834673+0000","last_unstale":"2023-10-06T11:36:08.839807+0000","last_undegraded":"2023-10-06T11:36:08.839807+0000","last_fullsized":"2023-10-06T11:36:08.839807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:45.178435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843699+0000","last_change":"2023-10-06T11:36:08.839427+0000","last_active":"2023-10-06T11:36:08.843699+0000","last_peered":"2023-10-06T11:36:08.843699+0000","last_clean":"2023-10-06T11:36:08.843699+0000","last_became_active":"2023-10-06T11:36:08.839074+0000","last_became_peered":"2023-10-06T11:36:08.839074+0000","last_unstale":"2023-10-06T11:36:08.843699+0000","last_undegraded":"2023-10-06T11:36:08.843699+0000","last_fullsized":"2023-10-06T11:36:08.843699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:44.389800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.826608+0000","last_change":"2023-10-06T11:36:08.822154+0000","last_active":"2023-10-06T11:36:08.826608+0000","last_peered":"2023-10-06T11:36:08.826608+0000","last_clean":"2023-10-06T11:36:08.826608+0000","last_became_active":"2023-10-06T11:36:08.821837+0000","last_became_peered":"2023-10-06T11:36:08.821837+0000","last_unstale":"2023-10-06T11:36:08.826608+0000","last_undegraded":"2023-10-06T11:36:08.826608+0000","last_fullsized":"2023-10-06T11:36:08.826608+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.339584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841877+0000","last_change":"2023-10-06T11:36:08.835584+0000","last_active":"2023-10-06T11:36:08.841877+0000","last_peered":"2023-10-06T11:36:08.841877+0000","last_clean":"2023-10-06T11:36:08.841877+0000","last_became_active":"2023-10-06T11:36:08.835284+0000","last_became_peered":"2023-10-06T11:36:08.835284+0000","last_unstale":"2023-10-06T11:36:08.841877+0000","last_undegraded":"2023-10-06T11:36:08.841877+0000","last_fullsized":"2023-10-06T11:36:08.841877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842628+0000","last_change":"2023-10-06T11:36:08.837618+0000","last_active":"2023-10-06T11:36:08.842628+0000","last_peered":"2023-10-06T11:36:08.842628+0000","last_clean":"2023-10-06T11:36:08.842628+0000","last_became_active":"2023-10-06T11:36:08.837297+0000","last_became_peered":"2023-10-06T11:36:08.837297+0000","last_unstale":"2023-10-06T11:36:08.842628+0000","last_undegraded":"2023-10-06T11:36:08.842628+0000","last_fullsized":"2023-10-06T11:36:08.842628+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850512+0000","last_change":"2023-10-06T11:36:08.847872+0000","last_active":"2023-10-06T11:36:08.850512+0000","last_peered":"2023-10-06T11:36:08.850512+0000","last_clean":"2023-10-06T11:36:08.850512+0000","last_became_active":"2023-10-06T11:36:08.847547+0000","last_became_peered":"2023-10-06T11:36:08.847547+0000","last_unstale":"2023-10-06T11:36:08.850512+0000","last_undegraded":"2023-10-06T11:36:08.850512+0000","last_fullsized":"2023-10-06T11:36:08.850512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:43.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849958+0000","last_change":"2023-10-06T11:36:08.846412+0000","last_active":"2023-10-06T11:36:08.849958+0000","last_peered":"2023-10-06T11:36:08.849958+0000","last_clean":"2023-10-06T11:36:08.849958+0000","last_became_active":"2023-10-06T11:36:08.843620+0000","last_became_peered":"2023-10-06T11:36:08.843620+0000","last_unstale":"2023-10-06T11:36:08.849958+0000","last_undegraded":"2023-10-06T11:36:08.849958+0000","last_fullsized":"2023-10-06T11:36:08.849958+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:23.081039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850850+0000","last_change":"2023-10-06T11:36:08.846553+0000","last_active":"2023-10-06T11:36:08.850850+0000","last_peered":"2023-10-06T11:36:08.850850+0000","last_clean":"2023-10-06T11:36:08.850850+0000","last_became_active":"2023-10-06T11:36:08.846228+0000","last_became_peered":"2023-10-06T11:36:08.846228+0000","last_unstale":"2023-10-06T11:36:08.850850+0000","last_undegraded":"2023-10-06T11:36:08.850850+0000","last_fullsized":"2023-10-06T11:36:08.850850+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849890+0000","last_change":"2023-10-06T11:36:08.846081+0000","last_active":"2023-10-06T11:36:08.849890+0000","last_peered":"2023-10-06T11:36:08.849890+0000","last_clean":"2023-10-06T11:36:08.849890+0000","last_became_active":"2023-10-06T11:36:08.844160+0000","last_became_peered":"2023-10-06T11:36:08.844160+0000","last_unstale":"2023-10-06T11:36:08.849890+0000","last_undegraded":"2023-10-06T11:36:08.849890+0000","last_fullsized":"2023-10-06T11:36:08.849890+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:32.586598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835394+0000","last_change":"2023-10-06T11:36:08.832132+0000","last_active":"2023-10-06T11:36:08.835394+0000","last_peered":"2023-10-06T11:36:08.835394+0000","last_clean":"2023-10-06T11:36:08.835394+0000","last_became_active":"2023-10-06T11:36:08.831813+0000","last_became_peered":"2023-10-06T11:36:08.831813+0000","last_unstale":"2023-10-06T11:36:08.835394+0000","last_undegraded":"2023-10-06T11:36:08.835394+0000","last_fullsized":"2023-10-06T11:36:08.835394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847126+0000","last_change":"2023-10-06T11:36:08.839463+0000","last_active":"2023-10-06T11:36:08.847126+0000","last_peered":"2023-10-06T11:36:08.847126+0000","last_clean":"2023-10-06T11:36:08.847126+0000","last_became_active":"2023-10-06T11:36:08.839166+0000","last_became_peered":"2023-10-06T11:36:08.839166+0000","last_unstale":"2023-10-06T11:36:08.847126+0000","last_undegraded":"2023-10-06T11:36:08.847126+0000","last_fullsized":"2023-10-06T11:36:08.847126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:41.417191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848083+0000","last_change":"2023-10-06T11:36:08.845554+0000","last_active":"2023-10-06T11:36:08.848083+0000","last_peered":"2023-10-06T11:36:08.848083+0000","last_clean":"2023-10-06T11:36:08.848083+0000","last_became_active":"2023-10-06T11:36:08.844765+0000","last_became_peered":"2023-10-06T11:36:08.844765+0000","last_unstale":"2023-10-06T11:36:08.848083+0000","last_undegraded":"2023-10-06T11:36:08.848083+0000","last_fullsized":"2023-10-06T11:36:08.848083+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:56.739651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834928+0000","last_change":"2023-10-06T11:36:08.829614+0000","last_active":"2023-10-06T11:36:08.834928+0000","last_peered":"2023-10-06T11:36:08.834928+0000","last_clean":"2023-10-06T11:36:08.834928+0000","last_became_active":"2023-10-06T11:36:08.829269+0000","last_became_peered":"2023-10-06T11:36:08.829269+0000","last_unstale":"2023-10-06T11:36:08.834928+0000","last_undegraded":"2023-10-06T11:36:08.834928+0000","last_fullsized":"2023-10-06T11:36:08.834928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847934+0000","last_change":"2023-10-06T11:36:08.845000+0000","last_active":"2023-10-06T11:36:08.847934+0000","last_peered":"2023-10-06T11:36:08.847934+0000","last_clean":"2023-10-06T11:36:08.847934+0000","last_became_active":"2023-10-06T11:36:08.844706+0000","last_became_peered":"2023-10-06T11:36:08.844706+0000","last_unstale":"2023-10-06T11:36:08.847934+0000","last_undegraded":"2023-10-06T11:36:08.847934+0000","last_fullsized":"2023-10-06T11:36:08.847934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849167+0000","last_change":"2023-10-06T11:36:08.847706+0000","last_active":"2023-10-06T11:36:08.849167+0000","last_peered":"2023-10-06T11:36:08.849167+0000","last_clean":"2023-10-06T11:36:08.849167+0000","last_became_active":"2023-10-06T11:36:08.847386+0000","last_became_peered":"2023-10-06T11:36:08.847386+0000","last_unstale":"2023-10-06T11:36:08.849167+0000","last_undegraded":"2023-10-06T11:36:08.849167+0000","last_fullsized":"2023-10-06T11:36:08.849167+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.844371+0000","last_change":"2023-10-06T11:36:08.840984+0000","last_active":"2023-10-06T11:36:08.844371+0000","last_peered":"2023-10-06T11:36:08.844371+0000","last_clean":"2023-10-06T11:36:08.844371+0000","last_became_active":"2023-10-06T11:36:08.840669+0000","last_became_peered":"2023-10-06T11:36:08.840669+0000","last_unstale":"2023-10-06T11:36:08.844371+0000","last_undegraded":"2023-10-06T11:36:08.844371+0000","last_fullsized":"2023-10-06T11:36:08.844371+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.821913+0000","last_change":"2023-10-06T11:36:08.820676+0000","last_active":"2023-10-06T11:36:08.821913+0000","last_peered":"2023-10-06T11:36:08.821913+0000","last_clean":"2023-10-06T11:36:08.821913+0000","last_became_active":"2023-10-06T11:36:08.820319+0000","last_became_peered":"2023-10-06T11:36:08.820319+0000","last_unstale":"2023-10-06T11:36:08.821913+0000","last_undegraded":"2023-10-06T11:36:08.821913+0000","last_fullsized":"2023-10-06T11:36:08.821913+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:53:05.714850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850475+0000","last_change":"2023-10-06T11:36:08.847440+0000","last_active":"2023-10-06T11:36:08.850475+0000","last_peered":"2023-10-06T11:36:08.850475+0000","last_clean":"2023-10-06T11:36:08.850475+0000","last_became_active":"2023-10-06T11:36:08.844060+0000","last_became_peered":"2023-10-06T11:36:08.844060+0000","last_unstale":"2023-10-06T11:36:08.850475+0000","last_undegraded":"2023-10-06T11:36:08.850475+0000","last_fullsized":"2023-10-06T11:36:08.850475+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:10.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835909+0000","last_change":"2023-10-06T11:36:08.833419+0000","last_active":"2023-10-06T11:36:08.835909+0000","last_peered":"2023-10-06T11:36:08.835909+0000","last_clean":"2023-10-06T11:36:08.835909+0000","last_became_active":"2023-10-06T11:36:08.832970+0000","last_became_peered":"2023-10-06T11:36:08.832970+0000","last_unstale":"2023-10-06T11:36:08.835909+0000","last_undegraded":"2023-10-06T11:36:08.835909+0000","last_fullsized":"2023-10-06T11:36:08.835909+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:28:24.073017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843554+0000","last_change":"2023-10-06T11:36:08.839073+0000","last_active":"2023-10-06T11:36:08.843554+0000","last_peered":"2023-10-06T11:36:08.843554+0000","last_clean":"2023-10-06T11:36:08.843554+0000","last_became_active":"2023-10-06T11:36:08.838729+0000","last_became_peered":"2023-10-06T11:36:08.838729+0000","last_unstale":"2023-10-06T11:36:08.843554+0000","last_undegraded":"2023-10-06T11:36:08.843554+0000","last_fullsized":"2023-10-06T11:36:08.843554+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:43.453476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847734+0000","last_change":"2023-10-06T11:36:08.834706+0000","last_active":"2023-10-06T11:36:08.847734+0000","last_peered":"2023-10-06T11:36:08.847734+0000","last_clean":"2023-10-06T11:36:08.847734+0000","last_became_active":"2023-10-06T11:36:08.834392+0000","last_became_peered":"2023-10-06T11:36:08.834392+0000","last_unstale":"2023-10-06T11:36:08.847734+0000","last_undegraded":"2023-10-06T11:36:08.847734+0000","last_fullsized":"2023-10-06T11:36:08.847734+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835267+0000","last_change":"2023-10-06T11:36:08.827263+0000","last_active":"2023-10-06T11:36:08.835267+0000","last_peered":"2023-10-06T11:36:08.835267+0000","last_clean":"2023-10-06T11:36:08.835267+0000","last_became_active":"2023-10-06T11:36:08.826927+0000","last_became_peered":"2023-10-06T11:36:08.826927+0000","last_unstale":"2023-10-06T11:36:08.835267+0000","last_undegraded":"2023-10-06T11:36:08.835267+0000","last_fullsized":"2023-10-06T11:36:08.835267+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:09.477110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832710+0000","last_change":"2023-10-06T11:36:08.825112+0000","last_active":"2023-10-06T11:36:08.832710+0000","last_peered":"2023-10-06T11:36:08.832710+0000","last_clean":"2023-10-06T11:36:08.832710+0000","last_became_active":"2023-10-06T11:36:08.824817+0000","last_became_peered":"2023-10-06T11:36:08.824817+0000","last_unstale":"2023-10-06T11:36:08.832710+0000","last_undegraded":"2023-10-06T11:36:08.832710+0000","last_fullsized":"2023-10-06T11:36:08.832710+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:47.661809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840092+0000","last_change":"2023-10-06T11:36:08.833482+0000","last_active":"2023-10-06T11:36:08.840092+0000","last_peered":"2023-10-06T11:36:08.840092+0000","last_clean":"2023-10-06T11:36:08.840092+0000","last_became_active":"2023-10-06T11:36:08.833172+0000","last_became_peered":"2023-10-06T11:36:08.833172+0000","last_unstale":"2023-10-06T11:36:08.840092+0000","last_undegraded":"2023-10-06T11:36:08.840092+0000","last_fullsized":"2023-10-06T11:36:08.840092+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837751+0000","last_change":"2023-10-06T11:36:08.834331+0000","last_active":"2023-10-06T11:36:08.837751+0000","last_peered":"2023-10-06T11:36:08.837751+0000","last_clean":"2023-10-06T11:36:08.837751+0000","last_became_active":"2023-10-06T11:36:08.834010+0000","last_became_peered":"2023-10-06T11:36:08.834010+0000","last_unstale":"2023-10-06T11:36:08.837751+0000","last_undegraded":"2023-10-06T11:36:08.837751+0000","last_fullsized":"2023-10-06T11:36:08.837751+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:37.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843481+0000","last_change":"2023-10-06T11:36:08.839346+0000","last_active":"2023-10-06T11:36:08.843481+0000","last_peered":"2023-10-06T11:36:08.843481+0000","last_clean":"2023-10-06T11:36:08.843481+0000","last_became_active":"2023-10-06T11:36:08.839069+0000","last_became_peered":"2023-10-06T11:36:08.839069+0000","last_unstale":"2023-10-06T11:36:08.843481+0000","last_undegraded":"2023-10-06T11:36:08.843481+0000","last_fullsized":"2023-10-06T11:36:08.843481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:43.156987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847105+0000","last_change":"2023-10-06T11:36:08.838625+0000","last_active":"2023-10-06T11:36:08.847105+0000","last_peered":"2023-10-06T11:36:08.847105+0000","last_clean":"2023-10-06T11:36:08.847105+0000","last_became_active":"2023-10-06T11:36:08.838342+0000","last_became_peered":"2023-10-06T11:36:08.838342+0000","last_unstale":"2023-10-06T11:36:08.847105+0000","last_undegraded":"2023-10-06T11:36:08.847105+0000","last_fullsized":"2023-10-06T11:36:08.847105+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849748+0000","last_change":"2023-10-06T11:36:08.843721+0000","last_active":"2023-10-06T11:36:08.849748+0000","last_peered":"2023-10-06T11:36:08.849748+0000","last_clean":"2023-10-06T11:36:08.849748+0000","last_became_active":"2023-10-06T11:36:08.843407+0000","last_became_peered":"2023-10-06T11:36:08.843407+0000","last_unstale":"2023-10-06T11:36:08.849748+0000","last_undegraded":"2023-10-06T11:36:08.849748+0000","last_fullsized":"2023-10-06T11:36:08.849748+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847711+0000","last_change":"2023-10-06T11:36:08.830867+0000","last_active":"2023-10-06T11:36:08.847711+0000","last_peered":"2023-10-06T11:36:08.847711+0000","last_clean":"2023-10-06T11:36:08.847711+0000","last_became_active":"2023-10-06T11:36:08.830226+0000","last_became_peered":"2023-10-06T11:36:08.830226+0000","last_unstale":"2023-10-06T11:36:08.847711+0000","last_undegraded":"2023-10-06T11:36:08.847711+0000","last_fullsized":"2023-10-06T11:36:08.847711+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835740+0000","last_change":"2023-10-06T11:36:08.833495+0000","last_active":"2023-10-06T11:36:08.835740+0000","last_peered":"2023-10-06T11:36:08.835740+0000","last_clean":"2023-10-06T11:36:08.835740+0000","last_became_active":"2023-10-06T11:36:08.833119+0000","last_became_peered":"2023-10-06T11:36:08.833119+0000","last_unstale":"2023-10-06T11:36:08.835740+0000","last_undegraded":"2023-10-06T11:36:08.835740+0000","last_fullsized":"2023-10-06T11:36:08.835740+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:49.438750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849754+0000","last_change":"2023-10-06T11:36:08.843345+0000","last_active":"2023-10-06T11:36:08.849754+0000","last_peered":"2023-10-06T11:36:08.849754+0000","last_clean":"2023-10-06T11:36:08.849754+0000","last_became_active":"2023-10-06T11:36:08.842930+0000","last_became_peered":"2023-10-06T11:36:08.842930+0000","last_unstale":"2023-10-06T11:36:08.849754+0000","last_undegraded":"2023-10-06T11:36:08.849754+0000","last_fullsized":"2023-10-06T11:36:08.849754+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:25.950732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827000+0000","last_change":"2023-10-06T11:36:08.825532+0000","last_active":"2023-10-06T11:36:08.827000+0000","last_peered":"2023-10-06T11:36:08.827000+0000","last_clean":"2023-10-06T11:36:08.827000+0000","last_became_active":"2023-10-06T11:36:08.825246+0000","last_became_peered":"2023-10-06T11:36:08.825246+0000","last_unstale":"2023-10-06T11:36:08.827000+0000","last_undegraded":"2023-10-06T11:36:08.827000+0000","last_fullsized":"2023-10-06T11:36:08.827000+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837616+0000","last_change":"2023-10-06T11:36:08.832215+0000","last_active":"2023-10-06T11:36:08.837616+0000","last_peered":"2023-10-06T11:36:08.837616+0000","last_clean":"2023-10-06T11:36:08.837616+0000","last_became_active":"2023-10-06T11:36:08.831906+0000","last_became_peered":"2023-10-06T11:36:08.831906+0000","last_unstale":"2023-10-06T11:36:08.837616+0000","last_undegraded":"2023-10-06T11:36:08.837616+0000","last_fullsized":"2023-10-06T11:36:08.837616+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:21.848572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"121'4","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.854694+0000","last_change":"2023-10-06T11:36:07.836398+0000","last_active":"2023-10-06T11:36:08.854694+0000","last_peered":"2023-10-06T11:36:08.854694+0000","last_clean":"2023-10-06T11:36:08.854694+0000","last_became_active":"2023-10-06T11:36:07.836063+0000","last_became_peered":"2023-10-06T11:36:07.836063+0000","last_unstale":"2023-10-06T11:36:08.854694+0000","last_undegraded":"2023-10-06T11:36:08.854694+0000","last_fullsized":"2023-10-06T11:36:08.854694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850887+0000","last_change":"2023-10-06T11:36:08.845254+0000","last_active":"2023-10-06T11:36:08.850887+0000","last_peered":"2023-10-06T11:36:08.850887+0000","last_clean":"2023-10-06T11:36:08.850887+0000","last_became_active":"2023-10-06T11:36:08.844729+0000","last_became_peered":"2023-10-06T11:36:08.844729+0000","last_unstale":"2023-10-06T11:36:08.850887+0000","last_undegraded":"2023-10-06T11:36:08.850887+0000","last_fullsized":"2023-10-06T11:36:08.850887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:46.014384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857089+0000","last_change":"2023-10-06T11:36:07.830544+0000","last_active":"2023-10-06T11:36:08.857089+0000","last_peered":"2023-10-06T11:36:08.857089+0000","last_clean":"2023-10-06T11:36:08.857089+0000","last_became_active":"2023-10-06T11:36:07.830229+0000","last_became_peered":"2023-10-06T11:36:07.830229+0000","last_unstale":"2023-10-06T11:36:08.857089+0000","last_undegraded":"2023-10-06T11:36:08.857089+0000","last_fullsized":"2023-10-06T11:36:08.857089+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842017+0000","last_change":"2023-10-06T11:36:08.837123+0000","last_active":"2023-10-06T11:36:08.842017+0000","last_peered":"2023-10-06T11:36:08.842017+0000","last_clean":"2023-10-06T11:36:08.842017+0000","last_became_active":"2023-10-06T11:36:08.836794+0000","last_became_peered":"2023-10-06T11:36:08.836794+0000","last_unstale":"2023-10-06T11:36:08.842017+0000","last_undegraded":"2023-10-06T11:36:08.842017+0000","last_fullsized":"2023-10-06T11:36:08.842017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:34.957119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864720+0000","last_change":"2023-10-06T11:36:07.843540+0000","last_active":"2023-10-06T11:36:08.864720+0000","last_peered":"2023-10-06T11:36:08.864720+0000","last_clean":"2023-10-06T11:36:08.864720+0000","last_became_active":"2023-10-06T11:36:07.843215+0000","last_became_peered":"2023-10-06T11:36:07.843215+0000","last_unstale":"2023-10-06T11:36:08.864720+0000","last_undegraded":"2023-10-06T11:36:08.864720+0000","last_fullsized":"2023-10-06T11:36:08.864720+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:22.066116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842320+0000","last_change":"2023-10-06T11:36:08.828471+0000","last_active":"2023-10-06T11:36:08.842320+0000","last_peered":"2023-10-06T11:36:08.842320+0000","last_clean":"2023-10-06T11:36:08.842320+0000","last_became_active":"2023-10-06T11:36:08.828169+0000","last_became_peered":"2023-10-06T11:36:08.828169+0000","last_unstale":"2023-10-06T11:36:08.842320+0000","last_undegraded":"2023-10-06T11:36:08.842320+0000","last_fullsized":"2023-10-06T11:36:08.842320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:32.232594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.855976+0000","last_change":"2023-10-06T11:36:07.834816+0000","last_active":"2023-10-06T11:36:08.855976+0000","last_peered":"2023-10-06T11:36:08.855976+0000","last_clean":"2023-10-06T11:36:08.855976+0000","last_became_active":"2023-10-06T11:36:07.834503+0000","last_became_peered":"2023-10-06T11:36:07.834503+0000","last_unstale":"2023-10-06T11:36:08.855976+0000","last_undegraded":"2023-10-06T11:36:08.855976+0000","last_fullsized":"2023-10-06T11:36:08.855976+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841854+0000","last_change":"2023-10-06T11:36:08.834817+0000","last_active":"2023-10-06T11:36:08.841854+0000","last_peered":"2023-10-06T11:36:08.841854+0000","last_clean":"2023-10-06T11:36:08.841854+0000","last_became_active":"2023-10-06T11:36:08.834510+0000","last_became_peered":"2023-10-06T11:36:08.834510+0000","last_unstale":"2023-10-06T11:36:08.841854+0000","last_undegraded":"2023-10-06T11:36:08.841854+0000","last_fullsized":"2023-10-06T11:36:08.841854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861794+0000","last_change":"2023-10-06T11:36:07.842608+0000","last_active":"2023-10-06T11:36:08.861794+0000","last_peered":"2023-10-06T11:36:08.861794+0000","last_clean":"2023-10-06T11:36:08.861794+0000","last_became_active":"2023-10-06T11:36:07.842325+0000","last_became_peered":"2023-10-06T11:36:07.842325+0000","last_unstale":"2023-10-06T11:36:08.861794+0000","last_undegraded":"2023-10-06T11:36:08.861794+0000","last_fullsized":"2023-10-06T11:36:08.861794+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835929+0000","last_change":"2023-10-06T11:36:08.833625+0000","last_active":"2023-10-06T11:36:08.835929+0000","last_peered":"2023-10-06T11:36:08.835929+0000","last_clean":"2023-10-06T11:36:08.835929+0000","last_became_active":"2023-10-06T11:36:08.833132+0000","last_became_peered":"2023-10-06T11:36:08.833132+0000","last_unstale":"2023-10-06T11:36:08.835929+0000","last_undegraded":"2023-10-06T11:36:08.835929+0000","last_fullsized":"2023-10-06T11:36:08.835929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.375194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861213+0000","last_change":"2023-10-06T11:36:07.836739+0000","last_active":"2023-10-06T11:36:08.861213+0000","last_peered":"2023-10-06T11:36:08.861213+0000","last_clean":"2023-10-06T11:36:08.861213+0000","last_became_active":"2023-10-06T11:36:07.836402+0000","last_became_peered":"2023-10-06T11:36:07.836402+0000","last_unstale":"2023-10-06T11:36:08.861213+0000","last_undegraded":"2023-10-06T11:36:08.861213+0000","last_fullsized":"2023-10-06T11:36:08.861213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:13.703496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849866+0000","last_change":"2023-10-06T11:36:08.845779+0000","last_active":"2023-10-06T11:36:08.849866+0000","last_peered":"2023-10-06T11:36:08.849866+0000","last_clean":"2023-10-06T11:36:08.849866+0000","last_became_active":"2023-10-06T11:36:08.843903+0000","last_became_peered":"2023-10-06T11:36:08.843903+0000","last_unstale":"2023-10-06T11:36:08.849866+0000","last_undegraded":"2023-10-06T11:36:08.849866+0000","last_fullsized":"2023-10-06T11:36:08.849866+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:49.631159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824932+0000","last_change":"2023-10-06T11:36:07.842155+0000","last_active":"2023-10-06T11:36:08.824932+0000","last_peered":"2023-10-06T11:36:08.824932+0000","last_clean":"2023-10-06T11:36:08.824932+0000","last_became_active":"2023-10-06T11:36:07.841850+0000","last_became_peered":"2023-10-06T11:36:07.841850+0000","last_unstale":"2023-10-06T11:36:08.824932+0000","last_undegraded":"2023-10-06T11:36:08.824932+0000","last_fullsized":"2023-10-06T11:36:08.824932+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:09.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833946+0000","last_change":"2023-10-06T11:36:08.822864+0000","last_active":"2023-10-06T11:36:08.833946+0000","last_peered":"2023-10-06T11:36:08.833946+0000","last_clean":"2023-10-06T11:36:08.833946+0000","last_became_active":"2023-10-06T11:36:08.822567+0000","last_became_peered":"2023-10-06T11:36:08.822567+0000","last_unstale":"2023-10-06T11:36:08.833946+0000","last_undegraded":"2023-10-06T11:36:08.833946+0000","last_fullsized":"2023-10-06T11:36:08.833946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827606+0000","last_change":"2023-10-06T11:36:07.838208+0000","last_active":"2023-10-06T11:36:08.827606+0000","last_peered":"2023-10-06T11:36:08.827606+0000","last_clean":"2023-10-06T11:36:08.827606+0000","last_became_active":"2023-10-06T11:36:07.837843+0000","last_became_peered":"2023-10-06T11:36:07.837843+0000","last_unstale":"2023-10-06T11:36:08.827606+0000","last_undegraded":"2023-10-06T11:36:08.827606+0000","last_fullsized":"2023-10-06T11:36:08.827606+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837595+0000","last_change":"2023-10-06T11:36:08.831541+0000","last_active":"2023-10-06T11:36:08.837595+0000","last_peered":"2023-10-06T11:36:08.837595+0000","last_clean":"2023-10-06T11:36:08.837595+0000","last_became_active":"2023-10-06T11:36:08.831234+0000","last_became_peered":"2023-10-06T11:36:08.831234+0000","last_unstale":"2023-10-06T11:36:08.837595+0000","last_undegraded":"2023-10-06T11:36:08.837595+0000","last_fullsized":"2023-10-06T11:36:08.837595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:26.203324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825150+0000","last_change":"2023-10-06T11:36:07.844770+0000","last_active":"2023-10-06T11:36:08.825150+0000","last_peered":"2023-10-06T11:36:08.825150+0000","last_clean":"2023-10-06T11:36:08.825150+0000","last_became_active":"2023-10-06T11:36:07.843489+0000","last_became_peered":"2023-10-06T11:36:07.843489+0000","last_unstale":"2023-10-06T11:36:08.825150+0000","last_undegraded":"2023-10-06T11:36:08.825150+0000","last_fullsized":"2023-10-06T11:36:08.825150+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:55.205873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835302+0000","last_change":"2023-10-06T11:36:08.828494+0000","last_active":"2023-10-06T11:36:08.835302+0000","last_peered":"2023-10-06T11:36:08.835302+0000","last_clean":"2023-10-06T11:36:08.835302+0000","last_became_active":"2023-10-06T11:36:08.828149+0000","last_became_peered":"2023-10-06T11:36:08.828149+0000","last_unstale":"2023-10-06T11:36:08.835302+0000","last_undegraded":"2023-10-06T11:36:08.835302+0000","last_fullsized":"2023-10-06T11:36:08.835302+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:25.498441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810127+0000","last_change":"2023-10-06T11:36:07.831652+0000","last_active":"2023-10-06T11:36:08.810127+0000","last_peered":"2023-10-06T11:36:08.810127+0000","last_clean":"2023-10-06T11:36:08.810127+0000","last_became_active":"2023-10-06T11:36:07.831331+0000","last_became_peered":"2023-10-06T11:36:07.831331+0000","last_unstale":"2023-10-06T11:36:08.810127+0000","last_undegraded":"2023-10-06T11:36:08.810127+0000","last_fullsized":"2023-10-06T11:36:08.810127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:30:03.936003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850794+0000","last_change":"2023-10-06T11:36:08.843325+0000","last_active":"2023-10-06T11:36:08.850794+0000","last_peered":"2023-10-06T11:36:08.850794+0000","last_clean":"2023-10-06T11:36:08.850794+0000","last_became_active":"2023-10-06T11:36:08.843032+0000","last_became_peered":"2023-10-06T11:36:08.843032+0000","last_unstale":"2023-10-06T11:36:08.850794+0000","last_undegraded":"2023-10-06T11:36:08.850794+0000","last_fullsized":"2023-10-06T11:36:08.850794+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:50:13.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"121'3","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861425+0000","last_change":"2023-10-06T11:36:07.845696+0000","last_active":"2023-10-06T11:36:08.861425+0000","last_peered":"2023-10-06T11:36:08.861425+0000","last_clean":"2023-10-06T11:36:08.861425+0000","last_became_active":"2023-10-06T11:36:07.845420+0000","last_became_peered":"2023-10-06T11:36:07.845420+0000","last_unstale":"2023-10-06T11:36:08.861425+0000","last_undegraded":"2023-10-06T11:36:08.861425+0000","last_fullsized":"2023-10-06T11:36:08.861425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837117+0000","last_change":"2023-10-06T11:36:08.827973+0000","last_active":"2023-10-06T11:36:08.837117+0000","last_peered":"2023-10-06T11:36:08.837117+0000","last_clean":"2023-10-06T11:36:08.837117+0000","last_became_active":"2023-10-06T11:36:08.827650+0000","last_became_peered":"2023-10-06T11:36:08.827650+0000","last_unstale":"2023-10-06T11:36:08.837117+0000","last_undegraded":"2023-10-06T11:36:08.837117+0000","last_fullsized":"2023-10-06T11:36:08.837117+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:48.124515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"121'6","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.865336+0000","last_change":"2023-10-06T11:36:07.845721+0000","last_active":"2023-10-06T11:36:08.865336+0000","last_peered":"2023-10-06T11:36:08.865336+0000","last_clean":"2023-10-06T11:36:08.865336+0000","last_became_active":"2023-10-06T11:36:07.845389+0000","last_became_peered":"2023-10-06T11:36:07.845389+0000","last_unstale":"2023-10-06T11:36:08.865336+0000","last_undegraded":"2023-10-06T11:36:08.865336+0000","last_fullsized":"2023-10-06T11:36:08.865336+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842565+0000","last_change":"2023-10-06T11:36:08.831561+0000","last_active":"2023-10-06T11:36:08.842565+0000","last_peered":"2023-10-06T11:36:08.842565+0000","last_clean":"2023-10-06T11:36:08.842565+0000","last_became_active":"2023-10-06T11:36:08.831286+0000","last_became_peered":"2023-10-06T11:36:08.831286+0000","last_unstale":"2023-10-06T11:36:08.842565+0000","last_undegraded":"2023-10-06T11:36:08.842565+0000","last_fullsized":"2023-10-06T11:36:08.842565+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857133+0000","last_change":"2023-10-06T11:36:07.836509+0000","last_active":"2023-10-06T11:36:08.857133+0000","last_peered":"2023-10-06T11:36:08.857133+0000","last_clean":"2023-10-06T11:36:08.857133+0000","last_became_active":"2023-10-06T11:36:07.835626+0000","last_became_peered":"2023-10-06T11:36:07.835626+0000","last_unstale":"2023-10-06T11:36:08.857133+0000","last_undegraded":"2023-10-06T11:36:08.857133+0000","last_fullsized":"2023-10-06T11:36:08.857133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849972+0000","last_change":"2023-10-06T11:36:08.846593+0000","last_active":"2023-10-06T11:36:08.849972+0000","last_peered":"2023-10-06T11:36:08.849972+0000","last_clean":"2023-10-06T11:36:08.849972+0000","last_became_active":"2023-10-06T11:36:08.846204+0000","last_became_peered":"2023-10-06T11:36:08.846204+0000","last_unstale":"2023-10-06T11:36:08.849972+0000","last_undegraded":"2023-10-06T11:36:08.849972+0000","last_fullsized":"2023-10-06T11:36:08.849972+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:46.893794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.830870+0000","last_change":"2023-10-06T11:36:07.833180+0000","last_active":"2023-10-06T11:36:08.830870+0000","last_peered":"2023-10-06T11:36:08.830870+0000","last_clean":"2023-10-06T11:36:08.830870+0000","last_became_active":"2023-10-06T11:36:07.832883+0000","last_became_peered":"2023-10-06T11:36:07.832883+0000","last_unstale":"2023-10-06T11:36:08.830870+0000","last_undegraded":"2023-10-06T11:36:08.830870+0000","last_fullsized":"2023-10-06T11:36:08.830870+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827807+0000","last_change":"2023-10-06T11:36:08.826278+0000","last_active":"2023-10-06T11:36:08.827807+0000","last_peered":"2023-10-06T11:36:08.827807+0000","last_clean":"2023-10-06T11:36:08.827807+0000","last_became_active":"2023-10-06T11:36:08.825985+0000","last_became_peered":"2023-10-06T11:36:08.825985+0000","last_unstale":"2023-10-06T11:36:08.827807+0000","last_undegraded":"2023-10-06T11:36:08.827807+0000","last_fullsized":"2023-10-06T11:36:08.827807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820190+0000","last_change":"2023-10-06T11:36:07.840409+0000","last_active":"2023-10-06T11:36:08.820190+0000","last_peered":"2023-10-06T11:36:08.820190+0000","last_clean":"2023-10-06T11:36:08.820190+0000","last_became_active":"2023-10-06T11:36:07.840059+0000","last_became_peered":"2023-10-06T11:36:07.840059+0000","last_unstale":"2023-10-06T11:36:08.820190+0000","last_undegraded":"2023-10-06T11:36:08.820190+0000","last_fullsized":"2023-10-06T11:36:08.820190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:54.481491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832875+0000","last_change":"2023-10-06T11:36:08.826025+0000","last_active":"2023-10-06T11:36:08.832875+0000","last_peered":"2023-10-06T11:36:08.832875+0000","last_clean":"2023-10-06T11:36:08.832875+0000","last_became_active":"2023-10-06T11:36:08.825431+0000","last_became_peered":"2023-10-06T11:36:08.825431+0000","last_unstale":"2023-10-06T11:36:08.832875+0000","last_undegraded":"2023-10-06T11:36:08.832875+0000","last_fullsized":"2023-10-06T11:36:08.832875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:56.221716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"121'8","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864905+0000","last_change":"2023-10-06T11:36:07.844146+0000","last_active":"2023-10-06T11:36:08.864905+0000","last_peered":"2023-10-06T11:36:08.864905+0000","last_clean":"2023-10-06T11:36:08.864905+0000","last_became_active":"2023-10-06T11:36:07.843809+0000","last_became_peered":"2023-10-06T11:36:07.843809+0000","last_unstale":"2023-10-06T11:36:08.864905+0000","last_undegraded":"2023-10-06T11:36:08.864905+0000","last_fullsized":"2023-10-06T11:36:08.864905+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839926+0000","last_change":"2023-10-06T11:36:08.837343+0000","last_active":"2023-10-06T11:36:08.839926+0000","last_peered":"2023-10-06T11:36:08.839926+0000","last_clean":"2023-10-06T11:36:08.839926+0000","last_became_active":"2023-10-06T11:36:08.835511+0000","last_became_peered":"2023-10-06T11:36:08.835511+0000","last_unstale":"2023-10-06T11:36:08.839926+0000","last_undegraded":"2023-10-06T11:36:08.839926+0000","last_fullsized":"2023-10-06T11:36:08.839926+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:20.565177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857120+0000","last_change":"2023-10-06T11:36:07.832997+0000","last_active":"2023-10-06T11:36:08.857120+0000","last_peered":"2023-10-06T11:36:08.857120+0000","last_clean":"2023-10-06T11:36:08.857120+0000","last_became_active":"2023-10-06T11:36:07.832693+0000","last_became_peered":"2023-10-06T11:36:07.832693+0000","last_unstale":"2023-10-06T11:36:08.857120+0000","last_undegraded":"2023-10-06T11:36:08.857120+0000","last_fullsized":"2023-10-06T11:36:08.857120+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833519+0000","last_change":"2023-10-06T11:36:08.831230+0000","last_active":"2023-10-06T11:36:08.833519+0000","last_peered":"2023-10-06T11:36:08.833519+0000","last_clean":"2023-10-06T11:36:08.833519+0000","last_became_active":"2023-10-06T11:36:08.830920+0000","last_became_peered":"2023-10-06T11:36:08.830920+0000","last_unstale":"2023-10-06T11:36:08.833519+0000","last_undegraded":"2023-10-06T11:36:08.833519+0000","last_fullsized":"2023-10-06T11:36:08.833519+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:01.109540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810349+0000","last_change":"2023-10-06T11:36:07.833889+0000","last_active":"2023-10-06T11:36:08.810349+0000","last_peered":"2023-10-06T11:36:08.810349+0000","last_clean":"2023-10-06T11:36:08.810349+0000","last_became_active":"2023-10-06T11:36:07.833564+0000","last_became_peered":"2023-10-06T11:36:07.833564+0000","last_unstale":"2023-10-06T11:36:08.810349+0000","last_undegraded":"2023-10-06T11:36:08.810349+0000","last_fullsized":"2023-10-06T11:36:08.810349+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:43.919700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835073+0000","last_change":"2023-10-06T11:36:08.832266+0000","last_active":"2023-10-06T11:36:08.835073+0000","last_peered":"2023-10-06T11:36:08.835073+0000","last_clean":"2023-10-06T11:36:08.835073+0000","last_became_active":"2023-10-06T11:36:08.830568+0000","last_became_peered":"2023-10-06T11:36:08.830568+0000","last_unstale":"2023-10-06T11:36:08.835073+0000","last_undegraded":"2023-10-06T11:36:08.835073+0000","last_fullsized":"2023-10-06T11:36:08.835073+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836702+0000","last_change":"2023-10-06T11:36:07.847402+0000","last_active":"2023-10-06T11:36:08.836702+0000","last_peered":"2023-10-06T11:36:08.836702+0000","last_clean":"2023-10-06T11:36:08.836702+0000","last_became_active":"2023-10-06T11:36:07.847014+0000","last_became_peered":"2023-10-06T11:36:07.847014+0000","last_unstale":"2023-10-06T11:36:08.836702+0000","last_undegraded":"2023-10-06T11:36:08.836702+0000","last_fullsized":"2023-10-06T11:36:08.836702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847206+0000","last_change":"2023-10-06T11:36:08.837814+0000","last_active":"2023-10-06T11:36:08.847206+0000","last_peered":"2023-10-06T11:36:08.847206+0000","last_clean":"2023-10-06T11:36:08.847206+0000","last_became_active":"2023-10-06T11:36:08.837531+0000","last_became_peered":"2023-10-06T11:36:08.837531+0000","last_unstale":"2023-10-06T11:36:08.847206+0000","last_undegraded":"2023-10-06T11:36:08.847206+0000","last_fullsized":"2023-10-06T11:36:08.847206+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:56.220796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816582+0000","last_change":"2023-10-06T11:36:07.834704+0000","last_active":"2023-10-06T11:36:08.816582+0000","last_peered":"2023-10-06T11:36:08.816582+0000","last_clean":"2023-10-06T11:36:08.816582+0000","last_became_active":"2023-10-06T11:36:07.834381+0000","last_became_peered":"2023-10-06T11:36:07.834381+0000","last_unstale":"2023-10-06T11:36:08.816582+0000","last_undegraded":"2023-10-06T11:36:08.816582+0000","last_fullsized":"2023-10-06T11:36:08.816582+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850140+0000","last_change":"2023-10-06T11:36:08.836975+0000","last_active":"2023-10-06T11:36:08.850140+0000","last_peered":"2023-10-06T11:36:08.850140+0000","last_clean":"2023-10-06T11:36:08.850140+0000","last_became_active":"2023-10-06T11:36:08.836692+0000","last_became_peered":"2023-10-06T11:36:08.836692+0000","last_unstale":"2023-10-06T11:36:08.850140+0000","last_undegraded":"2023-10-06T11:36:08.850140+0000","last_fullsized":"2023-10-06T11:36:08.850140+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:19.467727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825222+0000","last_change":"2023-10-06T11:36:07.843092+0000","last_active":"2023-10-06T11:36:08.825222+0000","last_peered":"2023-10-06T11:36:08.825222+0000","last_clean":"2023-10-06T11:36:08.825222+0000","last_became_active":"2023-10-06T11:36:07.842729+0000","last_became_peered":"2023-10-06T11:36:07.842729+0000","last_unstale":"2023-10-06T11:36:08.825222+0000","last_undegraded":"2023-10-06T11:36:08.825222+0000","last_fullsized":"2023-10-06T11:36:08.825222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850838+0000","last_change":"2023-10-06T11:36:08.843940+0000","last_active":"2023-10-06T11:36:08.850838+0000","last_peered":"2023-10-06T11:36:08.850838+0000","last_clean":"2023-10-06T11:36:08.850838+0000","last_became_active":"2023-10-06T11:36:08.843666+0000","last_became_peered":"2023-10-06T11:36:08.843666+0000","last_unstale":"2023-10-06T11:36:08.850838+0000","last_undegraded":"2023-10-06T11:36:08.850838+0000","last_fullsized":"2023-10-06T11:36:08.850838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822603+0000","last_change":"2023-10-06T11:36:07.840413+0000","last_active":"2023-10-06T11:36:08.822603+0000","last_peered":"2023-10-06T11:36:08.822603+0000","last_clean":"2023-10-06T11:36:08.822603+0000","last_became_active":"2023-10-06T11:36:07.840091+0000","last_became_peered":"2023-10-06T11:36:07.840091+0000","last_unstale":"2023-10-06T11:36:08.822603+0000","last_undegraded":"2023-10-06T11:36:08.822603+0000","last_fullsized":"2023-10-06T11:36:08.822603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:25.925632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849923+0000","last_change":"2023-10-06T11:36:08.848796+0000","last_active":"2023-10-06T11:36:08.849923+0000","last_peered":"2023-10-06T11:36:08.849923+0000","last_clean":"2023-10-06T11:36:08.849923+0000","last_became_active":"2023-10-06T11:36:08.848494+0000","last_became_peered":"2023-10-06T11:36:08.848494+0000","last_unstale":"2023-10-06T11:36:08.849923+0000","last_undegraded":"2023-10-06T11:36:08.849923+0000","last_fullsized":"2023-10-06T11:36:08.849923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835941+0000","last_change":"2023-10-06T11:36:07.847520+0000","last_active":"2023-10-06T11:36:08.835941+0000","last_peered":"2023-10-06T11:36:08.835941+0000","last_clean":"2023-10-06T11:36:08.835941+0000","last_became_active":"2023-10-06T11:36:07.847113+0000","last_became_peered":"2023-10-06T11:36:07.847113+0000","last_unstale":"2023-10-06T11:36:08.835941+0000","last_undegraded":"2023-10-06T11:36:08.835941+0000","last_fullsized":"2023-10-06T11:36:08.835941+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:44.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847166+0000","last_change":"2023-10-06T11:36:08.839414+0000","last_active":"2023-10-06T11:36:08.847166+0000","last_peered":"2023-10-06T11:36:08.847166+0000","last_clean":"2023-10-06T11:36:08.847166+0000","last_became_active":"2023-10-06T11:36:08.839089+0000","last_became_peered":"2023-10-06T11:36:08.839089+0000","last_unstale":"2023-10-06T11:36:08.847166+0000","last_undegraded":"2023-10-06T11:36:08.847166+0000","last_fullsized":"2023-10-06T11:36:08.847166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819931+0000","last_change":"2023-10-06T11:36:07.841033+0000","last_active":"2023-10-06T11:36:08.819931+0000","last_peered":"2023-10-06T11:36:08.819931+0000","last_clean":"2023-10-06T11:36:08.819931+0000","last_became_active":"2023-10-06T11:36:07.840720+0000","last_became_peered":"2023-10-06T11:36:07.840720+0000","last_unstale":"2023-10-06T11:36:08.819931+0000","last_undegraded":"2023-10-06T11:36:08.819931+0000","last_fullsized":"2023-10-06T11:36:08.819931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842397+0000","last_change":"2023-10-06T11:36:08.830558+0000","last_active":"2023-10-06T11:36:08.842397+0000","last_peered":"2023-10-06T11:36:08.842397+0000","last_clean":"2023-10-06T11:36:08.842397+0000","last_became_active":"2023-10-06T11:36:08.830244+0000","last_became_peered":"2023-10-06T11:36:08.830244+0000","last_unstale":"2023-10-06T11:36:08.842397+0000","last_undegraded":"2023-10-06T11:36:08.842397+0000","last_fullsized":"2023-10-06T11:36:08.842397+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857037+0000","last_change":"2023-10-06T11:36:07.838666+0000","last_active":"2023-10-06T11:36:08.857037+0000","last_peered":"2023-10-06T11:36:08.857037+0000","last_clean":"2023-10-06T11:36:08.857037+0000","last_became_active":"2023-10-06T11:36:07.837999+0000","last_became_peered":"2023-10-06T11:36:07.837999+0000","last_unstale":"2023-10-06T11:36:08.857037+0000","last_undegraded":"2023-10-06T11:36:08.857037+0000","last_fullsized":"2023-10-06T11:36:08.857037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"111'227","reported_seq":641,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833386+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:36:08.833386+0000","last_peered":"2023-10-06T11:36:08.833386+0000","last_clean":"2023-10-06T11:36:08.833386+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:36:08.833386+0000","last_undegraded":"2023-10-06T11:36:08.833386+0000","last_fullsized":"2023-10-06T11:36:08.833386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824390+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:36:08.824390+0000","last_peered":"2023-10-06T11:36:08.824390+0000","last_clean":"2023-10-06T11:36:08.824390+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:36:08.824390+0000","last_undegraded":"2023-10-06T11:36:08.824390+0000","last_fullsized":"2023-10-06T11:36:08.824390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840213+0000","last_change":"2023-10-06T11:36:08.836795+0000","last_active":"2023-10-06T11:36:08.840213+0000","last_peered":"2023-10-06T11:36:08.840213+0000","last_clean":"2023-10-06T11:36:08.840213+0000","last_became_active":"2023-10-06T11:36:08.836163+0000","last_became_peered":"2023-10-06T11:36:08.836163+0000","last_unstale":"2023-10-06T11:36:08.840213+0000","last_undegraded":"2023-10-06T11:36:08.840213+0000","last_fullsized":"2023-10-06T11:36:08.840213+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:55.806624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.863443+0000","last_change":"2023-10-06T11:36:07.839247+0000","last_active":"2023-10-06T11:36:08.863443+0000","last_peered":"2023-10-06T11:36:08.863443+0000","last_clean":"2023-10-06T11:36:08.863443+0000","last_became_active":"2023-10-06T11:36:07.838584+0000","last_became_peered":"2023-10-06T11:36:07.838584+0000","last_unstale":"2023-10-06T11:36:08.863443+0000","last_undegraded":"2023-10-06T11:36:08.863443+0000","last_fullsized":"2023-10-06T11:36:08.863443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:48.616236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833289+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:36:08.833289+0000","last_peered":"2023-10-06T11:36:08.833289+0000","last_clean":"2023-10-06T11:36:08.833289+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:36:08.833289+0000","last_undegraded":"2023-10-06T11:36:08.833289+0000","last_fullsized":"2023-10-06T11:36:08.833289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837137+0000","last_change":"2023-10-06T11:36:08.831245+0000","last_active":"2023-10-06T11:36:08.837137+0000","last_peered":"2023-10-06T11:36:08.837137+0000","last_clean":"2023-10-06T11:36:08.837137+0000","last_became_active":"2023-10-06T11:36:08.830977+0000","last_became_peered":"2023-10-06T11:36:08.830977+0000","last_unstale":"2023-10-06T11:36:08.837137+0000","last_undegraded":"2023-10-06T11:36:08.837137+0000","last_fullsized":"2023-10-06T11:36:08.837137+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:49.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824163+0000","last_change":"2023-10-06T11:36:07.841765+0000","last_active":"2023-10-06T11:36:08.824163+0000","last_peered":"2023-10-06T11:36:08.824163+0000","last_clean":"2023-10-06T11:36:08.824163+0000","last_became_active":"2023-10-06T11:36:07.841405+0000","last_became_peered":"2023-10-06T11:36:07.841405+0000","last_unstale":"2023-10-06T11:36:08.824163+0000","last_undegraded":"2023-10-06T11:36:08.824163+0000","last_fullsized":"2023-10-06T11:36:08.824163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:26.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820278+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:36:08.820278+0000","last_peered":"2023-10-06T11:36:08.820278+0000","last_clean":"2023-10-06T11:36:08.820278+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:36:08.820278+0000","last_undegraded":"2023-10-06T11:36:08.820278+0000","last_fullsized":"2023-10-06T11:36:08.820278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847753+0000","last_change":"2023-10-06T11:36:08.834917+0000","last_active":"2023-10-06T11:36:08.847753+0000","last_peered":"2023-10-06T11:36:08.847753+0000","last_clean":"2023-10-06T11:36:08.847753+0000","last_became_active":"2023-10-06T11:36:08.834522+0000","last_became_peered":"2023-10-06T11:36:08.834522+0000","last_unstale":"2023-10-06T11:36:08.847753+0000","last_undegraded":"2023-10-06T11:36:08.847753+0000","last_fullsized":"2023-10-06T11:36:08.847753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.860358+0000","last_change":"2023-10-06T11:36:07.840773+0000","last_active":"2023-10-06T11:36:08.860358+0000","last_peered":"2023-10-06T11:36:08.860358+0000","last_clean":"2023-10-06T11:36:08.860358+0000","last_became_active":"2023-10-06T11:36:07.840415+0000","last_became_peered":"2023-10-06T11:36:07.840415+0000","last_unstale":"2023-10-06T11:36:08.860358+0000","last_undegraded":"2023-10-06T11:36:08.860358+0000","last_fullsized":"2023-10-06T11:36:08.860358+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824689+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:36:08.824689+0000","last_peered":"2023-10-06T11:36:08.824689+0000","last_clean":"2023-10-06T11:36:08.824689+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:36:08.824689+0000","last_undegraded":"2023-10-06T11:36:08.824689+0000","last_fullsized":"2023-10-06T11:36:08.824689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834204+0000","last_change":"2023-10-06T11:36:08.831735+0000","last_active":"2023-10-06T11:36:08.834204+0000","last_peered":"2023-10-06T11:36:08.834204+0000","last_clean":"2023-10-06T11:36:08.834204+0000","last_became_active":"2023-10-06T11:36:08.831450+0000","last_became_peered":"2023-10-06T11:36:08.831450+0000","last_unstale":"2023-10-06T11:36:08.834204+0000","last_undegraded":"2023-10-06T11:36:08.834204+0000","last_fullsized":"2023-10-06T11:36:08.834204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835829+0000","last_change":"2023-10-06T11:36:07.847157+0000","last_active":"2023-10-06T11:36:08.835829+0000","last_peered":"2023-10-06T11:36:08.835829+0000","last_clean":"2023-10-06T11:36:08.835829+0000","last_became_active":"2023-10-06T11:36:07.846801+0000","last_became_peered":"2023-10-06T11:36:07.846801+0000","last_unstale":"2023-10-06T11:36:08.835829+0000","last_undegraded":"2023-10-06T11:36:08.835829+0000","last_fullsized":"2023-10-06T11:36:08.835829+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:08.462187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816701+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:36:08.816701+0000","last_peered":"2023-10-06T11:36:08.816701+0000","last_clean":"2023-10-06T11:36:08.816701+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:36:08.816701+0000","last_undegraded":"2023-10-06T11:36:08.816701+0000","last_fullsized":"2023-10-06T11:36:08.816701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848126+0000","last_change":"2023-10-06T11:36:08.836395+0000","last_active":"2023-10-06T11:36:08.848126+0000","last_peered":"2023-10-06T11:36:08.848126+0000","last_clean":"2023-10-06T11:36:08.848126+0000","last_became_active":"2023-10-06T11:36:08.836112+0000","last_became_peered":"2023-10-06T11:36:08.836112+0000","last_unstale":"2023-10-06T11:36:08.848126+0000","last_undegraded":"2023-10-06T11:36:08.848126+0000","last_fullsized":"2023-10-06T11:36:08.848126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819833+0000","last_change":"2023-10-06T11:36:07.840551+0000","last_active":"2023-10-06T11:36:08.819833+0000","last_peered":"2023-10-06T11:36:08.819833+0000","last_clean":"2023-10-06T11:36:08.819833+0000","last_became_active":"2023-10-06T11:36:07.840210+0000","last_became_peered":"2023-10-06T11:36:07.840210+0000","last_unstale":"2023-10-06T11:36:08.819833+0000","last_undegraded":"2023-10-06T11:36:08.819833+0000","last_fullsized":"2023-10-06T11:36:08.819833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833204+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:36:08.833204+0000","last_peered":"2023-10-06T11:36:08.833204+0000","last_clean":"2023-10-06T11:36:08.833204+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:36:08.833204+0000","last_undegraded":"2023-10-06T11:36:08.833204+0000","last_fullsized":"2023-10-06T11:36:08.833204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839738+0000","last_change":"2023-10-06T11:36:08.836510+0000","last_active":"2023-10-06T11:36:08.839738+0000","last_peered":"2023-10-06T11:36:08.839738+0000","last_clean":"2023-10-06T11:36:08.839738+0000","last_became_active":"2023-10-06T11:36:08.836215+0000","last_became_peered":"2023-10-06T11:36:08.836215+0000","last_unstale":"2023-10-06T11:36:08.839738+0000","last_undegraded":"2023-10-06T11:36:08.839738+0000","last_fullsized":"2023-10-06T11:36:08.839738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:32.797025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857016+0000","last_change":"2023-10-06T11:36:07.841111+0000","last_active":"2023-10-06T11:36:08.857016+0000","last_peered":"2023-10-06T11:36:08.857016+0000","last_clean":"2023-10-06T11:36:08.857016+0000","last_became_active":"2023-10-06T11:36:07.840810+0000","last_became_peered":"2023-10-06T11:36:07.840810+0000","last_unstale":"2023-10-06T11:36:08.857016+0000","last_undegraded":"2023-10-06T11:36:08.857016+0000","last_fullsized":"2023-10-06T11:36:08.857016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:03.725992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.818400+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:36:08.818400+0000","last_peered":"2023-10-06T11:36:08.818400+0000","last_clean":"2023-10-06T11:36:08.818400+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:36:08.818400+0000","last_undegraded":"2023-10-06T11:36:08.818400+0000","last_fullsized":"2023-10-06T11:36:08.818400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837031+0000","last_change":"2023-10-06T11:36:08.833641+0000","last_active":"2023-10-06T11:36:08.837031+0000","last_peered":"2023-10-06T11:36:08.837031+0000","last_clean":"2023-10-06T11:36:08.837031+0000","last_became_active":"2023-10-06T11:36:08.833372+0000","last_became_peered":"2023-10-06T11:36:08.833372+0000","last_unstale":"2023-10-06T11:36:08.837031+0000","last_undegraded":"2023-10-06T11:36:08.837031+0000","last_fullsized":"2023-10-06T11:36:08.837031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:45.164451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822466+0000","last_change":"2023-10-06T11:36:07.841750+0000","last_active":"2023-10-06T11:36:08.822466+0000","last_peered":"2023-10-06T11:36:08.822466+0000","last_clean":"2023-10-06T11:36:08.822466+0000","last_became_active":"2023-10-06T11:36:07.841432+0000","last_became_peered":"2023-10-06T11:36:07.841432+0000","last_unstale":"2023-10-06T11:36:08.822466+0000","last_undegraded":"2023-10-06T11:36:08.822466+0000","last_fullsized":"2023-10-06T11:36:08.822466+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:27:06.372507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836500+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:36:08.836500+0000","last_peered":"2023-10-06T11:36:08.836500+0000","last_clean":"2023-10-06T11:36:08.836500+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:36:08.836500+0000","last_undegraded":"2023-10-06T11:36:08.836500+0000","last_fullsized":"2023-10-06T11:36:08.836500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575060,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":2109440,"data_stored":1371418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1020000000000001,"15min":1.1339999999999999},"min":{"1min":0.84499999999999997,"5min":0.59599999999999997,"15min":0.495},"max":{"1min":1.5249999999999999,"5min":1.6579999999999999,"15min":1.724},"last":1.224},{"interface":"front","average":{"1min":1.159,"5min":1.143,"15min":1.139},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.56399999999999995},"max":{"1min":1.5229999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":1.4410000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.359,"15min":1.333},"min":{"1min":0.78100000000000003,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.651,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.232},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5349999999999999,"5min":1.927,"15min":1.927},"last":1.1679999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2,"15min":1.214},"min":{"1min":1.006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6040000000000001,"5min":1.827,"15min":1.827},"last":1.335},{"interface":"front","average":{"1min":1.353,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.294}]},{"osd":3,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.256,"15min":1.264},"min":{"1min":0.92900000000000005,"5min":0.79300000000000004,"15min":0.68500000000000005},"max":{"1min":1.7210000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.486},{"interface":"front","average":{"1min":1.454,"5min":1.3380000000000001,"15min":1.323},"min":{"1min":1.2829999999999999,"5min":0.89000000000000001,"15min":0.75600000000000001},"max":{"1min":1.8029999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1120000000000001},"min":{"1min":0.72799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.601,"5min":1.663,"15min":2.149},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.25},"min":{"1min":0.85199999999999998,"5min":0.55300000000000005,"15min":0.41699999999999998},"max":{"1min":1.7849999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3700000000000001,"15min":1.3109999999999999},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.58199999999999996},"max":{"1min":1.881,"5min":2.488,"15min":2.488},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":1.903,"15min":2.2480000000000002},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.347,"15min":1.349},"min":{"1min":0.93600000000000005,"5min":0.67000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7250000000000001,"5min":2.0219999999999998,"15min":2.4289999999999998},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.60099999999999998,"15min":0.54200000000000004},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575060,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":783002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2869999999999999,"15min":1.2889999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.891,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.3180000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.546,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1439999999999999,"15min":1.1579999999999999},"min":{"1min":1.024,"5min":0.76200000000000001,"15min":0.64000000000000001},"max":{"1min":1.48,"5min":1.5980000000000001,"15min":1.8169999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.214},"min":{"1min":0.78600000000000003,"5min":0.66200000000000003,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.046}]},{"osd":2,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.202,"15min":1.206},"min":{"1min":0.85399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.476,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.214},"min":{"1min":0.79300000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5009999999999999,"5min":1.736,"15min":1.736},"last":1.222}]},{"osd":3,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.306},"min":{"1min":0.89500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.625,"5min":2.121,"15min":2.121},"last":1.302},{"interface":"front","average":{"1min":1.321,"5min":1.3240000000000001,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.597,"5min":1.7430000000000001,"15min":1.7509999999999999},"last":1.345}]},{"osd":4,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.90700000000000003,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.159},{"interface":"front","average":{"1min":1.198,"5min":1.304,"15min":1.248},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":1.649,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.25}]},{"osd":5,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.196,"15min":1.2270000000000001},"min":{"1min":0.89200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5529999999999999,"5min":1.879,"15min":1.879},"last":1.369},{"interface":"front","average":{"1min":1.282,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.92100000000000004,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.9359999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.399}]},{"osd":7,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4390000000000001},"min":{"1min":0.84199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.77,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.581},{"interface":"front","average":{"1min":1.363,"5min":1.389,"15min":1.415},"min":{"1min":1.0129999999999999,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.508}]}]},{"osd":5,"up_from":13,"seq":55834575060,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":780890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3979999999999999,"15min":1.284},"min":{"1min":0.89900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.615,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.296},"min":{"1min":1.095,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":1.972,"15min":2.3959999999999999},"last":1.117}]},{"osd":1,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1479999999999999,"15min":1.1639999999999999},"min":{"1min":0.82999999999999996,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.3999999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.212},"min":{"1min":0.84799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5529999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":1.044}]},{"osd":2,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.88800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5149999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.181},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.6990000000000001,"15min":1.738},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.397,"15min":1.28},"min":{"1min":1.0109999999999999,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.454,"5min":18.074999999999999,"15min":18.074999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.2989999999999999},"min":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.581,"5min":1.8400000000000001,"15min":1.871},"last":1}]},{"osd":4,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3460000000000001,"15min":1.2609999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.5369999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5309999999999999,"15min":1.383},"min":{"1min":0.83899999999999997,"5min":0.72199999999999998,"15min":0.63500000000000001},"max":{"1min":1.6739999999999999,"5min":18.48,"15min":18.48},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.3999999999999999,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.4870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.488,"15min":1.3660000000000001},"min":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.609,"5min":18.379999999999999,"15min":18.379999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.268,"5min":1.4550000000000001,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.534,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.0620000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575061,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1515520,"data_stored":780982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.77400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.669,"5min":1.72,"15min":1.72},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.175,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.84499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.48,"5min":1.5669999999999999,"15min":1.6910000000000001},"last":0.998}]},{"osd":1,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":0.94399999999999995,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7709999999999999},"last":1.23},{"interface":"front","average":{"1min":1.331,"5min":1.26,"15min":1.272},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.0620000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2010000000000001,"15min":1.196},"min":{"1min":0.874,"5min":0.628,"15min":0.628},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.274,"15min":1.2669999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6619999999999999,"5min":1.6779999999999999,"15min":1.78},"last":1.353}]},{"osd":3,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1839999999999999,"15min":1.204},"min":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6040000000000001,"5min":1.617,"15min":1.6839999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.3180000000000001,"15min":1.3049999999999999},"min":{"1min":1.0369999999999999,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.77,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.82699999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.706,"5min":1.706,"15min":1.7390000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.571,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.2669999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2390000000000001,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.389},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.3480000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.831,"5min":1.845,"15min":1.8660000000000001},"last":1.522}]},{"osd":7,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.349,"15min":1.329},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4359999999999999,"15min":1.3879999999999999},"min":{"1min":0.96299999999999997,"5min":0.88700000000000001,"15min":0.73799999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.482}]}]},{"osd":3,"up_from":13,"seq":55834575062,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":781444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.052,"15min":0.93500000000000005},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":14.49,"15min":14.49},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.998,"5min":1.175,"15min":1.05},"min":{"1min":0.81200000000000006,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.98599999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98099999999999998,"15min":0.96699999999999997},"min":{"1min":0.73899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1890000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.1579999999999999,"15min":1.1140000000000001},"min":{"1min":0.72899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.698,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.23,"15min":1.1599999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.6080000000000001,"5min":12.446,"15min":12.446},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.76100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.5009999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.1559999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3169999999999999,"15min":1.2589999999999999},"min":{"1min":0.89200000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":10.460000000000001,"15min":10.460000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.343,"15min":1.3560000000000001},"min":{"1min":1.0289999999999999,"5min":0.80800000000000005,"15min":0.78400000000000003},"max":{"1min":1.5569999999999999,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.407,"15min":1.411},"min":{"1min":1.0860000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.73,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.111},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.874,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7509999999999999,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.3009999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.8080000000000001,"5min":11.362,"15min":11.362},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.4419999999999999,"15min":1.399},"min":{"1min":0.98499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6679999999999999,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.4950000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.52,"15min":1.4610000000000001},"min":{"1min":1.1759999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7689999999999999,"5min":10.561,"15min":10.561},"last":1.607},{"interface":"front","average":{"1min":1.48,"5min":1.4890000000000001,"15min":1.4870000000000001},"min":{"1min":1.1679999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.792,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.3320000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575060,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.56999999999999995,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4530000000000001,"5min":1.524,"15min":1.8380000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.52,"5min":1.5409999999999999,"15min":1.9650000000000001},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.018},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.5760000000000001,"5min":1.599,"15min":2.5390000000000001},"last":1.26}]},{"osd":3,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.84599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.546,"5min":1.758,"15min":1.8240000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.288,"5min":1.256,"15min":1.2989999999999999},"min":{"1min":0.85499999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"max":{"1min":1.6419999999999999,"5min":1.948,"15min":2.4550000000000001},"last":1.016}]},{"osd":4,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.96999999999999997,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.569,"5min":1.625,"15min":1.907},"last":1.093},{"interface":"front","average":{"1min":1.373,"5min":1.3,"15min":1.29},"min":{"1min":1.071,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3420000000000001,"15min":1.3580000000000001},"min":{"1min":0.94099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8049999999999999,"5min":1.8480000000000001,"15min":2.0680000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2909999999999999,"15min":1.3300000000000001},"min":{"1min":1.0089999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.8049999999999999,"15min":2.0630000000000002},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.411,"15min":1.403},"min":{"1min":1.121,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0150000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.76900000000000002},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":2.617},"last":1.369}]},{"osd":7,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.375,"15min":1.387},"min":{"1min":1.1160000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.05},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.468,"15min":1.4650000000000001},"min":{"1min":1.238,"5min":0.753,"15min":0.753},"max":{"1min":2.0830000000000002,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.399}]}]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.874,"15min":0.92100000000000004},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.724},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.59199999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.337,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":1.216},{"interface":"front","average":{"1min":1.107,"5min":1.077,"15min":1.093},"min":{"1min":0.79100000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":2.137},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1799999999999999,"15min":1.214},"min":{"1min":0.82999999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":1.028},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0640000000000001,"15min":1.0920000000000001},"min":{"1min":0.71299999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5549999999999999,"15min":1.756},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2270000000000001,"15min":1.2430000000000001},"min":{"1min":1.0149999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5529999999999999,"5min":1.891,"15min":2.2290000000000001},"last":1.071},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3560000000000001,"15min":1.375},"min":{"1min":0.90900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.944},"last":1.1100000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.94799999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.6970000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.3460000000000001,"15min":1.3440000000000001},"min":{"1min":1.173,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.871,"5min":1.923,"15min":2.1240000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4350000000000001,"15min":1.46},"min":{"1min":1.0289999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8979999999999999,"5min":1.966,"15min":2.0310000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.0369999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.901,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.3779999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4810000000000001,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0209999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.458},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.4870000000000001,"15min":1.454},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.8100000000000001,"5min":2.085,"15min":2.085},"last":1.286}]}]},{"osd":0,"up_from":13,"seq":55834575061,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2117632,"data_stored":1372884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.571},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.54200000000000004},"max":{"1min":1.5069999999999999,"5min":1.609,"15min":1.609},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.026,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.6699999999999999,"15min":1.6950000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.0640000000000001,"15min":1.0760000000000001},"min":{"1min":0.83199999999999996,"5min":0.58399999999999996,"15min":0.55800000000000005},"max":{"1min":1.3720000000000001,"5min":1.738,"15min":1.738},"last":0.622}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0800000000000001},"min":{"1min":0.80100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.615,"5min":1.6639999999999999,"15min":1.919},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2749999999999999,"15min":1.3169999999999999},"min":{"1min":0.90600000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.673,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.286}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3140000000000001,"15min":1.339},"min":{"1min":1.0309999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.7749999999999999,"5min":2.1720000000000002,"15min":2.294},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2609999999999999,"15min":1.2909999999999999},"min":{"1min":0.93300000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.454}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.409,"15min":1.377},"min":{"1min":1.0609999999999999,"5min":0.90100000000000002,"15min":0.65900000000000003},"max":{"1min":1.9350000000000001,"5min":2.0859999999999999,"15min":2.25},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3979999999999999,"15min":1.3959999999999999},"min":{"1min":1.103,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.893,"5min":1.893,"15min":2.5510000000000002},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.22,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.647,"5min":2.2240000000000002,"15min":2.52},"last":1.129},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.835,"5min":2.391,"15min":2.4750000000000001},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.4650000000000001},"min":{"1min":1.129,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.728,"5min":2.0219999999999998,"15min":2.4260000000000002},"last":1.236},{"interface":"front","average":{"1min":1.413,"5min":1.431,"15min":1.429},"min":{"1min":1.0820000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7589999999999999,"5min":2.1459999999999999,"15min":2.3809999999999998},"last":1.335}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:36:17.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:36:17.901 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:36:17.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:36:18.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.093+0000 7fed3ccf0700 1 Processor -- start 2023-10-06T11:36:18.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 -- start start 2023-10-06T11:36:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed381502f0 0x7fed380c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 0x7fed38159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 0x7fed38159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed380c67e0 con 0x7fed381502f0 2023-10-06T11:36:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3806e810 con 0x7fed3806e2a0 2023-10-06T11:36:18.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.097+0000 7fed3ccf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3806eaf0 con 0x7fed381506c0 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed381502f0 0x7fed380c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed381502f0 0x7fed380c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34940/0 (socket says 172.21.15.46:34940) 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 -- 172.21.15.46:0/1050971266 learned_addr learned my addr 172.21.15.46:0/1050971266 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed35d9b700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 0x7fed38159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed36d9d700 1 --2- 172.21.15.46:0/1050971266 >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 0x7fed38159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 -- 172.21.15.46:0/1050971266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 msgr2=0x7fed38159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 0x7fed38159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 -- 172.21.15.46:0/1050971266 >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 msgr2=0x7fed38159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 --2- 172.21.15.46:0/1050971266 >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 0x7fed38159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 -- 172.21.15.46:0/1050971266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3806edd0 con 0x7fed381502f0 2023-10-06T11:36:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed35d9b700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 0x7fed38159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed36d9d700 1 --2- 172.21.15.46:0/1050971266 >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 0x7fed38159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3659c700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381502f0 0x7fed380c0bd0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fed380ccf10 tx=0x7fed28009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54a5d56d544f53d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3559a700 1 -- 172.21.15.46:0/1050971266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28005c10 con 0x7fed381502f0 2023-10-06T11:36:18.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3559a700 1 -- 172.21.15.46:0/1050971266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28005d70 con 0x7fed381502f0 2023-10-06T11:36:18.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3559a700 1 -- 172.21.15.46:0/1050971266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800b880 con 0x7fed381502f0 2023-10-06T11:36:18.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3ccf0700 1 -- 172.21.15.46:0/1050971266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381502f0 msgr2=0x7fed380c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.101+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381502f0 0x7fed380c0bd0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fed380ccf10 tx=0x7fed28009220 comp rx=0 tx=0).stop 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 -- 172.21.15.46:0/1050971266 shutdown_connections 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/1050971266 >> v2:172.21.15.132:3300/0 conn(0x7fed3806e2a0 0x7fed38159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381506c0 0x7fed38159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/1050971266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381502f0 0x7fed380c0bd0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 -- 172.21.15.46:0/1050971266 >> 172.21.15.46:0/1050971266 conn(0x7fed380cbe30 msgr2=0x7fed380cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 -- 172.21.15.46:0/1050971266 shutdown_connections 2023-10-06T11:36:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 -- 172.21.15.46:0/1050971266 wait complete. 2023-10-06T11:36:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 Processor -- start 2023-10-06T11:36:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.105+0000 7fed3ccf0700 1 -- start start 2023-10-06T11:36:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed3806e2a0 0x7fed3814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381502f0 0x7fed380c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed380c4120 con 0x7fed381506c0 2023-10-06T11:36:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed380ac400 con 0x7fed3806e2a0 2023-10-06T11:36:18.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380ac6c0 con 0x7fed381502f0 2023-10-06T11:36:18.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34952/0 (socket says 172.21.15.46:34952) 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 -- 172.21.15.46:0/2760859436 learned_addr learned my addr 172.21.15.46:0/2760859436 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3659c700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed3806e2a0 0x7fed3814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed35d9b700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381502f0 0x7fed380c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 -- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381502f0 msgr2=0x7fed380c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381502f0 0x7fed380c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 -- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed3806e2a0 msgr2=0x7fed3814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed3806e2a0 0x7fed3814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 -- 172.21.15.46:0/2760859436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009480 con 0x7fed381506c0 2023-10-06T11:36:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed36d9d700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fed3000d180 tx=0x7fed30009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3000dcc0 con 0x7fed381506c0 2023-10-06T11:36:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3000f040 con 0x7fed381506c0 2023-10-06T11:36:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed380c3880 con 0x7fed381506c0 2023-10-06T11:36:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30015400 con 0x7fed381506c0 2023-10-06T11:36:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.109+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed380c3e40 con 0x7fed381506c0 2023-10-06T11:36:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.113+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed38063f90 con 0x7fed381506c0 2023-10-06T11:36:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.113+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fed300157d0 con 0x7fed381506c0 2023-10-06T11:36:18.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.113+0000 7fed267fc700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 0x7fed18062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.113+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fed3007d580 con 0x7fed381506c0 2023-10-06T11:36:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.113+0000 7fed3659c700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 0x7fed18062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.117+0000 7fed3659c700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 0x7fed18062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fed28007fd0 tx=0x7fed28016000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.121+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fed3004d590 con 0x7fed381506c0 2023-10-06T11:36:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.289+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed380ac800 con 0x7fed18060670 2023-10-06T11:36:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.301+0000 7fed267fc700 1 -- 172.21.15.46:0/2760859436 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309058 (secure 0 0 0) 0x7fed380ac800 con 0x7fed18060670 2023-10-06T11:36:18.322 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 msgr2=0x7fed18062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 0x7fed18062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fed28007fd0 tx=0x7fed28016000 comp rx=0 tx=0).stop 2023-10-06T11:36:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 msgr2=0x7fed380c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fed3000d180 tx=0x7fed30009a50 comp rx=0 tx=0).stop 2023-10-06T11:36:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 shutdown_connections 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fed18060670 0x7fed18062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed381506c0 0x7fed380c1370 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed381502f0 0x7fed380c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 --2- 172.21.15.46:0/2760859436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed3806e2a0 0x7fed3814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 >> 172.21.15.46:0/2760859436 conn(0x7fed380cbe30 msgr2=0x7fed380c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 shutdown_connections 2023-10-06T11:36:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.313+0000 7fed3ccf0700 1 -- 172.21.15.46:0/2760859436 wait complete. 2023-10-06T11:36:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:36:18.347 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T11:36:17.527460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":404,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460936,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752513784,"statfs":{"total":773094113280,"available":770574114816,"internally_reserved":0,"allocated":13312000,"data_stored":7432468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.721142"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839807+0000","last_change":"2023-10-06T11:36:08.834959+0000","last_active":"2023-10-06T11:36:08.839807+0000","last_peered":"2023-10-06T11:36:08.839807+0000","last_clean":"2023-10-06T11:36:08.839807+0000","last_became_active":"2023-10-06T11:36:08.834673+0000","last_became_peered":"2023-10-06T11:36:08.834673+0000","last_unstale":"2023-10-06T11:36:08.839807+0000","last_undegraded":"2023-10-06T11:36:08.839807+0000","last_fullsized":"2023-10-06T11:36:08.839807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:45.178435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843699+0000","last_change":"2023-10-06T11:36:08.839427+0000","last_active":"2023-10-06T11:36:08.843699+0000","last_peered":"2023-10-06T11:36:08.843699+0000","last_clean":"2023-10-06T11:36:08.843699+0000","last_became_active":"2023-10-06T11:36:08.839074+0000","last_became_peered":"2023-10-06T11:36:08.839074+0000","last_unstale":"2023-10-06T11:36:08.843699+0000","last_undegraded":"2023-10-06T11:36:08.843699+0000","last_fullsized":"2023-10-06T11:36:08.843699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:51:44.389800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.826608+0000","last_change":"2023-10-06T11:36:08.822154+0000","last_active":"2023-10-06T11:36:08.826608+0000","last_peered":"2023-10-06T11:36:08.826608+0000","last_clean":"2023-10-06T11:36:08.826608+0000","last_became_active":"2023-10-06T11:36:08.821837+0000","last_became_peered":"2023-10-06T11:36:08.821837+0000","last_unstale":"2023-10-06T11:36:08.826608+0000","last_undegraded":"2023-10-06T11:36:08.826608+0000","last_fullsized":"2023-10-06T11:36:08.826608+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:17.339584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841877+0000","last_change":"2023-10-06T11:36:08.835584+0000","last_active":"2023-10-06T11:36:08.841877+0000","last_peered":"2023-10-06T11:36:08.841877+0000","last_clean":"2023-10-06T11:36:08.841877+0000","last_became_active":"2023-10-06T11:36:08.835284+0000","last_became_peered":"2023-10-06T11:36:08.835284+0000","last_unstale":"2023-10-06T11:36:08.841877+0000","last_undegraded":"2023-10-06T11:36:08.841877+0000","last_fullsized":"2023-10-06T11:36:08.841877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842628+0000","last_change":"2023-10-06T11:36:08.837618+0000","last_active":"2023-10-06T11:36:08.842628+0000","last_peered":"2023-10-06T11:36:08.842628+0000","last_clean":"2023-10-06T11:36:08.842628+0000","last_became_active":"2023-10-06T11:36:08.837297+0000","last_became_peered":"2023-10-06T11:36:08.837297+0000","last_unstale":"2023-10-06T11:36:08.842628+0000","last_undegraded":"2023-10-06T11:36:08.842628+0000","last_fullsized":"2023-10-06T11:36:08.842628+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850512+0000","last_change":"2023-10-06T11:36:08.847872+0000","last_active":"2023-10-06T11:36:08.850512+0000","last_peered":"2023-10-06T11:36:08.850512+0000","last_clean":"2023-10-06T11:36:08.850512+0000","last_became_active":"2023-10-06T11:36:08.847547+0000","last_became_peered":"2023-10-06T11:36:08.847547+0000","last_unstale":"2023-10-06T11:36:08.850512+0000","last_undegraded":"2023-10-06T11:36:08.850512+0000","last_fullsized":"2023-10-06T11:36:08.850512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:43.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849958+0000","last_change":"2023-10-06T11:36:08.846412+0000","last_active":"2023-10-06T11:36:08.849958+0000","last_peered":"2023-10-06T11:36:08.849958+0000","last_clean":"2023-10-06T11:36:08.849958+0000","last_became_active":"2023-10-06T11:36:08.843620+0000","last_became_peered":"2023-10-06T11:36:08.843620+0000","last_unstale":"2023-10-06T11:36:08.849958+0000","last_undegraded":"2023-10-06T11:36:08.849958+0000","last_fullsized":"2023-10-06T11:36:08.849958+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:23.081039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850850+0000","last_change":"2023-10-06T11:36:08.846553+0000","last_active":"2023-10-06T11:36:08.850850+0000","last_peered":"2023-10-06T11:36:08.850850+0000","last_clean":"2023-10-06T11:36:08.850850+0000","last_became_active":"2023-10-06T11:36:08.846228+0000","last_became_peered":"2023-10-06T11:36:08.846228+0000","last_unstale":"2023-10-06T11:36:08.850850+0000","last_undegraded":"2023-10-06T11:36:08.850850+0000","last_fullsized":"2023-10-06T11:36:08.850850+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849890+0000","last_change":"2023-10-06T11:36:08.846081+0000","last_active":"2023-10-06T11:36:08.849890+0000","last_peered":"2023-10-06T11:36:08.849890+0000","last_clean":"2023-10-06T11:36:08.849890+0000","last_became_active":"2023-10-06T11:36:08.844160+0000","last_became_peered":"2023-10-06T11:36:08.844160+0000","last_unstale":"2023-10-06T11:36:08.849890+0000","last_undegraded":"2023-10-06T11:36:08.849890+0000","last_fullsized":"2023-10-06T11:36:08.849890+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:32.586598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835394+0000","last_change":"2023-10-06T11:36:08.832132+0000","last_active":"2023-10-06T11:36:08.835394+0000","last_peered":"2023-10-06T11:36:08.835394+0000","last_clean":"2023-10-06T11:36:08.835394+0000","last_became_active":"2023-10-06T11:36:08.831813+0000","last_became_peered":"2023-10-06T11:36:08.831813+0000","last_unstale":"2023-10-06T11:36:08.835394+0000","last_undegraded":"2023-10-06T11:36:08.835394+0000","last_fullsized":"2023-10-06T11:36:08.835394+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847126+0000","last_change":"2023-10-06T11:36:08.839463+0000","last_active":"2023-10-06T11:36:08.847126+0000","last_peered":"2023-10-06T11:36:08.847126+0000","last_clean":"2023-10-06T11:36:08.847126+0000","last_became_active":"2023-10-06T11:36:08.839166+0000","last_became_peered":"2023-10-06T11:36:08.839166+0000","last_unstale":"2023-10-06T11:36:08.847126+0000","last_undegraded":"2023-10-06T11:36:08.847126+0000","last_fullsized":"2023-10-06T11:36:08.847126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:41.417191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848083+0000","last_change":"2023-10-06T11:36:08.845554+0000","last_active":"2023-10-06T11:36:08.848083+0000","last_peered":"2023-10-06T11:36:08.848083+0000","last_clean":"2023-10-06T11:36:08.848083+0000","last_became_active":"2023-10-06T11:36:08.844765+0000","last_became_peered":"2023-10-06T11:36:08.844765+0000","last_unstale":"2023-10-06T11:36:08.848083+0000","last_undegraded":"2023-10-06T11:36:08.848083+0000","last_fullsized":"2023-10-06T11:36:08.848083+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:56.739651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834928+0000","last_change":"2023-10-06T11:36:08.829614+0000","last_active":"2023-10-06T11:36:08.834928+0000","last_peered":"2023-10-06T11:36:08.834928+0000","last_clean":"2023-10-06T11:36:08.834928+0000","last_became_active":"2023-10-06T11:36:08.829269+0000","last_became_peered":"2023-10-06T11:36:08.829269+0000","last_unstale":"2023-10-06T11:36:08.834928+0000","last_undegraded":"2023-10-06T11:36:08.834928+0000","last_fullsized":"2023-10-06T11:36:08.834928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847934+0000","last_change":"2023-10-06T11:36:08.845000+0000","last_active":"2023-10-06T11:36:08.847934+0000","last_peered":"2023-10-06T11:36:08.847934+0000","last_clean":"2023-10-06T11:36:08.847934+0000","last_became_active":"2023-10-06T11:36:08.844706+0000","last_became_peered":"2023-10-06T11:36:08.844706+0000","last_unstale":"2023-10-06T11:36:08.847934+0000","last_undegraded":"2023-10-06T11:36:08.847934+0000","last_fullsized":"2023-10-06T11:36:08.847934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849167+0000","last_change":"2023-10-06T11:36:08.847706+0000","last_active":"2023-10-06T11:36:08.849167+0000","last_peered":"2023-10-06T11:36:08.849167+0000","last_clean":"2023-10-06T11:36:08.849167+0000","last_became_active":"2023-10-06T11:36:08.847386+0000","last_became_peered":"2023-10-06T11:36:08.847386+0000","last_unstale":"2023-10-06T11:36:08.849167+0000","last_undegraded":"2023-10-06T11:36:08.849167+0000","last_fullsized":"2023-10-06T11:36:08.849167+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.844371+0000","last_change":"2023-10-06T11:36:08.840984+0000","last_active":"2023-10-06T11:36:08.844371+0000","last_peered":"2023-10-06T11:36:08.844371+0000","last_clean":"2023-10-06T11:36:08.844371+0000","last_became_active":"2023-10-06T11:36:08.840669+0000","last_became_peered":"2023-10-06T11:36:08.840669+0000","last_unstale":"2023-10-06T11:36:08.844371+0000","last_undegraded":"2023-10-06T11:36:08.844371+0000","last_fullsized":"2023-10-06T11:36:08.844371+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.821913+0000","last_change":"2023-10-06T11:36:08.820676+0000","last_active":"2023-10-06T11:36:08.821913+0000","last_peered":"2023-10-06T11:36:08.821913+0000","last_clean":"2023-10-06T11:36:08.821913+0000","last_became_active":"2023-10-06T11:36:08.820319+0000","last_became_peered":"2023-10-06T11:36:08.820319+0000","last_unstale":"2023-10-06T11:36:08.821913+0000","last_undegraded":"2023-10-06T11:36:08.821913+0000","last_fullsized":"2023-10-06T11:36:08.821913+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:53:05.714850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850475+0000","last_change":"2023-10-06T11:36:08.847440+0000","last_active":"2023-10-06T11:36:08.850475+0000","last_peered":"2023-10-06T11:36:08.850475+0000","last_clean":"2023-10-06T11:36:08.850475+0000","last_became_active":"2023-10-06T11:36:08.844060+0000","last_became_peered":"2023-10-06T11:36:08.844060+0000","last_unstale":"2023-10-06T11:36:08.850475+0000","last_undegraded":"2023-10-06T11:36:08.850475+0000","last_fullsized":"2023-10-06T11:36:08.850475+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:10.190069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835909+0000","last_change":"2023-10-06T11:36:08.833419+0000","last_active":"2023-10-06T11:36:08.835909+0000","last_peered":"2023-10-06T11:36:08.835909+0000","last_clean":"2023-10-06T11:36:08.835909+0000","last_became_active":"2023-10-06T11:36:08.832970+0000","last_became_peered":"2023-10-06T11:36:08.832970+0000","last_unstale":"2023-10-06T11:36:08.835909+0000","last_undegraded":"2023-10-06T11:36:08.835909+0000","last_fullsized":"2023-10-06T11:36:08.835909+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:28:24.073017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843554+0000","last_change":"2023-10-06T11:36:08.839073+0000","last_active":"2023-10-06T11:36:08.843554+0000","last_peered":"2023-10-06T11:36:08.843554+0000","last_clean":"2023-10-06T11:36:08.843554+0000","last_became_active":"2023-10-06T11:36:08.838729+0000","last_became_peered":"2023-10-06T11:36:08.838729+0000","last_unstale":"2023-10-06T11:36:08.843554+0000","last_undegraded":"2023-10-06T11:36:08.843554+0000","last_fullsized":"2023-10-06T11:36:08.843554+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:43.453476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847734+0000","last_change":"2023-10-06T11:36:08.834706+0000","last_active":"2023-10-06T11:36:08.847734+0000","last_peered":"2023-10-06T11:36:08.847734+0000","last_clean":"2023-10-06T11:36:08.847734+0000","last_became_active":"2023-10-06T11:36:08.834392+0000","last_became_peered":"2023-10-06T11:36:08.834392+0000","last_unstale":"2023-10-06T11:36:08.847734+0000","last_undegraded":"2023-10-06T11:36:08.847734+0000","last_fullsized":"2023-10-06T11:36:08.847734+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:34.991338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835267+0000","last_change":"2023-10-06T11:36:08.827263+0000","last_active":"2023-10-06T11:36:08.835267+0000","last_peered":"2023-10-06T11:36:08.835267+0000","last_clean":"2023-10-06T11:36:08.835267+0000","last_became_active":"2023-10-06T11:36:08.826927+0000","last_became_peered":"2023-10-06T11:36:08.826927+0000","last_unstale":"2023-10-06T11:36:08.835267+0000","last_undegraded":"2023-10-06T11:36:08.835267+0000","last_fullsized":"2023-10-06T11:36:08.835267+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:09.477110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832710+0000","last_change":"2023-10-06T11:36:08.825112+0000","last_active":"2023-10-06T11:36:08.832710+0000","last_peered":"2023-10-06T11:36:08.832710+0000","last_clean":"2023-10-06T11:36:08.832710+0000","last_became_active":"2023-10-06T11:36:08.824817+0000","last_became_peered":"2023-10-06T11:36:08.824817+0000","last_unstale":"2023-10-06T11:36:08.832710+0000","last_undegraded":"2023-10-06T11:36:08.832710+0000","last_fullsized":"2023-10-06T11:36:08.832710+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:47.661809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840092+0000","last_change":"2023-10-06T11:36:08.833482+0000","last_active":"2023-10-06T11:36:08.840092+0000","last_peered":"2023-10-06T11:36:08.840092+0000","last_clean":"2023-10-06T11:36:08.840092+0000","last_became_active":"2023-10-06T11:36:08.833172+0000","last_became_peered":"2023-10-06T11:36:08.833172+0000","last_unstale":"2023-10-06T11:36:08.840092+0000","last_undegraded":"2023-10-06T11:36:08.840092+0000","last_fullsized":"2023-10-06T11:36:08.840092+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837751+0000","last_change":"2023-10-06T11:36:08.834331+0000","last_active":"2023-10-06T11:36:08.837751+0000","last_peered":"2023-10-06T11:36:08.837751+0000","last_clean":"2023-10-06T11:36:08.837751+0000","last_became_active":"2023-10-06T11:36:08.834010+0000","last_became_peered":"2023-10-06T11:36:08.834010+0000","last_unstale":"2023-10-06T11:36:08.837751+0000","last_undegraded":"2023-10-06T11:36:08.837751+0000","last_fullsized":"2023-10-06T11:36:08.837751+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:37.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.843481+0000","last_change":"2023-10-06T11:36:08.839346+0000","last_active":"2023-10-06T11:36:08.843481+0000","last_peered":"2023-10-06T11:36:08.843481+0000","last_clean":"2023-10-06T11:36:08.843481+0000","last_became_active":"2023-10-06T11:36:08.839069+0000","last_became_peered":"2023-10-06T11:36:08.839069+0000","last_unstale":"2023-10-06T11:36:08.843481+0000","last_undegraded":"2023-10-06T11:36:08.843481+0000","last_fullsized":"2023-10-06T11:36:08.843481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:43.156987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847105+0000","last_change":"2023-10-06T11:36:08.838625+0000","last_active":"2023-10-06T11:36:08.847105+0000","last_peered":"2023-10-06T11:36:08.847105+0000","last_clean":"2023-10-06T11:36:08.847105+0000","last_became_active":"2023-10-06T11:36:08.838342+0000","last_became_peered":"2023-10-06T11:36:08.838342+0000","last_unstale":"2023-10-06T11:36:08.847105+0000","last_undegraded":"2023-10-06T11:36:08.847105+0000","last_fullsized":"2023-10-06T11:36:08.847105+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849748+0000","last_change":"2023-10-06T11:36:08.843721+0000","last_active":"2023-10-06T11:36:08.849748+0000","last_peered":"2023-10-06T11:36:08.849748+0000","last_clean":"2023-10-06T11:36:08.849748+0000","last_became_active":"2023-10-06T11:36:08.843407+0000","last_became_peered":"2023-10-06T11:36:08.843407+0000","last_unstale":"2023-10-06T11:36:08.849748+0000","last_undegraded":"2023-10-06T11:36:08.849748+0000","last_fullsized":"2023-10-06T11:36:08.849748+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847711+0000","last_change":"2023-10-06T11:36:08.830867+0000","last_active":"2023-10-06T11:36:08.847711+0000","last_peered":"2023-10-06T11:36:08.847711+0000","last_clean":"2023-10-06T11:36:08.847711+0000","last_became_active":"2023-10-06T11:36:08.830226+0000","last_became_peered":"2023-10-06T11:36:08.830226+0000","last_unstale":"2023-10-06T11:36:08.847711+0000","last_undegraded":"2023-10-06T11:36:08.847711+0000","last_fullsized":"2023-10-06T11:36:08.847711+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:58.162759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835740+0000","last_change":"2023-10-06T11:36:08.833495+0000","last_active":"2023-10-06T11:36:08.835740+0000","last_peered":"2023-10-06T11:36:08.835740+0000","last_clean":"2023-10-06T11:36:08.835740+0000","last_became_active":"2023-10-06T11:36:08.833119+0000","last_became_peered":"2023-10-06T11:36:08.833119+0000","last_unstale":"2023-10-06T11:36:08.835740+0000","last_undegraded":"2023-10-06T11:36:08.835740+0000","last_fullsized":"2023-10-06T11:36:08.835740+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:49.438750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849754+0000","last_change":"2023-10-06T11:36:08.843345+0000","last_active":"2023-10-06T11:36:08.849754+0000","last_peered":"2023-10-06T11:36:08.849754+0000","last_clean":"2023-10-06T11:36:08.849754+0000","last_became_active":"2023-10-06T11:36:08.842930+0000","last_became_peered":"2023-10-06T11:36:08.842930+0000","last_unstale":"2023-10-06T11:36:08.849754+0000","last_undegraded":"2023-10-06T11:36:08.849754+0000","last_fullsized":"2023-10-06T11:36:08.849754+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:25.950732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827000+0000","last_change":"2023-10-06T11:36:08.825532+0000","last_active":"2023-10-06T11:36:08.827000+0000","last_peered":"2023-10-06T11:36:08.827000+0000","last_clean":"2023-10-06T11:36:08.827000+0000","last_became_active":"2023-10-06T11:36:08.825246+0000","last_became_peered":"2023-10-06T11:36:08.825246+0000","last_unstale":"2023-10-06T11:36:08.827000+0000","last_undegraded":"2023-10-06T11:36:08.827000+0000","last_fullsized":"2023-10-06T11:36:08.827000+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837616+0000","last_change":"2023-10-06T11:36:08.832215+0000","last_active":"2023-10-06T11:36:08.837616+0000","last_peered":"2023-10-06T11:36:08.837616+0000","last_clean":"2023-10-06T11:36:08.837616+0000","last_became_active":"2023-10-06T11:36:08.831906+0000","last_became_peered":"2023-10-06T11:36:08.831906+0000","last_unstale":"2023-10-06T11:36:08.837616+0000","last_undegraded":"2023-10-06T11:36:08.837616+0000","last_fullsized":"2023-10-06T11:36:08.837616+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:21.848572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"121'4","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.854694+0000","last_change":"2023-10-06T11:36:07.836398+0000","last_active":"2023-10-06T11:36:08.854694+0000","last_peered":"2023-10-06T11:36:08.854694+0000","last_clean":"2023-10-06T11:36:08.854694+0000","last_became_active":"2023-10-06T11:36:07.836063+0000","last_became_peered":"2023-10-06T11:36:07.836063+0000","last_unstale":"2023-10-06T11:36:08.854694+0000","last_undegraded":"2023-10-06T11:36:08.854694+0000","last_fullsized":"2023-10-06T11:36:08.854694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850887+0000","last_change":"2023-10-06T11:36:08.845254+0000","last_active":"2023-10-06T11:36:08.850887+0000","last_peered":"2023-10-06T11:36:08.850887+0000","last_clean":"2023-10-06T11:36:08.850887+0000","last_became_active":"2023-10-06T11:36:08.844729+0000","last_became_peered":"2023-10-06T11:36:08.844729+0000","last_unstale":"2023-10-06T11:36:08.850887+0000","last_undegraded":"2023-10-06T11:36:08.850887+0000","last_fullsized":"2023-10-06T11:36:08.850887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:07:46.014384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857089+0000","last_change":"2023-10-06T11:36:07.830544+0000","last_active":"2023-10-06T11:36:08.857089+0000","last_peered":"2023-10-06T11:36:08.857089+0000","last_clean":"2023-10-06T11:36:08.857089+0000","last_became_active":"2023-10-06T11:36:07.830229+0000","last_became_peered":"2023-10-06T11:36:07.830229+0000","last_unstale":"2023-10-06T11:36:08.857089+0000","last_undegraded":"2023-10-06T11:36:08.857089+0000","last_fullsized":"2023-10-06T11:36:08.857089+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842017+0000","last_change":"2023-10-06T11:36:08.837123+0000","last_active":"2023-10-06T11:36:08.842017+0000","last_peered":"2023-10-06T11:36:08.842017+0000","last_clean":"2023-10-06T11:36:08.842017+0000","last_became_active":"2023-10-06T11:36:08.836794+0000","last_became_peered":"2023-10-06T11:36:08.836794+0000","last_unstale":"2023-10-06T11:36:08.842017+0000","last_undegraded":"2023-10-06T11:36:08.842017+0000","last_fullsized":"2023-10-06T11:36:08.842017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:29:34.957119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864720+0000","last_change":"2023-10-06T11:36:07.843540+0000","last_active":"2023-10-06T11:36:08.864720+0000","last_peered":"2023-10-06T11:36:08.864720+0000","last_clean":"2023-10-06T11:36:08.864720+0000","last_became_active":"2023-10-06T11:36:07.843215+0000","last_became_peered":"2023-10-06T11:36:07.843215+0000","last_unstale":"2023-10-06T11:36:08.864720+0000","last_undegraded":"2023-10-06T11:36:08.864720+0000","last_fullsized":"2023-10-06T11:36:08.864720+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:32:22.066116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842320+0000","last_change":"2023-10-06T11:36:08.828471+0000","last_active":"2023-10-06T11:36:08.842320+0000","last_peered":"2023-10-06T11:36:08.842320+0000","last_clean":"2023-10-06T11:36:08.842320+0000","last_became_active":"2023-10-06T11:36:08.828169+0000","last_became_peered":"2023-10-06T11:36:08.828169+0000","last_unstale":"2023-10-06T11:36:08.842320+0000","last_undegraded":"2023-10-06T11:36:08.842320+0000","last_fullsized":"2023-10-06T11:36:08.842320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:32.232594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.855976+0000","last_change":"2023-10-06T11:36:07.834816+0000","last_active":"2023-10-06T11:36:08.855976+0000","last_peered":"2023-10-06T11:36:08.855976+0000","last_clean":"2023-10-06T11:36:08.855976+0000","last_became_active":"2023-10-06T11:36:07.834503+0000","last_became_peered":"2023-10-06T11:36:07.834503+0000","last_unstale":"2023-10-06T11:36:08.855976+0000","last_undegraded":"2023-10-06T11:36:08.855976+0000","last_fullsized":"2023-10-06T11:36:08.855976+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.841854+0000","last_change":"2023-10-06T11:36:08.834817+0000","last_active":"2023-10-06T11:36:08.841854+0000","last_peered":"2023-10-06T11:36:08.841854+0000","last_clean":"2023-10-06T11:36:08.841854+0000","last_became_active":"2023-10-06T11:36:08.834510+0000","last_became_peered":"2023-10-06T11:36:08.834510+0000","last_unstale":"2023-10-06T11:36:08.841854+0000","last_undegraded":"2023-10-06T11:36:08.841854+0000","last_fullsized":"2023-10-06T11:36:08.841854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861794+0000","last_change":"2023-10-06T11:36:07.842608+0000","last_active":"2023-10-06T11:36:08.861794+0000","last_peered":"2023-10-06T11:36:08.861794+0000","last_clean":"2023-10-06T11:36:08.861794+0000","last_became_active":"2023-10-06T11:36:07.842325+0000","last_became_peered":"2023-10-06T11:36:07.842325+0000","last_unstale":"2023-10-06T11:36:08.861794+0000","last_undegraded":"2023-10-06T11:36:08.861794+0000","last_fullsized":"2023-10-06T11:36:08.861794+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835929+0000","last_change":"2023-10-06T11:36:08.833625+0000","last_active":"2023-10-06T11:36:08.835929+0000","last_peered":"2023-10-06T11:36:08.835929+0000","last_clean":"2023-10-06T11:36:08.835929+0000","last_became_active":"2023-10-06T11:36:08.833132+0000","last_became_peered":"2023-10-06T11:36:08.833132+0000","last_unstale":"2023-10-06T11:36:08.835929+0000","last_undegraded":"2023-10-06T11:36:08.835929+0000","last_fullsized":"2023-10-06T11:36:08.835929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:44.375194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861213+0000","last_change":"2023-10-06T11:36:07.836739+0000","last_active":"2023-10-06T11:36:08.861213+0000","last_peered":"2023-10-06T11:36:08.861213+0000","last_clean":"2023-10-06T11:36:08.861213+0000","last_became_active":"2023-10-06T11:36:07.836402+0000","last_became_peered":"2023-10-06T11:36:07.836402+0000","last_unstale":"2023-10-06T11:36:08.861213+0000","last_undegraded":"2023-10-06T11:36:08.861213+0000","last_fullsized":"2023-10-06T11:36:08.861213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:13.703496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849866+0000","last_change":"2023-10-06T11:36:08.845779+0000","last_active":"2023-10-06T11:36:08.849866+0000","last_peered":"2023-10-06T11:36:08.849866+0000","last_clean":"2023-10-06T11:36:08.849866+0000","last_became_active":"2023-10-06T11:36:08.843903+0000","last_became_peered":"2023-10-06T11:36:08.843903+0000","last_unstale":"2023-10-06T11:36:08.849866+0000","last_undegraded":"2023-10-06T11:36:08.849866+0000","last_fullsized":"2023-10-06T11:36:08.849866+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:49.631159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824932+0000","last_change":"2023-10-06T11:36:07.842155+0000","last_active":"2023-10-06T11:36:08.824932+0000","last_peered":"2023-10-06T11:36:08.824932+0000","last_clean":"2023-10-06T11:36:08.824932+0000","last_became_active":"2023-10-06T11:36:07.841850+0000","last_became_peered":"2023-10-06T11:36:07.841850+0000","last_unstale":"2023-10-06T11:36:08.824932+0000","last_undegraded":"2023-10-06T11:36:08.824932+0000","last_fullsized":"2023-10-06T11:36:08.824932+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:09.175146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833946+0000","last_change":"2023-10-06T11:36:08.822864+0000","last_active":"2023-10-06T11:36:08.833946+0000","last_peered":"2023-10-06T11:36:08.833946+0000","last_clean":"2023-10-06T11:36:08.833946+0000","last_became_active":"2023-10-06T11:36:08.822567+0000","last_became_peered":"2023-10-06T11:36:08.822567+0000","last_unstale":"2023-10-06T11:36:08.833946+0000","last_undegraded":"2023-10-06T11:36:08.833946+0000","last_fullsized":"2023-10-06T11:36:08.833946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:29.590875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827606+0000","last_change":"2023-10-06T11:36:07.838208+0000","last_active":"2023-10-06T11:36:08.827606+0000","last_peered":"2023-10-06T11:36:08.827606+0000","last_clean":"2023-10-06T11:36:08.827606+0000","last_became_active":"2023-10-06T11:36:07.837843+0000","last_became_peered":"2023-10-06T11:36:07.837843+0000","last_unstale":"2023-10-06T11:36:08.827606+0000","last_undegraded":"2023-10-06T11:36:08.827606+0000","last_fullsized":"2023-10-06T11:36:08.827606+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837595+0000","last_change":"2023-10-06T11:36:08.831541+0000","last_active":"2023-10-06T11:36:08.837595+0000","last_peered":"2023-10-06T11:36:08.837595+0000","last_clean":"2023-10-06T11:36:08.837595+0000","last_became_active":"2023-10-06T11:36:08.831234+0000","last_became_peered":"2023-10-06T11:36:08.831234+0000","last_unstale":"2023-10-06T11:36:08.837595+0000","last_undegraded":"2023-10-06T11:36:08.837595+0000","last_fullsized":"2023-10-06T11:36:08.837595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:26.203324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825150+0000","last_change":"2023-10-06T11:36:07.844770+0000","last_active":"2023-10-06T11:36:08.825150+0000","last_peered":"2023-10-06T11:36:08.825150+0000","last_clean":"2023-10-06T11:36:08.825150+0000","last_became_active":"2023-10-06T11:36:07.843489+0000","last_became_peered":"2023-10-06T11:36:07.843489+0000","last_unstale":"2023-10-06T11:36:08.825150+0000","last_undegraded":"2023-10-06T11:36:08.825150+0000","last_fullsized":"2023-10-06T11:36:08.825150+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:55.205873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835302+0000","last_change":"2023-10-06T11:36:08.828494+0000","last_active":"2023-10-06T11:36:08.835302+0000","last_peered":"2023-10-06T11:36:08.835302+0000","last_clean":"2023-10-06T11:36:08.835302+0000","last_became_active":"2023-10-06T11:36:08.828149+0000","last_became_peered":"2023-10-06T11:36:08.828149+0000","last_unstale":"2023-10-06T11:36:08.835302+0000","last_undegraded":"2023-10-06T11:36:08.835302+0000","last_fullsized":"2023-10-06T11:36:08.835302+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:25.498441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.810127+0000","last_change":"2023-10-06T11:36:07.831652+0000","last_active":"2023-10-06T11:36:08.810127+0000","last_peered":"2023-10-06T11:36:08.810127+0000","last_clean":"2023-10-06T11:36:08.810127+0000","last_became_active":"2023-10-06T11:36:07.831331+0000","last_became_peered":"2023-10-06T11:36:07.831331+0000","last_unstale":"2023-10-06T11:36:08.810127+0000","last_undegraded":"2023-10-06T11:36:08.810127+0000","last_fullsized":"2023-10-06T11:36:08.810127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:30:03.936003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850794+0000","last_change":"2023-10-06T11:36:08.843325+0000","last_active":"2023-10-06T11:36:08.850794+0000","last_peered":"2023-10-06T11:36:08.850794+0000","last_clean":"2023-10-06T11:36:08.850794+0000","last_became_active":"2023-10-06T11:36:08.843032+0000","last_became_peered":"2023-10-06T11:36:08.843032+0000","last_unstale":"2023-10-06T11:36:08.850794+0000","last_undegraded":"2023-10-06T11:36:08.850794+0000","last_fullsized":"2023-10-06T11:36:08.850794+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:50:13.479212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"121'3","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.861425+0000","last_change":"2023-10-06T11:36:07.845696+0000","last_active":"2023-10-06T11:36:08.861425+0000","last_peered":"2023-10-06T11:36:08.861425+0000","last_clean":"2023-10-06T11:36:08.861425+0000","last_became_active":"2023-10-06T11:36:07.845420+0000","last_became_peered":"2023-10-06T11:36:07.845420+0000","last_unstale":"2023-10-06T11:36:08.861425+0000","last_undegraded":"2023-10-06T11:36:08.861425+0000","last_fullsized":"2023-10-06T11:36:08.861425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837117+0000","last_change":"2023-10-06T11:36:08.827973+0000","last_active":"2023-10-06T11:36:08.837117+0000","last_peered":"2023-10-06T11:36:08.837117+0000","last_clean":"2023-10-06T11:36:08.837117+0000","last_became_active":"2023-10-06T11:36:08.827650+0000","last_became_peered":"2023-10-06T11:36:08.827650+0000","last_unstale":"2023-10-06T11:36:08.837117+0000","last_undegraded":"2023-10-06T11:36:08.837117+0000","last_fullsized":"2023-10-06T11:36:08.837117+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:48.124515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"121'6","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.865336+0000","last_change":"2023-10-06T11:36:07.845721+0000","last_active":"2023-10-06T11:36:08.865336+0000","last_peered":"2023-10-06T11:36:08.865336+0000","last_clean":"2023-10-06T11:36:08.865336+0000","last_became_active":"2023-10-06T11:36:07.845389+0000","last_became_peered":"2023-10-06T11:36:07.845389+0000","last_unstale":"2023-10-06T11:36:08.865336+0000","last_undegraded":"2023-10-06T11:36:08.865336+0000","last_fullsized":"2023-10-06T11:36:08.865336+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:24.483518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842565+0000","last_change":"2023-10-06T11:36:08.831561+0000","last_active":"2023-10-06T11:36:08.842565+0000","last_peered":"2023-10-06T11:36:08.842565+0000","last_clean":"2023-10-06T11:36:08.842565+0000","last_became_active":"2023-10-06T11:36:08.831286+0000","last_became_peered":"2023-10-06T11:36:08.831286+0000","last_unstale":"2023-10-06T11:36:08.842565+0000","last_undegraded":"2023-10-06T11:36:08.842565+0000","last_fullsized":"2023-10-06T11:36:08.842565+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:27:02.158809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857133+0000","last_change":"2023-10-06T11:36:07.836509+0000","last_active":"2023-10-06T11:36:08.857133+0000","last_peered":"2023-10-06T11:36:08.857133+0000","last_clean":"2023-10-06T11:36:08.857133+0000","last_became_active":"2023-10-06T11:36:07.835626+0000","last_became_peered":"2023-10-06T11:36:07.835626+0000","last_unstale":"2023-10-06T11:36:08.857133+0000","last_undegraded":"2023-10-06T11:36:08.857133+0000","last_fullsized":"2023-10-06T11:36:08.857133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:25.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849972+0000","last_change":"2023-10-06T11:36:08.846593+0000","last_active":"2023-10-06T11:36:08.849972+0000","last_peered":"2023-10-06T11:36:08.849972+0000","last_clean":"2023-10-06T11:36:08.849972+0000","last_became_active":"2023-10-06T11:36:08.846204+0000","last_became_peered":"2023-10-06T11:36:08.846204+0000","last_unstale":"2023-10-06T11:36:08.849972+0000","last_undegraded":"2023-10-06T11:36:08.849972+0000","last_fullsized":"2023-10-06T11:36:08.849972+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:35:46.893794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.830870+0000","last_change":"2023-10-06T11:36:07.833180+0000","last_active":"2023-10-06T11:36:08.830870+0000","last_peered":"2023-10-06T11:36:08.830870+0000","last_clean":"2023-10-06T11:36:08.830870+0000","last_became_active":"2023-10-06T11:36:07.832883+0000","last_became_peered":"2023-10-06T11:36:07.832883+0000","last_unstale":"2023-10-06T11:36:08.830870+0000","last_undegraded":"2023-10-06T11:36:08.830870+0000","last_fullsized":"2023-10-06T11:36:08.830870+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.827807+0000","last_change":"2023-10-06T11:36:08.826278+0000","last_active":"2023-10-06T11:36:08.827807+0000","last_peered":"2023-10-06T11:36:08.827807+0000","last_clean":"2023-10-06T11:36:08.827807+0000","last_became_active":"2023-10-06T11:36:08.825985+0000","last_became_peered":"2023-10-06T11:36:08.825985+0000","last_unstale":"2023-10-06T11:36:08.827807+0000","last_undegraded":"2023-10-06T11:36:08.827807+0000","last_fullsized":"2023-10-06T11:36:08.827807+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:17.774461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820190+0000","last_change":"2023-10-06T11:36:07.840409+0000","last_active":"2023-10-06T11:36:08.820190+0000","last_peered":"2023-10-06T11:36:08.820190+0000","last_clean":"2023-10-06T11:36:08.820190+0000","last_became_active":"2023-10-06T11:36:07.840059+0000","last_became_peered":"2023-10-06T11:36:07.840059+0000","last_unstale":"2023-10-06T11:36:08.820190+0000","last_undegraded":"2023-10-06T11:36:08.820190+0000","last_fullsized":"2023-10-06T11:36:08.820190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:54.481491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.832875+0000","last_change":"2023-10-06T11:36:08.826025+0000","last_active":"2023-10-06T11:36:08.832875+0000","last_peered":"2023-10-06T11:36:08.832875+0000","last_clean":"2023-10-06T11:36:08.832875+0000","last_became_active":"2023-10-06T11:36:08.825431+0000","last_became_peered":"2023-10-06T11:36:08.825431+0000","last_unstale":"2023-10-06T11:36:08.832875+0000","last_undegraded":"2023-10-06T11:36:08.832875+0000","last_fullsized":"2023-10-06T11:36:08.832875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:56.221716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"121'8","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.864905+0000","last_change":"2023-10-06T11:36:07.844146+0000","last_active":"2023-10-06T11:36:08.864905+0000","last_peered":"2023-10-06T11:36:08.864905+0000","last_clean":"2023-10-06T11:36:08.864905+0000","last_became_active":"2023-10-06T11:36:07.843809+0000","last_became_peered":"2023-10-06T11:36:07.843809+0000","last_unstale":"2023-10-06T11:36:08.864905+0000","last_undegraded":"2023-10-06T11:36:08.864905+0000","last_fullsized":"2023-10-06T11:36:08.864905+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:42:42.965351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839926+0000","last_change":"2023-10-06T11:36:08.837343+0000","last_active":"2023-10-06T11:36:08.839926+0000","last_peered":"2023-10-06T11:36:08.839926+0000","last_clean":"2023-10-06T11:36:08.839926+0000","last_became_active":"2023-10-06T11:36:08.835511+0000","last_became_peered":"2023-10-06T11:36:08.835511+0000","last_unstale":"2023-10-06T11:36:08.839926+0000","last_undegraded":"2023-10-06T11:36:08.839926+0000","last_fullsized":"2023-10-06T11:36:08.839926+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:18:20.565177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857120+0000","last_change":"2023-10-06T11:36:07.832997+0000","last_active":"2023-10-06T11:36:08.857120+0000","last_peered":"2023-10-06T11:36:08.857120+0000","last_clean":"2023-10-06T11:36:08.857120+0000","last_became_active":"2023-10-06T11:36:07.832693+0000","last_became_peered":"2023-10-06T11:36:07.832693+0000","last_unstale":"2023-10-06T11:36:08.857120+0000","last_undegraded":"2023-10-06T11:36:08.857120+0000","last_fullsized":"2023-10-06T11:36:08.857120+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833519+0000","last_change":"2023-10-06T11:36:08.831230+0000","last_active":"2023-10-06T11:36:08.833519+0000","last_peered":"2023-10-06T11:36:08.833519+0000","last_clean":"2023-10-06T11:36:08.833519+0000","last_became_active":"2023-10-06T11:36:08.830920+0000","last_became_peered":"2023-10-06T11:36:08.830920+0000","last_unstale":"2023-10-06T11:36:08.833519+0000","last_undegraded":"2023-10-06T11:36:08.833519+0000","last_fullsized":"2023-10-06T11:36:08.833519+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:01.109540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T11:36:09.812940+0000","last_change":"2023-10-06T11:36:07.833889+0000","last_active":"2023-10-06T11:36:09.812940+0000","last_peered":"2023-10-06T11:36:09.812940+0000","last_clean":"2023-10-06T11:36:09.812940+0000","last_became_active":"2023-10-06T11:36:07.833564+0000","last_became_peered":"2023-10-06T11:36:07.833564+0000","last_unstale":"2023-10-06T11:36:09.812940+0000","last_undegraded":"2023-10-06T11:36:09.812940+0000","last_fullsized":"2023-10-06T11:36:09.812940+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:43.919700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835073+0000","last_change":"2023-10-06T11:36:08.832266+0000","last_active":"2023-10-06T11:36:08.835073+0000","last_peered":"2023-10-06T11:36:08.835073+0000","last_clean":"2023-10-06T11:36:08.835073+0000","last_became_active":"2023-10-06T11:36:08.830568+0000","last_became_peered":"2023-10-06T11:36:08.830568+0000","last_unstale":"2023-10-06T11:36:08.835073+0000","last_undegraded":"2023-10-06T11:36:08.835073+0000","last_fullsized":"2023-10-06T11:36:08.835073+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:14.718419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836702+0000","last_change":"2023-10-06T11:36:07.847402+0000","last_active":"2023-10-06T11:36:08.836702+0000","last_peered":"2023-10-06T11:36:08.836702+0000","last_clean":"2023-10-06T11:36:08.836702+0000","last_became_active":"2023-10-06T11:36:07.847014+0000","last_became_peered":"2023-10-06T11:36:07.847014+0000","last_unstale":"2023-10-06T11:36:08.836702+0000","last_undegraded":"2023-10-06T11:36:08.836702+0000","last_fullsized":"2023-10-06T11:36:08.836702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:43.149577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847206+0000","last_change":"2023-10-06T11:36:08.837814+0000","last_active":"2023-10-06T11:36:08.847206+0000","last_peered":"2023-10-06T11:36:08.847206+0000","last_clean":"2023-10-06T11:36:08.847206+0000","last_became_active":"2023-10-06T11:36:08.837531+0000","last_became_peered":"2023-10-06T11:36:08.837531+0000","last_unstale":"2023-10-06T11:36:08.847206+0000","last_undegraded":"2023-10-06T11:36:08.847206+0000","last_fullsized":"2023-10-06T11:36:08.847206+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:16:56.220796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816582+0000","last_change":"2023-10-06T11:36:07.834704+0000","last_active":"2023-10-06T11:36:08.816582+0000","last_peered":"2023-10-06T11:36:08.816582+0000","last_clean":"2023-10-06T11:36:08.816582+0000","last_became_active":"2023-10-06T11:36:07.834381+0000","last_became_peered":"2023-10-06T11:36:07.834381+0000","last_unstale":"2023-10-06T11:36:08.816582+0000","last_undegraded":"2023-10-06T11:36:08.816582+0000","last_fullsized":"2023-10-06T11:36:08.816582+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850140+0000","last_change":"2023-10-06T11:36:08.836975+0000","last_active":"2023-10-06T11:36:08.850140+0000","last_peered":"2023-10-06T11:36:08.850140+0000","last_clean":"2023-10-06T11:36:08.850140+0000","last_became_active":"2023-10-06T11:36:08.836692+0000","last_became_peered":"2023-10-06T11:36:08.836692+0000","last_unstale":"2023-10-06T11:36:08.850140+0000","last_undegraded":"2023-10-06T11:36:08.850140+0000","last_fullsized":"2023-10-06T11:36:08.850140+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:19.467727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.825222+0000","last_change":"2023-10-06T11:36:07.843092+0000","last_active":"2023-10-06T11:36:08.825222+0000","last_peered":"2023-10-06T11:36:08.825222+0000","last_clean":"2023-10-06T11:36:08.825222+0000","last_became_active":"2023-10-06T11:36:07.842729+0000","last_became_peered":"2023-10-06T11:36:07.842729+0000","last_unstale":"2023-10-06T11:36:08.825222+0000","last_undegraded":"2023-10-06T11:36:08.825222+0000","last_fullsized":"2023-10-06T11:36:08.825222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:54.791701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.850838+0000","last_change":"2023-10-06T11:36:08.843940+0000","last_active":"2023-10-06T11:36:08.850838+0000","last_peered":"2023-10-06T11:36:08.850838+0000","last_clean":"2023-10-06T11:36:08.850838+0000","last_became_active":"2023-10-06T11:36:08.843666+0000","last_became_peered":"2023-10-06T11:36:08.843666+0000","last_unstale":"2023-10-06T11:36:08.850838+0000","last_undegraded":"2023-10-06T11:36:08.850838+0000","last_fullsized":"2023-10-06T11:36:08.850838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:57:06.422240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822603+0000","last_change":"2023-10-06T11:36:07.840413+0000","last_active":"2023-10-06T11:36:08.822603+0000","last_peered":"2023-10-06T11:36:08.822603+0000","last_clean":"2023-10-06T11:36:08.822603+0000","last_became_active":"2023-10-06T11:36:07.840091+0000","last_became_peered":"2023-10-06T11:36:07.840091+0000","last_unstale":"2023-10-06T11:36:08.822603+0000","last_undegraded":"2023-10-06T11:36:08.822603+0000","last_fullsized":"2023-10-06T11:36:08.822603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:25.925632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.849923+0000","last_change":"2023-10-06T11:36:08.848796+0000","last_active":"2023-10-06T11:36:08.849923+0000","last_peered":"2023-10-06T11:36:08.849923+0000","last_clean":"2023-10-06T11:36:08.849923+0000","last_became_active":"2023-10-06T11:36:08.848494+0000","last_became_peered":"2023-10-06T11:36:08.848494+0000","last_unstale":"2023-10-06T11:36:08.849923+0000","last_undegraded":"2023-10-06T11:36:08.849923+0000","last_fullsized":"2023-10-06T11:36:08.849923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:44.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835941+0000","last_change":"2023-10-06T11:36:07.847520+0000","last_active":"2023-10-06T11:36:08.835941+0000","last_peered":"2023-10-06T11:36:08.835941+0000","last_clean":"2023-10-06T11:36:08.835941+0000","last_became_active":"2023-10-06T11:36:07.847113+0000","last_became_peered":"2023-10-06T11:36:07.847113+0000","last_unstale":"2023-10-06T11:36:08.835941+0000","last_undegraded":"2023-10-06T11:36:08.835941+0000","last_fullsized":"2023-10-06T11:36:08.835941+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:44.110192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847166+0000","last_change":"2023-10-06T11:36:08.839414+0000","last_active":"2023-10-06T11:36:08.847166+0000","last_peered":"2023-10-06T11:36:08.847166+0000","last_clean":"2023-10-06T11:36:08.847166+0000","last_became_active":"2023-10-06T11:36:08.839089+0000","last_became_peered":"2023-10-06T11:36:08.839089+0000","last_unstale":"2023-10-06T11:36:08.847166+0000","last_undegraded":"2023-10-06T11:36:08.847166+0000","last_fullsized":"2023-10-06T11:36:08.847166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:39:18.745369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819931+0000","last_change":"2023-10-06T11:36:07.841033+0000","last_active":"2023-10-06T11:36:08.819931+0000","last_peered":"2023-10-06T11:36:08.819931+0000","last_clean":"2023-10-06T11:36:08.819931+0000","last_became_active":"2023-10-06T11:36:07.840720+0000","last_became_peered":"2023-10-06T11:36:07.840720+0000","last_unstale":"2023-10-06T11:36:08.819931+0000","last_undegraded":"2023-10-06T11:36:08.819931+0000","last_fullsized":"2023-10-06T11:36:08.819931+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:22:24.650172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.842397+0000","last_change":"2023-10-06T11:36:08.830558+0000","last_active":"2023-10-06T11:36:08.842397+0000","last_peered":"2023-10-06T11:36:08.842397+0000","last_clean":"2023-10-06T11:36:08.842397+0000","last_became_active":"2023-10-06T11:36:08.830244+0000","last_became_peered":"2023-10-06T11:36:08.830244+0000","last_unstale":"2023-10-06T11:36:08.842397+0000","last_undegraded":"2023-10-06T11:36:08.842397+0000","last_fullsized":"2023-10-06T11:36:08.842397+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:08.600157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857037+0000","last_change":"2023-10-06T11:36:07.838666+0000","last_active":"2023-10-06T11:36:08.857037+0000","last_peered":"2023-10-06T11:36:08.857037+0000","last_clean":"2023-10-06T11:36:08.857037+0000","last_became_active":"2023-10-06T11:36:07.837999+0000","last_became_peered":"2023-10-06T11:36:07.837999+0000","last_unstale":"2023-10-06T11:36:08.857037+0000","last_undegraded":"2023-10-06T11:36:08.857037+0000","last_fullsized":"2023-10-06T11:36:08.857037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:30.002060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"111'227","reported_seq":641,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833386+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:36:08.833386+0000","last_peered":"2023-10-06T11:36:08.833386+0000","last_clean":"2023-10-06T11:36:08.833386+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:36:08.833386+0000","last_undegraded":"2023-10-06T11:36:08.833386+0000","last_fullsized":"2023-10-06T11:36:08.833386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824390+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:36:08.824390+0000","last_peered":"2023-10-06T11:36:08.824390+0000","last_clean":"2023-10-06T11:36:08.824390+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:36:08.824390+0000","last_undegraded":"2023-10-06T11:36:08.824390+0000","last_fullsized":"2023-10-06T11:36:08.824390+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.840213+0000","last_change":"2023-10-06T11:36:08.836795+0000","last_active":"2023-10-06T11:36:08.840213+0000","last_peered":"2023-10-06T11:36:08.840213+0000","last_clean":"2023-10-06T11:36:08.840213+0000","last_became_active":"2023-10-06T11:36:08.836163+0000","last_became_peered":"2023-10-06T11:36:08.836163+0000","last_unstale":"2023-10-06T11:36:08.840213+0000","last_undegraded":"2023-10-06T11:36:08.840213+0000","last_fullsized":"2023-10-06T11:36:08.840213+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:37:55.806624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.863443+0000","last_change":"2023-10-06T11:36:07.839247+0000","last_active":"2023-10-06T11:36:08.863443+0000","last_peered":"2023-10-06T11:36:08.863443+0000","last_clean":"2023-10-06T11:36:08.863443+0000","last_became_active":"2023-10-06T11:36:07.838584+0000","last_became_peered":"2023-10-06T11:36:07.838584+0000","last_unstale":"2023-10-06T11:36:08.863443+0000","last_undegraded":"2023-10-06T11:36:08.863443+0000","last_fullsized":"2023-10-06T11:36:08.863443+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:32:48.616236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833289+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:36:08.833289+0000","last_peered":"2023-10-06T11:36:08.833289+0000","last_clean":"2023-10-06T11:36:08.833289+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:36:08.833289+0000","last_undegraded":"2023-10-06T11:36:08.833289+0000","last_fullsized":"2023-10-06T11:36:08.833289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837137+0000","last_change":"2023-10-06T11:36:08.831245+0000","last_active":"2023-10-06T11:36:08.837137+0000","last_peered":"2023-10-06T11:36:08.837137+0000","last_clean":"2023-10-06T11:36:08.837137+0000","last_became_active":"2023-10-06T11:36:08.830977+0000","last_became_peered":"2023-10-06T11:36:08.830977+0000","last_unstale":"2023-10-06T11:36:08.837137+0000","last_undegraded":"2023-10-06T11:36:08.837137+0000","last_fullsized":"2023-10-06T11:36:08.837137+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:49.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824163+0000","last_change":"2023-10-06T11:36:07.841765+0000","last_active":"2023-10-06T11:36:08.824163+0000","last_peered":"2023-10-06T11:36:08.824163+0000","last_clean":"2023-10-06T11:36:08.824163+0000","last_became_active":"2023-10-06T11:36:07.841405+0000","last_became_peered":"2023-10-06T11:36:07.841405+0000","last_unstale":"2023-10-06T11:36:08.824163+0000","last_undegraded":"2023-10-06T11:36:08.824163+0000","last_fullsized":"2023-10-06T11:36:08.824163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:26.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.820278+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:36:08.820278+0000","last_peered":"2023-10-06T11:36:08.820278+0000","last_clean":"2023-10-06T11:36:08.820278+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:36:08.820278+0000","last_undegraded":"2023-10-06T11:36:08.820278+0000","last_fullsized":"2023-10-06T11:36:08.820278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.847753+0000","last_change":"2023-10-06T11:36:08.834917+0000","last_active":"2023-10-06T11:36:08.847753+0000","last_peered":"2023-10-06T11:36:08.847753+0000","last_clean":"2023-10-06T11:36:08.847753+0000","last_became_active":"2023-10-06T11:36:08.834522+0000","last_became_peered":"2023-10-06T11:36:08.834522+0000","last_unstale":"2023-10-06T11:36:08.847753+0000","last_undegraded":"2023-10-06T11:36:08.847753+0000","last_fullsized":"2023-10-06T11:36:08.847753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:52:27.750176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"121'1","reported_seq":11,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.860358+0000","last_change":"2023-10-06T11:36:07.840773+0000","last_active":"2023-10-06T11:36:08.860358+0000","last_peered":"2023-10-06T11:36:08.860358+0000","last_clean":"2023-10-06T11:36:08.860358+0000","last_became_active":"2023-10-06T11:36:07.840415+0000","last_became_peered":"2023-10-06T11:36:07.840415+0000","last_unstale":"2023-10-06T11:36:08.860358+0000","last_undegraded":"2023-10-06T11:36:08.860358+0000","last_fullsized":"2023-10-06T11:36:08.860358+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:47.109592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.824689+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:36:08.824689+0000","last_peered":"2023-10-06T11:36:08.824689+0000","last_clean":"2023-10-06T11:36:08.824689+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:36:08.824689+0000","last_undegraded":"2023-10-06T11:36:08.824689+0000","last_fullsized":"2023-10-06T11:36:08.824689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.834204+0000","last_change":"2023-10-06T11:36:08.831735+0000","last_active":"2023-10-06T11:36:08.834204+0000","last_peered":"2023-10-06T11:36:08.834204+0000","last_clean":"2023-10-06T11:36:08.834204+0000","last_became_active":"2023-10-06T11:36:08.831450+0000","last_became_peered":"2023-10-06T11:36:08.831450+0000","last_unstale":"2023-10-06T11:36:08.834204+0000","last_undegraded":"2023-10-06T11:36:08.834204+0000","last_fullsized":"2023-10-06T11:36:08.834204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:38.251254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.835829+0000","last_change":"2023-10-06T11:36:07.847157+0000","last_active":"2023-10-06T11:36:08.835829+0000","last_peered":"2023-10-06T11:36:08.835829+0000","last_clean":"2023-10-06T11:36:08.835829+0000","last_became_active":"2023-10-06T11:36:07.846801+0000","last_became_peered":"2023-10-06T11:36:07.846801+0000","last_unstale":"2023-10-06T11:36:08.835829+0000","last_undegraded":"2023-10-06T11:36:08.835829+0000","last_fullsized":"2023-10-06T11:36:08.835829+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:08.462187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.816701+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:36:08.816701+0000","last_peered":"2023-10-06T11:36:08.816701+0000","last_clean":"2023-10-06T11:36:08.816701+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:36:08.816701+0000","last_undegraded":"2023-10-06T11:36:08.816701+0000","last_fullsized":"2023-10-06T11:36:08.816701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.848126+0000","last_change":"2023-10-06T11:36:08.836395+0000","last_active":"2023-10-06T11:36:08.848126+0000","last_peered":"2023-10-06T11:36:08.848126+0000","last_clean":"2023-10-06T11:36:08.848126+0000","last_became_active":"2023-10-06T11:36:08.836112+0000","last_became_peered":"2023-10-06T11:36:08.836112+0000","last_unstale":"2023-10-06T11:36:08.848126+0000","last_undegraded":"2023-10-06T11:36:08.848126+0000","last_fullsized":"2023-10-06T11:36:08.848126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:21.083112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.819833+0000","last_change":"2023-10-06T11:36:07.840551+0000","last_active":"2023-10-06T11:36:08.819833+0000","last_peered":"2023-10-06T11:36:08.819833+0000","last_clean":"2023-10-06T11:36:08.819833+0000","last_became_active":"2023-10-06T11:36:07.840210+0000","last_became_peered":"2023-10-06T11:36:07.840210+0000","last_unstale":"2023-10-06T11:36:08.819833+0000","last_undegraded":"2023-10-06T11:36:08.819833+0000","last_fullsized":"2023-10-06T11:36:08.819833+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:09.519398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.833204+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:36:08.833204+0000","last_peered":"2023-10-06T11:36:08.833204+0000","last_clean":"2023-10-06T11:36:08.833204+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:36:08.833204+0000","last_undegraded":"2023-10-06T11:36:08.833204+0000","last_fullsized":"2023-10-06T11:36:08.833204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.839738+0000","last_change":"2023-10-06T11:36:08.836510+0000","last_active":"2023-10-06T11:36:08.839738+0000","last_peered":"2023-10-06T11:36:08.839738+0000","last_clean":"2023-10-06T11:36:08.839738+0000","last_became_active":"2023-10-06T11:36:08.836215+0000","last_became_peered":"2023-10-06T11:36:08.836215+0000","last_unstale":"2023-10-06T11:36:08.839738+0000","last_undegraded":"2023-10-06T11:36:08.839738+0000","last_fullsized":"2023-10-06T11:36:08.839738+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:32.797025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"121'2","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.857016+0000","last_change":"2023-10-06T11:36:07.841111+0000","last_active":"2023-10-06T11:36:08.857016+0000","last_peered":"2023-10-06T11:36:08.857016+0000","last_clean":"2023-10-06T11:36:08.857016+0000","last_became_active":"2023-10-06T11:36:07.840810+0000","last_became_peered":"2023-10-06T11:36:07.840810+0000","last_unstale":"2023-10-06T11:36:08.857016+0000","last_undegraded":"2023-10-06T11:36:08.857016+0000","last_fullsized":"2023-10-06T11:36:08.857016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:03.725992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.818400+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:36:08.818400+0000","last_peered":"2023-10-06T11:36:08.818400+0000","last_clean":"2023-10-06T11:36:08.818400+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:36:08.818400+0000","last_undegraded":"2023-10-06T11:36:08.818400+0000","last_fullsized":"2023-10-06T11:36:08.818400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.837031+0000","last_change":"2023-10-06T11:36:08.833641+0000","last_active":"2023-10-06T11:36:08.837031+0000","last_peered":"2023-10-06T11:36:08.837031+0000","last_clean":"2023-10-06T11:36:08.837031+0000","last_became_active":"2023-10-06T11:36:08.833372+0000","last_became_peered":"2023-10-06T11:36:08.833372+0000","last_unstale":"2023-10-06T11:36:08.837031+0000","last_undegraded":"2023-10-06T11:36:08.837031+0000","last_fullsized":"2023-10-06T11:36:08.837031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:07.796936+0000","last_clean_scrub_stamp":"2023-10-06T11:36:07.796936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:45.164451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.822466+0000","last_change":"2023-10-06T11:36:07.841750+0000","last_active":"2023-10-06T11:36:08.822466+0000","last_peered":"2023-10-06T11:36:08.822466+0000","last_clean":"2023-10-06T11:36:08.822466+0000","last_became_active":"2023-10-06T11:36:07.841432+0000","last_became_peered":"2023-10-06T11:36:07.841432+0000","last_unstale":"2023-10-06T11:36:08.822466+0000","last_undegraded":"2023-10-06T11:36:08.822466+0000","last_fullsized":"2023-10-06T11:36:08.822466+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:36:06.782014+0000","last_clean_scrub_stamp":"2023-10-06T11:36:06.782014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:27:06.372507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T11:36:08.836500+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:36:08.836500+0000","last_peered":"2023-10-06T11:36:08.836500+0000","last_clean":"2023-10-06T11:36:08.836500+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:36:08.836500+0000","last_undegraded":"2023-10-06T11:36:08.836500+0000","last_fullsized":"2023-10-06T11:36:08.836500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575061,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":2109440,"data_stored":1371418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1020000000000001,"15min":1.1339999999999999},"min":{"1min":0.84499999999999997,"5min":0.59599999999999997,"15min":0.495},"max":{"1min":1.5249999999999999,"5min":1.6579999999999999,"15min":1.724},"last":1.135},{"interface":"front","average":{"1min":1.159,"5min":1.143,"15min":1.139},"min":{"1min":0.84799999999999998,"5min":0.66100000000000003,"15min":0.56399999999999995},"max":{"1min":1.5229999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":1.325}]},{"osd":1,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.359,"15min":1.333},"min":{"1min":0.78100000000000003,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.651,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.653},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2190000000000001,"15min":1.232},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5349999999999999,"5min":1.927,"15min":1.927},"last":1.482}]},{"osd":2,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.2,"15min":1.214},"min":{"1min":1.006,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6040000000000001,"5min":1.827,"15min":1.827},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.5649999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.256,"15min":1.264},"min":{"1min":0.92900000000000005,"5min":0.79300000000000004,"15min":0.68500000000000005},"max":{"1min":1.7210000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.3380000000000001,"15min":1.323},"min":{"1min":1.2829999999999999,"5min":0.89000000000000001,"15min":0.75600000000000001},"max":{"1min":1.8029999999999999,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1140000000000001,"15min":1.1120000000000001},"min":{"1min":0.72799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.601,"5min":1.663,"15min":2.149},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.2869999999999999,"15min":1.25},"min":{"1min":0.85199999999999998,"5min":0.55300000000000005,"15min":0.41699999999999998},"max":{"1min":1.7849999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.401}]},{"osd":5,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3700000000000001,"15min":1.3109999999999999},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.58199999999999996},"max":{"1min":1.881,"5min":2.488,"15min":2.488},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.65300000000000002,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.6559999999999999,"5min":1.903,"15min":2.2480000000000002},"last":1.3720000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.347,"15min":1.349},"min":{"1min":0.93600000000000005,"5min":0.67000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7250000000000001,"5min":2.0219999999999998,"15min":2.4289999999999998},"last":1.532},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4039999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.60099999999999998,"15min":0.54200000000000004},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.288}]}]},{"osd":6,"up_from":13,"seq":55834575061,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":783002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2869999999999999,"15min":1.2889999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.891,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.3180000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.546,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.3919999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1439999999999999,"15min":1.1579999999999999},"min":{"1min":1.024,"5min":0.76200000000000001,"15min":0.64000000000000001},"max":{"1min":1.48,"5min":1.5980000000000001,"15min":1.8169999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.214},"min":{"1min":0.78600000000000003,"5min":0.66200000000000003,"15min":0.63700000000000001},"max":{"1min":1.728,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.4339999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.202,"15min":1.206},"min":{"1min":0.85399999999999998,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.476,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.214},"min":{"1min":0.79300000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5009999999999999,"5min":1.736,"15min":1.736},"last":1.351}]},{"osd":3,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.306},"min":{"1min":0.89500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.625,"5min":2.121,"15min":2.121},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3240000000000001,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.597,"5min":1.7430000000000001,"15min":1.7509999999999999},"last":1.161}]},{"osd":4,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.90700000000000003,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.21},{"interface":"front","average":{"1min":1.198,"5min":1.304,"15min":1.248},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":1.649,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.268}]},{"osd":5,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.196,"15min":1.2270000000000001},"min":{"1min":0.89200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5529999999999999,"5min":1.879,"15min":1.879},"last":1.472},{"interface":"front","average":{"1min":1.282,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.92100000000000004,"5min":0.78300000000000003,"15min":0.71799999999999997},"max":{"1min":1.9359999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Oct 6 11:35:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4450000000000001,"15min":1.4390000000000001},"min":{"1min":0.84199999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.77,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.496},{"interface":"front","average":{"1min":1.363,"5min":1.389,"15min":1.415},"min":{"1min":1.0129999999999999,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.591}]}]},{"osd":5,"up_from":13,"seq":55834575061,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302856,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068984,"statfs":{"total":96636764160,"available":96326639616,"internally_reserved":0,"allocated":1507328,"data_stored":780890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3979999999999999,"15min":1.284},"min":{"1min":0.89900000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.615,"5min":17.626000000000001,"15min":17.626000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.296},"min":{"1min":1.095,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6339999999999999,"5min":1.972,"15min":2.3959999999999999},"last":1.4339999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1479999999999999,"15min":1.1639999999999999},"min":{"1min":0.82999999999999996,"5min":0.64500000000000002,"15min":0.53200000000000003},"max":{"1min":1.3999999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.341,"15min":1.212},"min":{"1min":0.84799999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5529999999999999,"5min":18.033000000000001,"15min":18.033000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.88800000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5149999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.202,"15min":1.21},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4319999999999999,"5min":1.6990000000000001,"15min":1.738},"last":1.407}]},{"osd":3,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.397,"15min":1.28},"min":{"1min":1.0109999999999999,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.454,"5min":18.074999999999999,"15min":18.074999999999999},"last":1.131},{"interface":"front","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.2989999999999999},"min":{"1min":1.0409999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.581,"5min":1.8400000000000001,"15min":1.871},"last":1.4850000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3460000000000001,"15min":1.2609999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.5369999999999999,"5min":8.5649999999999995,"15min":8.5649999999999995},"last":1.077},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5309999999999999,"15min":1.383},"min":{"1min":0.83899999999999997,"5min":0.72199999999999998,"15min":0.63500000000000001},"max":{"1min":1.6739999999999999,"5min":18.48,"15min":18.48},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3340000000000001,"15min":1.266},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.367},{"interface":"front","average":{"1min":1.252,"5min":1.3999999999999999,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.4870000000000001,"5min":9.8559999999999999,"15min":9.8559999999999999},"last":1.325}]},{"osd":7,"last update":"Fri Oct 6 11:35:42 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.488,"15min":1.3660000000000001},"min":{"1min":0.83499999999999996,"5min":0.76700000000000002,"15min":0.755},"max":{"1min":1.609,"5min":18.379999999999999,"15min":18.379999999999999},"last":1.232},{"interface":"front","average":{"1min":1.268,"5min":1.4550000000000001,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.534,"5min":8.4830000000000005,"15min":8.4830000000000005},"last":1.5329999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575062,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1515520,"data_stored":780982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.77400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.669,"5min":1.72,"15min":1.72},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.175,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.84499999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.48,"5min":1.5669999999999999,"15min":1.6910000000000001},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2869999999999999,"15min":1.3089999999999999},"min":{"1min":0.94399999999999995,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7709999999999999},"last":1.458},{"interface":"front","average":{"1min":1.331,"5min":1.26,"15min":1.272},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.1819999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2010000000000001,"15min":1.196},"min":{"1min":0.874,"5min":0.628,"15min":0.628},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.274,"15min":1.2669999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6619999999999999,"5min":1.6779999999999999,"15min":1.78},"last":1.095}]},{"osd":3,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1839999999999999,"15min":1.204},"min":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6040000000000001,"5min":1.617,"15min":1.6839999999999999},"last":1.393},{"interface":"front","average":{"1min":1.339,"5min":1.3180000000000001,"15min":1.3049999999999999},"min":{"1min":1.0369999999999999,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.77,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.018}]},{"osd":5,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.82699999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.706,"5min":1.706,"15min":1.7390000000000001},"last":1.27},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.2090000000000001,"15min":1.1870000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.571,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.607}]},{"osd":6,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2390000000000001,"15min":1.232},"min":{"1min":0.83199999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3560000000000001,"15min":1.3480000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.831,"5min":1.845,"15min":1.8660000000000001},"last":1.5489999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.349,"15min":1.329},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4359999999999999,"15min":1.3879999999999999},"min":{"1min":0.96299999999999997,"5min":0.88700000000000001,"15min":0.73799999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.494}]}]},{"osd":3,"up_from":13,"seq":55834575063,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":781444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.052,"15min":0.93500000000000005},"min":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7190000000000001,"5min":14.49,"15min":14.49},"last":1.276},{"interface":"front","average":{"1min":0.998,"5min":1.175,"15min":1.05},"min":{"1min":0.81200000000000006,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":19.800999999999998,"15min":19.800999999999998},"last":1.21}]},{"osd":1,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.98099999999999998,"15min":0.96699999999999997},"min":{"1min":0.73899999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1890000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.1579999999999999,"15min":1.1140000000000001},"min":{"1min":0.72899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.698,"5min":11.513999999999999,"15min":11.513999999999999},"last":1.552}]},{"osd":2,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.23,"15min":1.1599999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.6080000000000001,"5min":12.446,"15min":12.446},"last":1.488},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.76100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.5009999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.6699999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3169999999999999,"15min":1.2589999999999999},"min":{"1min":0.89200000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":10.460000000000001,"15min":10.460000000000001},"last":1.423},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.343,"15min":1.3560000000000001},"min":{"1min":1.0289999999999999,"5min":0.80800000000000005,"15min":0.78400000000000003},"max":{"1min":1.5569999999999999,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.2490000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.407,"15min":1.411},"min":{"1min":1.0860000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.73,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.127},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.874,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7509999999999999,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.8049999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.8080000000000001,"5min":11.362,"15min":11.362},"last":1.905},{"interface":"front","average":{"1min":1.381,"5min":1.4419999999999999,"15min":1.399},"min":{"1min":0.98499999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6679999999999999,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.643}]},{"osd":7,"last update":"Fri Oct 6 11:35:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.52,"15min":1.4610000000000001},"min":{"1min":1.1759999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7689999999999999,"5min":10.561,"15min":10.561},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.4890000000000001,"15min":1.4870000000000001},"min":{"1min":1.1679999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.792,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.6040000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575061,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311052,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060788,"statfs":{"total":96636764160,"available":96318246912,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.56999999999999995,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4530000000000001,"5min":1.524,"15min":1.8380000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.52,"5min":1.5409999999999999,"15min":1.9650000000000001},"last":1.077}]},{"osd":1,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.91900000000000004,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":1.002},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0069999999999999,"15min":1.018},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.439},"max":{"1min":1.5760000000000001,"5min":1.599,"15min":2.5390000000000001},"last":1.123}]},{"osd":3,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.135,"15min":1.1839999999999999},"min":{"1min":0.84599999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.546,"5min":1.758,"15min":1.8240000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.256,"15min":1.2989999999999999},"min":{"1min":0.85499999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"max":{"1min":1.6419999999999999,"5min":1.948,"15min":2.4550000000000001},"last":1.302}]},{"osd":4,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.96999999999999997,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.569,"5min":1.625,"15min":1.907},"last":1.478},{"interface":"front","average":{"1min":1.373,"5min":1.3,"15min":1.29},"min":{"1min":1.071,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3420000000000001,"15min":1.3580000000000001},"min":{"1min":0.94099999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8049999999999999,"5min":1.8480000000000001,"15min":2.0680000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.2909999999999999,"15min":1.3300000000000001},"min":{"1min":1.0089999999999999,"5min":0.751,"15min":0.751},"max":{"1min":1.5700000000000001,"5min":1.8049999999999999,"15min":2.0630000000000002},"last":1.5860000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.411,"15min":1.403},"min":{"1min":1.121,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0150000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.0960000000000001,"5min":0.78400000000000003,"15min":0.76900000000000002},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":2.617},"last":1.6339999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:35:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.375,"15min":1.387},"min":{"1min":1.1160000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.468,"15min":1.4650000000000001},"min":{"1min":1.238,"5min":0.753,"15min":0.753},"max":{"1min":2.0830000000000002,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.7370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575061,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1511424,"data_stored":780924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.874,"15min":0.92100000000000004},"min":{"1min":0.61799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.724},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.0669999999999999,"5min":0.98499999999999999,"15min":0.96599999999999997},"min":{"1min":0.59199999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.149}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.64500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.337,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.107,"5min":1.077,"15min":1.093},"min":{"1min":0.79100000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":2.137},"last":1.0349999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1799999999999999,"15min":1.214},"min":{"1min":0.82999999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0640000000000001,"15min":1.0920000000000001},"min":{"1min":0.71299999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5549999999999999,"15min":1.756},"last":1.0009999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2270000000000001,"15min":1.2430000000000001},"min":{"1min":1.0149999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5529999999999999,"5min":1.891,"15min":2.2290000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3560000000000001,"15min":1.375},"min":{"1min":0.90900000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.944},"last":1.383}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.94799999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.6970000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.3460000000000001,"15min":1.3440000000000001},"min":{"1min":1.173,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.871,"5min":1.923,"15min":2.1240000000000001},"last":1.5940000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4350000000000001,"15min":1.46},"min":{"1min":1.0289999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8979999999999999,"5min":1.966,"15min":2.0310000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":1.0369999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.901,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.321}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4810000000000001,"15min":1.49},"min":{"1min":1.1879999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0209999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.548},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.4870000000000001,"15min":1.454},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.8100000000000001,"5min":2.085,"15min":2.085},"last":1.6919999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307548,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064292,"statfs":{"total":96636764160,"available":96321835008,"internally_reserved":0,"allocated":2117632,"data_stored":1372884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.66100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.571},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.97699999999999998,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.54200000000000004},"max":{"1min":1.5069999999999999,"5min":1.609,"15min":1.609},"last":0.90000000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.026,"15min":1.0269999999999999},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.6699999999999999,"15min":1.6950000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.0640000000000001,"15min":1.0760000000000001},"min":{"1min":0.83199999999999996,"5min":0.58399999999999996,"15min":0.55800000000000005},"max":{"1min":1.3720000000000001,"5min":1.738,"15min":1.738},"last":1.143}]},{"osd":3,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0449999999999999,"15min":1.0800000000000001},"min":{"1min":0.80100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.615,"5min":1.6639999999999999,"15min":1.919},"last":1.403},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2749999999999999,"15min":1.3169999999999999},"min":{"1min":0.90600000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.673,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.532}]},{"osd":4,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3140000000000001,"15min":1.339},"min":{"1min":1.0309999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.7749999999999999,"5min":2.1720000000000002,"15min":2.294},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2609999999999999,"15min":1.2909999999999999},"min":{"1min":0.93300000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.5840000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.409,"15min":1.377},"min":{"1min":1.0609999999999999,"5min":0.90100000000000002,"15min":0.65900000000000003},"max":{"1min":1.9350000000000001,"5min":2.0859999999999999,"15min":2.25},"last":1.645},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3979999999999999,"15min":1.3959999999999999},"min":{"1min":1.103,"5min":1.024,"15min":0.84599999999999997},"max":{"1min":1.893,"5min":1.893,"15min":2.5510000000000002},"last":1.3700000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.22,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.647,"5min":2.2240000000000002,"15min":2.52},"last":2.1190000000000002},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.835,"5min":2.391,"15min":2.4750000000000001},"last":1.8740000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:35:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.4650000000000001},"min":{"1min":1.129,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.728,"5min":2.0219999999999998,"15min":2.4260000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.431,"15min":1.429},"min":{"1min":1.0820000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":1.7589999999999999,"5min":2.1459999999999999,"15min":2.3809999999999998},"last":2.2389999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:36:18.354 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:36:18.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:36:18.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.553+0000 7f58c759e700 1 Processor -- start 2023-10-06T11:36:18.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.553+0000 7f58c759e700 1 -- start start 2023-10-06T11:36:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58c80d0750 0x7f58c80cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c80cf050 con 0x7f58c80cf4e0 2023-10-06T11:36:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c80cf8f0 con 0x7f58c80cb020 2023-10-06T11:36:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c80ce440 con 0x7f58c80d0750 2023-10-06T11:36:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c6d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58c80cf4e0 0x7f58c8158bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56454/0 (socket says 172.21.15.46:56454) 2023-10-06T11:36:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c6d9d700 1 -- 172.21.15.46:0/1343898778 learned_addr learned my addr 172.21.15.46:0/1343898778 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c5d9b700 1 --2- 172.21.15.46:0/1343898778 >> v2:172.21.15.132:3300/0 conn(0x7f58c80d0750 0x7f58c80cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957650768 0 0) 0x7f58c80cf050 con 0x7f58c80cf4e0 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b0003570 con 0x7f58c80cf4e0 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860348368 0 0) 0x7f58c80cf8f0 con 0x7f58c80cb020 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c80cf050 con 0x7f58c80cb020 2023-10-06T11:36:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1343898778 >> v1:172.21.15.132:6789/0 conn(0x7f58c80cb020 legacy=0x7f58c80cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1343898778 >> v1:172.21.15.46:6789/0 conn(0x7f58c80cf4e0 legacy=0x7f58c8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1343898778 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80ce720 con 0x7f58c80d0750 2023-10-06T11:36:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.557+0000 7f58c5d9b700 1 --2- 172.21.15.46:0/1343898778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80d0750 0x7f58c80cb3f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098e0 tx=0x7f58bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed09888865b4866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011630 con 0x7f58c80d0750 2023-10-06T11:36:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc011790 con 0x7f58c80d0750 2023-10-06T11:36:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c559a700 1 -- 172.21.15.46:0/1343898778 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011b30 con 0x7f58c80d0750 2023-10-06T11:36:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- 172.21.15.46:0/1343898778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80d0750 msgr2=0x7f58c80cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 --2- 172.21.15.46:0/1343898778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80d0750 0x7f58c80cb3f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0098e0 tx=0x7f58bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- 172.21.15.46:0/1343898778 shutdown_connections 2023-10-06T11:36:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 --2- 172.21.15.46:0/1343898778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80d0750 0x7f58c80cb3f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- 172.21.15.46:0/1343898778 >> 172.21.15.46:0/1343898778 conn(0x7f58c80683d0 msgr2=0x7f58c80d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- 172.21.15.46:0/1343898778 shutdown_connections 2023-10-06T11:36:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- 172.21.15.46:0/1343898778 wait complete. 2023-10-06T11:36:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 Processor -- start 2023-10-06T11:36:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.561+0000 7f58c759e700 1 -- start start 2023-10-06T11:36:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 0x7f58c80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c80d0750 0x7f58c815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c816dd30 con 0x7f58c80cf4e0 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c816ce40 con 0x7f58c80cb020 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c816d120 con 0x7f58c80d0750 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34954/0 (socket says 172.21.15.46:34954) 2023-10-06T11:36:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1223479260 learned_addr learned my addr 172.21.15.46:0/1223479260 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c6d9d700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c80d0750 0x7f58c815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c659c700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 0x7f58c80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c80d0750 msgr2=0x7f58c815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c80d0750 0x7f58c815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 msgr2=0x7f58c80c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 0x7f58c80c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 -- 172.21.15.46:0/1223479260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc009420 con 0x7f58c80cf4e0 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c659c700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 0x7f58c80c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c5d9b700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f58c8088050 tx=0x7f58bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc011900 con 0x7f58c80cf4e0 2023-10-06T11:36:18.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc011a60 con 0x7f58c80cf4e0 2023-10-06T11:36:18.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc025670 con 0x7f58c80cf4e0 2023-10-06T11:36:18.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c816d400 con 0x7f58c80cf4e0 2023-10-06T11:36:18.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.565+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c816d9c0 con 0x7f58c80cf4e0 2023-10-06T11:36:18.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.569+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c8064590 con 0x7f58c80cf4e0 2023-10-06T11:36:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.569+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f58bc023070 con 0x7f58c80cf4e0 2023-10-06T11:36:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.569+0000 7f58b67fc700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 0x7f58a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.573+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f58bc07e250 con 0x7f58c80cf4e0 2023-10-06T11:36:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.573+0000 7f58c659c700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 0x7f58a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:18.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.573+0000 7f58c659c700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 0x7f58a8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58b8000c00 tx=0x7f58b8009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.577+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f58bc04e370 con 0x7f58c80cf4e0 2023-10-06T11:36:18.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.813+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f58c80cea00 con 0x7f58c80cf4e0 2023-10-06T11:36:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.821+0000 7f58b67fc700 1 -- 172.21.15.46:0/1223479260 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f58bc051990 con 0x7f58c80cf4e0 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.825+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 msgr2=0x7f58a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.825+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 0x7f58a8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58b8000c00 tx=0x7f58b8009040 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.825+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 msgr2=0x7f58c815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.825+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f58c8088050 tx=0x7f58bc017000 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.825+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 shutdown_connections 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f58a8060670 0x7f58a8062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58c80d0750 0x7f58c815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c80cf4e0 0x7f58c815af30 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 --2- 172.21.15.46:0/1223479260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58c80cb020 0x7f58c80c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 >> 172.21.15.46:0/1223479260 conn(0x7f58c80683d0 msgr2=0x7f58c80b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 shutdown_connections 2023-10-06T11:36:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:18.829+0000 7f58c759e700 1 -- 172.21.15.46:0/1223479260 wait complete. 2023-10-06T11:36:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:36:18.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:36:19.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 Processor -- start 2023-10-06T11:36:19.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 -- start start 2023-10-06T11:36:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 0x7fd8b8067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80680d0 con 0x7fd8b80af850 2023-10-06T11:36:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80b0d00 con 0x7fd8b8156d40 2023-10-06T11:36:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8bd6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80b03d0 con 0x7fd8b80c9f90 2023-10-06T11:36:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8affff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8b80af850 0x7fd8b80b1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56466/0 (socket says 172.21.15.46:56466) 2023-10-06T11:36:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.013+0000 7fd8affff700 1 -- 172.21.15.46:0/3013328366 learned_addr learned my addr 172.21.15.46:0/3013328366 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b7fff700 1 --2- 172.21.15.46:0/3013328366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 0x7fd8b8067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144424873 0 0) 0x7fd8b80680d0 con 0x7fd8b80af850 2023-10-06T11:36:19.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a4003570 con 0x7fd8b80af850 2023-10-06T11:36:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212548565 0 0) 0x7fd8b80b0d00 con 0x7fd8b8156d40 2023-10-06T11:36:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b7fff700 1 -- 172.21.15.46:0/3013328366 >> v1:172.21.15.132:6789/0 conn(0x7fd8b8156d40 legacy=0x7fd8b80af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b7fff700 1 -- 172.21.15.46:0/3013328366 >> v1:172.21.15.46:6789/0 conn(0x7fd8b80af850 legacy=0x7fd8b80b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b7fff700 1 -- 172.21.15.46:0/3013328366 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b80b06b0 con 0x7fd8b80c9f90 2023-10-06T11:36:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b7fff700 1 --2- 172.21.15.46:0/3013328366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 0x7fd8b8067a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009e90 tx=0x7fd8b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3149ac9b72a02d4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000e070 con 0x7fd8b80c9f90 2023-10-06T11:36:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b000bc30 con 0x7fd8b80c9f90 2023-10-06T11:36:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8b67fc700 1 -- 172.21.15.46:0/3013328366 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0012430 con 0x7fd8b80c9f90 2023-10-06T11:36:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3013328366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 msgr2=0x7fd8b8067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3013328366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 0x7fd8b8067a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009e90 tx=0x7fd8b0009780 comp rx=0 tx=0).stop 2023-10-06T11:36:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3013328366 shutdown_connections 2023-10-06T11:36:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3013328366 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b80c9f90 0x7fd8b8067a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3013328366 >> 172.21.15.46:0/3013328366 conn(0x7fd8b8069250 msgr2=0x7fd8b80cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3013328366 shutdown_connections 2023-10-06T11:36:19.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.017+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3013328366 wait complete. 2023-10-06T11:36:19.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 Processor -- start 2023-10-06T11:36:19.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 -- start start 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 0x7fd8b814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b8156d40 0x7fd8b8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81584c0 con 0x7fd8b80c9f90 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8157650 con 0x7fd8b80af850 2023-10-06T11:36:19.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8bd6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81578b0 con 0x7fd8b8156d40 2023-10-06T11:36:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34962/0 (socket says 172.21.15.46:34962) 2023-10-06T11:36:19.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.021+0000 7fd8affff700 1 -- 172.21.15.46:0/3300216744 learned_addr learned my addr 172.21.15.46:0/3300216744 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8b7fff700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b8156d40 0x7fd8b8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8b6ffd700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 0x7fd8b814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 -- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b8156d40 msgr2=0x7fd8b8157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b8156d40 0x7fd8b8157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 -- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 msgr2=0x7fd8b814fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 0x7fd8b814fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 -- 172.21.15.46:0/3300216744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0009a20 con 0x7fd8b80c9f90 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8b6ffd700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 0x7fd8b814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8affff700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c7c0 tx=0x7fd8a0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0005c00 con 0x7fd8b80c9f90 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0005d60 con 0x7fd8b80c9f90 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b8157b90 con 0x7fd8b80c9f90 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0007300 con 0x7fd8b80c9f90 2023-10-06T11:36:19.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8158150 con 0x7fd8b80c9f90 2023-10-06T11:36:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.025+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b80acc00 con 0x7fd8b80c9f90 2023-10-06T11:36:19.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.029+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd8a001b3b0 con 0x7fd8b80c9f90 2023-10-06T11:36:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.029+0000 7fd8aeffd700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 0x7fd898062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.029+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fd8a007c200 con 0x7fd8b80c9f90 2023-10-06T11:36:19.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.029+0000 7fd8b6ffd700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 0x7fd898062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.029+0000 7fd8b6ffd700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 0x7fd898062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8007fd0 tx=0x7fd8a8009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.033+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8a004c320 con 0x7fd8b80c9f90 2023-10-06T11:36:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.241+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8b8158600 con 0x7fd8b80c9f90 2023-10-06T11:36:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8aeffd700 1 -- 172.21.15.46:0/3300216744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd8b8158600 con 0x7fd8b80c9f90 2023-10-06T11:36:19.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:19.258 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:36:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 msgr2=0x7fd898062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 0x7fd898062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8007fd0 tx=0x7fd8a8009040 comp rx=0 tx=0).stop 2023-10-06T11:36:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 msgr2=0x7fd8b80cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c7c0 tx=0x7fd8a0007dd0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 shutdown_connections 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd8980606a0 0x7fd898062b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b8156d40 0x7fd8b8157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b80c9f90 0x7fd8b80cdfa0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 --2- 172.21.15.46:0/3300216744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b80af850 0x7fd8b814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.245+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 >> 172.21.15.46:0/3300216744 conn(0x7fd8b8069250 msgr2=0x7fd8b81525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.249+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 shutdown_connections 2023-10-06T11:36:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.249+0000 7fd8bd6f0700 1 -- 172.21.15.46:0/3300216744 wait complete. 2023-10-06T11:36:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:36:19.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:36:08.275313+0000', 'modified': '2023-10-06T11:36:11.843055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7824}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7824': {'gid': 7824, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/2696512883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2696512883}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 2696512883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:36:19.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:36:19.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T11:36:19.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.841+0000 7f0cf240c700 1 Processor -- start 2023-10-06T11:36:19.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.841+0000 7f0cf240c700 1 -- start start 2023-10-06T11:36:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.841+0000 7f0cf240c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec159290 0x7f0cec0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf240c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cec0b1300 0x7f0cec0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf240c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cec0ae340 con 0x7f0cec0d07b0 2023-10-06T11:36:19.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf240c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec0b09a0 con 0x7f0cec0b1300 2023-10-06T11:36:19.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf240c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec159660 con 0x7f0cec159290 2023-10-06T11:36:19.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cec0b1300 0x7f0cec0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cec0b1300 0x7f0cec0ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34972/0 (socket says 172.21.15.46:34972) 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/2503165549 learned_addr learned my addr 172.21.15.46:0/2503165549 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf140a700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec159290 0x7f0cec0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/2503165549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec159290 msgr2=0x7f0cec0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec159290 0x7f0cec0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/2503165549 >> v1:172.21.15.132:6789/0 conn(0x7f0cec0d07b0 legacy=0x7f0cec0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0ce3fff700 1 -- 172.21.15.46:0/2503165549 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577715786 0 0) 0x7f0cec0ae340 con 0x7f0cec0d07b0 2023-10-06T11:36:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/2503165549 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0b3450 con 0x7f0cec0b1300 2023-10-06T11:36:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec0b1300 0x7f0cec0ae7d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009f20 tx=0x7f0ce4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383cc7dd5bd8db83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0ce3fff700 1 -- 172.21.15.46:0/2503165549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400e070 con 0x7f0cec0b1300 2023-10-06T11:36:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0ce3fff700 1 -- 172.21.15.46:0/2503165549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400ba60 con 0x7f0cec0b1300 2023-10-06T11:36:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.845+0000 7f0ce3fff700 1 -- 172.21.15.46:0/2503165549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40092e0 con 0x7f0cec0b1300 2023-10-06T11:36:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 -- 172.21.15.46:0/2503165549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec0b1300 msgr2=0x7f0cec0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec0b1300 0x7f0cec0ae7d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009f20 tx=0x7f0ce4009840 comp rx=0 tx=0).stop 2023-10-06T11:36:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 -- 172.21.15.46:0/2503165549 shutdown_connections 2023-10-06T11:36:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec0b1300 0x7f0cec0ae7d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 --2- 172.21.15.46:0/2503165549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec159290 0x7f0cec0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 -- 172.21.15.46:0/2503165549 >> 172.21.15.46:0/2503165549 conn(0x7f0cec06ef60 msgr2=0x7f0cec0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 -- 172.21.15.46:0/2503165549 shutdown_connections 2023-10-06T11:36:19.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 -- 172.21.15.46:0/2503165549 wait complete. 2023-10-06T11:36:19.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.849+0000 7f0cf240c700 1 Processor -- start 2023-10-06T11:36:19.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 -- start start 2023-10-06T11:36:19.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec0b1300 0x7f0cec1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 0x7f0cec151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec152ec0 con 0x7f0cec159290 2023-10-06T11:36:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec1522b0 con 0x7f0cec0b1300 2023-10-06T11:36:19.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf240c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec152590 con 0x7f0cec0d07b0 2023-10-06T11:36:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34986/0 (socket says 172.21.15.46:34986) 2023-10-06T11:36:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/1862522419 learned_addr learned my addr 172.21.15.46:0/1862522419 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf140a700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec0b1300 0x7f0cec1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 0x7f0cec151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 msgr2=0x7f0cec151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 0x7f0cec151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec0b1300 msgr2=0x7f0cec1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec0b1300 0x7f0cec1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/1862522419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009ae0 con 0x7f0cec159290 2023-10-06T11:36:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.853+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 0x7f0cec151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009880 tx=0x7f0ce4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0ce17fa700 1 -- 172.21.15.46:0/1862522419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40092e0 con 0x7f0cec159290 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0ce17fa700 1 -- 172.21.15.46:0/1862522419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4008e10 con 0x7f0cec159290 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0ce17fa700 1 -- 172.21.15.46:0/1862522419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d650 con 0x7f0cec159290 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec152870 con 0x7f0cec159290 2023-10-06T11:36:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.857+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec151fc0 con 0x7f0cec159290 2023-10-06T11:36:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.861+0000 7f0ce17fa700 1 -- 172.21.15.46:0/1862522419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0ce4003040 con 0x7f0cec159290 2023-10-06T11:36:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.861+0000 7f0ce17fa700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 0x7f0cd0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.861+0000 7f0ce17fa700 1 -- 172.21.15.46:0/1862522419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0ce407e970 con 0x7f0cec159290 2023-10-06T11:36:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.861+0000 7f0cf140a700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 0x7f0cd0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.861+0000 7f0cf140a700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 0x7f0cd0062b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009d10 tx=0x7f0ce80091a0 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 Processor -- start 2023-10-06T11:36:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 -- start start 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f58e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f59900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e7ef30 con 0x1cd3710 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e7aed0 con 0x1d1e790 2023-10-06T11:36:19.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e410b0 con 0x1cd3ae0 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34994/0 (socket says 172.21.15.46:34994) 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 -- 172.21.15.46:0/472498268 learned_addr learned my addr 172.21.15.46:0/472498268 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf140a700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f58e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f59900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 -- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 msgr2=0x1f58e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f58e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 -- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 msgr2=0x1f59900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f59900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 -- 172.21.15.46:0/472498268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d1f9e0 con 0x1cd3710 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.865+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc001730 tx=0x7f0cdc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed182042e17f61b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cd67fc700 1 -- 172.21.15.46:0/472498268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00e680 con 0x1cd3710 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cd67fc700 1 -- 172.21.15.46:0/472498268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00e7e0 con 0x1cd3710 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cd67fc700 1 -- 172.21.15.46:0/472498268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc00eba0 con 0x1cd3710 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- 172.21.15.46:0/472498268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 msgr2=0x1f593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc001730 tx=0x7f0cdc009040 comp rx=0 tx=0).stop 2023-10-06T11:36:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- 172.21.15.46:0/472498268 shutdown_connections 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f59900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1f593c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 --2- 172.21.15.46:0/472498268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f58e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- 172.21.15.46:0/472498268 >> 172.21.15.46:0/472498268 conn(0x1cd3eb0 msgr2=0x1f0d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- 172.21.15.46:0/472498268 shutdown_connections 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- 172.21.15.46:0/472498268 wait complete. 2023-10-06T11:36:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 -- start start 2023-10-06T11:36:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.869+0000 7f0cf5929740 1 Processor -- start 2023-10-06T11:36:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e40d50 con 0x1d1e790 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e19970 con 0x1cd3710 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1db7430 con 0x1cd3ae0 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 10 client.0 _ll_register_callbacks cb 0x1e1db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35010/0 (socket says 172.21.15.46:35010) 2023-10-06T11:36:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/3258859084 learned_addr learned my addr 172.21.15.46:0/3258859084 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 msgr2=0x1f5d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f5d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 msgr2=0x1f5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/3258859084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc00f040 con 0x1d1e790 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b810 tx=0x7f0ce4008f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f2fa1c1a38f635 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400e040 con 0x1d1e790 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0ce4059620 con 0x1d1e790 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e19cc0 con 0x1d1e790 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce4052af0 con 0x1d1e790 2023-10-06T11:36:19.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 10 client.7986 fetch_fsmap learned FSMap version 59 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x1e49370 con 0x1d1e790 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce407c080 con 0x1d1e790 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0ce4051b30 con 0x1d1e790 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 10 client.7986 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 10 client.7986 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 20 client.7986 populate_metadata read hostname 'smithi046' 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1f0b8e0 0x1f68420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 4 client.7986 mds_command: new command op to 7824 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 --> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1e1d3e0 con 0x1f0b8e0 2023-10-06T11:36:19.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.873+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1f0b8e0 0x1f68420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1f0b8e0 0x1f68420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af6f4be4aa1aee2 server_cookie=2dcd5c15d37c92a4 in_seq=0 out_seq=0 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0ccb7fe700 10 client.7986 ms_handle_connect on v2:172.21.15.46:6834/2696512883 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3258859084 <== mds.0 v2:172.21.15.46:6834/2696512883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1e1d3e0 con 0x1f0b8e0 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0ccb7fe700 10 client.7986 handle_command_reply: tid=0 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 client.7986 shutdown 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 msgr2=0x1f0a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b810 tx=0x7f0ce4008f80 comp rx=0 tx=0).stop 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 shutdown_connections 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1f0b8e0 0x1f68420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f0a350 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0ccb7fe700 0 client.7986 ms_handle_reset on v2:172.21.15.46:6834/2696512883 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3258859084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 >> 172.21.15.46:0/3258859084 conn(0x1cd3eb0 msgr2=0x1f321d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 shutdown_connections 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 1 -- 172.21.15.46:0/3258859084 wait complete. 2023-10-06T11:36:19.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.877+0000 7f0cf5929740 20 client.7986 trim_cache size 0 max 16384 2023-10-06T11:36:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.897+0000 7f0cf5929740 1 Processor -- start 2023-10-06T11:36:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.897+0000 7f0cf5929740 1 -- start start 2023-10-06T11:36:19.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f0c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e81290 con 0x1d1e790 2023-10-06T11:36:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f639d0 con 0x1cd3710 2023-10-06T11:36:19.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e77150 con 0x1cd3ae0 2023-10-06T11:36:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35026/0 (socket says 172.21.15.46:35026) 2023-10-06T11:36:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/4004613769 learned_addr learned my addr 172.21.15.46:0/4004613769 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf140a700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f0c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 msgr2=0x1f0caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f0caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 msgr2=0x1f0c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f0c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 -- 172.21.15.46:0/4004613769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e2fd80 con 0x1d1e790 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf140a700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f0c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b7e0 tx=0x7f0ce4066000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd8e3735d17a338 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cd6ffd700 1 -- 172.21.15.46:0/4004613769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400e070 con 0x1d1e790 2023-10-06T11:36:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cd6ffd700 1 -- 172.21.15.46:0/4004613769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce407c040 con 0x1d1e790 2023-10-06T11:36:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.901+0000 7f0cd6ffd700 1 -- 172.21.15.46:0/4004613769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4009440 con 0x1d1e790 2023-10-06T11:36:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- 172.21.15.46:0/4004613769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 msgr2=0x1f08a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b7e0 tx=0x7f0ce4066000 comp rx=0 tx=0).stop 2023-10-06T11:36:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- 172.21.15.46:0/4004613769 shutdown_connections 2023-10-06T11:36:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d1e790 0x1f08a10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1f0caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- 172.21.15.46:0/4004613769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cd3710 0x1f0c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- 172.21.15.46:0/4004613769 >> 172.21.15.46:0/4004613769 conn(0x1cd3eb0 msgr2=0x1f341e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- 172.21.15.46:0/4004613769 shutdown_connections 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- 172.21.15.46:0/4004613769 wait complete. 2023-10-06T11:36:19.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- start start 2023-10-06T11:36:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 Processor -- start 2023-10-06T11:36:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1fb35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e7b770 con 0x1cd3710 2023-10-06T11:36:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e81400 con 0x1d1e790 2023-10-06T11:36:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e24ee0 con 0x1cd3ae0 2023-10-06T11:36:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.905+0000 7f0cf5929740 10 client.0 _ll_register_callbacks cb 0x1e46110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35034/0 (socket says 172.21.15.46:35034) 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 -- 172.21.15.46:0/3481835992 learned_addr learned my addr 172.21.15.46:0/3481835992 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1fb35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf1c0b700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 -- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 msgr2=0x1fb35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1fb35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 -- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 msgr2=0x1f673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 -- 172.21.15.46:0/3481835992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e49370 con 0x1cd3710 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf140a700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80119c0 tx=0x7f0ce8025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f49e3519ab07684 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8028050 con 0x1cd3710 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f0ce801a040 con 0x1cd3710 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cdc00e7e0 con 0x1cd3710 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce802c470 con 0x1cd3710 2023-10-06T11:36:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 10 client.7992 fetch_fsmap learned FSMap version 59 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x1e24130 con 0x1cd3710 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc00e7e0 con 0x1cd3710 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 59) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0ce8034350 con 0x1cd3710 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 10 client.7992 fetch_fsmap finished waiting for FSMap version 59 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 10 client.7992 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 20 client.7992 populate_metadata read hostname 'smithi046' 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1fb9fc0 0x1fbc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 4 client.7992 mds_command: new command op to 7824 tid=0[{"prefix": "status"}] 2023-10-06T11:36:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.909+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 --> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1e4f410 con 0x1fb9fc0 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1fb9fc0 0x1fbc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf0c09700 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1fb9fc0 0x1fbc3a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db264e2d731752ef server_cookie=63ff420086910476 in_seq=0 out_seq=0 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0ccb7fe700 10 client.7992 ms_handle_connect on v2:172.21.15.46:6834/2696512883 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0ccb7fe700 1 -- 172.21.15.46:0/3481835992 <== mds.0 v2:172.21.15.46:6834/2696512883 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x1e4f410 con 0x1fb9fc0 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0ccb7fe700 10 client.7992 handle_command_reply: tid=0 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 client.7992 shutdown 2023-10-06T11:36:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 msgr2=0x1fb30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80119c0 tx=0x7f0ce8025040 comp rx=0 tx=0).stop 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 shutdown_connections 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:6834/2696512883,v1:172.21.15.46:6836/2696512883] conn(0x1fb9fc0 0x1fbc3a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d1e790 0x1f673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cd3ae0 0x1fb35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 --2- 172.21.15.46:0/3481835992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1cd3710 0x1fb30b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0ccb7fe700 0 client.7992 ms_handle_reset on v2:172.21.15.46:6834/2696512883 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 >> 172.21.15.46:0/3481835992 conn(0x1cd3eb0 msgr2=0x1f0c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 shutdown_connections 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 1 -- 172.21.15.46:0/3481835992 wait complete. 2023-10-06T11:36:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf5929740 20 client.7992 trim_cache size 0 max 16384 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7824, 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 11.086737189999999, 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 59, 2023-10-06T11:36:19.929 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 122, 2023-10-06T11:36:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 122, 2023-10-06T11:36:19.930 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 17.018690649 2023-10-06T11:36:19.930 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 msgr2=0x7f0cd0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 0x7f0cd0062b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009d10 tx=0x7f0ce80091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 msgr2=0x7f0cec14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009880 tx=0x7f0ce4018000 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 shutdown_connections 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cd0060670 0x7f0cd0062b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec159290 0x7f0cec14ae80 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec0d07b0 0x7f0cec151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 --2- 172.21.15.46:0/1862522419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec0b1300 0x7f0cec1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.913+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 >> 172.21.15.46:0/1862522419 conn(0x7f0cec06ef60 msgr2=0x7f0cec14dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.917+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 shutdown_connections 2023-10-06T11:36:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:19.917+0000 7f0cf240c700 1 -- 172.21.15.46:0/1862522419 wait complete. 2023-10-06T11:36:19.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:36:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.105+0000 7f26944f0700 1 Processor -- start 2023-10-06T11:36:20.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.105+0000 7f26944f0700 1 -- start start 2023-10-06T11:36:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f26944f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f268c156c80 0x7f268c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f26944f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f268c0c7fc0 0x7f268c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f26944f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0c6470 con 0x7f268c0c7b50 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f26944f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0c7760 con 0x7f268c156c80 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f26944f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0ca710 con 0x7f268c0c7fc0 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f2691a8b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f268c0c7b50 0x7f268c15cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56470/0 (socket says 172.21.15.46:56470) 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f2691a8b700 1 -- 172.21.15.46:0/2883604211 learned_addr learned my addr 172.21.15.46:0/2883604211 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 --2- 172.21.15.46:0/2883604211 >> v2:172.21.15.46:3300/0 conn(0x7f268c156c80 0x7f268c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f2692a8d700 1 --2- 172.21.15.46:0/2883604211 >> v2:172.21.15.132:3300/0 conn(0x7f268c0c7fc0 0x7f268c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269128a700 1 -- 172.21.15.46:0/2883604211 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511566936 0 0) 0x7f268c0c6470 con 0x7f268c0c7b50 2023-10-06T11:36:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269128a700 1 -- 172.21.15.46:0/2883604211 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003570 con 0x7f268c0c7b50 2023-10-06T11:36:20.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 -- 172.21.15.46:0/2883604211 >> v2:172.21.15.132:3300/0 conn(0x7f268c0c7fc0 msgr2=0x7f268c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 --2- 172.21.15.46:0/2883604211 >> v2:172.21.15.132:3300/0 conn(0x7f268c0c7fc0 0x7f268c15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 -- 172.21.15.46:0/2883604211 >> v1:172.21.15.46:6789/0 conn(0x7f268c0c7b50 legacy=0x7f268c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 -- 172.21.15.46:0/2883604211 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0c66c0 con 0x7f268c156c80 2023-10-06T11:36:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.109+0000 7f269228c700 1 --2- 172.21.15.46:0/2883604211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c156c80 0x7f268c0ca1d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f268c06cbc0 tx=0x7f26880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9e881dc57dd367 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f269128a700 1 -- 172.21.15.46:0/2883604211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688011630 con 0x7f268c156c80 2023-10-06T11:36:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f269128a700 1 -- 172.21.15.46:0/2883604211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688011790 con 0x7f268c156c80 2023-10-06T11:36:20.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f269128a700 1 -- 172.21.15.46:0/2883604211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688011b10 con 0x7f268c156c80 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- 172.21.15.46:0/2883604211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c156c80 msgr2=0x7f268c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 --2- 172.21.15.46:0/2883604211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c156c80 0x7f268c0ca1d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f268c06cbc0 tx=0x7f26880091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- 172.21.15.46:0/2883604211 shutdown_connections 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 --2- 172.21.15.46:0/2883604211 >> v2:172.21.15.132:3300/0 conn(0x7f268c0c7fc0 0x7f268c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 --2- 172.21.15.46:0/2883604211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c156c80 0x7f268c0ca1d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- 172.21.15.46:0/2883604211 >> 172.21.15.46:0/2883604211 conn(0x7f268c0cbe30 msgr2=0x7f268c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- 172.21.15.46:0/2883604211 shutdown_connections 2023-10-06T11:36:20.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- 172.21.15.46:0/2883604211 wait complete. 2023-10-06T11:36:20.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 Processor -- start 2023-10-06T11:36:20.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.113+0000 7f26944f0700 1 -- start start 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 0x7f268c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c156c80 0x7f268c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b03c0 con 0x7f268c0c7fc0 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0c4b30 con 0x7f268c0c7b50 2023-10-06T11:36:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0af7b0 con 0x7f268c156c80 2023-10-06T11:36:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35050/0 (socket says 172.21.15.46:35050) 2023-10-06T11:36:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 -- 172.21.15.46:0/600945862 learned_addr learned my addr 172.21.15.46:0/600945862 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2692a8d700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c156c80 0x7f268c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f269228c700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 0x7f268c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 -- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c156c80 msgr2=0x7f268c0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c156c80 0x7f268c0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 -- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 msgr2=0x7f268c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 0x7f268c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 -- 172.21.15.46:0/600945862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880093c0 con 0x7f268c0c7fc0 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f269228c700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 0x7f268c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f2691a8b700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f267c0011e0 tx=0x7f267c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c005b30 con 0x7f268c0c7fc0 2023-10-06T11:36:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c005c90 con 0x7f268c0c7fc0 2023-10-06T11:36:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0afa90 con 0x7f268c0c7fc0 2023-10-06T11:36:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c007320 con 0x7f268c0c7fc0 2023-10-06T11:36:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.117+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0b0050 con 0x7f268c0c7fc0 2023-10-06T11:36:20.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.121+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c0cb020 con 0x7f268c0c7fc0 2023-10-06T11:36:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.121+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f267c007690 con 0x7f268c0c7fc0 2023-10-06T11:36:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.125+0000 7f26827fc700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 0x7f2670062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.125+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f267c07c0d0 con 0x7f268c0c7fc0 2023-10-06T11:36:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.125+0000 7f269228c700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 0x7f2670062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.125+0000 7f269228c700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 0x7f2670062b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2688009390 tx=0x7f2688017000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.129+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f268c0cb020 con 0x7f268c0c7fc0 2023-10-06T11:36:20.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.333+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f268c0cb020 con 0x7f268c0c7fc0 2023-10-06T11:36:20.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.333+0000 7f26827fc700 1 -- 172.21.15.46:0/600945862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7f268c0cb020 con 0x7f268c0c7fc0 2023-10-06T11:36:20.348 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:36:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 msgr2=0x7f2670062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 0x7f2670062b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2688009390 tx=0x7f2688017000 comp rx=0 tx=0).stop 2023-10-06T11:36:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 msgr2=0x7f268c0aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:20.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f267c0011e0 tx=0x7f267c007df0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 shutdown_connections 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2670060670 0x7f2670062b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f268c156c80 0x7f268c0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268c0c7fc0 0x7f268c0aebd0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 --2- 172.21.15.46:0/600945862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f268c0c7b50 0x7f268c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 >> 172.21.15.46:0/600945862 conn(0x7f268c0cbe30 msgr2=0x7f268c151960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 shutdown_connections 2023-10-06T11:36:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:20.337+0000 7f26944f0700 1 -- 172.21.15.46:0/600945862 wait complete. 2023-10-06T11:36:20.371 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:36:20.371 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:36:20.371 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:36:20.371 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:36:20.372 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:36:20.372 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:36:20.372 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:36:20.372 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:36:20.372 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:36:20.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.372 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:36:20.386 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:36:20.386 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:36:20.387 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41350sec preferred_lft 41350sec 2023-10-06T11:36:20.388 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.389 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.390 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.391 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:36:20.392 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:36:20.392 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:36:20.392 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:36:20.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.394 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:36:20.475 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:36:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.479 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:36:20.544 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:36:20.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.545 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:36:20.546 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:36:20.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:36:20.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:36:20.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:36:20.546 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:36:20.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.684 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:36:20.702 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:36:20.702 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:36:20.703 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:36:20.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:20.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:36:20.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:36:20.786 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:36:20.847 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:36:20.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:36:20.848 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:36:20.910 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:36:20.910 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:36:20.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:36:21.040 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:36:21.041 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:36:21.041 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:36:21.067 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:36:21.067 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:36:21.067 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:36:21.067 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:36:21.067 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:36:21.081 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:36:21.082 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:36:21.082 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:36:21.086 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:36:21.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:36:21.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:36:21.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:36:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.317+0000 7f6a3aa6e700 1 Processor -- start 2023-10-06T11:36:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.317+0000 7f6a3aa6e700 1 -- start start 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a3415c770 0x7f6a340cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a340ccb50 0x7f6a340ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6a340cea30 0x7f6a340c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cfbb0 con 0x7f6a340ccb50 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cdbf0 con 0x7f6a340cea30 2023-10-06T11:36:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3aa6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340cc150 con 0x7f6a3415c770 2023-10-06T11:36:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a340ccb50 0x7f6a340ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a340ccb50 0x7f6a340ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35060/0 (socket says 172.21.15.46:35060) 2023-10-06T11:36:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 -- 172.21.15.46:0/1745481927 learned_addr learned my addr 172.21.15.46:0/1745481927 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:21.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3880a700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a3415c770 0x7f6a340cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a3900b700 1 --2- 172.21.15.46:0/1745481927 >> v2:172.21.15.132:3300/0 conn(0x7f6a340cea30 0x7f6a340c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 -- 172.21.15.46:0/1745481927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a3415c770 msgr2=0x7f6a340cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a3415c770 0x7f6a340cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 -- 172.21.15.46:0/1745481927 >> v2:172.21.15.132:3300/0 conn(0x7f6a340cea30 msgr2=0x7f6a340c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 --2- 172.21.15.46:0/1745481927 >> v2:172.21.15.132:3300/0 conn(0x7f6a340cea30 0x7f6a340c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 -- 172.21.15.46:0/1745481927 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cee70 con 0x7f6a340ccb50 2023-10-06T11:36:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a33fff700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a340ce4f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6a340cf000 tx=0x7f6a20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24add7f60fc3f808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a337fe700 1 -- 172.21.15.46:0/1745481927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20011630 con 0x7f6a340ccb50 2023-10-06T11:36:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a337fe700 1 -- 172.21.15.46:0/1745481927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20011790 con 0x7f6a340ccb50 2023-10-06T11:36:21.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.321+0000 7f6a337fe700 1 -- 172.21.15.46:0/1745481927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20011b50 con 0x7f6a340ccb50 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/1745481927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 msgr2=0x7f6a340ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a340ce4f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6a340cf000 tx=0x7f6a20009250 comp rx=0 tx=0).stop 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/1745481927 shutdown_connections 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/1745481927 >> v2:172.21.15.132:3300/0 conn(0x7f6a340cea30 0x7f6a340c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a340ce4f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/1745481927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a3415c770 0x7f6a340cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/1745481927 >> 172.21.15.46:0/1745481927 conn(0x7f6a34099480 msgr2=0x7f6a3406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:21.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/1745481927 shutdown_connections 2023-10-06T11:36:21.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/1745481927 wait complete. 2023-10-06T11:36:21.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.325+0000 7f6a3aa6e700 1 Processor -- start 2023-10-06T11:36:21.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 -- start start 2023-10-06T11:36:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 0x7f6a34155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 0x7f6a341560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3416e020 con 0x7f6a340ccb50 2023-10-06T11:36:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3416ec80 con 0x7f6a3415c770 2023-10-06T11:36:21.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3aa6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c82c0 con 0x7f6a340cea30 2023-10-06T11:36:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35064/0 (socket says 172.21.15.46:35064) 2023-10-06T11:36:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 -- 172.21.15.46:0/173596460 learned_addr learned my addr 172.21.15.46:0/173596460 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:21.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a33fff700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 0x7f6a34155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3900b700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 0x7f6a341560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 -- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 msgr2=0x7f6a34155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 0x7f6a34155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 -- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 msgr2=0x7f6a341560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 0x7f6a341560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 -- 172.21.15.46:0/173596460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200094f0 con 0x7f6a340ccb50 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a33fff700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 0x7f6a34155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3900b700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 0x7f6a341560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.329+0000 7f6a3880a700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ea50 tx=0x7f6a28009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800c920 con 0x7f6a340ccb50 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2800ca80 con 0x7f6a340ccb50 2023-10-06T11:36:21.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800ce40 con 0x7f6a340ccb50 2023-10-06T11:36:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340c8880 con 0x7f6a340ccb50 2023-10-06T11:36:21.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a34158d60 con 0x7f6a340ccb50 2023-10-06T11:36:21.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.333+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a340660f0 con 0x7f6a340ccb50 2023-10-06T11:36:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.337+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6a28013680 con 0x7f6a340ccb50 2023-10-06T11:36:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.337+0000 7f6a30ff9700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 0x7f6a18062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.337+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f6a2807d400 con 0x7f6a340ccb50 2023-10-06T11:36:21.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.337+0000 7f6a33fff700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 0x7f6a18062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.337+0000 7f6a33fff700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 0x7f6a18062b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a340cf000 tx=0x7f6a20016000 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.345+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6a2804d520 con 0x7f6a340ccb50 2023-10-06T11:36:21.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.541+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a3404b5c0 con 0x7f6a18060670 2023-10-06T11:36:21.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.545+0000 7f6a30ff9700 1 -- 172.21.15.46:0/173596460 <== mgr.7408 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6a3404b5c0 con 0x7f6a18060670 2023-10-06T11:36:21.721 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:36:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:36:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:36:21.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:36:21.721 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:36:21.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 msgr2=0x7f6a18062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 0x7f6a18062b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a340cf000 tx=0x7f6a20016000 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 msgr2=0x7f6a3416e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ea50 tx=0x7f6a28009120 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 shutdown_connections 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6a18060670 0x7f6a18062b20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a3415c770 0x7f6a341560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a340cea30 0x7f6a34155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 --2- 172.21.15.46:0/173596460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a340ccb50 0x7f6a3416e740 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 >> 172.21.15.46:0/173596460 conn(0x7f6a34099480 msgr2=0x7f6a34068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:21.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 shutdown_connections 2023-10-06T11:36:21.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.549+0000 7f6a3aa6e700 1 -- 172.21.15.46:0/173596460 wait complete. 2023-10-06T11:36:21.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:36:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 Processor -- start 2023-10-06T11:36:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 -- start start 2023-10-06T11:36:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe4d80ca430 0x7fe4d80c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d80c1630 con 0x7fe4d80c7d80 2023-10-06T11:36:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d80c0d00 con 0x7fe4d80a9840 2023-10-06T11:36:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4df785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d80c0fe0 con 0x7fe4d80ca430 2023-10-06T11:36:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4dd521700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe4d80c7d80 0x7fe4d80c6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56474/0 (socket says 172.21.15.46:56474) 2023-10-06T11:36:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4dd521700 1 -- 172.21.15.46:0/3163009668 learned_addr learned my addr 172.21.15.46:0/3163009668 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.877+0000 7fe4dcd20700 1 --2- 172.21.15.46:0/3163009668 >> v2:172.21.15.132:3300/0 conn(0x7fe4d80ca430 0x7fe4d80c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719905949 0 0) 0x7fe4d80c1630 con 0x7fe4d80c7d80 2023-10-06T11:36:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4c4003570 con 0x7fe4d80c7d80 2023-10-06T11:36:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420709866 0 0) 0x7fe4d80c0d00 con 0x7fe4d80a9840 2023-10-06T11:36:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d80c1630 con 0x7fe4d80a9840 2023-10-06T11:36:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4dcd20700 1 -- 172.21.15.46:0/3163009668 >> v1:172.21.15.132:6789/0 conn(0x7fe4d80a9840 legacy=0x7fe4d80c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4dcd20700 1 -- 172.21.15.46:0/3163009668 >> v1:172.21.15.46:6789/0 conn(0x7fe4d80c7d80 legacy=0x7fe4d80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4dcd20700 1 -- 172.21.15.46:0/3163009668 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80c12c0 con 0x7fe4d80ca430 2023-10-06T11:36:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4dcd20700 1 --2- 172.21.15.46:0/3163009668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d80c8150 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80098e0 tx=0x7fe4c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581a91218ff37927 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8011630 con 0x7fe4d80ca430 2023-10-06T11:36:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8011790 con 0x7fe4d80ca430 2023-10-06T11:36:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4cffff700 1 -- 172.21.15.46:0/3163009668 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8011b30 con 0x7fe4d80ca430 2023-10-06T11:36:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 -- 172.21.15.46:0/3163009668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 msgr2=0x7fe4d80c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 --2- 172.21.15.46:0/3163009668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d80c8150 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80098e0 tx=0x7fe4c80091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 -- 172.21.15.46:0/3163009668 shutdown_connections 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 --2- 172.21.15.46:0/3163009668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d80c8150 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 -- 172.21.15.46:0/3163009668 >> 172.21.15.46:0/3163009668 conn(0x7fe4d80cbe30 msgr2=0x7fe4d80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 -- 172.21.15.46:0/3163009668 shutdown_connections 2023-10-06T11:36:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.881+0000 7fe4df785700 1 -- 172.21.15.46:0/3163009668 wait complete. 2023-10-06T11:36:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.885+0000 7fe4df785700 1 Processor -- start 2023-10-06T11:36:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.885+0000 7fe4df785700 1 -- start start 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4d80a9840 0x7fe4d8155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d815e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815d700 con 0x7fe4d80c7d80 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80ac180 con 0x7fe4d80ca430 2023-10-06T11:36:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80ab890 con 0x7fe4d80a9840 2023-10-06T11:36:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35076/0 (socket says 172.21.15.46:35076) 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 -- 172.21.15.46:0/4259504914 learned_addr learned my addr 172.21.15.46:0/4259504914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4ddd22700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dd521700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4d80a9840 0x7fe4d8155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 -- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4d80a9840 msgr2=0x7fe4d8155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4d80a9840 0x7fe4d8155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 -- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 msgr2=0x7fe4d815e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d815e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 -- 172.21.15.46:0/4259504914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8009420 con 0x7fe4d80c7d80 2023-10-06T11:36:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4dcd20700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe4d806c150 tx=0x7fe4c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c801f070 con 0x7fe4d80c7d80 2023-10-06T11:36:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8000bc0 con 0x7fe4d80c7d80 2023-10-06T11:36:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8025420 con 0x7fe4d80c7d80 2023-10-06T11:36:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d80abb30 con 0x7fe4d80c7d80 2023-10-06T11:36:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.889+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d8155ff0 con 0x7fe4d80c7d80 2023-10-06T11:36:21.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.893+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d80660f0 con 0x7fe4d80c7d80 2023-10-06T11:36:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.893+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 53) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe4c8023070 con 0x7fe4d80c7d80 2023-10-06T11:36:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.893+0000 7fe4cd7fa700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 0x7fe4bc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.893+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fe4c807dde0 con 0x7fe4d80c7d80 2023-10-06T11:36:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.893+0000 7fe4dd521700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 0x7fe4bc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.897+0000 7fe4dd521700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 0x7fe4bc062b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4007fd0 tx=0x7fe4d4009040 comp rx=0 tx=0).ready entity=mgr.7408 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:21.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:21.901+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe4c804df00 con 0x7fe4d80c7d80 2023-10-06T11:36:22.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.109+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe4d804b5c0 con 0x7fe4d80c7d80 2023-10-06T11:36:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.609+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 54) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fe4c804e2d0 con 0x7fe4d80c7d80 2023-10-06T11:36:22.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.613+0000 7fe4cd7fa700 1 -- 172.21.15.46:0/4259504914 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v54) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe4c8051990 con 0x7fe4d80c7d80 2023-10-06T11:36:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 msgr2=0x7fe4bc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 0x7fe4bc062b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4007fd0 tx=0x7fe4d4009040 comp rx=0 tx=0).stop 2023-10-06T11:36:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 msgr2=0x7fe4d8159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fe4d806c150 tx=0x7fe4c8017000 comp rx=0 tx=0).stop 2023-10-06T11:36:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 shutdown_connections 2023-10-06T11:36:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4bc060670 0x7fe4bc062b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4d80ca430 0x7fe4d815e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4d80c7d80 0x7fe4d8159830 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 --2- 172.21.15.46:0/4259504914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4d80a9840 0x7fe4d8155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 >> 172.21.15.46:0/4259504914 conn(0x7fe4d80cbe30 msgr2=0x7fe4d80c1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 shutdown_connections 2023-10-06T11:36:22.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.617+0000 7fe4df785700 1 -- 172.21.15.46:0/4259504914 wait complete. 2023-10-06T11:36:22.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:36:22.804 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:22.802+0000 7f3b559c3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:36:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 Processor -- start 2023-10-06T11:36:22.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 -- start start 2023-10-06T11:36:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 0x7fcc7c069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7c0ce6e0 con 0x7fcc7c159850 2023-10-06T11:36:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7c15a5c0 con 0x7fcc7c069270 2023-10-06T11:36:22.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8228e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c15a3b0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc8128c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 0x7fcc7c069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.801+0000 7fcc80a8b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:56114/0 (socket says 172.21.15.46:56114) 2023-10-06T11:36:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 -- 172.21.15.46:0/525562768 learned_addr learned my addr 172.21.15.46:0/525562768 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:22.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8128c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 0x7fcc7c069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35098/0 (socket says 172.21.15.46:35098) 2023-10-06T11:36:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 -- 172.21.15.46:0/525562768 >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 msgr2=0x7fcc7c069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 --2- 172.21.15.46:0/525562768 >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 0x7fcc7c069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 -- 172.21.15.46:0/525562768 >> v1:172.21.15.132:6789/0 conn(0x7fcc7c159850 legacy=0x7fcc7c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 -- 172.21.15.46:0/525562768 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0cddb0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc80a8b700 1 --2- 172.21.15.46:0/525562768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0099b0 tx=0x7fcc6c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f849bf6d0249ed53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc73fff700 1 -- 172.21.15.46:0/525562768 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c009ea0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc73fff700 1 -- 172.21.15.46:0/525562768 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c00b860 con 0x7fcc7c158aa0 2023-10-06T11:36:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc73fff700 1 -- 172.21.15.46:0/525562768 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00bc00 con 0x7fcc7c158aa0 2023-10-06T11:36:22.819 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:22.805+0000 7f2d96343000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:36:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 -- 172.21.15.46:0/525562768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 msgr2=0x7fcc7c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 --2- 172.21.15.46:0/525562768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0099b0 tx=0x7fcc6c009250 comp rx=0 tx=0).stop 2023-10-06T11:36:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 -- 172.21.15.46:0/525562768 shutdown_connections 2023-10-06T11:36:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 --2- 172.21.15.46:0/525562768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c158aa0 0x7fcc7c158f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 --2- 172.21.15.46:0/525562768 >> v2:172.21.15.46:3300/0 conn(0x7fcc7c069270 0x7fcc7c069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.805+0000 7fcc8228e700 1 -- 172.21.15.46:0/525562768 >> 172.21.15.46:0/525562768 conn(0x7fcc7c068390 msgr2=0x7fcc7c0d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:22.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.809+0000 7fcc8228e700 1 -- 172.21.15.46:0/525562768 shutdown_connections 2023-10-06T11:36:22.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.809+0000 7fcc8228e700 1 -- 172.21.15.46:0/525562768 wait complete. 2023-10-06T11:36:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.809+0000 7fcc8228e700 1 Processor -- start 2023-10-06T11:36:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.809+0000 7fcc8228e700 1 -- start start 2023-10-06T11:36:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c069270 0x7fcc7c16c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 0x7fcc7c16ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c15b190 con 0x7fcc7c158aa0 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c16cdd0 con 0x7fcc7c159850 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c15a860 con 0x7fcc7c069270 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35102/0 (socket says 172.21.15.46:35102) 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 -- 172.21.15.46:0/294512040 learned_addr learned my addr 172.21.15.46:0/294512040 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8128c700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c069270 0x7fcc7c16c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc81a8d700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 0x7fcc7c16ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 -- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c069270 msgr2=0x7fcc7c16c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c069270 0x7fcc7c16c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 -- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 msgr2=0x7fcc7c16ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 0x7fcc7c16ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 -- 172.21.15.46:0/294512040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0094f0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc81a8d700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 0x7fcc7c16ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc80a8b700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c15a4f0 tx=0x7fcc6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c00bbe0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c011420 con 0x7fcc7c158aa0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c15ab40 con 0x7fcc7c158aa0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.813+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0117c0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.817+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c0c70f0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.817+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c15b300 con 0x7fcc7c158aa0 2023-10-06T11:36:22.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.817+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fcc6c011bf0 con 0x7fcc7c158aa0 2023-10-06T11:36:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.821+0000 7fcc717fa700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 0x7fcc60062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.821+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fcc6c07e660 con 0x7fcc7c158aa0 2023-10-06T11:36:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.821+0000 7fcc8128c700 1 -- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 msgr2=0x7fcc60062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.821+0000 7fcc8128c700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 0x7fcc60062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:36:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.825+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc6c04e780 con 0x7fcc7c158aa0 2023-10-06T11:36:23.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:22.997+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fcc7c064590 con 0x7fcc7c158aa0 2023-10-06T11:36:23.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.005+0000 7fcc717fa700 1 -- 172.21.15.46:0/294512040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcc6c051da0 con 0x7fcc7c158aa0 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 msgr2=0x7fcc60062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 0x7fcc60062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 msgr2=0x7fcc7c16a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c15a4f0 tx=0x7fcc6c017000 comp rx=0 tx=0).stop 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 shutdown_connections 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc600606c0 0x7fcc60062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc7c159850 0x7fcc7c16ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc7c158aa0 0x7fcc7c16a8a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 --2- 172.21.15.46:0/294512040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc7c069270 0x7fcc7c16c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 >> 172.21.15.46:0/294512040 conn(0x7fcc7c068390 msgr2=0x7fcc7c157c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 shutdown_connections 2023-10-06T11:36:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.009+0000 7fcc8228e700 1 -- 172.21.15.46:0/294512040 wait complete. 2023-10-06T11:36:23.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:36:23.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 Processor -- start 2023-10-06T11:36:23.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 -- start start 2023-10-06T11:36:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 0x7f57740c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57740c6250 0x7f577406c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f577406cf30 0x7f57740c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57740c1d90 con 0x7f577406cf30 2023-10-06T11:36:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57740c1460 con 0x7f57740c97e0 2023-10-06T11:36:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.189+0000 7f577a7cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57740c1740 con 0x7f57740c6250 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f577406cf30 0x7f57740c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f577406cf30 0x7f57740c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48192/0 (socket says 172.21.15.46:48192) 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 -- 172.21.15.46:0/2326240348 learned_addr learned my addr 172.21.15.46:0/2326240348 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f57797ca700 1 --2- 172.21.15.46:0/2326240348 >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 0x7f57740c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5778fc9700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57740c6250 0x7f577406c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 -- 172.21.15.46:0/2326240348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57740c6250 msgr2=0x7f577406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57740c6250 0x7f577406c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 -- 172.21.15.46:0/2326240348 >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 msgr2=0x7f57740c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 --2- 172.21.15.46:0/2326240348 >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 0x7f57740c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 -- 172.21.15.46:0/2326240348 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57740c1a20 con 0x7f577406cf30 2023-10-06T11:36:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f57797ca700 1 --2- 172.21.15.46:0/2326240348 >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 0x7f57740c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f5779fcb700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577406cf30 0x7f57740c2220 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f57740cc8f0 tx=0x7f576c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3750eb8c661b1bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f576bfff700 1 -- 172.21.15.46:0/2326240348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00e070 con 0x7f577406cf30 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f576bfff700 1 -- 172.21.15.46:0/2326240348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c00ba20 con 0x7f577406cf30 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f576bfff700 1 -- 172.21.15.46:0/2326240348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c0092e0 con 0x7f577406cf30 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 -- 172.21.15.46:0/2326240348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577406cf30 msgr2=0x7f57740c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577406cf30 0x7f57740c2220 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f57740cc8f0 tx=0x7f576c009840 comp rx=0 tx=0).stop 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 -- 172.21.15.46:0/2326240348 shutdown_connections 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577406cf30 0x7f57740c2220 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 --2- 172.21.15.46:0/2326240348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57740c6250 0x7f577406c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 --2- 172.21.15.46:0/2326240348 >> v2:172.21.15.132:3300/0 conn(0x7f57740c97e0 0x7f57740c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 -- 172.21.15.46:0/2326240348 >> 172.21.15.46:0/2326240348 conn(0x7f57740cbc10 msgr2=0x7f57740cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:23.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.193+0000 7f577a7cc700 1 -- 172.21.15.46:0/2326240348 shutdown_connections 2023-10-06T11:36:23.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.197+0000 7f577a7cc700 1 -- 172.21.15.46:0/2326240348 wait complete. 2023-10-06T11:36:23.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.197+0000 7f577a7cc700 1 Processor -- start 2023-10-06T11:36:23.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.197+0000 7f577a7cc700 1 -- start start 2023-10-06T11:36:23.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 0x7f577415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57740c97e0 0x7f57740ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577415a7c0 con 0x7f57740c6250 2023-10-06T11:36:23.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577415cc40 con 0x7f57740c97e0 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57740ac450 con 0x7f577406cf30 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48206/0 (socket says 172.21.15.46:48206) 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 -- 172.21.15.46:0/3077367213 learned_addr learned my addr 172.21.15.46:0/3077367213 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5779fcb700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57740c97e0 0x7f57740ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f57797ca700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 0x7f577415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 -- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 msgr2=0x7f577415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 0x7f577415c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 -- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57740c97e0 msgr2=0x7f57740ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57740c97e0 0x7f57740ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 -- 172.21.15.46:0/3077367213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c009ae0 con 0x7f57740c6250 2023-10-06T11:36:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f57797ca700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 0x7f577415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f5778fc9700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f576400e950 tx=0x7f57640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764013680 con 0x7f57740c6250 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57640137e0 con 0x7f57740c6250 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764013ba0 con 0x7f57740c6250 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57740ac730 con 0x7f57740c6250 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.201+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57740accf0 con 0x7f57740c6250 2023-10-06T11:36:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.205+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577406c2d0 con 0x7f57740c6250 2023-10-06T11:36:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.205+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f576400c490 con 0x7f57740c6250 2023-10-06T11:36:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.205+0000 7f57697fa700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 0x7f5758062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.209+0000 7f57797ca700 1 -- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 msgr2=0x7f5758062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.209+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f576407cec0 con 0x7f57740c6250 2023-10-06T11:36:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.209+0000 7f57797ca700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 0x7f5758062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:36:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.213+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f577406c2d0 con 0x7f57740c6250 2023-10-06T11:36:23.351 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:23.350+0000 7f3b559c3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:36:23.400 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:23.385+0000 7f2d96343000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:36:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.389+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f577406c2d0 con 0x7f57740c6250 2023-10-06T11:36:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.397+0000 7f57697fa700 1 -- 172.21.15.46:0/3077367213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f576404cfe0 con 0x7f57740c6250 2023-10-06T11:36:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 msgr2=0x7f5758062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:36:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 0x7f5758062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 msgr2=0x7f57740c4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f576400e950 tx=0x7f57640091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 shutdown_connections 2023-10-06T11:36:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f57580606f0 0x7f5758062ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57740c97e0 0x7f57740ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57740c6250 0x7f57740c4bf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 --2- 172.21.15.46:0/3077367213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f577406cf30 0x7f577415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 >> 172.21.15.46:0/3077367213 conn(0x7f57740cbc10 msgr2=0x7f57740caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 shutdown_connections 2023-10-06T11:36:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.401+0000 7f577a7cc700 1 -- 172.21.15.46:0/3077367213 wait complete. 2023-10-06T11:36:23.428 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:23.426+0000 7f3b559c3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:36:23.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:23.436 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:36:23.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T11:36:23.481 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:23.469+0000 7f2d96343000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:36:23.567 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:23.566+0000 7f3b559c3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:36:23.637 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:23.625+0000 7f2d96343000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:36:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.637+0000 7fce748ad700 1 Processor -- start 2023-10-06T11:36:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce748ad700 1 -- start start 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce748ad700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 0x7fce7015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce748ad700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700cf360 con 0x7fce701595c0 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce748ad700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7015dd60 con 0x7fce701591f0 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce748ad700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700cff00 con 0x7fce700cc500 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.641+0000 7fce6e59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce701595c0 0x7fce700af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46116/0 (socket says 172.21.15.46:46116) 2023-10-06T11:36:23.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6e59c700 1 -- 172.21.15.46:0/711525494 learned_addr learned my addr 172.21.15.46:0/711525494 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:36:23.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6f59e700 1 --2- 172.21.15.46:0/711525494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 0x7fce7015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051746966 0 0) 0x7fce700cf360 con 0x7fce701595c0 2023-10-06T11:36:23.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce58003570 con 0x7fce701595c0 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400067280 0 0) 0x7fce7015dd60 con 0x7fce701591f0 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce700cf360 con 0x7fce701591f0 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6f59e700 1 -- 172.21.15.46:0/711525494 >> v1:172.21.15.132:6789/0 conn(0x7fce701591f0 legacy=0x7fce700cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6f59e700 1 -- 172.21.15.46:0/711525494 >> v1:172.21.15.46:6789/0 conn(0x7fce701595c0 legacy=0x7fce700af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6f59e700 1 -- 172.21.15.46:0/711525494 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700cea30 con 0x7fce700cc500 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6f59e700 1 --2- 172.21.15.46:0/711525494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 0x7fce7015ceb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e90 tx=0x7fce68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eede8cdfee6f1fe8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:23.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e070 con 0x7fce700cc500 2023-10-06T11:36:23.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800bc10 con 0x7fce700cc500 2023-10-06T11:36:23.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce6dd9b700 1 -- 172.21.15.46:0/711525494 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68012430 con 0x7fce700cc500 2023-10-06T11:36:23.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 -- 172.21.15.46:0/711525494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 msgr2=0x7fce7015ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 --2- 172.21.15.46:0/711525494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 0x7fce7015ceb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e90 tx=0x7fce68009780 comp rx=0 tx=0).stop 2023-10-06T11:36:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 -- 172.21.15.46:0/711525494 shutdown_connections 2023-10-06T11:36:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 --2- 172.21.15.46:0/711525494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700cc500 0x7fce7015ceb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 -- 172.21.15.46:0/711525494 >> 172.21.15.46:0/711525494 conn(0x7fce7006ef60 msgr2=0x7fce700d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 -- 172.21.15.46:0/711525494 shutdown_connections 2023-10-06T11:36:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.645+0000 7fce748ad700 1 -- 172.21.15.46:0/711525494 wait complete. 2023-10-06T11:36:23.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.649+0000 7fce748ad700 1 Processor -- start 2023-10-06T11:36:23.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.649+0000 7fce748ad700 1 -- start start 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce700cc500 0x7fce7014ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce701591f0 0x7fce7015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b2840 con 0x7fce7015ac40 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70155b50 con 0x7fce700cc500 2023-10-06T11:36:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b14c0 con 0x7fce701591f0 2023-10-06T11:36:23.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6e59c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce701591f0 0x7fce7015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48208/0 (socket says 172.21.15.46:48208) 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 -- 172.21.15.46:0/1262284363 learned_addr learned my addr 172.21.15.46:0/1262284363 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce700cc500 0x7fce7014ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce701591f0 msgr2=0x7fce7015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce701591f0 0x7fce7015a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce700cc500 msgr2=0x7fce7014ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce700cc500 0x7fce7014ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009a20 con 0x7fce7015ac40 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce6f59e700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fce70088770 tx=0x7fce68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6801b420 con 0x7fce7015ac40 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6801b5a0 con 0x7fce7015ac40 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700b17a0 con 0x7fce7015ac40 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6801b960 con 0x7fce7015ac40 2023-10-06T11:36:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.653+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7014b3a0 con 0x7fce7015ac40 2023-10-06T11:36:23.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.657+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce70155640 con 0x7fce7015ac40 2023-10-06T11:36:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.657+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 54) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fce6801bd50 con 0x7fce7015ac40 2023-10-06T11:36:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.661+0000 7fce56ffd700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.661+0000 7fce6ed9d700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.661+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6502+0+0 (secure 0 0 0) 0x7fce6807e720 con 0x7fce7015ac40 2023-10-06T11:36:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.661+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:36:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.669+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce70155640 con 0x7fce7015ac40 2023-10-06T11:36:23.845 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:23.842+0000 7f3b559c3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:36:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.865+0000 7fce6ed9d700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.865+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:36:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:23.865+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fce7014b5c0 con 0x7fce5c0606f0 2023-10-06T11:36:23.919 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:23.918+0000 7f3b559c3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:36:23.926 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:23.913+0000 7f2d96343000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:36:24.001 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:23.989+0000 7f2d96343000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:36:24.135 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:24.134+0000 7f3b559c3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:36:24.218 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:24.214+0000 7f3b559c3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:36:24.233 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:24.221+0000 7f2d96343000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:36:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:24.269+0000 7fce6ed9d700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:24.269+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:36:24.308 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:24.306+0000 7f3b559c3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:36:24.319 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:24.305+0000 7f2d96343000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:36:24.406 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:24.393+0000 7f2d96343000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:36:24.708 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:24.706+0000 7f3b559c3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:36:24.822 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:24.809+0000 7f2d96343000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:36:24.869 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:24.866+0000 7f3b559c3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:36:24.995 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:24.981+0000 7f2d96343000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:36:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:25.073+0000 7fce6ed9d700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:25.073+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:36:25.237 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.234+0000 7f3b559c3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:36:25.379 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:25.365+0000 7f2d96343000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:36:25.444 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.442+0000 7f3b559c3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:36:25.533 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.530+0000 7f3b559c3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:36:25.604 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:25.589+0000 7f2d96343000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:36:25.607 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.606+0000 7f3b559c3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:36:25.688 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.686+0000 7f3b559c3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:36:25.695 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:25.681+0000 7f2d96343000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:36:25.773 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:25.761+0000 7f2d96343000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:36:25.851 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:25.837+0000 7f2d96343000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:36:25.901 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.898+0000 7f3b559c3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:36:25.978 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:25.974+0000 7f3b559c3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:36:26.061 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:26.058+0000 7f3b559c3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:36:26.073 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:26.061+0000 7f2d96343000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:36:26.154 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:26.141+0000 7f2d96343000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:36:26.245 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:26.233+0000 7f2d96343000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:36:26.245 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:26.242+0000 7f3b559c3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:36:26.330 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:26.326+0000 7f3b559c3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:36:26.442 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:26.429+0000 7f2d96343000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:36:26.524 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:26.513+0000 7f2d96343000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:36:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:26.677+0000 7fce6ed9d700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:36:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:26.677+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:36:27.023 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:36:27.018+0000 7f3b559c3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:36:27.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:27.077+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 55) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fce6804ebb0 con 0x7fce7015ac40 2023-10-06T11:36:27.261 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.249+0000 7f2d96343000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:36:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:27.269+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 56) v1 ==== 79687+0+0 (secure 0 0 0) 0x7fce680211b0 con 0x7fce7015ac40 2023-10-06T11:36:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:27.269+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:36:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:27.269+0000 7fce56ffd700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:27.467 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d56a0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.467 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d56a0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.467 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d56a0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.467 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d56a0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.467 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d56a0c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.468 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d59211700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.468 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d59211700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.468 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d59211700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.468 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d59211700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:27.468 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:36:27.453+0000 7f2d59211700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:36:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.273+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 57) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fce6804f480 con 0x7fce7015ac40 2023-10-06T11:36:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.277+0000 7fce56ffd700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.277+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fce5c066a90 con 0x7fce5c0606f0 2023-10-06T11:36:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.277+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.281+0000 7fce6ed9d700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062b20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fce60007fd0 tx=0x7fce60007d50 comp rx=0 tx=0).ready entity=mgr.8025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.389+0000 7fce56ffd700 1 -- 172.21.15.46:0/1262284363 <== mgr.8025 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fce5c066a90 con 0x7fce5c0606f0 2023-10-06T11:36:28.403 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T11:36:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 msgr2=0x7fce5c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062b20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fce60007fd0 tx=0x7fce60007d50 comp rx=0 tx=0).stop 2023-10-06T11:36:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 msgr2=0x7fce700b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fce70088770 tx=0x7fce68019000 comp rx=0 tx=0).stop 2023-10-06T11:36:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 shutdown_connections 2023-10-06T11:36:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fce5c0606f0 0x7fce5c062b20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7015ac40 0x7fce700b08e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce701591f0 0x7fce7015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 --2- 172.21.15.46:0/1262284363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce700cc500 0x7fce7014ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 >> 172.21.15.46:0/1262284363 conn(0x7fce7006ef60 msgr2=0x7fce700ace20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 shutdown_connections 2023-10-06T11:36:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.393+0000 7fce748ad700 1 -- 172.21.15.46:0/1262284363 wait complete. 2023-10-06T11:36:28.430 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T11:36:28.431 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-11_37_00 in ~32s... 2023-10-06T11:36:28.431 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-06T11:36:28.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T11:36:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.589+0000 7f539cbcc700 1 Processor -- start 2023-10-06T11:36:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.589+0000 7f539cbcc700 1 -- start start 2023-10-06T11:36:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f53981591f0 0x7f53980cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53981595c0 0x7f53980af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53980cc500 0x7f53980afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53980cf360 con 0x7f53981595c0 2023-10-06T11:36:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53980b03c0 con 0x7f53981591f0 2023-10-06T11:36:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f539cbcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539815dd60 con 0x7f53980cc500 2023-10-06T11:36:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f5396ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53981595c0 0x7f53980af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f5396ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53981595c0 0x7f53980af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48368/0 (socket says 172.21.15.46:48368) 2023-10-06T11:36:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f5396ffd700 1 -- 172.21.15.46:0/2088964969 learned_addr learned my addr 172.21.15.46:0/2088964969 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f5397fff700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53980cc500 0x7f53980afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.593+0000 7f53977fe700 1 --2- 172.21.15.46:0/2088964969 >> v2:172.21.15.132:3300/0 conn(0x7f53981591f0 0x7f53980cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 -- 172.21.15.46:0/2088964969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53980cc500 msgr2=0x7f53980afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53980cc500 0x7f53980afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 -- 172.21.15.46:0/2088964969 >> v2:172.21.15.132:3300/0 conn(0x7f53981591f0 msgr2=0x7f53980cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 --2- 172.21.15.46:0/2088964969 >> v2:172.21.15.132:3300/0 conn(0x7f53981591f0 0x7f53980cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 -- 172.21.15.46:0/2088964969 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980cea30 con 0x7f53981595c0 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f5396ffd700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53981595c0 0x7f53980af940 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f53980892e0 tx=0x7f538c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8f854966b3e3fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f53967fc700 1 -- 172.21.15.46:0/2088964969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00b6d0 con 0x7f53981595c0 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f53967fc700 1 -- 172.21.15.46:0/2088964969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c00b830 con 0x7f53981595c0 2023-10-06T11:36:28.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f53967fc700 1 -- 172.21.15.46:0/2088964969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00bbf0 con 0x7f53981595c0 2023-10-06T11:36:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 -- 172.21.15.46:0/2088964969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53981595c0 msgr2=0x7f53980af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53981595c0 0x7f53980af940 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f53980892e0 tx=0x7f538c009250 comp rx=0 tx=0).stop 2023-10-06T11:36:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 -- 172.21.15.46:0/2088964969 shutdown_connections 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53980cc500 0x7f53980afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 --2- 172.21.15.46:0/2088964969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53981595c0 0x7f53980af940 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 --2- 172.21.15.46:0/2088964969 >> v2:172.21.15.132:3300/0 conn(0x7f53981591f0 0x7f53980cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 -- 172.21.15.46:0/2088964969 >> 172.21.15.46:0/2088964969 conn(0x7f539806ef60 msgr2=0x7f53980d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 -- 172.21.15.46:0/2088964969 shutdown_connections 2023-10-06T11:36:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.597+0000 7f539cbcc700 1 -- 172.21.15.46:0/2088964969 wait complete. 2023-10-06T11:36:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.601+0000 7f539cbcc700 1 Processor -- start 2023-10-06T11:36:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.601+0000 7f539cbcc700 1 -- start start 2023-10-06T11:36:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53981591f0 0x7f5398153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398154400 0x7f5398143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5398154fa0 con 0x7f53980cc500 2023-10-06T11:36:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5398156720 con 0x7f5398154400 2023-10-06T11:36:28.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539815b0c0 con 0x7f53981591f0 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48378/0 (socket says 172.21.15.46:48378) 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 -- 172.21.15.46:0/3421841590 learned_addr learned my addr 172.21.15.46:0/3421841590 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f5396ffd700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53981591f0 0x7f5398153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f5397fff700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398154400 0x7f5398143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 -- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53981591f0 msgr2=0x7f5398153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53981591f0 0x7f5398153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 -- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398154400 msgr2=0x7f5398143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398154400 0x7f5398143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 -- 172.21.15.46:0/3421841590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0094f0 con 0x7f53980cc500 2023-10-06T11:36:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f53977fe700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f538800e960 tx=0x7f53880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388013680 con 0x7f53980cc500 2023-10-06T11:36:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880137e0 con 0x7f53980cc500 2023-10-06T11:36:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388013b80 con 0x7f53980cc500 2023-10-06T11:36:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539815ab00 con 0x7f53980cc500 2023-10-06T11:36:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.605+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539816de10 con 0x7f53980cc500 2023-10-06T11:36:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.609+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53981564f0 con 0x7f53980cc500 2023-10-06T11:36:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.609+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 57) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f538800c490 con 0x7f53980cc500 2023-10-06T11:36:28.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.613+0000 7f537f7fe700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 0x7f53840628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:36:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.613+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f538807cb30 con 0x7f53980cc500 2023-10-06T11:36:28.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.613+0000 7f5396ffd700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 0x7f53840628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:36:28.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.613+0000 7f5396ffd700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 0x7f53840628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53980892e0 tx=0x7f538c017000 comp rx=0 tx=0).ready entity=mgr.8025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:36:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.617+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f538804ce90 con 0x7f53980cc500 2023-10-06T11:36:28.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.805+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5398156860 con 0x7f53840603f0 2023-10-06T11:36:28.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.821+0000 7f537f7fe700 1 -- 172.21.15.46:0/3421841590 <== mgr.8025 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f538c00b9a0 con 0x7f53840603f0 2023-10-06T11:36:28.833 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:36:28.834 INFO:teuthology.orchestra.run.smithi046.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-06T11:36:28.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 msgr2=0x7f53840628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 0x7f53840628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53980892e0 tx=0x7f538c017000 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 msgr2=0x7f539816d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f538800e960 tx=0x7f53880091a0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 shutdown_connections 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53840603f0 0x7f53840628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398154400 0x7f5398143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53981591f0 0x7f5398153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 --2- 172.21.15.46:0/3421841590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53980cc500 0x7f539816d8d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 >> 172.21.15.46:0/3421841590 conn(0x7f539806ef60 msgr2=0x7f5398152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:36:28.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 shutdown_connections 2023-10-06T11:36:28.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:36:28.825+0000 7f539cbcc700 1 -- 172.21.15.46:0/3421841590 wait complete. 2023-10-06T11:36:28.859 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-06T11:36:28.859 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-10-06T11:36:28.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:28.859 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:28.883 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:28.883 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:28.883 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:29.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:29.884 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:29.911 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:29.911 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:29.911 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:30.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:30.912 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:30.935 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:30.936 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:30.936 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:31.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:31.936 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:31.963 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:31.963 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:31.963 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:32.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:32.965 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:32.989 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:32.989 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:32.990 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:33.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:33.991 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:34.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:34.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:34.016 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:35.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:35.018 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:35.043 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:35.043 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:35.043 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:36.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:36.044 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:36.068 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:36.068 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:36.068 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:37.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:37.069 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:37.095 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:37.095 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:37.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:38.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:38.096 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:38.122 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:38.122 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:38.122 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:39.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:39.123 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:39.147 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:39.148 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:39.148 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:40.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:40.149 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:40.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:40.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:40.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:41.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:41.176 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:41.199 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:41.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:41.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:42.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:42.201 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:42.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:42.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:42.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:43.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:43.228 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:43.251 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:43.252 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:43.252 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:44.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:44.252 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:44.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:44.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:44.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:45.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:45.278 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:45.301 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:45.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:45.302 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:46.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:46.303 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:46.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:46.328 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:46.328 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:47.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:47.329 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:47.354 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:47.354 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:47.354 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:47.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:36:47.991 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:36:48.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:48.355 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:48.381 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:48.381 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:48.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:49.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:49.383 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:49.406 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:49.406 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:49.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:50.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:50.407 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:50.431 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:50.431 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:50.431 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:51.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:51.432 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:51.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:51.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:51.458 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:52.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:52.459 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:52.482 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:52.482 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:52.482 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:53.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:53.483 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:53.508 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:53.509 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:53.509 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:54.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:54.510 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:54.536 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:54.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:54.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:55.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:55.538 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:55.563 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:55.563 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:55.563 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:56.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:56.564 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:56.588 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:56.588 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:56.588 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:57.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:57.589 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:57.613 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:57.613 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:57.613 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:58.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:58.615 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:58.639 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:58.639 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:58.639 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:36:59.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:36:59.640 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:36:59.664 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T11:36:59.664 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T11:36:59.664 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:37:00.425 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:00.412+0000 7f2d491f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:00.425 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:00.412+0000 7f2d491f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:00.425 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:00.412+0000 7f2d491f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:00.425 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:00.412+0000 7f2d491f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:00.425 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:00.412+0000 7f2d491f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:00.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:00.665 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:37:00.688 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_37_00_UTC 2023-10-06T11:37:00.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-11_37_00_UTC'} 2023-10-06T11:37:00.689 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-11_37_00_UTC'} 2023-10-06T11:37:00.689 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T11:37:00.690 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-11_37_00_UTC'] 2023-10-06T11:37:00.690 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696592188.4309595, now = 1696592220.6901627, timo = 32 2023-10-06T11:37:01.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T11:37:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.836+0000 7fc55cee3700 1 Processor -- start 2023-10-06T11:37:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.836+0000 7fc55cee3700 1 -- start start 2023-10-06T11:37:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc55cee3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5580aea70 0x7fc5580cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc55cee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc558068400 0x7fc5580aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc55cee3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5580ab500 con 0x7fc55806f1f0 2023-10-06T11:37:01.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc55cee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc558067a10 con 0x7fc558068400 2023-10-06T11:37:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc55cee3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc558067f70 con 0x7fc5580aea70 2023-10-06T11:37:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc556ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc55806f1f0 0x7fc5580cc980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52378/0 (socket says 172.21.15.46:52378) 2023-10-06T11:37:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc556ffd700 1 -- 172.21.15.46:0/2094230711 learned_addr learned my addr 172.21.15.46:0/2094230711 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc557fff700 1 --2- 172.21.15.46:0/2094230711 >> v2:172.21.15.46:3300/0 conn(0x7fc558068400 0x7fc5580aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc5577fe700 1 --2- 172.21.15.46:0/2094230711 >> v2:172.21.15.132:3300/0 conn(0x7fc5580aea70 0x7fc5580cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.840+0000 7fc5567fc700 1 -- 172.21.15.46:0/2094230711 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772759358 0 0) 0x7fc5580ab500 con 0x7fc55806f1f0 2023-10-06T11:37:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc5567fc700 1 -- 172.21.15.46:0/2094230711 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc540003570 con 0x7fc55806f1f0 2023-10-06T11:37:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc557fff700 1 -- 172.21.15.46:0/2094230711 >> v2:172.21.15.132:3300/0 conn(0x7fc5580aea70 msgr2=0x7fc5580cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:01.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc557fff700 1 --2- 172.21.15.46:0/2094230711 >> v2:172.21.15.132:3300/0 conn(0x7fc5580aea70 0x7fc5580cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:01.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc557fff700 1 -- 172.21.15.46:0/2094230711 >> v1:172.21.15.46:6789/0 conn(0x7fc55806f1f0 legacy=0x7fc5580cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:01.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc557fff700 1 -- 172.21.15.46:0/2094230711 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580aee40 con 0x7fc558068400 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc557fff700 1 --2- 172.21.15.46:0/2094230711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580aceb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc55806c2e0 tx=0x7fc550009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1380c5ae8baa62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc5567fc700 1 -- 172.21.15.46:0/2094230711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000e070 con 0x7fc558068400 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc5567fc700 1 -- 172.21.15.46:0/2094230711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000bb40 con 0x7fc558068400 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc5567fc700 1 -- 172.21.15.46:0/2094230711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550012430 con 0x7fc558068400 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 -- 172.21.15.46:0/2094230711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 msgr2=0x7fc5580aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 --2- 172.21.15.46:0/2094230711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580aceb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc55806c2e0 tx=0x7fc550009780 comp rx=0 tx=0).stop 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 -- 172.21.15.46:0/2094230711 shutdown_connections 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 --2- 172.21.15.46:0/2094230711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580aceb0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 --2- 172.21.15.46:0/2094230711 >> v2:172.21.15.132:3300/0 conn(0x7fc5580aea70 0x7fc5580cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 -- 172.21.15.46:0/2094230711 >> 172.21.15.46:0/2094230711 conn(0x7fc5580afe40 msgr2=0x7fc55806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 -- 172.21.15.46:0/2094230711 shutdown_connections 2023-10-06T11:37:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.844+0000 7fc55cee3700 1 -- 172.21.15.46:0/2094230711 wait complete. 2023-10-06T11:37:01.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 Processor -- start 2023-10-06T11:37:01.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 -- start start 2023-10-06T11:37:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc55806f1f0 0x7fc5581696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5580aea70 0x7fc558169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55816a8e0 con 0x7fc558068400 2023-10-06T11:37:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55816aea0 con 0x7fc55806f1f0 2023-10-06T11:37:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc55cee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc558151f30 con 0x7fc5580aea70 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35998/0 (socket says 172.21.15.46:35998) 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 -- 172.21.15.46:0/1770020401 learned_addr learned my addr 172.21.15.46:0/1770020401 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc556ffd700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc55806f1f0 0x7fc5581696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc557fff700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5580aea70 0x7fc558169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 -- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5580aea70 msgr2=0x7fc558169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5580aea70 0x7fc558169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 -- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc55806f1f0 msgr2=0x7fc5581696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc55806f1f0 0x7fc5581696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:01.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.848+0000 7fc5577fe700 1 -- 172.21.15.46:0/1770020401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc550009a20 con 0x7fc558068400 2023-10-06T11:37:01.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc5577fe700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc54800ecb0 tx=0x7fc548009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:01.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800c850 con 0x7fc558068400 2023-10-06T11:37:01.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54800c9b0 con 0x7fc558068400 2023-10-06T11:37:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54800cd30 con 0x7fc558068400 2023-10-06T11:37:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5580aa510 con 0x7fc558068400 2023-10-06T11:37:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55816ab30 con 0x7fc558068400 2023-10-06T11:37:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc558064590 con 0x7fc558068400 2023-10-06T11:37:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.852+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) v1 ==== 80192+0+0 (secure 0 0 0) 0x7fc548013680 con 0x7fc558068400 2023-10-06T11:37:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.856+0000 7fc53f7fe700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 0x7fc544062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.856+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fc54807d610 con 0x7fc558068400 2023-10-06T11:37:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.856+0000 7fc556ffd700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 0x7fc544062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.856+0000 7fc556ffd700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 0x7fc544062c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5580995e0 tx=0x7fc54c009040 comp rx=0 tx=0).ready entity=mgr.8025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:01.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:01.864+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc54804d460 con 0x7fc558068400 2023-10-06T11:37:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.052+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc55816afe0 con 0x7fc5440607e0 2023-10-06T11:37:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.180+0000 7fc53f7fe700 1 -- 172.21.15.46:0/1770020401 <== mgr.8025 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc54c001720 con 0x7fc5440607e0 2023-10-06T11:37:02.197 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path snap_test_dir1 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 msgr2=0x7fc544062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 0x7fc544062c90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5580995e0 tx=0x7fc54c009040 comp rx=0 tx=0).stop 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 msgr2=0x7fc5580a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc54800ecb0 tx=0x7fc548009120 comp rx=0 tx=0).stop 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 shutdown_connections 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc5440607e0 0x7fc544062c90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5580aea70 0x7fc558169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.184+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc55806f1f0 0x7fc5581696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.188+0000 7fc55cee3700 1 --2- 172.21.15.46:0/1770020401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc558068400 0x7fc5580a9a00 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.188+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 >> 172.21.15.46:0/1770020401 conn(0x7fc5580afe40 msgr2=0x7fc558099950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.188+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 shutdown_connections 2023-10-06T11:37:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.188+0000 7fc55cee3700 1 -- 172.21.15.46:0/1770020401 wait complete. 2023-10-06T11:37:02.223 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T11:37:02.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:02.224 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T11:37:02.246 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_37_00_UTC 2023-10-06T11:37:02.247 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_37_00_UTC 2023-10-06T11:37:02.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:02.247 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_37_00_UTC) 2023-10-06T11:37:02.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:02.322 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T11:37:02.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T11:37:02.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.580+0000 7ff9b1323700 1 Processor -- start 2023-10-06T11:37:02.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.580+0000 7ff9b1323700 1 -- start start 2023-10-06T11:37:02.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9ac0b30f0 0x7ff9ac0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b3570 0x7ff9ac0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0afd20 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac159740 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b0000 con 0x7ff9ac0b3570 2023-10-06T11:37:02.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff9ac0b30f0 0x7ff9ac0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9abfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9ac0adbe0 0x7ff9ac0d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52394/0 (socket says 172.21.15.46:52394) 2023-10-06T11:37:02.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9abfff700 1 -- 172.21.15.46:0/1189403475 learned_addr learned my addr 172.21.15.46:0/1189403475 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:02.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b0b22700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b3570 0x7ff9ac0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/1189403475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b3570 msgr2=0x7ff9ac0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b3570 0x7ff9ac0b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/1189403475 >> v1:172.21.15.46:6789/0 conn(0x7ff9ac0adbe0 legacy=0x7ff9ac0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 -- 172.21.15.46:0/1189403475 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0b02e0 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9aaffd700 1 -- 172.21.15.46:0/1189403475 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564619431 0 0) 0x7ff9ac0afd20 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9ab7fe700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0b30f0 0x7ff9ac0b0dc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00098e0 tx=0x7ff9a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55e9bd7b71b099c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9aaffd700 1 -- 172.21.15.46:0/1189403475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000b640 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9aaffd700 1 -- 172.21.15.46:0/1189403475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000b7a0 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9aaffd700 1 -- 172.21.15.46:0/1189403475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000bb40 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 -- 172.21.15.46:0/1189403475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0b30f0 msgr2=0x7ff9ac0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.584+0000 7ff9b1323700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0b30f0 0x7ff9ac0b0dc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00098e0 tx=0x7ff9a00091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 -- 172.21.15.46:0/1189403475 shutdown_connections 2023-10-06T11:37:02.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b3570 0x7ff9ac0b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 --2- 172.21.15.46:0/1189403475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0b30f0 0x7ff9ac0b0dc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 -- 172.21.15.46:0/1189403475 >> 172.21.15.46:0/1189403475 conn(0x7ff9ac06ef60 msgr2=0x7ff9ac0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 -- 172.21.15.46:0/1189403475 shutdown_connections 2023-10-06T11:37:02.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 -- 172.21.15.46:0/1189403475 wait complete. 2023-10-06T11:37:02.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 Processor -- start 2023-10-06T11:37:02.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.588+0000 7ff9b1323700 1 -- start start 2023-10-06T11:37:02.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b30f0 0x7ff9ac1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac0b3570 0x7ff9ac14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac151990 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac151c70 con 0x7ff9ac0b3570 2023-10-06T11:37:02.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b1323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac151f50 con 0x7ff9ac0b30f0 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36026/0 (socket says 172.21.15.46:36026) 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 -- 172.21.15.46:0/936424222 learned_addr learned my addr 172.21.15.46:0/936424222 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9ab7fe700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b30f0 0x7ff9ac1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9b0b22700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac0b3570 0x7ff9ac14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 -- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b30f0 msgr2=0x7ff9ac1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b30f0 0x7ff9ac1510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 -- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac0b3570 msgr2=0x7ff9ac14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:02.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac0b3570 0x7ff9ac14f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:02.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 -- 172.21.15.46:0/936424222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009420 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.592+0000 7ff9abfff700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c7e0 tx=0x7ff99c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c005b30 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac152230 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c005c90 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c007300 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac151630 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac13b940 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.596+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 59) v1 ==== 80192+0+0 (secure 0 0 0) 0x7ff99c01b3b0 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.600+0000 7ff993fff700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 0x7ff998062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.600+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7ff99c07c6b0 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.600+0000 7ff9ab7fe700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 0x7ff998062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:02.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.600+0000 7ff9ab7fe700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 0x7ff998062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00093f0 tx=0x7ff9a0017000 comp rx=0 tx=0).ready entity=mgr.8025 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:02.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.608+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff9ac13b940 con 0x7ff9ac0adbe0 2023-10-06T11:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:02.828+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff9ac064590 con 0x7ff9ac0adbe0 2023-10-06T11:37:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.700+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7ff99c04c800 con 0x7ff9ac0adbe0 2023-10-06T11:37:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.700+0000 7ff993fff700 1 -- 172.21.15.46:0/936424222 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v60) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff99c04c430 con 0x7ff9ac0adbe0 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 msgr2=0x7ff998062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 0x7ff998062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00093f0 tx=0x7ff9a0017000 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 msgr2=0x7ff9ac1589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c7e0 tx=0x7ff99c007df0 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 shutdown_connections 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9980607e0 0x7ff998062c90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac0b3570 0x7ff9ac14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b30f0 0x7ff9ac1510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 --2- 172.21.15.46:0/936424222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9ac0adbe0 0x7ff9ac1589e0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.704+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 >> 172.21.15.46:0/936424222 conn(0x7ff9ac06ef60 msgr2=0x7ff9ac0adfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.708+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 shutdown_connections 2023-10-06T11:37:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:03.708+0000 7ff9b1323700 1 -- 172.21.15.46:0/936424222 wait complete. 2023-10-06T11:37:03.748 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:37:03.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:37:03.787 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:37:03.827 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:37:03.827 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T11:37:03.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:03.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T11:37:03.896 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T11:37:03.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:03.897 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:37:03.907 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:03.905+0000 7f855fdfe000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:37:03.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:03.923 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:03.908+0000 7fe5fa1c3000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:37:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.068+0000 7f320d8e3700 1 Processor -- start 2023-10-06T11:37:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.068+0000 7f320d8e3700 1 -- start start 2023-10-06T11:37:04.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080cbb40 0x7f32080cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32080abd90 0x7f32080cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080cce50 0x7f32080ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080cd270 con 0x7f32080cce50 2023-10-06T11:37:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320806f2a0 con 0x7f32080abd90 2023-10-06T11:37:04.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d8e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320806d650 con 0x7f32080cbb40 2023-10-06T11:37:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080cce50 0x7f32080ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080cce50 0x7f32080ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36044/0 (socket says 172.21.15.46:36044) 2023-10-06T11:37:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 -- 172.21.15.46:0/2342586723 learned_addr learned my addr 172.21.15.46:0/2342586723 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320c8e1700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080cbb40 0x7f32080cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f3207fff700 1 --2- 172.21.15.46:0/2342586723 >> v2:172.21.15.132:3300/0 conn(0x7f32080abd90 0x7f32080cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 -- 172.21.15.46:0/2342586723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080cbb40 msgr2=0x7f32080cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080cbb40 0x7f32080cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 -- 172.21.15.46:0/2342586723 >> v2:172.21.15.132:3300/0 conn(0x7f32080abd90 msgr2=0x7f32080cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 --2- 172.21.15.46:0/2342586723 >> v2:172.21.15.132:3300/0 conn(0x7f32080abd90 0x7f32080cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 -- 172.21.15.46:0/2342586723 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080aec60 con 0x7f32080cce50 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.072+0000 7f320d0e2700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f32080ad510 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f320806c770 tx=0x7f3200009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b527e66bc88b375 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f32077fe700 1 -- 172.21.15.46:0/2342586723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000e070 con 0x7f32080cce50 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f32077fe700 1 -- 172.21.15.46:0/2342586723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000b970 con 0x7f32080cce50 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f32077fe700 1 -- 172.21.15.46:0/2342586723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32000092e0 con 0x7f32080cce50 2023-10-06T11:37:04.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- 172.21.15.46:0/2342586723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 msgr2=0x7f32080ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f32080ad510 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f320806c770 tx=0x7f3200009840 comp rx=0 tx=0).stop 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- 172.21.15.46:0/2342586723 shutdown_connections 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2342586723 >> v2:172.21.15.132:3300/0 conn(0x7f32080abd90 0x7f32080cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f32080ad510 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2342586723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080cbb40 0x7f32080cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- 172.21.15.46:0/2342586723 >> 172.21.15.46:0/2342586723 conn(0x7f32080afd40 msgr2=0x7f320806e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- 172.21.15.46:0/2342586723 shutdown_connections 2023-10-06T11:37:04.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- 172.21.15.46:0/2342586723 wait complete. 2023-10-06T11:37:04.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 Processor -- start 2023-10-06T11:37:04.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.076+0000 7f320d8e3700 1 -- start start 2023-10-06T11:37:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 0x7f3208095980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080cbb40 0x7f3208098420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208095ec0 con 0x7f32080cce50 2023-10-06T11:37:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32081576e0 con 0x7f32080cbb40 2023-10-06T11:37:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3208156ad0 con 0x7f32080abd90 2023-10-06T11:37:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d0e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 0x7f3208095980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d0e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36050/0 (socket says 172.21.15.46:36050) 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d0e2700 1 -- 172.21.15.46:0/2819613388 learned_addr learned my addr 172.21.15.46:0/2819613388 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f3207fff700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080cbb40 0x7f3208098420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080cbb40 msgr2=0x7f3208098420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080cbb40 0x7f3208098420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 msgr2=0x7f3208067a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 -- 172.21.15.46:0/2819613388 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200009ae0 con 0x7f32080abd90 2023-10-06T11:37:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d0e2700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320c8e1700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 0x7f3208095980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ea30 tx=0x7f31fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00ee70 con 0x7f32080abd90 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00c850 con 0x7f32080abd90 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3208156db0 con 0x7f32080abd90 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.080+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00cc10 con 0x7f32080abd90 2023-10-06T11:37:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.084+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3208157370 con 0x7f32080abd90 2023-10-06T11:37:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.084+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208064590 con 0x7f32080abd90 2023-10-06T11:37:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.084+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f31fc005a00 con 0x7f32080abd90 2023-10-06T11:37:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.088+0000 7f3204ff9700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 0x7f31ec062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.088+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f31fc07d280 con 0x7f32080abd90 2023-10-06T11:37:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.088+0000 7f3207fff700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 msgr2=0x7f31ec062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.088+0000 7f3207fff700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 0x7f31ec062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:04.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.096+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f31fc050be0 con 0x7f32080abd90 2023-10-06T11:37:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.292+0000 7f3207fff700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 msgr2=0x7f31ec062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.292+0000 7f3207fff700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 0x7f31ec062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.300+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3208068980 con 0x7f32080abd90 2023-10-06T11:37:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.304+0000 7f3204ff9700 1 -- 172.21.15.46:0/2819613388 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f31fc050770 con 0x7f32080abd90 2023-10-06T11:37:04.318 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:04.318 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 msgr2=0x7f31ec062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 0x7f31ec062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 msgr2=0x7f3208095980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 0x7f3208095980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ea30 tx=0x7f31fc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 shutdown_connections 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f31ec060790 0x7f31ec062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080cce50 0x7f3208067a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080cbb40 0x7f3208098420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 --2- 172.21.15.46:0/2819613388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080abd90 0x7f3208095980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 >> 172.21.15.46:0/2819613388 conn(0x7f32080afd40 msgr2=0x7f3208069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 shutdown_connections 2023-10-06T11:37:04.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.308+0000 7f320d8e3700 1 -- 172.21.15.46:0/2819613388 wait complete. 2023-10-06T11:37:04.323 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:37:04.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:04.453 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:04.449+0000 7f855fdfe000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:37:04.497 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:04.480+0000 7fe5fa1c3000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:37:04.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.484+0000 7fbbb0545700 1 Processor -- start 2023-10-06T11:37:04.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.484+0000 7fbbb0545700 1 -- start start 2023-10-06T11:37:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbb0545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 0x7fbba806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbb0545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbb0545700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba806d460 con 0x7fbba80ad6c0 2023-10-06T11:37:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbb0545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80c7000 con 0x7fbba815ab60 2023-10-06T11:37:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbb0545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80c72c0 con 0x7fbba815af30 2023-10-06T11:37:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbadae0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbaeae2700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbba80ad6c0 0x7fbba80adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35468/0 (socket says 172.21.15.46:35468) 2023-10-06T11:37:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbae2e1700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 0x7fbba806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbaeae2700 1 -- 172.21.15.46:0/2414384119 learned_addr learned my addr 172.21.15.46:0/2414384119 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbad2df700 1 -- 172.21.15.46:0/2414384119 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267457692 0 0) 0x7fbba806d460 con 0x7fbba80ad6c0 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbad2df700 1 -- 172.21.15.46:0/2414384119 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb94003570 con 0x7fbba80ad6c0 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbadae0700 1 -- 172.21.15.46:0/2414384119 >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 msgr2=0x7fbba806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbadae0700 1 --2- 172.21.15.46:0/2414384119 >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 0x7fbba806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbadae0700 1 -- 172.21.15.46:0/2414384119 >> v1:172.21.15.132:6789/0 conn(0x7fbba80ad6c0 legacy=0x7fbba80adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.488+0000 7fbbadae0700 1 -- 172.21.15.46:0/2414384119 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80c75a0 con 0x7fbba815af30 2023-10-06T11:37:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbae2e1700 1 --2- 172.21.15.46:0/2414384119 >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 0x7fbba806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbadae0700 1 --2- 172.21.15.46:0/2414384119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba806cc30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391bec7ecc181797 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbad2df700 1 -- 172.21.15.46:0/2414384119 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011630 con 0x7fbba815af30 2023-10-06T11:37:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbad2df700 1 -- 172.21.15.46:0/2414384119 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98011790 con 0x7fbba815af30 2023-10-06T11:37:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbad2df700 1 -- 172.21.15.46:0/2414384119 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98011b30 con 0x7fbba815af30 2023-10-06T11:37:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 -- 172.21.15.46:0/2414384119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 msgr2=0x7fbba806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 --2- 172.21.15.46:0/2414384119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba806cc30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb980091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 -- 172.21.15.46:0/2414384119 shutdown_connections 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 --2- 172.21.15.46:0/2414384119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba806cc30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 --2- 172.21.15.46:0/2414384119 >> v2:172.21.15.132:3300/0 conn(0x7fbba815ab60 0x7fbba806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 -- 172.21.15.46:0/2414384119 >> 172.21.15.46:0/2414384119 conn(0x7fbba80cbe30 msgr2=0x7fbba80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 -- 172.21.15.46:0/2414384119 shutdown_connections 2023-10-06T11:37:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.492+0000 7fbbb0545700 1 -- 172.21.15.46:0/2414384119 wait complete. 2023-10-06T11:37:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 Processor -- start 2023-10-06T11:37:04.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 -- start start 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbba815ab60 0x7fbba816c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba8169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba816d6f0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba816a130 con 0x7fbba815ab60 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbb0545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80aee00 con 0x7fbba815af30 2023-10-06T11:37:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36062/0 (socket says 172.21.15.46:36062) 2023-10-06T11:37:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 -- 172.21.15.46:0/1729769340 learned_addr learned my addr 172.21.15.46:0/1729769340 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbadae0700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbba815ab60 0x7fbba816c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbaeae2700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba8169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 -- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 msgr2=0x7fbba8169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba8169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 -- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbba815ab60 msgr2=0x7fbba816c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbba815ab60 0x7fbba816c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.496+0000 7fbbae2e1700 1 -- 172.21.15.46:0/1729769340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40093c0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbbae2e1700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ec80 tx=0x7fbba4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4005a00 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba4005b60 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb98009420 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400c850 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba816d130 con 0x7fbba80ad6c0 2023-10-06T11:37:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.500+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba80aebd0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.504+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fbba400cbc0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.504+0000 7fbb9e7fc700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 0x7fbb8c062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.504+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fbba407d4d0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.504+0000 7fbbadae0700 1 -- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 msgr2=0x7fbb8c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.504+0000 7fbbadae0700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 0x7fbb8c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:04.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.508+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbba80aebd0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.532 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:04.529+0000 7f855fdfe000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:37:04.577 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:04.564+0000 7fe5fa1c3000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:37:04.677 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:04.673+0000 7f855fdfe000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.680+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbba80aebd0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.680+0000 7fbb9e7fc700 1 -- 172.21.15.46:0/1729769340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18042 (secure 0 0 0) 0x7fbba80aebd0 con 0x7fbba80ad6c0 2023-10-06T11:37:04.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:04.697 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:27.255187+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2960823291":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6833/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6832/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/4259972733":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3863334752":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3488620324":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/1900876313":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/2274789125":"2023-10-07T11:36:27.255068+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 msgr2=0x7fbb8c062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 0x7fbb8c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 msgr2=0x7fbba80a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ec80 tx=0x7fbba4009120 comp rx=0 tx=0).stop 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 shutdown_connections 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbb8c060790 0x7fbb8c062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbba815af30 0x7fbba8169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbba815ab60 0x7fbba816c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 --2- 172.21.15.46:0/1729769340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba80ad6c0 0x7fbba80a9840 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 >> 172.21.15.46:0/1729769340 conn(0x7fbba80cbe30 msgr2=0x7fbba81574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 shutdown_connections 2023-10-06T11:37:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.684+0000 7fbbb0545700 1 -- 172.21.15.46:0/1729769340 wait complete. 2023-10-06T11:37:04.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:36:06.248657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:36:07.254889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:37:04.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:37:04.726 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:04.708+0000 7fe5fa1c3000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 Processor -- start 2023-10-06T11:37:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 -- start start 2023-10-06T11:37:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 0x7f23200ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23200cc510 con 0x7f23200b23c0 2023-10-06T11:37:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23200cd500 con 0x7f23200b2790 2023-10-06T11:37:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.876+0000 7f2326078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23200b2be0 con 0x7f23200cd080 2023-10-06T11:37:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 0x7f23200ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 0x7f23200ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33792/0 (socket says 172.21.15.46:33792) 2023-10-06T11:37:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231effd700 1 -- 172.21.15.46:0/2801677813 learned_addr learned my addr 172.21.15.46:0/2801677813 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231f7fe700 1 --1- 172.21.15.46:0/2801677813 >> v1:172.21.15.132:6789/0 conn(0x7f23200b23c0 0x7f23200cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35480/0 (socket says 172.21.15.46:35480) 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 -- 172.21.15.46:0/2801677813 >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 msgr2=0x7f23200ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 --2- 172.21.15.46:0/2801677813 >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 0x7f23200ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 -- 172.21.15.46:0/2801677813 >> v1:172.21.15.132:6789/0 conn(0x7f23200b23c0 legacy=0x7f23200cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 -- 172.21.15.46:0/2801677813 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23200cb900 con 0x7f23200cd080 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231ffff700 1 --2- 172.21.15.46:0/2801677813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c4fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2318009e60 tx=0x7f2318009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b354c72db870e270 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231e7fc700 1 -- 172.21.15.46:0/2801677813 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800e070 con 0x7f23200cd080 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231e7fc700 1 -- 172.21.15.46:0/2801677813 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231800bb60 con 0x7f23200cd080 2023-10-06T11:37:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f231e7fc700 1 -- 172.21.15.46:0/2801677813 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318012430 con 0x7f23200cd080 2023-10-06T11:37:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 -- 172.21.15.46:0/2801677813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 msgr2=0x7f23200c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 --2- 172.21.15.46:0/2801677813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c4fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2318009e60 tx=0x7f2318009780 comp rx=0 tx=0).stop 2023-10-06T11:37:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 -- 172.21.15.46:0/2801677813 shutdown_connections 2023-10-06T11:37:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 --2- 172.21.15.46:0/2801677813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c4fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 --2- 172.21.15.46:0/2801677813 >> v2:172.21.15.132:3300/0 conn(0x7f23200b2790 0x7f23200ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 -- 172.21.15.46:0/2801677813 >> 172.21.15.46:0/2801677813 conn(0x7f2320099480 msgr2=0x7f232006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 -- 172.21.15.46:0/2801677813 shutdown_connections 2023-10-06T11:37:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.880+0000 7f2326078700 1 -- 172.21.15.46:0/2801677813 wait complete. 2023-10-06T11:37:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 Processor -- start 2023-10-06T11:37:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 -- start start 2023-10-06T11:37:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 0x7f232009d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23200b2790 0x7f232009bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232009df10 con 0x7f23200b2790 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232009d540 con 0x7f23200b23c0 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.884+0000 7f2326078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232009c930 con 0x7f23200cd080 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 0x7f232009d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23200b2790 0x7f232009bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:45102/0 (socket says 172.21.15.46:45102) 2023-10-06T11:37:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 -- 172.21.15.46:0/669424528 learned_addr learned my addr 172.21.15.46:0/669424528 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:04.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 0x7f232009d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33798/0 (socket says 172.21.15.46:33798) 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 -- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 msgr2=0x7f232009d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 0x7f232009d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 -- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23200b2790 msgr2=0x7f232009bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23200b2790 0x7f232009bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 -- 172.21.15.46:0/669424528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318009a20 con 0x7f23200cd080 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f231ffff700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f231800b6c0 tx=0x7f2318019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:04.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318012e90 con 0x7f23200cd080 2023-10-06T11:37:04.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318027470 con 0x7f23200cd080 2023-10-06T11:37:04.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232009cc10 con 0x7f23200cd080 2023-10-06T11:37:04.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318027850 con 0x7f23200cd080 2023-10-06T11:37:04.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.888+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232009d1d0 con 0x7f23200cd080 2023-10-06T11:37:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23200660f0 con 0x7f23200cd080 2023-10-06T11:37:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f2318024070 con 0x7f23200cd080 2023-10-06T11:37:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f23077fe700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 0x7f230c062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f231807faf0 con 0x7f23200cd080 2023-10-06T11:37:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f231f7fe700 1 -- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 msgr2=0x7f230c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.892+0000 7f231f7fe700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 0x7f230c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:04.900+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f231804f940 con 0x7f23200cd080 2023-10-06T11:37:04.960 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:04.957+0000 7f855fdfe000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:37:05.013 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:04.996+0000 7fe5fa1c3000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:37:05.035 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.033+0000 7f855fdfe000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:05.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.068+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f232009d680 con 0x7f23200cd080 2023-10-06T11:37:05.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f23077fe700 1 -- 172.21.15.46:0/669424528 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2318052f60 con 0x7f23200cd080 2023-10-06T11:37:05.086 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:37:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 msgr2=0x7f230c062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 0x7f230c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 msgr2=0x7f23200c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f231800b6c0 tx=0x7f2318019000 comp rx=0 tx=0).stop 2023-10-06T11:37:05.091 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:05.072+0000 7fe5fa1c3000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 shutdown_connections 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f230c060790 0x7f230c062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23200cd080 0x7f23200c72f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23200b2790 0x7f232009bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 --2- 172.21.15.46:0/669424528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23200b23c0 0x7f232009d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.072+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 >> 172.21.15.46:0/669424528 conn(0x7f2320099480 msgr2=0x7f2320067be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.076+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 shutdown_connections 2023-10-06T11:37:05.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.076+0000 7f2326078700 1 -- 172.21.15.46:0/669424528 wait complete. 2023-10-06T11:37:05.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:37:05.255 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.253+0000 7f855fdfe000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:37:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.264+0000 7f0932c6b700 1 Processor -- start 2023-10-06T11:37:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.264+0000 7f0932c6b700 1 -- start start 2023-10-06T11:37:05.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0932c6b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f092c0c7d40 0x7f092c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0932c6b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0c1d90 con 0x7f092c06cf30 2023-10-06T11:37:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0932c6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0c1460 con 0x7f092c0c97e0 2023-10-06T11:37:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0932c6b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c0c8190 con 0x7f092c0c7d40 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f092c0c7d40 0x7f092c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f092c0c7d40 0x7f092c06c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36070/0 (socket says 172.21.15.46:36070) 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 -- 172.21.15.46:0/312938917 learned_addr learned my addr 172.21.15.46:0/312938917 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 -- 172.21.15.46:0/312938917 >> v1:172.21.15.132:6789/0 conn(0x7f092c0c97e0 legacy=0x7f092c0c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 -- 172.21.15.46:0/312938917 >> v1:172.21.15.46:6789/0 conn(0x7f092c06cf30 legacy=0x7f092c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 -- 172.21.15.46:0/312938917 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c0c1740 con 0x7f092c0c7d40 2023-10-06T11:37:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0931468700 1 --2- 172.21.15.46:0/312938917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c06c9f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f091c0099b0 tx=0x7f091c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1dbfb04fb17a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0930c67700 1 -- 172.21.15.46:0/312938917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00b860 con 0x7f092c0c7d40 2023-10-06T11:37:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0930c67700 1 -- 172.21.15.46:0/312938917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00b9c0 con 0x7f092c0c7d40 2023-10-06T11:37:05.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.268+0000 7f0930c67700 1 -- 172.21.15.46:0/312938917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00bd60 con 0x7f092c0c7d40 2023-10-06T11:37:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- 172.21.15.46:0/312938917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 msgr2=0x7f092c06c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 --2- 172.21.15.46:0/312938917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c06c9f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f091c0099b0 tx=0x7f091c009250 comp rx=0 tx=0).stop 2023-10-06T11:37:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- 172.21.15.46:0/312938917 shutdown_connections 2023-10-06T11:37:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 --2- 172.21.15.46:0/312938917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c06c9f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- 172.21.15.46:0/312938917 >> 172.21.15.46:0/312938917 conn(0x7f092c0cbc10 msgr2=0x7f092c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- 172.21.15.46:0/312938917 shutdown_connections 2023-10-06T11:37:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- 172.21.15.46:0/312938917 wait complete. 2023-10-06T11:37:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 Processor -- start 2023-10-06T11:37:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.272+0000 7f0932c6b700 1 -- start start 2023-10-06T11:37:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 0x7f092c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f092c0c97e0 0x7f092c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0aeef0 con 0x7f092c0c7d40 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c3e60 con 0x7f092c0c97e0 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c4140 con 0x7f092c06cf30 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36086/0 (socket says 172.21.15.46:36086) 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 -- 172.21.15.46:0/3545702821 learned_addr learned my addr 172.21.15.46:0/3545702821 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931c69700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 0x7f092c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f093246a700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f092c0c97e0 0x7f092c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 -- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 msgr2=0x7f092c155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 0x7f092c155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 -- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f092c0c97e0 msgr2=0x7f092c0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f092c0c97e0 0x7f092c0adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 -- 172.21.15.46:0/3545702821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0094f0 con 0x7f092c0c7d40 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931c69700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 0x7f092c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0931468700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f092c0c9790 tx=0x7f091c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c011ea0 con 0x7f092c0c7d40 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c01a630 con 0x7f092c0c7d40 2023-10-06T11:37:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c0c4420 con 0x7f092c0c7d40 2023-10-06T11:37:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c01a9b0 con 0x7f092c0c7d40 2023-10-06T11:37:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.276+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0c49e0 con 0x7f092c0c7d40 2023-10-06T11:37:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.280+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c064590 con 0x7f092c0c7d40 2023-10-06T11:37:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.280+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f091c02d440 con 0x7f092c0c7d40 2023-10-06T11:37:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.284+0000 7f0921ffb700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 0x7f0910062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.284+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f091c07ec20 con 0x7f092c0c7d40 2023-10-06T11:37:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.284+0000 7f0931c69700 1 -- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 msgr2=0x7f0910062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.284+0000 7f0931c69700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 0x7f0910062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:05.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.292+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f091c04ea70 con 0x7f092c0c7d40 2023-10-06T11:37:05.318 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:05.304+0000 7fe5fa1c3000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:37:05.336 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.333+0000 7f855fdfe000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:37:05.404 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:05.388+0000 7fe5fa1c3000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:37:05.419 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.417+0000 7f855fdfe000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:37:05.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.460+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f092c0c1a20 con 0x7f092c0c7d40 2023-10-06T11:37:05.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.460+0000 7f0921ffb700 1 -- 172.21.15.46:0/3545702821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7f091c052090 con 0x7f092c0c7d40 2023-10-06T11:37:05.475 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:37:05.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 msgr2=0x7f0910062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 0x7f0910062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 msgr2=0x7f092c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f092c0c9790 tx=0x7f091c017000 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 shutdown_connections 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f09100607c0 0x7f0910062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f092c0c97e0 0x7f092c0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f092c0c7d40 0x7f092c0ae9b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 --2- 172.21.15.46:0/3545702821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f092c06cf30 0x7f092c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 >> 172.21.15.46:0/3545702821 conn(0x7f092c0cbc10 msgr2=0x7f092c157fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 shutdown_connections 2023-10-06T11:37:05.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.464+0000 7f0932c6b700 1 -- 172.21.15.46:0/3545702821 wait complete. 2023-10-06T11:37:05.492 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:05.476+0000 7fe5fa1c3000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:37:05.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:37:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.648+0000 7fd24cca5700 1 Processor -- start 2023-10-06T11:37:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.648+0000 7fd24cca5700 1 -- start start 2023-10-06T11:37:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24cca5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2480b23c0 0x7fd2480b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24cca5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24cca5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480cc510 con 0x7fd2480cae40 2023-10-06T11:37:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24cca5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2480cb900 con 0x7fd2480b23c0 2023-10-06T11:37:05.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24cca5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480cbbe0 con 0x7fd2480cd250 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24659c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2480b23c0 0x7fd2480b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:45114/0 (socket says 172.21.15.46:45114) 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 -- 172.21.15.46:0/4077963007 learned_addr learned my addr 172.21.15.46:0/4077963007 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd245d9b700 1 --1- 172.21.15.46:0/4077963007 >> v1:172.21.15.132:6789/0 conn(0x7fd2480cae40 0x7fd2480ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35490/0 (socket says 172.21.15.46:35490) 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24559a700 1 -- 172.21.15.46:0/4077963007 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640103828 0 0) 0x7fd2480cc510 con 0x7fd2480cae40 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24559a700 1 -- 172.21.15.46:0/4077963007 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd230003570 con 0x7fd2480cae40 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 -- 172.21.15.46:0/4077963007 >> v2:172.21.15.132:3300/0 conn(0x7fd2480b23c0 msgr2=0x7fd2480b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 --2- 172.21.15.46:0/4077963007 >> v2:172.21.15.132:3300/0 conn(0x7fd2480b23c0 0x7fd2480b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 -- 172.21.15.46:0/4077963007 >> v1:172.21.15.132:6789/0 conn(0x7fd2480cae40 legacy=0x7fd2480ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 -- 172.21.15.46:0/4077963007 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480cbec0 con 0x7fd2480cd250 2023-10-06T11:37:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd246d9d700 1 --2- 172.21.15.46:0/4077963007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd240009f20 tx=0x7fd240009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe90213756bfd1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24559a700 1 -- 172.21.15.46:0/4077963007 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24000e070 con 0x7fd2480cd250 2023-10-06T11:37:05.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24559a700 1 -- 172.21.15.46:0/4077963007 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24000b970 con 0x7fd2480cd250 2023-10-06T11:37:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.652+0000 7fd24559a700 1 -- 172.21.15.46:0/4077963007 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400092e0 con 0x7fd2480cd250 2023-10-06T11:37:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- 172.21.15.46:0/4077963007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 msgr2=0x7fd2480cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 --2- 172.21.15.46:0/4077963007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd240009f20 tx=0x7fd240009840 comp rx=0 tx=0).stop 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- 172.21.15.46:0/4077963007 shutdown_connections 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 --2- 172.21.15.46:0/4077963007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480cd250 0x7fd2480cb290 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 --2- 172.21.15.46:0/4077963007 >> v2:172.21.15.132:3300/0 conn(0x7fd2480b23c0 0x7fd2480b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- 172.21.15.46:0/4077963007 >> 172.21.15.46:0/4077963007 conn(0x7fd248099480 msgr2=0x7fd24806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- 172.21.15.46:0/4077963007 shutdown_connections 2023-10-06T11:37:05.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- 172.21.15.46:0/4077963007 wait complete. 2023-10-06T11:37:05.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 Processor -- start 2023-10-06T11:37:05.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.656+0000 7fd24cca5700 1 -- start start 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 0x7fd24815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24809dec0 con 0x7fd2480cd250 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd248169c60 con 0x7fd2480cae40 2023-10-06T11:37:05.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24cca5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd248158260 con 0x7fd2480b23c0 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd246d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:33820/0 (socket says 172.21.15.46:33820) 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 -- 172.21.15.46:0/1629551314 learned_addr learned my addr 172.21.15.46:0/1629551314 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd246d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36088/0 (socket says 172.21.15.46:36088) 2023-10-06T11:37:05.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24659c700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 0x7fd24815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 -- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 msgr2=0x7fd24815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 0x7fd24815a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 -- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 msgr2=0x7fd248157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd245d9b700 1 -- 172.21.15.46:0/1629551314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240009ae0 con 0x7fd2480cae40 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd24659c700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 0x7fd24815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.660+0000 7fd246d9d700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd245d9b700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c7c0 tx=0x7fd23c007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c005c10 con 0x7fd2480cae40 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c005d70 con 0x7fd2480cae40 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007320 con 0x7fd2480cae40 2023-10-06T11:37:05.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24815a930 con 0x7fd2480cae40 2023-10-06T11:37:05.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2481698f0 con 0x7fd2480cae40 2023-10-06T11:37:05.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248169df0 con 0x7fd2480cae40 2023-10-06T11:37:05.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.664+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd23c01b3b0 con 0x7fd2480cae40 2023-10-06T11:37:05.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.668+0000 7fd2367fc700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 0x7fd228062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.668+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fd23c07c680 con 0x7fd2480cae40 2023-10-06T11:37:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.668+0000 7fd24659c700 1 -- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 msgr2=0x7fd228062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.668+0000 7fd24659c700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 0x7fd228062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.676+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd23c04c4d0 con 0x7fd2480cae40 2023-10-06T11:37:05.814 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.813+0000 7f855fdfe000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:37:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.848+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2480660f0 con 0x7fd2480cae40 2023-10-06T11:37:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.848+0000 7fd2367fc700 1 -- 172.21.15.46:0/1629551314 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd23c04faf0 con 0x7fd2480cae40 2023-10-06T11:37:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T11:37:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 msgr2=0x7fd228062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 0x7fd228062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 msgr2=0x7fd24809d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c7c0 tx=0x7fd23c007e00 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 shutdown_connections 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd228060790 0x7fd228062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2480cd250 0x7fd248157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2480cae40 0x7fd24809d980 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 --2- 172.21.15.46:0/1629551314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2480b23c0 0x7fd24815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 >> 172.21.15.46:0/1629551314 conn(0x7fd248099480 msgr2=0x7fd2480c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 shutdown_connections 2023-10-06T11:37:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:05.852+0000 7fd24cca5700 1 -- 172.21.15.46:0/1629551314 wait complete. 2023-10-06T11:37:05.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:37:05.929 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:05.916+0000 7fe5fa1c3000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:37:05.974 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:05.973+0000 7f855fdfe000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:37:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 Processor -- start 2023-10-06T11:37:06.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 -- start start 2023-10-06T11:37:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd61c0a69c0 0x7fd61c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c06f650 con 0x7fd61c0abd90 2023-10-06T11:37:06.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0aecd0 con 0x7fd61c0add40 2023-10-06T11:37:06.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0b02d0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd61c0a69c0 0x7fd61c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd61c0a69c0 0x7fd61c06ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36102/0 (socket says 172.21.15.46:36102) 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 -- 172.21.15.46:0/2653258115 learned_addr learned my addr 172.21.15.46:0/2653258115 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd619d9b700 1 --1- 172.21.15.46:0/2653258115 >> v1:172.21.15.46:6789/0 conn(0x7fd61c0abd90 0x7fd61c0ad630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34436/0 (socket says 172.21.15.46:34436) 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 -- 172.21.15.46:0/2653258115 >> v1:172.21.15.132:6789/0 conn(0x7fd61c0add40 legacy=0x7fd61c096220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 -- 172.21.15.46:0/2653258115 >> v1:172.21.15.46:6789/0 conn(0x7fd61c0abd90 legacy=0x7fd61c0ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61a59c700 1 -- 172.21.15.46:0/2653258115 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0ae150 con 0x7fd61c0a69c0 2023-10-06T11:37:06.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.032+0000 7fd61959a700 1 -- 172.21.15.46:0/2653258115 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852813225 0 0) 0x7fd61c06f650 con 0x7fd61c0abd90 2023-10-06T11:37:06.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61a59c700 1 --2- 172.21.15.46:0/2653258115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c06ef00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0099b0 tx=0x7fd60c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0033a1267a9f8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:06.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61959a700 1 -- 172.21.15.46:0/2653258115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00b860 con 0x7fd61c0a69c0 2023-10-06T11:37:06.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61959a700 1 -- 172.21.15.46:0/2653258115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00b9c0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61959a700 1 -- 172.21.15.46:0/2653258115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00bd60 con 0x7fd61c0a69c0 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 -- 172.21.15.46:0/2653258115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 msgr2=0x7fd61c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 --2- 172.21.15.46:0/2653258115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c06ef00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0099b0 tx=0x7fd60c009250 comp rx=0 tx=0).stop 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 -- 172.21.15.46:0/2653258115 shutdown_connections 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 --2- 172.21.15.46:0/2653258115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c06ef00 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 -- 172.21.15.46:0/2653258115 >> 172.21.15.46:0/2653258115 conn(0x7fd61c0afd10 msgr2=0x7fd61c06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:06.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 -- 172.21.15.46:0/2653258115 shutdown_connections 2023-10-06T11:37:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.036+0000 7fd61b59e700 1 -- 172.21.15.46:0/2653258115 wait complete. 2023-10-06T11:37:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 Processor -- start 2023-10-06T11:37:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 -- start start 2023-10-06T11:37:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd61c0abd90 0x7fd61c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61c0add40 0x7fd61c0c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0c5560 con 0x7fd61c0a69c0 2023-10-06T11:37:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c15b040 con 0x7fd61c0add40 2023-10-06T11:37:06.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0c17b0 con 0x7fd61c0abd90 2023-10-06T11:37:06.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.040+0000 7fd61a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36112/0 (socket says 172.21.15.46:36112) 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 -- 172.21.15.46:0/585590913 learned_addr learned my addr 172.21.15.46:0/585590913 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd619d9b700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd61c0abd90 0x7fd61c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61ad9d700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61c0add40 0x7fd61c0c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 -- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd61c0abd90 msgr2=0x7fd61c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd61c0abd90 0x7fd61c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 -- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61c0add40 msgr2=0x7fd61c0c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61c0add40 0x7fd61c0c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 -- 172.21.15.46:0/585590913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0094f0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61a59c700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd60c006f50 tx=0x7fd60c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01f070 con 0x7fd61c0a69c0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c011bb0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c1694f0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c025420 con 0x7fd61c0a69c0 2023-10-06T11:37:06.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c169a60 con 0x7fd61c0a69c0 2023-10-06T11:37:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c064590 con 0x7fd61c0a69c0 2023-10-06T11:37:06.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.044+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd60c023070 con 0x7fd61c0a69c0 2023-10-06T11:37:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.048+0000 7fd60a7fc700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 0x7fd5fc062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.048+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fd60c07de70 con 0x7fd61c0a69c0 2023-10-06T11:37:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.048+0000 7fd619d9b700 1 -- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 msgr2=0x7fd5fc062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:06.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.048+0000 7fd619d9b700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 0x7fd5fc062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:06.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.052+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd60c04de50 con 0x7fd61c0a69c0 2023-10-06T11:37:06.102 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:06.088+0000 7fe5fa1c3000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:37:06.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.224+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd61c04b5c0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.224+0000 7fd60a7fc700 1 -- 172.21.15.46:0/585590913 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd60c0150a0 con 0x7fd61c0a69c0 2023-10-06T11:37:06.241 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T11:37:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 msgr2=0x7fd5fc062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 0x7fd5fc062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 msgr2=0x7fd61c0c1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd60c006f50 tx=0x7fd60c017000 comp rx=0 tx=0).stop 2023-10-06T11:37:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 shutdown_connections 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5fc060790 0x7fd5fc062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd61c0add40 0x7fd61c0c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd61c0abd90 0x7fd61c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 --2- 172.21.15.46:0/585590913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd61c0a69c0 0x7fd61c0c1a70 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 >> 172.21.15.46:0/585590913 conn(0x7fd61c0afd10 msgr2=0x7fd61c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 shutdown_connections 2023-10-06T11:37:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.228+0000 7fd61b59e700 1 -- 172.21.15.46:0/585590913 wait complete. 2023-10-06T11:37:06.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:06.318 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.317+0000 7f855fdfe000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:37:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 Processor -- start 2023-10-06T11:37:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 -- start start 2023-10-06T11:37:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e806d460 con 0x7fb4e815ab60 2023-10-06T11:37:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e80c7000 con 0x7fb4e80ad6c0 2023-10-06T11:37:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4ed785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e806ed10 con 0x7fb4e806e8a0 2023-10-06T11:37:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4e77fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb4e80ad6c0 0x7fb4e80adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35508/0 (socket says 172.21.15.46:35508) 2023-10-06T11:37:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.452+0000 7fb4e6ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4e815ab60 0x7fb4e815af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34446/0 (socket says 172.21.15.46:34446) 2023-10-06T11:37:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e77fe700 1 -- 172.21.15.46:0/1220236499 learned_addr learned my addr 172.21.15.46:0/1220236499 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e67fc700 1 --2- 172.21.15.46:0/1220236499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002510294 0 0) 0x7fb4e80c7000 con 0x7fb4e80ad6c0 2023-10-06T11:37:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d0003570 con 0x7fb4e80ad6c0 2023-10-06T11:37:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e67fc700 1 -- 172.21.15.46:0/1220236499 >> v1:172.21.15.132:6789/0 conn(0x7fb4e80ad6c0 legacy=0x7fb4e80adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e67fc700 1 -- 172.21.15.46:0/1220236499 >> v1:172.21.15.46:6789/0 conn(0x7fb4e815ab60 legacy=0x7fb4e815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513088087 0 0) 0x7fb4e806d460 con 0x7fb4e815ab60 2023-10-06T11:37:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e67fc700 1 -- 172.21.15.46:0/1220236499 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80c72c0 con 0x7fb4e806e8a0 2023-10-06T11:37:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e67fc700 1 --2- 172.21.15.46:0/1220236499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e806cc30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009910 tx=0x7fb4dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6acbc574a4f366f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00b680 con 0x7fb4e806e8a0 2023-10-06T11:37:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b7e0 con 0x7fb4e806e8a0 2023-10-06T11:37:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4e5ffb700 1 -- 172.21.15.46:0/1220236499 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00bba0 con 0x7fb4e806e8a0 2023-10-06T11:37:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 -- 172.21.15.46:0/1220236499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 msgr2=0x7fb4e806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 --2- 172.21.15.46:0/1220236499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e806cc30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009910 tx=0x7fb4dc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 -- 172.21.15.46:0/1220236499 shutdown_connections 2023-10-06T11:37:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 --2- 172.21.15.46:0/1220236499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e806cc30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 -- 172.21.15.46:0/1220236499 >> 172.21.15.46:0/1220236499 conn(0x7fb4e80cbe30 msgr2=0x7fb4e80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 -- 172.21.15.46:0/1220236499 shutdown_connections 2023-10-06T11:37:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.456+0000 7fb4ed785700 1 -- 172.21.15.46:0/1220236499 wait complete. 2023-10-06T11:37:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.460+0000 7fb4ed785700 1 Processor -- start 2023-10-06T11:37:06.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.460+0000 7fb4ed785700 1 -- start start 2023-10-06T11:37:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e81670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e80ad6c0 0x7fb4e8169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80b0570 con 0x7fb4e815ab60 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80c3680 con 0x7fb4e80ad6c0 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80c4df0 con 0x7fb4e806e8a0 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36124/0 (socket says 172.21.15.46:36124) 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 -- 172.21.15.46:0/3060123665 learned_addr learned my addr 172.21.15.46:0/3060123665 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e6ffd700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e81670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e67fc700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e80ad6c0 0x7fb4e8169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 msgr2=0x7fb4e81670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e81670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e80ad6c0 msgr2=0x7fb4e8169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e80ad6c0 0x7fb4e8169d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 -- 172.21.15.46:0/3060123665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009420 con 0x7fb4e815ab60 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e6ffd700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e81670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4e77fe700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0001460 tx=0x7fb4e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000a940 con 0x7fb4e815ab60 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000aaa0 con 0x7fb4e815ab60 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e80afc40 con 0x7fb4e815ab60 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000ae20 con 0x7fb4e815ab60 2023-10-06T11:37:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.464+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e80b0200 con 0x7fb4e815ab60 2023-10-06T11:37:06.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.468+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e80b06e0 con 0x7fb4e815ab60 2023-10-06T11:37:06.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.468+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fb4e0007300 con 0x7fb4e815ab60 2023-10-06T11:37:06.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.472+0000 7fb4ceffd700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 0x7fb4d4062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.472+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7fb4e007d430 con 0x7fb4e815ab60 2023-10-06T11:37:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.472+0000 7fb4e6ffd700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 msgr2=0x7fb4d4062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:06.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.472+0000 7fb4e6ffd700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 0x7fb4d4062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:06.487 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:06.472+0000 7fe5fa1c3000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:37:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.476+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb4e004d300 con 0x7fb4e815ab60 2023-10-06T11:37:06.532 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.529+0000 7f855fdfe000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:06.615 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.613+0000 7f855fdfe000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:37:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.676+0000 7fb4e6ffd700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 msgr2=0x7fb4d4062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.676+0000 7fb4e6ffd700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 0x7fb4d4062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:06.690 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.685+0000 7f855fdfe000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:37:06.766 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.761+0000 7f855fdfe000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:37:06.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:06.820+0000 7fe5fa1c3000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:06.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.828+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4e8063f90 con 0x7fb4e815ab60 2023-10-06T11:37:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.828+0000 7fb4ceffd700 1 -- 172.21.15.46:0/3060123665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb4e0050920 con 0x7fb4e815ab60 2023-10-06T11:37:06.843 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:06.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 msgr2=0x7fb4d4062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 0x7fb4d4062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 msgr2=0x7fb4e80c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0001460 tx=0x7fb4e0001ae0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 shutdown_connections 2023-10-06T11:37:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb4d4060790 0x7fb4d4062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e815ab60 0x7fb4e80c42e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4e80ad6c0 0x7fb4e8169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 --2- 172.21.15.46:0/3060123665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4e806e8a0 0x7fb4e81670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 >> 172.21.15.46:0/3060123665 conn(0x7fb4e80cbe30 msgr2=0x7fb4e80adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 shutdown_connections 2023-10-06T11:37:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:06.832+0000 7fb4ed785700 1 -- 172.21.15.46:0/3060123665 wait complete. 2023-10-06T11:37:06.849 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:37:06.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:06.932 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:06.916+0000 7fe5fa1c3000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:37:06.979 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:06.977+0000 7f855fdfe000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:07.014 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.000+0000 7fe5fa1c3000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:37:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.012+0000 7f9f3afc3700 1 Processor -- start 2023-10-06T11:37:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.012+0000 7f9f3afc3700 1 -- start start 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.012+0000 7f9f3afc3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f340aaf30 0x7f9f34067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.012+0000 7f9f3afc3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f34067590 0x7f9f34068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3afc3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f340679f0 con 0x7f9f34156e20 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3afc3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340acd60 con 0x7f9f34067590 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3afc3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f340ad040 con 0x7f9f340aaf30 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f39fc1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f34156e20 0x7f9f3406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34460/0 (socket says 172.21.15.46:34460) 2023-10-06T11:37:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f39fc1700 1 -- 172.21.15.46:0/943437157 learned_addr learned my addr 172.21.15.46:0/943437157 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 --2- 172.21.15.46:0/943437157 >> v2:172.21.15.46:3300/0 conn(0x7f9f34067590 0x7f9f34068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f397c0700 1 --2- 172.21.15.46:0/943437157 >> v2:172.21.15.132:3300/0 conn(0x7f9f340aaf30 0x7f9f34067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574426870 0 0) 0x7f9f30001110 con 0x7f9f34156e20 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f20003570 con 0x7f9f34156e20 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 -- 172.21.15.46:0/943437157 >> v2:172.21.15.132:3300/0 conn(0x7f9f340aaf30 msgr2=0x7f9f34067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 --2- 172.21.15.46:0/943437157 >> v2:172.21.15.132:3300/0 conn(0x7f9f340aaf30 0x7f9f34067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055812564 0 0) 0x7f9f20003570 con 0x7f9f34156e20 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 -- 172.21.15.46:0/943437157 >> v1:172.21.15.46:6789/0 conn(0x7f9f34156e20 legacy=0x7f9f3406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 -- 172.21.15.46:0/943437157 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340ad320 con 0x7f9f34067590 2023-10-06T11:37:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3a7c2700 1 --2- 172.21.15.46:0/943437157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f34068a20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9f340c2b50 tx=0x7f9f2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af05d6b314906976 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00e070 con 0x7f9f34067590 2023-10-06T11:37:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00b7a0 con 0x7f9f34067590 2023-10-06T11:37:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f38fbf700 1 -- 172.21.15.46:0/943437157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30001600 con 0x7f9f34067590 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3afc3700 1 -- 172.21.15.46:0/943437157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 msgr2=0x7f9f34068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.016+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/943437157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f34068a20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9f340c2b50 tx=0x7f9f2c009780 comp rx=0 tx=0).stop 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 -- 172.21.15.46:0/943437157 shutdown_connections 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/943437157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f34068a20 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/943437157 >> v2:172.21.15.132:3300/0 conn(0x7f9f340aaf30 0x7f9f34067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 -- 172.21.15.46:0/943437157 >> 172.21.15.46:0/943437157 conn(0x7f9f34062ef0 msgr2=0x7f9f340cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 -- 172.21.15.46:0/943437157 shutdown_connections 2023-10-06T11:37:07.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 -- 172.21.15.46:0/943437157 wait complete. 2023-10-06T11:37:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 Processor -- start 2023-10-06T11:37:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.020+0000 7f9f3afc3700 1 -- start start 2023-10-06T11:37:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f340aaf30 0x7f9f3415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f34156e20 0x7f9f34142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34166f00 con 0x7f9f34067590 2023-10-06T11:37:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f341571f0 con 0x7f9f34156e20 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34158050 con 0x7f9f340aaf30 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3a7c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f34156e20 0x7f9f34142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36132/0 (socket says 172.21.15.46:36132) 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 -- 172.21.15.46:0/2545419347 learned_addr learned my addr 172.21.15.46:0/2545419347 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f397c0700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f340aaf30 0x7f9f3415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 -- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f340aaf30 msgr2=0x7f9f3415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f340aaf30 0x7f9f3415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 -- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f34156e20 msgr2=0x7f9f34142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f34156e20 0x7f9f34142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 -- 172.21.15.46:0/2545419347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009a20 con 0x7f9f34067590 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f39fc1700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cbc0 tx=0x7f9f30001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30007150 con 0x7f9f34067590 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300072d0 con 0x7f9f34067590 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3415b930 con 0x7f9f34067590 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30007690 con 0x7f9f34067590 2023-10-06T11:37:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.024+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f34157720 con 0x7f9f34067590 2023-10-06T11:37:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.028+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f34064590 con 0x7f9f34067590 2023-10-06T11:37:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.028+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f9f30004a40 con 0x7f9f34067590 2023-10-06T11:37:07.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.032+0000 7f9f29ffb700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 0x7f9f18062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.032+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f9f3007c550 con 0x7f9f34067590 2023-10-06T11:37:07.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.032+0000 7f9f397c0700 1 -- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 msgr2=0x7f9f18062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:07.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.032+0000 7f9f397c0700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 0x7f9f18062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:07.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.036+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f3004c420 con 0x7f9f34067590 2023-10-06T11:37:07.053 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:07.049+0000 7f855fdfe000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:37:07.106 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.092+0000 7fe5fa1c3000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:37:07.137 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:07.133+0000 7f855fdfe000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:37:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.216+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f340679f0 con 0x7f9f34067590 2023-10-06T11:37:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.216+0000 7f9f29ffb700 1 -- 172.21.15.46:0/2545419347 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18042 (secure 0 0 0) 0x7f9f340679f0 con 0x7f9f34067590 2023-10-06T11:37:07.233 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:07.233 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:27.255187+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2960823291":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6833/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6832/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/4259972733":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3863334752":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3488620324":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/1900876313":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/2274789125":"2023-10-07T11:36:27.255068+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 msgr2=0x7f9f18062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 0x7f9f18062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 msgr2=0x7f9f341669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cbc0 tx=0x7f9f30001e90 comp rx=0 tx=0).stop 2023-10-06T11:37:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 shutdown_connections 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9f18060790 0x7f9f18062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.220+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f34156e20 0x7f9f34142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.224+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f340aaf30 0x7f9f3415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.224+0000 7f9f3afc3700 1 --2- 172.21.15.46:0/2545419347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34067590 0x7f9f341669c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.224+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 >> 172.21.15.46:0/2545419347 conn(0x7f9f34062ef0 msgr2=0x7f9f340cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:07.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.224+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 shutdown_connections 2023-10-06T11:37:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.224+0000 7f9f3afc3700 1 -- 172.21.15.46:0/2545419347 wait complete. 2023-10-06T11:37:07.261 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:37:07.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:07.319 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:07.317+0000 7f855fdfe000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:37:07.350 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.336+0000 7fe5fa1c3000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:07.402 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:07.397+0000 7f855fdfe000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:37:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 Processor -- start 2023-10-06T11:37:07.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 -- start start 2023-10-06T11:37:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 0x7f3d7c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cf5d0 con 0x7f3d7c0cb400 2023-10-06T11:37:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0af860 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d81745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0ce9c0 con 0x7f3d7c0aef50 2023-10-06T11:37:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36142/0 (socket says 172.21.15.46:36142) 2023-10-06T11:37:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 -- 172.21.15.46:0/1867487686 learned_addr learned my addr 172.21.15.46:0/1867487686 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:07.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 0x7f3d7c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 -- 172.21.15.46:0/1867487686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 msgr2=0x7f3d7c0af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:07.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 0x7f3d7c0af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.400+0000 7f3d7a7fc700 1 -- 172.21.15.46:0/1867487686 >> v1:172.21.15.132:6789/0 conn(0x7f3d7c0cb400 legacy=0x7f3d7c069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d7a7fc700 1 -- 172.21.15.46:0/1867487686 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0ceca0 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 0x7f3d7c0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d7a7fc700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009990 tx=0x7f3d70009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a04e5dca724dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d79ffb700 1 -- 172.21.15.46:0/1867487686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70005c10 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d79ffb700 1 -- 172.21.15.46:0/1867487686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d70005d70 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d79ffb700 1 -- 172.21.15.46:0/1867487686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000b840 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 -- 172.21.15.46:0/1867487686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 msgr2=0x7f3d7c0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009990 tx=0x7f3d70009220 comp rx=0 tx=0).stop 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 -- 172.21.15.46:0/1867487686 shutdown_connections 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c0caec0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 --2- 172.21.15.46:0/1867487686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0aef50 0x7f3d7c0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 -- 172.21.15.46:0/1867487686 >> 172.21.15.46:0/1867487686 conn(0x7f3d7c06f200 msgr2=0x7f3d7c0d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 -- 172.21.15.46:0/1867487686 shutdown_connections 2023-10-06T11:37:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.404+0000 7f3d81745700 1 -- 172.21.15.46:0/1867487686 wait complete. 2023-10-06T11:37:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 Processor -- start 2023-10-06T11:37:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 -- start start 2023-10-06T11:37:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 0x7f3d7c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0cb400 0x7f3d7c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0b5b10 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c154dc0 con 0x7f3d7c0aef50 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d81745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c155660 con 0x7f3d7c0cb400 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36146/0 (socket says 172.21.15.46:36146) 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 -- 172.21.15.46:0/1878702762 learned_addr learned my addr 172.21.15.46:0/1878702762 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7a7fc700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0cb400 0x7f3d7c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 0x7f3d7c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0cb400 msgr2=0x7f3d7c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0cb400 0x7f3d7c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 msgr2=0x7f3d7c0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 0x7f3d7c0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 -- 172.21.15.46:0/1878702762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009480 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 0x7f3d7c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d7b7fe700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c890 tx=0x7f3d74002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74012630 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74012790 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.408+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74012b50 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.412+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c15bec0 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.412+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c1552f0 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.412+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c064590 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.416+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f3d7401b400 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.416+0000 7f3d62ffd700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 0x7f3d68062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.416+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f3d7407d250 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.416+0000 7f3d7affd700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 msgr2=0x7f3d68062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:07.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.416+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 0x7f3d68062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:07.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.424+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3d7404d140 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.438 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.424+0000 7fe5fa1c3000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:37:07.533 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.516+0000 7fe5fa1c3000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:37:07.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.620+0000 7f3d7affd700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 msgr2=0x7f3d68062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:07.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.620+0000 7f3d7affd700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 0x7f3d68062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.640+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7c1557a0 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.640+0000 7f3d62ffd700 1 -- 172.21.15.46:0/1878702762 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f3d74050760 con 0x7f3d7c0cfa60 2023-10-06T11:37:07.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:07.656 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 msgr2=0x7f3d68062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 0x7f3d68062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 msgr2=0x7f3d7c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400c890 tx=0x7f3d74002680 comp rx=0 tx=0).stop 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 shutdown_connections 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3d68060790 0x7f3d68062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d7c0cfa60 0x7f3d7c15b7f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d7c0cb400 0x7f3d7c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 --2- 172.21.15.46:0/1878702762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7c0aef50 0x7f3d7c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 >> 172.21.15.46:0/1878702762 conn(0x7f3d7c06f200 msgr2=0x7f3d7c158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:07.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 shutdown_connections 2023-10-06T11:37:07.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.644+0000 7f3d81745700 1 -- 172.21.15.46:0/1878702762 wait complete. 2023-10-06T11:37:07.661 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:37:07.679 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:07.743 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.728+0000 7fe5fa1c3000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:37:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.812+0000 7f4af1905700 1 Processor -- start 2023-10-06T11:37:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.812+0000 7f4af1905700 1 -- start start 2023-10-06T11:37:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4af1905700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4aec06e8e0 0x7f4aec0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4af1905700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4af1905700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec0ae550 con 0x7f4aec154a60 2023-10-06T11:37:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4af1905700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec0adcb0 con 0x7f4aec06e8e0 2023-10-06T11:37:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4af1905700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0adf90 con 0x7f4aec0ad450 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4aec06e8e0 0x7f4aec0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aeaffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4aec154a60 0x7f4aec06e1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34464/0 (socket says 172.21.15.46:34464) 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aeaffd700 1 -- 172.21.15.46:0/3796101062 learned_addr learned my addr 172.21.15.46:0/3796101062 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aeb7fe700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 -- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 msgr2=0x7f4aec068c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec068c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 -- 172.21.15.46:0/3796101062 >> v1:172.21.15.46:6789/0 conn(0x7f4aec154a60 legacy=0x7f4aec06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 -- 172.21.15.46:0/3796101062 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0ae270 con 0x7f4aec06e8e0 2023-10-06T11:37:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4aea7fc700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec06e8e0 0x7f4aec0acf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00098e0 tx=0x7f4ae00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745c9a14059e8562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4ae9ffb700 1 -- 172.21.15.46:0/3796101062 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0009e80 con 0x7f4aec06e8e0 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4ae9ffb700 1 -- 172.21.15.46:0/3796101062 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000b860 con 0x7f4aec06e8e0 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.816+0000 7f4ae9ffb700 1 -- 172.21.15.46:0/3796101062 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000bbe0 con 0x7f4aec06e8e0 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec06e8e0 msgr2=0x7f4aec0acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec06e8e0 0x7f4aec0acf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00098e0 tx=0x7f4ae00091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- 172.21.15.46:0/3796101062 shutdown_connections 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec068c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 --2- 172.21.15.46:0/3796101062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec06e8e0 0x7f4aec0acf10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- 172.21.15.46:0/3796101062 >> 172.21.15.46:0/3796101062 conn(0x7f4aec0cc6b0 msgr2=0x7f4aec0c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- 172.21.15.46:0/3796101062 shutdown_connections 2023-10-06T11:37:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- 172.21.15.46:0/3796101062 wait complete. 2023-10-06T11:37:07.835 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:07.820+0000 7fe5fa1c3000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:37:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 Processor -- start 2023-10-06T11:37:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.820+0000 7f4af1905700 1 -- start start 2023-10-06T11:37:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 0x7f4aec159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec06a0b0 con 0x7f4aec06e8e0 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec158700 con 0x7f4aec154a60 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec16a140 con 0x7f4aec0ad450 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36160/0 (socket says 172.21.15.46:36160) 2023-10-06T11:37:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 -- 172.21.15.46:0/461421041 learned_addr learned my addr 172.21.15.46:0/461421041 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aea7fc700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeb7fe700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 0x7f4aec159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 msgr2=0x7f4aec152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 msgr2=0x7f4aec159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 0x7f4aec159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 -- 172.21.15.46:0/461421041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae0009420 con 0x7f4aec06e8e0 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeb7fe700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 0x7f4aec159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4aeaffd700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c560 tx=0x7f4adc001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc005b30 con 0x7f4aec06e8e0 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc005cb0 con 0x7f4aec06e8e0 2023-10-06T11:37:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc007320 con 0x7f4aec06e8e0 2023-10-06T11:37:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec158950 con 0x7f4aec06e8e0 2023-10-06T11:37:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.824+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec158f10 con 0x7f4aec06e8e0 2023-10-06T11:37:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec159130 con 0x7f4aec06e8e0 2023-10-06T11:37:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f4adc007690 con 0x7f4aec06e8e0 2023-10-06T11:37:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4ad2ffd700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 0x7f4ad8062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4aea7fc700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 msgr2=0x7f4ad8062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f4adc07c460 con 0x7f4aec06e8e0 2023-10-06T11:37:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.828+0000 7f4aea7fc700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 0x7f4ad8062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:07.836+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4adc04c2b0 con 0x7f4aec06e8e0 2023-10-06T11:37:08.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.032+0000 7f4aea7fc700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 msgr2=0x7f4ad8062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:08.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.032+0000 7f4aea7fc700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 0x7f4ad8062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:08.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aec064590 con 0x7f4aec06e8e0 2023-10-06T11:37:08.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4ad2ffd700 1 -- 172.21.15.46:0/461421041 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4adc04f8d0 con 0x7f4aec06e8e0 2023-10-06T11:37:08.058 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:08.058 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:36:08.275313+0000","modified":"2023-10-06T11:36:11.843055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7824},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7824":{"gid":7824,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/2696512883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2696512883},{"type":"v1","addr":"172.21.15.46:6836","nonce":2696512883}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 msgr2=0x7f4ad8062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 0x7f4ad8062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 msgr2=0x7f4aec069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c560 tx=0x7f4adc001020 comp rx=0 tx=0).stop 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 shutdown_connections 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4ad8060790 0x7f4ad8062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec154a60 0x7f4aec159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0ad450 0x7f4aec152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 --2- 172.21.15.46:0/461421041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec06e8e0 0x7f4aec069b70 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 >> 172.21.15.46:0/461421041 conn(0x7f4aec0cc6b0 msgr2=0x7f4aec153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.044+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 shutdown_connections 2023-10-06T11:37:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.048+0000 7f4af1905700 1 -- 172.21.15.46:0/461421041 wait complete. 2023-10-06T11:37:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2023-10-06T11:37:08.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:08.120 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:08.117+0000 7f855fdfe000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:37:08.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.228+0000 7f869b345700 1 Processor -- start 2023-10-06T11:37:08.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.228+0000 7f869b345700 1 -- start start 2023-10-06T11:37:08.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f869b345700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86941502f0 0x7f86940c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f869b345700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f869415a1a0 0x7f86940c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f869b345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940c67e0 con 0x7f86941506c0 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f869b345700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869406e810 con 0x7f86941502f0 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f869b345700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869406eaf0 con 0x7f869415a1a0 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86941502f0 0x7f86940c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86941502f0 0x7f86940c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36168/0 (socket says 172.21.15.46:36168) 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 -- 172.21.15.46:0/928214266 learned_addr learned my addr 172.21.15.46:0/928214266 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86998e2700 1 --2- 172.21.15.46:0/928214266 >> v2:172.21.15.132:3300/0 conn(0x7f869415a1a0 0x7f86940c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 -- 172.21.15.46:0/928214266 >> v2:172.21.15.132:3300/0 conn(0x7f869415a1a0 msgr2=0x7f86940c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 --2- 172.21.15.46:0/928214266 >> v2:172.21.15.132:3300/0 conn(0x7f869415a1a0 0x7f86940c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 -- 172.21.15.46:0/928214266 >> v1:172.21.15.132:6789/0 conn(0x7f86941506c0 legacy=0x7f8694159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 -- 172.21.15.46:0/928214266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869406edd0 con 0x7f86941502f0 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.232+0000 7f86990e1700 1 --2- 172.21.15.46:0/928214266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941502f0 0x7f86940c0bd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f86940c9990 tx=0x7f869000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528717fbfc543e18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f868bfff700 1 -- 172.21.15.46:0/928214266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000db70 con 0x7f86941502f0 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f868bfff700 1 -- 172.21.15.46:0/928214266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869000dcd0 con 0x7f86941502f0 2023-10-06T11:37:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f868bfff700 1 -- 172.21.15.46:0/928214266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900196b0 con 0x7f86941502f0 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 -- 172.21.15.46:0/928214266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941502f0 msgr2=0x7f86940c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 --2- 172.21.15.46:0/928214266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941502f0 0x7f86940c0bd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f86940c9990 tx=0x7f869000b4f0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 -- 172.21.15.46:0/928214266 shutdown_connections 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 --2- 172.21.15.46:0/928214266 >> v2:172.21.15.132:3300/0 conn(0x7f869415a1a0 0x7f86940c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 --2- 172.21.15.46:0/928214266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941502f0 0x7f86940c0bd0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 -- 172.21.15.46:0/928214266 >> 172.21.15.46:0/928214266 conn(0x7f86940cbe30 msgr2=0x7f86940cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 -- 172.21.15.46:0/928214266 shutdown_connections 2023-10-06T11:37:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.236+0000 7f869b345700 1 -- 172.21.15.46:0/928214266 wait complete. 2023-10-06T11:37:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 Processor -- start 2023-10-06T11:37:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 -- start start 2023-10-06T11:37:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 0x7f869414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f869415a1a0 0x7f86940ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940b0c20 con 0x7f86941506c0 2023-10-06T11:37:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869414f500 con 0x7f86941502f0 2023-10-06T11:37:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f869b345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86941567b0 con 0x7f869415a1a0 2023-10-06T11:37:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f86988e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f86988e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36180/0 (socket says 172.21.15.46:36180) 2023-10-06T11:37:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f86988e0700 1 -- 172.21.15.46:0/414199012 learned_addr learned my addr 172.21.15.46:0/414199012 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f86998e2700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f869415a1a0 0x7f86940ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.240+0000 7f86990e1700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 0x7f869414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 -- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f869415a1a0 msgr2=0x7f86940ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f869415a1a0 0x7f86940ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 -- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 msgr2=0x7f869414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 0x7f869414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 -- 172.21.15.46:0/414199012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869000b730 con 0x7f86941506c0 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86990e1700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 0x7f869414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:08.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86988e0700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f868400c590 tx=0x7f868400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011590 con 0x7f86941506c0 2023-10-06T11:37:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684011710 con 0x7f86941506c0 2023-10-06T11:37:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011ad0 con 0x7f86941506c0 2023-10-06T11:37:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940b05d0 con 0x7f86941506c0 2023-10-06T11:37:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941563a0 con 0x7f86941506c0 2023-10-06T11:37:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86940b0430 con 0x7f86941506c0 2023-10-06T11:37:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.244+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 60) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f868401b3b0 con 0x7f86941506c0 2023-10-06T11:37:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.248+0000 7f86897fa700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 0x7f8678062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.248+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6846+0+0 (secure 0 0 0) 0x7f868407c4a0 con 0x7f86941506c0 2023-10-06T11:37:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.248+0000 7f86990e1700 1 -- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 msgr2=0x7f8678062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:08.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.248+0000 7f86990e1700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 0x7f8678062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.252+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f86940b0430 con 0x7f86941506c0 2023-10-06T11:37:08.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.424+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86940b0430 con 0x7f86941506c0 2023-10-06T11:37:08.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.424+0000 7f86897fa700 1 -- 172.21.15.46:0/414199012 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+18042 (secure 0 0 0) 0x7f86940b0430 con 0x7f86941506c0 2023-10-06T11:37:08.440 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:08.440 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:36:27.255187+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:36:06.248657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T11:36:07.254889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2960823291":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6833/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:6832/32974":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/4259972733":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3863334752":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/3488620324":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/1900876313":"2023-10-07T11:36:27.255068+0000","172.21.15.46:0/2274789125":"2023-10-07T11:36:27.255068+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:08.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 msgr2=0x7f8678062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 0x7f8678062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 msgr2=0x7f8694155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f868400c590 tx=0x7f868400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 shutdown_connections 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8678060790 0x7f8678062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f869415a1a0 0x7f86940ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86941506c0 0x7f8694155e60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 --2- 172.21.15.46:0/414199012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86941502f0 0x7f869414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 >> 172.21.15.46:0/414199012 conn(0x7f86940cbe30 msgr2=0x7f86940c10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 shutdown_connections 2023-10-06T11:37:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.428+0000 7f869b345700 1 -- 172.21.15.46:0/414199012 wait complete. 2023-10-06T11:37:08.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:37:08.615 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.600+0000 7fe5fa1c3000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:37:08.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.608+0000 7fc05fcf0700 1 Processor -- start 2023-10-06T11:37:08.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.608+0000 7fc05fcf0700 1 -- start start 2023-10-06T11:37:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05fcf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc058156c80 0x7fc0580ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05fcf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0580c7fc0 0x7fc05815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05fcf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0580c6470 con 0x7fc0580c7b50 2023-10-06T11:37:08.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05fcf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580c7760 con 0x7fc0580c7fc0 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05fcf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580ca710 con 0x7fc058156c80 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0580c7fc0 0x7fc05815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0580c7fc0 0x7fc05815d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36190/0 (socket says 172.21.15.46:36190) 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 -- 172.21.15.46:0/1377284231 learned_addr learned my addr 172.21.15.46:0/1377284231 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05da8c700 1 --2- 172.21.15.46:0/1377284231 >> v2:172.21.15.132:3300/0 conn(0x7fc058156c80 0x7fc0580ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 -- 172.21.15.46:0/1377284231 >> v2:172.21.15.132:3300/0 conn(0x7fc058156c80 msgr2=0x7fc0580ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1377284231 >> v2:172.21.15.132:3300/0 conn(0x7fc058156c80 0x7fc0580ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 -- 172.21.15.46:0/1377284231 >> v1:172.21.15.132:6789/0 conn(0x7fc0580c7b50 legacy=0x7fc05815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:08.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.612+0000 7fc05e28d700 1 -- 172.21.15.46:0/1377284231 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580c66c0 con 0x7fc0580c7fc0 2023-10-06T11:37:08.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.616+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1377284231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc05815d4f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc05806c640 tx=0x7fc050009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c94ecf5784e7e6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05ca8a700 1 -- 172.21.15.46:0/1377284231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05000e070 con 0x7fc0580c7fc0 2023-10-06T11:37:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05ca8a700 1 -- 172.21.15.46:0/1377284231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05000ba10 con 0x7fc0580c7fc0 2023-10-06T11:37:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05ca8a700 1 -- 172.21.15.46:0/1377284231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500092e0 con 0x7fc0580c7fc0 2023-10-06T11:37:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1377284231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 msgr2=0x7fc05815d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1377284231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc05815d4f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc05806c640 tx=0x7fc050009840 comp rx=0 tx=0).stop 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1377284231 shutdown_connections 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1377284231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc05815d4f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1377284231 >> v2:172.21.15.132:3300/0 conn(0x7fc058156c80 0x7fc0580ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1377284231 >> 172.21.15.46:0/1377284231 conn(0x7fc0580cbe30 msgr2=0x7fc0580cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1377284231 shutdown_connections 2023-10-06T11:37:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.624+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1377284231 wait complete. 2023-10-06T11:37:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 Processor -- start 2023-10-06T11:37:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 -- start start 2023-10-06T11:37:08.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580c7b50 0x7fc05816d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc0580aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 0x7fc0580a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc058170b20 con 0x7fc0580c7fc0 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580ac230 con 0x7fc058156c80 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05fcf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580af260 con 0x7fc0580c7b50 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05d28b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc0580aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05d28b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc0580aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36216/0 (socket says 172.21.15.46:36216) 2023-10-06T11:37:08.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05d28b700 1 -- 172.21.15.46:0/1781673380 learned_addr learned my addr 172.21.15.46:0/1781673380 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 0x7fc0580a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.628+0000 7fc05da8c700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580c7b50 0x7fc05816d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 -- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580c7b50 msgr2=0x7fc05816d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580c7b50 0x7fc05816d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 -- 172.21.15.46:0/1781673380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 msgr2=0x7fc0580aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc0580aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 -- 172.21.15.46:0/1781673380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc050009ae0 con 0x7fc058156c80 2023-10-06T11:37:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05e28d700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 0x7fc0580a9f60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc05806bac0 tx=0x7fc050017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500092e0 con 0x7fc058156c80 2023-10-06T11:37:08.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.632+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0580ade90 con 0x7fc058156c80 2023-10-06T11:37:08.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.636+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0580ae450 con 0x7fc058156c80 2023-10-06T11:37:08.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.636+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05000be80 con 0x7fc058156c80 2023-10-06T11:37:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.636+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05001d420 con 0x7fc058156c80 2023-10-06T11:37:08.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.640+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 61) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fc05001d600 con 0x7fc058156c80 2023-10-06T11:37:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.640+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0380052b0 con 0x7fc058156c80 2023-10-06T11:37:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.640+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 7246+0+0 (secure 0 0 0) 0x7fc05007de00 con 0x7fc058156c80 2023-10-06T11:37:08.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.648+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc05004e150 con 0x7fc058156c80 2023-10-06T11:37:08.746 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c461a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c461a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c461a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c3e19700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c3e19700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c3e19700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c3e19700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c461a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.747 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c461a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.748 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:08.732+0000 7fe5c3e19700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:08.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.876+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc038005560 con 0x7fc058156c80 2023-10-06T11:37:08.893 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T11:37:08.880+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:37:08.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.892+0000 7fc04dffb700 1 -- 172.21.15.46:0/1781673380 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc050051770 con 0x7fc058156c80 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 msgr2=0x7fc0580a9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 0x7fc0580a9f60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc05806bac0 tx=0x7fc050017000 comp rx=0 tx=0).stop 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 shutdown_connections 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc058156c80 0x7fc0580a9f60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0580c7fc0 0x7fc0580aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 --2- 172.21.15.46:0/1781673380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0580c7b50 0x7fc05816d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 >> 172.21.15.46:0/1781673380 conn(0x7fc0580cbe30 msgr2=0x7fc058068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 shutdown_connections 2023-10-06T11:37:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:08.896+0000 7fc05fcf0700 1 -- 172.21.15.46:0/1781673380 wait complete. 2023-10-06T11:37:08.914 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:37:08.922 INFO:tasks.ceph.mds.a.smithi046.stderr: -6881> 2023-10-06T11:36:14.773+0000 7f5190ed2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:37:08.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:37:08.976 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:37:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 Processor -- start 2023-10-06T11:37:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 -- start start 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f0158da0 0x7fe5f015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 0x7fe5f00690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f0069610 con 0x7fe5f0158da0 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00cb850 con 0x7fe5f00d0af0 2023-10-06T11:37:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5f54f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f015d3c0 con 0x7fe5f00caeb0 2023-10-06T11:37:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f0158da0 0x7fe5f015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5f0158da0 0x7fe5f015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36360/0 (socket says 172.21.15.46:36360) 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 -- 172.21.15.46:0/115797014 learned_addr learned my addr 172.21.15.46:0/115797014 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5ef7fe700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/115797014 >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 0x7fe5f00690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 -- 172.21.15.46:0/115797014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f00caeb0 msgr2=0x7fe5f00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 -- 172.21.15.46:0/115797014 >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 msgr2=0x7fe5f00690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/115797014 >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 0x7fe5f00690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5eeffd700 1 -- 172.21.15.46:0/115797014 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00d0f40 con 0x7fe5f0158da0 2023-10-06T11:37:09.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.084+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/115797014 >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 0x7fe5f00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0158da0 0x7fe5f015d850 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00b2bd0 tx=0x7fe5e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b71c5b4874ce00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5edffb700 1 -- 172.21.15.46:0/115797014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0009e70 con 0x7fe5f0158da0 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5edffb700 1 -- 172.21.15.46:0/115797014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e000b860 con 0x7fe5f0158da0 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5edffb700 1 -- 172.21.15.46:0/115797014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000bc00 con 0x7fe5f0158da0 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 -- 172.21.15.46:0/115797014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0158da0 msgr2=0x7fe5f015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0158da0 0x7fe5f015d850 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00b2bd0 tx=0x7fe5e0009250 comp rx=0 tx=0).stop 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 -- 172.21.15.46:0/115797014 shutdown_connections 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/115797014 >> v2:172.21.15.132:3300/0 conn(0x7fe5f00d0af0 0x7fe5f00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f00caeb0 0x7fe5f00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/115797014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0158da0 0x7fe5f015d850 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 -- 172.21.15.46:0/115797014 >> 172.21.15.46:0/115797014 conn(0x7fe5f00683f0 msgr2=0x7fe5f00d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 -- 172.21.15.46:0/115797014 shutdown_connections 2023-10-06T11:37:09.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.088+0000 7fe5f54f0700 1 -- 172.21.15.46:0/115797014 wait complete. 2023-10-06T11:37:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 Processor -- start 2023-10-06T11:37:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 -- start start 2023-10-06T11:37:09.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 0x7fe5f00c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 0x7fe5f00c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f0159170 con 0x7fe5f00caeb0 2023-10-06T11:37:09.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00c9ca0 con 0x7fe5f00d0af0 2023-10-06T11:37:09.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5f54f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00c9f80 con 0x7fe5f0158da0 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36372/0 (socket says 172.21.15.46:36372) 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 -- 172.21.15.46:0/394758623 learned_addr learned my addr 172.21.15.46:0/394758623 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5ef7fe700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 0x7fe5f00c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 0x7fe5f00c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 -- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 msgr2=0x7fe5f00c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 0x7fe5f00c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 -- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 msgr2=0x7fe5f00c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:09.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 0x7fe5f00c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5eeffd700 1 -- 172.21.15.46:0/394758623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00094f0 con 0x7fe5f00caeb0 2023-10-06T11:37:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5ef7fe700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 0x7fe5f00c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.092+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 0x7fe5f00c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:09.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5eeffd700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009290 tx=0x7fe5e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000bbe0 con 0x7fe5f00caeb0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e001f070 con 0x7fe5f00caeb0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00266b0 con 0x7fe5f00caeb0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00ca260 con 0x7fe5f00caeb0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f00c9900 con 0x7fe5f00caeb0 2023-10-06T11:37:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.096+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0064590 con 0x7fe5f00caeb0 2023-10-06T11:37:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.100+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 61) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fe5e0026a60 con 0x7fe5f00caeb0 2023-10-06T11:37:09.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.100+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7304+0+0 (secure 0 0 0) 0x7fe5e0021070 con 0x7fe5f00caeb0 2023-10-06T11:37:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.104+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe5e004e940 con 0x7fe5f00caeb0 2023-10-06T11:37:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.308+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5f00880e0 con 0x7fe5f00caeb0 2023-10-06T11:37:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.632+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 62) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fe5e0026d10 con 0x7fe5f00caeb0 2023-10-06T11:37:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.636+0000 7fe5d6ffd700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 0x7fe5dc068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.636+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 0x7fe5dc068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.640+0000 7fe5ee7fc700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 0x7fe5dc068e10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006f00 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:09.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.892+0000 7fe5d6ffd700 1 -- 172.21.15.46:0/394758623 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5e0055190 con 0x7fe5f00caeb0 2023-10-06T11:37:09.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 msgr2=0x7fe5dc068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:09.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 0x7fe5dc068e10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006f00 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 msgr2=0x7fe5f014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009290 tx=0x7fe5e0017000 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 shutdown_connections 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe5dc066a30 0x7fe5dc068e10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5f0158da0 0x7fe5f00c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5f00d0af0 0x7fe5f00c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 --2- 172.21.15.46:0/394758623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f00caeb0 0x7fe5f014aa90 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 >> 172.21.15.46:0/394758623 conn(0x7fe5f00683f0 msgr2=0x7fe5f0156630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 shutdown_connections 2023-10-06T11:37:09.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:09.900+0000 7fe5f54f0700 1 -- 172.21.15.46:0/394758623 wait complete. 2023-10-06T11:37:09.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:37:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.136+0000 7f02f18ae700 1 Processor -- start 2023-10-06T11:37:10.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.136+0000 7f02f18ae700 1 -- start start 2023-10-06T11:37:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02f18ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02ec159500 0x7f02ec0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02f18ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 0x7f02ec0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02f18ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec069b30 con 0x7f02ec159130 2023-10-06T11:37:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02f18ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec0afb30 con 0x7f02ec159500 2023-10-06T11:37:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02f18ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0cc090 con 0x7f02ec069190 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02ec159500 0x7f02ec0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02eaffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f02ec159130 0x7f02ec0aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34492/0 (socket says 172.21.15.46:34492) 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02eaffd700 1 -- 172.21.15.46:0/4254143165 learned_addr learned my addr 172.21.15.46:0/4254143165 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02eb7fe700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 0x7f02ec0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 -- 172.21.15.46:0/4254143165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 msgr2=0x7f02ec0695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 0x7f02ec0695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 -- 172.21.15.46:0/4254143165 >> v1:172.21.15.46:6789/0 conn(0x7f02ec159130 legacy=0x7f02ec0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:10.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 -- 172.21.15.46:0/4254143165 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0ce2c0 con 0x7f02ec159500 2023-10-06T11:37:10.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02eb7fe700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 0x7f02ec0695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:10.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02ea7fc700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec0d0d10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098e0 tx=0x7f02e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb69ac923fdeaddf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02e9ffb700 1 -- 172.21.15.46:0/4254143165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0009e80 con 0x7f02ec159500 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.140+0000 7f02e9ffb700 1 -- 172.21.15.46:0/4254143165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e000b860 con 0x7f02ec159500 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02e9ffb700 1 -- 172.21.15.46:0/4254143165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000bbe0 con 0x7f02ec159500 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- 172.21.15.46:0/4254143165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 msgr2=0x7f02ec0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec0d0d10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098e0 tx=0x7f02e00091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- 172.21.15.46:0/4254143165 shutdown_connections 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec069190 0x7f02ec0695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 --2- 172.21.15.46:0/4254143165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec0d0d10 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- 172.21.15.46:0/4254143165 >> 172.21.15.46:0/4254143165 conn(0x7f02ec068510 msgr2=0x7f02ec0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- 172.21.15.46:0/4254143165 shutdown_connections 2023-10-06T11:37:10.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- 172.21.15.46:0/4254143165 wait complete. 2023-10-06T11:37:10.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 Processor -- start 2023-10-06T11:37:10.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.144+0000 7f02f18ae700 1 -- start start 2023-10-06T11:37:10.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02ec069190 0x7f02ec0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec159130 0x7f02ec153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0c8640 con 0x7f02ec159500 2023-10-06T11:37:10.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0c91f0 con 0x7f02ec069190 2023-10-06T11:37:10.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec0c7da0 con 0x7f02ec159130 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36380/0 (socket says 172.21.15.46:36380) 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 -- 172.21.15.46:0/2814865900 learned_addr learned my addr 172.21.15.46:0/2814865900 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02ea7fc700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec159130 0x7f02ec153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eaffd700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02ec069190 0x7f02ec0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 -- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec159130 msgr2=0x7f02ec153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec159130 0x7f02ec153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 -- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02ec069190 msgr2=0x7f02ec0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02ec069190 0x7f02ec0c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 -- 172.21.15.46:0/2814865900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0009420 con 0x7f02ec159500 2023-10-06T11:37:10.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02eb7fe700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f02e400d180 tx=0x7f02e4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:10.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400dcc0 con 0x7f02ec159500 2023-10-06T11:37:10.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400f040 con 0x7f02ec159500 2023-10-06T11:37:10.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec0c8080 con 0x7f02ec159500 2023-10-06T11:37:10.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4015690 con 0x7f02ec159500 2023-10-06T11:37:10.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.148+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec0c7a90 con 0x7f02ec159500 2023-10-06T11:37:10.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.152+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec0c8fc0 con 0x7f02ec159500 2023-10-06T11:37:10.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.152+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 62) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f02e401e3b0 con 0x7f02ec159500 2023-10-06T11:37:10.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.152+0000 7f02d2ffd700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 0x7f02d8062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.152+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7304+0+0 (secure 0 0 0) 0x7f02e407d080 con 0x7f02ec159500 2023-10-06T11:37:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.156+0000 7f02eaffd700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 0x7f02d8062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:10.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.156+0000 7f02eaffd700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 0x7f02d8062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f02dc007130 tx=0x7f02dc007eb0 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.160+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f02e40acbe0 con 0x7f02ec159500 2023-10-06T11:37:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.328+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02ec0c9330 con 0x7f02ec159500 2023-10-06T11:37:10.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.904+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f02e404d5e0 con 0x7f02ec159500 2023-10-06T11:37:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.912+0000 7f02d2ffd700 1 -- 172.21.15.46:0/2814865900 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7f02e40560c0 con 0x7f02ec159500 2023-10-06T11:37:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 msgr2=0x7f02d8062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 0x7f02d8062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f02dc007130 tx=0x7f02dc007eb0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 msgr2=0x7f02ec15ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f02e400d180 tx=0x7f02e4009a50 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 shutdown_connections 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02d80603a0 0x7f02d8062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02ec159500 0x7f02ec15ae10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02ec159130 0x7f02ec153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 --2- 172.21.15.46:0/2814865900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02ec069190 0x7f02ec0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 >> 172.21.15.46:0/2814865900 conn(0x7f02ec068510 msgr2=0x7f02ec154ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 shutdown_connections 2023-10-06T11:37:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:10.916+0000 7f02f18ae700 1 -- 172.21.15.46:0/2814865900 wait complete. 2023-10-06T11:37:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:37:10.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:37:11.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.108+0000 7f5d4fd45700 1 Processor -- start 2023-10-06T11:37:11.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.108+0000 7f5d4fd45700 1 -- start start 2023-10-06T11:37:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4fd45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159130 0x7f5d480d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4fd45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d48069190 0x7f5d48069610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4fd45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480ce2c0 con 0x7f5d48159500 2023-10-06T11:37:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4fd45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480afb30 con 0x7f5d48069190 2023-10-06T11:37:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4fd45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cc090 con 0x7f5d48159130 2023-10-06T11:37:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4d2e0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5d48159500 0x7f5d480aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34500/0 (socket says 172.21.15.46:34500) 2023-10-06T11:37:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4d2e0700 1 -- 172.21.15.46:0/450803682 learned_addr learned my addr 172.21.15.46:0/450803682 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.112+0000 7f5d4e2e2700 1 --2- 172.21.15.46:0/450803682 >> v2:172.21.15.46:3300/0 conn(0x7f5d48069190 0x7f5d48069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4dae1700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159130 0x7f5d480d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4cadf700 1 -- 172.21.15.46:0/450803682 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524522319 0 0) 0x7f5d480ce2c0 con 0x7f5d48159500 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4cadf700 1 -- 172.21.15.46:0/450803682 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d34003570 con 0x7f5d48159500 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4e2e2700 1 -- 172.21.15.46:0/450803682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159130 msgr2=0x7f5d480d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4e2e2700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159130 0x7f5d480d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4e2e2700 1 -- 172.21.15.46:0/450803682 >> v1:172.21.15.46:6789/0 conn(0x7f5d48159500 legacy=0x7f5d480aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4e2e2700 1 -- 172.21.15.46:0/450803682 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480af570 con 0x7f5d48069190 2023-10-06T11:37:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4e2e2700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48069190 0x7f5d48069610 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009e60 tx=0x7f5d40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa7b348853b5d3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4cadf700 1 -- 172.21.15.46:0/450803682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000e070 con 0x7f5d48069190 2023-10-06T11:37:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4cadf700 1 -- 172.21.15.46:0/450803682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4000bab0 con 0x7f5d48069190 2023-10-06T11:37:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4cadf700 1 -- 172.21.15.46:0/450803682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000be70 con 0x7f5d48069190 2023-10-06T11:37:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 -- 172.21.15.46:0/450803682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48069190 msgr2=0x7f5d48069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48069190 0x7f5d48069610 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009e60 tx=0x7f5d40009780 comp rx=0 tx=0).stop 2023-10-06T11:37:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 -- 172.21.15.46:0/450803682 shutdown_connections 2023-10-06T11:37:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48069190 0x7f5d48069610 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/450803682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159130 0x7f5d480d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 -- 172.21.15.46:0/450803682 >> 172.21.15.46:0/450803682 conn(0x7f5d48068510 msgr2=0x7f5d480d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 -- 172.21.15.46:0/450803682 shutdown_connections 2023-10-06T11:37:11.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.116+0000 7f5d4fd45700 1 -- 172.21.15.46:0/450803682 wait complete. 2023-10-06T11:37:11.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.120+0000 7f5d4fd45700 1 Processor -- start 2023-10-06T11:37:11.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.120+0000 7f5d4fd45700 1 -- start start 2023-10-06T11:37:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d48069190 0x7f5d48153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159500 0x7f5d480ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c7980 con 0x7f5d48159130 2023-10-06T11:37:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c5c00 con 0x7f5d48069190 2023-10-06T11:37:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c5ee0 con 0x7f5d48159500 2023-10-06T11:37:11.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36386/0 (socket says 172.21.15.46:36386) 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 -- 172.21.15.46:0/1311120053 learned_addr learned my addr 172.21.15.46:0/1311120053 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4e2e2700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159500 0x7f5d480ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4dae1700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d48069190 0x7f5d48153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 -- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159500 msgr2=0x7f5d480ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159500 0x7f5d480ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 -- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d48069190 msgr2=0x7f5d48153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d48069190 0x7f5d48153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 -- 172.21.15.46:0/1311120053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40009a20 con 0x7f5d48159130 2023-10-06T11:37:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4d2e0700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c590 tx=0x7f5d38007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005b30 con 0x7f5d48159130 2023-10-06T11:37:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38005cb0 con 0x7f5d48159130 2023-10-06T11:37:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480c7330 con 0x7f5d48159130 2023-10-06T11:37:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007320 con 0x7f5d48159130 2023-10-06T11:37:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.124+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d48154410 con 0x7f5d48159130 2023-10-06T11:37:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.128+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d480c7af0 con 0x7f5d48159130 2023-10-06T11:37:11.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.128+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5d3801b3b0 con 0x7f5d48159130 2023-10-06T11:37:11.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.128+0000 7f5d3dffb700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 0x7f5d2c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.132+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6866+0+0 (secure 0 0 0) 0x7f5d3807c210 con 0x7f5d48159130 2023-10-06T11:37:11.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.132+0000 7f5d4dae1700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 0x7f5d2c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:11.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.132+0000 7f5d4dae1700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 0x7f5d2c062b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009db0 tx=0x7f5d440091a0 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:11.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.136+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d3804c1c0 con 0x7f5d48159130 2023-10-06T11:37:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.308+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d4804b5c0 con 0x7f5d48159130 2023-10-06T11:37:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.916+0000 7f5d3dffb700 1 -- 172.21.15.46:0/1311120053 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d3804f7e0 con 0x7f5d48159130 2023-10-06T11:37:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 msgr2=0x7f5d2c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 0x7f5d2c062b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009db0 tx=0x7f5d440091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 msgr2=0x7f5d480c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800c590 tx=0x7f5d38007e00 comp rx=0 tx=0).stop 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 shutdown_connections 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d2c0606a0 0x7f5d2c062b50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d48159500 0x7f5d480ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d48159130 0x7f5d480c7e10 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 --2- 172.21.15.46:0/1311120053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d48069190 0x7f5d48153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 >> 172.21.15.46:0/1311120053 conn(0x7f5d48068510 msgr2=0x7f5d48156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 shutdown_connections 2023-10-06T11:37:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:11.924+0000 7f5d4fd45700 1 -- 172.21.15.46:0/1311120053 wait complete. 2023-10-06T11:37:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T11:37:11.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:37:12.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.108+0000 7f11a3aae700 1 Processor -- start 2023-10-06T11:37:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 -- start start 2023-10-06T11:37:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f119c0c6470 0x7f119c0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 0x7f119c0c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0c1bf0 con 0x7f119c06e8a0 2023-10-06T11:37:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0abbe0 con 0x7f119c06ed10 2023-10-06T11:37:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a3aae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c06cf80 con 0x7f119c0c6470 2023-10-06T11:37:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a1049700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f119c06e8a0 0x7f119c0c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34504/0 (socket says 172.21.15.46:34504) 2023-10-06T11:37:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a1049700 1 -- 172.21.15.46:0/1553131288 learned_addr learned my addr 172.21.15.46:0/1553131288 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a204b700 1 --2- 172.21.15.46:0/1553131288 >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 0x7f119c0c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.112+0000 7f11a184a700 1 --2- 172.21.15.46:0/1553131288 >> v2:172.21.15.132:3300/0 conn(0x7f119c0c6470 0x7f119c0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a184a700 1 -- 172.21.15.46:0/1553131288 >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 msgr2=0x7f119c0c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a184a700 1 --2- 172.21.15.46:0/1553131288 >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 0x7f119c0c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a184a700 1 -- 172.21.15.46:0/1553131288 >> v1:172.21.15.46:6789/0 conn(0x7f119c06e8a0 legacy=0x7f119c0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a0848700 1 -- 172.21.15.46:0/1553131288 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254455043 0 0) 0x7f119c0c1bf0 con 0x7f119c06e8a0 2023-10-06T11:37:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a184a700 1 -- 172.21.15.46:0/1553131288 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c06d260 con 0x7f119c0c6470 2023-10-06T11:37:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a204b700 1 --2- 172.21.15.46:0/1553131288 >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 0x7f119c0c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:12.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a184a700 1 --2- 172.21.15.46:0/1553131288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0ab6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c2e90 tx=0x7f11980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12f22762e42760b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a0848700 1 -- 172.21.15.46:0/1553131288 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011630 con 0x7f119c0c6470 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a0848700 1 -- 172.21.15.46:0/1553131288 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198011790 con 0x7f119c0c6470 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a0848700 1 -- 172.21.15.46:0/1553131288 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011b30 con 0x7f119c0c6470 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 -- 172.21.15.46:0/1553131288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 msgr2=0x7f119c0ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 --2- 172.21.15.46:0/1553131288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0ab6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f119c0c2e90 tx=0x7f11980091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 -- 172.21.15.46:0/1553131288 shutdown_connections 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 --2- 172.21.15.46:0/1553131288 >> v2:172.21.15.46:3300/0 conn(0x7f119c06ed10 0x7f119c0c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 --2- 172.21.15.46:0/1553131288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0ab6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 -- 172.21.15.46:0/1553131288 >> 172.21.15.46:0/1553131288 conn(0x7f119c0cbe30 msgr2=0x7f119c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 -- 172.21.15.46:0/1553131288 shutdown_connections 2023-10-06T11:37:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.116+0000 7f11a3aae700 1 -- 172.21.15.46:0/1553131288 wait complete. 2023-10-06T11:37:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 Processor -- start 2023-10-06T11:37:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 -- start start 2023-10-06T11:37:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f119c06ed10 0x7f119c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c3c90 con 0x7f119c06e8a0 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c4120 con 0x7f119c0c6470 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a3aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c3080 con 0x7f119c06ed10 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36410/0 (socket says 172.21.15.46:36410) 2023-10-06T11:37:12.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 -- 172.21.15.46:0/3654915024 learned_addr learned my addr 172.21.15.46:0/3654915024 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a204b700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a1049700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f119c06ed10 0x7f119c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 -- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f119c06ed10 msgr2=0x7f119c0c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f119c06ed10 0x7f119c0c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 -- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 msgr2=0x7f119c0aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 -- 172.21.15.46:0/3654915024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194009a20 con 0x7f119c06e8a0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.120+0000 7f11a184a700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f119c06d0c0 tx=0x7f1198017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198011b10 con 0x7f119c06e8a0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11980253f0 con 0x7f119c06e8a0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1198009420 con 0x7f119c06e8a0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198025790 con 0x7f119c06e8a0 2023-10-06T11:37:12.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0c3640 con 0x7f119c06e8a0 2023-10-06T11:37:12.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c0660f0 con 0x7f119c06e8a0 2023-10-06T11:37:12.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.124+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1198025b80 con 0x7f119c06e8a0 2023-10-06T11:37:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.128+0000 7f1191ffb700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 0x7f1180062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.128+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f119807e530 con 0x7f119c06e8a0 2023-10-06T11:37:12.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.128+0000 7f11a1049700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 0x7f1180062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.128+0000 7f11a1049700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 0x7f1180062b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f119c13f680 tx=0x7f118c009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.136+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f119804e720 con 0x7f119c06e8a0 2023-10-06T11:37:12.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.304+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f119c0c0e80 con 0x7f119c06e8a0 2023-10-06T11:37:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.316+0000 7f1191ffb700 1 -- 172.21.15.46:0/3654915024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1198051d40 con 0x7f119c06e8a0 2023-10-06T11:37:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 msgr2=0x7f1180062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 0x7f1180062b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f119c13f680 tx=0x7f118c009040 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 msgr2=0x7f119c13c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f119c06d0c0 tx=0x7f1198017000 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 shutdown_connections 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f11800606a0 0x7f1180062b50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f119c0c6470 0x7f119c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f119c06ed10 0x7f119c0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 --2- 172.21.15.46:0/3654915024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f119c06e8a0 0x7f119c13c8e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 >> 172.21.15.46:0/3654915024 conn(0x7f119c0cbe30 msgr2=0x7f119c15c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 shutdown_connections 2023-10-06T11:37:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.320+0000 7f11a3aae700 1 -- 172.21.15.46:0/3654915024 wait complete. 2023-10-06T11:37:12.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T11:37:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.508+0000 7f333b66e700 1 Processor -- start 2023-10-06T11:37:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.512+0000 7f333b66e700 1 -- start start 2023-10-06T11:37:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.512+0000 7f333b66e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340cebe0 0x7f3334164210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.512+0000 7f333b66e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340cb4a0 con 0x7f33340b23f0 2023-10-06T11:37:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.512+0000 7f333b66e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340cb6e0 con 0x7f33340ccc00 2023-10-06T11:37:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.512+0000 7f333b66e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340cc780 con 0x7f33340cebe0 2023-10-06T11:37:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f333940a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f33340b23f0 0x7f33340cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34508/0 (socket says 172.21.15.46:34508) 2023-10-06T11:37:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f333940a700 1 -- 172.21.15.46:0/60424966 learned_addr learned my addr 172.21.15.46:0/60424966 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340cebe0 0x7f3334164210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583410713 0 0) 0x7f3330001610 con 0x7f33340b23f0 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003570 con 0x7f33340b23f0 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383665260 0 0) 0x7f33340cb6e0 con 0x7f33340ccc00 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3330001610 con 0x7f33340ccc00 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010856180 0 0) 0x7f3320003570 con 0x7f33340b23f0 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340cebe0 msgr2=0x7f3334164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340cebe0 0x7f3334164210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 >> v1:172.21.15.132:6789/0 conn(0x7f33340ccc00 legacy=0x7f33340ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340cb9c0 con 0x7f33340b23f0 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f3330001b20 con 0x7f33340b23f0 2023-10-06T11:37:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 >> v1:172.21.15.46:6789/0 conn(0x7f33340b23f0 legacy=0x7f33340cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004c80 0x7f3320005070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200037b0 0x7f3320003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f332000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332000c080 con 0x7f3320004160 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332000c1f0 con 0x7f3320004c80 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332000c3b0 con 0x7f33200037b0 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340cebe0 0x7f3334164210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f332000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f333940a700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004c80 0x7f3320005070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3338c09700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200037b0 0x7f3320003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 -- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200037b0 msgr2=0x7f3320003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33200037b0 0x7f3320003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 -- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004c80 msgr2=0x7f3320005070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004c80 0x7f3320005070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.516+0000 7f3339c0b700 1 -- 172.21.15.46:0/60424966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320003570 con 0x7f3320004160 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f3339c0b700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f332000ba60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f33340d01d0 tx=0x7f332c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b843cf2bf6b6c20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00e6e0 con 0x7f3320004160 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c00e840 con 0x7f3320004160 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f332bfff700 1 -- 172.21.15.46:0/60424966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00ebe0 con 0x7f3320004160 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 -- 172.21.15.46:0/60424966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 msgr2=0x7f332000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 --2- 172.21.15.46:0/60424966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f332000ba60 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f33340d01d0 tx=0x7f332c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f3339c0b700 1 -- 172.21.15.46:0/60424966 reap_dead start 2023-10-06T11:37:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 -- 172.21.15.46:0/60424966 shutdown_connections 2023-10-06T11:37:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 -- 172.21.15.46:0/60424966 >> 172.21.15.46:0/60424966 conn(0x7f3334099440 msgr2=0x7f333406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 -- 172.21.15.46:0/60424966 shutdown_connections 2023-10-06T11:37:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.520+0000 7f333b66e700 1 -- 172.21.15.46:0/60424966 wait complete. 2023-10-06T11:37:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 Processor -- start 2023-10-06T11:37:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 -- start start 2023-10-06T11:37:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340c9fc0 0x7f33340c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 0x7f3334067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334159920 con 0x7f3320004160 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334159b90 con 0x7f333414bb80 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333b66e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3334169b80 con 0x7f33340c9fc0 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333940a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333940a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36426/0 (socket says 172.21.15.46:36426) 2023-10-06T11:37:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f333940a700 1 -- 172.21.15.46:0/2844512652 learned_addr learned my addr 172.21.15.46:0/2844512652 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.524+0000 7f3338c09700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340c9fc0 0x7f33340c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f3339c0b700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 0x7f3334067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 -- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340c9fc0 msgr2=0x7f33340c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340c9fc0 0x7f33340c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 -- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 msgr2=0x7f3334067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 0x7f3334067b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 -- 172.21.15.46:0/2844512652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340c8b10 con 0x7f3320004160 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f3339c0b700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 0x7f3334067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333940a700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ab0 tx=0x7f3330007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330001040 con 0x7f3320004160 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333000fc60 con 0x7f3320004160 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340c8df0 con 0x7f3320004160 2023-10-06T11:37:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330014470 con 0x7f3320004160 2023-10-06T11:37:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340c87c0 con 0x7f3320004160 2023-10-06T11:37:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.528+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340660f0 con 0x7f3320004160 2023-10-06T11:37:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.532+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3330014820 con 0x7f3320004160 2023-10-06T11:37:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.532+0000 7f33297fa700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 0x7f3318062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.532+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f333007d3f0 con 0x7f3320004160 2023-10-06T11:37:12.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.532+0000 7f3338c09700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 0x7f3318062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.536+0000 7f3338c09700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 0x7f3318062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33340687d0 tx=0x7f3324007d30 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.540+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f333004d540 con 0x7f3320004160 2023-10-06T11:37:12.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.712+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f33340cb4a0 con 0x7f3320004160 2023-10-06T11:37:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.724+0000 7f33297fa700 1 -- 172.21.15.46:0/2844512652 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7f33340cb4a0 con 0x7f3320004160 2023-10-06T11:37:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 msgr2=0x7f3318062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 0x7f3318062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33340687d0 tx=0x7f3324007d30 comp rx=0 tx=0).stop 2023-10-06T11:37:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 msgr2=0x7f3334169640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ab0 tx=0x7f3330007dd0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 shutdown_connections 2023-10-06T11:37:12.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3318060670 0x7f3318062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f333414bb80 0x7f3334067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33340c9fc0 0x7f33340c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 --2- 172.21.15.46:0/2844512652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3320004160 0x7f3334169640 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 >> 172.21.15.46:0/2844512652 conn(0x7f3334099440 msgr2=0x7f3334068ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 shutdown_connections 2023-10-06T11:37:12.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.728+0000 7f333b66e700 1 -- 172.21.15.46:0/2844512652 wait complete. 2023-10-06T11:37:12.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T11:37:12.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.916+0000 7f2498c16700 1 Processor -- start 2023-10-06T11:37:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.916+0000 7f2498c16700 1 -- start start 2023-10-06T11:37:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f24940abfc0 0x7f24940ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24940ae890 0x7f24940ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f249406f1d0 0x7f2494096140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940ada20 con 0x7f24940ae890 2023-10-06T11:37:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940aca00 con 0x7f24940abfc0 2023-10-06T11:37:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2498c16700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24940633b0 con 0x7f249406f1d0 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24940ae890 0x7f24940ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24940ae890 0x7f24940ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36436/0 (socket says 172.21.15.46:36436) 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2493fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f249406f1d0 0x7f2494096140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 -- 172.21.15.46:0/1679764986 learned_addr learned my addr 172.21.15.46:0/1679764986 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f24937fe700 1 --2- 172.21.15.46:0/1679764986 >> v2:172.21.15.132:3300/0 conn(0x7f24940abfc0 0x7f24940ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 -- 172.21.15.46:0/1679764986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f249406f1d0 msgr2=0x7f2494096140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 --2- 172.21.15.46:0/1679764986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f249406f1d0 0x7f2494096140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 -- 172.21.15.46:0/1679764986 >> v2:172.21.15.132:3300/0 conn(0x7f24940abfc0 msgr2=0x7f24940ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 --2- 172.21.15.46:0/1679764986 >> v2:172.21.15.132:3300/0 conn(0x7f24940abfc0 0x7f24940ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.920+0000 7f2492ffd700 1 -- 172.21.15.46:0/1679764986 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494062df0 con 0x7f24940ae890 2023-10-06T11:37:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2492ffd700 1 --2- 172.21.15.46:0/1679764986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940ae890 0x7f24940ace90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f249406c1b0 tx=0x7f2488009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a26bbb84143c470 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f24927fc700 1 -- 172.21.15.46:0/1679764986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800b6d0 con 0x7f24940ae890 2023-10-06T11:37:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f24927fc700 1 -- 172.21.15.46:0/1679764986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800b830 con 0x7f24940ae890 2023-10-06T11:37:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f24927fc700 1 -- 172.21.15.46:0/1679764986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800bbf0 con 0x7f24940ae890 2023-10-06T11:37:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 -- 172.21.15.46:0/1679764986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940ae890 msgr2=0x7f24940ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 --2- 172.21.15.46:0/1679764986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940ae890 0x7f24940ace90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f249406c1b0 tx=0x7f2488009250 comp rx=0 tx=0).stop 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 -- 172.21.15.46:0/1679764986 shutdown_connections 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 --2- 172.21.15.46:0/1679764986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f249406f1d0 0x7f2494096140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 --2- 172.21.15.46:0/1679764986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940ae890 0x7f24940ace90 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 --2- 172.21.15.46:0/1679764986 >> v2:172.21.15.132:3300/0 conn(0x7f24940abfc0 0x7f24940ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 -- 172.21.15.46:0/1679764986 >> 172.21.15.46:0/1679764986 conn(0x7f24940afd10 msgr2=0x7f249406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 -- 172.21.15.46:0/1679764986 shutdown_connections 2023-10-06T11:37:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.924+0000 7f2498c16700 1 -- 172.21.15.46:0/1679764986 wait complete. 2023-10-06T11:37:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 Processor -- start 2023-10-06T11:37:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- start start 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 0x7f2494155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24940ae890 0x7f249415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249416cc90 con 0x7f24940abfc0 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494169b90 con 0x7f249406f1d0 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2494169e70 con 0x7f24940ae890 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36442/0 (socket says 172.21.15.46:36442) 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 -- 172.21.15.46:0/2206274520 learned_addr learned my addr 172.21.15.46:0/2206274520 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f24937fe700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 0x7f2494155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2493fff700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24940ae890 0x7f249415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 -- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24940ae890 msgr2=0x7f249415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24940ae890 0x7f249415d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 -- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 msgr2=0x7f2494155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 0x7f2494155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 -- 172.21.15.46:0/2206274520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880094f0 con 0x7f24940abfc0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f24937fe700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 0x7f2494155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2492ffd700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2494070bf0 tx=0x7f2488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800bbb0 con 0x7f24940abfc0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248801a450 con 0x7f24940abfc0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248801a7f0 con 0x7f24940abfc0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249416a150 con 0x7f24940abfc0 2023-10-06T11:37:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.928+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2494156450 con 0x7f24940abfc0 2023-10-06T11:37:12.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.932+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2494064590 con 0x7f24940abfc0 2023-10-06T11:37:12.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.932+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f248801aba0 con 0x7f24940abfc0 2023-10-06T11:37:12.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.932+0000 7f247b7fe700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 0x7f2480062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.932+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f248807e690 con 0x7f24940abfc0 2023-10-06T11:37:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.936+0000 7f24937fe700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 0x7f2480062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:12.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.936+0000 7f24937fe700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 0x7f2480062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d80 tx=0x7f2484008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:12.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:12.940+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f248804e7e0 con 0x7f24940abfc0 2023-10-06T11:37:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.112+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f249404b5c0 con 0x7f24940abfc0 2023-10-06T11:37:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.116+0000 7f247b7fe700 1 -- 172.21.15.46:0/2206274520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2488051e00 con 0x7f24940abfc0 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 msgr2=0x7f2480062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 0x7f2480062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d80 tx=0x7f2484008040 comp rx=0 tx=0).stop 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 msgr2=0x7f24940c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2494070bf0 tx=0x7f2488017000 comp rx=0 tx=0).stop 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 shutdown_connections 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2480060670 0x7f2480062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24940ae890 0x7f249415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24940abfc0 0x7f24940c1ee0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 --2- 172.21.15.46:0/2206274520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f249406f1d0 0x7f2494155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.120+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 >> 172.21.15.46:0/2206274520 conn(0x7f24940afd10 msgr2=0x7f24941570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.124+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 shutdown_connections 2023-10-06T11:37:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.124+0000 7f2498c16700 1 -- 172.21.15.46:0/2206274520 wait complete. 2023-10-06T11:37:13.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-06T11:37:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 Processor -- start 2023-10-06T11:37:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 -- start start 2023-10-06T11:37:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 0x7f1c5c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 0x7f1c5c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c06d460 con 0x7f1c5c15af30 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0c7000 con 0x7f1c5c15ab60 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.308+0000 7f1c644f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0c72c0 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c61a8b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c5c15af30 0x7f1c5c06cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45336/0 (socket says 172.21.15.46:45336) 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c61a8b700 1 -- 172.21.15.46:0/1226309866 learned_addr learned my addr 172.21.15.46:0/1226309866 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 --2- 172.21.15.46:0/1226309866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 0x7f1c5c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1226309866 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 0x7f1c5c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6128a700 1 -- 172.21.15.46:0/1226309866 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532470414 0 0) 0x7f1c5c06d460 con 0x7f1c5c15af30 2023-10-06T11:37:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6128a700 1 -- 172.21.15.46:0/1226309866 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c48003570 con 0x7f1c5c15af30 2023-10-06T11:37:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 -- 172.21.15.46:0/1226309866 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 msgr2=0x7f1c5c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 --2- 172.21.15.46:0/1226309866 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 0x7f1c5c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 -- 172.21.15.46:0/1226309866 >> v1:172.21.15.46:6789/0 conn(0x7f1c5c15af30 legacy=0x7f1c5c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 -- 172.21.15.46:0/1226309866 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0c75a0 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1226309866 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 0x7f1c5c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c62a8d700 1 --2- 172.21.15.46:0/1226309866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 0x7f1c5c0adaf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009e60 tx=0x7f1c54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4787c99d2da39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6128a700 1 -- 172.21.15.46:0/1226309866 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400e070 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6128a700 1 -- 172.21.15.46:0/1226309866 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400bab0 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c6128a700 1 -- 172.21.15.46:0/1226309866 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400be70 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 -- 172.21.15.46:0/1226309866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 msgr2=0x7f1c5c0adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1226309866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 0x7f1c5c0adaf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009e60 tx=0x7f1c54009780 comp rx=0 tx=0).stop 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 -- 172.21.15.46:0/1226309866 shutdown_connections 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1226309866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c0ad6c0 0x7f1c5c0adaf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1226309866 >> v2:172.21.15.132:3300/0 conn(0x7f1c5c15ab60 0x7f1c5c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 -- 172.21.15.46:0/1226309866 >> 172.21.15.46:0/1226309866 conn(0x7f1c5c0cbe30 msgr2=0x7f1c5c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 -- 172.21.15.46:0/1226309866 shutdown_connections 2023-10-06T11:37:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.312+0000 7f1c644f0700 1 -- 172.21.15.46:0/1226309866 wait complete. 2023-10-06T11:37:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 Processor -- start 2023-10-06T11:37:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 -- start start 2023-10-06T11:37:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 0x7f1c5c14da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c15af30 0x7f1c5c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0aa310 con 0x7f1c5c15ab60 2023-10-06T11:37:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0aa7a0 con 0x7f1c5c0ad6c0 2023-10-06T11:37:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c644f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0a99e0 con 0x7f1c5c15af30 2023-10-06T11:37:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c61a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c61a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46556/0 (socket says 172.21.15.46:46556) 2023-10-06T11:37:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c61a8b700 1 -- 172.21.15.46:0/1308741230 learned_addr learned my addr 172.21.15.46:0/1308741230 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c62a8d700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c15af30 0x7f1c5c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.316+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 0x7f1c5c14da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 -- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c15af30 msgr2=0x7f1c5c14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c15af30 0x7f1c5c14fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 -- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 msgr2=0x7f1c5c14da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 0x7f1c5c14da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 -- 172.21.15.46:0/1308741230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580093c0 con 0x7f1c5c15ab60 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 0x7f1c5c14da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c61a8b700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c7c0 tx=0x7f1c4c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c005b30 con 0x7f1c5c15ab60 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c005c90 con 0x7f1c5c15ab60 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c007340 con 0x7f1c5c15ab60 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c54009a20 con 0x7f1c5c15ab60 2023-10-06T11:37:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0a9fa0 con 0x7f1c5c15ab60 2023-10-06T11:37:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.320+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c063f90 con 0x7f1c5c15ab60 2023-10-06T11:37:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.324+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1c4c01b400 con 0x7f1c5c15ab60 2023-10-06T11:37:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.324+0000 7f1c527fc700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 0x7f1c40062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.324+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f1c4c07c790 con 0x7f1c5c15ab60 2023-10-06T11:37:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.324+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 0x7f1c40062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.324+0000 7f1c6228c700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 0x7f1c40062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009390 tx=0x7f1c58008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.328+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c4c04c960 con 0x7f1c5c15ab60 2023-10-06T11:37:13.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.568+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f1c5c04b5c0 con 0x7f1c5c15ab60 2023-10-06T11:37:13.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.924+0000 7f1c527fc700 1 -- 172.21.15.46:0/1308741230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v718) v1 ==== 148+0+0 (secure 0 0 0) 0x7f1c4c04ff80 con 0x7f1c5c15ab60 2023-10-06T11:37:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 msgr2=0x7f1c40062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 0x7f1c40062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009390 tx=0x7f1c58008040 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 msgr2=0x7f1c5c14f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c7c0 tx=0x7f1c4c007e00 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 shutdown_connections 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1c40060670 0x7f1c40062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c5c15af30 0x7f1c5c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15ab60 0x7f1c5c14f880 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 --2- 172.21.15.46:0/1308741230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c5c0ad6c0 0x7f1c5c14da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 >> 172.21.15.46:0/1308741230 conn(0x7f1c5c0cbe30 msgr2=0x7f1c5c06a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 shutdown_connections 2023-10-06T11:37:13.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:13.928+0000 7f1c644f0700 1 -- 172.21.15.46:0/1308741230 wait complete. 2023-10-06T11:37:13.967 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-06T11:37:13.968 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-06T11:37:13.968 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T11:37:13.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-06T11:37:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.120+0000 7fd02ac78700 1 Processor -- start 2023-10-06T11:37:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.120+0000 7fd02ac78700 1 -- start start 2023-10-06T11:37:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd02ac78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 0x7fd0240af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd02ac78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd02415a7c0 0x7fd0240af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd02ac78700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240b0680 con 0x7fd0240cdf30 2023-10-06T11:37:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd02ac78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0240afde0 con 0x7fd02415a7c0 2023-10-06T11:37:14.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd02ac78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240b00c0 con 0x7fd02415a3f0 2023-10-06T11:37:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd029215700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd0240cdf30 0x7fd0240ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45340/0 (socket says 172.21.15.46:45340) 2023-10-06T11:37:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd029215700 1 -- 172.21.15.46:0/3401973178 learned_addr learned my addr 172.21.15.46:0/3401973178 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 --2- 172.21.15.46:0/3401973178 >> v2:172.21.15.132:3300/0 conn(0x7fd02415a7c0 0x7fd0240af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd028a14700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 0x7fd0240af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01b7fe700 1 -- 172.21.15.46:0/3401973178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707622559 0 0) 0x7fd0240b0680 con 0x7fd0240cdf30 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01b7fe700 1 -- 172.21.15.46:0/3401973178 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c003570 con 0x7fd0240cdf30 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 -- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 msgr2=0x7fd0240af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 0x7fd0240af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 -- 172.21.15.46:0/3401973178 >> v1:172.21.15.46:6789/0 conn(0x7fd0240cdf30 legacy=0x7fd0240ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 -- 172.21.15.46:0/3401973178 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240b03a0 con 0x7fd02415a7c0 2023-10-06T11:37:14.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd028a14700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 0x7fd0240af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:14.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01bfff700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd02415a7c0 0x7fd0240af7d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd0140098e0 tx=0x7fd0140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad56811a5f14a027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:14.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.124+0000 7fd01b7fe700 1 -- 172.21.15.46:0/3401973178 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014011630 con 0x7fd02415a7c0 2023-10-06T11:37:14.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd01b7fe700 1 -- 172.21.15.46:0/3401973178 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014011790 con 0x7fd02415a7c0 2023-10-06T11:37:14.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd01b7fe700 1 -- 172.21.15.46:0/3401973178 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014011b30 con 0x7fd02415a7c0 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd02415a7c0 msgr2=0x7fd0240af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd02415a7c0 0x7fd0240af7d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd0140098e0 tx=0x7fd0140091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- 172.21.15.46:0/3401973178 shutdown_connections 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd02415a7c0 0x7fd0240af7d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 --2- 172.21.15.46:0/3401973178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a3f0 0x7fd0240af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- 172.21.15.46:0/3401973178 >> 172.21.15.46:0/3401973178 conn(0x7fd024062d50 msgr2=0x7fd024063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- 172.21.15.46:0/3401973178 shutdown_connections 2023-10-06T11:37:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- 172.21.15.46:0/3401973178 wait complete. 2023-10-06T11:37:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 Processor -- start 2023-10-06T11:37:14.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.128+0000 7fd02ac78700 1 -- start start 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240cdf30 0x7fd02414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a7c0 0x7fd024153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c6310 con 0x7fd02415a3f0 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c65f0 con 0x7fd0240cdf30 2023-10-06T11:37:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c68d0 con 0x7fd02415a7c0 2023-10-06T11:37:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46560/0 (socket says 172.21.15.46:46560) 2023-10-06T11:37:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 -- 172.21.15.46:0/1991729445 learned_addr learned my addr 172.21.15.46:0/1991729445 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd028a14700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240cdf30 0x7fd02414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd029215700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a7c0 0x7fd024153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 -- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a7c0 msgr2=0x7fd024153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a7c0 0x7fd024153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 -- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240cdf30 msgr2=0x7fd02414e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240cdf30 0x7fd02414e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 -- 172.21.15.46:0/1991729445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020009420 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd01bfff700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd024063940 tx=0x7fd014017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014011b10 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014000cc0 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd014009420 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014023600 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.132+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02414ecc0 con 0x7fd02415a3f0 2023-10-06T11:37:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.136+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0240c6dd0 con 0x7fd02415a3f0 2023-10-06T11:37:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.136+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd01402c3b0 con 0x7fd02415a3f0 2023-10-06T11:37:14.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.136+0000 7fd018ff9700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 0x7fd010062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:14.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.136+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7fd01407df90 con 0x7fd02415a3f0 2023-10-06T11:37:14.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.136+0000 7fd028a14700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 0x7fd010062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:14.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.140+0000 7fd028a14700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 0x7fd010062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0240c4d80 tx=0x7fd0200091a0 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.144+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd0140adbe0 con 0x7fd02415a3f0 2023-10-06T11:37:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.372+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fd0240c6450 con 0x7fd02415a3f0 2023-10-06T11:37:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.936+0000 7fd018ff9700 1 -- 172.21.15.46:0/1991729445 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v719) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd0240c6450 con 0x7fd02415a3f0 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 msgr2=0x7fd010062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 0x7fd010062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0240c4d80 tx=0x7fd0200091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 msgr2=0x7fd0240c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd024063940 tx=0x7fd014017000 comp rx=0 tx=0).stop 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 shutdown_connections 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd010060670 0x7fd010062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd02415a7c0 0x7fd024153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd02415a3f0 0x7fd0240c70d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 --2- 172.21.15.46:0/1991729445 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240cdf30 0x7fd02414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 >> 172.21.15.46:0/1991729445 conn(0x7fd024062d50 msgr2=0x7fd024150670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 shutdown_connections 2023-10-06T11:37:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:14.940+0000 7fd02ac78700 1 -- 172.21.15.46:0/1991729445 wait complete. 2023-10-06T11:37:14.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:37:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 Processor -- start 2023-10-06T11:37:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 -- start start 2023-10-06T11:37:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb00b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 0x7f0cb00aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00b0190 con 0x7f0cb00af0a0 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00b1620 con 0x7f0cb015a8a0 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0cb5e78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00ae9f0 con 0x7f0cb015a4d0 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0caffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0caffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46564/0 (socket says 172.21.15.46:46564) 2023-10-06T11:37:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.140+0000 7f0caffff700 1 -- 172.21.15.46:0/4123540256 learned_addr learned my addr 172.21.15.46:0/4123540256 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:15.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb00b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caeffd700 1 --2- 172.21.15.46:0/4123540256 >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 0x7f0cb00aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 -- 172.21.15.46:0/4123540256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 msgr2=0x7f0cb00b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb00b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 -- 172.21.15.46:0/4123540256 >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 msgr2=0x7f0cb00aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 --2- 172.21.15.46:0/4123540256 >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 0x7f0cb00aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 -- 172.21.15.46:0/4123540256 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00ab390 con 0x7f0cb00af0a0 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb00b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caeffd700 1 --2- 172.21.15.46:0/4123540256 >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 0x7f0cb00aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0caffff700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0cb006e320 tx=0x7f0ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d775a18179bfb0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cae7fc700 1 -- 172.21.15.46:0/4123540256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca800e070 con 0x7f0cb00af0a0 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cae7fc700 1 -- 172.21.15.46:0/4123540256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca800ba40 con 0x7f0cb00af0a0 2023-10-06T11:37:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cae7fc700 1 -- 172.21.15.46:0/4123540256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80092e0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 -- 172.21.15.46:0/4123540256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 msgr2=0x7f0cb00ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0cb006e320 tx=0x7f0ca8009840 comp rx=0 tx=0).stop 2023-10-06T11:37:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 -- 172.21.15.46:0/4123540256 shutdown_connections 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/4123540256 >> v2:172.21.15.132:3300/0 conn(0x7f0cb015a8a0 0x7f0cb00aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb00ad9c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/4123540256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb00b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 -- 172.21.15.46:0/4123540256 >> 172.21.15.46:0/4123540256 conn(0x7f0cb0097330 msgr2=0x7f0cb0067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 -- 172.21.15.46:0/4123540256 shutdown_connections 2023-10-06T11:37:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.144+0000 7f0cb5e78700 1 -- 172.21.15.46:0/4123540256 wait complete. 2023-10-06T11:37:15.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 Processor -- start 2023-10-06T11:37:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 -- start start 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb009b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cb015a8a0 0x7f0cb00993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb0098660 con 0x7f0cb00af0a0 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.148+0000 7f0cb5e78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00b1900 con 0x7f0cb015a8a0 2023-10-06T11:37:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0cb5e78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00990f0 con 0x7f0cb015a4d0 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46580/0 (socket says 172.21.15.46:46580) 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 -- 172.21.15.46:0/3070542009 learned_addr learned my addr 172.21.15.46:0/3070542009 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caffff700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cb015a8a0 0x7f0cb00993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caeffd700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb009b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 -- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 msgr2=0x7f0cb009b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb009b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 -- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cb015a8a0 msgr2=0x7f0cb00993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cb015a8a0 0x7f0cb00993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 -- 172.21.15.46:0/3070542009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8009ae0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c8b0 tx=0x7f0ca0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0011600 con 0x7f0cb00af0a0 2023-10-06T11:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0011760 con 0x7f0cb00af0a0 2023-10-06T11:37:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb0098850 con 0x7f0cb00af0a0 2023-10-06T11:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0011b00 con 0x7f0cb00af0a0 2023-10-06T11:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.152+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb0098e10 con 0x7f0cb00af0a0 2023-10-06T11:37:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.156+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb006cfc0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.156+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0ca0018350 con 0x7f0cb00af0a0 2023-10-06T11:37:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.160+0000 7f0c977fe700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 0x7f0c9c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.160+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f0ca007cc70 con 0x7f0cb00af0a0 2023-10-06T11:37:15.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.160+0000 7f0caeffd700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 0x7f0c9c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.160+0000 7f0caeffd700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 0x7f0c9c062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0159770 tx=0x7f0ca4008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.168+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0ca004ccb0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.340+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0cb004b5c0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.352+0000 7f0c977fe700 1 -- 172.21.15.46:0/3070542009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ca001e0c0 con 0x7f0cb00af0a0 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 msgr2=0x7f0c9c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 0x7f0c9c062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0159770 tx=0x7f0ca4008040 comp rx=0 tx=0).stop 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 msgr2=0x7f0cb014bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000c8b0 tx=0x7f0ca0008040 comp rx=0 tx=0).stop 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 shutdown_connections 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0c9c0606a0 0x7f0c9c062b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cb015a8a0 0x7f0cb00993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cb015a4d0 0x7f0cb009b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 --2- 172.21.15.46:0/3070542009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb00af0a0 0x7f0cb014bd00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 >> 172.21.15.46:0/3070542009 conn(0x7f0cb0097330 msgr2=0x7f0cb0155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 shutdown_connections 2023-10-06T11:37:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.356+0000 7f0cb5e78700 1 -- 172.21.15.46:0/3070542009 wait complete. 2023-10-06T11:37:15.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:37:15.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:37:15.439 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:37:15.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:37:15.484 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:37:15.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.636+0000 7fd71c0ae700 1 Processor -- start 2023-10-06T11:37:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.636+0000 7fd71c0ae700 1 -- start start 2023-10-06T11:37:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71c0ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd71415c770 0x7fd7140cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71c0ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140ccb50 0x7fd7140ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71c0ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7140cc150 con 0x7fd7140cea30 2023-10-06T11:37:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71c0ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7140cfbb0 con 0x7fd71415c770 2023-10-06T11:37:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71c0ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140cdbf0 con 0x7fd7140ccb50 2023-10-06T11:37:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71a64b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7140cea30 0x7fd7140c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45352/0 (socket says 172.21.15.46:45352) 2023-10-06T11:37:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd71a64b700 1 -- 172.21.15.46:0/3494150051 learned_addr learned my addr 172.21.15.46:0/3494150051 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 --2- 172.21.15.46:0/3494150051 >> v2:172.21.15.132:3300/0 conn(0x7fd71415c770 0x7fd7140cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719649700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140ccb50 0x7fd7140ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd718e48700 1 -- 172.21.15.46:0/3494150051 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862054566 0 0) 0x7fd7140cc150 con 0x7fd7140cea30 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd718e48700 1 -- 172.21.15.46:0/3494150051 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd700003570 con 0x7fd7140cea30 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 -- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140ccb50 msgr2=0x7fd7140ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140ccb50 0x7fd7140ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 -- 172.21.15.46:0/3494150051 >> v1:172.21.15.46:6789/0 conn(0x7fd7140cea30 legacy=0x7fd7140c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 -- 172.21.15.46:0/3494150051 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140cee70 con 0x7fd71415c770 2023-10-06T11:37:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd719e4a700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71415c770 0x7fd7140cad40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd7100098d0 tx=0x7fd710009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=277d9604ae12569b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd718e48700 1 -- 172.21.15.46:0/3494150051 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710005c10 con 0x7fd71415c770 2023-10-06T11:37:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd718e48700 1 -- 172.21.15.46:0/3494150051 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710005d70 con 0x7fd71415c770 2023-10-06T11:37:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.640+0000 7fd718e48700 1 -- 172.21.15.46:0/3494150051 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710006280 con 0x7fd71415c770 2023-10-06T11:37:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71415c770 msgr2=0x7fd7140cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71415c770 0x7fd7140cad40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd7100098d0 tx=0x7fd710009120 comp rx=0 tx=0).stop 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- 172.21.15.46:0/3494150051 shutdown_connections 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140ccb50 0x7fd7140ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/3494150051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71415c770 0x7fd7140cad40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- 172.21.15.46:0/3494150051 >> 172.21.15.46:0/3494150051 conn(0x7fd714099480 msgr2=0x7fd71406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- 172.21.15.46:0/3494150051 shutdown_connections 2023-10-06T11:37:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- 172.21.15.46:0/3494150051 wait complete. 2023-10-06T11:37:15.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 Processor -- start 2023-10-06T11:37:15.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.644+0000 7fd71c0ae700 1 -- start start 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7140ccb50 0x7fd714151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140cea30 0x7fd71415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140b4290 con 0x7fd71415c770 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140b3680 con 0x7fd7140ccb50 2023-10-06T11:37:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71c0ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140b3960 con 0x7fd7140cea30 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd719649700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140cea30 0x7fd71415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46588/0 (socket says 172.21.15.46:46588) 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 -- 172.21.15.46:0/1382531167 learned_addr learned my addr 172.21.15.46:0/1382531167 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd719e4a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7140ccb50 0x7fd714151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 -- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140cea30 msgr2=0x7fd71415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140cea30 0x7fd71415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 -- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7140ccb50 msgr2=0x7fd714151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7140ccb50 0x7fd714151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 -- 172.21.15.46:0/1382531167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100093c0 con 0x7fd71415c770 2023-10-06T11:37:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.648+0000 7fd71a64b700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001460 tx=0x7fd70c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00a890 con 0x7fd71415c770 2023-10-06T11:37:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7140b3c40 con 0x7fd71415c770 2023-10-06T11:37:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00a9f0 con 0x7fd71415c770 2023-10-06T11:37:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00ad70 con 0x7fd71415c770 2023-10-06T11:37:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd714152430 con 0x7fd71415c770 2023-10-06T11:37:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.652+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7140699f0 con 0x7fd71415c770 2023-10-06T11:37:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.656+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd70c007300 con 0x7fd71415c770 2023-10-06T11:37:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.656+0000 7fd709ffb700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 0x7fd6f8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.656+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7fd70c07d0e0 con 0x7fd71415c770 2023-10-06T11:37:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.656+0000 7fd719e4a700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 0x7fd6f8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.656+0000 7fd719e4a700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 0x7fd6f8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd710009340 tx=0x7fd710016000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.660+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7140699f0 con 0x7fd71415c770 2023-10-06T11:37:15.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.868+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7140699f0 con 0x7fd71415c770 2023-10-06T11:37:15.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.868+0000 7fd709ffb700 1 -- 172.21.15.46:0/1382531167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fd7140699f0 con 0x7fd71415c770 2023-10-06T11:37:15.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:15.883 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7594,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4089401595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4089401595},{"type":"v1","addr":"172.21.15.132:6834","nonce":4089401595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":7606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1246937518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1246937518},{"type":"v1","addr":"172.21.15.46:6837","nonce":1246937518}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":7818,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1125207548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1125207548},{"type":"v1","addr":"172.21.15.132:6835","nonce":1125207548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":54},{"gid":8244,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/1612578728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1612578728},{"type":"v1","addr":"172.21.15.46:6836","nonce":1612578728}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[]} 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 msgr2=0x7fd6f8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 0x7fd6f8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd710009340 tx=0x7fd710016000 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 msgr2=0x7fd7141540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001460 tx=0x7fd70c001ae0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 shutdown_connections 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6f8060670 0x7fd6f8062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71415c770 0x7fd7141540e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7140cea30 0x7fd71415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 --2- 172.21.15.46:0/1382531167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7140ccb50 0x7fd714151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 >> 172.21.15.46:0/1382531167 conn(0x7fd714099480 msgr2=0x7fd714154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:15.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 shutdown_connections 2023-10-06T11:37:15.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:15.872+0000 7fd71c0ae700 1 -- 172.21.15.46:0/1382531167 wait complete. 2023-10-06T11:37:15.888 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 61 2023-10-06T11:37:15.903 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:37:15.904 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:37:15.904 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:37:15.904 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:37:15.904 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:37:15.904 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:37:15.905 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:37:15.905 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:37:15.905 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:37:15.905 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:37:15.905 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:37:15.905 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:37:15.905 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:37:15.906 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:37:15.906 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:37:15.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:37:15.922 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:37:15.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:37:15.927 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:37:15.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:37:15.928 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:37:15.928 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:37:15.929 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:37:15.929 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:37:15.979 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:37:15.979 DEBUG:teuthology.parallel:result is None 2023-10-06T11:37:15.980 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:37:15.980 DEBUG:teuthology.parallel:result is None 2023-10-06T11:37:15.982 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:37:15.983 DEBUG:teuthology.parallel:result is None 2023-10-06T11:37:15.984 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:37:15.985 DEBUG:teuthology.parallel:result is None 2023-10-06T11:37:15.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:37:16.083 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T11:37:16.089 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T11:37:16.090 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T11:37:16.092 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T11:37:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.140+0000 7f61aaa0c700 1 Processor -- start 2023-10-06T11:37:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.140+0000 7f61aaa0c700 1 -- start start 2023-10-06T11:37:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61a40b02a0 0x7f61a40c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 0x7f61a40cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 0x7f61a40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a40cb180 con 0x7f61a40b02a0 2023-10-06T11:37:16.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a40c90b0 con 0x7f61a40ca7c0 2023-10-06T11:37:16.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40cc980 con 0x7f61a4158100 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61a40b02a0 0x7f61a40c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61a40b02a0 0x7f61a40c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46618/0 (socket says 172.21.15.46:46618) 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 -- 172.21.15.46:0/530823626 learned_addr learned my addr 172.21.15.46:0/530823626 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aa20b700 1 --2- 172.21.15.46:0/530823626 >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 0x7f61a40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 -- 172.21.15.46:0/530823626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 msgr2=0x7f61a40cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9209700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 0x7f61a40cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 0x7f61a40cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 -- 172.21.15.46:0/530823626 >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 msgr2=0x7f61a40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:16.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/530823626 >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 0x7f61a40cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 -- 172.21.15.46:0/530823626 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40ccc60 con 0x7f61a40b02a0 2023-10-06T11:37:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aa20b700 1 --2- 172.21.15.46:0/530823626 >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 0x7f61a40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9209700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 0x7f61a40cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:16.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a40c2c20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f61a4063830 tx=0x7f61a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b174de6c1c04ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a8a08700 1 -- 172.21.15.46:0/530823626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0005c10 con 0x7f61a40b02a0 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a8a08700 1 -- 172.21.15.46:0/530823626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a0005d70 con 0x7f61a40b02a0 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a8a08700 1 -- 172.21.15.46:0/530823626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000b840 con 0x7f61a40b02a0 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- 172.21.15.46:0/530823626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 msgr2=0x7f61a40c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a40c2c20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f61a4063830 tx=0x7f61a0009220 comp rx=0 tx=0).stop 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- 172.21.15.46:0/530823626 shutdown_connections 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/530823626 >> v2:172.21.15.132:3300/0 conn(0x7f61a40ca7c0 0x7f61a40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a4158100 0x7f61a40cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/530823626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a40c2c20 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- 172.21.15.46:0/530823626 >> 172.21.15.46:0/530823626 conn(0x7f61a40970c0 msgr2=0x7f61a4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:16.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- 172.21.15.46:0/530823626 shutdown_connections 2023-10-06T11:37:16.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- 172.21.15.46:0/530823626 wait complete. 2023-10-06T11:37:16.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 Processor -- start 2023-10-06T11:37:16.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- start start 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 0x7f61a40b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 0x7f61a40b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a4097a30 con 0x7f61a40b02a0 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a409b990 con 0x7f61a4158100 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61aaa0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a4151180 con 0x7f61a40ca7c0 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.144+0000 7f61a9a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46630/0 (socket says 172.21.15.46:46630) 2023-10-06T11:37:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 -- 172.21.15.46:0/2250047177 learned_addr learned my addr 172.21.15.46:0/2250047177 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:16.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 -- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 msgr2=0x7f61a40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9209700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 0x7f61a40b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61aa20b700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 0x7f61a40b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 0x7f61a40b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 -- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 msgr2=0x7f61a40b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 0x7f61a40b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 -- 172.21.15.46:0/2250047177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009480 con 0x7f61a40b02a0 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61aa20b700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 0x7f61a40b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9209700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 0x7f61a40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61a9a0a700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f61a406d9b0 tx=0x7f61a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a00118d0 con 0x7f61a40b02a0 2023-10-06T11:37:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a0020070 con 0x7f61a40b02a0 2023-10-06T11:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0011cc0 con 0x7f61a40b02a0 2023-10-06T11:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a40c54f0 con 0x7f61a40b02a0 2023-10-06T11:37:16.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a40c5ab0 con 0x7f61a40b02a0 2023-10-06T11:37:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.148+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a4097ba0 con 0x7f61a40b02a0 2023-10-06T11:37:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.152+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f61a0022030 con 0x7f61a40b02a0 2023-10-06T11:37:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.152+0000 7f6199ffb700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 0x7f6188062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.152+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f61a007e730 con 0x7f61a40b02a0 2023-10-06T11:37:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.152+0000 7f61a9209700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 0x7f6188062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.156+0000 7f61a9209700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 0x7f6188062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6194006fd0 tx=0x7f6194008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.164+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f61a004e880 con 0x7f61a40b02a0 2023-10-06T11:37:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.344+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f61a4064590 con 0x7f61a40b02a0 2023-10-06T11:37:16.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.964+0000 7f6199ffb700 1 -- 172.21.15.46:0/2250047177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) v1 ==== 97+0+0 (secure 0 0 0) 0x7f61a002a080 con 0x7f61a40b02a0 2023-10-06T11:37:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 msgr2=0x7f6188062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:16.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 0x7f6188062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6194006fd0 tx=0x7f6194008040 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 msgr2=0x7f61a4150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f61a406d9b0 tx=0x7f61a0016000 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 shutdown_connections 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6188060670 0x7f6188062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61a4158100 0x7f61a40b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61a40ca7c0 0x7f61a40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 --2- 172.21.15.46:0/2250047177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a40b02a0 0x7f61a4150c40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 >> 172.21.15.46:0/2250047177 conn(0x7f61a40970c0 msgr2=0x7f61a40cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 shutdown_connections 2023-10-06T11:37:16.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:16.972+0000 7f61aaa0c700 1 -- 172.21.15.46:0/2250047177 wait complete. 2023-10-06T11:37:16.983 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T11:37:17.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:37:17.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.160+0000 7f41bbd45700 1 Processor -- start 2023-10-06T11:37:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 -- start start 2023-10-06T11:37:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41b40ccb50 0x7f41b40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41b40cea30 0x7f41b40c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b40cfbb0 con 0x7f41b40ccb50 2023-10-06T11:37:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b40cdbf0 con 0x7f41b40cea30 2023-10-06T11:37:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41bbd45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40cc150 con 0x7f41b415c770 2023-10-06T11:37:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b9ae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b92e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41b40ccb50 0x7f41b40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b9ae1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:52982/0 (socket says 172.21.15.46:52982) 2023-10-06T11:37:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b9ae1700 1 -- 172.21.15.46:0/381844233 learned_addr learned my addr 172.21.15.46:0/381844233 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41ba2e2700 1 --2- 172.21.15.46:0/381844233 >> v2:172.21.15.132:3300/0 conn(0x7f41b40cea30 0x7f41b40c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b92e0700 1 -- 172.21.15.46:0/381844233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 msgr2=0x7f41b40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.164+0000 7f41b92e0700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b92e0700 1 -- 172.21.15.46:0/381844233 >> v2:172.21.15.132:3300/0 conn(0x7f41b40cea30 msgr2=0x7f41b40c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b92e0700 1 --2- 172.21.15.46:0/381844233 >> v2:172.21.15.132:3300/0 conn(0x7f41b40cea30 0x7f41b40c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b92e0700 1 -- 172.21.15.46:0/381844233 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b40cee70 con 0x7f41b40ccb50 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b9ae1700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b92e0700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40ccb50 0x7f41b40ce4f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f41b40cf000 tx=0x7f41a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02acca3f31a275d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b8adf700 1 -- 172.21.15.46:0/381844233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011630 con 0x7f41b40ccb50 2023-10-06T11:37:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b8adf700 1 -- 172.21.15.46:0/381844233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a4011790 con 0x7f41b40ccb50 2023-10-06T11:37:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41b8adf700 1 -- 172.21.15.46:0/381844233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011b50 con 0x7f41b40ccb50 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 -- 172.21.15.46:0/381844233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40ccb50 msgr2=0x7f41b40ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40ccb50 0x7f41b40ce4f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f41b40cf000 tx=0x7f41a40091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 -- 172.21.15.46:0/381844233 shutdown_connections 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 --2- 172.21.15.46:0/381844233 >> v2:172.21.15.132:3300/0 conn(0x7f41b40cea30 0x7f41b40c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40ccb50 0x7f41b40ce4f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 --2- 172.21.15.46:0/381844233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 -- 172.21.15.46:0/381844233 >> 172.21.15.46:0/381844233 conn(0x7f41b4099480 msgr2=0x7f41b406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 -- 172.21.15.46:0/381844233 shutdown_connections 2023-10-06T11:37:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.168+0000 7f41bbd45700 1 -- 172.21.15.46:0/381844233 wait complete. 2023-10-06T11:37:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 Processor -- start 2023-10-06T11:37:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 -- start start 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b40ccb50 0x7f41b4167450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4155ce0 con 0x7f41b40cea30 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b416b9d0 con 0x7f41b40ccb50 2023-10-06T11:37:17.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.172+0000 7f41bbd45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b416a780 con 0x7f41b415c770 2023-10-06T11:37:17.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46652/0 (socket says 172.21.15.46:46652) 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 -- 172.21.15.46:0/2019869949 learned_addr learned my addr 172.21.15.46:0/2019869949 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b9ae1700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b40ccb50 0x7f41b4167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41ba2e2700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 -- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 msgr2=0x7f41b40b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 -- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b40ccb50 msgr2=0x7f41b4167450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b40ccb50 0x7f41b4167450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 -- 172.21.15.46:0/2019869949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b0009ae0 con 0x7f41b40cea30 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41b92e0700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f41a40091e0 tx=0x7f41a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a401abf0 con 0x7f41b40cea30 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a401f070 con 0x7f41b40cea30 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4026480 con 0x7f41b40cea30 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a4009420 con 0x7f41b40cea30 2023-10-06T11:37:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.176+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b416b0a0 con 0x7f41b40cea30 2023-10-06T11:37:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.180+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b416bb40 con 0x7f41b40cea30 2023-10-06T11:37:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.180+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f41a401b450 con 0x7f41b40cea30 2023-10-06T11:37:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.180+0000 7f41a9ffb700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 0x7f4198062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.180+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7f41a4021070 con 0x7f41b40cea30 2023-10-06T11:37:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.180+0000 7f41b9ae1700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 0x7f4198062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.184+0000 7f41b9ae1700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 0x7f4198062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41b00072d0 tx=0x7f41b000b040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.188+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f41a404e7e0 con 0x7f41b40cea30 2023-10-06T11:37:17.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.424+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f41b40660f0 con 0x7f41b40cea30 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.424+0000 7f41a9ffb700 1 -- 172.21.15.46:0/2019869949 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f41a4051e00 con 0x7f41b40cea30 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T11:37:17.437 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:37:17.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.441 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.442 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T11:37:17.444 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.445 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.446 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.447 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:37:17.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.450 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T11:37:17.451 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T11:37:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T11:37:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:17.452 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T11:37:17.452 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 msgr2=0x7f4198062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 0x7f4198062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f41b00072d0 tx=0x7f41b000b040 comp rx=0 tx=0).stop 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 msgr2=0x7f41b416a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f41a40091e0 tx=0x7f41a4016000 comp rx=0 tx=0).stop 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 shutdown_connections 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4198060670 0x7f4198062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b415c770 0x7f41b40b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41b40cea30 0x7f41b416a240 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 --2- 172.21.15.46:0/2019869949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b40ccb50 0x7f41b4167450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 >> 172.21.15.46:0/2019869949 conn(0x7f41b4099480 msgr2=0x7f41b40cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 shutdown_connections 2023-10-06T11:37:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.432+0000 7f41bbd45700 1 -- 172.21.15.46:0/2019869949 wait complete. 2023-10-06T11:37:17.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 Processor -- start 2023-10-06T11:37:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 -- start start 2023-10-06T11:37:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52f8156fb0 0x7f52f80ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f80b1fe0 0x7f52f80aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80ad830 con 0x7f52f8157380 2023-10-06T11:37:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f80ae4b0 con 0x7f52f8156fb0 2023-10-06T11:37:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52fd6ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80adb10 con 0x7f52f80b1fe0 2023-10-06T11:37:17.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52f6ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52f8156fb0 0x7f52f80ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52f6ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52f8156fb0 0x7f52f80ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46664/0 (socket says 172.21.15.46:46664) 2023-10-06T11:37:17.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52f6ffd700 1 -- 172.21.15.46:0/1982455965 learned_addr learned my addr 172.21.15.46:0/1982455965 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:17.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.612+0000 7f52f77fe700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f80b1fe0 0x7f52f80aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f6ffd700 1 -- 172.21.15.46:0/1982455965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f80b1fe0 msgr2=0x7f52f80aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:17.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f6ffd700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f80b1fe0 0x7f52f80aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f6ffd700 1 -- 172.21.15.46:0/1982455965 >> v1:172.21.15.132:6789/0 conn(0x7f52f8157380 legacy=0x7f52f80aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f6ffd700 1 -- 172.21.15.46:0/1982455965 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80addf0 con 0x7f52f8156fb0 2023-10-06T11:37:17.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f6ffd700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f80ad120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009f20 tx=0x7f52e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eca8963aada548e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f67fc700 1 -- 172.21.15.46:0/1982455965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800e070 con 0x7f52f8156fb0 2023-10-06T11:37:17.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f67fc700 1 -- 172.21.15.46:0/1982455965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e800ba80 con 0x7f52f8156fb0 2023-10-06T11:37:17.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52f67fc700 1 -- 172.21.15.46:0/1982455965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80092e0 con 0x7f52f8156fb0 2023-10-06T11:37:17.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1982455965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 msgr2=0x7f52f80ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f80ad120 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009f20 tx=0x7f52e8009840 comp rx=0 tx=0).stop 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1982455965 shutdown_connections 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f80b1fe0 0x7f52f80aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1982455965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f80ad120 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1982455965 >> 172.21.15.46:0/1982455965 conn(0x7f52f8097190 msgr2=0x7f52f8067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1982455965 shutdown_connections 2023-10-06T11:37:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.616+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1982455965 wait complete. 2023-10-06T11:37:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 Processor -- start 2023-10-06T11:37:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 -- start start 2023-10-06T11:37:17.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52f80b1fe0 0x7f52f8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f8157380 0x7f52f815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8099040 con 0x7f52f8156fb0 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80991d0 con 0x7f52f80b1fe0 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52fd6ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f815a790 con 0x7f52f8157380 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46678/0 (socket says 172.21.15.46:46678) 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52effff700 1 -- 172.21.15.46:0/1417510730 learned_addr learned my addr 172.21.15.46:0/1417510730 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52f6ffd700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52f80b1fe0 0x7f52f8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.620+0000 7f52f77fe700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f8157380 0x7f52f815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 -- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f8157380 msgr2=0x7f52f815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f8157380 0x7f52f815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 -- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52f80b1fe0 msgr2=0x7f52f8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52f80b1fe0 0x7f52f8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 -- 172.21.15.46:0/1417510730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8009ae0 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52effff700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c590 tx=0x7f52e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0011590 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e0011710 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f8158780 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0011ad0 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f8158d40 con 0x7f52f8156fb0 2023-10-06T11:37:17.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.624+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8099380 con 0x7f52f8156fb0 2023-10-06T11:37:17.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.628+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f52e001b3b0 con 0x7f52f8156fb0 2023-10-06T11:37:17.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.628+0000 7f52eeffd700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 0x7f52d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:17.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.628+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7f52e007bdc0 con 0x7f52f8156fb0 2023-10-06T11:37:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.628+0000 7f52f6ffd700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 0x7f52d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.628+0000 7f52f6ffd700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 0x7f52d8062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009880 tx=0x7f52e8018000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:17.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.636+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52f8099380 con 0x7f52f8156fb0 2023-10-06T11:37:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.804+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f8099340 con 0x7f52f8156fb0 2023-10-06T11:37:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.808+0000 7f52eeffd700 1 -- 172.21.15.46:0/1417510730 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+14376 (secure 0 0 0) 0x7f52e0017020 con 0x7f52f8156fb0 2023-10-06T11:37:17.819 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:17.819 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":131,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:37:16.978542+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 msgr2=0x7f52d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 0x7f52d8062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009880 tx=0x7f52e8018000 comp rx=0 tx=0).stop 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 msgr2=0x7f52f8152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c590 tx=0x7f52e000c8a0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 shutdown_connections 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f52d8060670 0x7f52d8062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52f8157380 0x7f52f815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8156fb0 0x7f52f8152320 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 --2- 172.21.15.46:0/1417510730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52f80b1fe0 0x7f52f8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:17.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 >> 172.21.15.46:0/1417510730 conn(0x7f52f8097190 msgr2=0x7f52f806bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:17.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 shutdown_connections 2023-10-06T11:37:17.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.812+0000 7f52fd6ae700 1 -- 172.21.15.46:0/1417510730 wait complete. 2023-10-06T11:37:17.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T11:37:17.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:37:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.992+0000 7f270ef23700 1 Processor -- start 2023-10-06T11:37:18.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.992+0000 7f270ef23700 1 -- start start 2023-10-06T11:37:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270ef23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2708069270 0x7f2708069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270ef23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708159850 0x7f2708159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270ef23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27080ce6e0 con 0x7f2708158aa0 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270ef23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270815a5c0 con 0x7f2708069270 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270ef23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27080cddb0 con 0x7f2708159850 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2708069270 0x7f2708069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2708069270 0x7f2708069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46682/0 (socket says 172.21.15.46:46682) 2023-10-06T11:37:18.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 -- 172.21.15.46:0/3101258382 learned_addr learned my addr 172.21.15.46:0/3101258382 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270e722700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708159850 0x7f2708159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 -- 172.21.15.46:0/3101258382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708159850 msgr2=0x7f2708159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708159850 0x7f2708159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 -- 172.21.15.46:0/3101258382 >> v1:172.21.15.132:6789/0 conn(0x7f2708158aa0 legacy=0x7f2708158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 -- 172.21.15.46:0/3101258382 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27080ce090 con 0x7f2708069270 2023-10-06T11:37:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270df21700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708069270 0x7f2708069640 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f27040099b0 tx=0x7f2704009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2542c980f26bebcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:17.996+0000 7f270cf1f700 1 -- 172.21.15.46:0/3101258382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704009ea0 con 0x7f2708069270 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270cf1f700 1 -- 172.21.15.46:0/3101258382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270400b860 con 0x7f2708069270 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270cf1f700 1 -- 172.21.15.46:0/3101258382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270400bc00 con 0x7f2708069270 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- 172.21.15.46:0/3101258382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708069270 msgr2=0x7f2708069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708069270 0x7f2708069640 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f27040099b0 tx=0x7f2704009250 comp rx=0 tx=0).stop 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- 172.21.15.46:0/3101258382 shutdown_connections 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708159850 0x7f2708159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 --2- 172.21.15.46:0/3101258382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708069270 0x7f2708069640 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- 172.21.15.46:0/3101258382 >> 172.21.15.46:0/3101258382 conn(0x7f2708068390 msgr2=0x7f27080d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- 172.21.15.46:0/3101258382 shutdown_connections 2023-10-06T11:37:18.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- 172.21.15.46:0/3101258382 wait complete. 2023-10-06T11:37:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 Processor -- start 2023-10-06T11:37:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.000+0000 7f270ef23700 1 -- start start 2023-10-06T11:37:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708069270 0x7f2708152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2708159850 0x7f27080c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080c7340 con 0x7f2708158aa0 2023-10-06T11:37:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080b4ce0 con 0x7f2708159850 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27080b40d0 con 0x7f2708069270 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46698/0 (socket says 172.21.15.46:46698) 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 -- 172.21.15.46:0/913492341 learned_addr learned my addr 172.21.15.46:0/913492341 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270df21700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708069270 0x7f2708152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270e722700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2708159850 0x7f27080c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 -- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708069270 msgr2=0x7f2708152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708069270 0x7f2708152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 -- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2708159850 msgr2=0x7f27080c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2708159850 0x7f27080c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 -- 172.21.15.46:0/913492341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040094f0 con 0x7f2708158aa0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270d720700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c590 tx=0x7f26f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8011590 con 0x7f2708158aa0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f8011710 con 0x7f2708158aa0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27080b43b0 con 0x7f2708158aa0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8011ad0 con 0x7f2708158aa0 2023-10-06T11:37:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.004+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27080b49f0 con 0x7f2708158aa0 2023-10-06T11:37:18.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.008+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2708064590 con 0x7f2708158aa0 2023-10-06T11:37:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.008+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f26f801b3b0 con 0x7f2708158aa0 2023-10-06T11:37:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.012+0000 7f26fdffb700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 0x7f26ec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.012+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7f26f807bda0 con 0x7f2708158aa0 2023-10-06T11:37:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.012+0000 7f270df21700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 0x7f26ec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.012+0000 7f270df21700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 0x7f26ec062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27040099b0 tx=0x7f2704017000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.016+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f26f804c280 con 0x7f2708158aa0 2023-10-06T11:37:18.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:37:18.088 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:37:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.192+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27080ce3a0 con 0x7f2708158aa0 2023-10-06T11:37:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.192+0000 7f26fdffb700 1 -- 172.21.15.46:0/913492341 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26f8017020 con 0x7f2708158aa0 2023-10-06T11:37:18.203 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T11:37:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 msgr2=0x7f26ec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 0x7f26ec062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27040099b0 tx=0x7f2704017000 comp rx=0 tx=0).stop 2023-10-06T11:37:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 msgr2=0x7f27080b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f26f800c590 tx=0x7f26f800c8a0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 shutdown_connections 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f26ec060670 0x7f26ec062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2708159850 0x7f27080c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2708158aa0 0x7f27080b37a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 --2- 172.21.15.46:0/913492341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2708069270 0x7f2708152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 >> 172.21.15.46:0/913492341 conn(0x7f2708068390 msgr2=0x7f2708155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 shutdown_connections 2023-10-06T11:37:18.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.196+0000 7f270ef23700 1 -- 172.21.15.46:0/913492341 wait complete. 2023-10-06T11:37:18.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:37:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.388+0000 7f554ac2e700 1 Processor -- start 2023-10-06T11:37:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.388+0000 7f554ac2e700 1 -- start start 2023-10-06T11:37:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f554ac2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5544147280 0x7f554406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f554ac2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440c1980 con 0x7f5544146eb0 2023-10-06T11:37:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f554ac2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440c2520 con 0x7f5544161fc0 2023-10-06T11:37:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f554ac2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440c0d70 con 0x7f5544147280 2023-10-06T11:37:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55489ca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5544146eb0 0x7f55440c1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45380/0 (socket says 172.21.15.46:45380) 2023-10-06T11:37:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55489ca700 1 -- 172.21.15.46:0/4199858754 learned_addr learned my addr 172.21.15.46:0/4199858754 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55491cb700 1 --2- 172.21.15.46:0/4199858754 >> v2:172.21.15.132:3300/0 conn(0x7f5544147280 0x7f554406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899857727 0 0) 0x7f55440c2520 con 0x7f5544161fc0 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5534003570 con 0x7f5544161fc0 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824490502 0 0) 0x7f55440c1980 con 0x7f5544146eb0 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55491cb700 1 -- 172.21.15.46:0/4199858754 >> v1:172.21.15.132:6789/0 conn(0x7f5544161fc0 legacy=0x7f55440caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55491cb700 1 -- 172.21.15.46:0/4199858754 >> v1:172.21.15.46:6789/0 conn(0x7f5544146eb0 legacy=0x7f55440c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55491cb700 1 -- 172.21.15.46:0/4199858754 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440c1050 con 0x7f5544147280 2023-10-06T11:37:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.392+0000 7f55491cb700 1 --2- 172.21.15.46:0/4199858754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544147280 0x7f554406cc30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e60 tx=0x7f553c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2ac7c57e9ba35c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00e070 con 0x7f5544147280 2023-10-06T11:37:18.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c00bb60 con 0x7f5544147280 2023-10-06T11:37:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f55437fe700 1 -- 172.21.15.46:0/4199858754 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c012430 con 0x7f5544147280 2023-10-06T11:37:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 -- 172.21.15.46:0/4199858754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544147280 msgr2=0x7f554406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 --2- 172.21.15.46:0/4199858754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544147280 0x7f554406cc30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f553c009e60 tx=0x7f553c009780 comp rx=0 tx=0).stop 2023-10-06T11:37:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 -- 172.21.15.46:0/4199858754 shutdown_connections 2023-10-06T11:37:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 --2- 172.21.15.46:0/4199858754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544147280 0x7f554406cc30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 -- 172.21.15.46:0/4199858754 >> 172.21.15.46:0/4199858754 conn(0x7f55440cbe30 msgr2=0x7f55440cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 -- 172.21.15.46:0/4199858754 shutdown_connections 2023-10-06T11:37:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.396+0000 7f554ac2e700 1 -- 172.21.15.46:0/4199858754 wait complete. 2023-10-06T11:37:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 Processor -- start 2023-10-06T11:37:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 -- start start 2023-10-06T11:37:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5544147280 0x7f554414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544161fc0 0x7f554414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554415ee10 con 0x7f5544146eb0 2023-10-06T11:37:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e200 con 0x7f5544161fc0 2023-10-06T11:37:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f554ac2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e4e0 con 0x7f5544147280 2023-10-06T11:37:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46714/0 (socket says 172.21.15.46:46714) 2023-10-06T11:37:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 -- 172.21.15.46:0/3651932394 learned_addr learned my addr 172.21.15.46:0/3651932394 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55491cb700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544161fc0 0x7f554414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f5543fff700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5544147280 0x7f554414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 -- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5544147280 msgr2=0x7f554414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5544147280 0x7f554414f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 -- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544161fc0 msgr2=0x7f554414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544161fc0 0x7f554414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.400+0000 7f55489ca700 1 -- 172.21.15.46:0/3651932394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c009a20 con 0x7f5544146eb0 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f55489ca700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7e0 tx=0x7f5538007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538005b30 con 0x7f5544146eb0 2023-10-06T11:37:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538005c90 con 0x7f5544146eb0 2023-10-06T11:37:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554415e7c0 con 0x7f5544146eb0 2023-10-06T11:37:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538007300 con 0x7f5544146eb0 2023-10-06T11:37:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554415d260 con 0x7f5544146eb0 2023-10-06T11:37:18.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.404+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440660f0 con 0x7f5544146eb0 2023-10-06T11:37:18.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.408+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f55380076b0 con 0x7f5544146eb0 2023-10-06T11:37:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.408+0000 7f5540ff9700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 0x7f5528062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.408+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7f553807bf40 con 0x7f5544146eb0 2023-10-06T11:37:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.408+0000 7f5543fff700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 0x7f5528062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.408+0000 7f5543fff700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 0x7f5528062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5544160e50 tx=0x7f5530009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.416+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f553804c3a0 con 0x7f5544146eb0 2023-10-06T11:37:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.584+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55440c1330 con 0x7f5544146eb0 2023-10-06T11:37:18.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.584+0000 7f5540ff9700 1 -- 172.21.15.46:0/3651932394 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v131) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55380170a0 con 0x7f5544146eb0 2023-10-06T11:37:18.595 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 msgr2=0x7f5528062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 0x7f5528062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5544160e50 tx=0x7f5530009040 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 msgr2=0x7f554415cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7e0 tx=0x7f5538007df0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 shutdown_connections 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5528060670 0x7f5528062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5544161fc0 0x7f554414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5544147280 0x7f554414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 --2- 172.21.15.46:0/3651932394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5544146eb0 0x7f554415cd20 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 >> 172.21.15.46:0/3651932394 conn(0x7f55440cbe30 msgr2=0x7f5544144e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 shutdown_connections 2023-10-06T11:37:18.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.588+0000 7f554ac2e700 1 -- 172.21.15.46:0/3651932394 wait complete. 2023-10-06T11:37:18.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:37:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.768+0000 7f9c7759e700 1 Processor -- start 2023-10-06T11:37:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.768+0000 7f9c7759e700 1 -- start start 2023-10-06T11:37:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 0x7f9c78067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c780bf1a0 0x7f9c78067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c78069660 con 0x7f9c780c7a00 2023-10-06T11:37:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c78068ac0 con 0x7f9c780bf1a0 2023-10-06T11:37:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c78068d30 con 0x7f9c780bedd0 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c780bf1a0 0x7f9c78067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c780bf1a0 0x7f9c78067590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46716/0 (socket says 172.21.15.46:46716) 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 -- 172.21.15.46:0/3609889628 learned_addr learned my addr 172.21.15.46:0/3609889628 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7659c700 1 --2- 172.21.15.46:0/3609889628 >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 0x7f9c78067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 -- 172.21.15.46:0/3609889628 >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 msgr2=0x7f9c78067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:18.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 --2- 172.21.15.46:0/3609889628 >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 0x7f9c78067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 -- 172.21.15.46:0/3609889628 >> v1:172.21.15.132:6789/0 conn(0x7f9c780c7a00 legacy=0x7f9c780c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 -- 172.21.15.46:0/3609889628 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78069010 con 0x7f9c780bf1a0 2023-10-06T11:37:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7659c700 1 --2- 172.21.15.46:0/3609889628 >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 0x7f9c78067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c6dd9b700 1 --2- 172.21.15.46:0/3609889628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bf1a0 0x7f9c78067590 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9c780cca40 tx=0x7f9c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618d64158d43b69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c75d9b700 1 -- 172.21.15.46:0/3609889628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000b820 con 0x7f9c780bf1a0 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c75d9b700 1 -- 172.21.15.46:0/3609889628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000b980 con 0x7f9c780bf1a0 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c75d9b700 1 -- 172.21.15.46:0/3609889628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000bd20 con 0x7f9c780bf1a0 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- 172.21.15.46:0/3609889628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bf1a0 msgr2=0x7f9c78067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 --2- 172.21.15.46:0/3609889628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bf1a0 0x7f9c78067590 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9c780cca40 tx=0x7f9c60009250 comp rx=0 tx=0).stop 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- 172.21.15.46:0/3609889628 shutdown_connections 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 --2- 172.21.15.46:0/3609889628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bf1a0 0x7f9c78067590 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 --2- 172.21.15.46:0/3609889628 >> v2:172.21.15.132:3300/0 conn(0x7f9c780bedd0 0x7f9c78067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.772+0000 7f9c7759e700 1 -- 172.21.15.46:0/3609889628 >> 172.21.15.46:0/3609889628 conn(0x7f9c78062ef0 msgr2=0x7f9c780cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.776+0000 7f9c7759e700 1 -- 172.21.15.46:0/3609889628 shutdown_connections 2023-10-06T11:37:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.776+0000 7f9c7759e700 1 -- 172.21.15.46:0/3609889628 wait complete. 2023-10-06T11:37:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.776+0000 7f9c7759e700 1 Processor -- start 2023-10-06T11:37:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.776+0000 7f9c7759e700 1 -- start start 2023-10-06T11:37:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c780bf1a0 0x7f9c780cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c780c7a00 0x7f9c7816dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780bdf20 con 0x7f9c780bedd0 2023-10-06T11:37:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7815ff10 con 0x7f9c780bf1a0 2023-10-06T11:37:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c781601f0 con 0x7f9c780c7a00 2023-10-06T11:37:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46722/0 (socket says 172.21.15.46:46722) 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 -- 172.21.15.46:0/4024602370 learned_addr learned my addr 172.21.15.46:0/4024602370 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c6dd9b700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c780bf1a0 0x7f9c780cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c76d9d700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c780c7a00 0x7f9c7816dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 -- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c780c7a00 msgr2=0x7f9c7816dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c780c7a00 0x7f9c7816dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 -- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c780bf1a0 msgr2=0x7f9c780cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c780bf1a0 0x7f9c780cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 -- 172.21.15.46:0/4024602370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600094f0 con 0x7f9c780bedd0 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7659c700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800cfb0 tx=0x7f9c68007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800ae30 con 0x7f9c780bedd0 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c68016420 con 0x7f9c780bedd0 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c680167e0 con 0x7f9c780bedd0 2023-10-06T11:37:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c781604d0 con 0x7f9c780bedd0 2023-10-06T11:37:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.780+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7815fa10 con 0x7f9c780bedd0 2023-10-06T11:37:18.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.784+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7815fc60 con 0x7f9c780bedd0 2023-10-06T11:37:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.784+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9c68016bd0 con 0x7f9c780bedd0 2023-10-06T11:37:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.784+0000 7f9c6effd700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 0x7f9c58062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.784+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7f9c68012070 con 0x7f9c780bedd0 2023-10-06T11:37:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.784+0000 7f9c6dd9b700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 0x7f9c58062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.788+0000 7f9c6dd9b700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 0x7f9c58062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c780cca40 tx=0x7f9c60016000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:18.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.792+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c6804cbe0 con 0x7f9c780bedd0 2023-10-06T11:37:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.960+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9c78064590 con 0x7f9c780bedd0 2023-10-06T11:37:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.968+0000 7f9c6effd700 1 -- 172.21.15.46:0/4024602370 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9c6804d6f0 con 0x7f9c780bedd0 2023-10-06T11:37:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T11:37:18.980 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T11:37:18.980 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:18.980 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:37:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.968+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 msgr2=0x7f9c58062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.968+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 0x7f9c58062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c780cca40 tx=0x7f9c60016000 comp rx=0 tx=0).stop 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 msgr2=0x7f9c780ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800cfb0 tx=0x7f9c68007870 comp rx=0 tx=0).stop 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 shutdown_connections 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9c58060670 0x7f9c58062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c780c7a00 0x7f9c7816dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c780bf1a0 0x7f9c780cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 --2- 172.21.15.46:0/4024602370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c780bedd0 0x7f9c780ca6b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 >> 172.21.15.46:0/4024602370 conn(0x7f9c78062ef0 msgr2=0x7f9c7815c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 shutdown_connections 2023-10-06T11:37:18.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:18.972+0000 7f9c7759e700 1 -- 172.21.15.46:0/4024602370 wait complete. 2023-10-06T11:37:19.001 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:37:19.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:37:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.164+0000 7fb8fc585700 1 Processor -- start 2023-10-06T11:37:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.168+0000 7fb8fc585700 1 -- start start 2023-10-06T11:37:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8f4161e50 0x7fb8f40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40c1800 con 0x7fb8f4147270 2023-10-06T11:37:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40c6ce0 con 0x7fb8f4146ea0 2023-10-06T11:37:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f40c1580 con 0x7fb8f4161e50 2023-10-06T11:37:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f9b20700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8f4161e50 0x7fb8f40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fab22700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb8f4147270 0x7fb8f40c1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45396/0 (socket says 172.21.15.46:45396) 2023-10-06T11:37:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fab22700 1 -- 172.21.15.46:0/2429692020 learned_addr learned my addr 172.21.15.46:0/2429692020 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f9b20700 1 -- 172.21.15.46:0/2429692020 >> v1:172.21.15.132:6789/0 conn(0x7fb8f4146ea0 legacy=0x7fb8f406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f9b20700 1 -- 172.21.15.46:0/2429692020 >> v1:172.21.15.46:6789/0 conn(0x7fb8f4147270 legacy=0x7fb8f40c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f931f700 1 -- 172.21.15.46:0/2429692020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916186959 0 0) 0x7fb8f40c1800 con 0x7fb8f4147270 2023-10-06T11:37:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f9b20700 1 -- 172.21.15.46:0/2429692020 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40c6460 con 0x7fb8f4161e50 2023-10-06T11:37:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f9b20700 1 --2- 172.21.15.46:0/2429692020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4161e50 0x7fb8f40caa00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098e0 tx=0x7fb8e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa69774a5aa7201 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f931f700 1 -- 172.21.15.46:0/2429692020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4009e80 con 0x7fb8f4161e50 2023-10-06T11:37:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f931f700 1 -- 172.21.15.46:0/2429692020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400b860 con 0x7fb8f4161e50 2023-10-06T11:37:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8f931f700 1 -- 172.21.15.46:0/2429692020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400bbe0 con 0x7fb8f4161e50 2023-10-06T11:37:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 -- 172.21.15.46:0/2429692020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4161e50 msgr2=0x7fb8f40caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.172+0000 7fb8fc585700 1 --2- 172.21.15.46:0/2429692020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4161e50 0x7fb8f40caa00 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098e0 tx=0x7fb8e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- 172.21.15.46:0/2429692020 shutdown_connections 2023-10-06T11:37:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 --2- 172.21.15.46:0/2429692020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4161e50 0x7fb8f40caa00 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- 172.21.15.46:0/2429692020 >> 172.21.15.46:0/2429692020 conn(0x7fb8f40cbe20 msgr2=0x7fb8f40cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- 172.21.15.46:0/2429692020 shutdown_connections 2023-10-06T11:37:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- 172.21.15.46:0/2429692020 wait complete. 2023-10-06T11:37:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 Processor -- start 2023-10-06T11:37:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- start start 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 0x7fb8f415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 0x7fb8f40af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f416f490 con 0x7fb8f4147270 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4141790 con 0x7fb8f4161e50 2023-10-06T11:37:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.176+0000 7fb8fc585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4141a70 con 0x7fb8f4146ea0 2023-10-06T11:37:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46748/0 (socket says 172.21.15.46:46748) 2023-10-06T11:37:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 -- 172.21.15.46:0/1134778348 learned_addr learned my addr 172.21.15.46:0/1134778348 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fa321700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 0x7fb8f415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fab22700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 0x7fb8f40af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 -- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 msgr2=0x7fb8f415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 0x7fb8f415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 -- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 msgr2=0x7fb8f40af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 0x7fb8f40af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 -- 172.21.15.46:0/1134778348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4009420 con 0x7fb8f4147270 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fa321700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 0x7fb8f415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fab22700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 0x7fb8f40af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8f9b20700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb8f406f470 tx=0x7fb8e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e401f070 con 0x7fb8f4147270 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4011420 con 0x7fb8f4147270 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f4141d50 con 0x7fb8f4147270 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400bbe0 con 0x7fb8f4147270 2023-10-06T11:37:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f40b09a0 con 0x7fb8f4147270 2023-10-06T11:37:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.180+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f416ef80 con 0x7fb8f4147270 2023-10-06T11:37:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.184+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb8e4011a50 con 0x7fb8f4147270 2023-10-06T11:37:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.184+0000 7fb8ea7fc700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 0x7fb8d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.184+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5678+0+0 (secure 0 0 0) 0x7fb8e407ded0 con 0x7fb8f4147270 2023-10-06T11:37:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.184+0000 7fb8fa321700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 0x7fb8d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.184+0000 7fb8fa321700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 0x7fb8d8062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0001840 tx=0x7fb8f0009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.192+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb8f416ef80 con 0x7fb8f4147270 2023-10-06T11:37:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.360+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb8f416ef80 con 0x7fb8f4147270 2023-10-06T11:37:19.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.980+0000 7fb8ea7fc700 1 -- 172.21.15.46:0/1134778348 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb8f416ef80 con 0x7fb8f4147270 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 msgr2=0x7fb8d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 0x7fb8d8062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0001840 tx=0x7fb8f0009040 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 msgr2=0x7fb8f40aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb8f406f470 tx=0x7fb8e4017000 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 shutdown_connections 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb8d8060670 0x7fb8d8062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f4161e50 0x7fb8f40af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f4147270 0x7fb8f40aebc0 secure :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb8f406f470 tx=0x7fb8e4017000 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 --2- 172.21.15.46:0/1134778348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8f4146ea0 0x7fb8f415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 >> 172.21.15.46:0/1134778348 conn(0x7fb8f40cbe20 msgr2=0x7fb8f406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:20.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 shutdown_connections 2023-10-06T11:37:20.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:19.992+0000 7fb8fc585700 1 -- 172.21.15.46:0/1134778348 wait complete. 2023-10-06T11:37:20.005 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T11:37:20.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:37:20.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.180+0000 7fc82982e700 1 Processor -- start 2023-10-06T11:37:20.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc82982e700 1 -- start start 2023-10-06T11:37:20.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc82982e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 0x7fc8240c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:20.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc82982e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240c1bf0 con 0x7fc82406ed10 2023-10-06T11:37:20.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc82982e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82406cf80 con 0x7fc8240c6470 2023-10-06T11:37:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc82982e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82406d260 con 0x7fc82406e8a0 2023-10-06T11:37:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8237fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc82406ed10 0x7fc8240c2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45400/0 (socket says 172.21.15.46:45400) 2023-10-06T11:37:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8237fe700 1 -- 172.21.15.46:0/2544501820 learned_addr learned my addr 172.21.15.46:0/2544501820 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8227fc700 1 --2- 172.21.15.46:0/2544501820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 0x7fc8240c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8227fc700 1 -- 172.21.15.46:0/2544501820 >> v1:172.21.15.132:6789/0 conn(0x7fc8240c6470 legacy=0x7fc8240ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8227fc700 1 -- 172.21.15.46:0/2544501820 >> v1:172.21.15.46:6789/0 conn(0x7fc82406ed10 legacy=0x7fc8240c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8227fc700 1 -- 172.21.15.46:0/2544501820 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240c1880 con 0x7fc82406e8a0 2023-10-06T11:37:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc821ffb700 1 -- 172.21.15.46:0/2544501820 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546588046 0 0) 0x7fc8240c1bf0 con 0x7fc82406ed10 2023-10-06T11:37:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.184+0000 7fc8227fc700 1 --2- 172.21.15.46:0/2544501820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 0x7fc8240c6840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc818009910 tx=0x7fc8180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467d4be6fe1074ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc821ffb700 1 -- 172.21.15.46:0/2544501820 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800b860 con 0x7fc82406e8a0 2023-10-06T11:37:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc821ffb700 1 -- 172.21.15.46:0/2544501820 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800b9c0 con 0x7fc82406e8a0 2023-10-06T11:37:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc821ffb700 1 -- 172.21.15.46:0/2544501820 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800bd60 con 0x7fc82406e8a0 2023-10-06T11:37:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 -- 172.21.15.46:0/2544501820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 msgr2=0x7fc8240c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 --2- 172.21.15.46:0/2544501820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 0x7fc8240c6840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc818009910 tx=0x7fc8180091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 -- 172.21.15.46:0/2544501820 shutdown_connections 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 --2- 172.21.15.46:0/2544501820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406e8a0 0x7fc8240c6840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 -- 172.21.15.46:0/2544501820 >> 172.21.15.46:0/2544501820 conn(0x7fc8240cbe30 msgr2=0x7fc8240cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 -- 172.21.15.46:0/2544501820 shutdown_connections 2023-10-06T11:37:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.188+0000 7fc82982e700 1 -- 172.21.15.46:0/2544501820 wait complete. 2023-10-06T11:37:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 Processor -- start 2023-10-06T11:37:20.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 -- start start 2023-10-06T11:37:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 0x7fc82415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406ed10 0x7fc8240c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 0x7fc824069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c52c0 con 0x7fc8240c6470 2023-10-06T11:37:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c4220 con 0x7fc82406e8a0 2023-10-06T11:37:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc82982e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c4500 con 0x7fc82406ed10 2023-10-06T11:37:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 0x7fc824069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 0x7fc824069da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46758/0 (socket says 172.21.15.46:46758) 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc8237fe700 1 -- 172.21.15.46:0/939015764 learned_addr learned my addr 172.21.15.46:0/939015764 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc822ffd700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 0x7fc82415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.192+0000 7fc8227fc700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406ed10 0x7fc8240c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 -- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406ed10 msgr2=0x7fc8240c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406ed10 0x7fc8240c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 -- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 msgr2=0x7fc824069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 0x7fc824069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:20.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 -- 172.21.15.46:0/939015764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009420 con 0x7fc82406e8a0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc822ffd700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 0x7fc82415ca10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c940 tx=0x7fc8140016f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814012630 con 0x7fc82406e8a0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140127b0 con 0x7fc82406e8a0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814012b70 con 0x7fc82406e8a0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240c47e0 con 0x7fc82406e8a0 2023-10-06T11:37:20.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.196+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240c4da0 con 0x7fc82406e8a0 2023-10-06T11:37:20.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.200+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240660f0 con 0x7fc82406e8a0 2023-10-06T11:37:20.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.200+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc814003030 con 0x7fc82406e8a0 2023-10-06T11:37:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.200+0000 7fc80affd700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 0x7fc810062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.200+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6048+0+0 (secure 0 0 0) 0x7fc81407c9c0 con 0x7fc82406e8a0 2023-10-06T11:37:20.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.200+0000 7fc8227fc700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 0x7fc810062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:20.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.204+0000 7fc8227fc700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 0x7fc810062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8180093f0 tx=0x7fc818017000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.208+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc81404ccd0 con 0x7fc82406e8a0 2023-10-06T11:37:20.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:20.379+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc82404b5c0 con 0x7fc82406e8a0 2023-10-06T11:37:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.011+0000 7fc80affd700 1 -- 172.21.15.46:0/939015764 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc8140502f0 con 0x7fc82406e8a0 2023-10-06T11:37:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 msgr2=0x7fc810062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 0x7fc810062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8180093f0 tx=0x7fc818017000 comp rx=0 tx=0).stop 2023-10-06T11:37:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 msgr2=0x7fc82415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 0x7fc82415ca10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c940 tx=0x7fc8140016f0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 shutdown_connections 2023-10-06T11:37:21.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc810060670 0x7fc810062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8240c6470 0x7fc824069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc82406ed10 0x7fc8240c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 --2- 172.21.15.46:0/939015764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc82406e8a0 0x7fc82415ca10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 >> 172.21.15.46:0/939015764 conn(0x7fc8240cbe30 msgr2=0x7fc82415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.023+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 shutdown_connections 2023-10-06T11:37:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.027+0000 7fc82982e700 1 -- 172.21.15.46:0/939015764 wait complete. 2023-10-06T11:37:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T11:37:21.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:37:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.211+0000 7f9cfdee3700 1 Processor -- start 2023-10-06T11:37:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.211+0000 7f9cfdee3700 1 -- start start 2023-10-06T11:37:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfdee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9cf80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfdee3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9cf815b2b0 0x7f9cf80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfdee3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80b04e0 con 0x7f9cf80ae920 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfdee3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf815b6f0 con 0x7f9cf815b2b0 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfdee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80af2b0 con 0x7f9cf80abc40 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9cf80ae920 0x7f9cf815aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45410/0 (socket says 172.21.15.46:45410) 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf7fff700 1 -- 172.21.15.46:0/1654129834 learned_addr learned my addr 172.21.15.46:0/1654129834 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfd6e2700 1 --2- 172.21.15.46:0/1654129834 >> v2:172.21.15.132:3300/0 conn(0x7f9cf815b2b0 0x7f9cf80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9cf80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057386201 0 0) 0x7f9cf80b04e0 con 0x7f9cf80ae920 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce8003570 con 0x7f9cf80ae920 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347686736 0 0) 0x7f9ce8003570 con 0x7f9cf80ae920 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 msgr2=0x7f9cf80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9cf80b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 >> v2:172.21.15.132:3300/0 conn(0x7f9cf815b2b0 msgr2=0x7f9cf80aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 --2- 172.21.15.46:0/1654129834 >> v2:172.21.15.132:3300/0 conn(0x7f9cf815b2b0 0x7f9cf80aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80afbb0 con 0x7f9cf80ae920 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.215+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f9ce4001470 con 0x7f9cf80ae920 2023-10-06T11:37:21.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 >> v1:172.21.15.46:6789/0 conn(0x7f9cf80ae920 legacy=0x7f9cf815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9ce8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8004d50 0x7f9ce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815b6f0 con 0x7f9ce8004d50 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80af2b0 con 0x7f9ce8003a60 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce800ba60 con 0x7f9cf80abc40 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfd6e2700 1 --2- 172.21.15.46:0/1654129834 >> v2:172.21.15.132:3300/0 conn(0x7f9cf815b2b0 0x7f9cf80aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfd6e2700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8004d50 0x7f9ce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf7fff700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9ce8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 -- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 msgr2=0x7f9ce8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf80abc40 0x7f9ce8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 -- 172.21.15.46:0/1654129834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8004d50 msgr2=0x7f9ce80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8004d50 0x7f9ce80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 -- 172.21.15.46:0/1654129834 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8003570 con 0x7f9ce8003a60 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009c00 tx=0x7f9cec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ccbe8ba8b0b1160 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e710 con 0x7f9ce8003a60 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00e870 con 0x7f9ce8003a60 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cf77fe700 1 -- 172.21.15.46:0/1654129834 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00ec30 con 0x7f9ce8003a60 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 -- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8003a60 msgr2=0x7f9ce8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/1654129834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce8003a60 0x7f9ce8003e70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009c00 tx=0x7f9cec0091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfd6e2700 1 -- 172.21.15.46:0/1654129834 reap_dead start 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 -- 172.21.15.46:0/1654129834 shutdown_connections 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 -- 172.21.15.46:0/1654129834 >> 172.21.15.46:0/1654129834 conn(0x7f9cf80ccd50 msgr2=0x7f9cf80cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 -- 172.21.15.46:0/1654129834 shutdown_connections 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.219+0000 7f9cfdee3700 1 -- 172.21.15.46:0/1654129834 wait complete. 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 Processor -- start 2023-10-06T11:37:21.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 -- start start 2023-10-06T11:37:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cf81509c0 0x7f9cf8158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf814f3b0 0x7f9cf814f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8151ef0 con 0x7f9ce8003a60 2023-10-06T11:37:21.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf814fd70 con 0x7f9cf81509c0 2023-10-06T11:37:21.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfdee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf8151a60 con 0x7f9cf814f3b0 2023-10-06T11:37:21.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfcee1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfcee1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46780/0 (socket says 172.21.15.46:46780) 2023-10-06T11:37:21.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfcee1700 1 -- 172.21.15.46:0/3269519684 learned_addr learned my addr 172.21.15.46:0/3269519684 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cf7fff700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cf81509c0 0x7f9cf8158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.223+0000 7f9cfd6e2700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf814f3b0 0x7f9cf814f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 -- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf814f3b0 msgr2=0x7f9cf814f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf814f3b0 0x7f9cf814f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 -- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cf81509c0 msgr2=0x7f9cf8158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cf81509c0 0x7f9cf8158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 -- 172.21.15.46:0/3269519684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf81684d0 con 0x7f9ce8003a60 2023-10-06T11:37:21.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfcee1700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f9cec013ef0 tx=0x7f9cec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec022070 con 0x7f9ce8003a60 2023-10-06T11:37:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00e9e0 con 0x7f9ce8003a60 2023-10-06T11:37:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e710 con 0x7f9ce8003a60 2023-10-06T11:37:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf8159fc0 con 0x7f9ce8003a60 2023-10-06T11:37:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf8151130 con 0x7f9ce8003a60 2023-10-06T11:37:21.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.227+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf8064590 con 0x7f9ce8003a60 2023-10-06T11:37:21.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.231+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9cec01c450 con 0x7f9ce8003a60 2023-10-06T11:37:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.231+0000 7f9cf4ff9700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 0x7f9cdc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:21.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.231+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6430+0+0 (secure 0 0 0) 0x7f9cec07e920 con 0x7f9ce8003a60 2023-10-06T11:37:21.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.231+0000 7f9cf7fff700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 0x7f9cdc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:21.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.231+0000 7f9cf7fff700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 0x7f9cdc062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4001620 tx=0x7f9ce4001e90 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.235+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9cec04f100 con 0x7f9ce8003a60 2023-10-06T11:37:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:21.443+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9cf80cb690 con 0x7f9ce8003a60 2023-10-06T11:37:22.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.019+0000 7f9cf4ff9700 1 -- 172.21.15.46:0/3269519684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9cec052130 con 0x7f9ce8003a60 2023-10-06T11:37:22.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.035+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 msgr2=0x7f9cdc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.035+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 0x7f9cdc062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4001620 tx=0x7f9ce4001e90 comp rx=0 tx=0).stop 2023-10-06T11:37:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.035+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 msgr2=0x7f9cf8167f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.035+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f9cec013ef0 tx=0x7f9cec01a000 comp rx=0 tx=0).stop 2023-10-06T11:37:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 shutdown_connections 2023-10-06T11:37:22.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9cdc060670 0x7f9cdc062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cf814f3b0 0x7f9cf814f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cf81509c0 0x7f9cf8158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 --2- 172.21.15.46:0/3269519684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce8003a60 0x7f9cf8167f90 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 >> 172.21.15.46:0/3269519684 conn(0x7f9cf80ccd50 msgr2=0x7f9cf80cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 shutdown_connections 2023-10-06T11:37:22.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.039+0000 7f9cfdee3700 1 -- 172.21.15.46:0/3269519684 wait complete. 2023-10-06T11:37:22.063 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T11:37:22.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 Processor -- start 2023-10-06T11:37:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 -- start start 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b8159290 0x7f12b80b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12b80d07b0 0x7f12b80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12b80b1130 0x7f12b80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80b15b0 con 0x7f12b80d07b0 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80ae340 con 0x7f12b80b1130 2023-10-06T11:37:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bf1cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80b09a0 con 0x7f12b8159290 2023-10-06T11:37:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bd9c9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12b80d07b0 0x7f12b80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bd9c9700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12b80d07b0 0x7f12b80b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46792/0 (socket says 172.21.15.46:46792) 2023-10-06T11:37:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.227+0000 7f12bd9c9700 1 -- 172.21.15.46:0/3365436209 learned_addr learned my addr 172.21.15.46:0/3365436209 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12be9cb700 1 --2- 172.21.15.46:0/3365436209 >> v2:172.21.15.132:3300/0 conn(0x7f12b80b1130 0x7f12b80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12be1ca700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b8159290 0x7f12b80b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 -- 172.21.15.46:0/3365436209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b8159290 msgr2=0x7f12b80b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b8159290 0x7f12b80b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 -- 172.21.15.46:0/3365436209 >> v2:172.21.15.132:3300/0 conn(0x7f12b80b1130 msgr2=0x7f12b80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/3365436209 >> v2:172.21.15.132:3300/0 conn(0x7f12b80b1130 0x7f12b80ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 -- 172.21.15.46:0/3365436209 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8159660 con 0x7f12b80d07b0 2023-10-06T11:37:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80d07b0 0x7f12b80b0bf0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cffd0 tx=0x7f12a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6ced9beb175721 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd1c8700 1 -- 172.21.15.46:0/3365436209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800b6d0 con 0x7f12b80d07b0 2023-10-06T11:37:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd1c8700 1 -- 172.21.15.46:0/3365436209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a800b830 con 0x7f12b80d07b0 2023-10-06T11:37:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bd1c8700 1 -- 172.21.15.46:0/3365436209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800bbf0 con 0x7f12b80d07b0 2023-10-06T11:37:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 -- 172.21.15.46:0/3365436209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80d07b0 msgr2=0x7f12b80b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80d07b0 0x7f12b80b0bf0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cffd0 tx=0x7f12a8009250 comp rx=0 tx=0).stop 2023-10-06T11:37:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 -- 172.21.15.46:0/3365436209 shutdown_connections 2023-10-06T11:37:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/3365436209 >> v2:172.21.15.132:3300/0 conn(0x7f12b80b1130 0x7f12b80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80d07b0 0x7f12b80b0bf0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/3365436209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b8159290 0x7f12b80b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 -- 172.21.15.46:0/3365436209 >> 172.21.15.46:0/3365436209 conn(0x7f12b806ef60 msgr2=0x7f12b80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 -- 172.21.15.46:0/3365436209 shutdown_connections 2023-10-06T11:37:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.231+0000 7f12bf1cc700 1 -- 172.21.15.46:0/3365436209 wait complete. 2023-10-06T11:37:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 Processor -- start 2023-10-06T11:37:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 -- start start 2023-10-06T11:37:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 0x7f12b8154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 0x7f12b8140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8155170 con 0x7f12b80b1130 2023-10-06T11:37:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8141050 con 0x7f12b8159290 2023-10-06T11:37:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.235+0000 7f12bf1cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b816d0b0 con 0x7f12b80d07b0 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46796/0 (socket says 172.21.15.46:46796) 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 -- 172.21.15.46:0/1538924673 learned_addr learned my addr 172.21.15.46:0/1538924673 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be9cb700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 0x7f12b8140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 0x7f12b8154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 -- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 msgr2=0x7f12b8154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 0x7f12b8154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 -- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 msgr2=0x7f12b8140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 0x7f12b8140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 -- 172.21.15.46:0/1538924673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80094f0 con 0x7f12b80b1130 2023-10-06T11:37:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be9cb700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 0x7f12b8140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 0x7f12b8154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12be1ca700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f12b400ec80 tx=0x7f12b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b400c9d0 con 0x7f12b80b1130 2023-10-06T11:37:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b400cb50 con 0x7f12b80b1130 2023-10-06T11:37:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8152760 con 0x7f12b80b1130 2023-10-06T11:37:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4013470 con 0x7f12b80b1130 2023-10-06T11:37:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b816cd40 con 0x7f12b80b1130 2023-10-06T11:37:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.239+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b816cba0 con 0x7f12b80b1130 2023-10-06T11:37:22.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.243+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f12b4013860 con 0x7f12b80b1130 2023-10-06T11:37:22.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.243+0000 7f12ae7fc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 0x7f129c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.247+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f12b407d020 con 0x7f12b80b1130 2023-10-06T11:37:22.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.247+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 0x7f129c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.247+0000 7f12bd9c9700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 0x7f129c062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cffd0 tx=0x7f12a8017000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.251+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12b816cba0 con 0x7f12b80b1130 2023-10-06T11:37:22.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.431+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b816cba0 con 0x7f12b80b1130 2023-10-06T11:37:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12ae7fc700 1 -- 172.21.15.46:0/1538924673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17609 (secure 0 0 0) 0x7f12b816cba0 con 0x7f12b80b1130 2023-10-06T11:37:22.446 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:22.446 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":134,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:37:21.446095+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 msgr2=0x7f129c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 0x7f129c062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cffd0 tx=0x7f12a8017000 comp rx=0 tx=0).stop 2023-10-06T11:37:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 msgr2=0x7f12b81516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f12b400ec80 tx=0x7f12b40091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 shutdown_connections 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f129c060670 0x7f129c062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b8159290 0x7f12b8140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b80d07b0 0x7f12b8154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 --2- 172.21.15.46:0/1538924673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80b1130 0x7f12b81516d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 >> 172.21.15.46:0/1538924673 conn(0x7f12b806ef60 msgr2=0x7f12b80acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 shutdown_connections 2023-10-06T11:37:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.435+0000 7f12bf1cc700 1 -- 172.21.15.46:0/1538924673 wait complete. 2023-10-06T11:37:22.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:22.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 Processor -- start 2023-10-06T11:37:22.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 -- start start 2023-10-06T11:37:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f860815ab60 0x7f860806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 0x7f86080adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860806d460 con 0x7f860815af30 2023-10-06T11:37:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080c7000 con 0x7f86080ad6c0 2023-10-06T11:37:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f86107c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080c72c0 con 0x7f860815ab60 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.627+0000 7f860ed62700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 0x7f86080adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f860815ab60 0x7f860806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f860815ab60 0x7f860806e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43320/0 (socket says 172.21.15.46:43320) 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860ed62700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 0x7f86080adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46800/0 (socket says 172.21.15.46:46800) 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 -- 172.21.15.46:0/614086666 learned_addr learned my addr 172.21.15.46:0/614086666 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 -- 172.21.15.46:0/614086666 >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 msgr2=0x7f86080adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 --2- 172.21.15.46:0/614086666 >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 0x7f86080adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 -- 172.21.15.46:0/614086666 >> v1:172.21.15.132:6789/0 conn(0x7f860815af30 legacy=0x7f860806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 -- 172.21.15.46:0/614086666 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080c75a0 con 0x7f860815ab60 2023-10-06T11:37:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860d55f700 1 -- 172.21.15.46:0/614086666 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34540200 0 0) 0x7f860806d460 con 0x7f860815af30 2023-10-06T11:37:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860e561700 1 --2- 172.21.15.46:0/614086666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f860815ab60 0x7f860806e8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8608063be0 tx=0x7f8604009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b5390e50959ef4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860d55f700 1 -- 172.21.15.46:0/614086666 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604005c10 con 0x7f860815ab60 2023-10-06T11:37:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860d55f700 1 -- 172.21.15.46:0/614086666 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8604005d70 con 0x7f860815ab60 2023-10-06T11:37:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f860d55f700 1 -- 172.21.15.46:0/614086666 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604006260 con 0x7f860815ab60 2023-10-06T11:37:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 -- 172.21.15.46:0/614086666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f860815ab60 msgr2=0x7f860806e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 --2- 172.21.15.46:0/614086666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f860815ab60 0x7f860806e8a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8608063be0 tx=0x7f8604009220 comp rx=0 tx=0).stop 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 -- 172.21.15.46:0/614086666 shutdown_connections 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 --2- 172.21.15.46:0/614086666 >> v2:172.21.15.46:3300/0 conn(0x7f86080ad6c0 0x7f86080adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 --2- 172.21.15.46:0/614086666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f860815ab60 0x7f860806e8a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 -- 172.21.15.46:0/614086666 >> 172.21.15.46:0/614086666 conn(0x7f86080cbe30 msgr2=0x7f86080cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 -- 172.21.15.46:0/614086666 shutdown_connections 2023-10-06T11:37:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.631+0000 7f86107c5700 1 -- 172.21.15.46:0/614086666 wait complete. 2023-10-06T11:37:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 Processor -- start 2023-10-06T11:37:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 -- start start 2023-10-06T11:37:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86080ad6c0 0x7f8608150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f860815ab60 0x7f86080c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c5310 con 0x7f860815af30 2023-10-06T11:37:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c55f0 con 0x7f86080ad6c0 2023-10-06T11:37:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.635+0000 7f86107c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080ca240 con 0x7f860815ab60 2023-10-06T11:37:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46816/0 (socket says 172.21.15.46:46816) 2023-10-06T11:37:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 -- 172.21.15.46:0/3451850428 learned_addr learned my addr 172.21.15.46:0/3451850428 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860e561700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86080ad6c0 0x7f8608150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860dd60700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f860815ab60 0x7f86080c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 -- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f860815ab60 msgr2=0x7f86080c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f860815ab60 0x7f86080c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 -- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86080ad6c0 msgr2=0x7f8608150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86080ad6c0 0x7f8608150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 -- 172.21.15.46:0/3451850428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009480 con 0x7f860815af30 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f860ed62700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8600009df0 tx=0x7f860000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860000caa0 con 0x7f860815af30 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860000f040 con 0x7f860815af30 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860000ce70 con 0x7f860815af30 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86080ca520 con 0x7f860815af30 2023-10-06T11:37:22.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.639+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860816ddf0 con 0x7f860815af30 2023-10-06T11:37:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.643+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86080ca0a0 con 0x7f860815af30 2023-10-06T11:37:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.643+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8600014690 con 0x7f860815af30 2023-10-06T11:37:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.643+0000 7f85fe7fc700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 0x7f85ec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.643+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f860007d6e0 con 0x7f860815af30 2023-10-06T11:37:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.647+0000 7f860e561700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 0x7f85ec062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.647+0000 7f860e561700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 0x7f85ec062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8604009260 tx=0x7f8604016000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:22.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.651+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f86080ca0a0 con 0x7f860815af30 2023-10-06T11:37:22.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.827+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86080ca0a0 con 0x7f860815af30 2023-10-06T11:37:22.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.827+0000 7f85fe7fc700 1 -- 172.21.15.46:0/3451850428 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17609 (secure 0 0 0) 0x7f86080ca0a0 con 0x7f860815af30 2023-10-06T11:37:22.852 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:22.852 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":134,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:37:21.446095+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 msgr2=0x7f85ec062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 0x7f85ec062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8604009260 tx=0x7f8604016000 comp rx=0 tx=0).stop 2023-10-06T11:37:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 msgr2=0x7f86081713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8600009df0 tx=0x7f860000b040 comp rx=0 tx=0).stop 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 shutdown_connections 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f85ec060670 0x7f85ec062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f860815af30 0x7f86081713f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f860815ab60 0x7f86080c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 --2- 172.21.15.46:0/3451850428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86080ad6c0 0x7f8608150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 >> 172.21.15.46:0/3451850428 conn(0x7f86080cbe30 msgr2=0x7f86080adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 shutdown_connections 2023-10-06T11:37:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:22.831+0000 7f86107c5700 1 -- 172.21.15.46:0/3451850428 wait complete. 2023-10-06T11:37:22.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:37:22.980 INFO:tasks.ceph.mds.c.smithi046.stderr:2023-10-06T11:37:22.967+0000 7f87d8944700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:37:23.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.015+0000 7f5c4b20c700 1 Processor -- start 2023-10-06T11:37:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.015+0000 7f5c4b20c700 1 -- start start 2023-10-06T11:37:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4b20c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c440b14d0 0x7f5c440adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4b20c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440afd20 con 0x7f5c440adbe0 2023-10-06T11:37:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4b20c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440b0000 con 0x7f5c440b30f0 2023-10-06T11:37:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4b20c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c44159740 con 0x7f5c440b14d0 2023-10-06T11:37:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4aa0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c440b14d0 0x7f5c440adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c49a09700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f5c440b30f0 0x7f5c440b0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:53460/0 (socket says 172.21.15.46:53460) 2023-10-06T11:37:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c49a09700 1 -- 172.21.15.46:0/2738759230 learned_addr learned my addr 172.21.15.46:0/2738759230 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c49208700 1 -- 172.21.15.46:0/2738759230 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705999019 0 0) 0x7f5c440b0000 con 0x7f5c440b30f0 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c49208700 1 -- 172.21.15.46:0/2738759230 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c30003570 con 0x7f5c440b30f0 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4aa0b700 1 -- 172.21.15.46:0/2738759230 >> v1:172.21.15.132:6789/0 conn(0x7f5c440b30f0 legacy=0x7f5c440b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4aa0b700 1 -- 172.21.15.46:0/2738759230 >> v1:172.21.15.46:6789/0 conn(0x7f5c440adbe0 legacy=0x7f5c440d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.019+0000 7f5c4aa0b700 1 -- 172.21.15.46:0/2738759230 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440b02e0 con 0x7f5c440b14d0 2023-10-06T11:37:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4aa0b700 1 --2- 172.21.15.46:0/2738759230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440b14d0 0x7f5c440adfb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009e60 tx=0x7f5c3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4204387f3e5d973 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c49208700 1 -- 172.21.15.46:0/2738759230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00e070 con 0x7f5c440b14d0 2023-10-06T11:37:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c49208700 1 -- 172.21.15.46:0/2738759230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00bb60 con 0x7f5c440b14d0 2023-10-06T11:37:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c49208700 1 -- 172.21.15.46:0/2738759230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c012430 con 0x7f5c440b14d0 2023-10-06T11:37:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 -- 172.21.15.46:0/2738759230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440b14d0 msgr2=0x7f5c440adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/2738759230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440b14d0 0x7f5c440adfb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009e60 tx=0x7f5c3c009780 comp rx=0 tx=0).stop 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 -- 172.21.15.46:0/2738759230 shutdown_connections 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/2738759230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440b14d0 0x7f5c440adfb0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 -- 172.21.15.46:0/2738759230 >> 172.21.15.46:0/2738759230 conn(0x7f5c4406ef60 msgr2=0x7f5c440d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 -- 172.21.15.46:0/2738759230 shutdown_connections 2023-10-06T11:37:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.023+0000 7f5c4b20c700 1 -- 172.21.15.46:0/2738759230 wait complete. 2023-10-06T11:37:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 Processor -- start 2023-10-06T11:37:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 -- start start 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c440b14d0 0x7f5c4415b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 0x7f5c44169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44155e70 con 0x7f5c440adbe0 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4409d330 con 0x7f5c440b14d0 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44169870 con 0x7f5c440b30f0 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46826/0 (socket says 172.21.15.46:46826) 2023-10-06T11:37:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 -- 172.21.15.46:0/4005355891 learned_addr learned my addr 172.21.15.46:0/4005355891 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c49a09700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c440b14d0 0x7f5c4415b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4aa0b700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 0x7f5c44169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 -- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 msgr2=0x7f5c44169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 0x7f5c44169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 -- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c440b14d0 msgr2=0x7f5c4415b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c440b14d0 0x7f5c4415b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 -- 172.21.15.46:0/4005355891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c009a20 con 0x7f5c440adbe0 2023-10-06T11:37:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4aa0b700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 0x7f5c44169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4a20a700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c4e0 tx=0x7f5c4000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40011590 con 0x7f5c440adbe0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c40011710 con 0x7f5c440adbe0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4409db20 con 0x7f5c440adbe0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.027+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40011ad0 con 0x7f5c440adbe0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.031+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4409d560 con 0x7f5c440adbe0 2023-10-06T11:37:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.031+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c44064590 con 0x7f5c440adbe0 2023-10-06T11:37:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.031+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5c4001b3b0 con 0x7f5c440adbe0 2023-10-06T11:37:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.035+0000 7f5c3a7fc700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 0x7f5c28062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.035+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f5c4007c1f0 con 0x7f5c440adbe0 2023-10-06T11:37:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.035+0000 7f5c49a09700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 0x7f5c28062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.035+0000 7f5c49a09700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 0x7f5c28062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cf650 tx=0x7f5c34009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.039+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c4004c2f0 con 0x7f5c440adbe0 2023-10-06T11:37:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:23.247+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5c440cf6d0 con 0x7f5c440adbe0 2023-10-06T11:37:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.059+0000 7f5c3a7fc700 1 -- 172.21.15.46:0/4005355891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5c4004f910 con 0x7f5c440adbe0 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 msgr2=0x7f5c28062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 0x7f5c28062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cf650 tx=0x7f5c34009040 comp rx=0 tx=0).stop 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 msgr2=0x7f5c44155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c4e0 tx=0x7f5c4000c7f0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 shutdown_connections 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c28060670 0x7f5c28062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c440b30f0 0x7f5c44169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c440b14d0 0x7f5c4415b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 --2- 172.21.15.46:0/4005355891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c440adbe0 0x7f5c44155930 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 >> 172.21.15.46:0/4005355891 conn(0x7f5c4406ef60 msgr2=0x7f5c440693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 shutdown_connections 2023-10-06T11:37:24.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.063+0000 7f5c4b20c700 1 -- 172.21.15.46:0/4005355891 wait complete. 2023-10-06T11:37:24.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:37:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.255+0000 7fd8b52e3700 1 Processor -- start 2023-10-06T11:37:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.255+0000 7fd8b52e3700 1 -- start start 2023-10-06T11:37:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b52e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b0154c50 0x7fd8b00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b52e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b00c7f20 0x7fd8b015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b52e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b00c6cf0 con 0x7fd8b0154880 2023-10-06T11:37:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b52e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b00c2a30 con 0x7fd8b00c7f20 2023-10-06T11:37:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b52e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00c83e0 con 0x7fd8b0154c50 2023-10-06T11:37:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b00c7f20 0x7fd8b015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8b00c7f20 0x7fd8b015aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47544/0 (socket says 172.21.15.46:47544) 2023-10-06T11:37:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 -- 172.21.15.46:0/120510635 learned_addr learned my addr 172.21.15.46:0/120510635 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8a77fe700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b0154c50 0x7fd8b00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 -- 172.21.15.46:0/120510635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b0154c50 msgr2=0x7fd8b00c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b0154c50 0x7fd8b00c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 -- 172.21.15.46:0/120510635 >> v1:172.21.15.132:6789/0 conn(0x7fd8b0154880 legacy=0x7fd8b006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8af7fe700 1 -- 172.21.15.46:0/120510635 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786491579 0 0) 0x7fd8b00c6cf0 con 0x7fd8b0154880 2023-10-06T11:37:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 -- 172.21.15.46:0/120510635 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b006e5d0 con 0x7fd8b00c7f20 2023-10-06T11:37:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8b4ae2700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b00c7f20 0x7fd8b015aa30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009f20 tx=0x7fd8a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d58b6622133952 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.259+0000 7fd8af7fe700 1 -- 172.21.15.46:0/120510635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800e070 con 0x7fd8b00c7f20 2023-10-06T11:37:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8af7fe700 1 -- 172.21.15.46:0/120510635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800b9d0 con 0x7fd8b00c7f20 2023-10-06T11:37:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8af7fe700 1 -- 172.21.15.46:0/120510635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80092e0 con 0x7fd8b00c7f20 2023-10-06T11:37:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 -- 172.21.15.46:0/120510635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b00c7f20 msgr2=0x7fd8b015aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b00c7f20 0x7fd8b015aa30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009f20 tx=0x7fd8a8009840 comp rx=0 tx=0).stop 2023-10-06T11:37:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 -- 172.21.15.46:0/120510635 shutdown_connections 2023-10-06T11:37:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b00c7f20 0x7fd8b015aa30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/120510635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b0154c50 0x7fd8b00c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 -- 172.21.15.46:0/120510635 >> 172.21.15.46:0/120510635 conn(0x7fd8b00cbc10 msgr2=0x7fd8b00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 -- 172.21.15.46:0/120510635 shutdown_connections 2023-10-06T11:37:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.263+0000 7fd8b52e3700 1 -- 172.21.15.46:0/120510635 wait complete. 2023-10-06T11:37:24.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 Processor -- start 2023-10-06T11:37:24.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 -- start start 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b00c7f20 0x7fd8b01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b0154c50 0x7fd8b00c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0153f10 con 0x7fd8b0154880 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00c3a60 con 0x7fd8b0154c50 2023-10-06T11:37:24.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b52e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00c3d40 con 0x7fd8b00c7f20 2023-10-06T11:37:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47548/0 (socket says 172.21.15.46:47548) 2023-10-06T11:37:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3339270576 learned_addr learned my addr 172.21.15.46:0/3339270576 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8affff700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b00c7f20 0x7fd8b01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8b4ae2700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b0154c50 0x7fd8b00c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b00c7f20 msgr2=0x7fd8b01539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b00c7f20 0x7fd8b01539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b0154c50 msgr2=0x7fd8b00c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b0154c50 0x7fd8b00c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.267+0000 7fd8a77fe700 1 -- 172.21.15.46:0/3339270576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009ae0 con 0x7fd8b0154880 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8a77fe700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e900 tx=0x7fd8980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898013680 con 0x7fd8b0154880 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8980137e0 con 0x7fd8b0154880 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b00c4020 con 0x7fd8b0154880 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898013ba0 con 0x7fd8b0154880 2023-10-06T11:37:24.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b00c45e0 con 0x7fd8b0154880 2023-10-06T11:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.271+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0064590 con 0x7fd8b0154880 2023-10-06T11:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.275+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd8980123d0 con 0x7fd8b0154880 2023-10-06T11:37:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.275+0000 7fd8acff9700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 0x7fd890062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.275+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fd89807cdb0 con 0x7fd8b0154880 2023-10-06T11:37:24.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.275+0000 7fd8affff700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 0x7fd890062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:24.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.275+0000 7fd8affff700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 0x7fd890062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007fd0 tx=0x7fd8a0007d30 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:24.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.279+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd898081070 con 0x7fd8b0154880 2023-10-06T11:37:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:24.487+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8b006e8b0 con 0x7fd8b0154880 2023-10-06T11:37:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.071+0000 7fd8acff9700 1 -- 172.21.15.46:0/3339270576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8980170c0 con 0x7fd8b0154880 2023-10-06T11:37:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 msgr2=0x7fd890062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 0x7fd890062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0007fd0 tx=0x7fd8a0007d30 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 msgr2=0x7fd8b00c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e900 tx=0x7fd8980091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 shutdown_connections 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd890060670 0x7fd890062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8b0154c50 0x7fd8b00c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8b0154880 0x7fd8b00c2fe0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 --2- 172.21.15.46:0/3339270576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8b00c7f20 0x7fd8b01539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 >> 172.21.15.46:0/3339270576 conn(0x7fd8b00cbc10 msgr2=0x7fd8b00af210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 shutdown_connections 2023-10-06T11:37:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.075+0000 7fd8b52e3700 1 -- 172.21.15.46:0/3339270576 wait complete. 2023-10-06T11:37:25.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:25.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.267+0000 7f0ce2d23700 1 Processor -- start 2023-10-06T11:37:25.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.267+0000 7f0ce2d23700 1 -- start start 2023-10-06T11:37:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce2d23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cdc0af000 0x7f0cdc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce2d23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0cdc0cb4c0 0x7f0cdc0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce2d23700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc0cf100 con 0x7f0cdc0af3d0 2023-10-06T11:37:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce2d23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc0ce4f0 con 0x7f0cdc0af000 2023-10-06T11:37:25.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce2d23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc0ce7d0 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cdc0af000 0x7f0cdc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cdc0af000 0x7f0cdc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47560/0 (socket says 172.21.15.46:47560) 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 -- 172.21.15.46:0/4086911385 learned_addr learned my addr 172.21.15.46:0/4086911385 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1520700 1 --2- 172.21.15.46:0/4086911385 >> v2:172.21.15.132:3300/0 conn(0x7f0cdc0cb4c0 0x7f0cdc0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 -- 172.21.15.46:0/4086911385 >> v2:172.21.15.132:3300/0 conn(0x7f0cdc0cb4c0 msgr2=0x7f0cdc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 --2- 172.21.15.46:0/4086911385 >> v2:172.21.15.132:3300/0 conn(0x7f0cdc0cb4c0 0x7f0cdc0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 -- 172.21.15.46:0/4086911385 >> v1:172.21.15.132:6789/0 conn(0x7f0cdc0af3d0 legacy=0x7f0cdc0690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 -- 172.21.15.46:0/4086911385 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0ceab0 con 0x7f0cdc0af000 2023-10-06T11:37:25.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.271+0000 7f0ce1d21700 1 --2- 172.21.15.46:0/4086911385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0af000 0x7f0cdc0cf590 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc087fd0 tx=0x7f0cd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fec0d66851a045c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce0d1f700 1 -- 172.21.15.46:0/4086911385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800b820 con 0x7f0cdc0af000 2023-10-06T11:37:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce0d1f700 1 -- 172.21.15.46:0/4086911385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd800b980 con 0x7f0cdc0af000 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce0d1f700 1 -- 172.21.15.46:0/4086911385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800bd20 con 0x7f0cdc0af000 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 -- 172.21.15.46:0/4086911385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0af000 msgr2=0x7f0cdc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/4086911385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0af000 0x7f0cdc0cf590 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc087fd0 tx=0x7f0cd8009250 comp rx=0 tx=0).stop 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 -- 172.21.15.46:0/4086911385 shutdown_connections 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/4086911385 >> v2:172.21.15.132:3300/0 conn(0x7f0cdc0cb4c0 0x7f0cdc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/4086911385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0af000 0x7f0cdc0cf590 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 -- 172.21.15.46:0/4086911385 >> 172.21.15.46:0/4086911385 conn(0x7f0cdc0683b0 msgr2=0x7f0cdc0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 -- 172.21.15.46:0/4086911385 shutdown_connections 2023-10-06T11:37:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.275+0000 7f0ce2d23700 1 -- 172.21.15.46:0/4086911385 wait complete. 2023-10-06T11:37:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 Processor -- start 2023-10-06T11:37:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 -- start start 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cdc0af000 0x7f0cdc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 0x7f0cdc169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15c3d0 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc0c7670 con 0x7f0cdc0af000 2023-10-06T11:37:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2d23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15b470 con 0x7f0cdc0af3d0 2023-10-06T11:37:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47570/0 (socket says 172.21.15.46:47570) 2023-10-06T11:37:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 -- 172.21.15.46:0/3866827594 learned_addr learned my addr 172.21.15.46:0/3866827594 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce1d21700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cdc0af000 0x7f0cdc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce1520700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 0x7f0cdc169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 -- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 msgr2=0x7f0cdc169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 0x7f0cdc169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 -- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cdc0af000 msgr2=0x7f0cdc156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cdc0af000 0x7f0cdc156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 -- 172.21.15.46:0/3866827594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd80094f0 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce1520700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 0x7f0cdc169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0ce2522700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c960 tx=0x7f0cd40026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.279+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4012630 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4012790 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc15b750 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4012b50 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc157470 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc0c7160 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.283+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0cd401b400 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.287+0000 7f0cd1ffb700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 0x7f0cc0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.287+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f0cd407cfe0 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.287+0000 7f0ce1d21700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 0x7f0cc0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.287+0000 7f0ce1d21700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 0x7f0cc0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc087fd0 tx=0x7f0cd8016000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.291+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0cdc0c7160 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.495+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cdc0c7160 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.495+0000 7f0cd1ffb700 1 -- 172.21.15.46:0/3866827594 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0cdc0c7160 con 0x7f0cdc0cb4c0 2023-10-06T11:37:25.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:25.509 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T11:37:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 msgr2=0x7f0cc0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 0x7f0cc0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc087fd0 tx=0x7f0cd8016000 comp rx=0 tx=0).stop 2023-10-06T11:37:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 msgr2=0x7f0cdc169e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c960 tx=0x7f0cd40026e0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 shutdown_connections 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0cc0060670 0x7f0cc0062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cdc0cb4c0 0x7f0cdc169e70 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cdc0af3d0 0x7f0cdc169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 --2- 172.21.15.46:0/3866827594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cdc0af000 0x7f0cdc156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 >> 172.21.15.46:0/3866827594 conn(0x7f0cdc0683b0 msgr2=0x7f0cdc0ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 shutdown_connections 2023-10-06T11:37:25.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.499+0000 7f0ce2d23700 1 -- 172.21.15.46:0/3866827594 wait complete. 2023-10-06T11:37:25.515 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T11:37:25.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:37:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 Processor -- start 2023-10-06T11:37:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 -- start start 2023-10-06T11:37:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe7ac15ab60 0x7fe7ac06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac06d460 con 0x7fe7ac15af30 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c7000 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7b0f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c72c0 con 0x7fe7ac15ab60 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7aad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7aad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47580/0 (socket says 172.21.15.46:47580) 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3314522069 learned_addr learned my addr 172.21.15.46:0/3314522069 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:25.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.683+0000 7fe7aa59c700 1 --2- 172.21.15.46:0/3314522069 >> v2:172.21.15.132:3300/0 conn(0x7fe7ac15ab60 0x7fe7ac06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3314522069 >> v2:172.21.15.132:3300/0 conn(0x7fe7ac15ab60 msgr2=0x7fe7ac06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7aad9d700 1 --2- 172.21.15.46:0/3314522069 >> v2:172.21.15.132:3300/0 conn(0x7fe7ac15ab60 0x7fe7ac06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3314522069 >> v1:172.21.15.132:6789/0 conn(0x7fe7ac15af30 legacy=0x7fe7ac06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3314522069 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0c75a0 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7aad9d700 1 --2- 172.21.15.46:0/3314522069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac063be0 tx=0x7fe7a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4b38301582936a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7a959a700 1 -- 172.21.15.46:0/3314522069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400e070 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7a959a700 1 -- 172.21.15.46:0/3314522069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a400ba20 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7a959a700 1 -- 172.21.15.46:0/3314522069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40092e0 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3314522069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac0ad6c0 msgr2=0x7fe7ac0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3314522069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac063be0 tx=0x7fe7a4009840 comp rx=0 tx=0).stop 2023-10-06T11:37:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3314522069 shutdown_connections 2023-10-06T11:37:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3314522069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac0adb20 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3314522069 >> v2:172.21.15.132:3300/0 conn(0x7fe7ac15ab60 0x7fe7ac06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3314522069 >> 172.21.15.46:0/3314522069 conn(0x7fe7ac0cbe30 msgr2=0x7fe7ac0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:25.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3314522069 shutdown_connections 2023-10-06T11:37:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.687+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3314522069 wait complete. 2023-10-06T11:37:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 Processor -- start 2023-10-06T11:37:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 -- start start 2023-10-06T11:37:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7ac15ab60 0x7fe7ac0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0a98e0 con 0x7fe7ac15af30 2023-10-06T11:37:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac151070 con 0x7fe7ac0ad6c0 2023-10-06T11:37:25.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.691+0000 7fe7b0f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cb320 con 0x7fe7ac15ab60 2023-10-06T11:37:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47590/0 (socket says 172.21.15.46:47590) 2023-10-06T11:37:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3821679089 learned_addr learned my addr 172.21.15.46:0/3821679089 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aa59c700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7a9d9b700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7ac15ab60 0x7fe7ac0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7ac15ab60 msgr2=0x7fe7ac0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7ac15ab60 0x7fe7ac0c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ac0ad6c0 msgr2=0x7fe7ac150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 -- 172.21.15.46:0/3821679089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4009ae0 con 0x7fe7ac15af30 2023-10-06T11:37:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7aad9d700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009880 tx=0x7fe7a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4008be0 con 0x7fe7ac15af30 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a4008d40 con 0x7fe7ac15af30 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac0cb600 con 0x7fe7ac15af30 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a40256b0 con 0x7fe7ac15af30 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0a9e30 con 0x7fe7ac15af30 2023-10-06T11:37:25.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.695+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac0aa020 con 0x7fe7ac15af30 2023-10-06T11:37:25.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.699+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe7a4003040 con 0x7fe7ac15af30 2023-10-06T11:37:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.699+0000 7fe79a7fc700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 0x7fe78c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.699+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fe7a407eae0 con 0x7fe7ac15af30 2023-10-06T11:37:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.699+0000 7fe7aa59c700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 0x7fe78c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.703+0000 7fe7aa59c700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 0x7fe78c062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009d30 tx=0x7fe79c008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.707+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe7a404ead0 con 0x7fe7ac15af30 2023-10-06T11:37:25.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.879+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7ac063f90 con 0x7fe7ac15af30 2023-10-06T11:37:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.883+0000 7fe79a7fc700 1 -- 172.21.15.46:0/3821679089 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17596 (secure 0 0 0) 0x7fe7a40520f0 con 0x7fe7ac15af30 2023-10-06T11:37:25.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:25.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":135,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T11:37:23.002626+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 msgr2=0x7fe78c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 0x7fe78c062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009d30 tx=0x7fe79c008040 comp rx=0 tx=0).stop 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 msgr2=0x7fe7ac0c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009880 tx=0x7fe7a4018000 comp rx=0 tx=0).stop 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 shutdown_connections 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe78c060670 0x7fe78c062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7ac15af30 0x7fe7ac0c4960 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe7ac15ab60 0x7fe7ac0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 --2- 172.21.15.46:0/3821679089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7ac0ad6c0 0x7fe7ac150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 >> 172.21.15.46:0/3821679089 conn(0x7fe7ac0cbe30 msgr2=0x7fe7ac157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 shutdown_connections 2023-10-06T11:37:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:25.887+0000 7fe7b0f85700 1 -- 172.21.15.46:0/3821679089 wait complete. 2023-10-06T11:37:25.918 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:37:25.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:37:26.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.075+0000 7fededa8e700 1 Processor -- start 2023-10-06T11:37:26.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.075+0000 7fededa8e700 1 -- start start 2023-10-06T11:37:26.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fede8159290 0x7fede80b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede80ae340 con 0x7fede80d07b0 2023-10-06T11:37:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede80b09a0 con 0x7fede8159290 2023-10-06T11:37:26.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede8159660 con 0x7fede80b1300 2023-10-06T11:37:26.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fede8159290 0x7fede80b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fede7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fede80d07b0 0x7fede80b0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50996/0 (socket says 172.21.15.46:50996) 2023-10-06T11:37:26.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fede7fff700 1 -- 172.21.15.46:0/2154889830 learned_addr learned my addr 172.21.15.46:0/2154889830 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:26.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7feded28d700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 -- 172.21.15.46:0/2154889830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 msgr2=0x7fede80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede80ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 -- 172.21.15.46:0/2154889830 >> v1:172.21.15.46:6789/0 conn(0x7fede80d07b0 legacy=0x7fede80b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 -- 172.21.15.46:0/2154889830 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80b3450 con 0x7fede8159290 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7feded28d700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fedeca8c700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede80b2f10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232f927103b0cf0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fede77fe700 1 -- 172.21.15.46:0/2154889830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc00e070 con 0x7fede8159290 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fede77fe700 1 -- 172.21.15.46:0/2154889830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc00bcc0 con 0x7fede8159290 2023-10-06T11:37:26.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fede77fe700 1 -- 172.21.15.46:0/2154889830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc012680 con 0x7fede8159290 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 -- 172.21.15.46:0/2154889830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 msgr2=0x7fede80b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede80b2f10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e60 tx=0x7feddc009780 comp rx=0 tx=0).stop 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 -- 172.21.15.46:0/2154889830 shutdown_connections 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.079+0000 7fededa8e700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 --2- 172.21.15.46:0/2154889830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede80b2f10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- 172.21.15.46:0/2154889830 >> 172.21.15.46:0/2154889830 conn(0x7fede806ef60 msgr2=0x7fede80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- 172.21.15.46:0/2154889830 shutdown_connections 2023-10-06T11:37:26.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- 172.21.15.46:0/2154889830 wait complete. 2023-10-06T11:37:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 Processor -- start 2023-10-06T11:37:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- start start 2023-10-06T11:37:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 0x7fede8169490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80b39c0 con 0x7fede8159290 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede80b3f80 con 0x7fede80d07b0 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7fededa8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede8169f50 con 0x7fede80b1300 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7feded28d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7feded28d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47614/0 (socket says 172.21.15.46:47614) 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.083+0000 7feded28d700 1 -- 172.21.15.46:0/3679661419 learned_addr learned my addr 172.21.15.46:0/3679661419 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fedeca8c700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fede7fff700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 0x7fede8169490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 -- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 msgr2=0x7fede814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 -- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 msgr2=0x7fede8169490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 0x7fede8169490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 -- 172.21.15.46:0/3679661419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc009a20 con 0x7fede8159290 2023-10-06T11:37:26.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fede7fff700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 0x7fede8169490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7feded28d700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fede000e8d0 tx=0x7fede0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0013410 con 0x7fede8159290 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede0013570 con 0x7fede8159290 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede816a230 con 0x7fede8159290 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0013930 con 0x7fede8159290 2023-10-06T11:37:26.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede814a9f0 con 0x7fede8159290 2023-10-06T11:37:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.087+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede8064590 con 0x7fede8159290 2023-10-06T11:37:26.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.091+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fede0013ce0 con 0x7fede8159290 2023-10-06T11:37:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.091+0000 7fede4ff9700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 0x7fedcc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.091+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fede007caa0 con 0x7fede8159290 2023-10-06T11:37:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.091+0000 7fedeca8c700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 0x7fedcc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.091+0000 7fedeca8c700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 0x7fedcc062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feddc0097c0 tx=0x7feddc019000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.099+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fede004cb10 con 0x7fede8159290 2023-10-06T11:37:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.271+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fede80b06c0 con 0x7fedcc060670 2023-10-06T11:37:26.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.279+0000 7fede4ff9700 1 -- 172.21.15.46:0/3679661419 <== mgr.6956 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309325 (secure 0 0 0) 0x7fede80b06c0 con 0x7fedcc060670 2023-10-06T11:37:26.296 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 msgr2=0x7fedcc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 0x7fedcc062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feddc0097c0 tx=0x7feddc019000 comp rx=0 tx=0).stop 2023-10-06T11:37:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 msgr2=0x7fede8152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fede000e8d0 tx=0x7fede0008040 comp rx=0 tx=0).stop 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 shutdown_connections 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fedcc060670 0x7fedcc062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fede8159290 0x7fede8152780 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fede80d07b0 0x7fede8169490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 --2- 172.21.15.46:0/3679661419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fede80b1300 0x7fede814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 >> 172.21.15.46:0/3679661419 conn(0x7fede806ef60 msgr2=0x7fede814dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 shutdown_connections 2023-10-06T11:37:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.291+0000 7fededa8e700 1 -- 172.21.15.46:0/3679661419 wait complete. 2023-10-06T11:37:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:37:26.322 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:37:24.652274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2470416,"kb_used_data":14288,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752504304,"statfs":{"total":773094113280,"available":770564407296,"internally_reserved":0,"allocated":14630912,"data_stored":8147234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672910"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037361+0000","last_change":"2023-10-06T11:37:22.031626+0000","last_active":"2023-10-06T11:37:22.037361+0000","last_peered":"2023-10-06T11:37:22.037361+0000","last_clean":"2023-10-06T11:37:22.037361+0000","last_became_active":"2023-10-06T11:37:22.031316+0000","last_became_peered":"2023-10-06T11:37:22.031316+0000","last_unstale":"2023-10-06T11:37:22.037361+0000","last_undegraded":"2023-10-06T11:37:22.037361+0000","last_fullsized":"2023-10-06T11:37:22.037361+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:24.545371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.023338+0000","last_change":"2023-10-06T11:37:22.019305+0000","last_active":"2023-10-06T11:37:22.023338+0000","last_peered":"2023-10-06T11:37:22.023338+0000","last_clean":"2023-10-06T11:37:22.023338+0000","last_became_active":"2023-10-06T11:37:22.018984+0000","last_became_peered":"2023-10-06T11:37:22.018984+0000","last_unstale":"2023-10-06T11:37:22.023338+0000","last_undegraded":"2023-10-06T11:37:22.023338+0000","last_fullsized":"2023-10-06T11:37:22.023338+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.531278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052300+0000","last_change":"2023-10-06T11:37:22.047336+0000","last_active":"2023-10-06T11:37:22.052300+0000","last_peered":"2023-10-06T11:37:22.052300+0000","last_clean":"2023-10-06T11:37:22.052300+0000","last_became_active":"2023-10-06T11:37:22.047036+0000","last_became_peered":"2023-10-06T11:37:22.047036+0000","last_unstale":"2023-10-06T11:37:22.052300+0000","last_undegraded":"2023-10-06T11:37:22.052300+0000","last_fullsized":"2023-10-06T11:37:22.052300+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:47.029371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042308+0000","last_change":"2023-10-06T11:37:22.032093+0000","last_active":"2023-10-06T11:37:22.042308+0000","last_peered":"2023-10-06T11:37:22.042308+0000","last_clean":"2023-10-06T11:37:22.042308+0000","last_became_active":"2023-10-06T11:37:22.031761+0000","last_became_peered":"2023-10-06T11:37:22.031761+0000","last_unstale":"2023-10-06T11:37:22.042308+0000","last_undegraded":"2023-10-06T11:37:22.042308+0000","last_fullsized":"2023-10-06T11:37:22.042308+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042617+0000","last_change":"2023-10-06T11:37:22.041309+0000","last_active":"2023-10-06T11:37:22.042617+0000","last_peered":"2023-10-06T11:37:22.042617+0000","last_clean":"2023-10-06T11:37:22.042617+0000","last_became_active":"2023-10-06T11:37:22.040981+0000","last_became_peered":"2023-10-06T11:37:22.040981+0000","last_unstale":"2023-10-06T11:37:22.042617+0000","last_undegraded":"2023-10-06T11:37:22.042617+0000","last_fullsized":"2023-10-06T11:37:22.042617+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054162+0000","last_change":"2023-10-06T11:37:22.053329+0000","last_active":"2023-10-06T11:37:22.054162+0000","last_peered":"2023-10-06T11:37:22.054162+0000","last_clean":"2023-10-06T11:37:22.054162+0000","last_became_active":"2023-10-06T11:37:22.052882+0000","last_became_peered":"2023-10-06T11:37:22.052882+0000","last_unstale":"2023-10-06T11:37:22.054162+0000","last_undegraded":"2023-10-06T11:37:22.054162+0000","last_fullsized":"2023-10-06T11:37:22.054162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:26.113043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043095+0000","last_change":"2023-10-06T11:37:22.038395+0000","last_active":"2023-10-06T11:37:22.043095+0000","last_peered":"2023-10-06T11:37:22.043095+0000","last_clean":"2023-10-06T11:37:22.043095+0000","last_became_active":"2023-10-06T11:37:22.038101+0000","last_became_peered":"2023-10-06T11:37:22.038101+0000","last_unstale":"2023-10-06T11:37:22.043095+0000","last_undegraded":"2023-10-06T11:37:22.043095+0000","last_fullsized":"2023-10-06T11:37:22.043095+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:22.522026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054626+0000","last_change":"2023-10-06T11:37:22.049679+0000","last_active":"2023-10-06T11:37:22.054626+0000","last_peered":"2023-10-06T11:37:22.054626+0000","last_clean":"2023-10-06T11:37:22.054626+0000","last_became_active":"2023-10-06T11:37:22.049343+0000","last_became_peered":"2023-10-06T11:37:22.049343+0000","last_unstale":"2023-10-06T11:37:22.054626+0000","last_undegraded":"2023-10-06T11:37:22.054626+0000","last_fullsized":"2023-10-06T11:37:22.054626+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:00.137182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.041996+0000","last_change":"2023-10-06T11:37:22.029546+0000","last_active":"2023-10-06T11:37:22.041996+0000","last_peered":"2023-10-06T11:37:22.041996+0000","last_clean":"2023-10-06T11:37:22.041996+0000","last_became_active":"2023-10-06T11:37:22.029249+0000","last_became_peered":"2023-10-06T11:37:22.029249+0000","last_unstale":"2023-10-06T11:37:22.041996+0000","last_undegraded":"2023-10-06T11:37:22.041996+0000","last_fullsized":"2023-10-06T11:37:22.041996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:14.305263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038238+0000","last_change":"2023-10-06T11:37:22.034103+0000","last_active":"2023-10-06T11:37:22.038238+0000","last_peered":"2023-10-06T11:37:22.038238+0000","last_clean":"2023-10-06T11:37:22.038238+0000","last_became_active":"2023-10-06T11:37:22.032705+0000","last_became_peered":"2023-10-06T11:37:22.032705+0000","last_unstale":"2023-10-06T11:37:22.038238+0000","last_undegraded":"2023-10-06T11:37:22.038238+0000","last_fullsized":"2023-10-06T11:37:22.038238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:38.464047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.033582+0000","last_change":"2023-10-06T11:37:22.031305+0000","last_active":"2023-10-06T11:37:22.033582+0000","last_peered":"2023-10-06T11:37:22.033582+0000","last_clean":"2023-10-06T11:37:22.033582+0000","last_became_active":"2023-10-06T11:37:22.030910+0000","last_became_peered":"2023-10-06T11:37:22.030910+0000","last_unstale":"2023-10-06T11:37:22.033582+0000","last_undegraded":"2023-10-06T11:37:22.033582+0000","last_fullsized":"2023-10-06T11:37:22.033582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:30.187438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039749+0000","last_change":"2023-10-06T11:37:22.035159+0000","last_active":"2023-10-06T11:37:22.039749+0000","last_peered":"2023-10-06T11:37:22.039749+0000","last_clean":"2023-10-06T11:37:22.039749+0000","last_became_active":"2023-10-06T11:37:22.034431+0000","last_became_peered":"2023-10-06T11:37:22.034431+0000","last_unstale":"2023-10-06T11:37:22.039749+0000","last_undegraded":"2023-10-06T11:37:22.039749+0000","last_fullsized":"2023-10-06T11:37:22.039749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:39.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.028872+0000","last_change":"2023-10-06T11:37:22.022283+0000","last_active":"2023-10-06T11:37:22.028872+0000","last_peered":"2023-10-06T11:37:22.028872+0000","last_clean":"2023-10-06T11:37:22.028872+0000","last_became_active":"2023-10-06T11:37:22.021776+0000","last_became_peered":"2023-10-06T11:37:22.021776+0000","last_unstale":"2023-10-06T11:37:22.028872+0000","last_undegraded":"2023-10-06T11:37:22.028872+0000","last_fullsized":"2023-10-06T11:37:22.028872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:28.975870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052293+0000","last_change":"2023-10-06T11:37:22.048496+0000","last_active":"2023-10-06T11:37:22.052293+0000","last_peered":"2023-10-06T11:37:22.052293+0000","last_clean":"2023-10-06T11:37:22.052293+0000","last_became_active":"2023-10-06T11:37:22.047873+0000","last_became_peered":"2023-10-06T11:37:22.047873+0000","last_unstale":"2023-10-06T11:37:22.052293+0000","last_undegraded":"2023-10-06T11:37:22.052293+0000","last_fullsized":"2023-10-06T11:37:22.052293+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:53.672153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.033423+0000","last_change":"2023-10-06T11:37:22.029792+0000","last_active":"2023-10-06T11:37:22.033423+0000","last_peered":"2023-10-06T11:37:22.033423+0000","last_clean":"2023-10-06T11:37:22.033423+0000","last_became_active":"2023-10-06T11:37:22.029499+0000","last_became_peered":"2023-10-06T11:37:22.029499+0000","last_unstale":"2023-10-06T11:37:22.033423+0000","last_undegraded":"2023-10-06T11:37:22.033423+0000","last_fullsized":"2023-10-06T11:37:22.033423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:15.148919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.049969+0000","last_change":"2023-10-06T11:37:22.048575+0000","last_active":"2023-10-06T11:37:22.049969+0000","last_peered":"2023-10-06T11:37:22.049969+0000","last_clean":"2023-10-06T11:37:22.049969+0000","last_became_active":"2023-10-06T11:37:22.048257+0000","last_became_peered":"2023-10-06T11:37:22.048257+0000","last_unstale":"2023-10-06T11:37:22.049969+0000","last_undegraded":"2023-10-06T11:37:22.049969+0000","last_fullsized":"2023-10-06T11:37:22.049969+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:14.345894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038875+0000","last_change":"2023-10-06T11:37:22.035907+0000","last_active":"2023-10-06T11:37:22.038875+0000","last_peered":"2023-10-06T11:37:22.038875+0000","last_clean":"2023-10-06T11:37:22.038875+0000","last_became_active":"2023-10-06T11:37:22.035598+0000","last_became_peered":"2023-10-06T11:37:22.035598+0000","last_unstale":"2023-10-06T11:37:22.038875+0000","last_undegraded":"2023-10-06T11:37:22.038875+0000","last_fullsized":"2023-10-06T11:37:22.038875+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:17.910505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034617+0000","last_change":"2023-10-06T11:37:22.026104+0000","last_active":"2023-10-06T11:37:22.034617+0000","last_peered":"2023-10-06T11:37:22.034617+0000","last_clean":"2023-10-06T11:37:22.034617+0000","last_became_active":"2023-10-06T11:37:22.025816+0000","last_became_peered":"2023-10-06T11:37:22.025816+0000","last_unstale":"2023-10-06T11:37:22.034617+0000","last_undegraded":"2023-10-06T11:37:22.034617+0000","last_fullsized":"2023-10-06T11:37:22.034617+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.479961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051823+0000","last_change":"2023-10-06T11:37:22.046837+0000","last_active":"2023-10-06T11:37:22.051823+0000","last_peered":"2023-10-06T11:37:22.051823+0000","last_clean":"2023-10-06T11:37:22.051823+0000","last_became_active":"2023-10-06T11:37:22.046546+0000","last_became_peered":"2023-10-06T11:37:22.046546+0000","last_unstale":"2023-10-06T11:37:22.051823+0000","last_undegraded":"2023-10-06T11:37:22.051823+0000","last_fullsized":"2023-10-06T11:37:22.051823+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.479961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047027+0000","last_change":"2023-10-06T11:37:22.031268+0000","last_active":"2023-10-06T11:37:22.047027+0000","last_peered":"2023-10-06T11:37:22.047027+0000","last_clean":"2023-10-06T11:37:22.047027+0000","last_became_active":"2023-10-06T11:37:22.030975+0000","last_became_peered":"2023-10-06T11:37:22.030975+0000","last_unstale":"2023-10-06T11:37:22.047027+0000","last_undegraded":"2023-10-06T11:37:22.047027+0000","last_fullsized":"2023-10-06T11:37:22.047027+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:28.975870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042310+0000","last_change":"2023-10-06T11:37:22.032421+0000","last_active":"2023-10-06T11:37:22.042310+0000","last_peered":"2023-10-06T11:37:22.042310+0000","last_clean":"2023-10-06T11:37:22.042310+0000","last_became_active":"2023-10-06T11:37:22.031935+0000","last_became_peered":"2023-10-06T11:37:22.031935+0000","last_unstale":"2023-10-06T11:37:22.042310+0000","last_undegraded":"2023-10-06T11:37:22.042310+0000","last_fullsized":"2023-10-06T11:37:22.042310+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:48.820206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039833+0000","last_change":"2023-10-06T11:37:22.034731+0000","last_active":"2023-10-06T11:37:22.039833+0000","last_peered":"2023-10-06T11:37:22.039833+0000","last_clean":"2023-10-06T11:37:22.039833+0000","last_became_active":"2023-10-06T11:37:22.031456+0000","last_became_peered":"2023-10-06T11:37:22.031456+0000","last_unstale":"2023-10-06T11:37:22.039833+0000","last_undegraded":"2023-10-06T11:37:22.039833+0000","last_fullsized":"2023-10-06T11:37:22.039833+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:39.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042188+0000","last_change":"2023-10-06T11:37:22.030170+0000","last_active":"2023-10-06T11:37:22.042188+0000","last_peered":"2023-10-06T11:37:22.042188+0000","last_clean":"2023-10-06T11:37:22.042188+0000","last_became_active":"2023-10-06T11:37:22.029827+0000","last_became_peered":"2023-10-06T11:37:22.029827+0000","last_unstale":"2023-10-06T11:37:22.042188+0000","last_undegraded":"2023-10-06T11:37:22.042188+0000","last_fullsized":"2023-10-06T11:37:22.042188+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:51.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050831+0000","last_change":"2023-10-06T11:37:22.046077+0000","last_active":"2023-10-06T11:37:22.050831+0000","last_peered":"2023-10-06T11:37:22.050831+0000","last_clean":"2023-10-06T11:37:22.050831+0000","last_became_active":"2023-10-06T11:37:22.045747+0000","last_became_peered":"2023-10-06T11:37:22.045747+0000","last_unstale":"2023-10-06T11:37:22.050831+0000","last_undegraded":"2023-10-06T11:37:22.050831+0000","last_fullsized":"2023-10-06T11:37:22.050831+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:48.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045053+0000","last_change":"2023-10-06T11:37:22.038325+0000","last_active":"2023-10-06T11:37:22.045053+0000","last_peered":"2023-10-06T11:37:22.045053+0000","last_clean":"2023-10-06T11:37:22.045053+0000","last_became_active":"2023-10-06T11:37:22.038041+0000","last_became_peered":"2023-10-06T11:37:22.038041+0000","last_unstale":"2023-10-06T11:37:22.045053+0000","last_undegraded":"2023-10-06T11:37:22.045053+0000","last_fullsized":"2023-10-06T11:37:22.045053+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047681+0000","last_change":"2023-10-06T11:37:22.027301+0000","last_active":"2023-10-06T11:37:22.047681+0000","last_peered":"2023-10-06T11:37:22.047681+0000","last_clean":"2023-10-06T11:37:22.047681+0000","last_became_active":"2023-10-06T11:37:22.026899+0000","last_became_peered":"2023-10-06T11:37:22.026899+0000","last_unstale":"2023-10-06T11:37:22.047681+0000","last_undegraded":"2023-10-06T11:37:22.047681+0000","last_fullsized":"2023-10-06T11:37:22.047681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:44.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051806+0000","last_change":"2023-10-06T11:37:22.046758+0000","last_active":"2023-10-06T11:37:22.051806+0000","last_peered":"2023-10-06T11:37:22.051806+0000","last_clean":"2023-10-06T11:37:22.051806+0000","last_became_active":"2023-10-06T11:37:22.046123+0000","last_became_peered":"2023-10-06T11:37:22.046123+0000","last_unstale":"2023-10-06T11:37:22.051806+0000","last_undegraded":"2023-10-06T11:37:22.051806+0000","last_fullsized":"2023-10-06T11:37:22.051806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:51.465644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043185+0000","last_change":"2023-10-06T11:37:22.038611+0000","last_active":"2023-10-06T11:37:22.043185+0000","last_peered":"2023-10-06T11:37:22.043185+0000","last_clean":"2023-10-06T11:37:22.043185+0000","last_became_active":"2023-10-06T11:37:22.038158+0000","last_became_peered":"2023-10-06T11:37:22.038158+0000","last_unstale":"2023-10-06T11:37:22.043185+0000","last_undegraded":"2023-10-06T11:37:22.043185+0000","last_fullsized":"2023-10-06T11:37:22.043185+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:22.314707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024944+0000","last_change":"2023-10-06T11:37:22.023177+0000","last_active":"2023-10-06T11:37:22.024944+0000","last_peered":"2023-10-06T11:37:22.024944+0000","last_clean":"2023-10-06T11:37:22.024944+0000","last_became_active":"2023-10-06T11:37:22.022877+0000","last_became_peered":"2023-10-06T11:37:22.022877+0000","last_unstale":"2023-10-06T11:37:22.024944+0000","last_undegraded":"2023-10-06T11:37:22.024944+0000","last_fullsized":"2023-10-06T11:37:22.024944+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:32.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054716+0000","last_change":"2023-10-06T11:37:22.051644+0000","last_active":"2023-10-06T11:37:22.054716+0000","last_peered":"2023-10-06T11:37:22.054716+0000","last_clean":"2023-10-06T11:37:22.054716+0000","last_became_active":"2023-10-06T11:37:22.040424+0000","last_became_peered":"2023-10-06T11:37:22.040424+0000","last_unstale":"2023-10-06T11:37:22.054716+0000","last_undegraded":"2023-10-06T11:37:22.054716+0000","last_fullsized":"2023-10-06T11:37:22.054716+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:30.066954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043594+0000","last_change":"2023-10-06T11:37:22.033913+0000","last_active":"2023-10-06T11:37:22.043594+0000","last_peered":"2023-10-06T11:37:22.043594+0000","last_clean":"2023-10-06T11:37:22.043594+0000","last_became_active":"2023-10-06T11:37:22.033625+0000","last_became_peered":"2023-10-06T11:37:22.033625+0000","last_unstale":"2023-10-06T11:37:22.043594+0000","last_undegraded":"2023-10-06T11:37:22.043594+0000","last_fullsized":"2023-10-06T11:37:22.043594+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:20:15.111882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047809+0000","last_change":"2023-10-06T11:37:22.044036+0000","last_active":"2023-10-06T11:37:22.047809+0000","last_peered":"2023-10-06T11:37:22.047809+0000","last_clean":"2023-10-06T11:37:22.047809+0000","last_became_active":"2023-10-06T11:37:22.043696+0000","last_became_peered":"2023-10-06T11:37:22.043696+0000","last_unstale":"2023-10-06T11:37:22.047809+0000","last_undegraded":"2023-10-06T11:37:22.047809+0000","last_fullsized":"2023-10-06T11:37:22.047809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:14.305263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034465+0000","last_change":"2023-10-06T11:37:22.033061+0000","last_active":"2023-10-06T11:37:22.034465+0000","last_peered":"2023-10-06T11:37:22.034465+0000","last_clean":"2023-10-06T11:37:22.034465+0000","last_became_active":"2023-10-06T11:37:22.032667+0000","last_became_peered":"2023-10-06T11:37:22.032667+0000","last_unstale":"2023-10-06T11:37:22.034465+0000","last_undegraded":"2023-10-06T11:37:22.034465+0000","last_fullsized":"2023-10-06T11:37:22.034465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:46.600662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"134'4","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057739+0000","last_change":"2023-10-06T11:37:21.031884+0000","last_active":"2023-10-06T11:37:22.057739+0000","last_peered":"2023-10-06T11:37:22.057739+0000","last_clean":"2023-10-06T11:37:22.057739+0000","last_became_active":"2023-10-06T11:37:21.031582+0000","last_became_peered":"2023-10-06T11:37:21.031582+0000","last_unstale":"2023-10-06T11:37:22.057739+0000","last_undegraded":"2023-10-06T11:37:22.057739+0000","last_fullsized":"2023-10-06T11:37:22.057739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:51.839254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050904+0000","last_change":"2023-10-06T11:37:22.049010+0000","last_active":"2023-10-06T11:37:22.050904+0000","last_peered":"2023-10-06T11:37:22.050904+0000","last_clean":"2023-10-06T11:37:22.050904+0000","last_became_active":"2023-10-06T11:37:22.048699+0000","last_became_peered":"2023-10-06T11:37:22.048699+0000","last_unstale":"2023-10-06T11:37:22.050904+0000","last_undegraded":"2023-10-06T11:37:22.050904+0000","last_fullsized":"2023-10-06T11:37:22.050904+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:57.434772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048025+0000","last_change":"2023-10-06T11:37:21.022792+0000","last_active":"2023-10-06T11:37:22.048025+0000","last_peered":"2023-10-06T11:37:22.048025+0000","last_clean":"2023-10-06T11:37:22.048025+0000","last_became_active":"2023-10-06T11:37:21.022508+0000","last_became_peered":"2023-10-06T11:37:21.022508+0000","last_unstale":"2023-10-06T11:37:22.048025+0000","last_undegraded":"2023-10-06T11:37:22.048025+0000","last_fullsized":"2023-10-06T11:37:22.048025+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:41.448824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048059+0000","last_change":"2023-10-06T11:37:22.043065+0000","last_active":"2023-10-06T11:37:22.048059+0000","last_peered":"2023-10-06T11:37:22.048059+0000","last_clean":"2023-10-06T11:37:22.048059+0000","last_became_active":"2023-10-06T11:37:22.041296+0000","last_became_peered":"2023-10-06T11:37:22.041296+0000","last_unstale":"2023-10-06T11:37:22.048059+0000","last_undegraded":"2023-10-06T11:37:22.048059+0000","last_fullsized":"2023-10-06T11:37:22.048059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057784+0000","last_change":"2023-10-06T11:37:21.031883+0000","last_active":"2023-10-06T11:37:22.057784+0000","last_peered":"2023-10-06T11:37:22.057784+0000","last_clean":"2023-10-06T11:37:22.057784+0000","last_became_active":"2023-10-06T11:37:21.031568+0000","last_became_peered":"2023-10-06T11:37:21.031568+0000","last_unstale":"2023-10-06T11:37:22.057784+0000","last_undegraded":"2023-10-06T11:37:22.057784+0000","last_fullsized":"2023-10-06T11:37:22.057784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:19.518142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042259+0000","last_change":"2023-10-06T11:37:22.032346+0000","last_active":"2023-10-06T11:37:22.042259+0000","last_peered":"2023-10-06T11:37:22.042259+0000","last_clean":"2023-10-06T11:37:22.042259+0000","last_became_active":"2023-10-06T11:37:22.031980+0000","last_became_peered":"2023-10-06T11:37:22.031980+0000","last_unstale":"2023-10-06T11:37:22.042259+0000","last_undegraded":"2023-10-06T11:37:22.042259+0000","last_fullsized":"2023-10-06T11:37:22.042259+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055521+0000","last_change":"2023-10-06T11:37:21.025853+0000","last_active":"2023-10-06T11:37:22.055521+0000","last_peered":"2023-10-06T11:37:22.055521+0000","last_clean":"2023-10-06T11:37:22.055521+0000","last_became_active":"2023-10-06T11:37:21.025492+0000","last_became_peered":"2023-10-06T11:37:21.025492+0000","last_unstale":"2023-10-06T11:37:22.055521+0000","last_undegraded":"2023-10-06T11:37:22.055521+0000","last_fullsized":"2023-10-06T11:37:22.055521+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:23:07.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.036884+0000","last_change":"2023-10-06T11:37:22.034707+0000","last_active":"2023-10-06T11:37:22.036884+0000","last_peered":"2023-10-06T11:37:22.036884+0000","last_clean":"2023-10-06T11:37:22.036884+0000","last_became_active":"2023-10-06T11:37:22.034341+0000","last_became_peered":"2023-10-06T11:37:22.034341+0000","last_unstale":"2023-10-06T11:37:22.036884+0000","last_undegraded":"2023-10-06T11:37:22.036884+0000","last_fullsized":"2023-10-06T11:37:22.036884+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:37.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055942+0000","last_change":"2023-10-06T11:37:21.033617+0000","last_active":"2023-10-06T11:37:22.055942+0000","last_peered":"2023-10-06T11:37:22.055942+0000","last_clean":"2023-10-06T11:37:22.055942+0000","last_became_active":"2023-10-06T11:37:21.033273+0000","last_became_peered":"2023-10-06T11:37:21.033273+0000","last_unstale":"2023-10-06T11:37:22.055942+0000","last_undegraded":"2023-10-06T11:37:22.055942+0000","last_fullsized":"2023-10-06T11:37:22.055942+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:52.667532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042164+0000","last_change":"2023-10-06T11:37:22.037095+0000","last_active":"2023-10-06T11:37:22.042164+0000","last_peered":"2023-10-06T11:37:22.042164+0000","last_clean":"2023-10-06T11:37:22.042164+0000","last_became_active":"2023-10-06T11:37:22.036799+0000","last_became_peered":"2023-10-06T11:37:22.036799+0000","last_unstale":"2023-10-06T11:37:22.042164+0000","last_undegraded":"2023-10-06T11:37:22.042164+0000","last_fullsized":"2023-10-06T11:37:22.042164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.053115+0000","last_change":"2023-10-06T11:37:21.032380+0000","last_active":"2023-10-06T11:37:22.053115+0000","last_peered":"2023-10-06T11:37:22.053115+0000","last_clean":"2023-10-06T11:37:22.053115+0000","last_became_active":"2023-10-06T11:37:21.032017+0000","last_became_peered":"2023-10-06T11:37:21.032017+0000","last_unstale":"2023-10-06T11:37:22.053115+0000","last_undegraded":"2023-10-06T11:37:22.053115+0000","last_fullsized":"2023-10-06T11:37:22.053115+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:47.385947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054648+0000","last_change":"2023-10-06T11:37:22.049911+0000","last_active":"2023-10-06T11:37:22.054648+0000","last_peered":"2023-10-06T11:37:22.054648+0000","last_clean":"2023-10-06T11:37:22.054648+0000","last_became_active":"2023-10-06T11:37:22.049597+0000","last_became_peered":"2023-10-06T11:37:22.049597+0000","last_unstale":"2023-10-06T11:37:22.054648+0000","last_undegraded":"2023-10-06T11:37:22.054648+0000","last_fullsized":"2023-10-06T11:37:22.054648+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:12.939136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021032+0000","last_change":"2023-10-06T11:37:21.030689+0000","last_active":"2023-10-06T11:37:22.021032+0000","last_peered":"2023-10-06T11:37:22.021032+0000","last_clean":"2023-10-06T11:37:22.021032+0000","last_became_active":"2023-10-06T11:37:21.030401+0000","last_became_peered":"2023-10-06T11:37:21.030401+0000","last_unstale":"2023-10-06T11:37:22.021032+0000","last_undegraded":"2023-10-06T11:37:22.021032+0000","last_fullsized":"2023-10-06T11:37:22.021032+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.652875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034880+0000","last_change":"2023-10-06T11:37:22.027444+0000","last_active":"2023-10-06T11:37:22.034880+0000","last_peered":"2023-10-06T11:37:22.034880+0000","last_clean":"2023-10-06T11:37:22.034880+0000","last_became_active":"2023-10-06T11:37:22.027110+0000","last_became_peered":"2023-10-06T11:37:22.027110+0000","last_unstale":"2023-10-06T11:37:22.034880+0000","last_undegraded":"2023-10-06T11:37:22.034880+0000","last_fullsized":"2023-10-06T11:37:22.034880+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:57.757746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024372+0000","last_change":"2023-10-06T11:37:21.030894+0000","last_active":"2023-10-06T11:37:22.024372+0000","last_peered":"2023-10-06T11:37:22.024372+0000","last_clean":"2023-10-06T11:37:22.024372+0000","last_became_active":"2023-10-06T11:37:21.027945+0000","last_became_peered":"2023-10-06T11:37:21.027945+0000","last_unstale":"2023-10-06T11:37:22.024372+0000","last_undegraded":"2023-10-06T11:37:22.024372+0000","last_fullsized":"2023-10-06T11:37:22.024372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:19.705850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045043+0000","last_change":"2023-10-06T11:37:22.042184+0000","last_active":"2023-10-06T11:37:22.045043+0000","last_peered":"2023-10-06T11:37:22.045043+0000","last_clean":"2023-10-06T11:37:22.045043+0000","last_became_active":"2023-10-06T11:37:22.041914+0000","last_became_peered":"2023-10-06T11:37:22.041914+0000","last_unstale":"2023-10-06T11:37:22.045043+0000","last_undegraded":"2023-10-06T11:37:22.045043+0000","last_fullsized":"2023-10-06T11:37:22.045043+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048629+0000","last_change":"2023-10-06T11:37:21.024842+0000","last_active":"2023-10-06T11:37:22.048629+0000","last_peered":"2023-10-06T11:37:22.048629+0000","last_clean":"2023-10-06T11:37:22.048629+0000","last_became_active":"2023-10-06T11:37:21.024560+0000","last_became_peered":"2023-10-06T11:37:21.024560+0000","last_unstale":"2023-10-06T11:37:22.048629+0000","last_undegraded":"2023-10-06T11:37:22.048629+0000","last_fullsized":"2023-10-06T11:37:22.048629+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:33.876498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039509+0000","last_change":"2023-10-06T11:37:22.032993+0000","last_active":"2023-10-06T11:37:22.039509+0000","last_peered":"2023-10-06T11:37:22.039509+0000","last_clean":"2023-10-06T11:37:22.039509+0000","last_became_active":"2023-10-06T11:37:22.032693+0000","last_became_peered":"2023-10-06T11:37:22.032693+0000","last_unstale":"2023-10-06T11:37:22.039509+0000","last_undegraded":"2023-10-06T11:37:22.039509+0000","last_fullsized":"2023-10-06T11:37:22.039509+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052494+0000","last_change":"2023-10-06T11:37:21.026396+0000","last_active":"2023-10-06T11:37:22.052494+0000","last_peered":"2023-10-06T11:37:22.052494+0000","last_clean":"2023-10-06T11:37:22.052494+0000","last_became_active":"2023-10-06T11:37:21.026068+0000","last_became_peered":"2023-10-06T11:37:21.026068+0000","last_unstale":"2023-10-06T11:37:22.052494+0000","last_undegraded":"2023-10-06T11:37:22.052494+0000","last_fullsized":"2023-10-06T11:37:22.052494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:28.408340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043165+0000","last_change":"2023-10-06T11:37:22.038995+0000","last_active":"2023-10-06T11:37:22.043165+0000","last_peered":"2023-10-06T11:37:22.043165+0000","last_clean":"2023-10-06T11:37:22.043165+0000","last_became_active":"2023-10-06T11:37:22.038702+0000","last_became_peered":"2023-10-06T11:37:22.038702+0000","last_unstale":"2023-10-06T11:37:22.043165+0000","last_undegraded":"2023-10-06T11:37:22.043165+0000","last_fullsized":"2023-10-06T11:37:22.043165+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:04.471611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057861+0000","last_change":"2023-10-06T11:37:21.034035+0000","last_active":"2023-10-06T11:37:22.057861+0000","last_peered":"2023-10-06T11:37:22.057861+0000","last_clean":"2023-10-06T11:37:22.057861+0000","last_became_active":"2023-10-06T11:37:21.033768+0000","last_became_peered":"2023-10-06T11:37:21.033768+0000","last_unstale":"2023-10-06T11:37:22.057861+0000","last_undegraded":"2023-10-06T11:37:22.057861+0000","last_fullsized":"2023-10-06T11:37:22.057861+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.526657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037749+0000","last_change":"2023-10-06T11:37:22.028561+0000","last_active":"2023-10-06T11:37:22.037749+0000","last_peered":"2023-10-06T11:37:22.037749+0000","last_clean":"2023-10-06T11:37:22.037749+0000","last_became_active":"2023-10-06T11:37:22.028203+0000","last_became_peered":"2023-10-06T11:37:22.028203+0000","last_unstale":"2023-10-06T11:37:22.037749+0000","last_undegraded":"2023-10-06T11:37:22.037749+0000","last_fullsized":"2023-10-06T11:37:22.037749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:51.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020754+0000","last_change":"2023-10-06T11:37:21.042516+0000","last_active":"2023-10-06T11:37:22.020754+0000","last_peered":"2023-10-06T11:37:22.020754+0000","last_clean":"2023-10-06T11:37:22.020754+0000","last_became_active":"2023-10-06T11:37:21.042180+0000","last_became_peered":"2023-10-06T11:37:21.042180+0000","last_unstale":"2023-10-06T11:37:22.020754+0000","last_undegraded":"2023-10-06T11:37:22.020754+0000","last_fullsized":"2023-10-06T11:37:22.020754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:50.461023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042707+0000","last_change":"2023-10-06T11:37:22.036063+0000","last_active":"2023-10-06T11:37:22.042707+0000","last_peered":"2023-10-06T11:37:22.042707+0000","last_clean":"2023-10-06T11:37:22.042707+0000","last_became_active":"2023-10-06T11:37:22.035693+0000","last_became_peered":"2023-10-06T11:37:22.035693+0000","last_unstale":"2023-10-06T11:37:22.042707+0000","last_undegraded":"2023-10-06T11:37:22.042707+0000","last_fullsized":"2023-10-06T11:37:22.042707+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:05.005101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047158+0000","last_change":"2023-10-06T11:37:21.031123+0000","last_active":"2023-10-06T11:37:22.047158+0000","last_peered":"2023-10-06T11:37:22.047158+0000","last_clean":"2023-10-06T11:37:22.047158+0000","last_became_active":"2023-10-06T11:37:21.030739+0000","last_became_peered":"2023-10-06T11:37:21.030739+0000","last_unstale":"2023-10-06T11:37:22.047158+0000","last_undegraded":"2023-10-06T11:37:22.047158+0000","last_fullsized":"2023-10-06T11:37:22.047158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:11.817250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021626+0000","last_change":"2023-10-06T11:37:22.019124+0000","last_active":"2023-10-06T11:37:22.021626+0000","last_peered":"2023-10-06T11:37:22.021626+0000","last_clean":"2023-10-06T11:37:22.021626+0000","last_became_active":"2023-10-06T11:37:22.018796+0000","last_became_peered":"2023-10-06T11:37:22.018796+0000","last_unstale":"2023-10-06T11:37:22.021626+0000","last_undegraded":"2023-10-06T11:37:22.021626+0000","last_fullsized":"2023-10-06T11:37:22.021626+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:39.785634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019788+0000","last_change":"2023-10-06T11:37:21.035595+0000","last_active":"2023-10-06T11:37:22.019788+0000","last_peered":"2023-10-06T11:37:22.019788+0000","last_clean":"2023-10-06T11:37:22.019788+0000","last_became_active":"2023-10-06T11:37:21.035289+0000","last_became_peered":"2023-10-06T11:37:21.035289+0000","last_unstale":"2023-10-06T11:37:22.019788+0000","last_undegraded":"2023-10-06T11:37:22.019788+0000","last_fullsized":"2023-10-06T11:37:22.019788+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:32.475340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052270+0000","last_change":"2023-10-06T11:37:22.048313+0000","last_active":"2023-10-06T11:37:22.052270+0000","last_peered":"2023-10-06T11:37:22.052270+0000","last_clean":"2023-10-06T11:37:22.052270+0000","last_became_active":"2023-10-06T11:37:22.047766+0000","last_became_peered":"2023-10-06T11:37:22.047766+0000","last_unstale":"2023-10-06T11:37:22.052270+0000","last_undegraded":"2023-10-06T11:37:22.052270+0000","last_fullsized":"2023-10-06T11:37:22.052270+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:20.522763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.013094+0000","last_change":"2023-10-06T11:37:21.024890+0000","last_active":"2023-10-06T11:37:22.013094+0000","last_peered":"2023-10-06T11:37:22.013094+0000","last_clean":"2023-10-06T11:37:22.013094+0000","last_became_active":"2023-10-06T11:37:21.024592+0000","last_became_peered":"2023-10-06T11:37:21.024592+0000","last_unstale":"2023-10-06T11:37:22.013094+0000","last_undegraded":"2023-10-06T11:37:22.013094+0000","last_fullsized":"2023-10-06T11:37:22.013094+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:56.753124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043489+0000","last_change":"2023-10-06T11:37:22.033090+0000","last_active":"2023-10-06T11:37:22.043489+0000","last_peered":"2023-10-06T11:37:22.043489+0000","last_clean":"2023-10-06T11:37:22.043489+0000","last_became_active":"2023-10-06T11:37:22.032689+0000","last_became_peered":"2023-10-06T11:37:22.032689+0000","last_unstale":"2023-10-06T11:37:22.043489+0000","last_undegraded":"2023-10-06T11:37:22.043489+0000","last_fullsized":"2023-10-06T11:37:22.043489+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:03.029324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019366+0000","last_change":"2023-10-06T11:37:21.031934+0000","last_active":"2023-10-06T11:37:22.019366+0000","last_peered":"2023-10-06T11:37:22.019366+0000","last_clean":"2023-10-06T11:37:22.019366+0000","last_became_active":"2023-10-06T11:37:21.031654+0000","last_became_peered":"2023-10-06T11:37:21.031654+0000","last_unstale":"2023-10-06T11:37:22.019366+0000","last_undegraded":"2023-10-06T11:37:22.019366+0000","last_fullsized":"2023-10-06T11:37:22.019366+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:11.934515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043162+0000","last_change":"2023-10-06T11:37:22.038403+0000","last_active":"2023-10-06T11:37:22.043162+0000","last_peered":"2023-10-06T11:37:22.043162+0000","last_clean":"2023-10-06T11:37:22.043162+0000","last_became_active":"2023-10-06T11:37:22.037643+0000","last_became_peered":"2023-10-06T11:37:22.037643+0000","last_unstale":"2023-10-06T11:37:22.043162+0000","last_undegraded":"2023-10-06T11:37:22.043162+0000","last_fullsized":"2023-10-06T11:37:22.043162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:17.680312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046499+0000","last_change":"2023-10-06T11:37:21.024723+0000","last_active":"2023-10-06T11:37:22.046499+0000","last_peered":"2023-10-06T11:37:22.046499+0000","last_clean":"2023-10-06T11:37:22.046499+0000","last_became_active":"2023-10-06T11:37:21.024451+0000","last_became_peered":"2023-10-06T11:37:21.024451+0000","last_unstale":"2023-10-06T11:37:22.046499+0000","last_undegraded":"2023-10-06T11:37:22.046499+0000","last_fullsized":"2023-10-06T11:37:22.046499+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:26.726631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.035922+0000","last_change":"2023-10-06T11:37:22.034046+0000","last_active":"2023-10-06T11:37:22.035922+0000","last_peered":"2023-10-06T11:37:22.035922+0000","last_clean":"2023-10-06T11:37:22.035922+0000","last_became_active":"2023-10-06T11:37:22.033768+0000","last_became_peered":"2023-10-06T11:37:22.033768+0000","last_unstale":"2023-10-06T11:37:22.035922+0000","last_undegraded":"2023-10-06T11:37:22.035922+0000","last_fullsized":"2023-10-06T11:37:22.035922+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:15.444744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038622+0000","last_change":"2023-10-06T11:37:21.042502+0000","last_active":"2023-10-06T11:37:22.038622+0000","last_peered":"2023-10-06T11:37:22.038622+0000","last_clean":"2023-10-06T11:37:22.038622+0000","last_became_active":"2023-10-06T11:37:21.042164+0000","last_became_peered":"2023-10-06T11:37:21.042164+0000","last_unstale":"2023-10-06T11:37:22.038622+0000","last_undegraded":"2023-10-06T11:37:22.038622+0000","last_fullsized":"2023-10-06T11:37:22.038622+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:14.440123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042323+0000","last_change":"2023-10-06T11:37:22.033816+0000","last_active":"2023-10-06T11:37:22.042323+0000","last_peered":"2023-10-06T11:37:22.042323+0000","last_clean":"2023-10-06T11:37:22.042323+0000","last_became_active":"2023-10-06T11:37:22.033436+0000","last_became_peered":"2023-10-06T11:37:22.033436+0000","last_unstale":"2023-10-06T11:37:22.042323+0000","last_undegraded":"2023-10-06T11:37:22.042323+0000","last_fullsized":"2023-10-06T11:37:22.042323+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038396+0000","last_change":"2023-10-06T11:37:21.042542+0000","last_active":"2023-10-06T11:37:22.038396+0000","last_peered":"2023-10-06T11:37:22.038396+0000","last_clean":"2023-10-06T11:37:22.038396+0000","last_became_active":"2023-10-06T11:37:21.042231+0000","last_became_peered":"2023-10-06T11:37:21.042231+0000","last_unstale":"2023-10-06T11:37:22.038396+0000","last_undegraded":"2023-10-06T11:37:22.038396+0000","last_fullsized":"2023-10-06T11:37:22.038396+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:37.459426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.025795+0000","last_change":"2023-10-06T11:37:22.021723+0000","last_active":"2023-10-06T11:37:22.025795+0000","last_peered":"2023-10-06T11:37:22.025795+0000","last_clean":"2023-10-06T11:37:22.025795+0000","last_became_active":"2023-10-06T11:37:22.021332+0000","last_became_peered":"2023-10-06T11:37:22.021332+0000","last_unstale":"2023-10-06T11:37:22.025795+0000","last_undegraded":"2023-10-06T11:37:22.025795+0000","last_fullsized":"2023-10-06T11:37:22.025795+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:17.910505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.028473+0000","last_change":"2023-10-06T11:37:21.027939+0000","last_active":"2023-10-06T11:37:22.028473+0000","last_peered":"2023-10-06T11:37:22.028473+0000","last_clean":"2023-10-06T11:37:22.028473+0000","last_became_active":"2023-10-06T11:37:21.027554+0000","last_became_peered":"2023-10-06T11:37:21.027554+0000","last_unstale":"2023-10-06T11:37:22.028473+0000","last_undegraded":"2023-10-06T11:37:22.028473+0000","last_fullsized":"2023-10-06T11:37:22.028473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:08.775982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037289+0000","last_change":"2023-10-06T11:37:22.031714+0000","last_active":"2023-10-06T11:37:22.037289+0000","last_peered":"2023-10-06T11:37:22.037289+0000","last_clean":"2023-10-06T11:37:22.037289+0000","last_became_active":"2023-10-06T11:37:22.031429+0000","last_became_peered":"2023-10-06T11:37:22.031429+0000","last_unstale":"2023-10-06T11:37:22.037289+0000","last_undegraded":"2023-10-06T11:37:22.037289+0000","last_fullsized":"2023-10-06T11:37:22.037289+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:23:10.721108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.025027+0000","last_change":"2023-10-06T11:37:21.034445+0000","last_active":"2023-10-06T11:37:22.025027+0000","last_peered":"2023-10-06T11:37:22.025027+0000","last_clean":"2023-10-06T11:37:22.025027+0000","last_became_active":"2023-10-06T11:37:21.034003+0000","last_became_peered":"2023-10-06T11:37:21.034003+0000","last_unstale":"2023-10-06T11:37:22.025027+0000","last_undegraded":"2023-10-06T11:37:22.025027+0000","last_fullsized":"2023-10-06T11:37:22.025027+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:20.903950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046801+0000","last_change":"2023-10-06T11:37:22.037038+0000","last_active":"2023-10-06T11:37:22.046801+0000","last_peered":"2023-10-06T11:37:22.046801+0000","last_clean":"2023-10-06T11:37:22.046801+0000","last_became_active":"2023-10-06T11:37:22.036750+0000","last_became_peered":"2023-10-06T11:37:22.036750+0000","last_unstale":"2023-10-06T11:37:22.046801+0000","last_undegraded":"2023-10-06T11:37:22.046801+0000","last_fullsized":"2023-10-06T11:37:22.046801+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:38.596362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.027015+0000","last_change":"2023-10-06T11:37:21.035362+0000","last_active":"2023-10-06T11:37:22.027015+0000","last_peered":"2023-10-06T11:37:22.027015+0000","last_clean":"2023-10-06T11:37:22.027015+0000","last_became_active":"2023-10-06T11:37:21.035042+0000","last_became_peered":"2023-10-06T11:37:21.035042+0000","last_unstale":"2023-10-06T11:37:22.027015+0000","last_undegraded":"2023-10-06T11:37:22.027015+0000","last_fullsized":"2023-10-06T11:37:22.027015+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:46.024750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045491+0000","last_change":"2023-10-06T11:37:22.039285+0000","last_active":"2023-10-06T11:37:22.045491+0000","last_peered":"2023-10-06T11:37:22.045491+0000","last_clean":"2023-10-06T11:37:22.045491+0000","last_became_active":"2023-10-06T11:37:22.038993+0000","last_became_peered":"2023-10-06T11:37:22.038993+0000","last_unstale":"2023-10-06T11:37:22.045491+0000","last_undegraded":"2023-10-06T11:37:22.045491+0000","last_fullsized":"2023-10-06T11:37:22.045491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:39.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024100+0000","last_change":"2023-10-06T11:37:21.027828+0000","last_active":"2023-10-06T11:37:22.024100+0000","last_peered":"2023-10-06T11:37:22.024100+0000","last_clean":"2023-10-06T11:37:22.024100+0000","last_became_active":"2023-10-06T11:37:21.027477+0000","last_became_peered":"2023-10-06T11:37:21.027477+0000","last_unstale":"2023-10-06T11:37:22.024100+0000","last_undegraded":"2023-10-06T11:37:22.024100+0000","last_fullsized":"2023-10-06T11:37:22.024100+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:41.517876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037036+0000","last_change":"2023-10-06T11:37:22.035002+0000","last_active":"2023-10-06T11:37:22.037036+0000","last_peered":"2023-10-06T11:37:22.037036+0000","last_clean":"2023-10-06T11:37:22.037036+0000","last_became_active":"2023-10-06T11:37:22.033041+0000","last_became_peered":"2023-10-06T11:37:22.033041+0000","last_unstale":"2023-10-06T11:37:22.037036+0000","last_undegraded":"2023-10-06T11:37:22.037036+0000","last_fullsized":"2023-10-06T11:37:22.037036+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:32.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019633+0000","last_change":"2023-10-06T11:37:21.030419+0000","last_active":"2023-10-06T11:37:22.019633+0000","last_peered":"2023-10-06T11:37:22.019633+0000","last_clean":"2023-10-06T11:37:22.019633+0000","last_became_active":"2023-10-06T11:37:21.030062+0000","last_became_peered":"2023-10-06T11:37:21.030062+0000","last_unstale":"2023-10-06T11:37:22.019633+0000","last_undegraded":"2023-10-06T11:37:22.019633+0000","last_fullsized":"2023-10-06T11:37:22.019633+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:10.625491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042330+0000","last_change":"2023-10-06T11:37:22.032780+0000","last_active":"2023-10-06T11:37:22.042330+0000","last_peered":"2023-10-06T11:37:22.042330+0000","last_clean":"2023-10-06T11:37:22.042330+0000","last_became_active":"2023-10-06T11:37:22.032358+0000","last_became_peered":"2023-10-06T11:37:22.032358+0000","last_unstale":"2023-10-06T11:37:22.042330+0000","last_undegraded":"2023-10-06T11:37:22.042330+0000","last_fullsized":"2023-10-06T11:37:22.042330+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:48.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"134'8","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051403+0000","last_change":"2023-10-06T11:37:21.025453+0000","last_active":"2023-10-06T11:37:22.051403+0000","last_peered":"2023-10-06T11:37:22.051403+0000","last_clean":"2023-10-06T11:37:22.051403+0000","last_became_active":"2023-10-06T11:37:21.025125+0000","last_became_peered":"2023-10-06T11:37:21.025125+0000","last_unstale":"2023-10-06T11:37:22.051403+0000","last_undegraded":"2023-10-06T11:37:22.051403+0000","last_fullsized":"2023-10-06T11:37:22.051403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:15.652129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":729,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021990+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:37:22.021990+0000","last_peered":"2023-10-06T11:37:22.021990+0000","last_clean":"2023-10-06T11:37:22.021990+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:37:22.021990+0000","last_undegraded":"2023-10-06T11:37:22.021990+0000","last_fullsized":"2023-10-06T11:37:22.021990+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":120,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.022541+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:37:22.022541+0000","last_peered":"2023-10-06T11:37:22.022541+0000","last_clean":"2023-10-06T11:37:22.022541+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:37:22.022541+0000","last_undegraded":"2023-10-06T11:37:22.022541+0000","last_fullsized":"2023-10-06T11:37:22.022541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.062238+0000","last_change":"2023-10-06T11:37:22.060573+0000","last_active":"2023-10-06T11:37:22.062238+0000","last_peered":"2023-10-06T11:37:22.062238+0000","last_clean":"2023-10-06T11:37:22.062238+0000","last_became_active":"2023-10-06T11:37:22.059899+0000","last_became_peered":"2023-10-06T11:37:22.059899+0000","last_unstale":"2023-10-06T11:37:22.062238+0000","last_undegraded":"2023-10-06T11:37:22.062238+0000","last_fullsized":"2023-10-06T11:37:22.062238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.531278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050904+0000","last_change":"2023-10-06T11:37:21.024571+0000","last_active":"2023-10-06T11:37:22.050904+0000","last_peered":"2023-10-06T11:37:22.050904+0000","last_clean":"2023-10-06T11:37:22.050904+0000","last_became_active":"2023-10-06T11:37:21.024254+0000","last_became_peered":"2023-10-06T11:37:21.024254+0000","last_unstale":"2023-10-06T11:37:22.050904+0000","last_undegraded":"2023-10-06T11:37:22.050904+0000","last_fullsized":"2023-10-06T11:37:22.050904+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:52.937702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021867+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:37:22.021867+0000","last_peered":"2023-10-06T11:37:22.021867+0000","last_clean":"2023-10-06T11:37:22.021867+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:37:22.021867+0000","last_undegraded":"2023-10-06T11:37:22.021867+0000","last_fullsized":"2023-10-06T11:37:22.021867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.036142+0000","last_change":"2023-10-06T11:37:22.033626+0000","last_active":"2023-10-06T11:37:22.036142+0000","last_peered":"2023-10-06T11:37:22.036142+0000","last_clean":"2023-10-06T11:37:22.036142+0000","last_became_active":"2023-10-06T11:37:22.033315+0000","last_became_peered":"2023-10-06T11:37:22.033315+0000","last_unstale":"2023-10-06T11:37:22.036142+0000","last_undegraded":"2023-10-06T11:37:22.036142+0000","last_fullsized":"2023-10-06T11:37:22.036142+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:44.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.012975+0000","last_change":"2023-10-06T11:37:21.025989+0000","last_active":"2023-10-06T11:37:22.012975+0000","last_peered":"2023-10-06T11:37:22.012975+0000","last_clean":"2023-10-06T11:37:22.012975+0000","last_became_active":"2023-10-06T11:37:21.025691+0000","last_became_peered":"2023-10-06T11:37:21.025691+0000","last_unstale":"2023-10-06T11:37:22.012975+0000","last_undegraded":"2023-10-06T11:37:22.012975+0000","last_fullsized":"2023-10-06T11:37:22.012975+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:13.178643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.009254+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:37:22.009254+0000","last_peered":"2023-10-06T11:37:22.009254+0000","last_clean":"2023-10-06T11:37:22.009254+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:37:22.009254+0000","last_undegraded":"2023-10-06T11:37:22.009254+0000","last_fullsized":"2023-10-06T11:37:22.009254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046343+0000","last_change":"2023-10-06T11:37:22.026243+0000","last_active":"2023-10-06T11:37:22.046343+0000","last_peered":"2023-10-06T11:37:22.046343+0000","last_clean":"2023-10-06T11:37:22.046343+0000","last_became_active":"2023-10-06T11:37:22.025927+0000","last_became_peered":"2023-10-06T11:37:22.025927+0000","last_unstale":"2023-10-06T11:37:22.046343+0000","last_undegraded":"2023-10-06T11:37:22.046343+0000","last_fullsized":"2023-10-06T11:37:22.046343+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:37.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.012900+0000","last_change":"2023-10-06T11:37:21.027257+0000","last_active":"2023-10-06T11:37:22.012900+0000","last_peered":"2023-10-06T11:37:22.012900+0000","last_clean":"2023-10-06T11:37:22.012900+0000","last_became_active":"2023-10-06T11:37:21.026962+0000","last_became_peered":"2023-10-06T11:37:21.026962+0000","last_unstale":"2023-10-06T11:37:22.012900+0000","last_undegraded":"2023-10-06T11:37:22.012900+0000","last_fullsized":"2023-10-06T11:37:22.012900+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:43.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":120,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020016+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:37:22.020016+0000","last_peered":"2023-10-06T11:37:22.020016+0000","last_clean":"2023-10-06T11:37:22.020016+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:37:22.020016+0000","last_undegraded":"2023-10-06T11:37:22.020016+0000","last_fullsized":"2023-10-06T11:37:22.020016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051776+0000","last_change":"2023-10-06T11:37:22.046575+0000","last_active":"2023-10-06T11:37:22.051776+0000","last_peered":"2023-10-06T11:37:22.051776+0000","last_clean":"2023-10-06T11:37:22.051776+0000","last_became_active":"2023-10-06T11:37:22.046280+0000","last_became_peered":"2023-10-06T11:37:22.046280+0000","last_unstale":"2023-10-06T11:37:22.051776+0000","last_undegraded":"2023-10-06T11:37:22.051776+0000","last_fullsized":"2023-10-06T11:37:22.051776+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:11.630112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038396+0000","last_change":"2023-10-06T11:37:21.044314+0000","last_active":"2023-10-06T11:37:22.038396+0000","last_peered":"2023-10-06T11:37:22.038396+0000","last_clean":"2023-10-06T11:37:22.038396+0000","last_became_active":"2023-10-06T11:37:21.044002+0000","last_became_peered":"2023-10-06T11:37:21.044002+0000","last_unstale":"2023-10-06T11:37:22.038396+0000","last_undegraded":"2023-10-06T11:37:22.038396+0000","last_fullsized":"2023-10-06T11:37:22.038396+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:45.596041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.018017+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:37:22.018017+0000","last_peered":"2023-10-06T11:37:22.018017+0000","last_clean":"2023-10-06T11:37:22.018017+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:37:22.018017+0000","last_undegraded":"2023-10-06T11:37:22.018017+0000","last_fullsized":"2023-10-06T11:37:22.018017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037670+0000","last_change":"2023-10-06T11:37:22.028981+0000","last_active":"2023-10-06T11:37:22.037670+0000","last_peered":"2023-10-06T11:37:22.037670+0000","last_clean":"2023-10-06T11:37:22.037670+0000","last_became_active":"2023-10-06T11:37:22.028619+0000","last_became_peered":"2023-10-06T11:37:22.028619+0000","last_unstale":"2023-10-06T11:37:22.037670+0000","last_undegraded":"2023-10-06T11:37:22.037670+0000","last_fullsized":"2023-10-06T11:37:22.037670+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:48.820206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020279+0000","last_change":"2023-10-06T11:37:21.032431+0000","last_active":"2023-10-06T11:37:22.020279+0000","last_peered":"2023-10-06T11:37:22.020279+0000","last_clean":"2023-10-06T11:37:22.020279+0000","last_became_active":"2023-10-06T11:37:21.031119+0000","last_became_peered":"2023-10-06T11:37:21.031119+0000","last_unstale":"2023-10-06T11:37:22.020279+0000","last_undegraded":"2023-10-06T11:37:22.020279+0000","last_fullsized":"2023-10-06T11:37:22.020279+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:10.625491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020701+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:37:22.020701+0000","last_peered":"2023-10-06T11:37:22.020701+0000","last_clean":"2023-10-06T11:37:22.020701+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:37:22.020701+0000","last_undegraded":"2023-10-06T11:37:22.020701+0000","last_fullsized":"2023-10-06T11:37:22.020701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046951+0000","last_change":"2023-10-06T11:37:22.038617+0000","last_active":"2023-10-06T11:37:22.046951+0000","last_peered":"2023-10-06T11:37:22.046951+0000","last_clean":"2023-10-06T11:37:22.046951+0000","last_became_active":"2023-10-06T11:37:22.038323+0000","last_became_peered":"2023-10-06T11:37:22.038323+0000","last_unstale":"2023-10-06T11:37:22.046951+0000","last_undegraded":"2023-10-06T11:37:22.046951+0000","last_fullsized":"2023-10-06T11:37:22.046951+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:21.908571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"134'3","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.056686+0000","last_change":"2023-10-06T11:37:21.030077+0000","last_active":"2023-10-06T11:37:22.056686+0000","last_peered":"2023-10-06T11:37:22.056686+0000","last_clean":"2023-10-06T11:37:22.056686+0000","last_became_active":"2023-10-06T11:37:21.029785+0000","last_became_peered":"2023-10-06T11:37:21.029785+0000","last_unstale":"2023-10-06T11:37:22.056686+0000","last_undegraded":"2023-10-06T11:37:22.056686+0000","last_fullsized":"2023-10-06T11:37:22.056686+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:50.993145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020235+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:37:22.020235+0000","last_peered":"2023-10-06T11:37:22.020235+0000","last_clean":"2023-10-06T11:37:22.020235+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:37:22.020235+0000","last_undegraded":"2023-10-06T11:37:22.020235+0000","last_fullsized":"2023-10-06T11:37:22.020235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043889+0000","last_change":"2023-10-06T11:37:22.033184+0000","last_active":"2023-10-06T11:37:22.043889+0000","last_peered":"2023-10-06T11:37:22.043889+0000","last_clean":"2023-10-06T11:37:22.043889+0000","last_became_active":"2023-10-06T11:37:22.032909+0000","last_became_peered":"2023-10-06T11:37:22.032909+0000","last_unstale":"2023-10-06T11:37:22.043889+0000","last_undegraded":"2023-10-06T11:37:22.043889+0000","last_fullsized":"2023-10-06T11:37:22.043889+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:24.545371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"134'6","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055863+0000","last_change":"2023-10-06T11:37:21.029833+0000","last_active":"2023-10-06T11:37:22.055863+0000","last_peered":"2023-10-06T11:37:22.055863+0000","last_clean":"2023-10-06T11:37:22.055863+0000","last_became_active":"2023-10-06T11:37:21.027009+0000","last_became_peered":"2023-10-06T11:37:21.027009+0000","last_unstale":"2023-10-06T11:37:22.055863+0000","last_undegraded":"2023-10-06T11:37:22.055863+0000","last_fullsized":"2023-10-06T11:37:22.055863+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:16.905884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038486+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:37:22.038486+0000","last_peered":"2023-10-06T11:37:22.038486+0000","last_clean":"2023-10-06T11:37:22.038486+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:37:22.038486+0000","last_undegraded":"2023-10-06T11:37:22.038486+0000","last_fullsized":"2023-10-06T11:37:22.038486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311808,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060032,"statfs":{"total":96636764160,"available":96317472768,"internally_reserved":0,"allocated":2285568,"data_stored":1463130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1120000000000001,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.724},"last":1.466},{"interface":"front","average":{"1min":1.101,"5min":1.1379999999999999,"15min":1.1359999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.56399999999999995},"max":{"1min":1.7090000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":1.877}]},{"osd":1,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.327},"min":{"1min":0.89200000000000002,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.653,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.786},{"interface":"front","average":{"1min":1.224,"5min":1.228,"15min":1.236},"min":{"1min":0.57099999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5069999999999999,"5min":1.927,"15min":1.927},"last":1.1140000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.94399999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.554,"5min":1.6040000000000001,"15min":1.827},"last":1.173},{"interface":"front","average":{"1min":1.21,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.5649999999999999,"5min":1.7210000000000001,"15min":1.9830000000000001},"last":1.423}]},{"osd":3,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2350000000000001,"15min":1.268},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.68500000000000005},"max":{"1min":1.665,"5min":1.8,"15min":1.9450000000000001},"last":1.325},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3240000000000001,"15min":1.3200000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8580000000000001,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.6910000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1080000000000001,"15min":1.1140000000000001},"min":{"1min":0.66500000000000004,"5min":0.624,"15min":0.44600000000000001},"max":{"1min":1.893,"5min":1.893,"15min":2.149},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.41699999999999998},"max":{"1min":1.73,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5509999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.329,"15min":1.3069999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.58199999999999996},"max":{"1min":1.9330000000000001,"5min":2.488,"15min":2.488},"last":1.26},{"interface":"front","average":{"1min":1.272,"5min":1.268,"15min":1.268},"min":{"1min":0.94899999999999995,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":2.2480000000000002},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.331,"15min":1.3480000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.8500000000000001,"5min":2.0190000000000001,"15min":2.4289999999999998},"last":1.373},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":0.90600000000000003,"5min":0.76000000000000001,"15min":0.54200000000000004},"max":{"1min":1.819,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.827}]}]},{"osd":6,"up_from":13,"seq":55834575074,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":872568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.286},"min":{"1min":0.74099999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.774,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3380000000000001,"15min":1.3169999999999999},"min":{"1min":0.77200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9670000000000001,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.226}]},{"osd":1,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1539999999999999},"min":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.64000000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8169999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1899999999999999,"15min":1.208},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5940000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.4099999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.226,"15min":1.2070000000000001},"min":{"1min":0.92300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1280000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.69099999999999995},"max":{"1min":1.675,"5min":1.736,"15min":1.736},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.262,"15min":1.3029999999999999},"min":{"1min":0.80600000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.671,"5min":2.121,"15min":2.121},"last":1.631},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.339,"15min":1.304},"min":{"1min":1.0549999999999999,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.143}]},{"osd":4,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2390000000000001,"15min":1.222},"min":{"1min":0.81499999999999995,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.296},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2989999999999999,"15min":1.258},"min":{"1min":0.81499999999999995,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":2.0659999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.546}]},{"osd":5,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.216,"15min":1.2290000000000001},"min":{"1min":0.84899999999999998,"5min":0.79800000000000004,"15min":0.53700000000000003},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.377},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3799999999999999,"15min":1.379},"min":{"1min":0.89600000000000002,"5min":0.86499999999999999,"15min":0.71799999999999997},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.782}]},{"osd":7,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":0.877,"5min":0.84199999999999997,"15min":0.73899999999999999},"max":{"1min":2.206,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.68899999999999995},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.601}]}]},{"osd":5,"up_from":13,"seq":55834575074,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":872174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.232,"15min":1.28},"min":{"1min":0.84399999999999997,"5min":0.67400000000000004,"15min":0.5},"max":{"1min":1.5649999999999999,"5min":1.772,"15min":17.626000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.302,"15min":1.304},"min":{"1min":0.95699999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6850000000000001,"5min":1.966,"15min":2.3959999999999999},"last":1.413}]},{"osd":1,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.155,"15min":1.161},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.53200000000000003},"max":{"1min":1.4490000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1859999999999999,"15min":1.2090000000000001},"min":{"1min":0.86099999999999999,"5min":0.73799999999999999,"15min":0.58699999999999997},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":18.033000000000001},"last":1.1339999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1559999999999999,"15min":1.1659999999999999},"min":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.042},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2,"15min":1.2110000000000001},"min":{"1min":0.88900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4730000000000001,"5min":1.6990000000000001,"15min":1.738},"last":1.194}]},{"osd":3,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2350000000000001,"15min":1.28},"min":{"1min":0.88500000000000001,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.5309999999999999,"5min":1.6599999999999999,"15min":18.074999999999999},"last":1.331},{"interface":"front","average":{"1min":1.3,"5min":1.266,"15min":1.2969999999999999},"min":{"1min":0.96899999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5509999999999999,"5min":1.8400000000000001,"15min":1.871},"last":1.2689999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2470000000000001,"15min":1.2629999999999999},"min":{"1min":0.80400000000000005,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.6080000000000001,"5min":1.7390000000000001,"15min":8.5649999999999995},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.347,"15min":1.377},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.63500000000000001},"max":{"1min":1.5960000000000001,"5min":1.887,"15min":18.48},"last":1.5529999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.2669999999999999},"min":{"1min":0.78400000000000003,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.7450000000000001,"5min":1.8069999999999999,"15min":8.5530000000000008},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2869999999999999,"15min":1.3180000000000001},"min":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.6759999999999999,"5min":1.885,"15min":9.8559999999999999},"last":1.375}]},{"osd":7,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3009999999999999,"15min":1.3620000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.755},"max":{"1min":1.647,"5min":1.8280000000000001,"15min":18.379999999999999},"last":1.47},{"interface":"front","average":{"1min":1.381,"5min":1.365,"15min":1.3939999999999999},"min":{"1min":0.89600000000000002,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":8.4830000000000005},"last":1.609}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307128,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064712,"statfs":{"total":96636764160,"available":96322265088,"internally_reserved":0,"allocated":1687552,"data_stored":873550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1719999999999999,"15min":1.151},"min":{"1min":0.77800000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5069999999999999,"5min":1.72,"15min":1.72},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.123,"15min":1.1279999999999999},"min":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.6299999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.294,"15min":1.3109999999999999},"min":{"1min":1.1060000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.034},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.996,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.5800000000000001,"5min":1.78,"15min":1.78},"last":1.254}]},{"osd":2,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.71099999999999997,"5min":0.67000000000000004,"15min":0.628},"max":{"1min":1.456,"5min":1.736,"15min":1.736},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.246,"15min":1.2609999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6220000000000001,"5min":1.6619999999999999,"15min":1.6779999999999999},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1899999999999999,"15min":1.2050000000000001},"min":{"1min":0.78800000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.508,"5min":1.617,"15min":1.6839999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":1.018,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.607,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.621,"15min":0.621},"max":{"1min":1.6379999999999999,"5min":1.706,"15min":1.7390000000000001},"last":1.143},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.204,"15min":1.1930000000000001},"min":{"1min":0.74099999999999999,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.635,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.4350000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.70399999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6619999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.528},{"interface":"front","average":{"1min":1.429,"5min":1.361,"15min":1.353},"min":{"1min":1.1279999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7669999999999999,"5min":1.845,"15min":1.845},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.351,"15min":1.341},"min":{"1min":0.97999999999999998,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.649,"5min":1.897,"15min":1.897},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.3979999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.73799999999999999},"max":{"1min":1.8300000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.617}]}]},{"osd":3,"up_from":13,"seq":55834575076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311216,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060624,"statfs":{"total":96636764160,"available":96318078976,"internally_reserved":0,"allocated":1679360,"data_stored":866471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0660000000000001,"15min":0.93500000000000005},"min":{"1min":0.50800000000000001,"5min":0.44700000000000001,"15min":0.36599999999999999},"max":{"1min":1.3500000000000001,"5min":14.49,"15min":14.49},"last":0.997},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.2,"15min":1.056},"min":{"1min":0.78100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.331,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.96399999999999997},"min":{"1min":0.69499999999999995,"5min":0.56499999999999995,"15min":0.55300000000000005},"max":{"1min":1.5389999999999999,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.1679999999999999,"15min":1.115},"min":{"1min":0.69999999999999996,"5min":0.501,"15min":0.48199999999999998},"max":{"1min":1.552,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.89400000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.2509999999999999,"15min":1.1599999999999999},"min":{"1min":0.85599999999999998,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.52,"5min":12.446,"15min":12.446},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2669999999999999,"15min":1.2230000000000001},"min":{"1min":0.77400000000000002,"5min":0.626,"15min":0.44},"max":{"1min":1.6699999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.3400000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.341,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.84199999999999997,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":10.460000000000001,"15min":10.460000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.071,"5min":0.92700000000000005,"15min":0.78400000000000003},"max":{"1min":1.627,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4259999999999999,"15min":1.405},"min":{"1min":1.111,"5min":0.92100000000000004,"15min":0.69999999999999996},"max":{"1min":1.99,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.448,"15min":1.403},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.69499999999999995},"max":{"1min":2.2490000000000001,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.1040000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.556,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3839999999999999,"5min":11.362,"15min":11.362},"last":1.22},{"interface":"front","average":{"1min":1.405,"5min":1.466,"15min":1.399},"min":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80200000000000005},"max":{"1min":2.0259999999999998,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.5449999999999999,"15min":1.4610000000000001},"min":{"1min":1.018,"5min":1.012,"15min":0.84999999999999998},"max":{"1min":1.9490000000000001,"5min":10.561,"15min":10.561},"last":1.157},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.51,"15min":1.486},"min":{"1min":1.177,"5min":0.85899999999999999,"15min":0.83999999999999997},"max":{"1min":1.9359999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.2}]}]},{"osd":2,"up_from":13,"seq":55834575074,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311200,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060640,"statfs":{"total":96636764160,"available":96318095360,"internally_reserved":0,"allocated":1662976,"data_stored":864901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.96899999999999997,"15min":0.96599999999999997},"min":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.36299999999999999},"max":{"1min":1.4299999999999999,"5min":1.524,"15min":1.8380000000000001},"last":1.454},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0680000000000001},"min":{"1min":0.74099999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4299999999999999,"5min":1.5409999999999999,"15min":1.5629999999999999},"last":0.95599999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92400000000000004},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.161,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0349999999999999,"15min":1.02},"min":{"1min":0.75800000000000001,"5min":0.59399999999999997,"15min":0.439},"max":{"1min":1.855,"5min":1.855,"15min":2.5390000000000001},"last":1.708}]},{"osd":3,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1599999999999999,"15min":1.1850000000000001},"min":{"1min":0.79000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.89700000000000002,"5min":0.745,"15min":0.60699999999999998},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.4550000000000001},"last":1.9670000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.226,"15min":1.2210000000000001},"min":{"1min":0.95999999999999996,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.647,"5min":1.647,"15min":1.754},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.2869999999999999},"min":{"1min":0.97899999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6279999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.7849999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3779999999999999,"15min":1.3640000000000001},"min":{"1min":1.1060000000000001,"5min":0.91800000000000004,"15min":0.85199999999999998},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.8440000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.302,"15min":1.327},"min":{"1min":0.871,"5min":0.751,"15min":0.751},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.0630000000000002},"last":1.196}]},{"osd":6,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.423,"15min":1.3999999999999999},"min":{"1min":1.0980000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7769999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.4370000000000001,"15min":1.4419999999999999},"min":{"1min":1.046,"5min":1,"15min":0.76900000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.617},"last":1.915}]},{"osd":7,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":1.05,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7809999999999999,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.4910000000000001,"15min":1.472},"min":{"1min":1.177,"5min":0.753,"15min":0.753},"max":{"1min":2.024,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":872036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86699999999999999,"15min":0.91400000000000003},"min":{"1min":0.66100000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.724},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.016,"5min":0.99199999999999999,"15min":0.96999999999999997},"min":{"1min":0.66600000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.595,"5min":1.827,"15min":1.827},"last":1.4410000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.93600000000000005,"15min":0.96799999999999997},"min":{"1min":0.60999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.429,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.056,"15min":1.0980000000000001},"min":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.569,"5min":1.6859999999999999,"15min":2.137},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1639999999999999,"15min":1.2070000000000001},"min":{"1min":0.81899999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6879999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0800000000000001,"15min":1.0940000000000001},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.544,"5min":1.5549999999999999,"15min":1.756},"last":1.133}]},{"osd":4,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.226,"15min":1.2410000000000001},"min":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5149999999999999,"5min":1.8580000000000001,"15min":2.2290000000000001},"last":1.349},{"interface":"front","average":{"1min":1.337,"5min":1.353,"15min":1.3740000000000001},"min":{"1min":0.997,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.748,"5min":1.8,"15min":1.944},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.3200000000000001},"min":{"1min":0.92800000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.599,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":1.923,"15min":2.1240000000000001},"last":1.544}]},{"osd":6,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4119999999999999,"15min":1.456},"min":{"1min":1.0660000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.0310000000000001},"last":1.494},{"interface":"front","average":{"1min":1.357,"5min":1.3819999999999999,"15min":1.3879999999999999},"min":{"1min":1.018,"5min":0.92200000000000004,"15min":0.72299999999999998},"max":{"1min":1.7230000000000001,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.845}]},{"osd":7,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.462,"15min":1.4910000000000001},"min":{"1min":1.0349999999999999,"5min":0.88,"15min":0.83599999999999997},"max":{"1min":1.819,"5min":2.0209999999999999,"15min":2.0390000000000001},"last":1.944},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.478,"15min":1.4570000000000001},"min":{"1min":0.94899999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.875,"5min":2.085,"15min":2.085},"last":1.619}]}]},{"osd":0,"up_from":13,"seq":55834575075,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311796,"kb_used_data":2220,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060044,"statfs":{"total":96636764160,"available":96317485056,"internally_reserved":0,"allocated":2273280,"data_stored":1462404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.874,"15min":0.88400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.3979999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.628,"5min":0.498,"15min":0.498},"max":{"1min":1.3180000000000001,"5min":1.6180000000000001,"15min":1.6950000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.0980000000000001,"15min":1.0840000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.7169999999999996,"5min":4.7169999999999996,"15min":4.7169999999999996},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.919},"last":0.997},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.274,"15min":1.3129999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.841,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.97499999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2829999999999999,"15min":1.3320000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.792,"5min":1.8440000000000001,"15min":2.294},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.978,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7909999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3819999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.65900000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.25},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.786,"5min":1.893,"15min":2.5510000000000002},"last":1.716}]},{"osd":6,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.3540000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.458,"15min":1.458},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4750000000000001},"last":1.349}]},{"osd":7,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.415,"15min":1.4530000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6799999999999999,"5min":2.012,"15min":2.4260000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.425,"15min":1.4239999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.3809999999999998},"last":1.897}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:37:26.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:37:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 Processor -- start 2023-10-06T11:37:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 -- start start 2023-10-06T11:37:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe4bc15c770 0x7fe4bc0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0ccb50 0x7fe4bc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4bc0cfbb0 con 0x7fe4bc0cea30 2023-10-06T11:37:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4bc0cdbf0 con 0x7fe4bc15c770 2023-10-06T11:37:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4c14ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0cc150 con 0x7fe4bc0ccb50 2023-10-06T11:37:26.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4bb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4bb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47626/0 (socket says 172.21.15.46:47626) 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.519+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4151334293 learned_addr learned my addr 172.21.15.46:0/4151334293 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4baffd700 1 --2- 172.21.15.46:0/4151334293 >> v2:172.21.15.132:3300/0 conn(0x7fe4bc15c770 0x7fe4bc0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4ba7fc700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0ccb50 0x7fe4bc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4151334293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0ccb50 msgr2=0x7fe4bc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0ccb50 0x7fe4bc0ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4151334293 >> v2:172.21.15.132:3300/0 conn(0x7fe4bc15c770 msgr2=0x7fe4bc0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4151334293 >> v2:172.21.15.132:3300/0 conn(0x7fe4bc15c770 0x7fe4bc0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4151334293 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc0cee70 con 0x7fe4bc0cea30 2023-10-06T11:37:26.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0cf000 tx=0x7fe4b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a3f2cb8ea30fbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4b9ffb700 1 -- 172.21.15.46:0/4151334293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400e070 con 0x7fe4bc0cea30 2023-10-06T11:37:26.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4b9ffb700 1 -- 172.21.15.46:0/4151334293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400ba50 con 0x7fe4bc0cea30 2023-10-06T11:37:26.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4b9ffb700 1 -- 172.21.15.46:0/4151334293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400be10 con 0x7fe4bc0cea30 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4151334293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc0cea30 msgr2=0x7fe4bc0c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0cf000 tx=0x7fe4b4009840 comp rx=0 tx=0).stop 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4151334293 shutdown_connections 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc0cea30 0x7fe4bc0c4fe0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4151334293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0ccb50 0x7fe4bc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4151334293 >> v2:172.21.15.132:3300/0 conn(0x7fe4bc15c770 0x7fe4bc0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4151334293 >> 172.21.15.46:0/4151334293 conn(0x7fe4bc099480 msgr2=0x7fe4bc06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4151334293 shutdown_connections 2023-10-06T11:37:26.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.523+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4151334293 wait complete. 2023-10-06T11:37:26.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 Processor -- start 2023-10-06T11:37:26.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 -- start start 2023-10-06T11:37:26.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4bc0ccb50 0x7fe4bc1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0cea30 0x7fe4bc0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0c8210 con 0x7fe4bc15c770 2023-10-06T11:37:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0c7600 con 0x7fe4bc0ccb50 2023-10-06T11:37:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4c14ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc0c78e0 con 0x7fe4bc0cea30 2023-10-06T11:37:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47636/0 (socket says 172.21.15.46:47636) 2023-10-06T11:37:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4202008946 learned_addr learned my addr 172.21.15.46:0/4202008946 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4baffd700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4bc0ccb50 0x7fe4bc1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4ba7fc700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0cea30 0x7fe4bc0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0cea30 msgr2=0x7fe4bc0c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0cea30 0x7fe4bc0c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4bc0ccb50 msgr2=0x7fe4bc1536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.527+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4bc0ccb50 0x7fe4bc1536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4bb7fe700 1 -- 172.21.15.46:0/4202008946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4009ae0 con 0x7fe4bc15c770 2023-10-06T11:37:26.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4bb7fe700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4003000 tx=0x7fe4b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40094e0 con 0x7fe4bc15c770 2023-10-06T11:37:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b40032f0 con 0x7fe4bc15c770 2023-10-06T11:37:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b401d420 con 0x7fe4bc15c770 2023-10-06T11:37:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc0c7bc0 con 0x7fe4bc15c770 2023-10-06T11:37:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc0c72f0 con 0x7fe4bc15c770 2023-10-06T11:37:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.531+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc0660f0 con 0x7fe4bc15c770 2023-10-06T11:37:26.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.535+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe4b401d810 con 0x7fe4bc15c770 2023-10-06T11:37:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.535+0000 7fe4a2ffd700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 0x7fe4a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.535+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fe4b407e480 con 0x7fe4bc15c770 2023-10-06T11:37:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.535+0000 7fe4baffd700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 0x7fe4a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.535+0000 7fe4baffd700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 0x7fe4a8062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0061f0 tx=0x7fe4ac008040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.543+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe4b404e470 con 0x7fe4bc15c770 2023-10-06T11:37:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.707+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4bc0cb280 con 0x7fe4a8060670 2023-10-06T11:37:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.723+0000 7fe4a2ffd700 1 -- 172.21.15.46:0/4202008946 <== mgr.6956 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309325 (secure 0 0 0) 0x7fe4bc0cb280 con 0x7fe4a8060670 2023-10-06T11:37:26.741 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 msgr2=0x7fe4a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 0x7fe4a8062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0061f0 tx=0x7fe4ac008040 comp rx=0 tx=0).stop 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 msgr2=0x7fe4bc157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4003000 tx=0x7fe4b4017000 comp rx=0 tx=0).stop 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 shutdown_connections 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe4a8060670 0x7fe4a8062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4bc15c770 0x7fe4bc157fa0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4bc0cea30 0x7fe4bc0c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 --2- 172.21.15.46:0/4202008946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4bc0ccb50 0x7fe4bc1536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.731+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 >> 172.21.15.46:0/4202008946 conn(0x7fe4bc099480 msgr2=0x7fe4bc0c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.735+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 shutdown_connections 2023-10-06T11:37:26.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.735+0000 7fe4c14ae700 1 -- 172.21.15.46:0/4202008946 wait complete. 2023-10-06T11:37:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T11:37:26.768 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T11:37:26.653045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2470416,"kb_used_data":14288,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752504304,"statfs":{"total":773094113280,"available":770564407296,"internally_reserved":0,"allocated":14630912,"data_stored":8147234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001832"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037361+0000","last_change":"2023-10-06T11:37:22.031626+0000","last_active":"2023-10-06T11:37:22.037361+0000","last_peered":"2023-10-06T11:37:22.037361+0000","last_clean":"2023-10-06T11:37:22.037361+0000","last_became_active":"2023-10-06T11:37:22.031316+0000","last_became_peered":"2023-10-06T11:37:22.031316+0000","last_unstale":"2023-10-06T11:37:22.037361+0000","last_undegraded":"2023-10-06T11:37:22.037361+0000","last_fullsized":"2023-10-06T11:37:22.037361+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:24.545371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.023338+0000","last_change":"2023-10-06T11:37:22.019305+0000","last_active":"2023-10-06T11:37:22.023338+0000","last_peered":"2023-10-06T11:37:22.023338+0000","last_clean":"2023-10-06T11:37:22.023338+0000","last_became_active":"2023-10-06T11:37:22.018984+0000","last_became_peered":"2023-10-06T11:37:22.018984+0000","last_unstale":"2023-10-06T11:37:22.023338+0000","last_undegraded":"2023-10-06T11:37:22.023338+0000","last_fullsized":"2023-10-06T11:37:22.023338+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.531278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052300+0000","last_change":"2023-10-06T11:37:22.047336+0000","last_active":"2023-10-06T11:37:22.052300+0000","last_peered":"2023-10-06T11:37:22.052300+0000","last_clean":"2023-10-06T11:37:22.052300+0000","last_became_active":"2023-10-06T11:37:22.047036+0000","last_became_peered":"2023-10-06T11:37:22.047036+0000","last_unstale":"2023-10-06T11:37:22.052300+0000","last_undegraded":"2023-10-06T11:37:22.052300+0000","last_fullsized":"2023-10-06T11:37:22.052300+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:47.029371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042308+0000","last_change":"2023-10-06T11:37:22.032093+0000","last_active":"2023-10-06T11:37:22.042308+0000","last_peered":"2023-10-06T11:37:22.042308+0000","last_clean":"2023-10-06T11:37:22.042308+0000","last_became_active":"2023-10-06T11:37:22.031761+0000","last_became_peered":"2023-10-06T11:37:22.031761+0000","last_unstale":"2023-10-06T11:37:22.042308+0000","last_undegraded":"2023-10-06T11:37:22.042308+0000","last_fullsized":"2023-10-06T11:37:22.042308+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042617+0000","last_change":"2023-10-06T11:37:22.041309+0000","last_active":"2023-10-06T11:37:22.042617+0000","last_peered":"2023-10-06T11:37:22.042617+0000","last_clean":"2023-10-06T11:37:22.042617+0000","last_became_active":"2023-10-06T11:37:22.040981+0000","last_became_peered":"2023-10-06T11:37:22.040981+0000","last_unstale":"2023-10-06T11:37:22.042617+0000","last_undegraded":"2023-10-06T11:37:22.042617+0000","last_fullsized":"2023-10-06T11:37:22.042617+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054162+0000","last_change":"2023-10-06T11:37:22.053329+0000","last_active":"2023-10-06T11:37:22.054162+0000","last_peered":"2023-10-06T11:37:22.054162+0000","last_clean":"2023-10-06T11:37:22.054162+0000","last_became_active":"2023-10-06T11:37:22.052882+0000","last_became_peered":"2023-10-06T11:37:22.052882+0000","last_unstale":"2023-10-06T11:37:22.054162+0000","last_undegraded":"2023-10-06T11:37:22.054162+0000","last_fullsized":"2023-10-06T11:37:22.054162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:26.113043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043095+0000","last_change":"2023-10-06T11:37:22.038395+0000","last_active":"2023-10-06T11:37:22.043095+0000","last_peered":"2023-10-06T11:37:22.043095+0000","last_clean":"2023-10-06T11:37:22.043095+0000","last_became_active":"2023-10-06T11:37:22.038101+0000","last_became_peered":"2023-10-06T11:37:22.038101+0000","last_unstale":"2023-10-06T11:37:22.043095+0000","last_undegraded":"2023-10-06T11:37:22.043095+0000","last_fullsized":"2023-10-06T11:37:22.043095+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:00:22.522026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054626+0000","last_change":"2023-10-06T11:37:22.049679+0000","last_active":"2023-10-06T11:37:22.054626+0000","last_peered":"2023-10-06T11:37:22.054626+0000","last_clean":"2023-10-06T11:37:22.054626+0000","last_became_active":"2023-10-06T11:37:22.049343+0000","last_became_peered":"2023-10-06T11:37:22.049343+0000","last_unstale":"2023-10-06T11:37:22.054626+0000","last_undegraded":"2023-10-06T11:37:22.054626+0000","last_fullsized":"2023-10-06T11:37:22.054626+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:00.137182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.041996+0000","last_change":"2023-10-06T11:37:22.029546+0000","last_active":"2023-10-06T11:37:22.041996+0000","last_peered":"2023-10-06T11:37:22.041996+0000","last_clean":"2023-10-06T11:37:22.041996+0000","last_became_active":"2023-10-06T11:37:22.029249+0000","last_became_peered":"2023-10-06T11:37:22.029249+0000","last_unstale":"2023-10-06T11:37:22.041996+0000","last_undegraded":"2023-10-06T11:37:22.041996+0000","last_fullsized":"2023-10-06T11:37:22.041996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:14.305263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038238+0000","last_change":"2023-10-06T11:37:22.034103+0000","last_active":"2023-10-06T11:37:22.038238+0000","last_peered":"2023-10-06T11:37:22.038238+0000","last_clean":"2023-10-06T11:37:22.038238+0000","last_became_active":"2023-10-06T11:37:22.032705+0000","last_became_peered":"2023-10-06T11:37:22.032705+0000","last_unstale":"2023-10-06T11:37:22.038238+0000","last_undegraded":"2023-10-06T11:37:22.038238+0000","last_fullsized":"2023-10-06T11:37:22.038238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:38.464047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.033582+0000","last_change":"2023-10-06T11:37:22.031305+0000","last_active":"2023-10-06T11:37:22.033582+0000","last_peered":"2023-10-06T11:37:22.033582+0000","last_clean":"2023-10-06T11:37:22.033582+0000","last_became_active":"2023-10-06T11:37:22.030910+0000","last_became_peered":"2023-10-06T11:37:22.030910+0000","last_unstale":"2023-10-06T11:37:22.033582+0000","last_undegraded":"2023-10-06T11:37:22.033582+0000","last_fullsized":"2023-10-06T11:37:22.033582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:30.187438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039749+0000","last_change":"2023-10-06T11:37:22.035159+0000","last_active":"2023-10-06T11:37:22.039749+0000","last_peered":"2023-10-06T11:37:22.039749+0000","last_clean":"2023-10-06T11:37:22.039749+0000","last_became_active":"2023-10-06T11:37:22.034431+0000","last_became_peered":"2023-10-06T11:37:22.034431+0000","last_unstale":"2023-10-06T11:37:22.039749+0000","last_undegraded":"2023-10-06T11:37:22.039749+0000","last_fullsized":"2023-10-06T11:37:22.039749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:17:39.042323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.028872+0000","last_change":"2023-10-06T11:37:22.022283+0000","last_active":"2023-10-06T11:37:22.028872+0000","last_peered":"2023-10-06T11:37:22.028872+0000","last_clean":"2023-10-06T11:37:22.028872+0000","last_became_active":"2023-10-06T11:37:22.021776+0000","last_became_peered":"2023-10-06T11:37:22.021776+0000","last_unstale":"2023-10-06T11:37:22.028872+0000","last_undegraded":"2023-10-06T11:37:22.028872+0000","last_fullsized":"2023-10-06T11:37:22.028872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:28.975870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052293+0000","last_change":"2023-10-06T11:37:22.048496+0000","last_active":"2023-10-06T11:37:22.052293+0000","last_peered":"2023-10-06T11:37:22.052293+0000","last_clean":"2023-10-06T11:37:22.052293+0000","last_became_active":"2023-10-06T11:37:22.047873+0000","last_became_peered":"2023-10-06T11:37:22.047873+0000","last_unstale":"2023-10-06T11:37:22.052293+0000","last_undegraded":"2023-10-06T11:37:22.052293+0000","last_fullsized":"2023-10-06T11:37:22.052293+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:53.672153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.033423+0000","last_change":"2023-10-06T11:37:22.029792+0000","last_active":"2023-10-06T11:37:22.033423+0000","last_peered":"2023-10-06T11:37:22.033423+0000","last_clean":"2023-10-06T11:37:22.033423+0000","last_became_active":"2023-10-06T11:37:22.029499+0000","last_became_peered":"2023-10-06T11:37:22.029499+0000","last_unstale":"2023-10-06T11:37:22.033423+0000","last_undegraded":"2023-10-06T11:37:22.033423+0000","last_fullsized":"2023-10-06T11:37:22.033423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:15.148919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.049969+0000","last_change":"2023-10-06T11:37:22.048575+0000","last_active":"2023-10-06T11:37:22.049969+0000","last_peered":"2023-10-06T11:37:22.049969+0000","last_clean":"2023-10-06T11:37:22.049969+0000","last_became_active":"2023-10-06T11:37:22.048257+0000","last_became_peered":"2023-10-06T11:37:22.048257+0000","last_unstale":"2023-10-06T11:37:22.049969+0000","last_undegraded":"2023-10-06T11:37:22.049969+0000","last_fullsized":"2023-10-06T11:37:22.049969+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:14.345894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038875+0000","last_change":"2023-10-06T11:37:22.035907+0000","last_active":"2023-10-06T11:37:22.038875+0000","last_peered":"2023-10-06T11:37:22.038875+0000","last_clean":"2023-10-06T11:37:22.038875+0000","last_became_active":"2023-10-06T11:37:22.035598+0000","last_became_peered":"2023-10-06T11:37:22.035598+0000","last_unstale":"2023-10-06T11:37:22.038875+0000","last_undegraded":"2023-10-06T11:37:22.038875+0000","last_fullsized":"2023-10-06T11:37:22.038875+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:17.910505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034617+0000","last_change":"2023-10-06T11:37:22.026104+0000","last_active":"2023-10-06T11:37:22.034617+0000","last_peered":"2023-10-06T11:37:22.034617+0000","last_clean":"2023-10-06T11:37:22.034617+0000","last_became_active":"2023-10-06T11:37:22.025816+0000","last_became_peered":"2023-10-06T11:37:22.025816+0000","last_unstale":"2023-10-06T11:37:22.034617+0000","last_undegraded":"2023-10-06T11:37:22.034617+0000","last_fullsized":"2023-10-06T11:37:22.034617+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.479961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051823+0000","last_change":"2023-10-06T11:37:22.046837+0000","last_active":"2023-10-06T11:37:22.051823+0000","last_peered":"2023-10-06T11:37:22.051823+0000","last_clean":"2023-10-06T11:37:22.051823+0000","last_became_active":"2023-10-06T11:37:22.046546+0000","last_became_peered":"2023-10-06T11:37:22.046546+0000","last_unstale":"2023-10-06T11:37:22.051823+0000","last_undegraded":"2023-10-06T11:37:22.051823+0000","last_fullsized":"2023-10-06T11:37:22.051823+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:33.479961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047027+0000","last_change":"2023-10-06T11:37:22.031268+0000","last_active":"2023-10-06T11:37:22.047027+0000","last_peered":"2023-10-06T11:37:22.047027+0000","last_clean":"2023-10-06T11:37:22.047027+0000","last_became_active":"2023-10-06T11:37:22.030975+0000","last_became_peered":"2023-10-06T11:37:22.030975+0000","last_unstale":"2023-10-06T11:37:22.047027+0000","last_undegraded":"2023-10-06T11:37:22.047027+0000","last_fullsized":"2023-10-06T11:37:22.047027+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:03:28.975870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042310+0000","last_change":"2023-10-06T11:37:22.032421+0000","last_active":"2023-10-06T11:37:22.042310+0000","last_peered":"2023-10-06T11:37:22.042310+0000","last_clean":"2023-10-06T11:37:22.042310+0000","last_became_active":"2023-10-06T11:37:22.031935+0000","last_became_peered":"2023-10-06T11:37:22.031935+0000","last_unstale":"2023-10-06T11:37:22.042310+0000","last_undegraded":"2023-10-06T11:37:22.042310+0000","last_fullsized":"2023-10-06T11:37:22.042310+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:48.820206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039833+0000","last_change":"2023-10-06T11:37:22.034731+0000","last_active":"2023-10-06T11:37:22.039833+0000","last_peered":"2023-10-06T11:37:22.039833+0000","last_clean":"2023-10-06T11:37:22.039833+0000","last_became_active":"2023-10-06T11:37:22.031456+0000","last_became_peered":"2023-10-06T11:37:22.031456+0000","last_unstale":"2023-10-06T11:37:22.039833+0000","last_undegraded":"2023-10-06T11:37:22.039833+0000","last_fullsized":"2023-10-06T11:37:22.039833+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:39.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042188+0000","last_change":"2023-10-06T11:37:22.030170+0000","last_active":"2023-10-06T11:37:22.042188+0000","last_peered":"2023-10-06T11:37:22.042188+0000","last_clean":"2023-10-06T11:37:22.042188+0000","last_became_active":"2023-10-06T11:37:22.029827+0000","last_became_peered":"2023-10-06T11:37:22.029827+0000","last_unstale":"2023-10-06T11:37:22.042188+0000","last_undegraded":"2023-10-06T11:37:22.042188+0000","last_fullsized":"2023-10-06T11:37:22.042188+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:51.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050831+0000","last_change":"2023-10-06T11:37:22.046077+0000","last_active":"2023-10-06T11:37:22.050831+0000","last_peered":"2023-10-06T11:37:22.050831+0000","last_clean":"2023-10-06T11:37:22.050831+0000","last_became_active":"2023-10-06T11:37:22.045747+0000","last_became_peered":"2023-10-06T11:37:22.045747+0000","last_unstale":"2023-10-06T11:37:22.050831+0000","last_undegraded":"2023-10-06T11:37:22.050831+0000","last_fullsized":"2023-10-06T11:37:22.050831+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:48.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045053+0000","last_change":"2023-10-06T11:37:22.038325+0000","last_active":"2023-10-06T11:37:22.045053+0000","last_peered":"2023-10-06T11:37:22.045053+0000","last_clean":"2023-10-06T11:37:22.045053+0000","last_became_active":"2023-10-06T11:37:22.038041+0000","last_became_peered":"2023-10-06T11:37:22.038041+0000","last_unstale":"2023-10-06T11:37:22.045053+0000","last_undegraded":"2023-10-06T11:37:22.045053+0000","last_fullsized":"2023-10-06T11:37:22.045053+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047681+0000","last_change":"2023-10-06T11:37:22.027301+0000","last_active":"2023-10-06T11:37:22.047681+0000","last_peered":"2023-10-06T11:37:22.047681+0000","last_clean":"2023-10-06T11:37:22.047681+0000","last_became_active":"2023-10-06T11:37:22.026899+0000","last_became_peered":"2023-10-06T11:37:22.026899+0000","last_unstale":"2023-10-06T11:37:22.047681+0000","last_undegraded":"2023-10-06T11:37:22.047681+0000","last_fullsized":"2023-10-06T11:37:22.047681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:44.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051806+0000","last_change":"2023-10-06T11:37:22.046758+0000","last_active":"2023-10-06T11:37:22.051806+0000","last_peered":"2023-10-06T11:37:22.051806+0000","last_clean":"2023-10-06T11:37:22.051806+0000","last_became_active":"2023-10-06T11:37:22.046123+0000","last_became_peered":"2023-10-06T11:37:22.046123+0000","last_unstale":"2023-10-06T11:37:22.051806+0000","last_undegraded":"2023-10-06T11:37:22.051806+0000","last_fullsized":"2023-10-06T11:37:22.051806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:51.465644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043185+0000","last_change":"2023-10-06T11:37:22.038611+0000","last_active":"2023-10-06T11:37:22.043185+0000","last_peered":"2023-10-06T11:37:22.043185+0000","last_clean":"2023-10-06T11:37:22.043185+0000","last_became_active":"2023-10-06T11:37:22.038158+0000","last_became_peered":"2023-10-06T11:37:22.038158+0000","last_unstale":"2023-10-06T11:37:22.043185+0000","last_undegraded":"2023-10-06T11:37:22.043185+0000","last_fullsized":"2023-10-06T11:37:22.043185+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:22.314707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024944+0000","last_change":"2023-10-06T11:37:22.023177+0000","last_active":"2023-10-06T11:37:22.024944+0000","last_peered":"2023-10-06T11:37:22.024944+0000","last_clean":"2023-10-06T11:37:22.024944+0000","last_became_active":"2023-10-06T11:37:22.022877+0000","last_became_peered":"2023-10-06T11:37:22.022877+0000","last_unstale":"2023-10-06T11:37:22.024944+0000","last_undegraded":"2023-10-06T11:37:22.024944+0000","last_fullsized":"2023-10-06T11:37:22.024944+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:32.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054716+0000","last_change":"2023-10-06T11:37:22.051644+0000","last_active":"2023-10-06T11:37:22.054716+0000","last_peered":"2023-10-06T11:37:22.054716+0000","last_clean":"2023-10-06T11:37:22.054716+0000","last_became_active":"2023-10-06T11:37:22.040424+0000","last_became_peered":"2023-10-06T11:37:22.040424+0000","last_unstale":"2023-10-06T11:37:22.054716+0000","last_undegraded":"2023-10-06T11:37:22.054716+0000","last_fullsized":"2023-10-06T11:37:22.054716+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:11:30.066954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043594+0000","last_change":"2023-10-06T11:37:22.033913+0000","last_active":"2023-10-06T11:37:22.043594+0000","last_peered":"2023-10-06T11:37:22.043594+0000","last_clean":"2023-10-06T11:37:22.043594+0000","last_became_active":"2023-10-06T11:37:22.033625+0000","last_became_peered":"2023-10-06T11:37:22.033625+0000","last_unstale":"2023-10-06T11:37:22.043594+0000","last_undegraded":"2023-10-06T11:37:22.043594+0000","last_fullsized":"2023-10-06T11:37:22.043594+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:20:15.111882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047809+0000","last_change":"2023-10-06T11:37:22.044036+0000","last_active":"2023-10-06T11:37:22.047809+0000","last_peered":"2023-10-06T11:37:22.047809+0000","last_clean":"2023-10-06T11:37:22.047809+0000","last_became_active":"2023-10-06T11:37:22.043696+0000","last_became_peered":"2023-10-06T11:37:22.043696+0000","last_unstale":"2023-10-06T11:37:22.047809+0000","last_undegraded":"2023-10-06T11:37:22.047809+0000","last_fullsized":"2023-10-06T11:37:22.047809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:14.305263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034465+0000","last_change":"2023-10-06T11:37:22.033061+0000","last_active":"2023-10-06T11:37:22.034465+0000","last_peered":"2023-10-06T11:37:22.034465+0000","last_clean":"2023-10-06T11:37:22.034465+0000","last_became_active":"2023-10-06T11:37:22.032667+0000","last_became_peered":"2023-10-06T11:37:22.032667+0000","last_unstale":"2023-10-06T11:37:22.034465+0000","last_undegraded":"2023-10-06T11:37:22.034465+0000","last_fullsized":"2023-10-06T11:37:22.034465+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:46.600662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"134'4","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057739+0000","last_change":"2023-10-06T11:37:21.031884+0000","last_active":"2023-10-06T11:37:22.057739+0000","last_peered":"2023-10-06T11:37:22.057739+0000","last_clean":"2023-10-06T11:37:22.057739+0000","last_became_active":"2023-10-06T11:37:21.031582+0000","last_became_peered":"2023-10-06T11:37:21.031582+0000","last_unstale":"2023-10-06T11:37:22.057739+0000","last_undegraded":"2023-10-06T11:37:22.057739+0000","last_fullsized":"2023-10-06T11:37:22.057739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:51.839254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050904+0000","last_change":"2023-10-06T11:37:22.049010+0000","last_active":"2023-10-06T11:37:22.050904+0000","last_peered":"2023-10-06T11:37:22.050904+0000","last_clean":"2023-10-06T11:37:22.050904+0000","last_became_active":"2023-10-06T11:37:22.048699+0000","last_became_peered":"2023-10-06T11:37:22.048699+0000","last_unstale":"2023-10-06T11:37:22.050904+0000","last_undegraded":"2023-10-06T11:37:22.050904+0000","last_fullsized":"2023-10-06T11:37:22.050904+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:57.434772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048025+0000","last_change":"2023-10-06T11:37:21.022792+0000","last_active":"2023-10-06T11:37:22.048025+0000","last_peered":"2023-10-06T11:37:22.048025+0000","last_clean":"2023-10-06T11:37:22.048025+0000","last_became_active":"2023-10-06T11:37:21.022508+0000","last_became_peered":"2023-10-06T11:37:21.022508+0000","last_unstale":"2023-10-06T11:37:22.048025+0000","last_undegraded":"2023-10-06T11:37:22.048025+0000","last_fullsized":"2023-10-06T11:37:22.048025+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:41.448824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048059+0000","last_change":"2023-10-06T11:37:22.043065+0000","last_active":"2023-10-06T11:37:22.048059+0000","last_peered":"2023-10-06T11:37:22.048059+0000","last_clean":"2023-10-06T11:37:22.048059+0000","last_became_active":"2023-10-06T11:37:22.041296+0000","last_became_peered":"2023-10-06T11:37:22.041296+0000","last_unstale":"2023-10-06T11:37:22.048059+0000","last_undegraded":"2023-10-06T11:37:22.048059+0000","last_fullsized":"2023-10-06T11:37:22.048059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:22.301160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057784+0000","last_change":"2023-10-06T11:37:21.031883+0000","last_active":"2023-10-06T11:37:22.057784+0000","last_peered":"2023-10-06T11:37:22.057784+0000","last_clean":"2023-10-06T11:37:22.057784+0000","last_became_active":"2023-10-06T11:37:21.031568+0000","last_became_peered":"2023-10-06T11:37:21.031568+0000","last_unstale":"2023-10-06T11:37:22.057784+0000","last_undegraded":"2023-10-06T11:37:22.057784+0000","last_fullsized":"2023-10-06T11:37:22.057784+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:19.518142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042259+0000","last_change":"2023-10-06T11:37:22.032346+0000","last_active":"2023-10-06T11:37:22.042259+0000","last_peered":"2023-10-06T11:37:22.042259+0000","last_clean":"2023-10-06T11:37:22.042259+0000","last_became_active":"2023-10-06T11:37:22.031980+0000","last_became_peered":"2023-10-06T11:37:22.031980+0000","last_unstale":"2023-10-06T11:37:22.042259+0000","last_undegraded":"2023-10-06T11:37:22.042259+0000","last_fullsized":"2023-10-06T11:37:22.042259+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055521+0000","last_change":"2023-10-06T11:37:21.025853+0000","last_active":"2023-10-06T11:37:22.055521+0000","last_peered":"2023-10-06T11:37:22.055521+0000","last_clean":"2023-10-06T11:37:22.055521+0000","last_became_active":"2023-10-06T11:37:21.025492+0000","last_became_peered":"2023-10-06T11:37:21.025492+0000","last_unstale":"2023-10-06T11:37:22.055521+0000","last_undegraded":"2023-10-06T11:37:22.055521+0000","last_fullsized":"2023-10-06T11:37:22.055521+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:23:07.018154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.036884+0000","last_change":"2023-10-06T11:37:22.034707+0000","last_active":"2023-10-06T11:37:22.036884+0000","last_peered":"2023-10-06T11:37:22.036884+0000","last_clean":"2023-10-06T11:37:22.036884+0000","last_became_active":"2023-10-06T11:37:22.034341+0000","last_became_peered":"2023-10-06T11:37:22.034341+0000","last_unstale":"2023-10-06T11:37:22.036884+0000","last_undegraded":"2023-10-06T11:37:22.036884+0000","last_fullsized":"2023-10-06T11:37:22.036884+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:37.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055942+0000","last_change":"2023-10-06T11:37:21.033617+0000","last_active":"2023-10-06T11:37:22.055942+0000","last_peered":"2023-10-06T11:37:22.055942+0000","last_clean":"2023-10-06T11:37:22.055942+0000","last_became_active":"2023-10-06T11:37:21.033273+0000","last_became_peered":"2023-10-06T11:37:21.033273+0000","last_unstale":"2023-10-06T11:37:22.055942+0000","last_undegraded":"2023-10-06T11:37:22.055942+0000","last_fullsized":"2023-10-06T11:37:22.055942+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:19:52.667532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042164+0000","last_change":"2023-10-06T11:37:22.037095+0000","last_active":"2023-10-06T11:37:22.042164+0000","last_peered":"2023-10-06T11:37:22.042164+0000","last_clean":"2023-10-06T11:37:22.042164+0000","last_became_active":"2023-10-06T11:37:22.036799+0000","last_became_peered":"2023-10-06T11:37:22.036799+0000","last_unstale":"2023-10-06T11:37:22.042164+0000","last_undegraded":"2023-10-06T11:37:22.042164+0000","last_fullsized":"2023-10-06T11:37:22.042164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.053115+0000","last_change":"2023-10-06T11:37:21.032380+0000","last_active":"2023-10-06T11:37:22.053115+0000","last_peered":"2023-10-06T11:37:22.053115+0000","last_clean":"2023-10-06T11:37:22.053115+0000","last_became_active":"2023-10-06T11:37:21.032017+0000","last_became_peered":"2023-10-06T11:37:21.032017+0000","last_unstale":"2023-10-06T11:37:22.053115+0000","last_undegraded":"2023-10-06T11:37:22.053115+0000","last_fullsized":"2023-10-06T11:37:22.053115+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:47.385947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.054648+0000","last_change":"2023-10-06T11:37:22.049911+0000","last_active":"2023-10-06T11:37:22.054648+0000","last_peered":"2023-10-06T11:37:22.054648+0000","last_clean":"2023-10-06T11:37:22.054648+0000","last_became_active":"2023-10-06T11:37:22.049597+0000","last_became_peered":"2023-10-06T11:37:22.049597+0000","last_unstale":"2023-10-06T11:37:22.054648+0000","last_undegraded":"2023-10-06T11:37:22.054648+0000","last_fullsized":"2023-10-06T11:37:22.054648+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:12.939136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021032+0000","last_change":"2023-10-06T11:37:21.030689+0000","last_active":"2023-10-06T11:37:22.021032+0000","last_peered":"2023-10-06T11:37:22.021032+0000","last_clean":"2023-10-06T11:37:22.021032+0000","last_became_active":"2023-10-06T11:37:21.030401+0000","last_became_peered":"2023-10-06T11:37:21.030401+0000","last_unstale":"2023-10-06T11:37:22.021032+0000","last_undegraded":"2023-10-06T11:37:22.021032+0000","last_fullsized":"2023-10-06T11:37:22.021032+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:10.652875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.034880+0000","last_change":"2023-10-06T11:37:22.027444+0000","last_active":"2023-10-06T11:37:22.034880+0000","last_peered":"2023-10-06T11:37:22.034880+0000","last_clean":"2023-10-06T11:37:22.034880+0000","last_became_active":"2023-10-06T11:37:22.027110+0000","last_became_peered":"2023-10-06T11:37:22.027110+0000","last_unstale":"2023-10-06T11:37:22.034880+0000","last_undegraded":"2023-10-06T11:37:22.034880+0000","last_fullsized":"2023-10-06T11:37:22.034880+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:57.757746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024372+0000","last_change":"2023-10-06T11:37:21.030894+0000","last_active":"2023-10-06T11:37:22.024372+0000","last_peered":"2023-10-06T11:37:22.024372+0000","last_clean":"2023-10-06T11:37:22.024372+0000","last_became_active":"2023-10-06T11:37:21.027945+0000","last_became_peered":"2023-10-06T11:37:21.027945+0000","last_unstale":"2023-10-06T11:37:22.024372+0000","last_undegraded":"2023-10-06T11:37:22.024372+0000","last_fullsized":"2023-10-06T11:37:22.024372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:19.705850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045043+0000","last_change":"2023-10-06T11:37:22.042184+0000","last_active":"2023-10-06T11:37:22.045043+0000","last_peered":"2023-10-06T11:37:22.045043+0000","last_clean":"2023-10-06T11:37:22.045043+0000","last_became_active":"2023-10-06T11:37:22.041914+0000","last_became_peered":"2023-10-06T11:37:22.041914+0000","last_unstale":"2023-10-06T11:37:22.045043+0000","last_undegraded":"2023-10-06T11:37:22.045043+0000","last_fullsized":"2023-10-06T11:37:22.045043+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.048629+0000","last_change":"2023-10-06T11:37:21.024842+0000","last_active":"2023-10-06T11:37:22.048629+0000","last_peered":"2023-10-06T11:37:22.048629+0000","last_clean":"2023-10-06T11:37:22.048629+0000","last_became_active":"2023-10-06T11:37:21.024560+0000","last_became_peered":"2023-10-06T11:37:21.024560+0000","last_unstale":"2023-10-06T11:37:22.048629+0000","last_undegraded":"2023-10-06T11:37:22.048629+0000","last_fullsized":"2023-10-06T11:37:22.048629+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:57:33.876498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.039509+0000","last_change":"2023-10-06T11:37:22.032993+0000","last_active":"2023-10-06T11:37:22.039509+0000","last_peered":"2023-10-06T11:37:22.039509+0000","last_clean":"2023-10-06T11:37:22.039509+0000","last_became_active":"2023-10-06T11:37:22.032693+0000","last_became_peered":"2023-10-06T11:37:22.032693+0000","last_unstale":"2023-10-06T11:37:22.039509+0000","last_undegraded":"2023-10-06T11:37:22.039509+0000","last_fullsized":"2023-10-06T11:37:22.039509+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:54:43.285017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052494+0000","last_change":"2023-10-06T11:37:21.026396+0000","last_active":"2023-10-06T11:37:22.052494+0000","last_peered":"2023-10-06T11:37:22.052494+0000","last_clean":"2023-10-06T11:37:22.052494+0000","last_became_active":"2023-10-06T11:37:21.026068+0000","last_became_peered":"2023-10-06T11:37:21.026068+0000","last_unstale":"2023-10-06T11:37:22.052494+0000","last_undegraded":"2023-10-06T11:37:22.052494+0000","last_fullsized":"2023-10-06T11:37:22.052494+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:28.408340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043165+0000","last_change":"2023-10-06T11:37:22.038995+0000","last_active":"2023-10-06T11:37:22.043165+0000","last_peered":"2023-10-06T11:37:22.043165+0000","last_clean":"2023-10-06T11:37:22.043165+0000","last_became_active":"2023-10-06T11:37:22.038702+0000","last_became_peered":"2023-10-06T11:37:22.038702+0000","last_unstale":"2023-10-06T11:37:22.043165+0000","last_undegraded":"2023-10-06T11:37:22.043165+0000","last_fullsized":"2023-10-06T11:37:22.043165+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:50:04.471611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.057861+0000","last_change":"2023-10-06T11:37:21.034035+0000","last_active":"2023-10-06T11:37:22.057861+0000","last_peered":"2023-10-06T11:37:22.057861+0000","last_clean":"2023-10-06T11:37:22.057861+0000","last_became_active":"2023-10-06T11:37:21.033768+0000","last_became_peered":"2023-10-06T11:37:21.033768+0000","last_unstale":"2023-10-06T11:37:22.057861+0000","last_undegraded":"2023-10-06T11:37:22.057861+0000","last_fullsized":"2023-10-06T11:37:22.057861+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:24.526657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037749+0000","last_change":"2023-10-06T11:37:22.028561+0000","last_active":"2023-10-06T11:37:22.037749+0000","last_peered":"2023-10-06T11:37:22.037749+0000","last_clean":"2023-10-06T11:37:22.037749+0000","last_became_active":"2023-10-06T11:37:22.028203+0000","last_became_peered":"2023-10-06T11:37:22.028203+0000","last_unstale":"2023-10-06T11:37:22.037749+0000","last_undegraded":"2023-10-06T11:37:22.037749+0000","last_fullsized":"2023-10-06T11:37:22.037749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:48:51.889674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020754+0000","last_change":"2023-10-06T11:37:21.042516+0000","last_active":"2023-10-06T11:37:22.020754+0000","last_peered":"2023-10-06T11:37:22.020754+0000","last_clean":"2023-10-06T11:37:22.020754+0000","last_became_active":"2023-10-06T11:37:21.042180+0000","last_became_peered":"2023-10-06T11:37:21.042180+0000","last_unstale":"2023-10-06T11:37:22.020754+0000","last_undegraded":"2023-10-06T11:37:22.020754+0000","last_fullsized":"2023-10-06T11:37:22.020754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:27:50.461023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042707+0000","last_change":"2023-10-06T11:37:22.036063+0000","last_active":"2023-10-06T11:37:22.042707+0000","last_peered":"2023-10-06T11:37:22.042707+0000","last_clean":"2023-10-06T11:37:22.042707+0000","last_became_active":"2023-10-06T11:37:22.035693+0000","last_became_peered":"2023-10-06T11:37:22.035693+0000","last_unstale":"2023-10-06T11:37:22.042707+0000","last_undegraded":"2023-10-06T11:37:22.042707+0000","last_fullsized":"2023-10-06T11:37:22.042707+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:05.005101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.047158+0000","last_change":"2023-10-06T11:37:21.031123+0000","last_active":"2023-10-06T11:37:22.047158+0000","last_peered":"2023-10-06T11:37:22.047158+0000","last_clean":"2023-10-06T11:37:22.047158+0000","last_became_active":"2023-10-06T11:37:21.030739+0000","last_became_peered":"2023-10-06T11:37:21.030739+0000","last_unstale":"2023-10-06T11:37:22.047158+0000","last_undegraded":"2023-10-06T11:37:22.047158+0000","last_fullsized":"2023-10-06T11:37:22.047158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:55:11.817250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021626+0000","last_change":"2023-10-06T11:37:22.019124+0000","last_active":"2023-10-06T11:37:22.021626+0000","last_peered":"2023-10-06T11:37:22.021626+0000","last_clean":"2023-10-06T11:37:22.021626+0000","last_became_active":"2023-10-06T11:37:22.018796+0000","last_became_peered":"2023-10-06T11:37:22.018796+0000","last_unstale":"2023-10-06T11:37:22.021626+0000","last_undegraded":"2023-10-06T11:37:22.021626+0000","last_fullsized":"2023-10-06T11:37:22.021626+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:39.785634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019788+0000","last_change":"2023-10-06T11:37:21.035595+0000","last_active":"2023-10-06T11:37:22.019788+0000","last_peered":"2023-10-06T11:37:22.019788+0000","last_clean":"2023-10-06T11:37:22.019788+0000","last_became_active":"2023-10-06T11:37:21.035289+0000","last_became_peered":"2023-10-06T11:37:21.035289+0000","last_unstale":"2023-10-06T11:37:22.019788+0000","last_undegraded":"2023-10-06T11:37:22.019788+0000","last_fullsized":"2023-10-06T11:37:22.019788+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:32.475340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.052270+0000","last_change":"2023-10-06T11:37:22.048313+0000","last_active":"2023-10-06T11:37:22.052270+0000","last_peered":"2023-10-06T11:37:22.052270+0000","last_clean":"2023-10-06T11:37:22.052270+0000","last_became_active":"2023-10-06T11:37:22.047766+0000","last_became_peered":"2023-10-06T11:37:22.047766+0000","last_unstale":"2023-10-06T11:37:22.052270+0000","last_undegraded":"2023-10-06T11:37:22.052270+0000","last_fullsized":"2023-10-06T11:37:22.052270+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:20.522763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.013094+0000","last_change":"2023-10-06T11:37:21.024890+0000","last_active":"2023-10-06T11:37:22.013094+0000","last_peered":"2023-10-06T11:37:22.013094+0000","last_clean":"2023-10-06T11:37:22.013094+0000","last_became_active":"2023-10-06T11:37:21.024592+0000","last_became_peered":"2023-10-06T11:37:21.024592+0000","last_unstale":"2023-10-06T11:37:22.013094+0000","last_undegraded":"2023-10-06T11:37:22.013094+0000","last_fullsized":"2023-10-06T11:37:22.013094+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:05:56.753124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043489+0000","last_change":"2023-10-06T11:37:22.033090+0000","last_active":"2023-10-06T11:37:22.043489+0000","last_peered":"2023-10-06T11:37:22.043489+0000","last_clean":"2023-10-06T11:37:22.043489+0000","last_became_active":"2023-10-06T11:37:22.032689+0000","last_became_peered":"2023-10-06T11:37:22.032689+0000","last_unstale":"2023-10-06T11:37:22.043489+0000","last_undegraded":"2023-10-06T11:37:22.043489+0000","last_fullsized":"2023-10-06T11:37:22.043489+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:31:03.029324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019366+0000","last_change":"2023-10-06T11:37:21.031934+0000","last_active":"2023-10-06T11:37:22.019366+0000","last_peered":"2023-10-06T11:37:22.019366+0000","last_clean":"2023-10-06T11:37:22.019366+0000","last_became_active":"2023-10-06T11:37:21.031654+0000","last_became_peered":"2023-10-06T11:37:21.031654+0000","last_unstale":"2023-10-06T11:37:22.019366+0000","last_undegraded":"2023-10-06T11:37:22.019366+0000","last_fullsized":"2023-10-06T11:37:22.019366+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:11.934515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043162+0000","last_change":"2023-10-06T11:37:22.038403+0000","last_active":"2023-10-06T11:37:22.043162+0000","last_peered":"2023-10-06T11:37:22.043162+0000","last_clean":"2023-10-06T11:37:22.043162+0000","last_became_active":"2023-10-06T11:37:22.037643+0000","last_became_peered":"2023-10-06T11:37:22.037643+0000","last_unstale":"2023-10-06T11:37:22.043162+0000","last_undegraded":"2023-10-06T11:37:22.043162+0000","last_fullsized":"2023-10-06T11:37:22.043162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:17.680312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046499+0000","last_change":"2023-10-06T11:37:21.024723+0000","last_active":"2023-10-06T11:37:22.046499+0000","last_peered":"2023-10-06T11:37:22.046499+0000","last_clean":"2023-10-06T11:37:22.046499+0000","last_became_active":"2023-10-06T11:37:21.024451+0000","last_became_peered":"2023-10-06T11:37:21.024451+0000","last_unstale":"2023-10-06T11:37:22.046499+0000","last_undegraded":"2023-10-06T11:37:22.046499+0000","last_fullsized":"2023-10-06T11:37:22.046499+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:26.726631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.035922+0000","last_change":"2023-10-06T11:37:22.034046+0000","last_active":"2023-10-06T11:37:22.035922+0000","last_peered":"2023-10-06T11:37:22.035922+0000","last_clean":"2023-10-06T11:37:22.035922+0000","last_became_active":"2023-10-06T11:37:22.033768+0000","last_became_peered":"2023-10-06T11:37:22.033768+0000","last_unstale":"2023-10-06T11:37:22.035922+0000","last_undegraded":"2023-10-06T11:37:22.035922+0000","last_fullsized":"2023-10-06T11:37:22.035922+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:15.444744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038622+0000","last_change":"2023-10-06T11:37:21.042502+0000","last_active":"2023-10-06T11:37:22.038622+0000","last_peered":"2023-10-06T11:37:22.038622+0000","last_clean":"2023-10-06T11:37:22.038622+0000","last_became_active":"2023-10-06T11:37:21.042164+0000","last_became_peered":"2023-10-06T11:37:21.042164+0000","last_unstale":"2023-10-06T11:37:22.038622+0000","last_undegraded":"2023-10-06T11:37:22.038622+0000","last_fullsized":"2023-10-06T11:37:22.038622+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:10:14.440123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042323+0000","last_change":"2023-10-06T11:37:22.033816+0000","last_active":"2023-10-06T11:37:22.042323+0000","last_peered":"2023-10-06T11:37:22.042323+0000","last_clean":"2023-10-06T11:37:22.042323+0000","last_became_active":"2023-10-06T11:37:22.033436+0000","last_became_peered":"2023-10-06T11:37:22.033436+0000","last_unstale":"2023-10-06T11:37:22.042323+0000","last_undegraded":"2023-10-06T11:37:22.042323+0000","last_fullsized":"2023-10-06T11:37:22.042323+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:04.566708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038396+0000","last_change":"2023-10-06T11:37:21.042542+0000","last_active":"2023-10-06T11:37:22.038396+0000","last_peered":"2023-10-06T11:37:22.038396+0000","last_clean":"2023-10-06T11:37:22.038396+0000","last_became_active":"2023-10-06T11:37:21.042231+0000","last_became_peered":"2023-10-06T11:37:21.042231+0000","last_unstale":"2023-10-06T11:37:22.038396+0000","last_undegraded":"2023-10-06T11:37:22.038396+0000","last_fullsized":"2023-10-06T11:37:22.038396+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:37.459426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.025795+0000","last_change":"2023-10-06T11:37:22.021723+0000","last_active":"2023-10-06T11:37:22.025795+0000","last_peered":"2023-10-06T11:37:22.025795+0000","last_clean":"2023-10-06T11:37:22.025795+0000","last_became_active":"2023-10-06T11:37:22.021332+0000","last_became_peered":"2023-10-06T11:37:22.021332+0000","last_unstale":"2023-10-06T11:37:22.025795+0000","last_undegraded":"2023-10-06T11:37:22.025795+0000","last_fullsized":"2023-10-06T11:37:22.025795+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:17.910505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.028473+0000","last_change":"2023-10-06T11:37:21.027939+0000","last_active":"2023-10-06T11:37:22.028473+0000","last_peered":"2023-10-06T11:37:22.028473+0000","last_clean":"2023-10-06T11:37:22.028473+0000","last_became_active":"2023-10-06T11:37:21.027554+0000","last_became_peered":"2023-10-06T11:37:21.027554+0000","last_unstale":"2023-10-06T11:37:22.028473+0000","last_undegraded":"2023-10-06T11:37:22.028473+0000","last_fullsized":"2023-10-06T11:37:22.028473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:08.775982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037289+0000","last_change":"2023-10-06T11:37:22.031714+0000","last_active":"2023-10-06T11:37:22.037289+0000","last_peered":"2023-10-06T11:37:22.037289+0000","last_clean":"2023-10-06T11:37:22.037289+0000","last_became_active":"2023-10-06T11:37:22.031429+0000","last_became_peered":"2023-10-06T11:37:22.031429+0000","last_unstale":"2023-10-06T11:37:22.037289+0000","last_undegraded":"2023-10-06T11:37:22.037289+0000","last_fullsized":"2023-10-06T11:37:22.037289+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:23:10.721108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.025027+0000","last_change":"2023-10-06T11:37:21.034445+0000","last_active":"2023-10-06T11:37:22.025027+0000","last_peered":"2023-10-06T11:37:22.025027+0000","last_clean":"2023-10-06T11:37:22.025027+0000","last_became_active":"2023-10-06T11:37:21.034003+0000","last_became_peered":"2023-10-06T11:37:21.034003+0000","last_unstale":"2023-10-06T11:37:22.025027+0000","last_undegraded":"2023-10-06T11:37:22.025027+0000","last_fullsized":"2023-10-06T11:37:22.025027+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:20.903950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046801+0000","last_change":"2023-10-06T11:37:22.037038+0000","last_active":"2023-10-06T11:37:22.046801+0000","last_peered":"2023-10-06T11:37:22.046801+0000","last_clean":"2023-10-06T11:37:22.046801+0000","last_became_active":"2023-10-06T11:37:22.036750+0000","last_became_peered":"2023-10-06T11:37:22.036750+0000","last_unstale":"2023-10-06T11:37:22.046801+0000","last_undegraded":"2023-10-06T11:37:22.046801+0000","last_fullsized":"2023-10-06T11:37:22.046801+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:38.596362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.027015+0000","last_change":"2023-10-06T11:37:21.035362+0000","last_active":"2023-10-06T11:37:22.027015+0000","last_peered":"2023-10-06T11:37:22.027015+0000","last_clean":"2023-10-06T11:37:22.027015+0000","last_became_active":"2023-10-06T11:37:21.035042+0000","last_became_peered":"2023-10-06T11:37:21.035042+0000","last_unstale":"2023-10-06T11:37:22.027015+0000","last_undegraded":"2023-10-06T11:37:22.027015+0000","last_fullsized":"2023-10-06T11:37:22.027015+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:46.024750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.045491+0000","last_change":"2023-10-06T11:37:22.039285+0000","last_active":"2023-10-06T11:37:22.045491+0000","last_peered":"2023-10-06T11:37:22.045491+0000","last_clean":"2023-10-06T11:37:22.045491+0000","last_became_active":"2023-10-06T11:37:22.038993+0000","last_became_peered":"2023-10-06T11:37:22.038993+0000","last_unstale":"2023-10-06T11:37:22.045491+0000","last_undegraded":"2023-10-06T11:37:22.045491+0000","last_fullsized":"2023-10-06T11:37:22.045491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:39.785254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.024100+0000","last_change":"2023-10-06T11:37:21.027828+0000","last_active":"2023-10-06T11:37:22.024100+0000","last_peered":"2023-10-06T11:37:22.024100+0000","last_clean":"2023-10-06T11:37:22.024100+0000","last_became_active":"2023-10-06T11:37:21.027477+0000","last_became_peered":"2023-10-06T11:37:21.027477+0000","last_unstale":"2023-10-06T11:37:22.024100+0000","last_undegraded":"2023-10-06T11:37:22.024100+0000","last_fullsized":"2023-10-06T11:37:22.024100+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:41.517876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037036+0000","last_change":"2023-10-06T11:37:22.035002+0000","last_active":"2023-10-06T11:37:22.037036+0000","last_peered":"2023-10-06T11:37:22.037036+0000","last_clean":"2023-10-06T11:37:22.037036+0000","last_became_active":"2023-10-06T11:37:22.033041+0000","last_became_peered":"2023-10-06T11:37:22.033041+0000","last_unstale":"2023-10-06T11:37:22.037036+0000","last_undegraded":"2023-10-06T11:37:22.037036+0000","last_fullsized":"2023-10-06T11:37:22.037036+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:04:32.255913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.019633+0000","last_change":"2023-10-06T11:37:21.030419+0000","last_active":"2023-10-06T11:37:22.019633+0000","last_peered":"2023-10-06T11:37:22.019633+0000","last_clean":"2023-10-06T11:37:22.019633+0000","last_became_active":"2023-10-06T11:37:21.030062+0000","last_became_peered":"2023-10-06T11:37:21.030062+0000","last_unstale":"2023-10-06T11:37:22.019633+0000","last_undegraded":"2023-10-06T11:37:22.019633+0000","last_fullsized":"2023-10-06T11:37:22.019633+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:10.625491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.042330+0000","last_change":"2023-10-06T11:37:22.032780+0000","last_active":"2023-10-06T11:37:22.042330+0000","last_peered":"2023-10-06T11:37:22.042330+0000","last_clean":"2023-10-06T11:37:22.042330+0000","last_became_active":"2023-10-06T11:37:22.032358+0000","last_became_peered":"2023-10-06T11:37:22.032358+0000","last_unstale":"2023-10-06T11:37:22.042330+0000","last_undegraded":"2023-10-06T11:37:22.042330+0000","last_fullsized":"2023-10-06T11:37:22.042330+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:01:48.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"134'8","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051403+0000","last_change":"2023-10-06T11:37:21.025453+0000","last_active":"2023-10-06T11:37:22.051403+0000","last_peered":"2023-10-06T11:37:22.051403+0000","last_clean":"2023-10-06T11:37:22.051403+0000","last_became_active":"2023-10-06T11:37:21.025125+0000","last_became_peered":"2023-10-06T11:37:21.025125+0000","last_unstale":"2023-10-06T11:37:22.051403+0000","last_undegraded":"2023-10-06T11:37:22.051403+0000","last_fullsized":"2023-10-06T11:37:22.051403+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:26:15.652129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":729,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021990+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T11:37:22.021990+0000","last_peered":"2023-10-06T11:37:22.021990+0000","last_clean":"2023-10-06T11:37:22.021990+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T11:37:22.021990+0000","last_undegraded":"2023-10-06T11:37:22.021990+0000","last_fullsized":"2023-10-06T11:37:22.021990+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":120,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.022541+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T11:37:22.022541+0000","last_peered":"2023-10-06T11:37:22.022541+0000","last_clean":"2023-10-06T11:37:22.022541+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T11:37:22.022541+0000","last_undegraded":"2023-10-06T11:37:22.022541+0000","last_fullsized":"2023-10-06T11:37:22.022541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.062238+0000","last_change":"2023-10-06T11:37:22.060573+0000","last_active":"2023-10-06T11:37:22.062238+0000","last_peered":"2023-10-06T11:37:22.062238+0000","last_clean":"2023-10-06T11:37:22.062238+0000","last_became_active":"2023-10-06T11:37:22.059899+0000","last_became_peered":"2023-10-06T11:37:22.059899+0000","last_unstale":"2023-10-06T11:37:22.062238+0000","last_undegraded":"2023-10-06T11:37:22.062238+0000","last_fullsized":"2023-10-06T11:37:22.062238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:25.531278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.050904+0000","last_change":"2023-10-06T11:37:21.024571+0000","last_active":"2023-10-06T11:37:22.050904+0000","last_peered":"2023-10-06T11:37:22.050904+0000","last_clean":"2023-10-06T11:37:22.050904+0000","last_became_active":"2023-10-06T11:37:21.024254+0000","last_became_peered":"2023-10-06T11:37:21.024254+0000","last_unstale":"2023-10-06T11:37:22.050904+0000","last_undegraded":"2023-10-06T11:37:22.050904+0000","last_fullsized":"2023-10-06T11:37:22.050904+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:52.937702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.021867+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T11:37:22.021867+0000","last_peered":"2023-10-06T11:37:22.021867+0000","last_clean":"2023-10-06T11:37:22.021867+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T11:37:22.021867+0000","last_undegraded":"2023-10-06T11:37:22.021867+0000","last_fullsized":"2023-10-06T11:37:22.021867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.036142+0000","last_change":"2023-10-06T11:37:22.033626+0000","last_active":"2023-10-06T11:37:22.036142+0000","last_peered":"2023-10-06T11:37:22.036142+0000","last_clean":"2023-10-06T11:37:22.036142+0000","last_became_active":"2023-10-06T11:37:22.033315+0000","last_became_peered":"2023-10-06T11:37:22.033315+0000","last_unstale":"2023-10-06T11:37:22.036142+0000","last_undegraded":"2023-10-06T11:37:22.036142+0000","last_fullsized":"2023-10-06T11:37:22.036142+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:44.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.012975+0000","last_change":"2023-10-06T11:37:21.025989+0000","last_active":"2023-10-06T11:37:22.012975+0000","last_peered":"2023-10-06T11:37:22.012975+0000","last_clean":"2023-10-06T11:37:22.012975+0000","last_became_active":"2023-10-06T11:37:21.025691+0000","last_became_peered":"2023-10-06T11:37:21.025691+0000","last_unstale":"2023-10-06T11:37:22.012975+0000","last_undegraded":"2023-10-06T11:37:22.012975+0000","last_fullsized":"2023-10-06T11:37:22.012975+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:38:13.178643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.009254+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T11:37:22.009254+0000","last_peered":"2023-10-06T11:37:22.009254+0000","last_clean":"2023-10-06T11:37:22.009254+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T11:37:22.009254+0000","last_undegraded":"2023-10-06T11:37:22.009254+0000","last_fullsized":"2023-10-06T11:37:22.009254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046343+0000","last_change":"2023-10-06T11:37:22.026243+0000","last_active":"2023-10-06T11:37:22.046343+0000","last_peered":"2023-10-06T11:37:22.046343+0000","last_clean":"2023-10-06T11:37:22.046343+0000","last_became_active":"2023-10-06T11:37:22.025927+0000","last_became_peered":"2023-10-06T11:37:22.025927+0000","last_unstale":"2023-10-06T11:37:22.046343+0000","last_undegraded":"2023-10-06T11:37:22.046343+0000","last_fullsized":"2023-10-06T11:37:22.046343+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:37.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.012900+0000","last_change":"2023-10-06T11:37:21.027257+0000","last_active":"2023-10-06T11:37:22.012900+0000","last_peered":"2023-10-06T11:37:22.012900+0000","last_clean":"2023-10-06T11:37:22.012900+0000","last_became_active":"2023-10-06T11:37:21.026962+0000","last_became_peered":"2023-10-06T11:37:21.026962+0000","last_unstale":"2023-10-06T11:37:22.012900+0000","last_undegraded":"2023-10-06T11:37:22.012900+0000","last_fullsized":"2023-10-06T11:37:22.012900+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:22:43.541381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":120,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020016+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T11:37:22.020016+0000","last_peered":"2023-10-06T11:37:22.020016+0000","last_clean":"2023-10-06T11:37:22.020016+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T11:37:22.020016+0000","last_undegraded":"2023-10-06T11:37:22.020016+0000","last_fullsized":"2023-10-06T11:37:22.020016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.051776+0000","last_change":"2023-10-06T11:37:22.046575+0000","last_active":"2023-10-06T11:37:22.051776+0000","last_peered":"2023-10-06T11:37:22.051776+0000","last_clean":"2023-10-06T11:37:22.051776+0000","last_became_active":"2023-10-06T11:37:22.046280+0000","last_became_peered":"2023-10-06T11:37:22.046280+0000","last_unstale":"2023-10-06T11:37:22.051776+0000","last_undegraded":"2023-10-06T11:37:22.051776+0000","last_fullsized":"2023-10-06T11:37:22.051776+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:11.630112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038396+0000","last_change":"2023-10-06T11:37:21.044314+0000","last_active":"2023-10-06T11:37:22.038396+0000","last_peered":"2023-10-06T11:37:22.038396+0000","last_clean":"2023-10-06T11:37:22.038396+0000","last_became_active":"2023-10-06T11:37:21.044002+0000","last_became_peered":"2023-10-06T11:37:21.044002+0000","last_unstale":"2023-10-06T11:37:22.038396+0000","last_undegraded":"2023-10-06T11:37:22.038396+0000","last_fullsized":"2023-10-06T11:37:22.038396+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:45.596041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.018017+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T11:37:22.018017+0000","last_peered":"2023-10-06T11:37:22.018017+0000","last_clean":"2023-10-06T11:37:22.018017+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T11:37:22.018017+0000","last_undegraded":"2023-10-06T11:37:22.018017+0000","last_fullsized":"2023-10-06T11:37:22.018017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.037670+0000","last_change":"2023-10-06T11:37:22.028981+0000","last_active":"2023-10-06T11:37:22.037670+0000","last_peered":"2023-10-06T11:37:22.037670+0000","last_clean":"2023-10-06T11:37:22.037670+0000","last_became_active":"2023-10-06T11:37:22.028619+0000","last_became_peered":"2023-10-06T11:37:22.028619+0000","last_unstale":"2023-10-06T11:37:22.037670+0000","last_undegraded":"2023-10-06T11:37:22.037670+0000","last_fullsized":"2023-10-06T11:37:22.037670+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:48.820206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020279+0000","last_change":"2023-10-06T11:37:21.032431+0000","last_active":"2023-10-06T11:37:22.020279+0000","last_peered":"2023-10-06T11:37:22.020279+0000","last_clean":"2023-10-06T11:37:22.020279+0000","last_became_active":"2023-10-06T11:37:21.031119+0000","last_became_peered":"2023-10-06T11:37:21.031119+0000","last_unstale":"2023-10-06T11:37:22.020279+0000","last_undegraded":"2023-10-06T11:37:22.020279+0000","last_fullsized":"2023-10-06T11:37:22.020279+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:10.625491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020701+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T11:37:22.020701+0000","last_peered":"2023-10-06T11:37:22.020701+0000","last_clean":"2023-10-06T11:37:22.020701+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T11:37:22.020701+0000","last_undegraded":"2023-10-06T11:37:22.020701+0000","last_fullsized":"2023-10-06T11:37:22.020701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.046951+0000","last_change":"2023-10-06T11:37:22.038617+0000","last_active":"2023-10-06T11:37:22.046951+0000","last_peered":"2023-10-06T11:37:22.046951+0000","last_clean":"2023-10-06T11:37:22.046951+0000","last_became_active":"2023-10-06T11:37:22.038323+0000","last_became_peered":"2023-10-06T11:37:22.038323+0000","last_unstale":"2023-10-06T11:37:22.046951+0000","last_undegraded":"2023-10-06T11:37:22.046951+0000","last_fullsized":"2023-10-06T11:37:22.046951+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:21.908571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"134'3","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.056686+0000","last_change":"2023-10-06T11:37:21.030077+0000","last_active":"2023-10-06T11:37:22.056686+0000","last_peered":"2023-10-06T11:37:22.056686+0000","last_clean":"2023-10-06T11:37:22.056686+0000","last_became_active":"2023-10-06T11:37:21.029785+0000","last_became_peered":"2023-10-06T11:37:21.029785+0000","last_unstale":"2023-10-06T11:37:22.056686+0000","last_undegraded":"2023-10-06T11:37:22.056686+0000","last_fullsized":"2023-10-06T11:37:22.056686+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:25:50.993145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.020235+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T11:37:22.020235+0000","last_peered":"2023-10-06T11:37:22.020235+0000","last_clean":"2023-10-06T11:37:22.020235+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T11:37:22.020235+0000","last_undegraded":"2023-10-06T11:37:22.020235+0000","last_fullsized":"2023-10-06T11:37:22.020235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.043889+0000","last_change":"2023-10-06T11:37:22.033184+0000","last_active":"2023-10-06T11:37:22.043889+0000","last_peered":"2023-10-06T11:37:22.043889+0000","last_clean":"2023-10-06T11:37:22.043889+0000","last_became_active":"2023-10-06T11:37:22.032909+0000","last_became_peered":"2023-10-06T11:37:22.032909+0000","last_unstale":"2023-10-06T11:37:22.043889+0000","last_undegraded":"2023-10-06T11:37:22.043889+0000","last_fullsized":"2023-10-06T11:37:22.043889+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:20.979953+0000","last_clean_scrub_stamp":"2023-10-06T11:37:20.979953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:24.545371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"134'6","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.055863+0000","last_change":"2023-10-06T11:37:21.029833+0000","last_active":"2023-10-06T11:37:22.055863+0000","last_peered":"2023-10-06T11:37:22.055863+0000","last_clean":"2023-10-06T11:37:22.055863+0000","last_became_active":"2023-10-06T11:37:21.027009+0000","last_became_peered":"2023-10-06T11:37:21.027009+0000","last_unstale":"2023-10-06T11:37:22.055863+0000","last_undegraded":"2023-10-06T11:37:22.055863+0000","last_fullsized":"2023-10-06T11:37:22.055863+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:37:19.975332+0000","last_clean_scrub_stamp":"2023-10-06T11:37:19.975332+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:16.905884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":108,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-06T11:37:22.038486+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T11:37:22.038486+0000","last_peered":"2023-10-06T11:37:22.038486+0000","last_clean":"2023-10-06T11:37:22.038486+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T11:37:22.038486+0000","last_undegraded":"2023-10-06T11:37:22.038486+0000","last_fullsized":"2023-10-06T11:37:22.038486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311808,"kb_used_data":2232,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060032,"statfs":{"total":96636764160,"available":96317472768,"internally_reserved":0,"allocated":2285568,"data_stored":1463130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1120000000000001,"15min":1.135},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.724},"last":1.466},{"interface":"front","average":{"1min":1.101,"5min":1.1379999999999999,"15min":1.1359999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.56399999999999995},"max":{"1min":1.7090000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":1.877}]},{"osd":1,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3340000000000001,"15min":1.327},"min":{"1min":0.89200000000000002,"5min":0.67100000000000004,"15min":0.65900000000000003},"max":{"1min":1.653,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.786},{"interface":"front","average":{"1min":1.224,"5min":1.228,"15min":1.236},"min":{"1min":0.57099999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5069999999999999,"5min":1.927,"15min":1.927},"last":1.1140000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.94399999999999995,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.554,"5min":1.6040000000000001,"15min":1.827},"last":1.173},{"interface":"front","average":{"1min":1.21,"5min":1.2509999999999999,"15min":1.2609999999999999},"min":{"1min":0.86299999999999999,"5min":0.82099999999999995,"15min":0.65000000000000002},"max":{"1min":1.5649999999999999,"5min":1.7210000000000001,"15min":1.9830000000000001},"last":1.423}]},{"osd":3,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2350000000000001,"15min":1.268},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.68500000000000005},"max":{"1min":1.665,"5min":1.8,"15min":1.9450000000000001},"last":1.325},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3240000000000001,"15min":1.3200000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8580000000000001,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.6910000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1080000000000001,"15min":1.1140000000000001},"min":{"1min":0.66500000000000004,"5min":0.624,"15min":0.44600000000000001},"max":{"1min":1.893,"5min":1.893,"15min":2.149},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.41699999999999998},"max":{"1min":1.73,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5509999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.329,"15min":1.3069999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.58199999999999996},"max":{"1min":1.9330000000000001,"5min":2.488,"15min":2.488},"last":1.26},{"interface":"front","average":{"1min":1.272,"5min":1.268,"15min":1.268},"min":{"1min":0.94899999999999995,"5min":0.59599999999999997,"15min":0.56699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":2.2480000000000002},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:36:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.331,"15min":1.3480000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.8500000000000001,"5min":2.0190000000000001,"15min":2.4289999999999998},"last":1.373},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3540000000000001,"15min":1.387},"min":{"1min":0.90600000000000003,"5min":0.76000000000000001,"15min":0.54200000000000004},"max":{"1min":1.819,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.827}]}]},{"osd":6,"up_from":13,"seq":55834575074,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":872568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.286},"min":{"1min":0.74099999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.774,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3380000000000001,"15min":1.3169999999999999},"min":{"1min":0.77200000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9670000000000001,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.226}]},{"osd":1,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1359999999999999,"15min":1.1539999999999999},"min":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.64000000000000001},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8169999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1899999999999999,"15min":1.208},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.5940000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.4099999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.226,"15min":1.2070000000000001},"min":{"1min":0.92300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.1280000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.69099999999999995},"max":{"1min":1.675,"5min":1.736,"15min":1.736},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.262,"15min":1.3029999999999999},"min":{"1min":0.80600000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.671,"5min":2.121,"15min":2.121},"last":1.631},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.339,"15min":1.304},"min":{"1min":1.0549999999999999,"5min":0.91300000000000003,"15min":0.85899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.143}]},{"osd":4,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2390000000000001,"15min":1.222},"min":{"1min":0.81499999999999995,"5min":0.75900000000000001,"15min":0.61399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.296},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2989999999999999,"15min":1.258},"min":{"1min":0.81499999999999995,"5min":0.74299999999999999,"15min":0.73899999999999999},"max":{"1min":2.0659999999999998,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.546}]},{"osd":5,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.216,"15min":1.2290000000000001},"min":{"1min":0.84899999999999998,"5min":0.79800000000000004,"15min":0.53700000000000003},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.377},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3799999999999999,"15min":1.379},"min":{"1min":0.89600000000000002,"5min":0.86499999999999999,"15min":0.71799999999999997},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.782}]},{"osd":7,"last update":"Fri Oct 6 11:36:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":0.877,"5min":0.84199999999999997,"15min":0.73899999999999999},"max":{"1min":2.206,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4099999999999999,"15min":1.4179999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.68899999999999995},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.601}]}]},{"osd":5,"up_from":13,"seq":55834575074,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":872174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.232,"15min":1.28},"min":{"1min":0.84399999999999997,"5min":0.67400000000000004,"15min":0.5},"max":{"1min":1.5649999999999999,"5min":1.772,"15min":17.626000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.302,"15min":1.304},"min":{"1min":0.95699999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6850000000000001,"5min":1.966,"15min":2.3959999999999999},"last":1.413}]},{"osd":1,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.155,"15min":1.161},"min":{"1min":0.73799999999999999,"5min":0.65800000000000003,"15min":0.53200000000000003},"max":{"1min":1.4490000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1859999999999999,"15min":1.2090000000000001},"min":{"1min":0.86099999999999999,"5min":0.73799999999999999,"15min":0.58699999999999997},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":18.033000000000001},"last":1.1339999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1559999999999999,"15min":1.1659999999999999},"min":{"1min":0.89600000000000002,"5min":0.77100000000000002,"15min":0.46999999999999997},"max":{"1min":1.554,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.042},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2,"15min":1.2110000000000001},"min":{"1min":0.88900000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4730000000000001,"5min":1.6990000000000001,"15min":1.738},"last":1.194}]},{"osd":3,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2350000000000001,"15min":1.28},"min":{"1min":0.88500000000000001,"5min":0.84699999999999998,"15min":0.80400000000000005},"max":{"1min":1.5309999999999999,"5min":1.6599999999999999,"15min":18.074999999999999},"last":1.331},{"interface":"front","average":{"1min":1.3,"5min":1.266,"15min":1.2969999999999999},"min":{"1min":0.96899999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5509999999999999,"5min":1.8400000000000001,"15min":1.871},"last":1.2689999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2470000000000001,"15min":1.2629999999999999},"min":{"1min":0.80400000000000005,"5min":0.70699999999999996,"15min":0.40699999999999997},"max":{"1min":1.6080000000000001,"5min":1.7390000000000001,"15min":8.5649999999999995},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.347,"15min":1.377},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.63500000000000001},"max":{"1min":1.5960000000000001,"5min":1.887,"15min":18.48},"last":1.5529999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.2669999999999999},"min":{"1min":0.78400000000000003,"5min":0.77700000000000002,"15min":0.622},"max":{"1min":1.7450000000000001,"5min":1.8069999999999999,"15min":8.5530000000000008},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2869999999999999,"15min":1.3180000000000001},"min":{"1min":0.72299999999999998,"5min":0.65900000000000003,"15min":0.46300000000000002},"max":{"1min":1.6759999999999999,"5min":1.885,"15min":9.8559999999999999},"last":1.375}]},{"osd":7,"last update":"Fri Oct 6 11:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3009999999999999,"15min":1.3620000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.755},"max":{"1min":1.647,"5min":1.8280000000000001,"15min":18.379999999999999},"last":1.47},{"interface":"front","average":{"1min":1.381,"5min":1.365,"15min":1.3939999999999999},"min":{"1min":0.89600000000000002,"5min":0.80000000000000004,"15min":0.65400000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":8.4830000000000005},"last":1.609}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307128,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064712,"statfs":{"total":96636764160,"available":96322265088,"internally_reserved":0,"allocated":1687552,"data_stored":873550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1719999999999999,"15min":1.151},"min":{"1min":0.77800000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5069999999999999,"5min":1.72,"15min":1.72},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.123,"15min":1.1279999999999999},"min":{"1min":0.66100000000000003,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.6299999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.294,"15min":1.3109999999999999},"min":{"1min":1.1060000000000001,"5min":0.85499999999999998,"15min":0.60199999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.034},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.996,"5min":0.75700000000000001,"15min":0.68300000000000005},"max":{"1min":1.5800000000000001,"5min":1.78,"15min":1.78},"last":1.254}]},{"osd":2,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1859999999999999,"15min":1.2010000000000001},"min":{"1min":0.71099999999999997,"5min":0.67000000000000004,"15min":0.628},"max":{"1min":1.456,"5min":1.736,"15min":1.736},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.246,"15min":1.2609999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.6220000000000001,"5min":1.6619999999999999,"15min":1.6779999999999999},"last":1.3819999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1899999999999999,"15min":1.2050000000000001},"min":{"1min":0.78800000000000003,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.508,"5min":1.617,"15min":1.6839999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":1.018,"5min":0.82399999999999995,"15min":0.77400000000000002},"max":{"1min":1.607,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2010000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.1899999999999999},"min":{"1min":0.752,"5min":0.621,"15min":0.621},"max":{"1min":1.6379999999999999,"5min":1.706,"15min":1.7390000000000001},"last":1.143},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.204,"15min":1.1930000000000001},"min":{"1min":0.74099999999999999,"5min":0.67100000000000004,"15min":0.40999999999999998},"max":{"1min":1.635,"5min":1.6539999999999999,"15min":1.7549999999999999},"last":1.4350000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.70399999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6619999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.528},{"interface":"front","average":{"1min":1.429,"5min":1.361,"15min":1.353},"min":{"1min":1.1279999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7669999999999999,"5min":1.845,"15min":1.845},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:37:00 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.351,"15min":1.341},"min":{"1min":0.97999999999999998,"5min":0.80600000000000005,"15min":0.69399999999999995},"max":{"1min":1.649,"5min":1.897,"15min":1.897},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4259999999999999,"15min":1.3979999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.73799999999999999},"max":{"1min":1.8300000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.617}]}]},{"osd":3,"up_from":13,"seq":55834575076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311216,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060624,"statfs":{"total":96636764160,"available":96318078976,"internally_reserved":0,"allocated":1679360,"data_stored":866471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0660000000000001,"15min":0.93500000000000005},"min":{"1min":0.50800000000000001,"5min":0.44700000000000001,"15min":0.36599999999999999},"max":{"1min":1.3500000000000001,"5min":14.49,"15min":14.49},"last":0.997},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.2,"15min":1.056},"min":{"1min":0.78100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.331,"5min":19.800999999999998,"15min":19.800999999999998},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.96399999999999997},"min":{"1min":0.69499999999999995,"5min":0.56499999999999995,"15min":0.55300000000000005},"max":{"1min":1.5389999999999999,"5min":6.6790000000000003,"15min":6.6790000000000003},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.1679999999999999,"15min":1.115},"min":{"1min":0.69999999999999996,"5min":0.501,"15min":0.48199999999999998},"max":{"1min":1.552,"5min":11.513999999999999,"15min":11.513999999999999},"last":0.89400000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.2509999999999999,"15min":1.1599999999999999},"min":{"1min":0.85599999999999998,"5min":0.65200000000000002,"15min":0.57399999999999995},"max":{"1min":1.52,"5min":12.446,"15min":12.446},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.2669999999999999,"15min":1.2230000000000001},"min":{"1min":0.77400000000000002,"5min":0.626,"15min":0.44},"max":{"1min":1.6699999999999999,"5min":7.4550000000000001,"15min":7.4550000000000001},"last":1.3400000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.341,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.84199999999999997,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":10.460000000000001,"15min":10.460000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.3560000000000001},"min":{"1min":1.071,"5min":0.92700000000000005,"15min":0.78400000000000003},"max":{"1min":1.627,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":1.0820000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4259999999999999,"15min":1.405},"min":{"1min":1.111,"5min":0.92100000000000004,"15min":0.69999999999999996},"max":{"1min":1.99,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.448,"15min":1.403},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.69499999999999995},"max":{"1min":2.2490000000000001,"5min":7.1180000000000003,"15min":7.1180000000000003},"last":1.1040000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.556,"15min":1.4810000000000001},"min":{"1min":0.90500000000000003,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3839999999999999,"5min":11.362,"15min":11.362},"last":1.22},{"interface":"front","average":{"1min":1.405,"5min":1.466,"15min":1.399},"min":{"1min":0.82399999999999995,"5min":0.80400000000000005,"15min":0.80200000000000005},"max":{"1min":2.0259999999999998,"5min":10.515000000000001,"15min":10.515000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.5449999999999999,"15min":1.4610000000000001},"min":{"1min":1.018,"5min":1.012,"15min":0.84999999999999998},"max":{"1min":1.9490000000000001,"5min":10.561,"15min":10.561},"last":1.157},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.51,"15min":1.486},"min":{"1min":1.177,"5min":0.85899999999999999,"15min":0.83999999999999997},"max":{"1min":1.9359999999999999,"5min":6.7629999999999999,"15min":6.7629999999999999},"last":1.2}]}]},{"osd":2,"up_from":13,"seq":55834575074,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311200,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060640,"statfs":{"total":96636764160,"available":96318095360,"internally_reserved":0,"allocated":1662976,"data_stored":864901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.96899999999999997,"15min":0.96599999999999997},"min":{"1min":0.65100000000000002,"5min":0.56999999999999995,"15min":0.36299999999999999},"max":{"1min":1.4299999999999999,"5min":1.524,"15min":1.8380000000000001},"last":1.454},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.052,"15min":1.0680000000000001},"min":{"1min":0.74099999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.4299999999999999,"5min":1.5409999999999999,"15min":1.5629999999999999},"last":0.95599999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.92700000000000005,"15min":0.92400000000000004},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.161,"5min":1.4690000000000001,"15min":2.3940000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0349999999999999,"15min":1.02},"min":{"1min":0.75800000000000001,"5min":0.59399999999999997,"15min":0.439},"max":{"1min":1.855,"5min":1.855,"15min":2.5390000000000001},"last":1.708}]},{"osd":3,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.1599999999999999,"15min":1.1850000000000001},"min":{"1min":0.79000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.294,"15min":1.3069999999999999},"min":{"1min":0.89700000000000002,"5min":0.745,"15min":0.60699999999999998},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.4550000000000001},"last":1.9670000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.226,"15min":1.2210000000000001},"min":{"1min":0.95999999999999996,"5min":0.86099999999999999,"15min":0.68300000000000005},"max":{"1min":1.647,"5min":1.647,"15min":1.754},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.2869999999999999},"min":{"1min":0.97899999999999998,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6279999999999999,"5min":1.9199999999999999,"15min":2.024},"last":1.7849999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3779999999999999,"15min":1.3640000000000001},"min":{"1min":1.1060000000000001,"5min":0.91800000000000004,"15min":0.85199999999999998},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.8440000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.302,"15min":1.327},"min":{"1min":0.871,"5min":0.751,"15min":0.751},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.0630000000000002},"last":1.196}]},{"osd":6,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.423,"15min":1.3999999999999999},"min":{"1min":1.0980000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7769999999999999,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.4370000000000001,"15min":1.4419999999999999},"min":{"1min":1.046,"5min":1,"15min":0.76900000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.617},"last":1.915}]},{"osd":7,"last update":"Fri Oct 6 11:36:40 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3939999999999999,"15min":1.3859999999999999},"min":{"1min":1.05,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7809999999999999,"5min":1.8240000000000001,"15min":2.1659999999999999},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.4910000000000001,"15min":1.472},"min":{"1min":1.177,"5min":0.753,"15min":0.753},"max":{"1min":2.024,"5min":2.1419999999999999,"15min":2.2829999999999999},"last":1.5289999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307120,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064720,"statfs":{"total":96636764160,"available":96322273280,"internally_reserved":0,"allocated":1679360,"data_stored":872036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86699999999999999,"15min":0.91400000000000003},"min":{"1min":0.66100000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.724},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.016,"5min":0.99199999999999999,"15min":0.96999999999999997},"min":{"1min":0.66600000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.595,"5min":1.827,"15min":1.827},"last":1.4410000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.93600000000000005,"15min":0.96799999999999997},"min":{"1min":0.60999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.429,"5min":1.5529999999999999,"15min":1.8540000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.075,"5min":1.056,"15min":1.0980000000000001},"min":{"1min":0.67900000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.569,"5min":1.6859999999999999,"15min":2.137},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1639999999999999,"15min":1.2070000000000001},"min":{"1min":0.81899999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6879999999999999,"5min":1.8939999999999999,"15min":4.2530000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0800000000000001,"15min":1.0940000000000001},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.544,"5min":1.5549999999999999,"15min":1.756},"last":1.133}]},{"osd":4,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.226,"15min":1.2410000000000001},"min":{"1min":0.69199999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.5149999999999999,"5min":1.8580000000000001,"15min":2.2290000000000001},"last":1.349},{"interface":"front","average":{"1min":1.337,"5min":1.353,"15min":1.3740000000000001},"min":{"1min":0.997,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.748,"5min":1.8,"15min":1.944},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.321,"15min":1.3200000000000001},"min":{"1min":0.92800000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.599,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.3440000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":1.923,"15min":2.1240000000000001},"last":1.544}]},{"osd":6,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4119999999999999,"15min":1.456},"min":{"1min":1.0660000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.0310000000000001},"last":1.494},{"interface":"front","average":{"1min":1.357,"5min":1.3819999999999999,"15min":1.3879999999999999},"min":{"1min":1.018,"5min":0.92200000000000004,"15min":0.72299999999999998},"max":{"1min":1.7230000000000001,"5min":2.0600000000000001,"15min":2.8719999999999999},"last":1.845}]},{"osd":7,"last update":"Fri Oct 6 11:36:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.462,"15min":1.4910000000000001},"min":{"1min":1.0349999999999999,"5min":0.88,"15min":0.83599999999999997},"max":{"1min":1.819,"5min":2.0209999999999999,"15min":2.0390000000000001},"last":1.944},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.478,"15min":1.4570000000000001},"min":{"1min":0.94899999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.875,"5min":2.085,"15min":2.085},"last":1.619}]}]},{"osd":0,"up_from":13,"seq":55834575075,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311796,"kb_used_data":2220,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060044,"statfs":{"total":96636764160,"available":96317485056,"internally_reserved":0,"allocated":2273280,"data_stored":1462404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.874,"15min":0.88400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96999999999999997,"15min":0.96199999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.3979999999999999}]},{"osd":2,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.628,"5min":0.498,"15min":0.498},"max":{"1min":1.3180000000000001,"5min":1.6180000000000001,"15min":1.6950000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.0980000000000001,"15min":1.0840000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.7169999999999996,"5min":4.7169999999999996,"15min":4.7169999999999996},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0429999999999999,"15min":1.0760000000000001},"min":{"1min":0.54600000000000004,"5min":0.433,"15min":0.433},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.919},"last":0.997},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.274,"15min":1.3129999999999999},"min":{"1min":0.73099999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.841,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.97499999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2829999999999999,"15min":1.3320000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.792,"5min":1.8440000000000001,"15min":2.294},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.978,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7909999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3979999999999999,"15min":1.3819999999999999},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.65900000000000003},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.25},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.786,"5min":1.893,"15min":2.5510000000000002},"last":1.716}]},{"osd":6,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.3540000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.458,"15min":1.458},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4750000000000001},"last":1.349}]},{"osd":7,"last update":"Fri Oct 6 11:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.415,"15min":1.4530000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6799999999999999,"5min":2.012,"15min":2.4260000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.425,"15min":1.4239999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.3809999999999998},"last":1.897}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:37:26.774 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:37:26.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:37:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.967+0000 7f1564c8e700 1 Processor -- start 2023-10-06T11:37:26.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.971+0000 7f1564c8e700 1 -- start start 2023-10-06T11:37:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.971+0000 7f1564c8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f15600cb020 0x7f15600cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.971+0000 7f1564c8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15600cf050 con 0x7f15600d0750 2023-10-06T11:37:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.971+0000 7f1564c8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15600cf8f0 con 0x7f15600cf4e0 2023-10-06T11:37:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.971+0000 7f1564c8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15600ce440 con 0x7f15600cb020 2023-10-06T11:37:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155effd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15600d0750 0x7f15600cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51012/0 (socket says 172.21.15.46:51012) 2023-10-06T11:37:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155effd700 1 -- 172.21.15.46:0/2156253557 learned_addr learned my addr 172.21.15.46:0/2156253557 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155f7fe700 1 --2- 172.21.15.46:0/2156253557 >> v2:172.21.15.132:3300/0 conn(0x7f15600cb020 0x7f15600cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668169037 0 0) 0x7f15600cf050 con 0x7f15600d0750 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1548003570 con 0x7f15600d0750 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713814449 0 0) 0x7f15600cf8f0 con 0x7f15600cf4e0 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15600cf050 con 0x7f15600cf4e0 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155f7fe700 1 -- 172.21.15.46:0/2156253557 >> v1:172.21.15.132:6789/0 conn(0x7f15600cf4e0 legacy=0x7f1560158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155f7fe700 1 -- 172.21.15.46:0/2156253557 >> v1:172.21.15.46:6789/0 conn(0x7f15600d0750 legacy=0x7f15600cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442931442 0 0) 0x7f1548003570 con 0x7f15600d0750 2023-10-06T11:37:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155f7fe700 1 -- 172.21.15.46:0/2156253557 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600ce720 con 0x7f15600cb020 2023-10-06T11:37:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155f7fe700 1 --2- 172.21.15.46:0/2156253557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cb020 0x7f15600cb3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f15500098e0 tx=0x7f15500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229612e24b0bc21a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554001470 con 0x7f15600cb020 2023-10-06T11:37:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550011420 con 0x7f15600cb020 2023-10-06T11:37:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.975+0000 7f155e7fc700 1 -- 172.21.15.46:0/2156253557 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15500117c0 con 0x7f15600cb020 2023-10-06T11:37:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- 172.21.15.46:0/2156253557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cb020 msgr2=0x7f15600cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2156253557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cb020 0x7f15600cb3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f15500098e0 tx=0x7f15500091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- 172.21.15.46:0/2156253557 shutdown_connections 2023-10-06T11:37:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2156253557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cb020 0x7f15600cb3f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- 172.21.15.46:0/2156253557 >> 172.21.15.46:0/2156253557 conn(0x7f15600683d0 msgr2=0x7f15600d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- 172.21.15.46:0/2156253557 shutdown_connections 2023-10-06T11:37:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- 172.21.15.46:0/2156253557 wait complete. 2023-10-06T11:37:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 Processor -- start 2023-10-06T11:37:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.979+0000 7f1564c8e700 1 -- start start 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cf4e0 0x7f1560140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15600d0750 0x7f156015e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600c9a20 con 0x7f15600cb020 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600c8440 con 0x7f15600cf4e0 2023-10-06T11:37:26.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15600c8720 con 0x7f15600d0750 2023-10-06T11:37:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47638/0 (socket says 172.21.15.46:47638) 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 -- 172.21.15.46:0/2919531722 learned_addr learned my addr 172.21.15.46:0/2919531722 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155ffff700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15600d0750 0x7f156015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155effd700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cf4e0 0x7f1560140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 -- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15600d0750 msgr2=0x7f156015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15600d0750 0x7f156015e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 -- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cf4e0 msgr2=0x7f1560140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cf4e0 0x7f1560140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 -- 172.21.15.46:0/2919531722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550009420 con 0x7f15600cb020 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f155f7fe700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f1560088050 tx=0x7f1550016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155001cb30 con 0x7f15600cb020 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155001cc90 con 0x7f15600cb020 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15600c8a00 con 0x7f15600cb020 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155001a6a0 con 0x7f15600cb020 2023-10-06T11:37:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.983+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15600c8fc0 con 0x7f15600cb020 2023-10-06T11:37:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.987+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560064590 con 0x7f15600cb020 2023-10-06T11:37:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.987+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f155001aa90 con 0x7f15600cb020 2023-10-06T11:37:27.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.987+0000 7f15477fe700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 0x7f154c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.987+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f155007e780 con 0x7f15600cb020 2023-10-06T11:37:27.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.991+0000 7f155effd700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 0x7f154c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.991+0000 7f155effd700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 0x7f154c062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15600c5460 tx=0x7f1554000bc0 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:27.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:26.995+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f155004e900 con 0x7f15600cb020 2023-10-06T11:37:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.227+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f15600cea00 con 0x7f15600cb020 2023-10-06T11:37:27.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.235+0000 7f15477fe700 1 -- 172.21.15.46:0/2919531722 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f15500290e0 con 0x7f15600cb020 2023-10-06T11:37:27.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 msgr2=0x7f154c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 0x7f154c062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15600c5460 tx=0x7f1554000bc0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 msgr2=0x7f15600c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f1560088050 tx=0x7f1550016000 comp rx=0 tx=0).stop 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 shutdown_connections 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f154c060670 0x7f154c062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f15600d0750 0x7f156015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15600cf4e0 0x7f1560140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 --2- 172.21.15.46:0/2919531722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15600cb020 0x7f15600c94e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 >> 172.21.15.46:0/2919531722 conn(0x7f15600683d0 msgr2=0x7f15600b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 shutdown_connections 2023-10-06T11:37:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.239+0000 7f1564c8e700 1 -- 172.21.15.46:0/2919531722 wait complete. 2023-10-06T11:37:27.255 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T11:37:27.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:37:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.423+0000 7fd70a46e700 1 Processor -- start 2023-10-06T11:37:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.423+0000 7fd70a46e700 1 -- start start 2023-10-06T11:37:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd70a46e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7041502f0 0x7fd7040c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd70a46e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7041506c0 0x7fd704159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd70a46e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7040c67e0 con 0x7fd70406e2a0 2023-10-06T11:37:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd70a46e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70406e810 con 0x7fd7041506c0 2023-10-06T11:37:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd70a46e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70406eaf0 con 0x7fd7041502f0 2023-10-06T11:37:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd6fb7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7041506c0 0x7fd704159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd6fb7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7041506c0 0x7fd704159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36202/0 (socket says 172.21.15.46:36202) 2023-10-06T11:37:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd6fb7fe700 1 -- 172.21.15.46:0/126349934 learned_addr learned my addr 172.21.15.46:0/126349934 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.427+0000 7fd703fff700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7041502f0 0x7fd7040c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd6fb7fe700 1 -- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7041502f0 msgr2=0x7fd7040c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd6fb7fe700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7041502f0 0x7fd7040c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd6fb7fe700 1 -- 172.21.15.46:0/126349934 >> v1:172.21.15.132:6789/0 conn(0x7fd70406e2a0 legacy=0x7fd704156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd6fb7fe700 1 -- 172.21.15.46:0/126349934 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70406edd0 con 0x7fd7041506c0 2023-10-06T11:37:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd7037fe700 1 -- 172.21.15.46:0/126349934 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729801786 0 0) 0x7fd7040c67e0 con 0x7fd70406e2a0 2023-10-06T11:37:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd6fb7fe700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041506c0 0x7fd704159a90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0adc3d33d8eac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd7037fe700 1 -- 172.21.15.46:0/126349934 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec009ea0 con 0x7fd7041506c0 2023-10-06T11:37:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd7037fe700 1 -- 172.21.15.46:0/126349934 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec00b860 con 0x7fd7041506c0 2023-10-06T11:37:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd7037fe700 1 -- 172.21.15.46:0/126349934 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00bc00 con 0x7fd7041506c0 2023-10-06T11:37:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 -- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041506c0 msgr2=0x7fd704159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041506c0 0x7fd704159a90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099b0 tx=0x7fd6ec009250 comp rx=0 tx=0).stop 2023-10-06T11:37:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 -- 172.21.15.46:0/126349934 shutdown_connections 2023-10-06T11:37:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041506c0 0x7fd704159a90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 --2- 172.21.15.46:0/126349934 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7041502f0 0x7fd7040c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 -- 172.21.15.46:0/126349934 >> 172.21.15.46:0/126349934 conn(0x7fd7040cbe30 msgr2=0x7fd7040cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 -- 172.21.15.46:0/126349934 shutdown_connections 2023-10-06T11:37:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.431+0000 7fd70a46e700 1 -- 172.21.15.46:0/126349934 wait complete. 2023-10-06T11:37:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 Processor -- start 2023-10-06T11:37:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 -- start start 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd70406e2a0 0x7fd70416ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041502f0 0x7fd704167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70416bac0 con 0x7fd7041506c0 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7041684b0 con 0x7fd7041502f0 2023-10-06T11:37:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd70a46e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd704168610 con 0x7fd70406e2a0 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd708a0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd708a0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47650/0 (socket says 172.21.15.46:47650) 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd708a0b700 1 -- 172.21.15.46:0/3841726461 learned_addr learned my addr 172.21.15.46:0/3841726461 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd703fff700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd70406e2a0 0x7fd70416ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.435+0000 7fd6fb7fe700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041502f0 0x7fd704167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 -- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd70406e2a0 msgr2=0x7fd70416ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd70406e2a0 0x7fd70416ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 -- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041502f0 msgr2=0x7fd704167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041502f0 0x7fd704167f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 -- 172.21.15.46:0/3841726461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0094f0 con 0x7fd7041506c0 2023-10-06T11:37:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd708a0b700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc002230 tx=0x7fd6fc0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc012630 con 0x7fd7041506c0 2023-10-06T11:37:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc012790 con 0x7fd7041506c0 2023-10-06T11:37:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7041688a0 con 0x7fd7041506c0 2023-10-06T11:37:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc012b50 con 0x7fd7041506c0 2023-10-06T11:37:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70416b580 con 0x7fd7041506c0 2023-10-06T11:37:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.439+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd704168a30 con 0x7fd7041506c0 2023-10-06T11:37:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.443+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd6fc01a3a0 con 0x7fd7041506c0 2023-10-06T11:37:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.443+0000 7fd700ff9700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 0x7fd6e4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.443+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fd6fc07cf00 con 0x7fd7041506c0 2023-10-06T11:37:27.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.443+0000 7fd703fff700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 0x7fd6e4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.447+0000 7fd703fff700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 0x7fd6e4062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009da0 tx=0x7fd6f4009120 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.447+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6fc04d000 con 0x7fd7041506c0 2023-10-06T11:37:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.659+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd704063f90 con 0x7fd7041506c0 2023-10-06T11:37:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.663+0000 7fd700ff9700 1 -- 172.21.15.46:0/3841726461 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fd6fc050620 con 0x7fd7041506c0 2023-10-06T11:37:27.673 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:27.673 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 msgr2=0x7fd6e4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 0x7fd6e4062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009da0 tx=0x7fd6f4009120 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 msgr2=0x7fd70414e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc002230 tx=0x7fd6fc0016f0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 shutdown_connections 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6e4060670 0x7fd6e4062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7041506c0 0x7fd70414e0a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7041502f0 0x7fd704167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 --2- 172.21.15.46:0/3841726461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd70406e2a0 0x7fd70416ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:27.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 >> 172.21.15.46:0/3841726461 conn(0x7fd7040cbe30 msgr2=0x7fd7040c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 shutdown_connections 2023-10-06T11:37:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:27.667+0000 7fd70a46e700 1 -- 172.21.15.46:0/3841726461 wait complete. 2023-10-06T11:37:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T11:37:28.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:37:21.446503+0000', 'modified': '2023-10-06T11:37:25.060450+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8340': {'gid': 8340, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6837/3974508311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6836', 'nonce': 3974508311}, {'type': 'v1', 'addr': '172.21.15.46:6837', 'nonce': 3974508311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:37:28.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:37:28.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T11:37:28.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 Processor -- start 2023-10-06T11:37:28.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 -- start start 2023-10-06T11:37:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 0x7f6b2c067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2c067590 0x7f6b2c068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0679f0 con 0x7f6b2c156e20 2023-10-06T11:37:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0acd60 con 0x7f6b2c067590 2023-10-06T11:37:28.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2b59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0ad040 con 0x7f6b2c0aaf30 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.259+0000 7f6b2ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2c067590 0x7f6b2c068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2c067590 0x7f6b2c068a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47664/0 (socket says 172.21.15.46:47664) 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/1833982427 learned_addr learned my addr 172.21.15.46:0/1833982427 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1833982427 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 0x7f6b2c067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/1833982427 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 msgr2=0x7f6b2c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/1833982427 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 0x7f6b2c067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/1833982427 >> v1:172.21.15.132:6789/0 conn(0x7f6b2c156e20 legacy=0x7f6b2c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/1833982427 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0ad320 con 0x7f6b2c067590 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1833982427 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 0x7f6b2c067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/1833982427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c068a20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c2b50 tx=0x7f6b24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26c227e08ebca57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2959a700 1 -- 172.21.15.46:0/1833982427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400e070 con 0x7f6b2c067590 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2959a700 1 -- 172.21.15.46:0/1833982427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400ba50 con 0x7f6b2c067590 2023-10-06T11:37:28.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2959a700 1 -- 172.21.15.46:0/1833982427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240092e0 con 0x7f6b2c067590 2023-10-06T11:37:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 -- 172.21.15.46:0/1833982427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 msgr2=0x7f6b2c068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/1833982427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c068a20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c2b50 tx=0x7f6b24009840 comp rx=0 tx=0).stop 2023-10-06T11:37:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 -- 172.21.15.46:0/1833982427 shutdown_connections 2023-10-06T11:37:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/1833982427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c068a20 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/1833982427 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0aaf30 0x7f6b2c067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 -- 172.21.15.46:0/1833982427 >> 172.21.15.46:0/1833982427 conn(0x7f6b2c062ef0 msgr2=0x7f6b2c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 -- 172.21.15.46:0/1833982427 shutdown_connections 2023-10-06T11:37:28.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.263+0000 7f6b2b59e700 1 -- 172.21.15.46:0/1833982427 wait complete. 2023-10-06T11:37:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 Processor -- start 2023-10-06T11:37:28.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 -- start start 2023-10-06T11:37:28.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c0aaf30 0x7f6b2c06cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c156e20 0x7f6b2c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c168bf0 con 0x7f6b2c067590 2023-10-06T11:37:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1699e0 con 0x7f6b2c0aaf30 2023-10-06T11:37:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c168dd0 con 0x7f6b2c156e20 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47680/0 (socket says 172.21.15.46:47680) 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2a59c700 1 -- 172.21.15.46:0/160823979 learned_addr learned my addr 172.21.15.46:0/160823979 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c156e20 0x7f6b2c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.267+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c0aaf30 0x7f6b2c06cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 -- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c156e20 msgr2=0x7f6b2c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c156e20 0x7f6b2c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 -- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c0aaf30 msgr2=0x7f6b2c06cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c0aaf30 0x7f6b2c06cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 -- 172.21.15.46:0/160823979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b24009ae0 con 0x7f6b2c067590 2023-10-06T11:37:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c4e0 tx=0x7f6b1c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b1a7fc700 1 -- 172.21.15.46:0/160823979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c011590 con 0x7f6b2c067590 2023-10-06T11:37:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b1a7fc700 1 -- 172.21.15.46:0/160823979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c011710 con 0x7f6b2c067590 2023-10-06T11:37:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c1690b0 con 0x7f6b2c067590 2023-10-06T11:37:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b1a7fc700 1 -- 172.21.15.46:0/160823979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c011ad0 con 0x7f6b2c067590 2023-10-06T11:37:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.271+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c169670 con 0x7f6b2c067590 2023-10-06T11:37:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.275+0000 7f6b1a7fc700 1 -- 172.21.15.46:0/160823979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6b1c01b3b0 con 0x7f6b2c067590 2023-10-06T11:37:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.275+0000 7f6b1a7fc700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 0x7f6b0c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.275+0000 7f6b1a7fc700 1 -- 172.21.15.46:0/160823979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f6b1c07bf10 con 0x7f6b2c067590 2023-10-06T11:37:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.275+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 0x7f6b0c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 0x7f6b0c062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c9f50 tx=0x7f6b20006f90 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 Processor -- start 2023-10-06T11:37:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 -- start start 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b84e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b85900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aaaf30 con 0x18ff710 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aa6ed0 con 0x194a790 2023-10-06T11:37:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a6d0b0 con 0x18ffae0 2023-10-06T11:37:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b29d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b29d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47690/0 (socket says 172.21.15.46:47690) 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b29d9b700 1 -- 172.21.15.46:0/4281264888 learned_addr learned my addr 172.21.15.46:0/4281264888 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b85900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b84e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 msgr2=0x1b84e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b84e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/4281264888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 msgr2=0x1b853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2ad9d700 1 -- 172.21.15.46:0/4281264888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x194b9e0 con 0x194a790 2023-10-06T11:37:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.279+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b84e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b85900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6b24015000 tx=0x7f6b24018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=850e1d88af33b6fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b0b7fe700 1 -- 172.21.15.46:0/4281264888 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240092e0 con 0x194a790 2023-10-06T11:37:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b0b7fe700 1 -- 172.21.15.46:0/4281264888 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400be60 con 0x194a790 2023-10-06T11:37:28.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b0b7fe700 1 -- 172.21.15.46:0/4281264888 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24007610 con 0x194a790 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 msgr2=0x1b85900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b85900 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6b24015000 tx=0x7f6b24018000 comp rx=0 tx=0).stop 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- 172.21.15.46:0/4281264888 shutdown_connections 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b85900 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 --2- 172.21.15.46:0/4281264888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b84e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- 172.21.15.46:0/4281264888 >> 172.21.15.46:0/4281264888 conn(0x18ffeb0 msgr2=0x1b39060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- 172.21.15.46:0/4281264888 shutdown_connections 2023-10-06T11:37:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- 172.21.15.46:0/4281264888 wait complete. 2023-10-06T11:37:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.283+0000 7f6b32c09740 1 -- start start 2023-10-06T11:37:28.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 Processor -- start 2023-10-06T11:37:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6cd50 con 0x18ff710 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a45970 con 0x194a790 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x19e3430 con 0x18ffae0 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b32c09740 10 client.0 _ll_register_callbacks cb 0x1a49b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47692/0 (socket says 172.21.15.46:47692) 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 -- 172.21.15.46:0/1691925706 learned_addr learned my addr 172.21.15.46:0/1691925706 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 -- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 msgr2=0x1b897c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b897c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 -- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 msgr2=0x1b36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.287+0000 7f6b2a59c700 1 -- 172.21.15.46:0/1691925706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c169670 con 0x18ff710 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c04fc60 tx=0x7f6b1c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d85fad6b853be3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c065100 con 0x18ff710 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f6b1c011e80 con 0x18ff710 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1a45cc0 con 0x18ff710 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1c04ff70 con 0x18ff710 2023-10-06T11:37:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 10 client.8475 fetch_fsmap learned FSMap version 67 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x7f6b2c0ad320 con 0x18ff710 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c078020 con 0x18ff710 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6b1c04f1c0 con 0x18ff710 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 10 client.8475 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 10 client.8475 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 20 client.8475 populate_metadata read hostname 'smithi046' 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b92410 0x1b947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 4 client.8475 mds_command: new command op to 8340 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 --> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1a493e0 con 0x1b92410 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b92410 0x1b947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b92410 0x1b947f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2e1716512548aa server_cookie=41448911eff4dd1a in_seq=0 out_seq=0 2023-10-06T11:37:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.291+0000 7f6b08ff9700 10 client.8475 ms_handle_connect on v2:172.21.15.46:6836/3974508311 2023-10-06T11:37:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b08ff9700 1 -- 172.21.15.46:0/1691925706 <== mds.0 v2:172.21.15.46:6836/3974508311 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1a493e0 con 0x1b92410 2023-10-06T11:37:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b08ff9700 10 client.8475 handle_command_reply: tid=0 2023-10-06T11:37:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 client.8475 shutdown 2023-10-06T11:37:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 msgr2=0x1b89280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c04fc60 tx=0x7f6b1c014040 comp rx=0 tx=0).stop 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 shutdown_connections 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1b897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b92410 0x1b947f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1691925706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b89280 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 >> 172.21.15.46:0/1691925706 conn(0x18ffeb0 msgr2=0x1b5e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b08ff9700 0 client.8475 ms_handle_reset on v2:172.21.15.46:6836/3974508311 2023-10-06T11:37:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 shutdown_connections 2023-10-06T11:37:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 1 -- 172.21.15.46:0/1691925706 wait complete. 2023-10-06T11:37:28.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.295+0000 7f6b32c09740 20 client.8475 trim_cache size 0 max 16384 2023-10-06T11:37:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 Processor -- start 2023-10-06T11:37:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 -- start start 2023-10-06T11:37:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 0x1b5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a55c90 con 0x18ff710 2023-10-06T11:37:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a70400 con 0x18ffae0 2023-10-06T11:37:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.315+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1aaa5d0 con 0x194a790 2023-10-06T11:37:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36260/0 (socket says 172.21.15.46:36260) 2023-10-06T11:37:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 -- 172.21.15.46:0/1982023924 learned_addr learned my addr 172.21.15.46:0/1982023924 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 0x1b5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 -- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 msgr2=0x1b5c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 0x1b5c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 -- 172.21.15.46:0/1982023924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 msgr2=0x1b34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b34a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 -- 172.21.15.46:0/1982023924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a75370 con 0x18ffae0 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 0x1b5c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000afd0 tx=0x7f6b20018040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec04c016a7d7d60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b0bfff700 1 -- 172.21.15.46:0/1982023924 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20013c40 con 0x18ffae0 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b0bfff700 1 -- 172.21.15.46:0/1982023924 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000c040 con 0x18ffae0 2023-10-06T11:37:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b0bfff700 1 -- 172.21.15.46:0/1982023924 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2002d480 con 0x18ffae0 2023-10-06T11:37:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 msgr2=0x1b34f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000afd0 tx=0x7f6b20018040 comp rx=0 tx=0).stop 2023-10-06T11:37:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- 172.21.15.46:0/1982023924 shutdown_connections 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x194a790 0x1b5c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x18ffae0 0x1b34f50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 --2- 172.21.15.46:0/1982023924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1b34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- 172.21.15.46:0/1982023924 >> 172.21.15.46:0/1982023924 conn(0x18ffeb0 msgr2=0x1b383a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- 172.21.15.46:0/1982023924 shutdown_connections 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- 172.21.15.46:0/1982023924 wait complete. 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 -- start start 2023-10-06T11:37:28.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.319+0000 7f6b32c09740 1 Processor -- start 2023-10-06T11:37:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1bdf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b93cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1aabef0 con 0x18ff710 2023-10-06T11:37:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a7c550 con 0x194a790 2023-10-06T11:37:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a7c010 con 0x18ffae0 2023-10-06T11:37:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 10 client.0 _ll_register_callbacks cb 0x1a72110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47708/0 (socket says 172.21.15.46:47708) 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 -- 172.21.15.46:0/2964870711 learned_addr learned my addr 172.21.15.46:0/2964870711 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b93cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1bdf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 -- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 msgr2=0x1bdf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1bdf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 -- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 msgr2=0x1b93cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b93cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 -- 172.21.15.46:0/2964870711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20013460 con 0x18ff710 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2ad9d700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b93cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b2a59c700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c04c1d0 tx=0x7f6b1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e612a98f8e2d301d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T11:37:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c065100 con 0x18ff710 2023-10-06T11:37:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f6b1c079080 con 0x18ff710 2023-10-06T11:37:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.323+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b2400be60 con 0x18ff710 2023-10-06T11:37:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1c04f630 con 0x18ff710 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 10 client.8481 fetch_fsmap learned FSMap version 67 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x1a5bd80 con 0x18ff710 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400be60 con 0x18ff710 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 67) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6b1c04ff70 con 0x18ff710 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 10 client.8481 fetch_fsmap finished waiting for FSMap version 67 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 10 client.8481 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 20 client.8481 populate_metadata read hostname 'smithi046' 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b94680 0x1b94a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 4 client.8481 mds_command: new command op to 8340 tid=0[{"prefix": "status"}] 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 --> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1a7b410 con 0x1b94680 2023-10-06T11:37:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b94680 0x1b94a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b29d9b700 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b94680 0x1b94a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5c7f9ec60c98f7 server_cookie=5d72da7248fd230d in_seq=0 out_seq=0 2023-10-06T11:37:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 10 client.8481 ms_handle_connect on v2:172.21.15.46:6836/3974508311 2023-10-06T11:37:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 1 -- 172.21.15.46:0/2964870711 <== mds.0 v2:172.21.15.46:6836/3974508311 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x1a7b410 con 0x1b94680 2023-10-06T11:37:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 10 client.8481 handle_command_reply: tid=0 2023-10-06T11:37:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 client.8481 shutdown 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 msgr2=0x1bdf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c04c1d0 tx=0x7f6b1c017000 comp rx=0 tx=0).stop 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 shutdown_connections 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x194a790 0x1b93cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x18ffae0 0x1bdf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:6836/3974508311,v1:172.21.15.46:6837/3974508311] conn(0x1b94680 0x1b94a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 --2- 172.21.15.46:0/2964870711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x18ff710 0x1bdf1d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 >> 172.21.15.46:0/2964870711 conn(0x18ffeb0 msgr2=0x1b34a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b08ff9700 0 client.8481 ms_handle_reset on v2:172.21.15.46:6836/3974508311 2023-10-06T11:37:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 shutdown_connections 2023-10-06T11:37:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.327+0000 7f6b32c09740 1 -- 172.21.15.46:0/2964870711 wait complete. 2023-10-06T11:37:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b32c09740 20 client.8481 trim_cache size 0 max 16384 2023-10-06T11:37:28.342 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T11:37:28.342 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8340, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.3257413189999996, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 67, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 135, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 135, 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.243154954 2023-10-06T11:37:28.343 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T11:37:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 msgr2=0x7f6b0c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 0x7f6b0c062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c9f50 tx=0x7f6b20006f90 comp rx=0 tx=0).stop 2023-10-06T11:37:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 msgr2=0x7f6b2c153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c4e0 tx=0x7f6b1c00c7f0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 shutdown_connections 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6b0c060670 0x7f6b0c062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c156e20 0x7f6b2c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c0aaf30 0x7f6b2c06cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 --2- 172.21.15.46:0/160823979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2c067590 0x7f6b2c153e80 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 >> 172.21.15.46:0/160823979 conn(0x7f6b2c062ef0 msgr2=0x7f6b2c0690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 shutdown_connections 2023-10-06T11:37:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.331+0000 7f6b2b59e700 1 -- 172.21.15.46:0/160823979 wait complete. 2023-10-06T11:37:28.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:37:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.515+0000 7f9858e6e700 1 Processor -- start 2023-10-06T11:37:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.519+0000 7f9858e6e700 1 -- start start 2023-10-06T11:37:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.519+0000 7f9858e6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98541502f0 0x7f98540c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.519+0000 7f9858e6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540c67e0 con 0x7f98541506c0 2023-10-06T11:37:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.519+0000 7f9858e6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985406e810 con 0x7f985415a1a0 2023-10-06T11:37:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.519+0000 7f9858e6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985406eaf0 con 0x7f98541502f0 2023-10-06T11:37:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98541502f0 0x7f98540c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98541502f0 0x7f98540c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47724/0 (socket says 172.21.15.46:47724) 2023-10-06T11:37:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 -- 172.21.15.46:0/919188902 learned_addr learned my addr 172.21.15.46:0/919188902 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 -- 172.21.15.46:0/919188902 >> v1:172.21.15.132:6789/0 conn(0x7f985415a1a0 legacy=0x7f98540c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 -- 172.21.15.46:0/919188902 >> v1:172.21.15.46:6789/0 conn(0x7f98541506c0 legacy=0x7f9854159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 -- 172.21.15.46:0/919188902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985406edd0 con 0x7f98541502f0 2023-10-06T11:37:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985259c700 1 --2- 172.21.15.46:0/919188902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f98540c0bd0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9844009990 tx=0x7f9844009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6073d4af27e9be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985159a700 1 -- 172.21.15.46:0/919188902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844005c10 con 0x7f98541502f0 2023-10-06T11:37:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985159a700 1 -- 172.21.15.46:0/919188902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844005d70 con 0x7f98541502f0 2023-10-06T11:37:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f985159a700 1 -- 172.21.15.46:0/919188902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984400b860 con 0x7f98541502f0 2023-10-06T11:37:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 -- 172.21.15.46:0/919188902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 msgr2=0x7f98540c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 --2- 172.21.15.46:0/919188902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f98540c0bd0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9844009990 tx=0x7f9844009220 comp rx=0 tx=0).stop 2023-10-06T11:37:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 -- 172.21.15.46:0/919188902 shutdown_connections 2023-10-06T11:37:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 --2- 172.21.15.46:0/919188902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f98540c0bd0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 -- 172.21.15.46:0/919188902 >> 172.21.15.46:0/919188902 conn(0x7f98540cbe30 msgr2=0x7f98540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 -- 172.21.15.46:0/919188902 shutdown_connections 2023-10-06T11:37:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.523+0000 7f9858e6e700 1 -- 172.21.15.46:0/919188902 wait complete. 2023-10-06T11:37:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 Processor -- start 2023-10-06T11:37:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 -- start start 2023-10-06T11:37:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 0x7f98540b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985415a1a0 0x7f98540b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98540c5400 con 0x7f98541502f0 2023-10-06T11:37:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854169d90 con 0x7f98541506c0 2023-10-06T11:37:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9858e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985416a010 con 0x7f985415a1a0 2023-10-06T11:37:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47736/0 (socket says 172.21.15.46:47736) 2023-10-06T11:37:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 -- 172.21.15.46:0/2452749055 learned_addr learned my addr 172.21.15.46:0/2452749055 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9852d9d700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985415a1a0 0x7f98540b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9851d9b700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 0x7f98540b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 -- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985415a1a0 msgr2=0x7f98540b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985415a1a0 0x7f98540b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 -- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 msgr2=0x7f98540b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 0x7f98540b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 -- 172.21.15.46:0/2452749055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009480 con 0x7f98541502f0 2023-10-06T11:37:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f9851d9b700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 0x7f98540b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:37:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.527+0000 7f985259c700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f985406c150 tx=0x7f9844016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844011b60 con 0x7f98541502f0 2023-10-06T11:37:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844011ce0 con 0x7f98541502f0 2023-10-06T11:37:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985416a2f0 con 0x7f98541502f0 2023-10-06T11:37:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984401b680 con 0x7f98541502f0 2023-10-06T11:37:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985416a8b0 con 0x7f98541502f0 2023-10-06T11:37:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9854063f90 con 0x7f98541502f0 2023-10-06T11:37:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.531+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f984401ba70 con 0x7f98541502f0 2023-10-06T11:37:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.535+0000 7f98427fc700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 0x7f9834062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.535+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f984407e900 con 0x7f98541502f0 2023-10-06T11:37:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.535+0000 7f9851d9b700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 0x7f9834062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.535+0000 7f9851d9b700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 0x7f9834062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9854159d90 tx=0x7f9848009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.539+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f984404ea00 con 0x7f98541502f0 2023-10-06T11:37:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.747+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98540c6470 con 0x7f98541502f0 2023-10-06T11:37:28.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.747+0000 7f98427fc700 1 -- 172.21.15.46:0/2452749055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f984402a020 con 0x7f98541502f0 2023-10-06T11:37:28.761 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 msgr2=0x7f9834062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 0x7f9834062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9854159d90 tx=0x7f9848009040 comp rx=0 tx=0).stop 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 msgr2=0x7f9854163a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f985406c150 tx=0x7f9844016000 comp rx=0 tx=0).stop 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 shutdown_connections 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9834060670 0x7f9834062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985415a1a0 0x7f98540b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98541506c0 0x7f98540b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 --2- 172.21.15.46:0/2452749055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98541502f0 0x7f9854163a20 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 >> 172.21.15.46:0/2452749055 conn(0x7f98540cbe30 msgr2=0x7f9854069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 shutdown_connections 2023-10-06T11:37:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:28.751+0000 7f9858e6e700 1 -- 172.21.15.46:0/2452749055 wait complete. 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:37:28.783 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:37:28.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:28.784 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.803 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41281sec preferred_lft 41281sec 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T11:37:28.804 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T11:37:28.805 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T11:37:28.806 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:37:28.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:28.807 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:37:28.890 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T11:37:28.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:28.894 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T11:37:28.960 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:37:28.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:28.961 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T11:37:28.962 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T11:37:28.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:37:28.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:37:28.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:37:28.962 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T11:37:29.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:29.093 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T11:37:29.111 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:37:29.111 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:37:29.112 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:37:29.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:29.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T11:37:29.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:37:29.190 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:37:29.252 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:37:29.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:37:29.252 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:37:29.311 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T11:37:29.311 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:37:29.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:37:29.434 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:37:29.434 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:37:29.434 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:37:29.460 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:37:29.460 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:37:29.460 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:37:29.460 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:37:29.461 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T11:37:29.473 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T11:37:29.474 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T11:37:29.474 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T11:37:29.478 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T11:37:29.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:37:29.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:37:29.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:37:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.723+0000 7ff1c0a4c700 1 Processor -- start 2023-10-06T11:37:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.723+0000 7ff1c0a4c700 1 -- start start 2023-10-06T11:37:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.723+0000 7ff1c0a4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1bc158dd0 0x7ff1bc067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1c0a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc0679f0 con 0x7ff1bc067590 2023-10-06T11:37:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1c0a4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc15d6c0 con 0x7ff1bc1591a0 2023-10-06T11:37:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1c0a4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1bc0699c0 con 0x7ff1bc158dd0 2023-10-06T11:37:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bb7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1bc158dd0 0x7ff1bc067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bbfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1bc067590 0x7ff1bc06c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51028/0 (socket says 172.21.15.46:51028) 2023-10-06T11:37:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bbfff700 1 -- 172.21.15.46:0/2095452311 learned_addr learned my addr 172.21.15.46:0/2095452311 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bb7fe700 1 -- 172.21.15.46:0/2095452311 >> v1:172.21.15.132:6789/0 conn(0x7ff1bc1591a0 legacy=0x7ff1bc068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bb7fe700 1 -- 172.21.15.46:0/2095452311 >> v1:172.21.15.46:6789/0 conn(0x7ff1bc067590 legacy=0x7ff1bc06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1ba7fc700 1 -- 172.21.15.46:0/2095452311 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998571717 0 0) 0x7ff1bc15d6c0 con 0x7ff1bc1591a0 2023-10-06T11:37:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bb7fe700 1 -- 172.21.15.46:0/2095452311 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc15cd90 con 0x7ff1bc158dd0 2023-10-06T11:37:29.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1bb7fe700 1 --2- 172.21.15.46:0/2095452311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc067050 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098e0 tx=0x7ff1ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59c1d7b623a8645 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1ba7fc700 1 -- 172.21.15.46:0/2095452311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac009e80 con 0x7ff1bc158dd0 2023-10-06T11:37:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1ba7fc700 1 -- 172.21.15.46:0/2095452311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac00b860 con 0x7ff1bc158dd0 2023-10-06T11:37:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1ba7fc700 1 -- 172.21.15.46:0/2095452311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac00bbe0 con 0x7ff1bc158dd0 2023-10-06T11:37:29.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2095452311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc158dd0 msgr2=0x7ff1bc067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.727+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2095452311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc067050 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0098e0 tx=0x7ff1ac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2095452311 shutdown_connections 2023-10-06T11:37:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2095452311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc067050 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2095452311 >> 172.21.15.46:0/2095452311 conn(0x7ff1bc062ef0 msgr2=0x7ff1bc0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2095452311 shutdown_connections 2023-10-06T11:37:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2095452311 wait complete. 2023-10-06T11:37:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 Processor -- start 2023-10-06T11:37:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- start start 2023-10-06T11:37:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 0x7ff1bc14d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc169610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc158530 con 0x7ff1bc1591a0 2023-10-06T11:37:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc16a4d0 con 0x7ff1bc158dd0 2023-10-06T11:37:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1c0a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc16a1f0 con 0x7ff1bc067590 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.731+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 0x7ff1bc14d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47752/0 (socket says 172.21.15.46:47752) 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 -- 172.21.15.46:0/2147432199 learned_addr learned my addr 172.21.15.46:0/2147432199 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1baffd700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc169610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 -- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 msgr2=0x7ff1bc14d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 0x7ff1bc14d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 -- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1bc158dd0 msgr2=0x7ff1bc169610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc169610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 -- 172.21.15.46:0/2147432199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac009420 con 0x7ff1bc1591a0 2023-10-06T11:37:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bb7fe700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 0x7ff1bc14d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1bbfff700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400cc40 tx=0x7ff1b4001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:29.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400a8b0 con 0x7ff1bc1591a0 2023-10-06T11:37:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b400aa10 con 0x7ff1bc1591a0 2023-10-06T11:37:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400add0 con 0x7ff1bc1591a0 2023-10-06T11:37:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc16ccb0 con 0x7ff1bc1591a0 2023-10-06T11:37:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc16d270 con 0x7ff1bc1591a0 2023-10-06T11:37:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.735+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc064830 con 0x7ff1bc1591a0 2023-10-06T11:37:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.739+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff1b4007300 con 0x7ff1bc1591a0 2023-10-06T11:37:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.739+0000 7ff1a37fe700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 0x7ff1a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.739+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7ff1b407cd60 con 0x7ff1bc1591a0 2023-10-06T11:37:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.739+0000 7ff1bb7fe700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 0x7ff1a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.739+0000 7ff1bb7fe700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 0x7ff1a8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0093f0 tx=0x7ff1ac017000 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:29.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.747+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff1bc064830 con 0x7ff1bc1591a0 2023-10-06T11:37:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.939+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1bc0679f0 con 0x7ff1a8060670 2023-10-06T11:37:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.939+0000 7ff1a37fe700 1 -- 172.21.15.46:0/2147432199 <== mgr.6956 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff1bc0679f0 con 0x7ff1a8060670 2023-10-06T11:37:29.952 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T11:37:29.952 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T11:37:29.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T11:37:29.952 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T11:37:29.952 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T11:37:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 msgr2=0x7ff1a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 0x7ff1a8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0093f0 tx=0x7ff1ac017000 comp rx=0 tx=0).stop 2023-10-06T11:37:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 msgr2=0x7ff1bc169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400cc40 tx=0x7ff1b4001a50 comp rx=0 tx=0).stop 2023-10-06T11:37:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 shutdown_connections 2023-10-06T11:37:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff1a8060670 0x7ff1a8062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1bc1591a0 0x7ff1bc169b50 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1bc158dd0 0x7ff1bc169610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 --2- 172.21.15.46:0/2147432199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1bc067590 0x7ff1bc14d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 >> 172.21.15.46:0/2147432199 conn(0x7ff1bc062ef0 msgr2=0x7ff1bc06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 shutdown_connections 2023-10-06T11:37:29.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:29.943+0000 7ff1c0a4c700 1 -- 172.21.15.46:0/2147432199 wait complete. 2023-10-06T11:37:29.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T11:37:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 Processor -- start 2023-10-06T11:37:30.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 -- start start 2023-10-06T11:37:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12b806ed10 0x7f12b80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b80c1bf0 con 0x7f12b80c6470 2023-10-06T11:37:30.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80c72c0 con 0x7f12b806ed10 2023-10-06T11:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.143+0000 7f12bdb85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b806cf80 con 0x7f12b806e8a0 2023-10-06T11:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12b806ed10 0x7f12b80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b77fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f12b80c6470 0x7f12b80ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:48196/0 (socket says 172.21.15.46:48196) 2023-10-06T11:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b77fe700 1 -- 172.21.15.46:0/2697441559 learned_addr learned my addr 172.21.15.46:0/2697441559 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12affff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b6ffd700 1 -- 172.21.15.46:0/2697441559 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798634543 0 0) 0x7f12b80c1bf0 con 0x7f12b80c6470 2023-10-06T11:37:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b6ffd700 1 -- 172.21.15.46:0/2697441559 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a4003570 con 0x7f12b80c6470 2023-10-06T11:37:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 -- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 msgr2=0x7f12b80c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:30.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 --2- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 -- 172.21.15.46:0/2697441559 >> v1:172.21.15.132:6789/0 conn(0x7f12b80c6470 legacy=0x7f12b80ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 -- 172.21.15.46:0/2697441559 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b806d260 con 0x7f12b806ed10 2023-10-06T11:37:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b7fff700 1 --2- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b80c6d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009e60 tx=0x7f12b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b52499d4175a02d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b6ffd700 1 -- 172.21.15.46:0/2697441559 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000e070 con 0x7f12b806ed10 2023-10-06T11:37:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b6ffd700 1 -- 172.21.15.46:0/2697441559 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000bab0 con 0x7f12b806ed10 2023-10-06T11:37:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12b6ffd700 1 -- 172.21.15.46:0/2697441559 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000be70 con 0x7f12b806ed10 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12bdb85700 1 -- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 msgr2=0x7f12b80c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.147+0000 7f12bdb85700 1 --2- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b80c6d80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009e60 tx=0x7f12b0009780 comp rx=0 tx=0).stop 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 -- 172.21.15.46:0/2697441559 shutdown_connections 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 --2- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b80c6d80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 --2- 172.21.15.46:0/2697441559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 -- 172.21.15.46:0/2697441559 >> 172.21.15.46:0/2697441559 conn(0x7f12b80cbe30 msgr2=0x7f12b80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 -- 172.21.15.46:0/2697441559 shutdown_connections 2023-10-06T11:37:30.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 -- 172.21.15.46:0/2697441559 wait complete. 2023-10-06T11:37:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 Processor -- start 2023-10-06T11:37:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.151+0000 7f12bdb85700 1 -- start start 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80c8ae0 con 0x7f12b80c6470 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b815e040 con 0x7f12b806ed10 2023-10-06T11:37:30.166 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80af7f0 con 0x7f12b806e8a0 2023-10-06T11:37:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47764/0 (socket says 172.21.15.46:47764) 2023-10-06T11:37:30.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 -- 172.21.15.46:0/3820925499 learned_addr learned my addr 172.21.15.46:0/3820925499 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b77fe700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12affff700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 -- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 msgr2=0x7f12b80c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 -- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 msgr2=0x7f12b815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 -- 172.21.15.46:0/3820925499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0009a20 con 0x7f12b80c6470 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12b7fff700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f12b0013000 tx=0x7f12b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0012e10 con 0x7f12b80c6470 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0027470 con 0x7f12b80c6470 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b80b0480 con 0x7f12b80c6470 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0027830 con 0x7f12b80c6470 2023-10-06T11:37:30.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.155+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b80b0a40 con 0x7f12b80c6470 2023-10-06T11:37:30.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.159+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b80660f0 con 0x7f12b80c6470 2023-10-06T11:37:30.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.159+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 63) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f12b0024070 con 0x7f12b80c6470 2023-10-06T11:37:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.163+0000 7f12af7fe700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 0x7f1298062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.163+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f12b007f640 con 0x7f12b80c6470 2023-10-06T11:37:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.163+0000 7f12b77fe700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 0x7f1298062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.163+0000 7f12b77fe700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 0x7f1298062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f12a8007fd0 tx=0x7f12a8009040 comp rx=0 tx=0).ready entity=mgr.6956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:30.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.167+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12b004f740 con 0x7f12b80c6470 2023-10-06T11:37:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.387+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f12b804b5c0 con 0x7f12b80c6470 2023-10-06T11:37:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.455+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 64) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f12b0003040 con 0x7f12b80c6470 2023-10-06T11:37:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.455+0000 7f12af7fe700 1 -- 172.21.15.46:0/3820925499 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v64) v1 ==== 92+0+0 (secure 0 0 0) 0x7f12b00531d0 con 0x7f12b80c6470 2023-10-06T11:37:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 msgr2=0x7f1298062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 0x7f1298062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f12a8007fd0 tx=0x7f12a8009040 comp rx=0 tx=0).stop 2023-10-06T11:37:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 msgr2=0x7f12b80c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f12b0013000 tx=0x7f12b0019000 comp rx=0 tx=0).stop 2023-10-06T11:37:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 shutdown_connections 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1298060670 0x7f1298062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12b80c6470 0x7f12b80c9ee0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b806ed10 0x7f12b815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 --2- 172.21.15.46:0/3820925499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12b806e8a0 0x7f12b80c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 >> 172.21.15.46:0/3820925499 conn(0x7f12b80cbe30 msgr2=0x7f12b80c1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 shutdown_connections 2023-10-06T11:37:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.459+0000 7f12bdb85700 1 -- 172.21.15.46:0/3820925499 wait complete. 2023-10-06T11:37:30.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T11:37:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 Processor -- start 2023-10-06T11:37:30.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 -- start start 2023-10-06T11:37:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 0x7fc140159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1400ce6e0 con 0x7fc140069270 2023-10-06T11:37:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14015a5c0 con 0x7fc140159850 2023-10-06T11:37:30.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1400cddb0 con 0x7fc1401591b0 2023-10-06T11:37:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13ed9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 0x7fc140159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:57038/0 (socket says 172.21.15.46:57038) 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 -- 172.21.15.46:0/4221866106 learned_addr learned my addr 172.21.15.46:0/4221866106 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13ed9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 0x7fc140159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47782/0 (socket says 172.21.15.46:47782) 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 -- 172.21.15.46:0/4221866106 >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 msgr2=0x7fc140159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 --2- 172.21.15.46:0/4221866106 >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 0x7fc140159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 -- 172.21.15.46:0/4221866106 >> v1:172.21.15.132:6789/0 conn(0x7fc140069270 legacy=0x7fc140158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 -- 172.21.15.46:0/4221866106 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1400ce090 con 0x7fc1401591b0 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.647+0000 7fc13dd9b700 1 --2- 172.21.15.46:0/4221866106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc1340099b0 tx=0x7fc134009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b6dfdcc96cdacf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13d59a700 1 -- 172.21.15.46:0/4221866106 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134009ea0 con 0x7fc1401591b0 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13d59a700 1 -- 172.21.15.46:0/4221866106 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13400b860 con 0x7fc1401591b0 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13d59a700 1 -- 172.21.15.46:0/4221866106 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400bc00 con 0x7fc1401591b0 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 -- 172.21.15.46:0/4221866106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 msgr2=0x7fc140069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 --2- 172.21.15.46:0/4221866106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc1340099b0 tx=0x7fc134009250 comp rx=0 tx=0).stop 2023-10-06T11:37:30.663 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:30.660+0000 7f298ff25000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:37:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 -- 172.21.15.46:0/4221866106 shutdown_connections 2023-10-06T11:37:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 --2- 172.21.15.46:0/4221866106 >> v2:172.21.15.46:3300/0 conn(0x7fc140159850 0x7fc140159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 --2- 172.21.15.46:0/4221866106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140069640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 -- 172.21.15.46:0/4221866106 >> 172.21.15.46:0/4221866106 conn(0x7fc140068390 msgr2=0x7fc1400d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 -- 172.21.15.46:0/4221866106 shutdown_connections 2023-10-06T11:37:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.651+0000 7fc13f59e700 1 -- 172.21.15.46:0/4221866106 wait complete. 2023-10-06T11:37:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 Processor -- start 2023-10-06T11:37:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 -- start start 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140069270 0x7fc1400c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14016ce70 con 0x7fc140159850 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140169db0 con 0x7fc140069270 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14016a030 con 0x7fc1401591b0 2023-10-06T11:37:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140069270 0x7fc1400c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47794/0 (socket says 172.21.15.46:47794) 2023-10-06T11:37:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 -- 172.21.15.46:0/2145729259 learned_addr learned my addr 172.21.15.46:0/2145729259 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13dd9b700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 msgr2=0x7fc140169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140069270 msgr2=0x7fc1400c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140069270 0x7fc1400c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.655+0000 7fc13ed9d700 1 -- 172.21.15.46:0/2145729259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340094f0 con 0x7fc140159850 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc13ed9d700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc138009df0 tx=0x7fc13800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800caa0 con 0x7fc140159850 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13800f040 con 0x7fc140159850 2023-10-06T11:37:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800ce70 con 0x7fc140159850 2023-10-06T11:37:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14016a310 con 0x7fc140159850 2023-10-06T11:37:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14016d080 con 0x7fc140159850 2023-10-06T11:37:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14016d270 con 0x7fc140159850 2023-10-06T11:37:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.659+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 64) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fc138014690 con 0x7fc140159850 2023-10-06T11:37:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.663+0000 7fc12e7fc700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 0x7fc120062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.663+0000 7fc13e59c700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 msgr2=0x7fc120062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.663+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fc13807d660 con 0x7fc140159850 2023-10-06T11:37:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.663+0000 7fc13e59c700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 0x7fc120062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:30.677 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:30.667+0000 7f635c32a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:37:30.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.671+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc1380510e0 con 0x7fc140159850 2023-10-06T11:37:30.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.859+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc140064590 con 0x7fc140159850 2023-10-06T11:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.867+0000 7fc13e59c700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 msgr2=0x7fc120062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:30.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.867+0000 7fc13e59c700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 0x7fc120062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:30.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.867+0000 7fc12e7fc700 1 -- 172.21.15.46:0/2145729259 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc138050c70 con 0x7fc140159850 2023-10-06T11:37:30.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 msgr2=0x7fc120062bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 0x7fc120062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 msgr2=0x7fc140146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc138009df0 tx=0x7fc13800b040 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 shutdown_connections 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc120060740 0x7fc120062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc140159850 0x7fc140146e70 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1401591b0 0x7fc140169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 --2- 172.21.15.46:0/2145729259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc140069270 0x7fc1400c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:30.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 >> 172.21.15.46:0/2145729259 conn(0x7fc140068390 msgr2=0x7fc1400b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 shutdown_connections 2023-10-06T11:37:30.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:30.871+0000 7fc13f59e700 1 -- 172.21.15.46:0/2145729259 wait complete. 2023-10-06T11:37:30.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T11:37:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.059+0000 7f08d922e700 1 Processor -- start 2023-10-06T11:37:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.059+0000 7f08d922e700 1 -- start start 2023-10-06T11:37:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d922e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d922e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d922e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d4069c10 con 0x7f08d4069410 2023-10-06T11:37:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d922e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d415a890 con 0x7f08d4159420 2023-10-06T11:37:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d922e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40ce470 con 0x7f08d40697e0 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d259c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d259c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36364/0 (socket says 172.21.15.46:36364) 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d259c700 1 -- 172.21.15.46:0/2601823883 learned_addr learned my addr 172.21.15.46:0/2601823883 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d2d9d700 1 --1- 172.21.15.46:0/2601823883 >> v1:172.21.15.132:6789/0 conn(0x7f08d4069410 0x7f08d4158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:48224/0 (socket says 172.21.15.46:48224) 2023-10-06T11:37:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 -- 172.21.15.46:0/2601823883 >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 msgr2=0x7f08d415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 --2- 172.21.15.46:0/2601823883 >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 -- 172.21.15.46:0/2601823883 >> v1:172.21.15.132:6789/0 conn(0x7f08d4069410 legacy=0x7f08d4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 -- 172.21.15.46:0/2601823883 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d415d470 con 0x7f08d40697e0 2023-10-06T11:37:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d259c700 1 --2- 172.21.15.46:0/2601823883 >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.063+0000 7f08d359e700 1 --2- 172.21.15.46:0/2601823883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d415a350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e60 tx=0x7f08cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3afec4e46c66975d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d1d9b700 1 -- 172.21.15.46:0/2601823883 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00e070 con 0x7f08d40697e0 2023-10-06T11:37:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d1d9b700 1 -- 172.21.15.46:0/2601823883 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc00bb60 con 0x7f08d40697e0 2023-10-06T11:37:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d1d9b700 1 -- 172.21.15.46:0/2601823883 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc012430 con 0x7f08d40697e0 2023-10-06T11:37:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- 172.21.15.46:0/2601823883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 msgr2=0x7f08d415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 --2- 172.21.15.46:0/2601823883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d415a350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e60 tx=0x7f08cc009780 comp rx=0 tx=0).stop 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- 172.21.15.46:0/2601823883 shutdown_connections 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 --2- 172.21.15.46:0/2601823883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d415a350 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 --2- 172.21.15.46:0/2601823883 >> v2:172.21.15.132:3300/0 conn(0x7f08d4159420 0x7f08d415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- 172.21.15.46:0/2601823883 >> 172.21.15.46:0/2601823883 conn(0x7f08d40d0210 msgr2=0x7f08d40d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- 172.21.15.46:0/2601823883 shutdown_connections 2023-10-06T11:37:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- 172.21.15.46:0/2601823883 wait complete. 2023-10-06T11:37:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 Processor -- start 2023-10-06T11:37:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.067+0000 7f08d922e700 1 -- start start 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 0x7f08d4169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 0x7f08d4156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d415df50 con 0x7f08d4069410 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d414e660 con 0x7f08d4159420 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d415ef60 con 0x7f08d40697e0 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 0x7f08d4169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:57066/0 (socket says 172.21.15.46:57066) 2023-10-06T11:37:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d259c700 1 -- 172.21.15.46:0/208926302 learned_addr learned my addr 172.21.15.46:0/208926302 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d359e700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 0x7f08d4156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 -- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 msgr2=0x7f08d414e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 -- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 msgr2=0x7f08d4156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 0x7f08d4156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 -- 172.21.15.46:0/208926302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80094f0 con 0x7f08d4069410 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d259c700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d359e700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 0x7f08d4156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d2d9d700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 0x7f08d4169e20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c7c0 tx=0x7f08c4007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4005a70 con 0x7f08d4069410 2023-10-06T11:37:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c4005bf0 con 0x7f08d4069410 2023-10-06T11:37:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40070d0 con 0x7f08d4069410 2023-10-06T11:37:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 -- 172.21.15.46:0/208926302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc009a20 con 0x7f08d4069410 2023-10-06T11:37:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.071+0000 7f08d922e700 1 -- 172.21.15.46:0/208926302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d415e480 con 0x7f08d4069410 2023-10-06T11:37:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.075+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d4088870 con 0x7f08d4069410 2023-10-06T11:37:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.075+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 64) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f08c4007480 con 0x7f08d4069410 2023-10-06T11:37:31.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.075+0000 7f08baffd700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 0x7f08c0062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.075+0000 7f08d259c700 1 -- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 msgr2=0x7f08c0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.075+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f08c407c050 con 0x7f08d4069410 2023-10-06T11:37:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.079+0000 7f08d259c700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 0x7f08c0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:31.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.083+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f08c404c150 con 0x7f08d4069410 2023-10-06T11:37:31.197 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.192+0000 7f298ff25000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:37:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.255+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f08d4069c10 con 0x7f08d4069410 2023-10-06T11:37:31.267 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.264+0000 7f298ff25000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:37:31.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.259+0000 7f08baffd700 1 -- 172.21.15.46:0/208926302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f08d4069c10 con 0x7f08d4069410 2023-10-06T11:37:31.272 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:31.259+0000 7f635c32a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:37:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 msgr2=0x7f08c0062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 0x7f08c0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 msgr2=0x7f08d4169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 0x7f08d4169e20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c7c0 tx=0x7f08c4007eb0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 shutdown_connections 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c00606c0 0x7f08c0062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d4159420 0x7f08d4156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d40697e0 0x7f08d414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 --2- 172.21.15.46:0/208926302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d4069410 0x7f08d4169e20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 >> 172.21.15.46:0/208926302 conn(0x7f08d40d0210 msgr2=0x7f08d40ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 shutdown_connections 2023-10-06T11:37:31.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.263+0000 7f08b8ff9700 1 -- 172.21.15.46:0/208926302 wait complete. 2023-10-06T11:37:31.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:37:31.297 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T11:37:31.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-10-06T11:37:31.354 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:31.343+0000 7f635c32a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:37:31.404 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.400+0000 7f298ff25000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:31.502 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:31.491+0000 7f635c32a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.507+0000 7fc055a8e700 1 Processor -- start 2023-10-06T11:37:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.507+0000 7fc055a8e700 1 -- start start 2023-10-06T11:37:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc055a8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc0500cc500 0x7fc05015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc055a8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0500cf360 con 0x7fc0501595c0 2023-10-06T11:37:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc055a8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05015dd60 con 0x7fc0501591f0 2023-10-06T11:37:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc055a8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0500cff00 con 0x7fc0500cc500 2023-10-06T11:37:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0501595c0 0x7fc0500af940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51034/0 (socket says 172.21.15.46:51034) 2023-10-06T11:37:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04ffff700 1 -- 172.21.15.46:0/2674903978 learned_addr learned my addr 172.21.15.46:0/2674903978 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T11:37:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc05528d700 1 --2- 172.21.15.46:0/2674903978 >> v2:172.21.15.132:3300/0 conn(0x7fc0500cc500 0x7fc05015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957331732 0 0) 0x7fc0500cf360 con 0x7fc0501595c0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc040003570 con 0x7fc0501595c0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64364138 0 0) 0x7fc05015dd60 con 0x7fc0501591f0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0500cf360 con 0x7fc0501591f0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212802731 0 0) 0x7fc040003570 con 0x7fc0501595c0 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 >> v2:172.21.15.132:3300/0 conn(0x7fc0500cc500 msgr2=0x7fc05015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2674903978 >> v2:172.21.15.132:3300/0 conn(0x7fc0500cc500 0x7fc05015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 >> v1:172.21.15.132:6789/0 conn(0x7fc0501591f0 legacy=0x7fc0500cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0500cea30 con 0x7fc0501595c0 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fc03c001470 con 0x7fc0501595c0 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 >> v1:172.21.15.46:6789/0 conn(0x7fc0501595c0 legacy=0x7fc0500af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040003bf0 0x7fc040003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 0x7fc040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0400051c0 0x7fc04000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500cff00 con 0x7fc040004810 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0400148c0 con 0x7fc040003bf0 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc040014a80 con 0x7fc0400051c0 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 0x7fc040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc054a8c700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040003bf0 0x7fc040003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.511+0000 7fc05528d700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0400051c0 0x7fc04000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 -- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0400051c0 msgr2=0x7fc04000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0400051c0 0x7fc04000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 -- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040003bf0 msgr2=0x7fc040003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040003bf0 0x7fc040003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 -- 172.21.15.46:0/2674903978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040003570 con 0x7fc040004810 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc05528d700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0400051c0 0x7fc04000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc05528d700 1 -- 172.21.15.46:0/2674903978 reap_dead start 2023-10-06T11:37:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 0x7fc040004c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c2d0 tx=0x7fc03c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e477fa056fc73cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00cbc0 con 0x7fc040004810 2023-10-06T11:37:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c00cd20 con 0x7fc040004810 2023-10-06T11:37:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2674903978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c005b70 con 0x7fc040004810 2023-10-06T11:37:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 -- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 msgr2=0x7fc040004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 0x7fc040004c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c2d0 tx=0x7fc03c001e10 comp rx=0 tx=0).stop 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 -- 172.21.15.46:0/2674903978 shutdown_connections 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 --2- 172.21.15.46:0/2674903978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc040004810 0x7fc040004c80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 -- 172.21.15.46:0/2674903978 >> 172.21.15.46:0/2674903978 conn(0x7fc05006ef60 msgr2=0x7fc0500d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 -- 172.21.15.46:0/2674903978 shutdown_connections 2023-10-06T11:37:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.515+0000 7fc055a8e700 1 -- 172.21.15.46:0/2674903978 wait complete. 2023-10-06T11:37:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 Processor -- start 2023-10-06T11:37:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 -- start start 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040004810 0x7fc050150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 0x7fc0501599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500b18b0 con 0x7fc05014dbc0 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05015c090 con 0x7fc040004810 2023-10-06T11:37:31.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc055a8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05015a660 con 0x7fc0500b1410 2023-10-06T11:37:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47824/0 (socket says 172.21.15.46:47824) 2023-10-06T11:37:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc04ffff700 1 -- 172.21.15.46:0/102608738 learned_addr learned my addr 172.21.15.46:0/102608738 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc05528d700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 0x7fc0501599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.519+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040004810 0x7fc050150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040004810 msgr2=0x7fc050150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040004810 0x7fc050150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 msgr2=0x7fc0500b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05015a940 con 0x7fc0500b1410 2023-10-06T11:37:31.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc04ffff700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc05528d700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 0x7fc0501599a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc050089290 tx=0x7fc04800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04801b070 con 0x7fc0500b1410 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800f040 con 0x7fc0500b1410 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800e050 con 0x7fc0500b1410 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05015c330 con 0x7fc0500b1410 2023-10-06T11:37:31.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc050150650 con 0x7fc0500b1410 2023-10-06T11:37:31.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.523+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05015ab30 con 0x7fc0500b1410 2023-10-06T11:37:31.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.527+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 64) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fc048003020 con 0x7fc0500b1410 2023-10-06T11:37:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.527+0000 7fc04cff9700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.527+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6530+0+0 (secure 0 0 0) 0x7fc04807d0d0 con 0x7fc0500b1410 2023-10-06T11:37:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.527+0000 7fc054a8c700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.527+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:37:31.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.535+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc04804d200 con 0x7fc0500b1410 2023-10-06T11:37:31.669 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.664+0000 7f298ff25000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:37:31.739 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.736+0000 7f298ff25000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.731+0000 7fc054a8c700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:31.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.731+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:37:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:31.735+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc050064590 con 0x7fc0340607c0 2023-10-06T11:37:31.781 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:31.767+0000 7f635c32a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:37:31.856 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:31.843+0000 7f635c32a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:31.942 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:31.940+0000 7f298ff25000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:37:32.017 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:32.012+0000 7f298ff25000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:37:32.084 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:32.071+0000 7f635c32a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:37:32.101 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:32.096+0000 7f298ff25000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:37:32.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:32.135+0000 7fc054a8c700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:32.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:32.135+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:37:32.175 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:32.163+0000 7f635c32a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:37:32.266 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:32.255+0000 7f635c32a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:37:32.466 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:32.460+0000 7f298ff25000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:37:32.623 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:32.620+0000 7f298ff25000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:37:32.681 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:32.667+0000 7f635c32a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:37:32.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:32.839+0000 7f635c32a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:37:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:32.939+0000 7fc054a8c700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:32.939+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:37:32.959 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:32.956+0000 7f298ff25000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:37:33.161 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.156+0000 7f298ff25000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:33.232 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.219+0000 7f635c32a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:37:33.249 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.244+0000 7f298ff25000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:37:33.325 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.320+0000 7f298ff25000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:37:33.400 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.396+0000 7f298ff25000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:37:33.458 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.447+0000 7f635c32a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:37:33.545 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.535+0000 7f635c32a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:37:33.616 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.612+0000 7f298ff25000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:33.623 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.611+0000 7f635c32a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:37:33.689 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.684+0000 7f298ff25000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:37:33.700 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.687+0000 7f635c32a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:37:33.773 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.768+0000 7f298ff25000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:37:33.919 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.907+0000 7f635c32a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:37:33.961 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:33.956+0000 7f298ff25000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:37:33.997 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:33.983+0000 7f635c32a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:37:34.042 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:34.040+0000 7f298ff25000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:37:34.087 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:34.075+0000 7f635c32a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:37:34.281 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:34.271+0000 7f635c32a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:37:34.367 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:34.355+0000 7f635c32a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:37:34.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:34.543+0000 7fc054a8c700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T11:37:34.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:34.543+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T11:37:34.737 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T11:37:34.732+0000 7f298ff25000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:37:34.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:34.795+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mgrmap(e 65) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fc0480560f0 con 0x7fc0500b1410 2023-10-06T11:37:35.101 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.087+0000 7f635c32a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:37:35.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:35.111+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 8 ==== mgrmap(e 66) v1 ==== 79687+0+0 (secure 0 0 0) 0x7fc04804d5d0 con 0x7fc0500b1410 2023-10-06T11:37:35.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:35.115+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:37:35.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:35.115+0000 7fc04cff9700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:35.301 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631e9f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.301 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631e9f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.302 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631e9f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.302 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631e9f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.302 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631e9f6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.303 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631b1ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.303 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631b1ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.303 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631b1ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.303 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631b1ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:35.303 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:37:35.291+0000 7f631b1ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:37:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.123+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mon.2 v2:172.21.15.132:3301/0 9 ==== mgrmap(e 67) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fc04805c210 con 0x7fc0500b1410 2023-10-06T11:37:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.123+0000 7fc04cff9700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.123+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc050064590 con 0x7fc0340607c0 2023-10-06T11:37:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.127+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.131+0000 7fc054a8c700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc044000c00 tx=0x7fc044007d30 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.239+0000 7fc04cff9700 1 -- 172.21.15.46:0/102608738 <== mgr.8514 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc050064590 con 0x7fc0340607c0 2023-10-06T11:37:36.252 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1 2023-10-06T11:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 msgr2=0x7fc034062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc044000c00 tx=0x7fc044007d30 comp rx=0 tx=0).stop 2023-10-06T11:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 msgr2=0x7fc0501599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 0x7fc0501599a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc050089290 tx=0x7fc04800b040 comp rx=0 tx=0).stop 2023-10-06T11:37:36.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 shutdown_connections 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc0340607c0 0x7fc034062bf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0500b1410 0x7fc0501599a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc05014dbc0 0x7fc0500b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 --2- 172.21.15.46:0/102608738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc040004810 0x7fc050150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 >> 172.21.15.46:0/102608738 conn(0x7fc05006ef60 msgr2=0x7fc0500b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 shutdown_connections 2023-10-06T11:37:36.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.243+0000 7fc055a8e700 1 -- 172.21.15.46:0/102608738 wait complete. 2023-10-06T11:37:36.279 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-06T11:37:36.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-10-06T11:37:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.447+0000 7f8a39ef0700 1 Processor -- start 2023-10-06T11:37:36.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.447+0000 7f8a39ef0700 1 -- start start 2023-10-06T11:37:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.447+0000 7f8a39ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a34159050 0x7f8a34159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a340b5410 0x7f8a340b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a340b4f80 con 0x7f8a340d0aa0 2023-10-06T11:37:36.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a340b40c0 con 0x7f8a340b5410 2023-10-06T11:37:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340b4370 con 0x7f8a34159050 2023-10-06T11:37:36.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a340b5410 0x7f8a340b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a340b5410 0x7f8a340b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55794/0 (socket says 172.21.15.46:55794) 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 -- 172.21.15.46:0/1760356767 learned_addr learned my addr 172.21.15.46:0/1760356767 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a337fe700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a34159050 0x7f8a34159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 -- 172.21.15.46:0/1760356767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a34159050 msgr2=0x7f8a34159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a34159050 0x7f8a34159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 -- 172.21.15.46:0/1760356767 >> v1:172.21.15.132:6789/0 conn(0x7f8a340d0aa0 legacy=0x7f8a340b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 -- 172.21.15.46:0/1760356767 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a340b4650 con 0x7f8a340b5410 2023-10-06T11:37:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a33fff700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a340b5410 0x7f8a340b1fc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009f20 tx=0x7f8a2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c32cab9446cfbb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a32ffd700 1 -- 172.21.15.46:0/1760356767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00e070 con 0x7f8a340b5410 2023-10-06T11:37:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a32ffd700 1 -- 172.21.15.46:0/1760356767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c00ba80 con 0x7f8a340b5410 2023-10-06T11:37:36.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a32ffd700 1 -- 172.21.15.46:0/1760356767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0092e0 con 0x7f8a340b5410 2023-10-06T11:37:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1760356767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a340b5410 msgr2=0x7f8a340b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.451+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a340b5410 0x7f8a340b1fc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009f20 tx=0x7f8a2c009840 comp rx=0 tx=0).stop 2023-10-06T11:37:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1760356767 shutdown_connections 2023-10-06T11:37:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a340b5410 0x7f8a340b1fc0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1760356767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a34159050 0x7f8a34159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1760356767 >> 172.21.15.46:0/1760356767 conn(0x7f8a3406e780 msgr2=0x7f8a3406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1760356767 shutdown_connections 2023-10-06T11:37:36.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1760356767 wait complete. 2023-10-06T11:37:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 Processor -- start 2023-10-06T11:37:36.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- start start 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a340b5410 0x7f8a3415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a340d0aa0 0x7f8a340c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340cf000 con 0x7f8a34159050 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340ce480 con 0x7f8a340d0aa0 2023-10-06T11:37:36.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.455+0000 7f8a39ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a340ce760 con 0x7f8a340b5410 2023-10-06T11:37:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55798/0 (socket says 172.21.15.46:55798) 2023-10-06T11:37:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 -- 172.21.15.46:0/1804801249 learned_addr learned my addr 172.21.15.46:0/1804801249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a337fe700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a340b5410 0x7f8a3415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a2bfff700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a340d0aa0 0x7f8a340c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 -- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a340b5410 msgr2=0x7f8a3415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a340b5410 0x7f8a3415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 -- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a340d0aa0 msgr2=0x7f8a340c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a340d0aa0 0x7f8a340c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 -- 172.21.15.46:0/1804801249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c009ae0 con 0x7f8a34159050 2023-10-06T11:37:36.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a33fff700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00b6c0 tx=0x7f8a2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0032f0 con 0x7f8a34159050 2023-10-06T11:37:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c008d20 con 0x7f8a34159050 2023-10-06T11:37:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c026670 con 0x7f8a34159050 2023-10-06T11:37:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a340cea40 con 0x7f8a34159050 2023-10-06T11:37:36.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.459+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3415a8e0 con 0x7f8a34159050 2023-10-06T11:37:36.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.463+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3415b5b0 con 0x7f8a34159050 2023-10-06T11:37:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.463+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 67) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f8a2c003040 con 0x7f8a34159050 2023-10-06T11:37:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.463+0000 7f8a2b7fe700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 0x7f8a140628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.463+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6874+0+0 (secure 0 0 0) 0x7f8a2c07e380 con 0x7f8a34159050 2023-10-06T11:37:36.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.463+0000 7f8a337fe700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 0x7f8a140628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.467+0000 7f8a337fe700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 0x7f8a140628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006fd0 tx=0x7f8a24008040 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:36.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.471+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a2c04e6c0 con 0x7f8a34159050 2023-10-06T11:37:36.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.667+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a3404b5c0 con 0x7f8a140603f0 2023-10-06T11:37:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.779+0000 7f8a2b7fe700 1 -- 172.21.15.46:0/1804801249 <== mgr.8514 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f8a3404b5c0 con 0x7f8a140603f0 2023-10-06T11:37:36.792 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path /snap_test_dir1 2023-10-06T11:37:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 msgr2=0x7f8a140628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 0x7f8a140628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006fd0 tx=0x7f8a24008040 comp rx=0 tx=0).stop 2023-10-06T11:37:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 msgr2=0x7f8a341532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00b6c0 tx=0x7f8a2c018000 comp rx=0 tx=0).stop 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 shutdown_connections 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8a140603f0 0x7f8a140628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a34159050 0x7f8a341532a0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a340d0aa0 0x7f8a340c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 --2- 172.21.15.46:0/1804801249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a340b5410 0x7f8a3415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 >> 172.21.15.46:0/1804801249 conn(0x7f8a3406e780 msgr2=0x7f8a341584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 shutdown_connections 2023-10-06T11:37:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.783+0000 7f8a39ef0700 1 -- 172.21.15.46:0/1804801249 wait complete. 2023-10-06T11:37:36.818 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-06T11:37:36.819 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-06T11:37:36.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-10-06T11:37:36.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.979+0000 7f3332ce3700 1 Processor -- start 2023-10-06T11:37:36.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.979+0000 7f3332ce3700 1 -- start start 2023-10-06T11:37:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3332ce3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f332c069770 0x7f332c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3332ce3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0b5eb0 con 0x7f332c158de0 2023-10-06T11:37:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3332ce3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0cef30 con 0x7f332c1591b0 2023-10-06T11:37:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3332ce3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c0cf4f0 con 0x7f332c069770 2023-10-06T11:37:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f332c069770 0x7f332c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f332c069770 0x7f332c0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55800/0 (socket says 172.21.15.46:55800) 2023-10-06T11:37:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 -- 172.21.15.46:0/3216913569 learned_addr learned my addr 172.21.15.46:0/3216913569 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3331ce1700 1 --1- 172.21.15.46:0/3216913569 >> v1:172.21.15.46:6789/0 conn(0x7f332c158de0 0x7f332c069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43530/0 (socket says 172.21.15.46:43530) 2023-10-06T11:37:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 -- 172.21.15.46:0/3216913569 >> v1:172.21.15.132:6789/0 conn(0x7f332c1591b0 legacy=0x7f332c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 -- 172.21.15.46:0/3216913569 >> v1:172.21.15.46:6789/0 conn(0x7f332c158de0 legacy=0x7f332c069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f3330cdf700 1 -- 172.21.15.46:0/3216913569 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621622873 0 0) 0x7f3328001110 con 0x7f332c158de0 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 -- 172.21.15.46:0/3216913569 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0ccac0 con 0x7f332c069770 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.983+0000 7f33314e0700 1 --2- 172.21.15.46:0/3216913569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0b5970 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099b0 tx=0x7f331c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d8437fe70e5902 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3330cdf700 1 -- 172.21.15.46:0/3216913569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c00b860 con 0x7f332c069770 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3330cdf700 1 -- 172.21.15.46:0/3216913569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c00b9c0 con 0x7f332c069770 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3330cdf700 1 -- 172.21.15.46:0/3216913569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c00bd60 con 0x7f332c069770 2023-10-06T11:37:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 -- 172.21.15.46:0/3216913569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 msgr2=0x7f332c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 --2- 172.21.15.46:0/3216913569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0b5970 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099b0 tx=0x7f331c009250 comp rx=0 tx=0).stop 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 -- 172.21.15.46:0/3216913569 shutdown_connections 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 --2- 172.21.15.46:0/3216913569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0b5970 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 -- 172.21.15.46:0/3216913569 >> 172.21.15.46:0/3216913569 conn(0x7f332c068390 msgr2=0x7f332c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 -- 172.21.15.46:0/3216913569 shutdown_connections 2023-10-06T11:37:37.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.987+0000 7f3332ce3700 1 -- 172.21.15.46:0/3216913569 wait complete. 2023-10-06T11:37:37.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 Processor -- start 2023-10-06T11:37:37.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 -- start start 2023-10-06T11:37:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332c158de0 0x7f332c144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 0x7f332c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c15aaa0 con 0x7f332c069770 2023-10-06T11:37:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c144fe0 con 0x7f332c158de0 2023-10-06T11:37:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3332ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c159e90 con 0x7f332c1591b0 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3331ce1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3331ce1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55802/0 (socket says 172.21.15.46:55802) 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f3331ce1700 1 -- 172.21.15.46:0/2671871929 learned_addr learned my addr 172.21.15.46:0/2671871929 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f33314e0700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332c158de0 0x7f332c144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.991+0000 7f33324e2700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 0x7f332c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 -- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 msgr2=0x7f332c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:37.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 0x7f332c159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 -- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332c158de0 msgr2=0x7f332c144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332c158de0 0x7f332c144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 -- 172.21.15.46:0/2671871929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0094f0 con 0x7f332c069770 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f33324e2700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 0x7f332c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3331ce1700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f332800c810 tx=0x7f3328007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328005a70 con 0x7f332c069770 2023-10-06T11:37:37.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328005bf0 con 0x7f332c069770 2023-10-06T11:37:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c15a170 con 0x7f332c069770 2023-10-06T11:37:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33280070d0 con 0x7f332c069770 2023-10-06T11:37:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c15a730 con 0x7f332c069770 2023-10-06T11:37:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.995+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c064590 con 0x7f332c069770 2023-10-06T11:37:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.999+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 67) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f3328007480 con 0x7f332c069770 2023-10-06T11:37:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.999+0000 7f3321ffb700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 0x7f33100628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:37:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.999+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6874+0+0 (secure 0 0 0) 0x7f332807bd30 con 0x7f332c069770 2023-10-06T11:37:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:36.999+0000 7f33314e0700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 0x7f33100628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:37:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.003+0000 7f33314e0700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 0x7f33100628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099b0 tx=0x7f331c017000 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:37:37.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.007+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f332804c070 con 0x7f332c069770 2023-10-06T11:37:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.135+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 68) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f332804c440 con 0x7f332c069770 2023-10-06T11:37:37.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.207+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f332c15abe0 con 0x7f33100603f0 2023-10-06T11:37:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.223+0000 7f3321ffb700 1 -- 172.21.15.46:0/2671871929 <== mgr.8514 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f331c020040 con 0x7f33100603f0 2023-10-06T11:37:37.236 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T11:37:37.236 INFO:teuthology.orchestra.run.smithi046.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-06T11:37:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 msgr2=0x7f33100628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 0x7f33100628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f331c0099b0 tx=0x7f331c017000 comp rx=0 tx=0).stop 2023-10-06T11:37:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 msgr2=0x7f332c0c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:37:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f332800c810 tx=0x7f3328007c10 comp rx=0 tx=0).stop 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 shutdown_connections 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33100603f0 0x7f33100628a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f332c1591b0 0x7f332c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f332c158de0 0x7f332c144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 --2- 172.21.15.46:0/2671871929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f332c069770 0x7f332c0c51c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:37:37.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 >> 172.21.15.46:0/2671871929 conn(0x7f332c068390 msgr2=0x7f332c0b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:37:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 shutdown_connections 2023-10-06T11:37:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T11:37:37.227+0000 7f3332ce3700 1 -- 172.21.15.46:0/2671871929 wait complete. 2023-10-06T11:37:37.260 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-06T11:37:37.260 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-10-06T11:37:48.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:37:48.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:38:00.814 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:38:00.803+0000 7f630e9d6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:38:00.814 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:38:00.803+0000 7f630e9d6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:38:00.814 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:38:00.803+0000 7f630e9d6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:38:00.814 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:38:00.803+0000 7f630e9d6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:38:00.814 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T11:38:00.803+0000 7f630e9d6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:38:18.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:38:18.250 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:38:48.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:38:48.331 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:39:18.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:39:18.422 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:39:48.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:39:48.501 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:18.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:18.591 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:48.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:48.689 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:18.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:18.786 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:48.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:48.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:18.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:18.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:49.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:49.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:19.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:19.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:49.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:49.200 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:44:19.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:44:19.290 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:44:49.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:44:49.368 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:45:19.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:45:19.453 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:45:49.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:45:49.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:46:19.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:46:19.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:46:49.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:46:49.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:47:19.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:47:19.808 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:47:49.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:47:49.923 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:48:19.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:48:20.001 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:48:50.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:48:50.085 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:49:20.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:49:20.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:49:50.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:49:50.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:50:20.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:50:20.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:50:50.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:50:50.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:51:20.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:51:20.557 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:51:50.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:51:50.703 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:52:20.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:52:20.851 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:52:50.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:52:50.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:53:21.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:53:21.061 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:53:51.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:53:51.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:54:21.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:54:21.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:54:51.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:54:51.537 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:55:21.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:55:21.701 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:55:51.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:55:51.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:56:21.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:56:21.884 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:56:51.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:56:51.961 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:57:22.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:57:22.041 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:57:52.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:57:52.155 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:58:22.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:58:22.327 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:58:52.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:58:52.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:59:22.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:59:22.614 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:59:52.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:59:52.766 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:00:22.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:00:22.859 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:00:52.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:00:52.957 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:01:23.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:01:23.093 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:01:53.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:01:53.168 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:02:23.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:02:23.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:02:53.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:02:53.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:03:23.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:03:23.594 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:03:53.751 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:03:53.760 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:04:23.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:04:23.889 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:04:54.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:04:54.033 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:05:24.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:05:24.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:05:54.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:05:54.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:06:24.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:06:24.331 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:06:54.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:06:54.488 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:07:24.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:07:24.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:07:54.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:07:54.713 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:08:24.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:08:24.862 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:08:54.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:08:54.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:09:25.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:09:25.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:09:55.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:09:55.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:10:25.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:10:25.335 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:10:55.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:10:55.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:11:25.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:11:25.606 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:11:55.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:11:55.711 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:12:25.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:12:25.864 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:12:56.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:12:56.015 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:13:26.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:13:26.087 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:13:56.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:13:56.294 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:14:26.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:14:26.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:14:56.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:14:56.472 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:15:26.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:15:26.551 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:15:56.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:15:56.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:16:26.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:16:26.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:16:56.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:16:56.869 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:17:27.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:17:27.041 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:17:57.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:17:57.189 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:18:27.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:18:27.327 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:18:57.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:18:57.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:19:27.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:19:27.673 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:19:57.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:19:57.752 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:20:27.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:20:27.907 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:20:57.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:20:57.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:21:28.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:21:28.091 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:21:58.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:21:58.249 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:22:28.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:22:28.346 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:22:58.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:22:58.476 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:23:28.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:23:28.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:23:58.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:23:58.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:24:29.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:24:29.053 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:24:59.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:24:59.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:25:29.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:25:29.372 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:25:59.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:25:59.471 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:26:29.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:26:29.683 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:26:59.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:26:59.839 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:27:29.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:27:29.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:28:00.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:28:00.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:28:30.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:28:30.191 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:29:00.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:29:00.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:29:30.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:29:30.446 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:30:00.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:30:00.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:30:30.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:30:30.818 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:31:01.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:31:01.095 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:31:31.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:31:31.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:32:01.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:32:01.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:32:31.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:32:31.493 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:33:01.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:33:01.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:33:16.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:16.262 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:33:16.292 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_39_00_UTC 2023-10-06T12:33:16.292 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_40_00_UTC 2023-10-06T12:33:16.292 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_41_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_42_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_43_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_44_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_45_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_46_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_47_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_48_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_49_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_50_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_51_00_UTC 2023-10-06T12:33:16.293 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_52_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_53_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_54_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_55_01_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_56_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_57_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_58_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_59_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_00_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_01_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_02_00_UTC 2023-10-06T12:33:16.294 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_03_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_04_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_05_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_06_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_07_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_08_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_09_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_10_01_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_11_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_12_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_13_00_UTC 2023-10-06T12:33:16.295 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_14_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_15_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_16_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_17_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_18_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_19_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_20_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_21_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_22_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_23_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_24_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_25_00_UTC 2023-10-06T12:33:16.296 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_26_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_27_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_28_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_29_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_30_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_31_00_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_32_01_UTC 2023-10-06T12:33:16.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_33_00_UTC 2023-10-06T12:33:16.298 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-11_39_00_UTC', 'scheduled-2023-10-06-11_40_00_UTC', 'scheduled-2023-10-06-11_41_00_UTC', 'scheduled-2023-10-06-11_42_00_UTC', 'scheduled-2023-10-06-11_43_00_UTC', 'scheduled-2023-10-06-11_44_00_UTC', 'scheduled-2023-10-06-11_45_00_UTC', 'scheduled-2023-10-06-11_46_00_UTC', 'scheduled-2023-10-06-11_47_00_UTC', 'scheduled-2023-10-06-11_48_00_UTC', 'scheduled-2023-10-06-11_49_00_UTC', 'scheduled-2023-10-06-11_50_00_UTC', 'scheduled-2023-10-06-11_51_00_UTC', 'scheduled-2023-10-06-11_52_00_UTC', 'scheduled-2023-10-06-11_53_00_UTC', 'scheduled-2023-10-06-11_54_00_UTC', 'scheduled-2023-10-06-11_55_01_UTC', 'scheduled-2023-10-06-11_56_00_UTC', 'scheduled-2023-10-06-11_57_00_UTC', 'scheduled-2023-10-06-11_58_00_UTC', 'scheduled-2023-10-06-11_59_00_UTC', 'scheduled-2023-10-06-12_00_00_UTC', 'scheduled-2023-10-06-12_01_00_UTC', 'scheduled-2023-10-06-12_02_00_UTC', 'scheduled-2023-10-06-12_03_00_UTC', 'scheduled-2023-10-06-12_04_00_UTC', 'scheduled-2023-10-06-12_05_00_UTC', 'scheduled-2023-10-06-12_06_00_UTC', 'scheduled-2023-10-06-12_07_00_UTC', 'scheduled-2023-10-06-12_08_00_UTC', 'scheduled-2023-10-06-12_09_00_UTC', 'scheduled-2023-10-06-12_10_01_UTC', 'scheduled-2023-10-06-12_11_00_UTC', 'scheduled-2023-10-06-12_12_00_UTC', 'scheduled-2023-10-06-12_13_00_UTC', 'scheduled-2023-10-06-12_14_00_UTC', 'scheduled-2023-10-06-12_15_00_UTC', 'scheduled-2023-10-06-12_16_00_UTC', 'scheduled-2023-10-06-12_17_00_UTC', 'scheduled-2023-10-06-12_18_00_UTC', 'scheduled-2023-10-06-12_19_00_UTC', 'scheduled-2023-10-06-12_20_00_UTC', 'scheduled-2023-10-06-12_21_00_UTC', 'scheduled-2023-10-06-12_22_00_UTC', 'scheduled-2023-10-06-12_23_00_UTC', 'scheduled-2023-10-06-12_24_00_UTC', 'scheduled-2023-10-06-12_25_00_UTC', 'scheduled-2023-10-06-12_26_00_UTC', 'scheduled-2023-10-06-12_27_00_UTC', 'scheduled-2023-10-06-12_28_00_UTC', 'scheduled-2023-10-06-12_29_00_UTC', 'scheduled-2023-10-06-12_30_00_UTC', 'scheduled-2023-10-06-12_31_00_UTC', 'scheduled-2023-10-06-12_32_01_UTC', 'scheduled-2023-10-06-12_33_00_UTC'] 2023-10-06T12:33:16.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-10-06T12:33:16.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.504+0000 7fd799785700 1 Processor -- start 2023-10-06T12:33:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.504+0000 7fd799785700 1 -- start start 2023-10-06T12:33:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd799785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd799785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd79406c9e0 0x7fd794162c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd799785700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7940cd510 con 0x7fd794069e60 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd799785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd794069190 con 0x7fd79406c9e0 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd799785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940cdab0 con 0x7fd794158fd0 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd792ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd792ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:49646/0 (socket says 172.21.15.46:49646) 2023-10-06T12:33:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd792ffd700 1 -- 172.21.15.46:0/3505620389 learned_addr learned my addr 172.21.15.46:0/3505620389 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.508+0000 7fd7937fe700 1 --2- 172.21.15.46:0/3505620389 >> v2:172.21.15.132:3300/0 conn(0x7fd79406c9e0 0x7fd794162c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd792ffd700 1 -- 172.21.15.46:0/3505620389 >> v2:172.21.15.132:3300/0 conn(0x7fd79406c9e0 msgr2=0x7fd794162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd792ffd700 1 --2- 172.21.15.46:0/3505620389 >> v2:172.21.15.132:3300/0 conn(0x7fd79406c9e0 0x7fd794162c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd791ffb700 1 -- 172.21.15.46:0/3505620389 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205391177 0 0) 0x7fd7940cd510 con 0x7fd794069e60 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd792ffd700 1 -- 172.21.15.46:0/3505620389 >> v1:172.21.15.132:6789/0 conn(0x7fd794069e60 legacy=0x7fd79406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd792ffd700 1 -- 172.21.15.46:0/3505620389 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794068c10 con 0x7fd794158fd0 2023-10-06T12:33:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd792ffd700 1 --2- 172.21.15.46:0/3505620389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd7840099b0 tx=0x7fd784009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721e2b71ecf5a241 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd791ffb700 1 -- 172.21.15.46:0/3505620389 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784009ea0 con 0x7fd794158fd0 2023-10-06T12:33:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd791ffb700 1 -- 172.21.15.46:0/3505620389 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400b860 con 0x7fd794158fd0 2023-10-06T12:33:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd791ffb700 1 -- 172.21.15.46:0/3505620389 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400bc00 con 0x7fd794158fd0 2023-10-06T12:33:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 -- 172.21.15.46:0/3505620389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 msgr2=0x7fd7940cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 --2- 172.21.15.46:0/3505620389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd7840099b0 tx=0x7fd784009250 comp rx=0 tx=0).stop 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 -- 172.21.15.46:0/3505620389 shutdown_connections 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 --2- 172.21.15.46:0/3505620389 >> v2:172.21.15.132:3300/0 conn(0x7fd79406c9e0 0x7fd794162c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 --2- 172.21.15.46:0/3505620389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd794158fd0 0x7fd7940cba20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 -- 172.21.15.46:0/3505620389 >> 172.21.15.46:0/3505620389 conn(0x7fd794067b20 msgr2=0x7fd79406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 -- 172.21.15.46:0/3505620389 shutdown_connections 2023-10-06T12:33:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.512+0000 7fd799785700 1 -- 172.21.15.46:0/3505620389 wait complete. 2023-10-06T12:33:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.516+0000 7fd799785700 1 Processor -- start 2023-10-06T12:33:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.516+0000 7fd799785700 1 -- start start 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79406c9e0 0x7fd794154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 0x7fd794170af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79415e650 con 0x7fd794069e60 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794171a20 con 0x7fd794158fd0 2023-10-06T12:33:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd799785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79415e320 con 0x7fd79406c9e0 2023-10-06T12:33:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44170/0 (socket says 172.21.15.46:44170) 2023-10-06T12:33:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 -- 172.21.15.46:0/980452673 learned_addr learned my addr 172.21.15.46:0/980452673 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd7927fc700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79406c9e0 0x7fd794154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd7937fe700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 0x7fd794170af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 -- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79406c9e0 msgr2=0x7fd794154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79406c9e0 0x7fd794154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 -- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 msgr2=0x7fd794170af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 0x7fd794170af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 -- 172.21.15.46:0/980452673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840094f0 con 0x7fd794069e60 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd7937fe700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 0x7fd794170af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.520+0000 7fd792ffd700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd7940cdbf0 tx=0x7fd784017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784000d10 con 0x7fd794069e60 2023-10-06T12:33:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400bbe0 con 0x7fd794069e60 2023-10-06T12:33:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78401a6c0 con 0x7fd794069e60 2023-10-06T12:33:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd794171460 con 0x7fd794069e60 2023-10-06T12:33:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794155230 con 0x7fd794069e60 2023-10-06T12:33:16.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79415e460 con 0x7fd794069e60 2023-10-06T12:33:16.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.524+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 70) v1 ==== 80090+0+0 (secure 0 0 0) 0x7fd78401aab0 con 0x7fd794069e60 2023-10-06T12:33:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.528+0000 7fd77affd700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 0x7fd780062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.528+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fd78407e7c0 con 0x7fd794069e60 2023-10-06T12:33:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.528+0000 7fd7927fc700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 0x7fd780062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.532+0000 7fd7927fc700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 0x7fd780062b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7940c9ce0 tx=0x7fd788007d30 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.536+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd79415e460 con 0x7fd794069e60 2023-10-06T12:33:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.728+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd79415e790 con 0x7fd7800606c0 2023-10-06T12:33:16.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.748+0000 7fd77affd700 1 -- 172.21.15.46:0/980452673 <== mgr.8514 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd7880027c0 con 0x7fd7800606c0 2023-10-06T12:33:16.755 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:16.755 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:37:36", "first": "2023-10-06T11:38:00", "last": "2023-10-06T12:33:00", "last_pruned": "2023-10-06T12:33:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-06T12:33:16.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 msgr2=0x7fd780062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 0x7fd780062b70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7940c9ce0 tx=0x7fd788007d30 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 msgr2=0x7fd79416d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd7940cdbf0 tx=0x7fd784017000 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 shutdown_connections 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd7800606c0 0x7fd780062b70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd794158fd0 0x7fd794170af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79406c9e0 0x7fd794154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 --2- 172.21.15.46:0/980452673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd794069e60 0x7fd79416d510 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 >> 172.21.15.46:0/980452673 conn(0x7fd794067b20 msgr2=0x7fd7940c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:16.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 shutdown_connections 2023-10-06T12:33:16.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.752+0000 7fd799785700 1 -- 172.21.15.46:0/980452673 wait complete. 2023-10-06T12:33:16.779 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T11:37:36", "first": "2023-10-06T11:38:00", "last": "2023-10-06T12:33:00", "last_pruned": "2023-10-06T12:33:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-06T12:33:16.780 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T11:37:36', 'first': '2023-10-06T11:38:00', 'last': '2023-10-06T12:33:00', 'last_pruned': '2023-10-06T12:33:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-06T12:33:16.780 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-06T12:33:16.780 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-06T12:33:16.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-10-06T12:33:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.948+0000 7fc149a0c700 1 Processor -- start 2023-10-06T12:33:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.948+0000 7fc149a0c700 1 -- start start 2023-10-06T12:33:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc149a0c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc14406cb80 0x7fc1440c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc149a0c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 0x7fc1440af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc149a0c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440aec20 con 0x7fc144155460 2023-10-06T12:33:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc149a0c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14406d8a0 con 0x7fc14406cb80 2023-10-06T12:33:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc149a0c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1440ae980 con 0x7fc144069130 2023-10-06T12:33:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc143fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc14406cb80 0x7fc1440c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc143fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc14406cb80 0x7fc1440c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44184/0 (socket says 172.21.15.46:44184) 2023-10-06T12:33:16.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc143fff700 1 -- 172.21.15.46:0/2242637140 learned_addr learned my addr 172.21.15.46:0/2242637140 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:16.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.952+0000 7fc14920b700 1 --2- 172.21.15.46:0/2242637140 >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 0x7fc1440af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc143fff700 1 -- 172.21.15.46:0/2242637140 >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 msgr2=0x7fc1440af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc143fff700 1 --2- 172.21.15.46:0/2242637140 >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 0x7fc1440af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc143fff700 1 -- 172.21.15.46:0/2242637140 >> v1:172.21.15.132:6789/0 conn(0x7fc144155460 legacy=0x7fc144068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc143fff700 1 -- 172.21.15.46:0/2242637140 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14406d000 con 0x7fc14406cb80 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc14920b700 1 --2- 172.21.15.46:0/2242637140 >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 0x7fc1440af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc143fff700 1 --2- 172.21.15.46:0/2242637140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc14406cb80 0x7fc1440c7b20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc1440aeb90 tx=0x7fc130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edf4b645021f286 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc1437fe700 1 -- 172.21.15.46:0/2242637140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000b860 con 0x7fc14406cb80 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc1437fe700 1 -- 172.21.15.46:0/2242637140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13000b9c0 con 0x7fc14406cb80 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc1437fe700 1 -- 172.21.15.46:0/2242637140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000bd60 con 0x7fc14406cb80 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 -- 172.21.15.46:0/2242637140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc14406cb80 msgr2=0x7fc1440c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 --2- 172.21.15.46:0/2242637140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc14406cb80 0x7fc1440c7b20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc1440aeb90 tx=0x7fc130009250 comp rx=0 tx=0).stop 2023-10-06T12:33:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 -- 172.21.15.46:0/2242637140 shutdown_connections 2023-10-06T12:33:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 --2- 172.21.15.46:0/2242637140 >> v2:172.21.15.132:3300/0 conn(0x7fc144069130 0x7fc1440af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 --2- 172.21.15.46:0/2242637140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc14406cb80 0x7fc1440c7b20 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 -- 172.21.15.46:0/2242637140 >> 172.21.15.46:0/2242637140 conn(0x7fc1440cbbb0 msgr2=0x7fc1440cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:16.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 -- 172.21.15.46:0/2242637140 shutdown_connections 2023-10-06T12:33:16.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.956+0000 7fc149a0c700 1 -- 172.21.15.46:0/2242637140 wait complete. 2023-10-06T12:33:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 Processor -- start 2023-10-06T12:33:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- start start 2023-10-06T12:33:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc144155460 0x7fc1440af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 0x7fc14414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440af7b0 con 0x7fc144069130 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440afa90 con 0x7fc14416e8b0 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440afd70 con 0x7fc144155460 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44188/0 (socket says 172.21.15.46:44188) 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 -- 172.21.15.46:0/338815643 learned_addr learned my addr 172.21.15.46:0/338815643 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc14920b700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 0x7fc14414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 -- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc144155460 msgr2=0x7fc1440af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc144155460 0x7fc1440af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 -- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 msgr2=0x7fc14414fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 0x7fc14414fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 -- 172.21.15.46:0/338815643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300094f0 con 0x7fc144069130 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc14920b700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 0x7fc14414fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc148a0a700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c4e0 tx=0x7fc13800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138011590 con 0x7fc144069130 2023-10-06T12:33:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1440b0050 con 0x7fc144069130 2023-10-06T12:33:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc138011710 con 0x7fc144069130 2023-10-06T12:33:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138011ad0 con 0x7fc144069130 2023-10-06T12:33:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.960+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14416ed20 con 0x7fc144069130 2023-10-06T12:33:16.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.964+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc144064590 con 0x7fc144069130 2023-10-06T12:33:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.964+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 70) v1 ==== 80090+0+0 (secure 0 0 0) 0x7fc13801b3b0 con 0x7fc144069130 2023-10-06T12:33:16.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.964+0000 7fc140ff9700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 0x7fc128062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.964+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fc13807c3b0 con 0x7fc144069130 2023-10-06T12:33:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.968+0000 7fc143fff700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 0x7fc128062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:16.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.968+0000 7fc143fff700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 0x7fc128062b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1440aeb90 tx=0x7fc130017000 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:16.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:16.972+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc13804c3d0 con 0x7fc144069130 2023-10-06T12:33:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.164+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1440afc00 con 0x7fc1280606c0 2023-10-06T12:33:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.296+0000 7fc140ff9700 1 -- 172.21.15.46:0/338815643 <== mgr.8514 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7fc1440afc00 con 0x7fc1280606c0 2023-10-06T12:33:17.306 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1 2023-10-06T12:33:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 msgr2=0x7fc128062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 0x7fc128062b70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1440aeb90 tx=0x7fc130017000 comp rx=0 tx=0).stop 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 msgr2=0x7fc14416b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c4e0 tx=0x7fc13800c8a0 comp rx=0 tx=0).stop 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 shutdown_connections 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1280606c0 0x7fc128062b70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc14416e8b0 0x7fc14414fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc144155460 0x7fc1440af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 --2- 172.21.15.46:0/338815643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc144069130 0x7fc14416b550 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.300+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 >> 172.21.15.46:0/338815643 conn(0x7fc1440cbbb0 msgr2=0x7fc14414f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.304+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 shutdown_connections 2023-10-06T12:33:17.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:17.304+0000 7fc149a0c700 1 -- 172.21.15.46:0/338815643 wait complete. 2023-10-06T12:33:17.332 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-06T12:33:17.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.333 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:33:17.359 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_39_00_UTC 2023-10-06T12:33:17.359 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_40_00_UTC 2023-10-06T12:33:17.359 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_41_00_UTC 2023-10-06T12:33:17.359 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_42_00_UTC 2023-10-06T12:33:17.360 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_43_00_UTC 2023-10-06T12:33:17.360 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_44_00_UTC 2023-10-06T12:33:17.360 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_45_00_UTC 2023-10-06T12:33:17.360 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_46_00_UTC 2023-10-06T12:33:17.360 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_47_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_48_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_49_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_50_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_51_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_52_00_UTC 2023-10-06T12:33:17.361 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_53_00_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_54_00_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_55_01_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_56_00_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_57_00_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_58_00_UTC 2023-10-06T12:33:17.362 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-11_59_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_00_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_01_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_02_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_03_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_04_00_UTC 2023-10-06T12:33:17.363 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_05_00_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_06_00_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_07_00_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_08_00_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_09_00_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_10_01_UTC 2023-10-06T12:33:17.364 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_11_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_12_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_13_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_14_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_15_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_16_00_UTC 2023-10-06T12:33:17.365 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_17_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_18_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_19_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_20_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_21_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_22_00_UTC 2023-10-06T12:33:17.366 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_23_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_24_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_25_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_26_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_27_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_28_00_UTC 2023-10-06T12:33:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_29_00_UTC 2023-10-06T12:33:17.368 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_30_00_UTC 2023-10-06T12:33:17.368 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_31_00_UTC 2023-10-06T12:33:17.368 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_32_01_UTC 2023-10-06T12:33:17.368 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_33_00_UTC 2023-10-06T12:33:17.369 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_39_00_UTC 2023-10-06T12:33:17.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.370 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_39_00_UTC) 2023-10-06T12:33:17.451 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_40_00_UTC 2023-10-06T12:33:17.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.451 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_40_00_UTC) 2023-10-06T12:33:17.525 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_41_00_UTC 2023-10-06T12:33:17.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.526 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_41_00_UTC) 2023-10-06T12:33:17.602 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_42_00_UTC 2023-10-06T12:33:17.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.602 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_42_00_UTC) 2023-10-06T12:33:17.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_43_00_UTC 2023-10-06T12:33:17.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.682 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_43_00_UTC) 2023-10-06T12:33:17.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_44_00_UTC 2023-10-06T12:33:17.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.761 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_44_00_UTC) 2023-10-06T12:33:17.843 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_45_00_UTC 2023-10-06T12:33:17.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.843 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_45_00_UTC) 2023-10-06T12:33:17.917 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_46_00_UTC 2023-10-06T12:33:17.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.918 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_46_00_UTC) 2023-10-06T12:33:17.997 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_47_00_UTC 2023-10-06T12:33:17.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:17.997 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_47_00_UTC) 2023-10-06T12:33:18.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_48_00_UTC 2023-10-06T12:33:18.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.078 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_48_00_UTC) 2023-10-06T12:33:18.157 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_49_00_UTC 2023-10-06T12:33:18.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.158 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_49_00_UTC) 2023-10-06T12:33:18.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_50_00_UTC 2023-10-06T12:33:18.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.241 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_50_00_UTC) 2023-10-06T12:33:18.316 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_51_00_UTC 2023-10-06T12:33:18.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.317 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_51_00_UTC) 2023-10-06T12:33:18.393 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_52_00_UTC 2023-10-06T12:33:18.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.393 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_52_00_UTC) 2023-10-06T12:33:18.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_53_00_UTC 2023-10-06T12:33:18.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.471 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_53_00_UTC) 2023-10-06T12:33:18.551 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_54_00_UTC 2023-10-06T12:33:18.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.551 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_54_00_UTC) 2023-10-06T12:33:18.626 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_55_01_UTC 2023-10-06T12:33:18.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.626 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_55_01_UTC) 2023-10-06T12:33:18.701 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_56_00_UTC 2023-10-06T12:33:18.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.701 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_56_00_UTC) 2023-10-06T12:33:18.775 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_57_00_UTC 2023-10-06T12:33:18.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.776 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_57_00_UTC) 2023-10-06T12:33:18.849 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_58_00_UTC 2023-10-06T12:33:18.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.849 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_58_00_UTC) 2023-10-06T12:33:18.927 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-11_59_00_UTC 2023-10-06T12:33:18.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:18.928 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-11_59_00_UTC) 2023-10-06T12:33:19.000 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_00_00_UTC 2023-10-06T12:33:19.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.001 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_00_00_UTC) 2023-10-06T12:33:19.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_01_00_UTC 2023-10-06T12:33:19.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.074 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_01_00_UTC) 2023-10-06T12:33:19.151 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_02_00_UTC 2023-10-06T12:33:19.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.151 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_02_00_UTC) 2023-10-06T12:33:19.226 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_03_00_UTC 2023-10-06T12:33:19.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.227 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_03_00_UTC) 2023-10-06T12:33:19.303 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_04_00_UTC 2023-10-06T12:33:19.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.304 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_04_00_UTC) 2023-10-06T12:33:19.382 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_05_00_UTC 2023-10-06T12:33:19.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.383 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_05_00_UTC) 2023-10-06T12:33:19.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_06_00_UTC 2023-10-06T12:33:19.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.458 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_06_00_UTC) 2023-10-06T12:33:19.531 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_07_00_UTC 2023-10-06T12:33:19.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.532 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_07_00_UTC) 2023-10-06T12:33:19.607 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_08_00_UTC 2023-10-06T12:33:19.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.607 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_08_00_UTC) 2023-10-06T12:33:19.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_09_00_UTC 2023-10-06T12:33:19.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.683 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_09_00_UTC) 2023-10-06T12:33:19.759 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_10_01_UTC 2023-10-06T12:33:19.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.759 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_10_01_UTC) 2023-10-06T12:33:19.841 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_11_00_UTC 2023-10-06T12:33:19.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.842 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_11_00_UTC) 2023-10-06T12:33:19.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_12_00_UTC 2023-10-06T12:33:19.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.918 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_12_00_UTC) 2023-10-06T12:33:19.996 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_13_00_UTC 2023-10-06T12:33:19.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:19.996 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_13_00_UTC) 2023-10-06T12:33:20.069 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_14_00_UTC 2023-10-06T12:33:20.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.069 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_14_00_UTC) 2023-10-06T12:33:20.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_15_00_UTC 2023-10-06T12:33:20.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.148 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_15_00_UTC) 2023-10-06T12:33:20.227 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_16_00_UTC 2023-10-06T12:33:20.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.227 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_16_00_UTC) 2023-10-06T12:33:20.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_17_00_UTC 2023-10-06T12:33:20.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.303 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_17_00_UTC) 2023-10-06T12:33:20.378 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_18_00_UTC 2023-10-06T12:33:20.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.378 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_18_00_UTC) 2023-10-06T12:33:20.454 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_19_00_UTC 2023-10-06T12:33:20.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.454 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_19_00_UTC) 2023-10-06T12:33:20.536 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_20_00_UTC 2023-10-06T12:33:20.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.536 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_20_00_UTC) 2023-10-06T12:33:20.609 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_21_00_UTC 2023-10-06T12:33:20.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.609 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_21_00_UTC) 2023-10-06T12:33:20.686 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_22_00_UTC 2023-10-06T12:33:20.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.686 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_22_00_UTC) 2023-10-06T12:33:20.765 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_23_00_UTC 2023-10-06T12:33:20.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.766 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_23_00_UTC) 2023-10-06T12:33:20.845 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_24_00_UTC 2023-10-06T12:33:20.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.846 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_24_00_UTC) 2023-10-06T12:33:20.928 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_25_00_UTC 2023-10-06T12:33:20.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:20.928 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_25_00_UTC) 2023-10-06T12:33:21.003 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_26_00_UTC 2023-10-06T12:33:21.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.003 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_26_00_UTC) 2023-10-06T12:33:21.077 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_27_00_UTC 2023-10-06T12:33:21.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.078 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_27_00_UTC) 2023-10-06T12:33:21.156 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_28_00_UTC 2023-10-06T12:33:21.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.156 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_28_00_UTC) 2023-10-06T12:33:21.227 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_29_00_UTC 2023-10-06T12:33:21.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.227 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_29_00_UTC) 2023-10-06T12:33:21.308 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_30_00_UTC 2023-10-06T12:33:21.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.308 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_30_00_UTC) 2023-10-06T12:33:21.385 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_31_00_UTC 2023-10-06T12:33:21.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.386 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_31_00_UTC) 2023-10-06T12:33:21.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_32_01_UTC 2023-10-06T12:33:21.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.468 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_32_01_UTC) 2023-10-06T12:33:21.540 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_33_00_UTC 2023-10-06T12:33:21.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.540 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_33_00_UTC) 2023-10-06T12:33:21.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:21.619 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T12:33:21.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T12:33:21.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.900+0000 7f77c7a2e700 1 Processor -- start 2023-10-06T12:33:21.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.900+0000 7f77c7a2e700 1 -- start start 2023-10-06T12:33:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.900+0000 7f77c7a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f77c0155890 0x7f77c00b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.900+0000 7f77c7a2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c00afb40 con 0x7f77c0068770 2023-10-06T12:33:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.900+0000 7f77c7a2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c00aca50 con 0x7f77c01554c0 2023-10-06T12:33:21.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c7a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c00cc370 con 0x7f77c0155890 2023-10-06T12:33:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c4fc9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f77c0068770 0x7f77c00ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44238/0 (socket says 172.21.15.46:44238) 2023-10-06T12:33:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c4fc9700 1 -- 172.21.15.46:0/2667603869 learned_addr learned my addr 172.21.15.46:0/2667603869 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:21.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c5fcb700 1 --2- 172.21.15.46:0/2667603869 >> v2:172.21.15.46:3300/0 conn(0x7f77c0155890 0x7f77c00b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821785447 0 0) 0x7f77c00afb40 con 0x7f77c0068770 2023-10-06T12:33:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77ac004680 con 0x7f77c0068770 2023-10-06T12:33:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092270913 0 0) 0x7f77c00aca50 con 0x7f77c01554c0 2023-10-06T12:33:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00afb40 con 0x7f77c01554c0 2023-10-06T12:33:21.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c5fcb700 1 -- 172.21.15.46:0/2667603869 >> v1:172.21.15.132:6789/0 conn(0x7f77c01554c0 legacy=0x7f77c0068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c5fcb700 1 -- 172.21.15.46:0/2667603869 >> v1:172.21.15.46:6789/0 conn(0x7f77c0068770 legacy=0x7f77c00ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c5fcb700 1 -- 172.21.15.46:0/2667603869 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00ac120 con 0x7f77c0155890 2023-10-06T12:33:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c5fcb700 1 --2- 172.21.15.46:0/2667603869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0155890 0x7f77c00b1ed0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e60 tx=0x7f77b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a264c00aff67d7f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:21.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b800bab0 con 0x7f77c0155890 2023-10-06T12:33:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b800bc10 con 0x7f77c0155890 2023-10-06T12:33:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77b7fff700 1 -- 172.21.15.46:0/2667603869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8011460 con 0x7f77c0155890 2023-10-06T12:33:21.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2667603869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0155890 msgr2=0x7f77c00b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2667603869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0155890 0x7f77c00b1ed0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e60 tx=0x7f77b8009780 comp rx=0 tx=0).stop 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2667603869 shutdown_connections 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.904+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2667603869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0155890 0x7f77c00b1ed0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.908+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2667603869 >> 172.21.15.46:0/2667603869 conn(0x7f77c006d490 msgr2=0x7f77c006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.908+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2667603869 shutdown_connections 2023-10-06T12:33:21.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.908+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2667603869 wait complete. 2023-10-06T12:33:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.908+0000 7f77c7a2e700 1 Processor -- start 2023-10-06T12:33:21.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 -- start start 2023-10-06T12:33:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 0x7f77c00c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c0155890 0x7f77c0154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:21.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00a7170 con 0x7f77c0068770 2023-10-06T12:33:21.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00ca180 con 0x7f77c01554c0 2023-10-06T12:33:21.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c7a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00a6b60 con 0x7f77c0155890 2023-10-06T12:33:21.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44206/0 (socket says 172.21.15.46:44206) 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 -- 172.21.15.46:0/2986785278 learned_addr learned my addr 172.21.15.46:0/2986785278 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c5fcb700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c0155890 0x7f77c0154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c4fc9700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 0x7f77c00c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 -- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c0155890 msgr2=0x7f77c0154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c0155890 0x7f77c0154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 -- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 msgr2=0x7f77c00c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 0x7f77c00c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 -- 172.21.15.46:0/2986785278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009a20 con 0x7f77c0068770 2023-10-06T12:33:21.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c4fc9700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 0x7f77c00c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.912+0000 7f77c57ca700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c7c0 tx=0x7f77bc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc010510 con 0x7f77c0068770 2023-10-06T12:33:21.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc010670 con 0x7f77c0068770 2023-10-06T12:33:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c01547c0 con 0x7f77c0068770 2023-10-06T12:33:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc010a10 con 0x7f77c0068770 2023-10-06T12:33:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c00a6e00 con 0x7f77c0068770 2023-10-06T12:33:21.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c006c110 con 0x7f77c0068770 2023-10-06T12:33:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.916+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 70) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f77bc01b3b0 con 0x7f77c0068770 2023-10-06T12:33:21.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.920+0000 7f77b57fa700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 0x7f77a4062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.920+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f77bc07c7c0 con 0x7f77c0068770 2023-10-06T12:33:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.920+0000 7f77c4fc9700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 0x7f77a4062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.920+0000 7f77c4fc9700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 0x7f77a4062ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77c00ae170 tx=0x7f77b0009040 comp rx=0 tx=0).ready entity=mgr.8514 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:21.928+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f77bc04c740 con 0x7f77c0068770 2023-10-06T12:33:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.136+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f77c00ac400 con 0x7f77c0068770 2023-10-06T12:33:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.208+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f77bc04cb10 con 0x7f77c0068770 2023-10-06T12:33:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.212+0000 7f77b57fa700 1 -- 172.21.15.46:0/2986785278 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v71) v1 ==== 93+0+0 (secure 0 0 0) 0x7f77bc04c740 con 0x7f77c0068770 2023-10-06T12:33:22.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 msgr2=0x7f77a4062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 0x7f77a4062ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77c00ae170 tx=0x7f77b0009040 comp rx=0 tx=0).stop 2023-10-06T12:33:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 msgr2=0x7f77c00a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c7c0 tx=0x7f77bc007e10 comp rx=0 tx=0).stop 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 shutdown_connections 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f77a40606f0 0x7f77a4062ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77c0155890 0x7f77c0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77c01554c0 0x7f77c00c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 --2- 172.21.15.46:0/2986785278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77c0068770 0x7f77c00a5ff0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 >> 172.21.15.46:0/2986785278 conn(0x7f77c006d490 msgr2=0x7f77c00cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:22.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 shutdown_connections 2023-10-06T12:33:22.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.216+0000 7f77c7a2e700 1 -- 172.21.15.46:0/2986785278 wait complete. 2023-10-06T12:33:22.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:33:22.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:33:22.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:33:22.368 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:33:22.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T12:33:22.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:22.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T12:33:22.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:33:22.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:22.514 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T12:33:22.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:22.583 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:22.576+0000 7f9aa1765000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:33:22.591 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:22.583+0000 7f525bf0e000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:33:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.832+0000 7f7a66ecd700 1 Processor -- start 2023-10-06T12:33:22.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 -- start start 2023-10-06T12:33:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a580360f0 0x7f7a580defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a58052a00 0x7f7a58052e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5804f470 con 0x7f7a580dec00 2023-10-06T12:33:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580063d0 con 0x7f7a58052a00 2023-10-06T12:33:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a66ecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a58054360 con 0x7f7a580360f0 2023-10-06T12:33:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a65c6b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a58052a00 0x7f7a58052e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a65c6b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a58052a00 0x7f7a58052e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44224/0 (socket says 172.21.15.46:44224) 2023-10-06T12:33:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a65c6b700 1 -- 172.21.15.46:0/2522552223 learned_addr learned my addr 172.21.15.46:0/2522552223 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.836+0000 7f7a5ffff700 1 --2- 172.21.15.46:0/2522552223 >> v2:172.21.15.132:3300/0 conn(0x7f7a580360f0 0x7f7a580defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a65c6b700 1 -- 172.21.15.46:0/2522552223 >> v2:172.21.15.132:3300/0 conn(0x7f7a580360f0 msgr2=0x7f7a580defd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/2522552223 >> v2:172.21.15.132:3300/0 conn(0x7f7a580360f0 0x7f7a580defd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a65c6b700 1 -- 172.21.15.46:0/2522552223 >> v1:172.21.15.46:6789/0 conn(0x7f7a580dec00 legacy=0x7f7a580359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a65c6b700 1 -- 172.21.15.46:0/2522552223 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5804f6c0 con 0x7f7a58052a00 2023-10-06T12:33:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/2522552223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a58052e10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7a6004c2a0 tx=0x7f7a60065a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b168825042e79ff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a5f7fe700 1 -- 172.21.15.46:0/2522552223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6004be60 con 0x7f7a58052a00 2023-10-06T12:33:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a5f7fe700 1 -- 172.21.15.46:0/2522552223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60063f90 con 0x7f7a58052a00 2023-10-06T12:33:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a5f7fe700 1 -- 172.21.15.46:0/2522552223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60064140 con 0x7f7a58052a00 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 -- 172.21.15.46:0/2522552223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 msgr2=0x7f7a58052e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/2522552223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a58052e10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7a6004c2a0 tx=0x7f7a60065a20 comp rx=0 tx=0).stop 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 -- 172.21.15.46:0/2522552223 shutdown_connections 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/2522552223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a58052e10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/2522552223 >> v2:172.21.15.132:3300/0 conn(0x7f7a580360f0 0x7f7a580defd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.840+0000 7f7a66ecd700 1 -- 172.21.15.46:0/2522552223 >> 172.21.15.46:0/2522552223 conn(0x7f7a58054e30 msgr2=0x7f7a58055230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:22.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.844+0000 7f7a66ecd700 1 -- 172.21.15.46:0/2522552223 shutdown_connections 2023-10-06T12:33:22.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.844+0000 7f7a66ecd700 1 -- 172.21.15.46:0/2522552223 wait complete. 2023-10-06T12:33:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.844+0000 7f7a66ecd700 1 Processor -- start 2023-10-06T12:33:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.844+0000 7f7a66ecd700 1 -- start start 2023-10-06T12:33:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 0x7f7a580515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a64c69700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a64c69700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:49686/0 (socket says 172.21.15.46:49686) 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44240/0 (socket says 172.21.15.46:44240) 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a5ffff700 1 -- 172.21.15.46:0/1888996356 learned_addr learned my addr 172.21.15.46:0/1888996356 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 0x7f7a580515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58049bd0 con 0x7f7a58052a00 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5804c420 con 0x7f7a580dec00 2023-10-06T12:33:22.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.848+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58051bd0 con 0x7f7a580360f0 2023-10-06T12:33:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 msgr2=0x7f7a5804bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 msgr2=0x7f7a580510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a60062b60 con 0x7f7a580dec00 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a5ffff700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a65c6b700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 0x7f7a580515e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7a6004ccb0 tx=0x7f7a6006b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a64c69700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60099070 con 0x7f7a580dec00 2023-10-06T12:33:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60094280 con 0x7f7a580dec00 2023-10-06T12:33:22.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60094b10 con 0x7f7a580dec00 2023-10-06T12:33:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a58049e20 con 0x7f7a580dec00 2023-10-06T12:33:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.852+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5804a3e0 con 0x7f7a580dec00 2023-10-06T12:33:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.856+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5804a600 con 0x7f7a580dec00 2023-10-06T12:33:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.856+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f7a60062d50 con 0x7f7a580dec00 2023-10-06T12:33:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.860+0000 7f7a5cff9700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 0x7f7a44062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.860+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f7a600f5100 con 0x7f7a580dec00 2023-10-06T12:33:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.860+0000 7f7a64c69700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 msgr2=0x7f7a44062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.860+0000 7f7a64c69700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 0x7f7a44062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:22.868+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7a600c4fc0 con 0x7f7a580dec00 2023-10-06T12:33:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.064+0000 7f7a64c69700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 msgr2=0x7f7a44062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.064+0000 7f7a64c69700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 0x7f7a44062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:33:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.144+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a58005530 con 0x7f7a580dec00 2023-10-06T12:33:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.148+0000 7f7a5cff9700 1 -- 172.21.15.46:0/1888996356 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f7a600c85e0 con 0x7f7a580dec00 2023-10-06T12:33:23.155 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:23.155 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T12:33:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 msgr2=0x7f7a44062bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 0x7f7a44062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 msgr2=0x7f7a580515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 0x7f7a580515e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7a6004ccb0 tx=0x7f7a6006b000 comp rx=0 tx=0).stop 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 shutdown_connections 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7a44060740 0x7f7a44062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a580dec00 0x7f7a580515e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58052a00 0x7f7a580510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 --2- 172.21.15.46:0/1888996356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a580360f0 0x7f7a5804bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 >> 172.21.15.46:0/1888996356 conn(0x7f7a58054e30 msgr2=0x7f7a58006950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 shutdown_connections 2023-10-06T12:33:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.152+0000 7f7a66ecd700 1 -- 172.21.15.46:0/1888996356 wait complete. 2023-10-06T12:33:23.161 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T12:33:23.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:23.249 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:23.243+0000 7f525bf0e000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:33:23.345 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:23.339+0000 7f525bf0e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:33:23.405 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:23.396+0000 7f9aa1765000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:33:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.408+0000 7fd6cd363700 1 Processor -- start 2023-10-06T12:33:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.408+0000 7fd6cd363700 1 -- start start 2023-10-06T12:33:23.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6cd363700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6c8159290 0x7fd6c80b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6cd363700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6cd363700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80ae340 con 0x7fd6c80b1130 2023-10-06T12:33:23.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6cd363700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80b15b0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6cd363700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80b09a0 con 0x7fd6c8159290 2023-10-06T12:33:23.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34200/0 (socket says 172.21.15.46:34200) 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 -- 172.21.15.46:0/4098131434 learned_addr learned my addr 172.21.15.46:0/4098131434 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c7fff700 1 --2- 172.21.15.46:0/4098131434 >> v2:172.21.15.132:3300/0 conn(0x7fd6c8159290 0x7fd6c80b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 -- 172.21.15.46:0/4098131434 >> v2:172.21.15.132:3300/0 conn(0x7fd6c8159290 msgr2=0x7fd6c80b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 --2- 172.21.15.46:0/4098131434 >> v2:172.21.15.132:3300/0 conn(0x7fd6c8159290 0x7fd6c80b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 -- 172.21.15.46:0/4098131434 >> v1:172.21.15.132:6789/0 conn(0x7fd6c80b1130 legacy=0x7fd6c80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 -- 172.21.15.46:0/4098131434 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8159660 con 0x7fd6c80d07b0 2023-10-06T12:33:23.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c77fe700 1 --2- 172.21.15.46:0/4098131434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8071530 tx=0x7fd6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e80a8bfe7b13ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c6ffd700 1 -- 172.21.15.46:0/4098131434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00b8f0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c6ffd700 1 -- 172.21.15.46:0/4098131434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc00ba50 con 0x7fd6c80d07b0 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.412+0000 7fd6c6ffd700 1 -- 172.21.15.46:0/4098131434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00bdf0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 -- 172.21.15.46:0/4098131434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80d07b0 msgr2=0x7fd6c80b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 --2- 172.21.15.46:0/4098131434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8071530 tx=0x7fd6bc009250 comp rx=0 tx=0).stop 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 -- 172.21.15.46:0/4098131434 shutdown_connections 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 --2- 172.21.15.46:0/4098131434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c80d07b0 0x7fd6c80b0bf0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 --2- 172.21.15.46:0/4098131434 >> v2:172.21.15.132:3300/0 conn(0x7fd6c8159290 0x7fd6c80b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 -- 172.21.15.46:0/4098131434 >> 172.21.15.46:0/4098131434 conn(0x7fd6c806ef60 msgr2=0x7fd6c80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:23.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 -- 172.21.15.46:0/4098131434 shutdown_connections 2023-10-06T12:33:23.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.416+0000 7fd6cd363700 1 -- 172.21.15.46:0/4098131434 wait complete. 2023-10-06T12:33:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 Processor -- start 2023-10-06T12:33:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 -- start start 2023-10-06T12:33:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 0x7fd6c8154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 0x7fd6c813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 0x7fd6c813e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8155100 con 0x7fd6c8159290 2023-10-06T12:33:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c813d590 con 0x7fd6c80d07b0 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6cd363700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c813d870 con 0x7fd6c80b1130 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6ccb62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 0x7fd6c813e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 0x7fd6c813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6ccb62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 0x7fd6c813e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34216/0 (socket says 172.21.15.46:34216) 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6ccb62700 1 -- 172.21.15.46:0/3355963459 learned_addr learned my addr 172.21.15.46:0/3355963459 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c7fff700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 0x7fd6c8154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 -- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 msgr2=0x7fd6c8154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 0x7fd6c8154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 -- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 msgr2=0x7fd6c813e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 0x7fd6c813e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c77fe700 1 -- 172.21.15.46:0/3355963459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0094f0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.420+0000 7fd6c7fff700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 0x7fd6c8154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6c77fe700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 0x7fd6c813e1e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8089490 tx=0x7fd6bc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01a420 con 0x7fd6c80d07b0 2023-10-06T12:33:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc01a5c0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc01a980 con 0x7fd6c80d07b0 2023-10-06T12:33:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c813db50 con 0x7fd6c80d07b0 2023-10-06T12:33:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80ce520 con 0x7fd6c80d07b0 2023-10-06T12:33:23.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.424+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c813d9b0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.428+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fd6bc01ad30 con 0x7fd6c80d07b0 2023-10-06T12:33:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.428+0000 7fd6affff700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 0x7fd6b4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.428+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd6bc07e870 con 0x7fd6c80d07b0 2023-10-06T12:33:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.428+0000 7fd6c7fff700 1 -- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 msgr2=0x7fd6b4062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.428+0000 7fd6c7fff700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 0x7fd6b4062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:23.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.436+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6c813d9b0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.479 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:23.475+0000 7f525bf0e000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:23.487 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:23.480+0000 7f9aa1765000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:33:23.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.608+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6c813d9b0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.612+0000 7fd6affff700 1 -- 172.21.15.46:0/3355963459 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+18252 (secure 0 0 0) 0x7fd6c813d9b0 con 0x7fd6c80d07b0 2023-10-06T12:33:23.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:23.619 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":140,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:23.206705+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":140,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2667975390":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6833/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6832/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1202911604":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2705107049":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2225233759":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1098696528":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/888393785":"2023-10-07T11:37:35.094017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":51,"length":6},{"begin":107,"length":6}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 msgr2=0x7fd6b4062b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 0x7fd6b4062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 msgr2=0x7fd6c813e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 0x7fd6c813e1e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8089490 tx=0x7fd6bc017000 comp rx=0 tx=0).stop 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 shutdown_connections 2023-10-06T12:33:23.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6b40606a0 0x7fd6b4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6c8159290 0x7fd6c813e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6c80d07b0 0x7fd6c813e1e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 --2- 172.21.15.46:0/3355963459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6c80b1130 0x7fd6c8154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 >> 172.21.15.46:0/3355963459 conn(0x7fd6c806ef60 msgr2=0x7fd6c815ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 shutdown_connections 2023-10-06T12:33:23.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.616+0000 7fd6cd363700 1 -- 172.21.15.46:0/3355963459 wait complete. 2023-10-06T12:33:23.646 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:23.640+0000 7f9aa1765000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:23.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:37:19.420143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:37:20.441003+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 140, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T12:33:23.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:33:23.740 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:23.735+0000 7f525bf0e000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:33:23.805 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:23.799+0000 7f525bf0e000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 Processor -- start 2023-10-06T12:33:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 -- start start 2023-10-06T12:33:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8069410 0x7fd4e815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e80697e0 0x7fd4e815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80ce470 con 0x7fd4e8158d10 2023-10-06T12:33:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e8069c50 con 0x7fd4e8069410 2023-10-06T12:33:23.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.812+0000 7fd4ed385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e815a890 con 0x7fd4e80697e0 2023-10-06T12:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8069410 0x7fd4e815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e67fc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e80697e0 0x7fd4e815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4e8069410 0x7fd4e815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:51666/0 (socket says 172.21.15.46:51666) 2023-10-06T12:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 -- 172.21.15.46:0/548196962 learned_addr learned my addr 172.21.15.46:0/548196962 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 -- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e80697e0 msgr2=0x7fd4e815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 --2- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e80697e0 0x7fd4e815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 -- 172.21.15.46:0/548196962 >> v1:172.21.15.132:6789/0 conn(0x7fd4e8158d10 legacy=0x7fd4e8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 -- 172.21.15.46:0/548196962 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e815d470 con 0x7fd4e8069410 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e6ffd700 1 --2- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8069410 0x7fd4e815ce90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80099b0 tx=0x7fd4d8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814f8b109529f474 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/548196962 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8009ea0 con 0x7fd4e8069410 2023-10-06T12:33:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/548196962 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d800b860 con 0x7fd4e8069410 2023-10-06T12:33:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4e5ffb700 1 -- 172.21.15.46:0/548196962 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800bc00 con 0x7fd4e8069410 2023-10-06T12:33:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 -- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8069410 msgr2=0x7fd4e815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 --2- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8069410 0x7fd4e815ce90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80099b0 tx=0x7fd4d8009250 comp rx=0 tx=0).stop 2023-10-06T12:33:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 -- 172.21.15.46:0/548196962 shutdown_connections 2023-10-06T12:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 --2- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e80697e0 0x7fd4e815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 --2- 172.21.15.46:0/548196962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8069410 0x7fd4e815ce90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 -- 172.21.15.46:0/548196962 >> 172.21.15.46:0/548196962 conn(0x7fd4e80d0210 msgr2=0x7fd4e80d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 -- 172.21.15.46:0/548196962 shutdown_connections 2023-10-06T12:33:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.816+0000 7fd4ed385700 1 -- 172.21.15.46:0/548196962 wait complete. 2023-10-06T12:33:23.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 Processor -- start 2023-10-06T12:33:23.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 -- start start 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e8069410 0x7fd4e80b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 0x7fd4e814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c6560 con 0x7fd4e80697e0 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c6840 con 0x7fd4e8158d10 2023-10-06T12:33:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4ed385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c80a0 con 0x7fd4e8069410 2023-10-06T12:33:23.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4e77fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 0x7fd4e814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4e67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34230/0 (socket says 172.21.15.46:34230) 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4e67fc700 1 -- 172.21.15.46:0/687885784 learned_addr learned my addr 172.21.15.46:0/687885784 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.820+0000 7fd4e6ffd700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e8069410 0x7fd4e80b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 -- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e8069410 msgr2=0x7fd4e80b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e8069410 0x7fd4e80b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 -- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 msgr2=0x7fd4e814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:23.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 0x7fd4e814a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 -- 172.21.15.46:0/687885784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80094f0 con 0x7fd4e80697e0 2023-10-06T12:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e77fe700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 0x7fd4e814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4e67fc700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00ec80 tx=0x7fd4dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00c850 con 0x7fd4e80697e0 2023-10-06T12:33:23.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00c9b0 con 0x7fd4e80697e0 2023-10-06T12:33:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00cd50 con 0x7fd4e80697e0 2023-10-06T12:33:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80c8380 con 0x7fd4e80697e0 2023-10-06T12:33:23.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80b5d50 con 0x7fd4e80697e0 2023-10-06T12:33:23.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.824+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80cbef0 con 0x7fd4e80697e0 2023-10-06T12:33:23.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.828+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fd4dc013680 con 0x7fd4e80697e0 2023-10-06T12:33:23.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.828+0000 7fd4ceffd700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 0x7fd4d4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:23.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.828+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd4dc07d0e0 con 0x7fd4e80697e0 2023-10-06T12:33:23.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.828+0000 7fd4e6ffd700 1 -- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 msgr2=0x7fd4d4062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:23.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.828+0000 7fd4e6ffd700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 0x7fd4d4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:23.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:23.836+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd4e80cbef0 con 0x7fd4e80697e0 2023-10-06T12:33:23.932 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:23.924+0000 7f9aa1765000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:33:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.000+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4e80cdcd0 con 0x7fd4e80697e0 2023-10-06T12:33:24.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.000+0000 7fd4ceffd700 1 -- 172.21.15.46:0/687885784 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4e80cdcd0 con 0x7fd4e80697e0 2023-10-06T12:33:24.007 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 msgr2=0x7fd4d4062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 0x7fd4d4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 msgr2=0x7fd4e816d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00ec80 tx=0x7fd4dc0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 shutdown_connections 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd4d4060670 0x7fd4d4062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8158d10 0x7fd4e814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4e80697e0 0x7fd4e816d8c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 --2- 172.21.15.46:0/687885784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4e8069410 0x7fd4e80b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 >> 172.21.15.46:0/687885784 conn(0x7fd4e80d0210 msgr2=0x7fd4e80d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 shutdown_connections 2023-10-06T12:33:24.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.004+0000 7fd4ed385700 1 -- 172.21.15.46:0/687885784 wait complete. 2023-10-06T12:33:24.013 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:24.004+0000 7f9aa1765000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:24.014 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:24.007+0000 7f525bf0e000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:33:24.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:33:24.090 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:24.083+0000 7f525bf0e000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:33:24.169 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:24.163+0000 7f525bf0e000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:33:24.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 Processor -- start 2023-10-06T12:33:24.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 -- start start 2023-10-06T12:33:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 0x7fd2b0069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2b0158aa0 0x7fd2b0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b015a5c0 con 0x7fd2b0158aa0 2023-10-06T12:33:24.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00ce6e0 con 0x7fd2b0069270 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2b486d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0159480 con 0x7fd2b0159850 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2ae59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2b0158aa0 0x7fd2b0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2ae59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2b0158aa0 0x7fd2b0158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34244/0 (socket says 172.21.15.46:34244) 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2ae59c700 1 -- 172.21.15.46:0/191404743 learned_addr learned my addr 172.21.15.46:0/191404743 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2aed9d700 1 --2- 172.21.15.46:0/191404743 >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 0x7fd2b0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.192+0000 7fd2af59e700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 -- 172.21.15.46:0/191404743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 msgr2=0x7fd2b0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:24.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 -- 172.21.15.46:0/191404743 >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 msgr2=0x7fd2b0069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 --2- 172.21.15.46:0/191404743 >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 0x7fd2b0069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 -- 172.21.15.46:0/191404743 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00cddb0 con 0x7fd2b0158aa0 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2aed9d700 1 --2- 172.21.15.46:0/191404743 >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 0x7fd2b0069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2ae59c700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0158aa0 0x7fd2b0158f40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0088010 tx=0x7fd2a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff925bc3c6d1d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2add9b700 1 -- 172.21.15.46:0/191404743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400b640 con 0x7fd2b0158aa0 2023-10-06T12:33:24.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2add9b700 1 -- 172.21.15.46:0/191404743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a400b7a0 con 0x7fd2b0158aa0 2023-10-06T12:33:24.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2add9b700 1 -- 172.21.15.46:0/191404743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400bb60 con 0x7fd2b0158aa0 2023-10-06T12:33:24.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 -- 172.21.15.46:0/191404743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0158aa0 msgr2=0x7fd2b0158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0158aa0 0x7fd2b0158f40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0088010 tx=0x7fd2a4009250 comp rx=0 tx=0).stop 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 -- 172.21.15.46:0/191404743 shutdown_connections 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 --2- 172.21.15.46:0/191404743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0158aa0 0x7fd2b0158f40 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 --2- 172.21.15.46:0/191404743 >> v2:172.21.15.132:3300/0 conn(0x7fd2b0069270 0x7fd2b0069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 -- 172.21.15.46:0/191404743 >> 172.21.15.46:0/191404743 conn(0x7fd2b0068390 msgr2=0x7fd2b00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 -- 172.21.15.46:0/191404743 shutdown_connections 2023-10-06T12:33:24.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.196+0000 7fd2b486d700 1 -- 172.21.15.46:0/191404743 wait complete. 2023-10-06T12:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 Processor -- start 2023-10-06T12:33:24.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 -- start start 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2b0158aa0 0x7fd2b00c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b00c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b016ccc0 con 0x7fd2b0069270 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b016d280 con 0x7fd2b0158aa0 2023-10-06T12:33:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.200+0000 7fd2b486d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b00c9f70 con 0x7fd2b0159850 2023-10-06T12:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34250/0 (socket says 172.21.15.46:34250) 2023-10-06T12:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 -- 172.21.15.46:0/4207119788 learned_addr learned my addr 172.21.15.46:0/4207119788 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2af59e700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b00c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2ae59c700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2b0158aa0 0x7fd2b00c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 -- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 msgr2=0x7fd2b00c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b00c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 -- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2b0158aa0 msgr2=0x7fd2b00c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2b0158aa0 0x7fd2b00c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 -- 172.21.15.46:0/4207119788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009420 con 0x7fd2b0069270 2023-10-06T12:33:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2aed9d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ecb0 tx=0x7fd2a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000c920 con 0x7fd2b0069270 2023-10-06T12:33:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000ca80 con 0x7fd2b0069270 2023-10-06T12:33:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000ce00 con 0x7fd2b0069270 2023-10-06T12:33:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40094f0 con 0x7fd2b0069270 2023-10-06T12:33:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.204+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b00c7e30 con 0x7fd2b0069270 2023-10-06T12:33:24.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b016dc50 con 0x7fd2b0069270 2023-10-06T12:33:24.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fd2a0013680 con 0x7fd2b0069270 2023-10-06T12:33:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd296ffd700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 0x7fd29c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fd2a007d120 con 0x7fd2b0069270 2023-10-06T12:33:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd2ae59c700 1 -- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 msgr2=0x7fd29c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.208+0000 7fd2ae59c700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 0x7fd29c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.216+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2a004d110 con 0x7fd2b0069270 2023-10-06T12:33:24.248 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:24.240+0000 7f9aa1765000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:33:24.338 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:24.332+0000 7f9aa1765000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:33:24.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.400+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2b0064590 con 0x7fd2b0069270 2023-10-06T12:33:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.400+0000 7fd296ffd700 1 -- 172.21.15.46:0/4207119788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2a0050730 con 0x7fd2b0069270 2023-10-06T12:33:24.409 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:33:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 msgr2=0x7fd29c062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 0x7fd29c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 msgr2=0x7fd2b016d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000ecb0 tx=0x7fd2a00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 shutdown_connections 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd29c060670 0x7fd29c062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2b0159850 0x7fd2b00c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2b0158aa0 0x7fd2b00c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 --2- 172.21.15.46:0/4207119788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2b0069270 0x7fd2b016d710 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 >> 172.21.15.46:0/4207119788 conn(0x7fd2b0068390 msgr2=0x7fd2b00b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 shutdown_connections 2023-10-06T12:33:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.404+0000 7fd2b486d700 1 -- 172.21.15.46:0/4207119788 wait complete. 2023-10-06T12:33:24.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T12:33:24.429 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:24.420+0000 7f9aa1765000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:33:24.555 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:24.547+0000 7f525bf0e000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:33:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 Processor -- start 2023-10-06T12:33:24.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 -- start start 2023-10-06T12:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c041591e0 0x7f4c04068060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c040685a0 0x7f4c040aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040ac4c0 con 0x7f4c04158e10 2023-10-06T12:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040aa610 con 0x7f4c041591e0 2023-10-06T12:33:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.588+0000 7f4c09945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040af290 con 0x7f4c040685a0 2023-10-06T12:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c041591e0 0x7f4c04068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c02ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4c04158e10 0x7f4c040abdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33038/0 (socket says 172.21.15.46:33038) 2023-10-06T12:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c02ffd700 1 -- 172.21.15.46:0/3960253443 learned_addr learned my addr 172.21.15.46:0/3960253443 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:24.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c037fe700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c040685a0 0x7f4c040aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 -- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c040685a0 msgr2=0x7f4c040aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c040685a0 0x7f4c040aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 -- 172.21.15.46:0/3960253443 >> v1:172.21.15.46:6789/0 conn(0x7f4c04158e10 legacy=0x7f4c040abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c01ffb700 1 -- 172.21.15.46:0/3960253443 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233903786 0 0) 0x7f4c040ac4c0 con 0x7f4c04158e10 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 -- 172.21.15.46:0/3960253443 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040689c0 con 0x7f4c041591e0 2023-10-06T12:33:24.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c027fc700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c041591e0 0x7f4c04068060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098e0 tx=0x7f4bf80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9d1e07d5cc29b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c01ffb700 1 -- 172.21.15.46:0/3960253443 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800b640 con 0x7f4c041591e0 2023-10-06T12:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c01ffb700 1 -- 172.21.15.46:0/3960253443 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800b7a0 con 0x7f4c041591e0 2023-10-06T12:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c01ffb700 1 -- 172.21.15.46:0/3960253443 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800bb40 con 0x7f4c041591e0 2023-10-06T12:33:24.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 -- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c041591e0 msgr2=0x7f4c04068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c041591e0 0x7f4c04068060 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80098e0 tx=0x7f4bf80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 -- 172.21.15.46:0/3960253443 shutdown_connections 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c040685a0 0x7f4c040aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 --2- 172.21.15.46:0/3960253443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c041591e0 0x7f4c04068060 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 -- 172.21.15.46:0/3960253443 >> 172.21.15.46:0/3960253443 conn(0x7f4c0406ac10 msgr2=0x7f4c0406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 -- 172.21.15.46:0/3960253443 shutdown_connections 2023-10-06T12:33:24.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.592+0000 7f4c09945700 1 -- 172.21.15.46:0/3960253443 wait complete. 2023-10-06T12:33:24.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 Processor -- start 2023-10-06T12:33:24.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 -- start start 2023-10-06T12:33:24.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c040685a0 0x7f4c040cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 0x7f4c0415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040c9670 con 0x7f4c04158e10 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040cb5c0 con 0x7f4c040685a0 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c09945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04154380 con 0x7f4c041591e0 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34252/0 (socket says 172.21.15.46:34252) 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 -- 172.21.15.46:0/940436397 learned_addr learned my addr 172.21.15.46:0/940436397 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c02ffd700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c040685a0 0x7f4c040cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c037fe700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 0x7f4c0415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 -- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 msgr2=0x7f4c0415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 0x7f4c0415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 -- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c040685a0 msgr2=0x7f4c040cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c040685a0 0x7f4c040cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c027fc700 1 -- 172.21.15.46:0/940436397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009420 con 0x7f4c04158e10 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.596+0000 7f4c037fe700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 0x7f4c0415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4c027fc700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009f80 tx=0x7f4bf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800bb20 con 0x7f4c04158e10 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8000c40 con 0x7f4c04158e10 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c040c90f0 con 0x7f4c04158e10 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8024650 con 0x7f4c04158e10 2023-10-06T12:33:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c040cba90 con 0x7f4c04158e10 2023-10-06T12:33:24.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.600+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040cbc80 con 0x7f4c04158e10 2023-10-06T12:33:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.604+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f4bf8024a80 con 0x7f4c04158e10 2023-10-06T12:33:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.604+0000 7f4beaffd700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 0x7f4bf0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.604+0000 7f4c02ffd700 1 -- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 msgr2=0x7f4bf0062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.604+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f4bf807e840 con 0x7f4c04158e10 2023-10-06T12:33:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.604+0000 7f4c02ffd700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 0x7f4bf0062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.612+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4bf804e790 con 0x7f4c04158e10 2023-10-06T12:33:24.725 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:24.719+0000 7f525bf0e000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:33:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.776+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c04064590 con 0x7f4c04158e10 2023-10-06T12:33:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4beaffd700 1 -- 172.21.15.46:0/940436397 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bf8051db0 con 0x7f4c04158e10 2023-10-06T12:33:24.786 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 msgr2=0x7f4bf0062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 0x7f4bf0062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 msgr2=0x7f4c0415e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009f80 tx=0x7f4bf8017000 comp rx=0 tx=0).stop 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 shutdown_connections 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4bf0060670 0x7f4bf0062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c041591e0 0x7f4c0415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04158e10 0x7f4c0415e8f0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 --2- 172.21.15.46:0/940436397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c040685a0 0x7f4c040cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 >> 172.21.15.46:0/940436397 conn(0x7f4c0406ac10 msgr2=0x7f4c04154860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 shutdown_connections 2023-10-06T12:33:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.780+0000 7f4c09945700 1 -- 172.21.15.46:0/940436397 wait complete. 2023-10-06T12:33:24.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T12:33:24.850 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:24.844+0000 7f9aa1765000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:33:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 Processor -- start 2023-10-06T12:33:24.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 -- start start 2023-10-06T12:33:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 0x7f447c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447c154c50 0x7f447c06c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.960+0000 7f4482ee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0c6cf0 con 0x7f447c154c50 2023-10-06T12:33:24.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0c2a30 con 0x7f447c154880 2023-10-06T12:33:24.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c83e0 con 0x7f447c0c7f20 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447c154c50 0x7f447c06c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f447c154c50 0x7f447c06c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34254/0 (socket says 172.21.15.46:34254) 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 -- 172.21.15.46:0/3039257456 learned_addr learned my addr 172.21.15.46:0/3039257456 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4481ee1700 1 --2- 172.21.15.46:0/3039257456 >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 0x7f447c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44826e2700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 -- 172.21.15.46:0/3039257456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 msgr2=0x7f447c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 -- 172.21.15.46:0/3039257456 >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 msgr2=0x7f447c0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:24.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 --2- 172.21.15.46:0/3039257456 >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 0x7f447c0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 -- 172.21.15.46:0/3039257456 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c06e5d0 con 0x7f447c154c50 2023-10-06T12:33:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4481ee1700 1 --2- 172.21.15.46:0/3039257456 >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 0x7f447c0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44826e2700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:24.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f44816e0700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154c50 0x7f447c06c9f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f447c06a5f0 tx=0x7f446c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cfd3c08f6f75be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4480edf700 1 -- 172.21.15.46:0/3039257456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00b860 con 0x7f447c154c50 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4480edf700 1 -- 172.21.15.46:0/3039257456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00b9c0 con 0x7f447c154c50 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4480edf700 1 -- 172.21.15.46:0/3039257456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00bd60 con 0x7f447c154c50 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- 172.21.15.46:0/3039257456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154c50 msgr2=0x7f447c06c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154c50 0x7f447c06c9f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f447c06a5f0 tx=0x7f446c009250 comp rx=0 tx=0).stop 2023-10-06T12:33:24.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- 172.21.15.46:0/3039257456 shutdown_connections 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 --2- 172.21.15.46:0/3039257456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154c50 0x7f447c06c9f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 --2- 172.21.15.46:0/3039257456 >> v2:172.21.15.132:3300/0 conn(0x7f447c154880 0x7f447c0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- 172.21.15.46:0/3039257456 >> 172.21.15.46:0/3039257456 conn(0x7f447c0cbc10 msgr2=0x7f447c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- 172.21.15.46:0/3039257456 shutdown_connections 2023-10-06T12:33:24.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.964+0000 7f4482ee3700 1 -- 172.21.15.46:0/3039257456 wait complete. 2023-10-06T12:33:24.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.968+0000 7f4482ee3700 1 Processor -- start 2023-10-06T12:33:24.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.968+0000 7f4482ee3700 1 -- start start 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c0c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c154c50 0x7f447c1466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c16a3b0 con 0x7f447c154880 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c169830 con 0x7f447c154c50 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c169b10 con 0x7f447c0c7f20 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34260/0 (socket says 172.21.15.46:34260) 2023-10-06T12:33:24.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 -- 172.21.15.46:0/396966362 learned_addr learned my addr 172.21.15.46:0/396966362 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:24.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4481ee1700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c0c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44826e2700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c154c50 0x7f447c1466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 -- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 msgr2=0x7f447c0c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c0c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 -- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c154c50 msgr2=0x7f447c1466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c154c50 0x7f447c1466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 -- 172.21.15.46:0/396966362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0094f0 con 0x7f447c154880 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f44816e0700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f446c009290 tx=0x7f446c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c011e40 con 0x7f447c154880 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c01f070 con 0x7f447c154880 2023-10-06T12:33:24.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c169df0 con 0x7f447c154880 2023-10-06T12:33:24.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0266d0 con 0x7f447c154880 2023-10-06T12:33:24.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0c6710 con 0x7f447c154880 2023-10-06T12:33:24.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.972+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c15bbc0 con 0x7f447c154880 2023-10-06T12:33:24.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.976+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f446c026a40 con 0x7f447c154880 2023-10-06T12:33:24.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.976+0000 7f4471ffb700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 0x7f4460062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:24.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.976+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f446c021070 con 0x7f447c154880 2023-10-06T12:33:24.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.976+0000 7f4481ee1700 1 -- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 msgr2=0x7f4460062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:24.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.976+0000 7f4481ee1700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 0x7f4460062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:24.984+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f446c04eac0 con 0x7f447c154880 2023-10-06T12:33:25.033 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.024+0000 7f9aa1765000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:33:25.096 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.091+0000 7f525bf0e000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:33:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.164+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f447c064590 con 0x7f447c154880 2023-10-06T12:33:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.164+0000 7f4471ffb700 1 -- 172.21.15.46:0/396966362 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) v1 ==== 100+0+11 (secure 0 0 0) 0x7f446c015020 con 0x7f447c154880 2023-10-06T12:33:25.171 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 msgr2=0x7f4460062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 0x7f4460062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 msgr2=0x7f447c0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f446c009290 tx=0x7f446c017000 comp rx=0 tx=0).stop 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 shutdown_connections 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4460060670 0x7f4460062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c154c50 0x7f447c1466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f447c154880 0x7f447c0c61d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 --2- 172.21.15.46:0/396966362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f447c0c7f20 0x7f447c0c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 >> 172.21.15.46:0/396966362 conn(0x7f447c0cbc10 msgr2=0x7f447c0c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:25.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 shutdown_connections 2023-10-06T12:33:25.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.168+0000 7f4482ee3700 1 -- 172.21.15.46:0/396966362 wait complete. 2023-10-06T12:33:25.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:25.324 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.319+0000 7f525bf0e000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:25.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.392+0000 7fc590ccd700 1 Processor -- start 2023-10-06T12:33:25.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.392+0000 7fc590ccd700 1 -- start start 2023-10-06T12:33:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc58c15c770 0x7fc58c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0cea30 0x7fc58c0c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c0cfbb0 con 0x7fc58c0ccb50 2023-10-06T12:33:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58c0cdbf0 con 0x7fc58c15c770 2023-10-06T12:33:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc590ccd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c0cc150 con 0x7fc58c0cea30 2023-10-06T12:33:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34264/0 (socket says 172.21.15.46:34264) 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 -- 172.21.15.46:0/1507264648 learned_addr learned my addr 172.21.15.46:0/1507264648 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc58a59c700 1 --2- 172.21.15.46:0/1507264648 >> v2:172.21.15.132:3300/0 conn(0x7fc58c15c770 0x7fc58c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc58ad9d700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0cea30 0x7fc58c0c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 -- 172.21.15.46:0/1507264648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0cea30 msgr2=0x7fc58c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0cea30 0x7fc58c0c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 -- 172.21.15.46:0/1507264648 >> v2:172.21.15.132:3300/0 conn(0x7fc58c15c770 msgr2=0x7fc58c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1507264648 >> v2:172.21.15.132:3300/0 conn(0x7fc58c15c770 0x7fc58c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 -- 172.21.15.46:0/1507264648 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c0cee50 con 0x7fc58c0ccb50 2023-10-06T12:33:25.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06f6d0 tx=0x7fc580009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2481132c9a483bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc58959a700 1 -- 172.21.15.46:0/1507264648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000b640 con 0x7fc58c0ccb50 2023-10-06T12:33:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc58959a700 1 -- 172.21.15.46:0/1507264648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58000b7a0 con 0x7fc58c0ccb50 2023-10-06T12:33:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.396+0000 7fc58959a700 1 -- 172.21.15.46:0/1507264648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000bb60 con 0x7fc58c0ccb50 2023-10-06T12:33:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- 172.21.15.46:0/1507264648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0ccb50 msgr2=0x7fc58c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc58c06f6d0 tx=0x7fc580009250 comp rx=0 tx=0).stop 2023-10-06T12:33:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- 172.21.15.46:0/1507264648 shutdown_connections 2023-10-06T12:33:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0cea30 0x7fc58c0c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1507264648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0ccb50 0x7fc58c0ce4f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1507264648 >> v2:172.21.15.132:3300/0 conn(0x7fc58c15c770 0x7fc58c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- 172.21.15.46:0/1507264648 >> 172.21.15.46:0/1507264648 conn(0x7fc58c099480 msgr2=0x7fc58c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- 172.21.15.46:0/1507264648 shutdown_connections 2023-10-06T12:33:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- 172.21.15.46:0/1507264648 wait complete. 2023-10-06T12:33:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 Processor -- start 2023-10-06T12:33:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.400+0000 7fc590ccd700 1 -- start start 2023-10-06T12:33:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0ccb50 0x7fc58c152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 0x7fc58c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c158e30 con 0x7fc58c0cea30 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c156250 con 0x7fc58c15c770 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c158b50 con 0x7fc58c0ccb50 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34276/0 (socket says 172.21.15.46:34276) 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 -- 172.21.15.46:0/1229385243 learned_addr learned my addr 172.21.15.46:0/1229385243 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc58ad9d700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 0x7fc58c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc58a59c700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0ccb50 0x7fc58c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0ccb50 msgr2=0x7fc58c152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0ccb50 0x7fc58c152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 msgr2=0x7fc58c154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 0x7fc58c154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 -- 172.21.15.46:0/1229385243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800094f0 con 0x7fc58c0cea30 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc58ad9d700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 0x7fc58c154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc589d9b700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc580006fd0 tx=0x7fc580017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580000d10 con 0x7fc58c0cea30 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58000bb20 con 0x7fc58c0cea30 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580011670 con 0x7fc58c0cea30 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58c155c00 con 0x7fc58c0cea30 2023-10-06T12:33:25.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.404+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c151640 con 0x7fc58c0cea30 2023-10-06T12:33:25.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.408+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c1563e0 con 0x7fc58c0cea30 2023-10-06T12:33:25.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.408+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fc5800119e0 con 0x7fc58c0cea30 2023-10-06T12:33:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.412+0000 7fc57a7fc700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 0x7fc56c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.412+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fc58007e5f0 con 0x7fc58c0cea30 2023-10-06T12:33:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.412+0000 7fc58a59c700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 msgr2=0x7fc56c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.412+0000 7fc58a59c700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 0x7fc56c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:25.418 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.411+0000 7f525bf0e000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:33:25.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.416+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc58004e540 con 0x7fc58c0cea30 2023-10-06T12:33:25.426 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.420+0000 7f9aa1765000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:33:25.495 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.487+0000 7f525bf0e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:33:25.574 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.567+0000 7f525bf0e000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:33:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.616+0000 7fc58a59c700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 msgr2=0x7fc56c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.616+0000 7fc58a59c700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 0x7fc56c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:33:25.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.620+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc58c0660f0 con 0x7fc58c0cea30 2023-10-06T12:33:25.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.620+0000 7fc57a7fc700 1 -- 172.21.15.46:0/1229385243 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fc580051b60 con 0x7fc58c0cea30 2023-10-06T12:33:25.627 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:25.627 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 msgr2=0x7fc56c062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 0x7fc56c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 msgr2=0x7fc58c151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc580006fd0 tx=0x7fc580017000 comp rx=0 tx=0).stop 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 shutdown_connections 2023-10-06T12:33:25.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc56c060670 0x7fc56c062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc58c15c770 0x7fc58c154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc58c0cea30 0x7fc58c151100 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 --2- 172.21.15.46:0/1229385243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc58c0ccb50 0x7fc58c152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 >> 172.21.15.46:0/1229385243 conn(0x7fc58c099480 msgr2=0x7fc58c0cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 shutdown_connections 2023-10-06T12:33:25.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.624+0000 7fc590ccd700 1 -- 172.21.15.46:0/1229385243 wait complete. 2023-10-06T12:33:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T12:33:25.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:25.657 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.648+0000 7f9aa1765000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:25.750 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.744+0000 7f9aa1765000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:33:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 Processor -- start 2023-10-06T12:33:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 -- start start 2023-10-06T12:33:25.807 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.799+0000 7f525bf0e000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 0x7f3c5c0c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c06dd30 con 0x7f3c5c0c2a10 2023-10-06T12:33:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5c06ce80 con 0x7f3c5c155850 2023-10-06T12:33:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c6442e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c06d120 con 0x7f3c5c155480 2023-10-06T12:33:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c629cb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c5c0c2a10 0x7f3c5c0c9a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33052/0 (socket says 172.21.15.46:33052) 2023-10-06T12:33:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c629cb700 1 -- 172.21.15.46:0/1959961987 learned_addr learned my addr 172.21.15.46:0/1959961987 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c621ca700 1 --2- 172.21.15.46:0/1959961987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 0x7f3c5c0c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c611c8700 1 -- 172.21.15.46:0/1959961987 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157995004 0 0) 0x7f3c5c06dd30 con 0x7f3c5c0c2a10 2023-10-06T12:33:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.800+0000 7f3c611c8700 1 -- 172.21.15.46:0/1959961987 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c48003570 con 0x7f3c5c0c2a10 2023-10-06T12:33:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c621ca700 1 -- 172.21.15.46:0/1959961987 >> v1:172.21.15.132:6789/0 conn(0x7f3c5c155850 legacy=0x7f3c5c0c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c621ca700 1 -- 172.21.15.46:0/1959961987 >> v1:172.21.15.46:6789/0 conn(0x7f3c5c0c2a10 legacy=0x7f3c5c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c621ca700 1 -- 172.21.15.46:0/1959961987 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c06d400 con 0x7f3c5c155480 2023-10-06T12:33:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c621ca700 1 --2- 172.21.15.46:0/1959961987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 0x7f3c5c0c1460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009ee0 tx=0x7f3c58009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b84e52e55ddf74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c611c8700 1 -- 172.21.15.46:0/1959961987 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800e070 con 0x7f3c5c155480 2023-10-06T12:33:25.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c611c8700 1 -- 172.21.15.46:0/1959961987 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5800bc30 con 0x7f3c5c155480 2023-10-06T12:33:25.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c611c8700 1 -- 172.21.15.46:0/1959961987 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58012430 con 0x7f3c5c155480 2023-10-06T12:33:25.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 -- 172.21.15.46:0/1959961987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 msgr2=0x7f3c5c0c1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 --2- 172.21.15.46:0/1959961987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 0x7f3c5c0c1460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009ee0 tx=0x7f3c58009780 comp rx=0 tx=0).stop 2023-10-06T12:33:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 -- 172.21.15.46:0/1959961987 shutdown_connections 2023-10-06T12:33:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 --2- 172.21.15.46:0/1959961987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c155480 0x7f3c5c0c1460 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 -- 172.21.15.46:0/1959961987 >> 172.21.15.46:0/1959961987 conn(0x7f3c5c0cbe30 msgr2=0x7f3c5c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 -- 172.21.15.46:0/1959961987 shutdown_connections 2023-10-06T12:33:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.804+0000 7f3c6442e700 1 -- 172.21.15.46:0/1959961987 wait complete. 2023-10-06T12:33:25.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 Processor -- start 2023-10-06T12:33:25.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 -- start start 2023-10-06T12:33:25.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 0x7f3c5c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c5c155480 0x7f3c5c0aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c143060 con 0x7f3c5c155850 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c0aaa60 con 0x7f3c5c155480 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c6442e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c142730 con 0x7f3c5c0c2a10 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34288/0 (socket says 172.21.15.46:34288) 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 -- 172.21.15.46:0/2407542436 learned_addr learned my addr 172.21.15.46:0/2407542436 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c619c9700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c5c155480 0x7f3c5c0aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c621ca700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 0x7f3c5c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 -- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 msgr2=0x7f3c5c0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 0x7f3c5c0c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 -- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c5c155480 msgr2=0x7f3c5c0aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c5c155480 0x7f3c5c0aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 -- 172.21.15.46:0/2407542436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58009a20 con 0x7f3c5c155850 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c621ca700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 0x7f3c5c0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.808+0000 7f3c629cb700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3c540010e0 tx=0x7f3c54007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54005c00 con 0x7f3c5c155850 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54005d60 con 0x7f3c5c155850 2023-10-06T12:33:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c142a10 con 0x7f3c5c155850 2023-10-06T12:33:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540072e0 con 0x7f3c5c155850 2023-10-06T12:33:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c0aa760 con 0x7f3c5c155850 2023-10-06T12:33:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c063f90 con 0x7f3c5c155850 2023-10-06T12:33:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.812+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f3c54007690 con 0x7f3c5c155850 2023-10-06T12:33:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.816+0000 7f3c527fc700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 0x7f3c40062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.816+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f3c5407c300 con 0x7f3c5c155850 2023-10-06T12:33:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.816+0000 7f3c621ca700 1 -- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 msgr2=0x7f3c40062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.816+0000 7f3c621ca700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 0x7f3c40062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:25.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.816+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3c54049be0 con 0x7f3c5c155850 2023-10-06T12:33:25.828 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.820+0000 7f9aa1765000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:33:25.886 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.879+0000 7f525bf0e000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:33:25.915 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:25.908+0000 7f9aa1765000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:33:25.979 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:25.975+0000 7f525bf0e000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:33:25.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.988+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c5c06d6e0 con 0x7f3c5c155850 2023-10-06T12:33:25.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.988+0000 7f3c527fc700 1 -- 172.21.15.46:0/2407542436 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+18252 (secure 0 0 0) 0x7f3c5404c300 con 0x7f3c5c155850 2023-10-06T12:33:25.997 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:25.997 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":141,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:24.221349+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2667975390":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6833/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6832/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1202911604":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2705107049":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2225233759":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1098696528":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/888393785":"2023-10-07T11:37:35.094017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:25.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 msgr2=0x7f3c40062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 0x7f3c40062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 msgr2=0x7f3c5c0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3c540010e0 tx=0x7f3c54007dd0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 shutdown_connections 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3c40060670 0x7f3c40062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c5c155850 0x7f3c5c0cabe0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c5c155480 0x7f3c5c0aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 --2- 172.21.15.46:0/2407542436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c5c0c2a10 0x7f3c5c0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 >> 172.21.15.46:0/2407542436 conn(0x7f3c5c0cbe30 msgr2=0x7f3c5c068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 shutdown_connections 2023-10-06T12:33:26.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:25.992+0000 7f3c6442e700 1 -- 172.21.15.46:0/2407542436 wait complete. 2023-10-06T12:33:26.021 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:33:26.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:26.157 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:26.148+0000 7f9aa1765000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:26.180 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:26.175+0000 7f525bf0e000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:33:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.184+0000 7f13d8a2e700 1 Processor -- start 2023-10-06T12:33:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.184+0000 7f13d8a2e700 1 -- start start 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f13d415ab60 0x7f13d406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13d415af30 0x7f13d406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d40ad6c0 0x7f13d40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d406d460 con 0x7f13d415af30 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40c7000 con 0x7f13d415ab60 2023-10-06T12:33:26.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c72c0 con 0x7f13d40ad6c0 2023-10-06T12:33:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13d415af30 0x7f13d406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13d415af30 0x7f13d406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34294/0 (socket says 172.21.15.46:34294) 2023-10-06T12:33:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 -- 172.21.15.46:0/3774588038 learned_addr learned my addr 172.21.15.46:0/3774588038 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:26.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d2d9d700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d40ad6c0 0x7f13d40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d259c700 1 --2- 172.21.15.46:0/3774588038 >> v2:172.21.15.132:3300/0 conn(0x7f13d415ab60 0x7f13d406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 -- 172.21.15.46:0/3774588038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d40ad6c0 msgr2=0x7f13d40adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d40ad6c0 0x7f13d40adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 -- 172.21.15.46:0/3774588038 >> v2:172.21.15.132:3300/0 conn(0x7f13d415ab60 msgr2=0x7f13d406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/3774588038 >> v2:172.21.15.132:3300/0 conn(0x7f13d415ab60 0x7f13d406e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 -- 172.21.15.46:0/3774588038 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40c75a0 con 0x7f13d415af30 2023-10-06T12:33:26.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d415af30 0x7f13d406cc30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f13d4063be0 tx=0x7f13c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f6320ac2da5b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:26.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d159a700 1 -- 172.21.15.46:0/3774588038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8009e70 con 0x7f13d415af30 2023-10-06T12:33:26.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d159a700 1 -- 172.21.15.46:0/3774588038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c800b860 con 0x7f13d415af30 2023-10-06T12:33:26.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d159a700 1 -- 172.21.15.46:0/3774588038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800bc00 con 0x7f13d415af30 2023-10-06T12:33:26.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- 172.21.15.46:0/3774588038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d415af30 msgr2=0x7f13d406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d415af30 0x7f13d406cc30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f13d4063be0 tx=0x7f13c8009250 comp rx=0 tx=0).stop 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- 172.21.15.46:0/3774588038 shutdown_connections 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d40ad6c0 0x7f13d40adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/3774588038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d415af30 0x7f13d406cc30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/3774588038 >> v2:172.21.15.132:3300/0 conn(0x7f13d415ab60 0x7f13d406e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- 172.21.15.46:0/3774588038 >> 172.21.15.46:0/3774588038 conn(0x7f13d40cbe30 msgr2=0x7f13d40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- 172.21.15.46:0/3774588038 shutdown_connections 2023-10-06T12:33:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.188+0000 7f13d8a2e700 1 -- 172.21.15.46:0/3774588038 wait complete. 2023-10-06T12:33:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 Processor -- start 2023-10-06T12:33:26.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 -- start start 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d415ab60 0x7f13d4069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13d415af30 0x7f13d40696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4167170 con 0x7f13d40ad6c0 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41672d0 con 0x7f13d415af30 2023-10-06T12:33:26.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d8a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c3da0 con 0x7f13d415ab60 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34310/0 (socket says 172.21.15.46:34310) 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d259c700 1 -- 172.21.15.46:0/1158018660 learned_addr learned my addr 172.21.15.46:0/1158018660 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.192+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d415ab60 0x7f13d4069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d2d9d700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13d415af30 0x7f13d40696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d415ab60 msgr2=0x7f13d4069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d415ab60 0x7f13d4069190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13d415af30 msgr2=0x7f13d40696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13d415af30 0x7f13d40696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 -- 172.21.15.46:0/1158018660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80094f0 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d259c700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13c400d840 tx=0x7f13c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4014620 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4014780 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d40c4080 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4014b40 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40c4640 con 0x7f13d40ad6c0 2023-10-06T12:33:26.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.196+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d4063f90 con 0x7f13d40ad6c0 2023-10-06T12:33:26.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.200+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f13c401d400 con 0x7f13d40ad6c0 2023-10-06T12:33:26.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.200+0000 7f13c27fc700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 0x7f13b4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.200+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f13c407d360 con 0x7f13d40ad6c0 2023-10-06T12:33:26.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.200+0000 7f13d1d9b700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 msgr2=0x7f13b4062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:26.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.200+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 0x7f13b4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:26.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.204+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f13c404d2b0 con 0x7f13d40ad6c0 2023-10-06T12:33:26.244 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:26.236+0000 7f9aa1765000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:33:26.271 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:26.263+0000 7f525bf0e000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:33:26.338 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:26.332+0000 7f9aa1765000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:33:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.404+0000 7f13d1d9b700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 msgr2=0x7f13b4062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.404+0000 7f13d1d9b700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 0x7f13b4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:33:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.420+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13d404b5c0 con 0x7f13d40ad6c0 2023-10-06T12:33:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13c27fc700 1 -- 172.21.15.46:0/1158018660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f13c40508d0 con 0x7f13d40ad6c0 2023-10-06T12:33:26.429 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:26.430 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 msgr2=0x7f13b4062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 0x7f13b4062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 msgr2=0x7f13d4150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13c400d840 tx=0x7f13c4008040 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 shutdown_connections 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13b4060670 0x7f13b4062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13d415af30 0x7f13d40696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13d415ab60 0x7f13d4069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 --2- 172.21.15.46:0/1158018660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d40ad6c0 0x7f13d4150a80 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 >> 172.21.15.46:0/1158018660 conn(0x7f13d40cbe30 msgr2=0x7f13d406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 shutdown_connections 2023-10-06T12:33:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.424+0000 7f13d8a2e700 1 -- 172.21.15.46:0/1158018660 wait complete. 2023-10-06T12:33:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T12:33:26.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:26.566 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:26.560+0000 7f9aa1765000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:33:26.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.612+0000 7f5e10cae700 1 Processor -- start 2023-10-06T12:33:26.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.612+0000 7f5e10cae700 1 -- start start 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e10cae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e0c0d0aa0 0x7f5e0c159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e10cae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c0b5830 con 0x7f5e0c0b5410 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e10cae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c0b4f80 con 0x7f5e0c159050 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e10cae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0c0b40c0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5e0c0b5410 0x7f5e0c0b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33062/0 (socket says 172.21.15.46:33062) 2023-10-06T12:33:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0ad9d700 1 -- 172.21.15.46:0/298187224 learned_addr learned my addr 172.21.15.46:0/298187224 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e09d9b700 1 --2- 172.21.15.46:0/298187224 >> v2:172.21.15.132:3300/0 conn(0x7f5e0c0d0aa0 0x7f5e0c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386861007 0 0) 0x7f5e0c0b5830 con 0x7f5e0c0b5410 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5df4003570 con 0x7f5e0c0b5410 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927889700 0 0) 0x7f5e0c0b4f80 con 0x7f5e0c159050 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e0c0b5830 con 0x7f5e0c159050 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192431196 0 0) 0x7f5df4003570 con 0x7f5e0c0b5410 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e09d9b700 1 -- 172.21.15.46:0/298187224 >> v1:172.21.15.132:6789/0 conn(0x7f5e0c159050 legacy=0x7f5e0c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.616+0000 7f5e09d9b700 1 -- 172.21.15.46:0/298187224 >> v1:172.21.15.46:6789/0 conn(0x7f5e0c0b5410 legacy=0x7f5e0c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e09d9b700 1 -- 172.21.15.46:0/298187224 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c0b4370 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e09d9b700 1 --2- 172.21.15.46:0/298187224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c159420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098e0 tx=0x7f5e000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d90b50ad8ad026 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e040017f0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00011420 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e0959a700 1 -- 172.21.15.46:0/298187224 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000117c0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 -- 172.21.15.46:0/298187224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0d0aa0 msgr2=0x7f5e0c159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 --2- 172.21.15.46:0/298187224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c159420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5e000098e0 tx=0x7f5e000091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 -- 172.21.15.46:0/298187224 shutdown_connections 2023-10-06T12:33:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 --2- 172.21.15.46:0/298187224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c159420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 -- 172.21.15.46:0/298187224 >> 172.21.15.46:0/298187224 conn(0x7f5e0c06e780 msgr2=0x7f5e0c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 -- 172.21.15.46:0/298187224 shutdown_connections 2023-10-06T12:33:26.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.620+0000 7f5e10cae700 1 -- 172.21.15.46:0/298187224 wait complete. 2023-10-06T12:33:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 Processor -- start 2023-10-06T12:33:26.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 -- start start 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0b5410 0x7f5e0c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e0c159050 0x7f5e0c0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0c4b10 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c15a640 con 0x7f5e0c0b5410 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e10cae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c156270 con 0x7f5e0c159050 2023-10-06T12:33:26.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34318/0 (socket says 172.21.15.46:34318) 2023-10-06T12:33:26.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.624+0000 7f5e09d9b700 1 -- 172.21.15.46:0/4195650271 learned_addr learned my addr 172.21.15.46:0/4195650271 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e0ad9d700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e0c159050 0x7f5e0c0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e0a59c700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0b5410 0x7f5e0c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e0c159050 msgr2=0x7f5e0c0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e0c159050 0x7f5e0c0c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0b5410 msgr2=0x7f5e0c14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0b5410 0x7f5e0c14f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 -- 172.21.15.46:0/4195650271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00009420 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e09d9b700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0892a0 tx=0x7f5e00016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0001cc90 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0001cdf0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c156550 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0001a6a0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c14f750 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.628+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0c1566e0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.632+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f5e0001aa10 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.632+0000 7f5dfa7fc700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 0x7f5dec062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.632+0000 7f5e0a59c700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 msgr2=0x7f5dec062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.632+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f5e0007e8c0 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.632+0000 7f5e0a59c700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 0x7f5dec062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:26.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.640+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5e0004e810 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.653 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:26.644+0000 7f9aa1765000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:33:26.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.836+0000 7f5e0a59c700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 msgr2=0x7f5dec062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:26.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.836+0000 7f5e0a59c700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 0x7f5dec062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:33:26.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.848+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e0c0b4650 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5dfa7fc700 1 -- 172.21.15.46:0/4195650271 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5e00051e30 con 0x7f5e0c0d0aa0 2023-10-06T12:33:26.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:26.858 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:37:21.446503+0000","modified":"2023-10-06T11:37:25.060450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6837/3974508311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3974508311},{"type":"v1","addr":"172.21.15.46:6837","nonce":3974508311}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T12:33:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 msgr2=0x7f5dec062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 0x7f5dec062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 msgr2=0x7f5e0c0c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:26.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0892a0 tx=0x7f5e00016000 comp rx=0 tx=0).stop 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 shutdown_connections 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5dec060670 0x7f5dec062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e0c159050 0x7f5e0c0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e0c0d0aa0 0x7f5e0c0c3c30 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 --2- 172.21.15.46:0/4195650271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e0c0b5410 0x7f5e0c14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 >> 172.21.15.46:0/4195650271 conn(0x7f5e0c06e780 msgr2=0x7f5e0c1572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 shutdown_connections 2023-10-06T12:33:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:26.852+0000 7f5e10cae700 1 -- 172.21.15.46:0/4195650271 wait complete. 2023-10-06T12:33:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2023-10-06T12:33:26.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:27.008 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:27.003+0000 7f525bf0e000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:33:27.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 Processor -- start 2023-10-06T12:33:27.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 -- start start 2023-10-06T12:33:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f70c00ccb50 0x7f70c00cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c00cc150 con 0x7f70c015c770 2023-10-06T12:33:27.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c00cfbb0 con 0x7f70c00ccb50 2023-10-06T12:33:27.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.032+0000 7f70c6c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c00cdbf0 con 0x7f70c00c5ff0 2023-10-06T12:33:27.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c4a0a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70c015c770 0x7f70c00cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33070/0 (socket says 172.21.15.46:33070) 2023-10-06T12:33:27.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c4a0a700 1 -- 172.21.15.46:0/534513999 learned_addr learned my addr 172.21.15.46:0/534513999 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:27.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 --2- 172.21.15.46:0/534513999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c520b700 1 --2- 172.21.15.46:0/534513999 >> v2:172.21.15.132:3300/0 conn(0x7f70c00ccb50 0x7f70c00cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b77fe700 1 -- 172.21.15.46:0/534513999 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783965531 0 0) 0x7f70c00cc150 con 0x7f70c015c770 2023-10-06T12:33:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 -- 172.21.15.46:0/534513999 >> v2:172.21.15.132:3300/0 conn(0x7f70c00ccb50 msgr2=0x7f70c00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 --2- 172.21.15.46:0/534513999 >> v2:172.21.15.132:3300/0 conn(0x7f70c00ccb50 0x7f70c00cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 -- 172.21.15.46:0/534513999 >> v1:172.21.15.46:6789/0 conn(0x7f70c015c770 legacy=0x7f70c00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 -- 172.21.15.46:0/534513999 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c00cb540 con 0x7f70c00c5ff0 2023-10-06T12:33:27.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b7fff700 1 --2- 172.21.15.46:0/534513999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c00ce4f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098e0 tx=0x7f70b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9bd181dd403d2eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b77fe700 1 -- 172.21.15.46:0/534513999 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000b640 con 0x7f70c00c5ff0 2023-10-06T12:33:27.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b77fe700 1 -- 172.21.15.46:0/534513999 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b000b7a0 con 0x7f70c00c5ff0 2023-10-06T12:33:27.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70b77fe700 1 -- 172.21.15.46:0/534513999 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000bb40 con 0x7f70c00c5ff0 2023-10-06T12:33:27.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 -- 172.21.15.46:0/534513999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 msgr2=0x7f70c00ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/534513999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c00ce4f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098e0 tx=0x7f70b00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 -- 172.21.15.46:0/534513999 shutdown_connections 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/534513999 >> v2:172.21.15.132:3300/0 conn(0x7f70c00ccb50 0x7f70c00cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/534513999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c00ce4f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 -- 172.21.15.46:0/534513999 >> 172.21.15.46:0/534513999 conn(0x7f70c0099480 msgr2=0x7f70c006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.036+0000 7f70c6c6e700 1 -- 172.21.15.46:0/534513999 shutdown_connections 2023-10-06T12:33:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.040+0000 7f70c6c6e700 1 -- 172.21.15.46:0/534513999 wait complete. 2023-10-06T12:33:27.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.040+0000 7f70c6c6e700 1 Processor -- start 2023-10-06T12:33:27.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.040+0000 7f70c6c6e700 1 -- start start 2023-10-06T12:33:27.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 0x7f70c015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c014b290 con 0x7f70c00ccb50 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c00b5e60 con 0x7f70c015c770 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c00b50a0 con 0x7f70c00c5ff0 2023-10-06T12:33:27.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34324/0 (socket says 172.21.15.46:34324) 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 -- 172.21.15.46:0/2861833802 learned_addr learned my addr 172.21.15.46:0/2861833802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c4a0a700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c520b700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 0x7f70c015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 -- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 msgr2=0x7f70c014ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c014ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 -- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 msgr2=0x7f70c015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 0x7f70c015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 -- 172.21.15.46:0/2861833802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0009420 con 0x7f70c00ccb50 2023-10-06T12:33:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c4a0a700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c520b700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 0x7f70c015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:27.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b7fff700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f70c0064360 tx=0x7f70b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000bb20 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0000c90 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c00b5380 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0011650 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.044+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c00b5940 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.048+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c00660f0 con 0x7f70c00ccb50 2023-10-06T12:33:27.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.048+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 71) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f70b0011a00 con 0x7f70c00ccb50 2023-10-06T12:33:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.052+0000 7f70b4ff9700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 0x7f70ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.052+0000 7f70c4a0a700 1 -- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 msgr2=0x7f70ac062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:27.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.052+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7f70b007e4c0 con 0x7f70c00ccb50 2023-10-06T12:33:27.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.052+0000 7f70c4a0a700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 0x7f70ac062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:27.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.056+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f70b004e410 con 0x7f70c00ccb50 2023-10-06T12:33:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.228+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c00cb820 con 0x7f70c00ccb50 2023-10-06T12:33:27.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70b4ff9700 1 -- 172.21.15.46:0/2861833802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+18252 (secure 0 0 0) 0x7f70b0011cb0 con 0x7f70c00ccb50 2023-10-06T12:33:27.239 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:27.239 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":141,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:24.221349+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:37:19.420143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T11:37:20.441003+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":141,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2667975390":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6833/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:6832/32974":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1202911604":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2705107049":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/2225233759":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/1098696528":"2023-10-07T11:37:35.094017+0000","172.21.15.46:0/888393785":"2023-10-07T11:37:35.094017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 msgr2=0x7f70ac062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 0x7f70ac062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 msgr2=0x7f70c0099df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f70c0064360 tx=0x7f70b0017000 comp rx=0 tx=0).stop 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 shutdown_connections 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f70ac060670 0x7f70ac062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70c015c770 0x7f70c015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c00ccb50 0x7f70c0099df0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 --2- 172.21.15.46:0/2861833802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70c00c5ff0 0x7f70c014ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.232+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 >> 172.21.15.46:0/2861833802 conn(0x7f70c0099480 msgr2=0x7f70c015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.236+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 shutdown_connections 2023-10-06T12:33:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.236+0000 7f70c6c6e700 1 -- 172.21.15.46:0/2861833802 wait complete. 2023-10-06T12:33:27.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T12:33:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 Processor -- start 2023-10-06T12:33:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 -- start start 2023-10-06T12:33:27.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb55c161470 0x7fb55c161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0c18f0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0cb120 con 0x7fb55c0c1d80 2023-10-06T12:33:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb563b45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c0c2660 con 0x7fb55c161470 2023-10-06T12:33:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.404+0000 7fb5620e2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb55c0c21e0 0x7fb55c0caa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33084/0 (socket says 172.21.15.46:33084) 2023-10-06T12:33:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5620e2700 1 -- 172.21.15.46:0/3327925160 learned_addr learned my addr 172.21.15.46:0/3327925160 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5618e1700 1 --2- 172.21.15.46:0/3327925160 >> v2:172.21.15.132:3300/0 conn(0x7fb55c161470 0x7fb55c161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212636753 0 0) 0x7fb55c0c18f0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb548003570 con 0x7fb55c0c21e0 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098602165 0 0) 0x7fb55c0cb120 con 0x7fb55c0c1d80 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb55c0c18f0 con 0x7fb55c0c1d80 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5618e1700 1 -- 172.21.15.46:0/3327925160 >> v1:172.21.15.132:6789/0 conn(0x7fb55c0c1d80 legacy=0x7fb55c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5618e1700 1 -- 172.21.15.46:0/3327925160 >> v1:172.21.15.46:6789/0 conn(0x7fb55c0c21e0 legacy=0x7fb55c0caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5618e1700 1 -- 172.21.15.46:0/3327925160 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0c0ce0 con 0x7fb55c161470 2023-10-06T12:33:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5618e1700 1 --2- 172.21.15.46:0/3327925160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c161470 0x7fb55c161840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098d0 tx=0x7fb558009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e91b5eb8637d23c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558005c10 con 0x7fb55c161470 2023-10-06T12:33:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb558005d70 con 0x7fb55c161470 2023-10-06T12:33:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb5608df700 1 -- 172.21.15.46:0/3327925160 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558006260 con 0x7fb55c161470 2023-10-06T12:33:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 -- 172.21.15.46:0/3327925160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c161470 msgr2=0x7fb55c161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 --2- 172.21.15.46:0/3327925160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c161470 0x7fb55c161840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098d0 tx=0x7fb558009120 comp rx=0 tx=0).stop 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 -- 172.21.15.46:0/3327925160 shutdown_connections 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 --2- 172.21.15.46:0/3327925160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c161470 0x7fb55c161840 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 -- 172.21.15.46:0/3327925160 >> 172.21.15.46:0/3327925160 conn(0x7fb55c0cbe30 msgr2=0x7fb55c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 -- 172.21.15.46:0/3327925160 shutdown_connections 2023-10-06T12:33:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.408+0000 7fb563b45700 1 -- 172.21.15.46:0/3327925160 wait complete. 2023-10-06T12:33:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 Processor -- start 2023-10-06T12:33:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 -- start start 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c0c1d80 0x7fb55c14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb55c161470 0x7fb55c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c16ea70 con 0x7fb55c0c21e0 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0af3c0 con 0x7fb55c0c1d80 2023-10-06T12:33:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb563b45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c16e790 con 0x7fb55c161470 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.412+0000 7fb5610e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34336/0 (socket says 172.21.15.46:34336) 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 -- 172.21.15.46:0/1493561227 learned_addr learned my addr 172.21.15.46:0/1493561227 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5620e2700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb55c161470 0x7fb55c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5618e1700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c0c1d80 0x7fb55c14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 -- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb55c161470 msgr2=0x7fb55c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb55c161470 0x7fb55c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 -- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c0c1d80 msgr2=0x7fb55c14d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c0c1d80 0x7fb55c14d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 -- 172.21.15.46:0/1493561227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5580093c0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb5610e0700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c790 tx=0x7fb54c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c010510 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c010670 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c0aed70 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c010a10 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c14d580 con 0x7fb55c0c21e0 2023-10-06T12:33:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.416+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c0aebd0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.420+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 72) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fb54c01b400 con 0x7fb55c0c21e0 2023-10-06T12:33:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.420+0000 7fb551ffb700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 0x7fb540062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.420+0000 7fb5618e1700 1 -- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 msgr2=0x7fb540062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.420+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6962+0+0 (secure 0 0 0) 0x7fb54c07cc50 con 0x7fb55c0c21e0 2023-10-06T12:33:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.420+0000 7fb5618e1700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 0x7fb540062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.428+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb55c0aebd0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.453 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.444+0000 7f9aa1765000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:33:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.464+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 73) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fb54c04cf70 con 0x7fb55c0c21e0 2023-10-06T12:33:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.464+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 msgr2=0x7fb540062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:27.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.464+0000 7fb551ffb700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 0x7fb540062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.559 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a6d3ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.559 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a6d3ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.559 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a6d3ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.560 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a6d3ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.560 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a6d3ac700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.560 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a69ba5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.560 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a69ba5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.561 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a69ba5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.561 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a69ba5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.561 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:27.552+0000 7f9a69ba5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.664+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb55c0aebd0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.671 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T12:33:27.664+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T12:33:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.672+0000 7fb551ffb700 1 -- 172.21.15.46:0/1493561227 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb55c0aebd0 con 0x7fb55c0c21e0 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 msgr2=0x7fb55c06d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00c790 tx=0x7fb54c007e10 comp rx=0 tx=0).stop 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 shutdown_connections 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb540060670 0x7fb540062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb55c161470 0x7fb55c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb55c0c21e0 0x7fb55c06d800 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 --2- 172.21.15.46:0/1493561227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb55c0c1d80 0x7fb55c14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 >> 172.21.15.46:0/1493561227 conn(0x7fb55c0cbe30 msgr2=0x7fb55c0c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 shutdown_connections 2023-10-06T12:33:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.676+0000 7fb563b45700 1 -- 172.21.15.46:0/1493561227 wait complete. 2023-10-06T12:33:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T12:33:27.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T12:33:27.743 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:33:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.856+0000 7fc03106e700 1 Processor -- start 2023-10-06T12:33:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.856+0000 7fc03106e700 1 -- start start 2023-10-06T12:33:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.856+0000 7fc03106e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 0x7fc02c0c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.856+0000 7fc03106e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc02c0c1bd0 0x7fc02c0c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c06de80 con 0x7fc02c0c2ab0 2023-10-06T12:33:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0c67a0 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c6d90 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc02c0c1bd0 0x7fc02c0c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02ad9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 0x7fc02c0c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02ad9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 0x7fc02c0c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:60152/0 (socket says 172.21.15.46:60152) 2023-10-06T12:33:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc02c0c1bd0 0x7fc02c0c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34480/0 (socket says 172.21.15.46:34480) 2023-10-06T12:33:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02ad9d700 1 -- 172.21.15.46:0/3533064635 learned_addr learned my addr 172.21.15.46:0/3533064635 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02b59e700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc02c0c2ab0 0x7fc02c155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55022/0 (socket says 172.21.15.46:55022) 2023-10-06T12:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 -- 172.21.15.46:0/3533064635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 msgr2=0x7fc02c0c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 --2- 172.21.15.46:0/3533064635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 0x7fc02c0c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 -- 172.21.15.46:0/3533064635 >> v1:172.21.15.132:6789/0 conn(0x7fc02c0c2ab0 legacy=0x7fc02c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 -- 172.21.15.46:0/3533064635 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0c7070 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc02a59c700 1 --2- 172.21.15.46:0/3533064635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c1bd0 0x7fc02c0c2570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098e0 tx=0x7fc0200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb45994c9f9d747 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc029d9b700 1 -- 172.21.15.46:0/3533064635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000b640 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc029d9b700 1 -- 172.21.15.46:0/3533064635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02000b7a0 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc029d9b700 1 -- 172.21.15.46:0/3533064635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000bb40 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- 172.21.15.46:0/3533064635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c1bd0 msgr2=0x7fc02c0c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 --2- 172.21.15.46:0/3533064635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c1bd0 0x7fc02c0c2570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098e0 tx=0x7fc0200091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- 172.21.15.46:0/3533064635 shutdown_connections 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 --2- 172.21.15.46:0/3533064635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c1bd0 0x7fc02c0c2570 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 --2- 172.21.15.46:0/3533064635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c0bd0 0x7fc02c0c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.860+0000 7fc03106e700 1 -- 172.21.15.46:0/3533064635 >> 172.21.15.46:0/3533064635 conn(0x7fc02c0cbe30 msgr2=0x7fc02c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.864+0000 7fc03106e700 1 -- 172.21.15.46:0/3533064635 shutdown_connections 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.864+0000 7fc03106e700 1 -- 172.21.15.46:0/3533064635 wait complete. 2023-10-06T12:33:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.864+0000 7fc03106e700 1 Processor -- start 2023-10-06T12:33:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.864+0000 7fc03106e700 1 -- start start 2023-10-06T12:33:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c1bd0 0x7fc02c145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 0x7fc02c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1580a0 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c155670 con 0x7fc02c0c2ab0 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c157770 con 0x7fc02c0c1bd0 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34488/0 (socket says 172.21.15.46:34488) 2023-10-06T12:33:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 -- 172.21.15.46:0/3860721629 learned_addr learned my addr 172.21.15.46:0/3860721629 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02a59c700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c1bd0 0x7fc02c145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02b59e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 0x7fc02c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 -- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c1bd0 msgr2=0x7fc02c145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c1bd0 0x7fc02c145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 -- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 msgr2=0x7fc02c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 0x7fc02c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 -- 172.21.15.46:0/3860721629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009420 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02b59e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 0x7fc02c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc02ad9d700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00ea50 tx=0x7fc01c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00eee0 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00c850 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00cbd0 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.868+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c157a50 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.872+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c1532b0 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.872+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0660f0 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.872+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 73) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fc01c013470 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.876+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7262+0+0 (secure 0 0 0) 0x7fc01c07cc60 con 0x7fc02c0c0bd0 2023-10-06T12:33:27.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:27.884+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc01c04cf10 con 0x7fc02c0c0bd0 2023-10-06T12:33:28.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.080+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc02c04b5c0 con 0x7fc02c0c0bd0 2023-10-06T12:33:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.472+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 74) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fc01c04c9c0 con 0x7fc02c0c0bd0 2023-10-06T12:33:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.472+0000 7fc012ffd700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 0x7fc018068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.476+0000 7fc02a59c700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 0x7fc018068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.480+0000 7fc02a59c700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 0x7fc018068e00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc0200091e0 tx=0x7fc020017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:28.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.676+0000 7fc012ffd700 1 -- 172.21.15.46:0/3860721629 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc01c068770 con 0x7fc02c0c0bd0 2023-10-06T12:33:28.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 msgr2=0x7fc018068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 0x7fc018068e00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc0200091e0 tx=0x7fc020017000 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 msgr2=0x7fc02c0c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00ea50 tx=0x7fc01c009250 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 shutdown_connections 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc018066a20 0x7fc018068e00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc02c0c2ab0 0x7fc02c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc02c0c1bd0 0x7fc02c145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 --2- 172.21.15.46:0/3860721629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc02c0c0bd0 0x7fc02c0c5910 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 >> 172.21.15.46:0/3860721629 conn(0x7fc02c0cbe30 msgr2=0x7fc02c0c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 shutdown_connections 2023-10-06T12:33:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.684+0000 7fc03106e700 1 -- 172.21.15.46:0/3860721629 wait complete. 2023-10-06T12:33:28.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T12:33:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.924+0000 7f81a60f0700 1 Processor -- start 2023-10-06T12:33:28.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.924+0000 7f81a60f0700 1 -- start start 2023-10-06T12:33:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f81a60f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f81a60f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f81a60f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00c71a0 con 0x7f81a0159050 2023-10-06T12:33:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f81a60f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a00c8650 con 0x7f81a0159420 2023-10-06T12:33:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f81a60f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00c1bd0 con 0x7f81a00ca3a0 2023-10-06T12:33:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:52010/0 (socket says 172.21.15.46:52010) 2023-10-06T12:33:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:60158/0 (socket says 172.21.15.46:60158) 2023-10-06T12:33:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819ffff700 1 -- 172.21.15.46:0/781636352 learned_addr learned my addr 172.21.15.46:0/781636352 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 -- 172.21.15.46:0/781636352 >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 msgr2=0x7f81a00c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 --2- 172.21.15.46:0/781636352 >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 -- 172.21.15.46:0/781636352 >> v1:172.21.15.132:6789/0 conn(0x7f81a0159050 legacy=0x7f81a00c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 -- 172.21.15.46:0/781636352 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00c6590 con 0x7f81a00ca3a0 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819ffff700 1 --2- 172.21.15.46:0/781636352 >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819effd700 1 --2- 172.21.15.46:0/781636352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81940098e0 tx=0x7f81940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d237d336553b5960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819e7fc700 1 -- 172.21.15.46:0/781636352 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400b640 con 0x7f81a00ca3a0 2023-10-06T12:33:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819e7fc700 1 -- 172.21.15.46:0/781636352 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819400b7a0 con 0x7f81a00ca3a0 2023-10-06T12:33:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.928+0000 7f819e7fc700 1 -- 172.21.15.46:0/781636352 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819400bb40 con 0x7f81a00ca3a0 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- 172.21.15.46:0/781636352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 msgr2=0x7f81a00c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 --2- 172.21.15.46:0/781636352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81940098e0 tx=0x7f81940091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- 172.21.15.46:0/781636352 shutdown_connections 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 --2- 172.21.15.46:0/781636352 >> v2:172.21.15.132:3300/0 conn(0x7f81a0159420 0x7f81a00c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 --2- 172.21.15.46:0/781636352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00c7bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- 172.21.15.46:0/781636352 >> 172.21.15.46:0/781636352 conn(0x7f81a00cbe20 msgr2=0x7f81a00cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- 172.21.15.46:0/781636352 shutdown_connections 2023-10-06T12:33:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- 172.21.15.46:0/781636352 wait complete. 2023-10-06T12:33:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 Processor -- start 2023-10-06T12:33:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.932+0000 7f81a60f0700 1 -- start start 2023-10-06T12:33:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 0x7f81a0153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0153fd0 con 0x7f81a0159050 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ad310 con 0x7f81a0159420 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ac700 con 0x7f81a00ca3a0 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34496/0 (socket says 172.21.15.46:34496) 2023-10-06T12:33:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 -- 172.21.15.46:0/124547903 learned_addr learned my addr 172.21.15.46:0/124547903 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819f7fe700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 -- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 msgr2=0x7f81a00aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 -- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 msgr2=0x7f81a0153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819ffff700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 0x7f81a0153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 0x7f81a0153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 -- 172.21.15.46:0/124547903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198009ae0 con 0x7f81a0159050 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819f7fe700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819ffff700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 0x7f81a0153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f819effd700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f81a00c2620 tx=0x7f8194017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819401bda0 con 0x7f81a0159050 2023-10-06T12:33:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194011420 con 0x7f81a0159050 2023-10-06T12:33:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8194009420 con 0x7f81a0159050 2023-10-06T12:33:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940117e0 con 0x7f81a0159050 2023-10-06T12:33:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.936+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a00accc0 con 0x7f81a0159050 2023-10-06T12:33:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.940+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a00ad4a0 con 0x7f81a0159050 2023-10-06T12:33:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.940+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 74) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f8194011b90 con 0x7f81a0159050 2023-10-06T12:33:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.940+0000 7f81877fe700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 0x7f818c062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:28.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.940+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 7262+0+0 (secure 0 0 0) 0x7f819407e200 con 0x7f81a0159050 2023-10-06T12:33:28.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.944+0000 7f819f7fe700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 0x7f818c062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.944+0000 7f819f7fe700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 0x7f818c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f81a00aa5a0 tx=0x7f8190009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:28.948+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f819404e3c0 con 0x7f81a0159050 2023-10-06T12:33:29.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.124+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81a006eed0 con 0x7f81a0159050 2023-10-06T12:33:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.684+0000 7f81877fe700 1 -- 172.21.15.46:0/124547903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 181+0+0 (secure 0 0 0) 0x7f81940519e0 con 0x7f81a0159050 2023-10-06T12:33:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 msgr2=0x7f818c062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 0x7f818c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f81a00aa5a0 tx=0x7f8190009040 comp rx=0 tx=0).stop 2023-10-06T12:33:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 msgr2=0x7f81a01559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f81a00c2620 tx=0x7f8194017000 comp rx=0 tx=0).stop 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 shutdown_connections 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f818c0603a0 0x7f818c062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a0159420 0x7f81a0153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81a0159050 0x7f81a01559c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 --2- 172.21.15.46:0/124547903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81a00ca3a0 0x7f81a00aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 >> 172.21.15.46:0/124547903 conn(0x7f81a00cbe20 msgr2=0x7f81a00aaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 shutdown_connections 2023-10-06T12:33:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.688+0000 7f81a60f0700 1 -- 172.21.15.46:0/124547903 wait complete. 2023-10-06T12:33:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T12:33:29.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T12:33:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.884+0000 7f33812a5700 1 Processor -- start 2023-10-06T12:33:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.884+0000 7f33812a5700 1 -- start start 2023-10-06T12:33:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.888+0000 7f33812a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c1596a0 0x7f337c0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.888+0000 7f33812a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337c0c5db0 con 0x7f337c0c6780 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.888+0000 7f33812a5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337c0c5ab0 con 0x7f337c1592d0 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.888+0000 7f33812a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0d0aa0 con 0x7f337c1596a0 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337affd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f337c1592d0 0x7f337c0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55038/0 (socket says 172.21.15.46:55038) 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337affd700 1 -- 172.21.15.46:0/288902557 learned_addr learned my addr 172.21.15.46:0/288902557 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f337c0c6780 0x7f337c0ce040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33098/0 (socket says 172.21.15.46:33098) 2023-10-06T12:33:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c1596a0 0x7f337c0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201208243 0 0) 0x7f337c0c5db0 con 0x7f337c0c6780 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3364003570 con 0x7f337c0c6780 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463092620 0 0) 0x7f337c0c5ab0 con 0x7f337c1592d0 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f337c0c5db0 con 0x7f337c1592d0 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2743127292 0 0) 0x7f3364003570 con 0x7f337c0c6780 2023-10-06T12:33:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c1596a0 msgr2=0x7f337c0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c1596a0 0x7f337c0c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 >> v1:172.21.15.132:6789/0 conn(0x7f337c1592d0 legacy=0x7f337c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c0d0d80 con 0x7f337c0c6780 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f33740017f0 con 0x7f337c0c6780 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 >> v1:172.21.15.46:6789/0 conn(0x7f337c0c6780 legacy=0x7f337c0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3364004c80 0x7f3364005070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640037b0 0x7f3364003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33640041b0 0x7f336400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0d0aa0 con 0x7f33640037b0 2023-10-06T12:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336400c160 con 0x7f3364004c80 2023-10-06T12:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336400c320 con 0x7f33640041b0 2023-10-06T12:33:29.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640037b0 0x7f3364003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337affd700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3364004c80 0x7f3364005070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337b7fe700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33640041b0 0x7f336400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 -- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33640041b0 msgr2=0x7f336400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33640041b0 0x7f336400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 -- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3364004c80 msgr2=0x7f3364005070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3364004c80 0x7f3364005070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 -- 172.21.15.46:0/288902557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364003570 con 0x7f33640037b0 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.892+0000 7f337a7fc700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640037b0 0x7f3364003c20 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f337c06a230 tx=0x7f3370009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43077752326d0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000e610 con 0x7f33640037b0 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337000e770 con 0x7f33640037b0 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f3379ffb700 1 -- 172.21.15.46:0/288902557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000eb10 con 0x7f33640037b0 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 -- 172.21.15.46:0/288902557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640037b0 msgr2=0x7f3364003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 --2- 172.21.15.46:0/288902557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640037b0 0x7f3364003c20 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f337c06a230 tx=0x7f3370009120 comp rx=0 tx=0).stop 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f337b7fe700 1 -- 172.21.15.46:0/288902557 reap_dead start 2023-10-06T12:33:29.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 -- 172.21.15.46:0/288902557 shutdown_connections 2023-10-06T12:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 -- 172.21.15.46:0/288902557 >> 172.21.15.46:0/288902557 conn(0x7f337c06ef60 msgr2=0x7f337c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 -- 172.21.15.46:0/288902557 shutdown_connections 2023-10-06T12:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.896+0000 7f33812a5700 1 -- 172.21.15.46:0/288902557 wait complete. 2023-10-06T12:33:29.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 Processor -- start 2023-10-06T12:33:29.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 -- start start 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33640037b0 0x7f337c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c153190 0x7f337c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0b4320 con 0x7f337c16cb20 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c155480 con 0x7f33640037b0 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f33812a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c155760 con 0x7f337c153190 2023-10-06T12:33:29.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34522/0 (socket says 172.21.15.46:34522) 2023-10-06T12:33:29.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 -- 172.21.15.46:0/330161249 learned_addr learned my addr 172.21.15.46:0/330161249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337b7fe700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c153190 0x7f337c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337affd700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33640037b0 0x7f337c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 -- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c153190 msgr2=0x7f337c151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c153190 0x7f337c151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 -- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33640037b0 msgr2=0x7f337c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33640037b0 0x7f337c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:29.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.900+0000 7f337a7fc700 1 -- 172.21.15.46:0/330161249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c155a40 con 0x7f337c16cb20 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f337a7fc700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f337c0d1350 tx=0x7f337001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370009ed0 con 0x7f337c16cb20 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370022040 con 0x7f337c16cb20 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000eb10 con 0x7f337c16cb20 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c155d20 con 0x7f337c16cb20 2023-10-06T12:33:29.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c151f60 con 0x7f337c16cb20 2023-10-06T12:33:29.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c069e00 con 0x7f337c16cb20 2023-10-06T12:33:29.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.904+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 74) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f337001c660 con 0x7f337c16cb20 2023-10-06T12:33:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.908+0000 7f3362ffd700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 0x7f3368062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:29.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.908+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f337007dfe0 con 0x7f337c16cb20 2023-10-06T12:33:29.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.908+0000 7f337affd700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 0x7f3368062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.908+0000 7f337affd700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 0x7f3368062880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f336c004fd0 tx=0x7f336c007e70 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:29.916+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f337004e370 con 0x7f337c16cb20 2023-10-06T12:33:30.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.092+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f337c0d1060 con 0x7f337c16cb20 2023-10-06T12:33:30.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.200+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f337004e740 con 0x7f337c16cb20 2023-10-06T12:33:30.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.696+0000 7f3362ffd700 1 -- 172.21.15.46:0/330161249 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3370051e00 con 0x7f337c16cb20 2023-10-06T12:33:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 msgr2=0x7f3368062880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 0x7f3368062880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f336c004fd0 tx=0x7f336c007e70 comp rx=0 tx=0).stop 2023-10-06T12:33:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 msgr2=0x7f337c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f337c0d1350 tx=0x7f337001a000 comp rx=0 tx=0).stop 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 shutdown_connections 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33680603d0 0x7f3368062880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f337c153190 0x7f337c151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f337c16cb20 0x7f337c152c50 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 --2- 172.21.15.46:0/330161249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33640037b0 0x7f337c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 >> 172.21.15.46:0/330161249 conn(0x7f337c06ef60 msgr2=0x7f337c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:30.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 shutdown_connections 2023-10-06T12:33:30.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.700+0000 7f33812a5700 1 -- 172.21.15.46:0/330161249 wait complete. 2023-10-06T12:33:30.711 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T12:33:30.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T12:33:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 Processor -- start 2023-10-06T12:33:30.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 -- start start 2023-10-06T12:33:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29c8159240 0x7f29c80a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80c10a0 con 0x7f29c8158e70 2023-10-06T12:33:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80ca240 con 0x7f29c80c72d0 2023-10-06T12:33:30.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29cd44c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c80ca520 con 0x7f29c8159240 2023-10-06T12:33:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29bf7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29c8159240 0x7f29c80a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29c7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29c8158e70 0x7f29c80c99f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33102/0 (socket says 172.21.15.46:33102) 2023-10-06T12:33:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29c7fff700 1 -- 172.21.15.46:0/996033966 learned_addr learned my addr 172.21.15.46:0/996033966 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:30.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.888+0000 7f29c77fe700 1 -- 172.21.15.46:0/996033966 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224245720 0 0) 0x7f29c80c10a0 con 0x7f29c8158e70 2023-10-06T12:33:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29c77fe700 1 -- 172.21.15.46:0/996033966 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b4003570 con 0x7f29c8158e70 2023-10-06T12:33:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29bf7fe700 1 -- 172.21.15.46:0/996033966 >> v1:172.21.15.132:6789/0 conn(0x7f29c80c72d0 legacy=0x7f29c80c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29bf7fe700 1 -- 172.21.15.46:0/996033966 >> v1:172.21.15.46:6789/0 conn(0x7f29c8158e70 legacy=0x7f29c80c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29bf7fe700 1 -- 172.21.15.46:0/996033966 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80c0a50 con 0x7f29c8159240 2023-10-06T12:33:30.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29bf7fe700 1 --2- 172.21.15.46:0/996033966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80a9840 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f29b00098e0 tx=0x7f29b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b2093d6e348ad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29c77fe700 1 -- 172.21.15.46:0/996033966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000b640 con 0x7f29c8159240 2023-10-06T12:33:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29c77fe700 1 -- 172.21.15.46:0/996033966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b000b7a0 con 0x7f29c8159240 2023-10-06T12:33:30.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29c77fe700 1 -- 172.21.15.46:0/996033966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000bb40 con 0x7f29c8159240 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 -- 172.21.15.46:0/996033966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 msgr2=0x7f29c80a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 --2- 172.21.15.46:0/996033966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80a9840 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f29b00098e0 tx=0x7f29b00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 -- 172.21.15.46:0/996033966 shutdown_connections 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 --2- 172.21.15.46:0/996033966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80a9840 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 -- 172.21.15.46:0/996033966 >> 172.21.15.46:0/996033966 conn(0x7f29c80cbc10 msgr2=0x7f29c80cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 -- 172.21.15.46:0/996033966 shutdown_connections 2023-10-06T12:33:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.892+0000 7f29cd44c700 1 -- 172.21.15.46:0/996033966 wait complete. 2023-10-06T12:33:30.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 Processor -- start 2023-10-06T12:33:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 -- start start 2023-10-06T12:33:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29c80c72d0 0x7f29c814bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29c8158e70 0x7f29c814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c814c220 con 0x7f29c8159240 2023-10-06T12:33:30.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806e7e0 con 0x7f29c80c72d0 2023-10-06T12:33:30.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29cd44c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81568f0 con 0x7f29c8158e70 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29ccc4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29ccc4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34546/0 (socket says 172.21.15.46:34546) 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29ccc4b700 1 -- 172.21.15.46:0/2091128486 learned_addr learned my addr 172.21.15.46:0/2091128486 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29c7fff700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29c80c72d0 0x7f29c814bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.896+0000 7f29bf7fe700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29c8158e70 0x7f29c814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 -- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29c8158e70 msgr2=0x7f29c814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29c8158e70 0x7f29c814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 -- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29c80c72d0 msgr2=0x7f29c814bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29c80c72d0 0x7f29c814bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 -- 172.21.15.46:0/2091128486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0009420 con 0x7f29c8159240 2023-10-06T12:33:30.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29ccc4b700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c910 tx=0x7f29c0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000a740 con 0x7f29c8159240 2023-10-06T12:33:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c000a8a0 con 0x7f29c8159240 2023-10-06T12:33:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c000ac60 con 0x7f29c8159240 2023-10-06T12:33:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c806df40 con 0x7f29c8159240 2023-10-06T12:33:30.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c806e500 con 0x7f29c8159240 2023-10-06T12:33:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.900+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c8064590 con 0x7f29c8159240 2023-10-06T12:33:30.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.904+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f29c0012420 con 0x7f29c8159240 2023-10-06T12:33:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.904+0000 7f29c4ff9700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 0x7f29a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.904+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f29c007cf70 con 0x7f29c8159240 2023-10-06T12:33:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.904+0000 7f29c7fff700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 0x7f29a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.904+0000 7f29c7fff700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 0x7f29a8062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29b80058d0 tx=0x7f29b8009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:30.912+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f29c004d0f0 con 0x7f29c8159240 2023-10-06T12:33:31.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.084+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f29c8156a30 con 0x7f29c8159240 2023-10-06T12:33:31.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.092+0000 7f29c4ff9700 1 -- 172.21.15.46:0/2091128486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29c0050710 con 0x7f29c8159240 2023-10-06T12:33:31.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 msgr2=0x7f29a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 0x7f29a8062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f29b80058d0 tx=0x7f29b8009040 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 msgr2=0x7f29c80b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c910 tx=0x7f29c0001710 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 shutdown_connections 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f29a8060670 0x7f29a8062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29c8159240 0x7f29c80b0dc0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29c8158e70 0x7f29c814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 --2- 172.21.15.46:0/2091128486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29c80c72d0 0x7f29c814bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.096+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 >> 172.21.15.46:0/2091128486 conn(0x7f29c80cbc10 msgr2=0x7f29c80a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:31.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.100+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 shutdown_connections 2023-10-06T12:33:31.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.100+0000 7f29cd44c700 1 -- 172.21.15.46:0/2091128486 wait complete. 2023-10-06T12:33:31.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T12:33:31.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 Processor -- start 2023-10-06T12:33:31.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 -- start start 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa510164210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5100cb4a0 con 0x7fa5100ccc00 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5100cb6e0 con 0x7fa5100b23f0 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa517aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100cc780 con 0x7fa5100cebe0 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa515049700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5100ccc00 0x7fa5100ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33104/0 (socket says 172.21.15.46:33104) 2023-10-06T12:33:31.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa515049700 1 -- 172.21.15.46:0/581112291 learned_addr learned my addr 172.21.15.46:0/581112291 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:31.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa51604b700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa510164210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277242675 0 0) 0x7fa500001110 con 0x7fa5100ccc00 2023-10-06T12:33:31.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4fc003570 con 0x7fa5100ccc00 2023-10-06T12:33:31.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.288+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704151418 0 0) 0x7fa5100cb6e0 con 0x7fa5100b23f0 2023-10-06T12:33:31.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa500001110 con 0x7fa5100b23f0 2023-10-06T12:33:31.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625651332 0 0) 0x7fa4fc003570 con 0x7fa5100ccc00 2023-10-06T12:33:31.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 msgr2=0x7fa510164210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa510164210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 >> v1:172.21.15.132:6789/0 conn(0x7fa5100b23f0 legacy=0x7fa5100cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100cb9c0 con 0x7fa5100ccc00 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fa500001620 con 0x7fa5100ccc00 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 >> v1:172.21.15.46:6789/0 conn(0x7fa5100ccc00 legacy=0x7fa5100ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa4fc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa4fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4fc004d50 0x7fa4fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc00bbf0 con 0x7fa4fc003c20 2023-10-06T12:33:31.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc00bd30 con 0x7fa4fc004d50 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc00bef0 con 0x7fa5100cebe0 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa4fc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa515049700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa4fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51604b700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4fc004d50 0x7fa4fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 -- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 msgr2=0x7fa4fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5100cebe0 0x7fa4fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 -- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4fc004d50 msgr2=0x7fa4fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4fc004d50 0x7fa4fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 -- 172.21.15.46:0/581112291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc003570 con 0x7fa4fc003c20 2023-10-06T12:33:31.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51584a700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa4fc004010 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af336d8675ec3d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0013f0 con 0x7fa4fc003c20 2023-10-06T12:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c00c5d0 con 0x7fa4fc003c20 2023-10-06T12:33:31.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa514848700 1 -- 172.21.15.46:0/581112291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00c990 con 0x7fa4fc003c20 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 -- 172.21.15.46:0/581112291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 msgr2=0x7fa4fc004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 --2- 172.21.15.46:0/581112291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa4fc004010 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c007d80 comp rx=0 tx=0).stop 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa51604b700 1 -- 172.21.15.46:0/581112291 reap_dead start 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 -- 172.21.15.46:0/581112291 shutdown_connections 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 -- 172.21.15.46:0/581112291 >> 172.21.15.46:0/581112291 conn(0x7fa510099440 msgr2=0x7fa51006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 -- 172.21.15.46:0/581112291 shutdown_connections 2023-10-06T12:33:31.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.292+0000 7fa517aae700 1 -- 172.21.15.46:0/581112291 wait complete. 2023-10-06T12:33:31.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.296+0000 7fa517aae700 1 Processor -- start 2023-10-06T12:33:31.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.296+0000 7fa517aae700 1 -- start start 2023-10-06T12:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 0x7fa51015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa51015a1c0 0x7fa51015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51014b380 con 0x7fa4fc003c20 2023-10-06T12:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015ab70 con 0x7fa51015a1c0 2023-10-06T12:33:31.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015c060 con 0x7fa510169a30 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34564/0 (socket says 172.21.15.46:34564) 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 -- 172.21.15.46:0/2779003609 learned_addr learned my addr 172.21.15.46:0/2779003609 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51604b700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa51015a1c0 0x7fa51015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa515049700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 0x7fa51015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 -- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 msgr2=0x7fa51015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 0x7fa51015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 -- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa51015a1c0 msgr2=0x7fa51015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa51015a1c0 0x7fa51015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 -- 172.21.15.46:0/2779003609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51015ec40 con 0x7fa4fc003c20 2023-10-06T12:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa515049700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 0x7fa51015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:31.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa51584a700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c3c0 tx=0x7fa50c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c024070 con 0x7fa4fc003c20 2023-10-06T12:33:31.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c001030 con 0x7fa4fc003c20 2023-10-06T12:33:31.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00c970 con 0x7fa4fc003c20 2023-10-06T12:33:31.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa510169f50 con 0x7fa4fc003c20 2023-10-06T12:33:31.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.300+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51016a510 con 0x7fa4fc003c20 2023-10-06T12:33:31.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.304+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5100660f0 con 0x7fa4fc003c20 2023-10-06T12:33:31.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.304+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa50c006a80 con 0x7fa4fc003c20 2023-10-06T12:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.304+0000 7fa505ffb700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 0x7fa4f4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.308+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa50c07df00 con 0x7fa4fc003c20 2023-10-06T12:33:31.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.308+0000 7fa515049700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 0x7fa4f4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.308+0000 7fa515049700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 0x7fa4f4062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa51009e270 tx=0x7fa500007040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.312+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa50c04e130 con 0x7fa4fc003c20 2023-10-06T12:33:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.480+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fa5100c87d0 con 0x7fa4fc003c20 2023-10-06T12:33:31.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.492+0000 7fa505ffb700 1 -- 172.21.15.46:0/2779003609 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa50c017020 con 0x7fa4fc003c20 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 msgr2=0x7fa4f4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 0x7fa4f4062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa51009e270 tx=0x7fa500007040 comp rx=0 tx=0).stop 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 msgr2=0x7fa51014ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c3c0 tx=0x7fa50c015000 comp rx=0 tx=0).stop 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 shutdown_connections 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa4f40606a0 0x7fa4f4062b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa51015a1c0 0x7fa51015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa510169a30 0x7fa51015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 --2- 172.21.15.46:0/2779003609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4fc003c20 0x7fa51014ae40 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 >> 172.21.15.46:0/2779003609 conn(0x7fa510099440 msgr2=0x7fa510068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 shutdown_connections 2023-10-06T12:33:31.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.496+0000 7fa517aae700 1 -- 172.21.15.46:0/2779003609 wait complete. 2023-10-06T12:33:31.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T12:33:31.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.692+0000 7fdf1228e700 1 Processor -- start 2023-10-06T12:33:31.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.692+0000 7fdf1228e700 1 -- start start 2023-10-06T12:33:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf0c0b14d0 0x7fdf0c0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0afd20 con 0x7fdf0c0adbe0 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0b0000 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c159740 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1128c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf0c0adbe0 0x7fdf0c0d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33118/0 (socket says 172.21.15.46:33118) 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1128c700 1 -- 172.21.15.46:0/3998780118 learned_addr learned my addr 172.21.15.46:0/3998780118 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf11a8d700 1 --2- 172.21.15.46:0/3998780118 >> v2:172.21.15.46:3300/0 conn(0x7fdf0c0b14d0 0x7fdf0c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf11a8d700 1 -- 172.21.15.46:0/3998780118 >> v1:172.21.15.132:6789/0 conn(0x7fdf0c0b30f0 legacy=0x7fdf0c0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf11a8d700 1 -- 172.21.15.46:0/3998780118 >> v1:172.21.15.46:6789/0 conn(0x7fdf0c0adbe0 legacy=0x7fdf0c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf11a8d700 1 -- 172.21.15.46:0/3998780118 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0b02e0 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf11a8d700 1 --2- 172.21.15.46:0/3998780118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b14d0 0x7fdf0c0adfb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009e60 tx=0x7fdf04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da07307733e4992d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf03fff700 1 -- 172.21.15.46:0/3998780118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400e070 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf03fff700 1 -- 172.21.15.46:0/3998780118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0400bbf0 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf03fff700 1 -- 172.21.15.46:0/3998780118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04012430 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- 172.21.15.46:0/3998780118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b14d0 msgr2=0x7fdf0c0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 --2- 172.21.15.46:0/3998780118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b14d0 0x7fdf0c0adfb0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009e60 tx=0x7fdf04009780 comp rx=0 tx=0).stop 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- 172.21.15.46:0/3998780118 shutdown_connections 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 --2- 172.21.15.46:0/3998780118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b14d0 0x7fdf0c0adfb0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- 172.21.15.46:0/3998780118 >> 172.21.15.46:0/3998780118 conn(0x7fdf0c06ef60 msgr2=0x7fdf0c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- 172.21.15.46:0/3998780118 shutdown_connections 2023-10-06T12:33:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.696+0000 7fdf1228e700 1 -- 172.21.15.46:0/3998780118 wait complete. 2023-10-06T12:33:31.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.700+0000 7fdf1228e700 1 Processor -- start 2023-10-06T12:33:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.700+0000 7fdf1228e700 1 -- start start 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 0x7fdf0c16a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf0c0b14d0 0x7fdf0c15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c15a810 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c159920 con 0x7fdf0c0adbe0 2023-10-06T12:33:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c159c00 con 0x7fdf0c0b14d0 2023-10-06T12:33:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1128c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 0x7fdf0c16a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34578/0 (socket says 172.21.15.46:34578) 2023-10-06T12:33:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 -- 172.21.15.46:0/2441311907 learned_addr learned my addr 172.21.15.46:0/2441311907 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf10a8b700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf0c0b14d0 0x7fdf0c15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 -- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf0c0b14d0 msgr2=0x7fdf0c15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf0c0b14d0 0x7fdf0c15b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 -- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 msgr2=0x7fdf0c16a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 0x7fdf0c16a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 -- 172.21.15.46:0/2441311907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009a20 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1128c700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 0x7fdf0c16a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf11a8d700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdf04003000 tx=0x7fdf04018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04012e90 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0401e640 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0401ea00 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c159ee0 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.704+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c15a4a0 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.708+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c15a9c0 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.708+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fdf04024070 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.708+0000 7fdf017fa700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 0x7fdef0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.708+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf0407f6d0 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.708+0000 7fdf1128c700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 0x7fdef0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:31.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.712+0000 7fdf1128c700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 0x7fdef0062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf08007110 tx=0x7fdf08007f10 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.716+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdf0c15a9c0 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:33:31.759 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:33:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.888+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fdf0c064590 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.896+0000 7fdf017fa700 1 -- 172.21.15.46:0/2441311907 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdf0404f850 con 0x7fdf0c0b30f0 2023-10-06T12:33:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.896+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 msgr2=0x7fdef0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.896+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 0x7fdef0062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf08007110 tx=0x7fdf08007f10 comp rx=0 tx=0).stop 2023-10-06T12:33:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.896+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 msgr2=0x7fdf0c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.896+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdf04003000 tx=0x7fdf04018000 comp rx=0 tx=0).stop 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 shutdown_connections 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fdef0060670 0x7fdef0062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c0b30f0 0x7fdf0c15bee0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf0c0b14d0 0x7fdf0c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 --2- 172.21.15.46:0/2441311907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf0c0adbe0 0x7fdf0c16a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 >> 172.21.15.46:0/2441311907 conn(0x7fdf0c06ef60 msgr2=0x7fdf0c14e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 shutdown_connections 2023-10-06T12:33:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:31.900+0000 7fdf1228e700 1 -- 172.21.15.46:0/2441311907 wait complete. 2023-10-06T12:33:31.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-06T12:33:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.076+0000 7f8f12816700 1 Processor -- start 2023-10-06T12:33:32.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.076+0000 7f8f12816700 1 -- start start 2023-10-06T12:33:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12816700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12816700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0ae340 con 0x7f8f0c0b1130 2023-10-06T12:33:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12816700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b15b0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0b09a0 con 0x7f8f0c159290 2023-10-06T12:33:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11814700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:52072/0 (socket says 172.21.15.46:52072) 2023-10-06T12:33:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f12015700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8f0c0b1130 0x7f8f0c0ae7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:55080/0 (socket says 172.21.15.46:55080) 2023-10-06T12:33:32.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 -- 172.21.15.46:0/2598557174 learned_addr learned my addr 172.21.15.46:0/2598557174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 -- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 msgr2=0x7f8f0c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 --2- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c0b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 -- 172.21.15.46:0/2598557174 >> v1:172.21.15.132:6789/0 conn(0x7f8f0c0b1130 legacy=0x7f8f0c0ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 -- 172.21.15.46:0/2598557174 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c159660 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f11013700 1 --2- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0099b0 tx=0x7f8efc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2919f8662b9eacf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f10812700 1 -- 172.21.15.46:0/2598557174 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc009ea0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f10812700 1 -- 172.21.15.46:0/2598557174 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00b8f0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.080+0000 7f8f10812700 1 -- 172.21.15.46:0/2598557174 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00bc90 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 -- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0d07b0 msgr2=0x7f8f0c0b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 --2- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0099b0 tx=0x7f8efc009250 comp rx=0 tx=0).stop 2023-10-06T12:33:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 -- 172.21.15.46:0/2598557174 shutdown_connections 2023-10-06T12:33:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 --2- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c0b0bf0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 --2- 172.21.15.46:0/2598557174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c0b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 -- 172.21.15.46:0/2598557174 >> 172.21.15.46:0/2598557174 conn(0x7f8f0c06ef60 msgr2=0x7f8f0c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:32.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 -- 172.21.15.46:0/2598557174 shutdown_connections 2023-10-06T12:33:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.084+0000 7f8f12816700 1 -- 172.21.15.46:0/2598557174 wait complete. 2023-10-06T12:33:32.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 Processor -- start 2023-10-06T12:33:32.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 -- start start 2023-10-06T12:33:32.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0b1130 0x7f8f0c152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c156bc0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1551a0 con 0x7f8f0c0b1130 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c16a330 con 0x7f8f0c159290 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f11013700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f11013700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34586/0 (socket says 172.21.15.46:34586) 2023-10-06T12:33:32.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f11013700 1 -- 172.21.15.46:0/1763289809 learned_addr learned my addr 172.21.15.46:0/1763289809 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:32.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f12015700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.088+0000 7f8f11814700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0b1130 0x7f8f0c152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 -- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 msgr2=0x7f8f0c13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c13fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 -- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0b1130 msgr2=0x7f8f0c152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0b1130 0x7f8f0c152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 -- 172.21.15.46:0/1763289809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0094f0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f11013700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c088920 tx=0x7f8efc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00ba60 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00bbe0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.092+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c13ff60 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.096+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc025670 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.096+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c169a00 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.096+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c064590 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.096+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8efc023070 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.100+0000 7f8f01ffb700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 0x7f8ef0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.100+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8efc07e390 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.100+0000 7f8f11814700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 0x7f8ef0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.100+0000 7f8f11814700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 0x7f8ef0062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009d40 tx=0x7f8f08008040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.108+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8efc04e510 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.336+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f8f0c0675d0 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.700+0000 7f8f01ffb700 1 -- 172.21.15.46:0/1763289809 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2422) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8efc051b30 con 0x7f8f0c0d07b0 2023-10-06T12:33:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 msgr2=0x7f8ef0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 0x7f8ef0062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009d40 tx=0x7f8f08008040 comp rx=0 tx=0).stop 2023-10-06T12:33:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 msgr2=0x7f8f0c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c088920 tx=0x7f8efc017000 comp rx=0 tx=0).stop 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 shutdown_connections 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8ef0060670 0x7f8ef0062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f0c159290 0x7f8f0c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0c0d07b0 0x7f8f0c154c60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 --2- 172.21.15.46:0/1763289809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0c0b1130 0x7f8f0c152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 >> 172.21.15.46:0/1763289809 conn(0x7f8f0c06ef60 msgr2=0x7f8f0c067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 shutdown_connections 2023-10-06T12:33:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.704+0000 7f8f12816700 1 -- 172.21.15.46:0/1763289809 wait complete. 2023-10-06T12:33:32.734 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-06T12:33:32.734 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-06T12:33:32.735 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T12:33:32.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-06T12:33:32.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 Processor -- start 2023-10-06T12:33:32.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 -- start start 2023-10-06T12:33:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 0x7ff3a4161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3a40c1d80 0x7ff3a40c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40c18f0 con 0x7ff3a40c1d80 2023-10-06T12:33:32.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40c0ce0 con 0x7ff3a4161470 2023-10-06T12:33:32.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.900+0000 7ff3abdc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c0fc0 con 0x7ff3a406cc30 2023-10-06T12:33:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3a40c1d80 0x7ff3a40c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3a40c1d80 0x7ff3a40c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34600/0 (socket says 172.21.15.46:34600) 2023-10-06T12:33:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 -- 172.21.15.46:0/1890881059 learned_addr learned my addr 172.21.15.46:0/1890881059 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:32.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 -- 172.21.15.46:0/1890881059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 msgr2=0x7ff3a406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9b61700 1 --2- 172.21.15.46:0/1890881059 >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 0x7ff3a4161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3aa362700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a406d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 -- 172.21.15.46:0/1890881059 >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 msgr2=0x7ff3a4161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 --2- 172.21.15.46:0/1890881059 >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 0x7ff3a4161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 -- 172.21.15.46:0/1890881059 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40c12a0 con 0x7ff3a40c1d80 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9b61700 1 --2- 172.21.15.46:0/1890881059 >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 0x7ff3a4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3aa362700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a9360700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c21e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4063c10 tx=0x7ff394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d4d1712eb410da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a8b5f700 1 -- 172.21.15.46:0/1890881059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39400b8f0 con 0x7ff3a40c1d80 2023-10-06T12:33:32.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a8b5f700 1 -- 172.21.15.46:0/1890881059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39400ba50 con 0x7ff3a40c1d80 2023-10-06T12:33:32.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3a8b5f700 1 -- 172.21.15.46:0/1890881059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39400bdf0 con 0x7ff3a40c1d80 2023-10-06T12:33:32.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 -- 172.21.15.46:0/1890881059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a40c1d80 msgr2=0x7ff3a40c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c21e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4063c10 tx=0x7ff394009250 comp rx=0 tx=0).stop 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 -- 172.21.15.46:0/1890881059 shutdown_connections 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/1890881059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c21e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/1890881059 >> v2:172.21.15.132:3300/0 conn(0x7ff3a4161470 0x7ff3a4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 -- 172.21.15.46:0/1890881059 >> 172.21.15.46:0/1890881059 conn(0x7ff3a40cbe30 msgr2=0x7ff3a40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 -- 172.21.15.46:0/1890881059 shutdown_connections 2023-10-06T12:33:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.904+0000 7ff3abdc5700 1 -- 172.21.15.46:0/1890881059 wait complete. 2023-10-06T12:33:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.908+0000 7ff3abdc5700 1 Processor -- start 2023-10-06T12:33:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.908+0000 7ff3abdc5700 1 -- start start 2023-10-06T12:33:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4161d10 con 0x7ff3a4161470 2023-10-06T12:33:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a415d1d0 con 0x7ff3a40c1d80 2023-10-06T12:33:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c63d0 con 0x7ff3a406cc30 2023-10-06T12:33:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3a9b61700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34616/0 (socket says 172.21.15.46:34616) 2023-10-06T12:33:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 -- 172.21.15.46:0/514764992 learned_addr learned my addr 172.21.15.46:0/514764992 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:32.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3a9360700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 -- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 msgr2=0x7ff3a415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 -- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3a40c1d80 msgr2=0x7ff3a40c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 -- 172.21.15.46:0/514764992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940094f0 con 0x7ff3a4161470 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3aa362700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00cf60 tx=0x7ff39c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00ade0 con 0x7ff3a4161470 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c015440 con 0x7ff3a4161470 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c015800 con 0x7ff3a4161470 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.912+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a4160160 con 0x7ff3a4161470 2023-10-06T12:33:32.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.916+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a415fe80 con 0x7ff3a4161470 2023-10-06T12:33:32.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.916+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a4161e50 con 0x7ff3a4161470 2023-10-06T12:33:32.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.916+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff39c00d440 con 0x7ff3a4161470 2023-10-06T12:33:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.920+0000 7ff399ffb700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 0x7ff388062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.920+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff39c07d380 con 0x7ff3a4161470 2023-10-06T12:33:32.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.920+0000 7ff3a9b61700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 0x7ff388062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:32.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.920+0000 7ff3a9b61700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 0x7ff388062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000aca0 tx=0x7ff3a0008040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:32.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:32.928+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff39c04d500 con 0x7ff3a4161470 2023-10-06T12:33:33.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.156+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7ff3a40c1580 con 0x7ff3a4161470 2023-10-06T12:33:33.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.708+0000 7ff399ffb700 1 -- 172.21.15.46:0/514764992 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2423) v1 ==== 177+0+0 (secure 0 0 0) 0x7ff39c01a0f0 con 0x7ff3a4161470 2023-10-06T12:33:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 msgr2=0x7ff388062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 0x7ff388062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000aca0 tx=0x7ff3a0008040 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 msgr2=0x7ff3a4162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00cf60 tx=0x7ff39c007870 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 shutdown_connections 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff388060670 0x7ff388062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3a4161470 0x7ff3a4162130 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3a40c1d80 0x7ff3a40c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 --2- 172.21.15.46:0/514764992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3a406cc30 0x7ff3a415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 >> 172.21.15.46:0/514764992 conn(0x7ff3a40cbe30 msgr2=0x7ff3a40b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:33.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 shutdown_connections 2023-10-06T12:33:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.716+0000 7ff3abdc5700 1 -- 172.21.15.46:0/514764992 wait complete. 2023-10-06T12:33:33.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T12:33:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.908+0000 7fa83dee3700 1 Processor -- start 2023-10-06T12:33:33.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.908+0000 7fa83dee3700 1 -- start start 2023-10-06T12:33:33.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83dee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 0x7fa8380c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83dee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 0x7fa83806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83dee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380c6cf0 con 0x7fa8380c7f20 2023-10-06T12:33:33.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83dee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380c2a30 con 0x7fa838154c50 2023-10-06T12:33:33.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83dee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380c83e0 con 0x7fa838154880 2023-10-06T12:33:33.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa837fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 0x7fa83806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83cee1700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 0x7fa8380c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83d6e2700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa8380c7f20 0x7fa83815aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:57188/0 (socket says 172.21.15.46:57188) 2023-10-06T12:33:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83d6e2700 1 -- 172.21.15.46:0/1951312310 learned_addr learned my addr 172.21.15.46:0/1951312310 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa8377fe700 1 -- 172.21.15.46:0/1951312310 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036935078 0 0) 0x7fa8380c6cf0 con 0x7fa8380c7f20 2023-10-06T12:33:33.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa8377fe700 1 -- 172.21.15.46:0/1951312310 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa828003570 con 0x7fa8380c7f20 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83cee1700 1 -- 172.21.15.46:0/1951312310 >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 msgr2=0x7fa83806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83cee1700 1 --2- 172.21.15.46:0/1951312310 >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 0x7fa83806c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83cee1700 1 -- 172.21.15.46:0/1951312310 >> v1:172.21.15.132:6789/0 conn(0x7fa8380c7f20 legacy=0x7fa83815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa83cee1700 1 -- 172.21.15.46:0/1951312310 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83806e5d0 con 0x7fa838154880 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.912+0000 7fa837fff700 1 --2- 172.21.15.46:0/1951312310 >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 0x7fa83806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:33.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83cee1700 1 --2- 172.21.15.46:0/1951312310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 0x7fa8380c7180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098e0 tx=0x7fa82c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9013664392f2fcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:33.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa8377fe700 1 -- 172.21.15.46:0/1951312310 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011630 con 0x7fa838154880 2023-10-06T12:33:33.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa8377fe700 1 -- 172.21.15.46:0/1951312310 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c011790 con 0x7fa838154880 2023-10-06T12:33:33.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa8377fe700 1 -- 172.21.15.46:0/1951312310 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011b30 con 0x7fa838154880 2023-10-06T12:33:33.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 -- 172.21.15.46:0/1951312310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 msgr2=0x7fa8380c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 --2- 172.21.15.46:0/1951312310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 0x7fa8380c7180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0098e0 tx=0x7fa82c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 -- 172.21.15.46:0/1951312310 shutdown_connections 2023-10-06T12:33:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 --2- 172.21.15.46:0/1951312310 >> v2:172.21.15.46:3300/0 conn(0x7fa838154c50 0x7fa83806c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 --2- 172.21.15.46:0/1951312310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154880 0x7fa8380c7180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 -- 172.21.15.46:0/1951312310 >> 172.21.15.46:0/1951312310 conn(0x7fa8380cbc10 msgr2=0x7fa8380cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 -- 172.21.15.46:0/1951312310 shutdown_connections 2023-10-06T12:33:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.916+0000 7fa83dee3700 1 -- 172.21.15.46:0/1951312310 wait complete. 2023-10-06T12:33:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.920+0000 7fa83dee3700 1 Processor -- start 2023-10-06T12:33:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.920+0000 7fa83dee3700 1 -- start start 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.920+0000 7fa83dee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8380c7f20 0x7fa83815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154c50 0x7fa838147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381484a0 con 0x7fa838154880 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381586c0 con 0x7fa8380c7f20 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83815c150 con 0x7fa838154c50 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59490/0 (socket says 172.21.15.46:59490) 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 -- 172.21.15.46:0/787277656 learned_addr learned my addr 172.21.15.46:0/787277656 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83cee1700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8380c7f20 0x7fa83815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83d6e2700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154c50 0x7fa838147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 -- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154c50 msgr2=0x7fa838147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154c50 0x7fa838147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 -- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8380c7f20 msgr2=0x7fa83815bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8380c7f20 0x7fa83815bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 -- 172.21.15.46:0/787277656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009420 con 0x7fa838154880 2023-10-06T12:33:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa837fff700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa82400ec80 tx=0x7fa8240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400c9d0 con 0x7fa838154880 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400cb30 con 0x7fa838154880 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c009420 con 0x7fa838154880 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824013470 con 0x7fa838154880 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83815c710 con 0x7fa838154880 2023-10-06T12:33:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.924+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83815c930 con 0x7fa838154880 2023-10-06T12:33:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.928+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa824013820 con 0x7fa838154880 2023-10-06T12:33:33.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.928+0000 7fa834ff9700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 0x7fa81c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:33.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.928+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa82407cd70 con 0x7fa838154880 2023-10-06T12:33:33.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.928+0000 7fa83cee1700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 0x7fa81c062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:33.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.932+0000 7fa83cee1700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 0x7fa81c062ba0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0091e0 tx=0x7fa82c017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:33.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:33.936+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa82404cef0 con 0x7fa838154880 2023-10-06T12:33:34.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.116+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa838158830 con 0x7fa838154880 2023-10-06T12:33:34.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.124+0000 7fa834ff9700 1 -- 172.21.15.46:0/787277656 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa824050510 con 0x7fa838154880 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 msgr2=0x7fa81c062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 0x7fa81c062ba0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0091e0 tx=0x7fa82c017000 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 msgr2=0x7fa8380b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa82400ec80 tx=0x7fa8240091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 shutdown_connections 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa81c0606f0 0x7fa81c062ba0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa838154c50 0x7fa838147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa838154880 0x7fa8380b1340 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 --2- 172.21.15.46:0/787277656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8380c7f20 0x7fa83815bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 >> 172.21.15.46:0/787277656 conn(0x7fa8380cbc10 msgr2=0x7fa8380c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:34.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 shutdown_connections 2023-10-06T12:33:34.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.128+0000 7fa83dee3700 1 -- 172.21.15.46:0/787277656 wait complete. 2023-10-06T12:33:34.156 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:33:34.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:33:34.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:33:34.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:33:34.242 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T12:33:34.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 Processor -- start 2023-10-06T12:33:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 -- start start 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 0x7f7d680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 0x7f7d680cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680cf100 con 0x7f7d680af3d0 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680ce4f0 con 0x7f7d680af000 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.404+0000 7f7d6c9c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680ce7d0 con 0x7f7d680cb4c0 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6759e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7d680af3d0 0x7f7d680690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33976/0 (socket says 172.21.15.46:33976) 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6759e700 1 -- 172.21.15.46:0/558579664 learned_addr learned my addr 172.21.15.46:0/558579664 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 --2- 172.21.15.46:0/558579664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 0x7f7d680cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d66d9d700 1 --2- 172.21.15.46:0/558579664 >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 0x7f7d680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d65d9b700 1 -- 172.21.15.46:0/558579664 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083943920 0 0) 0x7f7d680cf100 con 0x7f7d680af3d0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d65d9b700 1 -- 172.21.15.46:0/558579664 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d50003570 con 0x7f7d680af3d0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 -- 172.21.15.46:0/558579664 >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 msgr2=0x7f7d680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 --2- 172.21.15.46:0/558579664 >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 0x7f7d680cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 -- 172.21.15.46:0/558579664 >> v1:172.21.15.46:6789/0 conn(0x7f7d680af3d0 legacy=0x7f7d680690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 -- 172.21.15.46:0/558579664 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680ceab0 con 0x7f7d680cb4c0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d66d9d700 1 --2- 172.21.15.46:0/558579664 >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 0x7f7d680cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d6659c700 1 --2- 172.21.15.46:0/558579664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 0x7f7d680cdbf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098e0 tx=0x7f7d5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af3797d818d0622 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d65d9b700 1 -- 172.21.15.46:0/558579664 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c011630 con 0x7f7d680cb4c0 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d65d9b700 1 -- 172.21.15.46:0/558579664 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c011790 con 0x7f7d680cb4c0 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.408+0000 7f7d65d9b700 1 -- 172.21.15.46:0/558579664 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c011b30 con 0x7f7d680cb4c0 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/558579664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 msgr2=0x7f7d680cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/558579664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 0x7f7d680cdbf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098e0 tx=0x7f7d5c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/558579664 shutdown_connections 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/558579664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680cb4c0 0x7f7d680cdbf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/558579664 >> v2:172.21.15.132:3300/0 conn(0x7f7d680af000 0x7f7d680cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/558579664 >> 172.21.15.46:0/558579664 conn(0x7f7d680683b0 msgr2=0x7f7d680d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/558579664 shutdown_connections 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.412+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/558579664 wait complete. 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 Processor -- start 2023-10-06T12:33:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 -- start start 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680af3d0 0x7f7d68156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d680cb4c0 0x7f7d680b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68169250 con 0x7f7d680af000 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680ca3a0 con 0x7f7d680cb4c0 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6c9c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681599b0 con 0x7f7d680af3d0 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59504/0 (socket says 172.21.15.46:59504) 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d66d9d700 1 -- 172.21.15.46:0/3865602698 learned_addr learned my addr 172.21.15.46:0/3865602698 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:34.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6659c700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680af3d0 0x7f7d68156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.416+0000 7f7d6759e700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d680cb4c0 0x7f7d680b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 -- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680af3d0 msgr2=0x7f7d68156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680af3d0 0x7f7d68156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 -- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d680cb4c0 msgr2=0x7f7d680b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d680cb4c0 0x7f7d680b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 -- 172.21.15.46:0/3865602698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58009420 con 0x7f7d680af000 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d66d9d700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ec80 tx=0x7f7d580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800c850 con 0x7f7d680af000 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5800c9b0 con 0x7f7d680af000 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cd30 con 0x7f7d680af000 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c009420 con 0x7f7d680af000 2023-10-06T12:33:34.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d680ca0c0 con 0x7f7d680af000 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.420+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68159480 con 0x7f7d680af000 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.424+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7d58013680 con 0x7f7d680af000 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.424+0000 7f7d4effd700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 0x7f7d54062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.424+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7d5807d210 con 0x7f7d680af000 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.424+0000 7f7d6659c700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 0x7f7d54062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.428+0000 7f7d6659c700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 0x7f7d54062b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0091e0 tx=0x7f7d5c017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.432+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7d58050e20 con 0x7f7d680af000 2023-10-06T12:33:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.640+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d680ced90 con 0x7f7d680af000 2023-10-06T12:33:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.640+0000 7f7d4effd700 1 -- 172.21.15.46:0/3865602698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7d58017de0 con 0x7f7d680af000 2023-10-06T12:33:34.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:34.648 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2185980488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2185980488},{"type":"v1","addr":"172.21.15.132:6833","nonce":2185980488}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8089,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1015238641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1015238641},{"type":"v1","addr":"172.21.15.132:6835","nonce":1015238641}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/268387790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":268387790},{"type":"v1","addr":"172.21.15.46:6835","nonce":268387790}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":8727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/4278602895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":4278602895},{"type":"v1","addr":"172.21.15.46:6837","nonce":4278602895}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[]} 2023-10-06T12:33:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 msgr2=0x7f7d54062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 0x7f7d54062b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0091e0 tx=0x7f7d5c017000 comp rx=0 tx=0).stop 2023-10-06T12:33:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 msgr2=0x7f7d6815be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ec80 tx=0x7f7d580091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 shutdown_connections 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7d540606a0 0x7f7d54062b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d680cb4c0 0x7f7d680b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d680af3d0 0x7f7d68156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 --2- 172.21.15.46:0/3865602698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d680af000 0x7f7d6815be80 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 >> 172.21.15.46:0/3865602698 conn(0x7f7d680683b0 msgr2=0x7f7d681576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 shutdown_connections 2023-10-06T12:33:34.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.644+0000 7f7d6c9c4700 1 -- 172.21.15.46:0/3865602698 wait complete. 2023-10-06T12:33:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2023-10-06T12:33:34.673 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T12:33:34.673 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T12:33:34.673 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T12:33:34.673 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:33:34.673 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T12:33:34.673 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T12:33:34.674 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T12:33:34.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:33:34.674 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T12:33:34.674 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T12:33:34.674 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T12:33:34.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:33:34.675 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T12:33:34.675 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T12:33:34.675 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T12:33:34.675 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:33:34.693 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T12:33:34.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T12:33:34.694 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T12:33:34.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T12:33:34.706 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T12:33:34.706 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T12:33:34.707 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T12:33:34.707 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T12:33:34.747 INFO:tasks.ceph.mds.a:Started 2023-10-06T12:33:34.748 DEBUG:teuthology.parallel:result is None 2023-10-06T12:33:34.753 INFO:tasks.ceph.mds.c:Started 2023-10-06T12:33:34.754 DEBUG:teuthology.parallel:result is None 2023-10-06T12:33:34.754 INFO:tasks.ceph.mds.d:Started 2023-10-06T12:33:34.754 DEBUG:teuthology.parallel:result is None 2023-10-06T12:33:34.759 INFO:tasks.ceph.mds.b:Started 2023-10-06T12:33:34.760 DEBUG:teuthology.parallel:result is None 2023-10-06T12:33:34.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T12:33:34.853 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:33:34.855 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:33:34.866 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T12:33:34.866 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T12:33:34.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.916+0000 7fc8090cd700 1 Processor -- start 2023-10-06T12:33:34.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.916+0000 7fc8090cd700 1 -- start start 2023-10-06T12:33:34.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc8090cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc8040c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc8090cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc8041506c0 0x7fc804159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc8090cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040c67e0 con 0x7fc80406e2a0 2023-10-06T12:33:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc8090cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80406e810 con 0x7fc8041506c0 2023-10-06T12:33:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc8090cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80406eaf0 con 0x7fc8041502f0 2023-10-06T12:33:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80359e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc80406e2a0 0x7fc804156490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34004/0 (socket says 172.21.15.46:34004) 2023-10-06T12:33:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80359e700 1 -- 172.21.15.46:0/427366239 learned_addr learned my addr 172.21.15.46:0/427366239 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 --2- 172.21.15.46:0/427366239 >> v2:172.21.15.132:3300/0 conn(0x7fc8041506c0 0x7fc804159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc802d9d700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc8040c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc801d9b700 1 -- 172.21.15.46:0/427366239 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529704427 0 0) 0x7fc8040c67e0 con 0x7fc80406e2a0 2023-10-06T12:33:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc801d9b700 1 -- 172.21.15.46:0/427366239 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7ec003570 con 0x7fc80406e2a0 2023-10-06T12:33:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 -- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 msgr2=0x7fc8040c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc8040c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 -- 172.21.15.46:0/427366239 >> v1:172.21.15.46:6789/0 conn(0x7fc80406e2a0 legacy=0x7fc804156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 -- 172.21.15.46:0/427366239 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80406edd0 con 0x7fc8041506c0 2023-10-06T12:33:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc802d9d700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc8040c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:34.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc80259c700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc804159a90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80098e0 tx=0x7fc7f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3b1dc0e7542451 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.920+0000 7fc801d9b700 1 -- 172.21.15.46:0/427366239 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8011630 con 0x7fc8041506c0 2023-10-06T12:33:34.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc801d9b700 1 -- 172.21.15.46:0/427366239 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8011790 con 0x7fc8041506c0 2023-10-06T12:33:34.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc801d9b700 1 -- 172.21.15.46:0/427366239 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8011b30 con 0x7fc8041506c0 2023-10-06T12:33:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 -- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 msgr2=0x7fc804159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc804159a90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80098e0 tx=0x7fc7f80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 -- 172.21.15.46:0/427366239 shutdown_connections 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc804159a90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 --2- 172.21.15.46:0/427366239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc8040c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 -- 172.21.15.46:0/427366239 >> 172.21.15.46:0/427366239 conn(0x7fc8040cbe30 msgr2=0x7fc8040cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 -- 172.21.15.46:0/427366239 shutdown_connections 2023-10-06T12:33:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.924+0000 7fc8090cd700 1 -- 172.21.15.46:0/427366239 wait complete. 2023-10-06T12:33:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 Processor -- start 2023-10-06T12:33:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 -- start start 2023-10-06T12:33:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc804147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc80414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc804148fb0 con 0x7fc80406e2a0 2023-10-06T12:33:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc804157660 con 0x7fc8041506c0 2023-10-06T12:33:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc8090cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc804157940 con 0x7fc8041502f0 2023-10-06T12:33:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc802d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc802d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59552/0 (socket says 172.21.15.46:59552) 2023-10-06T12:33:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc802d9d700 1 -- 172.21.15.46:0/4063050638 learned_addr learned my addr 172.21.15.46:0/4063050638 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc80259c700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc804147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.928+0000 7fc80359e700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc80414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 -- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 msgr2=0x7fc804147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc804147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 -- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 msgr2=0x7fc80414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc80414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 -- 172.21.15.46:0/4063050638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40093c0 con 0x7fc80406e2a0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc80259c700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc804147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc802d9d700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ec80 tx=0x7fc7f4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4005a00 con 0x7fc80406e2a0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f4005b60 con 0x7fc80406e2a0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8009420 con 0x7fc80406e2a0 2023-10-06T12:33:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400c850 con 0x7fc80406e2a0 2023-10-06T12:33:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc804157f00 con 0x7fc80406e2a0 2023-10-06T12:33:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.932+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc804063f90 con 0x7fc80406e2a0 2023-10-06T12:33:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.936+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc7f400cbc0 con 0x7fc80406e2a0 2023-10-06T12:33:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.936+0000 7fc7eaffd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 0x7fc7f0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.936+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc7f407d070 con 0x7fc80406e2a0 2023-10-06T12:33:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.940+0000 7fc80259c700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 0x7fc7f0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.940+0000 7fc80259c700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 0x7fc7f0062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80098e0 tx=0x7fc7f8017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:34.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:34.944+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7f40acbe0 con 0x7fc80406e2a0 2023-10-06T12:33:35.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.132+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc8040c6470 con 0x7fc80406e2a0 2023-10-06T12:33:35.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.740+0000 7fc7eaffd700 1 -- 172.21.15.46:0/4063050638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v147) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc7f404d1f0 con 0x7fc80406e2a0 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.744+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 msgr2=0x7fc7f0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.744+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 0x7fc7f0062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80098e0 tx=0x7fc7f8017000 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.744+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 msgr2=0x7fc8041453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.744+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400ec80 tx=0x7fc7f4009120 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 shutdown_connections 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc7f0060670 0x7fc7f0062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041506c0 0x7fc80414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8041502f0 0x7fc804147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 --2- 172.21.15.46:0/4063050638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80406e2a0 0x7fc8041453e0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 >> 172.21.15.46:0/4063050638 conn(0x7fc8040cbe30 msgr2=0x7fc80414f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 shutdown_connections 2023-10-06T12:33:35.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.748+0000 7fc8090cd700 1 -- 172.21.15.46:0/4063050638 wait complete. 2023-10-06T12:33:35.756 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T12:33:35.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T12:33:35.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 Processor -- start 2023-10-06T12:33:35.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 -- start start 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f669015ab60 0x7f669006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669006d460 con 0x7f66900ad6c0 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900c7000 con 0x7f669015af30 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.944+0000 7f6695945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900c72c0 con 0x7f669015ab60 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66900ad6c0 0x7f66900adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34012/0 (socket says 172.21.15.46:34012) 2023-10-06T12:33:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668f7fe700 1 -- 172.21.15.46:0/2239116137 learned_addr learned my addr 172.21.15.46:0/2239116137 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668effd700 1 --2- 172.21.15.46:0/2239116137 >> v2:172.21.15.132:3300/0 conn(0x7f669015ab60 0x7f669006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668effd700 1 -- 172.21.15.46:0/2239116137 >> v1:172.21.15.132:6789/0 conn(0x7f669015af30 legacy=0x7f669006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668e7fc700 1 -- 172.21.15.46:0/2239116137 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251580649 0 0) 0x7f669006d460 con 0x7f66900ad6c0 2023-10-06T12:33:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668effd700 1 -- 172.21.15.46:0/2239116137 >> v1:172.21.15.46:6789/0 conn(0x7f66900ad6c0 legacy=0x7f66900adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668effd700 1 -- 172.21.15.46:0/2239116137 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900c75a0 con 0x7f669015ab60 2023-10-06T12:33:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668effd700 1 --2- 172.21.15.46:0/2239116137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f669015ab60 0x7f669006e8a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f66900c71f0 tx=0x7f6680009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5c94d1deeddac4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668e7fc700 1 -- 172.21.15.46:0/2239116137 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680016070 con 0x7f669015ab60 2023-10-06T12:33:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668e7fc700 1 -- 172.21.15.46:0/2239116137 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000bc30 con 0x7f669015ab60 2023-10-06T12:33:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f668e7fc700 1 -- 172.21.15.46:0/2239116137 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000e070 con 0x7f669015ab60 2023-10-06T12:33:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 -- 172.21.15.46:0/2239116137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f669015ab60 msgr2=0x7f669006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 --2- 172.21.15.46:0/2239116137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f669015ab60 0x7f669006e8a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f66900c71f0 tx=0x7f6680009780 comp rx=0 tx=0).stop 2023-10-06T12:33:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 -- 172.21.15.46:0/2239116137 shutdown_connections 2023-10-06T12:33:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 --2- 172.21.15.46:0/2239116137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f669015ab60 0x7f669006e8a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 -- 172.21.15.46:0/2239116137 >> 172.21.15.46:0/2239116137 conn(0x7f66900cbe30 msgr2=0x7f66900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 -- 172.21.15.46:0/2239116137 shutdown_connections 2023-10-06T12:33:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.948+0000 7f6695945700 1 -- 172.21.15.46:0/2239116137 wait complete. 2023-10-06T12:33:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 Processor -- start 2023-10-06T12:33:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 -- start start 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 0x7f6690168130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 0x7f66900c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900ca1a0 con 0x7f669015af30 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c9590 con 0x7f66900ad6c0 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6695945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c9870 con 0x7f669015ab60 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f668f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59560/0 (socket says 172.21.15.46:59560) 2023-10-06T12:33:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f668f7fe700 1 -- 172.21.15.46:0/2353120030 learned_addr learned my addr 172.21.15.46:0/2353120030 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f6687fff700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 0x7f66900c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.952+0000 7f668effd700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 0x7f6690168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:35.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 -- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 msgr2=0x7f66900c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 0x7f66900c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 -- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 msgr2=0x7f66900c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 -- 172.21.15.46:0/2353120030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009a20 con 0x7f66900ad6c0 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6687fff700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 0x7f66900c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668f7fe700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f668effd700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 0x7f6690168130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6680015000 tx=0x7f6680018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680012e50 con 0x7f66900ad6c0 2023-10-06T12:33:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680027470 con 0x7f66900ad6c0 2023-10-06T12:33:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680027830 con 0x7f66900ad6c0 2023-10-06T12:33:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900c9b50 con 0x7f66900ad6c0 2023-10-06T12:33:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900c88a0 con 0x7f66900ad6c0 2023-10-06T12:33:35.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.956+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690063f90 con 0x7f66900ad6c0 2023-10-06T12:33:35.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.960+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6680024070 con 0x7f66900ad6c0 2023-10-06T12:33:35.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.960+0000 7f6686ffd700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 0x7f6670062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:35.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.960+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f668007f100 con 0x7f66900ad6c0 2023-10-06T12:33:35.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.960+0000 7f6687fff700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 0x7f6670062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.964+0000 7f6687fff700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 0x7f6670062bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690168670 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:35.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:35.968+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f668004f670 con 0x7f66900ad6c0 2023-10-06T12:33:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.196+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f669006e860 con 0x7f66900ad6c0 2023-10-06T12:33:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.196+0000 7f6686ffd700 1 -- 172.21.15.46:0/2353120030 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6680052c50 con 0x7f66900ad6c0 2023-10-06T12:33:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:36.205 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:33:36.205 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:33:36.206 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:33:36.207 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:33:36.208 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.209 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.210 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T12:33:36.211 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.214 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:33:36.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T12:33:36.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:33:36.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.221 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T12:33:36.222 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T12:33:36.223 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:33:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.200+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 msgr2=0x7f6670062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.200+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 0x7f6670062bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6690168670 tx=0x7f6678009040 comp rx=0 tx=0).stop 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.200+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 msgr2=0x7f6690168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.200+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 0x7f6690168130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6680015000 tx=0x7f6680018000 comp rx=0 tx=0).stop 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 shutdown_connections 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6670060740 0x7f6670062bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f669015af30 0x7f66900c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f669015ab60 0x7f66900c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 --2- 172.21.15.46:0/2353120030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66900ad6c0 0x7f6690168130 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 >> 172.21.15.46:0/2353120030 conn(0x7f66900cbe30 msgr2=0x7f6690151400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 shutdown_connections 2023-10-06T12:33:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.204+0000 7f6695945700 1 -- 172.21.15.46:0/2353120030 wait complete. 2023-10-06T12:33:36.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:36.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.384+0000 7f6e44385700 1 Processor -- start 2023-10-06T12:33:36.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.384+0000 7f6e44385700 1 -- start start 2023-10-06T12:33:36.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e44385700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e44385700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e3c1506c0 0x7f6e3c0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e44385700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3c0c67e0 con 0x7f6e3c1502f0 2023-10-06T12:33:36.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e44385700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3c06e810 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e44385700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3c06eaf0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59572/0 (socket says 172.21.15.46:59572) 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 -- 172.21.15.46:0/4083780304 learned_addr learned my addr 172.21.15.46:0/4083780304 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e42922700 1 --2- 172.21.15.46:0/4083780304 >> v2:172.21.15.132:3300/0 conn(0x7f6e3c1506c0 0x7f6e3c0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 -- 172.21.15.46:0/4083780304 >> v2:172.21.15.132:3300/0 conn(0x7f6e3c1506c0 msgr2=0x7f6e3c0c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 --2- 172.21.15.46:0/4083780304 >> v2:172.21.15.132:3300/0 conn(0x7f6e3c1506c0 0x7f6e3c0c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 -- 172.21.15.46:0/4083780304 >> v1:172.21.15.46:6789/0 conn(0x7f6e3c1502f0 legacy=0x7f6e3c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 -- 172.21.15.46:0/4083780304 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c06edd0 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e41920700 1 --2- 172.21.15.46:0/4083780304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0ccf10 tx=0x7f6e2c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e731706d2448f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e4111f700 1 -- 172.21.15.46:0/4083780304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c00dc50 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e4111f700 1 -- 172.21.15.46:0/4083780304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c00ddb0 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.388+0000 7f6e4111f700 1 -- 172.21.15.46:0/4083780304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0196d0 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 -- 172.21.15.46:0/4083780304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c15a1a0 msgr2=0x7f6e3c0c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 --2- 172.21.15.46:0/4083780304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0ccf10 tx=0x7f6e2c00b4f0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 -- 172.21.15.46:0/4083780304 shutdown_connections 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 --2- 172.21.15.46:0/4083780304 >> v2:172.21.15.132:3300/0 conn(0x7f6e3c1506c0 0x7f6e3c0c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 --2- 172.21.15.46:0/4083780304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c0c0bd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 -- 172.21.15.46:0/4083780304 >> 172.21.15.46:0/4083780304 conn(0x7f6e3c0cbe30 msgr2=0x7f6e3c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 -- 172.21.15.46:0/4083780304 shutdown_connections 2023-10-06T12:33:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 -- 172.21.15.46:0/4083780304 wait complete. 2023-10-06T12:33:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.392+0000 7f6e44385700 1 Processor -- start 2023-10-06T12:33:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 -- start start 2023-10-06T12:33:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e3c1502f0 0x7f6e3c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c14dce0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c1544a0 con 0x7f6e3c15a1a0 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e44385700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c154780 con 0x7f6e3c1502f0 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59580/0 (socket says 172.21.15.46:59580) 2023-10-06T12:33:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 -- 172.21.15.46:0/2814034393 learned_addr learned my addr 172.21.15.46:0/2814034393 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e42121700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e3c1502f0 0x7f6e3c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e42922700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 -- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e3c1502f0 msgr2=0x7f6e3c14cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e3c1502f0 0x7f6e3c14cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 -- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e3c15a1a0 msgr2=0x7f6e3c069f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c069f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 -- 172.21.15.46:0/2814034393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c00b790 con 0x7f6e3c1506c0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.396+0000 7f6e41920700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c06c150 tx=0x7f6e2c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c010680 con 0x7f6e3c1506c0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c0107e0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c154a60 con 0x7f6e3c1506c0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c010ba0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c155020 con 0x7f6e3c1506c0 2023-10-06T12:33:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.400+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c155210 con 0x7f6e3c1506c0 2023-10-06T12:33:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.404+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6e2c0233f0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.404+0000 7f6e327fc700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 0x7f6e20062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.404+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6e2c014070 con 0x7f6e3c1506c0 2023-10-06T12:33:36.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.404+0000 7f6e42121700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 0x7f6e20062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.404+0000 7f6e42121700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 0x7f6e20062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e38003a10 tx=0x7f6e38007e60 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.408+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6e2c04eb90 con 0x7f6e3c1506c0 2023-10-06T12:33:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.580+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e3c063f90 con 0x7f6e3c1506c0 2023-10-06T12:33:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e327fc700 1 -- 172.21.15.46:0/2814034393 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+14447 (secure 0 0 0) 0x7f6e2c0521b0 con 0x7f6e3c1506c0 2023-10-06T12:33:36.591 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:36.591 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":149,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:35.757224+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:36.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 msgr2=0x7f6e20062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 0x7f6e20062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e38003a10 tx=0x7f6e38007e60 comp rx=0 tx=0).stop 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 msgr2=0x7f6e3c0aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c06c150 tx=0x7f6e2c01b000 comp rx=0 tx=0).stop 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 shutdown_connections 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6e20060670 0x7f6e20062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e3c15a1a0 0x7f6e3c069f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e3c1506c0 0x7f6e3c0aeff0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 --2- 172.21.15.46:0/2814034393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e3c1502f0 0x7f6e3c14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.584+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 >> 172.21.15.46:0/2814034393 conn(0x7f6e3c0cbe30 msgr2=0x7f6e3c0c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:36.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.588+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 shutdown_connections 2023-10-06T12:33:36.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.588+0000 7f6e44385700 1 -- 172.21.15.46:0/2814034393 wait complete. 2023-10-06T12:33:36.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T12:33:36.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:33:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.776+0000 7f04e722e700 1 Processor -- start 2023-10-06T12:33:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 -- start start 2023-10-06T12:33:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04e00b2790 0x7f04e00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00cd080 0x7f04e00c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00cc510 con 0x7f04e00b23c0 2023-10-06T12:33:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e00cd500 con 0x7f04e00b2790 2023-10-06T12:33:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e722e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b2be0 con 0x7f04e00cd080 2023-10-06T12:33:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04dffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04e00b2790 0x7f04e00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04dffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04e00b2790 0x7f04e00ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59590/0 (socket says 172.21.15.46:59590) 2023-10-06T12:33:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04dffff700 1 -- 172.21.15.46:0/2611913322 learned_addr learned my addr 172.21.15.46:0/2611913322 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04e57cb700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00cd080 0x7f04e00c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04dffff700 1 -- 172.21.15.46:0/2611913322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00cd080 msgr2=0x7f04e00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.780+0000 7f04dffff700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00cd080 0x7f04e00c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04dffff700 1 -- 172.21.15.46:0/2611913322 >> v1:172.21.15.132:6789/0 conn(0x7f04e00b23c0 legacy=0x7f04e00cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04dffff700 1 -- 172.21.15.46:0/2611913322 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00cb900 con 0x7f04e00b2790 2023-10-06T12:33:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04dffff700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00b2790 0x7f04e00ccb40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0099b0 tx=0x7f04cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b385a44d47d5ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04df7fe700 1 -- 172.21.15.46:0/2611913322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc009ea0 con 0x7f04e00b2790 2023-10-06T12:33:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04df7fe700 1 -- 172.21.15.46:0/2611913322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc00b860 con 0x7f04e00b2790 2023-10-06T12:33:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04df7fe700 1 -- 172.21.15.46:0/2611913322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00bc00 con 0x7f04e00b2790 2023-10-06T12:33:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 -- 172.21.15.46:0/2611913322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00b2790 msgr2=0x7f04e00ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00b2790 0x7f04e00ccb40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0099b0 tx=0x7f04cc009250 comp rx=0 tx=0).stop 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 -- 172.21.15.46:0/2611913322 shutdown_connections 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00cd080 0x7f04e00c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 --2- 172.21.15.46:0/2611913322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00b2790 0x7f04e00ccb40 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 -- 172.21.15.46:0/2611913322 >> 172.21.15.46:0/2611913322 conn(0x7f04e0099480 msgr2=0x7f04e006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 -- 172.21.15.46:0/2611913322 shutdown_connections 2023-10-06T12:33:36.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.784+0000 7f04e722e700 1 -- 172.21.15.46:0/2611913322 wait complete. 2023-10-06T12:33:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 Processor -- start 2023-10-06T12:33:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 -- start start 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e00b23c0 0x7f04e0155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00b2790 0x7f04e00b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b3a60 con 0x7f04e00cd080 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0156580 con 0x7f04e00b23c0 2023-10-06T12:33:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e722e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0155970 con 0x7f04e00b2790 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59594/0 (socket says 172.21.15.46:59594) 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 -- 172.21.15.46:0/3611375228 learned_addr learned my addr 172.21.15.46:0/3611375228 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04dffff700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00b2790 0x7f04e00b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e4fca700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e00b23c0 0x7f04e0155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 -- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00b2790 msgr2=0x7f04e00b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00b2790 0x7f04e00b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 -- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e00b23c0 msgr2=0x7f04e0155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e00b23c0 0x7f04e0155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 -- 172.21.15.46:0/3611375228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0094f0 con 0x7f04e00cd080 2023-10-06T12:33:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.788+0000 7f04e57cb700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f04d800ea50 tx=0x7f04d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d800ee90 con 0x7f04e00cd080 2023-10-06T12:33:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d800c850 con 0x7f04e00cd080 2023-10-06T12:33:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e0155c50 con 0x7f04e00cd080 2023-10-06T12:33:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d800cc10 con 0x7f04e00cd080 2023-10-06T12:33:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e0156210 con 0x7f04e00cd080 2023-10-06T12:33:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e0156710 con 0x7f04e00cd080 2023-10-06T12:33:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.792+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f04d8013470 con 0x7f04e00cd080 2023-10-06T12:33:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.796+0000 7f04dcff9700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 0x7f04c4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:36.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.796+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f04d807ca90 con 0x7f04e00cd080 2023-10-06T12:33:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.796+0000 7f04e4fca700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 0x7f04c4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.796+0000 7f04e4fca700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 0x7f04c4062b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c580 tx=0x7f04d4001a80 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:36.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.804+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f04d804cf60 con 0x7f04e00cd080 2023-10-06T12:33:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.972+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04e00660f0 con 0x7f04e00cd080 2023-10-06T12:33:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.972+0000 7f04dcff9700 1 -- 172.21.15.46:0/3611375228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04d8050580 con 0x7f04e00cd080 2023-10-06T12:33:36.979 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:33:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 msgr2=0x7f04c4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 0x7f04c4062b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c580 tx=0x7f04d4001a80 comp rx=0 tx=0).stop 2023-10-06T12:33:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 msgr2=0x7f04e00b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f04d800ea50 tx=0x7f04d80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 shutdown_connections 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04c40606a0 0x7f04c4062b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04e00cd080 0x7f04e00b3520 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e00b2790 0x7f04e00b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 --2- 172.21.15.46:0/3611375228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04e00b23c0 0x7f04e0155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:36.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 >> 172.21.15.46:0/3611375228 conn(0x7f04e0099480 msgr2=0x7f04e00c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:36.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 shutdown_connections 2023-10-06T12:33:36.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:36.976+0000 7f04e722e700 1 -- 172.21.15.46:0/3611375228 wait complete. 2023-10-06T12:33:37.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:33:37.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.168+0000 7f78d61c3700 1 Processor -- start 2023-10-06T12:33:37.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.168+0000 7f78d61c3700 1 -- start start 2023-10-06T12:33:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d61c3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f78d00adbe0 0x7f78d00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d61c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00b0dc0 0x7f78d00b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d61c3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00b1740 con 0x7f78d00d0d60 2023-10-06T12:33:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d61c3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d0159740 con 0x7f78d00adbe0 2023-10-06T12:33:37.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d61c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00afd20 con 0x7f78d00b0dc0 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d49c0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78d00d0d60 0x7f78d00b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34024/0 (socket says 172.21.15.46:34024) 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d49c0700 1 -- 172.21.15.46:0/188428075 learned_addr learned my addr 172.21.15.46:0/188428075 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 --2- 172.21.15.46:0/188428075 >> v2:172.21.15.132:3300/0 conn(0x7f78d00adbe0 0x7f78d00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d59c2700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00b0dc0 0x7f78d00b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78c7fff700 1 -- 172.21.15.46:0/188428075 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414217915 0 0) 0x7f78c0001110 con 0x7f78d00d0d60 2023-10-06T12:33:37.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78c7fff700 1 -- 172.21.15.46:0/188428075 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003570 con 0x7f78d00d0d60 2023-10-06T12:33:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 -- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00b0dc0 msgr2=0x7f78d00b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00b0dc0 0x7f78d00b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 -- 172.21.15.46:0/188428075 >> v1:172.21.15.46:6789/0 conn(0x7f78d00d0d60 legacy=0x7f78d00b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 -- 172.21.15.46:0/188428075 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00b0000 con 0x7f78d00adbe0 2023-10-06T12:33:37.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.172+0000 7f78d51c1700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00adbe0 0x7f78d00d0820 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0098e0 tx=0x7f78cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315bcb8bbdb23066 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78c7fff700 1 -- 172.21.15.46:0/188428075 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011630 con 0x7f78d00adbe0 2023-10-06T12:33:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78c7fff700 1 -- 172.21.15.46:0/188428075 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc011790 con 0x7f78d00adbe0 2023-10-06T12:33:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78c7fff700 1 -- 172.21.15.46:0/188428075 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011b30 con 0x7f78d00adbe0 2023-10-06T12:33:37.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 -- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00adbe0 msgr2=0x7f78d00d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00adbe0 0x7f78d00d0820 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0098e0 tx=0x7f78cc0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 -- 172.21.15.46:0/188428075 shutdown_connections 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00b0dc0 0x7f78d00b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 --2- 172.21.15.46:0/188428075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00adbe0 0x7f78d00d0820 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 -- 172.21.15.46:0/188428075 >> 172.21.15.46:0/188428075 conn(0x7f78d006ef60 msgr2=0x7f78d00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:37.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 -- 172.21.15.46:0/188428075 shutdown_connections 2023-10-06T12:33:37.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.176+0000 7f78d61c3700 1 -- 172.21.15.46:0/188428075 wait complete. 2023-10-06T12:33:37.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 Processor -- start 2023-10-06T12:33:37.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 -- start start 2023-10-06T12:33:37.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00adbe0 0x7f78d0140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 0x7f78d0150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0150f10 con 0x7f78d00b0dc0 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d01511f0 con 0x7f78d00d0d60 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d61c3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d01514d0 con 0x7f78d00adbe0 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d49c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d49c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59606/0 (socket says 172.21.15.46:59606) 2023-10-06T12:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d49c0700 1 -- 172.21.15.46:0/1117801726 learned_addr learned my addr 172.21.15.46:0/1117801726 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d59c2700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 0x7f78d0150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.180+0000 7f78d51c1700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00adbe0 0x7f78d0140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 -- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00adbe0 msgr2=0x7f78d0140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00adbe0 0x7f78d0140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 -- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 msgr2=0x7f78d0150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 -- 172.21.15.46:0/1117801726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc009420 con 0x7f78d00d0d60 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d49c0700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d59c2700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 0x7f78d0150890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009e50 tx=0x7f78c800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800caa0 con 0x7f78d00d0d60 2023-10-06T12:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800f040 con 0x7f78d00d0d60 2023-10-06T12:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d01517b0 con 0x7f78d00d0d60 2023-10-06T12:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800ce50 con 0x7f78d00d0d60 2023-10-06T12:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d0169a20 con 0x7f78d00d0d60 2023-10-06T12:33:37.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.184+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d0064590 con 0x7f78d00d0d60 2023-10-06T12:33:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.188+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f78c8014690 con 0x7f78d00d0d60 2023-10-06T12:33:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.188+0000 7f78c57fa700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 0x7f78b4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.188+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f78c807d670 con 0x7f78d00d0d60 2023-10-06T12:33:37.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.188+0000 7f78d51c1700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 0x7f78b4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.192+0000 7f78d51c1700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 0x7f78b4062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0091e0 tx=0x7f78cc017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.196+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f78c8051420 con 0x7f78d00d0d60 2023-10-06T12:33:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.372+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78d00675d0 con 0x7f78d00d0d60 2023-10-06T12:33:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.372+0000 7f78c57fa700 1 -- 172.21.15.46:0/1117801726 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v149) v1 ==== 92+0+10 (secure 0 0 0) 0x7f78c8050fb0 con 0x7f78d00d0d60 2023-10-06T12:33:37.379 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:33:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 msgr2=0x7f78b4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 0x7f78b4062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0091e0 tx=0x7f78cc017000 comp rx=0 tx=0).stop 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 msgr2=0x7f78d0150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 0x7f78d0150890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009e50 tx=0x7f78c800b040 comp rx=0 tx=0).stop 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 shutdown_connections 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f78b4060670 0x7f78b4062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78d00d0d60 0x7f78d0150890 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00b0dc0 0x7f78d0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 --2- 172.21.15.46:0/1117801726 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78d00adbe0 0x7f78d0140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 >> 172.21.15.46:0/1117801726 conn(0x7f78d006ef60 msgr2=0x7f78d014eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 shutdown_connections 2023-10-06T12:33:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.376+0000 7f78d61c3700 1 -- 172.21.15.46:0/1117801726 wait complete. 2023-10-06T12:33:37.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T12:33:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.556+0000 7f9b9346b700 1 Processor -- start 2023-10-06T12:33:37.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 -- start start 2023-10-06T12:33:37.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 0x7f9b8c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af3d0 0x7f9b8c0690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cf100 con 0x7f9b8c0af000 2023-10-06T12:33:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0ce4f0 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b9346b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0ce7d0 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b92469700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 0x7f9b8c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b91c68700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b92469700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 0x7f9b8c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59610/0 (socket says 172.21.15.46:59610) 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b91c68700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44594/0 (socket says 172.21.15.46:44594) 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b92469700 1 -- 172.21.15.46:0/752143926 learned_addr learned my addr 172.21.15.46:0/752143926 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:37.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.560+0000 7f9b92c6a700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af3d0 0x7f9b8c0690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 -- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af3d0 msgr2=0x7f9b8c0690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af3d0 0x7f9b8c0690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 -- 172.21.15.46:0/752143926 >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 msgr2=0x7f9b8c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 --2- 172.21.15.46:0/752143926 >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 0x7f9b8c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 -- 172.21.15.46:0/752143926 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0ceab0 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91c68700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c087fd0 tx=0x7f9b7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b5c8a9983c098c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91467700 1 -- 172.21.15.46:0/752143926 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c011630 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91467700 1 -- 172.21.15.46:0/752143926 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0117b0 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b91467700 1 -- 172.21.15.46:0/752143926 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c011b70 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 -- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 msgr2=0x7f9b8c0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c087fd0 tx=0x7f9b7c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 -- 172.21.15.46:0/752143926 shutdown_connections 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0cdbf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 --2- 172.21.15.46:0/752143926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af3d0 0x7f9b8c0690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 --2- 172.21.15.46:0/752143926 >> v2:172.21.15.46:3300/0 conn(0x7f9b8c0af000 0x7f9b8c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 -- 172.21.15.46:0/752143926 >> 172.21.15.46:0/752143926 conn(0x7f9b8c0683b0 msgr2=0x7f9b8c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:37.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 -- 172.21.15.46:0/752143926 shutdown_connections 2023-10-06T12:33:37.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.564+0000 7f9b9346b700 1 -- 172.21.15.46:0/752143926 wait complete. 2023-10-06T12:33:37.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 Processor -- start 2023-10-06T12:33:37.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 -- start start 2023-10-06T12:33:37.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af000 0x7f9b8c170b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c159a80 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c16e530 con 0x7f9b8c0cb4c0 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b9346b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c16e810 con 0x7f9b8c0af000 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.568+0000 7f9b91c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59626/0 (socket says 172.21.15.46:59626) 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 -- 172.21.15.46:0/1544895541 learned_addr learned my addr 172.21.15.46:0/1544895541 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b92469700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af000 0x7f9b8c170b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b92c6a700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 -- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af000 msgr2=0x7f9b8c170b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af000 0x7f9b8c170b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 -- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 msgr2=0x7f9b8c0b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 -- 172.21.15.46:0/1544895541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c009420 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b92c6a700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b91c68700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c013000 tx=0x7f9b7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c01f070 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c000bc0 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c16eaf0 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c025420 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c16e180 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.572+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c159c30 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.576+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9b7c023070 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.576+0000 7f9b827fc700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 0x7f9b70062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.576+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f9b7c07dbc0 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.580+0000 7f9b92469700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 0x7f9b70062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.580+0000 7f9b92469700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 0x7f9b70062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006d40 tx=0x7f9b88009250 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.584+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9b8c159c30 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.756+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9b8c1598b0 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.756+0000 7f9b827fc700 1 -- 172.21.15.46:0/1544895541 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9b7c02a090 con 0x7f9b8c0af3d0 2023-10-06T12:33:37.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:37.764 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:33:37.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 msgr2=0x7f9b70062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 0x7f9b70062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9b88006d40 tx=0x7f9b88009250 comp rx=0 tx=0).stop 2023-10-06T12:33:37.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 msgr2=0x7f9b8c16dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c013000 tx=0x7f9b7c017000 comp rx=0 tx=0).stop 2023-10-06T12:33:37.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 shutdown_connections 2023-10-06T12:33:37.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9b70060670 0x7f9b70062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b8c0cb4c0 0x7f9b8c0b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b8c0af3d0 0x7f9b8c16dc40 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 --2- 172.21.15.46:0/1544895541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9b8c0af000 0x7f9b8c170b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 >> 172.21.15.46:0/1544895541 conn(0x7f9b8c0683b0 msgr2=0x7f9b8c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:37.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 shutdown_connections 2023-10-06T12:33:37.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.760+0000 7f9b9346b700 1 -- 172.21.15.46:0/1544895541 wait complete. 2023-10-06T12:33:37.786 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T12:33:37.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T12:33:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 Processor -- start 2023-10-06T12:33:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 -- start start 2023-10-06T12:33:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a8068660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3a8159230 0x7fc3a80a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80aab80 con 0x7fc3a8062fe0 2023-10-06T12:33:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a80a9d80 con 0x7fc3a8159230 2023-10-06T12:33:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.948+0000 7fc3a759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8063430 con 0x7fc3a8068290 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3a8159230 0x7fc3a80a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a5d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3a8062fe0 0x7fc3a8158b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34030/0 (socket says 172.21.15.46:34030) 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a5d9b700 1 -- 172.21.15.46:0/3893041250 learned_addr learned my addr 172.21.15.46:0/3893041250 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a659c700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a8068660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a559a700 1 -- 172.21.15.46:0/3893041250 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714930307 0 0) 0x7fc3a80aab80 con 0x7fc3a8062fe0 2023-10-06T12:33:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a559a700 1 -- 172.21.15.46:0/3893041250 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003570 con 0x7fc3a8062fe0 2023-10-06T12:33:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/3893041250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 msgr2=0x7fc3a8068660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a8068660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/3893041250 >> v1:172.21.15.46:6789/0 conn(0x7fc3a8062fe0 legacy=0x7fc3a8158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/3893041250 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80a9f70 con 0x7fc3a8159230 2023-10-06T12:33:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a659c700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a8068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a6d9d700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80a9840 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e60 tx=0x7fc3a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d335b5c0898b7448 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a559a700 1 -- 172.21.15.46:0/3893041250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000e070 con 0x7fc3a8159230 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a559a700 1 -- 172.21.15.46:0/3893041250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a000bb60 con 0x7fc3a8159230 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a559a700 1 -- 172.21.15.46:0/3893041250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0012430 con 0x7fc3a8159230 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 -- 172.21.15.46:0/3893041250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 msgr2=0x7fc3a80a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80a9840 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009e60 tx=0x7fc3a0009780 comp rx=0 tx=0).stop 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 -- 172.21.15.46:0/3893041250 shutdown_connections 2023-10-06T12:33:37.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80a9840 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 --2- 172.21.15.46:0/3893041250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a8068660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 -- 172.21.15.46:0/3893041250 >> 172.21.15.46:0/3893041250 conn(0x7fc3a806ab10 msgr2=0x7fc3a806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:37.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 -- 172.21.15.46:0/3893041250 shutdown_connections 2023-10-06T12:33:37.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.952+0000 7fc3a759e700 1 -- 172.21.15.46:0/3893041250 wait complete. 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 Processor -- start 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 -- start start 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 0x7fc3a81515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a80ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80c93e0 con 0x7fc3a8159230 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80c8810 con 0x7fc3a8062fe0 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8151b00 con 0x7fc3a8068290 2023-10-06T12:33:37.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59652/0 (socket says 172.21.15.46:59652) 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/353792284 learned_addr learned my addr 172.21.15.46:0/353792284 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a659c700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 0x7fc3a81515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.956+0000 7fc3a5d9b700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a80ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 msgr2=0x7fc3a80ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a80ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 msgr2=0x7fc3a81515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 0x7fc3a81515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 -- 172.21.15.46:0/353792284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0009a20 con 0x7fc3a8159230 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a659c700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 0x7fc3a81515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:37.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a6d9d700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3a806c850 tx=0x7fc3a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0027470 con 0x7fc3a8159230 2023-10-06T12:33:37.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a00275d0 con 0x7fc3a8159230 2023-10-06T12:33:37.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a80c8ab0 con 0x7fc3a8159230 2023-10-06T12:33:37.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0027990 con 0x7fc3a8159230 2023-10-06T12:33:37.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a80c9070 con 0x7fc3a8159230 2023-10-06T12:33:37.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.960+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a80c9550 con 0x7fc3a8159230 2023-10-06T12:33:37.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.964+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc3a0024070 con 0x7fc3a8159230 2023-10-06T12:33:37.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.964+0000 7fc3967fc700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 0x7fc388062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:37.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.964+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc3a007f2f0 con 0x7fc3a8159230 2023-10-06T12:33:37.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.964+0000 7fc3a659c700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 0x7fc388062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:37.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.964+0000 7fc3a659c700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 0x7fc388062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc398006f00 tx=0x7fc398008040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:37.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:37.972+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc3a004f810 con 0x7fc3a8159230 2023-10-06T12:33:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.144+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc3a8064590 con 0x7fc3a8159230 2023-10-06T12:33:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.768+0000 7fc3967fc700 1 -- 172.21.15.46:0/353792284 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc3a0052e30 con 0x7fc3a8159230 2023-10-06T12:33:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.772+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 msgr2=0x7fc388062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.772+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 0x7fc388062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc398006f00 tx=0x7fc398008040 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.772+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 msgr2=0x7fc3a80acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.772+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc3a806c850 tx=0x7fc3a0019000 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 shutdown_connections 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc3880606a0 0x7fc388062b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a8159230 0x7fc3a80acd10 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3a8068290 0x7fc3a80ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 --2- 172.21.15.46:0/353792284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3a8062fe0 0x7fc3a81515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 >> 172.21.15.46:0/353792284 conn(0x7fc3a806ab10 msgr2=0x7fc3a80b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 shutdown_connections 2023-10-06T12:33:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.776+0000 7fc3a759e700 1 -- 172.21.15.46:0/353792284 wait complete. 2023-10-06T12:33:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T12:33:38.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T12:33:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.976+0000 7f33ea963700 1 Processor -- start 2023-10-06T12:33:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.976+0000 7f33ea963700 1 -- start start 2023-10-06T12:33:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea963700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 0x7f33e40cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea963700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f33e40d0750 0x7f33e40cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea963700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40cf050 con 0x7f33e40cdc10 2023-10-06T12:33:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea963700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e40ce440 con 0x7f33e40d0750 2023-10-06T12:33:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea963700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40ce720 con 0x7f33e40cb020 2023-10-06T12:33:38.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.980+0000 7f33ea162700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f33e40cdc10 0x7f33e4158bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34038/0 (socket says 172.21.15.46:34038) 2023-10-06T12:33:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea162700 1 -- 172.21.15.46:0/3301995542 learned_addr learned my addr 172.21.15.46:0/3301995542 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 --2- 172.21.15.46:0/3301995542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 0x7f33e40cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9160700 1 --2- 172.21.15.46:0/3301995542 >> v2:172.21.15.132:3300/0 conn(0x7f33e40d0750 0x7f33e40cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e895f700 1 -- 172.21.15.46:0/3301995542 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916192387 0 0) 0x7f33e40cf050 con 0x7f33e40cdc10 2023-10-06T12:33:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e895f700 1 -- 172.21.15.46:0/3301995542 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d0003570 con 0x7f33e40cdc10 2023-10-06T12:33:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 -- 172.21.15.46:0/3301995542 >> v2:172.21.15.132:3300/0 conn(0x7f33e40d0750 msgr2=0x7f33e40cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 --2- 172.21.15.46:0/3301995542 >> v2:172.21.15.132:3300/0 conn(0x7f33e40d0750 0x7f33e40cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 -- 172.21.15.46:0/3301995542 >> v1:172.21.15.46:6789/0 conn(0x7f33e40cdc10 legacy=0x7f33e4158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 -- 172.21.15.46:0/3301995542 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40cea00 con 0x7f33e40cb020 2023-10-06T12:33:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e9961700 1 --2- 172.21.15.46:0/3301995542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 0x7f33e40cb3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33e00098e0 tx=0x7f33e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fff00d3bd231bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e895f700 1 -- 172.21.15.46:0/3301995542 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011630 con 0x7f33e40cb020 2023-10-06T12:33:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e895f700 1 -- 172.21.15.46:0/3301995542 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0011790 con 0x7f33e40cb020 2023-10-06T12:33:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33e895f700 1 -- 172.21.15.46:0/3301995542 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011b30 con 0x7f33e40cb020 2023-10-06T12:33:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 -- 172.21.15.46:0/3301995542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 msgr2=0x7f33e40cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:38.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 --2- 172.21.15.46:0/3301995542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 0x7f33e40cb3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f33e00098e0 tx=0x7f33e00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 -- 172.21.15.46:0/3301995542 shutdown_connections 2023-10-06T12:33:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 --2- 172.21.15.46:0/3301995542 >> v2:172.21.15.132:3300/0 conn(0x7f33e40d0750 0x7f33e40cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 --2- 172.21.15.46:0/3301995542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cb020 0x7f33e40cb3f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 -- 172.21.15.46:0/3301995542 >> 172.21.15.46:0/3301995542 conn(0x7f33e40683d0 msgr2=0x7f33e40d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.984+0000 7f33ea963700 1 -- 172.21.15.46:0/3301995542 shutdown_connections 2023-10-06T12:33:39.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 -- 172.21.15.46:0/3301995542 wait complete. 2023-10-06T12:33:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 Processor -- start 2023-10-06T12:33:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 -- start start 2023-10-06T12:33:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cdc10 0x7f33e4156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33e40d0750 0x7f33e41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e416cdb0 con 0x7f33e40cb020 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e416d090 con 0x7f33e40d0750 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33ea963700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e416d370 con 0x7f33e40cdc10 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33e9961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33e9961700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59662/0 (socket says 172.21.15.46:59662) 2023-10-06T12:33:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.988+0000 7f33e9961700 1 -- 172.21.15.46:0/4055013334 learned_addr learned my addr 172.21.15.46:0/4055013334 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33ea162700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33e40d0750 0x7f33e41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9160700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cdc10 0x7f33e4156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 -- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cdc10 msgr2=0x7f33e4156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cdc10 0x7f33e4156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 -- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33e40d0750 msgr2=0x7f33e41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33e40d0750 0x7f33e41569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 -- 172.21.15.46:0/4055013334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0009420 con 0x7f33e40cb020 2023-10-06T12:33:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33e9961700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f33e40ce860 tx=0x7f33e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0000d10 con 0x7f33e40cb020 2023-10-06T12:33:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0011b10 con 0x7f33e40cb020 2023-10-06T12:33:39.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e416d650 con 0x7f33e40cb020 2023-10-06T12:33:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0025660 con 0x7f33e40cb020 2023-10-06T12:33:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e40c95f0 con 0x7f33e40cb020 2023-10-06T12:33:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.992+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e416d870 con 0x7f33e40cb020 2023-10-06T12:33:39.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:38.996+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f33e0025a50 con 0x7f33e40cb020 2023-10-06T12:33:39.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.000+0000 7f33d9ffb700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 0x7f33c8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:39.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.000+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6092+0+0 (secure 0 0 0) 0x7f33e007e8c0 con 0x7f33e40cb020 2023-10-06T12:33:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.000+0000 7f33e9160700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 0x7f33c8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.000+0000 7f33e9160700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 0x7f33c8062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33e415c0a0 tx=0x7f33d40091a0 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.004+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f33e004ebc0 con 0x7f33e40cb020 2023-10-06T12:33:39.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.172+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f33e4064590 con 0x7f33e40cb020 2023-10-06T12:33:39.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.784+0000 7f33d9ffb700 1 -- 172.21.15.46:0/4055013334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) v1 ==== 170+0+0 (secure 0 0 0) 0x7f33e00521a0 con 0x7f33e40cb020 2023-10-06T12:33:39.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 msgr2=0x7f33c8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:39.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 0x7f33c8062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33e415c0a0 tx=0x7f33d40091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 msgr2=0x7f33e4146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:39.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f33e40ce860 tx=0x7f33e0017000 comp rx=0 tx=0).stop 2023-10-06T12:33:39.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 shutdown_connections 2023-10-06T12:33:39.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f33c8060670 0x7f33c8062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33e40d0750 0x7f33e41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33e40cdc10 0x7f33e4156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 --2- 172.21.15.46:0/4055013334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33e40cb020 0x7f33e4146bb0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:39.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.792+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 >> 172.21.15.46:0/4055013334 conn(0x7f33e40683d0 msgr2=0x7f33e40cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:39.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.796+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 shutdown_connections 2023-10-06T12:33:39.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:39.796+0000 7f33ea963700 1 -- 172.21.15.46:0/4055013334 wait complete. 2023-10-06T12:33:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T12:33:39.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T12:33:40.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.000+0000 7f69029cc700 1 Processor -- start 2023-10-06T12:33:40.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.000+0000 7f69029cc700 1 -- start start 2023-10-06T12:33:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69029cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f68fc0adbe0 0x7f68fc0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69029cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0b0dc0 0x7f68fc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69029cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0b1740 con 0x7f68fc0d0d60 2023-10-06T12:33:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69029cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc159740 con 0x7f68fc0adbe0 2023-10-06T12:33:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69029cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0afd20 con 0x7f68fc0b0dc0 2023-10-06T12:33:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69011c9700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68fc0d0d60 0x7f68fc0b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34052/0 (socket says 172.21.15.46:34052) 2023-10-06T12:33:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69011c9700 1 -- 172.21.15.46:0/322356701 learned_addr learned my addr 172.21.15.46:0/322356701 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:40.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 --2- 172.21.15.46:0/322356701 >> v2:172.21.15.132:3300/0 conn(0x7f68fc0adbe0 0x7f68fc0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69021cb700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0b0dc0 0x7f68fc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69009c8700 1 -- 172.21.15.46:0/322356701 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192285044 0 0) 0x7f68ec001110 con 0x7f68fc0d0d60 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69009c8700 1 -- 172.21.15.46:0/322356701 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e8003570 con 0x7f68fc0d0d60 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 -- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0b0dc0 msgr2=0x7f68fc0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0b0dc0 0x7f68fc0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 -- 172.21.15.46:0/322356701 >> v1:172.21.15.46:6789/0 conn(0x7f68fc0d0d60 legacy=0x7f68fc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 -- 172.21.15.46:0/322356701 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0b0000 con 0x7f68fc0adbe0 2023-10-06T12:33:40.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69019ca700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0adbe0 0x7f68fc0d0820 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68f80098e0 tx=0x7f68f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914f77b8ab3e656d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:40.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.004+0000 7f69009c8700 1 -- 172.21.15.46:0/322356701 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011630 con 0x7f68fc0adbe0 2023-10-06T12:33:40.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69009c8700 1 -- 172.21.15.46:0/322356701 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8011790 con 0x7f68fc0adbe0 2023-10-06T12:33:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69009c8700 1 -- 172.21.15.46:0/322356701 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011b30 con 0x7f68fc0adbe0 2023-10-06T12:33:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0adbe0 msgr2=0x7f68fc0d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0adbe0 0x7f68fc0d0820 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f68f80098e0 tx=0x7f68f80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- 172.21.15.46:0/322356701 shutdown_connections 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0b0dc0 0x7f68fc0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 --2- 172.21.15.46:0/322356701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0adbe0 0x7f68fc0d0820 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- 172.21.15.46:0/322356701 >> 172.21.15.46:0/322356701 conn(0x7f68fc06ef60 msgr2=0x7f68fc0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- 172.21.15.46:0/322356701 shutdown_connections 2023-10-06T12:33:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- 172.21.15.46:0/322356701 wait complete. 2023-10-06T12:33:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 Processor -- start 2023-10-06T12:33:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.008+0000 7f69029cc700 1 -- start start 2023-10-06T12:33:40.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 0x7f68fc15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0d0d60 0x7f68fc14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15f200 con 0x7f68fc0adbe0 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15cb00 con 0x7f68fc0b0dc0 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc15ad10 con 0x7f68fc0d0d60 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59664/0 (socket says 172.21.15.46:59664) 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 -- 172.21.15.46:0/306674364 learned_addr learned my addr 172.21.15.46:0/306674364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69011c9700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 0x7f68fc15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69021cb700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0d0d60 0x7f68fc14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 -- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0d0d60 msgr2=0x7f68fc14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0d0d60 0x7f68fc14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 -- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 msgr2=0x7f68fc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 0x7f68fc15f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 -- 172.21.15.46:0/306674364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8009420 con 0x7f68fc0adbe0 2023-10-06T12:33:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69011c9700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 0x7f68fc15f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69019ca700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f68fc088270 tx=0x7f68f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011b10 con 0x7f68fc0adbe0 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f80253f0 con 0x7f68fc0adbe0 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f80257b0 con 0x7f68fc0adbe0 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc15e8d0 con 0x7f68fc0adbe0 2023-10-06T12:33:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.012+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc15ee90 con 0x7f68fc0adbe0 2023-10-06T12:33:40.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.016+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc064590 con 0x7f68fc0adbe0 2023-10-06T12:33:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.016+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f68f8025ba0 con 0x7f68fc0adbe0 2023-10-06T12:33:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.020+0000 7f68f1ffb700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 0x7f68e0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.020+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f68f807e540 con 0x7f68fc0adbe0 2023-10-06T12:33:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.020+0000 7f69011c9700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 0x7f68e0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.020+0000 7f69011c9700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 0x7f68e0062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68fc16d1b0 tx=0x7f68ec009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.028+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f68f804e680 con 0x7f68fc0adbe0 2023-10-06T12:33:40.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.236+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f68fc068c50 con 0x7f68fc0adbe0 2023-10-06T12:33:40.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.788+0000 7f68f1ffb700 1 -- 172.21.15.46:0/306674364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7f68f8051ca0 con 0x7f68fc0adbe0 2023-10-06T12:33:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 msgr2=0x7f68e0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 0x7f68e0062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68fc16d1b0 tx=0x7f68ec009040 comp rx=0 tx=0).stop 2023-10-06T12:33:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 msgr2=0x7f68fc15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f68fc088270 tx=0x7f68f8017000 comp rx=0 tx=0).stop 2023-10-06T12:33:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 shutdown_connections 2023-10-06T12:33:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f68e0060670 0x7f68e0062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68fc0d0d60 0x7f68fc14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68fc0b0dc0 0x7f68fc15f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 --2- 172.21.15.46:0/306674364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68fc0adbe0 0x7f68fc15c5c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 >> 172.21.15.46:0/306674364 conn(0x7f68fc06ef60 msgr2=0x7f68fc0b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.796+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 shutdown_connections 2023-10-06T12:33:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.800+0000 7f69029cc700 1 -- 172.21.15.46:0/306674364 wait complete. 2023-10-06T12:33:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T12:33:40.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:40.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 Processor -- start 2023-10-06T12:33:40.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 -- start start 2023-10-06T12:33:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff41502f0 0x7f7ff40c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 0x7f7ff4159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff40c67e0 con 0x7f7ff406e2a0 2023-10-06T12:33:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff406e810 con 0x7f7ff41502f0 2023-10-06T12:33:40.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ffa585700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff406eaf0 con 0x7f7ff41506c0 2023-10-06T12:33:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff41502f0 0x7f7ff40c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff41502f0 0x7f7ff40c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59680/0 (socket says 172.21.15.46:59680) 2023-10-06T12:33:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 -- 172.21.15.46:0/1279275674 learned_addr learned my addr 172.21.15.46:0/1279275674 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff37fe700 1 --2- 172.21.15.46:0/1279275674 >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 0x7f7ff4159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 -- 172.21.15.46:0/1279275674 >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 msgr2=0x7f7ff4159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 --2- 172.21.15.46:0/1279275674 >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 0x7f7ff4159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.984+0000 7f7ff3fff700 1 -- 172.21.15.46:0/1279275674 >> v1:172.21.15.46:6789/0 conn(0x7f7ff406e2a0 legacy=0x7f7ff4156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff3fff700 1 -- 172.21.15.46:0/1279275674 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff406edd0 con 0x7f7ff41502f0 2023-10-06T12:33:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff37fe700 1 --2- 172.21.15.46:0/1279275674 >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 0x7f7ff4159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff3fff700 1 --2- 172.21.15.46:0/1279275674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40c0bd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406ba90 tx=0x7f7fe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e083613455dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:40.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff2ffd700 1 -- 172.21.15.46:0/1279275674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400b860 con 0x7f7ff41502f0 2023-10-06T12:33:40.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff2ffd700 1 -- 172.21.15.46:0/1279275674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400b9c0 con 0x7f7ff41502f0 2023-10-06T12:33:40.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ff2ffd700 1 -- 172.21.15.46:0/1279275674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400bd60 con 0x7f7ff41502f0 2023-10-06T12:33:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 -- 172.21.15.46:0/1279275674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 msgr2=0x7f7ff40c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 --2- 172.21.15.46:0/1279275674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40c0bd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406ba90 tx=0x7f7fe4009250 comp rx=0 tx=0).stop 2023-10-06T12:33:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 -- 172.21.15.46:0/1279275674 shutdown_connections 2023-10-06T12:33:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 --2- 172.21.15.46:0/1279275674 >> v2:172.21.15.132:3300/0 conn(0x7f7ff41506c0 0x7f7ff4159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 --2- 172.21.15.46:0/1279275674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40c0bd0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 -- 172.21.15.46:0/1279275674 >> 172.21.15.46:0/1279275674 conn(0x7f7ff40cbe30 msgr2=0x7f7ff40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 -- 172.21.15.46:0/1279275674 shutdown_connections 2023-10-06T12:33:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.988+0000 7f7ffa585700 1 -- 172.21.15.46:0/1279275674 wait complete. 2023-10-06T12:33:40.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 Processor -- start 2023-10-06T12:33:40.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 -- start start 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ff406e2a0 0x7f7ff4144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ff41506c0 0x7f7ff414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff416a5d0 con 0x7f7ff41502f0 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40ad810 con 0x7f7ff406e2a0 2023-10-06T12:33:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.992+0000 7f7ffa585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4169f80 con 0x7f7ff41506c0 2023-10-06T12:33:41.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59696/0 (socket says 172.21.15.46:59696) 2023-10-06T12:33:41.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 -- 172.21.15.46:0/157696114 learned_addr learned my addr 172.21.15.46:0/157696114 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff3fff700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ff406e2a0 0x7f7ff4144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff8b22700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ff41506c0 0x7f7ff414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 -- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ff41506c0 msgr2=0x7f7ff414e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ff41506c0 0x7f7ff414e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 -- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ff406e2a0 msgr2=0x7f7ff4144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ff406e2a0 0x7f7ff4144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 -- 172.21.15.46:0/157696114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4009490 con 0x7f7ff41502f0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ff37fe700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d860 tx=0x7f7fe8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80125f0 con 0x7f7ff41502f0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8012750 con 0x7f7ff41502f0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff416a260 con 0x7f7ff41502f0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8012af0 con 0x7f7ff41502f0 2023-10-06T12:33:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff41457c0 con 0x7f7ff41502f0 2023-10-06T12:33:41.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:40.996+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff416a740 con 0x7f7ff41502f0 2023-10-06T12:33:41.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.000+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7fe8013480 con 0x7f7ff41502f0 2023-10-06T12:33:41.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.000+0000 7f7fdbfff700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 0x7f7fe0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.000+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f7fe807cea0 con 0x7f7ff41502f0 2023-10-06T12:33:41.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.000+0000 7f7ff3fff700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 0x7f7fe0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.000+0000 7f7ff3fff700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 0x7f7fe0062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406ba90 tx=0x7f7fe4017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.004+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7fe804cf80 con 0x7f7ff41502f0 2023-10-06T12:33:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.176+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ff4063f90 con 0x7f7ff41502f0 2023-10-06T12:33:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.176+0000 7f7fdbfff700 1 -- 172.21.15.46:0/157696114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+17680 (secure 0 0 0) 0x7f7fe80505a0 con 0x7f7ff41502f0 2023-10-06T12:33:41.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:41.185 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":152,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:40.244427+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 msgr2=0x7f7fe0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 0x7f7fe0062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7ff406ba90 tx=0x7f7fe4017000 comp rx=0 tx=0).stop 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 msgr2=0x7f7ff40acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d860 tx=0x7f7fe8008040 comp rx=0 tx=0).stop 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 shutdown_connections 2023-10-06T12:33:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7fe0060670 0x7f7fe0062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ff41506c0 0x7f7ff414e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff41502f0 0x7f7ff40acdc0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 --2- 172.21.15.46:0/157696114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ff406e2a0 0x7f7ff4144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.180+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 >> 172.21.15.46:0/157696114 conn(0x7f7ff40cbe30 msgr2=0x7f7ff40c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.184+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 shutdown_connections 2023-10-06T12:33:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.184+0000 7f7ffa585700 1 -- 172.21.15.46:0/157696114 wait complete. 2023-10-06T12:33:41.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.368+0000 7f293dfc5700 1 Processor -- start 2023-10-06T12:33:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.368+0000 7f293dfc5700 1 -- start start 2023-10-06T12:33:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f293dfc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f293dfc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f293dfc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380c67e0 con 0x7f29381506c0 2023-10-06T12:33:41.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f293dfc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293806e810 con 0x7f293815a1a0 2023-10-06T12:33:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f293dfc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293806eaf0 con 0x7f29381502f0 2023-10-06T12:33:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f29377fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.372+0000 7f2937fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2937fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:44682/0 (socket says 172.21.15.46:44682) 2023-10-06T12:33:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:46134/0 (socket says 172.21.15.46:46134) 2023-10-06T12:33:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2937fff700 1 -- 172.21.15.46:0/654534432 learned_addr learned my addr 172.21.15.46:0/654534432 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f292ffff700 1 --1- 172.21.15.46:0/654534432 >> v1:172.21.15.132:6789/0 conn(0x7f29381506c0 0x7f2938159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:57212/0 (socket says 172.21.15.46:57212) 2023-10-06T12:33:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 -- 172.21.15.46:0/654534432 >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 msgr2=0x7f29380c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:41.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 --2- 172.21.15.46:0/654534432 >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 -- 172.21.15.46:0/654534432 >> v1:172.21.15.132:6789/0 conn(0x7f29381506c0 legacy=0x7f2938159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 -- 172.21.15.46:0/654534432 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293806edd0 con 0x7f29381502f0 2023-10-06T12:33:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2937fff700 1 --2- 172.21.15.46:0/654534432 >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f29377fe700 1 --2- 172.21.15.46:0/654534432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29280098d0 tx=0x7f2928009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eff27df0729468e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2936ffd700 1 -- 172.21.15.46:0/654534432 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928005c10 con 0x7f29381502f0 2023-10-06T12:33:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2936ffd700 1 -- 172.21.15.46:0/654534432 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2928005d70 con 0x7f29381502f0 2023-10-06T12:33:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f2936ffd700 1 -- 172.21.15.46:0/654534432 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928006280 con 0x7f29381502f0 2023-10-06T12:33:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 -- 172.21.15.46:0/654534432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 msgr2=0x7f29380c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 --2- 172.21.15.46:0/654534432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29280098d0 tx=0x7f2928009120 comp rx=0 tx=0).stop 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 -- 172.21.15.46:0/654534432 shutdown_connections 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 --2- 172.21.15.46:0/654534432 >> v2:172.21.15.132:3300/0 conn(0x7f293815a1a0 0x7f29380c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 --2- 172.21.15.46:0/654534432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29381502f0 0x7f29380c0bd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 -- 172.21.15.46:0/654534432 >> 172.21.15.46:0/654534432 conn(0x7f29380cbe30 msgr2=0x7f29380cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 -- 172.21.15.46:0/654534432 shutdown_connections 2023-10-06T12:33:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.376+0000 7f293dfc5700 1 -- 172.21.15.46:0/654534432 wait complete. 2023-10-06T12:33:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 Processor -- start 2023-10-06T12:33:41.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 -- start start 2023-10-06T12:33:41.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29381506c0 0x7f293814ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815a1a0 0x7f29380c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380af880 con 0x7f29381502f0 2023-10-06T12:33:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380aec70 con 0x7f29381506c0 2023-10-06T12:33:41.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f293dfc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380aef50 con 0x7f293815a1a0 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59712/0 (socket says 172.21.15.46:59712) 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 -- 172.21.15.46:0/3341986815 learned_addr learned my addr 172.21.15.46:0/3341986815 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f292ffff700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29381506c0 0x7f293814ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f2937fff700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815a1a0 0x7f29380c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 -- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815a1a0 msgr2=0x7f29380c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815a1a0 0x7f29380c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 -- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29381506c0 msgr2=0x7f293814ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29381506c0 0x7f293814ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 -- 172.21.15.46:0/3341986815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930009ae0 con 0x7f29381502f0 2023-10-06T12:33:41.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.380+0000 7f29377fe700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f293806ec30 tx=0x7f2928016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292801d6b0 con 0x7f29381502f0 2023-10-06T12:33:41.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292801d810 con 0x7f29381502f0 2023-10-06T12:33:41.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29280093c0 con 0x7f29381502f0 2023-10-06T12:33:41.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292801dbd0 con 0x7f29381502f0 2023-10-06T12:33:41.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29380af510 con 0x7f29381502f0 2023-10-06T12:33:41.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938063f90 con 0x7f29381502f0 2023-10-06T12:33:41.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.384+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f292801b480 con 0x7f29381502f0 2023-10-06T12:33:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.388+0000 7f292f7fe700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 0x7f2918062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.388+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f292807e9a0 con 0x7f29381502f0 2023-10-06T12:33:41.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.388+0000 7f292ffff700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 0x7f2918062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.388+0000 7f292ffff700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 0x7f2918062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29380b1bf0 tx=0x7f2920007eb0 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.396+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f292804eb20 con 0x7f29381502f0 2023-10-06T12:33:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.572+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29380c6470 con 0x7f29381502f0 2023-10-06T12:33:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f292f7fe700 1 -- 172.21.15.46:0/3341986815 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+17667 (secure 0 0 0) 0x7f2928006c50 con 0x7f29381502f0 2023-10-06T12:33:41.582 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:41.583 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":153,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:41.256157+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 msgr2=0x7f2918062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 0x7f2918062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29380b1bf0 tx=0x7f2920007eb0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 msgr2=0x7f2938156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f293806ec30 tx=0x7f2928016000 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 shutdown_connections 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f2918060670 0x7f2918062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f293815a1a0 0x7f29380c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29381506c0 0x7f293814ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 --2- 172.21.15.46:0/3341986815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29381502f0 0x7f2938156110 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.576+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 >> 172.21.15.46:0/3341986815 conn(0x7f29380cbe30 msgr2=0x7f29380a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.580+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 shutdown_connections 2023-10-06T12:33:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.580+0000 7f293dfc5700 1 -- 172.21.15.46:0/3341986815 wait complete. 2023-10-06T12:33:41.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T12:33:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.768+0000 7f0307cf0700 1 Processor -- start 2023-10-06T12:33:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.768+0000 7f0307cf0700 1 -- start start 2023-10-06T12:33:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0307cf0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03000ca430 0x7f03000c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0307cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000a9840 0x7f03000c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0307cf0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000c1630 con 0x7f03000c7d80 2023-10-06T12:33:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0307cf0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03000c0d00 con 0x7f03000ca430 2023-10-06T12:33:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0307cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c0fe0 con 0x7f03000a9840 2023-10-06T12:33:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0305a8c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03000c7d80 0x7f03000c6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34082/0 (socket says 172.21.15.46:34082) 2023-10-06T12:33:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0305a8c700 1 -- 172.21.15.46:0/2933831103 learned_addr learned my addr 172.21.15.46:0/2933831103 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030528b700 1 --2- 172.21.15.46:0/2933831103 >> v2:172.21.15.46:3300/0 conn(0x7f03000ca430 0x7f03000c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030628d700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000a9840 0x7f03000c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0304a8a700 1 -- 172.21.15.46:0/2933831103 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257304656 0 0) 0x7f03000c1630 con 0x7f03000c7d80 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f0304a8a700 1 -- 172.21.15.46:0/2933831103 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ec003570 con 0x7f03000c7d80 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030528b700 1 -- 172.21.15.46:0/2933831103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000a9840 msgr2=0x7f03000c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030528b700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000a9840 0x7f03000c1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030528b700 1 -- 172.21.15.46:0/2933831103 >> v1:172.21.15.46:6789/0 conn(0x7f03000c7d80 legacy=0x7f03000c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.772+0000 7f030528b700 1 -- 172.21.15.46:0/2933831103 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000c12c0 con 0x7f03000ca430 2023-10-06T12:33:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f030528b700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000ca430 0x7f03000c8150 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098e0 tx=0x7f02f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91b8ce5fb142dbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0304a8a700 1 -- 172.21.15.46:0/2933831103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011630 con 0x7f03000ca430 2023-10-06T12:33:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0304a8a700 1 -- 172.21.15.46:0/2933831103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0011790 con 0x7f03000ca430 2023-10-06T12:33:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0304a8a700 1 -- 172.21.15.46:0/2933831103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011b30 con 0x7f03000ca430 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 -- 172.21.15.46:0/2933831103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000ca430 msgr2=0x7f03000c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000ca430 0x7f03000c8150 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098e0 tx=0x7f02f00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 -- 172.21.15.46:0/2933831103 shutdown_connections 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000a9840 0x7f03000c1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2933831103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000ca430 0x7f03000c8150 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 -- 172.21.15.46:0/2933831103 >> 172.21.15.46:0/2933831103 conn(0x7f03000cbe30 msgr2=0x7f03000cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 -- 172.21.15.46:0/2933831103 shutdown_connections 2023-10-06T12:33:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 -- 172.21.15.46:0/2933831103 wait complete. 2023-10-06T12:33:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.776+0000 7f0307cf0700 1 Processor -- start 2023-10-06T12:33:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 -- start start 2023-10-06T12:33:41.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 0x7f030016d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000ca430 0x7f03001589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030016e4b0 con 0x7f03000c7d80 2023-10-06T12:33:41.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030015adc0 con 0x7f03000a9840 2023-10-06T12:33:41.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030016def0 con 0x7f03000ca430 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59726/0 (socket says 172.21.15.46:59726) 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 -- 172.21.15.46:0/2822851753 learned_addr learned my addr 172.21.15.46:0/2822851753 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0305a8c700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 0x7f030016d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030628d700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000ca430 0x7f03001589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 -- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000ca430 msgr2=0x7f03001589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000ca430 0x7f03001589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 -- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 msgr2=0x7f030016d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 0x7f030016d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 -- 172.21.15.46:0/2822851753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009420 con 0x7f03000c7d80 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0305a8c700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 0x7f030016d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f030528b700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ed0 tx=0x7f02f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011b10 con 0x7f03000c7d80 2023-10-06T12:33:41.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f00253f0 con 0x7f03000c7d80 2023-10-06T12:33:41.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00257b0 con 0x7f03000c7d80 2023-10-06T12:33:41.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.780+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030016e1d0 con 0x7f03000c7d80 2023-10-06T12:33:41.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.784+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030016dac0 con 0x7f03000c7d80 2023-10-06T12:33:41.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.784+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030015a8b0 con 0x7f03000c7d80 2023-10-06T12:33:41.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.784+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f02f0025ba0 con 0x7f03000c7d80 2023-10-06T12:33:41.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.788+0000 7f02f5ffb700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 0x7f02e4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:41.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.788+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f02f007e5c0 con 0x7f03000c7d80 2023-10-06T12:33:41.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.788+0000 7f0305a8c700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 0x7f02e4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:41.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.788+0000 7f0305a8c700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 0x7f02e4062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0058d0 tx=0x7f02fc009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:41.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:41.796+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f030015a8b0 con 0x7f03000c7d80 2023-10-06T12:33:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.008+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f030015a8b0 con 0x7f03000c7d80 2023-10-06T12:33:42.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.820+0000 7f02f5ffb700 1 -- 172.21.15.46:0/2822851753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f030015a8b0 con 0x7f03000c7d80 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 msgr2=0x7f02e4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 0x7f02e4062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0058d0 tx=0x7f02fc009040 comp rx=0 tx=0).stop 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 msgr2=0x7f030006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ed0 tx=0x7f02f0017000 comp rx=0 tx=0).stop 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 shutdown_connections 2023-10-06T12:33:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f02e4060670 0x7f02e4062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03000ca430 0x7f03001589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03000c7d80 0x7f030006d570 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 --2- 172.21.15.46:0/2822851753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03000a9840 0x7f030016d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 >> 172.21.15.46:0/2822851753 conn(0x7f03000cbe30 msgr2=0x7f0300157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 shutdown_connections 2023-10-06T12:33:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:42.828+0000 7f0307cf0700 1 -- 172.21.15.46:0/2822851753 wait complete. 2023-10-06T12:33:42.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T12:33:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.028+0000 7f08e8f85700 1 Processor -- start 2023-10-06T12:33:43.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.028+0000 7f08e8f85700 1 -- start start 2023-10-06T12:33:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e8f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08e40c7d80 0x7f08e40c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e8f85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08e40a9840 0x7f08e40c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e8f85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e40c1630 con 0x7f08e40ca430 2023-10-06T12:33:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e8f85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40c0d00 con 0x7f08e40a9840 2023-10-06T12:33:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e8f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40c0fe0 con 0x7f08e40c7d80 2023-10-06T12:33:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08e40a9840 0x7f08e40c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08e40a9840 0x7f08e40c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59728/0 (socket says 172.21.15.46:59728) 2023-10-06T12:33:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 -- 172.21.15.46:0/28214826 learned_addr learned my addr 172.21.15.46:0/28214826 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e259c700 1 --2- 172.21.15.46:0/28214826 >> v2:172.21.15.132:3300/0 conn(0x7f08e40c7d80 0x7f08e40c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 -- 172.21.15.46:0/28214826 >> v2:172.21.15.132:3300/0 conn(0x7f08e40c7d80 msgr2=0x7f08e40c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 --2- 172.21.15.46:0/28214826 >> v2:172.21.15.132:3300/0 conn(0x7f08e40c7d80 0x7f08e40c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 -- 172.21.15.46:0/28214826 >> v1:172.21.15.132:6789/0 conn(0x7f08e40ca430 legacy=0x7f08e40c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.032+0000 7f08e2d9d700 1 -- 172.21.15.46:0/28214826 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e40c12c0 con 0x7f08e40a9840 2023-10-06T12:33:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e2d9d700 1 --2- 172.21.15.46:0/28214826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40a9840 0x7f08e40c1ac0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f08e406c640 tx=0x7f08dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb14a8a386482d08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e159a700 1 -- 172.21.15.46:0/28214826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00e070 con 0x7f08e40a9840 2023-10-06T12:33:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e159a700 1 -- 172.21.15.46:0/28214826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc00ba20 con 0x7f08e40a9840 2023-10-06T12:33:43.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e159a700 1 -- 172.21.15.46:0/28214826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0092e0 con 0x7f08e40a9840 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/28214826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40a9840 msgr2=0x7f08e40c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- 172.21.15.46:0/28214826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40a9840 0x7f08e40c1ac0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f08e406c640 tx=0x7f08dc009840 comp rx=0 tx=0).stop 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/28214826 shutdown_connections 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- 172.21.15.46:0/28214826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40a9840 0x7f08e40c1ac0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- 172.21.15.46:0/28214826 >> v2:172.21.15.132:3300/0 conn(0x7f08e40c7d80 0x7f08e40c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/28214826 >> 172.21.15.46:0/28214826 conn(0x7f08e40cbe30 msgr2=0x7f08e40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/28214826 shutdown_connections 2023-10-06T12:33:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/28214826 wait complete. 2023-10-06T12:33:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 Processor -- start 2023-10-06T12:33:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- start start 2023-10-06T12:33:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40a9840 0x7f08e40ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40ca430 0x7f08e40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4156360 con 0x7f08e40c7d80 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b19d0 con 0x7f08e40a9840 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40aebd0 con 0x7f08e40ca430 2023-10-06T12:33:43.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40a9840 0x7f08e40ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59738/0 (socket says 172.21.15.46:59738) 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 -- 172.21.15.46:0/1796478680 learned_addr learned my addr 172.21.15.46:0/1796478680 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e2d9d700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40ca430 0x7f08e40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 -- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40ca430 msgr2=0x7f08e40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40ca430 0x7f08e40b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 -- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40a9840 msgr2=0x7f08e40ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40a9840 0x7f08e40ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 -- 172.21.15.46:0/1796478680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc009ae0 con 0x7f08e40c7d80 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e1d9b700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f08d800c4e0 tx=0x7f08d800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d800cea0 con 0x7f08e40c7d80 2023-10-06T12:33:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d8011590 con 0x7f08e40c7d80 2023-10-06T12:33:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8011930 con 0x7f08e40c7d80 2023-10-06T12:33:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40af470 con 0x7f08e40c7d80 2023-10-06T12:33:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.040+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40aee20 con 0x7f08e40c7d80 2023-10-06T12:33:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.044+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e40af600 con 0x7f08e40c7d80 2023-10-06T12:33:43.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.044+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f08d8011d20 con 0x7f08e40c7d80 2023-10-06T12:33:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.048+0000 7f08d27fc700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 0x7f08c4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.048+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f08d807c020 con 0x7f08e40c7d80 2023-10-06T12:33:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.048+0000 7f08e259c700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 0x7f08c4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.048+0000 7f08e259c700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 0x7f08c4062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009db0 tx=0x7f08d40091a0 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:43.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.052+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f08d804c1a0 con 0x7f08e40c7d80 2023-10-06T12:33:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.256+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f08e40660f0 con 0x7f08e40c7d80 2023-10-06T12:33:43.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.836+0000 7f08d27fc700 1 -- 172.21.15.46:0/1796478680 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08d80170c0 con 0x7f08e40c7d80 2023-10-06T12:33:43.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 msgr2=0x7f08c4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 0x7f08c4062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009db0 tx=0x7f08d40091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 msgr2=0x7f08e4155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:43.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f08d800c4e0 tx=0x7f08d800c7f0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 shutdown_connections 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08c4060670 0x7f08c4062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08e40ca430 0x7f08e40b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08e40c7d80 0x7f08e4155e20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 --2- 172.21.15.46:0/1796478680 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08e40a9840 0x7f08e40ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 >> 172.21.15.46:0/1796478680 conn(0x7f08e40cbe30 msgr2=0x7f08e4154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 shutdown_connections 2023-10-06T12:33:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:43.840+0000 7f08e8f85700 1 -- 172.21.15.46:0/1796478680 wait complete. 2023-10-06T12:33:43.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 Processor -- start 2023-10-06T12:33:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 -- start start 2023-10-06T12:33:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f360c06e8a0 0x7f360c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c06d460 con 0x7f360c15ab60 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0c7000 con 0x7f360c0ad6c0 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f3610985700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c06ed40 con 0x7f360c06e8a0 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f360a59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f360c15ab60 0x7f360c15af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56644/0 (socket says 172.21.15.46:56644) 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.024+0000 7f360a59c700 1 -- 172.21.15.46:0/3981796044 learned_addr learned my addr 172.21.15.46:0/3981796044 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f3609d9b700 1 --2- 172.21.15.46:0/3981796044 >> v2:172.21.15.132:3300/0 conn(0x7f360c06e8a0 0x7f360c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773681384 0 0) 0x7f360c06d460 con 0x7f360c15ab60 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f4003570 con 0x7f360c15ab60 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102695723 0 0) 0x7f360c0c7000 con 0x7f360c0ad6c0 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f360c06d460 con 0x7f360c0ad6c0 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f3609d9b700 1 -- 172.21.15.46:0/3981796044 >> v1:172.21.15.132:6789/0 conn(0x7f360c0ad6c0 legacy=0x7f360c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f3609d9b700 1 -- 172.21.15.46:0/3981796044 >> v1:172.21.15.46:6789/0 conn(0x7f360c15ab60 legacy=0x7f360c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f3609d9b700 1 -- 172.21.15.46:0/3981796044 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0c72c0 con 0x7f360c06e8a0 2023-10-06T12:33:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f3609d9b700 1 --2- 172.21.15.46:0/3981796044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c06e8a0 0x7f360c06cc30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f36000098e0 tx=0x7f36000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afa67800f5a20fa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600011630 con 0x7f360c06e8a0 2023-10-06T12:33:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600011790 con 0x7f360c06e8a0 2023-10-06T12:33:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.028+0000 7f360959a700 1 -- 172.21.15.46:0/3981796044 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600011b30 con 0x7f360c06e8a0 2023-10-06T12:33:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- 172.21.15.46:0/3981796044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c06e8a0 msgr2=0x7f360c06cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 --2- 172.21.15.46:0/3981796044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c06e8a0 0x7f360c06cc30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f36000098e0 tx=0x7f36000091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- 172.21.15.46:0/3981796044 shutdown_connections 2023-10-06T12:33:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 --2- 172.21.15.46:0/3981796044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c06e8a0 0x7f360c06cc30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- 172.21.15.46:0/3981796044 >> 172.21.15.46:0/3981796044 conn(0x7f360c0cbe30 msgr2=0x7f360c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- 172.21.15.46:0/3981796044 shutdown_connections 2023-10-06T12:33:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- 172.21.15.46:0/3981796044 wait complete. 2023-10-06T12:33:44.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 Processor -- start 2023-10-06T12:33:44.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.032+0000 7f3610985700 1 -- start start 2023-10-06T12:33:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c06e8a0 0x7f360c0c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0ad6c0 0x7f360c15c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0c6a00 con 0x7f360c15ab60 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0c5c40 con 0x7f360c0ad6c0 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0c5f20 con 0x7f360c06e8a0 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38590/0 (socket says 172.21.15.46:38590) 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 -- 172.21.15.46:0/3118772133 learned_addr learned my addr 172.21.15.46:0/3118772133 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3609d9b700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0ad6c0 0x7f360c15c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360a59c700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c06e8a0 0x7f360c0c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 -- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c06e8a0 msgr2=0x7f360c0c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c06e8a0 0x7f360c0c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 -- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0ad6c0 msgr2=0x7f360c15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0ad6c0 0x7f360c15c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 -- 172.21.15.46:0/3118772133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009420 con 0x7f360c15ab60 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f360ad9d700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f360400cb10 tx=0x7f36040027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040040f0 con 0x7f360c15ab60 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360400a7e0 con 0x7f360c15ab60 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360400aba0 con 0x7f360c15ab60 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0c6200 con 0x7f360c15ab60 2023-10-06T12:33:44.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.036+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c0c58a0 con 0x7f360c15ab60 2023-10-06T12:33:44.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.040+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c063f90 con 0x7f360c15ab60 2023-10-06T12:33:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.040+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3604012420 con 0x7f360c15ab60 2023-10-06T12:33:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.040+0000 7f35fa7fc700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 0x7f35ec062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.040+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f360407cc90 con 0x7f360c15ab60 2023-10-06T12:33:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.040+0000 7f360a59c700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 0x7f35ec062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.044+0000 7f360a59c700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 0x7f35ec062b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35fc001360 tx=0x7f35fc009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.052+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f360404ce30 con 0x7f360c15ab60 2023-10-06T12:33:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.252+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f360c04b5c0 con 0x7f360c15ab60 2023-10-06T12:33:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f35fa7fc700 1 -- 172.21.15.46:0/3118772133 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f3604050450 con 0x7f360c15ab60 2023-10-06T12:33:44.262 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:44.263 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:33:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 msgr2=0x7f35ec062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 0x7f35ec062b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35fc001360 tx=0x7f35fc009040 comp rx=0 tx=0).stop 2023-10-06T12:33:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 msgr2=0x7f360c069df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f360400cb10 tx=0x7f36040027a0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 shutdown_connections 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35ec0606a0 0x7f35ec062b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360c15ab60 0x7f360c069df0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f360c0ad6c0 0x7f360c15c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 --2- 172.21.15.46:0/3118772133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f360c06e8a0 0x7f360c0c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.256+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 >> 172.21.15.46:0/3118772133 conn(0x7f360c0cbe30 msgr2=0x7f360c0ac5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.260+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 shutdown_connections 2023-10-06T12:33:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.260+0000 7f3610985700 1 -- 172.21.15.46:0/3118772133 wait complete. 2023-10-06T12:33:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:33:44.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:33:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.440+0000 7fd6de4e3700 1 Processor -- start 2023-10-06T12:33:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.440+0000 7fd6de4e3700 1 -- start start 2023-10-06T12:33:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6de4e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6d80cb4c0 0x7fd6d80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6de4e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80cf100 con 0x7fd6d80af3d0 2023-10-06T12:33:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6de4e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d80ce4f0 con 0x7fd6d80af000 2023-10-06T12:33:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6de4e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80ce300 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6ddce2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6d80af3d0 0x7fd6d80690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56648/0 (socket says 172.21.15.46:56648) 2023-10-06T12:33:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6ddce2700 1 -- 172.21.15.46:0/2746660208 learned_addr learned my addr 172.21.15.46:0/2746660208 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6dcce0700 1 --2- 172.21.15.46:0/2746660208 >> v2:172.21.15.132:3300/0 conn(0x7fd6d80cb4c0 0x7fd6d80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536323184 0 0) 0x7fd6d80cf100 con 0x7fd6d80af3d0 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c4003570 con 0x7fd6d80af3d0 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171468169 0 0) 0x7fd6d80ce4f0 con 0x7fd6d80af000 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d80cf100 con 0x7fd6d80af000 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6dcce0700 1 -- 172.21.15.46:0/2746660208 >> v1:172.21.15.132:6789/0 conn(0x7fd6d80af000 legacy=0x7fd6d80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6dcce0700 1 -- 172.21.15.46:0/2746660208 >> v1:172.21.15.46:6789/0 conn(0x7fd6d80af3d0 legacy=0x7fd6d80690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6dcce0700 1 -- 172.21.15.46:0/2746660208 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80ce7d0 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6dcce0700 1 --2- 172.21.15.46:0/2746660208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d80cf590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098e0 tx=0x7fd6c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95c87568e03c18a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.444+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8011630 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8011790 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6cffff700 1 -- 172.21.15.46:0/2746660208 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8011b30 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2746660208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 msgr2=0x7fd6d80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2746660208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d80cf590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098e0 tx=0x7fd6c80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2746660208 shutdown_connections 2023-10-06T12:33:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2746660208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d80cf590 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2746660208 >> 172.21.15.46:0/2746660208 conn(0x7fd6d80683b0 msgr2=0x7fd6d80d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2746660208 shutdown_connections 2023-10-06T12:33:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2746660208 wait complete. 2023-10-06T12:33:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 Processor -- start 2023-10-06T12:33:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.448+0000 7fd6de4e3700 1 -- start start 2023-10-06T12:33:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d80af3d0 0x7fd6d80b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d8156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b5740 con 0x7fd6d80af000 2023-10-06T12:33:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b4b30 con 0x7fd6d80cb4c0 2023-10-06T12:33:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b4e10 con 0x7fd6d80af3d0 2023-10-06T12:33:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38604/0 (socket says 172.21.15.46:38604) 2023-10-06T12:33:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 -- 172.21.15.46:0/2999440744 learned_addr learned my addr 172.21.15.46:0/2999440744 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dcce0700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d80af3d0 0x7fd6d80b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6ddce2700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d8156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 -- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d80af3d0 msgr2=0x7fd6d80b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d80af3d0 0x7fd6d80b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 -- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 msgr2=0x7fd6d8156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d8156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 -- 172.21.15.46:0/2999440744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009420 con 0x7fd6d80af000 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6ddce2700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:44.457 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6dd4e1700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c790 tx=0x7fd6d4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4010510 con 0x7fd6d80af000 2023-10-06T12:33:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4010670 con 0x7fd6d80af000 2023-10-06T12:33:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4010a30 con 0x7fd6d80af000 2023-10-06T12:33:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d80b50f0 con 0x7fd6d80af000 2023-10-06T12:33:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d80c9fd0 con 0x7fd6d80af000 2023-10-06T12:33:44.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d8064590 con 0x7fd6d80af000 2023-10-06T12:33:44.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.452+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd6d401b3b0 con 0x7fd6d80af000 2023-10-06T12:33:44.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.456+0000 7fd6cd7fa700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 0x7fd6bc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.456+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fd6d407c5d0 con 0x7fd6d80af000 2023-10-06T12:33:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.456+0000 7fd6dcce0700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 0x7fd6bc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.456+0000 7fd6dcce0700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 0x7fd6bc062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098e0 tx=0x7fd6c8017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.464+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6d404c6b0 con 0x7fd6d80af000 2023-10-06T12:33:44.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.632+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d80ceab0 con 0x7fd6d80af000 2023-10-06T12:33:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.640+0000 7fd6cd7fa700 1 -- 172.21.15.46:0/2999440744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+17667 (secure 0 0 0) 0x7fd6d4017090 con 0x7fd6d80af000 2023-10-06T12:33:44.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:44.643 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":153,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:41.256157+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:33:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 msgr2=0x7fd6bc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 0x7fd6bc062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098e0 tx=0x7fd6c8017000 comp rx=0 tx=0).stop 2023-10-06T12:33:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 msgr2=0x7fd6d814ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c790 tx=0x7fd6d4007e10 comp rx=0 tx=0).stop 2023-10-06T12:33:44.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 shutdown_connections 2023-10-06T12:33:44.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd6bc060670 0x7fd6bc062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d80cb4c0 0x7fd6d8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d80af3d0 0x7fd6d80b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 --2- 172.21.15.46:0/2999440744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d80af000 0x7fd6d814ef50 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 >> 172.21.15.46:0/2999440744 conn(0x7fd6d80683b0 msgr2=0x7fd6d80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 shutdown_connections 2023-10-06T12:33:44.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.644+0000 7fd6de4e3700 1 -- 172.21.15.46:0/2999440744 wait complete. 2023-10-06T12:33:44.667 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T12:33:44.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:33:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.824+0000 7fce1b9c5700 1 Processor -- start 2023-10-06T12:33:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.824+0000 7fce1b9c5700 1 -- start start 2023-10-06T12:33:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce1b9c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140c7b50 0x7fce140c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce1b9c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140c8500 con 0x7fce140c0bd0 2023-10-06T12:33:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce1b9c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140c6470 con 0x7fce14156c80 2023-10-06T12:33:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce1b9c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce140c7760 con 0x7fce140c7b50 2023-10-06T12:33:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce19f62700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce140c0bd0 0x7fce140c1050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56650/0 (socket says 172.21.15.46:56650) 2023-10-06T12:33:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce19f62700 1 -- 172.21.15.46:0/1642372729 learned_addr learned my addr 172.21.15.46:0/1642372729 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce18f60700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140c7b50 0x7fce140c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278318940 0 0) 0x7fce140c8500 con 0x7fce140c0bd0 2023-10-06T12:33:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce00003570 con 0x7fce140c0bd0 2023-10-06T12:33:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380655976 0 0) 0x7fce140c6470 con 0x7fce14156c80 2023-10-06T12:33:44.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce140c8500 con 0x7fce14156c80 2023-10-06T12:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008395411 0 0) 0x7fce00003570 con 0x7fce140c0bd0 2023-10-06T12:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140c7b50 msgr2=0x7fce140c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140c7b50 0x7fce140c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 >> v1:172.21.15.132:6789/0 conn(0x7fce14156c80 legacy=0x7fce140ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140c66c0 con 0x7fce140c0bd0 2023-10-06T12:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fce0c0017f0 con 0x7fce140c0bd0 2023-10-06T12:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 >> v1:172.21.15.46:6789/0 conn(0x7fce140c0bd0 legacy=0x7fce140c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce00003c20 0x7fce00004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce00004810 0x7fce00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140c7760 con 0x7fce000051c0 2023-10-06T12:33:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0000c020 con 0x7fce00004810 2023-10-06T12:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.828+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0000c1e0 con 0x7fce00003c20 2023-10-06T12:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce18f60700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140c7b50 0x7fce140c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:44.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce18f60700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce00004810 0x7fce00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19761700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce00003c20 0x7fce00004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 -- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce00003c20 msgr2=0x7fce00004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce00003c20 0x7fce00004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 -- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce00004810 msgr2=0x7fce00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce00004810 0x7fce00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 -- 172.21.15.46:0/1642372729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00003570 con 0x7fce000051c0 2023-10-06T12:33:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce18f60700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce00004810 0x7fce00004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce19f62700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce0000ba60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00a8d0 tx=0x7fce0c002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1559fad9378b3be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0070d0 con 0x7fce000051c0 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c007230 con 0x7fce000051c0 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.832+0000 7fce0bfff700 1 -- 172.21.15.46:0/1642372729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0075f0 con 0x7fce000051c0 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- 172.21.15.46:0/1642372729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 msgr2=0x7fce0000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/1642372729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce0000ba60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00a8d0 tx=0x7fce0c002190 comp rx=0 tx=0).stop 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce19f62700 1 -- 172.21.15.46:0/1642372729 reap_dead start 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- 172.21.15.46:0/1642372729 shutdown_connections 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- 172.21.15.46:0/1642372729 >> 172.21.15.46:0/1642372729 conn(0x7fce140cbe30 msgr2=0x7fce140cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- 172.21.15.46:0/1642372729 shutdown_connections 2023-10-06T12:33:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- 172.21.15.46:0/1642372729 wait complete. 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 Processor -- start 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.836+0000 7fce1b9c5700 1 -- start start 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce1415a400 0x7fce1415a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140ad3a0 0x7fce140ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140af960 con 0x7fce000051c0 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140aba80 con 0x7fce1415a400 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce140afee0 con 0x7fce140ad3a0 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38618/0 (socket says 172.21.15.46:38618) 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 -- 172.21.15.46:0/3803502213 learned_addr learned my addr 172.21.15.46:0/3803502213 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce18f60700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce1415a400 0x7fce1415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19f62700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140ad3a0 0x7fce140ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 -- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140ad3a0 msgr2=0x7fce140ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140ad3a0 0x7fce140ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 -- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce1415a400 msgr2=0x7fce1415a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce1415a400 0x7fce1415a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 -- 172.21.15.46:0/3803502213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1415ad90 con 0x7fce000051c0 2023-10-06T12:33:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce19761700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fce1000f880 tx=0x7fce10007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000d670 con 0x7fce000051c0 2023-10-06T12:33:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce14151f60 con 0x7fce000051c0 2023-10-06T12:33:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1000d9e0 con 0x7fce000051c0 2023-10-06T12:33:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1415b2f0 con 0x7fce000051c0 2023-10-06T12:33:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.840+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000db50 con 0x7fce000051c0 2023-10-06T12:33:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.844+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1415b480 con 0x7fce000051c0 2023-10-06T12:33:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.844+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fce100133f0 con 0x7fce000051c0 2023-10-06T12:33:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.844+0000 7fce097fa700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 0x7fcdf8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.844+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fce1007ca30 con 0x7fce000051c0 2023-10-06T12:33:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.844+0000 7fce18f60700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 0x7fcdf8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.848+0000 7fce18f60700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 0x7fcdf8062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce140ab350 tx=0x7fce04008040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:44.852+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce1004cb10 con 0x7fce000051c0 2023-10-06T12:33:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.020+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce140b0020 con 0x7fcdf8060670 2023-10-06T12:33:45.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.040+0000 7fce097fa700 1 -- 172.21.15.46:0/3803502213 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293523 (secure 0 0 0) 0x7fce04014c30 con 0x7fcdf8060670 2023-10-06T12:33:45.046 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.044+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 msgr2=0x7fcdf8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.044+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 0x7fcdf8062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce140ab350 tx=0x7fce04008040 comp rx=0 tx=0).stop 2023-10-06T12:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.044+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 msgr2=0x7fce14151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.044+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fce1000f880 tx=0x7fce10007d80 comp rx=0 tx=0).stop 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 shutdown_connections 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdf8060670 0x7fcdf8062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce140ad3a0 0x7fce140ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce1415a400 0x7fce1415a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 --2- 172.21.15.46:0/3803502213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce000051c0 0x7fce14151a20 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 >> 172.21.15.46:0/3803502213 conn(0x7fce140cbe30 msgr2=0x7fce141526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 shutdown_connections 2023-10-06T12:33:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.048+0000 7fce1b9c5700 1 -- 172.21.15.46:0/3803502213 wait complete. 2023-10-06T12:33:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:33:45.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T12:33:43.485647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2459496,"kb_used_data":15656,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752515224,"statfs":{"total":773094113280,"available":770575589376,"internally_reserved":0,"allocated":16031744,"data_stored":8953344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.712418"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"142'271","reported_seq":818,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.665656+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:33:31.665656+0000","last_peered":"2023-10-06T12:33:31.665656+0000","last_clean":"2023-10-06T12:33:31.665656+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:33:31.665656+0000","last_undegraded":"2023-10-06T12:33:31.665656+0000","last_fullsized":"2023-10-06T12:33:31.665656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.757714+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:33:35.757714+0000","last_peered":"2023-10-06T12:33:35.757714+0000","last_clean":"2023-10-06T12:33:35.757714+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:33:35.757714+0000","last_undegraded":"2023-10-06T12:33:35.757714+0000","last_fullsized":"2023-10-06T12:33:35.757714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.665607+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:33:31.665607+0000","last_peered":"2023-10-06T12:33:31.665607+0000","last_clean":"2023-10-06T12:33:31.665607+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:33:31.665607+0000","last_undegraded":"2023-10-06T12:33:31.665607+0000","last_fullsized":"2023-10-06T12:33:31.665607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.756776+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:33:35.756776+0000","last_peered":"2023-10-06T12:33:35.756776+0000","last_clean":"2023-10-06T12:33:35.756776+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:33:35.756776+0000","last_undegraded":"2023-10-06T12:33:35.756776+0000","last_fullsized":"2023-10-06T12:33:35.756776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.757713+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:33:35.757713+0000","last_peered":"2023-10-06T12:33:35.757713+0000","last_clean":"2023-10-06T12:33:35.757713+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:33:35.757713+0000","last_undegraded":"2023-10-06T12:33:35.757713+0000","last_fullsized":"2023-10-06T12:33:35.757713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.746977+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:33:35.746977+0000","last_peered":"2023-10-06T12:33:35.746977+0000","last_clean":"2023-10-06T12:33:35.746977+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:33:35.746977+0000","last_undegraded":"2023-10-06T12:33:35.746977+0000","last_fullsized":"2023-10-06T12:33:35.746977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.667823+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:33:31.667823+0000","last_peered":"2023-10-06T12:33:31.667823+0000","last_clean":"2023-10-06T12:33:31.667823+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:33:31.667823+0000","last_undegraded":"2023-10-06T12:33:31.667823+0000","last_fullsized":"2023-10-06T12:33:31.667823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":120,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.746976+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:33:35.746976+0000","last_peered":"2023-10-06T12:33:35.746976+0000","last_clean":"2023-10-06T12:33:35.746976+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:33:35.746976+0000","last_undegraded":"2023-10-06T12:33:35.746976+0000","last_fullsized":"2023-10-06T12:33:35.746976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:30.703456+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:33:30.703456+0000","last_peered":"2023-10-06T12:33:30.703456+0000","last_clean":"2023-10-06T12:33:30.703456+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:33:30.703456+0000","last_undegraded":"2023-10-06T12:33:30.703456+0000","last_fullsized":"2023-10-06T12:33:30.703456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":2449408,"data_stored":1561944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.155},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.423,"5min":1.6519999999999999,"15min":1.6819999999999999},"last":1.204},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1890000000000001,"15min":1.1719999999999999},"min":{"1min":0.86899999999999999,"5min":0.73099999999999998,"15min":0.64000000000000001},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.91800000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5860000000000001,"5min":1.6200000000000001,"15min":1.7949999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":1.1040000000000001,"5min":0.92500000000000004,"15min":0.85599999999999998},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.052}]},{"osd":2,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":1.855,"15min":1.855},"last":1.3},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.262},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.65500000000000003},"max":{"1min":1.5189999999999999,"5min":1.6240000000000001,"15min":1.7649999999999999},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.2889999999999999},"min":{"1min":0.95499999999999996,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.353},{"interface":"front","average":{"1min":1.363,"5min":1.345,"15min":1.329},"min":{"1min":1.0820000000000001,"5min":0.77500000000000002,"15min":0.70399999999999996},"max":{"1min":1.6619999999999999,"5min":1.7529999999999999,"15min":1.9019999999999999},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1120000000000001,"15min":1.107},"min":{"1min":0.77500000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.722,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.242,"15min":1.2370000000000001},"min":{"1min":0.871,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7649999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.399}]},{"osd":5,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.304,"15min":1.282},"min":{"1min":1.0409999999999999,"5min":0.626,"15min":0.49399999999999999},"max":{"1min":1.752,"5min":1.766,"15min":1.853},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.999,"5min":0.80600000000000005,"15min":0.55700000000000005},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1850000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.389,"15min":1.3680000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.141},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.427,"15min":1.4039999999999999},"min":{"1min":1.2130000000000001,"5min":0.98399999999999999,"15min":0.54000000000000004},"max":{"1min":1.827,"5min":2.0499999999999998,"15min":2.073},"last":1.4470000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.4590000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":1.3999999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.179},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.292},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.325}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.032}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.216}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.53}]}]},{"osd":5,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":1.006}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.258}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.534},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.581}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.6220000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.5800000000000001,"5min":1.6120000000000001,"15min":1.8839999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.85699999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.524,"5min":1.887,"15min":2.0720000000000001},"last":1.0209999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3700000000000001},"min":{"1min":1.012,"5min":0.93700000000000006,"15min":0.90300000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3420000000000001,"15min":1.329},"min":{"1min":1.0469999999999999,"5min":0.98799999999999999,"15min":0.84199999999999997},"max":{"1min":1.637,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3280000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.22,"15min":1.2490000000000001},"min":{"1min":1.012,"5min":0.74199999999999999,"15min":0.64400000000000002},"max":{"1min":1.5089999999999999,"5min":1.726,"15min":1.9330000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":1.077,"5min":0.93200000000000005,"15min":0.872},"max":{"1min":1.484,"5min":1.861,"15min":1.861},"last":1.153}]},{"osd":3,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":1,"5min":0.83699999999999997,"15min":0.82599999999999996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3069999999999999,"15min":1.3400000000000001},"min":{"1min":1.024,"5min":0.88900000000000001,"15min":0.76100000000000001},"max":{"1min":1.5069999999999999,"5min":1.7030000000000001,"15min":1.9119999999999999},"last":1.466}]},{"osd":5,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1950000000000001,"15min":1.2130000000000001},"min":{"1min":0.86399999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6419999999999999,"5min":1.7929999999999999,"15min":1.944},"last":1.407},{"interface":"front","average":{"1min":1.242,"5min":1.1839999999999999,"15min":1.2130000000000001},"min":{"1min":0.83299999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.819},"last":1.196}]},{"osd":6,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.78500000000000003,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6639999999999999,"5min":1.722,"15min":1.984},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3879999999999999,"15min":1.407},"min":{"1min":0.88900000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7330000000000001,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.2330000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3440000000000001,"15min":1.3660000000000001},"min":{"1min":0.81999999999999995,"5min":0.71499999999999997,"15min":0.60299999999999998},"max":{"1min":1.6990000000000001,"5min":1.929,"15min":2.0539999999999998},"last":1.431},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.1459999999999999,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.581,"5min":1.899,"15min":2.1030000000000002},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315484,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056356,"statfs":{"total":96636764160,"available":96313708544,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3049999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0049999999999999,"15min":1.018},"min":{"1min":0.67400000000000004,"5min":0.311,"15min":0.311},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.048,"15min":1.0129999999999999},"min":{"1min":0.752,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.345,"5min":1.728,"15min":1.8},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1080000000000001,"15min":1.1000000000000001},"min":{"1min":0.85899999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.639,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.091}]},{"osd":2,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.177,"15min":1.1459999999999999},"min":{"1min":0.70399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3839999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.246,"15min":1.228},"min":{"1min":0.90600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.649,"15min":2.379},"last":1.508}]},{"osd":4,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.302,"15min":1.2709999999999999},"min":{"1min":0.91800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.02},"last":1.232},{"interface":"front","average":{"1min":1.423,"5min":1.415,"15min":1.389},"min":{"1min":1.1140000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4850000000000001,"15min":1.452},"min":{"1min":1.1020000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9470000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.446,"15min":1.4239999999999999},"min":{"1min":1.0760000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6990000000000001,"5min":1.9419999999999999,"15min":2.0939999999999999},"last":2.0310000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5309999999999999,"15min":1.488},"min":{"1min":1.165,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.819,"5min":2.0449999999999999,"15min":2.234},"last":1.907},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.444,"15min":1.397},"min":{"1min":1.0169999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7929999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.7649999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.494,"15min":1.4730000000000001},"min":{"1min":1.1279999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.5489999999999999,"15min":1.532},"min":{"1min":1.1679999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.02,"5min":2.048,"15min":2.0600000000000001},"last":1.575}]}]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.996}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":1.2310000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.26},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.319}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.206},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.534}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.573},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.425}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.748}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.782}]}]},{"osd":1,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97899999999999998,"15min":0.96899999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.53900000000000003},"max":{"1min":1.7290000000000001,"5min":1.798,"15min":1.798},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.82399999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.655},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.52500000000000002},"max":{"1min":1.3069999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.1180000000000001},"min":{"1min":0.94199999999999995,"5min":0.77400000000000002,"15min":0.60099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.345}]},{"osd":3,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.2370000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.61599999999999999},"max":{"1min":1.673,"5min":2.008,"15min":2.008},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.89900000000000002,"5min":0.77100000000000002,"15min":0.58199999999999996},"max":{"1min":1.4830000000000001,"5min":1.625,"15min":1.806},"last":1.2450000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.274,"15min":1.268},"min":{"1min":1.0569999999999999,"5min":0.96599999999999997,"15min":0.88},"max":{"1min":1.5629999999999999,"5min":1.764,"15min":2.2490000000000001},"last":1.597},{"interface":"front","average":{"1min":1.601,"5min":1.4730000000000001,"15min":1.4359999999999999},"min":{"1min":1.319,"5min":1.0449999999999999,"15min":1.014},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.73}]},{"osd":5,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.403,"15min":1.3859999999999999},"min":{"1min":1.1220000000000001,"5min":1.0349999999999999,"15min":0.80900000000000005},"max":{"1min":1.704,"5min":1.9550000000000001,"15min":2.0350000000000001},"last":1.504},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3879999999999999,"15min":1.379},"min":{"1min":1.0600000000000001,"5min":0.97599999999999998,"15min":0.93400000000000005},"max":{"1min":1.696,"5min":1.917,"15min":2.0619999999999998},"last":1.296}]},{"osd":6,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.554,"15min":1.5109999999999999},"min":{"1min":1.2509999999999999,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.2360000000000002},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.536,"5min":1.4950000000000001,"15min":1.4710000000000001},"min":{"1min":1.236,"5min":1.129,"15min":1.0569999999999999},"max":{"1min":1.865,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.8979999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.5660000000000001},"min":{"1min":1.3140000000000001,"5min":1.282,"15min":0.99299999999999999},"max":{"1min":1.915,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":2.0070000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.5149999999999999,"15min":1.48},"min":{"1min":1.383,"5min":1.157,"15min":0.95799999999999996},"max":{"1min":1.847,"5min":2.0369999999999999,"15min":2.1819999999999999},"last":1.825}]}]},{"osd":0,"up_from":13,"seq":55834575749,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":2449408,"data_stored":1561944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.77200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.492},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.008,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.59799999999999998},"max":{"1min":1.4339999999999999,"5min":1.5549999999999999,"15min":1.595},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.78000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3340000000000001,"5min":1.5469999999999999,"15min":1.746},"last":1.494},{"interface":"front","average":{"1min":1.121,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.4970000000000001,"5min":1.762,"15min":1.762},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.099,"15min":1.0920000000000001},"min":{"1min":0.86199999999999999,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.202,"5min":0.84699999999999998,"15min":0.71999999999999997},"max":{"1min":1.7030000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.7949999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.371,"15min":1.369},"min":{"1min":1.1819999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.663,"5min":1.71,"15min":1.756},"last":1.367},{"interface":"front","average":{"1min":1.417,"5min":1.3340000000000001,"15min":1.3320000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.80500000000000005},"max":{"1min":1.704,"5min":1.7290000000000001,"15min":2.036},"last":1.5589999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.407,"15min":1.401},"min":{"1min":1.117,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9330000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.655},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4239999999999999,"15min":1.423},"min":{"1min":1.2350000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.865,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.619}]},{"osd":6,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.347,"15min":1.357},"min":{"1min":1.1879999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.649,"5min":1.732,"15min":2.2029999999999998},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.478,"15min":1.48},"min":{"1min":1.202,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.919,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.7350000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5029999999999999,"15min":1.482},"min":{"1min":1.2490000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8100000000000001,"5min":2.0099999999999998,"15min":2.1019999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4490000000000001},"min":{"1min":1.087,"5min":0.98599999999999999,"15min":0.95099999999999996},"max":{"1min":1.8480000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":2.4239999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:33:45.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.268+0000 7f4ea1a0c700 1 Processor -- start 2023-10-06T12:33:45.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.268+0000 7f4ea1a0c700 1 -- start start 2023-10-06T12:33:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea1a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea1a0c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e9c067760 0x7f4e9c068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea1a0c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c06d790 con 0x7f4e9c0aaf30 2023-10-06T12:33:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea1a0c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c0acd60 con 0x7f4e9c067760 2023-10-06T12:33:45.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea1a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ad040 con 0x7f4e9c156e20 2023-10-06T12:33:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea120b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e9c067760 0x7f4e9c068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:40542/0 (socket says 172.21.15.46:40542) 2023-10-06T12:33:45.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2705967964 learned_addr learned my addr 172.21.15.46:0/2705967964 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2705967964 >> v2:172.21.15.132:3300/0 conn(0x7f4e9c067760 msgr2=0x7f4e9c068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 --2- 172.21.15.46:0/2705967964 >> v2:172.21.15.132:3300/0 conn(0x7f4e9c067760 0x7f4e9c068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2705967964 >> v1:172.21.15.132:6789/0 conn(0x7f4e9c0aaf30 legacy=0x7f4e9c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2705967964 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0ad320 con 0x7f4e9c156e20 2023-10-06T12:33:45.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.272+0000 7f4ea0a0a700 1 --2- 172.21.15.46:0/2705967964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26fc61366ab3ea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4e9b7fe700 1 -- 172.21.15.46:0/2705967964 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90009ea0 con 0x7f4e9c156e20 2023-10-06T12:33:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4e9b7fe700 1 -- 172.21.15.46:0/2705967964 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9000b860 con 0x7f4e9c156e20 2023-10-06T12:33:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4e9b7fe700 1 -- 172.21.15.46:0/2705967964 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000bc00 con 0x7f4e9c156e20 2023-10-06T12:33:45.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2705967964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 msgr2=0x7f4e9c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2705967964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e900099b0 tx=0x7f4e90009250 comp rx=0 tx=0).stop 2023-10-06T12:33:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2705967964 shutdown_connections 2023-10-06T12:33:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2705967964 >> v2:172.21.15.132:3300/0 conn(0x7f4e9c067760 0x7f4e9c068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2705967964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c06d250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2705967964 >> 172.21.15.46:0/2705967964 conn(0x7f4e9c062ef0 msgr2=0x7f4e9c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2705967964 shutdown_connections 2023-10-06T12:33:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.276+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2705967964 wait complete. 2023-10-06T12:33:45.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.280+0000 7f4ea1a0c700 1 Processor -- start 2023-10-06T12:33:45.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.280+0000 7f4ea1a0c700 1 -- start start 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.280+0000 7f4ea1a0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.280+0000 7f4ea1a0c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e9c0aaf30 0x7f4e9c149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c159130 con 0x7f4e9c067760 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c7590 con 0x7f4e9c0aaf30 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c149f70 con 0x7f4e9c156e20 2023-10-06T12:33:45.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38634/0 (socket says 172.21.15.46:38634) 2023-10-06T12:33:45.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2144092392 learned_addr learned my addr 172.21.15.46:0/2144092392 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea120b700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4e9bfff700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e9c0aaf30 0x7f4e9c149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 msgr2=0x7f4e9c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e9c0aaf30 msgr2=0x7f4e9c149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e9c0aaf30 0x7f4e9c149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 -- 172.21.15.46:0/2144092392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e900094f0 con 0x7f4e9c067760 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea0a0a700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ad180 tx=0x7f4e90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000bbe0 con 0x7f4e9c067760 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9001a450 con 0x7f4e9c067760 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0c6980 con 0x7f4e9c067760 2023-10-06T12:33:45.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9001a7f0 con 0x7f4e9c067760 2023-10-06T12:33:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c0c6f40 con 0x7f4e9c067760 2023-10-06T12:33:45.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.284+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c064590 con 0x7f4e9c067760 2023-10-06T12:33:45.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.288+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4e9001ab60 con 0x7f4e9c067760 2023-10-06T12:33:45.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.288+0000 7f4e98ff9700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 0x7f4e80062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.288+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f4e9007e6c0 con 0x7f4e9c067760 2023-10-06T12:33:45.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.288+0000 7f4e9bfff700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 0x7f4e80062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.292+0000 7f4e9bfff700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 0x7f4e80062b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c170bc0 tx=0x7f4e88009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.296+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e9004e7a0 con 0x7f4e9c067760 2023-10-06T12:33:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.468+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e9c0ad600 con 0x7f4e800606a0 2023-10-06T12:33:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.480+0000 7f4e98ff9700 1 -- 172.21.15.46:0/2144092392 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293523 (secure 0 0 0) 0x7f4e9c0ad600 con 0x7f4e800606a0 2023-10-06T12:33:45.489 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 msgr2=0x7f4e80062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 0x7f4e80062b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c170bc0 tx=0x7f4e88009040 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 msgr2=0x7f4e9c0c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ad180 tx=0x7f4e90017000 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 shutdown_connections 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4e800606a0 0x7f4e80062b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9c156e20 0x7f4e9c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e9c0aaf30 0x7f4e9c149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 --2- 172.21.15.46:0/2144092392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e9c067760 0x7f4e9c0c7a20 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 >> 172.21.15.46:0/2144092392 conn(0x7f4e9c062ef0 msgr2=0x7f4e9c0c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.488+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 shutdown_connections 2023-10-06T12:33:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.492+0000 7f4ea1a0c700 1 -- 172.21.15.46:0/2144092392 wait complete. 2023-10-06T12:33:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:33:45.513 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T12:33:43.485647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2459496,"kb_used_data":15656,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752515224,"statfs":{"total":773094113280,"available":770575589376,"internally_reserved":0,"allocated":16031744,"data_stored":8953344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.712418"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"142'271","reported_seq":818,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.665656+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:33:31.665656+0000","last_peered":"2023-10-06T12:33:31.665656+0000","last_clean":"2023-10-06T12:33:31.665656+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:33:31.665656+0000","last_undegraded":"2023-10-06T12:33:31.665656+0000","last_fullsized":"2023-10-06T12:33:31.665656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":134,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.757714+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:33:35.757714+0000","last_peered":"2023-10-06T12:33:35.757714+0000","last_clean":"2023-10-06T12:33:35.757714+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:33:35.757714+0000","last_undegraded":"2023-10-06T12:33:35.757714+0000","last_fullsized":"2023-10-06T12:33:35.757714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":119,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.665607+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:33:31.665607+0000","last_peered":"2023-10-06T12:33:31.665607+0000","last_clean":"2023-10-06T12:33:31.665607+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:33:31.665607+0000","last_undegraded":"2023-10-06T12:33:31.665607+0000","last_fullsized":"2023-10-06T12:33:31.665607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":121,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.756776+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:33:35.756776+0000","last_peered":"2023-10-06T12:33:35.756776+0000","last_clean":"2023-10-06T12:33:35.756776+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:33:35.756776+0000","last_undegraded":"2023-10-06T12:33:35.756776+0000","last_fullsized":"2023-10-06T12:33:35.756776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":134,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.757713+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:33:35.757713+0000","last_peered":"2023-10-06T12:33:35.757713+0000","last_clean":"2023-10-06T12:33:35.757713+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:33:35.757713+0000","last_undegraded":"2023-10-06T12:33:35.757713+0000","last_fullsized":"2023-10-06T12:33:35.757713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.746977+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:33:35.746977+0000","last_peered":"2023-10-06T12:33:35.746977+0000","last_clean":"2023-10-06T12:33:35.746977+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:33:35.746977+0000","last_undegraded":"2023-10-06T12:33:35.746977+0000","last_fullsized":"2023-10-06T12:33:35.746977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:31.667823+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:33:31.667823+0000","last_peered":"2023-10-06T12:33:31.667823+0000","last_clean":"2023-10-06T12:33:31.667823+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:33:31.667823+0000","last_undegraded":"2023-10-06T12:33:31.667823+0000","last_fullsized":"2023-10-06T12:33:31.667823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":120,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-06T12:33:35.746976+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:33:35.746976+0000","last_peered":"2023-10-06T12:33:35.746976+0000","last_clean":"2023-10-06T12:33:35.746976+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:33:35.746976+0000","last_undegraded":"2023-10-06T12:33:35.746976+0000","last_fullsized":"2023-10-06T12:33:35.746976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:39.773182+0000","last_change":"2023-10-06T12:33:39.773182+0000","last_active":"2023-10-06T12:33:39.773182+0000","last_peered":"2023-10-06T12:33:39.773182+0000","last_clean":"2023-10-06T12:33:39.773182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:39.773182+0000","last_undegraded":"2023-10-06T12:33:39.773182+0000","last_fullsized":"2023-10-06T12:33:39.773182+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T12:33:38.758108+0000","last_change":"2023-10-06T12:33:38.758108+0000","last_active":"2023-10-06T12:33:38.758108+0000","last_peered":"2023-10-06T12:33:38.758108+0000","last_clean":"2023-10-06T12:33:38.758108+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T12:33:38.758108+0000","last_undegraded":"2023-10-06T12:33:38.758108+0000","last_fullsized":"2023-10-06T12:33:38.758108+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-06T12:33:30.703456+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:33:30.703456+0000","last_peered":"2023-10-06T12:33:30.703456+0000","last_clean":"2023-10-06T12:33:30.703456+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:33:30.703456+0000","last_undegraded":"2023-10-06T12:33:30.703456+0000","last_fullsized":"2023-10-06T12:33:30.703456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":2449408,"data_stored":1561944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.155},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.423,"5min":1.6519999999999999,"15min":1.6819999999999999},"last":1.204},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1890000000000001,"15min":1.1719999999999999},"min":{"1min":0.86899999999999999,"5min":0.73099999999999998,"15min":0.64000000000000001},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.91800000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5860000000000001,"5min":1.6200000000000001,"15min":1.7949999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":1.1040000000000001,"5min":0.92500000000000004,"15min":0.85599999999999998},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.052}]},{"osd":2,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":1.855,"15min":1.855},"last":1.3},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.262},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.65500000000000003},"max":{"1min":1.5189999999999999,"5min":1.6240000000000001,"15min":1.7649999999999999},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.2889999999999999},"min":{"1min":0.95499999999999996,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.353},{"interface":"front","average":{"1min":1.363,"5min":1.345,"15min":1.329},"min":{"1min":1.0820000000000001,"5min":0.77500000000000002,"15min":0.70399999999999996},"max":{"1min":1.6619999999999999,"5min":1.7529999999999999,"15min":1.9019999999999999},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1120000000000001,"15min":1.107},"min":{"1min":0.77500000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.722,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.242,"15min":1.2370000000000001},"min":{"1min":0.871,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7649999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.399}]},{"osd":5,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.304,"15min":1.282},"min":{"1min":1.0409999999999999,"5min":0.626,"15min":0.49399999999999999},"max":{"1min":1.752,"5min":1.766,"15min":1.853},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.999,"5min":0.80600000000000005,"15min":0.55700000000000005},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1850000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.389,"15min":1.3680000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.141},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.427,"15min":1.4039999999999999},"min":{"1min":1.2130000000000001,"5min":0.98399999999999999,"15min":0.54000000000000004},"max":{"1min":1.827,"5min":2.0499999999999998,"15min":2.073},"last":1.4470000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.4590000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":1.3999999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.179},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.292},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.325}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.032}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.216}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.53}]}]},{"osd":5,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":1.006}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.2969999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.258}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.534},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.581}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.6220000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575749,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.89500000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.5800000000000001,"5min":1.6120000000000001,"15min":1.8839999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.85699999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.524,"5min":1.887,"15min":2.0720000000000001},"last":1.0209999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3720000000000001,"15min":1.3700000000000001},"min":{"1min":1.012,"5min":0.93700000000000006,"15min":0.90300000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3420000000000001,"15min":1.329},"min":{"1min":1.0469999999999999,"5min":0.98799999999999999,"15min":0.84199999999999997},"max":{"1min":1.637,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3280000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.22,"15min":1.2490000000000001},"min":{"1min":1.012,"5min":0.74199999999999999,"15min":0.64400000000000002},"max":{"1min":1.5089999999999999,"5min":1.726,"15min":1.9330000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.2809999999999999,"15min":1.2969999999999999},"min":{"1min":1.077,"5min":0.93200000000000005,"15min":0.872},"max":{"1min":1.484,"5min":1.861,"15min":1.861},"last":1.153}]},{"osd":3,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":1,"5min":0.83699999999999997,"15min":0.82599999999999996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3069999999999999,"15min":1.3400000000000001},"min":{"1min":1.024,"5min":0.88900000000000001,"15min":0.76100000000000001},"max":{"1min":1.5069999999999999,"5min":1.7030000000000001,"15min":1.9119999999999999},"last":1.466}]},{"osd":5,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1950000000000001,"15min":1.2130000000000001},"min":{"1min":0.86399999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.6419999999999999,"5min":1.7929999999999999,"15min":1.944},"last":1.407},{"interface":"front","average":{"1min":1.242,"5min":1.1839999999999999,"15min":1.2130000000000001},"min":{"1min":0.83299999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.819},"last":1.196}]},{"osd":6,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2430000000000001,"15min":1.2749999999999999},"min":{"1min":0.78500000000000003,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6639999999999999,"5min":1.722,"15min":1.984},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3879999999999999,"15min":1.407},"min":{"1min":0.88900000000000001,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.7330000000000001,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.2330000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3440000000000001,"15min":1.3660000000000001},"min":{"1min":0.81999999999999995,"5min":0.71499999999999997,"15min":0.60299999999999998},"max":{"1min":1.6990000000000001,"5min":1.929,"15min":2.0539999999999998},"last":1.431},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":1.1459999999999999,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.581,"5min":1.899,"15min":2.1030000000000002},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315484,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056356,"statfs":{"total":96636764160,"available":96313708544,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91300000000000003},"min":{"1min":0.64800000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3049999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.0049999999999999,"15min":1.018},"min":{"1min":0.67400000000000004,"5min":0.311,"15min":0.311},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.048,"15min":1.0129999999999999},"min":{"1min":0.752,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.345,"5min":1.728,"15min":1.8},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1080000000000001,"15min":1.1000000000000001},"min":{"1min":0.85899999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.639,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.091}]},{"osd":2,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.177,"15min":1.1459999999999999},"min":{"1min":0.70399999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3839999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.246,"15min":1.228},"min":{"1min":0.90600000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6000000000000001,"5min":1.649,"15min":2.379},"last":1.508}]},{"osd":4,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.302,"15min":1.2709999999999999},"min":{"1min":0.91800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.02},"last":1.232},{"interface":"front","average":{"1min":1.423,"5min":1.415,"15min":1.389},"min":{"1min":1.1140000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4850000000000001,"15min":1.452},"min":{"1min":1.1020000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9470000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.446,"15min":1.4239999999999999},"min":{"1min":1.0760000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6990000000000001,"5min":1.9419999999999999,"15min":2.0939999999999999},"last":2.0310000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5309999999999999,"15min":1.488},"min":{"1min":1.165,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.819,"5min":2.0449999999999999,"15min":2.234},"last":1.907},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.444,"15min":1.397},"min":{"1min":1.0169999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7929999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.7649999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.494,"15min":1.4730000000000001},"min":{"1min":1.1279999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.5489999999999999,"15min":1.532},"min":{"1min":1.1679999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.02,"5min":2.048,"15min":2.0600000000000001},"last":1.575}]}]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311388,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060452,"statfs":{"total":96636764160,"available":96317902848,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.996}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":1.2310000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.26},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.319}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.206},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.534}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.573},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.425}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.748}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.782}]}]},{"osd":1,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307292,"kb_used_data":1812,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064548,"statfs":{"total":96636764160,"available":96322097152,"internally_reserved":0,"allocated":1855488,"data_stored":971576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97899999999999998,"15min":0.96899999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.53900000000000003},"max":{"1min":1.7290000000000001,"5min":1.798,"15min":1.798},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.82399999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.655},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.52500000000000002},"max":{"1min":1.3069999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.1180000000000001},"min":{"1min":0.94199999999999995,"5min":0.77400000000000002,"15min":0.60099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.345}]},{"osd":3,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.2370000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.61599999999999999},"max":{"1min":1.673,"5min":2.008,"15min":2.008},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.89900000000000002,"5min":0.77100000000000002,"15min":0.58199999999999996},"max":{"1min":1.4830000000000001,"5min":1.625,"15min":1.806},"last":1.2450000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.274,"15min":1.268},"min":{"1min":1.0569999999999999,"5min":0.96599999999999997,"15min":0.88},"max":{"1min":1.5629999999999999,"5min":1.764,"15min":2.2490000000000001},"last":1.597},{"interface":"front","average":{"1min":1.601,"5min":1.4730000000000001,"15min":1.4359999999999999},"min":{"1min":1.319,"5min":1.0449999999999999,"15min":1.014},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.73}]},{"osd":5,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.403,"15min":1.3859999999999999},"min":{"1min":1.1220000000000001,"5min":1.0349999999999999,"15min":0.80900000000000005},"max":{"1min":1.704,"5min":1.9550000000000001,"15min":2.0350000000000001},"last":1.504},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3879999999999999,"15min":1.379},"min":{"1min":1.0600000000000001,"5min":0.97599999999999998,"15min":0.93400000000000005},"max":{"1min":1.696,"5min":1.917,"15min":2.0619999999999998},"last":1.296}]},{"osd":6,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.554,"15min":1.5109999999999999},"min":{"1min":1.2509999999999999,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.2360000000000002},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.536,"5min":1.4950000000000001,"15min":1.4710000000000001},"min":{"1min":1.236,"5min":1.129,"15min":1.0569999999999999},"max":{"1min":1.865,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.8979999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.5660000000000001},"min":{"1min":1.3140000000000001,"5min":1.282,"15min":0.99299999999999999},"max":{"1min":1.915,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":2.0070000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.5149999999999999,"15min":1.48},"min":{"1min":1.383,"5min":1.157,"15min":0.95799999999999996},"max":{"1min":1.847,"5min":2.0369999999999999,"15min":2.1819999999999999},"last":1.825}]}]},{"osd":0,"up_from":13,"seq":55834575749,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299680,"kb_used_data":2392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072160,"statfs":{"total":96636764160,"available":96329891840,"internally_reserved":0,"allocated":2449408,"data_stored":1561944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.77200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.492},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.008,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.59799999999999998},"max":{"1min":1.4339999999999999,"5min":1.5549999999999999,"15min":1.595},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.78000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3340000000000001,"5min":1.5469999999999999,"15min":1.746},"last":1.494},{"interface":"front","average":{"1min":1.121,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.4970000000000001,"5min":1.762,"15min":1.762},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.099,"15min":1.0920000000000001},"min":{"1min":0.86199999999999999,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.202,"5min":0.84699999999999998,"15min":0.71999999999999997},"max":{"1min":1.7030000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.7949999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.371,"15min":1.369},"min":{"1min":1.1819999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.663,"5min":1.71,"15min":1.756},"last":1.367},{"interface":"front","average":{"1min":1.417,"5min":1.3340000000000001,"15min":1.3320000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.80500000000000005},"max":{"1min":1.704,"5min":1.7290000000000001,"15min":2.036},"last":1.5589999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.407,"15min":1.401},"min":{"1min":1.117,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9330000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.655},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4239999999999999,"15min":1.423},"min":{"1min":1.2350000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.865,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.619}]},{"osd":6,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.347,"15min":1.357},"min":{"1min":1.1879999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.649,"5min":1.732,"15min":2.2029999999999998},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.478,"15min":1.48},"min":{"1min":1.202,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.919,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.7350000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5029999999999999,"15min":1.482},"min":{"1min":1.2490000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8100000000000001,"5min":2.0099999999999998,"15min":2.1019999999999999},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4490000000000001},"min":{"1min":1.087,"5min":0.98599999999999999,"15min":0.95099999999999996},"max":{"1min":1.8480000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":2.4239999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:33:45.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:33:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 Processor -- start 2023-10-06T12:33:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 -- start start 2023-10-06T12:33:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f629015ab60 0x7f629006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62900ad6c0 0x7f62900adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629006d460 con 0x7f629015af30 2023-10-06T12:33:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900c7000 con 0x7f629015ab60 2023-10-06T12:33:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.724+0000 7f62961c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900c72c0 con 0x7f62900ad6c0 2023-10-06T12:33:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f629015ab60 0x7f629006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f6287fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f629015af30 0x7f629006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56654/0 (socket says 172.21.15.46:56654) 2023-10-06T12:33:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f6287fff700 1 -- 172.21.15.46:0/2916854392 learned_addr learned my addr 172.21.15.46:0/2916854392 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f629015ab60 0x7f629006e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38636/0 (socket says 172.21.15.46:38636) 2023-10-06T12:33:45.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628ffff700 1 --2- 172.21.15.46:0/2916854392 >> v2:172.21.15.132:3300/0 conn(0x7f62900ad6c0 0x7f62900adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 -- 172.21.15.46:0/2916854392 >> v2:172.21.15.132:3300/0 conn(0x7f62900ad6c0 msgr2=0x7f62900adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 --2- 172.21.15.46:0/2916854392 >> v2:172.21.15.132:3300/0 conn(0x7f62900ad6c0 0x7f62900adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 -- 172.21.15.46:0/2916854392 >> v1:172.21.15.46:6789/0 conn(0x7f629015af30 legacy=0x7f629006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628effd700 1 -- 172.21.15.46:0/2916854392 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253259218 0 0) 0x7f629006d460 con 0x7f629015af30 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 -- 172.21.15.46:0/2916854392 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900c75a0 con 0x7f629015ab60 2023-10-06T12:33:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628f7fe700 1 --2- 172.21.15.46:0/2916854392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015ab60 0x7f629006e8a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6290151ba0 tx=0x7f62800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e102307d2978864 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628effd700 1 -- 172.21.15.46:0/2916854392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000b6b0 con 0x7f629015ab60 2023-10-06T12:33:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628effd700 1 -- 172.21.15.46:0/2916854392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628000b810 con 0x7f629015ab60 2023-10-06T12:33:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f628effd700 1 -- 172.21.15.46:0/2916854392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628000bbb0 con 0x7f629015ab60 2023-10-06T12:33:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 -- 172.21.15.46:0/2916854392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015ab60 msgr2=0x7f629006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 --2- 172.21.15.46:0/2916854392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015ab60 0x7f629006e8a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6290151ba0 tx=0x7f62800091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 -- 172.21.15.46:0/2916854392 shutdown_connections 2023-10-06T12:33:45.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 --2- 172.21.15.46:0/2916854392 >> v2:172.21.15.132:3300/0 conn(0x7f62900ad6c0 0x7f62900adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 --2- 172.21.15.46:0/2916854392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015ab60 0x7f629006e8a0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 -- 172.21.15.46:0/2916854392 >> 172.21.15.46:0/2916854392 conn(0x7f62900cbe30 msgr2=0x7f62900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:45.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.728+0000 7f62961c5700 1 -- 172.21.15.46:0/2916854392 shutdown_connections 2023-10-06T12:33:45.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.732+0000 7f62961c5700 1 -- 172.21.15.46:0/2916854392 wait complete. 2023-10-06T12:33:45.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.732+0000 7f62961c5700 1 Processor -- start 2023-10-06T12:33:45.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.732+0000 7f62961c5700 1 -- start start 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62900ad6c0 0x7f62900b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f629015ab60 0x7f629016cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900b0e00 con 0x7f629015af30 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629016d720 con 0x7f62900ad6c0 2023-10-06T12:33:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900accb0 con 0x7f629015ab60 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38644/0 (socket says 172.21.15.46:38644) 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 -- 172.21.15.46:0/2074867899 learned_addr learned my addr 172.21.15.46:0/2074867899 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628f7fe700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62900ad6c0 0x7f62900b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f6287fff700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f629015ab60 0x7f629016cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 -- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f629015ab60 msgr2=0x7f629016cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f629015ab60 0x7f629016cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 -- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62900ad6c0 msgr2=0x7f62900b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62900ad6c0 0x7f62900b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 -- 172.21.15.46:0/2074867899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62800093c0 con 0x7f629015af30 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f628ffff700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6288007f20 tx=0x7f6288009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800ebf0 con 0x7f629015af30 2023-10-06T12:33:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800ed70 con 0x7f629015af30 2023-10-06T12:33:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288014670 con 0x7f629015af30 2023-10-06T12:33:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6290150de0 con 0x7f629015af30 2023-10-06T12:33:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.736+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900b1790 con 0x7f629015af30 2023-10-06T12:33:45.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.740+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62900acb10 con 0x7f629015af30 2023-10-06T12:33:45.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.740+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f62880149e0 con 0x7f629015af30 2023-10-06T12:33:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.740+0000 7f62877fe700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 0x7f6270062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.740+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f628807d2b0 con 0x7f629015af30 2023-10-06T12:33:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.740+0000 7f628f7fe700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 0x7f6270062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:45.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.744+0000 7f628f7fe700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 0x7f6270062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6280009390 tx=0x7f6280017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:45.750 INFO:tasks.ceph.mds.c.smithi046.stderr:2023-10-06T12:33:45.748+0000 7efe2792a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:33:45.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.748+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62900acb10 con 0x7f629015af30 2023-10-06T12:33:45.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.920+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62900acb10 con 0x7f62700606a0 2023-10-06T12:33:45.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.936+0000 7f62877fe700 1 -- 172.21.15.46:0/2074867899 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308586 (secure 0 0 0) 0x7f62900acb10 con 0x7f62700606a0 2023-10-06T12:33:45.943 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:45.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 msgr2=0x7f6270062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 0x7f6270062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6280009390 tx=0x7f6280017000 comp rx=0 tx=0).stop 2023-10-06T12:33:45.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 msgr2=0x7f62901500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:45.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6288007f20 tx=0x7f6288009860 comp rx=0 tx=0).stop 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 shutdown_connections 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f62700606a0 0x7f6270062b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f629015af30 0x7f62901500f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f629015ab60 0x7f629016cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 --2- 172.21.15.46:0/2074867899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62900ad6c0 0x7f62900b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 >> 172.21.15.46:0/2074867899 conn(0x7f62900cbe30 msgr2=0x7f62900aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:45.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 shutdown_connections 2023-10-06T12:33:45.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:45.944+0000 7f62961c5700 1 -- 172.21.15.46:0/2074867899 wait complete. 2023-10-06T12:33:45.949 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:33:45.968 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T12:33:45.486411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459944,"kb_used_data":16104,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514776,"statfs":{"total":773094113280,"available":770575130624,"internally_reserved":0,"allocated":16490496,"data_stored":9186652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002022"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280402+0000","last_change":"2023-10-06T12:33:40.278094+0000","last_active":"2023-10-06T12:33:40.280402+0000","last_peered":"2023-10-06T12:33:40.280402+0000","last_clean":"2023-10-06T12:33:40.280402+0000","last_became_active":"2023-10-06T12:33:40.277766+0000","last_became_peered":"2023-10-06T12:33:40.277766+0000","last_unstale":"2023-10-06T12:33:40.280402+0000","last_undegraded":"2023-10-06T12:33:40.280402+0000","last_fullsized":"2023-10-06T12:33:40.280402+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:40.996759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278844+0000","last_change":"2023-10-06T12:33:40.276811+0000","last_active":"2023-10-06T12:33:40.278844+0000","last_peered":"2023-10-06T12:33:40.278844+0000","last_clean":"2023-10-06T12:33:40.278844+0000","last_became_active":"2023-10-06T12:33:40.276490+0000","last_became_peered":"2023-10-06T12:33:40.276490+0000","last_unstale":"2023-10-06T12:33:40.278844+0000","last_undegraded":"2023-10-06T12:33:40.278844+0000","last_fullsized":"2023-10-06T12:33:40.278844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293228+0000","last_change":"2023-10-06T12:33:40.289198+0000","last_active":"2023-10-06T12:33:40.293228+0000","last_peered":"2023-10-06T12:33:40.293228+0000","last_clean":"2023-10-06T12:33:40.293228+0000","last_became_active":"2023-10-06T12:33:40.288858+0000","last_became_peered":"2023-10-06T12:33:40.288858+0000","last_unstale":"2023-10-06T12:33:40.293228+0000","last_undegraded":"2023-10-06T12:33:40.293228+0000","last_fullsized":"2023-10-06T12:33:40.293228+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278895+0000","last_change":"2023-10-06T12:33:40.274309+0000","last_active":"2023-10-06T12:33:40.278895+0000","last_peered":"2023-10-06T12:33:40.278895+0000","last_clean":"2023-10-06T12:33:40.278895+0000","last_became_active":"2023-10-06T12:33:40.274015+0000","last_became_peered":"2023-10-06T12:33:40.274015+0000","last_unstale":"2023-10-06T12:33:40.278895+0000","last_undegraded":"2023-10-06T12:33:40.278895+0000","last_fullsized":"2023-10-06T12:33:40.278895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287401+0000","last_change":"2023-10-06T12:33:40.285006+0000","last_active":"2023-10-06T12:33:40.287401+0000","last_peered":"2023-10-06T12:33:40.287401+0000","last_clean":"2023-10-06T12:33:40.287401+0000","last_became_active":"2023-10-06T12:33:40.284722+0000","last_became_peered":"2023-10-06T12:33:40.284722+0000","last_unstale":"2023-10-06T12:33:40.287401+0000","last_undegraded":"2023-10-06T12:33:40.287401+0000","last_fullsized":"2023-10-06T12:33:40.287401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293763+0000","last_change":"2023-10-06T12:33:40.292772+0000","last_active":"2023-10-06T12:33:40.293763+0000","last_peered":"2023-10-06T12:33:40.293763+0000","last_clean":"2023-10-06T12:33:40.293763+0000","last_became_active":"2023-10-06T12:33:40.292458+0000","last_became_peered":"2023-10-06T12:33:40.292458+0000","last_unstale":"2023-10-06T12:33:40.293763+0000","last_undegraded":"2023-10-06T12:33:40.293763+0000","last_fullsized":"2023-10-06T12:33:40.293763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:47.841304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292102+0000","last_change":"2023-10-06T12:33:40.286279+0000","last_active":"2023-10-06T12:33:40.292102+0000","last_peered":"2023-10-06T12:33:40.292102+0000","last_clean":"2023-10-06T12:33:40.292102+0000","last_became_active":"2023-10-06T12:33:40.285975+0000","last_became_peered":"2023-10-06T12:33:40.285975+0000","last_unstale":"2023-10-06T12:33:40.292102+0000","last_undegraded":"2023-10-06T12:33:40.292102+0000","last_fullsized":"2023-10-06T12:33:40.292102+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284738+0000","last_change":"2023-10-06T12:33:40.274745+0000","last_active":"2023-10-06T12:33:40.284738+0000","last_peered":"2023-10-06T12:33:40.284738+0000","last_clean":"2023-10-06T12:33:40.284738+0000","last_became_active":"2023-10-06T12:33:40.274253+0000","last_became_peered":"2023-10-06T12:33:40.274253+0000","last_unstale":"2023-10-06T12:33:40.284738+0000","last_undegraded":"2023-10-06T12:33:40.284738+0000","last_fullsized":"2023-10-06T12:33:40.284738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:37.726541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281509+0000","last_change":"2023-10-06T12:33:40.277693+0000","last_active":"2023-10-06T12:33:40.281509+0000","last_peered":"2023-10-06T12:33:40.281509+0000","last_clean":"2023-10-06T12:33:40.281509+0000","last_became_active":"2023-10-06T12:33:40.277394+0000","last_became_peered":"2023-10-06T12:33:40.277394+0000","last_unstale":"2023-10-06T12:33:40.281509+0000","last_undegraded":"2023-10-06T12:33:40.281509+0000","last_fullsized":"2023-10-06T12:33:40.281509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:34.307339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273435+0000","last_change":"2023-10-06T12:33:40.269594+0000","last_active":"2023-10-06T12:33:40.273435+0000","last_peered":"2023-10-06T12:33:40.273435+0000","last_clean":"2023-10-06T12:33:40.273435+0000","last_became_active":"2023-10-06T12:33:40.269292+0000","last_became_peered":"2023-10-06T12:33:40.269292+0000","last_unstale":"2023-10-06T12:33:40.273435+0000","last_undegraded":"2023-10-06T12:33:40.273435+0000","last_fullsized":"2023-10-06T12:33:40.273435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:48.712468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.290272+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289949+0000","last_became_peered":"2023-10-06T12:33:40.289949+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281966+0000","last_change":"2023-10-06T12:33:40.278884+0000","last_active":"2023-10-06T12:33:40.281966+0000","last_peered":"2023-10-06T12:33:40.281966+0000","last_clean":"2023-10-06T12:33:40.281966+0000","last_became_active":"2023-10-06T12:33:40.278547+0000","last_became_peered":"2023-10-06T12:33:40.278547+0000","last_unstale":"2023-10-06T12:33:40.281966+0000","last_undegraded":"2023-10-06T12:33:40.281966+0000","last_fullsized":"2023-10-06T12:33:40.281966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291124+0000","last_change":"2023-10-06T12:33:40.290706+0000","last_active":"2023-10-06T12:33:40.291124+0000","last_peered":"2023-10-06T12:33:40.291124+0000","last_clean":"2023-10-06T12:33:40.291124+0000","last_became_active":"2023-10-06T12:33:40.290189+0000","last_became_peered":"2023-10-06T12:33:40.290189+0000","last_unstale":"2023-10-06T12:33:40.291124+0000","last_undegraded":"2023-10-06T12:33:40.291124+0000","last_fullsized":"2023-10-06T12:33:40.291124+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:31.802812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287127+0000","last_change":"2023-10-06T12:33:40.285957+0000","last_active":"2023-10-06T12:33:40.287127+0000","last_peered":"2023-10-06T12:33:40.287127+0000","last_clean":"2023-10-06T12:33:40.287127+0000","last_became_active":"2023-10-06T12:33:40.285663+0000","last_became_peered":"2023-10-06T12:33:40.285663+0000","last_unstale":"2023-10-06T12:33:40.287127+0000","last_undegraded":"2023-10-06T12:33:40.287127+0000","last_fullsized":"2023-10-06T12:33:40.287127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287455+0000","last_change":"2023-10-06T12:33:40.286241+0000","last_active":"2023-10-06T12:33:40.287455+0000","last_peered":"2023-10-06T12:33:40.287455+0000","last_clean":"2023-10-06T12:33:40.287455+0000","last_became_active":"2023-10-06T12:33:40.285913+0000","last_became_peered":"2023-10-06T12:33:40.285913+0000","last_unstale":"2023-10-06T12:33:40.287455+0000","last_undegraded":"2023-10-06T12:33:40.287455+0000","last_fullsized":"2023-10-06T12:33:40.287455+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282791+0000","last_change":"2023-10-06T12:33:40.281557+0000","last_active":"2023-10-06T12:33:40.282791+0000","last_peered":"2023-10-06T12:33:40.282791+0000","last_clean":"2023-10-06T12:33:40.282791+0000","last_became_active":"2023-10-06T12:33:40.281272+0000","last_became_peered":"2023-10-06T12:33:40.281272+0000","last_unstale":"2023-10-06T12:33:40.282791+0000","last_undegraded":"2023-10-06T12:33:40.282791+0000","last_fullsized":"2023-10-06T12:33:40.282791+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:25.331270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282795+0000","last_change":"2023-10-06T12:33:40.279525+0000","last_active":"2023-10-06T12:33:40.282795+0000","last_peered":"2023-10-06T12:33:40.282795+0000","last_clean":"2023-10-06T12:33:40.282795+0000","last_became_active":"2023-10-06T12:33:40.279160+0000","last_became_peered":"2023-10-06T12:33:40.279160+0000","last_unstale":"2023-10-06T12:33:40.282795+0000","last_undegraded":"2023-10-06T12:33:40.282795+0000","last_fullsized":"2023-10-06T12:33:40.282795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:56.644708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287429+0000","last_change":"2023-10-06T12:33:40.284511+0000","last_active":"2023-10-06T12:33:40.287429+0000","last_peered":"2023-10-06T12:33:40.287429+0000","last_clean":"2023-10-06T12:33:40.287429+0000","last_became_active":"2023-10-06T12:33:40.284083+0000","last_became_peered":"2023-10-06T12:33:40.284083+0000","last_unstale":"2023-10-06T12:33:40.287429+0000","last_undegraded":"2023-10-06T12:33:40.287429+0000","last_fullsized":"2023-10-06T12:33:40.287429+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292384+0000","last_change":"2023-10-06T12:33:40.286091+0000","last_active":"2023-10-06T12:33:40.292384+0000","last_peered":"2023-10-06T12:33:40.292384+0000","last_clean":"2023-10-06T12:33:40.292384+0000","last_became_active":"2023-10-06T12:33:40.285783+0000","last_became_peered":"2023-10-06T12:33:40.285783+0000","last_unstale":"2023-10-06T12:33:40.292384+0000","last_undegraded":"2023-10-06T12:33:40.292384+0000","last_fullsized":"2023-10-06T12:33:40.292384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286060+0000","last_change":"2023-10-06T12:33:40.283403+0000","last_active":"2023-10-06T12:33:40.286060+0000","last_peered":"2023-10-06T12:33:40.286060+0000","last_clean":"2023-10-06T12:33:40.286060+0000","last_became_active":"2023-10-06T12:33:40.283010+0000","last_became_peered":"2023-10-06T12:33:40.283010+0000","last_unstale":"2023-10-06T12:33:40.286060+0000","last_undegraded":"2023-10-06T12:33:40.286060+0000","last_fullsized":"2023-10-06T12:33:40.286060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.288358+0000","last_change":"2023-10-06T12:33:40.285829+0000","last_active":"2023-10-06T12:33:40.288358+0000","last_peered":"2023-10-06T12:33:40.288358+0000","last_clean":"2023-10-06T12:33:40.288358+0000","last_became_active":"2023-10-06T12:33:40.284389+0000","last_became_peered":"2023-10-06T12:33:40.284389+0000","last_unstale":"2023-10-06T12:33:40.288358+0000","last_undegraded":"2023-10-06T12:33:40.288358+0000","last_fullsized":"2023-10-06T12:33:40.288358+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281945+0000","last_change":"2023-10-06T12:33:40.280118+0000","last_active":"2023-10-06T12:33:40.281945+0000","last_peered":"2023-10-06T12:33:40.281945+0000","last_clean":"2023-10-06T12:33:40.281945+0000","last_became_active":"2023-10-06T12:33:40.279604+0000","last_became_peered":"2023-10-06T12:33:40.279604+0000","last_unstale":"2023-10-06T12:33:40.281945+0000","last_undegraded":"2023-10-06T12:33:40.281945+0000","last_fullsized":"2023-10-06T12:33:40.281945+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:11:13.472429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283037+0000","last_change":"2023-10-06T12:33:40.280532+0000","last_active":"2023-10-06T12:33:40.283037+0000","last_peered":"2023-10-06T12:33:40.283037+0000","last_clean":"2023-10-06T12:33:40.283037+0000","last_became_active":"2023-10-06T12:33:40.280248+0000","last_became_peered":"2023-10-06T12:33:40.280248+0000","last_unstale":"2023-10-06T12:33:40.283037+0000","last_undegraded":"2023-10-06T12:33:40.283037+0000","last_fullsized":"2023-10-06T12:33:40.283037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281656+0000","last_change":"2023-10-06T12:33:40.277730+0000","last_active":"2023-10-06T12:33:40.281656+0000","last_peered":"2023-10-06T12:33:40.281656+0000","last_clean":"2023-10-06T12:33:40.281656+0000","last_became_active":"2023-10-06T12:33:40.277481+0000","last_became_peered":"2023-10-06T12:33:40.277481+0000","last_unstale":"2023-10-06T12:33:40.281656+0000","last_undegraded":"2023-10-06T12:33:40.281656+0000","last_fullsized":"2023-10-06T12:33:40.281656+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:55.533377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295166+0000","last_change":"2023-10-06T12:33:40.290594+0000","last_active":"2023-10-06T12:33:40.295166+0000","last_peered":"2023-10-06T12:33:40.295166+0000","last_clean":"2023-10-06T12:33:40.295166+0000","last_became_active":"2023-10-06T12:33:40.290274+0000","last_became_peered":"2023-10-06T12:33:40.290274+0000","last_unstale":"2023-10-06T12:33:40.295166+0000","last_undegraded":"2023-10-06T12:33:40.295166+0000","last_fullsized":"2023-10-06T12:33:40.295166+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:26.400947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293038+0000","last_change":"2023-10-06T12:33:40.288927+0000","last_active":"2023-10-06T12:33:40.293038+0000","last_peered":"2023-10-06T12:33:40.293038+0000","last_clean":"2023-10-06T12:33:40.293038+0000","last_became_active":"2023-10-06T12:33:40.288580+0000","last_became_peered":"2023-10-06T12:33:40.288580+0000","last_unstale":"2023-10-06T12:33:40.293038+0000","last_undegraded":"2023-10-06T12:33:40.293038+0000","last_fullsized":"2023-10-06T12:33:40.293038+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291369+0000","last_change":"2023-10-06T12:33:40.282511+0000","last_active":"2023-10-06T12:33:40.291369+0000","last_peered":"2023-10-06T12:33:40.291369+0000","last_clean":"2023-10-06T12:33:40.291369+0000","last_became_active":"2023-10-06T12:33:40.282205+0000","last_became_peered":"2023-10-06T12:33:40.282205+0000","last_unstale":"2023-10-06T12:33:40.291369+0000","last_undegraded":"2023-10-06T12:33:40.291369+0000","last_fullsized":"2023-10-06T12:33:40.291369+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273907+0000","last_change":"2023-10-06T12:33:40.271338+0000","last_active":"2023-10-06T12:33:40.273907+0000","last_peered":"2023-10-06T12:33:40.273907+0000","last_clean":"2023-10-06T12:33:40.273907+0000","last_became_active":"2023-10-06T12:33:40.271069+0000","last_became_peered":"2023-10-06T12:33:40.271069+0000","last_unstale":"2023-10-06T12:33:40.273907+0000","last_undegraded":"2023-10-06T12:33:40.273907+0000","last_fullsized":"2023-10-06T12:33:40.273907+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.758266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294979+0000","last_change":"2023-10-06T12:33:40.290057+0000","last_active":"2023-10-06T12:33:40.294979+0000","last_peered":"2023-10-06T12:33:40.294979+0000","last_clean":"2023-10-06T12:33:40.294979+0000","last_became_active":"2023-10-06T12:33:40.288561+0000","last_became_peered":"2023-10-06T12:33:40.288561+0000","last_unstale":"2023-10-06T12:33:40.294979+0000","last_undegraded":"2023-10-06T12:33:40.294979+0000","last_fullsized":"2023-10-06T12:33:40.294979+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:57.704963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296209+0000","last_change":"2023-10-06T12:33:40.292961+0000","last_active":"2023-10-06T12:33:40.296209+0000","last_peered":"2023-10-06T12:33:40.296209+0000","last_clean":"2023-10-06T12:33:40.296209+0000","last_became_active":"2023-10-06T12:33:40.292379+0000","last_became_peered":"2023-10-06T12:33:40.292379+0000","last_unstale":"2023-10-06T12:33:40.296209+0000","last_undegraded":"2023-10-06T12:33:40.296209+0000","last_fullsized":"2023-10-06T12:33:40.296209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:03.774409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289262+0000","last_change":"2023-10-06T12:33:40.284299+0000","last_active":"2023-10-06T12:33:40.289262+0000","last_peered":"2023-10-06T12:33:40.289262+0000","last_clean":"2023-10-06T12:33:40.289262+0000","last_became_active":"2023-10-06T12:33:40.284014+0000","last_became_peered":"2023-10-06T12:33:40.284014+0000","last_unstale":"2023-10-06T12:33:40.289262+0000","last_undegraded":"2023-10-06T12:33:40.289262+0000","last_fullsized":"2023-10-06T12:33:40.289262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281701+0000","last_change":"2023-10-06T12:33:40.274503+0000","last_active":"2023-10-06T12:33:40.281701+0000","last_peered":"2023-10-06T12:33:40.281701+0000","last_clean":"2023-10-06T12:33:40.281701+0000","last_became_active":"2023-10-06T12:33:40.274196+0000","last_became_peered":"2023-10-06T12:33:40.274196+0000","last_unstale":"2023-10-06T12:33:40.281701+0000","last_undegraded":"2023-10-06T12:33:40.281701+0000","last_fullsized":"2023-10-06T12:33:40.281701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:20.140551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289181+0000","last_change":"2023-10-06T12:33:40.277982+0000","last_active":"2023-10-06T12:33:40.289181+0000","last_peered":"2023-10-06T12:33:40.289181+0000","last_clean":"2023-10-06T12:33:40.289181+0000","last_became_active":"2023-10-06T12:33:40.277677+0000","last_became_peered":"2023-10-06T12:33:40.277677+0000","last_unstale":"2023-10-06T12:33:40.289181+0000","last_undegraded":"2023-10-06T12:33:40.289181+0000","last_fullsized":"2023-10-06T12:33:40.289181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811674+0000","last_change":"2023-10-06T12:33:39.805457+0000","last_active":"2023-10-06T12:33:40.811674+0000","last_peered":"2023-10-06T12:33:40.811674+0000","last_clean":"2023-10-06T12:33:40.811674+0000","last_became_active":"2023-10-06T12:33:39.805053+0000","last_became_peered":"2023-10-06T12:33:39.805053+0000","last_unstale":"2023-10-06T12:33:40.811674+0000","last_undegraded":"2023-10-06T12:33:40.811674+0000","last_fullsized":"2023-10-06T12:33:40.811674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281933+0000","last_change":"2023-10-06T12:33:40.279910+0000","last_active":"2023-10-06T12:33:40.281933+0000","last_peered":"2023-10-06T12:33:40.281933+0000","last_clean":"2023-10-06T12:33:40.281933+0000","last_became_active":"2023-10-06T12:33:40.279582+0000","last_became_peered":"2023-10-06T12:33:40.279582+0000","last_unstale":"2023-10-06T12:33:40.281933+0000","last_undegraded":"2023-10-06T12:33:40.281933+0000","last_fullsized":"2023-10-06T12:33:40.281933+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:58.370893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808163+0000","last_change":"2023-10-06T12:33:39.807053+0000","last_active":"2023-10-06T12:33:40.808163+0000","last_peered":"2023-10-06T12:33:40.808163+0000","last_clean":"2023-10-06T12:33:40.808163+0000","last_became_active":"2023-10-06T12:33:39.806637+0000","last_became_peered":"2023-10-06T12:33:39.806637+0000","last_unstale":"2023-10-06T12:33:40.808163+0000","last_undegraded":"2023-10-06T12:33:40.808163+0000","last_fullsized":"2023-10-06T12:33:40.808163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280827+0000","last_change":"2023-10-06T12:33:40.279263+0000","last_active":"2023-10-06T12:33:40.280827+0000","last_peered":"2023-10-06T12:33:40.280827+0000","last_clean":"2023-10-06T12:33:40.280827+0000","last_became_active":"2023-10-06T12:33:40.278933+0000","last_became_peered":"2023-10-06T12:33:40.278933+0000","last_unstale":"2023-10-06T12:33:40.280827+0000","last_undegraded":"2023-10-06T12:33:40.280827+0000","last_fullsized":"2023-10-06T12:33:40.280827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.810492+0000","last_change":"2023-10-06T12:33:39.816889+0000","last_active":"2023-10-06T12:33:40.810492+0000","last_peered":"2023-10-06T12:33:40.810492+0000","last_clean":"2023-10-06T12:33:40.810492+0000","last_became_active":"2023-10-06T12:33:39.816379+0000","last_became_peered":"2023-10-06T12:33:39.816379+0000","last_unstale":"2023-10-06T12:33:40.810492+0000","last_undegraded":"2023-10-06T12:33:40.810492+0000","last_fullsized":"2023-10-06T12:33:40.810492+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:33.237343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286357+0000","last_change":"2023-10-06T12:33:40.285187+0000","last_active":"2023-10-06T12:33:40.286357+0000","last_peered":"2023-10-06T12:33:40.286357+0000","last_clean":"2023-10-06T12:33:40.286357+0000","last_became_active":"2023-10-06T12:33:40.284897+0000","last_became_peered":"2023-10-06T12:33:40.284897+0000","last_unstale":"2023-10-06T12:33:40.286357+0000","last_undegraded":"2023-10-06T12:33:40.286357+0000","last_fullsized":"2023-10-06T12:33:40.286357+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811184+0000","last_change":"2023-10-06T12:33:39.812575+0000","last_active":"2023-10-06T12:33:40.811184+0000","last_peered":"2023-10-06T12:33:40.811184+0000","last_clean":"2023-10-06T12:33:40.811184+0000","last_became_active":"2023-10-06T12:33:39.812156+0000","last_became_peered":"2023-10-06T12:33:39.812156+0000","last_unstale":"2023-10-06T12:33:40.811184+0000","last_undegraded":"2023-10-06T12:33:40.811184+0000","last_fullsized":"2023-10-06T12:33:40.811184+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:32.614790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289164+0000","last_change":"2023-10-06T12:33:40.276822+0000","last_active":"2023-10-06T12:33:40.289164+0000","last_peered":"2023-10-06T12:33:40.289164+0000","last_clean":"2023-10-06T12:33:40.289164+0000","last_became_active":"2023-10-06T12:33:40.276482+0000","last_became_peered":"2023-10-06T12:33:40.276482+0000","last_unstale":"2023-10-06T12:33:40.289164+0000","last_undegraded":"2023-10-06T12:33:40.289164+0000","last_fullsized":"2023-10-06T12:33:40.289164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809703+0000","last_change":"2023-10-06T12:33:39.814112+0000","last_active":"2023-10-06T12:33:40.809703+0000","last_peered":"2023-10-06T12:33:40.809703+0000","last_clean":"2023-10-06T12:33:40.809703+0000","last_became_active":"2023-10-06T12:33:39.813197+0000","last_became_peered":"2023-10-06T12:33:39.813197+0000","last_unstale":"2023-10-06T12:33:40.809703+0000","last_undegraded":"2023-10-06T12:33:40.809703+0000","last_fullsized":"2023-10-06T12:33:40.809703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:02.759335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284735+0000","last_change":"2023-10-06T12:33:40.272534+0000","last_active":"2023-10-06T12:33:40.284735+0000","last_peered":"2023-10-06T12:33:40.284735+0000","last_clean":"2023-10-06T12:33:40.284735+0000","last_became_active":"2023-10-06T12:33:40.272184+0000","last_became_peered":"2023-10-06T12:33:40.272184+0000","last_unstale":"2023-10-06T12:33:40.284735+0000","last_undegraded":"2023-10-06T12:33:40.284735+0000","last_fullsized":"2023-10-06T12:33:40.284735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:05.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809086+0000","last_change":"2023-10-06T12:33:39.812707+0000","last_active":"2023-10-06T12:33:40.809086+0000","last_peered":"2023-10-06T12:33:40.809086+0000","last_clean":"2023-10-06T12:33:40.809086+0000","last_became_active":"2023-10-06T12:33:39.812406+0000","last_became_peered":"2023-10-06T12:33:39.812406+0000","last_unstale":"2023-10-06T12:33:40.809086+0000","last_undegraded":"2023-10-06T12:33:40.809086+0000","last_fullsized":"2023-10-06T12:33:40.809086+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286027+0000","last_change":"2023-10-06T12:33:40.283254+0000","last_active":"2023-10-06T12:33:40.286027+0000","last_peered":"2023-10-06T12:33:40.286027+0000","last_clean":"2023-10-06T12:33:40.286027+0000","last_became_active":"2023-10-06T12:33:40.282952+0000","last_became_peered":"2023-10-06T12:33:40.282952+0000","last_unstale":"2023-10-06T12:33:40.286027+0000","last_undegraded":"2023-10-06T12:33:40.286027+0000","last_fullsized":"2023-10-06T12:33:40.286027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274181+0000","last_change":"2023-10-06T12:33:39.816556+0000","last_active":"2023-10-06T12:33:40.274181+0000","last_peered":"2023-10-06T12:33:40.274181+0000","last_clean":"2023-10-06T12:33:40.274181+0000","last_became_active":"2023-10-06T12:33:39.816213+0000","last_became_peered":"2023-10-06T12:33:39.816213+0000","last_unstale":"2023-10-06T12:33:40.274181+0000","last_undegraded":"2023-10-06T12:33:40.274181+0000","last_fullsized":"2023-10-06T12:33:40.274181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:56.132241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290868+0000","last_change":"2023-10-06T12:33:40.283081+0000","last_active":"2023-10-06T12:33:40.290868+0000","last_peered":"2023-10-06T12:33:40.290868+0000","last_clean":"2023-10-06T12:33:40.290868+0000","last_became_active":"2023-10-06T12:33:40.282501+0000","last_became_peered":"2023-10-06T12:33:40.282501+0000","last_unstale":"2023-10-06T12:33:40.290868+0000","last_undegraded":"2023-10-06T12:33:40.290868+0000","last_fullsized":"2023-10-06T12:33:40.290868+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286696+0000","last_change":"2023-10-06T12:33:39.810216+0000","last_active":"2023-10-06T12:33:40.286696+0000","last_peered":"2023-10-06T12:33:40.286696+0000","last_clean":"2023-10-06T12:33:40.286696+0000","last_became_active":"2023-10-06T12:33:39.809885+0000","last_became_peered":"2023-10-06T12:33:39.809885+0000","last_unstale":"2023-10-06T12:33:40.286696+0000","last_undegraded":"2023-10-06T12:33:40.286696+0000","last_fullsized":"2023-10-06T12:33:40.286696+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296114+0000","last_change":"2023-10-06T12:33:40.292823+0000","last_active":"2023-10-06T12:33:40.296114+0000","last_peered":"2023-10-06T12:33:40.296114+0000","last_clean":"2023-10-06T12:33:40.296114+0000","last_became_active":"2023-10-06T12:33:40.292457+0000","last_became_peered":"2023-10-06T12:33:40.292457+0000","last_unstale":"2023-10-06T12:33:40.296114+0000","last_undegraded":"2023-10-06T12:33:40.296114+0000","last_fullsized":"2023-10-06T12:33:40.296114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809539+0000","last_change":"2023-10-06T12:33:39.804873+0000","last_active":"2023-10-06T12:33:40.809539+0000","last_peered":"2023-10-06T12:33:40.809539+0000","last_clean":"2023-10-06T12:33:40.809539+0000","last_became_active":"2023-10-06T12:33:39.803841+0000","last_became_peered":"2023-10-06T12:33:39.803841+0000","last_unstale":"2023-10-06T12:33:40.809539+0000","last_undegraded":"2023-10-06T12:33:40.809539+0000","last_fullsized":"2023-10-06T12:33:40.809539+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:56.689888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291255+0000","last_change":"2023-10-06T12:33:40.284048+0000","last_active":"2023-10-06T12:33:40.291255+0000","last_peered":"2023-10-06T12:33:40.291255+0000","last_clean":"2023-10-06T12:33:40.291255+0000","last_became_active":"2023-10-06T12:33:40.283765+0000","last_became_peered":"2023-10-06T12:33:40.283765+0000","last_unstale":"2023-10-06T12:33:40.291255+0000","last_undegraded":"2023-10-06T12:33:40.291255+0000","last_fullsized":"2023-10-06T12:33:40.291255+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811535+0000","last_change":"2023-10-06T12:33:39.805882+0000","last_active":"2023-10-06T12:33:40.811535+0000","last_peered":"2023-10-06T12:33:40.811535+0000","last_clean":"2023-10-06T12:33:40.811535+0000","last_became_active":"2023-10-06T12:33:39.805035+0000","last_became_peered":"2023-10-06T12:33:39.805035+0000","last_unstale":"2023-10-06T12:33:40.811535+0000","last_undegraded":"2023-10-06T12:33:40.811535+0000","last_fullsized":"2023-10-06T12:33:40.811535+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290722+0000","last_change":"2023-10-06T12:33:40.282920+0000","last_active":"2023-10-06T12:33:40.290722+0000","last_peered":"2023-10-06T12:33:40.290722+0000","last_clean":"2023-10-06T12:33:40.290722+0000","last_became_active":"2023-10-06T12:33:40.282581+0000","last_became_peered":"2023-10-06T12:33:40.282581+0000","last_unstale":"2023-10-06T12:33:40.290722+0000","last_undegraded":"2023-10-06T12:33:40.290722+0000","last_fullsized":"2023-10-06T12:33:40.290722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.806365+0000","last_change":"2023-10-06T12:33:39.815967+0000","last_active":"2023-10-06T12:33:40.806365+0000","last_peered":"2023-10-06T12:33:40.806365+0000","last_clean":"2023-10-06T12:33:40.806365+0000","last_became_active":"2023-10-06T12:33:39.815667+0000","last_became_peered":"2023-10-06T12:33:39.815667+0000","last_unstale":"2023-10-06T12:33:40.806365+0000","last_undegraded":"2023-10-06T12:33:40.806365+0000","last_fullsized":"2023-10-06T12:33:40.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:05.141371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291363+0000","last_change":"2023-10-06T12:33:40.290471+0000","last_active":"2023-10-06T12:33:40.291363+0000","last_peered":"2023-10-06T12:33:40.291363+0000","last_clean":"2023-10-06T12:33:40.291363+0000","last_became_active":"2023-10-06T12:33:40.290101+0000","last_became_peered":"2023-10-06T12:33:40.290101+0000","last_unstale":"2023-10-06T12:33:40.291363+0000","last_undegraded":"2023-10-06T12:33:40.291363+0000","last_fullsized":"2023-10-06T12:33:40.291363+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:20.204914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266996+0000","last_change":"2023-10-06T12:33:39.811123+0000","last_active":"2023-10-06T12:33:40.266996+0000","last_peered":"2023-10-06T12:33:40.266996+0000","last_clean":"2023-10-06T12:33:40.266996+0000","last_became_active":"2023-10-06T12:33:39.810818+0000","last_became_peered":"2023-10-06T12:33:39.810818+0000","last_unstale":"2023-10-06T12:33:40.266996+0000","last_undegraded":"2023-10-06T12:33:40.266996+0000","last_fullsized":"2023-10-06T12:33:40.266996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280655+0000","last_change":"2023-10-06T12:33:40.275550+0000","last_active":"2023-10-06T12:33:40.280655+0000","last_peered":"2023-10-06T12:33:40.280655+0000","last_clean":"2023-10-06T12:33:40.280655+0000","last_became_active":"2023-10-06T12:33:40.275264+0000","last_became_peered":"2023-10-06T12:33:40.275264+0000","last_unstale":"2023-10-06T12:33:40.280655+0000","last_undegraded":"2023-10-06T12:33:40.280655+0000","last_fullsized":"2023-10-06T12:33:40.280655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812298+0000","last_change":"2023-10-06T12:33:39.811347+0000","last_active":"2023-10-06T12:33:40.812298+0000","last_peered":"2023-10-06T12:33:40.812298+0000","last_clean":"2023-10-06T12:33:40.812298+0000","last_became_active":"2023-10-06T12:33:39.810887+0000","last_became_peered":"2023-10-06T12:33:39.810887+0000","last_unstale":"2023-10-06T12:33:40.812298+0000","last_undegraded":"2023-10-06T12:33:40.812298+0000","last_fullsized":"2023-10-06T12:33:40.812298+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286384+0000","last_change":"2023-10-06T12:33:40.285226+0000","last_active":"2023-10-06T12:33:40.286384+0000","last_peered":"2023-10-06T12:33:40.286384+0000","last_clean":"2023-10-06T12:33:40.286384+0000","last_became_active":"2023-10-06T12:33:40.284880+0000","last_became_peered":"2023-10-06T12:33:40.284880+0000","last_unstale":"2023-10-06T12:33:40.286384+0000","last_undegraded":"2023-10-06T12:33:40.286384+0000","last_fullsized":"2023-10-06T12:33:40.286384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267489+0000","last_change":"2023-10-06T12:33:39.809422+0000","last_active":"2023-10-06T12:33:40.267489+0000","last_peered":"2023-10-06T12:33:40.267489+0000","last_clean":"2023-10-06T12:33:40.267489+0000","last_became_active":"2023-10-06T12:33:39.809122+0000","last_became_peered":"2023-10-06T12:33:39.809122+0000","last_unstale":"2023-10-06T12:33:40.267489+0000","last_undegraded":"2023-10-06T12:33:40.267489+0000","last_fullsized":"2023-10-06T12:33:40.267489+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:30.787738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289782+0000","last_change":"2023-10-06T12:33:40.289044+0000","last_active":"2023-10-06T12:33:40.289782+0000","last_peered":"2023-10-06T12:33:40.289782+0000","last_clean":"2023-10-06T12:33:40.289782+0000","last_became_active":"2023-10-06T12:33:40.288736+0000","last_became_peered":"2023-10-06T12:33:40.288736+0000","last_unstale":"2023-10-06T12:33:40.289782+0000","last_undegraded":"2023-10-06T12:33:40.289782+0000","last_fullsized":"2023-10-06T12:33:40.289782+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273368+0000","last_change":"2023-10-06T12:33:39.812514+0000","last_active":"2023-10-06T12:33:40.273368+0000","last_peered":"2023-10-06T12:33:40.273368+0000","last_clean":"2023-10-06T12:33:40.273368+0000","last_became_active":"2023-10-06T12:33:39.812148+0000","last_became_peered":"2023-10-06T12:33:39.812148+0000","last_unstale":"2023-10-06T12:33:40.273368+0000","last_undegraded":"2023-10-06T12:33:40.273368+0000","last_fullsized":"2023-10-06T12:33:40.273368+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:44.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295909+0000","last_change":"2023-10-06T12:33:40.295022+0000","last_active":"2023-10-06T12:33:40.295909+0000","last_peered":"2023-10-06T12:33:40.295909+0000","last_clean":"2023-10-06T12:33:40.295909+0000","last_became_active":"2023-10-06T12:33:40.294627+0000","last_became_peered":"2023-10-06T12:33:40.294627+0000","last_unstale":"2023-10-06T12:33:40.295909+0000","last_undegraded":"2023-10-06T12:33:40.295909+0000","last_fullsized":"2023-10-06T12:33:40.295909+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813958+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813379+0000","last_became_peered":"2023-10-06T12:33:39.813379+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:02.709927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287410+0000","last_change":"2023-10-06T12:33:40.285429+0000","last_active":"2023-10-06T12:33:40.287410+0000","last_peered":"2023-10-06T12:33:40.287410+0000","last_clean":"2023-10-06T12:33:40.287410+0000","last_became_active":"2023-10-06T12:33:40.285156+0000","last_became_peered":"2023-10-06T12:33:40.285156+0000","last_unstale":"2023-10-06T12:33:40.287410+0000","last_undegraded":"2023-10-06T12:33:40.287410+0000","last_fullsized":"2023-10-06T12:33:40.287410+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811309+0000","last_change":"2023-10-06T12:33:39.807549+0000","last_active":"2023-10-06T12:33:40.811309+0000","last_peered":"2023-10-06T12:33:40.811309+0000","last_clean":"2023-10-06T12:33:40.811309+0000","last_became_active":"2023-10-06T12:33:39.807256+0000","last_became_peered":"2023-10-06T12:33:39.807256+0000","last_unstale":"2023-10-06T12:33:40.811309+0000","last_undegraded":"2023-10-06T12:33:40.811309+0000","last_fullsized":"2023-10-06T12:33:40.811309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290283+0000","last_change":"2023-10-06T12:33:40.286426+0000","last_active":"2023-10-06T12:33:40.290283+0000","last_peered":"2023-10-06T12:33:40.290283+0000","last_clean":"2023-10-06T12:33:40.290283+0000","last_became_active":"2023-10-06T12:33:40.282983+0000","last_became_peered":"2023-10-06T12:33:40.282983+0000","last_unstale":"2023-10-06T12:33:40.290283+0000","last_undegraded":"2023-10-06T12:33:40.290283+0000","last_fullsized":"2023-10-06T12:33:40.290283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266369+0000","last_change":"2023-10-06T12:33:39.810049+0000","last_active":"2023-10-06T12:33:40.266369+0000","last_peered":"2023-10-06T12:33:40.266369+0000","last_clean":"2023-10-06T12:33:40.266369+0000","last_became_active":"2023-10-06T12:33:39.809771+0000","last_became_peered":"2023-10-06T12:33:39.809771+0000","last_unstale":"2023-10-06T12:33:40.266369+0000","last_undegraded":"2023-10-06T12:33:40.266369+0000","last_fullsized":"2023-10-06T12:33:40.266369+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291999+0000","last_change":"2023-10-06T12:33:40.285446+0000","last_active":"2023-10-06T12:33:40.291999+0000","last_peered":"2023-10-06T12:33:40.291999+0000","last_clean":"2023-10-06T12:33:40.291999+0000","last_became_active":"2023-10-06T12:33:40.285119+0000","last_became_peered":"2023-10-06T12:33:40.285119+0000","last_unstale":"2023-10-06T12:33:40.291999+0000","last_undegraded":"2023-10-06T12:33:40.291999+0000","last_fullsized":"2023-10-06T12:33:40.291999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.264857+0000","last_change":"2023-10-06T12:33:39.811717+0000","last_active":"2023-10-06T12:33:40.264857+0000","last_peered":"2023-10-06T12:33:40.264857+0000","last_clean":"2023-10-06T12:33:40.264857+0000","last_became_active":"2023-10-06T12:33:39.811418+0000","last_became_peered":"2023-10-06T12:33:39.811418+0000","last_unstale":"2023-10-06T12:33:40.264857+0000","last_undegraded":"2023-10-06T12:33:40.264857+0000","last_fullsized":"2023-10-06T12:33:40.264857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273738+0000","last_change":"2023-10-06T12:33:40.270916+0000","last_active":"2023-10-06T12:33:40.273738+0000","last_peered":"2023-10-06T12:33:40.273738+0000","last_clean":"2023-10-06T12:33:40.273738+0000","last_became_active":"2023-10-06T12:33:40.270606+0000","last_became_peered":"2023-10-06T12:33:40.270606+0000","last_unstale":"2023-10-06T12:33:40.273738+0000","last_undegraded":"2023-10-06T12:33:40.273738+0000","last_fullsized":"2023-10-06T12:33:40.273738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:32.377910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274558+0000","last_change":"2023-10-06T12:33:39.811637+0000","last_active":"2023-10-06T12:33:40.274558+0000","last_peered":"2023-10-06T12:33:40.274558+0000","last_clean":"2023-10-06T12:33:40.274558+0000","last_became_active":"2023-10-06T12:33:39.811347+0000","last_became_peered":"2023-10-06T12:33:39.811347+0000","last_unstale":"2023-10-06T12:33:40.274558+0000","last_undegraded":"2023-10-06T12:33:40.274558+0000","last_fullsized":"2023-10-06T12:33:40.274558+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:08:20.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281905+0000","last_change":"2023-10-06T12:33:40.276797+0000","last_active":"2023-10-06T12:33:40.281905+0000","last_peered":"2023-10-06T12:33:40.281905+0000","last_clean":"2023-10-06T12:33:40.281905+0000","last_became_active":"2023-10-06T12:33:40.276340+0000","last_became_peered":"2023-10-06T12:33:40.276340+0000","last_unstale":"2023-10-06T12:33:40.281905+0000","last_undegraded":"2023-10-06T12:33:40.281905+0000","last_fullsized":"2023-10-06T12:33:40.281905+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:55.302717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265680+0000","last_change":"2023-10-06T12:33:39.805296+0000","last_active":"2023-10-06T12:33:40.265680+0000","last_peered":"2023-10-06T12:33:40.265680+0000","last_clean":"2023-10-06T12:33:40.265680+0000","last_became_active":"2023-10-06T12:33:39.805032+0000","last_became_peered":"2023-10-06T12:33:39.805032+0000","last_unstale":"2023-10-06T12:33:40.265680+0000","last_undegraded":"2023-10-06T12:33:40.265680+0000","last_fullsized":"2023-10-06T12:33:40.265680+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:24.950347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281457+0000","last_change":"2023-10-06T12:33:40.276294+0000","last_active":"2023-10-06T12:33:40.281457+0000","last_peered":"2023-10-06T12:33:40.281457+0000","last_clean":"2023-10-06T12:33:40.281457+0000","last_became_active":"2023-10-06T12:33:40.275993+0000","last_became_peered":"2023-10-06T12:33:40.275993+0000","last_unstale":"2023-10-06T12:33:40.281457+0000","last_undegraded":"2023-10-06T12:33:40.281457+0000","last_fullsized":"2023-10-06T12:33:40.281457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:19.194989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271424+0000","last_change":"2023-10-06T12:33:39.815931+0000","last_active":"2023-10-06T12:33:40.271424+0000","last_peered":"2023-10-06T12:33:40.271424+0000","last_clean":"2023-10-06T12:33:40.271424+0000","last_became_active":"2023-10-06T12:33:39.815344+0000","last_became_peered":"2023-10-06T12:33:39.815344+0000","last_unstale":"2023-10-06T12:33:40.271424+0000","last_undegraded":"2023-10-06T12:33:40.271424+0000","last_fullsized":"2023-10-06T12:33:40.271424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281426+0000","last_change":"2023-10-06T12:33:40.276887+0000","last_active":"2023-10-06T12:33:40.281426+0000","last_peered":"2023-10-06T12:33:40.281426+0000","last_clean":"2023-10-06T12:33:40.281426+0000","last_became_active":"2023-10-06T12:33:40.276612+0000","last_became_peered":"2023-10-06T12:33:40.276612+0000","last_unstale":"2023-10-06T12:33:40.281426+0000","last_undegraded":"2023-10-06T12:33:40.281426+0000","last_fullsized":"2023-10-06T12:33:40.281426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:27.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813959+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813372+0000","last_became_peered":"2023-10-06T12:33:39.813372+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:25.113408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294809+0000","last_change":"2023-10-06T12:33:40.292608+0000","last_active":"2023-10-06T12:33:40.294809+0000","last_peered":"2023-10-06T12:33:40.294809+0000","last_clean":"2023-10-06T12:33:40.294809+0000","last_became_active":"2023-10-06T12:33:40.292296+0000","last_became_peered":"2023-10-06T12:33:40.292296+0000","last_unstale":"2023-10-06T12:33:40.294809+0000","last_undegraded":"2023-10-06T12:33:40.294809+0000","last_fullsized":"2023-10-06T12:33:40.294809+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267886+0000","last_change":"2023-10-06T12:33:39.809086+0000","last_active":"2023-10-06T12:33:40.267886+0000","last_peered":"2023-10-06T12:33:40.267886+0000","last_clean":"2023-10-06T12:33:40.267886+0000","last_became_active":"2023-10-06T12:33:39.808809+0000","last_became_peered":"2023-10-06T12:33:39.808809+0000","last_unstale":"2023-10-06T12:33:40.267886+0000","last_undegraded":"2023-10-06T12:33:40.267886+0000","last_fullsized":"2023-10-06T12:33:40.267886+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290334+0000","last_change":"2023-10-06T12:33:40.286561+0000","last_active":"2023-10-06T12:33:40.290334+0000","last_peered":"2023-10-06T12:33:40.290334+0000","last_clean":"2023-10-06T12:33:40.290334+0000","last_became_active":"2023-10-06T12:33:40.283580+0000","last_became_peered":"2023-10-06T12:33:40.283580+0000","last_unstale":"2023-10-06T12:33:40.290334+0000","last_undegraded":"2023-10-06T12:33:40.290334+0000","last_fullsized":"2023-10-06T12:33:40.290334+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271548+0000","last_change":"2023-10-06T12:33:39.812250+0000","last_active":"2023-10-06T12:33:40.271548+0000","last_peered":"2023-10-06T12:33:40.271548+0000","last_clean":"2023-10-06T12:33:40.271548+0000","last_became_active":"2023-10-06T12:33:39.811809+0000","last_became_peered":"2023-10-06T12:33:39.811809+0000","last_unstale":"2023-10-06T12:33:40.271548+0000","last_undegraded":"2023-10-06T12:33:40.271548+0000","last_fullsized":"2023-10-06T12:33:40.271548+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:24.023484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"142'271","reported_seq":822,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271897+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:33:40.271897+0000","last_peered":"2023-10-06T12:33:40.271897+0000","last_clean":"2023-10-06T12:33:40.271897+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:33:40.271897+0000","last_undegraded":"2023-10-06T12:33:40.271897+0000","last_fullsized":"2023-10-06T12:33:40.271897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265798+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:33:40.265798+0000","last_peered":"2023-10-06T12:33:40.265798+0000","last_clean":"2023-10-06T12:33:40.265798+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:33:40.265798+0000","last_undegraded":"2023-10-06T12:33:40.265798+0000","last_fullsized":"2023-10-06T12:33:40.265798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286113+0000","last_change":"2023-10-06T12:33:40.280700+0000","last_active":"2023-10-06T12:33:40.286113+0000","last_peered":"2023-10-06T12:33:40.286113+0000","last_clean":"2023-10-06T12:33:40.286113+0000","last_became_active":"2023-10-06T12:33:40.280391+0000","last_became_peered":"2023-10-06T12:33:40.280391+0000","last_unstale":"2023-10-06T12:33:40.286113+0000","last_undegraded":"2023-10-06T12:33:40.286113+0000","last_fullsized":"2023-10-06T12:33:40.286113+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283297+0000","last_change":"2023-10-06T12:33:39.800527+0000","last_active":"2023-10-06T12:33:40.283297+0000","last_peered":"2023-10-06T12:33:40.283297+0000","last_clean":"2023-10-06T12:33:40.283297+0000","last_became_active":"2023-10-06T12:33:39.800175+0000","last_became_peered":"2023-10-06T12:33:39.800175+0000","last_unstale":"2023-10-06T12:33:40.283297+0000","last_undegraded":"2023-10-06T12:33:40.283297+0000","last_fullsized":"2023-10-06T12:33:40.283297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:54.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271783+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:33:40.271783+0000","last_peered":"2023-10-06T12:33:40.271783+0000","last_clean":"2023-10-06T12:33:40.271783+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:33:40.271783+0000","last_undegraded":"2023-10-06T12:33:40.271783+0000","last_fullsized":"2023-10-06T12:33:40.271783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287437+0000","last_change":"2023-10-06T12:33:40.283772+0000","last_active":"2023-10-06T12:33:40.287437+0000","last_peered":"2023-10-06T12:33:40.287437+0000","last_clean":"2023-10-06T12:33:40.287437+0000","last_became_active":"2023-10-06T12:33:40.283455+0000","last_became_peered":"2023-10-06T12:33:40.283455+0000","last_unstale":"2023-10-06T12:33:40.287437+0000","last_undegraded":"2023-10-06T12:33:40.287437+0000","last_fullsized":"2023-10-06T12:33:40.287437+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812211+0000","last_change":"2023-10-06T12:33:39.814953+0000","last_active":"2023-10-06T12:33:40.812211+0000","last_peered":"2023-10-06T12:33:40.812211+0000","last_clean":"2023-10-06T12:33:40.812211+0000","last_became_active":"2023-10-06T12:33:39.814675+0000","last_became_peered":"2023-10-06T12:33:39.814675+0000","last_unstale":"2023-10-06T12:33:40.812211+0000","last_undegraded":"2023-10-06T12:33:40.812211+0000","last_fullsized":"2023-10-06T12:33:40.812211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.872666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266451+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:33:40.266451+0000","last_peered":"2023-10-06T12:33:40.266451+0000","last_clean":"2023-10-06T12:33:40.266451+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:33:40.266451+0000","last_undegraded":"2023-10-06T12:33:40.266451+0000","last_fullsized":"2023-10-06T12:33:40.266451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281384+0000","last_change":"2023-10-06T12:33:40.275173+0000","last_active":"2023-10-06T12:33:40.281384+0000","last_peered":"2023-10-06T12:33:40.281384+0000","last_clean":"2023-10-06T12:33:40.281384+0000","last_became_active":"2023-10-06T12:33:40.274912+0000","last_became_peered":"2023-10-06T12:33:40.274912+0000","last_unstale":"2023-10-06T12:33:40.281384+0000","last_undegraded":"2023-10-06T12:33:40.281384+0000","last_fullsized":"2023-10-06T12:33:40.281384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:15.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808626+0000","last_change":"2023-10-06T12:33:39.802677+0000","last_active":"2023-10-06T12:33:40.808626+0000","last_peered":"2023-10-06T12:33:40.808626+0000","last_clean":"2023-10-06T12:33:40.808626+0000","last_became_active":"2023-10-06T12:33:39.802329+0000","last_became_peered":"2023-10-06T12:33:39.802329+0000","last_unstale":"2023-10-06T12:33:40.808626+0000","last_undegraded":"2023-10-06T12:33:40.808626+0000","last_fullsized":"2023-10-06T12:33:40.808626+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:46.826229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265221+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:33:40.265221+0000","last_peered":"2023-10-06T12:33:40.265221+0000","last_clean":"2023-10-06T12:33:40.265221+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:33:40.265221+0000","last_undegraded":"2023-10-06T12:33:40.265221+0000","last_fullsized":"2023-10-06T12:33:40.265221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295188+0000","last_change":"2023-10-06T12:33:40.292699+0000","last_active":"2023-10-06T12:33:40.295188+0000","last_peered":"2023-10-06T12:33:40.295188+0000","last_clean":"2023-10-06T12:33:40.295188+0000","last_became_active":"2023-10-06T12:33:40.288567+0000","last_became_peered":"2023-10-06T12:33:40.288567+0000","last_unstale":"2023-10-06T12:33:40.295188+0000","last_undegraded":"2023-10-06T12:33:40.295188+0000","last_fullsized":"2023-10-06T12:33:40.295188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812728+0000","last_change":"2023-10-06T12:33:39.808806+0000","last_active":"2023-10-06T12:33:40.812728+0000","last_peered":"2023-10-06T12:33:40.812728+0000","last_clean":"2023-10-06T12:33:40.812728+0000","last_became_active":"2023-10-06T12:33:39.808486+0000","last_became_peered":"2023-10-06T12:33:39.808486+0000","last_unstale":"2023-10-06T12:33:40.812728+0000","last_undegraded":"2023-10-06T12:33:40.812728+0000","last_fullsized":"2023-10-06T12:33:40.812728+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274637+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:33:40.274637+0000","last_peered":"2023-10-06T12:33:40.274637+0000","last_clean":"2023-10-06T12:33:40.274637+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:33:40.274637+0000","last_undegraded":"2023-10-06T12:33:40.274637+0000","last_fullsized":"2023-10-06T12:33:40.274637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.289777+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289325+0000","last_became_peered":"2023-10-06T12:33:40.289325+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812531+0000","last_change":"2023-10-06T12:33:39.806182+0000","last_active":"2023-10-06T12:33:40.812531+0000","last_peered":"2023-10-06T12:33:40.812531+0000","last_clean":"2023-10-06T12:33:40.812531+0000","last_became_active":"2023-10-06T12:33:39.805865+0000","last_became_peered":"2023-10-06T12:33:39.805865+0000","last_unstale":"2023-10-06T12:33:40.812531+0000","last_undegraded":"2023-10-06T12:33:40.812531+0000","last_fullsized":"2023-10-06T12:33:40.812531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:59.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.270847+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:33:40.270847+0000","last_peered":"2023-10-06T12:33:40.270847+0000","last_clean":"2023-10-06T12:33:40.270847+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:33:40.270847+0000","last_undegraded":"2023-10-06T12:33:40.270847+0000","last_fullsized":"2023-10-06T12:33:40.270847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293890+0000","last_change":"2023-10-06T12:33:40.293011+0000","last_active":"2023-10-06T12:33:40.293890+0000","last_peered":"2023-10-06T12:33:40.293890+0000","last_clean":"2023-10-06T12:33:40.293890+0000","last_became_active":"2023-10-06T12:33:40.292669+0000","last_became_peered":"2023-10-06T12:33:40.292669+0000","last_unstale":"2023-10-06T12:33:40.293890+0000","last_undegraded":"2023-10-06T12:33:40.293890+0000","last_fullsized":"2023-10-06T12:33:40.293890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:46.979791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267782+0000","last_change":"2023-10-06T12:33:39.805085+0000","last_active":"2023-10-06T12:33:40.267782+0000","last_peered":"2023-10-06T12:33:40.267782+0000","last_clean":"2023-10-06T12:33:40.267782+0000","last_became_active":"2023-10-06T12:33:39.804573+0000","last_became_peered":"2023-10-06T12:33:39.804573+0000","last_unstale":"2023-10-06T12:33:40.267782+0000","last_undegraded":"2023-10-06T12:33:40.267782+0000","last_fullsized":"2023-10-06T12:33:40.267782+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274987+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:33:40.274987+0000","last_peered":"2023-10-06T12:33:40.274987+0000","last_clean":"2023-10-06T12:33:40.274987+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:33:40.274987+0000","last_undegraded":"2023-10-06T12:33:40.274987+0000","last_fullsized":"2023-10-06T12:33:40.274987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.285156+0000","last_change":"2023-10-06T12:33:40.282088+0000","last_active":"2023-10-06T12:33:40.285156+0000","last_peered":"2023-10-06T12:33:40.285156+0000","last_clean":"2023-10-06T12:33:40.285156+0000","last_became_active":"2023-10-06T12:33:40.281790+0000","last_became_peered":"2023-10-06T12:33:40.281790+0000","last_unstale":"2023-10-06T12:33:40.285156+0000","last_undegraded":"2023-10-06T12:33:40.285156+0000","last_fullsized":"2023-10-06T12:33:40.285156+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:40.112545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266012+0000","last_change":"2023-10-06T12:33:39.803873+0000","last_active":"2023-10-06T12:33:40.266012+0000","last_peered":"2023-10-06T12:33:40.266012+0000","last_clean":"2023-10-06T12:33:40.266012+0000","last_became_active":"2023-10-06T12:33:39.803599+0000","last_became_peered":"2023-10-06T12:33:39.803599+0000","last_unstale":"2023-10-06T12:33:40.266012+0000","last_undegraded":"2023-10-06T12:33:40.266012+0000","last_fullsized":"2023-10-06T12:33:40.266012+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:39.097471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287145+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:33:40.287145+0000","last_peered":"2023-10-06T12:33:40.287145+0000","last_clean":"2023-10-06T12:33:40.287145+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:33:40.287145+0000","last_undegraded":"2023-10-06T12:33:40.287145+0000","last_fullsized":"2023-10-06T12:33:40.287145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575749,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299744,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072096,"statfs":{"total":96636764160,"available":96329826304,"internally_reserved":0,"allocated":2514944,"data_stored":1592157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.155},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.423,"5min":1.6519999999999999,"15min":1.6819999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1890000000000001,"15min":1.1719999999999999},"min":{"1min":0.86899999999999999,"5min":0.73099999999999998,"15min":0.64000000000000001},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.91800000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5860000000000001,"5min":1.6200000000000001,"15min":1.7949999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":1.1040000000000001,"5min":0.92500000000000004,"15min":0.85599999999999998},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":1.855,"15min":1.855},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.262},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.65500000000000003},"max":{"1min":1.5189999999999999,"5min":1.6240000000000001,"15min":1.7649999999999999},"last":1.319}]},{"osd":3,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.2889999999999999},"min":{"1min":0.95499999999999996,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.363,"5min":1.345,"15min":1.329},"min":{"1min":1.0820000000000001,"5min":0.77500000000000002,"15min":0.70399999999999996},"max":{"1min":1.6619999999999999,"5min":1.7529999999999999,"15min":1.9019999999999999},"last":1.0629999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1120000000000001,"15min":1.107},"min":{"1min":0.77500000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.722,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.242,"15min":1.2370000000000001},"min":{"1min":0.871,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7649999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.347}]},{"osd":5,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.304,"15min":1.282},"min":{"1min":1.0409999999999999,"5min":0.626,"15min":0.49399999999999999},"max":{"1min":1.752,"5min":1.766,"15min":1.853},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.999,"5min":0.80600000000000005,"15min":0.55700000000000005},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.5800000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.389,"15min":1.3680000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.667},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.427,"15min":1.4039999999999999},"min":{"1min":1.2130000000000001,"5min":0.98399999999999999,"15min":0.54000000000000004},"max":{"1min":1.827,"5min":2.0499999999999998,"15min":2.073},"last":1.482}]}]},{"osd":6,"up_from":13,"seq":55834575749,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.171}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":1.0449999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.093},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":1.329}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.25},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.008}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.149},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.292}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.4850000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575749,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":0.88800000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.028},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":1.502}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.327},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.573},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.629}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.302}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.671}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":1000163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5800000000000001,"15min":1.8839999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.167,"15min":1.1890000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.887,"15min":2.0720000000000001},"last":1.5169999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.365},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6240000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.115},{"interface":"front","average":{"1min":1.27,"5min":1.325,"15min":1.329},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.554}]},{"osd":2,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.204,"15min":1.2390000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.506,"5min":1.625,"15min":1.9330000000000001},"last":1.002},{"interface":"front","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.619,"5min":1.861,"15min":1.861},"last":1.2569999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2370000000000001,"15min":1.2490000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5429999999999999,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":1.446},{"interface":"front","average":{"1min":1.321,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6339999999999999,"5min":1.7030000000000001,"15min":1.8340000000000001},"last":1.623}]},{"osd":5,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1919999999999999,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.56999999999999995},"max":{"1min":1.552,"5min":1.7929999999999999,"15min":1.944},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.163,"5min":1.1819999999999999,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.819},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.22,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.984},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3680000000000001,"15min":1.4019999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68799999999999994},"max":{"1min":1.754,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.415}]},{"osd":7,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.339,"15min":1.3620000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.304},{"interface":"front","average":{"1min":1.397,"5min":1.399,"15min":1.4359999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.69399999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1030000000000002},"last":1.4850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315548,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056292,"statfs":{"total":96636764160,"available":96313643008,"internally_reserved":0,"allocated":1921024,"data_stored":1001769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.999,"15min":1.0089999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.311},"max":{"1min":1.3089999999999999,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.52500000000000002},"max":{"1min":1.258,"5min":1.6100000000000001,"15min":1.8},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.54500000000000004},"max":{"1min":1.3580000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.23}]},{"osd":2,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1439999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.56899999999999995},"max":{"1min":1.623,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.337},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.252,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.59199999999999997},"max":{"1min":1.508,"5min":1.649,"15min":2.379},"last":1.095}]},{"osd":4,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3029999999999999,"15min":1.27},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.498},"max":{"1min":1.714,"5min":1.8700000000000001,"15min":2.02},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4410000000000001,"15min":1.3979999999999999},"min":{"1min":1.026,"5min":1.026,"15min":0.72499999999999998},"max":{"1min":1.8959999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.1990000000000001,"5min":1.0740000000000001,"15min":0.76900000000000002},"max":{"1min":1.784,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4390000000000001,"15min":1.419},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.518,"15min":1.4830000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.67600000000000005},"max":{"1min":1.907,"5min":2.0270000000000001,"15min":2.0449999999999999},"last":1.532},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4510000000000001,"15min":1.3979999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7649999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.97799999999999998}]},{"osd":7,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.468},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7430000000000001,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":1.296},{"interface":"front","average":{"1min":1.625,"5min":1.5780000000000001,"15min":1.5429999999999999},"min":{"1min":1.2949999999999999,"5min":1.1679999999999999,"15min":0.80000000000000004},"max":{"1min":1.9690000000000001,"5min":2.048,"15min":2.0600000000000001},"last":1.49}]}]},{"osd":2,"up_from":13,"seq":55834575749,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.325},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.0569999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.381},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.446}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.478}]}]},{"osd":1,"up_from":13,"seq":55834575749,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97899999999999998,"15min":0.96899999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.53900000000000003},"max":{"1min":1.7290000000000001,"5min":1.798,"15min":1.798},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.995,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.82399999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.655},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.52500000000000002},"max":{"1min":1.3069999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":0.871},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.1180000000000001},"min":{"1min":0.94199999999999995,"5min":0.77400000000000002,"15min":0.60099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.2370000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.61599999999999999},"max":{"1min":1.673,"5min":2.008,"15min":2.008},"last":1.649},{"interface":"front","average":{"1min":1.179,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.89900000000000002,"5min":0.77100000000000002,"15min":0.58199999999999996},"max":{"1min":1.4830000000000001,"5min":1.625,"15min":1.806},"last":1.054}]},{"osd":4,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.274,"15min":1.268},"min":{"1min":1.0569999999999999,"5min":0.96599999999999997,"15min":0.88},"max":{"1min":1.5629999999999999,"5min":1.764,"15min":2.2490000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4730000000000001,"15min":1.4359999999999999},"min":{"1min":1.319,"5min":1.0449999999999999,"15min":1.014},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.415}]},{"osd":5,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.403,"15min":1.3859999999999999},"min":{"1min":1.1220000000000001,"5min":1.0349999999999999,"15min":0.80900000000000005},"max":{"1min":1.704,"5min":1.9550000000000001,"15min":2.0350000000000001},"last":1.569},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3879999999999999,"15min":1.379},"min":{"1min":1.0600000000000001,"5min":0.97599999999999998,"15min":0.93400000000000005},"max":{"1min":1.696,"5min":1.917,"15min":2.0619999999999998},"last":1.609}]},{"osd":6,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.554,"15min":1.5109999999999999},"min":{"1min":1.2509999999999999,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.2360000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.4950000000000001,"15min":1.4710000000000001},"min":{"1min":1.236,"5min":1.129,"15min":1.0569999999999999},"max":{"1min":1.865,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.7609999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.5660000000000001},"min":{"1min":1.3140000000000001,"5min":1.282,"15min":0.99299999999999999},"max":{"1min":1.915,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":1.679},{"interface":"front","average":{"1min":1.575,"5min":1.5149999999999999,"15min":1.48},"min":{"1min":1.383,"5min":1.157,"15min":0.95799999999999996},"max":{"1min":1.847,"5min":2.0369999999999999,"15min":2.1819999999999999},"last":1.712}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299736,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072104,"statfs":{"total":96636764160,"available":96329834496,"internally_reserved":0,"allocated":2506752,"data_stored":1590587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.77200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.492},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.008,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.59799999999999998},"max":{"1min":1.4339999999999999,"5min":1.5549999999999999,"15min":1.595},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.78000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3340000000000001,"5min":1.5469999999999999,"15min":1.746},"last":1.26},{"interface":"front","average":{"1min":1.121,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.4970000000000001,"5min":1.762,"15min":1.762},"last":1.3}]},{"osd":3,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.099,"15min":1.0920000000000001},"min":{"1min":0.86199999999999999,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.202,"5min":0.84699999999999998,"15min":0.71999999999999997},"max":{"1min":1.7030000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.52}]},{"osd":4,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.371,"15min":1.369},"min":{"1min":1.1819999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.663,"5min":1.71,"15min":1.756},"last":1.204},{"interface":"front","average":{"1min":1.417,"5min":1.3340000000000001,"15min":1.3320000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.80500000000000005},"max":{"1min":1.704,"5min":1.7290000000000001,"15min":2.036},"last":1.5900000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.407,"15min":1.401},"min":{"1min":1.117,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9330000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.641},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4239999999999999,"15min":1.423},"min":{"1min":1.2350000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.865,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.6719999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.347,"15min":1.357},"min":{"1min":1.1879999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.649,"5min":1.732,"15min":2.2029999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.478,"15min":1.48},"min":{"1min":1.202,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.919,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.734}]},{"osd":7,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5029999999999999,"15min":1.482},"min":{"1min":1.2490000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8100000000000001,"5min":2.0099999999999998,"15min":2.1019999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4490000000000001},"min":{"1min":1.087,"5min":0.98599999999999999,"15min":0.95099999999999996},"max":{"1min":1.8480000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.7529999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:33:45.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T12:33:46.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.168+0000 7f66dd785700 1 Processor -- start 2023-10-06T12:33:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 -- start start 2023-10-06T12:33:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66d80ad6c0 0x7f66d80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d806d460 con 0x7f66d815ab60 2023-10-06T12:33:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d806ed10 con 0x7f66d80ad6c0 2023-10-06T12:33:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66dd785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80c7000 con 0x7f66d806e8a0 2023-10-06T12:33:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66d80ad6c0 0x7f66d80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d6ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66d815ab60 0x7f66d815af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56664/0 (socket says 172.21.15.46:56664) 2023-10-06T12:33:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d6ffd700 1 -- 172.21.15.46:0/3089569586 learned_addr learned my addr 172.21.15.46:0/3089569586 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66ce7fc700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d77fe700 1 -- 172.21.15.46:0/3089569586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 msgr2=0x7f66d806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d77fe700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d77fe700 1 -- 172.21.15.46:0/3089569586 >> v1:172.21.15.46:6789/0 conn(0x7f66d815ab60 legacy=0x7f66d815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.172+0000 7f66d77fe700 1 -- 172.21.15.46:0/3089569586 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80c72c0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66d77fe700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d80adb20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009e60 tx=0x7f66d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205d71572c1e9eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66d67fc700 1 -- 172.21.15.46:0/3089569586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000e070 con 0x7f66d80ad6c0 2023-10-06T12:33:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66d67fc700 1 -- 172.21.15.46:0/3089569586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d000bb60 con 0x7f66d80ad6c0 2023-10-06T12:33:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66d67fc700 1 -- 172.21.15.46:0/3089569586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0012430 con 0x7f66d80ad6c0 2023-10-06T12:33:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 -- 172.21.15.46:0/3089569586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 msgr2=0x7f66d80adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d80adb20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009e60 tx=0x7f66d0009780 comp rx=0 tx=0).stop 2023-10-06T12:33:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 -- 172.21.15.46:0/3089569586 shutdown_connections 2023-10-06T12:33:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d80adb20 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 --2- 172.21.15.46:0/3089569586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 -- 172.21.15.46:0/3089569586 >> 172.21.15.46:0/3089569586 conn(0x7f66d80cbe30 msgr2=0x7f66d80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 -- 172.21.15.46:0/3089569586 shutdown_connections 2023-10-06T12:33:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.176+0000 7f66dd785700 1 -- 172.21.15.46:0/3089569586 wait complete. 2023-10-06T12:33:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 Processor -- start 2023-10-06T12:33:46.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 -- start start 2023-10-06T12:33:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d80aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 0x7f66d8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80af830 con 0x7f66d80ad6c0 2023-10-06T12:33:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80afb10 con 0x7f66d815ab60 2023-10-06T12:33:46.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66dd785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80afdf0 con 0x7f66d806e8a0 2023-10-06T12:33:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38652/0 (socket says 172.21.15.46:38652) 2023-10-06T12:33:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 -- 172.21.15.46:0/872397097 learned_addr learned my addr 172.21.15.46:0/872397097 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:46.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66d77fe700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 0x7f66d8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66d6ffd700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d80aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 -- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 msgr2=0x7f66d80aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d80aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 -- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 msgr2=0x7f66d8159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 0x7f66d8159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 -- 172.21.15.46:0/872397097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009a20 con 0x7f66d80ad6c0 2023-10-06T12:33:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66d77fe700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 0x7f66d8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:46.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.180+0000 7f66ce7fc700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e930 tx=0x7f66c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c000eed0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000c850 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d80b00d0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c000cbf0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d8150ed0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d8063f90 con 0x7f66d80ad6c0 2023-10-06T12:33:46.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.184+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f66c0013470 con 0x7f66d80ad6c0 2023-10-06T12:33:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.188+0000 7f66cf7fe700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 0x7f66b8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.188+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f66c007cf30 con 0x7f66d80ad6c0 2023-10-06T12:33:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.188+0000 7f66d6ffd700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 0x7f66b8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.188+0000 7f66d6ffd700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 0x7f66b8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66c80010e0 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.192+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f66c004d010 con 0x7f66d80ad6c0 2023-10-06T12:33:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.420+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f66d804b5c0 con 0x7f66d80ad6c0 2023-10-06T12:33:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.420+0000 7f66cf7fe700 1 -- 172.21.15.46:0/872397097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1020 (secure 0 0 0) 0x7f66c0050630 con 0x7f66d80ad6c0 2023-10-06T12:33:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:46.424 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"36b5f543-5934-4e63-898c-253575113a16","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4549,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":153,"num_osds":8,"num_up_osds":8,"osd_up_since":1696591089,"num_in_osds":8,"osd_in_since":1696591080,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2518982656,"bytes_avail":770575130624,"bytes_total":773094113280,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8796}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T12:23:36.203655+0000","services":{}},"progress_events":{}} 2023-10-06T12:33:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 msgr2=0x7f66b8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 0x7f66b8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66c80010e0 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2023-10-06T12:33:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 msgr2=0x7f66d8150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e930 tx=0x7f66c00091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 shutdown_connections 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f66b8060670 0x7f66b8062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d815ab60 0x7f66d8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66d80ad6c0 0x7f66d8150990 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 --2- 172.21.15.46:0/872397097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f66d806e8a0 0x7f66d80aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 >> 172.21.15.46:0/872397097 conn(0x7f66d80cbe30 msgr2=0x7f66d80ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 shutdown_connections 2023-10-06T12:33:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.424+0000 7f66dd785700 1 -- 172.21.15.46:0/872397097 wait complete. 2023-10-06T12:33:46.447 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T12:33:46.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:33:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 Processor -- start 2023-10-06T12:33:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 -- start start 2023-10-06T12:33:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 0x7fecdc068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc06d790 con 0x7fecdc0aaf30 2023-10-06T12:33:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0acd60 con 0x7fecdc156e20 2023-10-06T12:33:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fece0ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0ad040 con 0x7fecdc067760 2023-10-06T12:33:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fecdb7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fecdbfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 0x7fecdc068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fecdb7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45304/0 (socket says 172.21.15.46:45304) 2023-10-06T12:33:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.608+0000 7fecdb7fe700 1 -- 172.21.15.46:0/1704025089 learned_addr learned my addr 172.21.15.46:0/1704025089 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdaffd700 1 -- 172.21.15.46:0/1704025089 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174761680 0 0) 0x7fecdc06d790 con 0x7fecdc0aaf30 2023-10-06T12:33:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdaffd700 1 -- 172.21.15.46:0/1704025089 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc8003570 con 0x7fecdc0aaf30 2023-10-06T12:33:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdbfff700 1 -- 172.21.15.46:0/1704025089 >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 msgr2=0x7fecdc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdbfff700 1 --2- 172.21.15.46:0/1704025089 >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdbfff700 1 -- 172.21.15.46:0/1704025089 >> v1:172.21.15.132:6789/0 conn(0x7fecdc0aaf30 legacy=0x7fecdc067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdbfff700 1 -- 172.21.15.46:0/1704025089 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0ad320 con 0x7fecdc067760 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdb7fe700 1 --2- 172.21.15.46:0/1704025089 >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdbfff700 1 --2- 172.21.15.46:0/1704025089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 0x7fecdc068a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009e60 tx=0x7fecd4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d6be8f9b978441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdaffd700 1 -- 172.21.15.46:0/1704025089 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400e070 con 0x7fecdc067760 2023-10-06T12:33:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdaffd700 1 -- 172.21.15.46:0/1704025089 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400bab0 con 0x7fecdc067760 2023-10-06T12:33:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fecdaffd700 1 -- 172.21.15.46:0/1704025089 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400be70 con 0x7fecdc067760 2023-10-06T12:33:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 -- 172.21.15.46:0/1704025089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 msgr2=0x7fecdc068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 --2- 172.21.15.46:0/1704025089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 0x7fecdc068a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009e60 tx=0x7fecd4009780 comp rx=0 tx=0).stop 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 -- 172.21.15.46:0/1704025089 shutdown_connections 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 --2- 172.21.15.46:0/1704025089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc067760 0x7fecdc068a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 --2- 172.21.15.46:0/1704025089 >> v2:172.21.15.132:3300/0 conn(0x7fecdc156e20 0x7fecdc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 -- 172.21.15.46:0/1704025089 >> 172.21.15.46:0/1704025089 conn(0x7fecdc062ef0 msgr2=0x7fecdc0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 -- 172.21.15.46:0/1704025089 shutdown_connections 2023-10-06T12:33:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.612+0000 7fece0ce3700 1 -- 172.21.15.46:0/1704025089 wait complete. 2023-10-06T12:33:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.616+0000 7fece0ce3700 1 Processor -- start 2023-10-06T12:33:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.616+0000 7fece0ce3700 1 -- start start 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecdc067760 0x7fecdc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc156e20 0x7fecdc154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc158650 con 0x7fecdc0aaf30 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc157a40 con 0x7fecdc067760 2023-10-06T12:33:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc157d20 con 0x7fecdc156e20 2023-10-06T12:33:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38664/0 (socket says 172.21.15.46:38664) 2023-10-06T12:33:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 -- 172.21.15.46:0/3916560628 learned_addr learned my addr 172.21.15.46:0/3916560628 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecdb7fe700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecdc067760 0x7fecdc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecdbfff700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc156e20 0x7fecdc154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 -- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc156e20 msgr2=0x7fecdc154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc156e20 0x7fecdc154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 -- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecdc067760 msgr2=0x7fecdc158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecdc067760 0x7fecdc158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 -- 172.21.15.46:0/3916560628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc0094f0 con 0x7fecdc0aaf30 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd3fff700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c790 tx=0x7fecc4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4010510 con 0x7fecdc0aaf30 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4010670 con 0x7fecdc0aaf30 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd4009a20 con 0x7fecdc0aaf30 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4010a10 con 0x7fecdc0aaf30 2023-10-06T12:33:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.620+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc1582e0 con 0x7fecdc0aaf30 2023-10-06T12:33:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.624+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc1587c0 con 0x7fecdc0aaf30 2023-10-06T12:33:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.624+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fecc401b3b0 con 0x7fecdc0aaf30 2023-10-06T12:33:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.628+0000 7fecd37fe700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 0x7fecbc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.628+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fecc407c580 con 0x7fecdc0aaf30 2023-10-06T12:33:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.628+0000 7fecdb7fe700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 0x7fecbc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.628+0000 7fecdb7fe700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 0x7fecbc062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feccc006d20 tx=0x7feccc009250 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.632+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fecc404c660 con 0x7fecdc0aaf30 2023-10-06T12:33:46.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.804+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecdc064590 con 0x7fecbc060670 2023-10-06T12:33:46.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.820+0000 7fecd37fe700 1 -- 172.21.15.46:0/3916560628 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308586 (secure 0 0 0) 0x7fecdc064590 con 0x7fecbc060670 2023-10-06T12:33:46.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 msgr2=0x7fecbc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 0x7fecbc062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feccc006d20 tx=0x7feccc009250 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 msgr2=0x7fecdc0c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c790 tx=0x7fecc4007e10 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 shutdown_connections 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fecbc060670 0x7fecbc062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fecdc156e20 0x7fecdc154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc0aaf30 0x7fecdc0c53b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 --2- 172.21.15.46:0/3916560628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecdc067760 0x7fecdc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 >> 172.21.15.46:0/3916560628 conn(0x7fecdc062ef0 msgr2=0x7fecdc156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 shutdown_connections 2023-10-06T12:33:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:46.828+0000 7fece0ce3700 1 -- 172.21.15.46:0/3916560628 wait complete. 2023-10-06T12:33:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:33:46.854 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T12:33:45.486411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459944,"kb_used_data":16104,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514776,"statfs":{"total":773094113280,"available":770575130624,"internally_reserved":0,"allocated":16490496,"data_stored":9186652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002022"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280402+0000","last_change":"2023-10-06T12:33:40.278094+0000","last_active":"2023-10-06T12:33:40.280402+0000","last_peered":"2023-10-06T12:33:40.280402+0000","last_clean":"2023-10-06T12:33:40.280402+0000","last_became_active":"2023-10-06T12:33:40.277766+0000","last_became_peered":"2023-10-06T12:33:40.277766+0000","last_unstale":"2023-10-06T12:33:40.280402+0000","last_undegraded":"2023-10-06T12:33:40.280402+0000","last_fullsized":"2023-10-06T12:33:40.280402+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:40.996759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278844+0000","last_change":"2023-10-06T12:33:40.276811+0000","last_active":"2023-10-06T12:33:40.278844+0000","last_peered":"2023-10-06T12:33:40.278844+0000","last_clean":"2023-10-06T12:33:40.278844+0000","last_became_active":"2023-10-06T12:33:40.276490+0000","last_became_peered":"2023-10-06T12:33:40.276490+0000","last_unstale":"2023-10-06T12:33:40.278844+0000","last_undegraded":"2023-10-06T12:33:40.278844+0000","last_fullsized":"2023-10-06T12:33:40.278844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293228+0000","last_change":"2023-10-06T12:33:40.289198+0000","last_active":"2023-10-06T12:33:40.293228+0000","last_peered":"2023-10-06T12:33:40.293228+0000","last_clean":"2023-10-06T12:33:40.293228+0000","last_became_active":"2023-10-06T12:33:40.288858+0000","last_became_peered":"2023-10-06T12:33:40.288858+0000","last_unstale":"2023-10-06T12:33:40.293228+0000","last_undegraded":"2023-10-06T12:33:40.293228+0000","last_fullsized":"2023-10-06T12:33:40.293228+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278895+0000","last_change":"2023-10-06T12:33:40.274309+0000","last_active":"2023-10-06T12:33:40.278895+0000","last_peered":"2023-10-06T12:33:40.278895+0000","last_clean":"2023-10-06T12:33:40.278895+0000","last_became_active":"2023-10-06T12:33:40.274015+0000","last_became_peered":"2023-10-06T12:33:40.274015+0000","last_unstale":"2023-10-06T12:33:40.278895+0000","last_undegraded":"2023-10-06T12:33:40.278895+0000","last_fullsized":"2023-10-06T12:33:40.278895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287401+0000","last_change":"2023-10-06T12:33:40.285006+0000","last_active":"2023-10-06T12:33:40.287401+0000","last_peered":"2023-10-06T12:33:40.287401+0000","last_clean":"2023-10-06T12:33:40.287401+0000","last_became_active":"2023-10-06T12:33:40.284722+0000","last_became_peered":"2023-10-06T12:33:40.284722+0000","last_unstale":"2023-10-06T12:33:40.287401+0000","last_undegraded":"2023-10-06T12:33:40.287401+0000","last_fullsized":"2023-10-06T12:33:40.287401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293763+0000","last_change":"2023-10-06T12:33:40.292772+0000","last_active":"2023-10-06T12:33:40.293763+0000","last_peered":"2023-10-06T12:33:40.293763+0000","last_clean":"2023-10-06T12:33:40.293763+0000","last_became_active":"2023-10-06T12:33:40.292458+0000","last_became_peered":"2023-10-06T12:33:40.292458+0000","last_unstale":"2023-10-06T12:33:40.293763+0000","last_undegraded":"2023-10-06T12:33:40.293763+0000","last_fullsized":"2023-10-06T12:33:40.293763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:47.841304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292102+0000","last_change":"2023-10-06T12:33:40.286279+0000","last_active":"2023-10-06T12:33:40.292102+0000","last_peered":"2023-10-06T12:33:40.292102+0000","last_clean":"2023-10-06T12:33:40.292102+0000","last_became_active":"2023-10-06T12:33:40.285975+0000","last_became_peered":"2023-10-06T12:33:40.285975+0000","last_unstale":"2023-10-06T12:33:40.292102+0000","last_undegraded":"2023-10-06T12:33:40.292102+0000","last_fullsized":"2023-10-06T12:33:40.292102+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284738+0000","last_change":"2023-10-06T12:33:40.274745+0000","last_active":"2023-10-06T12:33:40.284738+0000","last_peered":"2023-10-06T12:33:40.284738+0000","last_clean":"2023-10-06T12:33:40.284738+0000","last_became_active":"2023-10-06T12:33:40.274253+0000","last_became_peered":"2023-10-06T12:33:40.274253+0000","last_unstale":"2023-10-06T12:33:40.284738+0000","last_undegraded":"2023-10-06T12:33:40.284738+0000","last_fullsized":"2023-10-06T12:33:40.284738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:37.726541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281509+0000","last_change":"2023-10-06T12:33:40.277693+0000","last_active":"2023-10-06T12:33:40.281509+0000","last_peered":"2023-10-06T12:33:40.281509+0000","last_clean":"2023-10-06T12:33:40.281509+0000","last_became_active":"2023-10-06T12:33:40.277394+0000","last_became_peered":"2023-10-06T12:33:40.277394+0000","last_unstale":"2023-10-06T12:33:40.281509+0000","last_undegraded":"2023-10-06T12:33:40.281509+0000","last_fullsized":"2023-10-06T12:33:40.281509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:34.307339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273435+0000","last_change":"2023-10-06T12:33:40.269594+0000","last_active":"2023-10-06T12:33:40.273435+0000","last_peered":"2023-10-06T12:33:40.273435+0000","last_clean":"2023-10-06T12:33:40.273435+0000","last_became_active":"2023-10-06T12:33:40.269292+0000","last_became_peered":"2023-10-06T12:33:40.269292+0000","last_unstale":"2023-10-06T12:33:40.273435+0000","last_undegraded":"2023-10-06T12:33:40.273435+0000","last_fullsized":"2023-10-06T12:33:40.273435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:48.712468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.290272+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289949+0000","last_became_peered":"2023-10-06T12:33:40.289949+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281966+0000","last_change":"2023-10-06T12:33:40.278884+0000","last_active":"2023-10-06T12:33:40.281966+0000","last_peered":"2023-10-06T12:33:40.281966+0000","last_clean":"2023-10-06T12:33:40.281966+0000","last_became_active":"2023-10-06T12:33:40.278547+0000","last_became_peered":"2023-10-06T12:33:40.278547+0000","last_unstale":"2023-10-06T12:33:40.281966+0000","last_undegraded":"2023-10-06T12:33:40.281966+0000","last_fullsized":"2023-10-06T12:33:40.281966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291124+0000","last_change":"2023-10-06T12:33:40.290706+0000","last_active":"2023-10-06T12:33:40.291124+0000","last_peered":"2023-10-06T12:33:40.291124+0000","last_clean":"2023-10-06T12:33:40.291124+0000","last_became_active":"2023-10-06T12:33:40.290189+0000","last_became_peered":"2023-10-06T12:33:40.290189+0000","last_unstale":"2023-10-06T12:33:40.291124+0000","last_undegraded":"2023-10-06T12:33:40.291124+0000","last_fullsized":"2023-10-06T12:33:40.291124+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:31.802812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287127+0000","last_change":"2023-10-06T12:33:40.285957+0000","last_active":"2023-10-06T12:33:40.287127+0000","last_peered":"2023-10-06T12:33:40.287127+0000","last_clean":"2023-10-06T12:33:40.287127+0000","last_became_active":"2023-10-06T12:33:40.285663+0000","last_became_peered":"2023-10-06T12:33:40.285663+0000","last_unstale":"2023-10-06T12:33:40.287127+0000","last_undegraded":"2023-10-06T12:33:40.287127+0000","last_fullsized":"2023-10-06T12:33:40.287127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287455+0000","last_change":"2023-10-06T12:33:40.286241+0000","last_active":"2023-10-06T12:33:40.287455+0000","last_peered":"2023-10-06T12:33:40.287455+0000","last_clean":"2023-10-06T12:33:40.287455+0000","last_became_active":"2023-10-06T12:33:40.285913+0000","last_became_peered":"2023-10-06T12:33:40.285913+0000","last_unstale":"2023-10-06T12:33:40.287455+0000","last_undegraded":"2023-10-06T12:33:40.287455+0000","last_fullsized":"2023-10-06T12:33:40.287455+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282791+0000","last_change":"2023-10-06T12:33:40.281557+0000","last_active":"2023-10-06T12:33:40.282791+0000","last_peered":"2023-10-06T12:33:40.282791+0000","last_clean":"2023-10-06T12:33:40.282791+0000","last_became_active":"2023-10-06T12:33:40.281272+0000","last_became_peered":"2023-10-06T12:33:40.281272+0000","last_unstale":"2023-10-06T12:33:40.282791+0000","last_undegraded":"2023-10-06T12:33:40.282791+0000","last_fullsized":"2023-10-06T12:33:40.282791+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:25.331270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282795+0000","last_change":"2023-10-06T12:33:40.279525+0000","last_active":"2023-10-06T12:33:40.282795+0000","last_peered":"2023-10-06T12:33:40.282795+0000","last_clean":"2023-10-06T12:33:40.282795+0000","last_became_active":"2023-10-06T12:33:40.279160+0000","last_became_peered":"2023-10-06T12:33:40.279160+0000","last_unstale":"2023-10-06T12:33:40.282795+0000","last_undegraded":"2023-10-06T12:33:40.282795+0000","last_fullsized":"2023-10-06T12:33:40.282795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:56.644708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287429+0000","last_change":"2023-10-06T12:33:40.284511+0000","last_active":"2023-10-06T12:33:40.287429+0000","last_peered":"2023-10-06T12:33:40.287429+0000","last_clean":"2023-10-06T12:33:40.287429+0000","last_became_active":"2023-10-06T12:33:40.284083+0000","last_became_peered":"2023-10-06T12:33:40.284083+0000","last_unstale":"2023-10-06T12:33:40.287429+0000","last_undegraded":"2023-10-06T12:33:40.287429+0000","last_fullsized":"2023-10-06T12:33:40.287429+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292384+0000","last_change":"2023-10-06T12:33:40.286091+0000","last_active":"2023-10-06T12:33:40.292384+0000","last_peered":"2023-10-06T12:33:40.292384+0000","last_clean":"2023-10-06T12:33:40.292384+0000","last_became_active":"2023-10-06T12:33:40.285783+0000","last_became_peered":"2023-10-06T12:33:40.285783+0000","last_unstale":"2023-10-06T12:33:40.292384+0000","last_undegraded":"2023-10-06T12:33:40.292384+0000","last_fullsized":"2023-10-06T12:33:40.292384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286060+0000","last_change":"2023-10-06T12:33:40.283403+0000","last_active":"2023-10-06T12:33:40.286060+0000","last_peered":"2023-10-06T12:33:40.286060+0000","last_clean":"2023-10-06T12:33:40.286060+0000","last_became_active":"2023-10-06T12:33:40.283010+0000","last_became_peered":"2023-10-06T12:33:40.283010+0000","last_unstale":"2023-10-06T12:33:40.286060+0000","last_undegraded":"2023-10-06T12:33:40.286060+0000","last_fullsized":"2023-10-06T12:33:40.286060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.288358+0000","last_change":"2023-10-06T12:33:40.285829+0000","last_active":"2023-10-06T12:33:40.288358+0000","last_peered":"2023-10-06T12:33:40.288358+0000","last_clean":"2023-10-06T12:33:40.288358+0000","last_became_active":"2023-10-06T12:33:40.284389+0000","last_became_peered":"2023-10-06T12:33:40.284389+0000","last_unstale":"2023-10-06T12:33:40.288358+0000","last_undegraded":"2023-10-06T12:33:40.288358+0000","last_fullsized":"2023-10-06T12:33:40.288358+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281945+0000","last_change":"2023-10-06T12:33:40.280118+0000","last_active":"2023-10-06T12:33:40.281945+0000","last_peered":"2023-10-06T12:33:40.281945+0000","last_clean":"2023-10-06T12:33:40.281945+0000","last_became_active":"2023-10-06T12:33:40.279604+0000","last_became_peered":"2023-10-06T12:33:40.279604+0000","last_unstale":"2023-10-06T12:33:40.281945+0000","last_undegraded":"2023-10-06T12:33:40.281945+0000","last_fullsized":"2023-10-06T12:33:40.281945+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:11:13.472429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283037+0000","last_change":"2023-10-06T12:33:40.280532+0000","last_active":"2023-10-06T12:33:40.283037+0000","last_peered":"2023-10-06T12:33:40.283037+0000","last_clean":"2023-10-06T12:33:40.283037+0000","last_became_active":"2023-10-06T12:33:40.280248+0000","last_became_peered":"2023-10-06T12:33:40.280248+0000","last_unstale":"2023-10-06T12:33:40.283037+0000","last_undegraded":"2023-10-06T12:33:40.283037+0000","last_fullsized":"2023-10-06T12:33:40.283037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281656+0000","last_change":"2023-10-06T12:33:40.277730+0000","last_active":"2023-10-06T12:33:40.281656+0000","last_peered":"2023-10-06T12:33:40.281656+0000","last_clean":"2023-10-06T12:33:40.281656+0000","last_became_active":"2023-10-06T12:33:40.277481+0000","last_became_peered":"2023-10-06T12:33:40.277481+0000","last_unstale":"2023-10-06T12:33:40.281656+0000","last_undegraded":"2023-10-06T12:33:40.281656+0000","last_fullsized":"2023-10-06T12:33:40.281656+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:55.533377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295166+0000","last_change":"2023-10-06T12:33:40.290594+0000","last_active":"2023-10-06T12:33:40.295166+0000","last_peered":"2023-10-06T12:33:40.295166+0000","last_clean":"2023-10-06T12:33:40.295166+0000","last_became_active":"2023-10-06T12:33:40.290274+0000","last_became_peered":"2023-10-06T12:33:40.290274+0000","last_unstale":"2023-10-06T12:33:40.295166+0000","last_undegraded":"2023-10-06T12:33:40.295166+0000","last_fullsized":"2023-10-06T12:33:40.295166+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:26.400947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293038+0000","last_change":"2023-10-06T12:33:40.288927+0000","last_active":"2023-10-06T12:33:40.293038+0000","last_peered":"2023-10-06T12:33:40.293038+0000","last_clean":"2023-10-06T12:33:40.293038+0000","last_became_active":"2023-10-06T12:33:40.288580+0000","last_became_peered":"2023-10-06T12:33:40.288580+0000","last_unstale":"2023-10-06T12:33:40.293038+0000","last_undegraded":"2023-10-06T12:33:40.293038+0000","last_fullsized":"2023-10-06T12:33:40.293038+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291369+0000","last_change":"2023-10-06T12:33:40.282511+0000","last_active":"2023-10-06T12:33:40.291369+0000","last_peered":"2023-10-06T12:33:40.291369+0000","last_clean":"2023-10-06T12:33:40.291369+0000","last_became_active":"2023-10-06T12:33:40.282205+0000","last_became_peered":"2023-10-06T12:33:40.282205+0000","last_unstale":"2023-10-06T12:33:40.291369+0000","last_undegraded":"2023-10-06T12:33:40.291369+0000","last_fullsized":"2023-10-06T12:33:40.291369+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273907+0000","last_change":"2023-10-06T12:33:40.271338+0000","last_active":"2023-10-06T12:33:40.273907+0000","last_peered":"2023-10-06T12:33:40.273907+0000","last_clean":"2023-10-06T12:33:40.273907+0000","last_became_active":"2023-10-06T12:33:40.271069+0000","last_became_peered":"2023-10-06T12:33:40.271069+0000","last_unstale":"2023-10-06T12:33:40.273907+0000","last_undegraded":"2023-10-06T12:33:40.273907+0000","last_fullsized":"2023-10-06T12:33:40.273907+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.758266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294979+0000","last_change":"2023-10-06T12:33:40.290057+0000","last_active":"2023-10-06T12:33:40.294979+0000","last_peered":"2023-10-06T12:33:40.294979+0000","last_clean":"2023-10-06T12:33:40.294979+0000","last_became_active":"2023-10-06T12:33:40.288561+0000","last_became_peered":"2023-10-06T12:33:40.288561+0000","last_unstale":"2023-10-06T12:33:40.294979+0000","last_undegraded":"2023-10-06T12:33:40.294979+0000","last_fullsized":"2023-10-06T12:33:40.294979+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:57.704963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296209+0000","last_change":"2023-10-06T12:33:40.292961+0000","last_active":"2023-10-06T12:33:40.296209+0000","last_peered":"2023-10-06T12:33:40.296209+0000","last_clean":"2023-10-06T12:33:40.296209+0000","last_became_active":"2023-10-06T12:33:40.292379+0000","last_became_peered":"2023-10-06T12:33:40.292379+0000","last_unstale":"2023-10-06T12:33:40.296209+0000","last_undegraded":"2023-10-06T12:33:40.296209+0000","last_fullsized":"2023-10-06T12:33:40.296209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:03.774409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289262+0000","last_change":"2023-10-06T12:33:40.284299+0000","last_active":"2023-10-06T12:33:40.289262+0000","last_peered":"2023-10-06T12:33:40.289262+0000","last_clean":"2023-10-06T12:33:40.289262+0000","last_became_active":"2023-10-06T12:33:40.284014+0000","last_became_peered":"2023-10-06T12:33:40.284014+0000","last_unstale":"2023-10-06T12:33:40.289262+0000","last_undegraded":"2023-10-06T12:33:40.289262+0000","last_fullsized":"2023-10-06T12:33:40.289262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281701+0000","last_change":"2023-10-06T12:33:40.274503+0000","last_active":"2023-10-06T12:33:40.281701+0000","last_peered":"2023-10-06T12:33:40.281701+0000","last_clean":"2023-10-06T12:33:40.281701+0000","last_became_active":"2023-10-06T12:33:40.274196+0000","last_became_peered":"2023-10-06T12:33:40.274196+0000","last_unstale":"2023-10-06T12:33:40.281701+0000","last_undegraded":"2023-10-06T12:33:40.281701+0000","last_fullsized":"2023-10-06T12:33:40.281701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:20.140551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289181+0000","last_change":"2023-10-06T12:33:40.277982+0000","last_active":"2023-10-06T12:33:40.289181+0000","last_peered":"2023-10-06T12:33:40.289181+0000","last_clean":"2023-10-06T12:33:40.289181+0000","last_became_active":"2023-10-06T12:33:40.277677+0000","last_became_peered":"2023-10-06T12:33:40.277677+0000","last_unstale":"2023-10-06T12:33:40.289181+0000","last_undegraded":"2023-10-06T12:33:40.289181+0000","last_fullsized":"2023-10-06T12:33:40.289181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811674+0000","last_change":"2023-10-06T12:33:39.805457+0000","last_active":"2023-10-06T12:33:40.811674+0000","last_peered":"2023-10-06T12:33:40.811674+0000","last_clean":"2023-10-06T12:33:40.811674+0000","last_became_active":"2023-10-06T12:33:39.805053+0000","last_became_peered":"2023-10-06T12:33:39.805053+0000","last_unstale":"2023-10-06T12:33:40.811674+0000","last_undegraded":"2023-10-06T12:33:40.811674+0000","last_fullsized":"2023-10-06T12:33:40.811674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281933+0000","last_change":"2023-10-06T12:33:40.279910+0000","last_active":"2023-10-06T12:33:40.281933+0000","last_peered":"2023-10-06T12:33:40.281933+0000","last_clean":"2023-10-06T12:33:40.281933+0000","last_became_active":"2023-10-06T12:33:40.279582+0000","last_became_peered":"2023-10-06T12:33:40.279582+0000","last_unstale":"2023-10-06T12:33:40.281933+0000","last_undegraded":"2023-10-06T12:33:40.281933+0000","last_fullsized":"2023-10-06T12:33:40.281933+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:58.370893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808163+0000","last_change":"2023-10-06T12:33:39.807053+0000","last_active":"2023-10-06T12:33:40.808163+0000","last_peered":"2023-10-06T12:33:40.808163+0000","last_clean":"2023-10-06T12:33:40.808163+0000","last_became_active":"2023-10-06T12:33:39.806637+0000","last_became_peered":"2023-10-06T12:33:39.806637+0000","last_unstale":"2023-10-06T12:33:40.808163+0000","last_undegraded":"2023-10-06T12:33:40.808163+0000","last_fullsized":"2023-10-06T12:33:40.808163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280827+0000","last_change":"2023-10-06T12:33:40.279263+0000","last_active":"2023-10-06T12:33:40.280827+0000","last_peered":"2023-10-06T12:33:40.280827+0000","last_clean":"2023-10-06T12:33:40.280827+0000","last_became_active":"2023-10-06T12:33:40.278933+0000","last_became_peered":"2023-10-06T12:33:40.278933+0000","last_unstale":"2023-10-06T12:33:40.280827+0000","last_undegraded":"2023-10-06T12:33:40.280827+0000","last_fullsized":"2023-10-06T12:33:40.280827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.810492+0000","last_change":"2023-10-06T12:33:39.816889+0000","last_active":"2023-10-06T12:33:40.810492+0000","last_peered":"2023-10-06T12:33:40.810492+0000","last_clean":"2023-10-06T12:33:40.810492+0000","last_became_active":"2023-10-06T12:33:39.816379+0000","last_became_peered":"2023-10-06T12:33:39.816379+0000","last_unstale":"2023-10-06T12:33:40.810492+0000","last_undegraded":"2023-10-06T12:33:40.810492+0000","last_fullsized":"2023-10-06T12:33:40.810492+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:33.237343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286357+0000","last_change":"2023-10-06T12:33:40.285187+0000","last_active":"2023-10-06T12:33:40.286357+0000","last_peered":"2023-10-06T12:33:40.286357+0000","last_clean":"2023-10-06T12:33:40.286357+0000","last_became_active":"2023-10-06T12:33:40.284897+0000","last_became_peered":"2023-10-06T12:33:40.284897+0000","last_unstale":"2023-10-06T12:33:40.286357+0000","last_undegraded":"2023-10-06T12:33:40.286357+0000","last_fullsized":"2023-10-06T12:33:40.286357+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811184+0000","last_change":"2023-10-06T12:33:39.812575+0000","last_active":"2023-10-06T12:33:40.811184+0000","last_peered":"2023-10-06T12:33:40.811184+0000","last_clean":"2023-10-06T12:33:40.811184+0000","last_became_active":"2023-10-06T12:33:39.812156+0000","last_became_peered":"2023-10-06T12:33:39.812156+0000","last_unstale":"2023-10-06T12:33:40.811184+0000","last_undegraded":"2023-10-06T12:33:40.811184+0000","last_fullsized":"2023-10-06T12:33:40.811184+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:32.614790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289164+0000","last_change":"2023-10-06T12:33:40.276822+0000","last_active":"2023-10-06T12:33:40.289164+0000","last_peered":"2023-10-06T12:33:40.289164+0000","last_clean":"2023-10-06T12:33:40.289164+0000","last_became_active":"2023-10-06T12:33:40.276482+0000","last_became_peered":"2023-10-06T12:33:40.276482+0000","last_unstale":"2023-10-06T12:33:40.289164+0000","last_undegraded":"2023-10-06T12:33:40.289164+0000","last_fullsized":"2023-10-06T12:33:40.289164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809703+0000","last_change":"2023-10-06T12:33:39.814112+0000","last_active":"2023-10-06T12:33:40.809703+0000","last_peered":"2023-10-06T12:33:40.809703+0000","last_clean":"2023-10-06T12:33:40.809703+0000","last_became_active":"2023-10-06T12:33:39.813197+0000","last_became_peered":"2023-10-06T12:33:39.813197+0000","last_unstale":"2023-10-06T12:33:40.809703+0000","last_undegraded":"2023-10-06T12:33:40.809703+0000","last_fullsized":"2023-10-06T12:33:40.809703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:02.759335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284735+0000","last_change":"2023-10-06T12:33:40.272534+0000","last_active":"2023-10-06T12:33:40.284735+0000","last_peered":"2023-10-06T12:33:40.284735+0000","last_clean":"2023-10-06T12:33:40.284735+0000","last_became_active":"2023-10-06T12:33:40.272184+0000","last_became_peered":"2023-10-06T12:33:40.272184+0000","last_unstale":"2023-10-06T12:33:40.284735+0000","last_undegraded":"2023-10-06T12:33:40.284735+0000","last_fullsized":"2023-10-06T12:33:40.284735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:05.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809086+0000","last_change":"2023-10-06T12:33:39.812707+0000","last_active":"2023-10-06T12:33:40.809086+0000","last_peered":"2023-10-06T12:33:40.809086+0000","last_clean":"2023-10-06T12:33:40.809086+0000","last_became_active":"2023-10-06T12:33:39.812406+0000","last_became_peered":"2023-10-06T12:33:39.812406+0000","last_unstale":"2023-10-06T12:33:40.809086+0000","last_undegraded":"2023-10-06T12:33:40.809086+0000","last_fullsized":"2023-10-06T12:33:40.809086+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286027+0000","last_change":"2023-10-06T12:33:40.283254+0000","last_active":"2023-10-06T12:33:40.286027+0000","last_peered":"2023-10-06T12:33:40.286027+0000","last_clean":"2023-10-06T12:33:40.286027+0000","last_became_active":"2023-10-06T12:33:40.282952+0000","last_became_peered":"2023-10-06T12:33:40.282952+0000","last_unstale":"2023-10-06T12:33:40.286027+0000","last_undegraded":"2023-10-06T12:33:40.286027+0000","last_fullsized":"2023-10-06T12:33:40.286027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274181+0000","last_change":"2023-10-06T12:33:39.816556+0000","last_active":"2023-10-06T12:33:40.274181+0000","last_peered":"2023-10-06T12:33:40.274181+0000","last_clean":"2023-10-06T12:33:40.274181+0000","last_became_active":"2023-10-06T12:33:39.816213+0000","last_became_peered":"2023-10-06T12:33:39.816213+0000","last_unstale":"2023-10-06T12:33:40.274181+0000","last_undegraded":"2023-10-06T12:33:40.274181+0000","last_fullsized":"2023-10-06T12:33:40.274181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:56.132241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290868+0000","last_change":"2023-10-06T12:33:40.283081+0000","last_active":"2023-10-06T12:33:40.290868+0000","last_peered":"2023-10-06T12:33:40.290868+0000","last_clean":"2023-10-06T12:33:40.290868+0000","last_became_active":"2023-10-06T12:33:40.282501+0000","last_became_peered":"2023-10-06T12:33:40.282501+0000","last_unstale":"2023-10-06T12:33:40.290868+0000","last_undegraded":"2023-10-06T12:33:40.290868+0000","last_fullsized":"2023-10-06T12:33:40.290868+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286696+0000","last_change":"2023-10-06T12:33:39.810216+0000","last_active":"2023-10-06T12:33:40.286696+0000","last_peered":"2023-10-06T12:33:40.286696+0000","last_clean":"2023-10-06T12:33:40.286696+0000","last_became_active":"2023-10-06T12:33:39.809885+0000","last_became_peered":"2023-10-06T12:33:39.809885+0000","last_unstale":"2023-10-06T12:33:40.286696+0000","last_undegraded":"2023-10-06T12:33:40.286696+0000","last_fullsized":"2023-10-06T12:33:40.286696+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296114+0000","last_change":"2023-10-06T12:33:40.292823+0000","last_active":"2023-10-06T12:33:40.296114+0000","last_peered":"2023-10-06T12:33:40.296114+0000","last_clean":"2023-10-06T12:33:40.296114+0000","last_became_active":"2023-10-06T12:33:40.292457+0000","last_became_peered":"2023-10-06T12:33:40.292457+0000","last_unstale":"2023-10-06T12:33:40.296114+0000","last_undegraded":"2023-10-06T12:33:40.296114+0000","last_fullsized":"2023-10-06T12:33:40.296114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809539+0000","last_change":"2023-10-06T12:33:39.804873+0000","last_active":"2023-10-06T12:33:40.809539+0000","last_peered":"2023-10-06T12:33:40.809539+0000","last_clean":"2023-10-06T12:33:40.809539+0000","last_became_active":"2023-10-06T12:33:39.803841+0000","last_became_peered":"2023-10-06T12:33:39.803841+0000","last_unstale":"2023-10-06T12:33:40.809539+0000","last_undegraded":"2023-10-06T12:33:40.809539+0000","last_fullsized":"2023-10-06T12:33:40.809539+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:56.689888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291255+0000","last_change":"2023-10-06T12:33:40.284048+0000","last_active":"2023-10-06T12:33:40.291255+0000","last_peered":"2023-10-06T12:33:40.291255+0000","last_clean":"2023-10-06T12:33:40.291255+0000","last_became_active":"2023-10-06T12:33:40.283765+0000","last_became_peered":"2023-10-06T12:33:40.283765+0000","last_unstale":"2023-10-06T12:33:40.291255+0000","last_undegraded":"2023-10-06T12:33:40.291255+0000","last_fullsized":"2023-10-06T12:33:40.291255+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811535+0000","last_change":"2023-10-06T12:33:39.805882+0000","last_active":"2023-10-06T12:33:40.811535+0000","last_peered":"2023-10-06T12:33:40.811535+0000","last_clean":"2023-10-06T12:33:40.811535+0000","last_became_active":"2023-10-06T12:33:39.805035+0000","last_became_peered":"2023-10-06T12:33:39.805035+0000","last_unstale":"2023-10-06T12:33:40.811535+0000","last_undegraded":"2023-10-06T12:33:40.811535+0000","last_fullsized":"2023-10-06T12:33:40.811535+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290722+0000","last_change":"2023-10-06T12:33:40.282920+0000","last_active":"2023-10-06T12:33:40.290722+0000","last_peered":"2023-10-06T12:33:40.290722+0000","last_clean":"2023-10-06T12:33:40.290722+0000","last_became_active":"2023-10-06T12:33:40.282581+0000","last_became_peered":"2023-10-06T12:33:40.282581+0000","last_unstale":"2023-10-06T12:33:40.290722+0000","last_undegraded":"2023-10-06T12:33:40.290722+0000","last_fullsized":"2023-10-06T12:33:40.290722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.806365+0000","last_change":"2023-10-06T12:33:39.815967+0000","last_active":"2023-10-06T12:33:40.806365+0000","last_peered":"2023-10-06T12:33:40.806365+0000","last_clean":"2023-10-06T12:33:40.806365+0000","last_became_active":"2023-10-06T12:33:39.815667+0000","last_became_peered":"2023-10-06T12:33:39.815667+0000","last_unstale":"2023-10-06T12:33:40.806365+0000","last_undegraded":"2023-10-06T12:33:40.806365+0000","last_fullsized":"2023-10-06T12:33:40.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:05.141371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291363+0000","last_change":"2023-10-06T12:33:40.290471+0000","last_active":"2023-10-06T12:33:40.291363+0000","last_peered":"2023-10-06T12:33:40.291363+0000","last_clean":"2023-10-06T12:33:40.291363+0000","last_became_active":"2023-10-06T12:33:40.290101+0000","last_became_peered":"2023-10-06T12:33:40.290101+0000","last_unstale":"2023-10-06T12:33:40.291363+0000","last_undegraded":"2023-10-06T12:33:40.291363+0000","last_fullsized":"2023-10-06T12:33:40.291363+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:20.204914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266996+0000","last_change":"2023-10-06T12:33:39.811123+0000","last_active":"2023-10-06T12:33:40.266996+0000","last_peered":"2023-10-06T12:33:40.266996+0000","last_clean":"2023-10-06T12:33:40.266996+0000","last_became_active":"2023-10-06T12:33:39.810818+0000","last_became_peered":"2023-10-06T12:33:39.810818+0000","last_unstale":"2023-10-06T12:33:40.266996+0000","last_undegraded":"2023-10-06T12:33:40.266996+0000","last_fullsized":"2023-10-06T12:33:40.266996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280655+0000","last_change":"2023-10-06T12:33:40.275550+0000","last_active":"2023-10-06T12:33:40.280655+0000","last_peered":"2023-10-06T12:33:40.280655+0000","last_clean":"2023-10-06T12:33:40.280655+0000","last_became_active":"2023-10-06T12:33:40.275264+0000","last_became_peered":"2023-10-06T12:33:40.275264+0000","last_unstale":"2023-10-06T12:33:40.280655+0000","last_undegraded":"2023-10-06T12:33:40.280655+0000","last_fullsized":"2023-10-06T12:33:40.280655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812298+0000","last_change":"2023-10-06T12:33:39.811347+0000","last_active":"2023-10-06T12:33:40.812298+0000","last_peered":"2023-10-06T12:33:40.812298+0000","last_clean":"2023-10-06T12:33:40.812298+0000","last_became_active":"2023-10-06T12:33:39.810887+0000","last_became_peered":"2023-10-06T12:33:39.810887+0000","last_unstale":"2023-10-06T12:33:40.812298+0000","last_undegraded":"2023-10-06T12:33:40.812298+0000","last_fullsized":"2023-10-06T12:33:40.812298+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286384+0000","last_change":"2023-10-06T12:33:40.285226+0000","last_active":"2023-10-06T12:33:40.286384+0000","last_peered":"2023-10-06T12:33:40.286384+0000","last_clean":"2023-10-06T12:33:40.286384+0000","last_became_active":"2023-10-06T12:33:40.284880+0000","last_became_peered":"2023-10-06T12:33:40.284880+0000","last_unstale":"2023-10-06T12:33:40.286384+0000","last_undegraded":"2023-10-06T12:33:40.286384+0000","last_fullsized":"2023-10-06T12:33:40.286384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267489+0000","last_change":"2023-10-06T12:33:39.809422+0000","last_active":"2023-10-06T12:33:40.267489+0000","last_peered":"2023-10-06T12:33:40.267489+0000","last_clean":"2023-10-06T12:33:40.267489+0000","last_became_active":"2023-10-06T12:33:39.809122+0000","last_became_peered":"2023-10-06T12:33:39.809122+0000","last_unstale":"2023-10-06T12:33:40.267489+0000","last_undegraded":"2023-10-06T12:33:40.267489+0000","last_fullsized":"2023-10-06T12:33:40.267489+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:30.787738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289782+0000","last_change":"2023-10-06T12:33:40.289044+0000","last_active":"2023-10-06T12:33:40.289782+0000","last_peered":"2023-10-06T12:33:40.289782+0000","last_clean":"2023-10-06T12:33:40.289782+0000","last_became_active":"2023-10-06T12:33:40.288736+0000","last_became_peered":"2023-10-06T12:33:40.288736+0000","last_unstale":"2023-10-06T12:33:40.289782+0000","last_undegraded":"2023-10-06T12:33:40.289782+0000","last_fullsized":"2023-10-06T12:33:40.289782+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273368+0000","last_change":"2023-10-06T12:33:39.812514+0000","last_active":"2023-10-06T12:33:40.273368+0000","last_peered":"2023-10-06T12:33:40.273368+0000","last_clean":"2023-10-06T12:33:40.273368+0000","last_became_active":"2023-10-06T12:33:39.812148+0000","last_became_peered":"2023-10-06T12:33:39.812148+0000","last_unstale":"2023-10-06T12:33:40.273368+0000","last_undegraded":"2023-10-06T12:33:40.273368+0000","last_fullsized":"2023-10-06T12:33:40.273368+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:44.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295909+0000","last_change":"2023-10-06T12:33:40.295022+0000","last_active":"2023-10-06T12:33:40.295909+0000","last_peered":"2023-10-06T12:33:40.295909+0000","last_clean":"2023-10-06T12:33:40.295909+0000","last_became_active":"2023-10-06T12:33:40.294627+0000","last_became_peered":"2023-10-06T12:33:40.294627+0000","last_unstale":"2023-10-06T12:33:40.295909+0000","last_undegraded":"2023-10-06T12:33:40.295909+0000","last_fullsized":"2023-10-06T12:33:40.295909+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813958+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813379+0000","last_became_peered":"2023-10-06T12:33:39.813379+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:02.709927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287410+0000","last_change":"2023-10-06T12:33:40.285429+0000","last_active":"2023-10-06T12:33:40.287410+0000","last_peered":"2023-10-06T12:33:40.287410+0000","last_clean":"2023-10-06T12:33:40.287410+0000","last_became_active":"2023-10-06T12:33:40.285156+0000","last_became_peered":"2023-10-06T12:33:40.285156+0000","last_unstale":"2023-10-06T12:33:40.287410+0000","last_undegraded":"2023-10-06T12:33:40.287410+0000","last_fullsized":"2023-10-06T12:33:40.287410+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811309+0000","last_change":"2023-10-06T12:33:39.807549+0000","last_active":"2023-10-06T12:33:40.811309+0000","last_peered":"2023-10-06T12:33:40.811309+0000","last_clean":"2023-10-06T12:33:40.811309+0000","last_became_active":"2023-10-06T12:33:39.807256+0000","last_became_peered":"2023-10-06T12:33:39.807256+0000","last_unstale":"2023-10-06T12:33:40.811309+0000","last_undegraded":"2023-10-06T12:33:40.811309+0000","last_fullsized":"2023-10-06T12:33:40.811309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290283+0000","last_change":"2023-10-06T12:33:40.286426+0000","last_active":"2023-10-06T12:33:40.290283+0000","last_peered":"2023-10-06T12:33:40.290283+0000","last_clean":"2023-10-06T12:33:40.290283+0000","last_became_active":"2023-10-06T12:33:40.282983+0000","last_became_peered":"2023-10-06T12:33:40.282983+0000","last_unstale":"2023-10-06T12:33:40.290283+0000","last_undegraded":"2023-10-06T12:33:40.290283+0000","last_fullsized":"2023-10-06T12:33:40.290283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266369+0000","last_change":"2023-10-06T12:33:39.810049+0000","last_active":"2023-10-06T12:33:40.266369+0000","last_peered":"2023-10-06T12:33:40.266369+0000","last_clean":"2023-10-06T12:33:40.266369+0000","last_became_active":"2023-10-06T12:33:39.809771+0000","last_became_peered":"2023-10-06T12:33:39.809771+0000","last_unstale":"2023-10-06T12:33:40.266369+0000","last_undegraded":"2023-10-06T12:33:40.266369+0000","last_fullsized":"2023-10-06T12:33:40.266369+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291999+0000","last_change":"2023-10-06T12:33:40.285446+0000","last_active":"2023-10-06T12:33:40.291999+0000","last_peered":"2023-10-06T12:33:40.291999+0000","last_clean":"2023-10-06T12:33:40.291999+0000","last_became_active":"2023-10-06T12:33:40.285119+0000","last_became_peered":"2023-10-06T12:33:40.285119+0000","last_unstale":"2023-10-06T12:33:40.291999+0000","last_undegraded":"2023-10-06T12:33:40.291999+0000","last_fullsized":"2023-10-06T12:33:40.291999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.264857+0000","last_change":"2023-10-06T12:33:39.811717+0000","last_active":"2023-10-06T12:33:40.264857+0000","last_peered":"2023-10-06T12:33:40.264857+0000","last_clean":"2023-10-06T12:33:40.264857+0000","last_became_active":"2023-10-06T12:33:39.811418+0000","last_became_peered":"2023-10-06T12:33:39.811418+0000","last_unstale":"2023-10-06T12:33:40.264857+0000","last_undegraded":"2023-10-06T12:33:40.264857+0000","last_fullsized":"2023-10-06T12:33:40.264857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273738+0000","last_change":"2023-10-06T12:33:40.270916+0000","last_active":"2023-10-06T12:33:40.273738+0000","last_peered":"2023-10-06T12:33:40.273738+0000","last_clean":"2023-10-06T12:33:40.273738+0000","last_became_active":"2023-10-06T12:33:40.270606+0000","last_became_peered":"2023-10-06T12:33:40.270606+0000","last_unstale":"2023-10-06T12:33:40.273738+0000","last_undegraded":"2023-10-06T12:33:40.273738+0000","last_fullsized":"2023-10-06T12:33:40.273738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:32.377910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274558+0000","last_change":"2023-10-06T12:33:39.811637+0000","last_active":"2023-10-06T12:33:40.274558+0000","last_peered":"2023-10-06T12:33:40.274558+0000","last_clean":"2023-10-06T12:33:40.274558+0000","last_became_active":"2023-10-06T12:33:39.811347+0000","last_became_peered":"2023-10-06T12:33:39.811347+0000","last_unstale":"2023-10-06T12:33:40.274558+0000","last_undegraded":"2023-10-06T12:33:40.274558+0000","last_fullsized":"2023-10-06T12:33:40.274558+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:08:20.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281905+0000","last_change":"2023-10-06T12:33:40.276797+0000","last_active":"2023-10-06T12:33:40.281905+0000","last_peered":"2023-10-06T12:33:40.281905+0000","last_clean":"2023-10-06T12:33:40.281905+0000","last_became_active":"2023-10-06T12:33:40.276340+0000","last_became_peered":"2023-10-06T12:33:40.276340+0000","last_unstale":"2023-10-06T12:33:40.281905+0000","last_undegraded":"2023-10-06T12:33:40.281905+0000","last_fullsized":"2023-10-06T12:33:40.281905+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:55.302717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265680+0000","last_change":"2023-10-06T12:33:39.805296+0000","last_active":"2023-10-06T12:33:40.265680+0000","last_peered":"2023-10-06T12:33:40.265680+0000","last_clean":"2023-10-06T12:33:40.265680+0000","last_became_active":"2023-10-06T12:33:39.805032+0000","last_became_peered":"2023-10-06T12:33:39.805032+0000","last_unstale":"2023-10-06T12:33:40.265680+0000","last_undegraded":"2023-10-06T12:33:40.265680+0000","last_fullsized":"2023-10-06T12:33:40.265680+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:24.950347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281457+0000","last_change":"2023-10-06T12:33:40.276294+0000","last_active":"2023-10-06T12:33:40.281457+0000","last_peered":"2023-10-06T12:33:40.281457+0000","last_clean":"2023-10-06T12:33:40.281457+0000","last_became_active":"2023-10-06T12:33:40.275993+0000","last_became_peered":"2023-10-06T12:33:40.275993+0000","last_unstale":"2023-10-06T12:33:40.281457+0000","last_undegraded":"2023-10-06T12:33:40.281457+0000","last_fullsized":"2023-10-06T12:33:40.281457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:19.194989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271424+0000","last_change":"2023-10-06T12:33:39.815931+0000","last_active":"2023-10-06T12:33:40.271424+0000","last_peered":"2023-10-06T12:33:40.271424+0000","last_clean":"2023-10-06T12:33:40.271424+0000","last_became_active":"2023-10-06T12:33:39.815344+0000","last_became_peered":"2023-10-06T12:33:39.815344+0000","last_unstale":"2023-10-06T12:33:40.271424+0000","last_undegraded":"2023-10-06T12:33:40.271424+0000","last_fullsized":"2023-10-06T12:33:40.271424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281426+0000","last_change":"2023-10-06T12:33:40.276887+0000","last_active":"2023-10-06T12:33:40.281426+0000","last_peered":"2023-10-06T12:33:40.281426+0000","last_clean":"2023-10-06T12:33:40.281426+0000","last_became_active":"2023-10-06T12:33:40.276612+0000","last_became_peered":"2023-10-06T12:33:40.276612+0000","last_unstale":"2023-10-06T12:33:40.281426+0000","last_undegraded":"2023-10-06T12:33:40.281426+0000","last_fullsized":"2023-10-06T12:33:40.281426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:27.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813959+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813372+0000","last_became_peered":"2023-10-06T12:33:39.813372+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:25.113408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294809+0000","last_change":"2023-10-06T12:33:40.292608+0000","last_active":"2023-10-06T12:33:40.294809+0000","last_peered":"2023-10-06T12:33:40.294809+0000","last_clean":"2023-10-06T12:33:40.294809+0000","last_became_active":"2023-10-06T12:33:40.292296+0000","last_became_peered":"2023-10-06T12:33:40.292296+0000","last_unstale":"2023-10-06T12:33:40.294809+0000","last_undegraded":"2023-10-06T12:33:40.294809+0000","last_fullsized":"2023-10-06T12:33:40.294809+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267886+0000","last_change":"2023-10-06T12:33:39.809086+0000","last_active":"2023-10-06T12:33:40.267886+0000","last_peered":"2023-10-06T12:33:40.267886+0000","last_clean":"2023-10-06T12:33:40.267886+0000","last_became_active":"2023-10-06T12:33:39.808809+0000","last_became_peered":"2023-10-06T12:33:39.808809+0000","last_unstale":"2023-10-06T12:33:40.267886+0000","last_undegraded":"2023-10-06T12:33:40.267886+0000","last_fullsized":"2023-10-06T12:33:40.267886+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290334+0000","last_change":"2023-10-06T12:33:40.286561+0000","last_active":"2023-10-06T12:33:40.290334+0000","last_peered":"2023-10-06T12:33:40.290334+0000","last_clean":"2023-10-06T12:33:40.290334+0000","last_became_active":"2023-10-06T12:33:40.283580+0000","last_became_peered":"2023-10-06T12:33:40.283580+0000","last_unstale":"2023-10-06T12:33:40.290334+0000","last_undegraded":"2023-10-06T12:33:40.290334+0000","last_fullsized":"2023-10-06T12:33:40.290334+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271548+0000","last_change":"2023-10-06T12:33:39.812250+0000","last_active":"2023-10-06T12:33:40.271548+0000","last_peered":"2023-10-06T12:33:40.271548+0000","last_clean":"2023-10-06T12:33:40.271548+0000","last_became_active":"2023-10-06T12:33:39.811809+0000","last_became_peered":"2023-10-06T12:33:39.811809+0000","last_unstale":"2023-10-06T12:33:40.271548+0000","last_undegraded":"2023-10-06T12:33:40.271548+0000","last_fullsized":"2023-10-06T12:33:40.271548+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:24.023484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"142'271","reported_seq":822,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271897+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:33:40.271897+0000","last_peered":"2023-10-06T12:33:40.271897+0000","last_clean":"2023-10-06T12:33:40.271897+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:33:40.271897+0000","last_undegraded":"2023-10-06T12:33:40.271897+0000","last_fullsized":"2023-10-06T12:33:40.271897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265798+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:33:40.265798+0000","last_peered":"2023-10-06T12:33:40.265798+0000","last_clean":"2023-10-06T12:33:40.265798+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:33:40.265798+0000","last_undegraded":"2023-10-06T12:33:40.265798+0000","last_fullsized":"2023-10-06T12:33:40.265798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286113+0000","last_change":"2023-10-06T12:33:40.280700+0000","last_active":"2023-10-06T12:33:40.286113+0000","last_peered":"2023-10-06T12:33:40.286113+0000","last_clean":"2023-10-06T12:33:40.286113+0000","last_became_active":"2023-10-06T12:33:40.280391+0000","last_became_peered":"2023-10-06T12:33:40.280391+0000","last_unstale":"2023-10-06T12:33:40.286113+0000","last_undegraded":"2023-10-06T12:33:40.286113+0000","last_fullsized":"2023-10-06T12:33:40.286113+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283297+0000","last_change":"2023-10-06T12:33:39.800527+0000","last_active":"2023-10-06T12:33:40.283297+0000","last_peered":"2023-10-06T12:33:40.283297+0000","last_clean":"2023-10-06T12:33:40.283297+0000","last_became_active":"2023-10-06T12:33:39.800175+0000","last_became_peered":"2023-10-06T12:33:39.800175+0000","last_unstale":"2023-10-06T12:33:40.283297+0000","last_undegraded":"2023-10-06T12:33:40.283297+0000","last_fullsized":"2023-10-06T12:33:40.283297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:54.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271783+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:33:40.271783+0000","last_peered":"2023-10-06T12:33:40.271783+0000","last_clean":"2023-10-06T12:33:40.271783+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:33:40.271783+0000","last_undegraded":"2023-10-06T12:33:40.271783+0000","last_fullsized":"2023-10-06T12:33:40.271783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287437+0000","last_change":"2023-10-06T12:33:40.283772+0000","last_active":"2023-10-06T12:33:40.287437+0000","last_peered":"2023-10-06T12:33:40.287437+0000","last_clean":"2023-10-06T12:33:40.287437+0000","last_became_active":"2023-10-06T12:33:40.283455+0000","last_became_peered":"2023-10-06T12:33:40.283455+0000","last_unstale":"2023-10-06T12:33:40.287437+0000","last_undegraded":"2023-10-06T12:33:40.287437+0000","last_fullsized":"2023-10-06T12:33:40.287437+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812211+0000","last_change":"2023-10-06T12:33:39.814953+0000","last_active":"2023-10-06T12:33:40.812211+0000","last_peered":"2023-10-06T12:33:40.812211+0000","last_clean":"2023-10-06T12:33:40.812211+0000","last_became_active":"2023-10-06T12:33:39.814675+0000","last_became_peered":"2023-10-06T12:33:39.814675+0000","last_unstale":"2023-10-06T12:33:40.812211+0000","last_undegraded":"2023-10-06T12:33:40.812211+0000","last_fullsized":"2023-10-06T12:33:40.812211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.872666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266451+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:33:40.266451+0000","last_peered":"2023-10-06T12:33:40.266451+0000","last_clean":"2023-10-06T12:33:40.266451+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:33:40.266451+0000","last_undegraded":"2023-10-06T12:33:40.266451+0000","last_fullsized":"2023-10-06T12:33:40.266451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281384+0000","last_change":"2023-10-06T12:33:40.275173+0000","last_active":"2023-10-06T12:33:40.281384+0000","last_peered":"2023-10-06T12:33:40.281384+0000","last_clean":"2023-10-06T12:33:40.281384+0000","last_became_active":"2023-10-06T12:33:40.274912+0000","last_became_peered":"2023-10-06T12:33:40.274912+0000","last_unstale":"2023-10-06T12:33:40.281384+0000","last_undegraded":"2023-10-06T12:33:40.281384+0000","last_fullsized":"2023-10-06T12:33:40.281384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:15.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808626+0000","last_change":"2023-10-06T12:33:39.802677+0000","last_active":"2023-10-06T12:33:40.808626+0000","last_peered":"2023-10-06T12:33:40.808626+0000","last_clean":"2023-10-06T12:33:40.808626+0000","last_became_active":"2023-10-06T12:33:39.802329+0000","last_became_peered":"2023-10-06T12:33:39.802329+0000","last_unstale":"2023-10-06T12:33:40.808626+0000","last_undegraded":"2023-10-06T12:33:40.808626+0000","last_fullsized":"2023-10-06T12:33:40.808626+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:46.826229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265221+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:33:40.265221+0000","last_peered":"2023-10-06T12:33:40.265221+0000","last_clean":"2023-10-06T12:33:40.265221+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:33:40.265221+0000","last_undegraded":"2023-10-06T12:33:40.265221+0000","last_fullsized":"2023-10-06T12:33:40.265221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295188+0000","last_change":"2023-10-06T12:33:40.292699+0000","last_active":"2023-10-06T12:33:40.295188+0000","last_peered":"2023-10-06T12:33:40.295188+0000","last_clean":"2023-10-06T12:33:40.295188+0000","last_became_active":"2023-10-06T12:33:40.288567+0000","last_became_peered":"2023-10-06T12:33:40.288567+0000","last_unstale":"2023-10-06T12:33:40.295188+0000","last_undegraded":"2023-10-06T12:33:40.295188+0000","last_fullsized":"2023-10-06T12:33:40.295188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812728+0000","last_change":"2023-10-06T12:33:39.808806+0000","last_active":"2023-10-06T12:33:40.812728+0000","last_peered":"2023-10-06T12:33:40.812728+0000","last_clean":"2023-10-06T12:33:40.812728+0000","last_became_active":"2023-10-06T12:33:39.808486+0000","last_became_peered":"2023-10-06T12:33:39.808486+0000","last_unstale":"2023-10-06T12:33:40.812728+0000","last_undegraded":"2023-10-06T12:33:40.812728+0000","last_fullsized":"2023-10-06T12:33:40.812728+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274637+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:33:40.274637+0000","last_peered":"2023-10-06T12:33:40.274637+0000","last_clean":"2023-10-06T12:33:40.274637+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:33:40.274637+0000","last_undegraded":"2023-10-06T12:33:40.274637+0000","last_fullsized":"2023-10-06T12:33:40.274637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.289777+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289325+0000","last_became_peered":"2023-10-06T12:33:40.289325+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812531+0000","last_change":"2023-10-06T12:33:39.806182+0000","last_active":"2023-10-06T12:33:40.812531+0000","last_peered":"2023-10-06T12:33:40.812531+0000","last_clean":"2023-10-06T12:33:40.812531+0000","last_became_active":"2023-10-06T12:33:39.805865+0000","last_became_peered":"2023-10-06T12:33:39.805865+0000","last_unstale":"2023-10-06T12:33:40.812531+0000","last_undegraded":"2023-10-06T12:33:40.812531+0000","last_fullsized":"2023-10-06T12:33:40.812531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:59.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.270847+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:33:40.270847+0000","last_peered":"2023-10-06T12:33:40.270847+0000","last_clean":"2023-10-06T12:33:40.270847+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:33:40.270847+0000","last_undegraded":"2023-10-06T12:33:40.270847+0000","last_fullsized":"2023-10-06T12:33:40.270847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293890+0000","last_change":"2023-10-06T12:33:40.293011+0000","last_active":"2023-10-06T12:33:40.293890+0000","last_peered":"2023-10-06T12:33:40.293890+0000","last_clean":"2023-10-06T12:33:40.293890+0000","last_became_active":"2023-10-06T12:33:40.292669+0000","last_became_peered":"2023-10-06T12:33:40.292669+0000","last_unstale":"2023-10-06T12:33:40.293890+0000","last_undegraded":"2023-10-06T12:33:40.293890+0000","last_fullsized":"2023-10-06T12:33:40.293890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:46.979791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267782+0000","last_change":"2023-10-06T12:33:39.805085+0000","last_active":"2023-10-06T12:33:40.267782+0000","last_peered":"2023-10-06T12:33:40.267782+0000","last_clean":"2023-10-06T12:33:40.267782+0000","last_became_active":"2023-10-06T12:33:39.804573+0000","last_became_peered":"2023-10-06T12:33:39.804573+0000","last_unstale":"2023-10-06T12:33:40.267782+0000","last_undegraded":"2023-10-06T12:33:40.267782+0000","last_fullsized":"2023-10-06T12:33:40.267782+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274987+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:33:40.274987+0000","last_peered":"2023-10-06T12:33:40.274987+0000","last_clean":"2023-10-06T12:33:40.274987+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:33:40.274987+0000","last_undegraded":"2023-10-06T12:33:40.274987+0000","last_fullsized":"2023-10-06T12:33:40.274987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.285156+0000","last_change":"2023-10-06T12:33:40.282088+0000","last_active":"2023-10-06T12:33:40.285156+0000","last_peered":"2023-10-06T12:33:40.285156+0000","last_clean":"2023-10-06T12:33:40.285156+0000","last_became_active":"2023-10-06T12:33:40.281790+0000","last_became_peered":"2023-10-06T12:33:40.281790+0000","last_unstale":"2023-10-06T12:33:40.285156+0000","last_undegraded":"2023-10-06T12:33:40.285156+0000","last_fullsized":"2023-10-06T12:33:40.285156+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:40.112545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266012+0000","last_change":"2023-10-06T12:33:39.803873+0000","last_active":"2023-10-06T12:33:40.266012+0000","last_peered":"2023-10-06T12:33:40.266012+0000","last_clean":"2023-10-06T12:33:40.266012+0000","last_became_active":"2023-10-06T12:33:39.803599+0000","last_became_peered":"2023-10-06T12:33:39.803599+0000","last_unstale":"2023-10-06T12:33:40.266012+0000","last_undegraded":"2023-10-06T12:33:40.266012+0000","last_fullsized":"2023-10-06T12:33:40.266012+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:39.097471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287145+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:33:40.287145+0000","last_peered":"2023-10-06T12:33:40.287145+0000","last_clean":"2023-10-06T12:33:40.287145+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:33:40.287145+0000","last_undegraded":"2023-10-06T12:33:40.287145+0000","last_fullsized":"2023-10-06T12:33:40.287145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575749,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299744,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072096,"statfs":{"total":96636764160,"available":96329826304,"internally_reserved":0,"allocated":2514944,"data_stored":1592157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.155},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.423,"5min":1.6519999999999999,"15min":1.6819999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1890000000000001,"15min":1.1719999999999999},"min":{"1min":0.86899999999999999,"5min":0.73099999999999998,"15min":0.64000000000000001},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.91800000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5860000000000001,"5min":1.6200000000000001,"15min":1.7949999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":1.1040000000000001,"5min":0.92500000000000004,"15min":0.85599999999999998},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":1.855,"15min":1.855},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.262},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.65500000000000003},"max":{"1min":1.5189999999999999,"5min":1.6240000000000001,"15min":1.7649999999999999},"last":1.319}]},{"osd":3,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.2889999999999999},"min":{"1min":0.95499999999999996,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.363,"5min":1.345,"15min":1.329},"min":{"1min":1.0820000000000001,"5min":0.77500000000000002,"15min":0.70399999999999996},"max":{"1min":1.6619999999999999,"5min":1.7529999999999999,"15min":1.9019999999999999},"last":1.0629999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1120000000000001,"15min":1.107},"min":{"1min":0.77500000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.722,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.242,"15min":1.2370000000000001},"min":{"1min":0.871,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7649999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.347}]},{"osd":5,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.304,"15min":1.282},"min":{"1min":1.0409999999999999,"5min":0.626,"15min":0.49399999999999999},"max":{"1min":1.752,"5min":1.766,"15min":1.853},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.999,"5min":0.80600000000000005,"15min":0.55700000000000005},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.5800000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.389,"15min":1.3680000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.667},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.427,"15min":1.4039999999999999},"min":{"1min":1.2130000000000001,"5min":0.98399999999999999,"15min":0.54000000000000004},"max":{"1min":1.827,"5min":2.0499999999999998,"15min":2.073},"last":1.482}]}]},{"osd":6,"up_from":13,"seq":55834575749,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.171}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":1.0449999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.093},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":1.329}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.25},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.008}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.149},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.292}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.4850000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575749,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":0.88800000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.028},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":1.502}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.327},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.573},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.629}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.302}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.671}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":1000163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5800000000000001,"15min":1.8839999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.167,"15min":1.1890000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.887,"15min":2.0720000000000001},"last":1.5169999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.365},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6240000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.115},{"interface":"front","average":{"1min":1.27,"5min":1.325,"15min":1.329},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.554}]},{"osd":2,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.204,"15min":1.2390000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.506,"5min":1.625,"15min":1.9330000000000001},"last":1.002},{"interface":"front","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.619,"5min":1.861,"15min":1.861},"last":1.2569999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2370000000000001,"15min":1.2490000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5429999999999999,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":1.446},{"interface":"front","average":{"1min":1.321,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6339999999999999,"5min":1.7030000000000001,"15min":1.8340000000000001},"last":1.623}]},{"osd":5,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1919999999999999,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.56999999999999995},"max":{"1min":1.552,"5min":1.7929999999999999,"15min":1.944},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.163,"5min":1.1819999999999999,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.819},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.22,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.984},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3680000000000001,"15min":1.4019999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68799999999999994},"max":{"1min":1.754,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.415}]},{"osd":7,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.339,"15min":1.3620000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.304},{"interface":"front","average":{"1min":1.397,"5min":1.399,"15min":1.4359999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.69399999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1030000000000002},"last":1.4850000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315548,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056292,"statfs":{"total":96636764160,"available":96313643008,"internally_reserved":0,"allocated":1921024,"data_stored":1001769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.999,"15min":1.0089999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.311},"max":{"1min":1.3089999999999999,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.52500000000000002},"max":{"1min":1.258,"5min":1.6100000000000001,"15min":1.8},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.54500000000000004},"max":{"1min":1.3580000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.23}]},{"osd":2,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1439999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.56899999999999995},"max":{"1min":1.623,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.337},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.252,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.59199999999999997},"max":{"1min":1.508,"5min":1.649,"15min":2.379},"last":1.095}]},{"osd":4,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3029999999999999,"15min":1.27},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.498},"max":{"1min":1.714,"5min":1.8700000000000001,"15min":2.02},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4410000000000001,"15min":1.3979999999999999},"min":{"1min":1.026,"5min":1.026,"15min":0.72499999999999998},"max":{"1min":1.8959999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.1990000000000001,"5min":1.0740000000000001,"15min":0.76900000000000002},"max":{"1min":1.784,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4390000000000001,"15min":1.419},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.518,"15min":1.4830000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.67600000000000005},"max":{"1min":1.907,"5min":2.0270000000000001,"15min":2.0449999999999999},"last":1.532},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4510000000000001,"15min":1.3979999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7649999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.97799999999999998}]},{"osd":7,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.468},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7430000000000001,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":1.296},{"interface":"front","average":{"1min":1.625,"5min":1.5780000000000001,"15min":1.5429999999999999},"min":{"1min":1.2949999999999999,"5min":1.1679999999999999,"15min":0.80000000000000004},"max":{"1min":1.9690000000000001,"5min":2.048,"15min":2.0600000000000001},"last":1.49}]}]},{"osd":2,"up_from":13,"seq":55834575749,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.325},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.2210000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.0569999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.5389999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.381},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.446}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.478}]}]},{"osd":1,"up_from":13,"seq":55834575749,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.97899999999999998,"15min":0.96899999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.53900000000000003},"max":{"1min":1.7290000000000001,"5min":1.798,"15min":1.798},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.995,"5min":1.0329999999999999,"15min":1.028},"min":{"1min":0.82399999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.2589999999999999,"5min":1.5620000000000001,"15min":1.655},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.999,"15min":1.0009999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.52500000000000002},"max":{"1min":1.3069999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":0.871},{"interface":"front","average":{"1min":1.173,"5min":1.151,"15min":1.1180000000000001},"min":{"1min":0.94199999999999995,"5min":0.77400000000000002,"15min":0.60099999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.26,"15min":1.2370000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.61599999999999999},"max":{"1min":1.673,"5min":2.008,"15min":2.008},"last":1.649},{"interface":"front","average":{"1min":1.179,"5min":1.1579999999999999,"15min":1.1479999999999999},"min":{"1min":0.89900000000000002,"5min":0.77100000000000002,"15min":0.58199999999999996},"max":{"1min":1.4830000000000001,"5min":1.625,"15min":1.806},"last":1.054}]},{"osd":4,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.274,"15min":1.268},"min":{"1min":1.0569999999999999,"5min":0.96599999999999997,"15min":0.88},"max":{"1min":1.5629999999999999,"5min":1.764,"15min":2.2490000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4730000000000001,"15min":1.4359999999999999},"min":{"1min":1.319,"5min":1.0449999999999999,"15min":1.014},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.415}]},{"osd":5,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.403,"15min":1.3859999999999999},"min":{"1min":1.1220000000000001,"5min":1.0349999999999999,"15min":0.80900000000000005},"max":{"1min":1.704,"5min":1.9550000000000001,"15min":2.0350000000000001},"last":1.569},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.3879999999999999,"15min":1.379},"min":{"1min":1.0600000000000001,"5min":0.97599999999999998,"15min":0.93400000000000005},"max":{"1min":1.696,"5min":1.917,"15min":2.0619999999999998},"last":1.609}]},{"osd":6,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.554,"15min":1.5109999999999999},"min":{"1min":1.2509999999999999,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.2360000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.4950000000000001,"15min":1.4710000000000001},"min":{"1min":1.236,"5min":1.129,"15min":1.0569999999999999},"max":{"1min":1.865,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.7609999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:32:47 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.601,"15min":1.5660000000000001},"min":{"1min":1.3140000000000001,"5min":1.282,"15min":0.99299999999999999},"max":{"1min":1.915,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":1.679},{"interface":"front","average":{"1min":1.575,"5min":1.5149999999999999,"15min":1.48},"min":{"1min":1.383,"5min":1.157,"15min":0.95799999999999996},"max":{"1min":1.847,"5min":2.0369999999999999,"15min":2.1819999999999999},"last":1.712}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299736,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072104,"statfs":{"total":96636764160,"available":96329834496,"internally_reserved":0,"allocated":2506752,"data_stored":1590587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.77200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.492},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.008,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.59799999999999998},"max":{"1min":1.4339999999999999,"5min":1.5549999999999999,"15min":1.595},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.78000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3340000000000001,"5min":1.5469999999999999,"15min":1.746},"last":1.26},{"interface":"front","average":{"1min":1.121,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.4970000000000001,"5min":1.762,"15min":1.762},"last":1.3}]},{"osd":3,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.099,"15min":1.0920000000000001},"min":{"1min":0.86199999999999999,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.202,"5min":0.84699999999999998,"15min":0.71999999999999997},"max":{"1min":1.7030000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.52}]},{"osd":4,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.371,"15min":1.369},"min":{"1min":1.1819999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.663,"5min":1.71,"15min":1.756},"last":1.204},{"interface":"front","average":{"1min":1.417,"5min":1.3340000000000001,"15min":1.3320000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.80500000000000005},"max":{"1min":1.704,"5min":1.7290000000000001,"15min":2.036},"last":1.5900000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.407,"15min":1.401},"min":{"1min":1.117,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9330000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.641},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4239999999999999,"15min":1.423},"min":{"1min":1.2350000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.865,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.6719999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.347,"15min":1.357},"min":{"1min":1.1879999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.649,"5min":1.732,"15min":2.2029999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.478,"15min":1.48},"min":{"1min":1.202,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.919,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.734}]},{"osd":7,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5029999999999999,"15min":1.482},"min":{"1min":1.2490000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8100000000000001,"5min":2.0099999999999998,"15min":2.1019999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4490000000000001},"min":{"1min":1.087,"5min":0.98599999999999999,"15min":0.95099999999999996},"max":{"1min":1.8480000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.7529999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:33:49.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.024+0000 7fcc7ed63700 1 Processor -- start 2023-10-06T12:33:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.024+0000 7fcc7ed63700 1 -- start start 2023-10-06T12:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7ed63700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcc78067760 0x7fcc780c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7ed63700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc78069660 con 0x7fcc780bf1a0 2023-10-06T12:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7ed63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc78068ac0 con 0x7fcc780bedd0 2023-10-06T12:33:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7ed63700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc78068d30 con 0x7fcc78067760 2023-10-06T12:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7e562700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcc780bf1a0 0x7fcc78161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56676/0 (socket says 172.21.15.46:56676) 2023-10-06T12:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7e562700 1 -- 172.21.15.46:0/3998750901 learned_addr learned my addr 172.21.15.46:0/3998750901 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7d560700 1 --2- 172.21.15.46:0/3998750901 >> v2:172.21.15.132:3300/0 conn(0x7fcc78067760 0x7fcc780c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205953540 0 0) 0x7fcc78069660 con 0x7fcc780bf1a0 2023-10-06T12:33:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc64003570 con 0x7fcc780bf1a0 2023-10-06T12:33:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216498680 0 0) 0x7fcc78068ac0 con 0x7fcc780bedd0 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc78069660 con 0x7fcc780bedd0 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2237462927 0 0) 0x7fcc64003570 con 0x7fcc780bf1a0 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 >> v2:172.21.15.132:3300/0 conn(0x7fcc78067760 msgr2=0x7fcc780c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 --2- 172.21.15.46:0/3998750901 >> v2:172.21.15.132:3300/0 conn(0x7fcc78067760 0x7fcc780c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 >> v1:172.21.15.132:6789/0 conn(0x7fcc780bedd0 legacy=0x7fcc78067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78069010 con 0x7fcc780bf1a0 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fcc700017f0 con 0x7fcc780bf1a0 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 >> v1:172.21.15.46:6789/0 conn(0x7fcc780bf1a0 legacy=0x7fcc78161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64003bf0 0x7fcc64003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 0x7fcc64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640051c0 0x7fcc6400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78068d30 con 0x7fcc64004810 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6400c0b0 con 0x7fcc640051c0 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.028+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6400c270 con 0x7fcc64003bf0 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 0x7fcc64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7e562700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640051c0 0x7fcc6400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7dd61700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64003bf0 0x7fcc64003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 -- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64003bf0 msgr2=0x7fcc64003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64003bf0 0x7fcc64003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 -- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640051c0 msgr2=0x7fcc6400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640051c0 0x7fcc6400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 -- 172.21.15.46:0/3998750901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64003570 con 0x7fcc64004810 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7e562700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640051c0 0x7fcc6400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7e562700 1 -- 172.21.15.46:0/3998750901 reap_dead start 2023-10-06T12:33:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7d560700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 0x7fcc64004c80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009c00 tx=0x7fcc680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80832d0045f4515d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800e710 con 0x7fcc64004810 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6800e870 con 0x7fcc64004810 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7cd5f700 1 -- 172.21.15.46:0/3998750901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800ec30 con 0x7fcc64004810 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 -- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 msgr2=0x7fcc64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 0x7fcc64004c80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009c00 tx=0x7fcc680091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 -- 172.21.15.46:0/3998750901 shutdown_connections 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/3998750901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc64004810 0x7fcc64004c80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 -- 172.21.15.46:0/3998750901 >> 172.21.15.46:0/3998750901 conn(0x7fcc78062ef0 msgr2=0x7fcc780cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 -- 172.21.15.46:0/3998750901 shutdown_connections 2023-10-06T12:33:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.032+0000 7fcc7ed63700 1 -- 172.21.15.46:0/3998750901 wait complete. 2023-10-06T12:33:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.036+0000 7fcc7ed63700 1 Processor -- start 2023-10-06T12:33:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.036+0000 7fcc7ed63700 1 -- start start 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.036+0000 7fcc7ed63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 0x7fcc78155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.036+0000 7fcc7ed63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.036+0000 7fcc7ed63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc78170eb0 0x7fcc78171300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7ed63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78171da0 con 0x7fcc780b1050 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7ed63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7815ef80 con 0x7fcc78170eb0 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7ed63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78171840 con 0x7fcc64004810 2023-10-06T12:33:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38678/0 (socket says 172.21.15.46:38678) 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 -- 172.21.15.46:0/798286673 learned_addr learned my addr 172.21.15.46:0/798286673 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7dd61700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 0x7fcc78155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7e562700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc78170eb0 0x7fcc78171300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 -- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 msgr2=0x7fcc78155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 0x7fcc78155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 -- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc78170eb0 msgr2=0x7fcc78171300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc78170eb0 0x7fcc78171300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 -- 172.21.15.46:0/798286673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780af610 con 0x7fcc780b1050 2023-10-06T12:33:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7dd61700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 0x7fcc78155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7d560700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800e4b0 tx=0x7fcc6801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68022070 con 0x7fcc780b1050 2023-10-06T12:33:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6800e9e0 con 0x7fcc780b1050 2023-10-06T12:33:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc780af8f0 con 0x7fcc780b1050 2023-10-06T12:33:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6801c450 con 0x7fcc780b1050 2023-10-06T12:33:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.040+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc780af310 con 0x7fcc780b1050 2023-10-06T12:33:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc780afb30 con 0x7fcc780b1050 2023-10-06T12:33:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fcc6801c860 con 0x7fcc780b1050 2023-10-06T12:33:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc6dffb700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 0x7fcc5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fcc6807e510 con 0x7fcc780b1050 2023-10-06T12:33:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc7dd61700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 0x7fcc5c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.044+0000 7fcc7dd61700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 0x7fcc5c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc74003aa0 tx=0x7fcc74009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.052+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc780afb30 con 0x7fcc780b1050 2023-10-06T12:33:50.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.216+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc7815ea70 con 0x7fcc5c060670 2023-10-06T12:33:50.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.232+0000 7fcc6dffb700 1 -- 172.21.15.46:0/798286673 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308612 (secure 0 0 0) 0x7fcc7815ea70 con 0x7fcc5c060670 2023-10-06T12:33:50.240 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 msgr2=0x7fcc5c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 0x7fcc5c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc74003aa0 tx=0x7fcc74009040 comp rx=0 tx=0).stop 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 msgr2=0x7fcc780b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800e4b0 tx=0x7fcc6801a000 comp rx=0 tx=0).stop 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 shutdown_connections 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcc5c060670 0x7fcc5c062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc78170eb0 0x7fcc78171300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc780b1050 0x7fcc780b14c0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 --2- 172.21.15.46:0/798286673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc64004810 0x7fcc78155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 >> 172.21.15.46:0/798286673 conn(0x7fcc78062ef0 msgr2=0x7fcc780ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 shutdown_connections 2023-10-06T12:33:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.244+0000 7fcc7ed63700 1 -- 172.21.15.46:0/798286673 wait complete. 2023-10-06T12:33:50.249 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:33:50.268 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T12:33:49.487963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":484,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459944,"kb_used_data":16104,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752514776,"statfs":{"total":773094113280,"available":770575130624,"internally_reserved":0,"allocated":16490496,"data_stored":9186652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.230590"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280402+0000","last_change":"2023-10-06T12:33:40.278094+0000","last_active":"2023-10-06T12:33:40.280402+0000","last_peered":"2023-10-06T12:33:40.280402+0000","last_clean":"2023-10-06T12:33:40.280402+0000","last_became_active":"2023-10-06T12:33:40.277766+0000","last_became_peered":"2023-10-06T12:33:40.277766+0000","last_unstale":"2023-10-06T12:33:40.280402+0000","last_undegraded":"2023-10-06T12:33:40.280402+0000","last_fullsized":"2023-10-06T12:33:40.280402+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:50:40.996759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278844+0000","last_change":"2023-10-06T12:33:40.276811+0000","last_active":"2023-10-06T12:33:40.278844+0000","last_peered":"2023-10-06T12:33:40.278844+0000","last_clean":"2023-10-06T12:33:40.278844+0000","last_became_active":"2023-10-06T12:33:40.276490+0000","last_became_peered":"2023-10-06T12:33:40.276490+0000","last_unstale":"2023-10-06T12:33:40.278844+0000","last_undegraded":"2023-10-06T12:33:40.278844+0000","last_fullsized":"2023-10-06T12:33:40.278844+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293228+0000","last_change":"2023-10-06T12:33:40.289198+0000","last_active":"2023-10-06T12:33:40.293228+0000","last_peered":"2023-10-06T12:33:40.293228+0000","last_clean":"2023-10-06T12:33:40.293228+0000","last_became_active":"2023-10-06T12:33:40.288858+0000","last_became_peered":"2023-10-06T12:33:40.288858+0000","last_unstale":"2023-10-06T12:33:40.293228+0000","last_undegraded":"2023-10-06T12:33:40.293228+0000","last_fullsized":"2023-10-06T12:33:40.293228+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.278895+0000","last_change":"2023-10-06T12:33:40.274309+0000","last_active":"2023-10-06T12:33:40.278895+0000","last_peered":"2023-10-06T12:33:40.278895+0000","last_clean":"2023-10-06T12:33:40.278895+0000","last_became_active":"2023-10-06T12:33:40.274015+0000","last_became_peered":"2023-10-06T12:33:40.274015+0000","last_unstale":"2023-10-06T12:33:40.278895+0000","last_undegraded":"2023-10-06T12:33:40.278895+0000","last_fullsized":"2023-10-06T12:33:40.278895+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287401+0000","last_change":"2023-10-06T12:33:40.285006+0000","last_active":"2023-10-06T12:33:40.287401+0000","last_peered":"2023-10-06T12:33:40.287401+0000","last_clean":"2023-10-06T12:33:40.287401+0000","last_became_active":"2023-10-06T12:33:40.284722+0000","last_became_peered":"2023-10-06T12:33:40.284722+0000","last_unstale":"2023-10-06T12:33:40.287401+0000","last_undegraded":"2023-10-06T12:33:40.287401+0000","last_fullsized":"2023-10-06T12:33:40.287401+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293763+0000","last_change":"2023-10-06T12:33:40.292772+0000","last_active":"2023-10-06T12:33:40.293763+0000","last_peered":"2023-10-06T12:33:40.293763+0000","last_clean":"2023-10-06T12:33:40.293763+0000","last_became_active":"2023-10-06T12:33:40.292458+0000","last_became_peered":"2023-10-06T12:33:40.292458+0000","last_unstale":"2023-10-06T12:33:40.293763+0000","last_undegraded":"2023-10-06T12:33:40.293763+0000","last_fullsized":"2023-10-06T12:33:40.293763+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:47.841304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292102+0000","last_change":"2023-10-06T12:33:40.286279+0000","last_active":"2023-10-06T12:33:40.292102+0000","last_peered":"2023-10-06T12:33:40.292102+0000","last_clean":"2023-10-06T12:33:40.292102+0000","last_became_active":"2023-10-06T12:33:40.285975+0000","last_became_peered":"2023-10-06T12:33:40.285975+0000","last_unstale":"2023-10-06T12:33:40.292102+0000","last_undegraded":"2023-10-06T12:33:40.292102+0000","last_fullsized":"2023-10-06T12:33:40.292102+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284738+0000","last_change":"2023-10-06T12:33:40.274745+0000","last_active":"2023-10-06T12:33:40.284738+0000","last_peered":"2023-10-06T12:33:40.284738+0000","last_clean":"2023-10-06T12:33:40.284738+0000","last_became_active":"2023-10-06T12:33:40.274253+0000","last_became_peered":"2023-10-06T12:33:40.274253+0000","last_unstale":"2023-10-06T12:33:40.284738+0000","last_undegraded":"2023-10-06T12:33:40.284738+0000","last_fullsized":"2023-10-06T12:33:40.284738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:42:37.726541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281509+0000","last_change":"2023-10-06T12:33:40.277693+0000","last_active":"2023-10-06T12:33:40.281509+0000","last_peered":"2023-10-06T12:33:40.281509+0000","last_clean":"2023-10-06T12:33:40.281509+0000","last_became_active":"2023-10-06T12:33:40.277394+0000","last_became_peered":"2023-10-06T12:33:40.277394+0000","last_unstale":"2023-10-06T12:33:40.281509+0000","last_undegraded":"2023-10-06T12:33:40.281509+0000","last_fullsized":"2023-10-06T12:33:40.281509+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:34.307339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273435+0000","last_change":"2023-10-06T12:33:40.269594+0000","last_active":"2023-10-06T12:33:40.273435+0000","last_peered":"2023-10-06T12:33:40.273435+0000","last_clean":"2023-10-06T12:33:40.273435+0000","last_became_active":"2023-10-06T12:33:40.269292+0000","last_became_peered":"2023-10-06T12:33:40.269292+0000","last_unstale":"2023-10-06T12:33:40.273435+0000","last_undegraded":"2023-10-06T12:33:40.273435+0000","last_fullsized":"2023-10-06T12:33:40.273435+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:48.712468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.290272+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289949+0000","last_became_peered":"2023-10-06T12:33:40.289949+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281966+0000","last_change":"2023-10-06T12:33:40.278884+0000","last_active":"2023-10-06T12:33:40.281966+0000","last_peered":"2023-10-06T12:33:40.281966+0000","last_clean":"2023-10-06T12:33:40.281966+0000","last_became_active":"2023-10-06T12:33:40.278547+0000","last_became_peered":"2023-10-06T12:33:40.278547+0000","last_unstale":"2023-10-06T12:33:40.281966+0000","last_undegraded":"2023-10-06T12:33:40.281966+0000","last_fullsized":"2023-10-06T12:33:40.281966+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291124+0000","last_change":"2023-10-06T12:33:40.290706+0000","last_active":"2023-10-06T12:33:40.291124+0000","last_peered":"2023-10-06T12:33:40.291124+0000","last_clean":"2023-10-06T12:33:40.291124+0000","last_became_active":"2023-10-06T12:33:40.290189+0000","last_became_peered":"2023-10-06T12:33:40.290189+0000","last_unstale":"2023-10-06T12:33:40.291124+0000","last_undegraded":"2023-10-06T12:33:40.291124+0000","last_fullsized":"2023-10-06T12:33:40.291124+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:31.802812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287127+0000","last_change":"2023-10-06T12:33:40.285957+0000","last_active":"2023-10-06T12:33:40.287127+0000","last_peered":"2023-10-06T12:33:40.287127+0000","last_clean":"2023-10-06T12:33:40.287127+0000","last_became_active":"2023-10-06T12:33:40.285663+0000","last_became_peered":"2023-10-06T12:33:40.285663+0000","last_unstale":"2023-10-06T12:33:40.287127+0000","last_undegraded":"2023-10-06T12:33:40.287127+0000","last_fullsized":"2023-10-06T12:33:40.287127+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287455+0000","last_change":"2023-10-06T12:33:40.286241+0000","last_active":"2023-10-06T12:33:40.287455+0000","last_peered":"2023-10-06T12:33:40.287455+0000","last_clean":"2023-10-06T12:33:40.287455+0000","last_became_active":"2023-10-06T12:33:40.285913+0000","last_became_peered":"2023-10-06T12:33:40.285913+0000","last_unstale":"2023-10-06T12:33:40.287455+0000","last_undegraded":"2023-10-06T12:33:40.287455+0000","last_fullsized":"2023-10-06T12:33:40.287455+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282791+0000","last_change":"2023-10-06T12:33:40.281557+0000","last_active":"2023-10-06T12:33:40.282791+0000","last_peered":"2023-10-06T12:33:40.282791+0000","last_clean":"2023-10-06T12:33:40.282791+0000","last_became_active":"2023-10-06T12:33:40.281272+0000","last_became_peered":"2023-10-06T12:33:40.281272+0000","last_unstale":"2023-10-06T12:33:40.282791+0000","last_undegraded":"2023-10-06T12:33:40.282791+0000","last_fullsized":"2023-10-06T12:33:40.282791+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:25.331270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.282795+0000","last_change":"2023-10-06T12:33:40.279525+0000","last_active":"2023-10-06T12:33:40.282795+0000","last_peered":"2023-10-06T12:33:40.282795+0000","last_clean":"2023-10-06T12:33:40.282795+0000","last_became_active":"2023-10-06T12:33:40.279160+0000","last_became_peered":"2023-10-06T12:33:40.279160+0000","last_unstale":"2023-10-06T12:33:40.282795+0000","last_undegraded":"2023-10-06T12:33:40.282795+0000","last_fullsized":"2023-10-06T12:33:40.282795+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:32:56.644708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287429+0000","last_change":"2023-10-06T12:33:40.284511+0000","last_active":"2023-10-06T12:33:40.287429+0000","last_peered":"2023-10-06T12:33:40.287429+0000","last_clean":"2023-10-06T12:33:40.287429+0000","last_became_active":"2023-10-06T12:33:40.284083+0000","last_became_peered":"2023-10-06T12:33:40.284083+0000","last_unstale":"2023-10-06T12:33:40.287429+0000","last_undegraded":"2023-10-06T12:33:40.287429+0000","last_fullsized":"2023-10-06T12:33:40.287429+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.292384+0000","last_change":"2023-10-06T12:33:40.286091+0000","last_active":"2023-10-06T12:33:40.292384+0000","last_peered":"2023-10-06T12:33:40.292384+0000","last_clean":"2023-10-06T12:33:40.292384+0000","last_became_active":"2023-10-06T12:33:40.285783+0000","last_became_peered":"2023-10-06T12:33:40.285783+0000","last_unstale":"2023-10-06T12:33:40.292384+0000","last_undegraded":"2023-10-06T12:33:40.292384+0000","last_fullsized":"2023-10-06T12:33:40.292384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:35.732375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286060+0000","last_change":"2023-10-06T12:33:40.283403+0000","last_active":"2023-10-06T12:33:40.286060+0000","last_peered":"2023-10-06T12:33:40.286060+0000","last_clean":"2023-10-06T12:33:40.286060+0000","last_became_active":"2023-10-06T12:33:40.283010+0000","last_became_peered":"2023-10-06T12:33:40.283010+0000","last_unstale":"2023-10-06T12:33:40.286060+0000","last_undegraded":"2023-10-06T12:33:40.286060+0000","last_fullsized":"2023-10-06T12:33:40.286060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.288358+0000","last_change":"2023-10-06T12:33:40.285829+0000","last_active":"2023-10-06T12:33:40.288358+0000","last_peered":"2023-10-06T12:33:40.288358+0000","last_clean":"2023-10-06T12:33:40.288358+0000","last_became_active":"2023-10-06T12:33:40.284389+0000","last_became_peered":"2023-10-06T12:33:40.284389+0000","last_unstale":"2023-10-06T12:33:40.288358+0000","last_undegraded":"2023-10-06T12:33:40.288358+0000","last_fullsized":"2023-10-06T12:33:40.288358+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281945+0000","last_change":"2023-10-06T12:33:40.280118+0000","last_active":"2023-10-06T12:33:40.281945+0000","last_peered":"2023-10-06T12:33:40.281945+0000","last_clean":"2023-10-06T12:33:40.281945+0000","last_became_active":"2023-10-06T12:33:40.279604+0000","last_became_peered":"2023-10-06T12:33:40.279604+0000","last_unstale":"2023-10-06T12:33:40.281945+0000","last_undegraded":"2023-10-06T12:33:40.281945+0000","last_fullsized":"2023-10-06T12:33:40.281945+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:11:13.472429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283037+0000","last_change":"2023-10-06T12:33:40.280532+0000","last_active":"2023-10-06T12:33:40.283037+0000","last_peered":"2023-10-06T12:33:40.283037+0000","last_clean":"2023-10-06T12:33:40.283037+0000","last_became_active":"2023-10-06T12:33:40.280248+0000","last_became_peered":"2023-10-06T12:33:40.280248+0000","last_unstale":"2023-10-06T12:33:40.283037+0000","last_undegraded":"2023-10-06T12:33:40.283037+0000","last_fullsized":"2023-10-06T12:33:40.283037+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281656+0000","last_change":"2023-10-06T12:33:40.277730+0000","last_active":"2023-10-06T12:33:40.281656+0000","last_peered":"2023-10-06T12:33:40.281656+0000","last_clean":"2023-10-06T12:33:40.281656+0000","last_became_active":"2023-10-06T12:33:40.277481+0000","last_became_peered":"2023-10-06T12:33:40.277481+0000","last_unstale":"2023-10-06T12:33:40.281656+0000","last_undegraded":"2023-10-06T12:33:40.281656+0000","last_fullsized":"2023-10-06T12:33:40.281656+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:55.533377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295166+0000","last_change":"2023-10-06T12:33:40.290594+0000","last_active":"2023-10-06T12:33:40.295166+0000","last_peered":"2023-10-06T12:33:40.295166+0000","last_clean":"2023-10-06T12:33:40.295166+0000","last_became_active":"2023-10-06T12:33:40.290274+0000","last_became_peered":"2023-10-06T12:33:40.290274+0000","last_unstale":"2023-10-06T12:33:40.295166+0000","last_undegraded":"2023-10-06T12:33:40.295166+0000","last_fullsized":"2023-10-06T12:33:40.295166+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:26.400947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293038+0000","last_change":"2023-10-06T12:33:40.288927+0000","last_active":"2023-10-06T12:33:40.293038+0000","last_peered":"2023-10-06T12:33:40.293038+0000","last_clean":"2023-10-06T12:33:40.293038+0000","last_became_active":"2023-10-06T12:33:40.288580+0000","last_became_peered":"2023-10-06T12:33:40.288580+0000","last_unstale":"2023-10-06T12:33:40.293038+0000","last_undegraded":"2023-10-06T12:33:40.293038+0000","last_fullsized":"2023-10-06T12:33:40.293038+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:34.252418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291369+0000","last_change":"2023-10-06T12:33:40.282511+0000","last_active":"2023-10-06T12:33:40.291369+0000","last_peered":"2023-10-06T12:33:40.291369+0000","last_clean":"2023-10-06T12:33:40.291369+0000","last_became_active":"2023-10-06T12:33:40.282205+0000","last_became_peered":"2023-10-06T12:33:40.282205+0000","last_unstale":"2023-10-06T12:33:40.291369+0000","last_undegraded":"2023-10-06T12:33:40.291369+0000","last_fullsized":"2023-10-06T12:33:40.291369+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:23.076341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273907+0000","last_change":"2023-10-06T12:33:40.271338+0000","last_active":"2023-10-06T12:33:40.273907+0000","last_peered":"2023-10-06T12:33:40.273907+0000","last_clean":"2023-10-06T12:33:40.273907+0000","last_became_active":"2023-10-06T12:33:40.271069+0000","last_became_peered":"2023-10-06T12:33:40.271069+0000","last_unstale":"2023-10-06T12:33:40.273907+0000","last_undegraded":"2023-10-06T12:33:40.273907+0000","last_fullsized":"2023-10-06T12:33:40.273907+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:37.758266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294979+0000","last_change":"2023-10-06T12:33:40.290057+0000","last_active":"2023-10-06T12:33:40.294979+0000","last_peered":"2023-10-06T12:33:40.294979+0000","last_clean":"2023-10-06T12:33:40.294979+0000","last_became_active":"2023-10-06T12:33:40.288561+0000","last_became_peered":"2023-10-06T12:33:40.288561+0000","last_unstale":"2023-10-06T12:33:40.294979+0000","last_undegraded":"2023-10-06T12:33:40.294979+0000","last_fullsized":"2023-10-06T12:33:40.294979+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:57.704963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296209+0000","last_change":"2023-10-06T12:33:40.292961+0000","last_active":"2023-10-06T12:33:40.296209+0000","last_peered":"2023-10-06T12:33:40.296209+0000","last_clean":"2023-10-06T12:33:40.296209+0000","last_became_active":"2023-10-06T12:33:40.292379+0000","last_became_peered":"2023-10-06T12:33:40.292379+0000","last_unstale":"2023-10-06T12:33:40.296209+0000","last_undegraded":"2023-10-06T12:33:40.296209+0000","last_fullsized":"2023-10-06T12:33:40.296209+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:03.774409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289262+0000","last_change":"2023-10-06T12:33:40.284299+0000","last_active":"2023-10-06T12:33:40.289262+0000","last_peered":"2023-10-06T12:33:40.289262+0000","last_clean":"2023-10-06T12:33:40.289262+0000","last_became_active":"2023-10-06T12:33:40.284014+0000","last_became_peered":"2023-10-06T12:33:40.284014+0000","last_unstale":"2023-10-06T12:33:40.289262+0000","last_undegraded":"2023-10-06T12:33:40.289262+0000","last_fullsized":"2023-10-06T12:33:40.289262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281701+0000","last_change":"2023-10-06T12:33:40.274503+0000","last_active":"2023-10-06T12:33:40.281701+0000","last_peered":"2023-10-06T12:33:40.281701+0000","last_clean":"2023-10-06T12:33:40.281701+0000","last_became_active":"2023-10-06T12:33:40.274196+0000","last_became_peered":"2023-10-06T12:33:40.274196+0000","last_unstale":"2023-10-06T12:33:40.281701+0000","last_undegraded":"2023-10-06T12:33:40.281701+0000","last_fullsized":"2023-10-06T12:33:40.281701+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:20.140551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289181+0000","last_change":"2023-10-06T12:33:40.277982+0000","last_active":"2023-10-06T12:33:40.289181+0000","last_peered":"2023-10-06T12:33:40.289181+0000","last_clean":"2023-10-06T12:33:40.289181+0000","last_became_active":"2023-10-06T12:33:40.277677+0000","last_became_peered":"2023-10-06T12:33:40.277677+0000","last_unstale":"2023-10-06T12:33:40.289181+0000","last_undegraded":"2023-10-06T12:33:40.289181+0000","last_fullsized":"2023-10-06T12:33:40.289181+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"152'4","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811674+0000","last_change":"2023-10-06T12:33:39.805457+0000","last_active":"2023-10-06T12:33:40.811674+0000","last_peered":"2023-10-06T12:33:40.811674+0000","last_clean":"2023-10-06T12:33:40.811674+0000","last_became_active":"2023-10-06T12:33:39.805053+0000","last_became_peered":"2023-10-06T12:33:39.805053+0000","last_unstale":"2023-10-06T12:33:40.811674+0000","last_undegraded":"2023-10-06T12:33:40.811674+0000","last_fullsized":"2023-10-06T12:33:40.811674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281933+0000","last_change":"2023-10-06T12:33:40.279910+0000","last_active":"2023-10-06T12:33:40.281933+0000","last_peered":"2023-10-06T12:33:40.281933+0000","last_clean":"2023-10-06T12:33:40.281933+0000","last_became_active":"2023-10-06T12:33:40.279582+0000","last_became_peered":"2023-10-06T12:33:40.279582+0000","last_unstale":"2023-10-06T12:33:40.281933+0000","last_undegraded":"2023-10-06T12:33:40.281933+0000","last_fullsized":"2023-10-06T12:33:40.281933+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:55:58.370893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808163+0000","last_change":"2023-10-06T12:33:39.807053+0000","last_active":"2023-10-06T12:33:40.808163+0000","last_peered":"2023-10-06T12:33:40.808163+0000","last_clean":"2023-10-06T12:33:40.808163+0000","last_became_active":"2023-10-06T12:33:39.806637+0000","last_became_peered":"2023-10-06T12:33:39.806637+0000","last_unstale":"2023-10-06T12:33:40.808163+0000","last_undegraded":"2023-10-06T12:33:40.808163+0000","last_fullsized":"2023-10-06T12:33:40.808163+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280827+0000","last_change":"2023-10-06T12:33:40.279263+0000","last_active":"2023-10-06T12:33:40.280827+0000","last_peered":"2023-10-06T12:33:40.280827+0000","last_clean":"2023-10-06T12:33:40.280827+0000","last_became_active":"2023-10-06T12:33:40.278933+0000","last_became_peered":"2023-10-06T12:33:40.278933+0000","last_unstale":"2023-10-06T12:33:40.280827+0000","last_undegraded":"2023-10-06T12:33:40.280827+0000","last_fullsized":"2023-10-06T12:33:40.280827+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:38.397513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.810492+0000","last_change":"2023-10-06T12:33:39.816889+0000","last_active":"2023-10-06T12:33:40.810492+0000","last_peered":"2023-10-06T12:33:40.810492+0000","last_clean":"2023-10-06T12:33:40.810492+0000","last_became_active":"2023-10-06T12:33:39.816379+0000","last_became_peered":"2023-10-06T12:33:39.816379+0000","last_unstale":"2023-10-06T12:33:40.810492+0000","last_undegraded":"2023-10-06T12:33:40.810492+0000","last_fullsized":"2023-10-06T12:33:40.810492+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:33.237343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286357+0000","last_change":"2023-10-06T12:33:40.285187+0000","last_active":"2023-10-06T12:33:40.286357+0000","last_peered":"2023-10-06T12:33:40.286357+0000","last_clean":"2023-10-06T12:33:40.286357+0000","last_became_active":"2023-10-06T12:33:40.284897+0000","last_became_peered":"2023-10-06T12:33:40.284897+0000","last_unstale":"2023-10-06T12:33:40.286357+0000","last_undegraded":"2023-10-06T12:33:40.286357+0000","last_fullsized":"2023-10-06T12:33:40.286357+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811184+0000","last_change":"2023-10-06T12:33:39.812575+0000","last_active":"2023-10-06T12:33:40.811184+0000","last_peered":"2023-10-06T12:33:40.811184+0000","last_clean":"2023-10-06T12:33:40.811184+0000","last_became_active":"2023-10-06T12:33:39.812156+0000","last_became_peered":"2023-10-06T12:33:39.812156+0000","last_unstale":"2023-10-06T12:33:40.811184+0000","last_undegraded":"2023-10-06T12:33:40.811184+0000","last_fullsized":"2023-10-06T12:33:40.811184+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:32.614790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289164+0000","last_change":"2023-10-06T12:33:40.276822+0000","last_active":"2023-10-06T12:33:40.289164+0000","last_peered":"2023-10-06T12:33:40.289164+0000","last_clean":"2023-10-06T12:33:40.289164+0000","last_became_active":"2023-10-06T12:33:40.276482+0000","last_became_peered":"2023-10-06T12:33:40.276482+0000","last_unstale":"2023-10-06T12:33:40.289164+0000","last_undegraded":"2023-10-06T12:33:40.289164+0000","last_fullsized":"2023-10-06T12:33:40.289164+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809703+0000","last_change":"2023-10-06T12:33:39.814112+0000","last_active":"2023-10-06T12:33:40.809703+0000","last_peered":"2023-10-06T12:33:40.809703+0000","last_clean":"2023-10-06T12:33:40.809703+0000","last_became_active":"2023-10-06T12:33:39.813197+0000","last_became_peered":"2023-10-06T12:33:39.813197+0000","last_unstale":"2023-10-06T12:33:40.809703+0000","last_undegraded":"2023-10-06T12:33:40.809703+0000","last_fullsized":"2023-10-06T12:33:40.809703+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:02.759335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.284735+0000","last_change":"2023-10-06T12:33:40.272534+0000","last_active":"2023-10-06T12:33:40.284735+0000","last_peered":"2023-10-06T12:33:40.284735+0000","last_clean":"2023-10-06T12:33:40.284735+0000","last_became_active":"2023-10-06T12:33:40.272184+0000","last_became_peered":"2023-10-06T12:33:40.272184+0000","last_unstale":"2023-10-06T12:33:40.284735+0000","last_undegraded":"2023-10-06T12:33:40.284735+0000","last_fullsized":"2023-10-06T12:33:40.284735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:05.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809086+0000","last_change":"2023-10-06T12:33:39.812707+0000","last_active":"2023-10-06T12:33:40.809086+0000","last_peered":"2023-10-06T12:33:40.809086+0000","last_clean":"2023-10-06T12:33:40.809086+0000","last_became_active":"2023-10-06T12:33:39.812406+0000","last_became_peered":"2023-10-06T12:33:39.812406+0000","last_unstale":"2023-10-06T12:33:40.809086+0000","last_undegraded":"2023-10-06T12:33:40.809086+0000","last_fullsized":"2023-10-06T12:33:40.809086+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286027+0000","last_change":"2023-10-06T12:33:40.283254+0000","last_active":"2023-10-06T12:33:40.286027+0000","last_peered":"2023-10-06T12:33:40.286027+0000","last_clean":"2023-10-06T12:33:40.286027+0000","last_became_active":"2023-10-06T12:33:40.282952+0000","last_became_peered":"2023-10-06T12:33:40.282952+0000","last_unstale":"2023-10-06T12:33:40.286027+0000","last_undegraded":"2023-10-06T12:33:40.286027+0000","last_fullsized":"2023-10-06T12:33:40.286027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274181+0000","last_change":"2023-10-06T12:33:39.816556+0000","last_active":"2023-10-06T12:33:40.274181+0000","last_peered":"2023-10-06T12:33:40.274181+0000","last_clean":"2023-10-06T12:33:40.274181+0000","last_became_active":"2023-10-06T12:33:39.816213+0000","last_became_peered":"2023-10-06T12:33:39.816213+0000","last_unstale":"2023-10-06T12:33:40.274181+0000","last_undegraded":"2023-10-06T12:33:40.274181+0000","last_fullsized":"2023-10-06T12:33:40.274181+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:56.132241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290868+0000","last_change":"2023-10-06T12:33:40.283081+0000","last_active":"2023-10-06T12:33:40.290868+0000","last_peered":"2023-10-06T12:33:40.290868+0000","last_clean":"2023-10-06T12:33:40.290868+0000","last_became_active":"2023-10-06T12:33:40.282501+0000","last_became_peered":"2023-10-06T12:33:40.282501+0000","last_unstale":"2023-10-06T12:33:40.290868+0000","last_undegraded":"2023-10-06T12:33:40.290868+0000","last_fullsized":"2023-10-06T12:33:40.290868+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:57.530674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286696+0000","last_change":"2023-10-06T12:33:39.810216+0000","last_active":"2023-10-06T12:33:40.286696+0000","last_peered":"2023-10-06T12:33:40.286696+0000","last_clean":"2023-10-06T12:33:40.286696+0000","last_became_active":"2023-10-06T12:33:39.809885+0000","last_became_peered":"2023-10-06T12:33:39.809885+0000","last_unstale":"2023-10-06T12:33:40.286696+0000","last_undegraded":"2023-10-06T12:33:40.286696+0000","last_fullsized":"2023-10-06T12:33:40.286696+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:18.179915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.296114+0000","last_change":"2023-10-06T12:33:40.292823+0000","last_active":"2023-10-06T12:33:40.296114+0000","last_peered":"2023-10-06T12:33:40.296114+0000","last_clean":"2023-10-06T12:33:40.296114+0000","last_became_active":"2023-10-06T12:33:40.292457+0000","last_became_peered":"2023-10-06T12:33:40.292457+0000","last_unstale":"2023-10-06T12:33:40.296114+0000","last_undegraded":"2023-10-06T12:33:40.296114+0000","last_fullsized":"2023-10-06T12:33:40.296114+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.809539+0000","last_change":"2023-10-06T12:33:39.804873+0000","last_active":"2023-10-06T12:33:40.809539+0000","last_peered":"2023-10-06T12:33:40.809539+0000","last_clean":"2023-10-06T12:33:40.809539+0000","last_became_active":"2023-10-06T12:33:39.803841+0000","last_became_peered":"2023-10-06T12:33:39.803841+0000","last_unstale":"2023-10-06T12:33:40.809539+0000","last_undegraded":"2023-10-06T12:33:40.809539+0000","last_fullsized":"2023-10-06T12:33:40.809539+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:56.689888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291255+0000","last_change":"2023-10-06T12:33:40.284048+0000","last_active":"2023-10-06T12:33:40.291255+0000","last_peered":"2023-10-06T12:33:40.291255+0000","last_clean":"2023-10-06T12:33:40.291255+0000","last_became_active":"2023-10-06T12:33:40.283765+0000","last_became_peered":"2023-10-06T12:33:40.283765+0000","last_unstale":"2023-10-06T12:33:40.291255+0000","last_undegraded":"2023-10-06T12:33:40.291255+0000","last_fullsized":"2023-10-06T12:33:40.291255+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:10.127582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811535+0000","last_change":"2023-10-06T12:33:39.805882+0000","last_active":"2023-10-06T12:33:40.811535+0000","last_peered":"2023-10-06T12:33:40.811535+0000","last_clean":"2023-10-06T12:33:40.811535+0000","last_became_active":"2023-10-06T12:33:39.805035+0000","last_became_peered":"2023-10-06T12:33:39.805035+0000","last_unstale":"2023-10-06T12:33:40.811535+0000","last_undegraded":"2023-10-06T12:33:40.811535+0000","last_fullsized":"2023-10-06T12:33:40.811535+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290722+0000","last_change":"2023-10-06T12:33:40.282920+0000","last_active":"2023-10-06T12:33:40.290722+0000","last_peered":"2023-10-06T12:33:40.290722+0000","last_clean":"2023-10-06T12:33:40.290722+0000","last_became_active":"2023-10-06T12:33:40.282581+0000","last_became_peered":"2023-10-06T12:33:40.282581+0000","last_unstale":"2023-10-06T12:33:40.290722+0000","last_undegraded":"2023-10-06T12:33:40.290722+0000","last_fullsized":"2023-10-06T12:33:40.290722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"152'1","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.806365+0000","last_change":"2023-10-06T12:33:39.815967+0000","last_active":"2023-10-06T12:33:40.806365+0000","last_peered":"2023-10-06T12:33:40.806365+0000","last_clean":"2023-10-06T12:33:40.806365+0000","last_became_active":"2023-10-06T12:33:39.815667+0000","last_became_peered":"2023-10-06T12:33:39.815667+0000","last_unstale":"2023-10-06T12:33:40.806365+0000","last_undegraded":"2023-10-06T12:33:40.806365+0000","last_fullsized":"2023-10-06T12:33:40.806365+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:05.141371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291363+0000","last_change":"2023-10-06T12:33:40.290471+0000","last_active":"2023-10-06T12:33:40.291363+0000","last_peered":"2023-10-06T12:33:40.291363+0000","last_clean":"2023-10-06T12:33:40.291363+0000","last_became_active":"2023-10-06T12:33:40.290101+0000","last_became_peered":"2023-10-06T12:33:40.290101+0000","last_unstale":"2023-10-06T12:33:40.291363+0000","last_undegraded":"2023-10-06T12:33:40.291363+0000","last_fullsized":"2023-10-06T12:33:40.291363+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:20.204914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266996+0000","last_change":"2023-10-06T12:33:39.811123+0000","last_active":"2023-10-06T12:33:40.266996+0000","last_peered":"2023-10-06T12:33:40.266996+0000","last_clean":"2023-10-06T12:33:40.266996+0000","last_became_active":"2023-10-06T12:33:39.810818+0000","last_became_peered":"2023-10-06T12:33:39.810818+0000","last_unstale":"2023-10-06T12:33:40.266996+0000","last_undegraded":"2023-10-06T12:33:40.266996+0000","last_fullsized":"2023-10-06T12:33:40.266996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:30:33.292264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.280655+0000","last_change":"2023-10-06T12:33:40.275550+0000","last_active":"2023-10-06T12:33:40.280655+0000","last_peered":"2023-10-06T12:33:40.280655+0000","last_clean":"2023-10-06T12:33:40.280655+0000","last_became_active":"2023-10-06T12:33:40.275264+0000","last_became_peered":"2023-10-06T12:33:40.275264+0000","last_unstale":"2023-10-06T12:33:40.280655+0000","last_undegraded":"2023-10-06T12:33:40.280655+0000","last_fullsized":"2023-10-06T12:33:40.280655+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:45.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812298+0000","last_change":"2023-10-06T12:33:39.811347+0000","last_active":"2023-10-06T12:33:40.812298+0000","last_peered":"2023-10-06T12:33:40.812298+0000","last_clean":"2023-10-06T12:33:40.812298+0000","last_became_active":"2023-10-06T12:33:39.810887+0000","last_became_peered":"2023-10-06T12:33:39.810887+0000","last_unstale":"2023-10-06T12:33:40.812298+0000","last_undegraded":"2023-10-06T12:33:40.812298+0000","last_fullsized":"2023-10-06T12:33:40.812298+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286384+0000","last_change":"2023-10-06T12:33:40.285226+0000","last_active":"2023-10-06T12:33:40.286384+0000","last_peered":"2023-10-06T12:33:40.286384+0000","last_clean":"2023-10-06T12:33:40.286384+0000","last_became_active":"2023-10-06T12:33:40.284880+0000","last_became_peered":"2023-10-06T12:33:40.284880+0000","last_unstale":"2023-10-06T12:33:40.286384+0000","last_undegraded":"2023-10-06T12:33:40.286384+0000","last_fullsized":"2023-10-06T12:33:40.286384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267489+0000","last_change":"2023-10-06T12:33:39.809422+0000","last_active":"2023-10-06T12:33:40.267489+0000","last_peered":"2023-10-06T12:33:40.267489+0000","last_clean":"2023-10-06T12:33:40.267489+0000","last_became_active":"2023-10-06T12:33:39.809122+0000","last_became_peered":"2023-10-06T12:33:39.809122+0000","last_unstale":"2023-10-06T12:33:40.267489+0000","last_undegraded":"2023-10-06T12:33:40.267489+0000","last_fullsized":"2023-10-06T12:33:40.267489+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:06:30.787738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.289782+0000","last_change":"2023-10-06T12:33:40.289044+0000","last_active":"2023-10-06T12:33:40.289782+0000","last_peered":"2023-10-06T12:33:40.289782+0000","last_clean":"2023-10-06T12:33:40.289782+0000","last_became_active":"2023-10-06T12:33:40.288736+0000","last_became_peered":"2023-10-06T12:33:40.288736+0000","last_unstale":"2023-10-06T12:33:40.289782+0000","last_undegraded":"2023-10-06T12:33:40.289782+0000","last_fullsized":"2023-10-06T12:33:40.289782+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:24.514267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273368+0000","last_change":"2023-10-06T12:33:39.812514+0000","last_active":"2023-10-06T12:33:40.273368+0000","last_peered":"2023-10-06T12:33:40.273368+0000","last_clean":"2023-10-06T12:33:40.273368+0000","last_became_active":"2023-10-06T12:33:39.812148+0000","last_became_peered":"2023-10-06T12:33:39.812148+0000","last_unstale":"2023-10-06T12:33:40.273368+0000","last_undegraded":"2023-10-06T12:33:40.273368+0000","last_fullsized":"2023-10-06T12:33:40.273368+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:44.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295909+0000","last_change":"2023-10-06T12:33:40.295022+0000","last_active":"2023-10-06T12:33:40.295909+0000","last_peered":"2023-10-06T12:33:40.295909+0000","last_clean":"2023-10-06T12:33:40.295909+0000","last_became_active":"2023-10-06T12:33:40.294627+0000","last_became_peered":"2023-10-06T12:33:40.294627+0000","last_unstale":"2023-10-06T12:33:40.295909+0000","last_undegraded":"2023-10-06T12:33:40.295909+0000","last_fullsized":"2023-10-06T12:33:40.295909+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:53.382350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813958+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813379+0000","last_became_peered":"2023-10-06T12:33:39.813379+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:02.709927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287410+0000","last_change":"2023-10-06T12:33:40.285429+0000","last_active":"2023-10-06T12:33:40.287410+0000","last_peered":"2023-10-06T12:33:40.287410+0000","last_clean":"2023-10-06T12:33:40.287410+0000","last_became_active":"2023-10-06T12:33:40.285156+0000","last_became_peered":"2023-10-06T12:33:40.285156+0000","last_unstale":"2023-10-06T12:33:40.287410+0000","last_undegraded":"2023-10-06T12:33:40.287410+0000","last_fullsized":"2023-10-06T12:33:40.287410+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.811309+0000","last_change":"2023-10-06T12:33:39.807549+0000","last_active":"2023-10-06T12:33:40.811309+0000","last_peered":"2023-10-06T12:33:40.811309+0000","last_clean":"2023-10-06T12:33:40.811309+0000","last_became_active":"2023-10-06T12:33:39.807256+0000","last_became_peered":"2023-10-06T12:33:39.807256+0000","last_unstale":"2023-10-06T12:33:40.811309+0000","last_undegraded":"2023-10-06T12:33:40.811309+0000","last_fullsized":"2023-10-06T12:33:40.811309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:26.750076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290283+0000","last_change":"2023-10-06T12:33:40.286426+0000","last_active":"2023-10-06T12:33:40.290283+0000","last_peered":"2023-10-06T12:33:40.290283+0000","last_clean":"2023-10-06T12:33:40.290283+0000","last_became_active":"2023-10-06T12:33:40.282983+0000","last_became_peered":"2023-10-06T12:33:40.282983+0000","last_unstale":"2023-10-06T12:33:40.290283+0000","last_undegraded":"2023-10-06T12:33:40.290283+0000","last_fullsized":"2023-10-06T12:33:40.290283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:33:45.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"153'1","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T12:33:41.271726+0000","last_change":"2023-10-06T12:33:39.810049+0000","last_active":"2023-10-06T12:33:41.271726+0000","last_peered":"2023-10-06T12:33:41.271726+0000","last_clean":"2023-10-06T12:33:41.271726+0000","last_became_active":"2023-10-06T12:33:39.809771+0000","last_became_peered":"2023-10-06T12:33:39.809771+0000","last_unstale":"2023-10-06T12:33:41.271726+0000","last_undegraded":"2023-10-06T12:33:41.271726+0000","last_fullsized":"2023-10-06T12:33:41.271726+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.291999+0000","last_change":"2023-10-06T12:33:40.285446+0000","last_active":"2023-10-06T12:33:40.291999+0000","last_peered":"2023-10-06T12:33:40.291999+0000","last_clean":"2023-10-06T12:33:40.291999+0000","last_became_active":"2023-10-06T12:33:40.285119+0000","last_became_peered":"2023-10-06T12:33:40.285119+0000","last_unstale":"2023-10-06T12:33:40.291999+0000","last_undegraded":"2023-10-06T12:33:40.291999+0000","last_fullsized":"2023-10-06T12:33:40.291999+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:53.256056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.264857+0000","last_change":"2023-10-06T12:33:39.811717+0000","last_active":"2023-10-06T12:33:40.264857+0000","last_peered":"2023-10-06T12:33:40.264857+0000","last_clean":"2023-10-06T12:33:40.264857+0000","last_became_active":"2023-10-06T12:33:39.811418+0000","last_became_peered":"2023-10-06T12:33:39.811418+0000","last_unstale":"2023-10-06T12:33:40.264857+0000","last_undegraded":"2023-10-06T12:33:40.264857+0000","last_fullsized":"2023-10-06T12:33:40.264857+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.273738+0000","last_change":"2023-10-06T12:33:40.270916+0000","last_active":"2023-10-06T12:33:40.273738+0000","last_peered":"2023-10-06T12:33:40.273738+0000","last_clean":"2023-10-06T12:33:40.273738+0000","last_became_active":"2023-10-06T12:33:40.270606+0000","last_became_peered":"2023-10-06T12:33:40.270606+0000","last_unstale":"2023-10-06T12:33:40.273738+0000","last_undegraded":"2023-10-06T12:33:40.273738+0000","last_fullsized":"2023-10-06T12:33:40.273738+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:32.377910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274558+0000","last_change":"2023-10-06T12:33:39.811637+0000","last_active":"2023-10-06T12:33:40.274558+0000","last_peered":"2023-10-06T12:33:40.274558+0000","last_clean":"2023-10-06T12:33:40.274558+0000","last_became_active":"2023-10-06T12:33:39.811347+0000","last_became_peered":"2023-10-06T12:33:39.811347+0000","last_unstale":"2023-10-06T12:33:40.274558+0000","last_undegraded":"2023-10-06T12:33:40.274558+0000","last_fullsized":"2023-10-06T12:33:40.274558+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:08:20.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281905+0000","last_change":"2023-10-06T12:33:40.276797+0000","last_active":"2023-10-06T12:33:40.281905+0000","last_peered":"2023-10-06T12:33:40.281905+0000","last_clean":"2023-10-06T12:33:40.281905+0000","last_became_active":"2023-10-06T12:33:40.276340+0000","last_became_peered":"2023-10-06T12:33:40.276340+0000","last_unstale":"2023-10-06T12:33:40.281905+0000","last_undegraded":"2023-10-06T12:33:40.281905+0000","last_fullsized":"2023-10-06T12:33:40.281905+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:55.302717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265680+0000","last_change":"2023-10-06T12:33:39.805296+0000","last_active":"2023-10-06T12:33:40.265680+0000","last_peered":"2023-10-06T12:33:40.265680+0000","last_clean":"2023-10-06T12:33:40.265680+0000","last_became_active":"2023-10-06T12:33:39.805032+0000","last_became_peered":"2023-10-06T12:33:39.805032+0000","last_unstale":"2023-10-06T12:33:40.265680+0000","last_undegraded":"2023-10-06T12:33:40.265680+0000","last_fullsized":"2023-10-06T12:33:40.265680+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:24.950347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281457+0000","last_change":"2023-10-06T12:33:40.276294+0000","last_active":"2023-10-06T12:33:40.281457+0000","last_peered":"2023-10-06T12:33:40.281457+0000","last_clean":"2023-10-06T12:33:40.281457+0000","last_became_active":"2023-10-06T12:33:40.275993+0000","last_became_peered":"2023-10-06T12:33:40.275993+0000","last_unstale":"2023-10-06T12:33:40.281457+0000","last_undegraded":"2023-10-06T12:33:40.281457+0000","last_fullsized":"2023-10-06T12:33:40.281457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:19.194989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271424+0000","last_change":"2023-10-06T12:33:39.815931+0000","last_active":"2023-10-06T12:33:40.271424+0000","last_peered":"2023-10-06T12:33:40.271424+0000","last_clean":"2023-10-06T12:33:40.271424+0000","last_became_active":"2023-10-06T12:33:39.815344+0000","last_became_peered":"2023-10-06T12:33:39.815344+0000","last_unstale":"2023-10-06T12:33:40.271424+0000","last_undegraded":"2023-10-06T12:33:40.271424+0000","last_fullsized":"2023-10-06T12:33:40.271424+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:44:23.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281426+0000","last_change":"2023-10-06T12:33:40.276887+0000","last_active":"2023-10-06T12:33:40.281426+0000","last_peered":"2023-10-06T12:33:40.281426+0000","last_clean":"2023-10-06T12:33:40.281426+0000","last_became_active":"2023-10-06T12:33:40.276612+0000","last_became_peered":"2023-10-06T12:33:40.276612+0000","last_unstale":"2023-10-06T12:33:40.281426+0000","last_undegraded":"2023-10-06T12:33:40.281426+0000","last_fullsized":"2023-10-06T12:33:40.281426+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:49:27.765151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274284+0000","last_change":"2023-10-06T12:33:39.813959+0000","last_active":"2023-10-06T12:33:40.274284+0000","last_peered":"2023-10-06T12:33:40.274284+0000","last_clean":"2023-10-06T12:33:40.274284+0000","last_became_active":"2023-10-06T12:33:39.813372+0000","last_became_peered":"2023-10-06T12:33:39.813372+0000","last_unstale":"2023-10-06T12:33:40.274284+0000","last_undegraded":"2023-10-06T12:33:40.274284+0000","last_fullsized":"2023-10-06T12:33:40.274284+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:13:25.113408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.294809+0000","last_change":"2023-10-06T12:33:40.292608+0000","last_active":"2023-10-06T12:33:40.294809+0000","last_peered":"2023-10-06T12:33:40.294809+0000","last_clean":"2023-10-06T12:33:40.294809+0000","last_became_active":"2023-10-06T12:33:40.292296+0000","last_became_peered":"2023-10-06T12:33:40.292296+0000","last_unstale":"2023-10-06T12:33:40.294809+0000","last_undegraded":"2023-10-06T12:33:40.294809+0000","last_fullsized":"2023-10-06T12:33:40.294809+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:34:21.486786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267886+0000","last_change":"2023-10-06T12:33:39.809086+0000","last_active":"2023-10-06T12:33:40.267886+0000","last_peered":"2023-10-06T12:33:40.267886+0000","last_clean":"2023-10-06T12:33:40.267886+0000","last_became_active":"2023-10-06T12:33:39.808809+0000","last_became_peered":"2023-10-06T12:33:39.808809+0000","last_unstale":"2023-10-06T12:33:40.267886+0000","last_undegraded":"2023-10-06T12:33:40.267886+0000","last_fullsized":"2023-10-06T12:33:40.267886+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:45:54.287643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.290334+0000","last_change":"2023-10-06T12:33:40.286561+0000","last_active":"2023-10-06T12:33:40.290334+0000","last_peered":"2023-10-06T12:33:40.290334+0000","last_clean":"2023-10-06T12:33:40.290334+0000","last_became_active":"2023-10-06T12:33:40.283580+0000","last_became_peered":"2023-10-06T12:33:40.283580+0000","last_unstale":"2023-10-06T12:33:40.290334+0000","last_undegraded":"2023-10-06T12:33:40.290334+0000","last_fullsized":"2023-10-06T12:33:40.290334+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:46:33.629865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271548+0000","last_change":"2023-10-06T12:33:39.812250+0000","last_active":"2023-10-06T12:33:40.271548+0000","last_peered":"2023-10-06T12:33:40.271548+0000","last_clean":"2023-10-06T12:33:40.271548+0000","last_became_active":"2023-10-06T12:33:39.811809+0000","last_became_peered":"2023-10-06T12:33:39.811809+0000","last_unstale":"2023-10-06T12:33:40.271548+0000","last_undegraded":"2023-10-06T12:33:40.271548+0000","last_fullsized":"2023-10-06T12:33:40.271548+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:24.023484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"142'271","reported_seq":822,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271897+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:33:40.271897+0000","last_peered":"2023-10-06T12:33:40.271897+0000","last_clean":"2023-10-06T12:33:40.271897+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:33:40.271897+0000","last_undegraded":"2023-10-06T12:33:40.271897+0000","last_fullsized":"2023-10-06T12:33:40.271897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265798+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:33:40.265798+0000","last_peered":"2023-10-06T12:33:40.265798+0000","last_clean":"2023-10-06T12:33:40.265798+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:33:40.265798+0000","last_undegraded":"2023-10-06T12:33:40.265798+0000","last_fullsized":"2023-10-06T12:33:40.265798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.286113+0000","last_change":"2023-10-06T12:33:40.280700+0000","last_active":"2023-10-06T12:33:40.286113+0000","last_peered":"2023-10-06T12:33:40.286113+0000","last_clean":"2023-10-06T12:33:40.286113+0000","last_became_active":"2023-10-06T12:33:40.280391+0000","last_became_peered":"2023-10-06T12:33:40.280391+0000","last_unstale":"2023-10-06T12:33:40.286113+0000","last_undegraded":"2023-10-06T12:33:40.286113+0000","last_fullsized":"2023-10-06T12:33:40.286113+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.839425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.283297+0000","last_change":"2023-10-06T12:33:39.800527+0000","last_active":"2023-10-06T12:33:40.283297+0000","last_peered":"2023-10-06T12:33:40.283297+0000","last_clean":"2023-10-06T12:33:40.283297+0000","last_became_active":"2023-10-06T12:33:39.800175+0000","last_became_peered":"2023-10-06T12:33:39.800175+0000","last_unstale":"2023-10-06T12:33:40.283297+0000","last_undegraded":"2023-10-06T12:33:40.283297+0000","last_fullsized":"2023-10-06T12:33:40.283297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:54.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.271783+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:33:40.271783+0000","last_peered":"2023-10-06T12:33:40.271783+0000","last_clean":"2023-10-06T12:33:40.271783+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:33:40.271783+0000","last_undegraded":"2023-10-06T12:33:40.271783+0000","last_fullsized":"2023-10-06T12:33:40.271783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287437+0000","last_change":"2023-10-06T12:33:40.283772+0000","last_active":"2023-10-06T12:33:40.287437+0000","last_peered":"2023-10-06T12:33:40.287437+0000","last_clean":"2023-10-06T12:33:40.287437+0000","last_became_active":"2023-10-06T12:33:40.283455+0000","last_became_peered":"2023-10-06T12:33:40.283455+0000","last_unstale":"2023-10-06T12:33:40.287437+0000","last_undegraded":"2023-10-06T12:33:40.287437+0000","last_fullsized":"2023-10-06T12:33:40.287437+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:34.226446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"152'8","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812211+0000","last_change":"2023-10-06T12:33:39.814953+0000","last_active":"2023-10-06T12:33:40.812211+0000","last_peered":"2023-10-06T12:33:40.812211+0000","last_clean":"2023-10-06T12:33:40.812211+0000","last_became_active":"2023-10-06T12:33:39.814675+0000","last_became_peered":"2023-10-06T12:33:39.814675+0000","last_unstale":"2023-10-06T12:33:40.812211+0000","last_undegraded":"2023-10-06T12:33:40.812211+0000","last_fullsized":"2023-10-06T12:33:40.812211+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:36.872666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266451+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:33:40.266451+0000","last_peered":"2023-10-06T12:33:40.266451+0000","last_clean":"2023-10-06T12:33:40.266451+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:33:40.266451+0000","last_undegraded":"2023-10-06T12:33:40.266451+0000","last_fullsized":"2023-10-06T12:33:40.266451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.281384+0000","last_change":"2023-10-06T12:33:40.275173+0000","last_active":"2023-10-06T12:33:40.281384+0000","last_peered":"2023-10-06T12:33:40.281384+0000","last_clean":"2023-10-06T12:33:40.281384+0000","last_became_active":"2023-10-06T12:33:40.274912+0000","last_became_peered":"2023-10-06T12:33:40.274912+0000","last_unstale":"2023-10-06T12:33:40.281384+0000","last_undegraded":"2023-10-06T12:33:40.281384+0000","last_fullsized":"2023-10-06T12:33:40.281384+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:15.677092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"152'2","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.808626+0000","last_change":"2023-10-06T12:33:39.802677+0000","last_active":"2023-10-06T12:33:40.808626+0000","last_peered":"2023-10-06T12:33:40.808626+0000","last_clean":"2023-10-06T12:33:40.808626+0000","last_became_active":"2023-10-06T12:33:39.802329+0000","last_became_peered":"2023-10-06T12:33:39.802329+0000","last_unstale":"2023-10-06T12:33:40.808626+0000","last_undegraded":"2023-10-06T12:33:40.808626+0000","last_fullsized":"2023-10-06T12:33:40.808626+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:08:46.826229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":138,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.265221+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:33:40.265221+0000","last_peered":"2023-10-06T12:33:40.265221+0000","last_clean":"2023-10-06T12:33:40.265221+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:33:40.265221+0000","last_undegraded":"2023-10-06T12:33:40.265221+0000","last_fullsized":"2023-10-06T12:33:40.265221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.295188+0000","last_change":"2023-10-06T12:33:40.292699+0000","last_active":"2023-10-06T12:33:40.295188+0000","last_peered":"2023-10-06T12:33:40.295188+0000","last_clean":"2023-10-06T12:33:40.295188+0000","last_became_active":"2023-10-06T12:33:40.288567+0000","last_became_peered":"2023-10-06T12:33:40.288567+0000","last_unstale":"2023-10-06T12:33:40.295188+0000","last_undegraded":"2023-10-06T12:33:40.295188+0000","last_fullsized":"2023-10-06T12:33:40.295188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:00.323237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"152'3","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812728+0000","last_change":"2023-10-06T12:33:39.808806+0000","last_active":"2023-10-06T12:33:40.812728+0000","last_peered":"2023-10-06T12:33:40.812728+0000","last_clean":"2023-10-06T12:33:40.812728+0000","last_became_active":"2023-10-06T12:33:39.808486+0000","last_became_peered":"2023-10-06T12:33:39.808486+0000","last_unstale":"2023-10-06T12:33:40.812728+0000","last_undegraded":"2023-10-06T12:33:40.812728+0000","last_fullsized":"2023-10-06T12:33:40.812728+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:23:33.211372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274637+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:33:40.274637+0000","last_peered":"2023-10-06T12:33:40.274637+0000","last_clean":"2023-10-06T12:33:40.274637+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:33:40.274637+0000","last_undegraded":"2023-10-06T12:33:40.274637+0000","last_fullsized":"2023-10-06T12:33:40.274637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293560+0000","last_change":"2023-10-06T12:33:40.289777+0000","last_active":"2023-10-06T12:33:40.293560+0000","last_peered":"2023-10-06T12:33:40.293560+0000","last_clean":"2023-10-06T12:33:40.293560+0000","last_became_active":"2023-10-06T12:33:40.289325+0000","last_became_peered":"2023-10-06T12:33:40.289325+0000","last_unstale":"2023-10-06T12:33:40.293560+0000","last_undegraded":"2023-10-06T12:33:40.293560+0000","last_fullsized":"2023-10-06T12:33:40.293560+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:25.038558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"152'6","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.812531+0000","last_change":"2023-10-06T12:33:39.806182+0000","last_active":"2023-10-06T12:33:40.812531+0000","last_peered":"2023-10-06T12:33:40.812531+0000","last_clean":"2023-10-06T12:33:40.812531+0000","last_became_active":"2023-10-06T12:33:39.805865+0000","last_became_peered":"2023-10-06T12:33:39.805865+0000","last_unstale":"2023-10-06T12:33:40.812531+0000","last_undegraded":"2023-10-06T12:33:40.812531+0000","last_fullsized":"2023-10-06T12:33:40.812531+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:59.308163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.270847+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:33:40.270847+0000","last_peered":"2023-10-06T12:33:40.270847+0000","last_clean":"2023-10-06T12:33:40.270847+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:33:40.270847+0000","last_undegraded":"2023-10-06T12:33:40.270847+0000","last_fullsized":"2023-10-06T12:33:40.270847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.293890+0000","last_change":"2023-10-06T12:33:40.293011+0000","last_active":"2023-10-06T12:33:40.293890+0000","last_peered":"2023-10-06T12:33:40.293890+0000","last_clean":"2023-10-06T12:33:40.293890+0000","last_became_active":"2023-10-06T12:33:40.292669+0000","last_became_peered":"2023-10-06T12:33:40.292669+0000","last_unstale":"2023-10-06T12:33:40.293890+0000","last_undegraded":"2023-10-06T12:33:40.293890+0000","last_fullsized":"2023-10-06T12:33:40.293890+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:46.979791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.267782+0000","last_change":"2023-10-06T12:33:39.805085+0000","last_active":"2023-10-06T12:33:40.267782+0000","last_peered":"2023-10-06T12:33:40.267782+0000","last_clean":"2023-10-06T12:33:40.267782+0000","last_became_active":"2023-10-06T12:33:39.804573+0000","last_became_peered":"2023-10-06T12:33:39.804573+0000","last_unstale":"2023-10-06T12:33:40.267782+0000","last_undegraded":"2023-10-06T12:33:40.267782+0000","last_fullsized":"2023-10-06T12:33:40.267782+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:25.385873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.274987+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:33:40.274987+0000","last_peered":"2023-10-06T12:33:40.274987+0000","last_clean":"2023-10-06T12:33:40.274987+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:33:40.274987+0000","last_undegraded":"2023-10-06T12:33:40.274987+0000","last_fullsized":"2023-10-06T12:33:40.274987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.285156+0000","last_change":"2023-10-06T12:33:40.282088+0000","last_active":"2023-10-06T12:33:40.285156+0000","last_peered":"2023-10-06T12:33:40.285156+0000","last_clean":"2023-10-06T12:33:40.285156+0000","last_became_active":"2023-10-06T12:33:40.281790+0000","last_became_peered":"2023-10-06T12:33:40.281790+0000","last_unstale":"2023-10-06T12:33:40.285156+0000","last_undegraded":"2023-10-06T12:33:40.285156+0000","last_fullsized":"2023-10-06T12:33:40.285156+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:39.773182+0000","last_clean_scrub_stamp":"2023-10-06T12:33:39.773182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:40.112545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.266012+0000","last_change":"2023-10-06T12:33:39.803873+0000","last_active":"2023-10-06T12:33:40.266012+0000","last_peered":"2023-10-06T12:33:40.266012+0000","last_clean":"2023-10-06T12:33:40.266012+0000","last_became_active":"2023-10-06T12:33:39.803599+0000","last_became_peered":"2023-10-06T12:33:39.803599+0000","last_unstale":"2023-10-06T12:33:40.266012+0000","last_undegraded":"2023-10-06T12:33:40.266012+0000","last_fullsized":"2023-10-06T12:33:40.266012+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:33:38.758108+0000","last_clean_scrub_stamp":"2023-10-06T12:33:38.758108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:39.097471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-06T12:33:40.287145+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:33:40.287145+0000","last_peered":"2023-10-06T12:33:40.287145+0000","last_clean":"2023-10-06T12:33:40.287145+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:33:40.287145+0000","last_undegraded":"2023-10-06T12:33:40.287145+0000","last_fullsized":"2023-10-06T12:33:40.287145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299744,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072096,"statfs":{"total":96636764160,"available":96329826304,"internally_reserved":0,"allocated":2514944,"data_stored":1592157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.155},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.423,"5min":1.6519999999999999,"15min":1.6819999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1890000000000001,"15min":1.1719999999999999},"min":{"1min":0.86899999999999999,"5min":0.73099999999999998,"15min":0.64000000000000001},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.91800000000000004,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.5860000000000001,"5min":1.6200000000000001,"15min":1.7949999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2789999999999999,"15min":1.2829999999999999},"min":{"1min":1.1040000000000001,"5min":0.92500000000000004,"15min":0.85599999999999998},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.335}]},{"osd":2,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.6719999999999999,"5min":1.855,"15min":1.855},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2549999999999999,"15min":1.262},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.65500000000000003},"max":{"1min":1.5189999999999999,"5min":1.6240000000000001,"15min":1.7649999999999999},"last":1.046}]},{"osd":3,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.2889999999999999},"min":{"1min":0.95499999999999996,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.51},{"interface":"front","average":{"1min":1.363,"5min":1.345,"15min":1.329},"min":{"1min":1.0820000000000001,"5min":0.77500000000000002,"15min":0.70399999999999996},"max":{"1min":1.6619999999999999,"5min":1.7529999999999999,"15min":1.9019999999999999},"last":1.1910000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1120000000000001,"15min":1.107},"min":{"1min":0.77500000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.722,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":1.292},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.242,"15min":1.2370000000000001},"min":{"1min":0.871,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7649999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.5569999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.304,"15min":1.282},"min":{"1min":1.0409999999999999,"5min":0.626,"15min":0.49399999999999999},"max":{"1min":1.752,"5min":1.766,"15min":1.853},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.3200000000000001,"15min":1.3049999999999999},"min":{"1min":0.999,"5min":0.80600000000000005,"15min":0.55700000000000005},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.2170000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:49 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.389,"15min":1.3680000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.427,"15min":1.4039999999999999},"min":{"1min":1.2130000000000001,"5min":0.98399999999999999,"15min":0.54000000000000004},"max":{"1min":1.827,"5min":2.0499999999999998,"15min":2.073},"last":1.607}]}]},{"osd":6,"up_from":13,"seq":55834575750,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.27},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.304}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":1.389}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.421},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":1.161}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.5169999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.452},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.01}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":1.046},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.1240000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.2},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.3520000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575750,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":1.3029999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":1.091}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.3640000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.4359999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.571},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.403}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.6379999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.532}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":1000163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5800000000000001,"15min":1.8839999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.167,"15min":1.1890000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.887,"15min":2.0720000000000001},"last":1.0820000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.365},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6240000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.51},{"interface":"front","average":{"1min":1.27,"5min":1.325,"15min":1.329},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.5840000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.204,"15min":1.2390000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.506,"5min":1.625,"15min":1.9330000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.619,"5min":1.861,"15min":1.861},"last":1.335}]},{"osd":3,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2370000000000001,"15min":1.2490000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5429999999999999,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":1.46},{"interface":"front","average":{"1min":1.321,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6339999999999999,"5min":1.7030000000000001,"15min":1.8340000000000001},"last":1.1799999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1919999999999999,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.56999999999999995},"max":{"1min":1.552,"5min":1.7929999999999999,"15min":1.944},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1819999999999999,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.819},"last":1.276}]},{"osd":6,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.22,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.984},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3680000000000001,"15min":1.4019999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68799999999999994},"max":{"1min":1.754,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.339,"15min":1.3620000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.399,"15min":1.4359999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.69399999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1030000000000002},"last":1.6240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575752,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315548,"kb_used_data":1876,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056292,"statfs":{"total":96636764160,"available":96313643008,"internally_reserved":0,"allocated":1921024,"data_stored":1001769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.999,"15min":1.0089999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.311},"max":{"1min":1.3089999999999999,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.52500000000000002},"max":{"1min":1.258,"5min":1.6100000000000001,"15min":1.8},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.54500000000000004},"max":{"1min":1.3580000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.23}]},{"osd":2,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1439999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.56899999999999995},"max":{"1min":1.623,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.337},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.252,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.59199999999999997},"max":{"1min":1.508,"5min":1.649,"15min":2.379},"last":1.095}]},{"osd":4,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3029999999999999,"15min":1.27},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.498},"max":{"1min":1.714,"5min":1.8700000000000001,"15min":2.02},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4410000000000001,"15min":1.3979999999999999},"min":{"1min":1.026,"5min":1.026,"15min":0.72499999999999998},"max":{"1min":1.8959999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.1990000000000001,"5min":1.0740000000000001,"15min":0.76900000000000002},"max":{"1min":1.784,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4390000000000001,"15min":1.419},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.518,"15min":1.4830000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.67600000000000005},"max":{"1min":1.907,"5min":2.0270000000000001,"15min":2.0449999999999999},"last":1.532},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4510000000000001,"15min":1.3979999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7649999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.97799999999999998}]},{"osd":7,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.468},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7430000000000001,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":1.296},{"interface":"front","average":{"1min":1.625,"5min":1.5780000000000001,"15min":1.5429999999999999},"min":{"1min":1.2949999999999999,"5min":1.1679999999999999,"15min":0.80000000000000004},"max":{"1min":1.9690000000000001,"5min":2.048,"15min":2.0600000000000001},"last":1.49}]}]},{"osd":2,"up_from":13,"seq":55834575750,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311444,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060396,"statfs":{"total":96636764160,"available":96317845504,"internally_reserved":0,"allocated":1912832,"data_stored":1000751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":1.246}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.4650000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.319},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.5549999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.698},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.4970000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.8260000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.893}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.663},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.748}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307344,"kb_used_data":1864,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064496,"statfs":{"total":96636764160,"available":96322043904,"internally_reserved":0,"allocated":1908736,"data_stored":1000233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.673,"5min":1.798,"15min":1.798},"last":1.03},{"interface":"front","average":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0269999999999999},"min":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3500000000000001,"5min":1.5620000000000001,"15min":1.655},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.97499999999999998,"15min":0.99299999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.2909999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.129,"15min":1.1180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.60099999999999998},"max":{"1min":1.397,"5min":1.681,"15min":1.681},"last":1.29}]},{"osd":3,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.61599999999999999},"max":{"1min":1.8580000000000001,"5min":2.008,"15min":2.008},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1519999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.796,"5min":1.796,"15min":1.806},"last":1.5589999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.268,"15min":1.27},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.597,"5min":1.764,"15min":2.2490000000000001},"last":1.135},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.73,"5min":2.226,"15min":2.226},"last":1.3999999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3939999999999999,"15min":1.387},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8069999999999999,"5min":1.8460000000000001,"15min":2.0350000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.381},"min":{"1min":1,"5min":1,"15min":0.93400000000000005},"max":{"1min":1.74,"5min":1.917,"15min":2.0619999999999998},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5529999999999999,"15min":1.5149999999999999},"min":{"1min":1.2270000000000001,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":1.9470000000000001,"5min":2.0800000000000001,"15min":2.206},"last":1.508},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.4710000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.9450000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.462}]},{"osd":7,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.5860000000000001,"15min":1.571},"min":{"1min":1.038,"5min":1.038,"15min":0.99299999999999999},"max":{"1min":2.028,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":1.633},{"interface":"front","average":{"1min":1.516,"5min":1.5149999999999999,"15min":1.49},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":0.95799999999999996},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1819999999999999},"last":1.478}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299736,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072104,"statfs":{"total":96636764160,"available":96329834496,"internally_reserved":0,"allocated":2506752,"data_stored":1590587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.91000000000000003,"15min":0.90700000000000003},"min":{"1min":0.77200000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.492},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.008,"15min":1.0189999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.59799999999999998},"max":{"1min":1.4339999999999999,"5min":1.5549999999999999,"15min":1.595},"last":1.042}]},{"osd":2,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.78000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3340000000000001,"5min":1.5469999999999999,"15min":1.746},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.4970000000000001,"5min":1.762,"15min":1.762},"last":1.3420000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.099,"15min":1.0920000000000001},"min":{"1min":0.86199999999999999,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.202,"5min":0.84699999999999998,"15min":0.71999999999999997},"max":{"1min":1.7030000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.2330000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.371,"15min":1.369},"min":{"1min":1.1819999999999999,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.663,"5min":1.71,"15min":1.756},"last":1.444},{"interface":"front","average":{"1min":1.417,"5min":1.3340000000000001,"15min":1.3320000000000001},"min":{"1min":1.0760000000000001,"5min":0.81100000000000005,"15min":0.80500000000000005},"max":{"1min":1.704,"5min":1.7290000000000001,"15min":2.036},"last":1.506}]},{"osd":5,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.407,"15min":1.401},"min":{"1min":1.117,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9330000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4239999999999999,"15min":1.423},"min":{"1min":1.2350000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.865,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.7010000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.347,"15min":1.357},"min":{"1min":1.1879999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.649,"5min":1.732,"15min":2.2029999999999998},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.478,"15min":1.48},"min":{"1min":1.202,"5min":1.0600000000000001,"15min":1.0600000000000001},"max":{"1min":1.919,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.478}]},{"osd":7,"last update":"Fri Oct 6 12:32:44 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5029999999999999,"15min":1.482},"min":{"1min":1.2490000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.8100000000000001,"5min":2.0099999999999998,"15min":2.1019999999999999},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4570000000000001,"15min":1.4490000000000001},"min":{"1min":1.087,"5min":0.98599999999999999,"15min":0.95099999999999996},"max":{"1min":1.8480000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.675}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:33:50.278 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T12:33:50.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T12:33:50.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 Processor -- start 2023-10-06T12:33:50.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 -- start start 2023-10-06T12:33:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3af41594a0 0x7f3af40ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3af40cac10 0x7f3af40afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af4068c30 con 0x7f3af41590d0 2023-10-06T12:33:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af40cb090 con 0x7f3af40cac10 2023-10-06T12:33:50.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.484+0000 7f3af89c4700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af40cd0a0 con 0x7f3af41594a0 2023-10-06T12:33:50.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3af40cac10 0x7f3af40afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3af40cac10 0x7f3af40afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38684/0 (socket says 172.21.15.46:38684) 2023-10-06T12:33:50.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 -- 172.21.15.46:0/4250271989 learned_addr learned my addr 172.21.15.46:0/4250271989 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:50.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af259c700 1 --2- 172.21.15.46:0/4250271989 >> v2:172.21.15.132:3300/0 conn(0x7f3af41594a0 0x7f3af40ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 -- 172.21.15.46:0/4250271989 >> v2:172.21.15.132:3300/0 conn(0x7f3af41594a0 msgr2=0x7f3af40ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 --2- 172.21.15.46:0/4250271989 >> v2:172.21.15.132:3300/0 conn(0x7f3af41594a0 0x7f3af40ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 -- 172.21.15.46:0/4250271989 >> v1:172.21.15.132:6789/0 conn(0x7f3af41590d0 legacy=0x7f3af40cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 -- 172.21.15.46:0/4250271989 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af406a0d0 con 0x7f3af40cac10 2023-10-06T12:33:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af359e700 1 --2- 172.21.15.46:0/4250271989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af40cac10 0x7f3af40afa10 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3af415ce50 tx=0x7f3aec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77e15590f9aafc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af1d9b700 1 -- 172.21.15.46:0/4250271989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec00e070 con 0x7f3af40cac10 2023-10-06T12:33:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af1d9b700 1 -- 172.21.15.46:0/4250271989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec00ba20 con 0x7f3af40cac10 2023-10-06T12:33:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af1d9b700 1 -- 172.21.15.46:0/4250271989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0092e0 con 0x7f3af40cac10 2023-10-06T12:33:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 -- 172.21.15.46:0/4250271989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af40cac10 msgr2=0x7f3af40afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 --2- 172.21.15.46:0/4250271989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af40cac10 0x7f3af40afa10 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3af415ce50 tx=0x7f3aec009840 comp rx=0 tx=0).stop 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 -- 172.21.15.46:0/4250271989 shutdown_connections 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 --2- 172.21.15.46:0/4250271989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af40cac10 0x7f3af40afa10 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 --2- 172.21.15.46:0/4250271989 >> v2:172.21.15.132:3300/0 conn(0x7f3af41594a0 0x7f3af40ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 -- 172.21.15.46:0/4250271989 >> 172.21.15.46:0/4250271989 conn(0x7f3af40970c0 msgr2=0x7f3af4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 -- 172.21.15.46:0/4250271989 shutdown_connections 2023-10-06T12:33:50.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.488+0000 7f3af89c4700 1 -- 172.21.15.46:0/4250271989 wait complete. 2023-10-06T12:33:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 Processor -- start 2023-10-06T12:33:50.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 -- start start 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 0x7f3af40b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af41590d0 0x7f3af40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40c4550 con 0x7f3af41594a0 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40b39f0 con 0x7f3af40cac10 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af89c4700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af40c3f00 con 0x7f3af41590d0 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38690/0 (socket says 172.21.15.46:38690) 2023-10-06T12:33:50.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af359e700 1 -- 172.21.15.46:0/854813933 learned_addr learned my addr 172.21.15.46:0/854813933 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.492+0000 7f3af2d9d700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 0x7f3af40b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af259c700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af41590d0 0x7f3af40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 -- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af41590d0 msgr2=0x7f3af40afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af41590d0 0x7f3af40afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 -- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 msgr2=0x7f3af40b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 0x7f3af40b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 -- 172.21.15.46:0/854813933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec009ae0 con 0x7f3af41594a0 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af2d9d700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 0x7f3af40b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:50.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af359e700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009880 tx=0x7f3aec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec008be0 con 0x7f3af41594a0 2023-10-06T12:33:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec008d60 con 0x7f3af41594a0 2023-10-06T12:33:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec01c660 con 0x7f3af41594a0 2023-10-06T12:33:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af40c41e0 con 0x7f3af41594a0 2023-10-06T12:33:50.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af40c3bd0 con 0x7f3af41594a0 2023-10-06T12:33:50.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.496+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af4064590 con 0x7f3af41594a0 2023-10-06T12:33:50.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.500+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3aec003040 con 0x7f3af41594a0 2023-10-06T12:33:50.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.500+0000 7f3adaffd700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 0x7f3ae0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.500+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f3aec07e6f0 con 0x7f3af41594a0 2023-10-06T12:33:50.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.500+0000 7f3af2d9d700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 0x7f3ae0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.504+0000 7f3af2d9d700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 0x7f3ae0062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40018f0 tx=0x7f3ae4009040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.508+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3aec04e6c0 con 0x7f3af41594a0 2023-10-06T12:33:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.736+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3af404b5c0 con 0x7f3af41594a0 2023-10-06T12:33:50.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.744+0000 7f3adaffd700 1 -- 172.21.15.46:0/854813933 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3aec051ce0 con 0x7f3af41594a0 2023-10-06T12:33:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 msgr2=0x7f3ae0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 0x7f3ae0062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40018f0 tx=0x7f3ae4009040 comp rx=0 tx=0).stop 2023-10-06T12:33:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 msgr2=0x7f3af40b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009880 tx=0x7f3aec018000 comp rx=0 tx=0).stop 2023-10-06T12:33:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 shutdown_connections 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3ae0060670 0x7f3ae0062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3af41594a0 0x7f3af40b0160 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af41590d0 0x7f3af40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 --2- 172.21.15.46:0/854813933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40cac10 0x7f3af40b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 >> 172.21.15.46:0/854813933 conn(0x7f3af40970c0 msgr2=0x7f3af406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 shutdown_connections 2023-10-06T12:33:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.748+0000 7f3af89c4700 1 -- 172.21.15.46:0/854813933 wait complete. 2023-10-06T12:33:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T12:33:50.771 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:33:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.940+0000 7f9da1145700 1 Processor -- start 2023-10-06T12:33:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.940+0000 7f9da1145700 1 -- start start 2023-10-06T12:33:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9da1145700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9da1145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9da1145700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9c06d460 con 0x7f9d9c15af30 2023-10-06T12:33:50.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9da1145700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c7000 con 0x7f9d9c15ab60 2023-10-06T12:33:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9da1145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c72c0 con 0x7f9d9c0ad6c0 2023-10-06T12:33:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:45372/0 (socket says 172.21.15.46:45372) 2023-10-06T12:33:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 -- 172.21.15.46:0/534742844 learned_addr learned my addr 172.21.15.46:0/534742844 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 -- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 msgr2=0x7f9d9c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 --2- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c0adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 -- 172.21.15.46:0/534742844 >> v1:172.21.15.132:6789/0 conn(0x7f9d9c15af30 legacy=0x7f9d9c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.944+0000 7f9d9ad9d700 1 -- 172.21.15.46:0/534742844 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c0c75a0 con 0x7f9d9c15ab60 2023-10-06T12:33:50.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9d9ad9d700 1 --2- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53425df387441ee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9d9a59c700 1 -- 172.21.15.46:0/534742844 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c009e80 con 0x7f9d9c15ab60 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9d9a59c700 1 -- 172.21.15.46:0/534742844 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c005c10 con 0x7f9d9c15ab60 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9d9a59c700 1 -- 172.21.15.46:0/534742844 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00b650 con 0x7f9d9c15ab60 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 -- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15ab60 msgr2=0x7f9d9c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 --2- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c009220 comp rx=0 tx=0).stop 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 -- 172.21.15.46:0/534742844 shutdown_connections 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 --2- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 --2- 172.21.15.46:0/534742844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c06e8a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 -- 172.21.15.46:0/534742844 >> 172.21.15.46:0/534742844 conn(0x7f9d9c0cbe30 msgr2=0x7f9d9c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 -- 172.21.15.46:0/534742844 shutdown_connections 2023-10-06T12:33:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.948+0000 7f9da1145700 1 -- 172.21.15.46:0/534742844 wait complete. 2023-10-06T12:33:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 Processor -- start 2023-10-06T12:33:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 -- start start 2023-10-06T12:33:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c13fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15af30 0x7f9d9c16aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c4c00 con 0x7f9d9c15ab60 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c3ff0 con 0x7f9d9c15af30 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9da1145700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c0c42d0 con 0x7f9d9c0ad6c0 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38696/0 (socket says 172.21.15.46:38696) 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 -- 172.21.15.46:0/957823405 learned_addr learned my addr 172.21.15.46:0/957823405 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15af30 0x7f9d9c16aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9ad9d700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c13fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 -- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 msgr2=0x7f9d9c13fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c13fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 -- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15af30 msgr2=0x7f9d9c16aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15af30 0x7f9d9c16aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.952+0000 7f9d9259c700 1 -- 172.21.15.46:0/957823405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c009480 con 0x7f9d9c15ab60 2023-10-06T12:33:50.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9d9259c700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c590 tx=0x7f9d8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84011590 con 0x7f9d9c15ab60 2023-10-06T12:33:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84011710 con 0x7f9d9c15ab60 2023-10-06T12:33:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c0c45b0 con 0x7f9d9c15ab60 2023-10-06T12:33:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84011ad0 con 0x7f9d9c15ab60 2023-10-06T12:33:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c1400d0 con 0x7f9d9c15ab60 2023-10-06T12:33:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.956+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9c063f90 con 0x7f9d9c15ab60 2023-10-06T12:33:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.960+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9d8401b3b0 con 0x7f9d9c15ab60 2023-10-06T12:33:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.960+0000 7f9d937fe700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 0x7f9d7c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.960+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f9d8407c140 con 0x7f9d9c15ab60 2023-10-06T12:33:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.960+0000 7f9d9ad9d700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 0x7f9d7c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:50.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.964+0000 7f9d9ad9d700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 0x7f9d7c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009260 tx=0x7f9d8c016000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:50.968+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9d8404c220 con 0x7f9d9c15ab60 2023-10-06T12:33:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.172+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d9c04b5c0 con 0x7f9d9c15ab60 2023-10-06T12:33:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.172+0000 7f9d937fe700 1 -- 172.21.15.46:0/957823405 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9d8404f840 con 0x7f9d9c15ab60 2023-10-06T12:33:51.177 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:33:51.177 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:33:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 msgr2=0x7f9d7c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 0x7f9d7c062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009260 tx=0x7f9d8c016000 comp rx=0 tx=0).stop 2023-10-06T12:33:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 msgr2=0x7f9d9c16a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c590 tx=0x7f9d8400c8a0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 shutdown_connections 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9d7c060670 0x7f9d7c062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d9c15af30 0x7f9d9c16aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d9c15ab60 0x7f9d9c16a9a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 --2- 172.21.15.46:0/957823405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9c0ad6c0 0x7f9d9c13fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 >> 172.21.15.46:0/957823405 conn(0x7f9d9c0cbe30 msgr2=0x7f9d9c0c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 shutdown_connections 2023-10-06T12:33:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.176+0000 7f9da1145700 1 -- 172.21.15.46:0/957823405 wait complete. 2023-10-06T12:33:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:33:51.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T12:33:40.245290+0000', 'modified': '2023-10-06T12:33:43.824858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8796}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8796': {'gid': 8796, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/790293012', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 790293012}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 790293012}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T12:33:51.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T12:33:51.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T12:33:51.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.768+0000 7fcdf4878700 1 Processor -- start 2023-10-06T12:33:51.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 -- start start 2023-10-06T12:33:51.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 0x7fcdf006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcdf015af30 0x7fcdf006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf006d460 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00c7000 con 0x7fcdf015ab60 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdf4878700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00c72c0 con 0x7fcdf015af30 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdee59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 0x7fcdf006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdee59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 0x7fcdf006e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38710/0 (socket says 172.21.15.46:38710) 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcdee59c700 1 -- 172.21.15.46:0/1921315276 learned_addr learned my addr 172.21.15.46:0/1921315276 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.772+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1921315276 >> v2:172.21.15.132:3300/0 conn(0x7fcdf015af30 0x7fcdf006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcde7fff700 1 -- 172.21.15.46:0/1921315276 >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 msgr2=0x7fcdf006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1921315276 >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 0x7fcdf006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcde7fff700 1 -- 172.21.15.46:0/1921315276 >> v1:172.21.15.132:6789/0 conn(0x7fcdf00ad6c0 legacy=0x7fcdf00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcde7fff700 1 -- 172.21.15.46:0/1921315276 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00c75a0 con 0x7fcdf015af30 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcde7fff700 1 --2- 172.21.15.46:0/1921315276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015af30 0x7fcdf006cc30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcdf006f0b0 tx=0x7fcdd80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922116792c12cb69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdedd9b700 1 -- 172.21.15.46:0/1921315276 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011630 con 0x7fcdf015af30 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdedd9b700 1 -- 172.21.15.46:0/1921315276 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8011790 con 0x7fcdf015af30 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdedd9b700 1 -- 172.21.15.46:0/1921315276 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011b30 con 0x7fcdf015af30 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 -- 172.21.15.46:0/1921315276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015af30 msgr2=0x7fcdf006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 --2- 172.21.15.46:0/1921315276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015af30 0x7fcdf006cc30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcdf006f0b0 tx=0x7fcdd80091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 -- 172.21.15.46:0/1921315276 shutdown_connections 2023-10-06T12:33:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 --2- 172.21.15.46:0/1921315276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015af30 0x7fcdf006cc30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 --2- 172.21.15.46:0/1921315276 >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ab60 0x7fcdf006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 -- 172.21.15.46:0/1921315276 >> 172.21.15.46:0/1921315276 conn(0x7fcdf00cbe30 msgr2=0x7fcdf00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 -- 172.21.15.46:0/1921315276 shutdown_connections 2023-10-06T12:33:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.776+0000 7fcdf4878700 1 -- 172.21.15.46:0/1921315276 wait complete. 2023-10-06T12:33:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 Processor -- start 2023-10-06T12:33:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 -- start start 2023-10-06T12:33:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015ab60 0x7fcdf00ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcdf015af30 0x7fcdf00c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cb2b0 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cb500 con 0x7fcdf015ab60 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdf4878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cb7e0 con 0x7fcdf015af30 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38712/0 (socket says 172.21.15.46:38712) 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 -- 172.21.15.46:0/3396770836 learned_addr learned my addr 172.21.15.46:0/3396770836 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015ab60 0x7fcdf00ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcdf015af30 0x7fcdf00c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 -- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcdf015af30 msgr2=0x7fcdf00c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcdf015af30 0x7fcdf00c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 -- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015ab60 msgr2=0x7fcdf00ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015ab60 0x7fcdf00ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 -- 172.21.15.46:0/3396770836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8009420 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.780+0000 7fcdee59c700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fcde000ea20 tx=0x7fcde0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.784+0000 7fcde67fc700 1 -- 172.21.15.46:0/3396770836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000c850 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.784+0000 7fcde67fc700 1 -- 172.21.15.46:0/3396770836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000c9b0 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.784+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00c9ba0 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.784+0000 7fcde67fc700 1 -- 172.21.15.46:0/3396770836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000cd70 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.784+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf015a400 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.788+0000 7fcde67fc700 1 -- 172.21.15.46:0/3396770836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fcde0013680 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.788+0000 7fcde67fc700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 0x7fcdd0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.788+0000 7fcde67fc700 1 -- 172.21.15.46:0/3396770836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fcde007d2e0 con 0x7fcdf00ad6c0 2023-10-06T12:33:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.788+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 0x7fcdd0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 Processor -- start 2023-10-06T12:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 0x7fcdd0062b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80093f0 tx=0x7fcdd8017000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 -- start start 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fd60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f03f30 con 0x1d58710 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1effed0 con 0x1d58ae0 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ec60b0 con 0x1da3790 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38728/0 (socket says 172.21.15.46:38728) 2023-10-06T12:33:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 -- 172.21.15.46:0/3347105250 learned_addr learned my addr 172.21.15.46:0/3347105250 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdee59c700 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fd60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 -- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 msgr2=0x1fd60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fd60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 -- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 msgr2=0x1fd5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fd5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 -- 172.21.15.46:0/3347105250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1da49e0 con 0x1d58710 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.792+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcdf006bf10 tx=0x7fcdd8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a30e5bc57b29cfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdcf7fe700 1 -- 172.21.15.46:0/3347105250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8020070 con 0x1d58710 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdcf7fe700 1 -- 172.21.15.46:0/3347105250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8029890 con 0x1d58710 2023-10-06T12:33:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdcf7fe700 1 -- 172.21.15.46:0/3347105250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8029a20 con 0x1d58710 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3347105250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 msgr2=0x1fd5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcdf006bf10 tx=0x7fcdd8020000 comp rx=0 tx=0).stop 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3347105250 shutdown_connections 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fd60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd5b70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3347105250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fd5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3347105250 >> 172.21.15.46:0/3347105250 conn(0x1d58eb0 msgr2=0x1f92060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3347105250 shutdown_connections 2023-10-06T12:33:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3347105250 wait complete. 2023-10-06T12:33:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 -- start start 2023-10-06T12:33:51.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.796+0000 7fcdf6b68740 1 Processor -- start 2023-10-06T12:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1f8fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ec5d50 con 0x1d58710 2023-10-06T12:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e9e970 con 0x1d58ae0 2023-10-06T12:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ec5420 con 0x1da3790 2023-10-06T12:33:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 10 client.0 _ll_register_callbacks cb 0x1ea2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38732/0 (socket says 172.21.15.46:38732) 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 -- 172.21.15.46:0/2820301675 learned_addr learned my addr 172.21.15.46:0/2820301675 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcde7fff700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1f8fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 -- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 msgr2=0x1f8fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1f8fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 -- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 msgr2=0x1fda180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fda180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 -- 172.21.15.46:0/2820301675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00c75a0 con 0x1d58710 2023-10-06T12:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcde7fff700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fda180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdee59c700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcde007b370 tx=0x7fcde007d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a449f9b3d82bbe18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdccff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0051320 con 0x1d58710 2023-10-06T12:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e9ecc0 con 0x1d58710 2023-10-06T12:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fcde0051480 con 0x1d58710 2023-10-06T12:33:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde0050780 con 0x1d58710 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 10 client.8970 fetch_fsmap learned FSMap version 76 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x1ece370 con 0x1d58710 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.800+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e9ecc0 con 0x1d58710 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcde0057bd0 con 0x1d58710 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 10 client.8970 fetch_fsmap finished waiting for FSMap version 76 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 10 client.8970 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 20 client.8970 populate_metadata read hostname 'smithi046' 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1f91260 0x1fe4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 4 client.8970 mds_command: new command op to 8796 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 --> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ea23e0 con 0x1f91260 2023-10-06T12:33:51.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcde7fff700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1f91260 0x1fe4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcde7fff700 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1f91260 0x1fe4de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243cbc6b8dfa220b server_cookie=b40ea4478a4ed90d in_seq=0 out_seq=0 2023-10-06T12:33:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdccff9700 10 client.8970 ms_handle_connect on v2:172.21.15.46:6834/790293012 2023-10-06T12:33:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdccff9700 1 -- 172.21.15.46:0/2820301675 <== mds.0 v2:172.21.15.46:6834/790293012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1ea23e0 con 0x1f91260 2023-10-06T12:33:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdccff9700 10 client.8970 handle_command_reply: tid=0 2023-10-06T12:33:51.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 client.8970 shutdown 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 msgr2=0x1fd9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcde007b370 tx=0x7fcde007d040 comp rx=0 tx=0).stop 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 shutdown_connections 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1f8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1f91260 0x1fe4de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdccff9700 0 client.8970 ms_handle_reset on v2:172.21.15.46:6834/790293012 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fda180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/2820301675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fd9c40 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.804+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 >> 172.21.15.46:0/2820301675 conn(0x1d58eb0 msgr2=0x1fb91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.808+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 shutdown_connections 2023-10-06T12:33:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.808+0000 7fcdf6b68740 1 -- 172.21.15.46:0/2820301675 wait complete. 2023-10-06T12:33:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.808+0000 7fcdf6b68740 20 client.8970 trim_cache size 0 max 16384 2023-10-06T12:33:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 Processor -- start 2023-10-06T12:33:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 -- start start 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1d58710 0x2033220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x2033760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1efbb50 con 0x1da3790 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ed4610 con 0x1d58ae0 2023-10-06T12:33:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dd17c0 con 0x1d58710 2023-10-06T12:33:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdeed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdeed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38740/0 (socket says 172.21.15.46:38740) 2023-10-06T12:33:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdeed9d700 1 -- 172.21.15.46:0/3824076343 learned_addr learned my addr 172.21.15.46:0/3824076343 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcde7fff700 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x2033760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.828+0000 7fcdee59c700 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1d58710 0x2033220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 -- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1d58710 msgr2=0x2033220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1d58710 0x2033220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 -- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 msgr2=0x2033760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x2033760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 -- 172.21.15.46:0/3824076343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1eb4d80 con 0x1da3790 2023-10-06T12:33:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcde8002e80 tx=0x7fcde80015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb6d2a970a9c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdcffff700 1 -- 172.21.15.46:0/3824076343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8006db0 con 0x1da3790 2023-10-06T12:33:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdcffff700 1 -- 172.21.15.46:0/3824076343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8013ad0 con 0x1da3790 2023-10-06T12:33:51.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdcffff700 1 -- 172.21.15.46:0/3824076343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8013e70 con 0x1da3790 2023-10-06T12:33:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3824076343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 msgr2=0x2033ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcde8002e80 tx=0x7fcde80015f0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3824076343 shutdown_connections 2023-10-06T12:33:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1da3790 0x2033ca0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x2033760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/3824076343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1d58710 0x2033220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3824076343 >> 172.21.15.46:0/3824076343 conn(0x1d58eb0 msgr2=0x1fd8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3824076343 shutdown_connections 2023-10-06T12:33:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.832+0000 7fcdf6b68740 1 -- 172.21.15.46:0/3824076343 wait complete. 2023-10-06T12:33:51.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 -- start start 2023-10-06T12:33:51.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 Processor -- start 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fb30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eaee00 con 0x1d58710 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f04810 con 0x1d58ae0 2023-10-06T12:33:51.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eabec0 con 0x1da3790 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdf6b68740 10 client.0 _ll_register_callbacks cb 0x1ecb110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38754/0 (socket says 172.21.15.46:38754) 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdee59c700 1 -- 172.21.15.46:0/442546611 learned_addr learned my addr 172.21.15.46:0/442546611 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcdeed9d700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.836+0000 7fcde7fff700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fb30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 -- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 msgr2=0x1fb3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fb3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 -- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 msgr2=0x1fb30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fb30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 -- 172.21.15.46:0/442546611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde800f040 con 0x1d58710 2023-10-06T12:33:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdee59c700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcde004d580 tx=0x7fcde0065000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90da6480a0912ec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0050e50 con 0x1d58710 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fcde0058060 con 0x1d58710 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdd8029890 con 0x1d58710 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde007b0f0 con 0x1d58710 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 10 client.8976 fetch_fsmap learned FSMap version 76 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x1ea9130 con 0x1d58710 2023-10-06T12:33:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8029890 con 0x1d58710 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fcde0059030 con 0x1d58710 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 10 client.8976 fetch_fsmap finished waiting for FSMap version 76 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 10 client.8976 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 20 client.8976 populate_metadata read hostname 'smithi046' 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1fb3b40 0x203f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 4 client.8976 mds_command: new command op to 8796 tid=0[{"prefix": "status"}] 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 --> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1ed4410 con 0x1fb3b40 2023-10-06T12:33:51.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcde7fff700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1fb3b40 0x203f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcdccff9700 10 client.8976 ms_handle_connect on v2:172.21.15.46:6834/790293012 2023-10-06T12:33:51.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.840+0000 7fcde7fff700 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1fb3b40 0x203f4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91863271bccfd09 server_cookie=47429ddc300a8549 in_seq=0 out_seq=0 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdccff9700 1 -- 172.21.15.46:0/442546611 <== mds.0 v2:172.21.15.46:6834/790293012 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x1ed4410 con 0x1fb3b40 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdccff9700 10 client.8976 handle_command_reply: tid=0 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 client.8976 shutdown 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 msgr2=0x1fb64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fcde004d580 tx=0x7fcde0065000 comp rx=0 tx=0).stop 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 shutdown_connections 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1da3790 0x1fb3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:6834/790293012,v1:172.21.15.46:6835/790293012] conn(0x1fb3b40 0x203f4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1d58ae0 0x1fb30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 --2- 172.21.15.46:0/442546611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1d58710 0x1fb64b0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdccff9700 0 client.8976 ms_handle_reset on v2:172.21.15.46:6834/790293012 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 >> 172.21.15.46:0/442546611 conn(0x1d58eb0 msgr2=0x1f91ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 shutdown_connections 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 1 -- 172.21.15.46:0/442546611 wait complete. 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf6b68740 20 client.8976 trim_cache size 0 max 16384 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T12:33:51.849 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8796, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 11.053118497, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 76, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 153, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 153, 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 16.987949196999999 2023-10-06T12:33:51.850 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 msgr2=0x7fcdd0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 0x7fcdd0062b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80093f0 tx=0x7fcdd8017000 comp rx=0 tx=0).stop 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 msgr2=0x7fcdf0159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fcde000ea20 tx=0x7fcde0009220 comp rx=0 tx=0).stop 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 shutdown_connections 2023-10-06T12:33:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcdd0060670 0x7fcdd0062b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcdf015af30 0x7fcdf00c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdf015ab60 0x7fcdf00ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 --2- 172.21.15.46:0/3396770836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00ad6c0 0x7fcdf0159ec0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 >> 172.21.15.46:0/3396770836 conn(0x7fcdf00cbe30 msgr2=0x7fcdf00c6120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 shutdown_connections 2023-10-06T12:33:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:51.844+0000 7fcdf4878700 1 -- 172.21.15.46:0/3396770836 wait complete. 2023-10-06T12:33:51.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T12:33:52.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 Processor -- start 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 -- start start 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9d80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d80c7bf0 con 0x7fa9d8087740 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d806d8e0 con 0x7fa9d80c69b0 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9df6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80c6fe0 con 0x7fa9d8087b10 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9dd48c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9d8087740 0x7fa9d80c8080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56692/0 (socket says 172.21.15.46:56692) 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9dd48c700 1 -- 172.21.15.46:0/125258588 learned_addr learned my addr 172.21.15.46:0/125258588 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9dcc8b700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9d80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776297481 0 0) 0x7fa9d80c7bf0 con 0x7fa9d8087740 2023-10-06T12:33:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c4003570 con 0x7fa9d8087740 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.040+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509461057 0 0) 0x7fa9c4003570 con 0x7fa9d8087740 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 msgr2=0x7fa9d80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9d80c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 >> v1:172.21.15.132:6789/0 conn(0x7fa9d80c69b0 legacy=0x7fa9d80caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80c72c0 con 0x7fa9d8087740 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fa9d4001470 con 0x7fa9d8087740 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 >> v1:172.21.15.46:6789/0 conn(0x7fa9d8087740 legacy=0x7fa9d80c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c4003a90 0x7fa9c4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4004d50 0x7fa9c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80c6fe0 con 0x7fa9c4003a90 2023-10-06T12:33:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4010000 con 0x7fa9c4004d50 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4010170 con 0x7fa9d8087b10 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c4003a90 0x7fa9c4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dcc8b700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9ddc8d700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4004d50 0x7fa9c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 -- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 msgr2=0x7fa9c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d8087b10 0x7fa9c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 -- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4004d50 msgr2=0x7fa9c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4004d50 0x7fa9c40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 -- 172.21.15.46:0/125258588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c4003570 con 0x7fa9c4003a90 2023-10-06T12:33:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c4003a90 0x7fa9c4003e60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c2d0 tx=0x7fa9d4001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f029f794e82948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400cc00 con 0x7fa9c4003a90 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d400cd80 con 0x7fa9c4003a90 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9cffff700 1 -- 172.21.15.46:0/125258588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4005b70 con 0x7fa9c4003a90 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9df6f0700 1 -- 172.21.15.46:0/125258588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c4003a90 msgr2=0x7fa9c4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/125258588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c4003a90 0x7fa9c4003e60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c2d0 tx=0x7fa9d4001e30 comp rx=0 tx=0).stop 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.200+0000 7fa9ddc8d700 1 -- 172.21.15.46:0/125258588 reap_dead start 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- 172.21.15.46:0/125258588 shutdown_connections 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- 172.21.15.46:0/125258588 >> 172.21.15.46:0/125258588 conn(0x7fa9d80cc860 msgr2=0x7fa9d80c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- 172.21.15.46:0/125258588 shutdown_connections 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- 172.21.15.46:0/125258588 wait complete. 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 Processor -- start 2023-10-06T12:33:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- start start 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4003a90 0x7fa9d8162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d80af5c0 0x7fa9d80b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80c4f40 con 0x7fa9d80b1680 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8162aa0 con 0x7fa9c4003a90 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9df6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80c5190 con 0x7fa9d80af5c0 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.204+0000 7fa9ddc8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38770/0 (socket says 172.21.15.46:38770) 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 -- 172.21.15.46:0/2353650574 learned_addr learned my addr 172.21.15.46:0/2353650574 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9dcc8b700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d80af5c0 0x7fa9d80b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4003a90 0x7fa9d8162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 -- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d80af5c0 msgr2=0x7fa9d80b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d80af5c0 0x7fa9d80b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 -- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4003a90 msgr2=0x7fa9d8162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4003a90 0x7fa9d8162560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 -- 172.21.15.46:0/2353650574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80c5470 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9ddc8d700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ccc80 tx=0x7fa9d0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000dec0 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000b7a0 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000bb60 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d80c5750 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d80c5d10 con 0x7fa9d80b1680 2023-10-06T12:33:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.208+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d80660f0 con 0x7fa9d80b1680 2023-10-06T12:33:52.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.212+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa9d0003050 con 0x7fa9d80b1680 2023-10-06T12:33:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.212+0000 7fa9cd7fa700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 0x7fa9bc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.212+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fa9d007cd50 con 0x7fa9d80b1680 2023-10-06T12:33:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.212+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 0x7fa9bc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.212+0000 7fa9dd48c700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 0x7fa9bc062b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007cb0 tx=0x7fa9d4015000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.215+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa9d004cd20 con 0x7fa9d80b1680 2023-10-06T12:33:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.431+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9d80c9070 con 0x7fa9d80b1680 2023-10-06T12:33:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9cd7fa700 1 -- 172.21.15.46:0/2353650574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9d0050340 con 0x7fa9d80b1680 2023-10-06T12:33:52.438 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T12:33:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 msgr2=0x7fa9bc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 0x7fa9bc062b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007cb0 tx=0x7fa9d4015000 comp rx=0 tx=0).stop 2023-10-06T12:33:52.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 msgr2=0x7fa9d80ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ccc80 tx=0x7fa9d0009040 comp rx=0 tx=0).stop 2023-10-06T12:33:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 shutdown_connections 2023-10-06T12:33:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa9bc0606a0 0x7fa9bc062b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9d80b1680 0x7fa9d80ac870 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9d80af5c0 0x7fa9d80b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 --2- 172.21.15.46:0/2353650574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c4003a90 0x7fa9d8162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 >> 172.21.15.46:0/2353650574 conn(0x7fa9d80cc860 msgr2=0x7fa9d8163320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 shutdown_connections 2023-10-06T12:33:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:52.435+0000 7fa9df6f0700 1 -- 172.21.15.46:0/2353650574 wait complete. 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T12:33:52.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T12:33:52.461 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T12:33:52.461 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:33:52.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.461 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T12:33:52.479 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T12:33:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T12:33:52.479 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T12:33:52.480 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 37898sec preferred_lft 37898sec 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T12:33:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:33:52.482 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T12:33:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T12:33:52.483 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:33:52.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.484 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T12:33:52.560 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T12:33:52.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.563 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T12:33:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T12:33:52.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.631 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T12:33:52.632 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T12:33:52.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T12:33:52.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T12:33:52.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T12:33:52.632 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T12:33:52.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.777 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:33:52.795 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T12:33:52.795 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T12:33:52.796 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:33:52.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:52.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T12:33:52.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:33:52.878 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T12:33:52.945 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:33:52.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:33:52.945 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:33:53.006 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T12:33:53.006 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T12:33:53.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:33:53.135 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:33:53.136 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:33:53.136 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:33:53.159 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:33:53.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:33:53.160 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:33:53.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:33:53.160 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:33:53.175 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:33:53.175 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:33:53.175 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:33:53.176 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:33:53.176 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:33:53.176 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T12:33:53.176 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T12:33:53.176 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T12:33:53.181 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T12:33:53.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:33:53.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:33:53.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T12:33:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.427+0000 7ff0b3dc5700 1 Processor -- start 2023-10-06T12:33:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.427+0000 7ff0b3dc5700 1 -- start start 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac0c7b50 0x7ff0ac0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff0ac15cde0 0x7ff0ac15d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c8500 con 0x7ff0ac156c80 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c6470 con 0x7ff0ac15cde0 2023-10-06T12:33:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b3dc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c7760 con 0x7ff0ac0c7b50 2023-10-06T12:33:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44660/0 (socket says 172.21.15.46:44660) 2023-10-06T12:33:53.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 -- 172.21.15.46:0/1595788970 learned_addr learned my addr 172.21.15.46:0/1595788970 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:53.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1360700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac0c7b50 0x7ff0ac0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b2362700 1 --2- 172.21.15.46:0/1595788970 >> v2:172.21.15.132:3300/0 conn(0x7ff0ac15cde0 0x7ff0ac15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 -- 172.21.15.46:0/1595788970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac0c7b50 msgr2=0x7ff0ac0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac0c7b50 0x7ff0ac0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 -- 172.21.15.46:0/1595788970 >> v2:172.21.15.132:3300/0 conn(0x7ff0ac15cde0 msgr2=0x7ff0ac15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/1595788970 >> v2:172.21.15.132:3300/0 conn(0x7ff0ac15cde0 0x7ff0ac15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 -- 172.21.15.46:0/1595788970 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0ca710 con 0x7ff0ac156c80 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac06bf00 tx=0x7ff0a8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b7ed8133b56627 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b0b5f700 1 -- 172.21.15.46:0/1595788970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8005c10 con 0x7ff0ac156c80 2023-10-06T12:33:53.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b0b5f700 1 -- 172.21.15.46:0/1595788970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8005d70 con 0x7ff0ac156c80 2023-10-06T12:33:53.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.431+0000 7ff0b0b5f700 1 -- 172.21.15.46:0/1595788970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8006260 con 0x7ff0ac156c80 2023-10-06T12:33:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/1595788970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 msgr2=0x7ff0ac0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac06bf00 tx=0x7ff0a8009220 comp rx=0 tx=0).stop 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/1595788970 shutdown_connections 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/1595788970 >> v2:172.21.15.132:3300/0 conn(0x7ff0ac15cde0 0x7ff0ac15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac0c7b50 0x7ff0ac0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/1595788970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0ca1d0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/1595788970 >> 172.21.15.46:0/1595788970 conn(0x7ff0ac0cbe30 msgr2=0x7ff0ac0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/1595788970 shutdown_connections 2023-10-06T12:33:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.435+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/1595788970 wait complete. 2023-10-06T12:33:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 Processor -- start 2023-10-06T12:33:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 -- start start 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0ac0c7b50 0x7ff0ac155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac15cde0 0x7ff0ac150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0ac9c0 con 0x7ff0ac156c80 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0ad6a0 con 0x7ff0ac0c7b50 2023-10-06T12:33:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b3dc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0acd70 con 0x7ff0ac15cde0 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44666/0 (socket says 172.21.15.46:44666) 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 -- 172.21.15.46:0/2740354485 learned_addr learned my addr 172.21.15.46:0/2740354485 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0ac0c7b50 0x7ff0ac155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b2362700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac15cde0 0x7ff0ac150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 -- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac15cde0 msgr2=0x7ff0ac150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac15cde0 0x7ff0ac150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 -- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0ac0c7b50 msgr2=0x7ff0ac155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0ac0c7b50 0x7ff0ac155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.439+0000 7ff0b1360700 1 -- 172.21.15.46:0/2740354485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8009480 con 0x7ff0ac156c80 2023-10-06T12:33:53.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0b1360700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e930 tx=0x7ff09c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c013680 con 0x7ff0ac156c80 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c0137e0 con 0x7ff0ac156c80 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c013b80 con 0x7ff0ac156c80 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac0ad050 con 0x7ff0ac156c80 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac151320 con 0x7ff0ac156c80 2023-10-06T12:33:53.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.443+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac0ad850 con 0x7ff0ac156c80 2023-10-06T12:33:53.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.447+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff09c00c3e0 con 0x7ff0ac156c80 2023-10-06T12:33:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.447+0000 7ff0a1ffb700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 0x7ff090062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.447+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7ff09c07ce50 con 0x7ff0ac156c80 2023-10-06T12:33:53.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.447+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 0x7ff090062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.447+0000 7ff0b1b61700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 0x7ff090062b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009260 tx=0x7ff0a8016000 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.455+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff0ac0ad850 con 0x7ff0ac156c80 2023-10-06T12:33:53.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.647+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0ac0ad810 con 0x7ff090060670 2023-10-06T12:33:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.651+0000 7ff0a1ffb700 1 -- 172.21.15.46:0/2740354485 <== mgr.8365 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff0ac0ad810 con 0x7ff090060670 2023-10-06T12:33:53.657 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:33:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:33:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T12:33:53.657 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:33:53.657 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:33:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 msgr2=0x7ff090062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 0x7ff090062b20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009260 tx=0x7ff0a8016000 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 msgr2=0x7ff0ac0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e930 tx=0x7ff09c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 shutdown_connections 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff090060670 0x7ff090062b20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ac15cde0 0x7ff0ac150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0ac156c80 0x7ff0ac0b1590 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 --2- 172.21.15.46:0/2740354485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0ac0c7b50 0x7ff0ac155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 >> 172.21.15.46:0/2740354485 conn(0x7ff0ac0cbe30 msgr2=0x7ff0ac15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 shutdown_connections 2023-10-06T12:33:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.655+0000 7ff0b3dc5700 1 -- 172.21.15.46:0/2740354485 wait complete. 2023-10-06T12:33:53.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T12:33:53.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.843+0000 7f93a5265700 1 Processor -- start 2023-10-06T12:33:53.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.843+0000 7f93a5265700 1 -- start start 2023-10-06T12:33:53.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93a5265700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f93a00b23f0 0x7f93a00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93a5265700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 0x7f93a00c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93a5265700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00cb6e0 con 0x7f93a00ccc00 2023-10-06T12:33:53.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93a5265700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a00cc780 con 0x7f93a00b23f0 2023-10-06T12:33:53.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93a5265700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00cb9c0 con 0x7f93a00cebe0 2023-10-06T12:33:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93967fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f93a00ccc00 0x7f93a00ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42358/0 (socket says 172.21.15.46:42358) 2023-10-06T12:33:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f93967fc700 1 -- 172.21.15.46:0/1508398602 learned_addr learned my addr 172.21.15.46:0/1508398602 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939effd700 1 --2- 172.21.15.46:0/1508398602 >> v2:172.21.15.132:3300/0 conn(0x7f93a00b23f0 0x7f93a00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 --2- 172.21.15.46:0/1508398602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 0x7f93a00c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523852217 0 0) 0x7f93a00cb6e0 con 0x7f93a00ccc00 2023-10-06T12:33:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938c003570 con 0x7f93a00ccc00 2023-10-06T12:33:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 -- 172.21.15.46:0/1508398602 >> v2:172.21.15.132:3300/0 conn(0x7f93a00b23f0 msgr2=0x7f93a00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 --2- 172.21.15.46:0/1508398602 >> v2:172.21.15.132:3300/0 conn(0x7f93a00b23f0 0x7f93a00cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 -- 172.21.15.46:0/1508398602 >> v1:172.21.15.46:6789/0 conn(0x7f93a00ccc00 legacy=0x7f93a00ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772349628 0 0) 0x7f938c003570 con 0x7f93a00ccc00 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 -- 172.21.15.46:0/1508398602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a00cbca0 con 0x7f93a00cebe0 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.847+0000 7f939f7fe700 1 --2- 172.21.15.46:0/1508398602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 0x7f93a00c4fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93980098e0 tx=0x7f93980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbe0e9e666f1944 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388001490 con 0x7f93a00cebe0 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398011420 con 0x7f93a00cebe0 2023-10-06T12:33:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f939e7fc700 1 -- 172.21.15.46:0/1508398602 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980117c0 con 0x7f93a00cebe0 2023-10-06T12:33:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- 172.21.15.46:0/1508398602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 msgr2=0x7f93a00c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 --2- 172.21.15.46:0/1508398602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 0x7f93a00c4fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93980098e0 tx=0x7f93980091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- 172.21.15.46:0/1508398602 shutdown_connections 2023-10-06T12:33:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 --2- 172.21.15.46:0/1508398602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00cebe0 0x7f93a00c4fa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 --2- 172.21.15.46:0/1508398602 >> v2:172.21.15.132:3300/0 conn(0x7f93a00b23f0 0x7f93a00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- 172.21.15.46:0/1508398602 >> 172.21.15.46:0/1508398602 conn(0x7f93a0099440 msgr2=0x7f93a006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- 172.21.15.46:0/1508398602 shutdown_connections 2023-10-06T12:33:53.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- 172.21.15.46:0/1508398602 wait complete. 2023-10-06T12:33:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 Processor -- start 2023-10-06T12:33:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.851+0000 7f93a5265700 1 -- start start 2023-10-06T12:33:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00b23f0 0x7f93a0156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 0x7f93a01694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015ae40 con 0x7f93a00cebe0 2023-10-06T12:33:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015bca0 con 0x7f93a00ccc00 2023-10-06T12:33:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015b090 con 0x7f93a00b23f0 2023-10-06T12:33:53.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44680/0 (socket says 172.21.15.46:44680) 2023-10-06T12:33:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 -- 172.21.15.46:0/802107559 learned_addr learned my addr 172.21.15.46:0/802107559 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939effd700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00b23f0 0x7f93a0156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93967fc700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 0x7f93a01694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 -- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00b23f0 msgr2=0x7f93a0156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00b23f0 0x7f93a0156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 -- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 msgr2=0x7f93a01694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 0x7f93a01694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 -- 172.21.15.46:0/802107559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398009420 con 0x7f93a00cebe0 2023-10-06T12:33:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93967fc700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 0x7f93a01694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f939f7fe700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9398002000 tx=0x7f9398016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939801cd00 con 0x7f93a00cebe0 2023-10-06T12:33:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939801ce60 con 0x7f93a00cebe0 2023-10-06T12:33:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a015b370 con 0x7f93a00cebe0 2023-10-06T12:33:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398025660 con 0x7f93a00cebe0 2023-10-06T12:33:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.855+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a015b930 con 0x7f93a00cebe0 2023-10-06T12:33:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.859+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a015be30 con 0x7f93a00cebe0 2023-10-06T12:33:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.859+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 75) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f93980193f0 con 0x7f93a00cebe0 2023-10-06T12:33:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.863+0000 7f93977fe700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 0x7f9380062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.863+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f939807e140 con 0x7f93a00cebe0 2023-10-06T12:33:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.863+0000 7f939effd700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 0x7f9380062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.863+0000 7f939effd700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 0x7f9380062b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f939000f510 tx=0x7f939000b040 comp rx=0 tx=0).ready entity=mgr.8365 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:53.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:53.871+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f939804e2a0 con 0x7f93a00cebe0 2023-10-06T12:33:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.083+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f93a00660f0 con 0x7f93a00cebe0 2023-10-06T12:33:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.291+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 76) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f939804e670 con 0x7f93a00cebe0 2023-10-06T12:33:54.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93977fe700 1 -- 172.21.15.46:0/802107559 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v76) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9398063800 con 0x7f93a00cebe0 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 msgr2=0x7f9380062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 0x7f9380062b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f939000f510 tx=0x7f939000b040 comp rx=0 tx=0).stop 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 msgr2=0x7f93a00c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9398002000 tx=0x7f9398016000 comp rx=0 tx=0).stop 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 shutdown_connections 2023-10-06T12:33:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9380060670 0x7f9380062b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93a00cebe0 0x7f93a00c9de0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93a00ccc00 0x7f93a01694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 --2- 172.21.15.46:0/802107559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93a00b23f0 0x7f93a0156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.295+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 >> 172.21.15.46:0/802107559 conn(0x7f93a0099440 msgr2=0x7f93a0158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.299+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 shutdown_connections 2023-10-06T12:33:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.299+0000 7f93a5265700 1 -- 172.21.15.46:0/802107559 wait complete. 2023-10-06T12:33:54.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T12:33:54.471 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:54.467+0000 7f7a5395b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:33:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.483+0000 7f137a1cc700 1 Processor -- start 2023-10-06T12:33:54.489 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:54.483+0000 7f9eeab6a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:33:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 -- start start 2023-10-06T12:33:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f13740abfc0 0x7f13740ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13740aca00 con 0x7f13740ae890 2023-10-06T12:33:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740ada20 con 0x7f13740abfc0 2023-10-06T12:33:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f137a1cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740633b0 con 0x7f137406f1d0 2023-10-06T12:33:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f13799cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f13799cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:56304/0 (socket says 172.21.15.46:56304) 2023-10-06T12:33:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f13799cb700 1 -- 172.21.15.46:0/1325948391 learned_addr learned my addr 172.21.15.46:0/1325948391 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.487+0000 7f13791ca700 1 --2- 172.21.15.46:0/1325948391 >> v2:172.21.15.132:3300/0 conn(0x7f13740abfc0 0x7f13740ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f13799cb700 1 -- 172.21.15.46:0/1325948391 >> v2:172.21.15.132:3300/0 conn(0x7f13740abfc0 msgr2=0x7f13740ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f13799cb700 1 --2- 172.21.15.46:0/1325948391 >> v2:172.21.15.132:3300/0 conn(0x7f13740abfc0 0x7f13740ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f13799cb700 1 -- 172.21.15.46:0/1325948391 >> v1:172.21.15.132:6789/0 conn(0x7f13740ae890 legacy=0x7f1374096140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f13799cb700 1 -- 172.21.15.46:0/1325948391 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374062df0 con 0x7f137406f1d0 2023-10-06T12:33:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f13799cb700 1 --2- 172.21.15.46:0/1325948391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f136c0099b0 tx=0x7f136c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e78b0bc1154f37e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:54.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f136bfff700 1 -- 172.21.15.46:0/1325948391 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c009ea0 con 0x7f137406f1d0 2023-10-06T12:33:54.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f136bfff700 1 -- 172.21.15.46:0/1325948391 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c00b8f0 con 0x7f137406f1d0 2023-10-06T12:33:54.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f136bfff700 1 -- 172.21.15.46:0/1325948391 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c00bc90 con 0x7f137406f1d0 2023-10-06T12:33:54.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 -- 172.21.15.46:0/1325948391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 msgr2=0x7f13740ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1325948391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f136c0099b0 tx=0x7f136c009250 comp rx=0 tx=0).stop 2023-10-06T12:33:54.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 -- 172.21.15.46:0/1325948391 shutdown_connections 2023-10-06T12:33:54.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1325948391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f137406f1d0 0x7f13740ace90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1325948391 >> v2:172.21.15.132:3300/0 conn(0x7f13740abfc0 0x7f13740ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 -- 172.21.15.46:0/1325948391 >> 172.21.15.46:0/1325948391 conn(0x7f13740afd10 msgr2=0x7f137406e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:54.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 -- 172.21.15.46:0/1325948391 shutdown_connections 2023-10-06T12:33:54.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.491+0000 7f137a1cc700 1 -- 172.21.15.46:0/1325948391 wait complete. 2023-10-06T12:33:54.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.495+0000 7f137a1cc700 1 Processor -- start 2023-10-06T12:33:54.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.495+0000 7f137a1cc700 1 -- start start 2023-10-06T12:33:54.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13740abfc0 0x7f13740a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 0x7f137414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740a8e40 con 0x7f137406f1d0 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740a9120 con 0x7f13740ae890 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13740a9400 con 0x7f13740abfc0 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44700/0 (socket says 172.21.15.46:44700) 2023-10-06T12:33:54.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 -- 172.21.15.46:0/1946272257 learned_addr learned my addr 172.21.15.46:0/1946272257 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:54.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13789c9700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13740abfc0 0x7f13740a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13799cb700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 0x7f137414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 -- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13740abfc0 msgr2=0x7f13740a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13740abfc0 0x7f13740a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 -- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 msgr2=0x7f137414deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 0x7f137414deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 -- 172.21.15.46:0/1946272257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0094f0 con 0x7f137406f1d0 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13799cb700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 0x7f137414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:33:54.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13791ca700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1370009ef0 tx=0x7f137000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000ccb0 con 0x7f137406f1d0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137000f040 con 0x7f137406f1d0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13740a96e0 con 0x7f137406f1d0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700146b0 con 0x7f137406f1d0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.499+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13740a8b00 con 0x7f137406f1d0 2023-10-06T12:33:54.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.503+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13740a9b60 con 0x7f137406f1d0 2023-10-06T12:33:54.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.503+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 76) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f1370014a20 con 0x7f137406f1d0 2023-10-06T12:33:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.507+0000 7f13697fa700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 0x7f1358062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.507+0000 7f13789c9700 1 -- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 msgr2=0x7f1358062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.507+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f137007d8c0 con 0x7f137406f1d0 2023-10-06T12:33:54.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.507+0000 7f13789c9700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 0x7f1358062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:54.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.515+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f137004d890 con 0x7f137406f1d0 2023-10-06T12:33:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.691+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1374064590 con 0x7f137406f1d0 2023-10-06T12:33:54.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.699+0000 7f13697fa700 1 -- 172.21.15.46:0/1946272257 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1370050eb0 con 0x7f137406f1d0 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 msgr2=0x7f1358062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 0x7f1358062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 msgr2=0x7f1374099ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1370009ef0 tx=0x7f137000b040 comp rx=0 tx=0).stop 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 shutdown_connections 2023-10-06T12:33:54.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f13580606c0 0x7f1358062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13740ae890 0x7f137414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13740abfc0 0x7f13740a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 --2- 172.21.15.46:0/1946272257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f137406f1d0 0x7f1374099ad0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 >> 172.21.15.46:0/1946272257 conn(0x7f13740afd10 msgr2=0x7f1374068660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 shutdown_connections 2023-10-06T12:33:54.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.703+0000 7f137a1cc700 1 -- 172.21.15.46:0/1946272257 wait complete. 2023-10-06T12:33:54.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T12:33:54.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 Processor -- start 2023-10-06T12:33:54.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 -- start start 2023-10-06T12:33:54.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f467c069c10 0x7f467c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 0x7f467c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0b1b60 con 0x7f467c068160 2023-10-06T12:33:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c1597e0 con 0x7f467c069c10 2023-10-06T12:33:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4680945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c15a5c0 con 0x7f467c0b1710 2023-10-06T12:33:54.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f467c069c10 0x7f467c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4673fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f467c068160 0x7f467c0b1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40216/0 (socket says 172.21.15.46:40216) 2023-10-06T12:33:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f4673fff700 1 -- 172.21.15.46:0/2571294720 learned_addr learned my addr 172.21.15.46:0/2571294720 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:33:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467ad9d700 1 --2- 172.21.15.46:0/2571294720 >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 0x7f467c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467a59c700 1 -- 172.21.15.46:0/2571294720 >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 msgr2=0x7f467c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467a59c700 1 --2- 172.21.15.46:0/2571294720 >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 0x7f467c0aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467a59c700 1 -- 172.21.15.46:0/2571294720 >> v1:172.21.15.132:6789/0 conn(0x7f467c068160 legacy=0x7f467c0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467a59c700 1 -- 172.21.15.46:0/2571294720 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c1599f0 con 0x7f467c069c10 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.895+0000 7f467ad9d700 1 --2- 172.21.15.46:0/2571294720 >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 0x7f467c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f467a59c700 1 --2- 172.21.15.46:0/2571294720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c069c10 0x7f467c0af9e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f467c159920 tx=0x7f466c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2a18f298efff39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4679d9b700 1 -- 172.21.15.46:0/2571294720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c00b680 con 0x7f467c069c10 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4679d9b700 1 -- 172.21.15.46:0/2571294720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c00b7e0 con 0x7f467c069c10 2023-10-06T12:33:54.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4679d9b700 1 -- 172.21.15.46:0/2571294720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c00bba0 con 0x7f467c069c10 2023-10-06T12:33:54.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 -- 172.21.15.46:0/2571294720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c069c10 msgr2=0x7f467c0af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 --2- 172.21.15.46:0/2571294720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c069c10 0x7f467c0af9e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f467c159920 tx=0x7f466c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 -- 172.21.15.46:0/2571294720 shutdown_connections 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 --2- 172.21.15.46:0/2571294720 >> v2:172.21.15.132:3300/0 conn(0x7f467c0b1710 0x7f467c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 --2- 172.21.15.46:0/2571294720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c069c10 0x7f467c0af9e0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 -- 172.21.15.46:0/2571294720 >> 172.21.15.46:0/2571294720 conn(0x7f467c06ae60 msgr2=0x7f467c06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 -- 172.21.15.46:0/2571294720 shutdown_connections 2023-10-06T12:33:54.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 -- 172.21.15.46:0/2571294720 wait complete. 2023-10-06T12:33:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.899+0000 7f4680945700 1 Processor -- start 2023-10-06T12:33:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- start start 2023-10-06T12:33:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 0x7f467c16d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f467c0b1710 0x7f467c12f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c13e6f0 con 0x7f467c068160 2023-10-06T12:33:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c13e9d0 con 0x7f467c0b1710 2023-10-06T12:33:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c13ecb0 con 0x7f467c069c10 2023-10-06T12:33:54.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44730/0 (socket says 172.21.15.46:44730) 2023-10-06T12:33:54.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 -- 172.21.15.46:0/2432043252 learned_addr learned my addr 172.21.15.46:0/2432043252 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:54.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467ad9d700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f467c0b1710 0x7f467c12f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4673fff700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 0x7f467c16d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 -- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 msgr2=0x7f467c16d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 0x7f467c16d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 -- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f467c0b1710 msgr2=0x7f467c12f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f467c0b1710 0x7f467c12f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 -- 172.21.15.46:0/2432043252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009420 con 0x7f467c068160 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4673fff700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 0x7f467c16d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f467a59c700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f467c15df10 tx=0x7f466c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c011e30 con 0x7f467c068160 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c01b670 con 0x7f467c068160 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c13ef90 con 0x7f467c068160 2023-10-06T12:33:54.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c01ba30 con 0x7f467c068160 2023-10-06T12:33:54.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.903+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c1583d0 con 0x7f467c068160 2023-10-06T12:33:54.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.907+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c06da40 con 0x7f467c068160 2023-10-06T12:33:54.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.907+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 76) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f466c019400 con 0x7f467c068160 2023-10-06T12:33:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.911+0000 7f46727fc700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 0x7f465c062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.911+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f466c07e9b0 con 0x7f467c068160 2023-10-06T12:33:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.911+0000 7f4673fff700 1 -- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 msgr2=0x7f465c062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.911+0000 7f4673fff700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 0x7f465c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:54.915+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f466c04ea90 con 0x7f467c068160 2023-10-06T12:33:55.007 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.002+0000 7f7a5395b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:33:55.065 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.059+0000 7f9eeab6a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:33:55.077 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.074+0000 7f7a5395b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:33:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.091+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f467c159c90 con 0x7f467c068160 2023-10-06T12:33:55.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.099+0000 7f46727fc700 1 -- 172.21.15.46:0/2432043252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f466c0520b0 con 0x7f467c068160 2023-10-06T12:33:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 msgr2=0x7f465c062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 0x7f465c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 msgr2=0x7f467c13e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f467c15df10 tx=0x7f466c016000 comp rx=0 tx=0).stop 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 shutdown_connections 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f465c0606c0 0x7f465c062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f467c0b1710 0x7f467c12f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f467c069c10 0x7f467c16d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 --2- 172.21.15.46:0/2432043252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f467c068160 0x7f467c13e110 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 >> 172.21.15.46:0/2432043252 conn(0x7f467c06ae60 msgr2=0x7f467c13b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:55.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 shutdown_connections 2023-10-06T12:33:55.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.103+0000 7f4680945700 1 -- 172.21.15.46:0/2432043252 wait complete. 2023-10-06T12:33:55.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:33:55.128 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T12:33:55.144 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.139+0000 7f9eeab6a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:33:55.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-10-06T12:33:55.215 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.210+0000 7f7a5395b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:55.291 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.287+0000 7f9eeab6a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:55.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 Processor -- start 2023-10-06T12:33:55.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 -- start start 2023-10-06T12:33:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efde81591f0 0x7efde80cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80cf360 con 0x7efde81595c0 2023-10-06T12:33:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde815dd60 con 0x7efde81591f0 2023-10-06T12:33:55.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.367+0000 7efdec82d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde80cea30 con 0x7efde80cc500 2023-10-06T12:33:55.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:55.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:56324/0 (socket says 172.21.15.46:56324) 2023-10-06T12:33:55.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 -- 172.21.15.46:0/338511528 learned_addr learned my addr 172.21.15.46:0/338511528 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:55.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde6d9d700 1 --2- 172.21.15.46:0/338511528 >> v2:172.21.15.132:3300/0 conn(0x7efde81591f0 0x7efde80cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 -- 172.21.15.46:0/338511528 >> v2:172.21.15.132:3300/0 conn(0x7efde81591f0 msgr2=0x7efde80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 --2- 172.21.15.46:0/338511528 >> v2:172.21.15.132:3300/0 conn(0x7efde81591f0 0x7efde80cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 -- 172.21.15.46:0/338511528 >> v1:172.21.15.132:6789/0 conn(0x7efde81595c0 legacy=0x7efde80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 -- 172.21.15.46:0/338511528 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80ced10 con 0x7efde80cc500 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde759e700 1 --2- 172.21.15.46:0/338511528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efde0009f20 tx=0x7efde0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdc666f7e2213a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde5d9b700 1 -- 172.21.15.46:0/338511528 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000e070 con 0x7efde80cc500 2023-10-06T12:33:55.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde5d9b700 1 -- 172.21.15.46:0/338511528 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde000ba60 con 0x7efde80cc500 2023-10-06T12:33:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efde5d9b700 1 -- 172.21.15.46:0/338511528 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00092e0 con 0x7efde80cc500 2023-10-06T12:33:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 -- 172.21.15.46:0/338511528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 msgr2=0x7efde815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 --2- 172.21.15.46:0/338511528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efde0009f20 tx=0x7efde0009840 comp rx=0 tx=0).stop 2023-10-06T12:33:55.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 -- 172.21.15.46:0/338511528 shutdown_connections 2023-10-06T12:33:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 --2- 172.21.15.46:0/338511528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde815ceb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 --2- 172.21.15.46:0/338511528 >> v2:172.21.15.132:3300/0 conn(0x7efde81591f0 0x7efde80cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 -- 172.21.15.46:0/338511528 >> 172.21.15.46:0/338511528 conn(0x7efde806ef60 msgr2=0x7efde80d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:33:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 -- 172.21.15.46:0/338511528 shutdown_connections 2023-10-06T12:33:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.371+0000 7efdec82d700 1 -- 172.21.15.46:0/338511528 wait complete. 2023-10-06T12:33:55.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 Processor -- start 2023-10-06T12:33:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 -- start start 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efde81591f0 0x7efde8169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80afce0 con 0x7efde814fe00 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80af9b0 con 0x7efde81591f0 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efdec82d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde8169fc0 con 0x7efde80cc500 2023-10-06T12:33:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efde759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efde6d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efde759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44744/0 (socket says 172.21.15.46:44744) 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efde759e700 1 -- 172.21.15.46:0/1342434541 learned_addr learned my addr 172.21.15.46:0/1342434541 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.375+0000 7efde659c700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efde81591f0 0x7efde8169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 msgr2=0x7efde80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde80b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efde81591f0 msgr2=0x7efde8169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:33:55.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efde81591f0 0x7efde8169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:55.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0009ae0 con 0x7efde814fe00 2023-10-06T12:33:55.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:33:55.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efde759e700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7efde000b610 tx=0x7efde0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:33:55.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00032f0 con 0x7efde814fe00 2023-10-06T12:33:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde0008d20 con 0x7efde814fe00 2023-10-06T12:33:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde816a2a0 con 0x7efde814fe00 2023-10-06T12:33:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde001c620 con 0x7efde814fe00 2023-10-06T12:33:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde80b26a0 con 0x7efde814fe00 2023-10-06T12:33:55.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.379+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde80afe50 con 0x7efde814fe00 2023-10-06T12:33:55.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.383+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 76) v1 ==== 80078+0+0 (secure 0 0 0) 0x7efde0003040 con 0x7efde814fe00 2023-10-06T12:33:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.383+0000 7efdceffd700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:33:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.383+0000 7efde6d9d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.383+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6574+0+0 (secure 0 0 0) 0x7efde007e4b0 con 0x7efde814fe00 2023-10-06T12:33:55.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.383+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:33:55.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.387+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efde004e480 con 0x7efde814fe00 2023-10-06T12:33:55.463 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.458+0000 7f7a5395b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:33:55.531 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.526+0000 7f7a5395b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:55.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.583+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7efde8064590 con 0x7efdd40606f0 2023-10-06T12:33:55.588 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.583+0000 7f9eeab6a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:33:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.587+0000 7efde6d9d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:55.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.587+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:33:55.665 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.659+0000 7f9eeab6a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:55.734 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.730+0000 7f7a5395b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:33:55.812 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.810+0000 7f7a5395b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:33:55.891 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.887+0000 7f9eeab6a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:33:55.892 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:55.890+0000 7f7a5395b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:33:55.979 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:55.975+0000 7f9eeab6a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:33:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.991+0000 7efde6d9d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:55.991+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T12:33:56.071 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:56.067+0000 7f9eeab6a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:33:56.260 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:56.258+0000 7f7a5395b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:33:56.420 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:56.418+0000 7f7a5395b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:33:56.488 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:56.483+0000 7f9eeab6a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:33:56.667 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:56.663+0000 7f9eeab6a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:33:56.759 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:56.754+0000 7f7a5395b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:33:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:56.795+0000 7efde6d9d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:56.795+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T12:33:56.967 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:56.962+0000 7f7a5395b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:57.049 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.043+0000 7f9eeab6a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:33:57.053 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.050+0000 7f7a5395b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:33:57.129 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.126+0000 7f7a5395b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:33:57.194 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.190+0000 7f7a5395b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:33:57.282 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.279+0000 7f9eeab6a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:33:57.381 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.375+0000 7f9eeab6a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:33:57.409 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.406+0000 7f7a5395b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:57.459 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.455+0000 7f9eeab6a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:33:57.487 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.482+0000 7f7a5395b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:33:57.542 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.539+0000 7f9eeab6a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:33:57.570 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.566+0000 7f7a5395b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:33:57.752 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.750+0000 7f7a5395b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:33:57.776 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.771+0000 7f9eeab6a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:33:57.834 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:57.830+0000 7f7a5395b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:33:57.863 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.859+0000 7f9eeab6a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:33:57.953 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:57.951+0000 7f9eeab6a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:33:58.162 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:58.159+0000 7f9eeab6a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:33:58.248 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:58.243+0000 7f9eeab6a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:33:58.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:58.399+0000 7efde6d9d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:33:58.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:58.399+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T12:33:58.528 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:33:58.526+0000 7f7a5395b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:33:58.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:58.591+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 77) v1 ==== 80078+0+0 (secure 0 0 0) 0x7efde004e850 con 0x7efde814fe00 2023-10-06T12:33:59.005 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:58.999+0000 7f9eeab6a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:33:59.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:59.019+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 78) v1 ==== 79687+0+0 (secure 0 0 0) 0x7efde0083d40 con 0x7efde814fe00 2023-10-06T12:33:59.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:59.019+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:33:59.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:33:59.019+0000 7efdceffd700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:33:59.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9eac692700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9eac692700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9eac692700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9eac692700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9eac692700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9ea968c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9ea968c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9ea968c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9ea968c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:33:59.221 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:33:59.215+0000 7f9ea968c700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.031+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 79) v1 ==== 79814+0+0 (secure 0 0 0) 0x7efde004f120 con 0x7efde814fe00 2023-10-06T12:34:00.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.031+0000 7efdceffd700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.035+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7efde8064590 con 0x7efdd40606f0 2023-10-06T12:34:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.039+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.039+0000 7efde6d9d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009a50 tx=0x7efdd80091a0 comp rx=0 tx=0).ready entity=mgr.9009 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:00.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.147+0000 7efdceffd700 1 -- 172.21.15.46:0/1342434541 <== mgr.9009 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efde8064590 con 0x7efdd40606f0 2023-10-06T12:34:00.334 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.151+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 msgr2=0x7efdd4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.151+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009a50 tx=0x7efdd80091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.151+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 msgr2=0x7efde80b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.151+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7efde000b610 tx=0x7efde0018000 comp rx=0 tx=0).stop 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 shutdown_connections 2023-10-06T12:34:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7efdd40606f0 0x7efdd4062b20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efde814fe00 0x7efde80b0170 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efde81591f0 0x7efde8169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 --2- 172.21.15.46:0/1342434541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efde80cc500 0x7efde80b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 >> 172.21.15.46:0/1342434541 conn(0x7efde806ef60 msgr2=0x7efde8150840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 shutdown_connections 2023-10-06T12:34:00.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.155+0000 7efdec82d700 1 -- 172.21.15.46:0/1342434541 wait complete. 2023-10-06T12:34:00.336 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T12:34:00.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T12:34:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.511+0000 7f0516563700 1 Processor -- start 2023-10-06T12:34:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.511+0000 7f0516563700 1 -- start start 2023-10-06T12:34:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0516563700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 0x7f05100b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0516563700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 0x7f05100b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0516563700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05100b5eb0 con 0x7f05101591b0 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0516563700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05100cef30 con 0x7f0510069770 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0516563700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05100cf4f0 con 0x7f0510158de0 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0514d60700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f05101591b0 0x7f0510069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40236/0 (socket says 172.21.15.46:40236) 2023-10-06T12:34:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0514d60700 1 -- 172.21.15.46:0/1207204437 learned_addr learned my addr 172.21.15.46:0/1207204437 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0515561700 1 --2- 172.21.15.46:0/1207204437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 0x7f05100b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0515d62700 1 --2- 172.21.15.46:0/1207204437 >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 0x7f05100b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0507fff700 1 -- 172.21.15.46:0/1207204437 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061329052 0 0) 0x7f0500001110 con 0x7f05101591b0 2023-10-06T12:34:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.515+0000 7f0507fff700 1 -- 172.21.15.46:0/1207204437 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04fc003570 con 0x7f05101591b0 2023-10-06T12:34:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515561700 1 -- 172.21.15.46:0/1207204437 >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 msgr2=0x7f05100b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515561700 1 --2- 172.21.15.46:0/1207204437 >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 0x7f05100b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515561700 1 -- 172.21.15.46:0/1207204437 >> v1:172.21.15.132:6789/0 conn(0x7f05101591b0 legacy=0x7f0510069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515561700 1 -- 172.21.15.46:0/1207204437 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05100ccac0 con 0x7f0510158de0 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515d62700 1 --2- 172.21.15.46:0/1207204437 >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 0x7f05100b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0515561700 1 --2- 172.21.15.46:0/1207204437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 0x7f05100b5970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f050c0098e0 tx=0x7f050c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b7241e866f6bb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0507fff700 1 -- 172.21.15.46:0/1207204437 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c011630 con 0x7f0510158de0 2023-10-06T12:34:00.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0507fff700 1 -- 172.21.15.46:0/1207204437 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c011790 con 0x7f0510158de0 2023-10-06T12:34:00.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0507fff700 1 -- 172.21.15.46:0/1207204437 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c011b30 con 0x7f0510158de0 2023-10-06T12:34:00.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 -- 172.21.15.46:0/1207204437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 msgr2=0x7f05100b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 --2- 172.21.15.46:0/1207204437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 0x7f05100b5970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f050c0098e0 tx=0x7f050c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 -- 172.21.15.46:0/1207204437 shutdown_connections 2023-10-06T12:34:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 --2- 172.21.15.46:0/1207204437 >> v2:172.21.15.132:3300/0 conn(0x7f0510069770 0x7f05100b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 --2- 172.21.15.46:0/1207204437 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0510158de0 0x7f05100b5970 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 -- 172.21.15.46:0/1207204437 >> 172.21.15.46:0/1207204437 conn(0x7f0510068390 msgr2=0x7f05100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 -- 172.21.15.46:0/1207204437 shutdown_connections 2023-10-06T12:34:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.519+0000 7f0516563700 1 -- 172.21.15.46:0/1207204437 wait complete. 2023-10-06T12:34:00.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 Processor -- start 2023-10-06T12:34:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 -- start start 2023-10-06T12:34:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0510158de0 0x7f05100b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05101591b0 0x7f05100b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0510169640 con 0x7f0510069770 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0510169850 con 0x7f0510158de0 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0516563700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0510169b30 con 0x7f05101591b0 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44876/0 (socket says 172.21.15.46:44876) 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 -- 172.21.15.46:0/4286772888 learned_addr learned my addr 172.21.15.46:0/4286772888 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515d62700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05101591b0 0x7f05100b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0514d60700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0510158de0 0x7f05100b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 -- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05101591b0 msgr2=0x7f05100b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05101591b0 0x7f05100b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 -- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0510158de0 msgr2=0x7f05100b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0510158de0 0x7f05100b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.523+0000 7f0515561700 1 -- 172.21.15.46:0/4286772888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508009a20 con 0x7f0510069770 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f0515561700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f05100b5840 tx=0x7f050c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c000d10 con 0x7f0510069770 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c011b10 con 0x7f0510069770 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c025660 con 0x7f0510069770 2023-10-06T12:34:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c009420 con 0x7f0510069770 2023-10-06T12:34:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051016a0f0 con 0x7f0510069770 2023-10-06T12:34:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.527+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0510064590 con 0x7f0510069770 2023-10-06T12:34:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.531+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 79) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f050c025a50 con 0x7f0510069770 2023-10-06T12:34:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.531+0000 7f05057fa700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 0x7f04f40628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.531+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f050c07e220 con 0x7f0510069770 2023-10-06T12:34:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.531+0000 7f0514d60700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 0x7f04f40628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.531+0000 7f0514d60700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 0x7f04f40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0500007fd0 tx=0x7f0500007bf0 comp rx=0 tx=0).ready entity=mgr.9009 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.535+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f050c04e530 con 0x7f0510069770 2023-10-06T12:34:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.735+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f05100ccda0 con 0x7f04f40603f0 2023-10-06T12:34:00.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.855+0000 7f05057fa700 1 -- 172.21.15.46:0/4286772888 <== mgr.9009 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f05100ccda0 con 0x7f04f40603f0 2023-10-06T12:34:00.861 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path snap_test_dir1 2023-10-06T12:34:00.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.859+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 msgr2=0x7f04f40628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.859+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 0x7f04f40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0500007fd0 tx=0x7f0500007bf0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.859+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 msgr2=0x7f0510153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:00.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.859+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f05100b5840 tx=0x7f050c017000 comp rx=0 tx=0).stop 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 shutdown_connections 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f04f40603f0 0x7f04f40628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05101591b0 0x7f05100b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0510158de0 0x7f05100b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 --2- 172.21.15.46:0/4286772888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0510069770 0x7f0510153ec0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 >> 172.21.15.46:0/4286772888 conn(0x7f0510068390 msgr2=0x7f0510154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 shutdown_connections 2023-10-06T12:34:00.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:00.863+0000 7f0516563700 1 -- 172.21.15.46:0/4286772888 wait complete. 2023-10-06T12:34:00.889 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T12:34:00.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T12:34:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 Processor -- start 2023-10-06T12:34:01.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 -- start start 2023-10-06T12:34:01.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e0156bf0 0x7fd5e00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd5e0156fc0 0x7fd5e006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e006c8f0 0x7fd5e015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cdcd0 con 0x7fd5e006c8f0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e006cd70 con 0x7fd5e0156fc0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5df59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00cd4c0 con 0x7fd5e0156bf0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e006c8f0 0x7fd5e015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e006c8f0 0x7fd5e015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44884/0 (socket says 172.21.15.46:44884) 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 -- 172.21.15.46:0/1851910094 learned_addr learned my addr 172.21.15.46:0/1851910094 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5de59c700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e0156bf0 0x7fd5e00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ddd9b700 1 --2- 172.21.15.46:0/1851910094 >> v2:172.21.15.132:3300/0 conn(0x7fd5e0156fc0 0x7fd5e006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 -- 172.21.15.46:0/1851910094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e0156bf0 msgr2=0x7fd5e00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e0156bf0 0x7fd5e00cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 -- 172.21.15.46:0/1851910094 >> v2:172.21.15.132:3300/0 conn(0x7fd5e0156fc0 msgr2=0x7fd5e006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 --2- 172.21.15.46:0/1851910094 >> v2:172.21.15.132:3300/0 conn(0x7fd5e0156fc0 0x7fd5e006c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 -- 172.21.15.46:0/1851910094 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00ca2d0 con 0x7fd5e006c8f0 2023-10-06T12:34:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.059+0000 7fd5ded9d700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006c8f0 0x7fd5e015aa00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd5e006e6f0 tx=0x7fd5d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535cca3f14520f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5dd59a700 1 -- 172.21.15.46:0/1851910094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800e070 con 0x7fd5e006c8f0 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5dd59a700 1 -- 172.21.15.46:0/1851910094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800b930 con 0x7fd5e006c8f0 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5dd59a700 1 -- 172.21.15.46:0/1851910094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80092e0 con 0x7fd5e006c8f0 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 -- 172.21.15.46:0/1851910094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006c8f0 msgr2=0x7fd5e015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006c8f0 0x7fd5e015aa00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd5e006e6f0 tx=0x7fd5d8009840 comp rx=0 tx=0).stop 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 -- 172.21.15.46:0/1851910094 shutdown_connections 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 --2- 172.21.15.46:0/1851910094 >> v2:172.21.15.132:3300/0 conn(0x7fd5e0156fc0 0x7fd5e006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006c8f0 0x7fd5e015aa00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 --2- 172.21.15.46:0/1851910094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e0156bf0 0x7fd5e00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 -- 172.21.15.46:0/1851910094 >> 172.21.15.46:0/1851910094 conn(0x7fd5e006ad70 msgr2=0x7fd5e006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 -- 172.21.15.46:0/1851910094 shutdown_connections 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.063+0000 7fd5df59e700 1 -- 172.21.15.46:0/1851910094 wait complete. 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 Processor -- start 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 -- start start 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e006c8f0 0x7fd5e00c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0156bf0 0x7fd5e00ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5e0156fc0 0x7fd5e01525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00b1ce0 con 0x7fd5e0156bf0 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00c59a0 con 0x7fd5e0156fc0 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5df59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00b10d0 con 0x7fd5e006c8f0 2023-10-06T12:34:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0156bf0 0x7fd5e00ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0156bf0 0x7fd5e00ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44898/0 (socket says 172.21.15.46:44898) 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 -- 172.21.15.46:0/4266448629 learned_addr learned my addr 172.21.15.46:0/4266448629 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5de59c700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e006c8f0 0x7fd5e00c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ded9d700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5e0156fc0 0x7fd5e01525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 -- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e006c8f0 msgr2=0x7fd5e00c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5e006c8f0 0x7fd5e00c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 -- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5e0156fc0 msgr2=0x7fd5e01525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5e0156fc0 0x7fd5e01525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 -- 172.21.15.46:0/4266448629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d8009ae0 con 0x7fd5e0156bf0 2023-10-06T12:34:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ded9d700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5e0156fc0 0x7fd5e01525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.067+0000 7fd5ddd9b700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0156bf0 0x7fd5e00ae260 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400e960 tx=0x7fd5d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400c850 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d400c9b0 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d400cd50 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5df59e700 1 -- 172.21.15.46:0/4266448629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e00b13b0 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5df59e700 1 -- 172.21.15.46:0/4266448629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e00b1970 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5df59e700 1 -- 172.21.15.46:0/4266448629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00b1e90 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.071+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 80) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fd5d4013680 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.075+0000 7fd5ce7fc700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5c0060780 0x7fd5c0062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.075+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fd5d407d260 con 0x7fd5e0156bf0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.075+0000 7fd5de59c700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5c0060780 0x7fd5c0062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.075+0000 7fd5de59c700 1 --2- 172.21.15.46:0/4266448629 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd5c0060780 0x7fd5c0062c30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009d40 tx=0x7fd5d0008040 comp rx=0 tx=0).ready entity=mgr.9009 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.079+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd5e00b1e90 con 0x7fd5e0156bf0 2023-10-06T12:34:01.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.271+0000 7fd5df59e700 1 -- 172.21.15.46:0/4266448629 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e00b1e90 con 0x7fd5c0060780 2023-10-06T12:34:01.290 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:01.287+0000 7f9eb36a0700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T12:34:01.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.287+0000 7fd5ce7fc700 1 -- 172.21.15.46:0/4266448629 <== mgr.9009 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fd5e00b1e90 con 0x7fd5c0060780 2023-10-06T12:34:01.293 INFO:teuthology.orchestra.run.smithi046.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T12:34:01.304 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T12:34:01.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:34:01.305 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T12:34:01.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T12:34:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.543+0000 7fd88a278700 1 Processor -- start 2023-10-06T12:34:01.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.543+0000 7fd88a278700 1 -- start start 2023-10-06T12:34:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd88a278700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 0x7fd884160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd88a278700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88406de80 con 0x7fd8840c1a60 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd88a278700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8840c6d90 con 0x7fd8840c1690 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd88a278700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c67a0 con 0x7fd884155a60 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd87bfff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8840c1a60 0x7fd884155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42380/0 (socket says 172.21.15.46:42380) 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd87bfff700 1 -- 172.21.15.46:0/3531108487 learned_addr learned my addr 172.21.15.46:0/3531108487 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd888815700 1 --2- 172.21.15.46:0/3531108487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 0x7fd884160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747588956 0 0) 0x7fd88406de80 con 0x7fd8840c1a60 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd870003570 con 0x7fd8840c1a60 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146370959 0 0) 0x7fd8840c6d90 con 0x7fd8840c1690 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd88406de80 con 0x7fd8840c1690 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248334675 0 0) 0x7fd870003570 con 0x7fd8840c1a60 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd888815700 1 -- 172.21.15.46:0/3531108487 >> v1:172.21.15.132:6789/0 conn(0x7fd8840c1690 legacy=0x7fd8840c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd888815700 1 -- 172.21.15.46:0/3531108487 >> v1:172.21.15.46:6789/0 conn(0x7fd8840c1a60 legacy=0x7fd884155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd888815700 1 -- 172.21.15.46:0/3531108487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840c7070 con 0x7fd884155a60 2023-10-06T12:34:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.547+0000 7fd888815700 1 --2- 172.21.15.46:0/3531108487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 0x7fd884160790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00afa0 tx=0x7fd87c00a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6a3cc753d276a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00f070 con 0x7fd884155a60 2023-10-06T12:34:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00a0d0 con 0x7fd884155a60 2023-10-06T12:34:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd8837fe700 1 -- 172.21.15.46:0/3531108487 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c001470 con 0x7fd884155a60 2023-10-06T12:34:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 -- 172.21.15.46:0/3531108487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 msgr2=0x7fd884160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 --2- 172.21.15.46:0/3531108487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 0x7fd884160790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00afa0 tx=0x7fd87c00a890 comp rx=0 tx=0).stop 2023-10-06T12:34:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 -- 172.21.15.46:0/3531108487 shutdown_connections 2023-10-06T12:34:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 --2- 172.21.15.46:0/3531108487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd884155a60 0x7fd884160790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 -- 172.21.15.46:0/3531108487 >> 172.21.15.46:0/3531108487 conn(0x7fd8840cbe30 msgr2=0x7fd8840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 -- 172.21.15.46:0/3531108487 shutdown_connections 2023-10-06T12:34:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.551+0000 7fd88a278700 1 -- 172.21.15.46:0/3531108487 wait complete. 2023-10-06T12:34:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 Processor -- start 2023-10-06T12:34:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 -- start start 2023-10-06T12:34:01.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8840c1a60 0x7fd884153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd884155a60 0x7fd8840b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c5b80 con 0x7fd8840c1690 2023-10-06T12:34:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884153070 con 0x7fd884155a60 2023-10-06T12:34:01.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd88a278700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd884152a60 con 0x7fd8840c1a60 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44906/0 (socket says 172.21.15.46:44906) 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 -- 172.21.15.46:0/6077407 learned_addr learned my addr 172.21.15.46:0/6077407 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd87bfff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8840c1a60 0x7fd884153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd888815700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd884155a60 0x7fd8840b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 -- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8840c1a60 msgr2=0x7fd884153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8840c1a60 0x7fd884153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 -- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd884155a60 msgr2=0x7fd8840b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd884155a60 0x7fd8840b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:01.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.555+0000 7fd883fff700 1 -- 172.21.15.46:0/6077407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c00ab30 con 0x7fd8840c1690 2023-10-06T12:34:01.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd883fff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c790 tx=0x7fd874007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874010510 con 0x7fd8840c1690 2023-10-06T12:34:01.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874010670 con 0x7fd8840c1690 2023-10-06T12:34:01.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd884150d90 con 0x7fd8840c1690 2023-10-06T12:34:01.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874010a10 con 0x7fd8840c1690 2023-10-06T12:34:01.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd884152d00 con 0x7fd8840c1690 2023-10-06T12:34:01.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.559+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8841528c0 con 0x7fd8840c1690 2023-10-06T12:34:01.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.563+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 80) v1 ==== 80135+0+0 (secure 0 0 0) 0x7fd87401b3b0 con 0x7fd8840c1690 2023-10-06T12:34:01.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.563+0000 7fd880ff9700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 0x7fd864062c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:01.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.563+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fd87407c8e0 con 0x7fd8840c1690 2023-10-06T12:34:01.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.563+0000 7fd87bfff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 0x7fd864062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.563+0000 7fd87bfff700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 0x7fd864062c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd88406a5b0 tx=0x7fd86c000bc0 comp rx=0 tx=0).ready entity=mgr.9009 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.571+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8841528c0 con 0x7fd8840c1690 2023-10-06T12:34:01.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:01.787+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd8841528c0 con 0x7fd8840c1690 2023-10-06T12:34:01.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:34:01.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:34:02.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.055+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fd874010dc0 con 0x7fd8840c1690 2023-10-06T12:34:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.055+0000 7fd880ff9700 1 -- 172.21.15.46:0/6077407 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v81) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd874046880 con 0x7fd8840c1690 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 msgr2=0x7fd864062c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 0x7fd864062c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd88406a5b0 tx=0x7fd86c000bc0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 msgr2=0x7fd884150850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c790 tx=0x7fd874007e10 comp rx=0 tx=0).stop 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 shutdown_connections 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd864060750 0x7fd864062c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd884155a60 0x7fd8840b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8840c1a60 0x7fd884153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 --2- 172.21.15.46:0/6077407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8840c1690 0x7fd884150850 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 >> 172.21.15.46:0/6077407 conn(0x7fd8840cbe30 msgr2=0x7fd8840c0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 shutdown_connections 2023-10-06T12:34:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.059+0000 7fd88a278700 1 -- 172.21.15.46:0/6077407 wait complete. 2023-10-06T12:34:02.081 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:34:02.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:34:02.122 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:34:02.154 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:34:02.154 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T12:34:02.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:34:02.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T12:34:02.234 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:02.230+0000 7f122a7af000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:34:02.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:34:02.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:02.254 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T12:34:02.255 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:02.251+0000 7f7a375c5000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:34:02.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.463+0000 7f39fa616700 1 Processor -- start 2023-10-06T12:34:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.463+0000 7f39fa616700 1 -- start start 2023-10-06T12:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39fa616700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f40cb4c0 0x7f39f40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39fa616700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39f40af3d0 0x7f39f40690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39fa616700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f40cf100 con 0x7f39f40af000 2023-10-06T12:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39fa616700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40ce300 con 0x7f39f40cb4c0 2023-10-06T12:34:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39fa616700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40ce4f0 con 0x7f39f40af3d0 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f40cb4c0 0x7f39f40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39f40cb4c0 0x7f39f40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44926/0 (socket says 172.21.15.46:44926) 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 -- 172.21.15.46:0/395886315 learned_addr learned my addr 172.21.15.46:0/395886315 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f9e15700 1 --2- 172.21.15.46:0/395886315 >> v2:172.21.15.132:3300/0 conn(0x7f39f40af3d0 0x7f39f40690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 -- 172.21.15.46:0/395886315 >> v2:172.21.15.132:3300/0 conn(0x7f39f40af3d0 msgr2=0x7f39f40690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 --2- 172.21.15.46:0/395886315 >> v2:172.21.15.132:3300/0 conn(0x7f39f40af3d0 0x7f39f40690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 -- 172.21.15.46:0/395886315 >> v1:172.21.15.132:6789/0 conn(0x7f39f40af000 legacy=0x7f39f40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 -- 172.21.15.46:0/395886315 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40ce7d0 con 0x7f39f40cb4c0 2023-10-06T12:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39f8e13700 1 --2- 172.21.15.46:0/395886315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40cb4c0 0x7f39f40cf590 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f39f40cf070 tx=0x7f39e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294cc9b7bb6be334 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39ebfff700 1 -- 172.21.15.46:0/395886315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4009e70 con 0x7f39f40cb4c0 2023-10-06T12:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39ebfff700 1 -- 172.21.15.46:0/395886315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e400b860 con 0x7f39f40cb4c0 2023-10-06T12:34:02.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.467+0000 7f39ebfff700 1 -- 172.21.15.46:0/395886315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400bc00 con 0x7f39f40cb4c0 2023-10-06T12:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- 172.21.15.46:0/395886315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40cb4c0 msgr2=0x7f39f40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 --2- 172.21.15.46:0/395886315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40cb4c0 0x7f39f40cf590 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f39f40cf070 tx=0x7f39e4009250 comp rx=0 tx=0).stop 2023-10-06T12:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- 172.21.15.46:0/395886315 shutdown_connections 2023-10-06T12:34:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 --2- 172.21.15.46:0/395886315 >> v2:172.21.15.132:3300/0 conn(0x7f39f40af3d0 0x7f39f40690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 --2- 172.21.15.46:0/395886315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40cb4c0 0x7f39f40cf590 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- 172.21.15.46:0/395886315 >> 172.21.15.46:0/395886315 conn(0x7f39f40683b0 msgr2=0x7f39f40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- 172.21.15.46:0/395886315 shutdown_connections 2023-10-06T12:34:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- 172.21.15.46:0/395886315 wait complete. 2023-10-06T12:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 Processor -- start 2023-10-06T12:34:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.471+0000 7f39fa616700 1 -- start start 2023-10-06T12:34:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f40af3d0 0x7f39f415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f40cb4c0 0x7f39f40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40c9c70 con 0x7f39f40af000 2023-10-06T12:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4157730 con 0x7f39f40af3d0 2023-10-06T12:34:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40c97e0 con 0x7f39f40cb4c0 2023-10-06T12:34:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44938/0 (socket says 172.21.15.46:44938) 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 -- 172.21.15.46:0/42315975 learned_addr learned my addr 172.21.15.46:0/42315975 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f8e13700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f40af3d0 0x7f39f415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9e15700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f40cb4c0 0x7f39f40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f40cb4c0 msgr2=0x7f39f40c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f40cb4c0 0x7f39f40c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f40af3d0 msgr2=0x7f39f415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f40af3d0 0x7f39f415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 -- 172.21.15.46:0/42315975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e40094f0 con 0x7f39f40af000 2023-10-06T12:34:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39f9614700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f39f000c5c0 tx=0x7f39f000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0011590 con 0x7f39f40af000 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f0011710 con 0x7f39f40af000 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f40c8c10 con 0x7f39f40af000 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f0011ad0 con 0x7f39f40af000 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.475+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f40c9190 con 0x7f39f40af000 2023-10-06T12:34:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f4064590 con 0x7f39f40af000 2023-10-06T12:34:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f39f001b3b0 con 0x7f39f40af000 2023-10-06T12:34:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39e97fa700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 0x7f39d8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f39f007c5d0 con 0x7f39f40af000 2023-10-06T12:34:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39f8e13700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 msgr2=0x7f39d8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.479+0000 7f39f8e13700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 0x7f39d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.487+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f39f004c3e0 con 0x7f39f40af000 2023-10-06T12:34:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.683+0000 7f39f8e13700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 msgr2=0x7f39d8062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.683+0000 7f39f8e13700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 0x7f39d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.695+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39f40ceab0 con 0x7f39f40af000 2023-10-06T12:34:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.695+0000 7f39e97fa700 1 -- 172.21.15.46:0/42315975 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f39f0017090 con 0x7f39f40af000 2023-10-06T12:34:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 msgr2=0x7f39d8062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 0x7f39d8062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 msgr2=0x7f39f41571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f39f000c5c0 tx=0x7f39f000c980 comp rx=0 tx=0).stop 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 shutdown_connections 2023-10-06T12:34:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f39d8060700 0x7f39d8062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f40cb4c0 0x7f39f40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f40af3d0 0x7f39f415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 --2- 172.21.15.46:0/42315975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39f40af000 0x7f39f41571f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 >> 172.21.15.46:0/42315975 conn(0x7f39f40683b0 msgr2=0x7f39f40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 shutdown_connections 2023-10-06T12:34:02.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.699+0000 7f39fa616700 1 -- 172.21.15.46:0/42315975 wait complete. 2023-10-06T12:34:02.706 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:34:02.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:02.776 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:02.774+0000 7f122a7af000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:34:02.830 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:02.827+0000 7f7a375c5000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:34:02.852 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:02.850+0000 7f122a7af000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:34:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 Processor -- start 2023-10-06T12:34:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 -- start start 2023-10-06T12:34:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faaac156c20 0x7faaac0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac0c96d0 con 0x7faaac0c9300 2023-10-06T12:34:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaac15aa40 con 0x7faaac156ff0 2023-10-06T12:34:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab1bcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaac0ca3f0 con 0x7faaac156c20 2023-10-06T12:34:02.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.875+0000 7faab0bca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faaac156c20 0x7faaac0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faaac156c20 0x7faaac0ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44954/0 (socket says 172.21.15.46:44954) 2023-10-06T12:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 -- 172.21.15.46:0/2207204866 learned_addr learned my addr 172.21.15.46:0/2207204866 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab13cb700 1 --1- 172.21.15.46:0/2207204866 >> v1:172.21.15.46:6789/0 conn(0x7faaac0c9300 0x7faaac0afb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42402/0 (socket says 172.21.15.46:42402) 2023-10-06T12:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 -- 172.21.15.46:0/2207204866 >> v1:172.21.15.132:6789/0 conn(0x7faaac156ff0 legacy=0x7faaac0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 -- 172.21.15.46:0/2207204866 >> v1:172.21.15.46:6789/0 conn(0x7faaac0c9300 legacy=0x7faaac0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 -- 172.21.15.46:0/2207204866 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac15ad20 con 0x7faaac156c20 2023-10-06T12:34:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab0bca700 1 --2- 172.21.15.46:0/2207204866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156c20 0x7faaac0ae920 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7faaac0ca530 tx=0x7faaa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e23a7a3f517c1f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faaab7fe700 1 -- 172.21.15.46:0/2207204866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000b860 con 0x7faaac156c20 2023-10-06T12:34:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faaab7fe700 1 -- 172.21.15.46:0/2207204866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa000b9c0 con 0x7faaac156c20 2023-10-06T12:34:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faaab7fe700 1 -- 172.21.15.46:0/2207204866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000bd80 con 0x7faaac156c20 2023-10-06T12:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 -- 172.21.15.46:0/2207204866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156c20 msgr2=0x7faaac0ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 --2- 172.21.15.46:0/2207204866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156c20 0x7faaac0ae920 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7faaac0ca530 tx=0x7faaa0009250 comp rx=0 tx=0).stop 2023-10-06T12:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 -- 172.21.15.46:0/2207204866 shutdown_connections 2023-10-06T12:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 --2- 172.21.15.46:0/2207204866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156c20 0x7faaac0ae920 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 -- 172.21.15.46:0/2207204866 >> 172.21.15.46:0/2207204866 conn(0x7faaac0ccd50 msgr2=0x7faaac0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 -- 172.21.15.46:0/2207204866 shutdown_connections 2023-10-06T12:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.879+0000 7faab1bcc700 1 -- 172.21.15.46:0/2207204866 wait complete. 2023-10-06T12:34:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 Processor -- start 2023-10-06T12:34:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 -- start start 2023-10-06T12:34:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaac0c9300 0x7faaac1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaac156c20 0x7faaac15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac0aeed0 con 0x7faaac156ff0 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac15e090 con 0x7faaac0c9300 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab1bcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac157650 con 0x7faaac156c20 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab13cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab13cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44970/0 (socket says 172.21.15.46:44970) 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab13cb700 1 -- 172.21.15.46:0/1363948223 learned_addr learned my addr 172.21.15.46:0/1363948223 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faab0bca700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaac0c9300 0x7faaac1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.883+0000 7faaabfff700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaac156c20 0x7faaac15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 -- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaac156c20 msgr2=0x7faaac15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaac156c20 0x7faaac15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 -- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaac0c9300 msgr2=0x7faaac1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaac0c9300 0x7faaac1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 -- 172.21.15.46:0/1363948223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa00094f0 con 0x7faaac156ff0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab13cb700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7faaa400cd30 tx=0x7faaa4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400ce90 con 0x7faaac156ff0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4005df0 con 0x7faaac156ff0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac15d7f0 con 0x7faaac156ff0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400aac0 con 0x7faaac156ff0 2023-10-06T12:34:02.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac15ddb0 con 0x7faaac156ff0 2023-10-06T12:34:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.887+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac064590 con 0x7faaac156ff0 2023-10-06T12:34:02.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.891+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7faaa40070f0 con 0x7faaac156ff0 2023-10-06T12:34:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.891+0000 7faaa8ff9700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 0x7faa90062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:02.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.891+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7faaa407cfe0 con 0x7faaac156ff0 2023-10-06T12:34:02.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.891+0000 7faab0bca700 1 -- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 msgr2=0x7faa90062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:02.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.891+0000 7faab0bca700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 0x7faa90062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:02.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:02.899+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faaa404ced0 con 0x7faaac156ff0 2023-10-06T12:34:02.916 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:02.911+0000 7f7a375c5000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:34:02.994 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:02.990+0000 7f122a7af000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:03.066 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.063+0000 7f7a375c5000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:03.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.067+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaac157790 con 0x7faaac156ff0 2023-10-06T12:34:03.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faaa8ff9700 1 -- 172.21.15.46:0/1363948223 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18156 (secure 0 0 0) 0x7faaa40504f0 con 0x7faaac156ff0 2023-10-06T12:34:03.075 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:03.075 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":154,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:59.005299+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/4189294583":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6833/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6832/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/1906903242":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/934339781":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/2228099525":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/3221426589":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/511843182":"2023-10-07T12:33:59.005230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 msgr2=0x7faa90062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 0x7faa90062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 msgr2=0x7faaac15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7faaa400cd30 tx=0x7faaa4001b10 comp rx=0 tx=0).stop 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 shutdown_connections 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7faa90060700 0x7faa90062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaac156ff0 0x7faaac15ea30 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaac156c20 0x7faaac15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 --2- 172.21.15.46:0/1363948223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaac0c9300 0x7faaac1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 >> 172.21.15.46:0/1363948223 conn(0x7faaac0ccd50 msgr2=0x7faaac0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.071+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 shutdown_connections 2023-10-06T12:34:03.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.075+0000 7faab1bcc700 1 -- 172.21.15.46:0/1363948223 wait complete. 2023-10-06T12:34:03.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T12:33:38.215933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T12:33:39.244553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T12:34:03.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:34:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 Processor -- start 2023-10-06T12:34:03.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 -- start start 2023-10-06T12:34:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7754159d90 0x7f775415a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540c6470 con 0x7f7754153c60 2023-10-06T12:34:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540c6750 con 0x7f775406e290 2023-10-06T12:34:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.263+0000 7f775aa2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775415a9c0 con 0x7f7754159d90 2023-10-06T12:34:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7758fcb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7754153c60 0x7f77540c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36362/0 (socket says 172.21.15.46:36362) 2023-10-06T12:34:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7758fcb700 1 -- 172.21.15.46:0/2469756381 learned_addr learned my addr 172.21.15.46:0/2469756381 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7753fff700 1 --2- 172.21.15.46:0/2469756381 >> v2:172.21.15.132:3300/0 conn(0x7f7754159d90 0x7f775415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559610286 0 0) 0x7f77540c6470 con 0x7f7754153c60 2023-10-06T12:34:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7740003570 con 0x7f7754153c60 2023-10-06T12:34:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1248652611 0 0) 0x7f7740003570 con 0x7f7754153c60 2023-10-06T12:34:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7753fff700 1 -- 172.21.15.46:0/2469756381 >> v1:172.21.15.132:6789/0 conn(0x7f775406e290 legacy=0x7f7754153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7753fff700 1 -- 172.21.15.46:0/2469756381 >> v1:172.21.15.46:6789/0 conn(0x7f7754153c60 legacy=0x7f77540c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7753fff700 1 -- 172.21.15.46:0/2469756381 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775406e7f0 con 0x7f7754159d90 2023-10-06T12:34:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f7753fff700 1 --2- 172.21.15.46:0/2469756381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754159d90 0x7f775415a160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f77440098e0 tx=0x7f77440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f1f05f39f593b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0017f0 con 0x7f7754159d90 2023-10-06T12:34:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7744011420 con 0x7f7754159d90 2023-10-06T12:34:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.267+0000 7f77537fe700 1 -- 172.21.15.46:0/2469756381 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77440117c0 con 0x7f7754159d90 2023-10-06T12:34:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- 172.21.15.46:0/2469756381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754159d90 msgr2=0x7f775415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 --2- 172.21.15.46:0/2469756381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754159d90 0x7f775415a160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f77440098e0 tx=0x7f77440091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- 172.21.15.46:0/2469756381 shutdown_connections 2023-10-06T12:34:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 --2- 172.21.15.46:0/2469756381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754159d90 0x7f775415a160 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- 172.21.15.46:0/2469756381 >> 172.21.15.46:0/2469756381 conn(0x7f77540cbe30 msgr2=0x7f77540cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- 172.21.15.46:0/2469756381 shutdown_connections 2023-10-06T12:34:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- 172.21.15.46:0/2469756381 wait complete. 2023-10-06T12:34:03.274 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:03.270+0000 7f122a7af000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:34:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 Processor -- start 2023-10-06T12:34:03.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.271+0000 7f775aa2e700 1 -- start start 2023-10-06T12:34:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 0x7f7754151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7754159d90 0x7f775416d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77540ae490 con 0x7f775406e290 2023-10-06T12:34:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775416a6b0 con 0x7f7754153c60 2023-10-06T12:34:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f775aa2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775416e3a0 con 0x7f7754159d90 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7753fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7753fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35252/0 (socket says 172.21.15.46:35252) 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 0x7f7754151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7758fcb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7754159d90 0x7f775416d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7753fff700 1 -- 172.21.15.46:0/3692613133 learned_addr learned my addr 172.21.15.46:0/3692613133 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7754159d90 msgr2=0x7f775416d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7754159d90 0x7f775416d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 msgr2=0x7f775416ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 -- 172.21.15.46:0/3692613133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744009420 con 0x7f7754153c60 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7753fff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f774bfff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 0x7f7754151b90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c5b0 tx=0x7f773c007de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c011590 con 0x7f7754153c60 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c0116f0 con 0x7f7754153c60 2023-10-06T12:34:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.275+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c011af0 con 0x7f7754153c60 2023-10-06T12:34:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.279+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775416e0c0 con 0x7f7754153c60 2023-10-06T12:34:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.279+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775416a340 con 0x7f7754153c60 2023-10-06T12:34:03.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.279+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77540660f0 con 0x7f7754153c60 2023-10-06T12:34:03.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.279+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f773c01b3b0 con 0x7f7754153c60 2023-10-06T12:34:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.283+0000 7f7750ff9700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 0x7f7734062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.283+0000 7f7753fff700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 msgr2=0x7f7734062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.283+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f773c07c420 con 0x7f7754153c60 2023-10-06T12:34:03.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.283+0000 7f7753fff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 0x7f7734062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:03.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.291+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f773c04c230 con 0x7f7754153c60 2023-10-06T12:34:03.351 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:03.346+0000 7f122a7af000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:03.422 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.419+0000 7f7a375c5000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:34:03.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.487+0000 7f7753fff700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 msgr2=0x7f7734062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:03.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.487+0000 7f7753fff700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 0x7f7734062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:03.541 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.535+0000 7f7a375c5000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:03.573 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:03.570+0000 7f122a7af000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:34:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.631+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f775406ead0 con 0x7f7754153c60 2023-10-06T12:34:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.631+0000 7f7750ff9700 1 -- 172.21.15.46:0/3692613133 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f773c017020 con 0x7f7754153c60 2023-10-06T12:34:03.637 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:34:03.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.635+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 msgr2=0x7f7734062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:03.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.635+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 0x7f7734062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.635+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 msgr2=0x7f7754151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.635+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 0x7f7754151b90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c5b0 tx=0x7f773c007de0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 shutdown_connections 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7734060730 0x7f7734062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7754159d90 0x7f775416d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7754153c60 0x7f7754151b90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 --2- 172.21.15.46:0/3692613133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f775406e290 0x7f775416ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 >> 172.21.15.46:0/3692613133 conn(0x7f77540cbe30 msgr2=0x7f77541527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 shutdown_connections 2023-10-06T12:34:03.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.639+0000 7f775aa2e700 1 -- 172.21.15.46:0/3692613133 wait complete. 2023-10-06T12:34:03.657 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:03.654+0000 7f122a7af000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:34:03.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:34:03.746 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:03.742+0000 7f122a7af000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:34:03.802 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.799+0000 7f7a375c5000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:34:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.823+0000 7f506a8e3700 1 Processor -- start 2023-10-06T12:34:03.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.823+0000 7f506a8e3700 1 -- start start 2023-10-06T12:34:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a8e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f506406ca80 0x7f50640cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a8e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50640cc190 con 0x7f50640cb180 2023-10-06T12:34:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a8e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50640cd9a0 con 0x7f50640cd4c0 2023-10-06T12:34:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a8e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640cdc80 con 0x7f506406ca80 2023-10-06T12:34:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50698e1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f50640cb180 0x7f5064157e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36366/0 (socket says 172.21.15.46:36366) 2023-10-06T12:34:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50698e1700 1 -- 172.21.15.46:0/1150670380 learned_addr learned my addr 172.21.15.46:0/1150670380 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50690e0700 1 --1- 172.21.15.46:0/1150670380 >> v1:172.21.15.132:6789/0 conn(0x7f50640cd4c0 0x7f506406c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34280/0 (socket says 172.21.15.46:34280) 2023-10-06T12:34:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a0e2700 1 --2- 172.21.15.46:0/1150670380 >> v2:172.21.15.46:3300/0 conn(0x7f506406ca80 0x7f50640cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a0e2700 1 -- 172.21.15.46:0/1150670380 >> v1:172.21.15.132:6789/0 conn(0x7f50640cd4c0 legacy=0x7f506406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a0e2700 1 -- 172.21.15.46:0/1150670380 >> v1:172.21.15.46:6789/0 conn(0x7f50640cb180 legacy=0x7f5064157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50688df700 1 -- 172.21.15.46:0/1150670380 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983114558 0 0) 0x7f50640cd9a0 con 0x7f50640cd4c0 2023-10-06T12:34:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a0e2700 1 -- 172.21.15.46:0/1150670380 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640cbb40 con 0x7f506406ca80 2023-10-06T12:34:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f506a0e2700 1 --2- 172.21.15.46:0/1150670380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f506406ca80 0x7f50640cc620 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f505c009e60 tx=0x7f505c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3b94189a3ca40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50688df700 1 -- 172.21.15.46:0/1150670380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00e070 con 0x7f506406ca80 2023-10-06T12:34:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50688df700 1 -- 172.21.15.46:0/1150670380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c00bb30 con 0x7f506406ca80 2023-10-06T12:34:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.827+0000 7f50688df700 1 -- 172.21.15.46:0/1150670380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00bed0 con 0x7f506406ca80 2023-10-06T12:34:03.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- 172.21.15.46:0/1150670380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f506406ca80 msgr2=0x7f50640cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 --2- 172.21.15.46:0/1150670380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f506406ca80 0x7f50640cc620 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f505c009e60 tx=0x7f505c009780 comp rx=0 tx=0).stop 2023-10-06T12:34:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- 172.21.15.46:0/1150670380 shutdown_connections 2023-10-06T12:34:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 --2- 172.21.15.46:0/1150670380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f506406ca80 0x7f50640cc620 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- 172.21.15.46:0/1150670380 >> 172.21.15.46:0/1150670380 conn(0x7f506406ad30 msgr2=0x7f506406b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- 172.21.15.46:0/1150670380 shutdown_connections 2023-10-06T12:34:03.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- 172.21.15.46:0/1150670380 wait complete. 2023-10-06T12:34:03.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 Processor -- start 2023-10-06T12:34:03.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.831+0000 7f506a8e3700 1 -- start start 2023-10-06T12:34:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506406ca80 0x7f506414dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640cd4c0 0x7f5064158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640b0540 con 0x7f50640cb180 2023-10-06T12:34:03.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640c6ed0 con 0x7f50640cd4c0 2023-10-06T12:34:03.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50640c71b0 con 0x7f506406ca80 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35270/0 (socket says 172.21.15.46:35270) 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 -- 172.21.15.46:0/3145153003 learned_addr learned my addr 172.21.15.46:0/3145153003 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50698e1700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506406ca80 0x7f506414dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a0e2700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640cd4c0 0x7f5064158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 -- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506406ca80 msgr2=0x7f506414dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506406ca80 0x7f506414dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 -- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640cd4c0 msgr2=0x7f5064158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640cd4c0 0x7f5064158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:03.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 -- 172.21.15.46:0/3145153003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c009a20 con 0x7f50640cb180 2023-10-06T12:34:03.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f50690e0700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f505400c510 tx=0x7f505400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:03.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054005a70 con 0x7f50640cb180 2023-10-06T12:34:03.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054005bf0 con 0x7f50640cb180 2023-10-06T12:34:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640c7490 con 0x7f50640cb180 2023-10-06T12:34:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540070d0 con 0x7f50640cb180 2023-10-06T12:34:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.835+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50640c7a50 con 0x7f50640cb180 2023-10-06T12:34:03.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.839+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50640ad750 con 0x7f50640cb180 2023-10-06T12:34:03.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.839+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f5054007440 con 0x7f50640cb180 2023-10-06T12:34:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.843+0000 7f5059ffb700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 0x7f5048062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.843+0000 7f50698e1700 1 -- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 msgr2=0x7f5048062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.843+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f505407c400 con 0x7f50640cb180 2023-10-06T12:34:03.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.843+0000 7f50698e1700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 0x7f5048062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:03.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:03.847+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f50640ad750 con 0x7f50640cb180 2023-10-06T12:34:03.895 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.891+0000 7f7a375c5000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:34:03.984 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:03.979+0000 7f7a375c5000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:34:04.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.015+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f50640ad750 con 0x7f50640cb180 2023-10-06T12:34:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.019+0000 7f5059ffb700 1 -- 172.21.15.46:0/3145153003 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v154) v1 ==== 92+0+10 (secure 0 0 0) 0x7f50640ad750 con 0x7f50640cb180 2023-10-06T12:34:04.022 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:34:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 msgr2=0x7f5048062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:04.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 0x7f5048062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 msgr2=0x7f5064158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f505400c510 tx=0x7f505400c820 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 shutdown_connections 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5048060700 0x7f5048062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640cd4c0 0x7f5064158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50640cb180 0x7f5064158a10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 --2- 172.21.15.46:0/3145153003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506406ca80 0x7f506414dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 >> 172.21.15.46:0/3145153003 conn(0x7f506406ad30 msgr2=0x7f50640c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 shutdown_connections 2023-10-06T12:34:04.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.023+0000 7f506a8e3700 1 -- 172.21.15.46:0/3145153003 wait complete. 2023-10-06T12:34:04.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T12:34:04.152 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:04.150+0000 7f122a7af000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:34:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.199+0000 7f14d922e700 1 Processor -- start 2023-10-06T12:34:04.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.199+0000 7f14d922e700 1 -- start start 2023-10-06T12:34:04.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d922e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d4069410 0x7f14d415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d922e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14d40697e0 0x7f14d415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d922e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d40ce470 con 0x7f14d4158d10 2023-10-06T12:34:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d922e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d4069c50 con 0x7f14d40697e0 2023-10-06T12:34:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d922e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d415a890 con 0x7f14d4069410 2023-10-06T12:34:04.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14d40697e0 0x7f14d415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d2d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d4069410 0x7f14d415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14d40697e0 0x7f14d415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36130/0 (socket says 172.21.15.46:36130) 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 -- 172.21.15.46:0/51567410 learned_addr learned my addr 172.21.15.46:0/51567410 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d359e700 1 --1- 172.21.15.46:0/51567410 >> v1:172.21.15.132:6789/0 conn(0x7f14d4158d10 0x7f14d4159170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34286/0 (socket says 172.21.15.46:34286) 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 -- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d4069410 msgr2=0x7f14d415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 --2- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d4069410 0x7f14d415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 -- 172.21.15.46:0/51567410 >> v1:172.21.15.132:6789/0 conn(0x7f14d4158d10 legacy=0x7f14d4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.203+0000 7f14d259c700 1 -- 172.21.15.46:0/51567410 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d415d470 con 0x7f14d40697e0 2023-10-06T12:34:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d259c700 1 --2- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d40697e0 0x7f14d415a350 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14c80099b0 tx=0x7f14c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba95e5d4a42a1bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d1d9b700 1 -- 172.21.15.46:0/51567410 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8009ea0 con 0x7f14d40697e0 2023-10-06T12:34:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d1d9b700 1 -- 172.21.15.46:0/51567410 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800b860 con 0x7f14d40697e0 2023-10-06T12:34:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d1d9b700 1 -- 172.21.15.46:0/51567410 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800bc00 con 0x7f14d40697e0 2023-10-06T12:34:04.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 -- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d40697e0 msgr2=0x7f14d415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 --2- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d40697e0 0x7f14d415a350 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14c80099b0 tx=0x7f14c8009250 comp rx=0 tx=0).stop 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 -- 172.21.15.46:0/51567410 shutdown_connections 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 --2- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d40697e0 0x7f14d415a350 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 --2- 172.21.15.46:0/51567410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d4069410 0x7f14d415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 -- 172.21.15.46:0/51567410 >> 172.21.15.46:0/51567410 conn(0x7f14d40d0210 msgr2=0x7f14d40d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 -- 172.21.15.46:0/51567410 shutdown_connections 2023-10-06T12:34:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.207+0000 7f14d922e700 1 -- 172.21.15.46:0/51567410 wait complete. 2023-10-06T12:34:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 Processor -- start 2023-10-06T12:34:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 -- start start 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d4069410 0x7f14d40b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d4158d10 0x7f14d40c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40b5e80 con 0x7f14d4158d10 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40c9610 con 0x7f14d4069410 2023-10-06T12:34:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d922e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40c8a00 con 0x7f14d40697e0 2023-10-06T12:34:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d4158d10 0x7f14d40c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d4069410 0x7f14d40b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:48682/0 (socket says 172.21.15.46:48682) 2023-10-06T12:34:04.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 -- 172.21.15.46:0/1390206790 learned_addr learned my addr 172.21.15.46:0/1390206790 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 -- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d4069410 msgr2=0x7f14d40b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d4069410 0x7f14d40b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 -- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d4158d10 msgr2=0x7f14d40c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d4158d10 0x7f14d40c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 -- 172.21.15.46:0/1390206790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80094f0 con 0x7f14d40697e0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.211+0000 7f14d259c700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14d4088000 tx=0x7f14c8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8000d10 con 0x7f14d40697e0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800bbe0 con 0x7f14d40697e0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40c8ce0 con 0x7f14d40697e0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8011670 con 0x7f14d40697e0 2023-10-06T12:34:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40c92a0 con 0x7f14d40697e0 2023-10-06T12:34:04.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d40c97c0 con 0x7f14d40697e0 2023-10-06T12:34:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.215+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f14c8011aa0 con 0x7f14d40697e0 2023-10-06T12:34:04.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.219+0000 7f14baffd700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 0x7f14c0062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.219+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f14c807e430 con 0x7f14d40697e0 2023-10-06T12:34:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.219+0000 7f14d2d9d700 1 -- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 msgr2=0x7f14c0062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:04.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.219+0000 7f14d2d9d700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 0x7f14c0062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:04.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.227+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f14d40c97c0 con 0x7f14d40697e0 2023-10-06T12:34:04.317 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:04.314+0000 7f122a7af000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:34:04.407 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:04.403+0000 7f7a375c5000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:34:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.403+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14d40c9780 con 0x7f14d40697e0 2023-10-06T12:34:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.407+0000 7f14baffd700 1 -- 172.21.15.46:0/1390206790 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14c804e2c0 con 0x7f14d40697e0 2023-10-06T12:34:04.410 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T12:34:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 msgr2=0x7f14c0062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 0x7f14c0062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 msgr2=0x7f14d4147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14d4088000 tx=0x7f14c8017000 comp rx=0 tx=0).stop 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 shutdown_connections 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f14c0060730 0x7f14c0062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14d4158d10 0x7f14d40c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14d40697e0 0x7f14d4147c00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 --2- 172.21.15.46:0/1390206790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14d4069410 0x7f14d40b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 >> 172.21.15.46:0/1390206790 conn(0x7f14d40d0210 msgr2=0x7f14d4159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 shutdown_connections 2023-10-06T12:34:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.411+0000 7f14d922e700 1 -- 172.21.15.46:0/1390206790 wait complete. 2023-10-06T12:34:04.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T12:34:04.582 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:04.579+0000 7f7a375c5000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:34:04.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.595+0000 7f9440ee3700 1 Processor -- start 2023-10-06T12:34:04.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.595+0000 7f9440ee3700 1 -- start start 2023-10-06T12:34:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f9440ee3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f943c0cd500 0x7f943c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f9440ee3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c068160 con 0x7f943c158fc0 2023-10-06T12:34:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f9440ee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c068440 con 0x7f943c159390 2023-10-06T12:34:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f9440ee3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c1597e0 con 0x7f943c0cd500 2023-10-06T12:34:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f943c0cd500 0x7f943c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943b7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f943c158fc0 0x7f943c15d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36368/0 (socket says 172.21.15.46:36368) 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943b7fe700 1 -- 172.21.15.46:0/3172335108 learned_addr learned my addr 172.21.15.46:0/3172335108 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877250932 0 0) 0x7f943c068160 con 0x7f943c158fc0 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9424003570 con 0x7f943c158fc0 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943bfff700 1 -- 172.21.15.46:0/3172335108 >> v1:172.21.15.132:6789/0 conn(0x7f943c159390 legacy=0x7f943c0c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216514946 0 0) 0x7f943c068440 con 0x7f943c159390 2023-10-06T12:34:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943bfff700 1 -- 172.21.15.46:0/3172335108 >> v1:172.21.15.46:6789/0 conn(0x7f943c158fc0 legacy=0x7f943c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943bfff700 1 -- 172.21.15.46:0/3172335108 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0cbbb0 con 0x7f943c0cd500 2023-10-06T12:34:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943bfff700 1 --2- 172.21.15.46:0/3172335108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c0cd500 0x7f943c0acb20 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9434009e60 tx=0x7f9434009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2404f3daf4b743 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400e070 con 0x7f943c0cd500 2023-10-06T12:34:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400bb60 con 0x7f943c0cd500 2023-10-06T12:34:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.599+0000 7f943a7fc700 1 -- 172.21.15.46:0/3172335108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434012430 con 0x7f943c0cd500 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- 172.21.15.46:0/3172335108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c0cd500 msgr2=0x7f943c0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 --2- 172.21.15.46:0/3172335108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c0cd500 0x7f943c0acb20 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9434009e60 tx=0x7f9434009780 comp rx=0 tx=0).stop 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- 172.21.15.46:0/3172335108 shutdown_connections 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 --2- 172.21.15.46:0/3172335108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c0cd500 0x7f943c0acb20 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- 172.21.15.46:0/3172335108 >> 172.21.15.46:0/3172335108 conn(0x7f943c06ad30 msgr2=0x7f943c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- 172.21.15.46:0/3172335108 shutdown_connections 2023-10-06T12:34:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- 172.21.15.46:0/3172335108 wait complete. 2023-10-06T12:34:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 Processor -- start 2023-10-06T12:34:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.603+0000 7f9440ee3700 1 -- start start 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f943c0cd500 0x7f943c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f943c159390 0x7f943c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0adfe0 con 0x7f943c158fc0 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0ad6b0 con 0x7f943c0cd500 2023-10-06T12:34:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f9440ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0ad990 con 0x7f943c159390 2023-10-06T12:34:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35288/0 (socket says 172.21.15.46:35288) 2023-10-06T12:34:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 -- 172.21.15.46:0/237339228 learned_addr learned my addr 172.21.15.46:0/237339228 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943b7fe700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f943c0cd500 0x7f943c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943bfff700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f943c159390 0x7f943c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 -- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f943c159390 msgr2=0x7f943c1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f943c159390 0x7f943c1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 -- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f943c0cd500 msgr2=0x7f943c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f943c0cd500 0x7f943c0b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 -- 172.21.15.46:0/237339228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434009a20 con 0x7f943c158fc0 2023-10-06T12:34:04.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f943affd700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f943000c7e0 tx=0x7f9430007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.607+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430011590 con 0x7f943c158fc0 2023-10-06T12:34:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430011710 con 0x7f943c158fc0 2023-10-06T12:34:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c0adc70 con 0x7f943c158fc0 2023-10-06T12:34:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430011ad0 con 0x7f943c158fc0 2023-10-06T12:34:04.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c153fb0 con 0x7f943c158fc0 2023-10-06T12:34:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c0b3170 con 0x7f943c158fc0 2023-10-06T12:34:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.611+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f943001b3b0 con 0x7f943c158fc0 2023-10-06T12:34:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.615+0000 7f94237fe700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 0x7f9428062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.615+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f943007c480 con 0x7f943c158fc0 2023-10-06T12:34:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.615+0000 7f943b7fe700 1 -- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 msgr2=0x7f9428062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.615+0000 7f943b7fe700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 0x7f9428062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:04.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.623+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f943004c290 con 0x7f943c158fc0 2023-10-06T12:34:04.689 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:04.686+0000 7f122a7af000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:34:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.791+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f943c064590 con 0x7f943c158fc0 2023-10-06T12:34:04.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.791+0000 7f94237fe700 1 -- 172.21.15.46:0/237339228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9430017020 con 0x7f943c158fc0 2023-10-06T12:34:04.797 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T12:34:04.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.795+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 msgr2=0x7f9428062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:04.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.795+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 0x7f9428062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.795+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 msgr2=0x7f943c153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.795+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f943000c7e0 tx=0x7f9430007dc0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 shutdown_connections 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9428060730 0x7f9428062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f943c159390 0x7f943c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f943c158fc0 0x7f943c153800 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 --2- 172.21.15.46:0/237339228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f943c0cd500 0x7f943c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 >> 172.21.15.46:0/237339228 conn(0x7f943c06ad30 msgr2=0x7f943c069810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 shutdown_connections 2023-10-06T12:34:04.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.799+0000 7f9440ee3700 1 -- 172.21.15.46:0/237339228 wait complete. 2023-10-06T12:34:04.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:04.905 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:04.902+0000 7f122a7af000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:04.976 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:04.971+0000 7f7a375c5000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:34:04.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.979+0000 7f9e4d4f0700 1 Processor -- start 2023-10-06T12:34:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.979+0000 7f9e4d4f0700 1 -- start start 2023-10-06T12:34:04.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e4d4f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e48159740 0x7f9e4806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e4d4f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806dbc0 con 0x7f9e480c6aa0 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e4d4f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806cfb0 con 0x7f9e48156cb0 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e4d4f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4806e050 con 0x7f9e48159740 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e467fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9e480c6aa0 0x7f9e48160d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36378/0 (socket says 172.21.15.46:36378) 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e467fc700 1 -- 172.21.15.46:0/1021006413 learned_addr learned my addr 172.21.15.46:0/1021006413 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:04.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e477fe700 1 --2- 172.21.15.46:0/1021006413 >> v2:172.21.15.46:3300/0 conn(0x7f9e48159740 0x7f9e4806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111208182 0 0) 0x7f9e4806dbc0 con 0x7f9e480c6aa0 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e34003570 con 0x7f9e480c6aa0 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e477fe700 1 -- 172.21.15.46:0/1021006413 >> v1:172.21.15.132:6789/0 conn(0x7f9e48156cb0 legacy=0x7f9e480c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947906152 0 0) 0x7f9e4806cfb0 con 0x7f9e48156cb0 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e477fe700 1 -- 172.21.15.46:0/1021006413 >> v1:172.21.15.46:6789/0 conn(0x7f9e480c6aa0 legacy=0x7f9e48160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e477fe700 1 -- 172.21.15.46:0/1021006413 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4806d290 con 0x7f9e48159740 2023-10-06T12:34:04.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.983+0000 7f9e477fe700 1 --2- 172.21.15.46:0/1021006413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e48159740 0x7f9e4806c9b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009e60 tx=0x7f9e40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592e5387fc241f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.989 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:04.986+0000 7f122a7af000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:34:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000e070 con 0x7f9e48159740 2023-10-06T12:34:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000bb60 con 0x7f9e48159740 2023-10-06T12:34:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e45ffb700 1 -- 172.21.15.46:0/1021006413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40012430 con 0x7f9e48159740 2023-10-06T12:34:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/1021006413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e48159740 msgr2=0x7f9e4806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/1021006413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e48159740 0x7f9e4806c9b0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009e60 tx=0x7f9e40009780 comp rx=0 tx=0).stop 2023-10-06T12:34:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/1021006413 shutdown_connections 2023-10-06T12:34:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/1021006413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e48159740 0x7f9e4806c9b0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/1021006413 >> 172.21.15.46:0/1021006413 conn(0x7f9e480cc6b0 msgr2=0x7f9e480c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/1021006413 shutdown_connections 2023-10-06T12:34:04.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.987+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/1021006413 wait complete. 2023-10-06T12:34:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 Processor -- start 2023-10-06T12:34:04.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 -- start start 2023-10-06T12:34:04.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e48156cb0 0x7f9e480a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 0x7f9e48152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480c24d0 con 0x7f9e480c6aa0 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4816c8c0 con 0x7f9e48156cb0 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e4d4f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e481697f0 con 0x7f9e48159740 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35308/0 (socket says 172.21.15.46:35308) 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 -- 172.21.15.46:0/2843226108 learned_addr learned my addr 172.21.15.46:0/2843226108 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e467fc700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e48156cb0 0x7f9e480a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e477fe700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 0x7f9e48152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 msgr2=0x7f9e48152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 0x7f9e48152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e48156cb0 msgr2=0x7f9e480a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:04.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e48156cb0 0x7f9e480a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 -- 172.21.15.46:0/2843226108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40009a20 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e477fe700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 0x7f9e48152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.991+0000 7f9e46ffd700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c7e0 tx=0x7f9e38007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38011590 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38011710 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e48169ad0 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38011ad0 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4816a090 con 0x7f9e480c6aa0 2023-10-06T12:34:04.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.995+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e48064590 con 0x7f9e480c6aa0 2023-10-06T12:34:05.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.999+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f9e3801b3b0 con 0x7f9e480c6aa0 2023-10-06T12:34:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.999+0000 7f9e3effd700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 0x7f9e28062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.999+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f9e3807c580 con 0x7f9e480c6aa0 2023-10-06T12:34:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.999+0000 7f9e467fc700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 msgr2=0x7f9e28062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:05.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:04.999+0000 7f9e467fc700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 0x7f9e28062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:05.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.007+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9e3804c390 con 0x7f9e480c6aa0 2023-10-06T12:34:05.066 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.062+0000 7f122a7af000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:34:05.150 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.138+0000 7f122a7af000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:34:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.203+0000 7f9e467fc700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 msgr2=0x7f9e28062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.203+0000 7f9e467fc700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 0x7f9e28062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:05.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.215+0000 7f7a375c5000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.219+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e480bf3c0 con 0x7f9e480c6aa0 2023-10-06T12:34:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.219+0000 7f9e3effd700 1 -- 172.21.15.46:0/2843226108 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9e38017090 con 0x7f9e480c6aa0 2023-10-06T12:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:05.223 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:34:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 msgr2=0x7f9e28062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 0x7f9e28062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 msgr2=0x7f9e480c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c7e0 tx=0x7f9e38007df0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 shutdown_connections 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9e28060700 0x7f9e28062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e48159740 0x7f9e48152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e48156cb0 0x7f9e480a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 --2- 172.21.15.46:0/2843226108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e480c6aa0 0x7f9e480c1f90 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 >> 172.21.15.46:0/2843226108 conn(0x7f9e480cc6b0 msgr2=0x7f9e48156050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 shutdown_connections 2023-10-06T12:34:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.223+0000 7f9e4d4f0700 1 -- 172.21.15.46:0/2843226108 wait complete. 2023-10-06T12:34:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:34:05.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:05.316 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.311+0000 7f7a375c5000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:34:05.352 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.350+0000 7f122a7af000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:05.393 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.387+0000 7f7a375c5000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:34:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 Processor -- start 2023-10-06T12:34:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 -- start start 2023-10-06T12:34:05.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b100c90b0 con 0x7f4b10158100 2023-10-06T12:34:05.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b100cc980 con 0x7f4b100b02a0 2023-10-06T12:34:05.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.399+0000 7f4b148cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100ccc60 con 0x7f4b100caed0 2023-10-06T12:34:05.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b07fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b10158100 0x7f4b100ca7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36392/0 (socket says 172.21.15.46:36392) 2023-10-06T12:34:05.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b07fff700 1 -- 172.21.15.46:0/477501741 learned_addr learned my addr 172.21.15.46:0/477501741 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0f59e700 1 --2- 172.21.15.46:0/477501741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930982906 0 0) 0x7f4b100c90b0 con 0x7f4b10158100 2023-10-06T12:34:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc003570 con 0x7f4b10158100 2023-10-06T12:34:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568339837 0 0) 0x7f4b100cc980 con 0x7f4b100b02a0 2023-10-06T12:34:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0f59e700 1 -- 172.21.15.46:0/477501741 >> v1:172.21.15.132:6789/0 conn(0x7f4b100b02a0 legacy=0x7f4b100cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0f59e700 1 -- 172.21.15.46:0/477501741 >> v1:172.21.15.46:6789/0 conn(0x7f4b10158100 legacy=0x7f4b100ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0f59e700 1 -- 172.21.15.46:0/477501741 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b100c8d40 con 0x7f4b100caed0 2023-10-06T12:34:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0f59e700 1 --2- 172.21.15.46:0/477501741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009e90 tx=0x7f4b08009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f463106f586d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800e070 con 0x7f4b100caed0 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0800bc30 con 0x7f4b100caed0 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b0e59c700 1 -- 172.21.15.46:0/477501741 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08012430 con 0x7f4b100caed0 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 -- 172.21.15.46:0/477501741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 msgr2=0x7f4b100c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 --2- 172.21.15.46:0/477501741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009e90 tx=0x7f4b08009780 comp rx=0 tx=0).stop 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 -- 172.21.15.46:0/477501741 shutdown_connections 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 --2- 172.21.15.46:0/477501741 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 -- 172.21.15.46:0/477501741 >> 172.21.15.46:0/477501741 conn(0x7f4b100970c0 msgr2=0x7f4b10067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 -- 172.21.15.46:0/477501741 shutdown_connections 2023-10-06T12:34:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.403+0000 7f4b148cc700 1 -- 172.21.15.46:0/477501741 wait complete. 2023-10-06T12:34:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 Processor -- start 2023-10-06T12:34:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 -- start start 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b10158100 0x7f4b100c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b100c4f30 con 0x7f4b100b02a0 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1016a3d0 con 0x7f4b10158100 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b148cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1016cef0 con 0x7f4b100caed0 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b0ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b0ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35322/0 (socket says 172.21.15.46:35322) 2023-10-06T12:34:05.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b0f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b10158100 0x7f4b100c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b0ed9d700 1 -- 172.21.15.46:0/646266065 learned_addr learned my addr 172.21.15.46:0/646266065 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:05.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.407+0000 7f4b07fff700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 -- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 msgr2=0x7f4b100c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 -- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b10158100 msgr2=0x7f4b100c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b10158100 0x7f4b100c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 -- 172.21.15.46:0/646266065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08009a20 con 0x7f4b100b02a0 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b0ed9d700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4b00001f00 tx=0x7f4b00001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00011590 con 0x7f4b100b02a0 2023-10-06T12:34:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000116f0 con 0x7f4b100b02a0 2023-10-06T12:34:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1016d1d0 con 0x7f4b100b02a0 2023-10-06T12:34:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00011ab0 con 0x7f4b100b02a0 2023-10-06T12:34:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1016d790 con 0x7f4b100b02a0 2023-10-06T12:34:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.411+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b10064590 con 0x7f4b100b02a0 2023-10-06T12:34:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.415+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f4b0001b3b0 con 0x7f4b100b02a0 2023-10-06T12:34:05.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.415+0000 7f4b06ffd700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 0x7f4af0062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.415+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f4b0007c320 con 0x7f4b100b02a0 2023-10-06T12:34:05.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.415+0000 7f4b07fff700 1 -- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 msgr2=0x7f4af0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:05.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.415+0000 7f4b07fff700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 0x7f4af0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:05.426 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.422+0000 7f122a7af000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:34:05.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.423+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b0004c280 con 0x7f4b100b02a0 2023-10-06T12:34:05.491 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.487+0000 7f7a375c5000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:34:05.507 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.506+0000 7f122a7af000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:34:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.595+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b100c8960 con 0x7f4b100b02a0 2023-10-06T12:34:05.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.599+0000 7f4b06ffd700 1 -- 172.21.15.46:0/646266065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18156 (secure 0 0 0) 0x7f4b00053e60 con 0x7f4b100b02a0 2023-10-06T12:34:05.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:05.602 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":154,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:59.005299+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/4189294583":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6833/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6832/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/1906903242":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/934339781":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/2228099525":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/3221426589":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/511843182":"2023-10-07T12:33:59.005230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.599+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 msgr2=0x7f4af0062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.599+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 0x7f4af0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.599+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 msgr2=0x7f4b10169e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.599+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4b00001f00 tx=0x7f4b00001060 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 shutdown_connections 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4af0060700 0x7f4af0062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b10158100 0x7f4b100c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b100caed0 0x7f4b100c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 --2- 172.21.15.46:0/646266065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b100b02a0 0x7f4b10169e90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 >> 172.21.15.46:0/646266065 conn(0x7f4b100970c0 msgr2=0x7f4b100afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 shutdown_connections 2023-10-06T12:34:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.603+0000 7f4b148cc700 1 -- 172.21.15.46:0/646266065 wait complete. 2023-10-06T12:34:05.625 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:34:05.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:05.693 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.690+0000 7f122a7af000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:34:05.734 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.731+0000 7f7a375c5000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:05.777 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:05.774+0000 7f122a7af000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:34:05.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 Processor -- start 2023-10-06T12:34:05.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 -- start start 2023-10-06T12:34:05.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb0500aaf30 0x7fb050067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0500679f0 con 0x7fb050156e20 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0500acd60 con 0x7fb050067590 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb05648e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0500ad040 con 0x7fb0500aaf30 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.779+0000 7fb055c8d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb050067590 0x7fb050068a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34318/0 (socket says 172.21.15.46:34318) 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb055c8d700 1 -- 172.21.15.46:0/2567746637 learned_addr learned my addr 172.21.15.46:0/2567746637 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:05.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb054c8b700 1 --2- 172.21.15.46:0/2567746637 >> v2:172.21.15.132:3300/0 conn(0x7fb0500aaf30 0x7fb050067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526122531 0 0) 0x7fb04c001040 con 0x7fb050156e20 2023-10-06T12:34:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb03c003570 con 0x7fb050156e20 2023-10-06T12:34:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968893549 0 0) 0x7fb0500acd60 con 0x7fb050067590 2023-10-06T12:34:05.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c001040 con 0x7fb050067590 2023-10-06T12:34:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb054c8b700 1 -- 172.21.15.46:0/2567746637 >> v1:172.21.15.132:6789/0 conn(0x7fb050067590 legacy=0x7fb050068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb054c8b700 1 -- 172.21.15.46:0/2567746637 >> v1:172.21.15.46:6789/0 conn(0x7fb050156e20 legacy=0x7fb05006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb054c8b700 1 -- 172.21.15.46:0/2567746637 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500ad320 con 0x7fb0500aaf30 2023-10-06T12:34:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb054c8b700 1 --2- 172.21.15.46:0/2567746637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050067050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098e0 tx=0x7fb0400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8386ae0bf05732 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:05.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011630 con 0x7fb0500aaf30 2023-10-06T12:34:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040011790 con 0x7fb0500aaf30 2023-10-06T12:34:05.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb047fff700 1 -- 172.21.15.46:0/2567746637 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011b30 con 0x7fb0500aaf30 2023-10-06T12:34:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 -- 172.21.15.46:0/2567746637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 msgr2=0x7fb050067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 --2- 172.21.15.46:0/2567746637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050067050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb0400098e0 tx=0x7fb0400091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 -- 172.21.15.46:0/2567746637 shutdown_connections 2023-10-06T12:34:05.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 --2- 172.21.15.46:0/2567746637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050067050 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 -- 172.21.15.46:0/2567746637 >> 172.21.15.46:0/2567746637 conn(0x7fb050062ef0 msgr2=0x7fb0500cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 -- 172.21.15.46:0/2567746637 shutdown_connections 2023-10-06T12:34:05.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.783+0000 7fb05648e700 1 -- 172.21.15.46:0/2567746637 wait complete. 2023-10-06T12:34:05.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 Processor -- start 2023-10-06T12:34:05.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 -- start start 2023-10-06T12:34:05.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 0x7fb050169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb050156e20 0x7fb0500ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501542f0 con 0x7fb050156e20 2023-10-06T12:34:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05016a250 con 0x7fb0500aaf30 2023-10-06T12:34:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb05648e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05016ce40 con 0x7fb050067590 2023-10-06T12:34:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.787+0000 7fb054c8b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb055c8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb050156e20 0x7fb0500ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb054c8b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36192/0 (socket says 172.21.15.46:36192) 2023-10-06T12:34:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb054c8b700 1 -- 172.21.15.46:0/1500623059 learned_addr learned my addr 172.21.15.46:0/1500623059 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 0x7fb050169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 msgr2=0x7fb050153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb050156e20 msgr2=0x7fb0500ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb050156e20 0x7fb0500ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:05.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 -- 172.21.15.46:0/1500623059 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009420 con 0x7fb050067590 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb054c8b700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05548c700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 0x7fb050169d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7c0 tx=0x7fb04c007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c010510 con 0x7fb050067590 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c010690 con 0x7fb050067590 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05016d120 con 0x7fb050067590 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c010a50 con 0x7fb050067590 2023-10-06T12:34:05.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0500c4560 con 0x7fb050067590 2023-10-06T12:34:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.791+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0500c4780 con 0x7fb050067590 2023-10-06T12:34:05.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.795+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fb04c01b3b0 con 0x7fb050067590 2023-10-06T12:34:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.795+0000 7fb0457fa700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 0x7fb034062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.795+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fb04c07c690 con 0x7fb050067590 2023-10-06T12:34:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.795+0000 7fb054c8b700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 msgr2=0x7fb034062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.795+0000 7fb054c8b700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 0x7fb034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:05.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.803+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb04c04c4a0 con 0x7fb050067590 2023-10-06T12:34:05.819 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.815+0000 7f7a375c5000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:34:05.913 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:05.907+0000 7f7a375c5000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:34:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.999+0000 7fb054c8b700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 msgr2=0x7fb034062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:05.999+0000 7fb054c8b700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 0x7fb034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:06.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.011+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0500679f0 con 0x7fb050067590 2023-10-06T12:34:06.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.011+0000 7fb0457fa700 1 -- 172.21.15.46:0/1500623059 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb0500679f0 con 0x7fb050067590 2023-10-06T12:34:06.017 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 msgr2=0x7fb034062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 0x7fb034062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 msgr2=0x7fb050169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 0x7fb050169d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7c0 tx=0x7fb04c007dd0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 shutdown_connections 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb034060700 0x7fb034062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb050156e20 0x7fb0500ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0500aaf30 0x7fb050153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 --2- 172.21.15.46:0/1500623059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb050067590 0x7fb050169d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 >> 172.21.15.46:0/1500623059 conn(0x7fb050062ef0 msgr2=0x7fb050068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 shutdown_connections 2023-10-06T12:34:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.015+0000 7fb05648e700 1 -- 172.21.15.46:0/1500623059 wait complete. 2023-10-06T12:34:06.022 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:34:06.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:06.126 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:06.123+0000 7f7a375c5000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:34:06.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.191+0000 7f9683fff700 1 Processor -- start 2023-10-06T12:34:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.191+0000 7f9683fff700 1 -- start start 2023-10-06T12:34:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f9683fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9684158dd0 0x7f9684067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f9683fff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840679f0 con 0x7f9684067590 2023-10-06T12:34:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f9683fff700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968415d6c0 con 0x7f96841591a0 2023-10-06T12:34:06.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f9683fff700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840699c0 con 0x7f9684158dd0 2023-10-06T12:34:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f96837fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9684067590 0x7f968406c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36406/0 (socket says 172.21.15.46:36406) 2023-10-06T12:34:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f96837fe700 1 -- 172.21.15.46:0/1752909118 learned_addr learned my addr 172.21.15.46:0/1752909118 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f9682ffd700 1 --2- 172.21.15.46:0/1752909118 >> v2:172.21.15.132:3300/0 conn(0x7f9684158dd0 0x7f9684067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.195+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862665011 0 0) 0x7f967c001490 con 0x7f9684067590 2023-10-06T12:34:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9670003570 con 0x7f9684067590 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640917537 0 0) 0x7f968415d6c0 con 0x7f96841591a0 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f967c001490 con 0x7f96841591a0 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9682ffd700 1 -- 172.21.15.46:0/1752909118 >> v1:172.21.15.132:6789/0 conn(0x7f96841591a0 legacy=0x7f9684068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9682ffd700 1 -- 172.21.15.46:0/1752909118 >> v1:172.21.15.46:6789/0 conn(0x7f9684067590 legacy=0x7f968406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9682ffd700 1 -- 172.21.15.46:0/1752909118 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968415cd90 con 0x7f9684158dd0 2023-10-06T12:34:06.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9682ffd700 1 --2- 172.21.15.46:0/1752909118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9684158dd0 0x7f9684067050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f96740098e0 tx=0x7f96740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf2bb4b7404e77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:06.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674011630 con 0x7f9684158dd0 2023-10-06T12:34:06.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674011790 con 0x7f9684158dd0 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f96827fc700 1 -- 172.21.15.46:0/1752909118 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674011b30 con 0x7f9684158dd0 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 -- 172.21.15.46:0/1752909118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9684158dd0 msgr2=0x7f9684067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 --2- 172.21.15.46:0/1752909118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9684158dd0 0x7f9684067050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f96740098e0 tx=0x7f96740091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 -- 172.21.15.46:0/1752909118 shutdown_connections 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 --2- 172.21.15.46:0/1752909118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9684158dd0 0x7f9684067050 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 -- 172.21.15.46:0/1752909118 >> 172.21.15.46:0/1752909118 conn(0x7f9684062ef0 msgr2=0x7f96840cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 -- 172.21.15.46:0/1752909118 shutdown_connections 2023-10-06T12:34:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.199+0000 7f9683fff700 1 -- 172.21.15.46:0/1752909118 wait complete. 2023-10-06T12:34:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 Processor -- start 2023-10-06T12:34:06.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 -- start start 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9684158dd0 0x7f9684157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96841591a0 0x7f968406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c0d10 con 0x7f9684067590 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c0100 con 0x7f96841591a0 2023-10-06T12:34:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9683fff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c03e0 con 0x7f9684158dd0 2023-10-06T12:34:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9682ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9682ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35344/0 (socket says 172.21.15.46:35344) 2023-10-06T12:34:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f9682ffd700 1 -- 172.21.15.46:0/2805745398 learned_addr learned my addr 172.21.15.46:0/2805745398 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f96837fe700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96841591a0 0x7f968406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.203+0000 7f967bfff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9684158dd0 0x7f9684157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9684158dd0 msgr2=0x7f9684157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9684158dd0 0x7f9684157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96841591a0 msgr2=0x7f968406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96841591a0 0x7f968406dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 -- 172.21.15.46:0/2805745398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674009420 con 0x7f9684067590 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9682ffd700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f968406ab70 tx=0x7f9674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967401f070 con 0x7f9684067590 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96840c06c0 con 0x7f9684067590 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674000bc0 con 0x7f9684067590 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674025420 con 0x7f9684067590 2023-10-06T12:34:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968415bc00 con 0x7f9684067590 2023-10-06T12:34:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.207+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9684064590 con 0x7f9684067590 2023-10-06T12:34:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f9674023070 con 0x7f9684067590 2023-10-06T12:34:06.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f967affd700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 0x7f9664062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f967407e2f0 con 0x7f9684067590 2023-10-06T12:34:06.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f967bfff700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 msgr2=0x7f9664062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:06.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f967bfff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 0x7f9664062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:06.216 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:06.211+0000 7f7a375c5000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:34:06.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.215+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f967404e100 con 0x7f9684067590 2023-10-06T12:34:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.415+0000 7f967bfff700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 msgr2=0x7f9664062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.415+0000 7f967bfff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 0x7f9664062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:06.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.423+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f968404b5c0 con 0x7f9684067590 2023-10-06T12:34:06.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f967affd700 1 -- 172.21.15.46:0/2805745398 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f9674051720 con 0x7f9684067590 2023-10-06T12:34:06.431 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:06.431 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:33:40.245290+0000","modified":"2023-10-06T12:33:43.824858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8796},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8796":{"gid":8796,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/790293012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":790293012},{"type":"v1","addr":"172.21.15.46:6835","nonce":790293012}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T12:34:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 msgr2=0x7f9664062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 0x7f9664062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 msgr2=0x7f96840c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f968406ab70 tx=0x7f9674017000 comp rx=0 tx=0).stop 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 shutdown_connections 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f9664060730 0x7f9664062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96841591a0 0x7f968406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9684158dd0 0x7f9684157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 --2- 172.21.15.46:0/2805745398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9684067590 0x7f96840c2450 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.427+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 >> 172.21.15.46:0/2805745398 conn(0x7f9684062ef0 msgr2=0x7f96840c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.431+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 shutdown_connections 2023-10-06T12:34:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.431+0000 7f9683fff700 1 -- 172.21.15.46:0/2805745398 wait complete. 2023-10-06T12:34:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 76 2023-10-06T12:34:06.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:06.502 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:06.498+0000 7f122a7af000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:34:06.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.611+0000 7fa79f46e700 1 Processor -- start 2023-10-06T12:34:06.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.611+0000 7fa79f46e700 1 -- start start 2023-10-06T12:34:06.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79f46e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa798155480 0x7fa7980c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79f46e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa7980c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79f46e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79806dd30 con 0x7fa7980c2300 2023-10-06T12:34:06.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79f46e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7980caa10 con 0x7fa798155480 2023-10-06T12:34:06.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79f46e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79806ce80 con 0x7fa798155850 2023-10-06T12:34:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79da0b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7980c2300 0x7fa7980c2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36422/0 (socket says 172.21.15.46:36422) 2023-10-06T12:34:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79da0b700 1 -- 172.21.15.46:0/2486292880 learned_addr learned my addr 172.21.15.46:0/2486292880 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79ca09700 1 --2- 172.21.15.46:0/2486292880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa7980c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.615+0000 7fa79d20a700 1 --2- 172.21.15.46:0/2486292880 >> v2:172.21.15.46:3300/0 conn(0x7fa798155480 0x7fa7980c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa78ffff700 1 -- 172.21.15.46:0/2486292880 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605550544 0 0) 0x7fa79806dd30 con 0x7fa7980c2300 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa78ffff700 1 -- 172.21.15.46:0/2486292880 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa784003570 con 0x7fa7980c2300 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79ca09700 1 -- 172.21.15.46:0/2486292880 >> v2:172.21.15.46:3300/0 conn(0x7fa798155480 msgr2=0x7fa7980c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79ca09700 1 --2- 172.21.15.46:0/2486292880 >> v2:172.21.15.46:3300/0 conn(0x7fa798155480 0x7fa7980c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79ca09700 1 -- 172.21.15.46:0/2486292880 >> v1:172.21.15.46:6789/0 conn(0x7fa7980c2300 legacy=0x7fa7980c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79ca09700 1 -- 172.21.15.46:0/2486292880 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79806d120 con 0x7fa798155850 2023-10-06T12:34:06.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79ca09700 1 --2- 172.21.15.46:0/2486292880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa7980c19a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098e0 tx=0x7fa7880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bab012e610a359 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:06.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa78ffff700 1 -- 172.21.15.46:0/2486292880 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788011630 con 0x7fa798155850 2023-10-06T12:34:06.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa78ffff700 1 -- 172.21.15.46:0/2486292880 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788011790 con 0x7fa798155850 2023-10-06T12:34:06.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa78ffff700 1 -- 172.21.15.46:0/2486292880 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788011b30 con 0x7fa798155850 2023-10-06T12:34:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 -- 172.21.15.46:0/2486292880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 msgr2=0x7fa7980c19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 --2- 172.21.15.46:0/2486292880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa7980c19a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098e0 tx=0x7fa7880091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 -- 172.21.15.46:0/2486292880 shutdown_connections 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 --2- 172.21.15.46:0/2486292880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa7980c19a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 --2- 172.21.15.46:0/2486292880 >> v2:172.21.15.46:3300/0 conn(0x7fa798155480 0x7fa7980c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 -- 172.21.15.46:0/2486292880 >> 172.21.15.46:0/2486292880 conn(0x7fa7980cbe30 msgr2=0x7fa7980cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 -- 172.21.15.46:0/2486292880 shutdown_connections 2023-10-06T12:34:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.619+0000 7fa79f46e700 1 -- 172.21.15.46:0/2486292880 wait complete. 2023-10-06T12:34:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.623+0000 7fa79f46e700 1 Processor -- start 2023-10-06T12:34:06.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.623+0000 7fa79f46e700 1 -- start start 2023-10-06T12:34:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.623+0000 7fa79f46e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7980c2300 0x7fa7980ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.623+0000 7fa79f46e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.623+0000 7fa79f46e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa798069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79f46e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798156660 con 0x7fa798155480 2023-10-06T12:34:06.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79f46e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980abbe0 con 0x7fa7980c2300 2023-10-06T12:34:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79f46e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b18a0 con 0x7fa798155850 2023-10-06T12:34:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35358/0 (socket says 172.21.15.46:35358) 2023-10-06T12:34:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 -- 172.21.15.46:0/707675859 learned_addr learned my addr 172.21.15.46:0/707675859 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:06.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79da0b700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa798069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79d20a700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7980c2300 0x7fa7980ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 -- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 msgr2=0x7fa798069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa798069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 -- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7980c2300 msgr2=0x7fa7980ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7980c2300 0x7fa7980ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 -- 172.21.15.46:0/707675859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009420 con 0x7fa798155480 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79ca09700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa788009ed0 tx=0x7fa788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788000d10 con 0x7fa798155480 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788011b10 con 0x7fa798155480 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788025660 con 0x7fa798155480 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa798155d30 con 0x7fa798155480 2023-10-06T12:34:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.627+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981562f0 con 0x7fa798155480 2023-10-06T12:34:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.631+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa798156810 con 0x7fa798155480 2023-10-06T12:34:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.631+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 81) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fa788025a50 con 0x7fa798155480 2023-10-06T12:34:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.631+0000 7fa78d7fa700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 0x7fa77c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.631+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fa78807e760 con 0x7fa798155480 2023-10-06T12:34:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.635+0000 7fa79d20a700 1 -- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 msgr2=0x7fa77c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.635+0000 7fa79d20a700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 0x7fa77c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.639+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa78804e570 con 0x7fa798155480 2023-10-06T12:34:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.807+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa798063f90 con 0x7fa798155480 2023-10-06T12:34:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.811+0000 7fa78d7fa700 1 -- 172.21.15.46:0/707675859 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18156 (secure 0 0 0) 0x7fa788051b90 con 0x7fa798155480 2023-10-06T12:34:06.815 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:06.815 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":154,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:33:59.005299+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:33:38.215933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T12:33:39.244553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/4189294583":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6833/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:6832/32974":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/1906903242":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/934339781":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/2228099525":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/3221426589":"2023-10-07T12:33:59.005230+0000","172.21.15.46:0/511843182":"2023-10-07T12:33:59.005230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 msgr2=0x7fa77c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 0x7fa77c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 msgr2=0x7fa79813e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa788009ed0 tx=0x7fa788017000 comp rx=0 tx=0).stop 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 shutdown_connections 2023-10-06T12:34:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa77c060700 0x7fa77c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa798155850 0x7fa798069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa798155480 0x7fa79813e400 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 --2- 172.21.15.46:0/707675859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7980c2300 0x7fa7980ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 >> 172.21.15.46:0/707675859 conn(0x7fa7980cbe30 msgr2=0x7fa798068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 shutdown_connections 2023-10-06T12:34:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.815+0000 7fa79f46e700 1 -- 172.21.15.46:0/707675859 wait complete. 2023-10-06T12:34:06.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T12:34:06.993 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:06.987+0000 7f7a375c5000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:34:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.991+0000 7f8cec2cd700 1 Processor -- start 2023-10-06T12:34:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.991+0000 7f8cec2cd700 1 -- start start 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cec2cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8ce40cec00 0x7f8ce4163e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cec2cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce40cc150 con 0x7f8ce415c770 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cec2cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce40cb540 con 0x7f8ce40c5ff0 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cec2cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce40cfbb0 con 0x7f8ce40cec00 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea069700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ce415c770 0x7f8ce40cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36430/0 (socket says 172.21.15.46:36430) 2023-10-06T12:34:06.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea069700 1 -- 172.21.15.46:0/2897520502 learned_addr learned my addr 172.21.15.46:0/2897520502 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:06.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea86a700 1 --2- 172.21.15.46:0/2897520502 >> v2:172.21.15.132:3300/0 conn(0x7f8ce40cec00 0x7f8ce4163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:06.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea86a700 1 -- 172.21.15.46:0/2897520502 >> v1:172.21.15.132:6789/0 conn(0x7f8ce40c5ff0 legacy=0x7f8ce40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea86a700 1 -- 172.21.15.46:0/2897520502 >> v1:172.21.15.46:6789/0 conn(0x7f8ce415c770 legacy=0x7f8ce40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:06.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea86a700 1 -- 172.21.15.46:0/2897520502 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce40cb820 con 0x7f8ce40cec00 2023-10-06T12:34:07.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.995+0000 7f8cea86a700 1 --2- 172.21.15.46:0/2897520502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40cec00 0x7f8ce4163e60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009e60 tx=0x7f8cdc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c39872178c98f90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8ce9067700 1 -- 172.21.15.46:0/2897520502 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00e070 con 0x7f8ce40cec00 2023-10-06T12:34:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8ce9067700 1 -- 172.21.15.46:0/2897520502 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00bc30 con 0x7f8ce40cec00 2023-10-06T12:34:07.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8ce9067700 1 -- 172.21.15.46:0/2897520502 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc012430 con 0x7f8ce40cec00 2023-10-06T12:34:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2897520502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40cec00 msgr2=0x7f8ce4163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2897520502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40cec00 0x7f8ce4163e60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009e60 tx=0x7f8cdc009780 comp rx=0 tx=0).stop 2023-10-06T12:34:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2897520502 shutdown_connections 2023-10-06T12:34:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2897520502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce40cec00 0x7f8ce4163e60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2897520502 >> 172.21.15.46:0/2897520502 conn(0x7f8ce4099480 msgr2=0x7f8ce406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2897520502 shutdown_connections 2023-10-06T12:34:07.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:06.999+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2897520502 wait complete. 2023-10-06T12:34:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.003+0000 7f8cec2cd700 1 Processor -- start 2023-10-06T12:34:07.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.003+0000 7f8cec2cd700 1 -- start start 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.003+0000 7f8cec2cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce40cec00 0x7f8ce41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce415c770 0x7f8ce41537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4153d20 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce416db30 con 0x7f8ce415c770 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce415ad70 con 0x7f8ce40cec00 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35360/0 (socket says 172.21.15.46:35360) 2023-10-06T12:34:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 -- 172.21.15.46:0/2606204318 learned_addr learned my addr 172.21.15.46:0/2606204318 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 -- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce40cec00 msgr2=0x7f8ce41532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce40cec00 0x7f8ce41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 -- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce415c770 msgr2=0x7f8ce41537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce415c770 0x7f8ce41537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 -- 172.21.15.46:0/2606204318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc009a20 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cea069700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c5c0 tx=0x7f8ce0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00072e0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce0007460 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce416d6a0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00057f0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce416d050 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.007+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce40660f0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.011+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 82) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f8ce0005be0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.011+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 7219+0+0 (secure 0 0 0) 0x7f8ce007bee0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.015+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ce004c1b0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.107+0000 7f7a020b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.107+0000 7f7a020b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.107+0000 7f7a020b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.107+0000 7f7a020b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.113 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.107+0000 7f7a020b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.111+0000 7f79ff8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.111+0000 7f79ff8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.111+0000 7f79ff8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.111+0000 7f79ff8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:07.111+0000 7f79ff8b0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:07.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.243+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8ce40686e0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.250 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T12:34:07.247+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T12:34:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cda7fc700 1 -- 172.21.15.46:0/2606204318 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8ce004f7d0 con 0x7f8ce40c5ff0 2023-10-06T12:34:07.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 msgr2=0x7f8ce40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c5c0 tx=0x7f8ce0007e40 comp rx=0 tx=0).stop 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 shutdown_connections 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ce415c770 0x7f8ce41537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ce40cec00 0x7f8ce41532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 --2- 172.21.15.46:0/2606204318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce40c5ff0 0x7f8ce40b4dc0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.259+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 >> 172.21.15.46:0/2606204318 conn(0x7f8ce4099480 msgr2=0x7f8ce4155020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.263+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 shutdown_connections 2023-10-06T12:34:07.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.263+0000 7f8cec2cd700 1 -- 172.21.15.46:0/2606204318 wait complete. 2023-10-06T12:34:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T12:34:07.269 INFO:tasks.ceph.mds.c.smithi046.stderr: -2497> 2023-10-06T12:33:45.748+0000 7efe2792a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:34:07.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T12:34:07.312 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:34:07.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.435+0000 7ff829b85700 1 Processor -- start 2023-10-06T12:34:07.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.435+0000 7ff829b85700 1 -- start start 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff829b85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8240ccc00 0x7ff8240ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff829b85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 0x7ff8240cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff829b85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240cb4a0 con 0x7ff8240b23f0 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff829b85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8240cc780 con 0x7ff8240ccc00 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff829b85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240cb6e0 con 0x7ff8240b27c0 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8240ccc00 0x7ff8240ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff8237fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8240b23f0 0x7ff8240cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36450/0 (socket says 172.21.15.46:36450) 2023-10-06T12:34:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff8237fe700 1 -- 172.21.15.46:0/2268901082 learned_addr learned my addr 172.21.15.46:0/2268901082 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff823fff700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 0x7ff8240cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 -- 172.21.15.46:0/2268901082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 msgr2=0x7ff8240cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 0x7ff8240cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 -- 172.21.15.46:0/2268901082 >> v1:172.21.15.46:6789/0 conn(0x7ff8240b23f0 legacy=0x7ff8240cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 -- 172.21.15.46:0/2268901082 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240cb9c0 con 0x7ff8240ccc00 2023-10-06T12:34:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff822ffd700 1 -- 172.21.15.46:0/2268901082 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206074801 0 0) 0x7ff814001290 con 0x7ff8240b23f0 2023-10-06T12:34:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff823fff700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 0x7ff8240cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff81bfff700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ccc00 0x7ff8240ce4d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009e60 tx=0x7ff80c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbacf5d101e619 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff822ffd700 1 -- 172.21.15.46:0/2268901082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00e070 con 0x7ff8240ccc00 2023-10-06T12:34:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff822ffd700 1 -- 172.21.15.46:0/2268901082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c00bc30 con 0x7ff8240ccc00 2023-10-06T12:34:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.439+0000 7ff822ffd700 1 -- 172.21.15.46:0/2268901082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c012430 con 0x7ff8240ccc00 2023-10-06T12:34:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- 172.21.15.46:0/2268901082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ccc00 msgr2=0x7ff8240ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ccc00 0x7ff8240ce4d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009e60 tx=0x7ff80c009780 comp rx=0 tx=0).stop 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- 172.21.15.46:0/2268901082 shutdown_connections 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240b27c0 0x7ff8240cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 --2- 172.21.15.46:0/2268901082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240ccc00 0x7ff8240ce4d0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- 172.21.15.46:0/2268901082 >> 172.21.15.46:0/2268901082 conn(0x7ff824099440 msgr2=0x7ff82406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- 172.21.15.46:0/2268901082 shutdown_connections 2023-10-06T12:34:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- 172.21.15.46:0/2268901082 wait complete. 2023-10-06T12:34:07.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 Processor -- start 2023-10-06T12:34:07.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 -- start start 2023-10-06T12:34:07.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.443+0000 7ff829b85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 0x7ff82416a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 0x7ff824159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:07.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82415c140 con 0x7ff8240b23f0 2023-10-06T12:34:07.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82415b530 con 0x7ff8240b27c0 2023-10-06T12:34:07.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82415b810 con 0x7ff8240ccc00 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35496/0 (socket says 172.21.15.46:35496) 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 -- 172.21.15.46:0/3252997489 learned_addr learned my addr 172.21.15.46:0/3252997489 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff823fff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 0x7ff824159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff81bfff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 0x7ff82416a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 -- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 msgr2=0x7ff824159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 0x7ff824159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 -- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 msgr2=0x7ff82416a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 0x7ff82416a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 -- 172.21.15.46:0/3252997489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c009a20 con 0x7ff8240b23f0 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff81bfff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 0x7ff82416a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff823fff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 0x7ff824159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:07.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff8237fe700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c5e0 tx=0x7ff814007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:07.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140072e0 con 0x7ff8240b23f0 2023-10-06T12:34:07.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff814007440 con 0x7ff8240b23f0 2023-10-06T12:34:07.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82415baf0 con 0x7ff8240b23f0 2023-10-06T12:34:07.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8140057f0 con 0x7ff8240b23f0 2023-10-06T12:34:07.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8240ca040 con 0x7ff8240b23f0 2023-10-06T12:34:07.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.447+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8240660f0 con 0x7ff8240b23f0 2023-10-06T12:34:07.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.451+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 82) v1 ==== 79670+0+0 (secure 0 0 0) 0x7ff814005c00 con 0x7ff8240b23f0 2023-10-06T12:34:07.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.451+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7305+0+0 (secure 0 0 0) 0x7ff81407be10 con 0x7ff8240b23f0 2023-10-06T12:34:07.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.459+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff81407f030 con 0x7ff8240b23f0 2023-10-06T12:34:07.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:07.659+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8240cb4a0 con 0x7ff8240b23f0 2023-10-06T12:34:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.019+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 83) v1 ==== 79797+0+0 (secure 0 0 0) 0x7ff81404bb40 con 0x7ff8240b23f0 2023-10-06T12:34:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.019+0000 7ff81b7fe700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 0x7ff804068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.023+0000 7ff81bfff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 0x7ff804068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.027+0000 7ff81bfff700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 0x7ff804068e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009e60 tx=0x7ff80c019000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.287+0000 7ff81b7fe700 1 -- 172.21.15.46:0/3252997489 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff814068e70 con 0x7ff8240b23f0 2023-10-06T12:34:08.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 msgr2=0x7ff804068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 0x7ff804068e50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009e60 tx=0x7ff80c019000 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 msgr2=0x7ff8240c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c5e0 tx=0x7ff814007e80 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 shutdown_connections 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff804066a70 0x7ff804068e50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8240ccc00 0x7ff824159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8240b27c0 0x7ff82416a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 --2- 172.21.15.46:0/3252997489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8240b23f0 0x7ff8240c9b00 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 >> 172.21.15.46:0/3252997489 conn(0x7ff824099440 msgr2=0x7ff8240c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 shutdown_connections 2023-10-06T12:34:08.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.295+0000 7ff829b85700 1 -- 172.21.15.46:0/3252997489 wait complete. 2023-10-06T12:34:08.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T12:34:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.543+0000 7f12a8fcc700 1 Processor -- start 2023-10-06T12:34:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.543+0000 7f12a8fcc700 1 -- start start 2023-10-06T12:34:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a8fcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12a4069770 0x7f12a40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a8fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 0x7f12a40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a8fcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40b5eb0 con 0x7f12a4158de0 2023-10-06T12:34:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a8fcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40cef30 con 0x7f12a4069770 2023-10-06T12:34:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a8fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40cf4f0 con 0x7f12a41591b0 2023-10-06T12:34:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a2ffd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12a4069770 0x7f12a40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 0x7f12a40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a37fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f12a4158de0 0x7f12a4069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:34354/0 (socket says 172.21.15.46:34354) 2023-10-06T12:34:08.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a37fe700 1 -- 172.21.15.46:0/29444760 learned_addr learned my addr 172.21.15.46:0/29444760 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 -- 172.21.15.46:0/29444760 >> v2:172.21.15.132:3300/0 conn(0x7f12a4069770 msgr2=0x7f12a40b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 --2- 172.21.15.46:0/29444760 >> v2:172.21.15.132:3300/0 conn(0x7f12a4069770 0x7f12a40b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 -- 172.21.15.46:0/29444760 >> v1:172.21.15.132:6789/0 conn(0x7f12a4158de0 legacy=0x7f12a4069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 -- 172.21.15.46:0/29444760 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40ccac0 con 0x7f12a41591b0 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a27fc700 1 -- 172.21.15.46:0/29444760 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516329013 0 0) 0x7f1294001110 con 0x7f12a4158de0 2023-10-06T12:34:08.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a3fff700 1 --2- 172.21.15.46:0/29444760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 0x7f12a40b2e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e60 tx=0x7f129c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378b4f5657deeec7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a27fc700 1 -- 172.21.15.46:0/29444760 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00e070 con 0x7f12a41591b0 2023-10-06T12:34:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a27fc700 1 -- 172.21.15.46:0/29444760 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00bb60 con 0x7f12a41591b0 2023-10-06T12:34:08.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.547+0000 7f12a27fc700 1 -- 172.21.15.46:0/29444760 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c012430 con 0x7f12a41591b0 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- 172.21.15.46:0/29444760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 msgr2=0x7f12a40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/29444760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 0x7f12a40b2e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e60 tx=0x7f129c009780 comp rx=0 tx=0).stop 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- 172.21.15.46:0/29444760 shutdown_connections 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/29444760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a41591b0 0x7f12a40b2e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/29444760 >> v2:172.21.15.132:3300/0 conn(0x7f12a4069770 0x7f12a40b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- 172.21.15.46:0/29444760 >> 172.21.15.46:0/29444760 conn(0x7f12a4068390 msgr2=0x7f12a40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:08.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- 172.21.15.46:0/29444760 shutdown_connections 2023-10-06T12:34:08.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- 172.21.15.46:0/29444760 wait complete. 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 Processor -- start 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.551+0000 7f12a8fcc700 1 -- start start 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a4069770 0x7f12a41411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 0x7f12a4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a416b500 con 0x7f12a41591b0 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a416a8f0 con 0x7f12a4158de0 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a416abd0 con 0x7f12a4069770 2023-10-06T12:34:08.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35508/0 (socket says 172.21.15.46:35508) 2023-10-06T12:34:08.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 -- 172.21.15.46:0/1789119084 learned_addr learned my addr 172.21.15.46:0/1789119084 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a37fe700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a4069770 0x7f12a41411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a2ffd700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 0x7f12a4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 -- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a4069770 msgr2=0x7f12a41411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a4069770 0x7f12a41411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 -- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 msgr2=0x7f12a4157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 0x7f12a4157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 -- 172.21.15.46:0/1789119084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009a20 con 0x7f12a41591b0 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a2ffd700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 0x7f12a4157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:08.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a3fff700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b2710 tx=0x7f129c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c027470 con 0x7f12a41591b0 2023-10-06T12:34:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c027610 con 0x7f12a41591b0 2023-10-06T12:34:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a416aeb0 con 0x7f12a41591b0 2023-10-06T12:34:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0279d0 con 0x7f12a41591b0 2023-10-06T12:34:08.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.555+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a41416f0 con 0x7f12a41591b0 2023-10-06T12:34:08.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.559+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4064590 con 0x7f12a41591b0 2023-10-06T12:34:08.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.559+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 83) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f129c024070 con 0x7f12a41591b0 2023-10-06T12:34:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.559+0000 7f128b7fe700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 0x7f1290062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:08.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.559+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f129c07f680 con 0x7f12a41591b0 2023-10-06T12:34:08.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.563+0000 7f12a37fe700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 0x7f1290062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.563+0000 7f12a37fe700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 0x7f1290062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12940018a0 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.567+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f129c04f810 con 0x7f12a41591b0 2023-10-06T12:34:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:08.739+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f12a40ccda0 con 0x7f12a41591b0 2023-10-06T12:34:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.319+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f129c003040 con 0x7f12a41591b0 2023-10-06T12:34:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.319+0000 7f128b7fe700 1 -- 172.21.15.46:0/1789119084 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) v1 ==== 181+0+0 (secure 0 0 0) 0x7f129c062110 con 0x7f12a41591b0 2023-10-06T12:34:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 msgr2=0x7f1290062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 0x7f1290062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f12940018a0 tx=0x7f1294009040 comp rx=0 tx=0).stop 2023-10-06T12:34:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 msgr2=0x7f12a41583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b2710 tx=0x7f129c019000 comp rx=0 tx=0).stop 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 shutdown_connections 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f12900603a0 0x7f1290062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a41591b0 0x7f12a41583b0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12a4158de0 0x7f12a4157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 --2- 172.21.15.46:0/1789119084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12a4069770 0x7f12a41411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 >> 172.21.15.46:0/1789119084 conn(0x7f12a4068390 msgr2=0x7f12a40b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 shutdown_connections 2023-10-06T12:34:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.323+0000 7f12a8fcc700 1 -- 172.21.15.46:0/1789119084 wait complete. 2023-10-06T12:34:09.329 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T12:34:09.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T12:34:09.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.515+0000 7f5c8022e700 1 Processor -- start 2023-10-06T12:34:09.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.515+0000 7f5c8022e700 1 -- start start 2023-10-06T12:34:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5c781592d0 0x7f5c780acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c780c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c780c6780 0x7f5c780cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c5db0 con 0x7f5c780c6780 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c5ab0 con 0x7f5c781592d0 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c8022e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780ad2f0 con 0x7f5c781596a0 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7e7cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c780c6780 0x7f5c780cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5c781592d0 0x7f5c780acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7d7c9700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c780c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5c781592d0 0x7f5c780acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:36386/0 (socket says 172.21.15.46:36386) 2023-10-06T12:34:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 -- 172.21.15.46:0/2868335153 learned_addr learned my addr 172.21.15.46:0/2868335153 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 -- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 msgr2=0x7f5c780c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c780c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.519+0000 7f5c7dfca700 1 -- 172.21.15.46:0/2868335153 >> v2:172.21.15.46:3300/0 conn(0x7f5c780c6780 msgr2=0x7f5c780cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/2868335153 >> v2:172.21.15.46:3300/0 conn(0x7f5c780c6780 0x7f5c780cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7dfca700 1 -- 172.21.15.46:0/2868335153 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780d0aa0 con 0x7f5c781592d0 2023-10-06T12:34:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c780acdb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5c78064310 tx=0x7f5c74009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8de903174226157 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7cfc8700 1 -- 172.21.15.46:0/2868335153 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74005c10 con 0x7f5c781592d0 2023-10-06T12:34:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7cfc8700 1 -- 172.21.15.46:0/2868335153 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74005d70 con 0x7f5c781592d0 2023-10-06T12:34:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c7cfc8700 1 -- 172.21.15.46:0/2868335153 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74006280 con 0x7f5c781592d0 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 -- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 msgr2=0x7f5c780acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 --2- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c780acdb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5c78064310 tx=0x7f5c74009220 comp rx=0 tx=0).stop 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 -- 172.21.15.46:0/2868335153 shutdown_connections 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 --2- 172.21.15.46:0/2868335153 >> v2:172.21.15.46:3300/0 conn(0x7f5c780c6780 0x7f5c780cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 --2- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c780c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 --2- 172.21.15.46:0/2868335153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c780acdb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 -- 172.21.15.46:0/2868335153 >> 172.21.15.46:0/2868335153 conn(0x7f5c7806ef60 msgr2=0x7f5c780d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:09.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 -- 172.21.15.46:0/2868335153 shutdown_connections 2023-10-06T12:34:09.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.523+0000 7f5c8022e700 1 -- 172.21.15.46:0/2868335153 wait complete. 2023-10-06T12:34:09.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 Processor -- start 2023-10-06T12:34:09.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 -- start start 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c7815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c7815db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c9000 con 0x7f5c780c6780 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c92e0 con 0x7f5c781592d0 2023-10-06T12:34:09.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c8022e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c95c0 con 0x7f5c781596a0 2023-10-06T12:34:09.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35528/0 (socket says 172.21.15.46:35528) 2023-10-06T12:34:09.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 -- 172.21.15.46:0/384590616 learned_addr learned my addr 172.21.15.46:0/384590616 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:09.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7e7cb700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c7815db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7d7c9700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c7815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 -- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 msgr2=0x7f5c7815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c7815db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 -- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 msgr2=0x7f5c7815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c7815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 -- 172.21.15.46:0/384590616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009480 con 0x7f5c780c6780 2023-10-06T12:34:09.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.527+0000 7f5c7dfca700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c5bf0 tx=0x7f5c74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74006b70 con 0x7f5c780c6780 2023-10-06T12:34:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c740061c0 con 0x7f5c780c6780 2023-10-06T12:34:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7401a670 con 0x7f5c780c6780 2023-10-06T12:34:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780c98a0 con 0x7f5c780c6780 2023-10-06T12:34:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780c8ca0 con 0x7f5c780c6780 2023-10-06T12:34:09.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.531+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78069e00 con 0x7f5c780c6780 2023-10-06T12:34:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.535+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5c7401aa20 con 0x7f5c780c6780 2023-10-06T12:34:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.535+0000 7f5c6dffb700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 0x7f5c5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.535+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6867+0+0 (secure 0 0 0) 0x7f5c7407dfc0 con 0x7f5c780c6780 2023-10-06T12:34:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.535+0000 7f5c7d7c9700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 0x7f5c5c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:09.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.535+0000 7f5c7d7c9700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 0x7f5c5c062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c9ec0 tx=0x7f5c68009120 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.543+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c7404e020 con 0x7f5c780c6780 2023-10-06T12:34:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:09.711+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c780d0d80 con 0x7f5c780c6780 2023-10-06T12:34:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.319+0000 7f5c6dffb700 1 -- 172.21.15.46:0/384590616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5c74051640 con 0x7f5c780c6780 2023-10-06T12:34:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 msgr2=0x7f5c5c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 0x7f5c5c062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c9ec0 tx=0x7f5c68009120 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 msgr2=0x7f5c7815be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c5bf0 tx=0x7f5c74016000 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 shutdown_connections 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5c5c060670 0x7f5c5c062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5c781596a0 0x7f5c7815db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c781592d0 0x7f5c7815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 --2- 172.21.15.46:0/384590616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c780c6780 0x7f5c7815be20 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 >> 172.21.15.46:0/384590616 conn(0x7f5c7806ef60 msgr2=0x7f5c780cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 shutdown_connections 2023-10-06T12:34:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.327+0000 7f5c8022e700 1 -- 172.21.15.46:0/384590616 wait complete. 2023-10-06T12:34:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T12:34:10.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T12:34:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.527+0000 7ff9fc816700 1 Processor -- start 2023-10-06T12:34:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 -- start start 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff9f80c7930 0x7ff9f80c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80c84b0 con 0x7ff9f80c9c20 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80c23d0 con 0x7ff9f80c7930 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9fc816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c7ef0 con 0x7ff9f81589e0 2023-10-06T12:34:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9f77fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9f80c9c20 0x7ff9f80c6ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36462/0 (socket says 172.21.15.46:36462) 2023-10-06T12:34:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9f77fe700 1 -- 172.21.15.46:0/917576406 learned_addr learned my addr 172.21.15.46:0/917576406 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/917576406 >> v2:172.21.15.132:3300/0 conn(0x7ff9f80c7930 0x7ff9f80c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.531+0000 7ff9f7fff700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 msgr2=0x7ff9f8158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/917576406 >> v1:172.21.15.46:6789/0 conn(0x7ff9f80c9c20 legacy=0x7ff9f80c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/917576406 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80c81d0 con 0x7ff9f80c7930 2023-10-06T12:34:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f80c9ff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009e60 tx=0x7ff9e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10d7974f30e6b90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f67fc700 1 -- 172.21.15.46:0/917576406 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000e070 con 0x7ff9f80c7930 2023-10-06T12:34:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f67fc700 1 -- 172.21.15.46:0/917576406 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000bc30 con 0x7ff9f80c7930 2023-10-06T12:34:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9f67fc700 1 -- 172.21.15.46:0/917576406 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0012430 con 0x7ff9f80c7930 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 -- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 msgr2=0x7ff9f80c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f80c9ff0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009e60 tx=0x7ff9e0009780 comp rx=0 tx=0).stop 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 -- 172.21.15.46:0/917576406 shutdown_connections 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 --2- 172.21.15.46:0/917576406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f80c9ff0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.535+0000 7ff9fc816700 1 -- 172.21.15.46:0/917576406 >> 172.21.15.46:0/917576406 conn(0x7ff9f80cbc10 msgr2=0x7ff9f80cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- 172.21.15.46:0/917576406 shutdown_connections 2023-10-06T12:34:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- 172.21.15.46:0/917576406 wait complete. 2023-10-06T12:34:10.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 Processor -- start 2023-10-06T12:34:10.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- start start 2023-10-06T12:34:10.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f8169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8149e60 con 0x7ff9f80c9c20 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c3970 con 0x7ff9f80c7930 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.539+0000 7ff9fc816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f816a490 con 0x7ff9f81589e0 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35532/0 (socket says 172.21.15.46:35532) 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/726144542 learned_addr learned my addr 172.21.15.46:0/726144542 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f7fff700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f77fe700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f8169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 msgr2=0x7ff9f8152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 msgr2=0x7ff9f8169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f8169f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 -- 172.21.15.46:0/726144542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009a20 con 0x7ff9f80c9c20 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9f6ffd700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0013000 tx=0x7ff9e0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0012e80 con 0x7ff9f80c9c20 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0027470 con 0x7ff9f80c9c20 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0027810 con 0x7ff9f80c9c20 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f80c2d60 con 0x7ff9f80c9c20 2023-10-06T12:34:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f80c3320 con 0x7ff9f80c9c20 2023-10-06T12:34:10.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f80c3b20 con 0x7ff9f80c9c20 2023-10-06T12:34:10.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.543+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff9e0024070 con 0x7ff9f80c9c20 2023-10-06T12:34:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.547+0000 7ff9ef7fe700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 0x7ff9d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.547+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7ff9e007f230 con 0x7ff9f80c9c20 2023-10-06T12:34:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.547+0000 7ff9f77fe700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 0x7ff9d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.547+0000 7ff9f77fe700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 0x7ff9d8062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8007130 tx=0x7ff9e8007eb0 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.551+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff9f80c3b20 con 0x7ff9f80c9c20 2023-10-06T12:34:10.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.719+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff9f80c3ae0 con 0x7ff9f80c9c20 2023-10-06T12:34:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.731+0000 7ff9ef7fe700 1 -- 172.21.15.46:0/726144542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff9e004f380 con 0x7ff9f80c9c20 2023-10-06T12:34:10.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 msgr2=0x7ff9d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 0x7ff9d8062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8007130 tx=0x7ff9e8007eb0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 msgr2=0x7ff9f8150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0013000 tx=0x7ff9e0019000 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 shutdown_connections 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff9d8060670 0x7ff9d8062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9f81589e0 0x7ff9f8152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9f80c9c20 0x7ff9f8150b90 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 --2- 172.21.15.46:0/726144542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9f80c7930 0x7ff9f8169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 >> 172.21.15.46:0/726144542 conn(0x7ff9f80cbc10 msgr2=0x7ff9f8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 shutdown_connections 2023-10-06T12:34:10.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.735+0000 7ff9fc816700 1 -- 172.21.15.46:0/726144542 wait complete. 2023-10-06T12:34:10.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T12:34:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 Processor -- start 2023-10-06T12:34:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 -- start start 2023-10-06T12:34:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cf80cebe0 0x7f6cf8164210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80cb4a0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80cb6e0 con 0x7f6cf80ccc00 2023-10-06T12:34:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6d004ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80cc780 con 0x7f6cf80cebe0 2023-10-06T12:34:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6cfea4b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cf80cebe0 0x7f6cf8164210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6cfea4b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cf80cebe0 0x7f6cf8164210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35540/0 (socket says 172.21.15.46:35540) 2023-10-06T12:34:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6cfe24a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6cf80b23f0 0x7f6cf80cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36476/0 (socket says 172.21.15.46:36476) 2023-10-06T12:34:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.927+0000 7f6cfea4b700 1 -- 172.21.15.46:0/2139608079 learned_addr learned my addr 172.21.15.46:0/2139608079 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749030253 0 0) 0x7f6cf80cb6e0 con 0x7f6cf80ccc00 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce4003570 con 0x7f6cf80ccc00 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfea4b700 1 -- 172.21.15.46:0/2139608079 >> v1:172.21.15.132:6789/0 conn(0x7f6cf80ccc00 legacy=0x7f6cf80ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfea4b700 1 -- 172.21.15.46:0/2139608079 >> v1:172.21.15.46:6789/0 conn(0x7f6cf80b23f0 legacy=0x7f6cf80cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262200669 0 0) 0x7f6cf40011d0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfea4b700 1 -- 172.21.15.46:0/2139608079 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80cb9c0 con 0x7f6cf80cebe0 2023-10-06T12:34:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfea4b700 1 --2- 172.21.15.46:0/2139608079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80cebe0 0x7f6cf8164210 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009f20 tx=0x7f6cf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe9a5b9e15bea4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf000e070 con 0x7f6cf80cebe0 2023-10-06T12:34:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf000b9d0 con 0x7f6cf80cebe0 2023-10-06T12:34:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6cfd248700 1 -- 172.21.15.46:0/2139608079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00092e0 con 0x7f6cf80cebe0 2023-10-06T12:34:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 -- 172.21.15.46:0/2139608079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80cebe0 msgr2=0x7f6cf8164210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 --2- 172.21.15.46:0/2139608079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80cebe0 0x7f6cf8164210 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009f20 tx=0x7f6cf0009840 comp rx=0 tx=0).stop 2023-10-06T12:34:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 -- 172.21.15.46:0/2139608079 shutdown_connections 2023-10-06T12:34:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 --2- 172.21.15.46:0/2139608079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80cebe0 0x7f6cf8164210 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 -- 172.21.15.46:0/2139608079 >> 172.21.15.46:0/2139608079 conn(0x7f6cf8099440 msgr2=0x7f6cf806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 -- 172.21.15.46:0/2139608079 shutdown_connections 2023-10-06T12:34:10.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.931+0000 7f6d004ae700 1 -- 172.21.15.46:0/2139608079 wait complete. 2023-10-06T12:34:10.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.935+0000 7f6d004ae700 1 Processor -- start 2023-10-06T12:34:10.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.935+0000 7f6d004ae700 1 -- start start 2023-10-06T12:34:10.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cf80ccc00 0x7f6cf80c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cf80cebe0 0x7f6cf814be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c77f0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c8a80 con 0x7f6cf80cebe0 2023-10-06T12:34:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c79e0 con 0x7f6cf80ccc00 2023-10-06T12:34:10.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35546/0 (socket says 172.21.15.46:35546) 2023-10-06T12:34:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 -- 172.21.15.46:0/1271809748 learned_addr learned my addr 172.21.15.46:0/1271809748 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfea4b700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cf80cebe0 0x7f6cf814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfda49700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cf80ccc00 0x7f6cf80c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 -- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cf80ccc00 msgr2=0x7f6cf80c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cf80ccc00 0x7f6cf80c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 -- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cf80cebe0 msgr2=0x7f6cf814be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cf80cebe0 0x7f6cf814be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 -- 172.21.15.46:0/1271809748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0009ae0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cfe24a700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c7e0 tx=0x7f6cf4007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40073b0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4007510 con 0x7f6cf80b23f0 2023-10-06T12:34:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4005a20 con 0x7f6cf80b23f0 2023-10-06T12:34:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf80c7cc0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.939+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80c8280 con 0x7f6cf80b23f0 2023-10-06T12:34:10.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.943+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf80660f0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.943+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6cf401b3b0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.943+0000 7f6cee7fc700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 0x7f6cdc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.947+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f6cf407c0b0 con 0x7f6cf80b23f0 2023-10-06T12:34:10.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.947+0000 7f6cfda49700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 0x7f6cdc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:10.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.947+0000 7f6cfda49700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 0x7f6cdc062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cf809e050 tx=0x7f6ce8007d30 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:10.951+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6cf404c200 con 0x7f6cf80b23f0 2023-10-06T12:34:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.123+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f6cf80cb4a0 con 0x7f6cf80b23f0 2023-10-06T12:34:11.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.131+0000 7f6cee7fc700 1 -- 172.21.15.46:0/1271809748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6cf80cb4a0 con 0x7f6cf80b23f0 2023-10-06T12:34:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 msgr2=0x7f6cdc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 0x7f6cdc062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cf809e050 tx=0x7f6ce8007d30 comp rx=0 tx=0).stop 2023-10-06T12:34:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 msgr2=0x7f6cf815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c7e0 tx=0x7f6cf4007b80 comp rx=0 tx=0).stop 2023-10-06T12:34:11.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 shutdown_connections 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6cdc060670 0x7f6cdc062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6cf80cebe0 0x7f6cf814be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6cf80ccc00 0x7f6cf80c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 --2- 172.21.15.46:0/1271809748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cf80b23f0 0x7f6cf815ea20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.135+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 >> 172.21.15.46:0/1271809748 conn(0x7f6cf8099440 msgr2=0x7f6cf81598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.139+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 shutdown_connections 2023-10-06T12:34:11.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.139+0000 7f6d004ae700 1 -- 172.21.15.46:0/1271809748 wait complete. 2023-10-06T12:34:11.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T12:34:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.311+0000 7f7e59945700 1 Processor -- start 2023-10-06T12:34:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.311+0000 7f7e59945700 1 -- start start 2023-10-06T12:34:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e59945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e59945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7e540aee00 0x7f7e540afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e59945700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540af6f0 con 0x7f7e54158e40 2023-10-06T12:34:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e59945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540b43e0 con 0x7f7e540aee00 2023-10-06T12:34:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e59945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540b04b0 con 0x7f7e540b2360 2023-10-06T12:34:11.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:49002/0 (socket says 172.21.15.46:49002) 2023-10-06T12:34:11.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e52ffd700 1 -- 172.21.15.46:0/2694072348 learned_addr learned my addr 172.21.15.46:0/2694072348 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e527fc700 1 --2- 172.21.15.46:0/2694072348 >> v2:172.21.15.132:3300/0 conn(0x7f7e540aee00 0x7f7e540afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e52ffd700 1 -- 172.21.15.46:0/2694072348 >> v2:172.21.15.132:3300/0 conn(0x7f7e540aee00 msgr2=0x7f7e540afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.315+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2694072348 >> v2:172.21.15.132:3300/0 conn(0x7f7e540aee00 0x7f7e540afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e52ffd700 1 -- 172.21.15.46:0/2694072348 >> v1:172.21.15.132:6789/0 conn(0x7f7e54158e40 legacy=0x7f7e541592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e52ffd700 1 -- 172.21.15.46:0/2694072348 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540b0790 con 0x7f7e540b2360 2023-10-06T12:34:11.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2694072348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e440099e0 tx=0x7f7e44009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b0913aad5b3d2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e51ffb700 1 -- 172.21.15.46:0/2694072348 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400b820 con 0x7f7e540b2360 2023-10-06T12:34:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e51ffb700 1 -- 172.21.15.46:0/2694072348 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400b980 con 0x7f7e540b2360 2023-10-06T12:34:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e51ffb700 1 -- 172.21.15.46:0/2694072348 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400bd40 con 0x7f7e540b2360 2023-10-06T12:34:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 -- 172.21.15.46:0/2694072348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 msgr2=0x7f7e540b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 --2- 172.21.15.46:0/2694072348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e440099e0 tx=0x7f7e44009250 comp rx=0 tx=0).stop 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 -- 172.21.15.46:0/2694072348 shutdown_connections 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 --2- 172.21.15.46:0/2694072348 >> v2:172.21.15.132:3300/0 conn(0x7f7e540aee00 0x7f7e540afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 --2- 172.21.15.46:0/2694072348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540b2730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 -- 172.21.15.46:0/2694072348 >> 172.21.15.46:0/2694072348 conn(0x7f7e5406e780 msgr2=0x7f7e5406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 -- 172.21.15.46:0/2694072348 shutdown_connections 2023-10-06T12:34:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.319+0000 7f7e59945700 1 -- 172.21.15.46:0/2694072348 wait complete. 2023-10-06T12:34:11.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.323+0000 7f7e59945700 1 Processor -- start 2023-10-06T12:34:11.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.323+0000 7f7e59945700 1 -- start start 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 0x7f7e5415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540a7e90 con 0x7f7e54158e40 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540a7000 con 0x7f7e540aee00 2023-10-06T12:34:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540a7280 con 0x7f7e540b2360 2023-10-06T12:34:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35560/0 (socket says 172.21.15.46:35560) 2023-10-06T12:34:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 -- 172.21.15.46:0/2181970174 learned_addr learned my addr 172.21.15.46:0/2181970174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 0x7f7e5415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e527fc700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 -- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 msgr2=0x7f7e540a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 -- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 msgr2=0x7f7e5415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 0x7f7e5415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 -- 172.21.15.46:0/2181970174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440094f0 con 0x7f7e54158e40 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 0x7f7e5415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e537fe700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c910 tx=0x7f7e4c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c012630 con 0x7f7e54158e40 2023-10-06T12:34:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c012790 con 0x7f7e54158e40 2023-10-06T12:34:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c012b50 con 0x7f7e54158e40 2023-10-06T12:34:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.327+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e540a7560 con 0x7f7e54158e40 2023-10-06T12:34:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.331+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540a7b20 con 0x7f7e54158e40 2023-10-06T12:34:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.331+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e540a8040 con 0x7f7e54158e40 2023-10-06T12:34:11.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.331+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7e4c01a3a0 con 0x7f7e54158e40 2023-10-06T12:34:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.335+0000 7f7e3affd700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 0x7f7e40062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.335+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f7e4c07cec0 con 0x7f7e54158e40 2023-10-06T12:34:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.335+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 0x7f7e40062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.335+0000 7f7e52ffd700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 0x7f7e40062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e440099e0 tx=0x7f7e44017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.343+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7e540a8040 con 0x7f7e54158e40 2023-10-06T12:34:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.511+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f7e540a8040 con 0x7f7e54158e40 2023-10-06T12:34:11.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.523+0000 7f7e3affd700 1 -- 172.21.15.46:0/2181970174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7e540a8040 con 0x7f7e54158e40 2023-10-06T12:34:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.523+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 msgr2=0x7f7e40062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.523+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 0x7f7e40062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e440099e0 tx=0x7f7e44017000 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.523+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 msgr2=0x7f7e54158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.523+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00c910 tx=0x7f7e4c001710 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 shutdown_connections 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f7e40060670 0x7f7e40062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54158e40 0x7f7e54158340 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e540b2360 0x7f7e540a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 --2- 172.21.15.46:0/2181970174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e540aee00 0x7f7e5415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 >> 172.21.15.46:0/2181970174 conn(0x7f7e5406e780 msgr2=0x7f7e540cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 shutdown_connections 2023-10-06T12:34:11.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.527+0000 7f7e59945700 1 -- 172.21.15.46:0/2181970174 wait complete. 2023-10-06T12:34:11.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-06T12:34:11.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.711+0000 7f334f123700 1 Processor -- start 2023-10-06T12:34:11.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.711+0000 7f334f123700 1 -- start start 2023-10-06T12:34:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.711+0000 7f334f123700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3348067280 0x7f3348067650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334f123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3348159a00 con 0x7f334806cfd0 2023-10-06T12:34:11.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334f123700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3348159e90 con 0x7f334806cb30 2023-10-06T12:34:11.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334f123700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334806d580 con 0x7f3348067280 2023-10-06T12:34:11.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e922700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f334806cfd0 0x7f33481563e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36490/0 (socket says 172.21.15.46:36490) 2023-10-06T12:34:11.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e922700 1 -- 172.21.15.46:0/750828628 learned_addr learned my addr 172.21.15.46:0/750828628 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:11.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e121700 1 --2- 172.21.15.46:0/750828628 >> v2:172.21.15.132:3300/0 conn(0x7f3348067280 0x7f3348067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844698582 0 0) 0x7f3348159a00 con 0x7f334806cfd0 2023-10-06T12:34:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3334003570 con 0x7f334806cfd0 2023-10-06T12:34:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300172605 0 0) 0x7f3348159e90 con 0x7f334806cb30 2023-10-06T12:34:11.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3348159a00 con 0x7f334806cb30 2023-10-06T12:34:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e121700 1 -- 172.21.15.46:0/750828628 >> v1:172.21.15.132:6789/0 conn(0x7f334806cb30 legacy=0x7f3348155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e121700 1 -- 172.21.15.46:0/750828628 >> v1:172.21.15.46:6789/0 conn(0x7f334806cfd0 legacy=0x7f33481563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634310790 0 0) 0x7f3334003570 con 0x7f334806cfd0 2023-10-06T12:34:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e121700 1 -- 172.21.15.46:0/750828628 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33481590d0 con 0x7f3348067280 2023-10-06T12:34:11.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334e121700 1 --2- 172.21.15.46:0/750828628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3348067280 0x7f3348067650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f334400aa90 tx=0x7f334400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20cd1b7270587a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.715+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400017f0 con 0x7f3348067280 2023-10-06T12:34:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334400aed0 con 0x7f3348067280 2023-10-06T12:34:11.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334d11f700 1 -- 172.21.15.46:0/750828628 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400c890 con 0x7f3348067280 2023-10-06T12:34:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- 172.21.15.46:0/750828628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3348067280 msgr2=0x7f3348067650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 --2- 172.21.15.46:0/750828628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3348067280 0x7f3348067650 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f334400aa90 tx=0x7f334400a330 comp rx=0 tx=0).stop 2023-10-06T12:34:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- 172.21.15.46:0/750828628 shutdown_connections 2023-10-06T12:34:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 --2- 172.21.15.46:0/750828628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3348067280 0x7f3348067650 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- 172.21.15.46:0/750828628 >> 172.21.15.46:0/750828628 conn(0x7f3348062ef0 msgr2=0x7f33480cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- 172.21.15.46:0/750828628 shutdown_connections 2023-10-06T12:34:11.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- 172.21.15.46:0/750828628 wait complete. 2023-10-06T12:34:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 Processor -- start 2023-10-06T12:34:11.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.719+0000 7f334f123700 1 -- start start 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f334806cb30 0x7f33480c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 0x7f3348151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33480c21c0 con 0x7f3348067280 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33480c31c0 con 0x7f334806cfd0 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33480c1920 con 0x7f334806cb30 2023-10-06T12:34:11.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35564/0 (socket says 172.21.15.46:35564) 2023-10-06T12:34:11.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 -- 172.21.15.46:0/1326263192 learned_addr learned my addr 172.21.15.46:0/1326263192 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334d920700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f334806cb30 0x7f33480c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e922700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 0x7f3348151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 -- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f334806cb30 msgr2=0x7f33480c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f334806cb30 0x7f33480c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 -- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 msgr2=0x7f3348151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 0x7f3348151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 -- 172.21.15.46:0/1326263192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334400a5d0 con 0x7f3348067280 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e922700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 0x7f3348151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334e121700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f334400a370 tx=0x7f3344019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344018a80 con 0x7f3348067280 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344018be0 con 0x7f3348067280 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344022420 con 0x7f3348067280 2023-10-06T12:34:11.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.723+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33480c1c00 con 0x7f3348067280 2023-10-06T12:34:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.727+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3348169530 con 0x7f3348067280 2023-10-06T12:34:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.727+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33480c3020 con 0x7f3348067280 2023-10-06T12:34:11.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.727+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3344020070 con 0x7f3348067280 2023-10-06T12:34:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.731+0000 7f333e7fc700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 0x7f332c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.731+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f334407db50 con 0x7f3348067280 2023-10-06T12:34:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.731+0000 7f334d920700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 0x7f332c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:11.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.731+0000 7f334d920700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 0x7f332c062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3338003870 tx=0x7f3338009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.739+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f33480c3020 con 0x7f3348067280 2023-10-06T12:34:11.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:11.967+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f33480c3020 con 0x7f3348067280 2023-10-06T12:34:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.331+0000 7f333e7fc700 1 -- 172.21.15.46:0/1326263192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2456) v1 ==== 174+0+0 (secure 0 0 0) 0x7f33480c3020 con 0x7f3348067280 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 msgr2=0x7f332c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 0x7f332c062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3338003870 tx=0x7f3338009040 comp rx=0 tx=0).stop 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 msgr2=0x7f33480c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f334400a370 tx=0x7f3344019000 comp rx=0 tx=0).stop 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 shutdown_connections 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f332c060670 0x7f332c062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f334806cfd0 0x7f3348151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f334806cb30 0x7f33480c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 --2- 172.21.15.46:0/1326263192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3348067280 0x7f33480c5370 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 >> 172.21.15.46:0/1326263192 conn(0x7f3348062ef0 msgr2=0x7f33480b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 shutdown_connections 2023-10-06T12:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.335+0000 7f334f123700 1 -- 172.21.15.46:0/1326263192 wait complete. 2023-10-06T12:34:12.360 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-06T12:34:12.361 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-06T12:34:12.361 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T12:34:12.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-06T12:34:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.519+0000 7f867d1c5700 1 Processor -- start 2023-10-06T12:34:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.519+0000 7f867d1c5700 1 -- start start 2023-10-06T12:34:12.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f867d1c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86780c7b50 0x7f86780c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f867d1c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780c8500 con 0x7f8678156c80 2023-10-06T12:34:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f867d1c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780c6470 con 0x7f86780c0bd0 2023-10-06T12:34:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f867d1c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780c7760 con 0x7f86780c7b50 2023-10-06T12:34:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f8676d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8678156c80 0x7f86780ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36498/0 (socket says 172.21.15.46:36498) 2023-10-06T12:34:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f8676d9d700 1 -- 172.21.15.46:0/3365942658 learned_addr learned my addr 172.21.15.46:0/3365942658 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f866e59c700 1 --2- 172.21.15.46:0/3365942658 >> v2:172.21.15.46:3300/0 conn(0x7f86780c7b50 0x7f86780c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.523+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277738661 0 0) 0x7f86780c8500 con 0x7f8678156c80 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8664003570 con 0x7f8678156c80 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271675378 0 0) 0x7f86780c6470 con 0x7f86780c0bd0 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86780c8500 con 0x7f86780c0bd0 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f866e59c700 1 -- 172.21.15.46:0/3365942658 >> v1:172.21.15.132:6789/0 conn(0x7f86780c0bd0 legacy=0x7f86780c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f866e59c700 1 -- 172.21.15.46:0/3365942658 >> v1:172.21.15.46:6789/0 conn(0x7f8678156c80 legacy=0x7f86780ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f866e59c700 1 -- 172.21.15.46:0/3365942658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780c66c0 con 0x7f86780c7b50 2023-10-06T12:34:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f866e59c700 1 --2- 172.21.15.46:0/3365942658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c7b50 0x7f86780c7fc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f86600098e0 tx=0x7f86600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809c0dc28828d130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011630 con 0x7f86780c7b50 2023-10-06T12:34:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660011790 con 0x7f86780c7b50 2023-10-06T12:34:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867659c700 1 -- 172.21.15.46:0/3365942658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011b30 con 0x7f86780c7b50 2023-10-06T12:34:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 -- 172.21.15.46:0/3365942658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c7b50 msgr2=0x7f86780c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 --2- 172.21.15.46:0/3365942658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c7b50 0x7f86780c7fc0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f86600098e0 tx=0x7f86600091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 -- 172.21.15.46:0/3365942658 shutdown_connections 2023-10-06T12:34:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 --2- 172.21.15.46:0/3365942658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c7b50 0x7f86780c7fc0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 -- 172.21.15.46:0/3365942658 >> 172.21.15.46:0/3365942658 conn(0x7f86780cbe30 msgr2=0x7f86780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 -- 172.21.15.46:0/3365942658 shutdown_connections 2023-10-06T12:34:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.527+0000 7f867d1c5700 1 -- 172.21.15.46:0/3365942658 wait complete. 2023-10-06T12:34:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 Processor -- start 2023-10-06T12:34:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 -- start start 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 0x7f86780ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8678156c80 0x7f86781698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8678150280 con 0x7f86780c0bd0 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86781542a0 con 0x7f86780c7b50 2023-10-06T12:34:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.531+0000 7f867d1c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867816a140 con 0x7f8678156c80 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35572/0 (socket says 172.21.15.46:35572) 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 -- 172.21.15.46:0/2837428421 learned_addr learned my addr 172.21.15.46:0/2837428421 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f866e59c700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 0x7f86780ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f867759e700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8678156c80 0x7f86781698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 -- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8678156c80 msgr2=0x7f86781698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8678156c80 0x7f86781698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 -- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 msgr2=0x7f86780ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 0x7f86780ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 -- 172.21.15.46:0/2837428421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660009420 con 0x7f86780c0bd0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f866e59c700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 0x7f86780ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f8676d9d700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f866800c790 tx=0x7f8668007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680072e0 con 0x7f86780c0bd0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668007440 con 0x7f86780c0bd0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867816a420 con 0x7f86780c0bd0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680057f0 con 0x7f86780c0bd0 2023-10-06T12:34:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86781544f0 con 0x7f86780c0bd0 2023-10-06T12:34:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.535+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678063f90 con 0x7f86780c0bd0 2023-10-06T12:34:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.539+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8668005be0 con 0x7f86780c0bd0 2023-10-06T12:34:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.539+0000 7f866f7fe700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 0x7f8658062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.539+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f866807c290 con 0x7f86780c0bd0 2023-10-06T12:34:12.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.539+0000 7f866e59c700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 0x7f8658062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.539+0000 7f866e59c700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 0x7f8658062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f86600093f0 tx=0x7f8660017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.547+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f866804c480 con 0x7f86780c0bd0 2023-10-06T12:34:12.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:12.775+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f867804b5c0 con 0x7f86780c0bd0 2023-10-06T12:34:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.343+0000 7f866f7fe700 1 -- 172.21.15.46:0/2837428421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2457) v1 ==== 168+0+0 (secure 0 0 0) 0x7f866804faa0 con 0x7f86780c0bd0 2023-10-06T12:34:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 msgr2=0x7f8658062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 0x7f8658062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f86600093f0 tx=0x7f8660017000 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 msgr2=0x7f86780ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f866800c790 tx=0x7f8668007ce0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 shutdown_connections 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8658060670 0x7f8658062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8678156c80 0x7f86781698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86780c7b50 0x7f86780ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 --2- 172.21.15.46:0/2837428421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86780c0bd0 0x7f86780ad570 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 >> 172.21.15.46:0/2837428421 conn(0x7f86780cbe30 msgr2=0x7f8678152420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 shutdown_connections 2023-10-06T12:34:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.347+0000 7f867d1c5700 1 -- 172.21.15.46:0/2837428421 wait complete. 2023-10-06T12:34:13.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T12:34:13.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 Processor -- start 2023-10-06T12:34:13.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 -- start start 2023-10-06T12:34:13.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 0x7fbe3c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0cc510 con 0x7fbe3c0cd080 2023-10-06T12:34:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0b2be0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe430f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cd500 con 0x7fbe3c0b2790 2023-10-06T12:34:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe4168d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbe3c0cd080 0x7fbe3c0c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50092/0 (socket says 172.21.15.46:50092) 2023-10-06T12:34:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe4168d700 1 -- 172.21.15.46:0/1779939494 learned_addr learned my addr 172.21.15.46:0/1779939494 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:13.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.539+0000 7fbe3bfff700 1 --2- 172.21.15.46:0/1779939494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 0x7fbe3c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269543592 0 0) 0x7fbe3c0cc510 con 0x7fbe3c0cd080 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe2c003570 con 0x7fbe3c0cd080 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839082159 0 0) 0x7fbe3c0b2be0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3c0cc510 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3bfff700 1 -- 172.21.15.46:0/1779939494 >> v1:172.21.15.132:6789/0 conn(0x7fbe3c0b23c0 legacy=0x7fbe3c0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3bfff700 1 -- 172.21.15.46:0/1779939494 >> v1:172.21.15.46:6789/0 conn(0x7fbe3c0cd080 legacy=0x7fbe3c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3bfff700 1 -- 172.21.15.46:0/1779939494 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0cb900 con 0x7fbe3c0b2790 2023-10-06T12:34:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3bfff700 1 --2- 172.21.15.46:0/1779939494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 0x7fbe3c0ccb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009910 tx=0x7fbe280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1e063b07dec00b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:13.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28011630 con 0x7fbe3c0b2790 2023-10-06T12:34:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28011790 con 0x7fbe3c0b2790 2023-10-06T12:34:13.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe3b7fe700 1 -- 172.21.15.46:0/1779939494 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28011b30 con 0x7fbe3c0b2790 2023-10-06T12:34:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 -- 172.21.15.46:0/1779939494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 msgr2=0x7fbe3c0ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 --2- 172.21.15.46:0/1779939494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 0x7fbe3c0ccb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009910 tx=0x7fbe280091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 -- 172.21.15.46:0/1779939494 shutdown_connections 2023-10-06T12:34:13.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 --2- 172.21.15.46:0/1779939494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0b2790 0x7fbe3c0ccb40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 -- 172.21.15.46:0/1779939494 >> 172.21.15.46:0/1779939494 conn(0x7fbe3c099480 msgr2=0x7fbe3c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:13.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.543+0000 7fbe430f0700 1 -- 172.21.15.46:0/1779939494 shutdown_connections 2023-10-06T12:34:13.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.547+0000 7fbe430f0700 1 -- 172.21.15.46:0/1779939494 wait complete. 2023-10-06T12:34:13.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.547+0000 7fbe430f0700 1 Processor -- start 2023-10-06T12:34:13.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.547+0000 7fbe430f0700 1 -- start start 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0b2790 0x7fbe3c0c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cd080 0x7fbe3c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c171650 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c15a730 con 0x7fbe3c0b2790 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c159b20 con 0x7fbe3c0cd080 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58542/0 (socket says 172.21.15.46:58542) 2023-10-06T12:34:13.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 -- 172.21.15.46:0/2782024681 learned_addr learned my addr 172.21.15.46:0/2782024681 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe3bfff700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0b2790 0x7fbe3c0c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe4168d700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cd080 0x7fbe3c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 -- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cd080 msgr2=0x7fbe3c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cd080 0x7fbe3c0d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 -- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0b2790 msgr2=0x7fbe3c0c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0b2790 0x7fbe3c0c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 -- 172.21.15.46:0/2782024681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009420 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe40e8c700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c560 tx=0x7fbe30007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:13.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30010510 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30010670 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c159e00 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30010a30 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.551+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c15a3c0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.555+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c0660f0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.555+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbe30002050 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.559+0000 7fbe38ff9700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 0x7fbe20062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.559+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7fbe3007c5a0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.559+0000 7fbe3bfff700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 0x7fbe20062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:13.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.559+0000 7fbe3bfff700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 0x7fbe20062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009860 tx=0x7fbe28017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:13.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.563+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbe3004c6f0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.735+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbe3c0cbbe0 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.747+0000 7fbe38ff9700 1 -- 172.21.15.46:0/2782024681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbe3004fd10 con 0x7fbe3c0b23c0 2023-10-06T12:34:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 msgr2=0x7fbe20062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 0x7fbe20062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009860 tx=0x7fbe28017000 comp rx=0 tx=0).stop 2023-10-06T12:34:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 msgr2=0x7fbe3c16e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c560 tx=0x7fbe30007e00 comp rx=0 tx=0).stop 2023-10-06T12:34:13.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 shutdown_connections 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbe20060670 0x7fbe20062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cd080 0x7fbe3c0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0b2790 0x7fbe3c0c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 --2- 172.21.15.46:0/2782024681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe3c0b23c0 0x7fbe3c16e200 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 >> 172.21.15.46:0/2782024681 conn(0x7fbe3c099480 msgr2=0x7fbe3c158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 shutdown_connections 2023-10-06T12:34:13.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:13.751+0000 7fbe430f0700 1 -- 172.21.15.46:0/2782024681 wait complete. 2023-10-06T12:34:13.775 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:34:13.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:34:13.816 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:34:13.860 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:34:13.860 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T12:34:13.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.023+0000 7f588aae3700 1 Processor -- start 2023-10-06T12:34:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588aae3700 1 -- start start 2023-10-06T12:34:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588aae3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5884067760 0x7f5884161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588aae3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840acd60 con 0x7f58840aaf30 2023-10-06T12:34:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588aae3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840ad040 con 0x7f5884156e20 2023-10-06T12:34:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588aae3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840ad320 con 0x7f5884067760 2023-10-06T12:34:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f58892e0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58840aaf30 0x7f5884067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50108/0 (socket says 172.21.15.46:50108) 2023-10-06T12:34:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f58892e0700 1 -- 172.21.15.46:0/179335205 learned_addr learned my addr 172.21.15.46:0/179335205 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.027+0000 7f588a2e2700 1 --2- 172.21.15.46:0/179335205 >> v2:172.21.15.46:3300/0 conn(0x7f5884067760 0x7f5884161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f5888adf700 1 -- 172.21.15.46:0/179335205 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355120781 0 0) 0x7f58840acd60 con 0x7f58840aaf30 2023-10-06T12:34:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f5888adf700 1 -- 172.21.15.46:0/179335205 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5870003570 con 0x7f58840aaf30 2023-10-06T12:34:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588a2e2700 1 -- 172.21.15.46:0/179335205 >> v1:172.21.15.132:6789/0 conn(0x7f5884156e20 legacy=0x7f588406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588a2e2700 1 -- 172.21.15.46:0/179335205 >> v1:172.21.15.46:6789/0 conn(0x7f58840aaf30 legacy=0x7f5884067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588a2e2700 1 -- 172.21.15.46:0/179335205 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840ad600 con 0x7f5884067760 2023-10-06T12:34:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588a2e2700 1 --2- 172.21.15.46:0/179335205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884067760 0x7f5884161ab0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e60 tx=0x7f587c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5f01357116715f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f5888adf700 1 -- 172.21.15.46:0/179335205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00e070 con 0x7f5884067760 2023-10-06T12:34:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f5888adf700 1 -- 172.21.15.46:0/179335205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00bb60 con 0x7f5884067760 2023-10-06T12:34:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f5888adf700 1 -- 172.21.15.46:0/179335205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c012430 con 0x7f5884067760 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 -- 172.21.15.46:0/179335205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884067760 msgr2=0x7f5884161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 --2- 172.21.15.46:0/179335205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884067760 0x7f5884161ab0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f587c009e60 tx=0x7f587c009780 comp rx=0 tx=0).stop 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 -- 172.21.15.46:0/179335205 shutdown_connections 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 --2- 172.21.15.46:0/179335205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884067760 0x7f5884161ab0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 -- 172.21.15.46:0/179335205 >> 172.21.15.46:0/179335205 conn(0x7f5884062ef0 msgr2=0x7f58840cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 -- 172.21.15.46:0/179335205 shutdown_connections 2023-10-06T12:34:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.031+0000 7f588aae3700 1 -- 172.21.15.46:0/179335205 wait complete. 2023-10-06T12:34:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 Processor -- start 2023-10-06T12:34:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 -- start start 2023-10-06T12:34:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 0x7f58840c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840aaf30 0x7f5884154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840aedb0 con 0x7f5884156e20 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840af090 con 0x7f58840aaf30 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588aae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58840af370 con 0x7f5884067760 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58568/0 (socket says 172.21.15.46:58568) 2023-10-06T12:34:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 -- 172.21.15.46:0/3434539349 learned_addr learned my addr 172.21.15.46:0/3434539349 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f58892e0700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840aaf30 0x7f5884154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f5889ae1700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 0x7f58840c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 -- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 msgr2=0x7f58840c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 0x7f58840c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 -- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840aaf30 msgr2=0x7f5884154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840aaf30 0x7f5884154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 -- 172.21.15.46:0/3434539349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009a20 con 0x7f5884156e20 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f5889ae1700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 0x7f58840c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.035+0000 7f588a2e2700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f587c013000 tx=0x7f587c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c027470 con 0x7f5884156e20 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58840af650 con 0x7f5884156e20 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c0275f0 con 0x7f5884156e20 2023-10-06T12:34:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0279b0 con 0x7f5884156e20 2023-10-06T12:34:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588416a7d0 con 0x7f5884156e20 2023-10-06T12:34:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.039+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884064590 con 0x7f5884156e20 2023-10-06T12:34:14.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.043+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f587c024070 con 0x7f5884156e20 2023-10-06T12:34:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.043+0000 7f5879ffb700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 0x7f5868062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.043+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f587c07f600 con 0x7f5884156e20 2023-10-06T12:34:14.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.043+0000 7f5889ae1700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 0x7f5868062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.043+0000 7f5889ae1700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 0x7f5868062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5880007fd0 tx=0x7f5880009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.047+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f587c04f7f0 con 0x7f5884156e20 2023-10-06T12:34:14.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.259+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58840ad8e0 con 0x7f5884156e20 2023-10-06T12:34:14.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.259+0000 7f5879ffb700 1 -- 172.21.15.46:0/3434539349 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f587c052e10 con 0x7f5884156e20 2023-10-06T12:34:14.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:14.265 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3533921805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3533921805},{"type":"v1","addr":"172.21.15.132:6834","nonce":3533921805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3083371908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3083371908},{"type":"v1","addr":"172.21.15.46:6837","nonce":3083371908}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":8569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3594444492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3594444492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3594444492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71},{"gid":9213,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3986617138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3986617138},{"type":"v1","addr":"172.21.15.46:6835","nonce":3986617138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[]} 2023-10-06T12:34:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 msgr2=0x7f5868062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 0x7f5868062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5880007fd0 tx=0x7f5880009040 comp rx=0 tx=0).stop 2023-10-06T12:34:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 msgr2=0x7f58840a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f587c013000 tx=0x7f587c019000 comp rx=0 tx=0).stop 2023-10-06T12:34:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 shutdown_connections 2023-10-06T12:34:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5868060670 0x7f5868062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5884156e20 0x7f58840a9850 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58840aaf30 0x7f5884154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 --2- 172.21.15.46:0/3434539349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5884067760 0x7f58840c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 >> 172.21.15.46:0/3434539349 conn(0x7f5884062ef0 msgr2=0x7f5884156030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 shutdown_connections 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.267+0000 7f588aae3700 1 -- 172.21.15.46:0/3434539349 wait complete. 2023-10-06T12:34:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 78 2023-10-06T12:34:14.290 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T12:34:14.290 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T12:34:14.291 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T12:34:14.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:34:14.291 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T12:34:14.291 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T12:34:14.291 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T12:34:14.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:34:14.292 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T12:34:14.292 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T12:34:14.292 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T12:34:14.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:34:14.292 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T12:34:14.292 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T12:34:14.292 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T12:34:14.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:34:14.310 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T12:34:14.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T12:34:14.312 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T12:34:14.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T12:34:14.313 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T12:34:14.313 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T12:34:14.314 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T12:34:14.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T12:34:14.360 INFO:tasks.ceph.mds.b:Started 2023-10-06T12:34:14.361 DEBUG:teuthology.parallel:result is None 2023-10-06T12:34:14.362 INFO:tasks.ceph.mds.c:Started 2023-10-06T12:34:14.362 DEBUG:teuthology.parallel:result is None 2023-10-06T12:34:14.364 INFO:tasks.ceph.mds.d:Started 2023-10-06T12:34:14.364 DEBUG:teuthology.parallel:result is None 2023-10-06T12:34:14.366 INFO:tasks.ceph.mds.a:Started 2023-10-06T12:34:14.366 DEBUG:teuthology.parallel:result is None 2023-10-06T12:34:14.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T12:34:14.464 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T12:34:14.471 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:34:14.471 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:34:14.475 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T12:34:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.519+0000 7f8c07545700 1 Processor -- start 2023-10-06T12:34:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.519+0000 7f8c07545700 1 -- start start 2023-10-06T12:34:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c07545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8c00161470 0x7f8c00161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c07545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8c000c1d80 0x7f8c000c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c07545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c000c18f0 con 0x7f8c0006cc30 2023-10-06T12:34:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c07545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c000c0ce0 con 0x7f8c00161470 2023-10-06T12:34:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c07545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c000c0fc0 con 0x7f8c000c1d80 2023-10-06T12:34:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c05ae2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8c0006cc30 0x7f8c0015d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50126/0 (socket says 172.21.15.46:50126) 2023-10-06T12:34:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c05ae2700 1 -- 172.21.15.46:0/674168568 learned_addr learned my addr 172.21.15.46:0/674168568 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/674168568 >> v2:172.21.15.132:3300/0 conn(0x7f8c000c1d80 0x7f8c000c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c052e1700 1 --2- 172.21.15.46:0/674168568 >> v2:172.21.15.46:3300/0 conn(0x7f8c00161470 0x7f8c00161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 -- 172.21.15.46:0/674168568 >> v2:172.21.15.46:3300/0 conn(0x7f8c00161470 msgr2=0x7f8c00161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/674168568 >> v2:172.21.15.46:3300/0 conn(0x7f8c00161470 0x7f8c00161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 -- 172.21.15.46:0/674168568 >> v1:172.21.15.46:6789/0 conn(0x7f8c0006cc30 legacy=0x7f8c0015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 -- 172.21.15.46:0/674168568 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000c12a0 con 0x7f8c000c1d80 2023-10-06T12:34:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/674168568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c000c21e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8c0006ba90 tx=0x7f8bf00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3430e8871d2104db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8bf7fff700 1 -- 172.21.15.46:0/674168568 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000b690 con 0x7f8c000c1d80 2023-10-06T12:34:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8bf7fff700 1 -- 172.21.15.46:0/674168568 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000b7f0 con 0x7f8c000c1d80 2023-10-06T12:34:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.523+0000 7f8bf7fff700 1 -- 172.21.15.46:0/674168568 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000bb90 con 0x7f8c000c1d80 2023-10-06T12:34:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 -- 172.21.15.46:0/674168568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 msgr2=0x7f8c000c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 --2- 172.21.15.46:0/674168568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c000c21e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8c0006ba90 tx=0x7f8bf00091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 -- 172.21.15.46:0/674168568 shutdown_connections 2023-10-06T12:34:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 --2- 172.21.15.46:0/674168568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c000c21e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 --2- 172.21.15.46:0/674168568 >> v2:172.21.15.46:3300/0 conn(0x7f8c00161470 0x7f8c00161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 -- 172.21.15.46:0/674168568 >> 172.21.15.46:0/674168568 conn(0x7f8c000cbe30 msgr2=0x7f8c000cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 -- 172.21.15.46:0/674168568 shutdown_connections 2023-10-06T12:34:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 -- 172.21.15.46:0/674168568 wait complete. 2023-10-06T12:34:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.527+0000 7f8c07545700 1 Processor -- start 2023-10-06T12:34:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 -- start start 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c0014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 0x7f8c0014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00172150 con 0x7f8c0006cc30 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0015d940 con 0x7f8c000c1d80 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c07545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0014b900 con 0x7f8c00161470 2023-10-06T12:34:14.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58624/0 (socket says 172.21.15.46:58624) 2023-10-06T12:34:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 -- 172.21.15.46:0/1081842645 learned_addr learned my addr 172.21.15.46:0/1081842645 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c05ae2700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 0x7f8c0014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 -- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 msgr2=0x7f8c0014b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c0014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 0x7f8c0014b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 -- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 msgr2=0x7f8c0014ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c0014ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 -- 172.21.15.46:0/1081842645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0009420 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c0014ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c05ae2700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 0x7f8c0014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.531+0000 7f8c052e1700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ea00 tx=0x7f8bfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00c850 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00c9b0 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0015d010 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc00cd50 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0015d5d0 con 0x7f8c0006cc30 2023-10-06T12:34:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c00063f90 con 0x7f8c0006cc30 2023-10-06T12:34:14.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.535+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8bfc013680 con 0x7f8c0006cc30 2023-10-06T12:34:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.539+0000 7f8bf57fa700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 0x7f8be4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.539+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f8bfc07cf20 con 0x7f8c0006cc30 2023-10-06T12:34:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.539+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 0x7f8be4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.539+0000 7f8c04ae0700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 0x7f8be4062b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8c000b1990 tx=0x7f8bf0017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.543+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8bfc04d070 con 0x7f8c0006cc30 2023-10-06T12:34:14.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:14.723+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c000c1580 con 0x7f8c0006cc30 2023-10-06T12:34:15.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.367+0000 7f8bf57fa700 1 -- 172.21.15.46:0/1081842645 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v160) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8bfc050690 con 0x7f8c0006cc30 2023-10-06T12:34:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 msgr2=0x7f8be4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 0x7f8be4062b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8c000b1990 tx=0x7f8bf0017000 comp rx=0 tx=0).stop 2023-10-06T12:34:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 msgr2=0x7f8c00163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00ea00 tx=0x7f8bfc0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 shutdown_connections 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f8be40606a0 0x7f8be4062b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c00161470 0x7f8c0014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c000c1d80 0x7f8c0014ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 --2- 172.21.15.46:0/1081842645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c0006cc30 0x7f8c00163bc0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 >> 172.21.15.46:0/1081842645 conn(0x7f8c000cbe30 msgr2=0x7f8c000acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 shutdown_connections 2023-10-06T12:34:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.371+0000 7f8c07545700 1 -- 172.21.15.46:0/1081842645 wait complete. 2023-10-06T12:34:15.377 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T12:34:15.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T12:34:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 Processor -- start 2023-10-06T12:34:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 -- start start 2023-10-06T12:34:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 0x7f1e7c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0cc150 con 0x7f1e7c0cea30 2023-10-06T12:34:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0cfbb0 con 0x7f1e7c0ccb50 2023-10-06T12:34:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.559+0000 7f1e81aae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0cdbf0 con 0x7f1e7c15c770 2023-10-06T12:34:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7b7fe700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 0x7f1e7c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7bfff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1e7c0cea30 0x7f1e7c0c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47562/0 (socket says 172.21.15.46:47562) 2023-10-06T12:34:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7bfff700 1 -- 172.21.15.46:0/802383662 learned_addr learned my addr 172.21.15.46:0/802383662 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58638/0 (socket says 172.21.15.46:58638) 2023-10-06T12:34:15.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 -- 172.21.15.46:0/802383662 >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 msgr2=0x7f1e7c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:15.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 --2- 172.21.15.46:0/802383662 >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 0x7f1e7c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 -- 172.21.15.46:0/802383662 >> v1:172.21.15.132:6789/0 conn(0x7f1e7c0cea30 legacy=0x7f1e7c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 -- 172.21.15.46:0/802383662 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0cee50 con 0x7f1e7c0ccb50 2023-10-06T12:34:15.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7b7fe700 1 --2- 172.21.15.46:0/802383662 >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 0x7f1e7c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7affd700 1 -- 172.21.15.46:0/802383662 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945144403 0 0) 0x7f1e7c0cc150 con 0x7f1e7c0cea30 2023-10-06T12:34:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e73fff700 1 --2- 172.21.15.46:0/802383662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c06f6d0 tx=0x7f1e640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0311c282f9df131 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7affd700 1 -- 172.21.15.46:0/802383662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64011630 con 0x7f1e7c0ccb50 2023-10-06T12:34:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7affd700 1 -- 172.21.15.46:0/802383662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e64011790 con 0x7f1e7c0ccb50 2023-10-06T12:34:15.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e7affd700 1 -- 172.21.15.46:0/802383662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64011b30 con 0x7f1e7c0ccb50 2023-10-06T12:34:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e81aae700 1 -- 172.21.15.46:0/802383662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0ccb50 msgr2=0x7f1e7c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.563+0000 7f1e81aae700 1 --2- 172.21.15.46:0/802383662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c06f6d0 tx=0x7f1e640091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 -- 172.21.15.46:0/802383662 shutdown_connections 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 --2- 172.21.15.46:0/802383662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0ccb50 0x7f1e7c0ce4f0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 --2- 172.21.15.46:0/802383662 >> v2:172.21.15.132:3300/0 conn(0x7f1e7c15c770 0x7f1e7c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 -- 172.21.15.46:0/802383662 >> 172.21.15.46:0/802383662 conn(0x7f1e7c099480 msgr2=0x7f1e7c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 -- 172.21.15.46:0/802383662 shutdown_connections 2023-10-06T12:34:15.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 -- 172.21.15.46:0/802383662 wait complete. 2023-10-06T12:34:15.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 Processor -- start 2023-10-06T12:34:15.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.567+0000 7f1e81aae700 1 -- start start 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e7c15c770 0x7f1e7c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e7c09b1c0 0x7f1e7c067b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c09c2b0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c15a980 con 0x7f1e7c09b1c0 2023-10-06T12:34:15.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c169810 con 0x7f1e7c15c770 2023-10-06T12:34:15.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58648/0 (socket says 172.21.15.46:58648) 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 -- 172.21.15.46:0/1490214759 learned_addr learned my addr 172.21.15.46:0/1490214759 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7bfff700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e7c09b1c0 0x7f1e7c067b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e73fff700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e7c15c770 0x7f1e7c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 -- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e7c15c770 msgr2=0x7f1e7c0b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e7c15c770 0x7f1e7c0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 -- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e7c09b1c0 msgr2=0x7f1e7c067b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e7c09b1c0 0x7f1e7c067b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 -- 172.21.15.46:0/1490214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c0093c0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e7b7fe700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00ecb0 tx=0x7f1e6c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c005a00 con 0x7f1e7c0cea30 2023-10-06T12:34:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005b60 con 0x7f1e7c0cea30 2023-10-06T12:34:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e64009420 con 0x7f1e7c0cea30 2023-10-06T12:34:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00c9d0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.571+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c09bac0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.575+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c09c0f0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.575+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1e6c00cd80 con 0x7f1e7c0cea30 2023-10-06T12:34:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.575+0000 7f1e737fe700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 0x7f1e5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:15.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.575+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f1e6c07cf30 con 0x7f1e7c0cea30 2023-10-06T12:34:15.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.579+0000 7f1e73fff700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 0x7f1e5c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:15.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.579+0000 7f1e73fff700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 0x7f1e5c062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0ccf20 tx=0x7f1e64017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.583+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1e6c050df0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.811+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1e7c0660f0 con 0x7f1e7c0cea30 2023-10-06T12:34:15.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.811+0000 7f1e737fe700 1 -- 172.21.15.46:0/1490214759 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1e6c020020 con 0x7f1e7c0cea30 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T12:34:15.817 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T12:34:15.818 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T12:34:15.819 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.820 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:34:15.822 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.826 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.828 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.829 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T12:34:15.830 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:34:15.831 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:34:15.832 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:15.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T12:34:15.834 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:34:15.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.815+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 msgr2=0x7f1e5c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.815+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 0x7f1e5c062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0ccf20 tx=0x7f1e64017000 comp rx=0 tx=0).stop 2023-10-06T12:34:15.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.815+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 msgr2=0x7f1e7c09d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.815+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00ecb0 tx=0x7f1e6c009120 comp rx=0 tx=0).stop 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 shutdown_connections 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f1e5c060670 0x7f1e5c062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e7c09b1c0 0x7f1e7c067b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e7c15c770 0x7f1e7c0b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 --2- 172.21.15.46:0/1490214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e7c0cea30 0x7f1e7c09d990 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 >> 172.21.15.46:0/1490214759 conn(0x7f1e7c099480 msgr2=0x7f1e7c0c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 shutdown_connections 2023-10-06T12:34:15.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.819+0000 7f1e81aae700 1 -- 172.21.15.46:0/1490214759 wait complete. 2023-10-06T12:34:15.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.999+0000 7fd92c26e700 1 Processor -- start 2023-10-06T12:34:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:15.999+0000 7fd92c26e700 1 -- start start 2023-10-06T12:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.003+0000 7fd92c26e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd9241502f0 0x7fd9240c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.003+0000 7fd92c26e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9240c67e0 con 0x7fd9241506c0 2023-10-06T12:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.003+0000 7fd92c26e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92406e810 con 0x7fd92415a1a0 2023-10-06T12:34:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.003+0000 7fd92c26e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92406eaf0 con 0x7fd9241502f0 2023-10-06T12:34:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a00a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd9241502f0 0x7fd9240c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a80b700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd92415a1a0 0x7fd9240c6c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47564/0 (socket says 172.21.15.46:47564) 2023-10-06T12:34:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a80b700 1 -- 172.21.15.46:0/768760125 learned_addr learned my addr 172.21.15.46:0/768760125 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929809700 1 --1- 172.21.15.46:0/768760125 >> v1:172.21.15.46:6789/0 conn(0x7fd9241506c0 0x7fd924159a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50142/0 (socket says 172.21.15.46:50142) 2023-10-06T12:34:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992641920 0 0) 0x7fd92406e810 con 0x7fd92415a1a0 2023-10-06T12:34:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd910003570 con 0x7fd92415a1a0 2023-10-06T12:34:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753878760 0 0) 0x7fd9240c67e0 con 0x7fd9241506c0 2023-10-06T12:34:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92406e810 con 0x7fd9241506c0 2023-10-06T12:34:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a00a700 1 -- 172.21.15.46:0/768760125 >> v1:172.21.15.132:6789/0 conn(0x7fd92415a1a0 legacy=0x7fd9240c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a00a700 1 -- 172.21.15.46:0/768760125 >> v1:172.21.15.46:6789/0 conn(0x7fd9241506c0 legacy=0x7fd924159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a00a700 1 -- 172.21.15.46:0/768760125 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92406edd0 con 0x7fd9241502f0 2023-10-06T12:34:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92a00a700 1 --2- 172.21.15.46:0/768760125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241502f0 0x7fd9240c0bd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd9200098d0 tx=0x7fd920009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a89685f3c477bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920005c10 con 0x7fd9241502f0 2023-10-06T12:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920005d70 con 0x7fd9241502f0 2023-10-06T12:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd929008700 1 -- 172.21.15.46:0/768760125 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920006260 con 0x7fd9241502f0 2023-10-06T12:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92c26e700 1 -- 172.21.15.46:0/768760125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241502f0 msgr2=0x7fd9240c0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92c26e700 1 --2- 172.21.15.46:0/768760125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241502f0 0x7fd9240c0bd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd9200098d0 tx=0x7fd920009120 comp rx=0 tx=0).stop 2023-10-06T12:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92c26e700 1 -- 172.21.15.46:0/768760125 shutdown_connections 2023-10-06T12:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.007+0000 7fd92c26e700 1 --2- 172.21.15.46:0/768760125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241502f0 0x7fd9240c0bd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.011+0000 7fd92c26e700 1 -- 172.21.15.46:0/768760125 >> 172.21.15.46:0/768760125 conn(0x7fd9240cbe30 msgr2=0x7fd9240cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.011+0000 7fd92c26e700 1 -- 172.21.15.46:0/768760125 shutdown_connections 2023-10-06T12:34:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.011+0000 7fd92c26e700 1 -- 172.21.15.46:0/768760125 wait complete. 2023-10-06T12:34:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.011+0000 7fd92c26e700 1 Processor -- start 2023-10-06T12:34:16.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.011+0000 7fd92c26e700 1 -- start start 2023-10-06T12:34:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9241502f0 0x7fd9240b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241506c0 0x7fd9240af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92416def0 con 0x7fd92415a1a0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240ac7f0 con 0x7fd9241506c0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240acad0 con 0x7fd9241502f0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58662/0 (socket says 172.21.15.46:58662) 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 -- 172.21.15.46:0/875501353 learned_addr learned my addr 172.21.15.46:0/875501353 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a00a700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9241502f0 0x7fd9240b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd929809700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241506c0 0x7fd9240af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 -- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9241502f0 msgr2=0x7fd9240b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9241502f0 0x7fd9240b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 -- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241506c0 msgr2=0x7fd9240af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241506c0 0x7fd9240af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 -- 172.21.15.46:0/875501353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200093c0 con 0x7fd92415a1a0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92a80b700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00cb70 tx=0x7fd91c0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00a7e0 con 0x7fd92415a1a0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c00a960 con 0x7fd92415a1a0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9240acdb0 con 0x7fd92415a1a0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.015+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00ad20 con 0x7fd92415a1a0 2023-10-06T12:34:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.019+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9240afb30 con 0x7fd92415a1a0 2023-10-06T12:34:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.019+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240acff0 con 0x7fd92415a1a0 2023-10-06T12:34:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.019+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd91c007300 con 0x7fd92415a1a0 2023-10-06T12:34:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.023+0000 7fd91a7fc700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 0x7fd908062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.023+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd91c07cc80 con 0x7fd92415a1a0 2023-10-06T12:34:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.023+0000 7fd92a00a700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 0x7fd908062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.023+0000 7fd92a00a700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 0x7fd908062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd920009160 tx=0x7fd920016000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.027+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd9240acff0 con 0x7fd92415a1a0 2023-10-06T12:34:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.199+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9240acff0 con 0x7fd92415a1a0 2023-10-06T12:34:16.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.203+0000 7fd91a7fc700 1 -- 172.21.15.46:0/875501353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+14447 (secure 0 0 0) 0x7fd9240acff0 con 0x7fd92415a1a0 2023-10-06T12:34:16.207 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:16.207 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":162,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:34:15.377734+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 msgr2=0x7fd908062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 0x7fd908062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd920009160 tx=0x7fd920016000 comp rx=0 tx=0).stop 2023-10-06T12:34:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 msgr2=0x7fd92414f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00cb70 tx=0x7fd91c0019e0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 shutdown_connections 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd908060670 0x7fd908062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd92415a1a0 0x7fd92414f3f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9241506c0 0x7fd9240af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 --2- 172.21.15.46:0/875501353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9241502f0 0x7fd9240b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 >> 172.21.15.46:0/875501353 conn(0x7fd9240cbe30 msgr2=0x7fd9240c0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 shutdown_connections 2023-10-06T12:34:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.207+0000 7fd92c26e700 1 -- 172.21.15.46:0/875501353 wait complete. 2023-10-06T12:34:16.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T12:34:16.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:34:16.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.395+0000 7f53e8e8e700 1 Processor -- start 2023-10-06T12:34:16.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.395+0000 7f53e8e8e700 1 -- start start 2023-10-06T12:34:16.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.395+0000 7f53e8e8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f53e40c7f20 0x7f53e40c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.395+0000 7f53e8e8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40c6cf0 con 0x7f53e4154c50 2023-10-06T12:34:16.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e8e8e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40c2a30 con 0x7f53e4154880 2023-10-06T12:34:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e8e8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40c83e0 con 0x7f53e40c7f20 2023-10-06T12:34:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e2ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f53e4154c50 0x7f53e415aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50144/0 (socket says 172.21.15.46:50144) 2023-10-06T12:34:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e2ffd700 1 -- 172.21.15.46:0/1812466569 learned_addr learned my addr 172.21.15.46:0/1812466569 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e3fff700 1 --2- 172.21.15.46:0/1812466569 >> v2:172.21.15.132:3300/0 conn(0x7f53e40c7f20 0x7f53e40c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128746804 0 0) 0x7f53e40c6cf0 con 0x7f53e4154c50 2023-10-06T12:34:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53cc003570 con 0x7f53e4154c50 2023-10-06T12:34:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472700734 0 0) 0x7f53e40c2a30 con 0x7f53e4154880 2023-10-06T12:34:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e40c6cf0 con 0x7f53e4154880 2023-10-06T12:34:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e3fff700 1 -- 172.21.15.46:0/1812466569 >> v1:172.21.15.132:6789/0 conn(0x7f53e4154880 legacy=0x7f53e406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e3fff700 1 -- 172.21.15.46:0/1812466569 >> v1:172.21.15.46:6789/0 conn(0x7f53e4154c50 legacy=0x7f53e415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e3fff700 1 -- 172.21.15.46:0/1812466569 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e406e5d0 con 0x7f53e40c7f20 2023-10-06T12:34:16.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e3fff700 1 --2- 172.21.15.46:0/1812466569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e40c7180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009e60 tx=0x7f53dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf4ccf119a5a04c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00e070 con 0x7f53e40c7f20 2023-10-06T12:34:16.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc00bab0 con 0x7f53e40c7f20 2023-10-06T12:34:16.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.399+0000 7f53e27fc700 1 -- 172.21.15.46:0/1812466569 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00be70 con 0x7f53e40c7f20 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- 172.21.15.46:0/1812466569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 msgr2=0x7f53e40c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/1812466569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e40c7180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009e60 tx=0x7f53dc009780 comp rx=0 tx=0).stop 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- 172.21.15.46:0/1812466569 shutdown_connections 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/1812466569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e40c7180 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- 172.21.15.46:0/1812466569 >> 172.21.15.46:0/1812466569 conn(0x7f53e40cbc10 msgr2=0x7f53e40cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- 172.21.15.46:0/1812466569 shutdown_connections 2023-10-06T12:34:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- 172.21.15.46:0/1812466569 wait complete. 2023-10-06T12:34:16.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 Processor -- start 2023-10-06T12:34:16.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- start start 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b12b0 con 0x7f53e4154c50 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b0170 con 0x7f53e40c7f20 2023-10-06T12:34:16.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.403+0000 7f53e8e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40af560 con 0x7f53e4154880 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e2ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58670/0 (socket says 172.21.15.46:58670) 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e2ffd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:52022/0 (socket says 172.21.15.46:52022) 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 -- 172.21.15.46:0/2702601544 learned_addr learned my addr 172.21.15.46:0/2702601544 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e37fe700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 -- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 msgr2=0x7f53e40b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 -- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 msgr2=0x7f53e414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 -- 172.21.15.46:0/2702601544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc009a20 con 0x7f53e4154c50 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e2ffd700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e37fe700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e3fff700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009460 tx=0x7f53dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc012e80 con 0x7f53e4154c50 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc026480 con 0x7f53e4154c50 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e40af840 con 0x7f53e4154c50 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc026840 con 0x7f53e4154c50 2023-10-06T12:34:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.407+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e40afe00 con 0x7f53e4154c50 2023-10-06T12:34:16.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.411+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4064590 con 0x7f53e4154c50 2023-10-06T12:34:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.411+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f53dc01c400 con 0x7f53e4154c50 2023-10-06T12:34:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.411+0000 7f53cb7fe700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 0x7f53d0062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.411+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f53dc07eaa0 con 0x7f53e4154c50 2023-10-06T12:34:16.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.411+0000 7f53e37fe700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 0x7f53d0062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.415+0000 7f53e37fe700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 0x7f53d0062ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53d4003990 tx=0x7f53d4009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.419+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f53dc04edc0 con 0x7f53e4154c50 2023-10-06T12:34:16.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.587+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53e40b02b0 con 0x7f53e4154c50 2023-10-06T12:34:16.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.587+0000 7f53cb7fe700 1 -- 172.21.15.46:0/2702601544 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53dc0523e0 con 0x7f53e4154c50 2023-10-06T12:34:16.593 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 msgr2=0x7f53d0062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 0x7f53d0062ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53d4003990 tx=0x7f53d4009040 comp rx=0 tx=0).stop 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 msgr2=0x7f53e415a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009460 tx=0x7f53dc019000 comp rx=0 tx=0).stop 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 shutdown_connections 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f53d00606f0 0x7f53d0062ba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4154c50 0x7f53e415a060 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53e4154880 0x7f53e40b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 --2- 172.21.15.46:0/2702601544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53e40c7f20 0x7f53e414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 >> 172.21.15.46:0/2702601544 conn(0x7f53e40cbc10 msgr2=0x7f53e41539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 shutdown_connections 2023-10-06T12:34:16.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.591+0000 7f53e8e8e700 1 -- 172.21.15.46:0/2702601544 wait complete. 2023-10-06T12:34:16.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.779+0000 7f4c3359e700 1 Processor -- start 2023-10-06T12:34:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.779+0000 7f4c3359e700 1 -- start start 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3359e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c340b0dc0 0x7f4c340b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3359e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340afd20 con 0x7f4c340adbe0 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3359e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c34159740 con 0x7f4c340b0dc0 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340b1760 con 0x7f4c340b30f0 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4c340adbe0 0x7f4c340d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50148/0 (socket says 172.21.15.46:50148) 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c3259c700 1 -- 172.21.15.46:0/2580006976 learned_addr learned my addr 172.21.15.46:0/2580006976 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c29d9b700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c32d9d700 1 --2- 172.21.15.46:0/2580006976 >> v2:172.21.15.132:3300/0 conn(0x7f4c340b0dc0 0x7f4c340b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.783+0000 7f4c31d9b700 1 -- 172.21.15.46:0/2580006976 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884638860 0 0) 0x7f4c340afd20 con 0x7f4c340adbe0 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c31d9b700 1 -- 172.21.15.46:0/2580006976 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c20003570 con 0x7f4c340adbe0 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c32d9d700 1 -- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 msgr2=0x7f4c340b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c32d9d700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c32d9d700 1 -- 172.21.15.46:0/2580006976 >> v1:172.21.15.46:6789/0 conn(0x7f4c340adbe0 legacy=0x7f4c340d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c32d9d700 1 -- 172.21.15.46:0/2580006976 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340b0000 con 0x7f4c340b0dc0 2023-10-06T12:34:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c32d9d700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340b0dc0 0x7f4c340b1220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009910 tx=0x7f4c2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2de3040e6edfa74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c31d9b700 1 -- 172.21.15.46:0/2580006976 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c011630 con 0x7f4c340b0dc0 2023-10-06T12:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c31d9b700 1 -- 172.21.15.46:0/2580006976 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c011790 con 0x7f4c340b0dc0 2023-10-06T12:34:16.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c31d9b700 1 -- 172.21.15.46:0/2580006976 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c011b30 con 0x7f4c340b0dc0 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 -- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340b0dc0 msgr2=0x7f4c340b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340b0dc0 0x7f4c340b1220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009910 tx=0x7f4c2c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 -- 172.21.15.46:0/2580006976 shutdown_connections 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340b0dc0 0x7f4c340b1220 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 --2- 172.21.15.46:0/2580006976 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 -- 172.21.15.46:0/2580006976 >> 172.21.15.46:0/2580006976 conn(0x7f4c3406ef60 msgr2=0x7f4c340d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 -- 172.21.15.46:0/2580006976 shutdown_connections 2023-10-06T12:34:16.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.787+0000 7f4c3359e700 1 -- 172.21.15.46:0/2580006976 wait complete. 2023-10-06T12:34:16.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 Processor -- start 2023-10-06T12:34:16.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 -- start start 2023-10-06T12:34:16.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340adbe0 0x7f4c34151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3415a740 con 0x7f4c340b0dc0 2023-10-06T12:34:16.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3415aa20 con 0x7f4c340adbe0 2023-10-06T12:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3415ad00 con 0x7f4c340b30f0 2023-10-06T12:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c29d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c29d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58686/0 (socket says 172.21.15.46:58686) 2023-10-06T12:34:16.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c29d9b700 1 -- 172.21.15.46:0/68361989 learned_addr learned my addr 172.21.15.46:0/68361989 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c3259c700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340adbe0 0x7f4c34151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.791+0000 7f4c32d9d700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 -- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 msgr2=0x7f4c340d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 -- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340adbe0 msgr2=0x7f4c34151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340adbe0 0x7f4c34151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 -- 172.21.15.46:0/68361989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009420 con 0x7f4c340b0dc0 2023-10-06T12:34:16.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c29d9b700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ec80 tx=0x7f4c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00c850 con 0x7f4c340b0dc0 2023-10-06T12:34:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00c9b0 con 0x7f4c340b0dc0 2023-10-06T12:34:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3415afe0 con 0x7f4c340b0dc0 2023-10-06T12:34:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00cd50 con 0x7f4c340b0dc0 2023-10-06T12:34:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34152290 con 0x7f4c340b0dc0 2023-10-06T12:34:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.795+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34153a00 con 0x7f4c340b0dc0 2023-10-06T12:34:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.799+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4c1c013680 con 0x7f4c340b0dc0 2023-10-06T12:34:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.799+0000 7f4c2affd700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 0x7f4c14062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.799+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4c1c07cc40 con 0x7f4c340b0dc0 2023-10-06T12:34:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.799+0000 7f4c3259c700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 0x7f4c14062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:16.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.799+0000 7f4c3259c700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 0x7f4c14062ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005c50 tx=0x7f4c24009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:16.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.807+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c34153a00 con 0x7f4c340b0dc0 2023-10-06T12:34:16.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.983+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c34153a00 con 0x7f4c340b0dc0 2023-10-06T12:34:16.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.983+0000 7f4c2affd700 1 -- 172.21.15.46:0/68361989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v162) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c34153a00 con 0x7f4c340b0dc0 2023-10-06T12:34:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:34:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 msgr2=0x7f4c14062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 0x7f4c14062ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005c50 tx=0x7f4c24009040 comp rx=0 tx=0).stop 2023-10-06T12:34:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 msgr2=0x7f4c340d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ec80 tx=0x7f4c1c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 shutdown_connections 2023-10-06T12:34:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f4c140606f0 0x7f4c14062ba0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c340b30f0 0x7f4c340d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c340b0dc0 0x7f4c340d0330 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 --2- 172.21.15.46:0/68361989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c340adbe0 0x7f4c34151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 >> 172.21.15.46:0/68361989 conn(0x7f4c3406ef60 msgr2=0x7f4c34158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 shutdown_connections 2023-10-06T12:34:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:16.987+0000 7f4c3359e700 1 -- 172.21.15.46:0/68361989 wait complete. 2023-10-06T12:34:17.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T12:34:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.167+0000 7fa8dcecd700 1 Processor -- start 2023-10-06T12:34:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 -- start start 2023-10-06T12:34:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 0x7fa8d8161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d80c18f0 con 0x7fa8d806cc30 2023-10-06T12:34:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d80c0ce0 con 0x7fa8d80c1d80 2023-10-06T12:34:17.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8dcecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d80c0fc0 con 0x7fa8d8161470 2023-10-06T12:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8cffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8d659c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 0x7fa8d8161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8d6d9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fa8d806cc30 0x7fa8d815d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47586/0 (socket says 172.21.15.46:47586) 2023-10-06T12:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8cffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:46488/0 (socket says 172.21.15.46:46488) 2023-10-06T12:34:17.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.171+0000 7fa8d6d9d700 1 -- 172.21.15.46:0/1238119400 learned_addr learned my addr 172.21.15.46:0/1238119400 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d659c700 1 -- 172.21.15.46:0/1238119400 >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 msgr2=0x7fa8d80c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d659c700 1 --2- 172.21.15.46:0/1238119400 >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d659c700 1 -- 172.21.15.46:0/1238119400 >> v1:172.21.15.132:6789/0 conn(0x7fa8d806cc30 legacy=0x7fa8d815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d659c700 1 -- 172.21.15.46:0/1238119400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80c12a0 con 0x7fa8d8161470 2023-10-06T12:34:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8cffff700 1 --2- 172.21.15.46:0/1238119400 >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d659c700 1 --2- 172.21.15.46:0/1238119400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 0x7fa8d8161840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80098d0 tx=0x7fa8c8009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1368bf833647fc13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d5d9b700 1 -- 172.21.15.46:0/1238119400 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8005c10 con 0x7fa8d8161470 2023-10-06T12:34:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d5d9b700 1 -- 172.21.15.46:0/1238119400 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c8005d70 con 0x7fa8d8161470 2023-10-06T12:34:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8d5d9b700 1 -- 172.21.15.46:0/1238119400 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8006280 con 0x7fa8d8161470 2023-10-06T12:34:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 -- 172.21.15.46:0/1238119400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 msgr2=0x7fa8d8161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/1238119400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 0x7fa8d8161840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80098d0 tx=0x7fa8c8009120 comp rx=0 tx=0).stop 2023-10-06T12:34:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 -- 172.21.15.46:0/1238119400 shutdown_connections 2023-10-06T12:34:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/1238119400 >> v2:172.21.15.132:3300/0 conn(0x7fa8d80c1d80 0x7fa8d80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/1238119400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d8161470 0x7fa8d8161840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 -- 172.21.15.46:0/1238119400 >> 172.21.15.46:0/1238119400 conn(0x7fa8d80cbe30 msgr2=0x7fa8d80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 -- 172.21.15.46:0/1238119400 shutdown_connections 2023-10-06T12:34:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.175+0000 7fa8dcecd700 1 -- 172.21.15.46:0/1238119400 wait complete. 2023-10-06T12:34:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 Processor -- start 2023-10-06T12:34:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 -- start start 2023-10-06T12:34:17.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 0x7fa8d80c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 0x7fa8d80cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d81591e0 con 0x7fa8d806cc30 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d815dc20 con 0x7fa8d8161470 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.179+0000 7fa8dcecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d8160540 con 0x7fa8d80c1d80 2023-10-06T12:34:17.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58690/0 (socket says 172.21.15.46:58690) 2023-10-06T12:34:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 -- 172.21.15.46:0/3706794718 learned_addr learned my addr 172.21.15.46:0/3706794718 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d6d9d700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 0x7fa8d80cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8cffff700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 0x7fa8d80c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 -- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 msgr2=0x7fa8d80c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 0x7fa8d80c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 -- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 msgr2=0x7fa8d80cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 0x7fa8d80cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 -- 172.21.15.46:0/3706794718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0009420 con 0x7fa8d806cc30 2023-10-06T12:34:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d6d9d700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 0x7fa8d80cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8cffff700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 0x7fa8d80c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8d659c700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009f70 tx=0x7fa8c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800b7d0 con 0x7fa8d806cc30 2023-10-06T12:34:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c800b950 con 0x7fa8d806cc30 2023-10-06T12:34:17.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800bd10 con 0x7fa8d806cc30 2023-10-06T12:34:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c80093c0 con 0x7fa8d806cc30 2023-10-06T12:34:17.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d815d2f0 con 0x7fa8d806cc30 2023-10-06T12:34:17.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.183+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d8063f90 con 0x7fa8d806cc30 2023-10-06T12:34:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.187+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa8c801b690 con 0x7fa8d806cc30 2023-10-06T12:34:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.187+0000 7fa8ce7fc700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 0x7fa8b8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.187+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa8c807ebc0 con 0x7fa8d806cc30 2023-10-06T12:34:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.187+0000 7fa8cffff700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 0x7fa8b8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.187+0000 7fa8cffff700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 0x7fa8b8062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00093f0 tx=0x7fa8c0008040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.195+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa8c804eff0 con 0x7fa8d806cc30 2023-10-06T12:34:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.367+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8d80c1580 con 0x7fa8d806cc30 2023-10-06T12:34:17.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.371+0000 7fa8ce7fc700 1 -- 172.21.15.46:0/3706794718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v162) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa8c8052610 con 0x7fa8d806cc30 2023-10-06T12:34:17.374 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:17.374 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T12:34:17.375 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T12:34:17.376 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:17.376 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:34:17.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 msgr2=0x7fa8b8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 0x7fa8b8062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00093f0 tx=0x7fa8c0008040 comp rx=0 tx=0).stop 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 msgr2=0x7fa8d8158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009f70 tx=0x7fa8c8016000 comp rx=0 tx=0).stop 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 shutdown_connections 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fa8b8060670 0x7fa8b8062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8d8161470 0x7fa8d80cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8d80c1d80 0x7fa8d80c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 --2- 172.21.15.46:0/3706794718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8d806cc30 0x7fa8d8158ca0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 >> 172.21.15.46:0/3706794718 conn(0x7fa8d80cbe30 msgr2=0x7fa8d80ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 shutdown_connections 2023-10-06T12:34:17.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.375+0000 7fa8dcecd700 1 -- 172.21.15.46:0/3706794718 wait complete. 2023-10-06T12:34:17.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T12:34:17.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T12:34:17.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.567+0000 7fcf7a0a5700 1 Processor -- start 2023-10-06T12:34:17.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.567+0000 7fcf7a0a5700 1 -- start start 2023-10-06T12:34:17.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf7a0a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740b2600 0x7fcf740ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf7a0a5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcf740cae50 0x7fcf740c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf7a0a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740cbff0 con 0x7fcf740b29d0 2023-10-06T12:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf7a0a5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf741598f0 con 0x7fcf740cae50 2023-10-06T12:34:17.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf7a0a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740cb3e0 con 0x7fcf740b2600 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf72ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf740b29d0 0x7fcf740ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50154/0 (socket says 172.21.15.46:50154) 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf72ffd700 1 -- 172.21.15.46:0/4051341979 learned_addr learned my addr 172.21.15.46:0/4051341979 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf737fe700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740b2600 0x7fcf740ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 --2- 172.21.15.46:0/4051341979 >> v2:172.21.15.132:3300/0 conn(0x7fcf740cae50 0x7fcf740c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf727fc700 1 -- 172.21.15.46:0/4051341979 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063509100 0 0) 0x7fcf740cbff0 con 0x7fcf740b29d0 2023-10-06T12:34:17.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf727fc700 1 -- 172.21.15.46:0/4051341979 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c003570 con 0x7fcf740b29d0 2023-10-06T12:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 -- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740b2600 msgr2=0x7fcf740ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740b2600 0x7fcf740ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 -- 172.21.15.46:0/4051341979 >> v1:172.21.15.46:6789/0 conn(0x7fcf740b29d0 legacy=0x7fcf740ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 -- 172.21.15.46:0/4051341979 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740cb6c0 con 0x7fcf740cae50 2023-10-06T12:34:17.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.571+0000 7fcf73fff700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740cae50 0x7fcf740c4ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009e60 tx=0x7fcf6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5618dc8a9c0bfe72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf727fc700 1 -- 172.21.15.46:0/4051341979 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e070 con 0x7fcf740cae50 2023-10-06T12:34:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf727fc700 1 -- 172.21.15.46:0/4051341979 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00bab0 con 0x7fcf740cae50 2023-10-06T12:34:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf727fc700 1 -- 172.21.15.46:0/4051341979 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00be70 con 0x7fcf740cae50 2023-10-06T12:34:17.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740cae50 msgr2=0x7fcf740c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740cae50 0x7fcf740c4ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009e60 tx=0x7fcf6c009780 comp rx=0 tx=0).stop 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/4051341979 shutdown_connections 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740cae50 0x7fcf740c4ff0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/4051341979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740b2600 0x7fcf740ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/4051341979 >> 172.21.15.46:0/4051341979 conn(0x7fcf7406ef60 msgr2=0x7fcf740d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/4051341979 shutdown_connections 2023-10-06T12:34:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.575+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/4051341979 wait complete. 2023-10-06T12:34:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 Processor -- start 2023-10-06T12:34:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 -- start start 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740b29d0 0x7fcf7414c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740cae50 0x7fcf741558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7414e330 con 0x7fcf740b2600 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7414cbc0 con 0x7fcf740b29d0 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf7a0a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c81f0 con 0x7fcf740cae50 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58698/0 (socket says 172.21.15.46:58698) 2023-10-06T12:34:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf737fe700 1 -- 172.21.15.46:0/2357384237 learned_addr learned my addr 172.21.15.46:0/2357384237 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf72ffd700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740b29d0 0x7fcf7414c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.579+0000 7fcf73fff700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740cae50 0x7fcf741558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 -- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740cae50 msgr2=0x7fcf741558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740cae50 0x7fcf741558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 -- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740b29d0 msgr2=0x7fcf7414c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740b29d0 0x7fcf7414c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 -- 172.21.15.46:0/2357384237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009a20 con 0x7fcf740b2600 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf737fe700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400ea00 tx=0x7fcf64009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400c730 con 0x7fcf740b2600 2023-10-06T12:34:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400c890 con 0x7fcf740b2600 2023-10-06T12:34:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740b2ff0 con 0x7fcf740b2600 2023-10-06T12:34:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400cc10 con 0x7fcf740b2600 2023-10-06T12:34:17.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740b35b0 con 0x7fcf740b2600 2023-10-06T12:34:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.583+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf74069b50 con 0x7fcf740b2600 2023-10-06T12:34:17.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.587+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fcf64013470 con 0x7fcf740b2600 2023-10-06T12:34:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.587+0000 7fcf5b7fe700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 0x7fcf60062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.587+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fcf6407cef0 con 0x7fcf740b2600 2023-10-06T12:34:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.587+0000 7fcf72ffd700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 0x7fcf60062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.591+0000 7fcf72ffd700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 0x7fcf60062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007fd0 tx=0x7fcf68009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:17.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.595+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcf6404d3c0 con 0x7fcf740b2600 2023-10-06T12:34:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:17.767+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcf740cb9a0 con 0x7fcf740b2600 2023-10-06T12:34:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.383+0000 7fcf5b7fe700 1 -- 172.21.15.46:0/2357384237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v163) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf640210b0 con 0x7fcf740b2600 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 msgr2=0x7fcf60062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 0x7fcf60062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007fd0 tx=0x7fcf68009040 comp rx=0 tx=0).stop 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 msgr2=0x7fcf7414ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400ea00 tx=0x7fcf64009120 comp rx=0 tx=0).stop 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 shutdown_connections 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fcf60060670 0x7fcf60062b20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf740cae50 0x7fcf741558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf740b29d0 0x7fcf7414c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 --2- 172.21.15.46:0/2357384237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf740b2600 0x7fcf7414ddf0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.387+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 >> 172.21.15.46:0/2357384237 conn(0x7fcf7406ef60 msgr2=0x7fcf740c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.391+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 shutdown_connections 2023-10-06T12:34:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.391+0000 7fcf7a0a5700 1 -- 172.21.15.46:0/2357384237 wait complete. 2023-10-06T12:34:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T12:34:18.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T12:34:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.591+0000 7fc00359e700 1 Processor -- start 2023-10-06T12:34:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.591+0000 7fc00359e700 1 -- start start 2023-10-06T12:34:18.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc00359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 0x7fc0041594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:18.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc00359e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc004068c30 con 0x7fc0040cc5f0 2023-10-06T12:34:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc00359e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040cd0a0 con 0x7fc0040cade0 2023-10-06T12:34:18.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc00359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00406a0d0 con 0x7fc0041590d0 2023-10-06T12:34:18.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc00259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 0x7fc0041594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:18.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc001d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0040cc5f0 0x7fc0040ca6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50160/0 (socket says 172.21.15.46:50160) 2023-10-06T12:34:18.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc002d9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc0040cade0 0x7fc0040afa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47598/0 (socket says 172.21.15.46:47598) 2023-10-06T12:34:18.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.595+0000 7fc001d9b700 1 -- 172.21.15.46:0/3818669148 learned_addr learned my addr 172.21.15.46:0/3818669148 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835533730 0 0) 0x7fc004068c30 con 0x7fc0040cc5f0 2023-10-06T12:34:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfec003570 con 0x7fc0040cc5f0 2023-10-06T12:34:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363000214 0 0) 0x7fc0040cd0a0 con 0x7fc0040cade0 2023-10-06T12:34:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00259c700 1 -- 172.21.15.46:0/3818669148 >> v1:172.21.15.132:6789/0 conn(0x7fc0040cade0 legacy=0x7fc0040afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00259c700 1 -- 172.21.15.46:0/3818669148 >> v1:172.21.15.46:6789/0 conn(0x7fc0040cc5f0 legacy=0x7fc0040ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00259c700 1 -- 172.21.15.46:0/3818669148 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004068300 con 0x7fc0041590d0 2023-10-06T12:34:18.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00259c700 1 --2- 172.21.15.46:0/3818669148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 0x7fc0041594a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009900 tx=0x7fbff4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96e1cd632998906 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4005c10 con 0x7fc0041590d0 2023-10-06T12:34:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4005d70 con 0x7fc0041590d0 2023-10-06T12:34:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00159a700 1 -- 172.21.15.46:0/3818669148 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4006260 con 0x7fc0041590d0 2023-10-06T12:34:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 -- 172.21.15.46:0/3818669148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 msgr2=0x7fc0041594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 --2- 172.21.15.46:0/3818669148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 0x7fc0041594a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009900 tx=0x7fbff4009120 comp rx=0 tx=0).stop 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 -- 172.21.15.46:0/3818669148 shutdown_connections 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 --2- 172.21.15.46:0/3818669148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0041590d0 0x7fc0041594a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 -- 172.21.15.46:0/3818669148 >> 172.21.15.46:0/3818669148 conn(0x7fc0040970c0 msgr2=0x7fc004067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 -- 172.21.15.46:0/3818669148 shutdown_connections 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.599+0000 7fc00359e700 1 -- 172.21.15.46:0/3818669148 wait complete. 2023-10-06T12:34:18.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 Processor -- start 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 -- start start 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 0x7fc004154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0041590d0 0x7fc0040ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040c2290 con 0x7fc0040cc5f0 2023-10-06T12:34:18.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040c4900 con 0x7fc0041590d0 2023-10-06T12:34:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040c1960 con 0x7fc0040cade0 2023-10-06T12:34:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc001d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc001d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58710/0 (socket says 172.21.15.46:58710) 2023-10-06T12:34:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc001d9b700 1 -- 172.21.15.46:0/3525171234 learned_addr learned my addr 172.21.15.46:0/3525171234 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:18.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc002d9d700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0041590d0 0x7fc0040ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc00259c700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 0x7fc004154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.603+0000 7fc001d9b700 1 -- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 msgr2=0x7fc004154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc001d9b700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 0x7fc004154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc001d9b700 1 -- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0041590d0 msgr2=0x7fc0040ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc001d9b700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0041590d0 0x7fc0040ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:18.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc001d9b700 1 -- 172.21.15.46:0/3525171234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40093c0 con 0x7fc0040cc5f0 2023-10-06T12:34:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc00259c700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 0x7fc004154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:18.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc001d9b700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbff80011e0 tx=0x7fbff8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8005b30 con 0x7fc0040cc5f0 2023-10-06T12:34:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8005c90 con 0x7fc0040cc5f0 2023-10-06T12:34:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8007320 con 0x7fc0040cc5f0 2023-10-06T12:34:18.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040c1c40 con 0x7fc0040cc5f0 2023-10-06T12:34:18.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc004155440 con 0x7fc0040cc5f0 2023-10-06T12:34:18.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.607+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004064590 con 0x7fc0040cc5f0 2023-10-06T12:34:18.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.611+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fbff8007690 con 0x7fc0040cc5f0 2023-10-06T12:34:18.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.611+0000 7fbff27fc700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 0x7fbfe4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:18.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.611+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6092+0+0 (secure 0 0 0) 0x7fbff807bfe0 con 0x7fc0040cc5f0 2023-10-06T12:34:18.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.611+0000 7fc00259c700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 0x7fbfe4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:18.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.615+0000 7fc00259c700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 0x7fbfe4062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009160 tx=0x7fbff4016000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:18.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.619+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbff804c2c0 con 0x7fc0040cc5f0 2023-10-06T12:34:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:18.787+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc0040685e0 con 0x7fc0040cc5f0 2023-10-06T12:34:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.399+0000 7fbff27fc700 1 -- 172.21.15.46:0/3525171234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v164) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbff80170d0 con 0x7fc0040cc5f0 2023-10-06T12:34:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 msgr2=0x7fbfe4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 0x7fbfe4062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009160 tx=0x7fbff4016000 comp rx=0 tx=0).stop 2023-10-06T12:34:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 msgr2=0x7fc0040c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbff80011e0 tx=0x7fbff8007df0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 shutdown_connections 2023-10-06T12:34:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fbfe4060670 0x7fbfe4062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0041590d0 0x7fc0040ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040cc5f0 0x7fc0040c3f20 secure :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fbff80011e0 tx=0x7fbff8007df0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 --2- 172.21.15.46:0/3525171234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0040cade0 0x7fc004154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.407+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 >> 172.21.15.46:0/3525171234 conn(0x7fc0040970c0 msgr2=0x7fc00406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.415+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 shutdown_connections 2023-10-06T12:34:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.415+0000 7fc00359e700 1 -- 172.21.15.46:0/3525171234 wait complete. 2023-10-06T12:34:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T12:34:19.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T12:34:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.603+0000 7f299e2e3700 1 Processor -- start 2023-10-06T12:34:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.607+0000 7f299e2e3700 1 -- start start 2023-10-06T12:34:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299e2e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 0x7f29980b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299e2e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29980b3570 0x7f29980b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299e2e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29980afd20 con 0x7f29980adbe0 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299e2e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2998159740 con 0x7f29980b3570 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299e2e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29980b0000 con 0x7f29980b30f0 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299cae0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 0x7f29980b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299d2e1700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f29980adbe0 0x7f29980d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47610/0 (socket says 172.21.15.46:47610) 2023-10-06T12:34:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299d2e1700 1 -- 172.21.15.46:0/2544468709 learned_addr learned my addr 172.21.15.46:0/2544468709 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 --2- 172.21.15.46:0/2544468709 >> v2:172.21.15.46:3300/0 conn(0x7f29980b3570 0x7f29980b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f298ffff700 1 -- 172.21.15.46:0/2544468709 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550065205 0 0) 0x7f29980afd20 con 0x7f29980adbe0 2023-10-06T12:34:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f298ffff700 1 -- 172.21.15.46:0/2544468709 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2984003570 con 0x7f29980adbe0 2023-10-06T12:34:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 -- 172.21.15.46:0/2544468709 >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 msgr2=0x7f29980b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 --2- 172.21.15.46:0/2544468709 >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 0x7f29980b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 -- 172.21.15.46:0/2544468709 >> v1:172.21.15.132:6789/0 conn(0x7f29980adbe0 legacy=0x7f29980d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 -- 172.21.15.46:0/2544468709 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980b02e0 con 0x7f29980b3570 2023-10-06T12:34:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299cae0700 1 --2- 172.21.15.46:0/2544468709 >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 0x7f29980b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f299dae2700 1 --2- 172.21.15.46:0/2544468709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b3570 0x7f29980b1300 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f29980881c0 tx=0x7f2990009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb158dc79bcc2780 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f298ffff700 1 -- 172.21.15.46:0/2544468709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000e070 con 0x7f29980b3570 2023-10-06T12:34:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f298ffff700 1 -- 172.21.15.46:0/2544468709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299000bab0 con 0x7f29980b3570 2023-10-06T12:34:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.611+0000 7f298ffff700 1 -- 172.21.15.46:0/2544468709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000be70 con 0x7f29980b3570 2023-10-06T12:34:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 -- 172.21.15.46:0/2544468709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b3570 msgr2=0x7f29980b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2544468709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b3570 0x7f29980b1300 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f29980881c0 tx=0x7f2990009780 comp rx=0 tx=0).stop 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 -- 172.21.15.46:0/2544468709 shutdown_connections 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2544468709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b3570 0x7f29980b1300 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2544468709 >> v2:172.21.15.132:3300/0 conn(0x7f29980b30f0 0x7f29980b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 -- 172.21.15.46:0/2544468709 >> 172.21.15.46:0/2544468709 conn(0x7f299806ef60 msgr2=0x7f29980d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 -- 172.21.15.46:0/2544468709 shutdown_connections 2023-10-06T12:34:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 -- 172.21.15.46:0/2544468709 wait complete. 2023-10-06T12:34:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.615+0000 7f299e2e3700 1 Processor -- start 2023-10-06T12:34:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 -- start start 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29980adbe0 0x7f299815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 0x7f2998140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29981553c0 con 0x7f29980b30f0 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29981547b0 con 0x7f29980b3570 2023-10-06T12:34:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299e2e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2998154a90 con 0x7f29980adbe0 2023-10-06T12:34:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58734/0 (socket says 172.21.15.46:58734) 2023-10-06T12:34:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 -- 172.21.15.46:0/2888463022 learned_addr learned my addr 172.21.15.46:0/2888463022 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299d2e1700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29980adbe0 0x7f299815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299dae2700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 0x7f2998140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 -- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29980adbe0 msgr2=0x7f299815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29980adbe0 0x7f299815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 -- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 msgr2=0x7f2998140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 0x7f2998140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299cae0700 1 -- 172.21.15.46:0/2888463022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009420 con 0x7f29980b30f0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.619+0000 7f299dae2700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 0x7f2998140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f299cae0700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f298800ec80 tx=0x7f29880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800c8f0 con 0x7f29980b30f0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298800ca50 con 0x7f29980b30f0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990009a20 con 0x7f29980b30f0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800cdd0 con 0x7f29980b30f0 2023-10-06T12:34:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2998155050 con 0x7f29980b30f0 2023-10-06T12:34:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.623+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2998064590 con 0x7f29980b30f0 2023-10-06T12:34:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.627+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2988013680 con 0x7f29980b30f0 2023-10-06T12:34:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.627+0000 7f298d7fa700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 0x7f297c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.627+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6474+0+0 (secure 0 0 0) 0x7f298807d7c0 con 0x7f29980b30f0 2023-10-06T12:34:19.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.627+0000 7f299d2e1700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 0x7f297c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:19.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.631+0000 7f299d2e1700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 0x7f297c062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2994007fd0 tx=0x7f2994009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:19.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.635+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f298804d980 con 0x7f29980b30f0 2023-10-06T12:34:19.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:19.843+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f29980675d0 con 0x7f29980b30f0 2023-10-06T12:34:20.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.411+0000 7f298d7fa700 1 -- 172.21.15.46:0/2888463022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2988050fa0 con 0x7f29980b30f0 2023-10-06T12:34:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.423+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 msgr2=0x7f297c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.423+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 0x7f297c062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2994007fd0 tx=0x7f2994009040 comp rx=0 tx=0).stop 2023-10-06T12:34:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.427+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 msgr2=0x7f2998153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.427+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f298800ec80 tx=0x7f29880091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 shutdown_connections 2023-10-06T12:34:20.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f297c060670 0x7f297c062b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980b3570 0x7f2998140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29980b30f0 0x7f2998153f60 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 --2- 172.21.15.46:0/2888463022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29980adbe0 0x7f299815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 >> 172.21.15.46:0/2888463022 conn(0x7f299806ef60 msgr2=0x7f29980b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 shutdown_connections 2023-10-06T12:34:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.431+0000 7f299e2e3700 1 -- 172.21.15.46:0/2888463022 wait complete. 2023-10-06T12:34:20.444 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T12:34:20.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:20.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.611+0000 7f60354e3700 1 Processor -- start 2023-10-06T12:34:20.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.611+0000 7f60354e3700 1 -- start start 2023-10-06T12:34:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f60354e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 0x7f6030067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f60354e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f60301591a0 0x7f6030067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f60354e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6030069780 con 0x7f6030068bf0 2023-10-06T12:34:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f60354e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300699c0 con 0x7f60301591a0 2023-10-06T12:34:20.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f60354e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603015db50 con 0x7f6030158dd0 2023-10-06T12:34:20.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f6034ce2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6030068bf0 0x7f6030069070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50162/0 (socket says 172.21.15.46:50162) 2023-10-06T12:34:20.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f6034ce2700 1 -- 172.21.15.46:0/4069524823 learned_addr learned my addr 172.21.15.46:0/4069524823 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602ffff700 1 --2- 172.21.15.46:0/4069524823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 0x7f6030067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f6027fff700 1 --2- 172.21.15.46:0/4069524823 >> v2:172.21.15.132:3300/0 conn(0x7f60301591a0 0x7f6030067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602f7fe700 1 -- 172.21.15.46:0/4069524823 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060129430 0 0) 0x7f6028001490 con 0x7f6030068bf0 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602f7fe700 1 -- 172.21.15.46:0/4069524823 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f601c003570 con 0x7f6030068bf0 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602ffff700 1 -- 172.21.15.46:0/4069524823 >> v2:172.21.15.132:3300/0 conn(0x7f60301591a0 msgr2=0x7f6030067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602ffff700 1 --2- 172.21.15.46:0/4069524823 >> v2:172.21.15.132:3300/0 conn(0x7f60301591a0 0x7f6030067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602ffff700 1 -- 172.21.15.46:0/4069524823 >> v1:172.21.15.46:6789/0 conn(0x7f6030068bf0 legacy=0x7f6030069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.615+0000 7f602ffff700 1 -- 172.21.15.46:0/4069524823 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603015d6c0 con 0x7f6030158dd0 2023-10-06T12:34:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f602ffff700 1 --2- 172.21.15.46:0/4069524823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 0x7f6030067050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60200098e0 tx=0x7f60200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93abbc83914feb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f602f7fe700 1 -- 172.21.15.46:0/4069524823 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000b640 con 0x7f6030158dd0 2023-10-06T12:34:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f602f7fe700 1 -- 172.21.15.46:0/4069524823 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000b7a0 con 0x7f6030158dd0 2023-10-06T12:34:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f602f7fe700 1 -- 172.21.15.46:0/4069524823 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000bb40 con 0x7f6030158dd0 2023-10-06T12:34:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 -- 172.21.15.46:0/4069524823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 msgr2=0x7f6030067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 --2- 172.21.15.46:0/4069524823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 0x7f6030067050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f60200098e0 tx=0x7f60200091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 -- 172.21.15.46:0/4069524823 shutdown_connections 2023-10-06T12:34:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 --2- 172.21.15.46:0/4069524823 >> v2:172.21.15.132:3300/0 conn(0x7f60301591a0 0x7f6030067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 --2- 172.21.15.46:0/4069524823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030158dd0 0x7f6030067050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 -- 172.21.15.46:0/4069524823 >> 172.21.15.46:0/4069524823 conn(0x7f6030062ef0 msgr2=0x7f60300cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 -- 172.21.15.46:0/4069524823 shutdown_connections 2023-10-06T12:34:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.619+0000 7f60354e3700 1 -- 172.21.15.46:0/4069524823 wait complete. 2023-10-06T12:34:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 Processor -- start 2023-10-06T12:34:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 -- start start 2023-10-06T12:34:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 0x7f6030145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60301591a0 0x7f603006da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300c25d0 con 0x7f6030158dd0 2023-10-06T12:34:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60300c28b0 con 0x7f60301591a0 2023-10-06T12:34:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f60354e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60300c38f0 con 0x7f6030068bf0 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58744/0 (socket says 172.21.15.46:58744) 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 -- 172.21.15.46:0/2091467424 learned_addr learned my addr 172.21.15.46:0/2091467424 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f602ffff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 0x7f6030145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 -- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 msgr2=0x7f6030145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 0x7f6030145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 -- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60301591a0 msgr2=0x7f603006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60301591a0 0x7f603006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f6027fff700 1 -- 172.21.15.46:0/2091467424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009420 con 0x7f6030158dd0 2023-10-06T12:34:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.623+0000 7f602ffff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 0x7f6030145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f6027fff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f601800d860 tx=0x7f6018008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018009610 con 0x7f6030158dd0 2023-10-06T12:34:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6018009770 con 0x7f6030158dd0 2023-10-06T12:34:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60300c3bd0 con 0x7f6030158dd0 2023-10-06T12:34:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018009b30 con 0x7f6030158dd0 2023-10-06T12:34:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6030145c50 con 0x7f6030158dd0 2023-10-06T12:34:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.627+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030064590 con 0x7f6030158dd0 2023-10-06T12:34:20.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.631+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f601801e460 con 0x7f6030158dd0 2023-10-06T12:34:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.631+0000 7f602cff9700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 0x7f6010062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.631+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f601807d360 con 0x7f6030158dd0 2023-10-06T12:34:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.631+0000 7f602ffff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 0x7f6010062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.631+0000 7f602ffff700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 0x7f6010062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f60200093f0 tx=0x7f6020017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.639+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f601804d440 con 0x7f6030158dd0 2023-10-06T12:34:20.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.815+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f603015cd90 con 0x7f6030158dd0 2023-10-06T12:34:20.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f602cff9700 1 -- 172.21.15.46:0/2091467424 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+17680 (secure 0 0 0) 0x7f6018050a60 con 0x7f6030158dd0 2023-10-06T12:34:20.822 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:20.822 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":165,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:34:19.846189+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:20.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 msgr2=0x7f6010062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 0x7f6010062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f60200093f0 tx=0x7f6020017000 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 msgr2=0x7f60300c1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f601800d860 tx=0x7f6018008040 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 shutdown_connections 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f6010060670 0x7f6010062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60301591a0 0x7f603006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6030158dd0 0x7f60300c1d70 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 --2- 172.21.15.46:0/2091467424 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6030068bf0 0x7f6030145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 >> 172.21.15.46:0/2091467424 conn(0x7f6030062ef0 msgr2=0x7f6030158030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 shutdown_connections 2023-10-06T12:34:20.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:20.819+0000 7f60354e3700 1 -- 172.21.15.46:0/2091467424 wait complete. 2023-10-06T12:34:20.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.007+0000 7f54a1185700 1 Processor -- start 2023-10-06T12:34:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.007+0000 7f54a1185700 1 -- start start 2023-10-06T12:34:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f54a1185700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f549c15ab60 0x7f549c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f54a1185700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c06d460 con 0x7f549c0ad6c0 2023-10-06T12:34:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f54a1185700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0c7000 con 0x7f549c15af30 2023-10-06T12:34:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f54a1185700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0c72c0 con 0x7f549c15ab60 2023-10-06T12:34:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549ad9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f549c15ab60 0x7f549c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549b59e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f549c0ad6c0 0x7f549c0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50166/0 (socket says 172.21.15.46:50166) 2023-10-06T12:34:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549b59e700 1 -- 172.21.15.46:0/3217493090 learned_addr learned my addr 172.21.15.46:0/3217493090 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294013185 0 0) 0x7f549c06d460 con 0x7f549c0ad6c0 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5488003570 con 0x7f549c0ad6c0 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549ad9d700 1 -- 172.21.15.46:0/3217493090 >> v1:172.21.15.132:6789/0 conn(0x7f549c15af30 legacy=0x7f549c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531556032 0 0) 0x7f549c0c7000 con 0x7f549c15af30 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549ad9d700 1 -- 172.21.15.46:0/3217493090 >> v1:172.21.15.46:6789/0 conn(0x7f549c0ad6c0 legacy=0x7f549c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549ad9d700 1 -- 172.21.15.46:0/3217493090 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0c75a0 con 0x7f549c15ab60 2023-10-06T12:34:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549ad9d700 1 --2- 172.21.15.46:0/3217493090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c06e8a0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f549c0c71f0 tx=0x7f548c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8dddc1a37fd3d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c011630 con 0x7f549c15ab60 2023-10-06T12:34:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c011790 con 0x7f549c15ab60 2023-10-06T12:34:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.011+0000 7f549a59c700 1 -- 172.21.15.46:0/3217493090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c011b50 con 0x7f549c15ab60 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- 172.21.15.46:0/3217493090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 msgr2=0x7f549c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 --2- 172.21.15.46:0/3217493090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c06e8a0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f549c0c71f0 tx=0x7f548c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- 172.21.15.46:0/3217493090 shutdown_connections 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 --2- 172.21.15.46:0/3217493090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c06e8a0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- 172.21.15.46:0/3217493090 >> 172.21.15.46:0/3217493090 conn(0x7f549c0cbe30 msgr2=0x7f549c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:21.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- 172.21.15.46:0/3217493090 shutdown_connections 2023-10-06T12:34:21.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- 172.21.15.46:0/3217493090 wait complete. 2023-10-06T12:34:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 Processor -- start 2023-10-06T12:34:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.015+0000 7f54a1185700 1 -- start start 2023-10-06T12:34:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c0ad6c0 0x7f549c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f549c15af30 0x7f549c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c14f470 con 0x7f549c15ab60 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c16de50 con 0x7f549c0ad6c0 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c16cf60 con 0x7f549c15af30 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58774/0 (socket says 172.21.15.46:58774) 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 -- 172.21.15.46:0/1327326355 learned_addr learned my addr 172.21.15.46:0/1327326355 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f549ad9d700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c0ad6c0 0x7f549c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f549b59e700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f549c15af30 0x7f549c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 -- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f549c15af30 msgr2=0x7f549c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f549c15af30 0x7f549c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 -- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c0ad6c0 msgr2=0x7f549c14ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c0ad6c0 0x7f549c14ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 -- 172.21.15.46:0/1327326355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0093c0 con 0x7f549c15ab60 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5493fff700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f548400c7e0 tx=0x7f5484007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484012700 con 0x7f549c15ab60 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484012880 con 0x7f549c15ab60 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c16d240 con 0x7f549c15ab60 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484012c40 con 0x7f549c15ab60 2023-10-06T12:34:21.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.019+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c16d800 con 0x7f549c15ab60 2023-10-06T12:34:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.023+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c063f90 con 0x7f549c15ab60 2023-10-06T12:34:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.023+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f548401b400 con 0x7f549c15ab60 2023-10-06T12:34:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.027+0000 7f5492ffd700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 0x7f547c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.027+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f548407c8f0 con 0x7f549c15ab60 2023-10-06T12:34:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.027+0000 7f549ad9d700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 0x7f547c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.027+0000 7f549ad9d700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 0x7f547c062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c016000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.031+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f548404c9d0 con 0x7f549c15ab60 2023-10-06T12:34:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.207+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f549c04b5c0 con 0x7f549c15ab60 2023-10-06T12:34:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.207+0000 7f5492ffd700 1 -- 172.21.15.46:0/1327326355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+17680 (secure 0 0 0) 0x7f548404fff0 con 0x7f549c15ab60 2023-10-06T12:34:21.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:21.213 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":165,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:34:19.846189+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 msgr2=0x7f547c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 0x7f547c062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c016000 comp rx=0 tx=0).stop 2023-10-06T12:34:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 msgr2=0x7f549c0b1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f548400c7e0 tx=0x7f5484007df0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 shutdown_connections 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f547c060670 0x7f547c062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f549c15af30 0x7f549c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c15ab60 0x7f549c0b1140 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 --2- 172.21.15.46:0/1327326355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f549c0ad6c0 0x7f549c14ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 >> 172.21.15.46:0/1327326355 conn(0x7f549c0cbe30 msgr2=0x7f549c0adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 shutdown_connections 2023-10-06T12:34:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.211+0000 7f54a1185700 1 -- 172.21.15.46:0/1327326355 wait complete. 2023-10-06T12:34:21.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T12:34:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 Processor -- start 2023-10-06T12:34:21.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 -- start start 2023-10-06T12:34:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f588c0c6470 0x7f588c0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f588c06e8a0 0x7f588c0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0c1bf0 con 0x7f588c06ed10 2023-10-06T12:34:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0abbe0 con 0x7f588c06e8a0 2023-10-06T12:34:21.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.395+0000 7f5891d45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c06cf80 con 0x7f588c0c6470 2023-10-06T12:34:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f588c06e8a0 0x7f588c0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f588c06e8a0 0x7f588c0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58788/0 (socket says 172.21.15.46:58788) 2023-10-06T12:34:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 -- 172.21.15.46:0/1918289342 learned_addr learned my addr 172.21.15.46:0/1918289342 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588bfff700 1 --1- 172.21.15.46:0/1918289342 >> v1:172.21.15.46:6789/0 conn(0x7f588c06ed10 0x7f588c0c2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50172/0 (socket says 172.21.15.46:50172) 2023-10-06T12:34:21.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588b7fe700 1 --2- 172.21.15.46:0/1918289342 >> v2:172.21.15.132:3300/0 conn(0x7f588c0c6470 0x7f588c0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 -- 172.21.15.46:0/1918289342 >> v2:172.21.15.132:3300/0 conn(0x7f588c0c6470 msgr2=0x7f588c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 --2- 172.21.15.46:0/1918289342 >> v2:172.21.15.132:3300/0 conn(0x7f588c0c6470 0x7f588c0ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 -- 172.21.15.46:0/1918289342 >> v1:172.21.15.46:6789/0 conn(0x7f588c06ed10 legacy=0x7f588c0c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 -- 172.21.15.46:0/1918289342 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c06d260 con 0x7f588c06e8a0 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588affd700 1 -- 172.21.15.46:0/1918289342 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078253891 0 0) 0x7f588c0c1bf0 con 0x7f588c06ed10 2023-10-06T12:34:21.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5882ffd700 1 --2- 172.21.15.46:0/1918289342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c0c6840 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f588c0c2e90 tx=0x7f5874009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7877cd49a0aff935 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588affd700 1 -- 172.21.15.46:0/1918289342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874009e70 con 0x7f588c06e8a0 2023-10-06T12:34:21.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588affd700 1 -- 172.21.15.46:0/1918289342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587400b860 con 0x7f588c06e8a0 2023-10-06T12:34:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f588affd700 1 -- 172.21.15.46:0/1918289342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400bc00 con 0x7f588c06e8a0 2023-10-06T12:34:21.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5891d45700 1 -- 172.21.15.46:0/1918289342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 msgr2=0x7f588c0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.399+0000 7f5891d45700 1 --2- 172.21.15.46:0/1918289342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c0c6840 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f588c0c2e90 tx=0x7f5874009250 comp rx=0 tx=0).stop 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 -- 172.21.15.46:0/1918289342 shutdown_connections 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 --2- 172.21.15.46:0/1918289342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c0c6840 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 --2- 172.21.15.46:0/1918289342 >> v2:172.21.15.132:3300/0 conn(0x7f588c0c6470 0x7f588c0ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 -- 172.21.15.46:0/1918289342 >> 172.21.15.46:0/1918289342 conn(0x7f588c0cbe30 msgr2=0x7f588c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 -- 172.21.15.46:0/1918289342 shutdown_connections 2023-10-06T12:34:21.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 -- 172.21.15.46:0/1918289342 wait complete. 2023-10-06T12:34:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 Processor -- start 2023-10-06T12:34:21.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.403+0000 7f5891d45700 1 -- start start 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f588c06ed10 0x7f588c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f588c0c6470 0x7f588c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0aaa40 con 0x7f588c06e8a0 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1553e0 con 0x7f588c0c6470 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c154810 con 0x7f588c06ed10 2023-10-06T12:34:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58790/0 (socket says 172.21.15.46:58790) 2023-10-06T12:34:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 -- 172.21.15.46:0/1651747926 learned_addr learned my addr 172.21.15.46:0/1651747926 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588bfff700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f588c0c6470 0x7f588c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5882ffd700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f588c06ed10 0x7f588c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 -- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f588c06ed10 msgr2=0x7f588c16a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f588c06ed10 0x7f588c16a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 -- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f588c0c6470 msgr2=0x7f588c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f588c0c6470 0x7f588c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 -- 172.21.15.46:0/1651747926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740094f0 con 0x7f588c06e8a0 2023-10-06T12:34:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f588b7fe700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ecb0 tx=0x7f587c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00c9d0 con 0x7f588c06e8a0 2023-10-06T12:34:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00cb30 con 0x7f588c06e8a0 2023-10-06T12:34:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c0cb770 con 0x7f588c06e8a0 2023-10-06T12:34:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.407+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00ced0 con 0x7f588c06e8a0 2023-10-06T12:34:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.411+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c154d90 con 0x7f588c06e8a0 2023-10-06T12:34:21.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.411+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c0660f0 con 0x7f588c06e8a0 2023-10-06T12:34:21.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.411+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f587c013680 con 0x7f588c06e8a0 2023-10-06T12:34:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.411+0000 7f5883fff700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 0x7f5870062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.415+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f587c07d0d0 con 0x7f588c06e8a0 2023-10-06T12:34:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.415+0000 7f5882ffd700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 0x7f5870062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.415+0000 7f5882ffd700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 0x7f5870062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f588c0c2e90 tx=0x7f5874017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.419+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f587c04d1b0 con 0x7f588c06e8a0 2023-10-06T12:34:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:21.627+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f588c04b5c0 con 0x7f588c06e8a0 2023-10-06T12:34:22.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.455+0000 7f5883fff700 1 -- 172.21.15.46:0/1651747926 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7f587c0507d0 con 0x7f588c06e8a0 2023-10-06T12:34:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 msgr2=0x7f5870062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 0x7f5870062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f588c0c2e90 tx=0x7f5874017000 comp rx=0 tx=0).stop 2023-10-06T12:34:22.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 msgr2=0x7f588c155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ecb0 tx=0x7f587c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 shutdown_connections 2023-10-06T12:34:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5870060670 0x7f5870062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f588c0c6470 0x7f588c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f588c06ed10 0x7f588c16a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 --2- 172.21.15.46:0/1651747926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f588c06e8a0 0x7f588c155870 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 >> 172.21.15.46:0/1651747926 conn(0x7f588c0cbe30 msgr2=0x7f588c069400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 shutdown_connections 2023-10-06T12:34:22.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.459+0000 7f5891d45700 1 -- 172.21.15.46:0/1651747926 wait complete. 2023-10-06T12:34:22.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T12:34:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.663+0000 7f093886d700 1 Processor -- start 2023-10-06T12:34:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.663+0000 7f093886d700 1 -- start start 2023-10-06T12:34:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093886d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0934069270 0x7f0934069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093886d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0934158aa0 0x7f0934158f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093886d700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340ce6e0 con 0x7f0934159850 2023-10-06T12:34:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093886d700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093415a5c0 con 0x7f0934158aa0 2023-10-06T12:34:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093886d700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093415a3b0 con 0x7f0934069270 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0934158aa0 0x7f0934158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0934158aa0 0x7f0934158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58796/0 (socket says 172.21.15.46:58796) 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 -- 172.21.15.46:0/3020569115 learned_addr learned my addr 172.21.15.46:0/3020569115 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093359e700 1 --1- 172.21.15.46:0/3020569115 >> v1:172.21.15.46:6789/0 conn(0x7f0934159850 0x7f0934159ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50176/0 (socket says 172.21.15.46:50176) 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f0932d9d700 1 --2- 172.21.15.46:0/3020569115 >> v2:172.21.15.132:3300/0 conn(0x7f0934069270 0x7f0934069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 -- 172.21.15.46:0/3020569115 >> v2:172.21.15.132:3300/0 conn(0x7f0934069270 msgr2=0x7f0934069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 --2- 172.21.15.46:0/3020569115 >> v2:172.21.15.132:3300/0 conn(0x7f0934069270 0x7f0934069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 -- 172.21.15.46:0/3020569115 >> v1:172.21.15.46:6789/0 conn(0x7f0934159850 legacy=0x7f0934159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 -- 172.21.15.46:0/3020569115 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934159480 con 0x7f0934158aa0 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f093259c700 1 --2- 172.21.15.46:0/3020569115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934158aa0 0x7f0934158f40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0934088010 tx=0x7f0928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9d16ac660638d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.667+0000 7f0931d9b700 1 -- 172.21.15.46:0/3020569115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800b860 con 0x7f0934158aa0 2023-10-06T12:34:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f0931d9b700 1 -- 172.21.15.46:0/3020569115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092800b9c0 con 0x7f0934158aa0 2023-10-06T12:34:22.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f0931d9b700 1 -- 172.21.15.46:0/3020569115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800bd60 con 0x7f0934158aa0 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 -- 172.21.15.46:0/3020569115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934158aa0 msgr2=0x7f0934158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 --2- 172.21.15.46:0/3020569115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934158aa0 0x7f0934158f40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0934088010 tx=0x7f0928009250 comp rx=0 tx=0).stop 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 -- 172.21.15.46:0/3020569115 shutdown_connections 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 --2- 172.21.15.46:0/3020569115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934158aa0 0x7f0934158f40 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 --2- 172.21.15.46:0/3020569115 >> v2:172.21.15.132:3300/0 conn(0x7f0934069270 0x7f0934069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 -- 172.21.15.46:0/3020569115 >> 172.21.15.46:0/3020569115 conn(0x7f0934068390 msgr2=0x7f09340d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 -- 172.21.15.46:0/3020569115 shutdown_connections 2023-10-06T12:34:22.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.671+0000 7f093886d700 1 -- 172.21.15.46:0/3020569115 wait complete. 2023-10-06T12:34:22.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 Processor -- start 2023-10-06T12:34:22.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 -- start start 2023-10-06T12:34:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0934158aa0 0x7f0934154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0934159850 0x7f09340b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c94e0 con 0x7f0934069270 2023-10-06T12:34:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c88d0 con 0x7f0934159850 2023-10-06T12:34:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093886d700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c8bb0 con 0x7f0934158aa0 2023-10-06T12:34:22.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58800/0 (socket says 172.21.15.46:58800) 2023-10-06T12:34:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 -- 172.21.15.46:0/2806973302 learned_addr learned my addr 172.21.15.46:0/2806973302 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093359e700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0934159850 0x7f09340b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f093259c700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0934158aa0 0x7f0934154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 -- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0934158aa0 msgr2=0x7f0934154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0934158aa0 0x7f0934154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 -- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0934159850 msgr2=0x7f09340b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0934159850 0x7f09340b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.675+0000 7f0932d9d700 1 -- 172.21.15.46:0/2806973302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280094f0 con 0x7f0934069270 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f0932d9d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f092400d890 tx=0x7f0924008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240125f0 con 0x7f0934069270 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924012750 con 0x7f0934069270 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340c8e90 con 0x7f0934069270 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924012af0 con 0x7f0934069270 2023-10-06T12:34:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340b39b0 con 0x7f0934069270 2023-10-06T12:34:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.679+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340b3ba0 con 0x7f0934069270 2023-10-06T12:34:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.683+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f092401e460 con 0x7f0934069270 2023-10-06T12:34:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.683+0000 7f091affd700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 0x7f0920062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.683+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f092407d2a0 con 0x7f0934069270 2023-10-06T12:34:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.683+0000 7f093259c700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 0x7f0920062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.687+0000 7f093259c700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 0x7f0920062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0934088010 tx=0x7f0928017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.691+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f09340b3ba0 con 0x7f0934069270 2023-10-06T12:34:22.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:22.903+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f09340b3ba0 con 0x7f0934069270 2023-10-06T12:34:23.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.467+0000 7f091affd700 1 -- 172.21.15.46:0/2806973302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09340b3ba0 con 0x7f0934069270 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.471+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 msgr2=0x7f0920062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.471+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 0x7f0920062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0934088010 tx=0x7f0928017000 comp rx=0 tx=0).stop 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.471+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 msgr2=0x7f09340c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.471+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f092400d890 tx=0x7f0924008040 comp rx=0 tx=0).stop 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 shutdown_connections 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f0920060670 0x7f0920062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0934159850 0x7f09340b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0934158aa0 0x7f0934154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 --2- 172.21.15.46:0/2806973302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934069270 0x7f09340c7f60 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 >> 172.21.15.46:0/2806973302 conn(0x7f0934068390 msgr2=0x7f0934155db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 shutdown_connections 2023-10-06T12:34:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.475+0000 7f093886d700 1 -- 172.21.15.46:0/2806973302 wait complete. 2023-10-06T12:34:23.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.663+0000 7fd0f2ae3700 1 Processor -- start 2023-10-06T12:34:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.663+0000 7fd0f2ae3700 1 -- start start 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f2ae3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd0ec067760 0x7fd0ec0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f2ae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f2ae3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ec0c8480 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f2ae3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ec069660 con 0x7fd0ec067760 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f2ae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec068ac0 con 0x7fd0ec0bf1a0 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f1ae1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd0ec0bedd0 0x7fd0ec067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60996/0 (socket says 172.21.15.46:60996) 2023-10-06T12:34:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f1ae1700 1 -- 172.21.15.46:0/1026248400 learned_addr learned my addr 172.21.15.46:0/1026248400 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1026248400 >> v2:172.21.15.132:3300/0 conn(0x7fd0ec067760 0x7fd0ec0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f22e2700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f0adf700 1 -- 172.21.15.46:0/1026248400 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442398468 0 0) 0x7fd0e8001490 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f0adf700 1 -- 172.21.15.46:0/1026248400 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d8003570 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec0bf1a0 msgr2=0x7fd0ec0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1026248400 >> v1:172.21.15.46:6789/0 conn(0x7fd0ec0bedd0 legacy=0x7fd0ec067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1026248400 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec068d30 con 0x7fd0ec067760 2023-10-06T12:34:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.667+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec067760 0x7fd0ec0c7a00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098e0 tx=0x7fd0dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde561fdc5761d66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f0adf700 1 -- 172.21.15.46:0/1026248400 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc011630 con 0x7fd0ec067760 2023-10-06T12:34:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f0adf700 1 -- 172.21.15.46:0/1026248400 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc011790 con 0x7fd0ec067760 2023-10-06T12:34:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f0adf700 1 -- 172.21.15.46:0/1026248400 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc011b30 con 0x7fd0ec067760 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec067760 msgr2=0x7fd0ec0c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec067760 0x7fd0ec0c7a00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098e0 tx=0x7fd0dc0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1026248400 shutdown_connections 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1026248400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec067760 0x7fd0ec0c7a00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1026248400 >> 172.21.15.46:0/1026248400 conn(0x7fd0ec062ef0 msgr2=0x7fd0ec0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1026248400 shutdown_connections 2023-10-06T12:34:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.671+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1026248400 wait complete. 2023-10-06T12:34:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 Processor -- start 2023-10-06T12:34:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 -- start start 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec067760 0x7fd0ec0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec15dc90 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec1714b0 con 0x7fd0ec0bf1a0 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f2ae3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec171790 con 0x7fd0ec067760 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50182/0 (socket says 172.21.15.46:50182) 2023-10-06T12:34:23.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1288613681 learned_addr learned my addr 172.21.15.46:0/1288613681 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f1ae1700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec067760 0x7fd0ec0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f22e2700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec067760 msgr2=0x7fd0ec0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec067760 0x7fd0ec0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 msgr2=0x7fd0ec0bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 -- 172.21.15.46:0/1288613681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009420 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f22e2700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.675+0000 7fd0f12e0700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec06ab70 tx=0x7fd0dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:23.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc011b10 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc0253f0 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0257b0 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec171a70 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec0c7430 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.679+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ec15de00 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.683+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd0dc025ba0 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.683+0000 7fd0e1ffb700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 0x7fd0d0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:23.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.683+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fd0dc07e4e0 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.683+0000 7fd0f1ae1700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 0x7fd0d0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.683+0000 7fd0f1ae1700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 0x7fd0d0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80015f0 tx=0x7fd0e8009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.691+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd0ec15de00 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.903+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0ec15da30 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.903+0000 7fd0e1ffb700 1 -- 172.21.15.46:0/1288613681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fd0dc04e5c0 con 0x7fd0ec0bedd0 2023-10-06T12:34:23.909 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:23.909 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:34:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 msgr2=0x7fd0d0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 0x7fd0d0062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80015f0 tx=0x7fd0e8009040 comp rx=0 tx=0).stop 2023-10-06T12:34:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 msgr2=0x7fd0ec0c0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec06ab70 tx=0x7fd0dc017000 comp rx=0 tx=0).stop 2023-10-06T12:34:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 shutdown_connections 2023-10-06T12:34:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd0d0060670 0x7fd0d0062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0ec0bf1a0 0x7fd0ec0bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0ec0bedd0 0x7fd0ec0c0850 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 --2- 172.21.15.46:0/1288613681 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0ec067760 0x7fd0ec0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 >> 172.21.15.46:0/1288613681 conn(0x7fd0ec062ef0 msgr2=0x7fd0ec0cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 shutdown_connections 2023-10-06T12:34:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:23.907+0000 7fd0f2ae3700 1 -- 172.21.15.46:0/1288613681 wait complete. 2023-10-06T12:34:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:34:23.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:34:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 Processor -- start 2023-10-06T12:34:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 -- start start 2023-10-06T12:34:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400aaf30 0x7f3340067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340067590 0x7f3340068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400679f0 con 0x7f3340156e20 2023-10-06T12:34:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400acd60 con 0x7f33400aaf30 2023-10-06T12:34:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.095+0000 7f333f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ad040 con 0x7f3340067590 2023-10-06T12:34:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400aaf30 0x7f3340067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400aaf30 0x7f3340067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50186/0 (socket says 172.21.15.46:50186) 2023-10-06T12:34:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 -- 172.21.15.46:0/4025947964 learned_addr learned my addr 172.21.15.46:0/4025947964 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333e59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3340156e20 0x7f334006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32770/0 (socket says 172.21.15.46:32770) 2023-10-06T12:34:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333ed9d700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340067590 0x7f3340068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 -- 172.21.15.46:0/4025947964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340067590 msgr2=0x7f3340068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340067590 0x7f3340068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 -- 172.21.15.46:0/4025947964 >> v1:172.21.15.46:6789/0 conn(0x7f3340156e20 legacy=0x7f334006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 -- 172.21.15.46:0/4025947964 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400ad320 con 0x7f33400aaf30 2023-10-06T12:34:24.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f3335d9b700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340067050 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f33280099b0 tx=0x7f3328009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73567f949a4b9cfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333dd9b700 1 -- 172.21.15.46:0/4025947964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328009ea0 con 0x7f33400aaf30 2023-10-06T12:34:24.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333dd9b700 1 -- 172.21.15.46:0/4025947964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800b860 con 0x7f33400aaf30 2023-10-06T12:34:24.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333dd9b700 1 -- 172.21.15.46:0/4025947964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800bc00 con 0x7f33400aaf30 2023-10-06T12:34:24.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333f59e700 1 -- 172.21.15.46:0/4025947964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 msgr2=0x7f3340067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.099+0000 7f333f59e700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340067050 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f33280099b0 tx=0x7f3328009250 comp rx=0 tx=0).stop 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 -- 172.21.15.46:0/4025947964 shutdown_connections 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340067590 0x7f3340068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 --2- 172.21.15.46:0/4025947964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340067050 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 -- 172.21.15.46:0/4025947964 >> 172.21.15.46:0/4025947964 conn(0x7f3340062ef0 msgr2=0x7f33400cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 -- 172.21.15.46:0/4025947964 shutdown_connections 2023-10-06T12:34:24.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.103+0000 7f333f59e700 1 -- 172.21.15.46:0/4025947964 wait complete. 2023-10-06T12:34:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 Processor -- start 2023-10-06T12:34:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 -- start start 2023-10-06T12:34:24.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3340067590 0x7f334016a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340156e20 0x7f33400cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33401546d0 con 0x7f33400aaf30 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33401590d0 con 0x7f3340067590 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400cb0b0 con 0x7f3340156e20 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50202/0 (socket says 172.21.15.46:50202) 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 -- 172.21.15.46:0/1319324530 learned_addr learned my addr 172.21.15.46:0/1319324530 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333e59c700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3340067590 0x7f334016a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333ed9d700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340156e20 0x7f33400cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 -- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340156e20 msgr2=0x7f33400cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340156e20 0x7f33400cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 -- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3340067590 msgr2=0x7f334016a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3340067590 0x7f334016a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 -- 172.21.15.46:0/1319324530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33280094f0 con 0x7f33400aaf30 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3335d9b700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3340063770 tx=0x7f3328017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328000d10 con 0x7f33400aaf30 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800bbe0 con 0x7f33400aaf30 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400cb350 con 0x7f33400aaf30 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.107+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328011670 con 0x7f33400aaf30 2023-10-06T12:34:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.111+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3340158d60 con 0x7f33400aaf30 2023-10-06T12:34:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.111+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3340159260 con 0x7f33400aaf30 2023-10-06T12:34:24.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.111+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3328011aa0 con 0x7f33400aaf30 2023-10-06T12:34:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.115+0000 7f3336ffd700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.115+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f332807e5b0 con 0x7f33400aaf30 2023-10-06T12:34:24.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.115+0000 7f333e59c700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.115+0000 7f333e59c700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 0x7f3320062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3330007fd0 tx=0x7f3330009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.119+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f332804e690 con 0x7f33400aaf30 2023-10-06T12:34:24.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.291+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3340064590 con 0x7f33400aaf30 2023-10-06T12:34:24.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.295+0000 7f3336ffd700 1 -- 172.21.15.46:0/1319324530 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+17667 (secure 0 0 0) 0x7f3328051cb0 con 0x7f33400aaf30 2023-10-06T12:34:24.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:24.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":166,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:34:21.401645+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:34:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 msgr2=0x7f3320062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 0x7f3320062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3330007fd0 tx=0x7f3330009040 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 msgr2=0x7f3340154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3340063770 tx=0x7f3328017000 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 shutdown_connections 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3340156e20 0x7f33400cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400aaf30 0x7f3340154190 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 --2- 172.21.15.46:0/1319324530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3340067590 0x7f334016a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 >> 172.21.15.46:0/1319324530 conn(0x7f3340062ef0 msgr2=0x7f3340155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 shutdown_connections 2023-10-06T12:34:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.299+0000 7f333f59e700 1 -- 172.21.15.46:0/1319324530 wait complete. 2023-10-06T12:34:24.324 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T12:34:24.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:34:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.483+0000 7fe666f85700 1 Processor -- start 2023-10-06T12:34:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.483+0000 7fe666f85700 1 -- start start 2023-10-06T12:34:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.483+0000 7fe666f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe66006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.483+0000 7fe666f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6600ad6c0 0x7fe6600adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66006d460 con 0x7fe66015af30 2023-10-06T12:34:24.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6600c7000 con 0x7fe6600ad6c0 2023-10-06T12:34:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600c72c0 con 0x7fe66015ab60 2023-10-06T12:34:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6600ad6c0 0x7fe6600adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe664d21700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe66006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6600ad6c0 0x7fe6600adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54846/0 (socket says 172.21.15.46:54846) 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 -- 172.21.15.46:0/1236074731 learned_addr learned my addr 172.21.15.46:0/1236074731 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 -- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 msgr2=0x7fe66006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 --2- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe66006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 -- 172.21.15.46:0/1236074731 >> v1:172.21.15.132:6789/0 conn(0x7fe66015af30 legacy=0x7fe66006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 -- 172.21.15.46:0/1236074731 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600c75a0 con 0x7fe6600ad6c0 2023-10-06T12:34:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe665522700 1 --2- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600adb20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe658009f20 tx=0x7fe658009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7b14b5a12f9c1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe6577fe700 1 -- 172.21.15.46:0/1236074731 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800e070 con 0x7fe6600ad6c0 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe6577fe700 1 -- 172.21.15.46:0/1236074731 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65800b9d0 con 0x7fe6600ad6c0 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe6577fe700 1 -- 172.21.15.46:0/1236074731 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6580092e0 con 0x7fe6600ad6c0 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 -- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 msgr2=0x7fe6600adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 --2- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600adb20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe658009f20 tx=0x7fe658009840 comp rx=0 tx=0).stop 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 -- 172.21.15.46:0/1236074731 shutdown_connections 2023-10-06T12:34:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 --2- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600adb20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.487+0000 7fe666f85700 1 --2- 172.21.15.46:0/1236074731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe66006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.491+0000 7fe666f85700 1 -- 172.21.15.46:0/1236074731 >> 172.21.15.46:0/1236074731 conn(0x7fe6600cbe30 msgr2=0x7fe6600cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.491+0000 7fe666f85700 1 -- 172.21.15.46:0/1236074731 shutdown_connections 2023-10-06T12:34:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.491+0000 7fe666f85700 1 -- 172.21.15.46:0/1236074731 wait complete. 2023-10-06T12:34:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.491+0000 7fe666f85700 1 Processor -- start 2023-10-06T12:34:24.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.491+0000 7fe666f85700 1 -- start start 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe660150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600c3710 con 0x7fe66015af30 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600c39f0 con 0x7fe6600ad6c0 2023-10-06T12:34:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6600af3c0 con 0x7fe66015ab60 2023-10-06T12:34:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50214/0 (socket says 172.21.15.46:50214) 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 -- 172.21.15.46:0/2282207072 learned_addr learned my addr 172.21.15.46:0/2282207072 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe664d21700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe657fff700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe660150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 -- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 msgr2=0x7fe660150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe660150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 -- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 msgr2=0x7fe6600c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 -- 172.21.15.46:0/2282207072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe658009ae0 con 0x7fe66015af30 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe657fff700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe660150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:24.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe665522700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b610 tx=0x7fe658018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658008c30 con 0x7fe66015af30 2023-10-06T12:34:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe658008d90 con 0x7fe66015af30 2023-10-06T12:34:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6600af6a0 con 0x7fe66015af30 2023-10-06T12:34:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658025690 con 0x7fe66015af30 2023-10-06T12:34:24.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.495+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6600c3420 con 0x7fe66015af30 2023-10-06T12:34:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.499+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6600abb30 con 0x7fe66015af30 2023-10-06T12:34:24.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.499+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe658003040 con 0x7fe66015af30 2023-10-06T12:34:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.503+0000 7fe654ff9700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 0x7fe64c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.503+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fe65807e7b0 con 0x7fe66015af30 2023-10-06T12:34:24.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.503+0000 7fe664d21700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 0x7fe64c062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.503+0000 7fe664d21700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 0x7fe64c062ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009c90 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.507+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe65804e780 con 0x7fe66015af30 2023-10-06T12:34:24.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.671+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe660063f90 con 0x7fe64c0606f0 2023-10-06T12:34:24.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.683+0000 7fe654ff9700 1 -- 172.21.15.46:0/2282207072 <== mgr.8875 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308744 (secure 0 0 0) 0x7fe660063f90 con 0x7fe64c0606f0 2023-10-06T12:34:24.693 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 msgr2=0x7fe64c062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 0x7fe64c062ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009c90 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2023-10-06T12:34:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 msgr2=0x7fe6600a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b610 tx=0x7fe658018000 comp rx=0 tx=0).stop 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 shutdown_connections 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fe64c0606f0 0x7fe64c062ba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe66015af30 0x7fe6600a9840 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe66015ab60 0x7fe660150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 --2- 172.21.15.46:0/2282207072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6600ad6c0 0x7fe6600c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 >> 172.21.15.46:0/2282207072 conn(0x7fe6600cbe30 msgr2=0x7fe6600aaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 shutdown_connections 2023-10-06T12:34:24.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.691+0000 7fe666f85700 1 -- 172.21.15.46:0/2282207072 wait complete. 2023-10-06T12:34:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:34:24.717 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T12:34:23.035887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2469448,"kb_used_data":17416,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752505272,"statfs":{"total":773094113280,"available":770565398528,"internally_reserved":0,"allocated":17833984,"data_stored":9914524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.650578"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.427345+0000","last_change":"2023-10-06T12:34:20.422056+0000","last_active":"2023-10-06T12:34:20.427345+0000","last_peered":"2023-10-06T12:34:20.427345+0000","last_clean":"2023-10-06T12:34:20.427345+0000","last_became_active":"2023-10-06T12:34:20.421765+0000","last_became_peered":"2023-10-06T12:34:20.421765+0000","last_unstale":"2023-10-06T12:34:20.427345+0000","last_undegraded":"2023-10-06T12:34:20.427345+0000","last_fullsized":"2023-10-06T12:34:20.427345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:45.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443722+0000","last_change":"2023-10-06T12:34:20.435452+0000","last_active":"2023-10-06T12:34:20.443722+0000","last_peered":"2023-10-06T12:34:20.443722+0000","last_clean":"2023-10-06T12:34:20.443722+0000","last_became_active":"2023-10-06T12:34:20.435160+0000","last_became_peered":"2023-10-06T12:34:20.435160+0000","last_unstale":"2023-10-06T12:34:20.443722+0000","last_undegraded":"2023-10-06T12:34:20.443722+0000","last_fullsized":"2023-10-06T12:34:20.443722+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:52.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444107+0000","last_change":"2023-10-06T12:34:20.438651+0000","last_active":"2023-10-06T12:34:20.444107+0000","last_peered":"2023-10-06T12:34:20.444107+0000","last_clean":"2023-10-06T12:34:20.444107+0000","last_became_active":"2023-10-06T12:34:20.438365+0000","last_became_peered":"2023-10-06T12:34:20.438365+0000","last_unstale":"2023-10-06T12:34:20.444107+0000","last_undegraded":"2023-10-06T12:34:20.444107+0000","last_fullsized":"2023-10-06T12:34:20.444107+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:17.370357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437313+0000","last_change":"2023-10-06T12:34:20.431779+0000","last_active":"2023-10-06T12:34:20.437313+0000","last_peered":"2023-10-06T12:34:20.437313+0000","last_clean":"2023-10-06T12:34:20.437313+0000","last_became_active":"2023-10-06T12:34:20.431462+0000","last_became_peered":"2023-10-06T12:34:20.431462+0000","last_unstale":"2023-10-06T12:34:20.437313+0000","last_undegraded":"2023-10-06T12:34:20.437313+0000","last_fullsized":"2023-10-06T12:34:20.437313+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.068158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436913+0000","last_change":"2023-10-06T12:34:20.418531+0000","last_active":"2023-10-06T12:34:20.436913+0000","last_peered":"2023-10-06T12:34:20.436913+0000","last_clean":"2023-10-06T12:34:20.436913+0000","last_became_active":"2023-10-06T12:34:20.418187+0000","last_became_peered":"2023-10-06T12:34:20.418187+0000","last_unstale":"2023-10-06T12:34:20.436913+0000","last_undegraded":"2023-10-06T12:34:20.436913+0000","last_fullsized":"2023-10-06T12:34:20.436913+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:50.891730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431717+0000","last_change":"2023-10-06T12:34:20.419115+0000","last_active":"2023-10-06T12:34:20.431717+0000","last_peered":"2023-10-06T12:34:20.431717+0000","last_clean":"2023-10-06T12:34:20.431717+0000","last_became_active":"2023-10-06T12:34:20.418730+0000","last_became_peered":"2023-10-06T12:34:20.418730+0000","last_unstale":"2023-10-06T12:34:20.431717+0000","last_undegraded":"2023-10-06T12:34:20.431717+0000","last_fullsized":"2023-10-06T12:34:20.431717+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:24.221665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.439697+0000","last_change":"2023-10-06T12:34:20.433587+0000","last_active":"2023-10-06T12:34:20.439697+0000","last_peered":"2023-10-06T12:34:20.439697+0000","last_clean":"2023-10-06T12:34:20.439697+0000","last_became_active":"2023-10-06T12:34:20.433207+0000","last_became_peered":"2023-10-06T12:34:20.433207+0000","last_unstale":"2023-10-06T12:34:20.439697+0000","last_undegraded":"2023-10-06T12:34:20.439697+0000","last_fullsized":"2023-10-06T12:34:20.439697+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:41:36.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434616+0000","last_change":"2023-10-06T12:34:20.430936+0000","last_active":"2023-10-06T12:34:20.434616+0000","last_peered":"2023-10-06T12:34:20.434616+0000","last_clean":"2023-10-06T12:34:20.434616+0000","last_became_active":"2023-10-06T12:34:20.430571+0000","last_became_peered":"2023-10-06T12:34:20.430571+0000","last_unstale":"2023-10-06T12:34:20.434616+0000","last_undegraded":"2023-10-06T12:34:20.434616+0000","last_fullsized":"2023-10-06T12:34:20.434616+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:32.700937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441127+0000","last_change":"2023-10-06T12:34:20.436943+0000","last_active":"2023-10-06T12:34:20.441127+0000","last_peered":"2023-10-06T12:34:20.441127+0000","last_clean":"2023-10-06T12:34:20.441127+0000","last_became_active":"2023-10-06T12:34:20.436573+0000","last_became_peered":"2023-10-06T12:34:20.436573+0000","last_unstale":"2023-10-06T12:34:20.441127+0000","last_undegraded":"2023-10-06T12:34:20.441127+0000","last_fullsized":"2023-10-06T12:34:20.441127+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.240723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436663+0000","last_change":"2023-10-06T12:34:20.429878+0000","last_active":"2023-10-06T12:34:20.436663+0000","last_peered":"2023-10-06T12:34:20.436663+0000","last_clean":"2023-10-06T12:34:20.436663+0000","last_became_active":"2023-10-06T12:34:20.429354+0000","last_became_peered":"2023-10-06T12:34:20.429354+0000","last_unstale":"2023-10-06T12:34:20.436663+0000","last_undegraded":"2023-10-06T12:34:20.436663+0000","last_fullsized":"2023-10-06T12:34:20.436663+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.484389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.445039+0000","last_change":"2023-10-06T12:34:20.442426+0000","last_active":"2023-10-06T12:34:20.445039+0000","last_peered":"2023-10-06T12:34:20.445039+0000","last_clean":"2023-10-06T12:34:20.445039+0000","last_became_active":"2023-10-06T12:34:20.442104+0000","last_became_peered":"2023-10-06T12:34:20.442104+0000","last_unstale":"2023-10-06T12:34:20.445039+0000","last_undegraded":"2023-10-06T12:34:20.445039+0000","last_fullsized":"2023-10-06T12:34:20.445039+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:39.215054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440808+0000","last_change":"2023-10-06T12:34:20.434972+0000","last_active":"2023-10-06T12:34:20.440808+0000","last_peered":"2023-10-06T12:34:20.440808+0000","last_clean":"2023-10-06T12:34:20.440808+0000","last_became_active":"2023-10-06T12:34:20.434516+0000","last_became_peered":"2023-10-06T12:34:20.434516+0000","last_unstale":"2023-10-06T12:34:20.440808+0000","last_undegraded":"2023-10-06T12:34:20.440808+0000","last_fullsized":"2023-10-06T12:34:20.440808+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:30.094495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436944+0000","last_change":"2023-10-06T12:34:20.427214+0000","last_active":"2023-10-06T12:34:20.436944+0000","last_peered":"2023-10-06T12:34:20.436944+0000","last_clean":"2023-10-06T12:34:20.436944+0000","last_became_active":"2023-10-06T12:34:20.426867+0000","last_became_peered":"2023-10-06T12:34:20.426867+0000","last_unstale":"2023-10-06T12:34:20.436944+0000","last_undegraded":"2023-10-06T12:34:20.436944+0000","last_fullsized":"2023-10-06T12:34:20.436944+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:48.740650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444983+0000","last_change":"2023-10-06T12:34:20.438617+0000","last_active":"2023-10-06T12:34:20.444983+0000","last_peered":"2023-10-06T12:34:20.444983+0000","last_clean":"2023-10-06T12:34:20.444983+0000","last_became_active":"2023-10-06T12:34:20.438287+0000","last_became_peered":"2023-10-06T12:34:20.438287+0000","last_unstale":"2023-10-06T12:34:20.444983+0000","last_undegraded":"2023-10-06T12:34:20.444983+0000","last_fullsized":"2023-10-06T12:34:20.444983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.484389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444603+0000","last_change":"2023-10-06T12:34:20.440645+0000","last_active":"2023-10-06T12:34:20.444603+0000","last_peered":"2023-10-06T12:34:20.444603+0000","last_clean":"2023-10-06T12:34:20.444603+0000","last_became_active":"2023-10-06T12:34:20.439744+0000","last_became_peered":"2023-10-06T12:34:20.439744+0000","last_unstale":"2023-10-06T12:34:20.444603+0000","last_undegraded":"2023-10-06T12:34:20.444603+0000","last_fullsized":"2023-10-06T12:34:20.444603+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:29.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436102+0000","last_change":"2023-10-06T12:34:20.424317+0000","last_active":"2023-10-06T12:34:20.436102+0000","last_peered":"2023-10-06T12:34:20.436102+0000","last_clean":"2023-10-06T12:34:20.436102+0000","last_became_active":"2023-10-06T12:34:20.423484+0000","last_became_peered":"2023-10-06T12:34:20.423484+0000","last_unstale":"2023-10-06T12:34:20.436102+0000","last_undegraded":"2023-10-06T12:34:20.436102+0000","last_fullsized":"2023-10-06T12:34:20.436102+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:01.509365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437345+0000","last_change":"2023-10-06T12:34:20.430396+0000","last_active":"2023-10-06T12:34:20.437345+0000","last_peered":"2023-10-06T12:34:20.437345+0000","last_clean":"2023-10-06T12:34:20.437345+0000","last_became_active":"2023-10-06T12:34:20.429852+0000","last_became_peered":"2023-10-06T12:34:20.429852+0000","last_unstale":"2023-10-06T12:34:20.437345+0000","last_undegraded":"2023-10-06T12:34:20.437345+0000","last_fullsized":"2023-10-06T12:34:20.437345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:12.391470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441660+0000","last_change":"2023-10-06T12:34:20.431126+0000","last_active":"2023-10-06T12:34:20.441660+0000","last_peered":"2023-10-06T12:34:20.441660+0000","last_clean":"2023-10-06T12:34:20.441660+0000","last_became_active":"2023-10-06T12:34:20.430524+0000","last_became_peered":"2023-10-06T12:34:20.430524+0000","last_unstale":"2023-10-06T12:34:20.441660+0000","last_undegraded":"2023-10-06T12:34:20.441660+0000","last_fullsized":"2023-10-06T12:34:20.441660+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:46.328245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449382+0000","last_change":"2023-10-06T12:34:20.442182+0000","last_active":"2023-10-06T12:34:20.449382+0000","last_peered":"2023-10-06T12:34:20.449382+0000","last_clean":"2023-10-06T12:34:20.449382+0000","last_became_active":"2023-10-06T12:34:20.437829+0000","last_became_peered":"2023-10-06T12:34:20.437829+0000","last_unstale":"2023-10-06T12:34:20.449382+0000","last_undegraded":"2023-10-06T12:34:20.449382+0000","last_fullsized":"2023-10-06T12:34:20.449382+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:15:13.185350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440411+0000","last_change":"2023-10-06T12:34:20.438559+0000","last_active":"2023-10-06T12:34:20.440411+0000","last_peered":"2023-10-06T12:34:20.440411+0000","last_clean":"2023-10-06T12:34:20.440411+0000","last_became_active":"2023-10-06T12:34:20.438180+0000","last_became_peered":"2023-10-06T12:34:20.438180+0000","last_unstale":"2023-10-06T12:34:20.440411+0000","last_undegraded":"2023-10-06T12:34:20.440411+0000","last_fullsized":"2023-10-06T12:34:20.440411+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:51:51.690525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443808+0000","last_change":"2023-10-06T12:34:20.437067+0000","last_active":"2023-10-06T12:34:20.443808+0000","last_peered":"2023-10-06T12:34:20.443808+0000","last_clean":"2023-10-06T12:34:20.443808+0000","last_became_active":"2023-10-06T12:34:20.436702+0000","last_became_peered":"2023-10-06T12:34:20.436702+0000","last_unstale":"2023-10-06T12:34:20.443808+0000","last_undegraded":"2023-10-06T12:34:20.443808+0000","last_fullsized":"2023-10-06T12:34:20.443808+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:50.536480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.439825+0000","last_change":"2023-10-06T12:34:20.433593+0000","last_active":"2023-10-06T12:34:20.439825+0000","last_peered":"2023-10-06T12:34:20.439825+0000","last_clean":"2023-10-06T12:34:20.439825+0000","last_became_active":"2023-10-06T12:34:20.433284+0000","last_became_peered":"2023-10-06T12:34:20.433284+0000","last_unstale":"2023-10-06T12:34:20.439825+0000","last_undegraded":"2023-10-06T12:34:20.439825+0000","last_fullsized":"2023-10-06T12:34:20.439825+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:21:39.297865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.422008+0000","last_change":"2023-10-06T12:34:20.420936+0000","last_active":"2023-10-06T12:34:20.422008+0000","last_peered":"2023-10-06T12:34:20.422008+0000","last_clean":"2023-10-06T12:34:20.422008+0000","last_became_active":"2023-10-06T12:34:20.420612+0000","last_became_peered":"2023-10-06T12:34:20.420612+0000","last_unstale":"2023-10-06T12:34:20.422008+0000","last_undegraded":"2023-10-06T12:34:20.422008+0000","last_fullsized":"2023-10-06T12:34:20.422008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:24.221665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443253+0000","last_change":"2023-10-06T12:34:20.438071+0000","last_active":"2023-10-06T12:34:20.443253+0000","last_peered":"2023-10-06T12:34:20.443253+0000","last_clean":"2023-10-06T12:34:20.443253+0000","last_became_active":"2023-10-06T12:34:20.437757+0000","last_became_peered":"2023-10-06T12:34:20.437757+0000","last_unstale":"2023-10-06T12:34:20.443253+0000","last_undegraded":"2023-10-06T12:34:20.443253+0000","last_fullsized":"2023-10-06T12:34:20.443253+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:33.012175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431530+0000","last_change":"2023-10-06T12:34:20.429594+0000","last_active":"2023-10-06T12:34:20.431530+0000","last_peered":"2023-10-06T12:34:20.431530+0000","last_clean":"2023-10-06T12:34:20.431530+0000","last_became_active":"2023-10-06T12:34:20.429178+0000","last_became_peered":"2023-10-06T12:34:20.429178+0000","last_unstale":"2023-10-06T12:34:20.431530+0000","last_undegraded":"2023-10-06T12:34:20.431530+0000","last_fullsized":"2023-10-06T12:34:20.431530+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:38.434455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441058+0000","last_change":"2023-10-06T12:34:20.436485+0000","last_active":"2023-10-06T12:34:20.441058+0000","last_peered":"2023-10-06T12:34:20.441058+0000","last_clean":"2023-10-06T12:34:20.441058+0000","last_became_active":"2023-10-06T12:34:20.436131+0000","last_became_peered":"2023-10-06T12:34:20.436131+0000","last_unstale":"2023-10-06T12:34:20.441058+0000","last_undegraded":"2023-10-06T12:34:20.441058+0000","last_fullsized":"2023-10-06T12:34:20.441058+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:06.542282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440857+0000","last_change":"2023-10-06T12:34:20.437086+0000","last_active":"2023-10-06T12:34:20.440857+0000","last_peered":"2023-10-06T12:34:20.440857+0000","last_clean":"2023-10-06T12:34:20.440857+0000","last_became_active":"2023-10-06T12:34:20.436763+0000","last_became_peered":"2023-10-06T12:34:20.436763+0000","last_unstale":"2023-10-06T12:34:20.440857+0000","last_undegraded":"2023-10-06T12:34:20.440857+0000","last_fullsized":"2023-10-06T12:34:20.440857+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:08.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430059+0000","last_change":"2023-10-06T12:34:20.428484+0000","last_active":"2023-10-06T12:34:20.430059+0000","last_peered":"2023-10-06T12:34:20.430059+0000","last_clean":"2023-10-06T12:34:20.430059+0000","last_became_active":"2023-10-06T12:34:20.426905+0000","last_became_peered":"2023-10-06T12:34:20.426905+0000","last_unstale":"2023-10-06T12:34:20.430059+0000","last_undegraded":"2023-10-06T12:34:20.430059+0000","last_fullsized":"2023-10-06T12:34:20.430059+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:52.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431798+0000","last_change":"2023-10-06T12:34:20.428064+0000","last_active":"2023-10-06T12:34:20.431798+0000","last_peered":"2023-10-06T12:34:20.431798+0000","last_clean":"2023-10-06T12:34:20.431798+0000","last_became_active":"2023-10-06T12:34:20.427727+0000","last_became_peered":"2023-10-06T12:34:20.427727+0000","last_unstale":"2023-10-06T12:34:20.431798+0000","last_undegraded":"2023-10-06T12:34:20.431798+0000","last_fullsized":"2023-10-06T12:34:20.431798+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:46.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.435942+0000","last_change":"2023-10-06T12:34:20.429745+0000","last_active":"2023-10-06T12:34:20.435942+0000","last_peered":"2023-10-06T12:34:20.435942+0000","last_clean":"2023-10-06T12:34:20.435942+0000","last_became_active":"2023-10-06T12:34:20.429370+0000","last_became_peered":"2023-10-06T12:34:20.429370+0000","last_unstale":"2023-10-06T12:34:20.435942+0000","last_undegraded":"2023-10-06T12:34:20.435942+0000","last_fullsized":"2023-10-06T12:34:20.435942+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:59.315328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.433006+0000","last_change":"2023-10-06T12:34:20.423174+0000","last_active":"2023-10-06T12:34:20.433006+0000","last_peered":"2023-10-06T12:34:20.433006+0000","last_clean":"2023-10-06T12:34:20.433006+0000","last_became_active":"2023-10-06T12:34:20.422176+0000","last_became_peered":"2023-10-06T12:34:20.422176+0000","last_unstale":"2023-10-06T12:34:20.433006+0000","last_undegraded":"2023-10-06T12:34:20.433006+0000","last_fullsized":"2023-10-06T12:34:20.433006+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:37.281440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434596+0000","last_change":"2023-10-06T12:34:20.430628+0000","last_active":"2023-10-06T12:34:20.434596+0000","last_peered":"2023-10-06T12:34:20.434596+0000","last_clean":"2023-10-06T12:34:20.434596+0000","last_became_active":"2023-10-06T12:34:20.430270+0000","last_became_peered":"2023-10-06T12:34:20.430270+0000","last_unstale":"2023-10-06T12:34:20.434596+0000","last_undegraded":"2023-10-06T12:34:20.434596+0000","last_fullsized":"2023-10-06T12:34:20.434596+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:48.681099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443807+0000","last_change":"2023-10-06T12:34:20.436890+0000","last_active":"2023-10-06T12:34:20.443807+0000","last_peered":"2023-10-06T12:34:20.443807+0000","last_clean":"2023-10-06T12:34:20.443807+0000","last_became_active":"2023-10-06T12:34:20.436521+0000","last_became_peered":"2023-10-06T12:34:20.436521+0000","last_unstale":"2023-10-06T12:34:20.443807+0000","last_undegraded":"2023-10-06T12:34:20.443807+0000","last_fullsized":"2023-10-06T12:34:20.443807+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:55.289183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"165'4","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450452+0000","last_change":"2023-10-06T12:34:19.420577+0000","last_active":"2023-10-06T12:34:20.450452+0000","last_peered":"2023-10-06T12:34:20.450452+0000","last_clean":"2023-10-06T12:34:20.450452+0000","last_became_active":"2023-10-06T12:34:19.420176+0000","last_became_peered":"2023-10-06T12:34:19.420176+0000","last_unstale":"2023-10-06T12:34:20.450452+0000","last_undegraded":"2023-10-06T12:34:20.450452+0000","last_fullsized":"2023-10-06T12:34:20.450452+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:54.285225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444721+0000","last_change":"2023-10-06T12:34:20.441785+0000","last_active":"2023-10-06T12:34:20.444721+0000","last_peered":"2023-10-06T12:34:20.444721+0000","last_clean":"2023-10-06T12:34:20.444721+0000","last_became_active":"2023-10-06T12:34:20.441520+0000","last_became_peered":"2023-10-06T12:34:20.441520+0000","last_unstale":"2023-10-06T12:34:20.444721+0000","last_undegraded":"2023-10-06T12:34:20.444721+0000","last_fullsized":"2023-10-06T12:34:20.444721+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:28.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449223+0000","last_change":"2023-10-06T12:34:19.425747+0000","last_active":"2023-10-06T12:34:20.449223+0000","last_peered":"2023-10-06T12:34:20.449223+0000","last_clean":"2023-10-06T12:34:20.449223+0000","last_became_active":"2023-10-06T12:34:19.425206+0000","last_became_peered":"2023-10-06T12:34:19.425206+0000","last_unstale":"2023-10-06T12:34:20.449223+0000","last_undegraded":"2023-10-06T12:34:20.449223+0000","last_fullsized":"2023-10-06T12:34:20.449223+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:49.532522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.433904+0000","last_change":"2023-10-06T12:34:20.430696+0000","last_active":"2023-10-06T12:34:20.433904+0000","last_peered":"2023-10-06T12:34:20.433904+0000","last_clean":"2023-10-06T12:34:20.433904+0000","last_became_active":"2023-10-06T12:34:20.430263+0000","last_became_peered":"2023-10-06T12:34:20.430263+0000","last_unstale":"2023-10-06T12:34:20.433904+0000","last_undegraded":"2023-10-06T12:34:20.433904+0000","last_fullsized":"2023-10-06T12:34:20.433904+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:23.761652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449585+0000","last_change":"2023-10-06T12:34:19.426251+0000","last_active":"2023-10-06T12:34:20.449585+0000","last_peered":"2023-10-06T12:34:20.449585+0000","last_clean":"2023-10-06T12:34:20.449585+0000","last_became_active":"2023-10-06T12:34:19.425935+0000","last_became_peered":"2023-10-06T12:34:19.425935+0000","last_unstale":"2023-10-06T12:34:20.449585+0000","last_undegraded":"2023-10-06T12:34:20.449585+0000","last_fullsized":"2023-10-06T12:34:20.449585+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:47.736692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442434+0000","last_change":"2023-10-06T12:34:20.433485+0000","last_active":"2023-10-06T12:34:20.442434+0000","last_peered":"2023-10-06T12:34:20.442434+0000","last_clean":"2023-10-06T12:34:20.442434+0000","last_became_active":"2023-10-06T12:34:20.433198+0000","last_became_peered":"2023-10-06T12:34:20.433198+0000","last_unstale":"2023-10-06T12:34:20.442434+0000","last_undegraded":"2023-10-06T12:34:20.442434+0000","last_fullsized":"2023-10-06T12:34:20.442434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:56.657861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448479+0000","last_change":"2023-10-06T12:34:19.429070+0000","last_active":"2023-10-06T12:34:20.448479+0000","last_peered":"2023-10-06T12:34:20.448479+0000","last_clean":"2023-10-06T12:34:20.448479+0000","last_became_active":"2023-10-06T12:34:19.428644+0000","last_became_peered":"2023-10-06T12:34:19.428644+0000","last_unstale":"2023-10-06T12:34:20.448479+0000","last_undegraded":"2023-10-06T12:34:20.448479+0000","last_fullsized":"2023-10-06T12:34:20.448479+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:05.433196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434195+0000","last_change":"2023-10-06T12:34:20.431289+0000","last_active":"2023-10-06T12:34:20.434195+0000","last_peered":"2023-10-06T12:34:20.434195+0000","last_clean":"2023-10-06T12:34:20.434195+0000","last_became_active":"2023-10-06T12:34:20.430906+0000","last_became_peered":"2023-10-06T12:34:20.430906+0000","last_unstale":"2023-10-06T12:34:20.434195+0000","last_undegraded":"2023-10-06T12:34:20.434195+0000","last_fullsized":"2023-10-06T12:34:20.434195+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:30:33.336476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447890+0000","last_change":"2023-10-06T12:34:19.416154+0000","last_active":"2023-10-06T12:34:20.447890+0000","last_peered":"2023-10-06T12:34:20.447890+0000","last_clean":"2023-10-06T12:34:20.447890+0000","last_became_active":"2023-10-06T12:34:19.415730+0000","last_became_peered":"2023-10-06T12:34:19.415730+0000","last_unstale":"2023-10-06T12:34:20.447890+0000","last_undegraded":"2023-10-06T12:34:20.447890+0000","last_fullsized":"2023-10-06T12:34:20.447890+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:32.376167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.446460+0000","last_change":"2023-10-06T12:34:20.442742+0000","last_active":"2023-10-06T12:34:20.446460+0000","last_peered":"2023-10-06T12:34:20.446460+0000","last_clean":"2023-10-06T12:34:20.446460+0000","last_became_active":"2023-10-06T12:34:20.442153+0000","last_became_peered":"2023-10-06T12:34:20.442153+0000","last_unstale":"2023-10-06T12:34:20.446460+0000","last_undegraded":"2023-10-06T12:34:20.446460+0000","last_fullsized":"2023-10-06T12:34:20.446460+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:33.498249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448263+0000","last_change":"2023-10-06T12:34:19.427439+0000","last_active":"2023-10-06T12:34:20.448263+0000","last_peered":"2023-10-06T12:34:20.448263+0000","last_clean":"2023-10-06T12:34:20.448263+0000","last_became_active":"2023-10-06T12:34:19.427021+0000","last_became_peered":"2023-10-06T12:34:19.427021+0000","last_unstale":"2023-10-06T12:34:20.448263+0000","last_undegraded":"2023-10-06T12:34:20.448263+0000","last_fullsized":"2023-10-06T12:34:20.448263+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.366399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444001+0000","last_change":"2023-10-06T12:34:20.439155+0000","last_active":"2023-10-06T12:34:20.444001+0000","last_peered":"2023-10-06T12:34:20.444001+0000","last_clean":"2023-10-06T12:34:20.444001+0000","last_became_active":"2023-10-06T12:34:20.438850+0000","last_became_peered":"2023-10-06T12:34:20.438850+0000","last_unstale":"2023-10-06T12:34:20.444001+0000","last_undegraded":"2023-10-06T12:34:20.444001+0000","last_fullsized":"2023-10-06T12:34:20.444001+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:19.740524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413271+0000","last_change":"2023-10-06T12:34:19.417853+0000","last_active":"2023-10-06T12:34:20.413271+0000","last_peered":"2023-10-06T12:34:20.413271+0000","last_clean":"2023-10-06T12:34:20.413271+0000","last_became_active":"2023-10-06T12:34:19.417499+0000","last_became_peered":"2023-10-06T12:34:19.417499+0000","last_unstale":"2023-10-06T12:34:20.413271+0000","last_undegraded":"2023-10-06T12:34:20.413271+0000","last_fullsized":"2023-10-06T12:34:20.413271+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:04.390083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.426823+0000","last_change":"2023-10-06T12:34:20.422610+0000","last_active":"2023-10-06T12:34:20.426823+0000","last_peered":"2023-10-06T12:34:20.426823+0000","last_clean":"2023-10-06T12:34:20.426823+0000","last_became_active":"2023-10-06T12:34:20.422317+0000","last_became_peered":"2023-10-06T12:34:20.422317+0000","last_unstale":"2023-10-06T12:34:20.426823+0000","last_undegraded":"2023-10-06T12:34:20.426823+0000","last_fullsized":"2023-10-06T12:34:20.426823+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:52.924767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.409416+0000","last_change":"2023-10-06T12:34:19.422008+0000","last_active":"2023-10-06T12:34:20.409416+0000","last_peered":"2023-10-06T12:34:20.409416+0000","last_clean":"2023-10-06T12:34:20.409416+0000","last_became_active":"2023-10-06T12:34:19.421669+0000","last_became_peered":"2023-10-06T12:34:19.421669+0000","last_unstale":"2023-10-06T12:34:20.409416+0000","last_undegraded":"2023-10-06T12:34:20.409416+0000","last_fullsized":"2023-10-06T12:34:20.409416+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:40.633582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449413+0000","last_change":"2023-10-06T12:34:20.442539+0000","last_active":"2023-10-06T12:34:20.449413+0000","last_peered":"2023-10-06T12:34:20.449413+0000","last_clean":"2023-10-06T12:34:20.449413+0000","last_became_active":"2023-10-06T12:34:20.441876+0000","last_became_peered":"2023-10-06T12:34:20.441876+0000","last_unstale":"2023-10-06T12:34:20.449413+0000","last_undegraded":"2023-10-06T12:34:20.449413+0000","last_fullsized":"2023-10-06T12:34:20.449413+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:13.068286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449036+0000","last_change":"2023-10-06T12:34:19.419099+0000","last_active":"2023-10-06T12:34:20.449036+0000","last_peered":"2023-10-06T12:34:20.449036+0000","last_clean":"2023-10-06T12:34:20.449036+0000","last_became_active":"2023-10-06T12:34:19.418742+0000","last_became_peered":"2023-10-06T12:34:19.418742+0000","last_unstale":"2023-10-06T12:34:20.449036+0000","last_undegraded":"2023-10-06T12:34:20.449036+0000","last_fullsized":"2023-10-06T12:34:20.449036+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:35.738952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442180+0000","last_change":"2023-10-06T12:34:20.436475+0000","last_active":"2023-10-06T12:34:20.442180+0000","last_peered":"2023-10-06T12:34:20.442180+0000","last_clean":"2023-10-06T12:34:20.442180+0000","last_became_active":"2023-10-06T12:34:20.433947+0000","last_became_peered":"2023-10-06T12:34:20.433947+0000","last_unstale":"2023-10-06T12:34:20.442180+0000","last_undegraded":"2023-10-06T12:34:20.442180+0000","last_fullsized":"2023-10-06T12:34:20.442180+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:32.700937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450548+0000","last_change":"2023-10-06T12:34:19.421522+0000","last_active":"2023-10-06T12:34:20.450548+0000","last_peered":"2023-10-06T12:34:20.450548+0000","last_clean":"2023-10-06T12:34:20.450548+0000","last_became_active":"2023-10-06T12:34:19.420510+0000","last_became_peered":"2023-10-06T12:34:19.420510+0000","last_unstale":"2023-10-06T12:34:20.450548+0000","last_undegraded":"2023-10-06T12:34:20.450548+0000","last_fullsized":"2023-10-06T12:34:20.450548+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:29.090537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430532+0000","last_change":"2023-10-06T12:34:20.427223+0000","last_active":"2023-10-06T12:34:20.430532+0000","last_peered":"2023-10-06T12:34:20.430532+0000","last_clean":"2023-10-06T12:34:20.430532+0000","last_became_active":"2023-10-06T12:34:20.426907+0000","last_became_peered":"2023-10-06T12:34:20.426907+0000","last_unstale":"2023-10-06T12:34:20.430532+0000","last_undegraded":"2023-10-06T12:34:20.430532+0000","last_fullsized":"2023-10-06T12:34:20.430532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:54.100539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447199+0000","last_change":"2023-10-06T12:34:19.421541+0000","last_active":"2023-10-06T12:34:20.447199+0000","last_peered":"2023-10-06T12:34:20.447199+0000","last_clean":"2023-10-06T12:34:20.447199+0000","last_became_active":"2023-10-06T12:34:19.421177+0000","last_became_peered":"2023-10-06T12:34:19.421177+0000","last_unstale":"2023-10-06T12:34:20.447199+0000","last_undegraded":"2023-10-06T12:34:20.447199+0000","last_fullsized":"2023-10-06T12:34:20.447199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:57.760611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430939+0000","last_change":"2023-10-06T12:34:20.427193+0000","last_active":"2023-10-06T12:34:20.430939+0000","last_peered":"2023-10-06T12:34:20.430939+0000","last_clean":"2023-10-06T12:34:20.430939+0000","last_became_active":"2023-10-06T12:34:20.426860+0000","last_became_peered":"2023-10-06T12:34:20.426860+0000","last_unstale":"2023-10-06T12:34:20.430939+0000","last_undegraded":"2023-10-06T12:34:20.430939+0000","last_fullsized":"2023-10-06T12:34:20.430939+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:56.417675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.424211+0000","last_change":"2023-10-06T12:34:19.428875+0000","last_active":"2023-10-06T12:34:20.424211+0000","last_peered":"2023-10-06T12:34:20.424211+0000","last_clean":"2023-10-06T12:34:20.424211+0000","last_became_active":"2023-10-06T12:34:19.428456+0000","last_became_peered":"2023-10-06T12:34:19.428456+0000","last_unstale":"2023-10-06T12:34:20.424211+0000","last_undegraded":"2023-10-06T12:34:20.424211+0000","last_fullsized":"2023-10-06T12:34:20.424211+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.366399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436152+0000","last_change":"2023-10-06T12:34:20.433039+0000","last_active":"2023-10-06T12:34:20.436152+0000","last_peered":"2023-10-06T12:34:20.436152+0000","last_clean":"2023-10-06T12:34:20.436152+0000","last_became_active":"2023-10-06T12:34:20.432658+0000","last_became_peered":"2023-10-06T12:34:20.432658+0000","last_unstale":"2023-10-06T12:34:20.436152+0000","last_undegraded":"2023-10-06T12:34:20.436152+0000","last_fullsized":"2023-10-06T12:34:20.436152+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:46.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447115+0000","last_change":"2023-10-06T12:34:19.416622+0000","last_active":"2023-10-06T12:34:20.447115+0000","last_peered":"2023-10-06T12:34:20.447115+0000","last_clean":"2023-10-06T12:34:20.447115+0000","last_became_active":"2023-10-06T12:34:19.416257+0000","last_became_peered":"2023-10-06T12:34:19.416257+0000","last_unstale":"2023-10-06T12:34:20.447115+0000","last_undegraded":"2023-10-06T12:34:20.447115+0000","last_fullsized":"2023-10-06T12:34:20.447115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:20:50.547836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443946+0000","last_change":"2023-10-06T12:34:20.439834+0000","last_active":"2023-10-06T12:34:20.443946+0000","last_peered":"2023-10-06T12:34:20.443946+0000","last_clean":"2023-10-06T12:34:20.443946+0000","last_became_active":"2023-10-06T12:34:20.439409+0000","last_became_peered":"2023-10-06T12:34:20.439409+0000","last_unstale":"2023-10-06T12:34:20.443946+0000","last_undegraded":"2023-10-06T12:34:20.443946+0000","last_fullsized":"2023-10-06T12:34:20.443946+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:47:35.885220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.411360+0000","last_change":"2023-10-06T12:34:19.418254+0000","last_active":"2023-10-06T12:34:20.411360+0000","last_peered":"2023-10-06T12:34:20.411360+0000","last_clean":"2023-10-06T12:34:20.411360+0000","last_became_active":"2023-10-06T12:34:19.417955+0000","last_became_peered":"2023-10-06T12:34:19.417955+0000","last_unstale":"2023-10-06T12:34:20.411360+0000","last_undegraded":"2023-10-06T12:34:20.411360+0000","last_fullsized":"2023-10-06T12:34:20.411360+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:38.211096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442744+0000","last_change":"2023-10-06T12:34:20.433719+0000","last_active":"2023-10-06T12:34:20.442744+0000","last_peered":"2023-10-06T12:34:20.442744+0000","last_clean":"2023-10-06T12:34:20.442744+0000","last_became_active":"2023-10-06T12:34:20.432668+0000","last_became_peered":"2023-10-06T12:34:20.432668+0000","last_unstale":"2023-10-06T12:34:20.442744+0000","last_undegraded":"2023-10-06T12:34:20.442744+0000","last_fullsized":"2023-10-06T12:34:20.442744+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:05.790077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.417313+0000","last_change":"2023-10-06T12:34:19.415923+0000","last_active":"2023-10-06T12:34:20.417313+0000","last_peered":"2023-10-06T12:34:20.417313+0000","last_clean":"2023-10-06T12:34:20.417313+0000","last_became_active":"2023-10-06T12:34:19.415600+0000","last_became_peered":"2023-10-06T12:34:19.415600+0000","last_unstale":"2023-10-06T12:34:20.417313+0000","last_undegraded":"2023-10-06T12:34:20.417313+0000","last_fullsized":"2023-10-06T12:34:20.417313+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:10.481688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443766+0000","last_change":"2023-10-06T12:34:20.435419+0000","last_active":"2023-10-06T12:34:20.443766+0000","last_peered":"2023-10-06T12:34:20.443766+0000","last_clean":"2023-10-06T12:34:20.443766+0000","last_became_active":"2023-10-06T12:34:20.435038+0000","last_became_peered":"2023-10-06T12:34:20.435038+0000","last_unstale":"2023-10-06T12:34:20.443766+0000","last_undegraded":"2023-10-06T12:34:20.443766+0000","last_fullsized":"2023-10-06T12:34:20.443766+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:36.742910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.412864+0000","last_change":"2023-10-06T12:34:19.420179+0000","last_active":"2023-10-06T12:34:20.412864+0000","last_peered":"2023-10-06T12:34:20.412864+0000","last_clean":"2023-10-06T12:34:20.412864+0000","last_became_active":"2023-10-06T12:34:19.419880+0000","last_became_peered":"2023-10-06T12:34:19.419880+0000","last_unstale":"2023-10-06T12:34:20.412864+0000","last_undegraded":"2023-10-06T12:34:20.412864+0000","last_fullsized":"2023-10-06T12:34:20.412864+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:32.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431820+0000","last_change":"2023-10-06T12:34:20.419350+0000","last_active":"2023-10-06T12:34:20.431820+0000","last_peered":"2023-10-06T12:34:20.431820+0000","last_clean":"2023-10-06T12:34:20.431820+0000","last_became_active":"2023-10-06T12:34:20.417735+0000","last_became_peered":"2023-10-06T12:34:20.417735+0000","last_unstale":"2023-10-06T12:34:20.431820+0000","last_undegraded":"2023-10-06T12:34:20.431820+0000","last_fullsized":"2023-10-06T12:34:20.431820+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:05.394041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448311+0000","last_change":"2023-10-06T12:34:19.425948+0000","last_active":"2023-10-06T12:34:20.448311+0000","last_peered":"2023-10-06T12:34:20.448311+0000","last_clean":"2023-10-06T12:34:20.448311+0000","last_became_active":"2023-10-06T12:34:19.425654+0000","last_became_peered":"2023-10-06T12:34:19.425654+0000","last_unstale":"2023-10-06T12:34:20.448311+0000","last_undegraded":"2023-10-06T12:34:20.448311+0000","last_fullsized":"2023-10-06T12:34:20.448311+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:58.748685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444898+0000","last_change":"2023-10-06T12:34:20.438189+0000","last_active":"2023-10-06T12:34:20.444898+0000","last_peered":"2023-10-06T12:34:20.444898+0000","last_clean":"2023-10-06T12:34:20.444898+0000","last_became_active":"2023-10-06T12:34:20.437869+0000","last_became_peered":"2023-10-06T12:34:20.437869+0000","last_unstale":"2023-10-06T12:34:20.444898+0000","last_undegraded":"2023-10-06T12:34:20.444898+0000","last_fullsized":"2023-10-06T12:34:20.444898+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.240723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.414130+0000","last_change":"2023-10-06T12:34:19.425054+0000","last_active":"2023-10-06T12:34:20.414130+0000","last_peered":"2023-10-06T12:34:20.414130+0000","last_clean":"2023-10-06T12:34:20.414130+0000","last_became_active":"2023-10-06T12:34:19.424749+0000","last_became_peered":"2023-10-06T12:34:19.424749+0000","last_unstale":"2023-10-06T12:34:20.414130+0000","last_undegraded":"2023-10-06T12:34:20.414130+0000","last_fullsized":"2023-10-06T12:34:20.414130+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:36.277482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.427434+0000","last_change":"2023-10-06T12:34:20.423311+0000","last_active":"2023-10-06T12:34:20.427434+0000","last_peered":"2023-10-06T12:34:20.427434+0000","last_clean":"2023-10-06T12:34:20.427434+0000","last_became_active":"2023-10-06T12:34:20.422961+0000","last_became_peered":"2023-10-06T12:34:20.422961+0000","last_unstale":"2023-10-06T12:34:20.427434+0000","last_undegraded":"2023-10-06T12:34:20.427434+0000","last_fullsized":"2023-10-06T12:34:20.427434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:52.429307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421828+0000","last_change":"2023-10-06T12:34:19.427583+0000","last_active":"2023-10-06T12:34:20.421828+0000","last_peered":"2023-10-06T12:34:20.421828+0000","last_clean":"2023-10-06T12:34:20.421828+0000","last_became_active":"2023-10-06T12:34:19.427143+0000","last_became_peered":"2023-10-06T12:34:19.427143+0000","last_unstale":"2023-10-06T12:34:20.421828+0000","last_undegraded":"2023-10-06T12:34:20.421828+0000","last_fullsized":"2023-10-06T12:34:20.421828+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:30:56.234420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437048+0000","last_change":"2023-10-06T12:34:20.428349+0000","last_active":"2023-10-06T12:34:20.437048+0000","last_peered":"2023-10-06T12:34:20.437048+0000","last_clean":"2023-10-06T12:34:20.437048+0000","last_became_active":"2023-10-06T12:34:20.427988+0000","last_became_peered":"2023-10-06T12:34:20.427988+0000","last_unstale":"2023-10-06T12:34:20.437048+0000","last_undegraded":"2023-10-06T12:34:20.437048+0000","last_fullsized":"2023-10-06T12:34:20.437048+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:08.249347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.411690+0000","last_change":"2023-10-06T12:34:19.428160+0000","last_active":"2023-10-06T12:34:20.411690+0000","last_peered":"2023-10-06T12:34:20.411690+0000","last_clean":"2023-10-06T12:34:20.411690+0000","last_became_active":"2023-10-06T12:34:19.427852+0000","last_became_peered":"2023-10-06T12:34:19.427852+0000","last_unstale":"2023-10-06T12:34:20.411690+0000","last_undegraded":"2023-10-06T12:34:20.411690+0000","last_fullsized":"2023-10-06T12:34:20.411690+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:27.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444478+0000","last_change":"2023-10-06T12:34:20.434453+0000","last_active":"2023-10-06T12:34:20.444478+0000","last_peered":"2023-10-06T12:34:20.444478+0000","last_clean":"2023-10-06T12:34:20.444478+0000","last_became_active":"2023-10-06T12:34:20.434112+0000","last_became_peered":"2023-10-06T12:34:20.434112+0000","last_unstale":"2023-10-06T12:34:20.444478+0000","last_undegraded":"2023-10-06T12:34:20.444478+0000","last_fullsized":"2023-10-06T12:34:20.444478+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:01.509365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413490+0000","last_change":"2023-10-06T12:34:19.419962+0000","last_active":"2023-10-06T12:34:20.413490+0000","last_peered":"2023-10-06T12:34:20.413490+0000","last_clean":"2023-10-06T12:34:20.413490+0000","last_became_active":"2023-10-06T12:34:19.419675+0000","last_became_peered":"2023-10-06T12:34:19.419675+0000","last_unstale":"2023-10-06T12:34:20.413490+0000","last_undegraded":"2023-10-06T12:34:20.413490+0000","last_fullsized":"2023-10-06T12:34:20.413490+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:00.505407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444932+0000","last_change":"2023-10-06T12:34:20.438070+0000","last_active":"2023-10-06T12:34:20.444932+0000","last_peered":"2023-10-06T12:34:20.444932+0000","last_clean":"2023-10-06T12:34:20.444932+0000","last_became_active":"2023-10-06T12:34:20.437752+0000","last_became_peered":"2023-10-06T12:34:20.437752+0000","last_unstale":"2023-10-06T12:34:20.444932+0000","last_undegraded":"2023-10-06T12:34:20.444932+0000","last_fullsized":"2023-10-06T12:34:20.444932+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:08.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.419912+0000","last_change":"2023-10-06T12:34:19.425217+0000","last_active":"2023-10-06T12:34:20.419912+0000","last_peered":"2023-10-06T12:34:20.419912+0000","last_clean":"2023-10-06T12:34:20.419912+0000","last_became_active":"2023-10-06T12:34:19.424678+0000","last_became_peered":"2023-10-06T12:34:19.424678+0000","last_unstale":"2023-10-06T12:34:20.419912+0000","last_undegraded":"2023-10-06T12:34:20.419912+0000","last_fullsized":"2023-10-06T12:34:20.419912+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:14.496639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443982+0000","last_change":"2023-10-06T12:34:20.438856+0000","last_active":"2023-10-06T12:34:20.443982+0000","last_peered":"2023-10-06T12:34:20.443982+0000","last_clean":"2023-10-06T12:34:20.443982+0000","last_became_active":"2023-10-06T12:34:20.438513+0000","last_became_peered":"2023-10-06T12:34:20.438513+0000","last_unstale":"2023-10-06T12:34:20.443982+0000","last_undegraded":"2023-10-06T12:34:20.443982+0000","last_fullsized":"2023-10-06T12:34:20.443982+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:11.990001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413395+0000","last_change":"2023-10-06T12:34:19.420615+0000","last_active":"2023-10-06T12:34:20.413395+0000","last_peered":"2023-10-06T12:34:20.413395+0000","last_clean":"2023-10-06T12:34:20.413395+0000","last_became_active":"2023-10-06T12:34:19.420299+0000","last_became_peered":"2023-10-06T12:34:19.420299+0000","last_unstale":"2023-10-06T12:34:20.413395+0000","last_undegraded":"2023-10-06T12:34:20.413395+0000","last_fullsized":"2023-10-06T12:34:20.413395+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:49.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447985+0000","last_change":"2023-10-06T12:34:20.441584+0000","last_active":"2023-10-06T12:34:20.447985+0000","last_peered":"2023-10-06T12:34:20.447985+0000","last_clean":"2023-10-06T12:34:20.447985+0000","last_became_active":"2023-10-06T12:34:20.441086+0000","last_became_peered":"2023-10-06T12:34:20.441086+0000","last_unstale":"2023-10-06T12:34:20.447985+0000","last_undegraded":"2023-10-06T12:34:20.447985+0000","last_fullsized":"2023-10-06T12:34:20.447985+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:05.790077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421596+0000","last_change":"2023-10-06T12:34:19.423439+0000","last_active":"2023-10-06T12:34:20.421596+0000","last_peered":"2023-10-06T12:34:20.421596+0000","last_clean":"2023-10-06T12:34:20.421596+0000","last_became_active":"2023-10-06T12:34:19.423041+0000","last_became_peered":"2023-10-06T12:34:19.423041+0000","last_unstale":"2023-10-06T12:34:20.421596+0000","last_undegraded":"2023-10-06T12:34:20.421596+0000","last_fullsized":"2023-10-06T12:34:20.421596+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.236765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448002+0000","last_change":"2023-10-06T12:34:20.443798+0000","last_active":"2023-10-06T12:34:20.448002+0000","last_peered":"2023-10-06T12:34:20.448002+0000","last_clean":"2023-10-06T12:34:20.448002+0000","last_became_active":"2023-10-06T12:34:20.443473+0000","last_became_peered":"2023-10-06T12:34:20.443473+0000","last_unstale":"2023-10-06T12:34:20.448002+0000","last_undegraded":"2023-10-06T12:34:20.448002+0000","last_fullsized":"2023-10-06T12:34:20.448002+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:15:13.185350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"165'6","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.451775+0000","last_change":"2023-10-06T12:34:19.416197+0000","last_active":"2023-10-06T12:34:20.451775+0000","last_peered":"2023-10-06T12:34:20.451775+0000","last_clean":"2023-10-06T12:34:20.451775+0000","last_became_active":"2023-10-06T12:34:19.415826+0000","last_became_peered":"2023-10-06T12:34:19.415826+0000","last_unstale":"2023-10-06T12:34:20.451775+0000","last_undegraded":"2023-10-06T12:34:20.451775+0000","last_fullsized":"2023-10-06T12:34:20.451775+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:25.333862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"155'293","reported_seq":909,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421777+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:34:20.421777+0000","last_peered":"2023-10-06T12:34:20.421777+0000","last_clean":"2023-10-06T12:34:20.421777+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:34:20.421777+0000","last_undegraded":"2023-10-06T12:34:20.421777+0000","last_fullsized":"2023-10-06T12:34:20.421777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413593+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:34:20.413593+0000","last_peered":"2023-10-06T12:34:20.413593+0000","last_clean":"2023-10-06T12:34:20.413593+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:34:20.413593+0000","last_undegraded":"2023-10-06T12:34:20.413593+0000","last_fullsized":"2023-10-06T12:34:20.413593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448119+0000","last_change":"2023-10-06T12:34:20.439197+0000","last_active":"2023-10-06T12:34:20.448119+0000","last_peered":"2023-10-06T12:34:20.448119+0000","last_clean":"2023-10-06T12:34:20.448119+0000","last_became_active":"2023-10-06T12:34:20.438899+0000","last_became_peered":"2023-10-06T12:34:20.438899+0000","last_unstale":"2023-10-06T12:34:20.448119+0000","last_undegraded":"2023-10-06T12:34:20.448119+0000","last_fullsized":"2023-10-06T12:34:20.448119+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:37.281440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.425638+0000","last_change":"2023-10-06T12:34:19.424475+0000","last_active":"2023-10-06T12:34:20.425638+0000","last_peered":"2023-10-06T12:34:20.425638+0000","last_clean":"2023-10-06T12:34:20.425638+0000","last_became_active":"2023-10-06T12:34:19.424053+0000","last_became_peered":"2023-10-06T12:34:19.424053+0000","last_unstale":"2023-10-06T12:34:20.425638+0000","last_undegraded":"2023-10-06T12:34:20.425638+0000","last_fullsized":"2023-10-06T12:34:20.425638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:04.390083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":132,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421682+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:34:20.421682+0000","last_peered":"2023-10-06T12:34:20.421682+0000","last_clean":"2023-10-06T12:34:20.421682+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:34:20.421682+0000","last_undegraded":"2023-10-06T12:34:20.421682+0000","last_fullsized":"2023-10-06T12:34:20.421682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443192+0000","last_change":"2023-10-06T12:34:20.437936+0000","last_active":"2023-10-06T12:34:20.443192+0000","last_peered":"2023-10-06T12:34:20.443192+0000","last_clean":"2023-10-06T12:34:20.443192+0000","last_became_active":"2023-10-06T12:34:20.437636+0000","last_became_peered":"2023-10-06T12:34:20.437636+0000","last_unstale":"2023-10-06T12:34:20.443192+0000","last_undegraded":"2023-10-06T12:34:20.443192+0000","last_fullsized":"2023-10-06T12:34:20.443192+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:17.808471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.414081+0000","last_change":"2023-10-06T12:34:19.424907+0000","last_active":"2023-10-06T12:34:20.414081+0000","last_peered":"2023-10-06T12:34:20.414081+0000","last_clean":"2023-10-06T12:34:20.414081+0000","last_became_active":"2023-10-06T12:34:19.424545+0000","last_became_peered":"2023-10-06T12:34:19.424545+0000","last_unstale":"2023-10-06T12:34:20.414081+0000","last_undegraded":"2023-10-06T12:34:20.414081+0000","last_fullsized":"2023-10-06T12:34:20.414081+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:23.217707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.418628+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:34:20.418628+0000","last_peered":"2023-10-06T12:34:20.418628+0000","last_clean":"2023-10-06T12:34:20.418628+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:34:20.418628+0000","last_undegraded":"2023-10-06T12:34:20.418628+0000","last_fullsized":"2023-10-06T12:34:20.418628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441638+0000","last_change":"2023-10-06T12:34:20.430929+0000","last_active":"2023-10-06T12:34:20.441638+0000","last_peered":"2023-10-06T12:34:20.441638+0000","last_clean":"2023-10-06T12:34:20.441638+0000","last_became_active":"2023-10-06T12:34:20.430193+0000","last_became_peered":"2023-10-06T12:34:20.430193+0000","last_unstale":"2023-10-06T12:34:20.441638+0000","last_undegraded":"2023-10-06T12:34:20.441638+0000","last_fullsized":"2023-10-06T12:34:20.441638+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:19.586310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"165'3","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443139+0000","last_change":"2023-10-06T12:34:19.423221+0000","last_active":"2023-10-06T12:34:20.443139+0000","last_peered":"2023-10-06T12:34:20.443139+0000","last_clean":"2023-10-06T12:34:20.443139+0000","last_became_active":"2023-10-06T12:34:19.422927+0000","last_became_peered":"2023-10-06T12:34:19.422927+0000","last_unstale":"2023-10-06T12:34:20.443139+0000","last_undegraded":"2023-10-06T12:34:20.443139+0000","last_fullsized":"2023-10-06T12:34:20.443139+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:53.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413783+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:34:20.413783+0000","last_peered":"2023-10-06T12:34:20.413783+0000","last_clean":"2023-10-06T12:34:20.413783+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:34:20.413783+0000","last_undegraded":"2023-10-06T12:34:20.413783+0000","last_fullsized":"2023-10-06T12:34:20.413783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447935+0000","last_change":"2023-10-06T12:34:20.439172+0000","last_active":"2023-10-06T12:34:20.447935+0000","last_peered":"2023-10-06T12:34:20.447935+0000","last_clean":"2023-10-06T12:34:20.447935+0000","last_became_active":"2023-10-06T12:34:20.438727+0000","last_became_peered":"2023-10-06T12:34:20.438727+0000","last_unstale":"2023-10-06T12:34:20.447935+0000","last_undegraded":"2023-10-06T12:34:20.447935+0000","last_fullsized":"2023-10-06T12:34:20.447935+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:08.249347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450357+0000","last_change":"2023-10-06T12:34:19.421713+0000","last_active":"2023-10-06T12:34:20.450357+0000","last_peered":"2023-10-06T12:34:20.450357+0000","last_clean":"2023-10-06T12:34:20.450357+0000","last_became_active":"2023-10-06T12:34:19.421399+0000","last_became_peered":"2023-10-06T12:34:19.421399+0000","last_unstale":"2023-10-06T12:34:20.450357+0000","last_undegraded":"2023-10-06T12:34:20.450357+0000","last_fullsized":"2023-10-06T12:34:20.450357+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:58.813048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.420207+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:34:20.420207+0000","last_peered":"2023-10-06T12:34:20.420207+0000","last_clean":"2023-10-06T12:34:20.420207+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:34:20.420207+0000","last_undegraded":"2023-10-06T12:34:20.420207+0000","last_fullsized":"2023-10-06T12:34:20.420207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443967+0000","last_change":"2023-10-06T12:34:20.440059+0000","last_active":"2023-10-06T12:34:20.443967+0000","last_peered":"2023-10-06T12:34:20.443967+0000","last_clean":"2023-10-06T12:34:20.443967+0000","last_became_active":"2023-10-06T12:34:20.439495+0000","last_became_peered":"2023-10-06T12:34:20.439495+0000","last_unstale":"2023-10-06T12:34:20.443967+0000","last_undegraded":"2023-10-06T12:34:20.443967+0000","last_fullsized":"2023-10-06T12:34:20.443967+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:59.752643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423942+0000","last_change":"2023-10-06T12:34:19.426392+0000","last_active":"2023-10-06T12:34:20.423942+0000","last_peered":"2023-10-06T12:34:20.423942+0000","last_clean":"2023-10-06T12:34:20.423942+0000","last_became_active":"2023-10-06T12:34:19.426104+0000","last_became_peered":"2023-10-06T12:34:19.426104+0000","last_unstale":"2023-10-06T12:34:20.423942+0000","last_undegraded":"2023-10-06T12:34:20.423942+0000","last_fullsized":"2023-10-06T12:34:20.423942+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:10.986043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":132,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.425245+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:34:20.425245+0000","last_peered":"2023-10-06T12:34:20.425245+0000","last_clean":"2023-10-06T12:34:20.425245+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:34:20.425245+0000","last_undegraded":"2023-10-06T12:34:20.425245+0000","last_fullsized":"2023-10-06T12:34:20.425245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436628+0000","last_change":"2023-10-06T12:34:20.428960+0000","last_active":"2023-10-06T12:34:20.436628+0000","last_peered":"2023-10-06T12:34:20.436628+0000","last_clean":"2023-10-06T12:34:20.436628+0000","last_became_active":"2023-10-06T12:34:20.428677+0000","last_became_peered":"2023-10-06T12:34:20.428677+0000","last_unstale":"2023-10-06T12:34:20.436628+0000","last_undegraded":"2023-10-06T12:34:20.436628+0000","last_fullsized":"2023-10-06T12:34:20.436628+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:30.094495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423744+0000","last_change":"2023-10-06T12:34:19.421945+0000","last_active":"2023-10-06T12:34:20.423744+0000","last_peered":"2023-10-06T12:34:20.423744+0000","last_clean":"2023-10-06T12:34:20.423744+0000","last_became_active":"2023-10-06T12:34:19.421628+0000","last_became_peered":"2023-10-06T12:34:19.421628+0000","last_unstale":"2023-10-06T12:34:20.423744+0000","last_undegraded":"2023-10-06T12:34:20.423744+0000","last_fullsized":"2023-10-06T12:34:20.423744+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:27.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423047+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:34:20.423047+0000","last_peered":"2023-10-06T12:34:20.423047+0000","last_clean":"2023-10-06T12:34:20.423047+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:34:20.423047+0000","last_undegraded":"2023-10-06T12:34:20.423047+0000","last_fullsized":"2023-10-06T12:34:20.423047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444680+0000","last_change":"2023-10-06T12:34:20.441203+0000","last_active":"2023-10-06T12:34:20.444680+0000","last_peered":"2023-10-06T12:34:20.444680+0000","last_clean":"2023-10-06T12:34:20.444680+0000","last_became_active":"2023-10-06T12:34:20.440933+0000","last_became_peered":"2023-10-06T12:34:20.440933+0000","last_unstale":"2023-10-06T12:34:20.444680+0000","last_undegraded":"2023-10-06T12:34:20.444680+0000","last_fullsized":"2023-10-06T12:34:20.444680+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:17.808471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"165'8","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450064+0000","last_change":"2023-10-06T12:34:19.418654+0000","last_active":"2023-10-06T12:34:20.450064+0000","last_peered":"2023-10-06T12:34:20.450064+0000","last_clean":"2023-10-06T12:34:20.450064+0000","last_became_active":"2023-10-06T12:34:19.418251+0000","last_became_peered":"2023-10-06T12:34:19.418251+0000","last_unstale":"2023-10-06T12:34:20.450064+0000","last_undegraded":"2023-10-06T12:34:20.450064+0000","last_fullsized":"2023-10-06T12:34:20.450064+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:07.245389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.424380+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:34:20.424380+0000","last_peered":"2023-10-06T12:34:20.424380+0000","last_clean":"2023-10-06T12:34:20.424380+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:34:20.424380+0000","last_undegraded":"2023-10-06T12:34:20.424380+0000","last_fullsized":"2023-10-06T12:34:20.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575756,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071944,"statfs":{"total":96636764160,"available":96329670656,"internally_reserved":0,"allocated":2670592,"data_stored":1681537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.60599999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.202,"15min":1.175},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.64000000000000001},"max":{"1min":1.6659999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.266}]},{"osd":1,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.359,"15min":1.363},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.68500000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.476},{"interface":"front","average":{"1min":1.21,"5min":1.268,"15min":1.282},"min":{"1min":0.875,"5min":0.875,"15min":0.85599999999999998},"max":{"1min":1.5880000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.84699999999999998,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.7370000000000001,"5min":1.855,"15min":1.855},"last":1.216},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2569999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.65500000000000003},"max":{"1min":1.726,"5min":1.726,"15min":1.7649999999999999},"last":1.083}]},{"osd":3,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3140000000000001,"15min":1.2949999999999999},"min":{"1min":0.78800000000000003,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7070000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.70399999999999996},"max":{"1min":1.679,"5min":1.7130000000000001,"15min":1.9019999999999999},"last":1.381}]},{"osd":4,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.77400000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.573,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.2769999999999999,"15min":1.252},"min":{"1min":0.89800000000000002,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.8200000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.92100000000000004,"5min":0.82199999999999995,"15min":0.49399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.853},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.88500000000000001,"5min":0.81599999999999995,"15min":0.55700000000000005},"max":{"1min":1.702,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.393,"15min":1.375},"min":{"1min":1.081,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":1.766,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4430000000000001,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.54000000000000004},"max":{"1min":1.895,"5min":2.0499999999999998,"15min":2.073},"last":1.3340000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575756,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311604,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060236,"statfs":{"total":96636764160,"available":96317681664,"internally_reserved":0,"allocated":2076672,"data_stored":1091217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.214}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.131},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":0.99299999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.014}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.427},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.304}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.26}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.0660000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307504,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064336,"statfs":{"total":96636764160,"available":96321880064,"internally_reserved":0,"allocated":2072576,"data_stored":1091147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":1.6399999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.004},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":1.6899999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.863},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.242}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.7509999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.385},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.409}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.272},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575757,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1092683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5800000000000001,"15min":1.8839999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.167,"15min":1.1890000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.887,"15min":2.0720000000000001},"last":1.232}]},{"osd":1,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.365},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6240000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.325,"15min":1.329},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.204,"15min":1.2390000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.506,"5min":1.625,"15min":1.9330000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.619,"5min":1.861,"15min":1.861},"last":1.29}]},{"osd":3,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2370000000000001,"15min":1.2490000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5429999999999999,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6339999999999999,"5min":1.7030000000000001,"15min":1.8340000000000001},"last":1.3069999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1919999999999999,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.56999999999999995},"max":{"1min":1.552,"5min":1.7929999999999999,"15min":1.944},"last":1.127},{"interface":"front","average":{"1min":1.163,"5min":1.1819999999999999,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.819},"last":1.071}]},{"osd":6,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.22,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.984},"last":1.411},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3680000000000001,"15min":1.4019999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68799999999999994},"max":{"1min":1.754,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.379}]},{"osd":7,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.339,"15min":1.3620000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.266},{"interface":"front","average":{"1min":1.397,"5min":1.399,"15min":1.4359999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.69399999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1030000000000002},"last":1.5589999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315704,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056136,"statfs":{"total":96636764160,"available":96313483264,"internally_reserved":0,"allocated":2080768,"data_stored":1091735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.376},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.999,"15min":1.0089999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.311},"max":{"1min":1.3089999999999999,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.52500000000000002},"max":{"1min":1.258,"5min":1.6100000000000001,"15min":1.8},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.54500000000000004},"max":{"1min":1.3580000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1439999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.56899999999999995},"max":{"1min":1.623,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.252,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.59199999999999997},"max":{"1min":1.508,"5min":1.649,"15min":2.379},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3029999999999999,"15min":1.27},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.498},"max":{"1min":1.714,"5min":1.8700000000000001,"15min":2.02},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4410000000000001,"15min":1.3979999999999999},"min":{"1min":1.026,"5min":1.026,"15min":0.72499999999999998},"max":{"1min":1.8959999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.1990000000000001,"5min":1.0740000000000001,"15min":0.76900000000000002},"max":{"1min":1.784,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.747},{"interface":"front","average":{"1min":1.419,"5min":1.4390000000000001,"15min":1.419},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.518,"15min":1.4830000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.67600000000000005},"max":{"1min":1.907,"5min":2.0270000000000001,"15min":2.0449999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4510000000000001,"15min":1.3979999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7649999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.468},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7430000000000001,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":0.88},{"interface":"front","average":{"1min":1.625,"5min":1.5780000000000001,"15min":1.5429999999999999},"min":{"1min":1.2949999999999999,"5min":1.1679999999999999,"15min":0.80000000000000004},"max":{"1min":1.9690000000000001,"5min":2.048,"15min":2.0600000000000001},"last":0.85699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575756,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311608,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060232,"statfs":{"total":96636764160,"available":96317677568,"internally_reserved":0,"allocated":2080768,"data_stored":1091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":0.79100000000000004}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.091},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.351},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.3939999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.423},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.5840000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.492},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.623}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.462},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.5529999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575756,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311604,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060236,"statfs":{"total":96636764160,"available":96317681664,"internally_reserved":0,"allocated":2076672,"data_stored":1091217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.673,"5min":1.798,"15min":1.798},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0269999999999999},"min":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3500000000000001,"5min":1.5620000000000001,"15min":1.655},"last":1.1839999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.97499999999999998,"15min":0.99299999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.2909999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.129,"15min":1.1180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.60099999999999998},"max":{"1min":1.397,"5min":1.681,"15min":1.681},"last":1.651}]},{"osd":3,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.61599999999999999},"max":{"1min":1.8580000000000001,"5min":2.008,"15min":2.008},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1519999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.796,"5min":1.796,"15min":1.806},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.268,"15min":1.27},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.597,"5min":1.764,"15min":2.2490000000000001},"last":1.488},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.73,"5min":2.226,"15min":2.226},"last":1.319}]},{"osd":5,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3939999999999999,"15min":1.387},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8069999999999999,"5min":1.8460000000000001,"15min":2.0350000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.381},"min":{"1min":1,"5min":1,"15min":0.93400000000000005},"max":{"1min":1.74,"5min":1.917,"15min":2.0619999999999998},"last":1.0860000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5529999999999999,"15min":1.5149999999999999},"min":{"1min":1.2270000000000001,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":1.9470000000000001,"5min":2.0800000000000001,"15min":2.206},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.4710000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.9450000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.5860000000000001,"15min":1.571},"min":{"1min":1.038,"5min":1.038,"15min":0.99299999999999999},"max":{"1min":2.028,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":1.448},{"interface":"front","average":{"1min":1.516,"5min":1.5149999999999999,"15min":1.49},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":0.95799999999999996},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1819999999999999},"last":1.4099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575757,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299916,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071924,"statfs":{"total":96636764160,"available":96329650176,"internally_reserved":0,"allocated":2691072,"data_stored":1683729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90600000000000003,"15min":0.90400000000000003},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2150000000000001,"5min":1.4410000000000001,"15min":1.492},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3069999999999999,"5min":1.5549999999999999,"15min":1.595},"last":1.089}]},{"osd":2,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.028,"15min":1.042},"min":{"1min":0.72799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.746},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.0840000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.599,"5min":1.762,"15min":1.762},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.554},{"interface":"front","average":{"1min":1.331,"5min":1.3640000000000001,"15min":1.367},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.71999999999999997},"max":{"1min":1.7949999999999999,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.657}]},{"osd":4,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3460000000000001,"15min":1.3580000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6850000000000001,"5min":1.71,"15min":1.756},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.3220000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6759999999999999,"5min":1.704,"15min":1.8560000000000001},"last":1.28}]},{"osd":5,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.393,"15min":1.401},"min":{"1min":0.93000000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.863,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.9630000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4219999999999999,"15min":1.4159999999999999},"min":{"1min":0.94899999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.7470000000000001,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.607}]},{"osd":6,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.1419999999999999},"last":1.351},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.472,"15min":1.476},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.77,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.857}]},{"osd":7,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.474,"15min":1.476},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.8089999999999999,"5min":1.9790000000000001,"15min":2.1019999999999999},"last":1.903},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4390000000000001,"15min":1.4430000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.776}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:34:24.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:34:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.919+0000 7fc1cf6f0700 1 Processor -- start 2023-10-06T12:34:24.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.919+0000 7fc1cf6f0700 1 -- start start 2023-10-06T12:34:24.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cf6f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc1c8156c80 0x7fc1c80ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cf6f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 0x7fc1c80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cf6f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80c8500 con 0x7fc1c815cde0 2023-10-06T12:34:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cf6f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c80c6470 con 0x7fc1c80c7b50 2023-10-06T12:34:24.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cf6f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c80c7760 con 0x7fc1c8156c80 2023-10-06T12:34:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1ccc8b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 0x7fc1c80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1ccc8b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 0x7fc1c80c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50218/0 (socket says 172.21.15.46:50218) 2023-10-06T12:34:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/3279577855 >> v2:172.21.15.132:3300/0 conn(0x7fc1c8156c80 0x7fc1c80ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1ccc8b700 1 -- 172.21.15.46:0/3279577855 learned_addr learned my addr 172.21.15.46:0/3279577855 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cd48c700 1 -- 172.21.15.46:0/3279577855 >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 msgr2=0x7fc1c80c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/3279577855 >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 0x7fc1c80c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cd48c700 1 -- 172.21.15.46:0/3279577855 >> v1:172.21.15.132:6789/0 conn(0x7fc1c815cde0 legacy=0x7fc1c815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.923+0000 7fc1cd48c700 1 -- 172.21.15.46:0/3279577855 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80ca710 con 0x7fc1c8156c80 2023-10-06T12:34:24.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/3279577855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8156c80 0x7fc1c80ca1d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc1c806c640 tx=0x7fc1c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75fca13e618b58d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1bffff700 1 -- 172.21.15.46:0/3279577855 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400e070 con 0x7fc1c8156c80 2023-10-06T12:34:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1bffff700 1 -- 172.21.15.46:0/3279577855 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c400bb60 con 0x7fc1c8156c80 2023-10-06T12:34:24.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1bffff700 1 -- 172.21.15.46:0/3279577855 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4012430 con 0x7fc1c8156c80 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/3279577855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8156c80 msgr2=0x7fc1c80ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/3279577855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8156c80 0x7fc1c80ca1d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc1c806c640 tx=0x7fc1c4009780 comp rx=0 tx=0).stop 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/3279577855 shutdown_connections 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/3279577855 >> v2:172.21.15.46:3300/0 conn(0x7fc1c80c7b50 0x7fc1c80c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/3279577855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8156c80 0x7fc1c80ca1d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/3279577855 >> 172.21.15.46:0/3279577855 conn(0x7fc1c80cbe30 msgr2=0x7fc1c80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/3279577855 shutdown_connections 2023-10-06T12:34:24.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.927+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/3279577855 wait complete. 2023-10-06T12:34:24.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 Processor -- start 2023-10-06T12:34:24.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 -- start start 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80c7b50 0x7fc1c806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8156c80 0x7fc1c81510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c806e520 con 0x7fc1c815cde0 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80aebd0 con 0x7fc1c80c7b50 2023-10-06T12:34:24.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.931+0000 7fc1cf6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c8151610 con 0x7fc1c8156c80 2023-10-06T12:34:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50232/0 (socket says 172.21.15.46:50232) 2023-10-06T12:34:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 -- 172.21.15.46:0/981166237 learned_addr learned my addr 172.21.15.46:0/981166237 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80c7b50 0x7fc1c806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1ccc8b700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8156c80 0x7fc1c81510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 -- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8156c80 msgr2=0x7fc1c81510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8156c80 0x7fc1c81510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 -- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80c7b50 msgr2=0x7fc1c806dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80c7b50 0x7fc1c806dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 -- 172.21.15.46:0/981166237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4009a20 con 0x7fc1c815cde0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cdc8d700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000c560 tx=0x7fc1c0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011590 con 0x7fc1c815cde0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c00116f0 con 0x7fc1c815cde0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011ab0 con 0x7fc1c815cde0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80aee20 con 0x7fc1c815cde0 2023-10-06T12:34:24.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c80af3e0 con 0x7fc1c815cde0 2023-10-06T12:34:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.935+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c8063f90 con 0x7fc1c815cde0 2023-10-06T12:34:24.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.939+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc1c001b3b0 con 0x7fc1c815cde0 2023-10-06T12:34:24.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.939+0000 7fc1bd7fa700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 0x7fc1ac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:24.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.939+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fc1c007c150 con 0x7fc1c815cde0 2023-10-06T12:34:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.939+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 0x7fc1ac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:24.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.939+0000 7fc1cd48c700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 0x7fc1ac062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097c0 tx=0x7fc1c4019000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:24.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:24.943+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc1c004c230 con 0x7fc1c815cde0 2023-10-06T12:34:25.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.107+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1c804b5c0 con 0x7fc1ac060670 2023-10-06T12:34:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.123+0000 7fc1bd7fa700 1 -- 172.21.15.46:0/981166237 <== mgr.8875 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308744 (secure 0 0 0) 0x7fc1c804b5c0 con 0x7fc1ac060670 2023-10-06T12:34:25.132 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 msgr2=0x7fc1ac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 0x7fc1ac062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097c0 tx=0x7fc1c4019000 comp rx=0 tx=0).stop 2023-10-06T12:34:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 msgr2=0x7fc1c816a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000c560 tx=0x7fc1c0007ef0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 shutdown_connections 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fc1ac060670 0x7fc1ac062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c815cde0 0x7fc1c816a760 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8156c80 0x7fc1c81510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 --2- 172.21.15.46:0/981166237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80c7b50 0x7fc1c806dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 >> 172.21.15.46:0/981166237 conn(0x7fc1c80cbe30 msgr2=0x7fc1c806e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:25.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 shutdown_connections 2023-10-06T12:34:25.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.131+0000 7fc1cf6f0700 1 -- 172.21.15.46:0/981166237 wait complete. 2023-10-06T12:34:25.138 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:34:25.156 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T12:34:25.036804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2469448,"kb_used_data":17416,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752505272,"statfs":{"total":773094113280,"available":770565398528,"internally_reserved":0,"allocated":17833984,"data_stored":9914524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002075"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.427345+0000","last_change":"2023-10-06T12:34:20.422056+0000","last_active":"2023-10-06T12:34:20.427345+0000","last_peered":"2023-10-06T12:34:20.427345+0000","last_clean":"2023-10-06T12:34:20.427345+0000","last_became_active":"2023-10-06T12:34:20.421765+0000","last_became_peered":"2023-10-06T12:34:20.421765+0000","last_unstale":"2023-10-06T12:34:20.427345+0000","last_undegraded":"2023-10-06T12:34:20.427345+0000","last_fullsized":"2023-10-06T12:34:20.427345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:45.930963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443722+0000","last_change":"2023-10-06T12:34:20.435452+0000","last_active":"2023-10-06T12:34:20.443722+0000","last_peered":"2023-10-06T12:34:20.443722+0000","last_clean":"2023-10-06T12:34:20.443722+0000","last_became_active":"2023-10-06T12:34:20.435160+0000","last_became_peered":"2023-10-06T12:34:20.435160+0000","last_unstale":"2023-10-06T12:34:20.443722+0000","last_undegraded":"2023-10-06T12:34:20.443722+0000","last_fullsized":"2023-10-06T12:34:20.443722+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:52.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444107+0000","last_change":"2023-10-06T12:34:20.438651+0000","last_active":"2023-10-06T12:34:20.444107+0000","last_peered":"2023-10-06T12:34:20.444107+0000","last_clean":"2023-10-06T12:34:20.444107+0000","last_became_active":"2023-10-06T12:34:20.438365+0000","last_became_peered":"2023-10-06T12:34:20.438365+0000","last_unstale":"2023-10-06T12:34:20.444107+0000","last_undegraded":"2023-10-06T12:34:20.444107+0000","last_fullsized":"2023-10-06T12:34:20.444107+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:17.370357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437313+0000","last_change":"2023-10-06T12:34:20.431779+0000","last_active":"2023-10-06T12:34:20.437313+0000","last_peered":"2023-10-06T12:34:20.437313+0000","last_clean":"2023-10-06T12:34:20.437313+0000","last_became_active":"2023-10-06T12:34:20.431462+0000","last_became_peered":"2023-10-06T12:34:20.431462+0000","last_unstale":"2023-10-06T12:34:20.437313+0000","last_undegraded":"2023-10-06T12:34:20.437313+0000","last_fullsized":"2023-10-06T12:34:20.437313+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:18:17.068158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436913+0000","last_change":"2023-10-06T12:34:20.418531+0000","last_active":"2023-10-06T12:34:20.436913+0000","last_peered":"2023-10-06T12:34:20.436913+0000","last_clean":"2023-10-06T12:34:20.436913+0000","last_became_active":"2023-10-06T12:34:20.418187+0000","last_became_peered":"2023-10-06T12:34:20.418187+0000","last_unstale":"2023-10-06T12:34:20.436913+0000","last_undegraded":"2023-10-06T12:34:20.436913+0000","last_fullsized":"2023-10-06T12:34:20.436913+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:50.891730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431717+0000","last_change":"2023-10-06T12:34:20.419115+0000","last_active":"2023-10-06T12:34:20.431717+0000","last_peered":"2023-10-06T12:34:20.431717+0000","last_clean":"2023-10-06T12:34:20.431717+0000","last_became_active":"2023-10-06T12:34:20.418730+0000","last_became_peered":"2023-10-06T12:34:20.418730+0000","last_unstale":"2023-10-06T12:34:20.431717+0000","last_undegraded":"2023-10-06T12:34:20.431717+0000","last_fullsized":"2023-10-06T12:34:20.431717+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:24.221665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.439697+0000","last_change":"2023-10-06T12:34:20.433587+0000","last_active":"2023-10-06T12:34:20.439697+0000","last_peered":"2023-10-06T12:34:20.439697+0000","last_clean":"2023-10-06T12:34:20.439697+0000","last_became_active":"2023-10-06T12:34:20.433207+0000","last_became_peered":"2023-10-06T12:34:20.433207+0000","last_unstale":"2023-10-06T12:34:20.439697+0000","last_undegraded":"2023-10-06T12:34:20.439697+0000","last_fullsized":"2023-10-06T12:34:20.439697+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:41:36.587895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434616+0000","last_change":"2023-10-06T12:34:20.430936+0000","last_active":"2023-10-06T12:34:20.434616+0000","last_peered":"2023-10-06T12:34:20.434616+0000","last_clean":"2023-10-06T12:34:20.434616+0000","last_became_active":"2023-10-06T12:34:20.430571+0000","last_became_peered":"2023-10-06T12:34:20.430571+0000","last_unstale":"2023-10-06T12:34:20.434616+0000","last_undegraded":"2023-10-06T12:34:20.434616+0000","last_fullsized":"2023-10-06T12:34:20.434616+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:32.700937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441127+0000","last_change":"2023-10-06T12:34:20.436943+0000","last_active":"2023-10-06T12:34:20.441127+0000","last_peered":"2023-10-06T12:34:20.441127+0000","last_clean":"2023-10-06T12:34:20.441127+0000","last_became_active":"2023-10-06T12:34:20.436573+0000","last_became_peered":"2023-10-06T12:34:20.436573+0000","last_unstale":"2023-10-06T12:34:20.441127+0000","last_undegraded":"2023-10-06T12:34:20.441127+0000","last_fullsized":"2023-10-06T12:34:20.441127+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.240723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436663+0000","last_change":"2023-10-06T12:34:20.429878+0000","last_active":"2023-10-06T12:34:20.436663+0000","last_peered":"2023-10-06T12:34:20.436663+0000","last_clean":"2023-10-06T12:34:20.436663+0000","last_became_active":"2023-10-06T12:34:20.429354+0000","last_became_peered":"2023-10-06T12:34:20.429354+0000","last_unstale":"2023-10-06T12:34:20.436663+0000","last_undegraded":"2023-10-06T12:34:20.436663+0000","last_fullsized":"2023-10-06T12:34:20.436663+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.484389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.445039+0000","last_change":"2023-10-06T12:34:20.442426+0000","last_active":"2023-10-06T12:34:20.445039+0000","last_peered":"2023-10-06T12:34:20.445039+0000","last_clean":"2023-10-06T12:34:20.445039+0000","last_became_active":"2023-10-06T12:34:20.442104+0000","last_became_peered":"2023-10-06T12:34:20.442104+0000","last_unstale":"2023-10-06T12:34:20.445039+0000","last_undegraded":"2023-10-06T12:34:20.445039+0000","last_fullsized":"2023-10-06T12:34:20.445039+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:39.215054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440808+0000","last_change":"2023-10-06T12:34:20.434972+0000","last_active":"2023-10-06T12:34:20.440808+0000","last_peered":"2023-10-06T12:34:20.440808+0000","last_clean":"2023-10-06T12:34:20.440808+0000","last_became_active":"2023-10-06T12:34:20.434516+0000","last_became_peered":"2023-10-06T12:34:20.434516+0000","last_unstale":"2023-10-06T12:34:20.440808+0000","last_undegraded":"2023-10-06T12:34:20.440808+0000","last_fullsized":"2023-10-06T12:34:20.440808+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:30.094495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436944+0000","last_change":"2023-10-06T12:34:20.427214+0000","last_active":"2023-10-06T12:34:20.436944+0000","last_peered":"2023-10-06T12:34:20.436944+0000","last_clean":"2023-10-06T12:34:20.436944+0000","last_became_active":"2023-10-06T12:34:20.426867+0000","last_became_peered":"2023-10-06T12:34:20.426867+0000","last_unstale":"2023-10-06T12:34:20.436944+0000","last_undegraded":"2023-10-06T12:34:20.436944+0000","last_fullsized":"2023-10-06T12:34:20.436944+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:48.740650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444983+0000","last_change":"2023-10-06T12:34:20.438617+0000","last_active":"2023-10-06T12:34:20.444983+0000","last_peered":"2023-10-06T12:34:20.444983+0000","last_clean":"2023-10-06T12:34:20.444983+0000","last_became_active":"2023-10-06T12:34:20.438287+0000","last_became_peered":"2023-10-06T12:34:20.438287+0000","last_unstale":"2023-10-06T12:34:20.444983+0000","last_undegraded":"2023-10-06T12:34:20.444983+0000","last_fullsized":"2023-10-06T12:34:20.444983+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:53:33.484389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444603+0000","last_change":"2023-10-06T12:34:20.440645+0000","last_active":"2023-10-06T12:34:20.444603+0000","last_peered":"2023-10-06T12:34:20.444603+0000","last_clean":"2023-10-06T12:34:20.444603+0000","last_became_active":"2023-10-06T12:34:20.439744+0000","last_became_peered":"2023-10-06T12:34:20.439744+0000","last_unstale":"2023-10-06T12:34:20.444603+0000","last_undegraded":"2023-10-06T12:34:20.444603+0000","last_fullsized":"2023-10-06T12:34:20.444603+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:39:29.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436102+0000","last_change":"2023-10-06T12:34:20.424317+0000","last_active":"2023-10-06T12:34:20.436102+0000","last_peered":"2023-10-06T12:34:20.436102+0000","last_clean":"2023-10-06T12:34:20.436102+0000","last_became_active":"2023-10-06T12:34:20.423484+0000","last_became_peered":"2023-10-06T12:34:20.423484+0000","last_unstale":"2023-10-06T12:34:20.436102+0000","last_undegraded":"2023-10-06T12:34:20.436102+0000","last_fullsized":"2023-10-06T12:34:20.436102+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:01.509365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437345+0000","last_change":"2023-10-06T12:34:20.430396+0000","last_active":"2023-10-06T12:34:20.437345+0000","last_peered":"2023-10-06T12:34:20.437345+0000","last_clean":"2023-10-06T12:34:20.437345+0000","last_became_active":"2023-10-06T12:34:20.429852+0000","last_became_peered":"2023-10-06T12:34:20.429852+0000","last_unstale":"2023-10-06T12:34:20.437345+0000","last_undegraded":"2023-10-06T12:34:20.437345+0000","last_fullsized":"2023-10-06T12:34:20.437345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:12.391470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441660+0000","last_change":"2023-10-06T12:34:20.431126+0000","last_active":"2023-10-06T12:34:20.441660+0000","last_peered":"2023-10-06T12:34:20.441660+0000","last_clean":"2023-10-06T12:34:20.441660+0000","last_became_active":"2023-10-06T12:34:20.430524+0000","last_became_peered":"2023-10-06T12:34:20.430524+0000","last_unstale":"2023-10-06T12:34:20.441660+0000","last_undegraded":"2023-10-06T12:34:20.441660+0000","last_fullsized":"2023-10-06T12:34:20.441660+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:46.328245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449382+0000","last_change":"2023-10-06T12:34:20.442182+0000","last_active":"2023-10-06T12:34:20.449382+0000","last_peered":"2023-10-06T12:34:20.449382+0000","last_clean":"2023-10-06T12:34:20.449382+0000","last_became_active":"2023-10-06T12:34:20.437829+0000","last_became_peered":"2023-10-06T12:34:20.437829+0000","last_unstale":"2023-10-06T12:34:20.449382+0000","last_undegraded":"2023-10-06T12:34:20.449382+0000","last_fullsized":"2023-10-06T12:34:20.449382+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:15:13.185350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440411+0000","last_change":"2023-10-06T12:34:20.438559+0000","last_active":"2023-10-06T12:34:20.440411+0000","last_peered":"2023-10-06T12:34:20.440411+0000","last_clean":"2023-10-06T12:34:20.440411+0000","last_became_active":"2023-10-06T12:34:20.438180+0000","last_became_peered":"2023-10-06T12:34:20.438180+0000","last_unstale":"2023-10-06T12:34:20.440411+0000","last_undegraded":"2023-10-06T12:34:20.440411+0000","last_fullsized":"2023-10-06T12:34:20.440411+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:51:51.690525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443808+0000","last_change":"2023-10-06T12:34:20.437067+0000","last_active":"2023-10-06T12:34:20.443808+0000","last_peered":"2023-10-06T12:34:20.443808+0000","last_clean":"2023-10-06T12:34:20.443808+0000","last_became_active":"2023-10-06T12:34:20.436702+0000","last_became_peered":"2023-10-06T12:34:20.436702+0000","last_unstale":"2023-10-06T12:34:20.443808+0000","last_undegraded":"2023-10-06T12:34:20.443808+0000","last_fullsized":"2023-10-06T12:34:20.443808+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:50.536480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.439825+0000","last_change":"2023-10-06T12:34:20.433593+0000","last_active":"2023-10-06T12:34:20.439825+0000","last_peered":"2023-10-06T12:34:20.439825+0000","last_clean":"2023-10-06T12:34:20.439825+0000","last_became_active":"2023-10-06T12:34:20.433284+0000","last_became_peered":"2023-10-06T12:34:20.433284+0000","last_unstale":"2023-10-06T12:34:20.439825+0000","last_undegraded":"2023-10-06T12:34:20.439825+0000","last_fullsized":"2023-10-06T12:34:20.439825+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:21:39.297865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.422008+0000","last_change":"2023-10-06T12:34:20.420936+0000","last_active":"2023-10-06T12:34:20.422008+0000","last_peered":"2023-10-06T12:34:20.422008+0000","last_clean":"2023-10-06T12:34:20.422008+0000","last_became_active":"2023-10-06T12:34:20.420612+0000","last_became_peered":"2023-10-06T12:34:20.420612+0000","last_unstale":"2023-10-06T12:34:20.422008+0000","last_undegraded":"2023-10-06T12:34:20.422008+0000","last_fullsized":"2023-10-06T12:34:20.422008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:24.221665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443253+0000","last_change":"2023-10-06T12:34:20.438071+0000","last_active":"2023-10-06T12:34:20.443253+0000","last_peered":"2023-10-06T12:34:20.443253+0000","last_clean":"2023-10-06T12:34:20.443253+0000","last_became_active":"2023-10-06T12:34:20.437757+0000","last_became_peered":"2023-10-06T12:34:20.437757+0000","last_unstale":"2023-10-06T12:34:20.443253+0000","last_undegraded":"2023-10-06T12:34:20.443253+0000","last_fullsized":"2023-10-06T12:34:20.443253+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:33.012175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431530+0000","last_change":"2023-10-06T12:34:20.429594+0000","last_active":"2023-10-06T12:34:20.431530+0000","last_peered":"2023-10-06T12:34:20.431530+0000","last_clean":"2023-10-06T12:34:20.431530+0000","last_became_active":"2023-10-06T12:34:20.429178+0000","last_became_peered":"2023-10-06T12:34:20.429178+0000","last_unstale":"2023-10-06T12:34:20.431530+0000","last_undegraded":"2023-10-06T12:34:20.431530+0000","last_fullsized":"2023-10-06T12:34:20.431530+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:38.434455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441058+0000","last_change":"2023-10-06T12:34:20.436485+0000","last_active":"2023-10-06T12:34:20.441058+0000","last_peered":"2023-10-06T12:34:20.441058+0000","last_clean":"2023-10-06T12:34:20.441058+0000","last_became_active":"2023-10-06T12:34:20.436131+0000","last_became_peered":"2023-10-06T12:34:20.436131+0000","last_unstale":"2023-10-06T12:34:20.441058+0000","last_undegraded":"2023-10-06T12:34:20.441058+0000","last_fullsized":"2023-10-06T12:34:20.441058+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:06.542282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.440857+0000","last_change":"2023-10-06T12:34:20.437086+0000","last_active":"2023-10-06T12:34:20.440857+0000","last_peered":"2023-10-06T12:34:20.440857+0000","last_clean":"2023-10-06T12:34:20.440857+0000","last_became_active":"2023-10-06T12:34:20.436763+0000","last_became_peered":"2023-10-06T12:34:20.436763+0000","last_unstale":"2023-10-06T12:34:20.440857+0000","last_undegraded":"2023-10-06T12:34:20.440857+0000","last_fullsized":"2023-10-06T12:34:20.440857+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:08.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430059+0000","last_change":"2023-10-06T12:34:20.428484+0000","last_active":"2023-10-06T12:34:20.430059+0000","last_peered":"2023-10-06T12:34:20.430059+0000","last_clean":"2023-10-06T12:34:20.430059+0000","last_became_active":"2023-10-06T12:34:20.426905+0000","last_became_peered":"2023-10-06T12:34:20.426905+0000","last_unstale":"2023-10-06T12:34:20.430059+0000","last_undegraded":"2023-10-06T12:34:20.430059+0000","last_fullsized":"2023-10-06T12:34:20.430059+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:52.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431798+0000","last_change":"2023-10-06T12:34:20.428064+0000","last_active":"2023-10-06T12:34:20.431798+0000","last_peered":"2023-10-06T12:34:20.431798+0000","last_clean":"2023-10-06T12:34:20.431798+0000","last_became_active":"2023-10-06T12:34:20.427727+0000","last_became_peered":"2023-10-06T12:34:20.427727+0000","last_unstale":"2023-10-06T12:34:20.431798+0000","last_undegraded":"2023-10-06T12:34:20.431798+0000","last_fullsized":"2023-10-06T12:34:20.431798+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:46.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.435942+0000","last_change":"2023-10-06T12:34:20.429745+0000","last_active":"2023-10-06T12:34:20.435942+0000","last_peered":"2023-10-06T12:34:20.435942+0000","last_clean":"2023-10-06T12:34:20.435942+0000","last_became_active":"2023-10-06T12:34:20.429370+0000","last_became_peered":"2023-10-06T12:34:20.429370+0000","last_unstale":"2023-10-06T12:34:20.435942+0000","last_undegraded":"2023-10-06T12:34:20.435942+0000","last_fullsized":"2023-10-06T12:34:20.435942+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:33:59.315328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.433006+0000","last_change":"2023-10-06T12:34:20.423174+0000","last_active":"2023-10-06T12:34:20.433006+0000","last_peered":"2023-10-06T12:34:20.433006+0000","last_clean":"2023-10-06T12:34:20.433006+0000","last_became_active":"2023-10-06T12:34:20.422176+0000","last_became_peered":"2023-10-06T12:34:20.422176+0000","last_unstale":"2023-10-06T12:34:20.433006+0000","last_undegraded":"2023-10-06T12:34:20.433006+0000","last_fullsized":"2023-10-06T12:34:20.433006+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:37.281440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434596+0000","last_change":"2023-10-06T12:34:20.430628+0000","last_active":"2023-10-06T12:34:20.434596+0000","last_peered":"2023-10-06T12:34:20.434596+0000","last_clean":"2023-10-06T12:34:20.434596+0000","last_became_active":"2023-10-06T12:34:20.430270+0000","last_became_peered":"2023-10-06T12:34:20.430270+0000","last_unstale":"2023-10-06T12:34:20.434596+0000","last_undegraded":"2023-10-06T12:34:20.434596+0000","last_fullsized":"2023-10-06T12:34:20.434596+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:48.681099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443807+0000","last_change":"2023-10-06T12:34:20.436890+0000","last_active":"2023-10-06T12:34:20.443807+0000","last_peered":"2023-10-06T12:34:20.443807+0000","last_clean":"2023-10-06T12:34:20.443807+0000","last_became_active":"2023-10-06T12:34:20.436521+0000","last_became_peered":"2023-10-06T12:34:20.436521+0000","last_unstale":"2023-10-06T12:34:20.443807+0000","last_undegraded":"2023-10-06T12:34:20.443807+0000","last_fullsized":"2023-10-06T12:34:20.443807+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:55.289183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"165'4","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450452+0000","last_change":"2023-10-06T12:34:19.420577+0000","last_active":"2023-10-06T12:34:20.450452+0000","last_peered":"2023-10-06T12:34:20.450452+0000","last_clean":"2023-10-06T12:34:20.450452+0000","last_became_active":"2023-10-06T12:34:19.420176+0000","last_became_peered":"2023-10-06T12:34:19.420176+0000","last_unstale":"2023-10-06T12:34:20.450452+0000","last_undegraded":"2023-10-06T12:34:20.450452+0000","last_fullsized":"2023-10-06T12:34:20.450452+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:22:54.285225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444721+0000","last_change":"2023-10-06T12:34:20.441785+0000","last_active":"2023-10-06T12:34:20.444721+0000","last_peered":"2023-10-06T12:34:20.444721+0000","last_clean":"2023-10-06T12:34:20.444721+0000","last_became_active":"2023-10-06T12:34:20.441520+0000","last_became_peered":"2023-10-06T12:34:20.441520+0000","last_unstale":"2023-10-06T12:34:20.444721+0000","last_undegraded":"2023-10-06T12:34:20.444721+0000","last_fullsized":"2023-10-06T12:34:20.444721+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:28.324559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449223+0000","last_change":"2023-10-06T12:34:19.425747+0000","last_active":"2023-10-06T12:34:20.449223+0000","last_peered":"2023-10-06T12:34:20.449223+0000","last_clean":"2023-10-06T12:34:20.449223+0000","last_became_active":"2023-10-06T12:34:19.425206+0000","last_became_peered":"2023-10-06T12:34:19.425206+0000","last_unstale":"2023-10-06T12:34:20.449223+0000","last_undegraded":"2023-10-06T12:34:20.449223+0000","last_fullsized":"2023-10-06T12:34:20.449223+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:31:49.532522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.433904+0000","last_change":"2023-10-06T12:34:20.430696+0000","last_active":"2023-10-06T12:34:20.433904+0000","last_peered":"2023-10-06T12:34:20.433904+0000","last_clean":"2023-10-06T12:34:20.433904+0000","last_became_active":"2023-10-06T12:34:20.430263+0000","last_became_peered":"2023-10-06T12:34:20.430263+0000","last_unstale":"2023-10-06T12:34:20.433904+0000","last_undegraded":"2023-10-06T12:34:20.433904+0000","last_fullsized":"2023-10-06T12:34:20.433904+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:23.761652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449585+0000","last_change":"2023-10-06T12:34:19.426251+0000","last_active":"2023-10-06T12:34:20.449585+0000","last_peered":"2023-10-06T12:34:20.449585+0000","last_clean":"2023-10-06T12:34:20.449585+0000","last_became_active":"2023-10-06T12:34:19.425935+0000","last_became_peered":"2023-10-06T12:34:19.425935+0000","last_unstale":"2023-10-06T12:34:20.449585+0000","last_undegraded":"2023-10-06T12:34:20.449585+0000","last_fullsized":"2023-10-06T12:34:20.449585+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:00:47.736692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442434+0000","last_change":"2023-10-06T12:34:20.433485+0000","last_active":"2023-10-06T12:34:20.442434+0000","last_peered":"2023-10-06T12:34:20.442434+0000","last_clean":"2023-10-06T12:34:20.442434+0000","last_became_active":"2023-10-06T12:34:20.433198+0000","last_became_peered":"2023-10-06T12:34:20.433198+0000","last_unstale":"2023-10-06T12:34:20.442434+0000","last_undegraded":"2023-10-06T12:34:20.442434+0000","last_fullsized":"2023-10-06T12:34:20.442434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:56.657861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448479+0000","last_change":"2023-10-06T12:34:19.429070+0000","last_active":"2023-10-06T12:34:20.448479+0000","last_peered":"2023-10-06T12:34:20.448479+0000","last_clean":"2023-10-06T12:34:20.448479+0000","last_became_active":"2023-10-06T12:34:19.428644+0000","last_became_peered":"2023-10-06T12:34:19.428644+0000","last_unstale":"2023-10-06T12:34:20.448479+0000","last_undegraded":"2023-10-06T12:34:20.448479+0000","last_fullsized":"2023-10-06T12:34:20.448479+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:05.433196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.434195+0000","last_change":"2023-10-06T12:34:20.431289+0000","last_active":"2023-10-06T12:34:20.434195+0000","last_peered":"2023-10-06T12:34:20.434195+0000","last_clean":"2023-10-06T12:34:20.434195+0000","last_became_active":"2023-10-06T12:34:20.430906+0000","last_became_peered":"2023-10-06T12:34:20.430906+0000","last_unstale":"2023-10-06T12:34:20.434195+0000","last_undegraded":"2023-10-06T12:34:20.434195+0000","last_fullsized":"2023-10-06T12:34:20.434195+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:30:33.336476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447890+0000","last_change":"2023-10-06T12:34:19.416154+0000","last_active":"2023-10-06T12:34:20.447890+0000","last_peered":"2023-10-06T12:34:20.447890+0000","last_clean":"2023-10-06T12:34:20.447890+0000","last_became_active":"2023-10-06T12:34:19.415730+0000","last_became_peered":"2023-10-06T12:34:19.415730+0000","last_unstale":"2023-10-06T12:34:20.447890+0000","last_undegraded":"2023-10-06T12:34:20.447890+0000","last_fullsized":"2023-10-06T12:34:20.447890+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:24:32.376167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.446460+0000","last_change":"2023-10-06T12:34:20.442742+0000","last_active":"2023-10-06T12:34:20.446460+0000","last_peered":"2023-10-06T12:34:20.446460+0000","last_clean":"2023-10-06T12:34:20.446460+0000","last_became_active":"2023-10-06T12:34:20.442153+0000","last_became_peered":"2023-10-06T12:34:20.442153+0000","last_unstale":"2023-10-06T12:34:20.446460+0000","last_undegraded":"2023-10-06T12:34:20.446460+0000","last_fullsized":"2023-10-06T12:34:20.446460+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:29:33.498249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448263+0000","last_change":"2023-10-06T12:34:19.427439+0000","last_active":"2023-10-06T12:34:20.448263+0000","last_peered":"2023-10-06T12:34:20.448263+0000","last_clean":"2023-10-06T12:34:20.448263+0000","last_became_active":"2023-10-06T12:34:19.427021+0000","last_became_peered":"2023-10-06T12:34:19.427021+0000","last_unstale":"2023-10-06T12:34:20.448263+0000","last_undegraded":"2023-10-06T12:34:20.448263+0000","last_fullsized":"2023-10-06T12:34:20.448263+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.366399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444001+0000","last_change":"2023-10-06T12:34:20.439155+0000","last_active":"2023-10-06T12:34:20.444001+0000","last_peered":"2023-10-06T12:34:20.444001+0000","last_clean":"2023-10-06T12:34:20.444001+0000","last_became_active":"2023-10-06T12:34:20.438850+0000","last_became_peered":"2023-10-06T12:34:20.438850+0000","last_unstale":"2023-10-06T12:34:20.444001+0000","last_undegraded":"2023-10-06T12:34:20.444001+0000","last_fullsized":"2023-10-06T12:34:20.444001+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:19.740524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413271+0000","last_change":"2023-10-06T12:34:19.417853+0000","last_active":"2023-10-06T12:34:20.413271+0000","last_peered":"2023-10-06T12:34:20.413271+0000","last_clean":"2023-10-06T12:34:20.413271+0000","last_became_active":"2023-10-06T12:34:19.417499+0000","last_became_peered":"2023-10-06T12:34:19.417499+0000","last_unstale":"2023-10-06T12:34:20.413271+0000","last_undegraded":"2023-10-06T12:34:20.413271+0000","last_fullsized":"2023-10-06T12:34:20.413271+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:04.390083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.426823+0000","last_change":"2023-10-06T12:34:20.422610+0000","last_active":"2023-10-06T12:34:20.426823+0000","last_peered":"2023-10-06T12:34:20.426823+0000","last_clean":"2023-10-06T12:34:20.426823+0000","last_became_active":"2023-10-06T12:34:20.422317+0000","last_became_peered":"2023-10-06T12:34:20.422317+0000","last_unstale":"2023-10-06T12:34:20.426823+0000","last_undegraded":"2023-10-06T12:34:20.426823+0000","last_fullsized":"2023-10-06T12:34:20.426823+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:52.924767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.409416+0000","last_change":"2023-10-06T12:34:19.422008+0000","last_active":"2023-10-06T12:34:20.409416+0000","last_peered":"2023-10-06T12:34:20.409416+0000","last_clean":"2023-10-06T12:34:20.409416+0000","last_became_active":"2023-10-06T12:34:19.421669+0000","last_became_peered":"2023-10-06T12:34:19.421669+0000","last_unstale":"2023-10-06T12:34:20.409416+0000","last_undegraded":"2023-10-06T12:34:20.409416+0000","last_fullsized":"2023-10-06T12:34:20.409416+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:40.633582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449413+0000","last_change":"2023-10-06T12:34:20.442539+0000","last_active":"2023-10-06T12:34:20.449413+0000","last_peered":"2023-10-06T12:34:20.449413+0000","last_clean":"2023-10-06T12:34:20.449413+0000","last_became_active":"2023-10-06T12:34:20.441876+0000","last_became_peered":"2023-10-06T12:34:20.441876+0000","last_unstale":"2023-10-06T12:34:20.449413+0000","last_undegraded":"2023-10-06T12:34:20.449413+0000","last_fullsized":"2023-10-06T12:34:20.449413+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:49:13.068286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.449036+0000","last_change":"2023-10-06T12:34:19.419099+0000","last_active":"2023-10-06T12:34:20.449036+0000","last_peered":"2023-10-06T12:34:20.449036+0000","last_clean":"2023-10-06T12:34:20.449036+0000","last_became_active":"2023-10-06T12:34:19.418742+0000","last_became_peered":"2023-10-06T12:34:19.418742+0000","last_unstale":"2023-10-06T12:34:20.449036+0000","last_undegraded":"2023-10-06T12:34:20.449036+0000","last_fullsized":"2023-10-06T12:34:20.449036+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:35.738952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442180+0000","last_change":"2023-10-06T12:34:20.436475+0000","last_active":"2023-10-06T12:34:20.442180+0000","last_peered":"2023-10-06T12:34:20.442180+0000","last_clean":"2023-10-06T12:34:20.442180+0000","last_became_active":"2023-10-06T12:34:20.433947+0000","last_became_peered":"2023-10-06T12:34:20.433947+0000","last_unstale":"2023-10-06T12:34:20.442180+0000","last_undegraded":"2023-10-06T12:34:20.442180+0000","last_fullsized":"2023-10-06T12:34:20.442180+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:32.700937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450548+0000","last_change":"2023-10-06T12:34:19.421522+0000","last_active":"2023-10-06T12:34:20.450548+0000","last_peered":"2023-10-06T12:34:20.450548+0000","last_clean":"2023-10-06T12:34:20.450548+0000","last_became_active":"2023-10-06T12:34:19.420510+0000","last_became_peered":"2023-10-06T12:34:19.420510+0000","last_unstale":"2023-10-06T12:34:20.450548+0000","last_undegraded":"2023-10-06T12:34:20.450548+0000","last_fullsized":"2023-10-06T12:34:20.450548+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:29.090537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430532+0000","last_change":"2023-10-06T12:34:20.427223+0000","last_active":"2023-10-06T12:34:20.430532+0000","last_peered":"2023-10-06T12:34:20.430532+0000","last_clean":"2023-10-06T12:34:20.430532+0000","last_became_active":"2023-10-06T12:34:20.426907+0000","last_became_peered":"2023-10-06T12:34:20.426907+0000","last_unstale":"2023-10-06T12:34:20.430532+0000","last_undegraded":"2023-10-06T12:34:20.430532+0000","last_fullsized":"2023-10-06T12:34:20.430532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:12:54.100539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447199+0000","last_change":"2023-10-06T12:34:19.421541+0000","last_active":"2023-10-06T12:34:20.447199+0000","last_peered":"2023-10-06T12:34:20.447199+0000","last_clean":"2023-10-06T12:34:20.447199+0000","last_became_active":"2023-10-06T12:34:19.421177+0000","last_became_peered":"2023-10-06T12:34:19.421177+0000","last_unstale":"2023-10-06T12:34:20.447199+0000","last_undegraded":"2023-10-06T12:34:20.447199+0000","last_fullsized":"2023-10-06T12:34:20.447199+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:57.760611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.430939+0000","last_change":"2023-10-06T12:34:20.427193+0000","last_active":"2023-10-06T12:34:20.430939+0000","last_peered":"2023-10-06T12:34:20.430939+0000","last_clean":"2023-10-06T12:34:20.430939+0000","last_became_active":"2023-10-06T12:34:20.426860+0000","last_became_peered":"2023-10-06T12:34:20.426860+0000","last_unstale":"2023-10-06T12:34:20.430939+0000","last_undegraded":"2023-10-06T12:34:20.430939+0000","last_fullsized":"2023-10-06T12:34:20.430939+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:28:56.417675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.424211+0000","last_change":"2023-10-06T12:34:19.428875+0000","last_active":"2023-10-06T12:34:20.424211+0000","last_peered":"2023-10-06T12:34:20.424211+0000","last_clean":"2023-10-06T12:34:20.424211+0000","last_became_active":"2023-10-06T12:34:19.428456+0000","last_became_peered":"2023-10-06T12:34:19.428456+0000","last_unstale":"2023-10-06T12:34:20.424211+0000","last_undegraded":"2023-10-06T12:34:20.424211+0000","last_fullsized":"2023-10-06T12:34:20.424211+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:16.366399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436152+0000","last_change":"2023-10-06T12:34:20.433039+0000","last_active":"2023-10-06T12:34:20.436152+0000","last_peered":"2023-10-06T12:34:20.436152+0000","last_clean":"2023-10-06T12:34:20.436152+0000","last_became_active":"2023-10-06T12:34:20.432658+0000","last_became_peered":"2023-10-06T12:34:20.432658+0000","last_unstale":"2023-10-06T12:34:20.436152+0000","last_undegraded":"2023-10-06T12:34:20.436152+0000","last_fullsized":"2023-10-06T12:34:20.436152+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:46.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447115+0000","last_change":"2023-10-06T12:34:19.416622+0000","last_active":"2023-10-06T12:34:20.447115+0000","last_peered":"2023-10-06T12:34:20.447115+0000","last_clean":"2023-10-06T12:34:20.447115+0000","last_became_active":"2023-10-06T12:34:19.416257+0000","last_became_peered":"2023-10-06T12:34:19.416257+0000","last_unstale":"2023-10-06T12:34:20.447115+0000","last_undegraded":"2023-10-06T12:34:20.447115+0000","last_fullsized":"2023-10-06T12:34:20.447115+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:20:50.547836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443946+0000","last_change":"2023-10-06T12:34:20.439834+0000","last_active":"2023-10-06T12:34:20.443946+0000","last_peered":"2023-10-06T12:34:20.443946+0000","last_clean":"2023-10-06T12:34:20.443946+0000","last_became_active":"2023-10-06T12:34:20.439409+0000","last_became_peered":"2023-10-06T12:34:20.439409+0000","last_unstale":"2023-10-06T12:34:20.443946+0000","last_undegraded":"2023-10-06T12:34:20.443946+0000","last_fullsized":"2023-10-06T12:34:20.443946+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:47:35.885220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.411360+0000","last_change":"2023-10-06T12:34:19.418254+0000","last_active":"2023-10-06T12:34:20.411360+0000","last_peered":"2023-10-06T12:34:20.411360+0000","last_clean":"2023-10-06T12:34:20.411360+0000","last_became_active":"2023-10-06T12:34:19.417955+0000","last_became_peered":"2023-10-06T12:34:19.417955+0000","last_unstale":"2023-10-06T12:34:20.411360+0000","last_undegraded":"2023-10-06T12:34:20.411360+0000","last_fullsized":"2023-10-06T12:34:20.411360+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:38.211096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.442744+0000","last_change":"2023-10-06T12:34:20.433719+0000","last_active":"2023-10-06T12:34:20.442744+0000","last_peered":"2023-10-06T12:34:20.442744+0000","last_clean":"2023-10-06T12:34:20.442744+0000","last_became_active":"2023-10-06T12:34:20.432668+0000","last_became_peered":"2023-10-06T12:34:20.432668+0000","last_unstale":"2023-10-06T12:34:20.442744+0000","last_undegraded":"2023-10-06T12:34:20.442744+0000","last_fullsized":"2023-10-06T12:34:20.442744+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:05.790077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.417313+0000","last_change":"2023-10-06T12:34:19.415923+0000","last_active":"2023-10-06T12:34:20.417313+0000","last_peered":"2023-10-06T12:34:20.417313+0000","last_clean":"2023-10-06T12:34:20.417313+0000","last_became_active":"2023-10-06T12:34:19.415600+0000","last_became_peered":"2023-10-06T12:34:19.415600+0000","last_unstale":"2023-10-06T12:34:20.417313+0000","last_undegraded":"2023-10-06T12:34:20.417313+0000","last_fullsized":"2023-10-06T12:34:20.417313+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:10.481688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443766+0000","last_change":"2023-10-06T12:34:20.435419+0000","last_active":"2023-10-06T12:34:20.443766+0000","last_peered":"2023-10-06T12:34:20.443766+0000","last_clean":"2023-10-06T12:34:20.443766+0000","last_became_active":"2023-10-06T12:34:20.435038+0000","last_became_peered":"2023-10-06T12:34:20.435038+0000","last_unstale":"2023-10-06T12:34:20.443766+0000","last_undegraded":"2023-10-06T12:34:20.443766+0000","last_fullsized":"2023-10-06T12:34:20.443766+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:36.742910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.412864+0000","last_change":"2023-10-06T12:34:19.420179+0000","last_active":"2023-10-06T12:34:20.412864+0000","last_peered":"2023-10-06T12:34:20.412864+0000","last_clean":"2023-10-06T12:34:20.412864+0000","last_became_active":"2023-10-06T12:34:19.419880+0000","last_became_peered":"2023-10-06T12:34:19.419880+0000","last_unstale":"2023-10-06T12:34:20.412864+0000","last_undegraded":"2023-10-06T12:34:20.412864+0000","last_fullsized":"2023-10-06T12:34:20.412864+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:32.008217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.431820+0000","last_change":"2023-10-06T12:34:20.419350+0000","last_active":"2023-10-06T12:34:20.431820+0000","last_peered":"2023-10-06T12:34:20.431820+0000","last_clean":"2023-10-06T12:34:20.431820+0000","last_became_active":"2023-10-06T12:34:20.417735+0000","last_became_peered":"2023-10-06T12:34:20.417735+0000","last_unstale":"2023-10-06T12:34:20.431820+0000","last_undegraded":"2023-10-06T12:34:20.431820+0000","last_fullsized":"2023-10-06T12:34:20.431820+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:05.394041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448311+0000","last_change":"2023-10-06T12:34:19.425948+0000","last_active":"2023-10-06T12:34:20.448311+0000","last_peered":"2023-10-06T12:34:20.448311+0000","last_clean":"2023-10-06T12:34:20.448311+0000","last_became_active":"2023-10-06T12:34:19.425654+0000","last_became_peered":"2023-10-06T12:34:19.425654+0000","last_unstale":"2023-10-06T12:34:20.448311+0000","last_undegraded":"2023-10-06T12:34:20.448311+0000","last_fullsized":"2023-10-06T12:34:20.448311+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:58.748685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444898+0000","last_change":"2023-10-06T12:34:20.438189+0000","last_active":"2023-10-06T12:34:20.444898+0000","last_peered":"2023-10-06T12:34:20.444898+0000","last_clean":"2023-10-06T12:34:20.444898+0000","last_became_active":"2023-10-06T12:34:20.437869+0000","last_became_peered":"2023-10-06T12:34:20.437869+0000","last_unstale":"2023-10-06T12:34:20.444898+0000","last_undegraded":"2023-10-06T12:34:20.444898+0000","last_fullsized":"2023-10-06T12:34:20.444898+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:32.240723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.414130+0000","last_change":"2023-10-06T12:34:19.425054+0000","last_active":"2023-10-06T12:34:20.414130+0000","last_peered":"2023-10-06T12:34:20.414130+0000","last_clean":"2023-10-06T12:34:20.414130+0000","last_became_active":"2023-10-06T12:34:19.424749+0000","last_became_peered":"2023-10-06T12:34:19.424749+0000","last_unstale":"2023-10-06T12:34:20.414130+0000","last_undegraded":"2023-10-06T12:34:20.414130+0000","last_fullsized":"2023-10-06T12:34:20.414130+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:36.277482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.427434+0000","last_change":"2023-10-06T12:34:20.423311+0000","last_active":"2023-10-06T12:34:20.427434+0000","last_peered":"2023-10-06T12:34:20.427434+0000","last_clean":"2023-10-06T12:34:20.427434+0000","last_became_active":"2023-10-06T12:34:20.422961+0000","last_became_peered":"2023-10-06T12:34:20.422961+0000","last_unstale":"2023-10-06T12:34:20.427434+0000","last_undegraded":"2023-10-06T12:34:20.427434+0000","last_fullsized":"2023-10-06T12:34:20.427434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:52.429307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421828+0000","last_change":"2023-10-06T12:34:19.427583+0000","last_active":"2023-10-06T12:34:20.421828+0000","last_peered":"2023-10-06T12:34:20.421828+0000","last_clean":"2023-10-06T12:34:20.421828+0000","last_became_active":"2023-10-06T12:34:19.427143+0000","last_became_peered":"2023-10-06T12:34:19.427143+0000","last_unstale":"2023-10-06T12:34:20.421828+0000","last_undegraded":"2023-10-06T12:34:20.421828+0000","last_fullsized":"2023-10-06T12:34:20.421828+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:30:56.234420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.437048+0000","last_change":"2023-10-06T12:34:20.428349+0000","last_active":"2023-10-06T12:34:20.437048+0000","last_peered":"2023-10-06T12:34:20.437048+0000","last_clean":"2023-10-06T12:34:20.437048+0000","last_became_active":"2023-10-06T12:34:20.427988+0000","last_became_peered":"2023-10-06T12:34:20.427988+0000","last_unstale":"2023-10-06T12:34:20.437048+0000","last_undegraded":"2023-10-06T12:34:20.437048+0000","last_fullsized":"2023-10-06T12:34:20.437048+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:08.249347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.411690+0000","last_change":"2023-10-06T12:34:19.428160+0000","last_active":"2023-10-06T12:34:20.411690+0000","last_peered":"2023-10-06T12:34:20.411690+0000","last_clean":"2023-10-06T12:34:20.411690+0000","last_became_active":"2023-10-06T12:34:19.427852+0000","last_became_peered":"2023-10-06T12:34:19.427852+0000","last_unstale":"2023-10-06T12:34:20.411690+0000","last_undegraded":"2023-10-06T12:34:20.411690+0000","last_fullsized":"2023-10-06T12:34:20.411690+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:27.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444478+0000","last_change":"2023-10-06T12:34:20.434453+0000","last_active":"2023-10-06T12:34:20.444478+0000","last_peered":"2023-10-06T12:34:20.444478+0000","last_clean":"2023-10-06T12:34:20.444478+0000","last_became_active":"2023-10-06T12:34:20.434112+0000","last_became_peered":"2023-10-06T12:34:20.434112+0000","last_unstale":"2023-10-06T12:34:20.444478+0000","last_undegraded":"2023-10-06T12:34:20.444478+0000","last_fullsized":"2023-10-06T12:34:20.444478+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:01.509365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413490+0000","last_change":"2023-10-06T12:34:19.419962+0000","last_active":"2023-10-06T12:34:20.413490+0000","last_peered":"2023-10-06T12:34:20.413490+0000","last_clean":"2023-10-06T12:34:20.413490+0000","last_became_active":"2023-10-06T12:34:19.419675+0000","last_became_peered":"2023-10-06T12:34:19.419675+0000","last_unstale":"2023-10-06T12:34:20.413490+0000","last_undegraded":"2023-10-06T12:34:20.413490+0000","last_fullsized":"2023-10-06T12:34:20.413490+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:00.505407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444932+0000","last_change":"2023-10-06T12:34:20.438070+0000","last_active":"2023-10-06T12:34:20.444932+0000","last_peered":"2023-10-06T12:34:20.444932+0000","last_clean":"2023-10-06T12:34:20.444932+0000","last_became_active":"2023-10-06T12:34:20.437752+0000","last_became_peered":"2023-10-06T12:34:20.437752+0000","last_unstale":"2023-10-06T12:34:20.444932+0000","last_undegraded":"2023-10-06T12:34:20.444932+0000","last_fullsized":"2023-10-06T12:34:20.444932+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:08.404117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.419912+0000","last_change":"2023-10-06T12:34:19.425217+0000","last_active":"2023-10-06T12:34:20.419912+0000","last_peered":"2023-10-06T12:34:20.419912+0000","last_clean":"2023-10-06T12:34:20.419912+0000","last_became_active":"2023-10-06T12:34:19.424678+0000","last_became_peered":"2023-10-06T12:34:19.424678+0000","last_unstale":"2023-10-06T12:34:20.419912+0000","last_undegraded":"2023-10-06T12:34:20.419912+0000","last_fullsized":"2023-10-06T12:34:20.419912+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:14.496639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443982+0000","last_change":"2023-10-06T12:34:20.438856+0000","last_active":"2023-10-06T12:34:20.443982+0000","last_peered":"2023-10-06T12:34:20.443982+0000","last_clean":"2023-10-06T12:34:20.443982+0000","last_became_active":"2023-10-06T12:34:20.438513+0000","last_became_peered":"2023-10-06T12:34:20.438513+0000","last_unstale":"2023-10-06T12:34:20.443982+0000","last_undegraded":"2023-10-06T12:34:20.443982+0000","last_fullsized":"2023-10-06T12:34:20.443982+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:11.990001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413395+0000","last_change":"2023-10-06T12:34:19.420615+0000","last_active":"2023-10-06T12:34:20.413395+0000","last_peered":"2023-10-06T12:34:20.413395+0000","last_clean":"2023-10-06T12:34:20.413395+0000","last_became_active":"2023-10-06T12:34:19.420299+0000","last_became_peered":"2023-10-06T12:34:19.420299+0000","last_unstale":"2023-10-06T12:34:20.413395+0000","last_undegraded":"2023-10-06T12:34:20.413395+0000","last_fullsized":"2023-10-06T12:34:20.413395+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:49.887772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447985+0000","last_change":"2023-10-06T12:34:20.441584+0000","last_active":"2023-10-06T12:34:20.447985+0000","last_peered":"2023-10-06T12:34:20.447985+0000","last_clean":"2023-10-06T12:34:20.447985+0000","last_became_active":"2023-10-06T12:34:20.441086+0000","last_became_peered":"2023-10-06T12:34:20.441086+0000","last_unstale":"2023-10-06T12:34:20.447985+0000","last_undegraded":"2023-10-06T12:34:20.447985+0000","last_fullsized":"2023-10-06T12:34:20.447985+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:05:05.790077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421596+0000","last_change":"2023-10-06T12:34:19.423439+0000","last_active":"2023-10-06T12:34:20.421596+0000","last_peered":"2023-10-06T12:34:20.421596+0000","last_clean":"2023-10-06T12:34:20.421596+0000","last_became_active":"2023-10-06T12:34:19.423041+0000","last_became_peered":"2023-10-06T12:34:19.423041+0000","last_unstale":"2023-10-06T12:34:20.421596+0000","last_undegraded":"2023-10-06T12:34:20.421596+0000","last_fullsized":"2023-10-06T12:34:20.421596+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:47:31.236765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448002+0000","last_change":"2023-10-06T12:34:20.443798+0000","last_active":"2023-10-06T12:34:20.448002+0000","last_peered":"2023-10-06T12:34:20.448002+0000","last_clean":"2023-10-06T12:34:20.448002+0000","last_became_active":"2023-10-06T12:34:20.443473+0000","last_became_peered":"2023-10-06T12:34:20.443473+0000","last_unstale":"2023-10-06T12:34:20.448002+0000","last_undegraded":"2023-10-06T12:34:20.448002+0000","last_fullsized":"2023-10-06T12:34:20.448002+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:15:13.185350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"165'6","reported_seq":16,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.451775+0000","last_change":"2023-10-06T12:34:19.416197+0000","last_active":"2023-10-06T12:34:20.451775+0000","last_peered":"2023-10-06T12:34:20.451775+0000","last_clean":"2023-10-06T12:34:20.451775+0000","last_became_active":"2023-10-06T12:34:19.415826+0000","last_became_peered":"2023-10-06T12:34:19.415826+0000","last_unstale":"2023-10-06T12:34:20.451775+0000","last_undegraded":"2023-10-06T12:34:20.451775+0000","last_fullsized":"2023-10-06T12:34:20.451775+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:25.333862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"155'293","reported_seq":909,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421777+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:34:20.421777+0000","last_peered":"2023-10-06T12:34:20.421777+0000","last_clean":"2023-10-06T12:34:20.421777+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:34:20.421777+0000","last_undegraded":"2023-10-06T12:34:20.421777+0000","last_fullsized":"2023-10-06T12:34:20.421777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":151,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413593+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:34:20.413593+0000","last_peered":"2023-10-06T12:34:20.413593+0000","last_clean":"2023-10-06T12:34:20.413593+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:34:20.413593+0000","last_undegraded":"2023-10-06T12:34:20.413593+0000","last_fullsized":"2023-10-06T12:34:20.413593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.448119+0000","last_change":"2023-10-06T12:34:20.439197+0000","last_active":"2023-10-06T12:34:20.448119+0000","last_peered":"2023-10-06T12:34:20.448119+0000","last_clean":"2023-10-06T12:34:20.448119+0000","last_became_active":"2023-10-06T12:34:20.438899+0000","last_became_peered":"2023-10-06T12:34:20.438899+0000","last_unstale":"2023-10-06T12:34:20.448119+0000","last_undegraded":"2023-10-06T12:34:20.448119+0000","last_fullsized":"2023-10-06T12:34:20.448119+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:37.281440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.425638+0000","last_change":"2023-10-06T12:34:19.424475+0000","last_active":"2023-10-06T12:34:20.425638+0000","last_peered":"2023-10-06T12:34:20.425638+0000","last_clean":"2023-10-06T12:34:20.425638+0000","last_became_active":"2023-10-06T12:34:19.424053+0000","last_became_peered":"2023-10-06T12:34:19.424053+0000","last_unstale":"2023-10-06T12:34:20.425638+0000","last_undegraded":"2023-10-06T12:34:20.425638+0000","last_fullsized":"2023-10-06T12:34:20.425638+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:04.390083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":132,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.421682+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:34:20.421682+0000","last_peered":"2023-10-06T12:34:20.421682+0000","last_clean":"2023-10-06T12:34:20.421682+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:34:20.421682+0000","last_undegraded":"2023-10-06T12:34:20.421682+0000","last_fullsized":"2023-10-06T12:34:20.421682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443192+0000","last_change":"2023-10-06T12:34:20.437936+0000","last_active":"2023-10-06T12:34:20.443192+0000","last_peered":"2023-10-06T12:34:20.443192+0000","last_clean":"2023-10-06T12:34:20.443192+0000","last_became_active":"2023-10-06T12:34:20.437636+0000","last_became_peered":"2023-10-06T12:34:20.437636+0000","last_unstale":"2023-10-06T12:34:20.443192+0000","last_undegraded":"2023-10-06T12:34:20.443192+0000","last_fullsized":"2023-10-06T12:34:20.443192+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:17.808471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.414081+0000","last_change":"2023-10-06T12:34:19.424907+0000","last_active":"2023-10-06T12:34:20.414081+0000","last_peered":"2023-10-06T12:34:20.414081+0000","last_clean":"2023-10-06T12:34:20.414081+0000","last_became_active":"2023-10-06T12:34:19.424545+0000","last_became_peered":"2023-10-06T12:34:19.424545+0000","last_unstale":"2023-10-06T12:34:20.414081+0000","last_undegraded":"2023-10-06T12:34:20.414081+0000","last_fullsized":"2023-10-06T12:34:20.414081+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:26:23.217707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.418628+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:34:20.418628+0000","last_peered":"2023-10-06T12:34:20.418628+0000","last_clean":"2023-10-06T12:34:20.418628+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:34:20.418628+0000","last_undegraded":"2023-10-06T12:34:20.418628+0000","last_fullsized":"2023-10-06T12:34:20.418628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.441638+0000","last_change":"2023-10-06T12:34:20.430929+0000","last_active":"2023-10-06T12:34:20.441638+0000","last_peered":"2023-10-06T12:34:20.441638+0000","last_clean":"2023-10-06T12:34:20.441638+0000","last_became_active":"2023-10-06T12:34:20.430193+0000","last_became_peered":"2023-10-06T12:34:20.430193+0000","last_unstale":"2023-10-06T12:34:20.441638+0000","last_undegraded":"2023-10-06T12:34:20.441638+0000","last_fullsized":"2023-10-06T12:34:20.441638+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:19.586310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"165'3","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443139+0000","last_change":"2023-10-06T12:34:19.423221+0000","last_active":"2023-10-06T12:34:20.443139+0000","last_peered":"2023-10-06T12:34:20.443139+0000","last_clean":"2023-10-06T12:34:20.443139+0000","last_became_active":"2023-10-06T12:34:19.422927+0000","last_became_peered":"2023-10-06T12:34:19.422927+0000","last_unstale":"2023-10-06T12:34:20.443139+0000","last_undegraded":"2023-10-06T12:34:20.443139+0000","last_fullsized":"2023-10-06T12:34:20.443139+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:59:53.862331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.413783+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:34:20.413783+0000","last_peered":"2023-10-06T12:34:20.413783+0000","last_clean":"2023-10-06T12:34:20.413783+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:34:20.413783+0000","last_undegraded":"2023-10-06T12:34:20.413783+0000","last_fullsized":"2023-10-06T12:34:20.413783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.447935+0000","last_change":"2023-10-06T12:34:20.439172+0000","last_active":"2023-10-06T12:34:20.447935+0000","last_peered":"2023-10-06T12:34:20.447935+0000","last_clean":"2023-10-06T12:34:20.447935+0000","last_became_active":"2023-10-06T12:34:20.438727+0000","last_became_peered":"2023-10-06T12:34:20.438727+0000","last_unstale":"2023-10-06T12:34:20.447935+0000","last_undegraded":"2023-10-06T12:34:20.447935+0000","last_fullsized":"2023-10-06T12:34:20.447935+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:08.249347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"165'2","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450357+0000","last_change":"2023-10-06T12:34:19.421713+0000","last_active":"2023-10-06T12:34:20.450357+0000","last_peered":"2023-10-06T12:34:20.450357+0000","last_clean":"2023-10-06T12:34:20.450357+0000","last_became_active":"2023-10-06T12:34:19.421399+0000","last_became_peered":"2023-10-06T12:34:19.421399+0000","last_unstale":"2023-10-06T12:34:20.450357+0000","last_undegraded":"2023-10-06T12:34:20.450357+0000","last_fullsized":"2023-10-06T12:34:20.450357+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:58.813048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.420207+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:34:20.420207+0000","last_peered":"2023-10-06T12:34:20.420207+0000","last_clean":"2023-10-06T12:34:20.420207+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:34:20.420207+0000","last_undegraded":"2023-10-06T12:34:20.420207+0000","last_fullsized":"2023-10-06T12:34:20.420207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.443967+0000","last_change":"2023-10-06T12:34:20.440059+0000","last_active":"2023-10-06T12:34:20.443967+0000","last_peered":"2023-10-06T12:34:20.443967+0000","last_clean":"2023-10-06T12:34:20.443967+0000","last_became_active":"2023-10-06T12:34:20.439495+0000","last_became_peered":"2023-10-06T12:34:20.439495+0000","last_unstale":"2023-10-06T12:34:20.443967+0000","last_undegraded":"2023-10-06T12:34:20.443967+0000","last_fullsized":"2023-10-06T12:34:20.443967+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:59.752643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423942+0000","last_change":"2023-10-06T12:34:19.426392+0000","last_active":"2023-10-06T12:34:20.423942+0000","last_peered":"2023-10-06T12:34:20.423942+0000","last_clean":"2023-10-06T12:34:20.423942+0000","last_became_active":"2023-10-06T12:34:19.426104+0000","last_became_peered":"2023-10-06T12:34:19.426104+0000","last_unstale":"2023-10-06T12:34:20.423942+0000","last_undegraded":"2023-10-06T12:34:20.423942+0000","last_fullsized":"2023-10-06T12:34:20.423942+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:10.986043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":132,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.425245+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:34:20.425245+0000","last_peered":"2023-10-06T12:34:20.425245+0000","last_clean":"2023-10-06T12:34:20.425245+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:34:20.425245+0000","last_undegraded":"2023-10-06T12:34:20.425245+0000","last_fullsized":"2023-10-06T12:34:20.425245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.436628+0000","last_change":"2023-10-06T12:34:20.428960+0000","last_active":"2023-10-06T12:34:20.436628+0000","last_peered":"2023-10-06T12:34:20.436628+0000","last_clean":"2023-10-06T12:34:20.436628+0000","last_became_active":"2023-10-06T12:34:20.428677+0000","last_became_peered":"2023-10-06T12:34:20.428677+0000","last_unstale":"2023-10-06T12:34:20.436628+0000","last_undegraded":"2023-10-06T12:34:20.436628+0000","last_fullsized":"2023-10-06T12:34:20.436628+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:14:30.094495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423744+0000","last_change":"2023-10-06T12:34:19.421945+0000","last_active":"2023-10-06T12:34:20.423744+0000","last_peered":"2023-10-06T12:34:20.423744+0000","last_clean":"2023-10-06T12:34:20.423744+0000","last_became_active":"2023-10-06T12:34:19.421628+0000","last_became_peered":"2023-10-06T12:34:19.421628+0000","last_unstale":"2023-10-06T12:34:20.423744+0000","last_undegraded":"2023-10-06T12:34:20.423744+0000","last_fullsized":"2023-10-06T12:34:20.423744+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:27.320601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.423047+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:34:20.423047+0000","last_peered":"2023-10-06T12:34:20.423047+0000","last_clean":"2023-10-06T12:34:20.423047+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:34:20.423047+0000","last_undegraded":"2023-10-06T12:34:20.423047+0000","last_fullsized":"2023-10-06T12:34:20.423047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.444680+0000","last_change":"2023-10-06T12:34:20.441203+0000","last_active":"2023-10-06T12:34:20.444680+0000","last_peered":"2023-10-06T12:34:20.444680+0000","last_clean":"2023-10-06T12:34:20.444680+0000","last_became_active":"2023-10-06T12:34:20.440933+0000","last_became_peered":"2023-10-06T12:34:20.440933+0000","last_unstale":"2023-10-06T12:34:20.444680+0000","last_undegraded":"2023-10-06T12:34:20.444680+0000","last_fullsized":"2023-10-06T12:34:20.444680+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:19.385274+0000","last_clean_scrub_stamp":"2023-10-06T12:34:19.385274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:27:17.808471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"165'8","reported_seq":18,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.450064+0000","last_change":"2023-10-06T12:34:19.418654+0000","last_active":"2023-10-06T12:34:20.450064+0000","last_peered":"2023-10-06T12:34:20.450064+0000","last_clean":"2023-10-06T12:34:20.450064+0000","last_became_active":"2023-10-06T12:34:19.418251+0000","last_became_peered":"2023-10-06T12:34:19.418251+0000","last_unstale":"2023-10-06T12:34:20.450064+0000","last_undegraded":"2023-10-06T12:34:20.450064+0000","last_fullsized":"2023-10-06T12:34:20.450064+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:34:18.381316+0000","last_clean_scrub_stamp":"2023-10-06T12:34:18.381316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:07.245389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-06T12:34:20.424380+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:34:20.424380+0000","last_peered":"2023-10-06T12:34:20.424380+0000","last_clean":"2023-10-06T12:34:20.424380+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:34:20.424380+0000","last_undegraded":"2023-10-06T12:34:20.424380+0000","last_fullsized":"2023-10-06T12:34:20.424380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575756,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299896,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071944,"statfs":{"total":96636764160,"available":96329670656,"internally_reserved":0,"allocated":2670592,"data_stored":1681537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.60599999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.202,"15min":1.175},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.64000000000000001},"max":{"1min":1.6659999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.266}]},{"osd":1,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.359,"15min":1.363},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.68500000000000005},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.476},{"interface":"front","average":{"1min":1.21,"5min":1.268,"15min":1.282},"min":{"1min":0.875,"5min":0.875,"15min":0.85599999999999998},"max":{"1min":1.5880000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.84699999999999998,"5min":0.78200000000000003,"15min":0.74299999999999999},"max":{"1min":1.7370000000000001,"5min":1.855,"15min":1.855},"last":1.216},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2569999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.65500000000000003},"max":{"1min":1.726,"5min":1.726,"15min":1.7649999999999999},"last":1.083}]},{"osd":3,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3140000000000001,"15min":1.2949999999999999},"min":{"1min":0.78800000000000003,"5min":0.77400000000000002,"15min":0.71699999999999997},"max":{"1min":1.7070000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.70399999999999996},"max":{"1min":1.679,"5min":1.7130000000000001,"15min":1.9019999999999999},"last":1.381}]},{"osd":4,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.77400000000000002,"5min":0.52800000000000002,"15min":0.46999999999999997},"max":{"1min":1.573,"5min":1.7390000000000001,"15min":1.7410000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.2769999999999999,"15min":1.252},"min":{"1min":0.89800000000000002,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.8200000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.92100000000000004,"5min":0.82199999999999995,"15min":0.49399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.853},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3260000000000001,"15min":1.3109999999999999},"min":{"1min":0.88500000000000001,"5min":0.81599999999999995,"15min":0.55700000000000005},"max":{"1min":1.702,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:33:54 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.393,"15min":1.375},"min":{"1min":1.081,"5min":0.91900000000000004,"15min":0.65100000000000002},"max":{"1min":1.766,"5min":2.0910000000000002,"15min":2.0990000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4430000000000001,"15min":1.415},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.54000000000000004},"max":{"1min":1.895,"5min":2.0499999999999998,"15min":2.073},"last":1.3340000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575756,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311604,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060236,"statfs":{"total":96636764160,"available":96317681664,"internally_reserved":0,"allocated":2076672,"data_stored":1091217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3049999999999999,"15min":1.323},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6579999999999999,"5min":1.724,"15min":1.7889999999999999},"last":1.214}]},{"osd":1,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.1870000000000001,"15min":1.1839999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.51900000000000002},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.157,"5min":1.1910000000000001,"15min":1.206},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5800000000000001,"5min":1.76,"15min":1.76},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2290000000000001,"15min":1.2170000000000001},"min":{"1min":1.024,"5min":0.80800000000000005,"15min":0.69899999999999995},"max":{"1min":3.9159999999999999,"5min":3.9159999999999999,"15min":3.9159999999999999},"last":1.131},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.222,"15min":1.238},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.679,"5min":1.7509999999999999,"15min":1.863},"last":0.99299999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3009999999999999,"15min":1.3120000000000001},"min":{"1min":1.121,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.732,"5min":1.732,"15min":1.7709999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.353,"15min":1.329},"min":{"1min":1.008,"5min":0.95999999999999996,"15min":0.77200000000000002},"max":{"1min":1.833,"5min":1.833,"15min":1.847},"last":1.014}]},{"osd":4,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.202,"15min":1.1890000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.9139999999999999},"last":1.427},{"interface":"front","average":{"1min":1.252,"5min":1.24,"15min":1.248},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":1.8620000000000001,"15min":2.0449999999999999},"last":1.304}]},{"osd":5,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2210000000000001,"15min":1.2350000000000001},"min":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.47199999999999998},"max":{"1min":1.883,"5min":1.883,"15min":1.9770000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.353,"15min":1.3580000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7909999999999999,"5min":1.806,"15min":1.901},"last":1.26}]},{"osd":7,"last update":"Fri Oct 6 12:33:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4279999999999999,"15min":1.4470000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.802,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.421,"15min":1.4259999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6950000000000001,"5min":1.905,"15min":2.226},"last":1.0660000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307504,"kb_used_data":2024,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064336,"statfs":{"total":96636764160,"available":96321880064,"internally_reserved":0,"allocated":2072576,"data_stored":1091147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3280000000000001,"15min":1.304},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6200000000000001,"5min":1.6379999999999999,"15min":1.885},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.81000000000000005},"max":{"1min":1.579,"5min":1.728,"15min":1.728},"last":1.6399999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2070000000000001,"15min":1.196},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69599999999999995},"max":{"1min":1.546,"5min":1.758,"15min":1.758},"last":1.004},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.177},"min":{"1min":0.75600000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"max":{"1min":1.579,"5min":1.6459999999999999,"15min":1.6659999999999999},"last":1.6899999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.2190000000000001},"min":{"1min":0.86099999999999999,"5min":0.80300000000000005,"15min":0.68999999999999995},"max":{"1min":1.5820000000000001,"5min":1.5880000000000001,"15min":1.996},"last":1.863},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.276,"15min":1.25},"min":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":0.76500000000000001},"max":{"1min":1.599,"5min":1.6890000000000001,"15min":1.7909999999999999},"last":1.242}]},{"osd":3,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.86499999999999999,"15min":0.82499999999999996},"max":{"1min":1.6930000000000001,"5min":1.712,"15min":1.8939999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.3049999999999999,"15min":1.306},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.82399999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.7509999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.274,"15min":1.268},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.53600000000000003},"max":{"1min":1.649,"5min":1.6859999999999999,"15min":1.716},"last":1.385},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3839999999999999},"min":{"1min":1.0609999999999999,"5min":0.623,"15min":0.623},"max":{"1min":1.8169999999999999,"5min":1.835,"15min":1.835},"last":1.409}]},{"osd":6,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2849999999999999,"15min":1.244},"min":{"1min":0.97099999999999997,"5min":0.65400000000000003,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.98099999999999998,"5min":0.748,"15min":0.748},"max":{"1min":1.671,"5min":1.919,"15min":2.145},"last":1.3089999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.355,"15min":1.325},"min":{"1min":0.92100000000000004,"5min":0.73399999999999999,"15min":0.59899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7929999999999999,"15min":1.863},"last":1.272},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3999999999999999},"min":{"1min":1.0669999999999999,"5min":0.80300000000000005,"15min":0.754},"max":{"1min":1.7589999999999999,"5min":2.0219999999999998,"15min":2.0979999999999999},"last":1.1619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575757,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1092683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5109999999999999,"5min":1.5800000000000001,"15min":1.8839999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.167,"15min":1.1890000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.538,"5min":1.887,"15min":2.0720000000000001},"last":1.232}]},{"osd":1,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.365},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6240000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.325,"15min":1.329},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5640000000000001,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3460000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.204,"15min":1.2390000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.506,"5min":1.625,"15min":1.9330000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.619,"5min":1.861,"15min":1.861},"last":1.29}]},{"osd":3,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2370000000000001,"15min":1.2490000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5429999999999999,"5min":1.5980000000000001,"15min":1.9610000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6339999999999999,"5min":1.7030000000000001,"15min":1.8340000000000001},"last":1.3069999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1919999999999999,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.56999999999999995},"max":{"1min":1.552,"5min":1.7929999999999999,"15min":1.944},"last":1.127},{"interface":"front","average":{"1min":1.163,"5min":1.1819999999999999,"15min":1.2070000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.819},"last":1.071}]},{"osd":6,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.22,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.64200000000000002,"15min":0.63500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.984},"last":1.411},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.3680000000000001,"15min":1.4019999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68799999999999994},"max":{"1min":1.754,"5min":1.9850000000000001,"15min":2.0089999999999999},"last":1.379}]},{"osd":7,"last update":"Fri Oct 6 12:33:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.339,"15min":1.3620000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.266},{"interface":"front","average":{"1min":1.397,"5min":1.399,"15min":1.4359999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.69399999999999995},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.1030000000000002},"last":1.5589999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315704,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056136,"statfs":{"total":96636764160,"available":96313483264,"internally_reserved":0,"allocated":2080768,"data_stored":1091735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.376},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.999,"15min":1.0089999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.311},"max":{"1min":1.3089999999999999,"5min":1.5800000000000001,"15min":1.7210000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0249999999999999,"15min":1.0029999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.52500000000000002},"max":{"1min":1.258,"5min":1.6100000000000001,"15min":1.8},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.54500000000000004},"max":{"1min":1.3580000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.177,"15min":1.1439999999999999},"min":{"1min":0.80300000000000005,"5min":0.70399999999999996,"15min":0.56899999999999995},"max":{"1min":1.623,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.252,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.59199999999999997},"max":{"1min":1.508,"5min":1.649,"15min":2.379},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3029999999999999,"15min":1.27},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.498},"max":{"1min":1.714,"5min":1.8700000000000001,"15min":2.02},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4410000000000001,"15min":1.3979999999999999},"min":{"1min":1.026,"5min":1.026,"15min":0.72499999999999998},"max":{"1min":1.8959999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.1990000000000001,"5min":1.0740000000000001,"15min":0.76900000000000002},"max":{"1min":1.784,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.747},{"interface":"front","average":{"1min":1.419,"5min":1.4390000000000001,"15min":1.419},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.518,"15min":1.4830000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.67600000000000005},"max":{"1min":1.907,"5min":2.0270000000000001,"15min":2.0449999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4510000000000001,"15min":1.3979999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.7649999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Oct 6 12:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.468},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7430000000000001,"5min":1.9339999999999999,"15min":1.9870000000000001},"last":0.88},{"interface":"front","average":{"1min":1.625,"5min":1.5780000000000001,"15min":1.5429999999999999},"min":{"1min":1.2949999999999999,"5min":1.1679999999999999,"15min":0.80000000000000004},"max":{"1min":1.9690000000000001,"5min":2.048,"15min":2.0600000000000001},"last":0.85699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575756,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311608,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060232,"statfs":{"total":96636764160,"available":96317677568,"internally_reserved":0,"allocated":2080768,"data_stored":1091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.0069999999999999,"15min":0.97899999999999998},"min":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.50700000000000001},"max":{"1min":1.29,"5min":1.649,"15min":1.845},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.05,"5min":1.1120000000000001,"15min":1.125},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.64200000000000002},"max":{"1min":1.53,"5min":1.649,"15min":1.649},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.92100000000000004},"min":{"1min":0.72899999999999998,"5min":0.66100000000000003,"15min":0.59499999999999997},"max":{"1min":1.1599999999999999,"5min":1.4350000000000001,"15min":1.5580000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.006,"5min":1.0429999999999999,"15min":1.0620000000000001},"min":{"1min":0.76900000000000002,"5min":0.69699999999999995,"15min":0.67500000000000004},"max":{"1min":1.206,"5min":1.6080000000000001,"15min":2.8570000000000002},"last":0.79100000000000004}]},{"osd":3,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.206,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.593,"5min":1.6779999999999999,"15min":2.6309999999999998},"last":1.091},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3580000000000001,"15min":1.383},"min":{"1min":0.93200000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.673,"5min":1.8640000000000001,"15min":2.9700000000000002},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.256,"15min":1.268},"min":{"1min":1.0660000000000001,"5min":0.83499999999999996,"15min":0.77400000000000002},"max":{"1min":1.4690000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.351},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.331},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":1.6759999999999999,"5min":1.718,"15min":2.4849999999999999},"last":1.3939999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.377},"min":{"1min":1.2230000000000001,"5min":1.0269999999999999,"15min":0.747},"max":{"1min":1.663,"5min":1.9279999999999999,"15min":2.577},"last":1.423},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3919999999999999,"15min":1.399},"min":{"1min":0.97199999999999998,"5min":0.93799999999999994,"15min":0.85699999999999998},"max":{"1min":1.671,"5min":1.8320000000000001,"15min":3.5190000000000001},"last":1.5840000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.391,"15min":1.411},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5900000000000001,"5min":1.7629999999999999,"15min":2.948},"last":1.492},{"interface":"front","average":{"1min":1.446,"5min":1.508,"15min":1.5009999999999999},"min":{"1min":1.238,"5min":1.093,"15min":0.97899999999999998},"max":{"1min":1.6220000000000001,"5min":2.1280000000000001,"15min":2.9300000000000002},"last":1.623}]},{"osd":7,"last update":"Fri Oct 6 12:33:30 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4350000000000001,"15min":1.4359999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6399999999999999,"5min":2.089,"15min":2.1680000000000001},"last":1.462},{"interface":"front","average":{"1min":1.48,"5min":1.506,"15min":1.502},"min":{"1min":1.181,"5min":1.1559999999999999,"15min":1.0509999999999999},"max":{"1min":1.7230000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.5529999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575756,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311604,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060236,"statfs":{"total":96636764160,"available":96317681664,"internally_reserved":0,"allocated":2076672,"data_stored":1091217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.673,"5min":1.798,"15min":1.798},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.0189999999999999,"15min":1.0269999999999999},"min":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3500000000000001,"5min":1.5620000000000001,"15min":1.655},"last":1.1839999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.97499999999999998,"15min":0.99299999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.2909999999999999,"5min":1.5629999999999999,"15min":1.6679999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.129,"15min":1.1180000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.60099999999999998},"max":{"1min":1.397,"5min":1.681,"15min":1.681},"last":1.651}]},{"osd":3,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.258,"15min":1.2390000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.61599999999999999},"max":{"1min":1.8580000000000001,"5min":2.008,"15min":2.008},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.1519999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.796,"5min":1.796,"15min":1.806},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.268,"15min":1.27},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.597,"5min":1.764,"15min":2.2490000000000001},"last":1.488},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.4379999999999999},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":1.73,"5min":2.226,"15min":2.226},"last":1.319}]},{"osd":5,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3939999999999999,"15min":1.387},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8069999999999999,"5min":1.8460000000000001,"15min":2.0350000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.381},"min":{"1min":1,"5min":1,"15min":0.93400000000000005},"max":{"1min":1.74,"5min":1.917,"15min":2.0619999999999998},"last":1.0860000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5529999999999999,"15min":1.5149999999999999},"min":{"1min":1.2270000000000001,"5min":1.161,"15min":1.0269999999999999},"max":{"1min":1.9470000000000001,"5min":2.0800000000000001,"15min":2.206},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.4710000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.9450000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:33:47 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.5860000000000001,"15min":1.571},"min":{"1min":1.038,"5min":1.038,"15min":0.99299999999999999},"max":{"1min":2.028,"5min":2.2440000000000002,"15min":2.2570000000000001},"last":1.448},{"interface":"front","average":{"1min":1.516,"5min":1.5149999999999999,"15min":1.49},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":0.95799999999999996},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1819999999999999},"last":1.4099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575757,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299916,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071924,"statfs":{"total":96636764160,"available":96329650176,"internally_reserved":0,"allocated":2691072,"data_stored":1683729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90600000000000003,"15min":0.90400000000000003},"min":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2150000000000001,"5min":1.4410000000000001,"15min":1.492},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.996,"15min":1.0089999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3069999999999999,"5min":1.5549999999999999,"15min":1.595},"last":1.089}]},{"osd":2,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.028,"15min":1.042},"min":{"1min":0.72799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.746},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.0840000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.599,"5min":1.762,"15min":1.762},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.554},{"interface":"front","average":{"1min":1.331,"5min":1.3640000000000001,"15min":1.367},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.71999999999999997},"max":{"1min":1.7949999999999999,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.657}]},{"osd":4,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3460000000000001,"15min":1.3580000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6850000000000001,"5min":1.71,"15min":1.756},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.3220000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6759999999999999,"5min":1.704,"15min":1.8560000000000001},"last":1.28}]},{"osd":5,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.393,"15min":1.401},"min":{"1min":0.93000000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.863,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.9630000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4219999999999999,"15min":1.4159999999999999},"min":{"1min":0.94899999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.7470000000000001,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.607}]},{"osd":6,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.1419999999999999},"last":1.351},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.472,"15min":1.476},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.77,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.857}]},{"osd":7,"last update":"Fri Oct 6 12:33:49 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.474,"15min":1.476},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.8089999999999999,"5min":1.9790000000000001,"15min":2.1019999999999999},"last":1.903},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4390000000000001,"15min":1.4430000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.776}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:34:25.164 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T12:34:25.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T12:34:25.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.375+0000 7f361be6e700 1 Processor -- start 2023-10-06T12:34:25.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.375+0000 7f361be6e700 1 -- start start 2023-10-06T12:34:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f361be6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36140b23c0 0x7f36140b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f361be6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36140cc510 con 0x7f36140cae40 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f361be6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36140cb900 con 0x7f36140cd250 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f361be6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140cbbe0 con 0x7f36140b23c0 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36140b23c0 0x7f36140b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36140b23c0 0x7f36140b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50240/0 (socket says 172.21.15.46:50240) 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 -- 172.21.15.46:0/3799231003 learned_addr learned my addr 172.21.15.46:0/3799231003 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:25.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619409700 1 --1- 172.21.15.46:0/3799231003 >> v1:172.21.15.46:6789/0 conn(0x7f36140cae40 0x7f36140ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32774/0 (socket says 172.21.15.46:32774) 2023-10-06T12:34:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3618c08700 1 -- 172.21.15.46:0/3799231003 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359875117 0 0) 0x7f36140cc510 con 0x7f36140cae40 2023-10-06T12:34:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3618c08700 1 -- 172.21.15.46:0/3799231003 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3600003570 con 0x7f36140cae40 2023-10-06T12:34:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 -- 172.21.15.46:0/3799231003 >> v1:172.21.15.132:6789/0 conn(0x7f36140cd250 legacy=0x7f36140c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 -- 172.21.15.46:0/3799231003 >> v1:172.21.15.46:6789/0 conn(0x7f36140cae40 legacy=0x7f36140ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 -- 172.21.15.46:0/3799231003 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140cbec0 con 0x7f36140b23c0 2023-10-06T12:34:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3619c0a700 1 --2- 172.21.15.46:0/3799231003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140b23c0 0x7f36140b2790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3610009990 tx=0x7f3610009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf589e9648eb621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3618c08700 1 -- 172.21.15.46:0/3799231003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610009e80 con 0x7f36140b23c0 2023-10-06T12:34:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3618c08700 1 -- 172.21.15.46:0/3799231003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610005c10 con 0x7f36140b23c0 2023-10-06T12:34:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.379+0000 7f3618c08700 1 -- 172.21.15.46:0/3799231003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000b650 con 0x7f36140b23c0 2023-10-06T12:34:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 -- 172.21.15.46:0/3799231003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140b23c0 msgr2=0x7f36140b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 --2- 172.21.15.46:0/3799231003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140b23c0 0x7f36140b2790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3610009990 tx=0x7f3610009220 comp rx=0 tx=0).stop 2023-10-06T12:34:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 -- 172.21.15.46:0/3799231003 shutdown_connections 2023-10-06T12:34:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 --2- 172.21.15.46:0/3799231003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140b23c0 0x7f36140b2790 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 -- 172.21.15.46:0/3799231003 >> 172.21.15.46:0/3799231003 conn(0x7f3614099480 msgr2=0x7f361406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 -- 172.21.15.46:0/3799231003 shutdown_connections 2023-10-06T12:34:25.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 -- 172.21.15.46:0/3799231003 wait complete. 2023-10-06T12:34:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.383+0000 7f361be6e700 1 Processor -- start 2023-10-06T12:34:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 -- start start 2023-10-06T12:34:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36140b23c0 0x7f361416a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36140cd250 0x7f361409d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361415beb0 con 0x7f36140cae40 2023-10-06T12:34:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361415c340 con 0x7f36140cd250 2023-10-06T12:34:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361be6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361415b2a0 con 0x7f36140b23c0 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50252/0 (socket says 172.21.15.46:50252) 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 -- 172.21.15.46:0/1677387497 learned_addr learned my addr 172.21.15.46:0/1677387497 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619c0a700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36140b23c0 0x7f361416a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f361a40b700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36140cd250 0x7f361409d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 -- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36140b23c0 msgr2=0x7f361416a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36140b23c0 0x7f361416a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 -- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36140cd250 msgr2=0x7f361409d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36140cd250 0x7f361409d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.387+0000 7f3619409700 1 -- 172.21.15.46:0/1677387497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009480 con 0x7f36140cae40 2023-10-06T12:34:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f3619409700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f360400c7c0 tx=0x7f3604007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604005c10 con 0x7f36140cae40 2023-10-06T12:34:25.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604005d70 con 0x7f36140cae40 2023-10-06T12:34:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604007320 con 0x7f36140cae40 2023-10-06T12:34:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361415b580 con 0x7f36140cae40 2023-10-06T12:34:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361415bb40 con 0x7f36140cae40 2023-10-06T12:34:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.391+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361415c020 con 0x7f36140cae40 2023-10-06T12:34:25.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.395+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f360401b3b0 con 0x7f36140cae40 2023-10-06T12:34:25.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.395+0000 7f3609ffb700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 0x7f35f8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.395+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f360407c380 con 0x7f36140cae40 2023-10-06T12:34:25.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.395+0000 7f3619c0a700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 0x7f35f8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.399+0000 7f3619c0a700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 0x7f35f8062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3610009260 tx=0x7f3610016000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.403+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f360404c460 con 0x7f36140cae40 2023-10-06T12:34:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.639+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f36140660f0 con 0x7f36140cae40 2023-10-06T12:34:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.647+0000 7f3609ffb700 1 -- 172.21.15.46:0/1677387497 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v42) v1 ==== 190+0+0 (secure 0 0 0) 0x7f360404fa80 con 0x7f36140cae40 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.651+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 msgr2=0x7f35f8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 0x7f35f8062b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3610009260 tx=0x7f3610016000 comp rx=0 tx=0).stop 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 msgr2=0x7f361415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f360400c7c0 tx=0x7f3604007e00 comp rx=0 tx=0).stop 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 shutdown_connections 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f35f80606a0 0x7f35f8062b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36140cd250 0x7f361409d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36140cae40 0x7f361415aa30 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 --2- 172.21.15.46:0/1677387497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36140b23c0 0x7f361416a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 >> 172.21.15.46:0/1677387497 conn(0x7f3614099480 msgr2=0x7f3614067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 shutdown_connections 2023-10-06T12:34:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.655+0000 7f361be6e700 1 -- 172.21.15.46:0/1677387497 wait complete. 2023-10-06T12:34:25.660 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T12:34:25.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:34:25.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.835+0000 7fb4bea2e700 1 Processor -- start 2023-10-06T12:34:25.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.835+0000 7fb4bea2e700 1 -- start start 2023-10-06T12:34:25.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bea2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4b8159050 0x7fb4b8159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bea2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b80b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bea2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80b4f80 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bea2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80b5870 con 0x7fb4b8159050 2023-10-06T12:34:25.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bea2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80b40c0 con 0x7fb4b80b5410 2023-10-06T12:34:25.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4affff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4b80d0aa0 0x7fb4b80b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32788/0 (socket says 172.21.15.46:32788) 2023-10-06T12:34:25.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4affff700 1 -- 172.21.15.46:0/2386302955 learned_addr learned my addr 172.21.15.46:0/2386302955 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:25.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2386302955 >> v2:172.21.15.46:3300/0 conn(0x7fb4b8159050 0x7fb4b8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4bcfcb700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b80b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64086860 0 0) 0x7fb4b80b4f80 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.839+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a4003570 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b7fff700 1 -- 172.21.15.46:0/2386302955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 msgr2=0x7fb4b80b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b80b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b7fff700 1 -- 172.21.15.46:0/2386302955 >> v1:172.21.15.46:6789/0 conn(0x7fb4b80d0aa0 legacy=0x7fb4b80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253682086 0 0) 0x7fb4a4003570 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b7fff700 1 -- 172.21.15.46:0/2386302955 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80b4370 con 0x7fb4b8159050 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b8159050 0x7fb4b8159420 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80098e0 tx=0x7fb4a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad2d2c1a2cda307 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0001470 con 0x7fb4b8159050 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8011420 con 0x7fb4b8159050 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4b77fe700 1 -- 172.21.15.46:0/2386302955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80117c0 con 0x7fb4b8159050 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 -- 172.21.15.46:0/2386302955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b8159050 msgr2=0x7fb4b8159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b8159050 0x7fb4b8159420 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80098e0 tx=0x7fb4a80091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 -- 172.21.15.46:0/2386302955 shutdown_connections 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b80b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/2386302955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b8159050 0x7fb4b8159420 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 -- 172.21.15.46:0/2386302955 >> 172.21.15.46:0/2386302955 conn(0x7fb4b806e780 msgr2=0x7fb4b806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:25.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 -- 172.21.15.46:0/2386302955 shutdown_connections 2023-10-06T12:34:25.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.843+0000 7fb4bea2e700 1 -- 172.21.15.46:0/2386302955 wait complete. 2023-10-06T12:34:25.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 Processor -- start 2023-10-06T12:34:25.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 -- start start 2023-10-06T12:34:25.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4b8159050 0x7fb4b8153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80c71e0 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b815c290 con 0x7fb4b8159050 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8156eb0 con 0x7fb4b80b5410 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50272/0 (socket says 172.21.15.46:50272) 2023-10-06T12:34:25.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 -- 172.21.15.46:0/3232187616 learned_addr learned my addr 172.21.15.46:0/3232187616 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bcfcb700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4b8159050 0x7fb4b8153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 -- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 msgr2=0x7fb4b815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 -- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4b8159050 msgr2=0x7fb4b8153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4b8159050 0x7fb4b8153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 -- 172.21.15.46:0/3232187616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8009420 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4affff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000cb90 tx=0x7fb4a0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0007e90 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a0007360 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b815bcd0 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.847+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0007700 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.851+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b8157150 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.851+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b81587f0 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.851+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb4a0004a40 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.851+0000 7fb4b4ff9700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 0x7fb498062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:25.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.855+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fb4a007c600 con 0x7fb4b80d0aa0 2023-10-06T12:34:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.855+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 0x7fb498062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:25.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.855+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 0x7fb498062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80030c0 tx=0x7fb4a8016000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:25.859+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb4b81587f0 con 0x7fb4b80d0aa0 2023-10-06T12:34:26.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.063+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b81587f0 con 0x7fb4b80d0aa0 2023-10-06T12:34:26.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.063+0000 7fb4b4ff9700 1 -- 172.21.15.46:0/3232187616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fb4b81587f0 con 0x7fb4b80d0aa0 2023-10-06T12:34:26.068 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:34:26.068 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:34:26.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 msgr2=0x7fb498062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 0x7fb498062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80030c0 tx=0x7fb4a8016000 comp rx=0 tx=0).stop 2023-10-06T12:34:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 msgr2=0x7fb4b80c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000cb90 tx=0x7fb4a0001b50 comp rx=0 tx=0).stop 2023-10-06T12:34:26.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 shutdown_connections 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fb498060670 0x7fb498062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4b8159050 0x7fb4b8153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b80d0aa0 0x7fb4b80c6320 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 --2- 172.21.15.46:0/3232187616 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4b80b5410 0x7fb4b815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 >> 172.21.15.46:0/3232187616 conn(0x7fb4b806e780 msgr2=0x7fb4b8155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 shutdown_connections 2023-10-06T12:34:26.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.067+0000 7fb4bea2e700 1 -- 172.21.15.46:0/3232187616 wait complete. 2023-10-06T12:34:26.074 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:34:26.377 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-10-06T12:34:26.371+0000 7fe21bfd3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:34:26.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T12:34:19.846543+0000', 'modified': '2023-10-06T12:34:23.457018+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9294': {'gid': 9294, 'name': 'a', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6836/3241270718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3241270718}, {'type': 'v1', 'addr': '172.21.15.46:6836', 'nonce': 3241270718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T12:34:26.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T12:34:26.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T12:34:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.655+0000 7f5d68e65700 1 Processor -- start 2023-10-06T12:34:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.655+0000 7f5d68e65700 1 -- start start 2023-10-06T12:34:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d68e65700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d6415c770 0x7f5d640cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d68e65700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d640cec00 0x7f5d64162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d68e65700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d640cc150 con 0x7f5d640ccb50 2023-10-06T12:34:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d68e65700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640cfbb0 con 0x7f5d640cec00 2023-10-06T12:34:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d68e65700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640cdbf0 con 0x7f5d6415c770 2023-10-06T12:34:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d62d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d640cec00 0x7f5d64162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d62d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d640cec00 0x7f5d64162e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50288/0 (socket says 172.21.15.46:50288) 2023-10-06T12:34:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d62d9d700 1 -- 172.21.15.46:0/3277021833 learned_addr learned my addr 172.21.15.46:0/3277021833 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d61d9b700 1 --1- 172.21.15.46:0/3277021833 >> v1:172.21.15.46:6789/0 conn(0x7f5d640ccb50 0x7f5d640ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32796/0 (socket says 172.21.15.46:32796) 2023-10-06T12:34:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.659+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3277021833 >> v2:172.21.15.132:3300/0 conn(0x7f5d6415c770 0x7f5d640cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d62d9d700 1 -- 172.21.15.46:0/3277021833 >> v2:172.21.15.132:3300/0 conn(0x7f5d6415c770 msgr2=0x7f5d640cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3277021833 >> v2:172.21.15.132:3300/0 conn(0x7f5d6415c770 0x7f5d640cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d62d9d700 1 -- 172.21.15.46:0/3277021833 >> v1:172.21.15.46:6789/0 conn(0x7f5d640ccb50 legacy=0x7f5d640ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d62d9d700 1 -- 172.21.15.46:0/3277021833 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640cb280 con 0x7f5d640cec00 2023-10-06T12:34:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d6159a700 1 -- 172.21.15.46:0/3277021833 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703221975 0 0) 0x7f5d640cc150 con 0x7f5d640ccb50 2023-10-06T12:34:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3277021833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d64162e50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5d6406f6d0 tx=0x7f5d5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f144d7dbfd5f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d6159a700 1 -- 172.21.15.46:0/3277021833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c00e070 con 0x7f5d640cec00 2023-10-06T12:34:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d6159a700 1 -- 172.21.15.46:0/3277021833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5c00ba20 con 0x7f5d640cec00 2023-10-06T12:34:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d6159a700 1 -- 172.21.15.46:0/3277021833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0092e0 con 0x7f5d640cec00 2023-10-06T12:34:26.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 -- 172.21.15.46:0/3277021833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 msgr2=0x7f5d64162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3277021833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d64162e50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5d6406f6d0 tx=0x7f5d5c009840 comp rx=0 tx=0).stop 2023-10-06T12:34:26.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 -- 172.21.15.46:0/3277021833 shutdown_connections 2023-10-06T12:34:26.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3277021833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d64162e50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3277021833 >> v2:172.21.15.132:3300/0 conn(0x7f5d6415c770 0x7f5d640cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 -- 172.21.15.46:0/3277021833 >> 172.21.15.46:0/3277021833 conn(0x7f5d64099480 msgr2=0x7f5d6406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 -- 172.21.15.46:0/3277021833 shutdown_connections 2023-10-06T12:34:26.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.663+0000 7f5d68e65700 1 -- 172.21.15.46:0/3277021833 wait complete. 2023-10-06T12:34:26.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 Processor -- start 2023-10-06T12:34:26.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 -- start start 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d640ccb50 0x7f5d64158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d6415c770 0x7f5d64157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640c9720 con 0x7f5d640cec00 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6416a100 con 0x7f5d640ccb50 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.667+0000 7f5d68e65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d64169240 con 0x7f5d6415c770 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d6259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d640ccb50 0x7f5d64158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2023-10-06T12:34:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3596790545 learned_addr learned my addr 172.21.15.46:0/3596790545 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d6415c770 0x7f5d64157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d6415c770 msgr2=0x7f5d64157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d6415c770 0x7f5d64157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d640ccb50 msgr2=0x7f5d64158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d640ccb50 0x7f5d64158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3596790545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c009ae0 con 0x7f5d640cec00 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5d580011e0 tx=0x7f5d58007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d527fc700 1 -- 172.21.15.46:0/3596790545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58005a70 con 0x7f5d640cec00 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d527fc700 1 -- 172.21.15.46:0/3596790545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58005bd0 con 0x7f5d640cec00 2023-10-06T12:34:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d64158500 con 0x7f5d640cec00 2023-10-06T12:34:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d527fc700 1 -- 172.21.15.46:0/3596790545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580070d0 con 0x7f5d640cec00 2023-10-06T12:34:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.671+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d641697d0 con 0x7f5d640cec00 2023-10-06T12:34:26.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d527fc700 1 -- 172.21.15.46:0/3596790545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5d58007480 con 0x7f5d640cec00 2023-10-06T12:34:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d527fc700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 0x7f5d44062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d6b0ed740 1 Processor -- start 2023-10-06T12:34:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d527fc700 1 -- 172.21.15.46:0/3596790545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f5d5807c250 con 0x7f5d640cec00 2023-10-06T12:34:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d6b0ed740 1 -- start start 2023-10-06T12:34:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.679+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 0x7f5d44062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caa710 0x1f27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e55f30 con 0x1caaae0 2023-10-06T12:34:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e51ed0 con 0x1caa710 2023-10-06T12:34:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e180b0 con 0x1cf5790 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 0x7f5d44062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d640b5c80 tx=0x7f5d54009120 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50294/0 (socket says 172.21.15.46:50294) 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 -- 172.21.15.46:0/915017629 learned_addr learned my addr 172.21.15.46:0/915017629 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caa710 0x1f27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 -- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 msgr2=0x1f280b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f280b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 -- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caa710 msgr2=0x1f27b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caa710 0x1f27b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 -- 172.21.15.46:0/915017629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cf69e0 con 0x1caaae0 2023-10-06T12:34:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d6259c700 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011e40 tx=0x7f5d54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665879fcdc01ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d437fe700 1 -- 172.21.15.46:0/915017629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54005d90 con 0x1caaae0 2023-10-06T12:34:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d437fe700 1 -- 172.21.15.46:0/915017629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540064e0 con 0x1caaae0 2023-10-06T12:34:26.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.683+0000 7f5d437fe700 1 -- 172.21.15.46:0/915017629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54006860 con 0x1caaae0 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/915017629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 msgr2=0x1f27630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011e40 tx=0x7f5d54018000 comp rx=0 tx=0).stop 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/915017629 shutdown_connections 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f280b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caa710 0x1f27b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/915017629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f27630 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/915017629 >> 172.21.15.46:0/915017629 conn(0x1caaeb0 msgr2=0x1ee4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/915017629 shutdown_connections 2023-10-06T12:34:26.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/915017629 wait complete. 2023-10-06T12:34:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 -- start start 2023-10-06T12:34:26.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.687+0000 7f5d6b0ed740 1 Processor -- start 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 0x1f2bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 0x1ee1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e17d50 con 0x1caaae0 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1df0970 con 0x1cf5790 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e17420 con 0x1caa710 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 10 client.0 _ll_register_callbacks cb 0x1df4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50298/0 (socket says 172.21.15.46:50298) 2023-10-06T12:34:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3380638518 learned_addr learned my addr 172.21.15.46:0/3380638518 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 0x1ee1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 0x1f2bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 msgr2=0x1f2bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 0x1f2bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 msgr2=0x1ee1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 0x1ee1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 -- 172.21.15.46:0/3380638518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d54011a70 con 0x1caaae0 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 0x1ee1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 0x1f2bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f5d5804ff80 tx=0x7f5d58052040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1936aed250ae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58056fd0 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1df0cc0 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f5d5804f4e0 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5806a1f0 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 10 client.9444 fetch_fsmap learned FSMap version 85 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x1e20370 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1df0cc0 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 85) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5d58056b40 con 0x1caaae0 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 10 client.9444 fetch_fsmap finished waiting for FSMap version 85 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 10 client.9444 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 20 client.9444 populate_metadata read hostname 'smithi046' 2023-10-06T12:34:26.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1ee3260 0x1f36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 4 client.9444 mds_command: new command op to 9294 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T12:34:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 --> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1df43e0 con 0x1ee3260 2023-10-06T12:34:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.691+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1ee3260 0x1f36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1ee3260 0x1f36de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db64ca1c7e927345 server_cookie=78ca1b335658fb88 in_seq=0 out_seq=0 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d40ff9700 10 client.9444 ms_handle_connect on v2:172.21.15.46:6834/3241270718 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d40ff9700 1 -- 172.21.15.46:0/3380638518 <== mds.0 v2:172.21.15.46:6834/3241270718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1df43e0 con 0x1ee3260 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d40ff9700 10 client.9444 handle_command_reply: tid=0 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 client.9444 shutdown 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 msgr2=0x1f2c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f5d5804ff80 tx=0x7f5d58052040 comp rx=0 tx=0).stop 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 shutdown_connections 2023-10-06T12:34:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1ee3260 0x1f36de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1cf5790 0x1ee1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caaae0 0x1f2c180 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3380638518 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1caa710 0x1f2bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d40ff9700 0 client.9444 ms_handle_reset on v2:172.21.15.46:6834/3241270718 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 >> 172.21.15.46:0/3380638518 conn(0x1caaeb0 msgr2=0x1f0b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 shutdown_connections 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3380638518 wait complete. 2023-10-06T12:34:26.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.695+0000 7f5d6b0ed740 20 client.9444 trim_cache size 0 max 16384 2023-10-06T12:34:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 Processor -- start 2023-10-06T12:34:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 -- start start 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e1b400 con 0x1caa710 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e27010 con 0x1caaae0 2023-10-06T12:34:26.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.715+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e25040 con 0x1cf5790 2023-10-06T12:34:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50314/0 (socket says 172.21.15.46:50314) 2023-10-06T12:34:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 -- 172.21.15.46:0/3330888392 learned_addr learned my addr 172.21.15.46:0/3330888392 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d61d9b700 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 -- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 msgr2=0x1f36de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f36de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 -- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 msgr2=0x1f2ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f2ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 -- 172.21.15.46:0/3330888392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e06d80 con 0x1caa710 2023-10-06T12:34:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6259c700 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011930 tx=0x7f5d5401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b13d1acf80ce9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d43fff700 1 -- 172.21.15.46:0/3330888392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54031040 con 0x1caa710 2023-10-06T12:34:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d43fff700 1 -- 172.21.15.46:0/3330888392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54005d90 con 0x1caa710 2023-10-06T12:34:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d43fff700 1 -- 172.21.15.46:0/3330888392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54006650 con 0x1caa710 2023-10-06T12:34:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3330888392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 msgr2=0x1f2a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5d54011930 tx=0x7f5d5401b000 comp rx=0 tx=0).stop 2023-10-06T12:34:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3330888392 shutdown_connections 2023-10-06T12:34:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f2ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/3330888392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1caa710 0x1f2a650 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3330888392 >> 172.21.15.46:0/3330888392 conn(0x1caaeb0 msgr2=0x1f080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3330888392 shutdown_connections 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.719+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/3330888392 wait complete. 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 -- start start 2023-10-06T12:34:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 Processor -- start 2023-10-06T12:34:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f81f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e4f890 con 0x1f05520 2023-10-06T12:34:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e50d60 con 0x1caaae0 2023-10-06T12:34:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ca8430 con 0x1cf5790 2023-10-06T12:34:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 10 client.0 _ll_register_callbacks cb 0x1e1d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50326/0 (socket says 172.21.15.46:50326) 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 -- 172.21.15.46:0/128312263 learned_addr learned my addr 172.21.15.46:0/128312263 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6259c700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 -- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 msgr2=0x1f81f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f81f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 -- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 msgr2=0x1f81a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f81a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 -- 172.21.15.46:0/128312263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1cf69e0 con 0x1f05520 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d62d9d700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009880 tx=0x7f5d5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d53e5a2e8c43072 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c024040 con 0x1f05520 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f5d5c01d630 con 0x1f05520 2023-10-06T12:34:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.723+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d540064e0 con 0x1f05520 2023-10-06T12:34:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5c00bc60 con 0x1f05520 2023-10-06T12:34:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 10 client.9450 fetch_fsmap learned FSMap version 85 2023-10-06T12:34:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x1dfb130 con 0x1f05520 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540064e0 con 0x1f05520 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 85) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5d5c019070 con 0x1f05520 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 10 client.9450 fetch_fsmap finished waiting for FSMap version 85 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 10 client.9450 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 20 client.9450 populate_metadata read hostname 'smithi046' 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1f88890 0x1f8ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 4 client.9450 mds_command: new command op to 9294 tid=0[{"prefix": "status"}] 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 --> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1e26410 con 0x1f88890 2023-10-06T12:34:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6259c700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1f88890 0x1f8ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6259c700 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1f88890 0x1f8ac70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0521768d9b7ef58 server_cookie=fc7841370425d401 in_seq=0 out_seq=0 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 10 client.9450 ms_handle_connect on v2:172.21.15.46:6834/3241270718 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 1 -- 172.21.15.46:0/128312263 <== mds.0 v2:172.21.15.46:6834/3241270718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x1e26410 con 0x1f88890 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 10 client.9450 handle_command_reply: tid=0 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 client.9450 shutdown 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 msgr2=0x1f058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009880 tx=0x7f5d5c018000 comp rx=0 tx=0).stop 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 shutdown_connections 2023-10-06T12:34:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:6834/3241270718,v1:172.21.15.46:6836/3241270718] conn(0x1f88890 0x1f8ac70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1f05520 0x1f058f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1cf5790 0x1f81f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 --2- 172.21.15.46:0/128312263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1caaae0 0x1f81a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d40ff9700 0 client.9450 ms_handle_reset on v2:172.21.15.46:6834/3241270718 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.727+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 >> 172.21.15.46:0/128312263 conn(0x1caaeb0 msgr2=0x1ee3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 shutdown_connections 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d6b0ed740 1 -- 172.21.15.46:0/128312263 wait complete. 2023-10-06T12:34:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d6b0ed740 20 client.9450 trim_cache size 0 max 16384 2023-10-06T12:34:26.736 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:34:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T12:34:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T12:34:26.736 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9294, 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 6.3269330779999997, 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 85, 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 166, 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 166, 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 12.256905623 2023-10-06T12:34:26.737 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:34:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 msgr2=0x7f5d44062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 0x7f5d44062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d640b5c80 tx=0x7f5d54009120 comp rx=0 tx=0).stop 2023-10-06T12:34:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 msgr2=0x7f5d640c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5d580011e0 tx=0x7f5d58007e10 comp rx=0 tx=0).stop 2023-10-06T12:34:26.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 shutdown_connections 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f5d44060670 0x7f5d44062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d6415c770 0x7f5d64157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d640cec00 0x7f5d640c72f0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 --2- 172.21.15.46:0/3596790545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d640ccb50 0x7f5d64158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 >> 172.21.15.46:0/3596790545 conn(0x7f5d64099480 msgr2=0x7f5d64067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 shutdown_connections 2023-10-06T12:34:26.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.731+0000 7f5d68e65700 1 -- 172.21.15.46:0/3596790545 wait complete. 2023-10-06T12:34:26.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T12:34:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.915+0000 7f3234877700 1 Processor -- start 2023-10-06T12:34:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.915+0000 7f3234877700 1 -- start start 2023-10-06T12:34:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f3234877700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3230158dd0 0x7f3230067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f3234877700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32300679f0 con 0x7f32301591a0 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f3234877700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015d6c0 con 0x7f3230067590 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f3234877700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32300699c0 con 0x7f3230158dd0 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3230158dd0 0x7f3230067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3230158dd0 0x7f3230067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54956/0 (socket says 172.21.15.46:54956) 2023-10-06T12:34:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 -- 172.21.15.46:0/548000914 learned_addr learned my addr 172.21.15.46:0/548000914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 -- 172.21.15.46:0/548000914 >> v1:172.21.15.132:6789/0 conn(0x7f3230067590 legacy=0x7f323006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 -- 172.21.15.46:0/548000914 >> v1:172.21.15.46:6789/0 conn(0x7f32301591a0 legacy=0x7f3230068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 -- 172.21.15.46:0/548000914 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015cd90 con 0x7f3230158dd0 2023-10-06T12:34:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322ed9d700 1 --2- 172.21.15.46:0/548000914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3230158dd0 0x7f3230067050 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f32200099b0 tx=0x7f3220009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e1f26cae5dce14b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322e59c700 1 -- 172.21.15.46:0/548000914 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000b860 con 0x7f3230158dd0 2023-10-06T12:34:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.919+0000 7f322e59c700 1 -- 172.21.15.46:0/548000914 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322000b9c0 con 0x7f3230158dd0 2023-10-06T12:34:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f322e59c700 1 -- 172.21.15.46:0/548000914 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000bd60 con 0x7f3230158dd0 2023-10-06T12:34:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 -- 172.21.15.46:0/548000914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3230158dd0 msgr2=0x7f3230067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 --2- 172.21.15.46:0/548000914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3230158dd0 0x7f3230067050 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f32200099b0 tx=0x7f3220009250 comp rx=0 tx=0).stop 2023-10-06T12:34:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 -- 172.21.15.46:0/548000914 shutdown_connections 2023-10-06T12:34:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 --2- 172.21.15.46:0/548000914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3230158dd0 0x7f3230067050 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 -- 172.21.15.46:0/548000914 >> 172.21.15.46:0/548000914 conn(0x7f3230062ef0 msgr2=0x7f32300cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 -- 172.21.15.46:0/548000914 shutdown_connections 2023-10-06T12:34:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.923+0000 7f3234877700 1 -- 172.21.15.46:0/548000914 wait complete. 2023-10-06T12:34:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 Processor -- start 2023-10-06T12:34:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 -- start start 2023-10-06T12:34:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 0x7f323006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32301591a0 0x7f323006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323016f090 con 0x7f3230067590 2023-10-06T12:34:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015a750 con 0x7f32301591a0 2023-10-06T12:34:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f3234877700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3230159eb0 con 0x7f3230158dd0 2023-10-06T12:34:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50336/0 (socket says 172.21.15.46:50336) 2023-10-06T12:34:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 -- 172.21.15.46:0/1401256193 learned_addr learned my addr 172.21.15.46:0/1401256193 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322f59e700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32301591a0 0x7f323006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322659c700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 0x7f323006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 -- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 msgr2=0x7f323006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 0x7f323006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 -- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32301591a0 msgr2=0x7f323006e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32301591a0 0x7f323006e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 -- 172.21.15.46:0/1401256193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200094f0 con 0x7f3230067590 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322659c700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 0x7f323006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.927+0000 7f322ed9d700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f323006ab70 tx=0x7f3220017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220011ea0 con 0x7f3230067590 2023-10-06T12:34:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322001a630 con 0x7f3230067590 2023-10-06T12:34:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323015a190 con 0x7f3230067590 2023-10-06T12:34:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322001a9b0 con 0x7f3230067590 2023-10-06T12:34:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323015b280 con 0x7f3230067590 2023-10-06T12:34:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.931+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323015b4a0 con 0x7f3230067590 2023-10-06T12:34:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.935+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f322002d440 con 0x7f3230067590 2023-10-06T12:34:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.935+0000 7f32277fe700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 0x7f3210062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.935+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f322007e900 con 0x7f3230067590 2023-10-06T12:34:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.935+0000 7f322659c700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 0x7f3210062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.935+0000 7f322659c700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 0x7f3210062b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32300c9480 tx=0x7f3218009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:26.939+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f322004e9e0 con 0x7f3230067590 2023-10-06T12:34:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.147+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3230064590 con 0x7f3230067590 2023-10-06T12:34:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.147+0000 7f32277fe700 1 -- 172.21.15.46:0/1401256193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3220052000 con 0x7f3230067590 2023-10-06T12:34:27.152 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 msgr2=0x7f3210062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 0x7f3210062b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f32300c9480 tx=0x7f3218009040 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 msgr2=0x7f3230159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f323006ab70 tx=0x7f3220017000 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 shutdown_connections 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f32100606a0 0x7f3210062b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32301591a0 0x7f323006e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3230158dd0 0x7f323006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 --2- 172.21.15.46:0/1401256193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3230067590 0x7f3230159660 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 >> 172.21.15.46:0/1401256193 conn(0x7f3230062ef0 msgr2=0x7f32300c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:27.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 shutdown_connections 2023-10-06T12:34:27.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:27.151+0000 7f3234877700 1 -- 172.21.15.46:0/1401256193 wait complete. 2023-10-06T12:34:27.173 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T12:34:27.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T12:34:27.174 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:34:27.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.175 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:34:27.195 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 37863sec preferred_lft 37863sec 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T12:34:27.196 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.197 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T12:34:27.198 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:34:27.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.199 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T12:34:27.279 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T12:34:27.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.282 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T12:34:27.347 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T12:34:27.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T12:34:27.348 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T12:34:27.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.485 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:34:27.503 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T12:34:27.503 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T12:34:27.504 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:34:27.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T12:34:27.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:34:27.584 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T12:34:27.643 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:34:27.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:34:27.643 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:34:27.706 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T12:34:27.706 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T12:34:27.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:34:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:34:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:34:27.827 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:34:27.853 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:34:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:34:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:34:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:34:27.854 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:34:27.866 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:34:27.866 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:34:27.866 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:34:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:34:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:34:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T12:34:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T12:34:27.867 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T12:34:27.871 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T12:34:27.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:34:27.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:34:27.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T12:34:28.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.111+0000 7ff7fa323700 1 Processor -- start 2023-10-06T12:34:28.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.111+0000 7ff7fa323700 1 -- start start 2023-10-06T12:34:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7fa323700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7f4067760 0x7ff7f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7fa323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f4069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7fa323700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f415d6c0 con 0x7ff7f4158dd0 2023-10-06T12:34:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7fa323700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40699c0 con 0x7ff7f4067760 2023-10-06T12:34:28.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7fa323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f415db50 con 0x7ff7f41591a0 2023-10-06T12:34:28.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7f8b20700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7f4067760 0x7ff7f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7f8b20700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7f4067760 0x7ff7f4068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50338/0 (socket says 172.21.15.46:50338) 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7f8b20700 1 -- 172.21.15.46:0/3648514928 learned_addr learned my addr 172.21.15.46:0/3648514928 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.115+0000 7ff7f9b22700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f4069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7f8b20700 1 -- 172.21.15.46:0/3648514928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 msgr2=0x7ff7f4069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7f8b20700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f4069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7f8b20700 1 -- 172.21.15.46:0/3648514928 >> v1:172.21.15.132:6789/0 conn(0x7ff7f4158dd0 legacy=0x7ff7f4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7f8b20700 1 -- 172.21.15.46:0/3648514928 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f415cd90 con 0x7ff7f4067760 2023-10-06T12:34:28.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7f8b20700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4067760 0x7ff7f4068bf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40099e0 tx=0x7ff7e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e2e44ec66bc66e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7ebfff700 1 -- 172.21.15.46:0/3648514928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400b820 con 0x7ff7f4067760 2023-10-06T12:34:28.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7ebfff700 1 -- 172.21.15.46:0/3648514928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400b980 con 0x7ff7f4067760 2023-10-06T12:34:28.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7ebfff700 1 -- 172.21.15.46:0/3648514928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400bd20 con 0x7ff7f4067760 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 -- 172.21.15.46:0/3648514928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4067760 msgr2=0x7ff7f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4067760 0x7ff7f4068bf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40099e0 tx=0x7ff7e4009250 comp rx=0 tx=0).stop 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 -- 172.21.15.46:0/3648514928 shutdown_connections 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f4069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 --2- 172.21.15.46:0/3648514928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4067760 0x7ff7f4068bf0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 -- 172.21.15.46:0/3648514928 >> 172.21.15.46:0/3648514928 conn(0x7ff7f4062ef0 msgr2=0x7ff7f40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:28.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 -- 172.21.15.46:0/3648514928 shutdown_connections 2023-10-06T12:34:28.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.119+0000 7ff7fa323700 1 -- 172.21.15.46:0/3648514928 wait complete. 2023-10-06T12:34:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 Processor -- start 2023-10-06T12:34:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 -- start start 2023-10-06T12:34:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f4067760 0x7ff7f406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f40ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f416a520 con 0x7ff7f4158dd0 2023-10-06T12:34:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40c58f0 con 0x7ff7f4067760 2023-10-06T12:34:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7fa323700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4169760 con 0x7ff7f41591a0 2023-10-06T12:34:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7f8b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7f8b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50348/0 (socket says 172.21.15.46:50348) 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7f8b20700 1 -- 172.21.15.46:0/2991809612 learned_addr learned my addr 172.21.15.46:0/2991809612 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7f9b22700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f40ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.123+0000 7ff7f9321700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f4067760 0x7ff7f406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 -- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 msgr2=0x7ff7f40ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f40ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 -- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f4067760 msgr2=0x7ff7f406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f4067760 0x7ff7f406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 -- 172.21.15.46:0/2991809612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40094f0 con 0x7ff7f4158dd0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7f8b20700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009290 tx=0x7ff7e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4011dc0 con 0x7ff7f4158dd0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4026450 con 0x7ff7f4158dd0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f4169a40 con 0x7ff7f4158dd0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4026810 con 0x7ff7f4158dd0 2023-10-06T12:34:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f416a000 con 0x7ff7f4158dd0 2023-10-06T12:34:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.127+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f416a1f0 con 0x7ff7f4158dd0 2023-10-06T12:34:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.131+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff7e401a420 con 0x7ff7f4158dd0 2023-10-06T12:34:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.131+0000 7ff7e97fa700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 0x7ff7d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.131+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7ff7e407e970 con 0x7ff7f4158dd0 2023-10-06T12:34:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.131+0000 7ff7f9321700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 0x7ff7d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.131+0000 7ff7f9321700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 0x7ff7d8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0003860 tx=0x7ff7f0009040 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.139+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff7e404ead0 con 0x7ff7f4158dd0 2023-10-06T12:34:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.339+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7f4064590 con 0x7ff7d8060670 2023-10-06T12:34:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.339+0000 7ff7e97fa700 1 -- 172.21.15.46:0/2991809612 <== mgr.8875 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff7f4064590 con 0x7ff7d8060670 2023-10-06T12:34:28.345 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:34:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:34:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T12:34:28.346 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:34:28.346 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:34:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 msgr2=0x7ff7d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 0x7ff7d8062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0003860 tx=0x7ff7f0009040 comp rx=0 tx=0).stop 2023-10-06T12:34:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 msgr2=0x7ff7f40aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009290 tx=0x7ff7e4017000 comp rx=0 tx=0).stop 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 shutdown_connections 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7ff7d8060670 0x7ff7d8062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f41591a0 0x7ff7f40ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f4158dd0 0x7ff7f40aaef0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 --2- 172.21.15.46:0/2991809612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f4067760 0x7ff7f406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 >> 172.21.15.46:0/2991809612 conn(0x7ff7f4062ef0 msgr2=0x7ff7f40add80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 shutdown_connections 2023-10-06T12:34:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.343+0000 7ff7fa323700 1 -- 172.21.15.46:0/2991809612 wait complete. 2023-10-06T12:34:28.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T12:34:28.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 Processor -- start 2023-10-06T12:34:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 -- start start 2023-10-06T12:34:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2800afc40 0x7fd2800b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd280068f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd280068a90 con 0x7fd280159820 2023-10-06T12:34:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800b18c0 con 0x7fd2800afc40 2023-10-06T12:34:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.539+0000 7fd286e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd280068160 con 0x7fd280159c70 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2800afc40 0x7fd2800b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2800afc40 0x7fd2800b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50350/0 (socket says 172.21.15.46:50350) 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 -- 172.21.15.46:0/1668730367 learned_addr learned my addr 172.21.15.46:0/1668730367 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 -- 172.21.15.46:0/1668730367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 msgr2=0x7fd280068f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd28540b700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd280068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd280068f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 -- 172.21.15.46:0/1668730367 >> v1:172.21.15.132:6789/0 conn(0x7fd280159820 legacy=0x7fd2800b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 -- 172.21.15.46:0/1668730367 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280068440 con 0x7fd2800afc40 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd28540b700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd280068f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd284c0a700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2800afc40 0x7fd2800b0010 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd2740099b0 tx=0x7fd274009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb4f0dcd7900e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd27f7fe700 1 -- 172.21.15.46:0/1668730367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400b860 con 0x7fd2800afc40 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd27f7fe700 1 -- 172.21.15.46:0/1668730367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27400b9c0 con 0x7fd2800afc40 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd27f7fe700 1 -- 172.21.15.46:0/1668730367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27400bd60 con 0x7fd2800afc40 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 -- 172.21.15.46:0/1668730367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2800afc40 msgr2=0x7fd2800b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2800afc40 0x7fd2800b0010 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd2740099b0 tx=0x7fd274009250 comp rx=0 tx=0).stop 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 -- 172.21.15.46:0/1668730367 shutdown_connections 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd280068f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 --2- 172.21.15.46:0/1668730367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2800afc40 0x7fd2800b0010 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 -- 172.21.15.46:0/1668730367 >> 172.21.15.46:0/1668730367 conn(0x7fd28006aea0 msgr2=0x7fd28006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 -- 172.21.15.46:0/1668730367 shutdown_connections 2023-10-06T12:34:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.543+0000 7fd286e6e700 1 -- 172.21.15.46:0/1668730367 wait complete. 2023-10-06T12:34:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.547+0000 7fd286e6e700 1 Processor -- start 2023-10-06T12:34:28.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.547+0000 7fd286e6e700 1 -- start start 2023-10-06T12:34:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2800afc40 0x7fd280158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd28015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28016aea0 con 0x7fd280159820 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28016b740 con 0x7fd2800afc40 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28016ab20 con 0x7fd280159c70 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50362/0 (socket says 172.21.15.46:50362) 2023-10-06T12:34:28.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 -- 172.21.15.46:0/2006210088 learned_addr learned my addr 172.21.15.46:0/2006210088 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd284c0a700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2800afc40 0x7fd280158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd28540b700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd28015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 -- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 msgr2=0x7fd28015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd28015c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 -- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2800afc40 msgr2=0x7fd280158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2800afc40 0x7fd280158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 -- 172.21.15.46:0/2006210088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2740094f0 con 0x7fd280159820 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27ffff700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00c5c0 tx=0x7fd26c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c011590 con 0x7fd280159820 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c011710 con 0x7fd280159820 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c011ad0 con 0x7fd280159820 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28016b0f0 con 0x7fd280159820 2023-10-06T12:34:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.551+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280159340 con 0x7fd280159820 2023-10-06T12:34:28.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.555+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2800660f0 con 0x7fd280159820 2023-10-06T12:34:28.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.555+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 84) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd26c01b3b0 con 0x7fd280159820 2023-10-06T12:34:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.559+0000 7fd27cff9700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 0x7fd264062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.559+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7fd26c07c1e0 con 0x7fd280159820 2023-10-06T12:34:28.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.559+0000 7fd284c0a700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 0x7fd264062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:28.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.559+0000 7fd284c0a700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 0x7fd264062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd2740099b0 tx=0x7fd274017000 comp rx=0 tx=0).ready entity=mgr.8875 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.563+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd26c04c2c0 con 0x7fd280159820 2023-10-06T12:34:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.775+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fd280068720 con 0x7fd280159820 2023-10-06T12:34:28.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.843+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 85) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fd26c04c690 con 0x7fd280159820 2023-10-06T12:34:28.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.847+0000 7fd27cff9700 1 -- 172.21.15.46:0/2006210088 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v85) v1 ==== 92+0+0 (secure 0 0 0) 0x7fd26c06a140 con 0x7fd280159820 2023-10-06T12:34:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 msgr2=0x7fd264062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 0x7fd264062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd2740099b0 tx=0x7fd274017000 comp rx=0 tx=0).stop 2023-10-06T12:34:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 msgr2=0x7fd28015c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00c5c0 tx=0x7fd26c00c8d0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 shutdown_connections 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7fd264060670 0x7fd264062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd280159c70 0x7fd28015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd280159820 0x7fd28015c020 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 --2- 172.21.15.46:0/2006210088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2800afc40 0x7fd280158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 >> 172.21.15.46:0/2006210088 conn(0x7fd28006aea0 msgr2=0x7fd28015ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 shutdown_connections 2023-10-06T12:34:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:28.851+0000 7fd286e6e700 1 -- 172.21.15.46:0/2006210088 wait complete. 2023-10-06T12:34:28.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T12:34:29.031 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:29.026+0000 7f08c6984000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:34:29.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.039+0000 7f73b55cc700 1 Processor -- start 2023-10-06T12:34:29.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.039+0000 7f73b55cc700 1 -- start start 2023-10-06T12:34:29.044 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:29.039+0000 7f34c58e7000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:34:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b55cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb180 0x7f73b00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b55cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73b00cb550 0x7f73b006c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b55cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b00cc190 con 0x7f73b00cd4c0 2023-10-06T12:34:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b55cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b00cdc80 con 0x7f73b00cb550 2023-10-06T12:34:29.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b55cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b006d260 con 0x7f73b00cb180 2023-10-06T12:34:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73af7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f73b00cd4c0 0x7f73b0157e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32816/0 (socket says 172.21.15.46:32816) 2023-10-06T12:34:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73af7fe700 1 -- 172.21.15.46:0/667908715 learned_addr learned my addr 172.21.15.46:0/667908715 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/667908715 >> v2:172.21.15.46:3300/0 conn(0x7f73b00cb550 0x7f73b006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73affff700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb180 0x7f73b00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73aeffd700 1 -- 172.21.15.46:0/667908715 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274188819 0 0) 0x7f73b00cc190 con 0x7f73b00cd4c0 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73aeffd700 1 -- 172.21.15.46:0/667908715 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7398003570 con 0x7f73b00cd4c0 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 -- 172.21.15.46:0/667908715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb180 msgr2=0x7f73b00cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb180 0x7f73b00cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 -- 172.21.15.46:0/667908715 >> v1:172.21.15.46:6789/0 conn(0x7f73b00cd4c0 legacy=0x7f73b0157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 -- 172.21.15.46:0/667908715 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00cd9a0 con 0x7f73b00cb550 2023-10-06T12:34:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.043+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cb550 0x7f73b006c370 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009e60 tx=0x7f73a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a474e1215912a67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73aeffd700 1 -- 172.21.15.46:0/667908715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800e070 con 0x7f73b00cb550 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73aeffd700 1 -- 172.21.15.46:0/667908715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a800bab0 con 0x7f73b00cb550 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73aeffd700 1 -- 172.21.15.46:0/667908715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a800be70 con 0x7f73b00cb550 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 -- 172.21.15.46:0/667908715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cb550 msgr2=0x7f73b006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cb550 0x7f73b006c370 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009e60 tx=0x7f73a8009780 comp rx=0 tx=0).stop 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 -- 172.21.15.46:0/667908715 shutdown_connections 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cb550 0x7f73b006c370 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 --2- 172.21.15.46:0/667908715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb180 0x7f73b00cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 -- 172.21.15.46:0/667908715 >> 172.21.15.46:0/667908715 conn(0x7f73b006ad30 msgr2=0x7f73b006b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 -- 172.21.15.46:0/667908715 shutdown_connections 2023-10-06T12:34:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 -- 172.21.15.46:0/667908715 wait complete. 2023-10-06T12:34:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.047+0000 7f73b55cc700 1 Processor -- start 2023-10-06T12:34:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 -- start start 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73b00cb180 0x7f73b014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb550 0x7f73b00c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0168f30 con 0x7f73b00cd4c0 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00c7b60 con 0x7f73b00cb180 2023-10-06T12:34:29.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b55cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b01691f0 con 0x7f73b00cb550 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50390/0 (socket says 172.21.15.46:50390) 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 -- 172.21.15.46:0/1865148384 learned_addr learned my addr 172.21.15.46:0/1865148384 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73affff700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73b00cb180 0x7f73b014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73af7fe700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb550 0x7f73b00c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 -- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb550 msgr2=0x7f73b00c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb550 0x7f73b00c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 -- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73b00cb180 msgr2=0x7f73b014f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73b00cb180 0x7f73b014f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.051+0000 7f73b4dcb700 1 -- 172.21.15.46:0/1865148384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8009a20 con 0x7f73b00cd4c0 2023-10-06T12:34:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f73b4dcb700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f73b0063750 tx=0x7f73a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8012e50 con 0x7f73b00cd4c0 2023-10-06T12:34:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8027470 con 0x7f73b00cd4c0 2023-10-06T12:34:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b01694d0 con 0x7f73b00cd4c0 2023-10-06T12:34:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a8027830 con 0x7f73b00cd4c0 2023-10-06T12:34:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b014fd70 con 0x7f73b00cd4c0 2023-10-06T12:34:29.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.055+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0064590 con 0x7f73b00cd4c0 2023-10-06T12:34:29.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.059+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 85) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f73a8024070 con 0x7f73b00cd4c0 2023-10-06T12:34:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.059+0000 7f7397fff700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 0x7f739c062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.059+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f73a807f3a0 con 0x7f73b00cd4c0 2023-10-06T12:34:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.059+0000 7f73affff700 1 -- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 msgr2=0x7f739c062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.059+0000 7f73affff700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 0x7f739c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:29.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.063+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f73a804f480 con 0x7f73b00cd4c0 2023-10-06T12:34:29.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.235+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f73b00cbb40 con 0x7f73b00cd4c0 2023-10-06T12:34:29.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.243+0000 7f7397fff700 1 -- 172.21.15.46:0/1865148384 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73a8052aa0 con 0x7f73b00cd4c0 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 msgr2=0x7f739c062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 0x7f739c062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 msgr2=0x7f73b00b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f73b0063750 tx=0x7f73a8019000 comp rx=0 tx=0).stop 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 shutdown_connections 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f739c0606c0 0x7f739c062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73b00cd4c0 0x7f73b00b0be0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73b00cb550 0x7f73b00c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 --2- 172.21.15.46:0/1865148384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73b00cb180 0x7f73b014f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 >> 172.21.15.46:0/1865148384 conn(0x7f73b006ad30 msgr2=0x7f73b0150750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 shutdown_connections 2023-10-06T12:34:29.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.247+0000 7f73b55cc700 1 -- 172.21.15.46:0/1865148384 wait complete. 2023-10-06T12:34:29.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T12:34:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 Processor -- start 2023-10-06T12:34:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 -- start start 2023-10-06T12:34:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f08ec156e40 0x7f08ec069d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec15b880 con 0x7f08ec0b27c0 2023-10-06T12:34:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec15ac70 con 0x7f08ec157210 2023-10-06T12:34:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f44f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0cab50 con 0x7f08ec156e40 2023-10-06T12:34:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f2a8d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08ec0b27c0 0x7f08ec06dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32830/0 (socket says 172.21.15.46:32830) 2023-10-06T12:34:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f2a8d700 1 -- 172.21.15.46:0/1170016819 learned_addr learned my addr 172.21.15.46:0/1170016819 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f228c700 1 --2- 172.21.15.46:0/1170016819 >> v2:172.21.15.132:3300/0 conn(0x7f08ec156e40 0x7f08ec069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075873155 0 0) 0x7f08ec15b880 con 0x7f08ec0b27c0 2023-10-06T12:34:29.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d8003570 con 0x7f08ec0b27c0 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1737727539 0 0) 0x7f08d8003570 con 0x7f08ec0b27c0 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 >> v2:172.21.15.132:3300/0 conn(0x7f08ec156e40 msgr2=0x7f08ec069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 --2- 172.21.15.46:0/1170016819 >> v2:172.21.15.132:3300/0 conn(0x7f08ec156e40 0x7f08ec069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 >> v1:172.21.15.132:6789/0 conn(0x7f08ec157210 legacy=0x7f08ec0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec15af50 con 0x7f08ec0b27c0 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.431+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f08e40017f0 con 0x7f08ec0b27c0 2023-10-06T12:34:29.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 >> v1:172.21.15.46:6789/0 conn(0x7f08ec0b27c0 legacy=0x7f08ec06dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d8004810 0x7f08d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d80051c0 0x7f08d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0cab50 con 0x7f08d8004810 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d800c0e0 con 0x7f08d8003a60 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d800c2a0 con 0x7f08d80051c0 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f1a8b700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d8004810 0x7f08d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f2a8d700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d80051c0 0x7f08d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 -- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d80051c0 msgr2=0x7f08d800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08d80051c0 0x7f08d800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 -- 172.21.15.46:0/1170016819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d8004810 msgr2=0x7f08d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08d8004810 0x7f08d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 -- 172.21.15.46:0/1170016819 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8003570 con 0x7f08d8003a60 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f2a8d700 1 -- 172.21.15.46:0/1170016819 reap_dead start 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f228c700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08d8003e30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009c00 tx=0x7f08e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d385ecc1ace0dde4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800e710 con 0x7f08d8003a60 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800e870 con 0x7f08d8003a60 2023-10-06T12:34:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f128a700 1 -- 172.21.15.46:0/1170016819 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800ec30 con 0x7f08d8003a60 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 -- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 msgr2=0x7f08d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08d8003e30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009c00 tx=0x7f08e80091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 -- 172.21.15.46:0/1170016819 shutdown_connections 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 --2- 172.21.15.46:0/1170016819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08d8003e30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 -- 172.21.15.46:0/1170016819 >> 172.21.15.46:0/1170016819 conn(0x7f08ec068c30 msgr2=0x7f08ec0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.435+0000 7f08f44f0700 1 -- 172.21.15.46:0/1170016819 shutdown_connections 2023-10-06T12:34:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.439+0000 7f08f44f0700 1 -- 172.21.15.46:0/1170016819 wait complete. 2023-10-06T12:34:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.439+0000 7f08f44f0700 1 Processor -- start 2023-10-06T12:34:29.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.439+0000 7f08f44f0700 1 -- start start 2023-10-06T12:34:29.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08ec0cac90 0x7f08ec0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 0x7f08ec0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0b1af0 con 0x7f08ec0b1670 2023-10-06T12:34:29.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec169740 con 0x7f08d8003a60 2023-10-06T12:34:29.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec169a20 con 0x7f08ec0cac90 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f2a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 0x7f08ec0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:55022/0 (socket says 172.21.15.46:55022) 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 -- 172.21.15.46:0/488094919 learned_addr learned my addr 172.21.15.46:0/488094919 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f1a8b700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08ec0cac90 0x7f08ec0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08ec0cac90 msgr2=0x7f08ec0b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08ec0cac90 0x7f08ec0b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 msgr2=0x7f08ec0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 0x7f08ec0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 -- 172.21.15.46:0/488094919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec169d00 con 0x7f08d8003a60 2023-10-06T12:34:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f2a8d700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 0x7f08ec0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:34:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f228c700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f08e800e4b0 tx=0x7f08e801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8022070 con 0x7f08d8003a60 2023-10-06T12:34:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800e9e0 con 0x7f08d8003a60 2023-10-06T12:34:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e80276a0 con 0x7f08d8003a60 2023-10-06T12:34:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec169fe0 con 0x7f08d8003a60 2023-10-06T12:34:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.443+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0cc7b0 con 0x7f08d8003a60 2023-10-06T12:34:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec063820 con 0x7f08d8003a60 2023-10-06T12:34:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 85) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f08e801c450 con 0x7f08d8003a60 2023-10-06T12:34:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08e27fc700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 0x7f08d0062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08f1a8b700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 msgr2=0x7f08d0062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f08e807e710 con 0x7f08d8003a60 2023-10-06T12:34:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.447+0000 7f08f1a8b700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 0x7f08d0062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.459+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f08e804e970 con 0x7f08d8003a60 2023-10-06T12:34:29.595 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:29.594+0000 7f08c6984000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:34:29.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.639+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f08ec04b5c0 con 0x7f08d8003a60 2023-10-06T12:34:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.647+0000 7f08e27fc700 1 -- 172.21.15.46:0/488094919 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f08e8051ed0 con 0x7f08d8003a60 2023-10-06T12:34:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f1a8b700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 msgr2=0x7f08d0062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f1a8b700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 0x7f08d0062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 msgr2=0x7f08d0062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 0x7f08d0062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 msgr2=0x7f08ec0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f08e800e4b0 tx=0x7f08e801a000 comp rx=0 tx=0).stop 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 shutdown_connections 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f08d00606f0 0x7f08d0062ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08ec0b1670 0x7f08ec0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08ec0cac90 0x7f08ec0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 --2- 172.21.15.46:0/488094919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08d8003a60 0x7f08ec0cbaa0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 >> 172.21.15.46:0/488094919 conn(0x7f08ec068c30 msgr2=0x7f08ec155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 shutdown_connections 2023-10-06T12:34:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.651+0000 7f08f44f0700 1 -- 172.21.15.46:0/488094919 wait complete. 2023-10-06T12:34:29.668 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:29.666+0000 7f08c6984000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:34:29.671 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:29.667+0000 7f34c58e7000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:34:29.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:34:29.674 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T12:34:29.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:34:29.704 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-06T12:34:29.752 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:29.747+0000 7f34c58e7000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:34:29.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T12:34:29.809 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:29.806+0000 7f08c6984000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:29.892 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:29.887+0000 7f34c58e7000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:29.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 Processor -- start 2023-10-06T12:34:29.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 -- start start 2023-10-06T12:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98f41595c0 0x7f98f40af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98f40cc500 0x7f98f40afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f40cf360 con 0x7f98f41591f0 2023-10-06T12:34:29.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40cff00 con 0x7f98f40cc500 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98fa68e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40b03c0 con 0x7f98f41595c0 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98f40cc500 0x7f98f40afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98f40cc500 0x7f98f40afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50424/0 (socket says 172.21.15.46:50424) 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 -- 172.21.15.46:0/412532698 learned_addr learned my addr 172.21.15.46:0/412532698 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f968c700 1 --1- 172.21.15.46:0/412532698 >> v1:172.21.15.46:6789/0 conn(0x7f98f41591f0 0x7f98f40cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32842/0 (socket says 172.21.15.46:32842) 2023-10-06T12:34:29.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f8e8b700 1 --2- 172.21.15.46:0/412532698 >> v2:172.21.15.132:3300/0 conn(0x7f98f41595c0 0x7f98f40af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 -- 172.21.15.46:0/412532698 >> v2:172.21.15.132:3300/0 conn(0x7f98f41595c0 msgr2=0x7f98f40af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 --2- 172.21.15.46:0/412532698 >> v2:172.21.15.132:3300/0 conn(0x7f98f41595c0 0x7f98f40af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 -- 172.21.15.46:0/412532698 >> v1:172.21.15.46:6789/0 conn(0x7f98f41591f0 legacy=0x7f98f40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.971+0000 7f98f9e8d700 1 -- 172.21.15.46:0/412532698 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f415dd60 con 0x7f98f40cc500 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98f9e8d700 1 --2- 172.21.15.46:0/412532698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f40cc500 0x7f98f40afe80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f98f4088580 tx=0x7f98ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579019df0ce5f7e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98ebfff700 1 -- 172.21.15.46:0/412532698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00e070 con 0x7f98f40cc500 2023-10-06T12:34:29.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98ebfff700 1 -- 172.21.15.46:0/412532698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00bae0 con 0x7f98f40cc500 2023-10-06T12:34:29.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98ebfff700 1 -- 172.21.15.46:0/412532698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0092e0 con 0x7f98f40cc500 2023-10-06T12:34:29.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- 172.21.15.46:0/412532698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f40cc500 msgr2=0x7f98f40afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 --2- 172.21.15.46:0/412532698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f40cc500 0x7f98f40afe80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f98f4088580 tx=0x7f98ec009840 comp rx=0 tx=0).stop 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- 172.21.15.46:0/412532698 shutdown_connections 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 --2- 172.21.15.46:0/412532698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f40cc500 0x7f98f40afe80 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 --2- 172.21.15.46:0/412532698 >> v2:172.21.15.132:3300/0 conn(0x7f98f41595c0 0x7f98f40af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- 172.21.15.46:0/412532698 >> 172.21.15.46:0/412532698 conn(0x7f98f406ef60 msgr2=0x7f98f40d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- 172.21.15.46:0/412532698 shutdown_connections 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- 172.21.15.46:0/412532698 wait complete. 2023-10-06T12:34:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 Processor -- start 2023-10-06T12:34:29.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.975+0000 7f98fa68e700 1 -- start start 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 0x7f98f415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98f4169ee0 0x7f98f416a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4099dc0 con 0x7f98f41591f0 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f416b570 con 0x7f98f40cc500 2023-10-06T12:34:29.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f416a960 con 0x7f98f4169ee0 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50440/0 (socket says 172.21.15.46:50440) 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 -- 172.21.15.46:0/929153944 learned_addr learned my addr 172.21.15.46:0/929153944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 0x7f98f415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98f4169ee0 msgr2=0x7f98f416a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98f4169ee0 0x7f98f416a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 msgr2=0x7f98f415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 0x7f98f415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009ae0 con 0x7f98f41591f0 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 0x7f98f415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:34:29.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98f8e8b700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d890 tx=0x7f98e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40125f0 con 0x7f98f41591f0 2023-10-06T12:34:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4012750 con 0x7f98f41591f0 2023-10-06T12:34:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f416ac40 con 0x7f98f41591f0 2023-10-06T12:34:29.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4012ad0 con 0x7f98f41591f0 2023-10-06T12:34:29.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f416b200 con 0x7f98f41591f0 2023-10-06T12:34:29.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.979+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4064590 con 0x7f98f41591f0 2023-10-06T12:34:29.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.983+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 85) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f98e401e460 con 0x7f98f41591f0 2023-10-06T12:34:29.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.983+0000 7f98e97fa700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.983+0000 7f98f968c700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.983+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6574+0+0 (secure 0 0 0) 0x7f98e407d3f0 con 0x7f98f41591f0 2023-10-06T12:34:29.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.983+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:34:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:29.987+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98e404d4d0 con 0x7f98f41591f0 2023-10-06T12:34:30.074 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.070+0000 7f08c6984000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:34:30.148 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.146+0000 7f08c6984000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:30.164 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:30.159+0000 7f34c58e7000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:34:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:30.179+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f416b6b0 con 0x7f98d80606f0 2023-10-06T12:34:30.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:30.187+0000 7f98f968c700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:30.187+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:34:30.236 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:30.231+0000 7f34c58e7000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:30.370 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.366+0000 7f08c6984000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:34:30.450 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.446+0000 7f08c6984000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:34:30.465 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:30.459+0000 7f34c58e7000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:34:30.543 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.538+0000 7f08c6984000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:34:30.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:30.547+0000 7f34c58e7000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:34:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:30.591+0000 7f98f968c700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:30.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:30.591+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T12:34:30.644 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:30.639+0000 7f34c58e7000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:34:30.942 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:30.938+0000 7f08c6984000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:34:31.054 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:31.051+0000 7f34c58e7000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:34:31.109 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.106+0000 7f08c6984000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:34:31.226 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:31.223+0000 7f34c58e7000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:34:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:31.395+0000 7f98f968c700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:31.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:31.395+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T12:34:31.472 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.470+0000 7f08c6984000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:34:31.612 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:31.607+0000 7f34c58e7000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:34:31.677 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.674+0000 7f08c6984000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:31.764 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.762+0000 7f08c6984000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:34:31.838 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.834+0000 7f08c6984000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:34:31.840 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:31.835+0000 7f34c58e7000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:34:31.910 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:31.906+0000 7f08c6984000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:34:31.935 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:31.931+0000 7f34c58e7000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:34:32.017 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.011+0000 7f34c58e7000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:34:32.029 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:34:32.037 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:34:32.091 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.087+0000 7f34c58e7000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:34:32.123 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:32.118+0000 7f08c6984000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:32.192 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:32.190+0000 7f08c6984000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:34:32.273 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:32.270+0000 7f08c6984000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:34:32.305 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.299+0000 7f34c58e7000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:34:32.383 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.379+0000 7f34c58e7000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:34:32.459 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:32.454+0000 7f08c6984000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:34:32.473 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.471+0000 7f34c58e7000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:34:32.543 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:32.538+0000 7f08c6984000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:34:32.669 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.663+0000 7f34c58e7000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:34:32.754 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:32.751+0000 7f34c58e7000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:34:33.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:32.999+0000 7f98f968c700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/32974 2023-10-06T12:34:33.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:32.999+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T12:34:33.238 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:34:33.234+0000 7f08c6984000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:34:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:33.303+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 86) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f98e404d8a0 con 0x7f98f41591f0 2023-10-06T12:34:33.508 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.503+0000 7f34c58e7000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:34:33.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:33.519+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 87) v1 ==== 79687+0+0 (secure 0 0 0) 0x7f98e401c360 con 0x7f98f41591f0 2023-10-06T12:34:33.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:33.523+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:34:33.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:33.523+0000 7f98e97fa700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:33.719 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3485ec9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.719 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3485ec9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.719 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3485ec9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.719 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3485ec9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.719 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3485ec9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3488ecf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3488ecf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3488ecf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3488ecf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:33.720 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:34:33.715+0000 7f3488ecf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:34:34.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.531+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 88) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f98e404e170 con 0x7f98f41591f0 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.531+0000 7f98e97fa700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.531+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f98f416b6b0 con 0x7f98d80606f0 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.531+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.539+0000 7f98f968c700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98f4166900 tx=0x7f98f0001490 comp rx=0 tx=0).ready entity=mgr.9480 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.647+0000 7f98e97fa700 1 -- 172.21.15.46:0/929153944 <== mgr.9480 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f98f416b6b0 con 0x7f98d80606f0 2023-10-06T12:34:34.890 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-06T12:34:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 msgr2=0x7f98d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:34.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062b20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98f4166900 tx=0x7f98f0001490 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 msgr2=0x7f98f416ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d890 tx=0x7f98e4008040 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 shutdown_connections 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f98d80606f0 0x7f98d8062b20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98f4169ee0 0x7f98f416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f41591f0 0x7f98f416ba00 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 --2- 172.21.15.46:0/929153944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98f40cc500 0x7f98f415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 >> 172.21.15.46:0/929153944 conn(0x7f98f406ef60 msgr2=0x7f98f415df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 shutdown_connections 2023-10-06T12:34:34.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:34.651+0000 7f98fa68e700 1 -- 172.21.15.46:0/929153944 wait complete. 2023-10-06T12:34:34.894 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-06T12:34:34.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T12:34:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.067+0000 7f37f5bc3700 1 Processor -- start 2023-10-06T12:34:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.071+0000 7f37f5bc3700 1 -- start start 2023-10-06T12:34:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.071+0000 7f37f5bc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 0x7f37f00b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.071+0000 7f37f5bc3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f00b5eb0 con 0x7f37f0158de0 2023-10-06T12:34:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.071+0000 7f37f5bc3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f00cef30 con 0x7f37f01591b0 2023-10-06T12:34:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37f5bc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00cf4f0 con 0x7f37f0069770 2023-10-06T12:34:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37f4bc1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f37f0158de0 0x7f37f0069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46444/0 (socket says 172.21.15.46:46444) 2023-10-06T12:34:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37f4bc1700 1 -- 172.21.15.46:0/2685816631 learned_addr learned my addr 172.21.15.46:0/2685816631 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:34:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37effff700 1 --2- 172.21.15.46:0/2685816631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 0x7f37f00b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218891285 0 0) 0x7f37e4001110 con 0x7f37f0158de0 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e0003570 con 0x7f37f0158de0 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619779427 0 0) 0x7f37f00cef30 con 0x7f37f01591b0 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e4001110 con 0x7f37f01591b0 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37effff700 1 -- 172.21.15.46:0/2685816631 >> v1:172.21.15.132:6789/0 conn(0x7f37f01591b0 legacy=0x7f37f00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591401198 0 0) 0x7f37e0003570 con 0x7f37f0158de0 2023-10-06T12:34:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37effff700 1 -- 172.21.15.46:0/2685816631 >> v1:172.21.15.46:6789/0 conn(0x7f37f0158de0 legacy=0x7f37f0069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37effff700 1 -- 172.21.15.46:0/2685816631 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00ccac0 con 0x7f37f0069770 2023-10-06T12:34:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37effff700 1 --2- 172.21.15.46:0/2685816631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 0x7f37f00b5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009910 tx=0x7f37dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23e9d1f208062e17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4001600 con 0x7f37f0069770 2023-10-06T12:34:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc011420 con 0x7f37f0069770 2023-10-06T12:34:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.075+0000 7f37ef7fe700 1 -- 172.21.15.46:0/2685816631 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0117c0 con 0x7f37f0069770 2023-10-06T12:34:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 -- 172.21.15.46:0/2685816631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 msgr2=0x7f37f00b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/2685816631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 0x7f37f00b5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009910 tx=0x7f37dc0091a0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 -- 172.21.15.46:0/2685816631 shutdown_connections 2023-10-06T12:34:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/2685816631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f0069770 0x7f37f00b5970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 -- 172.21.15.46:0/2685816631 >> 172.21.15.46:0/2685816631 conn(0x7f37f0068390 msgr2=0x7f37f00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 -- 172.21.15.46:0/2685816631 shutdown_connections 2023-10-06T12:34:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 -- 172.21.15.46:0/2685816631 wait complete. 2023-10-06T12:34:35.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.079+0000 7f37f5bc3700 1 Processor -- start 2023-10-06T12:34:35.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 -- start start 2023-10-06T12:34:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f0069770 0x7f37f016a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f01591b0 0x7f37f01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f015d3d0 con 0x7f37f0158de0 2023-10-06T12:34:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f015aff0 con 0x7f37f0069770 2023-10-06T12:34:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f5bc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f016a8c0 con 0x7f37f01591b0 2023-10-06T12:34:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58680/0 (socket says 172.21.15.46:58680) 2023-10-06T12:34:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f4bc1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f0069770 0x7f37f016a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 -- 172.21.15.46:0/3819710035 learned_addr learned my addr 172.21.15.46:0/3819710035 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:34:35.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37f53c2700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f01591b0 0x7f37f01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 -- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f01591b0 msgr2=0x7f37f01539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f01591b0 0x7f37f01539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 -- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f0069770 msgr2=0x7f37f016a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f0069770 0x7f37f016a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.083+0000 7f37effff700 1 -- 172.21.15.46:0/3819710035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc009420 con 0x7f37f0158de0 2023-10-06T12:34:35.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37effff700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0091e0 tx=0x7f37dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc01cb30 con 0x7f37f0158de0 2023-10-06T12:34:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc01cc90 con 0x7f37f0158de0 2023-10-06T12:34:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f015a730 con 0x7f37f0158de0 2023-10-06T12:34:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc01a6a0 con 0x7f37f0158de0 2023-10-06T12:34:35.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f015ac80 con 0x7f37f0158de0 2023-10-06T12:34:35.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.087+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f015b180 con 0x7f37f0158de0 2023-10-06T12:34:35.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.091+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 88) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f37dc01aad0 con 0x7f37f0158de0 2023-10-06T12:34:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.091+0000 7f37ecff9700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 0x7f37d40628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:34:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.091+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f37dc07e7e0 con 0x7f37f0158de0 2023-10-06T12:34:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.091+0000 7f37f4bc1700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 0x7f37d40628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:34:35.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.091+0000 7f37f4bc1700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 0x7f37d40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f37f016ceb0 tx=0x7f37e4001330 comp rx=0 tx=0).ready entity=mgr.9480 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:34:35.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.095+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f37dc04eaf0 con 0x7f37f0158de0 2023-10-06T12:34:35.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.291+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37f0064590 con 0x7f37d40603f0 2023-10-06T12:34:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.383+0000 7f37ecff9700 1 -- 172.21.15.46:0/3819710035 <== mgr.9480 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f37f0064590 con 0x7f37d40603f0 2023-10-06T12:34:35.389 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-06T12:34:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 msgr2=0x7f37d40628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 0x7f37d40628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f37f016ceb0 tx=0x7f37e4001330 comp rx=0 tx=0).stop 2023-10-06T12:34:35.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 msgr2=0x7f37f00c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:34:35.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0091e0 tx=0x7f37dc016000 comp rx=0 tx=0).stop 2023-10-06T12:34:35.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 shutdown_connections 2023-10-06T12:34:35.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f37d40603f0 0x7f37d40628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37f01591b0 0x7f37f01539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f0158de0 0x7f37f00c4df0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 --2- 172.21.15.46:0/3819710035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37f0069770 0x7f37f016a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 >> 172.21.15.46:0/3819710035 conn(0x7f37f0068390 msgr2=0x7f37f0154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 shutdown_connections 2023-10-06T12:34:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:34:35.387+0000 7f37f5bc3700 1 -- 172.21.15.46:0/3819710035 wait complete. 2023-10-06T12:34:35.416 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-06T12:35:00.409 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:35:00.403+0000 7f3477ead700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:35:00.409 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:35:00.403+0000 7f3477ead700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:35:00.410 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:35:00.403+0000 7f3477ead700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:35:00.410 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:35:00.403+0000 7f3477ead700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:35:00.410 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:35:00.403+0000 7f3477ead700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:35:02.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:35:02.177 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:35:32.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:35:32.332 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:36:02.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:36:02.472 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:36:32.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:36:32.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:37:02.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:37:02.673 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:37:32.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:37:32.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:38:02.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:38:02.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:38:33.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:38:33.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:39:03.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:39:03.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:39:33.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:39:33.321 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:40:03.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:40:03.484 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:40:33.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:40:33.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:41:03.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:41:03.746 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:41:33.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:41:33.836 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:42:03.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:42:03.999 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:42:34.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:42:34.146 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:43:04.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:43:04.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:43:34.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:43:34.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:44:04.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:44:04.620 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:44:34.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:44:34.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:45:04.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:45:04.865 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:45:16.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:45:16.417 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_35_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_36_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_37_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_38_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_39_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_40_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_41_00_UTC 2023-10-06T12:45:16.443 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_42_00_UTC 2023-10-06T12:45:16.444 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_43_00_UTC 2023-10-06T12:45:16.444 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_44_00_UTC 2023-10-06T12:45:16.444 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_45_00_UTC 2023-10-06T12:45:16.445 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-12_35_00_UTC', 'scheduled-2023-10-06-12_36_00_UTC', 'scheduled-2023-10-06-12_37_00_UTC', 'scheduled-2023-10-06-12_38_00_UTC', 'scheduled-2023-10-06-12_39_00_UTC', 'scheduled-2023-10-06-12_40_00_UTC', 'scheduled-2023-10-06-12_41_00_UTC', 'scheduled-2023-10-06-12_42_00_UTC', 'scheduled-2023-10-06-12_43_00_UTC', 'scheduled-2023-10-06-12_44_00_UTC', 'scheduled-2023-10-06-12_45_00_UTC'] 2023-10-06T12:45:16.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-06T12:45:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.652+0000 7f494e4f0700 1 Processor -- start 2023-10-06T12:45:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.652+0000 7f494e4f0700 1 -- start start 2023-10-06T12:45:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f494e4f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4948159050 0x7f49480c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f494e4f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4948159420 0x7f49480ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f494e4f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49480c71a0 con 0x7f49480c7bd0 2023-10-06T12:45:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f494e4f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480c86e0 con 0x7f4948159420 2023-10-06T12:45:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f494e4f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49480c1bd0 con 0x7f4948159050 2023-10-06T12:45:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4948159420 0x7f49480ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4948159420 0x7f49480ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50258/0 (socket says 172.21.15.46:50258) 2023-10-06T12:45:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 -- 172.21.15.46:0/838925327 learned_addr learned my addr 172.21.15.46:0/838925327 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:45:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f4947fff700 1 --2- 172.21.15.46:0/838925327 >> v2:172.21.15.132:3300/0 conn(0x7f4948159050 0x7f49480c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 -- 172.21.15.46:0/838925327 >> v2:172.21.15.132:3300/0 conn(0x7f4948159050 msgr2=0x7f49480c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:45:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f4946ffd700 1 -- 172.21.15.46:0/838925327 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490191230 0 0) 0x7f49480c71a0 con 0x7f49480c7bd0 2023-10-06T12:45:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 --2- 172.21.15.46:0/838925327 >> v2:172.21.15.132:3300/0 conn(0x7f4948159050 0x7f49480c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 -- 172.21.15.46:0/838925327 >> v1:172.21.15.132:6789/0 conn(0x7f49480c7bd0 legacy=0x7f49480c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 -- 172.21.15.46:0/838925327 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480c6590 con 0x7f4948159420 2023-10-06T12:45:16.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f49477fe700 1 --2- 172.21.15.46:0/838925327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480ca1d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f494806c340 tx=0x7f493c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ec0a4e4197891 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f4946ffd700 1 -- 172.21.15.46:0/838925327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c009e70 con 0x7f4948159420 2023-10-06T12:45:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f4946ffd700 1 -- 172.21.15.46:0/838925327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00b860 con 0x7f4948159420 2023-10-06T12:45:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.656+0000 7f4946ffd700 1 -- 172.21.15.46:0/838925327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00bc00 con 0x7f4948159420 2023-10-06T12:45:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- 172.21.15.46:0/838925327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 msgr2=0x7f49480ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 --2- 172.21.15.46:0/838925327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480ca1d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f494806c340 tx=0x7f493c009250 comp rx=0 tx=0).stop 2023-10-06T12:45:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- 172.21.15.46:0/838925327 shutdown_connections 2023-10-06T12:45:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 --2- 172.21.15.46:0/838925327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480ca1d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 --2- 172.21.15.46:0/838925327 >> v2:172.21.15.132:3300/0 conn(0x7f4948159050 0x7f49480c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- 172.21.15.46:0/838925327 >> 172.21.15.46:0/838925327 conn(0x7f49480cbe20 msgr2=0x7f49480cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:45:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- 172.21.15.46:0/838925327 shutdown_connections 2023-10-06T12:45:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- 172.21.15.46:0/838925327 wait complete. 2023-10-06T12:45:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 Processor -- start 2023-10-06T12:45:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.660+0000 7f494e4f0700 1 -- start start 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49480c7bd0 0x7f49480c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4948159050 0x7f49480c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480ac650 con 0x7f4948159420 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49480ad320 con 0x7f49480c7bd0 2023-10-06T12:45:16.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49480ac370 con 0x7f4948159050 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50268/0 (socket says 172.21.15.46:50268) 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 -- 172.21.15.46:0/2289186561 learned_addr learned my addr 172.21.15.46:0/2289186561 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f49477fe700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4948159050 0x7f49480c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f4947fff700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49480c7bd0 0x7f49480c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 -- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4948159050 msgr2=0x7f49480c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4948159050 0x7f49480c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 -- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49480c7bd0 msgr2=0x7f49480c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49480c7bd0 0x7f49480c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 -- 172.21.15.46:0/2289186561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0094f0 con 0x7f4948159420 2023-10-06T12:45:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494ca8d700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f494000c8c0 tx=0x7f4940001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940012630 con 0x7f4948159420 2023-10-06T12:45:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4940012790 con 0x7f4948159420 2023-10-06T12:45:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940012b50 con 0x7f4948159420 2023-10-06T12:45:16.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49480accd0 con 0x7f4948159420 2023-10-06T12:45:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.664+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49480c6010 con 0x7f4948159420 2023-10-06T12:45:16.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.668+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49480ad4b0 con 0x7f4948159420 2023-10-06T12:45:16.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.668+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 91) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f494001a3a0 con 0x7f4948159420 2023-10-06T12:45:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.672+0000 7f492ffff700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 0x7f4934062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.672+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6976+0+0 (secure 0 0 0) 0x7f494007d090 con 0x7f4948159420 2023-10-06T12:45:16.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.672+0000 7f4947fff700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 0x7f4934062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.672+0000 7f4947fff700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 0x7f4934062b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4938009db0 tx=0x7f49380091a0 comp rx=0 tx=0).ready entity=mgr.9480 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:16.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.680+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f494004d000 con 0x7f4948159420 2023-10-06T12:45:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.872+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 --> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f494806eed0 con 0x7f49340606c0 2023-10-06T12:45:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.888+0000 7f492ffff700 1 -- 172.21.15.46:0/2289186561 <== mgr.9480 v2:172.21.15.46:6832/32974 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f494806eed0 con 0x7f49340606c0 2023-10-06T12:45:16.894 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:45:16.894 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T12:34:34", "first": "2023-10-06T12:35:00", "last": "2023-10-06T12:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-06T12:45:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 msgr2=0x7f4934062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 0x7f4934062b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4938009db0 tx=0x7f49380091a0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 msgr2=0x7f49480aa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f494000c8c0 tx=0x7f4940001670 comp rx=0 tx=0).stop 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 shutdown_connections 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f49340606c0 0x7f4934062b70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4948159420 0x7f49480aa720 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4948159050 0x7f49480c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 --2- 172.21.15.46:0/2289186561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49480c7bd0 0x7f49480c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 >> 172.21.15.46:0/2289186561 conn(0x7f49480cbe20 msgr2=0x7f4948069490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:45:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 shutdown_connections 2023-10-06T12:45:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:16.892+0000 7f494e4f0700 1 -- 172.21.15.46:0/2289186561 wait complete. 2023-10-06T12:45:16.923 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T12:34:34", "first": "2023-10-06T12:35:00", "last": "2023-10-06T12:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-06T12:45:16.923 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T12:34:34', 'first': '2023-10-06T12:35:00', 'last': '2023-10-06T12:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-06T12:45:16.924 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T12:45:16.924 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T12:45:16.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T12:45:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 Processor -- start 2023-10-06T12:45:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 -- start start 2023-10-06T12:45:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7568161840 0x7f75680caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75680caf50 0x7f756806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75680c18f0 con 0x7f7568161470 2023-10-06T12:45:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75680c0ce0 con 0x7f7568161840 2023-10-06T12:45:17.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f756db45700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75680c0fc0 con 0x7f75680caf50 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f755ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7568161840 0x7f75680caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f755ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7568161840 0x7f75680caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50276/0 (socket says 172.21.15.46:50276) 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f755ffff700 1 -- 172.21.15.46:0/2500734280 learned_addr learned my addr 172.21.15.46:0/2500734280 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.092+0000 7f7567fff700 1 --2- 172.21.15.46:0/2500734280 >> v2:172.21.15.132:3300/0 conn(0x7f75680caf50 0x7f756806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f755ffff700 1 -- 172.21.15.46:0/2500734280 >> v2:172.21.15.132:3300/0 conn(0x7f75680caf50 msgr2=0x7f756806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:45:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f755ffff700 1 --2- 172.21.15.46:0/2500734280 >> v2:172.21.15.132:3300/0 conn(0x7f75680caf50 0x7f756806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f755ffff700 1 -- 172.21.15.46:0/2500734280 >> v1:172.21.15.132:6789/0 conn(0x7f7568161470 legacy=0x7f75680c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:45:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f755ffff700 1 -- 172.21.15.46:0/2500734280 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680c12a0 con 0x7f7568161840 2023-10-06T12:45:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f755ffff700 1 --2- 172.21.15.46:0/2500734280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7568161840 0x7f75680caa10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f75680c2e90 tx=0x7f7550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63df3aa1998945df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f7566ffd700 1 -- 172.21.15.46:0/2500734280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000b8b0 con 0x7f7568161840 2023-10-06T12:45:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f7566ffd700 1 -- 172.21.15.46:0/2500734280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755000ba10 con 0x7f7568161840 2023-10-06T12:45:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f7566ffd700 1 -- 172.21.15.46:0/2500734280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000bdb0 con 0x7f7568161840 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 -- 172.21.15.46:0/2500734280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7568161840 msgr2=0x7f75680caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 --2- 172.21.15.46:0/2500734280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7568161840 0x7f75680caa10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f75680c2e90 tx=0x7f7550009250 comp rx=0 tx=0).stop 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 -- 172.21.15.46:0/2500734280 shutdown_connections 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 --2- 172.21.15.46:0/2500734280 >> v2:172.21.15.132:3300/0 conn(0x7f75680caf50 0x7f756806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 --2- 172.21.15.46:0/2500734280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7568161840 0x7f75680caa10 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 -- 172.21.15.46:0/2500734280 >> 172.21.15.46:0/2500734280 conn(0x7f75680cbe30 msgr2=0x7f75680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 -- 172.21.15.46:0/2500734280 shutdown_connections 2023-10-06T12:45:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.096+0000 7f756db45700 1 -- 172.21.15.46:0/2500734280 wait complete. 2023-10-06T12:45:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 Processor -- start 2023-10-06T12:45:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 -- start start 2023-10-06T12:45:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 0x7f756815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7568161840 0x7f756815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756816eaf0 con 0x7f75680caf50 2023-10-06T12:45:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756815f190 con 0x7f7568161840 2023-10-06T12:45:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f756db45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756815e860 con 0x7f7568161470 2023-10-06T12:45:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50280/0 (socket says 172.21.15.46:50280) 2023-10-06T12:45:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 -- 172.21.15.46:0/274227174 learned_addr learned my addr 172.21.15.46:0/274227174 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f7567fff700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7568161840 0x7f756815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f755ffff700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 0x7f756815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 -- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 msgr2=0x7f756815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 0x7f756815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 -- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7568161840 msgr2=0x7f756815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7568161840 0x7f756815d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f75677fe700 1 -- 172.21.15.46:0/274227174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500094f0 con 0x7f75680caf50 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.100+0000 7f755ffff700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 0x7f756815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f75677fe700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f755800c5b0 tx=0x7f75580020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558011590 con 0x7f75680caf50 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75580116f0 con 0x7f75680caf50 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756815eb40 con 0x7f75680caf50 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558011a90 con 0x7f75680caf50 2023-10-06T12:45:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756815a650 con 0x7f75680caf50 2023-10-06T12:45:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.104+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568063f90 con 0x7f75680caf50 2023-10-06T12:45:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.108+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 91) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f7558012420 con 0x7f75680caf50 2023-10-06T12:45:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.108+0000 7f755f7fe700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 0x7f7548062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:45:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.108+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6976+0+0 (secure 0 0 0) 0x7f755807c9b0 con 0x7f75680caf50 2023-10-06T12:45:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.108+0000 7f755ffff700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 0x7f7548062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:45:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.108+0000 7f755ffff700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 0x7f7548062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75680c2e90 tx=0x7f7550016000 comp rx=0 tx=0).ready entity=mgr.9480 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:45:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.112+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f755804c920 con 0x7f75680caf50 2023-10-06T12:45:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.324+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f75680c1580 con 0x7f75680caf50 2023-10-06T12:45:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.332+0000 7f755f7fe700 1 -- 172.21.15.46:0/274227174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v91) v1 ==== 74+0+154338 (secure 0 0 0) 0x7f755804ff40 con 0x7f75680caf50 2023-10-06T12:45:17.342 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:45:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 msgr2=0x7f7548062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 0x7f7548062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75680c2e90 tx=0x7f7550016000 comp rx=0 tx=0).stop 2023-10-06T12:45:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 msgr2=0x7f756815a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:45:17.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f755800c5b0 tx=0x7f75580020e0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 shutdown_connections 2023-10-06T12:45:17.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:6832/32974,v1:172.21.15.46:6833/32974] conn(0x7f75480606c0 0x7f7548062b70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7568161840 0x7f756815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7568161470 0x7f756815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 --2- 172.21.15.46:0/274227174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75680caf50 0x7f756815a110 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 >> 172.21.15.46:0/274227174 conn(0x7f75680cbe30 msgr2=0x7f75680c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 shutdown_connections 2023-10-06T12:45:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:45:17.336+0000 7f756db45700 1 -- 172.21.15.46:0/274227174 wait complete. 2023-10-06T12:45:17.367 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"active_gid":9480,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":32974},{"type":"v1","addr":"172.21.15.46:6833","nonce":32974}]},"active_addr":"172.21.15.46:6833/32974","active_change":"2023-10-06T12:34:33.509901+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":7967,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":167,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1559249335}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2485742217}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":4045473864}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2361316107}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1003208918}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2241077176}]}]} 2023-10-06T12:45:17.371 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-10-06T12:45:18.220 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T12:45:18.220 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-10-06T12:45:18.220 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T12:45:18.221 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:45:18.288 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T12:45:18.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T12:45:18.678 INFO:tasks.ceph.mgr.y:Started 2023-10-06T12:45:18.908 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:18.900+0000 7f56bd9cc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:45:19.462 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:19.452+0000 7f56bd9cc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:45:19.542 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:19.536+0000 7f56bd9cc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:45:19.690 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:19.684+0000 7f56bd9cc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:45:19.969 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:19.960+0000 7f56bd9cc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:45:20.049 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:20.040+0000 7f56bd9cc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:45:20.277 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:20.268+0000 7f56bd9cc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:45:20.366 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:20.360+0000 7f56bd9cc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:45:20.458 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:20.448+0000 7f56bd9cc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:45:20.867 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:20.860+0000 7f56bd9cc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:45:21.033 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.024+0000 7f56bd9cc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:45:21.416 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.408+0000 7f56bd9cc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:45:21.637 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.628+0000 7f56bd9cc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:45:21.728 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.720+0000 7f56bd9cc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:45:21.805 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.796+0000 7f56bd9cc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:45:21.887 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:21.880+0000 7f56bd9cc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:45:22.107 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:22.100+0000 7f56bd9cc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:45:22.186 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:22.180+0000 7f56bd9cc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:45:22.276 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:22.268+0000 7f56bd9cc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:45:22.471 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:22.464+0000 7f56bd9cc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:45:23.299 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:22.548+0000 7f56bd9cc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:45:23.299 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.276+0000 7f56bd9cc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:45:23.564 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567c58e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.725 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567c58e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.725 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567c58e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.725 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567c58e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.725 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567c58e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.726 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567f594700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.726 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567f594700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.726 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567f594700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.874 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567f594700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:23.874 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:45:23.556+0000 7f567f594700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:45:35.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:45:35.029 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:46:00.551 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:46:00.544+0000 7f56875a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:46:00.552 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:46:00.544+0000 7f56875a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:46:00.552 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:46:00.544+0000 7f56875a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:46:00.552 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:46:00.544+0000 7f56875a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:46:00.552 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:46:00.544+0000 7f56875a4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:46:05.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:46:05.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:46:35.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:46:35.297 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:47:05.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:47:05.380 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:47:35.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:47:35.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:48:05.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:48:05.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:48:35.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:48:35.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:49:06.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:49:06.045 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:49:36.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:49:36.214 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:50:06.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:50:06.310 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:50:18.819 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T12:50:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.981+0000 7f95dafcc700 1 Processor -- start 2023-10-06T12:50:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.981+0000 7f95dafcc700 1 -- start start 2023-10-06T12:50:18.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95dafcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95d4158de0 0x7f95d40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95dafcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95d4069770 0x7f95d40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95dafcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d40b5eb0 con 0x7f95d41591b0 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95dafcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40cef30 con 0x7f95d4069770 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95dafcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40cf4f0 con 0x7f95d4158de0 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95d4069770 0x7f95d40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95d9fca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95d4158de0 0x7f95d40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95d4069770 0x7f95d40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60592/0 (socket says 172.21.15.46:60592) 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 -- 172.21.15.46:0/1070145495 learned_addr learned my addr 172.21.15.46:0/1070145495 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 -- 172.21.15.46:0/1070145495 >> v2:172.21.15.132:3300/0 conn(0x7f95d4158de0 msgr2=0x7f95d40b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 --2- 172.21.15.46:0/1070145495 >> v2:172.21.15.132:3300/0 conn(0x7f95d4158de0 0x7f95d40b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 -- 172.21.15.46:0/1070145495 >> v1:172.21.15.132:6789/0 conn(0x7f95d41591b0 legacy=0x7f95d4069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:18.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 -- 172.21.15.46:0/1070145495 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40ccac0 con 0x7f95d4069770 2023-10-06T12:50:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95da7cb700 1 --2- 172.21.15.46:0/1070145495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d40b2e10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f95d4088830 tx=0x7f95cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628c019b214c7522 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95d8fc8700 1 -- 172.21.15.46:0/1070145495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00e070 con 0x7f95d4069770 2023-10-06T12:50:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95d8fc8700 1 -- 172.21.15.46:0/1070145495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc00ba20 con 0x7f95d4069770 2023-10-06T12:50:18.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.985+0000 7f95d8fc8700 1 -- 172.21.15.46:0/1070145495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0092e0 con 0x7f95d4069770 2023-10-06T12:50:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 -- 172.21.15.46:0/1070145495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 msgr2=0x7f95d40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 --2- 172.21.15.46:0/1070145495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d40b2e10 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f95d4088830 tx=0x7f95cc009840 comp rx=0 tx=0).stop 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 -- 172.21.15.46:0/1070145495 shutdown_connections 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 --2- 172.21.15.46:0/1070145495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d40b2e10 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 --2- 172.21.15.46:0/1070145495 >> v2:172.21.15.132:3300/0 conn(0x7f95d4158de0 0x7f95d40b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 -- 172.21.15.46:0/1070145495 >> 172.21.15.46:0/1070145495 conn(0x7f95d4068390 msgr2=0x7f95d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 -- 172.21.15.46:0/1070145495 shutdown_connections 2023-10-06T12:50:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 -- 172.21.15.46:0/1070145495 wait complete. 2023-10-06T12:50:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.989+0000 7f95dafcc700 1 Processor -- start 2023-10-06T12:50:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 -- start start 2023-10-06T12:50:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d4158de0 0x7f95d415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95d41591b0 0x7f95d406c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d414a0a0 con 0x7f95d4069770 2023-10-06T12:50:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415a770 con 0x7f95d4158de0 2023-10-06T12:50:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d406bb90 con 0x7f95d41591b0 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60608/0 (socket says 172.21.15.46:60608) 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 -- 172.21.15.46:0/2123829338 learned_addr learned my addr 172.21.15.46:0/2123829338 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d97c9700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d4158de0 0x7f95d415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95da7cb700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95d41591b0 0x7f95d406c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 -- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95d41591b0 msgr2=0x7f95d406c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95d41591b0 0x7f95d406c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 -- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d4158de0 msgr2=0x7f95d415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d4158de0 0x7f95d415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 -- 172.21.15.46:0/2123829338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc009ae0 con 0x7f95d4069770 2023-10-06T12:50:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95d9fca700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ec80 tx=0x7f95d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c7f0 con 0x7f95d4069770 2023-10-06T12:50:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d000c950 con 0x7f95d4069770 2023-10-06T12:50:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d406b2f0 con 0x7f95d4069770 2023-10-06T12:50:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.993+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000ccf0 con 0x7f95d4069770 2023-10-06T12:50:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.997+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d406b8b0 con 0x7f95d4069770 2023-10-06T12:50:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.997+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d4064590 con 0x7f95d4069770 2023-10-06T12:50:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:18.997+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 96) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f95d0013680 con 0x7f95d4069770 2023-10-06T12:50:19.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.001+0000 7f95c9ffb700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 0x7f95b8062cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.001+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f95d007d510 con 0x7f95d4069770 2023-10-06T12:50:19.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.001+0000 7f95d97c9700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 0x7f95b8062cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.001+0000 7f95d97c9700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 0x7f95b8062cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95d40b3fc0 tx=0x7f95c4007ec0 comp rx=0 tx=0).ready entity=mgr.9576 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:19.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.009+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f95d004d3d0 con 0x7f95d4069770 2023-10-06T12:50:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.205+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f95d415a8b0 con 0x7f95b8060840 2023-10-06T12:50:19.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.321+0000 7f95c9ffb700 1 -- 172.21.15.46:0/2123829338 <== mgr.9576 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f95c4003670 con 0x7f95b8060840 2023-10-06T12:50:19.327 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 msgr2=0x7f95b8062cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 0x7f95b8062cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f95d40b3fc0 tx=0x7f95c4007ec0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 msgr2=0x7f95d4149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ec80 tx=0x7f95d00091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 shutdown_connections 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95d41591b0 0x7f95d406c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f95b8060840 0x7f95b8062cf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95d4158de0 0x7f95d415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 --2- 172.21.15.46:0/2123829338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95d4069770 0x7f95d4149b60 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 >> 172.21.15.46:0/2123829338 conn(0x7f95d4068390 msgr2=0x7f95d40b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:19.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 shutdown_connections 2023-10-06T12:50:19.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.325+0000 7f95dafcc700 1 -- 172.21.15.46:0/2123829338 wait complete. 2023-10-06T12:50:19.357 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-06T12:50:19.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:19.357 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T12:50:19.379 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_35_00_UTC 2023-10-06T12:50:19.379 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_36_00_UTC 2023-10-06T12:50:19.379 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_37_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_38_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_39_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_40_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_41_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_42_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_43_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_44_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_45_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_46_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_47_00_UTC 2023-10-06T12:50:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_48_00_UTC 2023-10-06T12:50:19.381 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_49_00_UTC 2023-10-06T12:50:19.381 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_50_00_UTC 2023-10-06T12:50:19.381 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-12_35_00_UTC', 'scheduled-2023-10-06-12_36_00_UTC', 'scheduled-2023-10-06-12_37_00_UTC', 'scheduled-2023-10-06-12_38_00_UTC', 'scheduled-2023-10-06-12_39_00_UTC', 'scheduled-2023-10-06-12_40_00_UTC', 'scheduled-2023-10-06-12_41_00_UTC', 'scheduled-2023-10-06-12_42_00_UTC', 'scheduled-2023-10-06-12_43_00_UTC', 'scheduled-2023-10-06-12_44_00_UTC', 'scheduled-2023-10-06-12_45_00_UTC', 'scheduled-2023-10-06-12_46_00_UTC', 'scheduled-2023-10-06-12_47_00_UTC', 'scheduled-2023-10-06-12_48_00_UTC', 'scheduled-2023-10-06-12_49_00_UTC', 'scheduled-2023-10-06-12_50_00_UTC'] 2023-10-06T12:50:19.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-06T12:50:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.589+0000 7f269e745700 1 Processor -- start 2023-10-06T12:50:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.589+0000 7f269e745700 1 -- start start 2023-10-06T12:50:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269e745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f269806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269e745700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 0x7f269806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269e745700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980c25a0 con 0x7f2698140800 2023-10-06T12:50:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269e745700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269806efa0 con 0x7f26980c6b70 2023-10-06T12:50:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269e745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c1c70 con 0x7f2698140bd0 2023-10-06T12:50:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f2697fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2698140800 0x7f26980c6460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54586/0 (socket says 172.21.15.46:54586) 2023-10-06T12:50:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f2697fff700 1 -- 172.21.15.46:0/1803853531 learned_addr learned my addr 172.21.15.46:0/1803853531 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f26977fe700 1 --2- 172.21.15.46:0/1803853531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f269806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269cce2700 1 --2- 172.21.15.46:0/1803853531 >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 0x7f269806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f2696ffd700 1 -- 172.21.15.46:0/1803853531 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828358996 0 0) 0x7f26980c25a0 con 0x7f2698140800 2023-10-06T12:50:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f2696ffd700 1 -- 172.21.15.46:0/1803853531 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2680003570 con 0x7f2698140800 2023-10-06T12:50:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f26977fe700 1 -- 172.21.15.46:0/1803853531 >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 msgr2=0x7f269806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f26977fe700 1 --2- 172.21.15.46:0/1803853531 >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 0x7f269806ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f26977fe700 1 -- 172.21.15.46:0/1803853531 >> v1:172.21.15.46:6789/0 conn(0x7f2698140800 legacy=0x7f26980c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f26977fe700 1 -- 172.21.15.46:0/1803853531 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980c1f50 con 0x7f2698140bd0 2023-10-06T12:50:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.593+0000 7f269cce2700 1 --2- 172.21.15.46:0/1803853531 >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 0x7f269806ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f26977fe700 1 --2- 172.21.15.46:0/1803853531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f269806cc10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098e0 tx=0x7f268c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e19340e3902f67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f2696ffd700 1 -- 172.21.15.46:0/1803853531 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00b640 con 0x7f2698140bd0 2023-10-06T12:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f2696ffd700 1 -- 172.21.15.46:0/1803853531 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c00b7a0 con 0x7f2698140bd0 2023-10-06T12:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f2696ffd700 1 -- 172.21.15.46:0/1803853531 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c00bb40 con 0x7f2698140bd0 2023-10-06T12:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 -- 172.21.15.46:0/1803853531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 msgr2=0x7f269806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 --2- 172.21.15.46:0/1803853531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f269806cc10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f268c0098e0 tx=0x7f268c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 -- 172.21.15.46:0/1803853531 shutdown_connections 2023-10-06T12:50:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 --2- 172.21.15.46:0/1803853531 >> v2:172.21.15.132:3300/0 conn(0x7f26980c6b70 0x7f269806ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 --2- 172.21.15.46:0/1803853531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f269806cc10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 -- 172.21.15.46:0/1803853531 >> 172.21.15.46:0/1803853531 conn(0x7f26980cbe20 msgr2=0x7f26980cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 -- 172.21.15.46:0/1803853531 shutdown_connections 2023-10-06T12:50:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.597+0000 7f269e745700 1 -- 172.21.15.46:0/1803853531 wait complete. 2023-10-06T12:50:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 Processor -- start 2023-10-06T12:50:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 -- start start 2023-10-06T12:50:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26980c6b70 0x7f269813d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f26980c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269816a500 con 0x7f2698140800 2023-10-06T12:50:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980c3ee0 con 0x7f26980c6b70 2023-10-06T12:50:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.601+0000 7f269e745700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980af2b0 con 0x7f2698140bd0 2023-10-06T12:50:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f26977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f26980c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f26977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60616/0 (socket says 172.21.15.46:60616) 2023-10-06T12:50:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f26977fe700 1 -- 172.21.15.46:0/321743582 learned_addr learned my addr 172.21.15.46:0/321743582 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f2697fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26980c6b70 0x7f269813d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 -- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26980c6b70 msgr2=0x7f269813d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26980c6b70 0x7f269813d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 -- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 msgr2=0x7f26980c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 -- 172.21.15.46:0/321743582 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c009420 con 0x7f2698140bd0 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f26977fe700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269cce2700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f26980c4fd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26980c6f70 tx=0x7f2690009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269000db80 con 0x7f2698140bd0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269000f040 con 0x7f2698140bd0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26980c3c00 con 0x7f2698140bd0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690015440 con 0x7f2698140bd0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26980aef40 con 0x7f2698140bd0 2023-10-06T12:50:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.605+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980af460 con 0x7f2698140bd0 2023-10-06T12:50:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.609+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 96) v1 ==== 80090+0+0 (secure 0 0 0) 0x7f269001e3b0 con 0x7f2698140bd0 2023-10-06T12:50:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.609+0000 7f267ffff700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 0x7f2684062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.609+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f269007d590 con 0x7f2698140bd0 2023-10-06T12:50:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.609+0000 7f2697fff700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 0x7f2684062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.613+0000 7f2697fff700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 0x7f2684062b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2688007fd0 tx=0x7f2688009040 comp rx=0 tx=0).ready entity=mgr.9576 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.617+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f26980af460 con 0x7f2698140bd0 2023-10-06T12:50:19.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.809+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26980af3f0 con 0x7f26840606c0 2023-10-06T12:50:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.825+0000 7f267ffff700 1 -- 172.21.15.46:0/321743582 <== mgr.9576 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f26980af3f0 con 0x7f26840606c0 2023-10-06T12:50:19.832 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:19.832 INFO:teuthology.orchestra.run.smithi046.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T12:34:34", "first": "2023-10-06T12:35:00", "last": "2023-10-06T12:50:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 msgr2=0x7f2684062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 0x7f2684062b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2688007fd0 tx=0x7f2688009040 comp rx=0 tx=0).stop 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 msgr2=0x7f26980c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f26980c4fd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26980c6f70 tx=0x7f2690009990 comp rx=0 tx=0).stop 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 shutdown_connections 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2698140bd0 0x7f26980c4fd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2698140800 0x7f26980c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26840606c0 0x7f2684062b70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 --2- 172.21.15.46:0/321743582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26980c6b70 0x7f269813d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 >> 172.21.15.46:0/321743582 conn(0x7f26980cbe20 msgr2=0x7f26980cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 shutdown_connections 2023-10-06T12:50:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:19.829+0000 7f269e745700 1 -- 172.21.15.46:0/321743582 wait complete. 2023-10-06T12:50:19.857 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T12:34:34", "first": "2023-10-06T12:35:00", "last": "2023-10-06T12:50:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-06T12:50:19.857 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T12:34:34', 'first': '2023-10-06T12:35:00', 'last': '2023-10-06T12:50:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-06T12:50:19.857 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-06T12:50:19.857 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-06T12:50:19.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T12:50:20.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.025+0000 7fa018e78700 1 Processor -- start 2023-10-06T12:50:20.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.025+0000 7fa018e78700 1 -- start start 2023-10-06T12:50:20.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa018e78700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa014159130 0x7fa0140d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:20.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa018e78700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0140ce2c0 con 0x7fa014069190 2023-10-06T12:50:20.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa018e78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0140af570 con 0x7fa014159500 2023-10-06T12:50:20.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa018e78700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140afb30 con 0x7fa014159130 2023-10-06T12:50:20.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa01259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa014159130 0x7fa0140d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:20.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa012d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa014069190 0x7fa0140695b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54596/0 (socket says 172.21.15.46:54596) 2023-10-06T12:50:20.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa012d9d700 1 -- 172.21.15.46:0/1670789971 learned_addr learned my addr 172.21.15.46:0/1670789971 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:20.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa011d9b700 1 --1- 172.21.15.46:0/1670789971 >> v1:172.21.15.132:6789/0 conn(0x7fa014159500 0x7fa0140aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35464/0 (socket says 172.21.15.46:35464) 2023-10-06T12:50:20.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa01259c700 1 -- 172.21.15.46:0/1670789971 >> v1:172.21.15.132:6789/0 conn(0x7fa014159500 legacy=0x7fa0140aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa01259c700 1 -- 172.21.15.46:0/1670789971 >> v1:172.21.15.46:6789/0 conn(0x7fa014069190 legacy=0x7fa0140695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa01259c700 1 -- 172.21.15.46:0/1670789971 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140af850 con 0x7fa014159130 2023-10-06T12:50:20.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.029+0000 7fa01159a700 1 -- 172.21.15.46:0/1670789971 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466167901 0 0) 0x7fa0140af570 con 0x7fa014159500 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa01259c700 1 --2- 172.21.15.46:0/1670789971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140d0d10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098e0 tx=0x7fa0040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801afccbd72b8445 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa01159a700 1 -- 172.21.15.46:0/1670789971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004009e80 con 0x7fa014159130 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa01159a700 1 -- 172.21.15.46:0/1670789971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00400b860 con 0x7fa014159130 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa01159a700 1 -- 172.21.15.46:0/1670789971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00400bbe0 con 0x7fa014159130 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 -- 172.21.15.46:0/1670789971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 msgr2=0x7fa0140d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 --2- 172.21.15.46:0/1670789971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140d0d10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098e0 tx=0x7fa0040091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 -- 172.21.15.46:0/1670789971 shutdown_connections 2023-10-06T12:50:20.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 --2- 172.21.15.46:0/1670789971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140d0d10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 -- 172.21.15.46:0/1670789971 >> 172.21.15.46:0/1670789971 conn(0x7fa014068510 msgr2=0x7fa0140d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:20.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 -- 172.21.15.46:0/1670789971 shutdown_connections 2023-10-06T12:50:20.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.033+0000 7fa018e78700 1 -- 172.21.15.46:0/1670789971 wait complete. 2023-10-06T12:50:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 Processor -- start 2023-10-06T12:50:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 -- start start 2023-10-06T12:50:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa014069190 0x7fa0140b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014159500 0x7fa0141695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:20.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140c87a0 con 0x7fa014159130 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140c8a40 con 0x7fa014159500 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa018e78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140c8d20 con 0x7fa014069190 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60644/0 (socket says 172.21.15.46:60644) 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 -- 172.21.15.46:0/3965273972 learned_addr learned my addr 172.21.15.46:0/3965273972 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:20.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa012d9d700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014159500 0x7fa0141695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa01259c700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa014069190 0x7fa0140b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 -- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa014069190 msgr2=0x7fa0140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa014069190 0x7fa0140b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 -- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014159500 msgr2=0x7fa0141695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014159500 0x7fa0141695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.037+0000 7fa011d9b700 1 -- 172.21.15.46:0/3965273972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004009420 con 0x7fa014159130 2023-10-06T12:50:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa011d9b700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c5b0 tx=0x7fa008001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008005a70 con 0x7fa014159130 2023-10-06T12:50:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008005bd0 con 0x7fa014159130 2023-10-06T12:50:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140c9000 con 0x7fa014159130 2023-10-06T12:50:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080070d0 con 0x7fa014159130 2023-10-06T12:50:20.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014169b30 con 0x7fa014159130 2023-10-06T12:50:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.041+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0140c9240 con 0x7fa014159130 2023-10-06T12:50:20.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.045+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 96) v1 ==== 80090+0+0 (secure 0 0 0) 0x7fa008007480 con 0x7fa014159130 2023-10-06T12:50:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.045+0000 7fa0027fc700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 0x7f9ff4062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.045+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7fa00807c490 con 0x7fa014159130 2023-10-06T12:50:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.045+0000 7fa01259c700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 0x7f9ff4062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:20.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.045+0000 7fa01259c700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 0x7f9ff4062b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0040093f0 tx=0x7fa004017000 comp rx=0 tx=0).ready entity=mgr.9576 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:20.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.049+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa0140c9240 con 0x7fa014159130 2023-10-06T12:50:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.249+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0140c9240 con 0x7f9ff40606c0 2023-10-06T12:50:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.373+0000 7fa0027fc700 1 -- 172.21.15.46:0/3965273972 <== mgr.9576 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fa0140c9240 con 0x7f9ff40606c0 2023-10-06T12:50:20.380 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-06T12:50:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 msgr2=0x7f9ff4062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 0x7f9ff4062b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0040093f0 tx=0x7fa004017000 comp rx=0 tx=0).stop 2023-10-06T12:50:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 msgr2=0x7fa0140c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c5b0 tx=0x7fa008001040 comp rx=0 tx=0).stop 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 shutdown_connections 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ff40606c0 0x7f9ff4062b70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa014159500 0x7fa0141695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa014159130 0x7fa0140c8260 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 --2- 172.21.15.46:0/3965273972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa014069190 0x7fa0140b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 >> 172.21.15.46:0/3965273972 conn(0x7fa014068510 msgr2=0x7fa014153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 shutdown_connections 2023-10-06T12:50:20.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:20.377+0000 7fa018e78700 1 -- 172.21.15.46:0/3965273972 wait complete. 2023-10-06T12:50:20.409 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-06T12:50:20.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.409 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T12:50:20.432 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_35_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_36_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_37_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_38_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_39_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_40_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_41_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_42_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_43_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_44_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_45_00_UTC 2023-10-06T12:50:20.433 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_46_00_UTC 2023-10-06T12:50:20.434 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_47_00_UTC 2023-10-06T12:50:20.434 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_48_00_UTC 2023-10-06T12:50:20.434 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_49_00_UTC 2023-10-06T12:50:20.434 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_50_00_UTC 2023-10-06T12:50:20.435 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_35_00_UTC 2023-10-06T12:50:20.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.435 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_35_00_UTC) 2023-10-06T12:50:20.505 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_36_00_UTC 2023-10-06T12:50:20.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.505 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_36_00_UTC) 2023-10-06T12:50:20.578 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_37_00_UTC 2023-10-06T12:50:20.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.579 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_37_00_UTC) 2023-10-06T12:50:20.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_38_00_UTC 2023-10-06T12:50:20.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.657 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_38_00_UTC) 2023-10-06T12:50:20.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_39_00_UTC 2023-10-06T12:50:20.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.731 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_39_00_UTC) 2023-10-06T12:50:20.806 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_40_00_UTC 2023-10-06T12:50:20.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.807 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_40_00_UTC) 2023-10-06T12:50:20.883 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_41_00_UTC 2023-10-06T12:50:20.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.883 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_41_00_UTC) 2023-10-06T12:50:20.958 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_42_00_UTC 2023-10-06T12:50:20.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:20.959 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_42_00_UTC) 2023-10-06T12:50:21.033 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_43_00_UTC 2023-10-06T12:50:21.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.034 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_43_00_UTC) 2023-10-06T12:50:21.109 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_44_00_UTC 2023-10-06T12:50:21.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.110 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_44_00_UTC) 2023-10-06T12:50:21.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_45_00_UTC 2023-10-06T12:50:21.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.183 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_45_00_UTC) 2023-10-06T12:50:21.257 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_46_00_UTC 2023-10-06T12:50:21.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.257 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_46_00_UTC) 2023-10-06T12:50:21.329 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_47_00_UTC 2023-10-06T12:50:21.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.329 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_47_00_UTC) 2023-10-06T12:50:21.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_48_00_UTC 2023-10-06T12:50:21.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.407 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_48_00_UTC) 2023-10-06T12:50:21.483 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_49_00_UTC 2023-10-06T12:50:21.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.483 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_49_00_UTC) 2023-10-06T12:50:21.563 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_50_00_UTC 2023-10-06T12:50:21.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.563 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-12_50_00_UTC) 2023-10-06T12:50:21.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:21.639 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-06T12:50:21.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T12:50:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.917+0000 7fccf6865700 1 Processor -- start 2023-10-06T12:50:21.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.917+0000 7fccf6865700 1 -- start start 2023-10-06T12:50:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccf6865700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fccf0068160 0x7fccf00afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccf6865700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00692e0 con 0x7fccf00b3180 2023-10-06T12:50:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccf6865700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00b0370 con 0x7fccf00aff50 2023-10-06T12:50:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccf6865700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf00686d0 con 0x7fccf0068160 2023-10-06T12:50:21.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcceffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fccf00b3180 0x7fccf013c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54610/0 (socket says 172.21.15.46:54610) 2023-10-06T12:50:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcce77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fccf0068160 0x7fccf00afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcceffff700 1 -- 172.21.15.46:0/144857161 learned_addr learned my addr 172.21.15.46:0/144857161 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccef7fe700 1 -- 172.21.15.46:0/144857161 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077423508 0 0) 0x7fccf00692e0 con 0x7fccf00b3180 2023-10-06T12:50:21.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fccef7fe700 1 -- 172.21.15.46:0/144857161 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc003570 con 0x7fccf00b3180 2023-10-06T12:50:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcce77fe700 1 -- 172.21.15.46:0/144857161 >> v1:172.21.15.132:6789/0 conn(0x7fccf00aff50 legacy=0x7fccf00b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcce77fe700 1 -- 172.21.15.46:0/144857161 >> v1:172.21.15.46:6789/0 conn(0x7fccf00b3180 legacy=0x7fccf013c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.921+0000 7fcce77fe700 1 -- 172.21.15.46:0/144857161 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00689b0 con 0x7fccf0068160 2023-10-06T12:50:21.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fcce77fe700 1 --2- 172.21.15.46:0/144857161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf0068160 0x7fccf00afa10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fccd80098e0 tx=0x7fccd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fae1ad74020d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccef7fe700 1 -- 172.21.15.46:0/144857161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800b640 con 0x7fccf0068160 2023-10-06T12:50:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccef7fe700 1 -- 172.21.15.46:0/144857161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd800b7a0 con 0x7fccf0068160 2023-10-06T12:50:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccef7fe700 1 -- 172.21.15.46:0/144857161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800bb40 con 0x7fccf0068160 2023-10-06T12:50:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 -- 172.21.15.46:0/144857161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf0068160 msgr2=0x7fccf00afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:21.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 --2- 172.21.15.46:0/144857161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf0068160 0x7fccf00afa10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fccd80098e0 tx=0x7fccd80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 -- 172.21.15.46:0/144857161 shutdown_connections 2023-10-06T12:50:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 --2- 172.21.15.46:0/144857161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf0068160 0x7fccf00afa10 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 -- 172.21.15.46:0/144857161 >> 172.21.15.46:0/144857161 conn(0x7fccf006aea0 msgr2=0x7fccf006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 -- 172.21.15.46:0/144857161 shutdown_connections 2023-10-06T12:50:21.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.925+0000 7fccf6865700 1 -- 172.21.15.46:0/144857161 wait complete. 2023-10-06T12:50:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 Processor -- start 2023-10-06T12:50:21.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 -- start start 2023-10-06T12:50:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccf0068160 0x7fccf00a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccf00b3180 0x7fccf00a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf01591a0 con 0x7fccf00aff50 2023-10-06T12:50:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00a8d00 con 0x7fccf0068160 2023-10-06T12:50:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf6865700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0171c00 con 0x7fccf00b3180 2023-10-06T12:50:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:21.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60662/0 (socket says 172.21.15.46:60662) 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 -- 172.21.15.46:0/261485091 learned_addr learned my addr 172.21.15.46:0/261485091 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcceffff700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccf0068160 0x7fccf00a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fccf4e02700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccf00b3180 0x7fccf00a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 -- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccf00b3180 msgr2=0x7fccf00a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccf00b3180 0x7fccf00a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 -- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccf0068160 msgr2=0x7fccf00a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccf0068160 0x7fccf00a6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:21.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.929+0000 7fcce77fe700 1 -- 172.21.15.46:0/261485091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd8009420 con 0x7fccf00aff50 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fcce77fe700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fccf006f800 tx=0x7fccd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800bb20 con 0x7fccf00aff50 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8000c90 con 0x7fccf00aff50 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf00ae090 con 0x7fccf00aff50 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8011670 con 0x7fccf00aff50 2023-10-06T12:50:21.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf00ae6d0 con 0x7fccf00aff50 2023-10-06T12:50:21.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.933+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf00660f0 con 0x7fccf00aff50 2023-10-06T12:50:21.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.937+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 96) v1 ==== 80090+0+0 (secure 0 0 0) 0x7fccd80119e0 con 0x7fccf00aff50 2023-10-06T12:50:21.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.937+0000 7fccecff9700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 0x7fccd0062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.937+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7fccd807e7c0 con 0x7fccf00aff50 2023-10-06T12:50:21.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.937+0000 7fcceffff700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 0x7fccd0062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:21.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.937+0000 7fcceffff700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 0x7fccd0062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007fd0 tx=0x7fcce0009040 comp rx=0 tx=0).ready entity=mgr.9576 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:21.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:21.945+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fccd804e600 con 0x7fccf00aff50 2023-10-06T12:50:22.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.153+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fccf0068c90 con 0x7fccf00aff50 2023-10-06T12:50:22.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.853+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fccd804e9d0 con 0x7fccf00aff50 2023-10-06T12:50:22.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccecff9700 1 -- 172.21.15.46:0/261485091 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v97) v1 ==== 93+0+0 (secure 0 0 0) 0x7fccd80253f0 con 0x7fccf00aff50 2023-10-06T12:50:22.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 msgr2=0x7fccd0062b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 0x7fccd0062b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcce0007fd0 tx=0x7fcce0009040 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 msgr2=0x7fccf00ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fccf006f800 tx=0x7fccd8017000 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 shutdown_connections 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccf00b3180 0x7fccf00a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf00aff50 0x7fccf00ada00 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fccd00606c0 0x7fccd0062b70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 --2- 172.21.15.46:0/261485091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccf0068160 0x7fccf00a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 >> 172.21.15.46:0/261485091 conn(0x7fccf006aea0 msgr2=0x7fccf013e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:22.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 shutdown_connections 2023-10-06T12:50:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:22.857+0000 7fccf6865700 1 -- 172.21.15.46:0/261485091 wait complete. 2023-10-06T12:50:22.887 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:50:22.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:50:22.925 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:50:22.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:50:22.960 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T12:50:22.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:22.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T12:50:23.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:50:23.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:23.040 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T12:50:23.041 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:23.038+0000 7f67263ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:50:23.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:23.062 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:23.057+0000 7f3a89763000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:50:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.217+0000 7f3d7cc65700 1 Processor -- start 2023-10-06T12:50:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.217+0000 7f3d7cc65700 1 -- start start 2023-10-06T12:50:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d7cc65700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 0x7f3d780c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d7cc65700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d781506c0 0x7f3d780c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d7cc65700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d780c67e0 con 0x7f3d781502f0 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d7cc65700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7806e810 con 0x7f3d781506c0 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d7cc65700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7806eaf0 con 0x7f3d7815a1a0 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d781506c0 0x7f3d780c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3d781506c0 0x7f3d780c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47540/0 (socket says 172.21.15.46:47540) 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 -- 172.21.15.46:0/2723681532 learned_addr learned my addr 172.21.15.46:0/2723681532 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d6dd9b700 1 --2- 172.21.15.46:0/2723681532 >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 0x7f3d780c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 -- 172.21.15.46:0/2723681532 >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 msgr2=0x7f3d780c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 --2- 172.21.15.46:0/2723681532 >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 0x7f3d780c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 -- 172.21.15.46:0/2723681532 >> v1:172.21.15.46:6789/0 conn(0x7f3d781502f0 legacy=0x7f3d78159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 -- 172.21.15.46:0/2723681532 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7806edd0 con 0x7f3d781506c0 2023-10-06T12:50:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d6dd9b700 1 --2- 172.21.15.46:0/2723681532 >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 0x7f3d780c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d76d9d700 1 --2- 172.21.15.46:0/2723681532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c6c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3d7806ba90 tx=0x7f3d70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e093dac093c90d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.221+0000 7f3d75d9b700 1 -- 172.21.15.46:0/2723681532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000e070 con 0x7f3d781506c0 2023-10-06T12:50:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d75d9b700 1 -- 172.21.15.46:0/2723681532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000ba50 con 0x7f3d781506c0 2023-10-06T12:50:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d75d9b700 1 -- 172.21.15.46:0/2723681532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700092e0 con 0x7f3d781506c0 2023-10-06T12:50:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- 172.21.15.46:0/2723681532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 msgr2=0x7f3d780c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/2723681532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c6c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f3d7806ba90 tx=0x7f3d70009840 comp rx=0 tx=0).stop 2023-10-06T12:50:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- 172.21.15.46:0/2723681532 shutdown_connections 2023-10-06T12:50:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/2723681532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c6c70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/2723681532 >> v2:172.21.15.132:3300/0 conn(0x7f3d7815a1a0 0x7f3d780c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- 172.21.15.46:0/2723681532 >> 172.21.15.46:0/2723681532 conn(0x7f3d780cbe30 msgr2=0x7f3d780cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- 172.21.15.46:0/2723681532 shutdown_connections 2023-10-06T12:50:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- 172.21.15.46:0/2723681532 wait complete. 2023-10-06T12:50:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 Processor -- start 2023-10-06T12:50:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.225+0000 7f3d7cc65700 1 -- start start 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d781502f0 0x7f3d78168ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7815a1a0 0x7f3d780c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78151340 con 0x7f3d781506c0 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78144150 con 0x7f3d7815a1a0 2023-10-06T12:50:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b0ba0 con 0x7f3d781502f0 2023-10-06T12:50:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47542/0 (socket says 172.21.15.46:47542) 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 -- 172.21.15.46:0/546071695 learned_addr learned my addr 172.21.15.46:0/546071695 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d76d9d700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7815a1a0 0x7f3d780c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7659c700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d781502f0 0x7f3d78168ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d781502f0 msgr2=0x7f3d78168ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d781502f0 0x7f3d78168ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7815a1a0 msgr2=0x7f3d780c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7815a1a0 0x7f3d780c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 -- 172.21.15.46:0/546071695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009ae0 con 0x7f3d781506c0 2023-10-06T12:50:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6dd9b700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000d890 tx=0x7f3d60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600125f0 con 0x7f3d781506c0 2023-10-06T12:50:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d78143820 con 0x7f3d781506c0 2023-10-06T12:50:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60012750 con 0x7f3d781506c0 2023-10-06T12:50:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60012b10 con 0x7f3d781506c0 2023-10-06T12:50:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.229+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d78143de0 con 0x7f3d781506c0 2023-10-06T12:50:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.233+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d78063f90 con 0x7f3d781506c0 2023-10-06T12:50:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.233+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f3d60014430 con 0x7f3d781506c0 2023-10-06T12:50:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.237+0000 7f3d6effd700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 0x7f3d58062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.237+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f3d6007d2f0 con 0x7f3d781506c0 2023-10-06T12:50:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.237+0000 7f3d7659c700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 msgr2=0x7f3d58062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.237+0000 7f3d7659c700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 0x7f3d58062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.241+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3d60050bc0 con 0x7f3d781506c0 2023-10-06T12:50:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.441+0000 7f3d7659c700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 msgr2=0x7f3d58062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.441+0000 7f3d7659c700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 0x7f3d58062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:23.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.453+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d78144290 con 0x7f3d781506c0 2023-10-06T12:50:23.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.453+0000 7f3d6effd700 1 -- 172.21.15.46:0/546071695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f3d60050750 con 0x7f3d781506c0 2023-10-06T12:50:23.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:23.460 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:50:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 msgr2=0x7f3d58062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 0x7f3d58062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 msgr2=0x7f3d780c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000d890 tx=0x7f3d60008040 comp rx=0 tx=0).stop 2023-10-06T12:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 shutdown_connections 2023-10-06T12:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3d58060670 0x7f3d58062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d7815a1a0 0x7f3d780c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d781506c0 0x7f3d780c1370 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 --2- 172.21.15.46:0/546071695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d781502f0 0x7f3d78168ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.457+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 >> 172.21.15.46:0/546071695 conn(0x7f3d780cbe30 msgr2=0x7f3d781462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.461+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 shutdown_connections 2023-10-06T12:50:23.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.461+0000 7f3d7cc65700 1 -- 172.21.15.46:0/546071695 wait complete. 2023-10-06T12:50:23.468 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:50:23.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:23.584 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:23.582+0000 7f67263ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:50:23.631 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:23.625+0000 7f3a89763000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:50:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.641+0000 7f5946878700 1 Processor -- start 2023-10-06T12:50:23.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.641+0000 7f5946878700 1 -- start start 2023-10-06T12:50:23.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5946878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f59400ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5946878700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594006d460 con 0x7f594015ab60 2023-10-06T12:50:23.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5946878700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400c7000 con 0x7f594006e8a0 2023-10-06T12:50:23.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5946878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594006ed40 con 0x7f59400c1ec0 2023-10-06T12:50:23.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f594015ab60 0x7f594015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36192/0 (socket says 172.21.15.46:36192) 2023-10-06T12:50:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593ffff700 1 -- 172.21.15.46:0/1888080558 learned_addr learned my addr 172.21.15.46:0/1888080558 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:23.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5944e15700 1 --2- 172.21.15.46:0/1888080558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f59400ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503431895 0 0) 0x7f594006d460 con 0x7f594015ab60 2023-10-06T12:50:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5928003570 con 0x7f594015ab60 2023-10-06T12:50:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952866215 0 0) 0x7f59400c7000 con 0x7f594006e8a0 2023-10-06T12:50:23.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f594006d460 con 0x7f594006e8a0 2023-10-06T12:50:23.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.645+0000 7f5944e15700 1 -- 172.21.15.46:0/1888080558 >> v1:172.21.15.132:6789/0 conn(0x7f594006e8a0 legacy=0x7f594006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5944e15700 1 -- 172.21.15.46:0/1888080558 >> v1:172.21.15.46:6789/0 conn(0x7f594015ab60 legacy=0x7f594015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5944e15700 1 -- 172.21.15.46:0/1888080558 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400c72c0 con 0x7f59400c1ec0 2023-10-06T12:50:23.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5944e15700 1 --2- 172.21.15.46:0/1888080558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f59400ad6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e90 tx=0x7f5938009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353e784e5b9fac1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:23.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593800e070 con 0x7f59400c1ec0 2023-10-06T12:50:23.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593800bb80 con 0x7f59400c1ec0 2023-10-06T12:50:23.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f593effd700 1 -- 172.21.15.46:0/1888080558 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938012430 con 0x7f59400c1ec0 2023-10-06T12:50:23.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 -- 172.21.15.46:0/1888080558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 msgr2=0x7f59400ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 --2- 172.21.15.46:0/1888080558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f59400ad6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e90 tx=0x7f5938009780 comp rx=0 tx=0).stop 2023-10-06T12:50:23.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 -- 172.21.15.46:0/1888080558 shutdown_connections 2023-10-06T12:50:23.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 --2- 172.21.15.46:0/1888080558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f59400ad6c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 -- 172.21.15.46:0/1888080558 >> 172.21.15.46:0/1888080558 conn(0x7f59400cbe30 msgr2=0x7f59400cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:23.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 -- 172.21.15.46:0/1888080558 shutdown_connections 2023-10-06T12:50:23.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.649+0000 7f5946878700 1 -- 172.21.15.46:0/1888080558 wait complete. 2023-10-06T12:50:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 Processor -- start 2023-10-06T12:50:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 -- start start 2023-10-06T12:50:23.666 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:23.658+0000 7f67263ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:50:23.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f5940148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f594015ab60 0x7f59400c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400c47b0 con 0x7f594006e8a0 2023-10-06T12:50:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400cba30 con 0x7f594015ab60 2023-10-06T12:50:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5946878700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594006a100 con 0x7f59400c1ec0 2023-10-06T12:50:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47546/0 (socket says 172.21.15.46:47546) 2023-10-06T12:50:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 -- 172.21.15.46:0/3917302648 learned_addr learned my addr 172.21.15.46:0/3917302648 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f5944e15700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f594015ab60 0x7f59400c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593f7fe700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f5940148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 -- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 msgr2=0x7f5940148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f5940148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 -- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f594015ab60 msgr2=0x7f59400c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.653+0000 7f593ffff700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f594015ab60 0x7f59400c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f593ffff700 1 -- 172.21.15.46:0/3917302648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938009a20 con 0x7f594006e8a0 2023-10-06T12:50:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f593f7fe700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f5940148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f593ffff700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f593000c790 tx=0x7f5930007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930007300 con 0x7f594006e8a0 2023-10-06T12:50:23.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930007460 con 0x7f594006e8a0 2023-10-06T12:50:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300057f0 con 0x7f594006e8a0 2023-10-06T12:50:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59400cb750 con 0x7f594006e8a0 2023-10-06T12:50:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400ca7a0 con 0x7f594006e8a0 2023-10-06T12:50:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400ca330 con 0x7f594006e8a0 2023-10-06T12:50:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.657+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f5930005ba0 con 0x7f594006e8a0 2023-10-06T12:50:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.661+0000 7f5927fff700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 0x7f592c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.661+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f593007ca40 con 0x7f594006e8a0 2023-10-06T12:50:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.661+0000 7f593f7fe700 1 -- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 msgr2=0x7f592c062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.661+0000 7f593f7fe700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 0x7f592c062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.665+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f593004c900 con 0x7f594006e8a0 2023-10-06T12:50:23.709 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:23.701+0000 7f3a89763000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:50:23.808 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:23.806+0000 7f67263ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:23.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.849+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5940063f90 con 0x7f594006e8a0 2023-10-06T12:50:23.855 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:23.849+0000 7f3a89763000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.849+0000 7f5927fff700 1 -- 172.21.15.46:0/3917302648 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18626 (secure 0 0 0) 0x7f593004ff20 con 0x7f594006e8a0 2023-10-06T12:50:23.857 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:23.857 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":170,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:45:23.285264+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.2b","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1003208918":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2241077176":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2485742217":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2361316107":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4002742240":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6833/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6832/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1813456300":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1926732711":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1559249335":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4045473864":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/61884588":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/3084269464":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/2070008917":"2023-10-07T12:34:33.509742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:23.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 msgr2=0x7f592c062b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 0x7f592c062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 msgr2=0x7f5940069bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f593000c790 tx=0x7f5930007c80 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 shutdown_connections 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f592c0606a0 0x7f592c062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f594015ab60 0x7f59400c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59400c1ec0 0x7f5940148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 --2- 172.21.15.46:0/3917302648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f594006e8a0 0x7f5940069bc0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:23.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 >> 172.21.15.46:0/3917302648 conn(0x7f59400cbe30 msgr2=0x7f59400cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 shutdown_connections 2023-10-06T12:50:23.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:23.853+0000 7f5946878700 1 -- 172.21.15.46:0/3917302648 wait complete. 2023-10-06T12:50:23.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T12:34:17.832704+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T12:34:18.856666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T12:50:23.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:50:24.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 Processor -- start 2023-10-06T12:50:24.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 -- start start 2023-10-06T12:50:24.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 0x7fe0f00c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f00c1630 con 0x7fe0f00c7d80 2023-10-06T12:50:24.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f00c0d00 con 0x7fe0f00ca430 2023-10-06T12:50:24.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f722e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00c0fe0 con 0x7fe0f00c8150 2023-10-06T12:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f4fca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe0f00c7d80 0x7fe0f00c6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36202/0 (socket says 172.21.15.46:36202) 2023-10-06T12:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f4fca700 1 -- 172.21.15.46:0/3102911919 learned_addr learned my addr 172.21.15.46:0/3102911919 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:24.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f57cb700 1 --2- 172.21.15.46:0/3102911919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 0x7fe0f00c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578442166 0 0) 0x7fe0f00c1630 con 0x7fe0f00c7d80 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e0003570 con 0x7fe0f00c7d80 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0f57cb700 1 -- 172.21.15.46:0/3102911919 >> v1:172.21.15.132:6789/0 conn(0x7fe0f00ca430 legacy=0x7fe0f00a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.041+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958505076 0 0) 0x7fe0f00c0d00 con 0x7fe0f00ca430 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f57cb700 1 -- 172.21.15.46:0/3102911919 >> v1:172.21.15.46:6789/0 conn(0x7fe0f00c7d80 legacy=0x7fe0f00c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f57cb700 1 -- 172.21.15.46:0/3102911919 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00c12c0 con 0x7fe0f00c8150 2023-10-06T12:50:24.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f57cb700 1 --2- 172.21.15.46:0/3102911919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 0x7fe0f00c1ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009910 tx=0x7fe0e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88bbdcfda89189d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800b680 con 0x7fe0f00c8150 2023-10-06T12:50:24.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e800b7e0 con 0x7fe0f00c8150 2023-10-06T12:50:24.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0ef7fe700 1 -- 172.21.15.46:0/3102911919 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800bba0 con 0x7fe0f00c8150 2023-10-06T12:50:24.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 -- 172.21.15.46:0/3102911919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 msgr2=0x7fe0f00c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 --2- 172.21.15.46:0/3102911919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 0x7fe0f00c1ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009910 tx=0x7fe0e80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 -- 172.21.15.46:0/3102911919 shutdown_connections 2023-10-06T12:50:24.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 --2- 172.21.15.46:0/3102911919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c8150 0x7fe0f00c1ac0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 -- 172.21.15.46:0/3102911919 >> 172.21.15.46:0/3102911919 conn(0x7fe0f00cbe30 msgr2=0x7fe0f00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:24.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 -- 172.21.15.46:0/3102911919 shutdown_connections 2023-10-06T12:50:24.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.045+0000 7fe0f722e700 1 -- 172.21.15.46:0/3102911919 wait complete. 2023-10-06T12:50:24.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 Processor -- start 2023-10-06T12:50:24.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 -- start start 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 0x7fe0f0151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0f00ca430 0x7fe0f00ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015de60 con 0x7fe0f00c8150 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015d9d0 con 0x7fe0f00ca430 2023-10-06T12:50:24.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f722e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f0151fb0 con 0x7fe0f00c7d80 2023-10-06T12:50:24.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47562/0 (socket says 172.21.15.46:47562) 2023-10-06T12:50:24.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0effff700 1 -- 172.21.15.46:0/1441048326 learned_addr learned my addr 172.21.15.46:0/1441048326 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f4fca700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 0x7fe0f0151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.049+0000 7fe0f57cb700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0f00ca430 0x7fe0f00ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 -- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0f00ca430 msgr2=0x7fe0f00ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0f00ca430 0x7fe0f00ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 -- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 msgr2=0x7fe0f015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 -- 172.21.15.46:0/1441048326 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8009420 con 0x7fe0f00c7d80 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0effff700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f4fca700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 0x7fe0f0151a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400cc40 tx=0x7fe0e4001a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400a850 con 0x7fe0f00c7d80 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e400a9d0 con 0x7fe0f00c7d80 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f015cdc0 con 0x7fe0f00c7d80 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400ad90 con 0x7fe0f00c7d80 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f015d380 con 0x7fe0f00c7d80 2023-10-06T12:50:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.053+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f00660f0 con 0x7fe0f00c7d80 2023-10-06T12:50:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.057+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fe0e4007300 con 0x7fe0f00c7d80 2023-10-06T12:50:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.057+0000 7fe0ecff9700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 0x7fe0d4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.057+0000 7fe0effff700 1 -- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 msgr2=0x7fe0d4062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.057+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7fe0e407d2b0 con 0x7fe0f00c7d80 2023-10-06T12:50:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.057+0000 7fe0effff700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 0x7fe0d4062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.065+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe0e404d170 con 0x7fe0f00c7d80 2023-10-06T12:50:24.090 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.086+0000 7f67263ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:50:24.141 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:24.133+0000 7f3a89763000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:50:24.161 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.158+0000 7f67263ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:24.241 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:24.213+0000 7f3a89763000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:24.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.233+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0f00c2350 con 0x7fe0f00c7d80 2023-10-06T12:50:24.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.233+0000 7fe0ecff9700 1 -- 172.21.15.46:0/1441048326 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0e4050790 con 0x7fe0f00c7d80 2023-10-06T12:50:24.242 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:50:24.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 msgr2=0x7fe0d4062b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 0x7fe0d4062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 msgr2=0x7fe0f0151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 0x7fe0f0151a70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400cc40 tx=0x7fe0e4001a20 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 shutdown_connections 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe0d40606a0 0x7fe0d4062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0f00ca430 0x7fe0f00ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0f00c8150 0x7fe0f015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 --2- 172.21.15.46:0/1441048326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0f00c7d80 0x7fe0f0151a70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 >> 172.21.15.46:0/1441048326 conn(0x7fe0f00cbe30 msgr2=0x7fe0f0154890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 shutdown_connections 2023-10-06T12:50:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.237+0000 7fe0f722e700 1 -- 172.21.15.46:0/1441048326 wait complete. 2023-10-06T12:50:24.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:50:24.382 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.378+0000 7f67263ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:50:24.569 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.462+0000 7f67263ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:50:24.569 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.546+0000 7f67263ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.413+0000 7f46ebfff700 1 Processor -- start 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.413+0000 7f46ebfff700 1 -- start start 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ebfff700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec0afd20 con 0x7f46ec0b30f0 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ebfff700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec0b0000 con 0x7f46ec0adbe0 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ebfff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec159740 con 0x7f46ec0b14d0 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ea7fc700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f46ec0b30f0 0x7f46ec0b0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36208/0 (socket says 172.21.15.46:36208) 2023-10-06T12:50:24.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46ea7fc700 1 -- 172.21.15.46:0/4285794137 learned_addr learned my addr 172.21.15.46:0/4285794137 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46eb7fe700 1 --2- 172.21.15.46:0/4285794137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265297803 0 0) 0x7f46ec0afd20 con 0x7f46ec0b30f0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46d4003570 con 0x7f46ec0b30f0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572341251 0 0) 0x7f46ec0b0000 con 0x7f46ec0adbe0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ec0afd20 con 0x7f46ec0adbe0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46eb7fe700 1 -- 172.21.15.46:0/4285794137 >> v1:172.21.15.132:6789/0 conn(0x7f46ec0adbe0 legacy=0x7f46ec0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46eb7fe700 1 -- 172.21.15.46:0/4285794137 >> v1:172.21.15.46:6789/0 conn(0x7f46ec0b30f0 legacy=0x7f46ec0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46eb7fe700 1 -- 172.21.15.46:0/4285794137 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0b02e0 con 0x7f46ec0b14d0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46eb7fe700 1 --2- 172.21.15.46:0/4285794137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0adfb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009e90 tx=0x7f46e4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abaa0699a08a7a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400e070 con 0x7f46ec0b14d0 2023-10-06T12:50:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400bb80 con 0x7f46ec0b14d0 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.417+0000 7f46e9ffb700 1 -- 172.21.15.46:0/4285794137 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4012430 con 0x7f46ec0b14d0 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- 172.21.15.46:0/4285794137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 msgr2=0x7f46ec0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 --2- 172.21.15.46:0/4285794137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0adfb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009e90 tx=0x7f46e4009780 comp rx=0 tx=0).stop 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- 172.21.15.46:0/4285794137 shutdown_connections 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 --2- 172.21.15.46:0/4285794137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0adfb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- 172.21.15.46:0/4285794137 >> 172.21.15.46:0/4285794137 conn(0x7f46ec06ef60 msgr2=0x7f46ec0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- 172.21.15.46:0/4285794137 shutdown_connections 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- 172.21.15.46:0/4285794137 wait complete. 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 Processor -- start 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.421+0000 7f46ebfff700 1 -- start start 2023-10-06T12:50:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ec0b30f0 0x7f46ec0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec154980 con 0x7f46ec0adbe0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec153d70 con 0x7f46ec0b30f0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec154050 con 0x7f46ec0b14d0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47568/0 (socket says 172.21.15.46:47568) 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 -- 172.21.15.46:0/2629860025 learned_addr learned my addr 172.21.15.46:0/2629860025 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ea7fc700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eb7fe700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ec0b30f0 0x7f46ec0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 -- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 msgr2=0x7f46ec0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 -- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ec0b30f0 msgr2=0x7f46ec0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ec0b30f0 0x7f46ec0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 -- 172.21.15.46:0/2629860025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009a20 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46eaffd700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c580 tx=0x7f46dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc010510 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc010670 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec154330 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc010a30 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.425+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec14fc80 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.429+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec064590 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.429+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f46dc01b400 con 0x7f46ec0adbe0 2023-10-06T12:50:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.429+0000 7f46d2ffd700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 0x7f46d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.433+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7f46dc07cb20 con 0x7f46ec0adbe0 2023-10-06T12:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.433+0000 7f46ea7fc700 1 -- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 msgr2=0x7f46d8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.433+0000 7f46ea7fc700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 0x7f46d8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.437+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f46dc04c960 con 0x7f46ec0adbe0 2023-10-06T12:50:24.578 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:24.449+0000 7f3a89763000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:50:24.578 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:24.537+0000 7f3a89763000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:50:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.609+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f46ec0675d0 con 0x7f46ec0adbe0 2023-10-06T12:50:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.613+0000 7f46d2ffd700 1 -- 172.21.15.46:0/2629860025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v170) v1 ==== 92+0+10 (secure 0 0 0) 0x7f46dc04ff80 con 0x7f46ec0adbe0 2023-10-06T12:50:24.618 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:50:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 msgr2=0x7f46d8062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 0x7f46d8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 msgr2=0x7f46ec14f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00c580 tx=0x7f46dc007e10 comp rx=0 tx=0).stop 2023-10-06T12:50:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 shutdown_connections 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f46d8060670 0x7f46d8062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ec0b30f0 0x7f46ec0b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46ec0b14d0 0x7f46ec0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 --2- 172.21.15.46:0/2629860025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46ec0adbe0 0x7f46ec14f740 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 >> 172.21.15.46:0/2629860025 conn(0x7f46ec06ef60 msgr2=0x7f46ec0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 shutdown_connections 2023-10-06T12:50:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.617+0000 7f46ebfff700 1 -- 172.21.15.46:0/2629860025 wait complete. 2023-10-06T12:50:24.636 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:24.629+0000 7f3a89763000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:50:24.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T12:50:24.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.805+0000 7fba3b6f0700 1 Processor -- start 2023-10-06T12:50:24.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.805+0000 7fba3b6f0700 1 -- start start 2023-10-06T12:50:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3b6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba340c9f80 0x7fba340ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3b6f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba340c6850 0x7fba34142890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3b6f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba340c7b60 con 0x7fba3406a340 2023-10-06T12:50:24.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3b6f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba34069c40 con 0x7fba340c6850 2023-10-06T12:50:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3b6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba340c6cb0 con 0x7fba340c9f80 2023-10-06T12:50:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba340c6850 0x7fba34142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba340c6850 0x7fba34142890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47574/0 (socket says 172.21.15.46:47574) 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 -- 172.21.15.46:0/1454630075 learned_addr learned my addr 172.21.15.46:0/1454630075 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba3948c700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba340c9f80 0x7fba340ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 -- 172.21.15.46:0/1454630075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba340c9f80 msgr2=0x7fba340ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba340c9f80 0x7fba340ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.809+0000 7fba39c8d700 1 -- 172.21.15.46:0/1454630075 >> v1:172.21.15.132:6789/0 conn(0x7fba3406a340 legacy=0x7fba340c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba39c8d700 1 -- 172.21.15.46:0/1454630075 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba340c6f50 con 0x7fba340c6850 2023-10-06T12:50:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba39c8d700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c6850 0x7fba34142890 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009f20 tx=0x7fba2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b56cb627149244 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba2bfff700 1 -- 172.21.15.46:0/1454630075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00e070 con 0x7fba340c6850 2023-10-06T12:50:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba2bfff700 1 -- 172.21.15.46:0/1454630075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c00b9d0 con 0x7fba340c6850 2023-10-06T12:50:24.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba2bfff700 1 -- 172.21.15.46:0/1454630075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0092e0 con 0x7fba340c6850 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 -- 172.21.15.46:0/1454630075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c6850 msgr2=0x7fba34142890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c6850 0x7fba34142890 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009f20 tx=0x7fba2c009840 comp rx=0 tx=0).stop 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 -- 172.21.15.46:0/1454630075 shutdown_connections 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c6850 0x7fba34142890 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/1454630075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba340c9f80 0x7fba340ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 -- 172.21.15.46:0/1454630075 >> 172.21.15.46:0/1454630075 conn(0x7fba340ccc90 msgr2=0x7fba34069010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 -- 172.21.15.46:0/1454630075 shutdown_connections 2023-10-06T12:50:24.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 -- 172.21.15.46:0/1454630075 wait complete. 2023-10-06T12:50:24.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.813+0000 7fba3b6f0700 1 Processor -- start 2023-10-06T12:50:24.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 -- start start 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba3406a340 0x7fba3416e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba340c6850 0x7fba34171200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba34171a50 con 0x7fba340c9f80 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba34171d30 con 0x7fba340c6850 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba34172010 con 0x7fba3406a340 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47584/0 (socket says 172.21.15.46:47584) 2023-10-06T12:50:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 -- 172.21.15.46:0/3111429036 learned_addr learned my addr 172.21.15.46:0/3111429036 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3948c700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba3406a340 0x7fba3416e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba38c8b700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba340c6850 0x7fba34171200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba3406a340 msgr2=0x7fba3416e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba3406a340 0x7fba3416e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba340c6850 msgr2=0x7fba34171200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba340c6850 0x7fba34171200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 -- 172.21.15.46:0/3111429036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c009ae0 con 0x7fba340c9f80 2023-10-06T12:50:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba39c8d700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00b610 tx=0x7fba2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c008ad0 con 0x7fba340c9f80 2023-10-06T12:50:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c008c30 con 0x7fba340c9f80 2023-10-06T12:50:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c026460 con 0x7fba340c9f80 2023-10-06T12:50:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.817+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3415c820 con 0x7fba340c9f80 2023-10-06T12:50:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.821+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba34159f80 con 0x7fba340c9f80 2023-10-06T12:50:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.821+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3406c1c0 con 0x7fba340c9f80 2023-10-06T12:50:24.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.821+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fba2c003040 con 0x7fba340c9f80 2023-10-06T12:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.825+0000 7fba297fa700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 0x7fba18062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.825+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 7234+0+0 (secure 0 0 0) 0x7fba2c07ee70 con 0x7fba340c9f80 2023-10-06T12:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.825+0000 7fba3948c700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 msgr2=0x7fba18062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.825+0000 7fba3948c700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 0x7fba18062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:24.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:24.833+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fba2c04ec20 con 0x7fba340c9f80 2023-10-06T12:50:24.991 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:24.986+0000 7f67263ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:50:25.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.029+0000 7fba3948c700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 msgr2=0x7fba18062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:25.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.029+0000 7fba3948c700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 0x7fba18062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.077+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba3415c9b0 con 0x7fba340c9f80 2023-10-06T12:50:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.077+0000 7fba297fa700 1 -- 172.21.15.46:0/3111429036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba2c052240 con 0x7fba340c9f80 2023-10-06T12:50:25.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T12:50:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 msgr2=0x7fba18062bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 0x7fba18062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 msgr2=0x7fba34159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00b610 tx=0x7fba2c018000 comp rx=0 tx=0).stop 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 shutdown_connections 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba340c9f80 0x7fba34159a40 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fba18060740 0x7fba18062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba340c6850 0x7fba34171200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 --2- 172.21.15.46:0/3111429036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba3406a340 0x7fba3416e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 >> 172.21.15.46:0/3111429036 conn(0x7fba340ccc90 msgr2=0x7fba3406e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 shutdown_connections 2023-10-06T12:50:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.081+0000 7fba3b6f0700 1 -- 172.21.15.46:0/3111429036 wait complete. 2023-10-06T12:50:25.097 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:25.089+0000 7f3a89763000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:50:25.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T12:50:25.157 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:25.154+0000 7f67263ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:50:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.269+0000 7f94f50f0700 1 Processor -- start 2023-10-06T12:50:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.269+0000 7f94f50f0700 1 -- start start 2023-10-06T12:50:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94f50f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94f50f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94f006c8f0 0x7f94f00c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94f50f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f006d190 con 0x7f94f016a5c0 2023-10-06T12:50:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94f50f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f006ceb0 con 0x7f94f006c8f0 2023-10-06T12:50:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94f50f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0169ae0 con 0x7f94f0067760 2023-10-06T12:50:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94f006c8f0 0x7f94f00c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94eed9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f94f016a5c0 0x7f94f0067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:44494/0 (socket says 172.21.15.46:44494) 2023-10-06T12:50:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94eed9d700 1 -- 172.21.15.46:0/3360686832 learned_addr learned my addr 172.21.15.46:0/3360686832 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 -- 172.21.15.46:0/3360686832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 msgr2=0x7f94f006e5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T12:50:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 --2- 172.21.15.46:0/3360686832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f006e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 -- 172.21.15.46:0/3360686832 >> v1:172.21.15.132:6789/0 conn(0x7f94f016a5c0 legacy=0x7f94f0067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 -- 172.21.15.46:0/3360686832 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0169dc0 con 0x7f94f006c8f0 2023-10-06T12:50:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.273+0000 7f94ef59e700 1 --2- 172.21.15.46:0/3360686832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f006c8f0 0x7f94f00c7ad0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009b40 tx=0x7f94e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a044fa82a023e2cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.281+0000 7f94edd9b700 1 -- 172.21.15.46:0/3360686832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000b780 con 0x7f94f006c8f0 2023-10-06T12:50:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94edd9b700 1 -- 172.21.15.46:0/3360686832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e000b8e0 con 0x7f94f006c8f0 2023-10-06T12:50:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 -- 172.21.15.46:0/3360686832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f006c8f0 msgr2=0x7f94f00c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94edd9b700 1 -- 172.21.15.46:0/3360686832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e000b780 con 0x7f94f006c8f0 2023-10-06T12:50:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 --2- 172.21.15.46:0/3360686832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f006c8f0 0x7f94f00c7ad0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009b40 tx=0x7f94e00091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 -- 172.21.15.46:0/3360686832 shutdown_connections 2023-10-06T12:50:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 --2- 172.21.15.46:0/3360686832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f006c8f0 0x7f94f00c7ad0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 --2- 172.21.15.46:0/3360686832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 -- 172.21.15.46:0/3360686832 >> 172.21.15.46:0/3360686832 conn(0x7f94f00cbc10 msgr2=0x7f94f00cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:25.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 -- 172.21.15.46:0/3360686832 shutdown_connections 2023-10-06T12:50:25.297 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:25.289+0000 7f3a89763000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:50:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.285+0000 7f94f50f0700 1 -- 172.21.15.46:0/3360686832 wait complete. 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.293+0000 7f94f50f0700 1 Processor -- start 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.293+0000 7f94f50f0700 1 -- start start 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.293+0000 7f94f50f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f006c8f0 0x7f94f00c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 0x7f94f00c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c1b90 con 0x7f94f016a5c0 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0167420 con 0x7f94f006c8f0 2023-10-06T12:50:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0167700 con 0x7f94f0067760 2023-10-06T12:50:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:54294/0 (socket says 172.21.15.46:54294) 2023-10-06T12:50:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 -- 172.21.15.46:0/1957412013 learned_addr learned my addr 172.21.15.46:0/1957412013 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94ee59c700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f006c8f0 0x7f94f00c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94ef59e700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 0x7f94f00c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f006c8f0 msgr2=0x7f94f00c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f006c8f0 0x7f94f00c06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 msgr2=0x7f94f00c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 0x7f94f00c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 -- 172.21.15.46:0/1957412013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0009420 con 0x7f94f0067760 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94ef59e700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 0x7f94f00c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94eed9d700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94e800cc70 tx=0x7f94e8001a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:25.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800ce80 con 0x7f94f0067760 2023-10-06T12:50:25.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e800ab70 con 0x7f94f0067760 2023-10-06T12:50:25.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80070f0 con 0x7f94f0067760 2023-10-06T12:50:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.297+0000 7f94f50f0700 1 -- 172.21.15.46:0/1957412013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f01679e0 con 0x7f94f0067760 2023-10-06T12:50:25.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.301+0000 7f94f50f0700 1 -- 172.21.15.46:0/1957412013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f00c66b0 con 0x7f94f0067760 2023-10-06T12:50:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.301+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c80052b0 con 0x7f94f0067760 2023-10-06T12:50:25.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.305+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f94e8007520 con 0x7f94f0067760 2023-10-06T12:50:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.309+0000 7f94d6ffd700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 0x7f94dc062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.309+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7f94e807d370 con 0x7f94f0067760 2023-10-06T12:50:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.309+0000 7f94ee59c700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 msgr2=0x7f94dc062ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.309+0000 7f94ee59c700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 0x7f94dc062ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.313+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f94e804d280 con 0x7f94f0067760 2023-10-06T12:50:25.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.513+0000 7f94ee59c700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 msgr2=0x7f94dc062ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:25.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.513+0000 7f94ee59c700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 0x7f94dc062ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:25.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.517+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94c8005ae0 con 0x7f94f0067760 2023-10-06T12:50:25.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.517+0000 7f94d6ffd700 1 -- 172.21.15.46:0/1957412013 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94e80508a0 con 0x7f94f0067760 2023-10-06T12:50:25.526 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 msgr2=0x7f94dc062ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 0x7f94dc062ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 msgr2=0x7f94f00c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94e800cc70 tx=0x7f94e8001a30 comp rx=0 tx=0).stop 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 shutdown_connections 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94f016a5c0 0x7f94f00c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f94dc0607f0 0x7f94dc062ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f006c8f0 0x7f94f00c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 --2- 172.21.15.46:0/1957412013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94f0067760 0x7f94f00c6170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 >> 172.21.15.46:0/1957412013 conn(0x7f94f00cbc10 msgr2=0x7f94f00ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 shutdown_connections 2023-10-06T12:50:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.521+0000 7f94d4ff9700 1 -- 172.21.15.46:0/1957412013 wait complete. 2023-10-06T12:50:25.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:25.590 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:25.586+0000 7f67263ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:50:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 Processor -- start 2023-10-06T12:50:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 -- start start 2023-10-06T12:50:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd80cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdd815b700 0x7efdd8156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd815b270 con 0x7efdd8156e80 2023-10-06T12:50:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd80ad3d0 con 0x7efdd815b700 2023-10-06T12:50:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efddf163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd806e3a0 con 0x7efdd80ccea0 2023-10-06T12:50:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.709+0000 7efdde962700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efdd8156e80 0x7efdd80c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36218/0 (socket says 172.21.15.46:36218) 2023-10-06T12:50:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efdde962700 1 -- 172.21.15.46:0/2698143761 learned_addr learned my addr 172.21.15.46:0/2698143761 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 --2- 172.21.15.46:0/2698143761 >> v2:172.21.15.132:3300/0 conn(0x7efdd815b700 0x7efdd8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efdde161700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd80cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd15f700 1 -- 172.21.15.46:0/2698143761 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705309189 0 0) 0x7efdd815b270 con 0x7efdd8156e80 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd15f700 1 -- 172.21.15.46:0/2698143761 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdc4003570 con 0x7efdd8156e80 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 -- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 msgr2=0x7efdd80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd80cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 -- 172.21.15.46:0/2698143761 >> v1:172.21.15.46:6789/0 conn(0x7efdd8156e80 legacy=0x7efdd80c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 -- 172.21.15.46:0/2698143761 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd815ac40 con 0x7efdd815b700 2023-10-06T12:50:25.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd960700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd815b700 0x7efdd8156940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efdc80098e0 tx=0x7efdc80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119acdccce0fb7f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:25.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd15f700 1 -- 172.21.15.46:0/2698143761 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8011630 con 0x7efdd815b700 2023-10-06T12:50:25.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd15f700 1 -- 172.21.15.46:0/2698143761 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc8011790 con 0x7efdd815b700 2023-10-06T12:50:25.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddd15f700 1 -- 172.21.15.46:0/2698143761 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8011b30 con 0x7efdd815b700 2023-10-06T12:50:25.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddf163700 1 -- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd815b700 msgr2=0x7efdd8156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.713+0000 7efddf163700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd815b700 0x7efdd8156940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efdc80098e0 tx=0x7efdc80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- 172.21.15.46:0/2698143761 shutdown_connections 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd815b700 0x7efdd8156940 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 --2- 172.21.15.46:0/2698143761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd80cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- 172.21.15.46:0/2698143761 >> 172.21.15.46:0/2698143761 conn(0x7efdd8068160 msgr2=0x7efdd8068560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- 172.21.15.46:0/2698143761 shutdown_connections 2023-10-06T12:50:25.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- 172.21.15.46:0/2698143761 wait complete. 2023-10-06T12:50:25.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 Processor -- start 2023-10-06T12:50:25.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- start start 2023-10-06T12:50:25.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd8156e80 0x7efdd80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80b1d60 con 0x7efdd815b700 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80b07c0 con 0x7efdd8156e80 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.717+0000 7efddf163700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80b1710 con 0x7efdd80ccea0 2023-10-06T12:50:25.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47620/0 (socket says 172.21.15.46:47620) 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 -- 172.21.15.46:0/3370876045 learned_addr learned my addr 172.21.15.46:0/3370876045 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde161700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efddd960700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd8156e80 0x7efdd80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 msgr2=0x7efdd81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd81564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd8156e80 msgr2=0x7efdd80b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd8156e80 0x7efdd80b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 -- 172.21.15.46:0/3370876045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc8009420 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdde962700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7efdd0001460 tx=0x7efdd0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000a890 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd000a9f0 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd80b19f0 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000ad70 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd80b2730 con 0x7efdd815b700 2023-10-06T12:50:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.721+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd80b1570 con 0x7efdd815b700 2023-10-06T12:50:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.725+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7efdd0007300 con 0x7efdd815b700 2023-10-06T12:50:25.732 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:25.725+0000 7f3a89763000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:50:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.725+0000 7efdce7fc700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 0x7efdbc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.725+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7efdd007d3d0 con 0x7efdd815b700 2023-10-06T12:50:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.725+0000 7efdde161700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 msgr2=0x7efdbc062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.725+0000 7efdde161700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 0x7efdbc062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:25.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.729+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efdd80b1570 con 0x7efdd815b700 2023-10-06T12:50:25.819 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:25.814+0000 7f67263ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:25.910 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:25.906+0000 7f67263ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:50:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.929+0000 7efdde161700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 msgr2=0x7efdbc062b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.929+0000 7efdde161700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 0x7efdbc062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.937+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdd8156ce0 con 0x7efdd815b700 2023-10-06T12:50:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.937+0000 7efdce7fc700 1 -- 172.21.15.46:0/3370876045 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7efdd8156ce0 con 0x7efdd815b700 2023-10-06T12:50:25.945 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:25.946 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:50:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 msgr2=0x7efdbc062b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 0x7efdbc062b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 msgr2=0x7efdd81572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7efdd0001460 tx=0x7efdd0001ae0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 shutdown_connections 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd815b700 0x7efdd81572c0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efdbc0606a0 0x7efdbc062b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd8156e80 0x7efdd80b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 --2- 172.21.15.46:0/3370876045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80ccea0 0x7efdd81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 >> 172.21.15.46:0/3370876045 conn(0x7efdd8068160 msgr2=0x7efdd80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 shutdown_connections 2023-10-06T12:50:25.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:25.941+0000 7efddf163700 1 -- 172.21.15.46:0/3370876045 wait complete. 2023-10-06T12:50:25.951 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:50:25.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:25.975 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:25.965+0000 7f3a89763000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:25.989 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:25.986+0000 7f67263ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:50:26.069 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.066+0000 7f67263ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:50:26.071 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.065+0000 7f3a89763000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:50:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.121+0000 7f8d764ae700 1 Processor -- start 2023-10-06T12:50:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.121+0000 7f8d764ae700 1 -- start start 2023-10-06T12:50:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d764ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d700d0aa0 0x7f8d70159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d764ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 0x7f8d700b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d764ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700b4f80 con 0x7f8d70159050 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d764ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700b5870 con 0x7f8d700d0aa0 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d764ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b40c0 con 0x7f8d700b5410 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6ffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8d70159050 0x7f8d700b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:44510/0 (socket says 172.21.15.46:44510) 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6ffff700 1 -- 172.21.15.46:0/1367332263 learned_addr learned my addr 172.21.15.46:0/1367332263 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 --2- 172.21.15.46:0/1367332263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 0x7f8d700b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6f7fe700 1 --2- 172.21.15.46:0/1367332263 >> v2:172.21.15.132:3300/0 conn(0x7f8d700d0aa0 0x7f8d70159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6effd700 1 -- 172.21.15.46:0/1367332263 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710396963 0 0) 0x7f8d700b4f80 con 0x7f8d70159050 2023-10-06T12:50:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6effd700 1 -- 172.21.15.46:0/1367332263 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d58003570 con 0x7f8d70159050 2023-10-06T12:50:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 -- 172.21.15.46:0/1367332263 >> v2:172.21.15.132:3300/0 conn(0x7f8d700d0aa0 msgr2=0x7f8d70159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 --2- 172.21.15.46:0/1367332263 >> v2:172.21.15.132:3300/0 conn(0x7f8d700d0aa0 0x7f8d70159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 -- 172.21.15.46:0/1367332263 >> v1:172.21.15.132:6789/0 conn(0x7f8d70159050 legacy=0x7f8d700b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 -- 172.21.15.46:0/1367332263 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700b4370 con 0x7f8d700b5410 2023-10-06T12:50:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d74a4b700 1 --2- 172.21.15.46:0/1367332263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 0x7f8d700b1fc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e60 tx=0x7f8d68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a817dfb44e627fa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6effd700 1 -- 172.21.15.46:0/1367332263 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800e070 con 0x7f8d700b5410 2023-10-06T12:50:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6effd700 1 -- 172.21.15.46:0/1367332263 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6800bab0 con 0x7f8d700b5410 2023-10-06T12:50:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.125+0000 7f8d6effd700 1 -- 172.21.15.46:0/1367332263 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800be70 con 0x7f8d700b5410 2023-10-06T12:50:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- 172.21.15.46:0/1367332263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 msgr2=0x7f8d700b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 --2- 172.21.15.46:0/1367332263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 0x7f8d700b1fc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e60 tx=0x7f8d68009780 comp rx=0 tx=0).stop 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- 172.21.15.46:0/1367332263 shutdown_connections 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 --2- 172.21.15.46:0/1367332263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d700b5410 0x7f8d700b1fc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 --2- 172.21.15.46:0/1367332263 >> v2:172.21.15.132:3300/0 conn(0x7f8d700d0aa0 0x7f8d70159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- 172.21.15.46:0/1367332263 >> 172.21.15.46:0/1367332263 conn(0x7f8d7006e780 msgr2=0x7f8d7006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- 172.21.15.46:0/1367332263 shutdown_connections 2023-10-06T12:50:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- 172.21.15.46:0/1367332263 wait complete. 2023-10-06T12:50:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 Processor -- start 2023-10-06T12:50:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.129+0000 7f8d764ae700 1 -- start start 2023-10-06T12:50:26.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d700b5410 0x7f8d70146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d70159050 0x7f8d700c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70159b60 con 0x7f8d700d0aa0 2023-10-06T12:50:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70159e40 con 0x7f8d700b5410 2023-10-06T12:50:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015a120 con 0x7f8d70159050 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47634/0 (socket says 172.21.15.46:47634) 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 -- 172.21.15.46:0/400368656 learned_addr learned my addr 172.21.15.46:0/400368656 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6ffff700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d700b5410 0x7f8d70146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d74a4b700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d70159050 0x7f8d700c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 -- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d70159050 msgr2=0x7f8d700c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d70159050 0x7f8d700c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 -- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d700b5410 msgr2=0x7f8d70146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d700b5410 0x7f8d70146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 -- 172.21.15.46:0/400368656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009a20 con 0x7f8d700d0aa0 2023-10-06T12:50:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d6f7fe700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400ec80 tx=0x7f8d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64009690 con 0x7f8d700d0aa0 2023-10-06T12:50:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6400c850 con 0x7f8d700d0aa0 2023-10-06T12:50:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7015a400 con 0x7f8d700d0aa0 2023-10-06T12:50:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400cc10 con 0x7f8d700d0aa0 2023-10-06T12:50:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.133+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d70146a60 con 0x7f8d700d0aa0 2023-10-06T12:50:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7015a880 con 0x7f8d700d0aa0 2023-10-06T12:50:26.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f8d64013470 con 0x7f8d700d0aa0 2023-10-06T12:50:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d57fff700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 0x7f8d5c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d6ffff700 1 -- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 msgr2=0x7f8d5c062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7f8d6407d170 con 0x7f8d700d0aa0 2023-10-06T12:50:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.137+0000 7f8d6ffff700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 0x7f8d5c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.145+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8d64050a10 con 0x7f8d700d0aa0 2023-10-06T12:50:26.150 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.145+0000 7f3a89763000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:50:26.238 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.233+0000 7f3a89763000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:50:26.301 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.298+0000 7f67263ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.317+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d700b4650 con 0x7f8d700d0aa0 2023-10-06T12:50:26.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d57fff700 1 -- 172.21.15.46:0/400368656 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18741 (secure 0 0 0) 0x7f8d640505a0 con 0x7f8d700d0aa0 2023-10-06T12:50:26.326 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:26.326 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":171,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:24.865281+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.2b","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1003208918":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2241077176":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2485742217":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2361316107":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4002742240":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6833/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6832/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1813456300":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1926732711":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1559249335":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4045473864":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/61884588":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/3084269464":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/2070008917":"2023-10-07T12:34:33.509742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 msgr2=0x7f8d5c062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 0x7f8d5c062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 msgr2=0x7f8d7015c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400ec80 tx=0x7f8d640091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 shutdown_connections 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d70159050 0x7f8d700c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d700d0aa0 0x7f8d7015c1a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8d5c060670 0x7f8d5c062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 --2- 172.21.15.46:0/400368656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d700b5410 0x7f8d70146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 >> 172.21.15.46:0/400368656 conn(0x7f8d7006e780 msgr2=0x7f8d70155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:26.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 shutdown_connections 2023-10-06T12:50:26.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.321+0000 7f8d764ae700 1 -- 172.21.15.46:0/400368656 wait complete. 2023-10-06T12:50:26.349 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:50:26.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:26.377 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.374+0000 7f67263ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:50:26.468 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.466+0000 7f67263ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:50:26.505 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.473+0000 7f3a89763000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 Processor -- start 2023-10-06T12:50:26.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 -- start start 2023-10-06T12:50:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb310159290 0x7fb3100b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 0x7fb3100ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3100ae340 con 0x7fb3100d07b0 2023-10-06T12:50:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100b09a0 con 0x7fb310159290 2023-10-06T12:50:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.501+0000 7fb3156e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb310159660 con 0x7fb3100b1300 2023-10-06T12:50:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb307fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb3100d07b0 0x7fb3100b0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36220/0 (socket says 172.21.15.46:36220) 2023-10-06T12:50:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb307fff700 1 -- 172.21.15.46:0/3279038908 learned_addr learned my addr 172.21.15.46:0/3279038908 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 --2- 172.21.15.46:0/3279038908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 0x7fb3100ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30ffff700 1 --2- 172.21.15.46:0/3279038908 >> v2:172.21.15.132:3300/0 conn(0x7fb310159290 0x7fb3100b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30f7fe700 1 -- 172.21.15.46:0/3279038908 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520206688 0 0) 0x7fb3100ae340 con 0x7fb3100d07b0 2023-10-06T12:50:26.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30f7fe700 1 -- 172.21.15.46:0/3279038908 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc003570 con 0x7fb3100d07b0 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 -- 172.21.15.46:0/3279038908 >> v2:172.21.15.132:3300/0 conn(0x7fb310159290 msgr2=0x7fb3100b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 --2- 172.21.15.46:0/3279038908 >> v2:172.21.15.132:3300/0 conn(0x7fb310159290 0x7fb3100b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 -- 172.21.15.46:0/3279038908 >> v1:172.21.15.46:6789/0 conn(0x7fb3100d07b0 legacy=0x7fb3100b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 -- 172.21.15.46:0/3279038908 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100b3450 con 0x7fb3100b1300 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb314ee2700 1 --2- 172.21.15.46:0/3279038908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 0x7fb3100ae7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e60 tx=0x7fb308009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea793d99d1bb94d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30f7fe700 1 -- 172.21.15.46:0/3279038908 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30800e070 con 0x7fb3100b1300 2023-10-06T12:50:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30f7fe700 1 -- 172.21.15.46:0/3279038908 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30800bb40 con 0x7fb3100b1300 2023-10-06T12:50:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb30f7fe700 1 -- 172.21.15.46:0/3279038908 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308012430 con 0x7fb3100b1300 2023-10-06T12:50:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 -- 172.21.15.46:0/3279038908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 msgr2=0x7fb3100ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 --2- 172.21.15.46:0/3279038908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 0x7fb3100ae7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e60 tx=0x7fb308009780 comp rx=0 tx=0).stop 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 -- 172.21.15.46:0/3279038908 shutdown_connections 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 --2- 172.21.15.46:0/3279038908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100b1300 0x7fb3100ae7d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 --2- 172.21.15.46:0/3279038908 >> v2:172.21.15.132:3300/0 conn(0x7fb310159290 0x7fb3100b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.505+0000 7fb3156e3700 1 -- 172.21.15.46:0/3279038908 >> 172.21.15.46:0/3279038908 conn(0x7fb31006ef60 msgr2=0x7fb3100d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.509+0000 7fb3156e3700 1 -- 172.21.15.46:0/3279038908 shutdown_connections 2023-10-06T12:50:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.509+0000 7fb3156e3700 1 -- 172.21.15.46:0/3279038908 wait complete. 2023-10-06T12:50:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.509+0000 7fb3156e3700 1 Processor -- start 2023-10-06T12:50:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.509+0000 7fb3156e3700 1 -- start start 2023-10-06T12:50:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3100b1300 0x7fb31013fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100d07b0 0x7fb31015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31009cb20 con 0x7fb310159290 2023-10-06T12:50:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310140370 con 0x7fb3100b1300 2023-10-06T12:50:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31015bda0 con 0x7fb3100d07b0 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47646/0 (socket says 172.21.15.46:47646) 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 -- 172.21.15.46:0/1983061106 learned_addr learned my addr 172.21.15.46:0/1983061106 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb30ffff700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3100b1300 0x7fb31013fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb307fff700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100d07b0 0x7fb31015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100d07b0 msgr2=0x7fb31015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100d07b0 0x7fb31015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3100b1300 msgr2=0x7fb31013fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3100b1300 0x7fb31013fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 -- 172.21.15.46:0/1983061106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308009a20 con 0x7fb310159290 2023-10-06T12:50:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb314ee2700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b610 tx=0x7fb308019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308012e90 con 0x7fb310159290 2023-10-06T12:50:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308027470 con 0x7fb310159290 2023-10-06T12:50:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31009bf10 con 0x7fb310159290 2023-10-06T12:50:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308027810 con 0x7fb310159290 2023-10-06T12:50:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.513+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31009c4d0 con 0x7fb310159290 2023-10-06T12:50:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.517+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31009ccd0 con 0x7fb310159290 2023-10-06T12:50:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.517+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7fb308024070 con 0x7fb310159290 2023-10-06T12:50:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.521+0000 7fb30cff9700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 0x7fb2f0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.521+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7fb30807fa50 con 0x7fb310159290 2023-10-06T12:50:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.521+0000 7fb30ffff700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 msgr2=0x7fb2f0062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.521+0000 7fb30ffff700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 0x7fb2f0062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:26.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.525+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb31009ccd0 con 0x7fb310159290 2023-10-06T12:50:26.569 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.561+0000 7f3a89763000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:50:26.659 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.653+0000 7f3a89763000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:50:26.673 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.670+0000 7f67263ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:50:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.725+0000 7fb30ffff700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 msgr2=0x7fb2f0062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:26.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.725+0000 7fb30ffff700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 0x7fb2f0062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:26.774 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:26.754+0000 7f67263ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:50:26.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.737+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb31009ccd0 con 0x7fb310159290 2023-10-06T12:50:26.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.737+0000 7fb30cff9700 1 -- 172.21.15.46:0/1983061106 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fb31009ccd0 con 0x7fb310159290 2023-10-06T12:50:26.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:26.775 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:50:26.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 msgr2=0x7fb2f0062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 0x7fb2f0062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 msgr2=0x7fb31009b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb30800b610 tx=0x7fb308019000 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 shutdown_connections 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb310159290 0x7fb31009b490 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3100d07b0 0x7fb31015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb2f0060670 0x7fb2f0062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 --2- 172.21.15.46:0/1983061106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3100b1300 0x7fb31013fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 >> 172.21.15.46:0/1983061106 conn(0x7fb31006ef60 msgr2=0x7fb310154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 shutdown_connections 2023-10-06T12:50:26.777 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.741+0000 7fb3156e3700 1 -- 172.21.15.46:0/1983061106 wait complete. 2023-10-06T12:50:26.778 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:50:26.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:26.879 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.873+0000 7f3a89763000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:50:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.933+0000 7f4e8c6cd700 1 Processor -- start 2023-10-06T12:50:26.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.933+0000 7f4e8c6cd700 1 -- start start 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8c6cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 0x7f4e840c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8c6cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e8415a1a0 0x7f4e840c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8c6cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e840c67e0 con 0x7f4e841506c0 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8c6cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8406e810 con 0x7f4e8415a1a0 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8c6cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8406eaf0 con 0x7f4e841502f0 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e8415a1a0 0x7f4e840c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e8415a1a0 0x7f4e840c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47656/0 (socket says 172.21.15.46:47656) 2023-10-06T12:50:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/4171879694 learned_addr learned my addr 172.21.15.46:0/4171879694 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8a469700 1 --2- 172.21.15.46:0/4171879694 >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 0x7f4e840c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/4171879694 >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 msgr2=0x7f4e840c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 --2- 172.21.15.46:0/4171879694 >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 0x7f4e840c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/4171879694 >> v1:172.21.15.132:6789/0 conn(0x7f4e841506c0 legacy=0x7f4e84159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/4171879694 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8406edd0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8a469700 1 --2- 172.21.15.46:0/4171879694 >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 0x7f4e840c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e8ac6a700 1 --2- 172.21.15.46:0/4171879694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e840c6c70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406ca30 tx=0x7f4e7c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f0c4993a34cae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e89467700 1 -- 172.21.15.46:0/4171879694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c010070 con 0x7f4e8415a1a0 2023-10-06T12:50:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e89467700 1 -- 172.21.15.46:0/4171879694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c00de90 con 0x7f4e8415a1a0 2023-10-06T12:50:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.937+0000 7f4e89467700 1 -- 172.21.15.46:0/4171879694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00b320 con 0x7f4e8415a1a0 2023-10-06T12:50:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/4171879694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 msgr2=0x7f4e840c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/4171879694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e840c6c70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4e8406ca30 tx=0x7f4e7c00ba50 comp rx=0 tx=0).stop 2023-10-06T12:50:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/4171879694 shutdown_connections 2023-10-06T12:50:26.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/4171879694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e840c6c70 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/4171879694 >> v2:172.21.15.132:3300/0 conn(0x7f4e841502f0 0x7f4e840c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/4171879694 >> 172.21.15.46:0/4171879694 conn(0x7f4e840cbe30 msgr2=0x7f4e840cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/4171879694 shutdown_connections 2023-10-06T12:50:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/4171879694 wait complete. 2023-10-06T12:50:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 Processor -- start 2023-10-06T12:50:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.941+0000 7f4e8c6cd700 1 -- start start 2023-10-06T12:50:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e841502f0 0x7f4e8414ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 0x7f4e840af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840a9840 con 0x7f4e8415a1a0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840afc30 con 0x7f4e841506c0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840aec70 con 0x7f4e841502f0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47660/0 (socket says 172.21.15.46:47660) 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/593828119 learned_addr learned my addr 172.21.15.46:0/593828119 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8a469700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e841502f0 0x7f4e8414ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e89c68700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 0x7f4e840af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e841502f0 msgr2=0x7f4e8414ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e841502f0 0x7f4e8414ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 msgr2=0x7f4e840af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 0x7f4e840af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 -- 172.21.15.46:0/593828119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c00bcf0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e89c68700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 0x7f4e840af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8ac6a700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c014000 tx=0x7f4e7c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00ac10 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c00ad90 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c01d680 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e840aef50 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.945+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8414d5e0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.949+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e840a99f0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.953+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f4e7c003040 con 0x7f4e8415a1a0 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.953+0000 7f4e7a7fc700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 0x7f4e68062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.953+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7f4e7c07ead0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.953+0000 7f4e8a469700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 msgr2=0x7f4e68062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.953+0000 7f4e8a469700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 0x7f4e68062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:26.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:26.957+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e7c04e7d0 con 0x7f4e8415a1a0 2023-10-06T12:50:26.992 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:26.965+0000 7f3a89763000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:50:27.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.157+0000 7f4e8a469700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 msgr2=0x7f4e68062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:27.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.157+0000 7f4e8a469700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 0x7f4e68062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:27.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.165+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e840a99f0 con 0x7f4e8415a1a0 2023-10-06T12:50:27.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.169+0000 7f4e7a7fc700 1 -- 172.21.15.46:0/593828119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f4e840a99f0 con 0x7f4e8415a1a0 2023-10-06T12:50:27.175 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:27.175 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:34:19.846543+0000","modified":"2023-10-06T12:34:23.457018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9294":{"gid":9294,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6836/3241270718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3241270718},{"type":"v1","addr":"172.21.15.46:6836","nonce":3241270718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T12:50:27.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 msgr2=0x7f4e68062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:27.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 0x7f4e68062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 msgr2=0x7f4e8414d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c014000 tx=0x7f4e7c01a000 comp rx=0 tx=0).stop 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 shutdown_connections 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e8415a1a0 0x7f4e8414d0a0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e68060670 0x7f4e68062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e841506c0 0x7f4e840af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 --2- 172.21.15.46:0/593828119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e841502f0 0x7f4e8414ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 >> 172.21.15.46:0/593828119 conn(0x7f4e840cbe30 msgr2=0x7f4e8414e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 shutdown_connections 2023-10-06T12:50:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.173+0000 7f4e8c6cd700 1 -- 172.21.15.46:0/593828119 wait complete. 2023-10-06T12:50:27.289 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2023-10-06T12:50:27.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:27.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.449+0000 7f81cae78700 1 Processor -- start 2023-10-06T12:50:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.449+0000 7f81cae78700 1 -- start start 2023-10-06T12:50:27.632 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:27.490+0000 7f67263ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:50:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81cae78700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c40af3f0 0x7f81c406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81cae78700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c415bb70 con 0x7f81c4156ea0 2023-10-06T12:50:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81cae78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c40cab20 con 0x7f81c40b2750 2023-10-06T12:50:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81cae78700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c415aff0 con 0x7f81c40af3f0 2023-10-06T12:50:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c40af3f0 0x7f81c406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c40af3f0 0x7f81c406a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47666/0 (socket says 172.21.15.46:47666) 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 -- 172.21.15.46:0/3644993225 learned_addr learned my addr 172.21.15.46:0/3644993225 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c8c14700 1 --1- 172.21.15.46:0/3644993225 >> v1:172.21.15.46:6789/0 conn(0x7f81c4156ea0 0x7f81c40b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36232/0 (socket says 172.21.15.46:36232) 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 -- 172.21.15.46:0/3644993225 >> v1:172.21.15.132:6789/0 conn(0x7f81c40b2750 legacy=0x7f81c406a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 -- 172.21.15.46:0/3644993225 >> v1:172.21.15.46:6789/0 conn(0x7f81c4156ea0 legacy=0x7f81c40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 -- 172.21.15.46:0/3644993225 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c415b2d0 con 0x7f81c40af3f0 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c3fff700 1 --2- 172.21.15.46:0/3644993225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40af3f0 0x7f81c406a0f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f81b00099b0 tx=0x7f81b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e723412fdc1633 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c37fe700 1 -- 172.21.15.46:0/3644993225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000b860 con 0x7f81c40af3f0 2023-10-06T12:50:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c37fe700 1 -- 172.21.15.46:0/3644993225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000b9c0 con 0x7f81c40af3f0 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.453+0000 7f81c37fe700 1 -- 172.21.15.46:0/3644993225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000bd80 con 0x7f81c40af3f0 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- 172.21.15.46:0/3644993225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40af3f0 msgr2=0x7f81c406a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 --2- 172.21.15.46:0/3644993225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40af3f0 0x7f81c406a0f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f81b00099b0 tx=0x7f81b0009250 comp rx=0 tx=0).stop 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- 172.21.15.46:0/3644993225 shutdown_connections 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 --2- 172.21.15.46:0/3644993225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40af3f0 0x7f81c406a0f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- 172.21.15.46:0/3644993225 >> 172.21.15.46:0/3644993225 conn(0x7f81c4068bf0 msgr2=0x7f81c40ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- 172.21.15.46:0/3644993225 shutdown_connections 2023-10-06T12:50:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- 172.21.15.46:0/3644993225 wait complete. 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 Processor -- start 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.457+0000 7f81cae78700 1 -- start start 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81c40af3f0 0x7f81c4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 0x7f81c4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40ab720 con 0x7f81c40b2750 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40aab10 con 0x7f81c4156ea0 2023-10-06T12:50:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40aadf0 con 0x7f81c40af3f0 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47670/0 (socket says 172.21.15.46:47670) 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 -- 172.21.15.46:0/1901813642 learned_addr learned my addr 172.21.15.46:0/1901813642 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c8c14700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81c40af3f0 0x7f81c4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c9415700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 0x7f81c4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 -- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81c40af3f0 msgr2=0x7f81c4149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81c40af3f0 0x7f81c4149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 -- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 msgr2=0x7f81c4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 0x7f81c4151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 -- 172.21.15.46:0/1901813642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00094f0 con 0x7f81c40b2750 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c9415700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 0x7f81c4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c3fff700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f81c406d6e0 tx=0x7f81b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b001f070 con 0x7f81c40b2750 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b0002f30 con 0x7f81c40b2750 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0025420 con 0x7f81c40b2750 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c40ab0d0 con 0x7f81c40b2750 2023-10-06T12:50:27.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c414a100 con 0x7f81c40b2750 2023-10-06T12:50:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.461+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c40ab890 con 0x7f81c40b2750 2023-10-06T12:50:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.465+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 97) v1 ==== 80073+0+0 (secure 0 0 0) 0x7f81b0023070 con 0x7f81c40b2750 2023-10-06T12:50:27.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.469+0000 7f81c0ff9700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 0x7f81a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.469+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 7290+0+0 (secure 0 0 0) 0x7f81b007e150 con 0x7f81c40b2750 2023-10-06T12:50:27.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.469+0000 7f81c8c14700 1 -- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 msgr2=0x7f81a8062b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.469+0000 7f81c8c14700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 0x7f81a8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:27.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.473+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f81b004df60 con 0x7f81c40b2750 2023-10-06T12:50:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.641+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81c4153d30 con 0x7f81c40b2750 2023-10-06T12:50:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.645+0000 7f81c0ff9700 1 -- 172.21.15.46:0/1901813642 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18741 (secure 0 0 0) 0x7f81b0051580 con 0x7f81c40b2750 2023-10-06T12:50:27.739 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:27.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":171,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:24.865281+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:34:17.832704+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T12:34:18.856666+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.2b","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1003208918":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2241077176":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2485742217":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/2361316107":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4002742240":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6833/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:6832/32974":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1813456300":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1926732711":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/1559249335":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/4045473864":"2023-10-07T12:45:23.285199+0000","172.21.15.46:0/61884588":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/3084269464":"2023-10-07T12:34:33.509742+0000","172.21.15.46:0/2070008917":"2023-10-07T12:34:33.509742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:27.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 msgr2=0x7f81a8062b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 0x7f81a8062b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 msgr2=0x7f81c40aa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f81c406d6e0 tx=0x7f81b0017000 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 shutdown_connections 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f81a8060670 0x7f81a8062b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c4156ea0 0x7f81c4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c40b2750 0x7f81c40aa200 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 --2- 172.21.15.46:0/1901813642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81c40af3f0 0x7f81c4149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 >> 172.21.15.46:0/1901813642 conn(0x7f81c4068bf0 msgr2=0x7f81c406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:27.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 shutdown_connections 2023-10-06T12:50:27.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.649+0000 7f81cae78700 1 -- 172.21.15.46:0/1901813642 wait complete. 2023-10-06T12:50:27.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T12:50:27.769 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.749+0000 7f3a89763000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:50:27.862 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a50347700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a50347700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a50347700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a50347700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a50347700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.890 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a53b4e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.890 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a53b4e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.890 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a53b4e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.890 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a53b4e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.890 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:27.857+0000 7f3a53b4e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:27.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.925+0000 7f544c4ae700 1 Processor -- start 2023-10-06T12:50:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.925+0000 7f544c4ae700 1 -- start start 2023-10-06T12:50:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f54440c1d80 0x7f54440c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f544406cc30 0x7f544406d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440c18f0 con 0x7f544406cc30 2023-10-06T12:50:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440c0ce0 con 0x7f54440c1d80 2023-10-06T12:50:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544c4ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440c0fc0 con 0x7f5444161470 2023-10-06T12:50:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544aa4b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f544406cc30 0x7f544406d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:54482/0 (socket says 172.21.15.46:54482) 2023-10-06T12:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 -- 172.21.15.46:0/2983229914 learned_addr learned my addr 172.21.15.46:0/2983229914 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f5449a49700 1 --2- 172.21.15.46:0/2983229914 >> v2:172.21.15.132:3300/0 conn(0x7f54440c1d80 0x7f54440c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 -- 172.21.15.46:0/2983229914 >> v2:172.21.15.132:3300/0 conn(0x7f54440c1d80 msgr2=0x7f54440c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 --2- 172.21.15.46:0/2983229914 >> v2:172.21.15.132:3300/0 conn(0x7f54440c1d80 0x7f54440c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 -- 172.21.15.46:0/2983229914 >> v2:172.21.15.46:3300/0 conn(0x7f544406cc30 msgr2=0x7f544406d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 --2- 172.21.15.46:0/2983229914 >> v2:172.21.15.46:3300/0 conn(0x7f544406cc30 0x7f544406d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 -- 172.21.15.46:0/2983229914 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440c12a0 con 0x7f5444161470 2023-10-06T12:50:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f544a24a700 1 --2- 172.21.15.46:0/2983229914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f544406c640 tx=0x7f5440009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ea1334bd7f1b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f5449248700 1 -- 172.21.15.46:0/2983229914 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440005c10 con 0x7f5444161470 2023-10-06T12:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f5449248700 1 -- 172.21.15.46:0/2983229914 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440005d70 con 0x7f5444161470 2023-10-06T12:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.929+0000 7f5449248700 1 -- 172.21.15.46:0/2983229914 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440006260 con 0x7f5444161470 2023-10-06T12:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- 172.21.15.46:0/2983229914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 msgr2=0x7f5444161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 --2- 172.21.15.46:0/2983229914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f544406c640 tx=0x7f5440009220 comp rx=0 tx=0).stop 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- 172.21.15.46:0/2983229914 shutdown_connections 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 --2- 172.21.15.46:0/2983229914 >> v2:172.21.15.132:3300/0 conn(0x7f54440c1d80 0x7f54440c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 --2- 172.21.15.46:0/2983229914 >> v2:172.21.15.46:3300/0 conn(0x7f544406cc30 0x7f544406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 --2- 172.21.15.46:0/2983229914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f5444161840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- 172.21.15.46:0/2983229914 >> 172.21.15.46:0/2983229914 conn(0x7f54440cbe30 msgr2=0x7f54440cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- 172.21.15.46:0/2983229914 shutdown_connections 2023-10-06T12:50:27.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- 172.21.15.46:0/2983229914 wait complete. 2023-10-06T12:50:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 Processor -- start 2023-10-06T12:50:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.933+0000 7f544c4ae700 1 -- start start 2023-10-06T12:50:27.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 0x7f54440c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f54440b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444162ed0 con 0x7f54440c1d80 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440c95a0 con 0x7f544406cc30 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440c8fe0 con 0x7f5444161470 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f5449a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f5449a49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47822/0 (socket says 172.21.15.46:47822) 2023-10-06T12:50:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f5449a49700 1 -- 172.21.15.46:0/834043670 learned_addr learned my addr 172.21.15.46:0/834043670 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 0x7f54440c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544aa4b700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f54440b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 -- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 msgr2=0x7f54440b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f54440b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 -- 172.21.15.46:0/834043670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 msgr2=0x7f54440c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 -- 172.21.15.46:0/834043670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5440009480 con 0x7f544406cc30 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f5449a49700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544a24a700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 0x7f54440c2ee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5440015000 tx=0x7f5440016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000b930 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000ba90 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000be50 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54440c92c0 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.937+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54440c3420 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.941+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444063f90 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.941+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 98) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f544001b640 con 0x7f544406cc30 2023-10-06T12:50:27.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.941+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 7606+0+0 (secure 0 0 0) 0x7f544007e3d0 con 0x7f544406cc30 2023-10-06T12:50:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:27.949+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f544004e520 con 0x7f544406cc30 2023-10-06T12:50:28.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.157+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f54440c1580 con 0x7f544406cc30 2023-10-06T12:50:28.201 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T12:50:28.157+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.169+0000 7f543a7fc700 1 -- 172.21.15.46:0/834043670 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5440051b40 con 0x7f544406cc30 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 msgr2=0x7f54440c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 0x7f54440c2ee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5440015000 tx=0x7f5440016000 comp rx=0 tx=0).stop 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 shutdown_connections 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5444161470 0x7f54440b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54440c1d80 0x7f54440c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 --2- 172.21.15.46:0/834043670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f544406cc30 0x7f54440c2ee0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 >> 172.21.15.46:0/834043670 conn(0x7f54440cbe30 msgr2=0x7f54440acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 shutdown_connections 2023-10-06T12:50:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.173+0000 7f544c4ae700 1 -- 172.21.15.46:0/834043670 wait complete. 2023-10-06T12:50:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T12:50:28.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T12:50:28.235 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:50:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.353+0000 7fc5d4dcc700 1 Processor -- start 2023-10-06T12:50:28.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.353+0000 7fc5d4dcc700 1 -- start start 2023-10-06T12:50:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5d4dcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc5d01591b0 0x7fc5d0069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5d4dcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5d0159850 0x7fc5d0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5d4dcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d00ce6e0 con 0x7fc5d0069270 2023-10-06T12:50:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5d4dcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d015a5c0 con 0x7fc5d01591b0 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5d4dcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00cddb0 con 0x7fc5d0159850 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5d0159850 0x7fc5d0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cf7fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc5d0069270 0x7fc5d0158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:44550/0 (socket says 172.21.15.46:44550) 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cf7fe700 1 -- 172.21.15.46:0/3307471979 learned_addr learned my addr 172.21.15.46:0/3307471979 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5c6ffd700 1 --2- 172.21.15.46:0/3307471979 >> v2:172.21.15.46:3300/0 conn(0x7fc5d01591b0 0x7fc5d0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5ceffd700 1 -- 172.21.15.46:0/3307471979 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134433090 0 0) 0x7fc5d00ce6e0 con 0x7fc5d0069270 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5ceffd700 1 -- 172.21.15.46:0/3307471979 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5bc003570 con 0x7fc5d0069270 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cffff700 1 -- 172.21.15.46:0/3307471979 >> v2:172.21.15.46:3300/0 conn(0x7fc5d01591b0 msgr2=0x7fc5d0069640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cffff700 1 --2- 172.21.15.46:0/3307471979 >> v2:172.21.15.46:3300/0 conn(0x7fc5d01591b0 0x7fc5d0069640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cffff700 1 -- 172.21.15.46:0/3307471979 >> v1:172.21.15.132:6789/0 conn(0x7fc5d0069270 legacy=0x7fc5d0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.357+0000 7fc5cffff700 1 -- 172.21.15.46:0/3307471979 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00ce090 con 0x7fc5d0159850 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5cffff700 1 --2- 172.21.15.46:0/3307471979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d0159850 0x7fc5d0159cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0067710 tx=0x7fc5c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826614d46e2cd294 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5ceffd700 1 -- 172.21.15.46:0/3307471979 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8011630 con 0x7fc5d0159850 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5ceffd700 1 -- 172.21.15.46:0/3307471979 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c80117b0 con 0x7fc5d0159850 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5ceffd700 1 -- 172.21.15.46:0/3307471979 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8011b70 con 0x7fc5d0159850 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/3307471979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d0159850 msgr2=0x7fc5d0159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/3307471979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d0159850 0x7fc5d0159cd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0067710 tx=0x7fc5c80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/3307471979 shutdown_connections 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/3307471979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d0159850 0x7fc5d0159cd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/3307471979 >> v2:172.21.15.46:3300/0 conn(0x7fc5d01591b0 0x7fc5d0069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/3307471979 >> 172.21.15.46:0/3307471979 conn(0x7fc5d0068390 msgr2=0x7fc5d00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/3307471979 shutdown_connections 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.361+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/3307471979 wait complete. 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 Processor -- start 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 -- start start 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5d0069270 0x7fc5d0155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d01591b0 0x7fc5d00ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00b1890 con 0x7fc5d0159850 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00b0c80 con 0x7fc5d01591b0 2023-10-06T12:50:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5d4dcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00b0f60 con 0x7fc5d0069270 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.365+0000 7fc5cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47854/0 (socket says 172.21.15.46:47854) 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 -- 172.21.15.46:0/641016570 learned_addr learned my addr 172.21.15.46:0/641016570 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cf7fe700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5d0069270 0x7fc5d0155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5c6ffd700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d01591b0 0x7fc5d00ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 -- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5d0069270 msgr2=0x7fc5d0155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5d0069270 0x7fc5d0155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 -- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d01591b0 msgr2=0x7fc5d00ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d01591b0 0x7fc5d00ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 -- 172.21.15.46:0/641016570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8009420 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5cffff700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80093f0 tx=0x7fc5c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c801f070 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8011920 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d00b1240 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8025420 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d0155e40 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.369+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0064590 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.373+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 98) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fc5c8023070 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.373+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 7692+0+0 (secure 0 0 0) 0x7fc5c807df70 con 0x7fc5d0159850 2023-10-06T12:50:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.381+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc5c804e070 con 0x7fc5d0159850 2023-10-06T12:50:28.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.589+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc5d00ce370 con 0x7fc5d0159850 2023-10-06T12:50:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.777+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 99) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fc5c8006c70 con 0x7fc5d0159850 2023-10-06T12:50:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.781+0000 7fc5c7fff700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 0x7fc5b00691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.781+0000 7fc5cf7fe700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 0x7fc5b00691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:28.785+0000 7fc5cf7fe700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 0x7fc5b00691c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005c50 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.173+0000 7fc5c7fff700 1 -- 172.21.15.46:0/641016570 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc5c807ff50 con 0x7fc5d0159850 2023-10-06T12:50:29.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 msgr2=0x7fc5b00691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 0x7fc5b00691c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005c50 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 msgr2=0x7fc5d00cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80093f0 tx=0x7fc5c8017000 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 shutdown_connections 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5d0159850 0x7fc5d00cf2a0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc5b0066de0 0x7fc5b00691c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5d01591b0 0x7fc5d00ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 --2- 172.21.15.46:0/641016570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5d0069270 0x7fc5d0155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 >> 172.21.15.46:0/641016570 conn(0x7fc5d0068390 msgr2=0x7fc5d00c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 shutdown_connections 2023-10-06T12:50:29.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.177+0000 7fc5d4dcc700 1 -- 172.21.15.46:0/641016570 wait complete. 2023-10-06T12:50:29.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T12:50:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.445+0000 7f1c7b59e700 1 Processor -- start 2023-10-06T12:50:29.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.445+0000 7f1c7b59e700 1 -- start start 2023-10-06T12:50:29.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c7c1595c0 0x7f1c7c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7b59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c0cf360 con 0x7f1c7c0cc500 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c15dd60 con 0x7f1c7c1591f0 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0cff00 con 0x7f1c7c1595c0 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c79d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c7c1595c0 0x7f1c7c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c7c0cc500 0x7f1c7c15ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36256/0 (socket says 172.21.15.46:36256) 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7ad9d700 1 -- 172.21.15.46:0/973722651 learned_addr learned my addr 172.21.15.46:0/973722651 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c79d9b700 1 -- 172.21.15.46:0/973722651 >> v1:172.21.15.132:6789/0 conn(0x7f1c7c1591f0 legacy=0x7f1c7c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c79d9b700 1 -- 172.21.15.46:0/973722651 >> v1:172.21.15.46:6789/0 conn(0x7f1c7c0cc500 legacy=0x7f1c7c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c79d9b700 1 -- 172.21.15.46:0/973722651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0cea30 con 0x7f1c7c1595c0 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.449+0000 7f1c7959a700 1 -- 172.21.15.46:0/973722651 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199206403 0 0) 0x7f1c7c0cf360 con 0x7f1c7c0cc500 2023-10-06T12:50:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c79d9b700 1 --2- 172.21.15.46:0/973722651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c1595c0 0x7f1c7c0af940 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098e0 tx=0x7f1c700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e27465b705da73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7959a700 1 -- 172.21.15.46:0/973722651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70009e80 con 0x7f1c7c1595c0 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7959a700 1 -- 172.21.15.46:0/973722651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7000b8f0 con 0x7f1c7c1595c0 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7959a700 1 -- 172.21.15.46:0/973722651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000bc70 con 0x7f1c7c1595c0 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 -- 172.21.15.46:0/973722651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c1595c0 msgr2=0x7f1c7c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/973722651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c1595c0 0x7f1c7c0af940 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1c700098e0 tx=0x7f1c700091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 -- 172.21.15.46:0/973722651 shutdown_connections 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/973722651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c1595c0 0x7f1c7c0af940 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 -- 172.21.15.46:0/973722651 >> 172.21.15.46:0/973722651 conn(0x7f1c7c06ef60 msgr2=0x7f1c7c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 -- 172.21.15.46:0/973722651 shutdown_connections 2023-10-06T12:50:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.453+0000 7f1c7b59e700 1 -- 172.21.15.46:0/973722651 wait complete. 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 Processor -- start 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 -- start start 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c7c0cc500 0x7f1c7c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7c1591f0 0x7f1c7c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c16da10 con 0x7f1c7c0ad570 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c150f50 con 0x7f1c7c0cc500 2023-10-06T12:50:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c15a120 con 0x7f1c7c1591f0 2023-10-06T12:50:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47870/0 (socket says 172.21.15.46:47870) 2023-10-06T12:50:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 -- 172.21.15.46:0/1416115757 learned_addr learned my addr 172.21.15.46:0/1416115757 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c79d9b700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7c1591f0 0x7f1c7c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7a59c700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c7c0cc500 0x7f1c7c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 -- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7c1591f0 msgr2=0x7f1c7c1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7c1591f0 0x7f1c7c1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 -- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c7c0cc500 msgr2=0x7f1c7c0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c7c0cc500 0x7f1c7c0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 -- 172.21.15.46:0/1416115757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c70009420 con 0x7f1c7c0ad570 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.457+0000 7f1c7ad9d700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ccc0 tx=0x7f1c74001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400a8a0 con 0x7f1c7c0ad570 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7400aa00 con 0x7f1c7c0ad570 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c15a400 con 0x7f1c7c0ad570 2023-10-06T12:50:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400adc0 con 0x7f1c7c0ad570 2023-10-06T12:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c15a9c0 con 0x7f1c7c0ad570 2023-10-06T12:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c064590 con 0x7f1c7c0ad570 2023-10-06T12:50:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.461+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 99) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f1c74007300 con 0x7f1c7c0ad570 2023-10-06T12:50:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.465+0000 7f1c6a7fc700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 0x7f1c5c062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.465+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 7692+0+0 (secure 0 0 0) 0x7f1c7407d040 con 0x7f1c7c0ad570 2023-10-06T12:50:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.465+0000 7f1c7a59c700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 0x7f1c5c062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.465+0000 7f1c7a59c700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 0x7f1c5c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c003a80 tx=0x7f1c6c007ec0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.469+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c7404d0d0 con 0x7f1c7c0ad570 2023-10-06T12:50:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:29.645+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1c7c04b5c0 con 0x7f1c7c0ad570 2023-10-06T12:50:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.177+0000 7f1c6a7fc700 1 -- 172.21.15.46:0/1416115757 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v174) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1c740506f0 con 0x7f1c7c0ad570 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 msgr2=0x7f1c5c062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 0x7f1c5c062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c003a80 tx=0x7f1c6c007ec0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 msgr2=0x7f1c7c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ccc0 tx=0x7f1c74001a80 comp rx=0 tx=0).stop 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 shutdown_connections 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0ad570 0x7f1c7c150a10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7c1591f0 0x7f1c7c1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c5c0603a0 0x7f1c5c062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 --2- 172.21.15.46:0/1416115757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c7c0cc500 0x7f1c7c0ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.181+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 >> 172.21.15.46:0/1416115757 conn(0x7f1c7c06ef60 msgr2=0x7f1c7c15e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.185+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 shutdown_connections 2023-10-06T12:50:30.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.185+0000 7f1c7b59e700 1 -- 172.21.15.46:0/1416115757 wait complete. 2023-10-06T12:50:30.192 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T12:50:30.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T12:50:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.377+0000 7f26b31cc700 1 Processor -- start 2023-10-06T12:50:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.377+0000 7f26b31cc700 1 -- start start 2023-10-06T12:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.381+0000 7f26b31cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 0x7f26ac0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.381+0000 7f26b31cc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0cf360 con 0x7f26ac0cc500 2023-10-06T12:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.381+0000 7f26b31cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac15dd60 con 0x7f26ac1595c0 2023-10-06T12:50:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.381+0000 7f26b31cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0cea30 con 0x7f26ac1591f0 2023-10-06T12:50:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b29cb700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26ac0cc500 0x7f26ac15ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36266/0 (socket says 172.21.15.46:36266) 2023-10-06T12:50:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b29cb700 1 -- 172.21.15.46:0/4052595268 learned_addr learned my addr 172.21.15.46:0/4052595268 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b21ca700 1 --2- 172.21.15.46:0/4052595268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 0x7f26ac0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980631913 0 0) 0x7f26ac0cf360 con 0x7f26ac0cc500 2023-10-06T12:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2698003570 con 0x7f26ac0cc500 2023-10-06T12:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060824509 0 0) 0x7f26ac15dd60 con 0x7f26ac1595c0 2023-10-06T12:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac0cf360 con 0x7f26ac1595c0 2023-10-06T12:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b21ca700 1 -- 172.21.15.46:0/4052595268 >> v1:172.21.15.132:6789/0 conn(0x7f26ac1595c0 legacy=0x7f26ac0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b21ca700 1 -- 172.21.15.46:0/4052595268 >> v1:172.21.15.46:6789/0 conn(0x7f26ac0cc500 legacy=0x7f26ac15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b21ca700 1 -- 172.21.15.46:0/4052595268 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0ced10 con 0x7f26ac1591f0 2023-10-06T12:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b21ca700 1 --2- 172.21.15.46:0/4052595268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 0x7f26ac0cf7f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009910 tx=0x7f26a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45dc4edbee0aadda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800b690 con 0x7f26ac1591f0 2023-10-06T12:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a800b7f0 con 0x7f26ac1591f0 2023-10-06T12:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b11c8700 1 -- 172.21.15.46:0/4052595268 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800bb90 con 0x7f26ac1591f0 2023-10-06T12:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b31cc700 1 -- 172.21.15.46:0/4052595268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 msgr2=0x7f26ac0cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.385+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4052595268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 0x7f26ac0cf7f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009910 tx=0x7f26a80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- 172.21.15.46:0/4052595268 shutdown_connections 2023-10-06T12:50:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4052595268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac1591f0 0x7f26ac0cf7f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- 172.21.15.46:0/4052595268 >> 172.21.15.46:0/4052595268 conn(0x7f26ac06ef60 msgr2=0x7f26ac0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- 172.21.15.46:0/4052595268 shutdown_connections 2023-10-06T12:50:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- 172.21.15.46:0/4052595268 wait complete. 2023-10-06T12:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 Processor -- start 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- start start 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 0x7f26ac151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac1591f0 0x7f26ac067a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 0x7f26ac170c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0683c0 con 0x7f26ac067f40 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac1521d0 con 0x7f26ac1591f0 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.389+0000 7f26b31cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac1719a0 con 0x7f26ac0cc500 2023-10-06T12:50:30.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b29cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 0x7f26ac170c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b29cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 0x7f26ac170c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47886/0 (socket says 172.21.15.46:47886) 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 0x7f26ac151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b29cb700 1 -- 172.21.15.46:0/4158368153 learned_addr learned my addr 172.21.15.46:0/4158368153 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b19c9700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac1591f0 0x7f26ac067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 -- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac1591f0 msgr2=0x7f26ac067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac1591f0 0x7f26ac067a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 -- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 msgr2=0x7f26ac170c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 0x7f26ac170c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 -- 172.21.15.46:0/4158368153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a8009420 con 0x7f26ac0cc500 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b21ca700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 0x7f26ac151c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26ac089290 tx=0x7f26a8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a801f070 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8002f70 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800b960 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac09a0f0 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.393+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac09a6b0 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac064590 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 99) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f26a8023070 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26a27fc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 0x7f2690062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 7254+0+0 (secure 0 0 0) 0x7f26a807da40 con 0x7f26ac0cc500 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26b19c9700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 0x7f2690062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.397+0000 7f26b19c9700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 0x7f2690062850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26ac09c300 tx=0x7f269c009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.405+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f26a804dc00 con 0x7f26ac0cc500 2023-10-06T12:50:30.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.585+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26ac04b5c0 con 0x7f26ac0cc500 2023-10-06T12:50:30.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:30.753+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f26a804dfd0 con 0x7f26ac0cc500 2023-10-06T12:50:31.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.181+0000 7f26a27fc700 1 -- 172.21.15.46:0/4158368153 <== mon.2 v2:172.21.15.132:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7f26a8051690 con 0x7f26ac0cc500 2023-10-06T12:50:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.185+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 msgr2=0x7f2690062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.185+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 0x7f2690062850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26ac09c300 tx=0x7f269c009040 comp rx=0 tx=0).stop 2023-10-06T12:50:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.185+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 msgr2=0x7f26ac151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.185+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 0x7f26ac151c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26ac089290 tx=0x7f26a8017000 comp rx=0 tx=0).stop 2023-10-06T12:50:31.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 shutdown_connections 2023-10-06T12:50:31.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac067f40 0x7f26ac170c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f26900603a0 0x7f2690062850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac1591f0 0x7f26ac067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 --2- 172.21.15.46:0/4158368153 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26ac0cc500 0x7f26ac151c90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 >> 172.21.15.46:0/4158368153 conn(0x7f26ac06ef60 msgr2=0x7f26ac152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 shutdown_connections 2023-10-06T12:50:31.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.189+0000 7f26b31cc700 1 -- 172.21.15.46:0/4158368153 wait complete. 2023-10-06T12:50:31.196 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T12:50:31.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T12:50:31.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.369+0000 7f75b5dcc700 1 Processor -- start 2023-10-06T12:50:31.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.369+0000 7f75b5dcc700 1 -- start start 2023-10-06T12:50:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 0x7f75b00b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75b00b0dc0 0x7f75b00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00afd20 con 0x7f75b00adbe0 2023-10-06T12:50:31.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b0159740 con 0x7f75b00b0dc0 2023-10-06T12:50:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b1760 con 0x7f75b00b30f0 2023-10-06T12:50:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b4dca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75b00adbe0 0x7f75b00d0820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36280/0 (socket says 172.21.15.46:36280) 2023-10-06T12:50:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b4dca700 1 -- 172.21.15.46:0/2164432307 learned_addr learned my addr 172.21.15.46:0/2164432307 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 --2- 172.21.15.46:0/2164432307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 0x7f75b00b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b55cb700 1 --2- 172.21.15.46:0/2164432307 >> v2:172.21.15.132:3300/0 conn(0x7f75b00b0dc0 0x7f75b00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482372072 0 0) 0x7f75b00afd20 con 0x7f75b00adbe0 2023-10-06T12:50:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a0003570 con 0x7f75b00adbe0 2023-10-06T12:50:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2866065079 0 0) 0x7f75a0003570 con 0x7f75b00adbe0 2023-10-06T12:50:31.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 -- 172.21.15.46:0/2164432307 >> v2:172.21.15.132:3300/0 conn(0x7f75b00b0dc0 msgr2=0x7f75b00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:31.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 --2- 172.21.15.46:0/2164432307 >> v2:172.21.15.132:3300/0 conn(0x7f75b00b0dc0 0x7f75b00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 -- 172.21.15.46:0/2164432307 >> v1:172.21.15.46:6789/0 conn(0x7f75b00adbe0 legacy=0x7f75b00d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 -- 172.21.15.46:0/2164432307 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00b0000 con 0x7f75b00b30f0 2023-10-06T12:50:31.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75affff700 1 --2- 172.21.15.46:0/2164432307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 0x7f75b00b3540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098e0 tx=0x7f759c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5373c00bdabd03ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4001470 con 0x7f75b00b30f0 2023-10-06T12:50:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c011420 con 0x7f75b00b30f0 2023-10-06T12:50:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75af7fe700 1 -- 172.21.15.46:0/2164432307 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c0117c0 con 0x7f75b00b30f0 2023-10-06T12:50:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2164432307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 msgr2=0x7f75b00b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.373+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2164432307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 0x7f75b00b3540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f759c0098e0 tx=0x7f759c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2164432307 shutdown_connections 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2164432307 >> v2:172.21.15.132:3300/0 conn(0x7f75b00b0dc0 0x7f75b00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2164432307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b30f0 0x7f75b00b3540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2164432307 >> 172.21.15.46:0/2164432307 conn(0x7f75b006ef60 msgr2=0x7f75b00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2164432307 shutdown_connections 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2164432307 wait complete. 2023-10-06T12:50:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 Processor -- start 2023-10-06T12:50:31.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- start start 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75b00adbe0 0x7f75b0145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 0x7f75b009a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0155590 con 0x7f75b00b30f0 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b009cc20 con 0x7f75b00adbe0 2023-10-06T12:50:31.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b5dcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b009cf00 con 0x7f75b00b0dc0 2023-10-06T12:50:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75affff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.377+0000 7f75b55cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 0x7f75b009a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:54578/0 (socket says 172.21.15.46:54578) 2023-10-06T12:50:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 -- 172.21.15.46:0/2933450298 learned_addr learned my addr 172.21.15.46:0/2933450298 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75b4dca700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75b00adbe0 0x7f75b0145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 -- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75b00adbe0 msgr2=0x7f75b0145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75b00adbe0 0x7f75b0145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 -- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 msgr2=0x7f75b009a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 0x7f75b009a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 -- 172.21.15.46:0/2933450298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009420 con 0x7f75b00b0dc0 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75b55cb700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 0x7f75b009a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:31.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75affff700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f759c0093f0 tx=0x7f759c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c01cb30 con 0x7f75b00b0dc0 2023-10-06T12:50:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c01cc90 con 0x7f75b00b0dc0 2023-10-06T12:50:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b009d1e0 con 0x7f75b00b0dc0 2023-10-06T12:50:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c01a660 con 0x7f75b00b0dc0 2023-10-06T12:50:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b009d7a0 con 0x7f75b00b0dc0 2023-10-06T12:50:31.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.381+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b009d990 con 0x7f75b00b0dc0 2023-10-06T12:50:31.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.385+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f759c01aa50 con 0x7f75b00b0dc0 2023-10-06T12:50:31.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.385+0000 7f75acff9700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 0x7f7594062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.385+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f759c07e5d0 con 0x7f75b00b0dc0 2023-10-06T12:50:31.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.385+0000 7f75b4dca700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 0x7f7594062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.385+0000 7f75b4dca700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 0x7f7594062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75a40011e0 tx=0x7f75a4001a00 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.393+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f759c04e690 con 0x7f75b00b0dc0 2023-10-06T12:50:31.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.561+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f75b009d040 con 0x7f75b00b0dc0 2023-10-06T12:50:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.573+0000 7f75acff9700 1 -- 172.21.15.46:0/2933450298 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75b009d040 con 0x7f75b00b0dc0 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 msgr2=0x7f7594062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 0x7f7594062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75a40011e0 tx=0x7f75a4001a00 comp rx=0 tx=0).stop 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 msgr2=0x7f75b0099cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f759c0093f0 tx=0x7f759c016000 comp rx=0 tx=0).stop 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 shutdown_connections 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75b00b30f0 0x7f75b009a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75b00b0dc0 0x7f75b0099cd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7594060670 0x7f7594062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 --2- 172.21.15.46:0/2933450298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75b00adbe0 0x7f75b0145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 >> 172.21.15.46:0/2933450298 conn(0x7f75b006ef60 msgr2=0x7f75b00b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 shutdown_connections 2023-10-06T12:50:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.577+0000 7f75b5dcc700 1 -- 172.21.15.46:0/2933450298 wait complete. 2023-10-06T12:50:31.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T12:50:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.821+0000 7fc525416700 1 Processor -- start 2023-10-06T12:50:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 -- start start 2023-10-06T12:50:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc5200afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc520140b00 0x7fc5200b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5200cd120 con 0x7fc5200cd5b0 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5200cc510 con 0x7fc520140b00 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc525416700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200cc7f0 con 0x7fc5200af980 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc524c15700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc520140b00 0x7fc5200b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc5177fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc5200cd5b0 0x7fc5201403f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:44586/0 (socket says 172.21.15.46:44586) 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc5177fe700 1 -- 172.21.15.46:0/563482499 learned_addr learned my addr 172.21.15.46:0/563482499 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.825+0000 7fc51ffff700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc5200afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc51f7fe700 1 -- 172.21.15.46:0/563482499 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024679 0 0) 0x7fc5200cd120 con 0x7fc5200cd5b0 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc51f7fe700 1 -- 172.21.15.46:0/563482499 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc50c003570 con 0x7fc5200cd5b0 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc524c15700 1 -- 172.21.15.46:0/563482499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 msgr2=0x7fc5200afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc524c15700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc5200afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc524c15700 1 -- 172.21.15.46:0/563482499 >> v1:172.21.15.132:6789/0 conn(0x7fc5200cd5b0 legacy=0x7fc5201403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc524c15700 1 -- 172.21.15.46:0/563482499 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200ccad0 con 0x7fc520140b00 2023-10-06T12:50:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc524c15700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc5200b0bd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc518009e60 tx=0x7fc518009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb846ae4d706d2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc51f7fe700 1 -- 172.21.15.46:0/563482499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800e070 con 0x7fc520140b00 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc51f7fe700 1 -- 172.21.15.46:0/563482499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51800bab0 con 0x7fc520140b00 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc51f7fe700 1 -- 172.21.15.46:0/563482499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800be50 con 0x7fc520140b00 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 -- 172.21.15.46:0/563482499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 msgr2=0x7fc5200b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc5200b0bd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc518009e60 tx=0x7fc518009780 comp rx=0 tx=0).stop 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 -- 172.21.15.46:0/563482499 shutdown_connections 2023-10-06T12:50:31.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc5200b0bd0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 --2- 172.21.15.46:0/563482499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc5200afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 -- 172.21.15.46:0/563482499 >> 172.21.15.46:0/563482499 conn(0x7fc52006ac90 msgr2=0x7fc52006b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 -- 172.21.15.46:0/563482499 shutdown_connections 2023-10-06T12:50:31.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.829+0000 7fc525416700 1 -- 172.21.15.46:0/563482499 wait complete. 2023-10-06T12:50:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.833+0000 7fc525416700 1 Processor -- start 2023-10-06T12:50:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.833+0000 7fc525416700 1 -- start start 2023-10-06T12:50:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc52013e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5200cd5b0 0x7fc520068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52016e4e0 con 0x7fc520140b00 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520068840 con 0x7fc5200cd5b0 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc520068b20 con 0x7fc5200af980 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47902/0 (socket says 172.21.15.46:47902) 2023-10-06T12:50:31.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 -- 172.21.15.46:0/1851109323 learned_addr learned my addr 172.21.15.46:0/1851109323 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc51ffff700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc52013e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc5177fe700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5200cd5b0 0x7fc520068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 -- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 msgr2=0x7fc52013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc52013e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 -- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5200cd5b0 msgr2=0x7fc520068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:31.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5200cd5b0 0x7fc520068160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 -- 172.21.15.46:0/1851109323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518009a20 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc524c15700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fc52006df20 tx=0x7fc518019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518012d80 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5180263f0 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180267b0 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc520068e00 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.837+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5200693c0 con 0x7fc520140b00 2023-10-06T12:50:31.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.841+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5200695e0 con 0x7fc520140b00 2023-10-06T12:50:31.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.841+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc518026b60 con 0x7fc520140b00 2023-10-06T12:50:31.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.841+0000 7fc51cff9700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 0x7fc500062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:31.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.841+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7fc51807ece0 con 0x7fc520140b00 2023-10-06T12:50:31.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.845+0000 7fc51ffff700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 0x7fc500062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:31.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.845+0000 7fc51ffff700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 0x7fc500062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc510009d10 tx=0x7fc5100091a0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:31.853+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc51804ebe0 con 0x7fc520140b00 2023-10-06T12:50:32.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.021+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fc520064590 con 0x7fc520140b00 2023-10-06T12:50:32.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.029+0000 7fc51cff9700 1 -- 172.21.15.46:0/1851109323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc518052200 con 0x7fc520140b00 2023-10-06T12:50:32.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 msgr2=0x7fc500062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 0x7fc500062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc510009d10 tx=0x7fc5100091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 msgr2=0x7fc52016dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fc52006df20 tx=0x7fc518019000 comp rx=0 tx=0).stop 2023-10-06T12:50:32.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 shutdown_connections 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520140b00 0x7fc52016dfa0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc500060670 0x7fc500062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5200cd5b0 0x7fc520068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 --2- 172.21.15.46:0/1851109323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5200af980 0x7fc52013e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 >> 172.21.15.46:0/1851109323 conn(0x7fc52006ac90 msgr2=0x7fc5200b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 shutdown_connections 2023-10-06T12:50:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.037+0000 7fc525416700 1 -- 172.21.15.46:0/1851109323 wait complete. 2023-10-06T12:50:32.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T12:50:32.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 Processor -- start 2023-10-06T12:50:32.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 -- start start 2023-10-06T12:50:32.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 0x7f4bf00b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf00af6f0 con 0x7f4bf00aee00 2023-10-06T12:50:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00b43e0 con 0x7f4bf00b2360 2023-10-06T12:50:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.321+0000 7f4bf8078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00b04b0 con 0x7f4bf0159550 2023-10-06T12:50:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf5e14700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 0x7f4bf00b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:54606/0 (socket says 172.21.15.46:54606) 2023-10-06T12:50:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 -- 172.21.15.46:0/2530398850 learned_addr learned my addr 172.21.15.46:0/2530398850 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf5613700 1 --1- 172.21.15.46:0/2530398850 >> v1:172.21.15.132:6789/0 conn(0x7f4bf00aee00 0x7f4bf0158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:44596/0 (socket says 172.21.15.46:44596) 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf4e12700 1 -- 172.21.15.46:0/2530398850 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385662211 0 0) 0x7f4bf00af6f0 con 0x7f4bf00aee00 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 -- 172.21.15.46:0/2530398850 >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 msgr2=0x7f4bf00b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 --2- 172.21.15.46:0/2530398850 >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 0x7f4bf00b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 -- 172.21.15.46:0/2530398850 >> v1:172.21.15.132:6789/0 conn(0x7f4bf00aee00 legacy=0x7f4bf0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 -- 172.21.15.46:0/2530398850 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00b0790 con 0x7f4bf0159550 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf5e14700 1 --2- 172.21.15.46:0/2530398850 >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 0x7f4bf00b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf6615700 1 --2- 172.21.15.46:0/2530398850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009f20 tx=0x7f4be8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e2877596cfb03e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf4e12700 1 -- 172.21.15.46:0/2530398850 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800e070 con 0x7f4bf0159550 2023-10-06T12:50:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf4e12700 1 -- 172.21.15.46:0/2530398850 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800b9d0 con 0x7f4bf0159550 2023-10-06T12:50:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.325+0000 7f4bf4e12700 1 -- 172.21.15.46:0/2530398850 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80092e0 con 0x7f4bf0159550 2023-10-06T12:50:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 -- 172.21.15.46:0/2530398850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 msgr2=0x7f4bf00afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 --2- 172.21.15.46:0/2530398850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009f20 tx=0x7f4be8009840 comp rx=0 tx=0).stop 2023-10-06T12:50:32.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 -- 172.21.15.46:0/2530398850 shutdown_connections 2023-10-06T12:50:32.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 --2- 172.21.15.46:0/2530398850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf0159550 0x7f4bf00afdd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 --2- 172.21.15.46:0/2530398850 >> v2:172.21.15.132:3300/0 conn(0x7f4bf00b2360 0x7f4bf00b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 -- 172.21.15.46:0/2530398850 >> 172.21.15.46:0/2530398850 conn(0x7f4bf006e780 msgr2=0x7f4bf006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:32.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 -- 172.21.15.46:0/2530398850 shutdown_connections 2023-10-06T12:50:32.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 -- 172.21.15.46:0/2530398850 wait complete. 2023-10-06T12:50:32.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.329+0000 7f4bf8078700 1 Processor -- start 2023-10-06T12:50:32.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 -- start start 2023-10-06T12:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00aee00 0x7f4bf0158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00b2360 0x7f4bf015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf016ca70 con 0x7f4bf0159550 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf015da30 con 0x7f4bf00b2360 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf8078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf015dd10 con 0x7f4bf00aee00 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47914/0 (socket says 172.21.15.46:47914) 2023-10-06T12:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 -- 172.21.15.46:0/625667163 learned_addr learned my addr 172.21.15.46:0/625667163 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf5e14700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00aee00 0x7f4bf0158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf5613700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00b2360 0x7f4bf015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 -- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00aee00 msgr2=0x7f4bf0158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00aee00 0x7f4bf0158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 -- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00b2360 msgr2=0x7f4bf015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00b2360 0x7f4bf015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 -- 172.21.15.46:0/625667163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009420 con 0x7f4bf0159550 2023-10-06T12:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4bf6615700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00b05f0 tx=0x7f4be8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.333+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8007b00 con 0x7f4bf0159550 2023-10-06T12:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8007c80 con 0x7f4bf0159550 2023-10-06T12:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be8009ae0 con 0x7f4bf0159550 2023-10-06T12:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be801e430 con 0x7f4bf0159550 2023-10-06T12:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf015e2d0 con 0x7f4bf0159550 2023-10-06T12:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0088840 con 0x7f4bf0159550 2023-10-06T12:50:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.337+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4be8003040 con 0x7f4bf0159550 2023-10-06T12:50:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.341+0000 7f4be5ffb700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 0x7f4bd4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.341+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f4be807efc0 con 0x7f4bf0159550 2023-10-06T12:50:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.341+0000 7f4bf5e14700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 0x7f4bd4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.341+0000 7f4bf5e14700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 0x7f4bd4062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009f40 tx=0x7f4bec0091a0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.349+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4be804eec0 con 0x7f4bf0159550 2023-10-06T12:50:32.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.525+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f4bf00b0a70 con 0x7f4bf0159550 2023-10-06T12:50:32.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.537+0000 7f4be5ffb700 1 -- 172.21.15.46:0/625667163 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4be80524e0 con 0x7f4bf0159550 2023-10-06T12:50:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 msgr2=0x7f4bd4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 0x7f4bd4062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009f40 tx=0x7f4bec0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 msgr2=0x7f4bf015d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00b05f0 tx=0x7f4be8018000 comp rx=0 tx=0).stop 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 shutdown_connections 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bf0159550 0x7f4bf015d3d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4bd4060670 0x7f4bd4062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00b2360 0x7f4bf015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 --2- 172.21.15.46:0/625667163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00aee00 0x7f4bf0158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 >> 172.21.15.46:0/625667163 conn(0x7f4bf006e780 msgr2=0x7f4bf00b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 shutdown_connections 2023-10-06T12:50:32.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.541+0000 7f4bf8078700 1 -- 172.21.15.46:0/625667163 wait complete. 2023-10-06T12:50:32.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-06T12:50:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.773+0000 7ff0a3f85700 1 Processor -- start 2023-10-06T12:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.773+0000 7ff0a3f85700 1 -- start start 2023-10-06T12:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff09c161470 0x7ff09c161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c0c1d80 0x7ff09c0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff09c06cc30 0x7ff09c06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0c18f0 con 0x7ff09c161470 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0c0ce0 con 0x7ff09c06cc30 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a3f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0c0fc0 con 0x7ff09c0c1d80 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff09c161470 0x7ff09c161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff09c161470 0x7ff09c161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47928/0 (socket says 172.21.15.46:47928) 2023-10-06T12:50:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 -- 172.21.15.46:0/539068975 learned_addr learned my addr 172.21.15.46:0/539068975 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a2522700 1 --2- 172.21.15.46:0/539068975 >> v2:172.21.15.132:3300/0 conn(0x7ff09c06cc30 0x7ff09c06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1520700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c0c1d80 0x7ff09c0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 -- 172.21.15.46:0/539068975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c0c1d80 msgr2=0x7ff09c0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c0c1d80 0x7ff09c0c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 -- 172.21.15.46:0/539068975 >> v2:172.21.15.132:3300/0 conn(0x7ff09c06cc30 msgr2=0x7ff09c06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/539068975 >> v2:172.21.15.132:3300/0 conn(0x7ff09c06cc30 0x7ff09c06d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 -- 172.21.15.46:0/539068975 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0c12a0 con 0x7ff09c161470 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c161470 0x7ff09c161840 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0c0bd0 tx=0x7ff098009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68fdcc3c390d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a0d1f700 1 -- 172.21.15.46:0/539068975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800b6d0 con 0x7ff09c161470 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a0d1f700 1 -- 172.21.15.46:0/539068975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09800b830 con 0x7ff09c161470 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.777+0000 7ff0a0d1f700 1 -- 172.21.15.46:0/539068975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800bbf0 con 0x7ff09c161470 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 -- 172.21.15.46:0/539068975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c161470 msgr2=0x7ff09c161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c161470 0x7ff09c161840 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0c0bd0 tx=0x7ff098009250 comp rx=0 tx=0).stop 2023-10-06T12:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 -- 172.21.15.46:0/539068975 shutdown_connections 2023-10-06T12:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/539068975 >> v2:172.21.15.132:3300/0 conn(0x7ff09c06cc30 0x7ff09c06d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c0c1d80 0x7ff09c0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/539068975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c161470 0x7ff09c161840 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 -- 172.21.15.46:0/539068975 >> 172.21.15.46:0/539068975 conn(0x7ff09c0cbe30 msgr2=0x7ff09c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 -- 172.21.15.46:0/539068975 shutdown_connections 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.781+0000 7ff0a3f85700 1 -- 172.21.15.46:0/539068975 wait complete. 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 Processor -- start 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 -- start start 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c06cc30 0x7ff09c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff09c161470 0x7ff09c15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c16da10 con 0x7ff09c0c1d80 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c16e2b0 con 0x7ff09c161470 2023-10-06T12:50:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a3f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0c8a00 con 0x7ff09c06cc30 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47938/0 (socket says 172.21.15.46:47938) 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 -- 172.21.15.46:0/2686467806 learned_addr learned my addr 172.21.15.46:0/2686467806 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a2522700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff09c161470 0x7ff09c15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c06cc30 0x7ff09c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 -- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c06cc30 msgr2=0x7ff09c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c06cc30 0x7ff09c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 -- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff09c161470 msgr2=0x7ff09c15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff09c161470 0x7ff09c15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:32.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.785+0000 7ff0a1520700 1 -- 172.21.15.46:0/2686467806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009490 con 0x7ff09c0c1d80 2023-10-06T12:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff0a1520700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00ecb0 tx=0x7ff08c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00c850 con 0x7ff09c0c1d80 2023-10-06T12:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c00c9b0 con 0x7ff09c0c1d80 2023-10-06T12:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00cd50 con 0x7ff09c0c1d80 2023-10-06T12:50:32.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c16dc60 con 0x7ff09c0c1d80 2023-10-06T12:50:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0c68e0 con 0x7ff09c0c1d80 2023-10-06T12:50:32.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c063f90 con 0x7ff09c0c1d80 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.789+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff08c013680 con 0x7ff09c0c1d80 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.793+0000 7ff091ffb700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 0x7ff080062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.793+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7ff08c07cf90 con 0x7ff09c0c1d80 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.793+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 0x7ff080062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.793+0000 7ff0a1d21700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 0x7ff080062b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0c0bd0 tx=0x7ff098017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:32.797+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff08c04cfa0 con 0x7ff09c0c1d80 2023-10-06T12:50:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.041+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7ff09c0c1580 con 0x7ff09c0c1d80 2023-10-06T12:50:33.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.189+0000 7ff091ffb700 1 -- 172.21.15.46:0/2686467806 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2964) v1 ==== 165+0+0 (secure 0 0 0) 0x7ff08c0505c0 con 0x7ff09c0c1d80 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 msgr2=0x7ff080062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 0x7ff080062b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0c0bd0 tx=0x7ff098017000 comp rx=0 tx=0).stop 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 msgr2=0x7ff09c15e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00ecb0 tx=0x7ff08c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 shutdown_connections 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff0800606a0 0x7ff080062b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff09c161470 0x7ff09c15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff09c0c1d80 0x7ff09c15e710 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 --2- 172.21.15.46:0/2686467806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff09c06cc30 0x7ff09c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 >> 172.21.15.46:0/2686467806 conn(0x7ff09c0cbe30 msgr2=0x7ff09c0abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 shutdown_connections 2023-10-06T12:50:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.193+0000 7ff0a3f85700 1 -- 172.21.15.46:0/2686467806 wait complete. 2023-10-06T12:50:33.275 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-06T12:50:33.275 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-06T12:50:33.275 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T12:50:33.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-06T12:50:33.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 Processor -- start 2023-10-06T12:50:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 -- start start 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2fc4159420 0x7f2fc40b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2fc40d0130 0x7f2fc40d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc40b4f80 con 0x7f2fc4159050 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40b40c0 con 0x7f2fc40d0130 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.437+0000 7f2fc8bc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc40d0a80 con 0x7f2fc4159420 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc259c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2fc4159050 0x7f2fc40b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58698/0 (socket says 172.21.15.46:58698) 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc259c700 1 -- 172.21.15.46:0/2320707362 learned_addr learned my addr 172.21.15.46:0/2320707362 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 --2- 172.21.15.46:0/2320707362 >> v2:172.21.15.46:3300/0 conn(0x7f2fc40d0130 0x7f2fc40d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fbbfff700 1 --2- 172.21.15.46:0/2320707362 >> v2:172.21.15.132:3300/0 conn(0x7f2fc4159420 0x7f2fc40b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc1d9b700 1 -- 172.21.15.46:0/2320707362 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437260606 0 0) 0x7f2fc40b4f80 con 0x7f2fc4159050 2023-10-06T12:50:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 -- 172.21.15.46:0/2320707362 >> v2:172.21.15.132:3300/0 conn(0x7f2fc4159420 msgr2=0x7f2fc40b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 --2- 172.21.15.46:0/2320707362 >> v2:172.21.15.132:3300/0 conn(0x7f2fc4159420 0x7f2fc40b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 -- 172.21.15.46:0/2320707362 >> v1:172.21.15.46:6789/0 conn(0x7f2fc4159050 legacy=0x7f2fc40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 -- 172.21.15.46:0/2320707362 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40b4370 con 0x7f2fc40d0130 2023-10-06T12:50:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc2d9d700 1 --2- 172.21.15.46:0/2320707362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc40d0130 0x7f2fc40d0540 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40cdbf0 tx=0x7f2fbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6db0fe0f4768963 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:33.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc1d9b700 1 -- 172.21.15.46:0/2320707362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00e070 con 0x7f2fc40d0130 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc1d9b700 1 -- 172.21.15.46:0/2320707362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc00bb60 con 0x7f2fc40d0130 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc1d9b700 1 -- 172.21.15.46:0/2320707362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc012430 con 0x7f2fc40d0130 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/2320707362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc40d0130 msgr2=0x7f2fc40d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/2320707362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc40d0130 0x7f2fc40d0540 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40cdbf0 tx=0x7f2fbc009780 comp rx=0 tx=0).stop 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/2320707362 shutdown_connections 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/2320707362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc40d0130 0x7f2fc40d0540 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/2320707362 >> v2:172.21.15.132:3300/0 conn(0x7f2fc4159420 0x7f2fc40b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/2320707362 >> 172.21.15.46:0/2320707362 conn(0x7f2fc406e780 msgr2=0x7f2fc406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/2320707362 shutdown_connections 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.441+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/2320707362 wait complete. 2023-10-06T12:50:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 Processor -- start 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 -- start start 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc40d0130 0x7f2fc40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc4159420 0x7f2fc4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40c89d0 con 0x7f2fc4159050 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40c8cb0 con 0x7f2fc4159420 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fc8bc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc40c8f90 con 0x7f2fc40d0130 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60306/0 (socket says 172.21.15.46:60306) 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 -- 172.21.15.46:0/1063038685 learned_addr learned my addr 172.21.15.46:0/1063038685 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 -- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc40d0130 msgr2=0x7f2fc40cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc40d0130 0x7f2fc40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 -- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc4159420 msgr2=0x7f2fc4156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc4159420 0x7f2fc4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.445+0000 7f2fbbfff700 1 -- 172.21.15.46:0/1063038685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc009a20 con 0x7f2fc4159050 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fbbfff700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e930 tx=0x7f2fac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac009510 con 0x7f2fc4159050 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00c850 con 0x7f2fc4159050 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc40c9270 con 0x7f2fc4159050 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00cc10 con 0x7f2fc4159050 2023-10-06T12:50:33.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc4157380 con 0x7f2fc4159050 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.449+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc4088b00 con 0x7f2fc4159050 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.453+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2fac013470 con 0x7f2fc4159050 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.453+0000 7f2fba7fc700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 0x7f2fa4062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.453+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f2fac07cef0 con 0x7f2fc4159050 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.453+0000 7f2fc259c700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 0x7f2fa4062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.453+0000 7f2fc259c700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 0x7f2fa4062bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40011e0 tx=0x7f2fb40079d0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:33.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.461+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2fac04cf80 con 0x7f2fc4159050 2023-10-06T12:50:33.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:33.689+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f2fc4159950 con 0x7f2fc4159050 2023-10-06T12:50:34.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.197+0000 7f2fba7fc700 1 -- 172.21.15.46:0/1063038685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2965) v1 ==== 166+0+0 (secure 0 0 0) 0x7f2fc4159950 con 0x7f2fc4159050 2023-10-06T12:50:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 msgr2=0x7f2fa4062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 0x7f2fa4062bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40011e0 tx=0x7f2fb40079d0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 msgr2=0x7f2fc40c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00e930 tx=0x7f2fac0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 shutdown_connections 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2fa4060740 0x7f2fa4062bf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc4159420 0x7f2fc4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fc4159050 0x7f2fc40c8180 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 --2- 172.21.15.46:0/1063038685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fc40d0130 0x7f2fc40cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 >> 172.21.15.46:0/1063038685 conn(0x7f2fc406e780 msgr2=0x7f2fc406eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 shutdown_connections 2023-10-06T12:50:34.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.201+0000 7f2fc8bc5700 1 -- 172.21.15.46:0/1063038685 wait complete. 2023-10-06T12:50:34.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T12:50:34.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 Processor -- start 2023-10-06T12:50:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 -- start start 2023-10-06T12:50:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cd0156f60 0x7f4cd00cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cd0157330 0x7f4cd00cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd00cd220 con 0x7f4cd00af9e0 2023-10-06T12:50:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00cc610 con 0x7f4cd0156f60 2023-10-06T12:50:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.397+0000 7f4cd4af0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00cc8f0 con 0x7f4cd0157330 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cd0156f60 0x7f4cd00cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cd0156f60 0x7f4cd00cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60320/0 (socket says 172.21.15.46:60320) 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 -- 172.21.15.46:0/201475501 learned_addr learned my addr 172.21.15.46:0/201475501 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4ccdd9b700 1 --2- 172.21.15.46:0/201475501 >> v2:172.21.15.132:3300/0 conn(0x7f4cd0157330 0x7f4cd00cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 -- 172.21.15.46:0/201475501 >> v2:172.21.15.132:3300/0 conn(0x7f4cd0157330 msgr2=0x7f4cd00cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 --2- 172.21.15.46:0/201475501 >> v2:172.21.15.132:3300/0 conn(0x7f4cd0157330 0x7f4cd00cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 -- 172.21.15.46:0/201475501 >> v1:172.21.15.132:6789/0 conn(0x7f4cd00af9e0 legacy=0x7f4cd00cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 -- 172.21.15.46:0/201475501 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00ccbd0 con 0x7f4cd0156f60 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cce59c700 1 --2- 172.21.15.46:0/201475501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0156f60 0x7f4cd00cba50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006d930 tx=0x7f4cc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d684b46beac2be0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4ccd59a700 1 -- 172.21.15.46:0/201475501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0009e70 con 0x7f4cd0156f60 2023-10-06T12:50:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4ccd59a700 1 -- 172.21.15.46:0/201475501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc000b860 con 0x7f4cd0156f60 2023-10-06T12:50:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4ccd59a700 1 -- 172.21.15.46:0/201475501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000bc00 con 0x7f4cd0156f60 2023-10-06T12:50:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cd4af0700 1 -- 172.21.15.46:0/201475501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0156f60 msgr2=0x7f4cd00cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.401+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/201475501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0156f60 0x7f4cd00cba50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006d930 tx=0x7f4cc0009250 comp rx=0 tx=0).stop 2023-10-06T12:50:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 -- 172.21.15.46:0/201475501 shutdown_connections 2023-10-06T12:50:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/201475501 >> v2:172.21.15.132:3300/0 conn(0x7f4cd0157330 0x7f4cd00cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/201475501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0156f60 0x7f4cd00cba50 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 -- 172.21.15.46:0/201475501 >> 172.21.15.46:0/201475501 conn(0x7f4cd006ae60 msgr2=0x7f4cd006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 -- 172.21.15.46:0/201475501 shutdown_connections 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 -- 172.21.15.46:0/201475501 wait complete. 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 Processor -- start 2023-10-06T12:50:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.405+0000 7f4cd4af0700 1 -- start start 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd00af9e0 0x7f4cd015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 0x7f4cd015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd015c2f0 con 0x7f4cd0157330 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd015c5d0 con 0x7f4cd0156f60 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd015c8b0 con 0x7f4cd00af9e0 2023-10-06T12:50:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60330/0 (socket says 172.21.15.46:60330) 2023-10-06T12:50:34.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 -- 172.21.15.46:0/2725234559 learned_addr learned my addr 172.21.15.46:0/2725234559 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4ccdd9b700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 0x7f4cd015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cce59c700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd00af9e0 0x7f4cd015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 -- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd00af9e0 msgr2=0x7f4cd015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd00af9e0 0x7f4cd015bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 -- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 msgr2=0x7f4cd015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 0x7f4cd015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 -- 172.21.15.46:0/2725234559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00094f0 con 0x7f4cd0157330 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4ccdd9b700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 0x7f4cd015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:34.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cced9d700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800c8e0 tx=0x7f4cc8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc800ce80 con 0x7f4cd0157330 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc800a7e0 con 0x7f4cd0157330 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd015cb90 con 0x7f4cd0157330 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc800ab80 con 0x7f4cd0157330 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.409+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd015a8f0 con 0x7f4cd0157330 2023-10-06T12:50:34.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.413+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd015d090 con 0x7f4cd0157330 2023-10-06T12:50:34.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.413+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4cc8012420 con 0x7f4cd0157330 2023-10-06T12:50:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.413+0000 7f4cbe7fc700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 0x7f4cb0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.413+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f4cc807d110 con 0x7f4cd0157330 2023-10-06T12:50:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.413+0000 7f4cce59c700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 0x7f4cb0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.417+0000 7f4cce59c700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 0x7f4cb0062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006d930 tx=0x7f4cc0017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:34.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.421+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4cc804d120 con 0x7f4cd0157330 2023-10-06T12:50:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.597+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4cd00cceb0 con 0x7f4cd0157330 2023-10-06T12:50:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.609+0000 7f4cbe7fc700 1 -- 172.21.15.46:0/2725234559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4cc8050740 con 0x7f4cd0157330 2023-10-06T12:50:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 msgr2=0x7f4cb0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 0x7f4cb0062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006d930 tx=0x7f4cc0017000 comp rx=0 tx=0).stop 2023-10-06T12:50:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 msgr2=0x7f4cd014cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800c8e0 tx=0x7f4cc8001710 comp rx=0 tx=0).stop 2023-10-06T12:50:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 shutdown_connections 2023-10-06T12:50:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0157330 0x7f4cd014cb00 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4cb0060670 0x7f4cb0062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd0156f60 0x7f4cd015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 --2- 172.21.15.46:0/2725234559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cd00af9e0 0x7f4cd015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 >> 172.21.15.46:0/2725234559 conn(0x7f4cd006ae60 msgr2=0x7f4cd00cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 shutdown_connections 2023-10-06T12:50:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.613+0000 7f4cd4af0700 1 -- 172.21.15.46:0/2725234559 wait complete. 2023-10-06T12:50:34.695 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:50:34.695 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:50:34.740 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:50:34.785 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:50:34.828 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T12:50:34.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.989+0000 7f5ded1c5700 1 Processor -- start 2023-10-06T12:50:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.989+0000 7f5ded1c5700 1 -- start start 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5de80c1ec0 0x7f5de80ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de806d460 con 0x7f5de806e8a0 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de80c7000 con 0x7f5de815ab60 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de806ed40 con 0x7f5de80c1ec0 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5dde59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5de806e8a0 0x7f5de806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58702/0 (socket says 172.21.15.46:58702) 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5dde59c700 1 -- 172.21.15.46:0/1383069220 learned_addr learned my addr 172.21.15.46:0/1383069220 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de759e700 1 --2- 172.21.15.46:0/1383069220 >> v2:172.21.15.46:3300/0 conn(0x7f5de80c1ec0 0x7f5de80ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758831918 0 0) 0x7f5de806d460 con 0x7f5de806e8a0 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd4003570 con 0x7f5de806e8a0 2023-10-06T12:50:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607253776 0 0) 0x7f5de80c7000 con 0x7f5de815ab60 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de806d460 con 0x7f5de815ab60 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de759e700 1 -- 172.21.15.46:0/1383069220 >> v1:172.21.15.132:6789/0 conn(0x7f5de815ab60 legacy=0x7f5de815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de759e700 1 -- 172.21.15.46:0/1383069220 >> v1:172.21.15.46:6789/0 conn(0x7f5de806e8a0 legacy=0x7f5de806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de759e700 1 -- 172.21.15.46:0/1383069220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de80c72c0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de759e700 1 --2- 172.21.15.46:0/1383069220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80ad6c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098e0 tx=0x7f5de00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d58785d4147fd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0011630 con 0x7f5de80c1ec0 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0011790 con 0x7f5de80c1ec0 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5de659c700 1 -- 172.21.15.46:0/1383069220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0011b30 con 0x7f5de80c1ec0 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 -- 172.21.15.46:0/1383069220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 msgr2=0x7f5de80ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.993+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/1383069220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80ad6c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098e0 tx=0x7f5de00091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 -- 172.21.15.46:0/1383069220 shutdown_connections 2023-10-06T12:50:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/1383069220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80ad6c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 -- 172.21.15.46:0/1383069220 >> 172.21.15.46:0/1383069220 conn(0x7f5de80cbe30 msgr2=0x7f5de80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 -- 172.21.15.46:0/1383069220 shutdown_connections 2023-10-06T12:50:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 -- 172.21.15.46:0/1383069220 wait complete. 2023-10-06T12:50:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 Processor -- start 2023-10-06T12:50:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:34.997+0000 7f5ded1c5700 1 -- start start 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de806e8a0 0x7f5de815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 0x7f5de815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80c5750 con 0x7f5de80c1ec0 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80c5a30 con 0x7f5de806e8a0 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80c5d10 con 0x7f5de815ab60 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60342/0 (socket says 172.21.15.46:60342) 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 -- 172.21.15.46:0/2233554952 learned_addr learned my addr 172.21.15.46:0/2233554952 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5de6d9d700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de806e8a0 0x7f5de815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5de759e700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 0x7f5de815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 -- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 msgr2=0x7f5de815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 0x7f5de815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 -- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de806e8a0 msgr2=0x7f5de815d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de806e8a0 0x7f5de815d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 -- 172.21.15.46:0/2233554952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0009420 con 0x7f5de80c1ec0 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5de759e700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 0x7f5de815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c790 tx=0x7f5dd0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005c00 con 0x7f5de80c1ec0 2023-10-06T12:50:35.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0005d60 con 0x7f5de80c1ec0 2023-10-06T12:50:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00072e0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de80c5ff0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.001+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de80c67f0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.005+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de80c64f0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.005+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5dd0012400 con 0x7f5de80c1ec0 2023-10-06T12:50:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.005+0000 7f5ddf7fe700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 0x7f5dc8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.005+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f5dd007c290 con 0x7f5de80c1ec0 2023-10-06T12:50:35.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.005+0000 7f5de6d9d700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 0x7f5dc8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.009+0000 7f5de6d9d700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 0x7f5dc8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800f3c0 tx=0x7f5dd8009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.013+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5dd0049be0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.225+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5de8063f90 con 0x7f5de80c1ec0 2023-10-06T12:50:35.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.225+0000 7f5ddf7fe700 1 -- 172.21.15.46:0/2233554952 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f5dd0017de0 con 0x7f5de80c1ec0 2023-10-06T12:50:35.317 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:35.318 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/1576317186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6835","nonce":1576317186},{"type":"v1","addr":"172.21.15.46:6837","nonce":1576317186}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":7862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1882284588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1882284588},{"type":"v1","addr":"172.21.15.132:6835","nonce":1882284588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9067,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1801756951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1801756951},{"type":"v1","addr":"172.21.15.132:6833","nonce":1801756951}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79},{"gid":9544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6836/3083984896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3083984896},{"type":"v1","addr":"172.21.15.46:6836","nonce":3083984896}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-10-06T12:50:35.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 msgr2=0x7f5dc8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 0x7f5dc8062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800f3c0 tx=0x7f5dd8009040 comp rx=0 tx=0).stop 2023-10-06T12:50:35.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 msgr2=0x7f5de80c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c790 tx=0x7f5dd0007ea0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 shutdown_connections 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de815ab60 0x7f5de815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de80c1ec0 0x7f5de80c2ee0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc8060670 0x7f5dc8062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.229+0000 7f5ded1c5700 1 --2- 172.21.15.46:0/2233554952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de806e8a0 0x7f5de815d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.233+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 >> 172.21.15.46:0/2233554952 conn(0x7f5de80cbe30 msgr2=0x7f5de8157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.233+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 shutdown_connections 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.233+0000 7f5ded1c5700 1 -- 172.21.15.46:0/2233554952 wait complete. 2023-10-06T12:50:35.320 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 87 2023-10-06T12:50:35.321 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T12:50:35.321 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T12:50:35.321 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T12:50:35.321 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:50:35.322 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T12:50:35.322 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T12:50:35.322 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T12:50:35.322 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:50:35.322 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T12:50:35.322 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T12:50:35.323 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T12:50:35.323 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:50:35.323 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T12:50:35.323 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T12:50:35.323 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T12:50:35.323 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:50:35.340 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T12:50:35.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T12:50:35.341 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T12:50:35.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T12:50:35.343 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T12:50:35.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T12:50:35.344 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T12:50:35.345 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T12:50:35.391 INFO:tasks.ceph.mds.c:Started 2023-10-06T12:50:35.505 DEBUG:teuthology.parallel:result is None 2023-10-06T12:50:35.506 INFO:tasks.ceph.mds.b:Started 2023-10-06T12:50:35.506 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T12:50:35.507 DEBUG:teuthology.parallel:result is None 2023-10-06T12:50:35.509 INFO:tasks.ceph.mds.a:Started 2023-10-06T12:50:35.509 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:50:35.509 DEBUG:teuthology.parallel:result is None 2023-10-06T12:50:35.525 INFO:tasks.ceph.mds.d:Started 2023-10-06T12:50:35.526 DEBUG:teuthology.parallel:result is None 2023-10-06T12:50:35.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T12:50:35.614 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:50:35.668 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T12:50:35.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.693+0000 7f1c78bc5700 1 Processor -- start 2023-10-06T12:50:35.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.693+0000 7f1c78bc5700 1 -- start start 2023-10-06T12:50:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c78bc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7415c770 0x7f1c740cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c78bc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c740ccb50 0x7f1c740ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c78bc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740cc150 con 0x7f1c740cea30 2023-10-06T12:50:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c78bc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740cfbb0 con 0x7f1c740ccb50 2023-10-06T12:50:35.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c78bc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740cdbf0 con 0x7f1c7415c770 2023-10-06T12:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c72d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c740cea30 0x7f1c740c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58716/0 (socket says 172.21.15.46:58716) 2023-10-06T12:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c72d9d700 1 -- 172.21.15.46:0/3885396663 learned_addr learned my addr 172.21.15.46:0/3885396663 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7259c700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7415c770 0x7f1c740cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 --2- 172.21.15.46:0/3885396663 >> v2:172.21.15.132:3300/0 conn(0x7f1c740ccb50 0x7f1c740ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7159a700 1 -- 172.21.15.46:0/3885396663 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828610303 0 0) 0x7f1c740cc150 con 0x7f1c740cea30 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7159a700 1 -- 172.21.15.46:0/3885396663 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c5c003570 con 0x7f1c740cea30 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 -- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7415c770 msgr2=0x7f1c740cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7415c770 0x7f1c740cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 -- 172.21.15.46:0/3885396663 >> v1:172.21.15.46:6789/0 conn(0x7f1c740cea30 legacy=0x7f1c740c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 -- 172.21.15.46:0/3885396663 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740cee70 con 0x7f1c740ccb50 2023-10-06T12:50:35.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c71d9b700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c740ce4f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1c680098e0 tx=0x7f1c680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b331174aeb1ee1ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7159a700 1 -- 172.21.15.46:0/3885396663 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011630 con 0x7f1c740ccb50 2023-10-06T12:50:35.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7159a700 1 -- 172.21.15.46:0/3885396663 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011790 con 0x7f1c740ccb50 2023-10-06T12:50:35.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.697+0000 7f1c7159a700 1 -- 172.21.15.46:0/3885396663 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011b30 con 0x7f1c740ccb50 2023-10-06T12:50:35.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 msgr2=0x7f1c740ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c740ce4f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1c680098e0 tx=0x7f1c680091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- 172.21.15.46:0/3885396663 shutdown_connections 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c740ce4f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/3885396663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c7415c770 0x7f1c740cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- 172.21.15.46:0/3885396663 >> 172.21.15.46:0/3885396663 conn(0x7f1c74099480 msgr2=0x7f1c7406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- 172.21.15.46:0/3885396663 shutdown_connections 2023-10-06T12:50:35.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- 172.21.15.46:0/3885396663 wait complete. 2023-10-06T12:50:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 Processor -- start 2023-10-06T12:50:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.701+0000 7f1c78bc5700 1 -- start start 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c74158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740cea30 0x7f1c740c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740b5430 con 0x7f1c7415c770 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c74158840 con 0x7f1c740ccb50 2023-10-06T12:50:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7416a9e0 con 0x7f1c740cea30 2023-10-06T12:50:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60374/0 (socket says 172.21.15.46:60374) 2023-10-06T12:50:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 -- 172.21.15.46:0/241344277 learned_addr learned my addr 172.21.15.46:0/241344277 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:35.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c7259c700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c74158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c71d9b700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740cea30 0x7f1c740c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 -- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740cea30 msgr2=0x7f1c740c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740cea30 0x7f1c740c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 -- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 msgr2=0x7f1c74158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c74158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:35.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 -- 172.21.15.46:0/241344277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009420 con 0x7f1c7415c770 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c72d9d700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cca0 tx=0x7f1c6c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00a8d0 con 0x7f1c7415c770 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00aa30 con 0x7f1c7415c770 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00adb0 con 0x7f1c7415c770 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c74169dd0 con 0x7f1c7415c770 2023-10-06T12:50:35.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.705+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7416a390 con 0x7f1c7415c770 2023-10-06T12:50:35.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.709+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7416ab50 con 0x7f1c7415c770 2023-10-06T12:50:35.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.713+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1c6c007300 con 0x7f1c7415c770 2023-10-06T12:50:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.713+0000 7f1c627fc700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 0x7f1c54062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.713+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f1c6c07d1e0 con 0x7f1c7415c770 2023-10-06T12:50:35.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.713+0000 7f1c7259c700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 0x7f1c54062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:35.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.713+0000 7f1c7259c700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 0x7f1c54062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009d00 tx=0x7f1c64009120 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:35.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.717+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1c6c04d1e0 con 0x7f1c7415c770 2023-10-06T12:50:35.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:35.901+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1c740660f0 con 0x7f1c7415c770 2023-10-06T12:50:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.225+0000 7f1c627fc700 1 -- 172.21.15.46:0/241344277 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v177) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1c6c050800 con 0x7f1c7415c770 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 msgr2=0x7f1c54062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 0x7f1c54062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009d00 tx=0x7f1c64009120 comp rx=0 tx=0).stop 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 msgr2=0x7f1c740cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00cca0 tx=0x7f1c6c001ae0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 shutdown_connections 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7415c770 0x7f1c740cd030 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740cea30 0x7f1c740c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1c54060670 0x7f1c54062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 --2- 172.21.15.46:0/241344277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740ccb50 0x7f1c74158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 >> 172.21.15.46:0/241344277 conn(0x7f1c74099480 msgr2=0x7f1c740c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 shutdown_connections 2023-10-06T12:50:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.229+0000 7f1c78bc5700 1 -- 172.21.15.46:0/241344277 wait complete. 2023-10-06T12:50:36.238 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T12:50:36.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T12:50:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.421+0000 7fdb2dacd700 1 Processor -- start 2023-10-06T12:50:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.421+0000 7fdb2dacd700 1 -- start start 2023-10-06T12:50:36.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:50:36.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb2dacd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdb280c1ec0 0x7fdb280ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb2dacd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2806d460 con 0x7fdb2806e8a0 2023-10-06T12:50:36.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb2dacd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb280c7000 con 0x7fdb2815ab60 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb2dacd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2806ed40 con 0x7fdb280c1ec0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb26ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdb2806e8a0 0x7fdb2806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58732/0 (socket says 172.21.15.46:58732) 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb26ffd700 1 -- 172.21.15.46:0/3424644226 learned_addr learned my addr 172.21.15.46:0/3424644226 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb27fff700 1 --2- 172.21.15.46:0/3424644226 >> v2:172.21.15.132:3300/0 conn(0x7fdb280c1ec0 0x7fdb280ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb267fc700 1 -- 172.21.15.46:0/3424644226 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631783788 0 0) 0x7fdb2806d460 con 0x7fdb2806e8a0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb267fc700 1 -- 172.21.15.46:0/3424644226 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb10003570 con 0x7fdb2806e8a0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb27fff700 1 -- 172.21.15.46:0/3424644226 >> v1:172.21.15.132:6789/0 conn(0x7fdb2815ab60 legacy=0x7fdb2815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb27fff700 1 -- 172.21.15.46:0/3424644226 >> v1:172.21.15.46:6789/0 conn(0x7fdb2806e8a0 legacy=0x7fdb2806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb27fff700 1 -- 172.21.15.46:0/3424644226 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280c72c0 con 0x7fdb280c1ec0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb27fff700 1 --2- 172.21.15.46:0/3424644226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280ad6c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009e60 tx=0x7fdb20009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e15c6b2d70ca4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb267fc700 1 -- 172.21.15.46:0/3424644226 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000e070 con 0x7fdb280c1ec0 2023-10-06T12:50:36.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb267fc700 1 -- 172.21.15.46:0/3424644226 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000bb60 con 0x7fdb280c1ec0 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.425+0000 7fdb267fc700 1 -- 172.21.15.46:0/3424644226 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20012430 con 0x7fdb280c1ec0 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3424644226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 msgr2=0x7fdb280ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3424644226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280ad6c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009e60 tx=0x7fdb20009780 comp rx=0 tx=0).stop 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3424644226 shutdown_connections 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3424644226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280ad6c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3424644226 >> 172.21.15.46:0/3424644226 conn(0x7fdb280cbe30 msgr2=0x7fdb280cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3424644226 shutdown_connections 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3424644226 wait complete. 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 Processor -- start 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.429+0000 7fdb2dacd700 1 -- start start 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 0x7fdb280b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2813c3e0 con 0x7fdb2815ab60 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280cac90 con 0x7fdb280c1ec0 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb280cb660 con 0x7fdb2806e8a0 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60376/0 (socket says 172.21.15.46:60376) 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 -- 172.21.15.46:0/3223057132 learned_addr learned my addr 172.21.15.46:0/3223057132 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb26ffd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb277fe700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 0x7fdb280b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 -- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 msgr2=0x7fdb280b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 0x7fdb280b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 -- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 msgr2=0x7fdb280cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 -- 172.21.15.46:0/3223057132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20009a20 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb26ffd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb277fe700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 0x7fdb280b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb27fff700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fdb2806d5f0 tx=0x7fdb20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20012e80 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb200032b0 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2001b850 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb280ca080 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.433+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb280ca640 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.437+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb28063f90 con 0x7fdb2815ab60 2023-10-06T12:50:36.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.437+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fdb2001bc40 con 0x7fdb2815ab60 2023-10-06T12:50:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.441+0000 7fdb0f7fe700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 0x7fdb14062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.441+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7fdb2007eab0 con 0x7fdb2815ab60 2023-10-06T12:50:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.441+0000 7fdb277fe700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 0x7fdb14062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.441+0000 7fdb277fe700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 0x7fdb14062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb2813f770 tx=0x7fdb18009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:36.453 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:50:36.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.449+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdb2004edb0 con 0x7fdb2815ab60 2023-10-06T12:50:36.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.677+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdb2804b5c0 con 0x7fdb2815ab60 2023-10-06T12:50:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.677+0000 7fdb0f7fe700 1 -- 172.21.15.46:0/3223057132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fdb200523d0 con 0x7fdb2815ab60 2023-10-06T12:50:36.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:36.704 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:50:36.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:50:36.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:50:36.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.711 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.723 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.724 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.725 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:36.726 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T12:50:36.727 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:50:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 msgr2=0x7fdb14062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 0x7fdb14062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb2813f770 tx=0x7fdb18009040 comp rx=0 tx=0).stop 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 msgr2=0x7fdb280c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fdb2806d5f0 tx=0x7fdb20019000 comp rx=0 tx=0).stop 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 shutdown_connections 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2815ab60 0x7fdb280c9850 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb14060670 0x7fdb14062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb280c1ec0 0x7fdb280cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 --2- 172.21.15.46:0/3223057132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2806e8a0 0x7fdb280b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 >> 172.21.15.46:0/3223057132 conn(0x7fdb280cbe30 msgr2=0x7fdb280a9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.681+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 shutdown_connections 2023-10-06T12:50:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.685+0000 7fdb2dacd700 1 -- 172.21.15.46:0/3223057132 wait complete. 2023-10-06T12:50:36.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.901+0000 7fa6fd123700 1 Processor -- start 2023-10-06T12:50:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.901+0000 7fa6fd123700 1 -- start start 2023-10-06T12:50:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fd123700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6f8067760 0x7fa6f80c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fd123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8069660 con 0x7fa6f80bedd0 2023-10-06T12:50:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fd123700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f8068ac0 con 0x7fa6f80bf1a0 2023-10-06T12:50:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fd123700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f8068d30 con 0x7fa6f8067760 2023-10-06T12:50:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f7fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa6f80bedd0 0x7fa6f8067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58736/0 (socket says 172.21.15.46:58736) 2023-10-06T12:50:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f7fff700 1 -- 172.21.15.46:0/2332118319 learned_addr learned my addr 172.21.15.46:0/2332118319 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6ef7fe700 1 --2- 172.21.15.46:0/2332118319 >> v2:172.21.15.132:3300/0 conn(0x7fa6f8067760 0x7fa6f80c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205353914 0 0) 0x7fa6f8069660 con 0x7fa6f80bedd0 2023-10-06T12:50:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4003570 con 0x7fa6f80bedd0 2023-10-06T12:50:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127334862 0 0) 0x7fa6f8068ac0 con 0x7fa6f80bf1a0 2023-10-06T12:50:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8069660 con 0x7fa6f80bf1a0 2023-10-06T12:50:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2749178038 0 0) 0x7fa6e4003570 con 0x7fa6f80bedd0 2023-10-06T12:50:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 >> v2:172.21.15.132:3300/0 conn(0x7fa6f8067760 msgr2=0x7fa6f80c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 --2- 172.21.15.46:0/2332118319 >> v2:172.21.15.132:3300/0 conn(0x7fa6f8067760 0x7fa6f80c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 >> v1:172.21.15.132:6789/0 conn(0x7fa6f80bf1a0 legacy=0x7fa6f8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8069010 con 0x7fa6f80bedd0 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fa6e80017f0 con 0x7fa6f80bedd0 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 >> v1:172.21.15.46:6789/0 conn(0x7fa6f80bedd0 legacy=0x7fa6f8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6e4003bf0 0x7fa6e4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8068d30 con 0x7fa6e40051c0 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400c0b0 con 0x7fa6e4003bf0 2023-10-06T12:50:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400c270 con 0x7fa6e4004810 2023-10-06T12:50:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6f7fff700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6e4003bf0 0x7fa6e4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6ef7fe700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 -- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e4004810 msgr2=0x7fa6e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 -- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6e4003bf0 msgr2=0x7fa6e4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6e4003bf0 0x7fa6e4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6fc922700 1 -- 172.21.15.46:0/2332118319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4003570 con 0x7fa6e40051c0 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.905+0000 7fa6ef7fe700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fc922700 1 -- 172.21.15.46:0/2332118319 reap_dead start 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fc922700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00019e0 tx=0x7fa6f0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d94323123b63a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000c5d0 con 0x7fa6e40051c0 2023-10-06T12:50:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000c730 con 0x7fa6e40051c0 2023-10-06T12:50:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6f77fe700 1 -- 172.21.15.46:0/2332118319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000cab0 con 0x7fa6e40051c0 2023-10-06T12:50:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 -- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 msgr2=0x7fa6e400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00019e0 tx=0x7fa6f0007d30 comp rx=0 tx=0).stop 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 -- 172.21.15.46:0/2332118319 shutdown_connections 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 --2- 172.21.15.46:0/2332118319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 -- 172.21.15.46:0/2332118319 >> 172.21.15.46:0/2332118319 conn(0x7fa6f8062ef0 msgr2=0x7fa6f80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 -- 172.21.15.46:0/2332118319 shutdown_connections 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 -- 172.21.15.46:0/2332118319 wait complete. 2023-10-06T12:50:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.909+0000 7fa6fd123700 1 Processor -- start 2023-10-06T12:50:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 -- start start 2023-10-06T12:50:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e40051c0 0x7fa6f80bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f80bc6b0 0x7fa6f815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815ee80 con 0x7fa6f815a600 2023-10-06T12:50:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815f160 con 0x7fa6f80bc6b0 2023-10-06T12:50:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fd123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815f440 con 0x7fa6e40051c0 2023-10-06T12:50:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60394/0 (socket says 172.21.15.46:60394) 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 -- 172.21.15.46:0/1102251255 learned_addr learned my addr 172.21.15.46:0/1102251255 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6ef7fe700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f80bc6b0 0x7fa6f815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6f7fff700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e40051c0 0x7fa6f80bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 -- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e40051c0 msgr2=0x7fa6f80bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e40051c0 0x7fa6f80bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 -- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f80bc6b0 msgr2=0x7fa6f815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f80bc6b0 0x7fa6f815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 -- 172.21.15.46:0/1102251255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f815f720 con 0x7fa6f815a600 2023-10-06T12:50:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6fc922700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa6f806e060 tx=0x7fa6f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.913+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000c5d0 con 0x7fa6f815a600 2023-10-06T12:50:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f0001360 con 0x7fa6f815a600 2023-10-06T12:50:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f815fa00 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0006a70 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f8160460 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8064590 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.917+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa6f0007030 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.921+0000 7fa6f4ff9700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 0x7fa6d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.921+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7fa6f007d0e0 con 0x7fa6f815a600 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.921+0000 7fa6f7fff700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 0x7fa6d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.921+0000 7fa6f7fff700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 0x7fa6d8062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000fc0 tx=0x7fa6e8002190 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:36.925+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa6f004d660 con 0x7fa6f815a600 2023-10-06T12:50:37.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.093+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6f80c1d60 con 0x7fa6f815a600 2023-10-06T12:50:37.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.097+0000 7fa6f4ff9700 1 -- 172.21.15.46:0/1102251255 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+14493 (secure 0 0 0) 0x7fa6f0022090 con 0x7fa6f815a600 2023-10-06T12:50:37.169 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:37.169 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":179,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:36.237888+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:37.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.097+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 msgr2=0x7fa6d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.097+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 0x7fa6d8062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8000fc0 tx=0x7fa6e8002190 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.097+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 msgr2=0x7fa6f816dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.097+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa6f806e060 tx=0x7fa6f0015000 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 shutdown_connections 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6f815a600 0x7fa6f816dd00 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa6d8060670 0x7fa6d8062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f80bc6b0 0x7fa6f815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 --2- 172.21.15.46:0/1102251255 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6e40051c0 0x7fa6f80bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 >> 172.21.15.46:0/1102251255 conn(0x7fa6f8062ef0 msgr2=0x7fa6f815ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 shutdown_connections 2023-10-06T12:50:37.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.101+0000 7fa6fd123700 1 -- 172.21.15.46:0/1102251255 wait complete. 2023-10-06T12:50:37.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T12:50:37.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:50:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 Processor -- start 2023-10-06T12:50:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 -- start start 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f731c146eb0 0x7f731c0c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c0c2350 0x7f731c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f731c161fc0 0x7f731c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0c1980 con 0x7f731c146eb0 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0c0d70 con 0x7f731c161fc0 2023-10-06T12:50:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f7322a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0c1050 con 0x7f731c0c2350 2023-10-06T12:50:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f731bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f731c146eb0 0x7f731c0c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f731bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f731c146eb0 0x7f731c0c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60400/0 (socket says 172.21.15.46:60400) 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.337+0000 7f731bfff700 1 -- 172.21.15.46:0/1598653678 learned_addr learned my addr 172.21.15.46:0/1598653678 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7320fcb700 1 --2- 172.21.15.46:0/1598653678 >> v2:172.21.15.132:3300/0 conn(0x7f731c161fc0 0x7f731c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f73137fe700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c0c2350 0x7f731c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 -- 172.21.15.46:0/1598653678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c0c2350 msgr2=0x7f731c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c0c2350 0x7f731c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 -- 172.21.15.46:0/1598653678 >> v2:172.21.15.132:3300/0 conn(0x7f731c161fc0 msgr2=0x7f731c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 --2- 172.21.15.46:0/1598653678 >> v2:172.21.15.132:3300/0 conn(0x7f731c161fc0 0x7f731c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 -- 172.21.15.46:0/1598653678 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0c1330 con 0x7f731c146eb0 2023-10-06T12:50:37.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731bfff700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c146eb0 0x7f731c0c1e10 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f731c06ca30 tx=0x7f730c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd69b047d5c5a2e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731b7fe700 1 -- 172.21.15.46:0/1598653678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c009e70 con 0x7f731c146eb0 2023-10-06T12:50:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731b7fe700 1 -- 172.21.15.46:0/1598653678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c00b860 con 0x7f731c146eb0 2023-10-06T12:50:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f731b7fe700 1 -- 172.21.15.46:0/1598653678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c00bc00 con 0x7f731c146eb0 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 -- 172.21.15.46:0/1598653678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c146eb0 msgr2=0x7f731c0c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c146eb0 0x7f731c0c1e10 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f731c06ca30 tx=0x7f730c009250 comp rx=0 tx=0).stop 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 -- 172.21.15.46:0/1598653678 shutdown_connections 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 --2- 172.21.15.46:0/1598653678 >> v2:172.21.15.132:3300/0 conn(0x7f731c161fc0 0x7f731c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c0c2350 0x7f731c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 --2- 172.21.15.46:0/1598653678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c146eb0 0x7f731c0c1e10 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 -- 172.21.15.46:0/1598653678 >> 172.21.15.46:0/1598653678 conn(0x7f731c0cbe30 msgr2=0x7f731c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 -- 172.21.15.46:0/1598653678 shutdown_connections 2023-10-06T12:50:37.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.341+0000 7f7322a2e700 1 -- 172.21.15.46:0/1598653678 wait complete. 2023-10-06T12:50:37.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 Processor -- start 2023-10-06T12:50:37.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 -- start start 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f731c0c2350 0x7f731c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 0x7f731c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0c5c10 con 0x7f731c161fc0 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0c50f0 con 0x7f731c0c2350 2023-10-06T12:50:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7322a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0c44e0 con 0x7f731c146eb0 2023-10-06T12:50:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60414/0 (socket says 172.21.15.46:60414) 2023-10-06T12:50:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 -- 172.21.15.46:0/283052000 learned_addr learned my addr 172.21.15.46:0/283052000 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f731bfff700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f731c0c2350 0x7f731c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f73137fe700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 0x7f731c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 -- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 msgr2=0x7f731c15c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 0x7f731c15c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 -- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f731c0c2350 msgr2=0x7f731c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f731c0c2350 0x7f731c0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 -- 172.21.15.46:0/283052000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0094f0 con 0x7f731c161fc0 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f73137fe700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 0x7f731c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.345+0000 7f7320fcb700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7314009df0 tx=0x7f731400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400cb50 con 0x7f731c161fc0 2023-10-06T12:50:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731400f040 con 0x7f731c161fc0 2023-10-06T12:50:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c0c47c0 con 0x7f731c161fc0 2023-10-06T12:50:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314014480 con 0x7f731c161fc0 2023-10-06T12:50:37.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c0c4d80 con 0x7f731c161fc0 2023-10-06T12:50:37.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c06ab00 con 0x7f731c161fc0 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.349+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7314014830 con 0x7f731c161fc0 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.353+0000 7f7318ff9700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 0x7f72fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.353+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7f731407d6d0 con 0x7f731c161fc0 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.353+0000 7f731bfff700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 0x7f72fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.353+0000 7f731bfff700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 0x7f72fc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f731c06ca30 tx=0x7f730c017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.357+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f731c06ab00 con 0x7f731c161fc0 2023-10-06T12:50:37.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f731c06ab00 con 0x7f731c161fc0 2023-10-06T12:50:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7318ff9700 1 -- 172.21.15.46:0/283052000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7f731c06ab00 con 0x7f731c161fc0 2023-10-06T12:50:37.587 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:50:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 msgr2=0x7f72fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 0x7f72fc062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f731c06ca30 tx=0x7f730c017000 comp rx=0 tx=0).stop 2023-10-06T12:50:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 msgr2=0x7f731c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.529+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f7314009df0 tx=0x7f731400b040 comp rx=0 tx=0).stop 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 shutdown_connections 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f731c161fc0 0x7f731c15f120 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f731c146eb0 0x7f731c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f72fc060670 0x7f72fc062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 --2- 172.21.15.46:0/283052000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f731c0c2350 0x7f731c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 >> 172.21.15.46:0/283052000 conn(0x7f731c0cbe30 msgr2=0x7f731c144890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 shutdown_connections 2023-10-06T12:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.533+0000 7f7322a2e700 1 -- 172.21.15.46:0/283052000 wait complete. 2023-10-06T12:50:37.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:50:37.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.753+0000 7f13ffa6e700 1 Processor -- start 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.753+0000 7f13ffa6e700 1 -- start start 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.753+0000 7f13ffa6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8146eb0 0x7f13f80c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13ffa6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f80c2350 0x7f13f806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13ffa6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f80c1980 con 0x7f13f8161fc0 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13ffa6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f80c0d70 con 0x7f13f80c2350 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13ffa6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80c1050 con 0x7f13f8146eb0 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f80c2350 0x7f13f806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f80c2350 0x7f13f806cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60430/0 (socket says 172.21.15.46:60430) 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 -- 172.21.15.46:0/3399357666 learned_addr learned my addr 172.21.15.46:0/3399357666 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd80a700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8146eb0 0x7f13f80c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fe00b700 1 --1- 172.21.15.46:0/3399357666 >> v1:172.21.15.132:6789/0 conn(0x7f13f8161fc0 0x7f13f8161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:49188/0 (socket says 172.21.15.46:49188) 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 -- 172.21.15.46:0/3399357666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8146eb0 msgr2=0x7f13f80c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8146eb0 0x7f13f80c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 -- 172.21.15.46:0/3399357666 >> v1:172.21.15.132:6789/0 conn(0x7f13f8161fc0 legacy=0x7f13f8161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 -- 172.21.15.46:0/3399357666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80c1330 con 0x7f13f80c2350 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fd009700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f806cc30 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f13e80099b0 tx=0x7f13e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326c68188c9b579a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fc808700 1 -- 172.21.15.46:0/3399357666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8009ea0 con 0x7f13f80c2350 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fc808700 1 -- 172.21.15.46:0/3399357666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e800b860 con 0x7f13f80c2350 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.757+0000 7f13fc808700 1 -- 172.21.15.46:0/3399357666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800bc00 con 0x7f13f80c2350 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3399357666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 msgr2=0x7f13f806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f806cc30 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f13e80099b0 tx=0x7f13e8009250 comp rx=0 tx=0).stop 2023-10-06T12:50:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3399357666 shutdown_connections 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f806cc30 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3399357666 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8146eb0 0x7f13f80c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3399357666 >> 172.21.15.46:0/3399357666 conn(0x7f13f80cbe30 msgr2=0x7f13f80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3399357666 shutdown_connections 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3399357666 wait complete. 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.761+0000 7f13ffa6e700 1 Processor -- start 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 -- start start 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13f8146eb0 0x7f13f815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8161fc0 0x7f13f80cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80c3af0 con 0x7f13f80c2350 2023-10-06T12:50:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80c2ee0 con 0x7f13f8146eb0 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80c31c0 con 0x7f13f8161fc0 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60442/0 (socket says 172.21.15.46:60442) 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 -- 172.21.15.46:0/3021127858 learned_addr learned my addr 172.21.15.46:0/3021127858 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd009700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13f8146eb0 0x7f13f815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fe00b700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8161fc0 0x7f13f80cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 -- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8161fc0 msgr2=0x7f13f80cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8161fc0 0x7f13f80cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 -- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13f8146eb0 msgr2=0x7f13f815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13f8146eb0 0x7f13f815c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 -- 172.21.15.46:0/3021127858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80094f0 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13fd80a700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f13f400ec80 tx=0x7f13f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400c850 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f400c9b0 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.765+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f80c34a0 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.769+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f400cd50 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.769+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f815ed30 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.769+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f80660f0 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.769+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f13f4013680 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.773+0000 7f13edffb700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 0x7f13dc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.773+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7f13f407cff0 con 0x7f13f80c2350 2023-10-06T12:50:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.773+0000 7f13fd009700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 0x7f13dc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.773+0000 7f13fd009700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 0x7f13dc062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13e80099b0 tx=0x7f13e8017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.777+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f13f404d400 con 0x7f13f80c2350 2023-10-06T12:50:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.945+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13f80c1610 con 0x7f13f80c2350 2023-10-06T12:50:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.945+0000 7f13edffb700 1 -- 172.21.15.46:0/3021127858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v179) v1 ==== 92+0+10 (secure 0 0 0) 0x7f13f40170a0 con 0x7f13f80c2350 2023-10-06T12:50:37.952 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 msgr2=0x7f13dc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 0x7f13dc062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13e80099b0 tx=0x7f13e8017000 comp rx=0 tx=0).stop 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 msgr2=0x7f13f815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f13f400ec80 tx=0x7f13f40091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 shutdown_connections 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13f8161fc0 0x7f13f80cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f13dc060670 0x7f13dc062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13f8146eb0 0x7f13f815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 --2- 172.21.15.46:0/3021127858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f80c2350 0x7f13f815e7f0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 >> 172.21.15.46:0/3021127858 conn(0x7f13f80cbe30 msgr2=0x7f13f8068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 shutdown_connections 2023-10-06T12:50:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:37.949+0000 7f13ffa6e700 1 -- 172.21.15.46:0/3021127858 wait complete. 2023-10-06T12:50:37.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T12:50:38.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.125+0000 7fe250c4c700 1 Processor -- start 2023-10-06T12:50:38.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.125+0000 7fe250c4c700 1 -- start start 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe24c156e20 0x7fe24c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c06d790 con 0x7fe24c067760 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c0acd60 con 0x7fe24c0aaf30 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0ad040 con 0x7fe24c156e20 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe24c156e20 0x7fe24c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe24c156e20 0x7fe24c06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60452/0 (socket says 172.21.15.46:60452) 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 -- 172.21.15.46:0/2368509613 learned_addr learned my addr 172.21.15.46:0/2368509613 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24bfff700 1 --1- 172.21.15.46:0/2368509613 >> v1:172.21.15.46:6789/0 conn(0x7fe24c067760 0x7fe24c068a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58744/0 (socket says 172.21.15.46:58744) 2023-10-06T12:50:38.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 -- 172.21.15.46:0/2368509613 >> v1:172.21.15.132:6789/0 conn(0x7fe24c0aaf30 legacy=0x7fe24c067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 -- 172.21.15.46:0/2368509613 >> v1:172.21.15.46:6789/0 conn(0x7fe24c067760 legacy=0x7fe24c068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 -- 172.21.15.46:0/2368509613 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0ad320 con 0x7fe24c156e20 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24b7fe700 1 --2- 172.21.15.46:0/2368509613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c156e20 0x7fe24c06d250 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0099b0 tx=0x7fe23c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d9ecadeb5468 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24affd700 1 -- 172.21.15.46:0/2368509613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00b820 con 0x7fe24c156e20 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24affd700 1 -- 172.21.15.46:0/2368509613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00b980 con 0x7fe24c156e20 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe24affd700 1 -- 172.21.15.46:0/2368509613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00bd20 con 0x7fe24c156e20 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- 172.21.15.46:0/2368509613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c156e20 msgr2=0x7fe24c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 --2- 172.21.15.46:0/2368509613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c156e20 0x7fe24c06d250 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0099b0 tx=0x7fe23c009250 comp rx=0 tx=0).stop 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- 172.21.15.46:0/2368509613 shutdown_connections 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 --2- 172.21.15.46:0/2368509613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c156e20 0x7fe24c06d250 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- 172.21.15.46:0/2368509613 >> 172.21.15.46:0/2368509613 conn(0x7fe24c062ef0 msgr2=0x7fe24c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:38.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.129+0000 7fe250c4c700 1 -- 172.21.15.46:0/2368509613 shutdown_connections 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 -- 172.21.15.46:0/2368509613 wait complete. 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 Processor -- start 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 -- start start 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24c067760 0x7fe24c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe24c156e20 0x7fe24c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0af0d0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0c3100 con 0x7fe24c156e20 2023-10-06T12:50:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe250c4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0c56d0 con 0x7fe24c067760 2023-10-06T12:50:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.133+0000 7fe242ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60462/0 (socket says 172.21.15.46:60462) 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 -- 172.21.15.46:0/4216930331 learned_addr learned my addr 172.21.15.46:0/4216930331 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe24b7fe700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24c067760 0x7fe24c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe24bfff700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe24c156e20 0x7fe24c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 -- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24c067760 msgr2=0x7fe24c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24c067760 0x7fe24c0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 -- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe24c156e20 msgr2=0x7fe24c0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe24c156e20 0x7fe24c0b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 -- 172.21.15.46:0/4216930331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0094f0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe242ffd700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c5d0 tx=0x7fe234007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234011590 con 0x7fe24c0aaf30 2023-10-06T12:50:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234011730 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234011af0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c0c59b0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c0aecf0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.137+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c06ad30 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.141+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe234006820 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.141+0000 7fe243fff700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 0x7fe22c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.141+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7fe23407cf40 con 0x7fe24c0aaf30 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.141+0000 7fe24b7fe700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 0x7fe22c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.145+0000 7fe24b7fe700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 0x7fe22c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0099b0 tx=0x7fe23c016000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.145+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe23404d3d0 con 0x7fe24c0aaf30 2023-10-06T12:50:38.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe24c06ad30 con 0x7fe24c0aaf30 2023-10-06T12:50:38.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe243fff700 1 -- 172.21.15.46:0/4216930331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v179) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe24c06ad30 con 0x7fe24c0aaf30 2023-10-06T12:50:38.329 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:38.330 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 msgr2=0x7fe22c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 0x7fe22c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0099b0 tx=0x7fe23c016000 comp rx=0 tx=0).stop 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 msgr2=0x7fe24c0c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c5d0 tx=0x7fe234007f60 comp rx=0 tx=0).stop 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 shutdown_connections 2023-10-06T12:50:38.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe22c060670 0x7fe22c062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe24c156e20 0x7fe24c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe24c0aaf30 0x7fe24c0c2bc0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.321+0000 7fe250c4c700 1 --2- 172.21.15.46:0/4216930331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe24c067760 0x7fe24c0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.325+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 >> 172.21.15.46:0/4216930331 conn(0x7fe24c062ef0 msgr2=0x7fe24c156150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.325+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 shutdown_connections 2023-10-06T12:50:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.325+0000 7fe250c4c700 1 -- 172.21.15.46:0/4216930331 wait complete. 2023-10-06T12:50:38.349 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T12:50:38.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T12:50:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.585+0000 7fa905a4c700 1 Processor -- start 2023-10-06T12:50:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.585+0000 7fa905a4c700 1 -- start start 2023-10-06T12:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa905a4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9001537f0 0x7fa90006ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa905a4c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa900153bc0 0x7fa900156360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa905a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9000c6150 con 0x7fa900067050 2023-10-06T12:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa905a4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9000c6430 con 0x7fa9001537f0 2023-10-06T12:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa905a4c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa900156c00 con 0x7fa900153bc0 2023-10-06T12:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9001537f0 0x7fa90006ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa90524b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa900067050 0x7fa9001599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58758/0 (socket says 172.21.15.46:58758) 2023-10-06T12:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa90524b700 1 -- 172.21.15.46:0/2621557535 learned_addr learned my addr 172.21.15.46:0/2621557535 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8fffff700 1 --2- 172.21.15.46:0/2621557535 >> v2:172.21.15.132:3300/0 conn(0x7fa900153bc0 0x7fa900156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8ff7fe700 1 -- 172.21.15.46:0/2621557535 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321834279 0 0) 0x7fa9000c6150 con 0x7fa900067050 2023-10-06T12:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8ff7fe700 1 -- 172.21.15.46:0/2621557535 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f0003570 con 0x7fa900067050 2023-10-06T12:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 -- 172.21.15.46:0/2621557535 >> v2:172.21.15.132:3300/0 conn(0x7fa900153bc0 msgr2=0x7fa900156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 --2- 172.21.15.46:0/2621557535 >> v2:172.21.15.132:3300/0 conn(0x7fa900153bc0 0x7fa900156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 -- 172.21.15.46:0/2621557535 >> v1:172.21.15.46:6789/0 conn(0x7fa900067050 legacy=0x7fa9001599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 -- 172.21.15.46:0/2621557535 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900156ee0 con 0x7fa9001537f0 2023-10-06T12:50:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa904a4a700 1 --2- 172.21.15.46:0/2621557535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9001537f0 0x7fa90006ceb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccc20 tx=0x7fa8f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eed7771f4c544ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8ff7fe700 1 -- 172.21.15.46:0/2621557535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4011630 con 0x7fa9001537f0 2023-10-06T12:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8ff7fe700 1 -- 172.21.15.46:0/2621557535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f40117b0 con 0x7fa9001537f0 2023-10-06T12:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.589+0000 7fa8ff7fe700 1 -- 172.21.15.46:0/2621557535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4011b70 con 0x7fa9001537f0 2023-10-06T12:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- 172.21.15.46:0/2621557535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9001537f0 msgr2=0x7fa90006ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- 172.21.15.46:0/2621557535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9001537f0 0x7fa90006ceb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccc20 tx=0x7fa8f40091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- 172.21.15.46:0/2621557535 shutdown_connections 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- 172.21.15.46:0/2621557535 >> v2:172.21.15.132:3300/0 conn(0x7fa900153bc0 0x7fa900156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- 172.21.15.46:0/2621557535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9001537f0 0x7fa90006ceb0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- 172.21.15.46:0/2621557535 >> 172.21.15.46:0/2621557535 conn(0x7fa9000cbbb0 msgr2=0x7fa9000cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- 172.21.15.46:0/2621557535 shutdown_connections 2023-10-06T12:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- 172.21.15.46:0/2621557535 wait complete. 2023-10-06T12:50:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 Processor -- start 2023-10-06T12:50:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- start start 2023-10-06T12:50:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa900067050 0x7fa9000bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 0x7fa9000c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000c5270 con 0x7fa900153bc0 2023-10-06T12:50:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000bfcf0 con 0x7fa9001537f0 2023-10-06T12:50:38.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.593+0000 7fa905a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000bffd0 con 0x7fa900067050 2023-10-06T12:50:38.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa90524b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa90524b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60468/0 (socket says 172.21.15.46:60468) 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa90524b700 1 -- 172.21.15.46:0/3537974076 learned_addr learned my addr 172.21.15.46:0/3537974076 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 0x7fa9000c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa904a4a700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa900067050 0x7fa9000bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 -- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa900067050 msgr2=0x7fa9000bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa900067050 0x7fa9000bf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 -- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 msgr2=0x7fa900152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 -- 172.21.15.46:0/3537974076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f4009420 con 0x7fa9001537f0 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa90524b700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:38.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fffff700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 0x7fa9000c4d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e900 tx=0x7fa8ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec013680 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0137e0 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9000c02b0 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec013b80 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.597+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9000af100 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.601+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9000af320 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.601+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa8ec005a00 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.601+0000 7fa8fcff9700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 0x7fa8e4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.605+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5750+0+0 (secure 0 0 0) 0x7fa8ec07cd70 con 0x7fa9001537f0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.605+0000 7fa904a4a700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 0x7fa8e4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.605+0000 7fa904a4a700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 0x7fa8e4062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccc20 tx=0x7fa8f4017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:38.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.609+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa8ec04d180 con 0x7fa9001537f0 2023-10-06T12:50:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:38.781+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa900064590 con 0x7fa9001537f0 2023-10-06T12:50:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.257+0000 7fa8fcff9700 1 -- 172.21.15.46:0/3537974076 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v180) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa8ec0507a0 con 0x7fa9001537f0 2023-10-06T12:50:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 msgr2=0x7fa8e4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 0x7fa8e4062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa9000ccc20 tx=0x7fa8f4017000 comp rx=0 tx=0).stop 2023-10-06T12:50:39.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 msgr2=0x7fa9000c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 0x7fa9000c4d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e900 tx=0x7fa8ec0091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 shutdown_connections 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa900153bc0 0x7fa900152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa8e40606a0 0x7fa8e4062b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9001537f0 0x7fa9000c4d30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 --2- 172.21.15.46:0/3537974076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa900067050 0x7fa9000bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.265+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 >> 172.21.15.46:0/3537974076 conn(0x7fa9000cbbb0 msgr2=0x7fa9000aab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:39.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.269+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 shutdown_connections 2023-10-06T12:50:39.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.269+0000 7fa905a4c700 1 -- 172.21.15.46:0/3537974076 wait complete. 2023-10-06T12:50:39.278 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T12:50:39.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T12:50:39.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.541+0000 7f4b576f0700 1 Processor -- start 2023-10-06T12:50:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.541+0000 7f4b576f0700 1 -- start start 2023-10-06T12:50:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b576f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b50068660 0x7f4b500b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b576f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b500ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b576f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b50159d70 con 0x7f4b50068290 2023-10-06T12:50:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b576f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500af930 con 0x7f4b50068660 2023-10-06T12:50:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b576f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500ad810 con 0x7f4b500af4b0 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b5548c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b50068290 0x7f4b500adaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58774/0 (socket says 172.21.15.46:58774) 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b5548c700 1 -- 172.21.15.46:0/3787338139 learned_addr learned my addr 172.21.15.46:0/3787338139 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b54c8b700 1 --2- 172.21.15.46:0/3787338139 >> v2:172.21.15.132:3300/0 conn(0x7f4b50068660 0x7f4b500b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770532892 0 0) 0x7f4b50159d70 con 0x7f4b50068290 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b500ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b3c003570 con 0x7f4b50068290 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939188827 0 0) 0x7f4b3c003570 con 0x7f4b50068290 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 msgr2=0x7f4b500ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b500ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 >> v2:172.21.15.132:3300/0 conn(0x7f4b50068660 msgr2=0x7f4b500b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 --2- 172.21.15.46:0/3787338139 >> v2:172.21.15.132:3300/0 conn(0x7f4b50068660 0x7f4b500b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500afb40 con 0x7f4b50068290 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f4b4c001470 con 0x7f4b50068290 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 >> v1:172.21.15.46:6789/0 conn(0x7f4b50068290 legacy=0x7f4b500adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b3c003a60 0x7f4b3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500af930 con 0x7f4b3c004d50 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500ad810 con 0x7f4b3c003a60 2023-10-06T12:50:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.545+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c00bae0 con 0x7f4b500af4b0 2023-10-06T12:50:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b5548c700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b3c003a60 0x7f4b3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b54c8b700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 -- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 msgr2=0x7f4b3c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b500af4b0 0x7f4b3c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 -- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b3c003a60 msgr2=0x7f4b3c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b3c003a60 0x7f4b3c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 -- 172.21.15.46:0/3787338139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c003570 con 0x7f4b3c004d50 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b3c0051c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007440 tx=0x7f4b48009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af13c3e938f99d20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:39.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48009a40 con 0x7f4b3c004d50 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48007cf0 con 0x7f4b3c004d50 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b47fff700 1 -- 172.21.15.46:0/3787338139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48014630 con 0x7f4b3c004d50 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b576f0700 1 -- 172.21.15.46:0/3787338139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 msgr2=0x7f4b3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b576f0700 1 --2- 172.21.15.46:0/3787338139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b3c0051c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4b48007440 tx=0x7f4b48009710 comp rx=0 tx=0).stop 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b55c8d700 1 -- 172.21.15.46:0/3787338139 reap_dead start 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b576f0700 1 -- 172.21.15.46:0/3787338139 shutdown_connections 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.549+0000 7f4b576f0700 1 -- 172.21.15.46:0/3787338139 >> 172.21.15.46:0/3787338139 conn(0x7f4b5006d490 msgr2=0x7f4b5006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.553+0000 7f4b576f0700 1 -- 172.21.15.46:0/3787338139 shutdown_connections 2023-10-06T12:50:39.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.553+0000 7f4b576f0700 1 -- 172.21.15.46:0/3787338139 wait complete. 2023-10-06T12:50:39.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.553+0000 7f4b576f0700 1 Processor -- start 2023-10-06T12:50:39.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.553+0000 7f4b576f0700 1 -- start start 2023-10-06T12:50:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b50150bf0 0x7f4b5014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b5015d5f0 0x7f4b50151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5014b010 con 0x7f4b3c004d50 2023-10-06T12:50:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5015e000 con 0x7f4b5015d5f0 2023-10-06T12:50:39.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5015c830 con 0x7f4b50150bf0 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60476/0 (socket says 172.21.15.46:60476) 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 -- 172.21.15.46:0/1055263334 learned_addr learned my addr 172.21.15.46:0/1055263334 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b55c8d700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b5015d5f0 0x7f4b50151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b54c8b700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b50150bf0 0x7f4b5014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 -- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b50150bf0 msgr2=0x7f4b5014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b50150bf0 0x7f4b5014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 -- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b5015d5f0 msgr2=0x7f4b50151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b5015d5f0 0x7f4b50151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 -- 172.21.15.46:0/1055263334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5015cb10 con 0x7f4b3c004d50 2023-10-06T12:50:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b5548c700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0096e0 tx=0x7f4b4c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c009e10 con 0x7f4b3c004d50 2023-10-06T12:50:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00f7f0 con 0x7f4b3c004d50 2023-10-06T12:50:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5015cdf0 con 0x7f4b3c004d50 2023-10-06T12:50:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00fbb0 con 0x7f4b3c004d50 2023-10-06T12:50:39.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.557+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50154830 con 0x7f4b3c004d50 2023-10-06T12:50:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.561+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5006c110 con 0x7f4b3c004d50 2023-10-06T12:50:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.561+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4b4c016430 con 0x7f4b3c004d50 2023-10-06T12:50:39.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.565+0000 7f4b457fa700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 0x7f4b34062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.565+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4b4c07cc40 con 0x7f4b3c004d50 2023-10-06T12:50:39.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.565+0000 7f4b54c8b700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 0x7f4b34062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:39.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.565+0000 7f4b54c8b700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 0x7f4b34062b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b5015c520 tx=0x7f4b400091a0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:39.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.573+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b4c04cee0 con 0x7f4b3c004d50 2023-10-06T12:50:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:39.745+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b50159440 con 0x7f4b3c004d50 2023-10-06T12:50:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.269+0000 7f4b457fa700 1 -- 172.21.15.46:0/1055263334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v181) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b4c050500 con 0x7f4b3c004d50 2023-10-06T12:50:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 msgr2=0x7f4b34062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 0x7f4b34062b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b5015c520 tx=0x7f4b400091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 msgr2=0x7f4b50154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:40.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0096e0 tx=0x7f4b4c001e90 comp rx=0 tx=0).stop 2023-10-06T12:50:40.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 shutdown_connections 2023-10-06T12:50:40.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b340606a0 0x7f4b34062b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b5015d5f0 0x7f4b50151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b50150bf0 0x7f4b5014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 --2- 172.21.15.46:0/1055263334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3c004d50 0x7f4b50154130 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.289+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 >> 172.21.15.46:0/1055263334 conn(0x7f4b5006d490 msgr2=0x7f4b500afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:40.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.293+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 shutdown_connections 2023-10-06T12:50:40.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.293+0000 7f4b576f0700 1 -- 172.21.15.46:0/1055263334 wait complete. 2023-10-06T12:50:40.306 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T12:50:40.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T12:50:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.485+0000 7f9bbf28e700 1 Processor -- start 2023-10-06T12:50:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.485+0000 7f9bbf28e700 1 -- start start 2023-10-06T12:50:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 0x7f9bb80d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bb80d0d60 0x7f9bb80b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb80afd20 con 0x7f9bb80b3630 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb8159740 con 0x7f9bb80d0d60 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80b0000 con 0x7f9bb80adbe0 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bb80d0d60 0x7f9bb80b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bb80d0d60 0x7f9bb80b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60490/0 (socket says 172.21.15.46:60490) 2023-10-06T12:50:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 -- 172.21.15.46:0/1880231248 learned_addr learned my addr 172.21.15.46:0/1880231248 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbe28c700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 0x7f9bb80d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 -- 172.21.15.46:0/1880231248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 msgr2=0x7f9bb80d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 0x7f9bb80d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 -- 172.21.15.46:0/1880231248 >> v1:172.21.15.132:6789/0 conn(0x7f9bb80b3630 legacy=0x7f9bb80b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 -- 172.21.15.46:0/1880231248 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80b02e0 con 0x7f9bb80d0d60 2023-10-06T12:50:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbe28c700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 0x7f9bb80d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:40.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80d0d60 0x7f9bb80b30f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80099b0 tx=0x7f9ba8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419b606d7942f066 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:40.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbd28a700 1 -- 172.21.15.46:0/1880231248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800b820 con 0x7f9bb80d0d60 2023-10-06T12:50:40.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbd28a700 1 -- 172.21.15.46:0/1880231248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba800b980 con 0x7f9bb80d0d60 2023-10-06T12:50:40.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbd28a700 1 -- 172.21.15.46:0/1880231248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800bd20 con 0x7f9bb80d0d60 2023-10-06T12:50:40.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1880231248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80d0d60 msgr2=0x7f9bb80b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:40.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80d0d60 0x7f9bb80b30f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80099b0 tx=0x7f9ba8009250 comp rx=0 tx=0).stop 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1880231248 shutdown_connections 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80d0d60 0x7f9bb80b30f0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1880231248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80adbe0 0x7f9bb80d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1880231248 >> 172.21.15.46:0/1880231248 conn(0x7f9bb806ef60 msgr2=0x7f9bb80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1880231248 shutdown_connections 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.489+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1880231248 wait complete. 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.493+0000 7f9bbf28e700 1 Processor -- start 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 -- start start 2023-10-06T12:50:40.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 0x7f9bb809ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bb80d0d60 0x7f9bb809b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb809db60 con 0x7f9bb80adbe0 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb815c600 con 0x7f9bb80d0d60 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbf28e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb815bcd0 con 0x7f9bb80b3630 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60494/0 (socket says 172.21.15.46:60494) 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 -- 172.21.15.46:0/1065993286 learned_addr learned my addr 172.21.15.46:0/1065993286 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 0x7f9bb809ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbea8d700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bb80d0d60 0x7f9bb809b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 -- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 msgr2=0x7f9bb809ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 0x7f9bb809ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 -- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bb80d0d60 msgr2=0x7f9bb809b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bb80d0d60 0x7f9bb809b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbe28c700 1 -- 172.21.15.46:0/1065993286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80094f0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.497+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 0x7f9bb809ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bbe28c700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c880 tx=0x7f9bb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400a2e0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4011600 con 0x7f9bb80adbe0 2023-10-06T12:50:40.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb815bfb0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40119a0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb815ca90 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.501+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8064590 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.505+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f9bb40223a0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.505+0000 7f9bae7fc700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 0x7f9b9c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.505+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6502+0+0 (secure 0 0 0) 0x7f9bb407d1f0 con 0x7f9bb80adbe0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.505+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 0x7f9b9c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.509+0000 7f9bbda8b700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 0x7f9b9c062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80099b0 tx=0x7f9ba8016000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:40.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.513+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9bb404d200 con 0x7f9bb80adbe0 2023-10-06T12:50:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:40.717+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9bb8159380 con 0x7f9bb80adbe0 2023-10-06T12:50:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.293+0000 7f9bae7fc700 1 -- 172.21.15.46:0/1065993286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9bb4050820 con 0x7f9bb80adbe0 2023-10-06T12:50:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 msgr2=0x7f9b9c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 0x7f9b9c062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80099b0 tx=0x7f9ba8016000 comp rx=0 tx=0).stop 2023-10-06T12:50:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 msgr2=0x7f9bb8154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c880 tx=0x7f9bb4008040 comp rx=0 tx=0).stop 2023-10-06T12:50:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 shutdown_connections 2023-10-06T12:50:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9b9c0606a0 0x7f9b9c062b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bb80d0d60 0x7f9bb809b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bb80b3630 0x7f9bb809ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 --2- 172.21.15.46:0/1065993286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb80adbe0 0x7f9bb8154cf0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 >> 172.21.15.46:0/1065993286 conn(0x7f9bb806ef60 msgr2=0x7f9bb8157690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 shutdown_connections 2023-10-06T12:50:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.297+0000 7f9bbf28e700 1 -- 172.21.15.46:0/1065993286 wait complete. 2023-10-06T12:50:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 23 and data pool 24 2023-10-06T12:50:41.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:41.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.481+0000 7f4e83825700 1 Processor -- start 2023-10-06T12:50:41.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.481+0000 7f4e83825700 1 -- start start 2023-10-06T12:50:41.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e83825700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e7c0c2a10 0x7f4e7c1605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e83825700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7c06dd30 con 0x7f4e7c155480 2023-10-06T12:50:41.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e83825700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7c0caa10 con 0x7f4e7c0c1b70 2023-10-06T12:50:41.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e83825700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7c06ce80 con 0x7f4e7c0c2a10 2023-10-06T12:50:41.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e815c1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4e7c155480 0x7f4e7c0c1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58790/0 (socket says 172.21.15.46:58790) 2023-10-06T12:50:41.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e815c1700 1 -- 172.21.15.46:0/3496438134 learned_addr learned my addr 172.21.15.46:0/3496438134 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:41.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.485+0000 7f4e81dc2700 1 --2- 172.21.15.46:0/3496438134 >> v2:172.21.15.46:3300/0 conn(0x7f4e7c0c2a10 0x7f4e7c1605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150788582 0 0) 0x7f4e7c06dd30 con 0x7f4e7c155480 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e68003570 con 0x7f4e7c155480 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854443554 0 0) 0x7f4e7c0caa10 con 0x7f4e7c0c1b70 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e7c06dd30 con 0x7f4e7c0c1b70 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1692531408 0 0) 0x7f4e68003570 con 0x7f4e7c155480 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 >> v2:172.21.15.46:3300/0 conn(0x7f4e7c0c2a10 msgr2=0x7f4e7c1605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 --2- 172.21.15.46:0/3496438134 >> v2:172.21.15.46:3300/0 conn(0x7f4e7c0c2a10 0x7f4e7c1605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 >> v1:172.21.15.132:6789/0 conn(0x7f4e7c0c1b70 legacy=0x7f4e7c0c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c06d120 con 0x7f4e7c155480 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f4e780015f0 con 0x7f4e7c155480 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 >> v1:172.21.15.46:6789/0 conn(0x7f4e7c155480 legacy=0x7f4e7c0c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68003bf0 0x7f4e68003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e680051c0 0x7f4e6800cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c06ce80 con 0x7f4e68004810 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6800d1c0 con 0x7f4e68003bf0 2023-10-06T12:50:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6800d380 con 0x7f4e680051c0 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e81dc2700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e680051c0 0x7f4e6800cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e815c1700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68003bf0 0x7f4e68003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 -- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e680051c0 msgr2=0x7f4e6800cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e680051c0 0x7f4e6800cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 -- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68003bf0 msgr2=0x7f4e68003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68003bf0 0x7f4e68003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 -- 172.21.15.46:0/3496438134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68003570 con 0x7f4e68004810 2023-10-06T12:50:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e81dc2700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e680051c0 0x7f4e6800cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e815c1700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68003bf0 0x7f4e68003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e81dc2700 1 -- 172.21.15.46:0/3496438134 reap_dead start 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e68004c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c001a10 tx=0x7f4e6c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b30ede7e5e4e7cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00c590 con 0x7f4e68004810 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00c6f0 con 0x7f4e68004810 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e73fff700 1 -- 172.21.15.46:0/3496438134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00ca90 con 0x7f4e68004810 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e83825700 1 -- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 msgr2=0x7f4e68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.489+0000 7f4e83825700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e68004c80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c001a10 tx=0x7f4e6c007d30 comp rx=0 tx=0).stop 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 -- 172.21.15.46:0/3496438134 shutdown_connections 2023-10-06T12:50:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 --2- 172.21.15.46:0/3496438134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e68004c80 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 -- 172.21.15.46:0/3496438134 >> 172.21.15.46:0/3496438134 conn(0x7f4e7c0cbe30 msgr2=0x7f4e7c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 -- 172.21.15.46:0/3496438134 shutdown_connections 2023-10-06T12:50:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 -- 172.21.15.46:0/3496438134 wait complete. 2023-10-06T12:50:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 Processor -- start 2023-10-06T12:50:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.493+0000 7f4e83825700 1 -- start start 2023-10-06T12:50:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e7c0b0d20 0x7f4e7c14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 0x7f4e7c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c150480 con 0x7f4e68004810 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c150bd0 con 0x7f4e7c0c8ad0 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c0abed0 con 0x7f4e7c0b0d20 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60524/0 (socket says 172.21.15.46:60524) 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 -- 172.21.15.46:0/3765933579 learned_addr learned my addr 172.21.15.46:0/3765933579 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e7c0b0d20 0x7f4e7c14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e81dc2700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 0x7f4e7c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 -- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e7c0b0d20 msgr2=0x7f4e7c14e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e7c0b0d20 0x7f4e7c14e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 -- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 msgr2=0x7f4e7c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 0x7f4e7c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 -- 172.21.15.46:0/3765933579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c0c9480 con 0x7f4e68004810 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e81dc2700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 0x7f4e7c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e815c1700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800d860 tx=0x7f4e78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78007300 con 0x7f4e68004810 2023-10-06T12:50:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e78007460 con 0x7f4e68004810 2023-10-06T12:50:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c0b1b10 con 0x7f4e68004810 2023-10-06T12:50:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e780057f0 con 0x7f4e68004810 2023-10-06T12:50:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.497+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c0b12f0 con 0x7f4e68004810 2023-10-06T12:50:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.501+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7c063f90 con 0x7f4e68004810 2023-10-06T12:50:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.501+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4e78005b60 con 0x7f4e68004810 2023-10-06T12:50:41.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.505+0000 7f4e717fa700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 0x7f4e60062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.505+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f4e7807cda0 con 0x7f4e68004810 2023-10-06T12:50:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.505+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 0x7f4e60062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.505+0000 7f4e80dc0700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 0x7f4e60062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c9720 tx=0x7f4e6c015000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.513+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e7804cd50 con 0x7f4e68004810 2023-10-06T12:50:41.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.681+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e7c06d400 con 0x7f4e68004810 2023-10-06T12:50:41.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e717fa700 1 -- 172.21.15.46:0/3765933579 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+17726 (secure 0 0 0) 0x7f4e78050370 con 0x7f4e68004810 2023-10-06T12:50:41.718 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:41.718 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":182,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:40.722142+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 msgr2=0x7f4e60062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 0x7f4e60062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c9720 tx=0x7f4e6c015000 comp rx=0 tx=0).stop 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 msgr2=0x7f4e7c150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800d860 tx=0x7f4e78009040 comp rx=0 tx=0).stop 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 shutdown_connections 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4e60060670 0x7f4e60062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e7c0c8ad0 0x7f4e7c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e7c0b0d20 0x7f4e7c14e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 --2- 172.21.15.46:0/3765933579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e68004810 0x7f4e7c150690 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 >> 172.21.15.46:0/3765933579 conn(0x7f4e7c0cbe30 msgr2=0x7f4e7c0c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 shutdown_connections 2023-10-06T12:50:41.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.685+0000 7f4e83825700 1 -- 172.21.15.46:0/3765933579 wait complete. 2023-10-06T12:50:41.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.877+0000 7f0d5b123700 1 Processor -- start 2023-10-06T12:50:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.877+0000 7f0d5b123700 1 -- start start 2023-10-06T12:50:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d540aaf30 0x7f0d54067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d54067590 0x7f0d54068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d540679f0 con 0x7f0d54156e20 2023-10-06T12:50:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d540acd60 con 0x7f0d54067590 2023-10-06T12:50:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540ad040 con 0x7f0d540aaf30 2023-10-06T12:50:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d54067590 0x7f0d54068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a121700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0d54156e20 0x7f0d5406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58792/0 (socket says 172.21.15.46:58792) 2023-10-06T12:50:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a121700 1 -- 172.21.15.46:0/1395663782 learned_addr learned my addr 172.21.15.46:0/1395663782 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d59920700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d540aaf30 0x7f0d54067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 -- 172.21.15.46:0/1395663782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d540aaf30 msgr2=0x7f0d54067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d540aaf30 0x7f0d54067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 -- 172.21.15.46:0/1395663782 >> v1:172.21.15.46:6789/0 conn(0x7f0d54156e20 legacy=0x7f0d5406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 -- 172.21.15.46:0/1395663782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540ad320 con 0x7f0d54067590 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5a922700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d54067590 0x7f0d54068a20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009e60 tx=0x7f0d4c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1883c05a81b8118 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5911f700 1 -- 172.21.15.46:0/1395663782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00e070 con 0x7f0d54067590 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5911f700 1 -- 172.21.15.46:0/1395663782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c00bb60 con 0x7f0d54067590 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5911f700 1 -- 172.21.15.46:0/1395663782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c012430 con 0x7f0d54067590 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 -- 172.21.15.46:0/1395663782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d54067590 msgr2=0x7f0d54068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.881+0000 7f0d5b123700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d54067590 0x7f0d54068a20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009e60 tx=0x7f0d4c009780 comp rx=0 tx=0).stop 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- 172.21.15.46:0/1395663782 shutdown_connections 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d54067590 0x7f0d54068a20 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 --2- 172.21.15.46:0/1395663782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d540aaf30 0x7f0d54067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- 172.21.15.46:0/1395663782 >> 172.21.15.46:0/1395663782 conn(0x7f0d54062ef0 msgr2=0x7f0d540cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- 172.21.15.46:0/1395663782 shutdown_connections 2023-10-06T12:50:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- 172.21.15.46:0/1395663782 wait complete. 2023-10-06T12:50:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 Processor -- start 2023-10-06T12:50:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- start start 2023-10-06T12:50:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d54067590 0x7f0d540c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d54156e20 0x7f0d5416ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540c3d80 con 0x7f0d540aaf30 2023-10-06T12:50:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540c3a50 con 0x7f0d54067590 2023-10-06T12:50:41.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.885+0000 7f0d5b123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5415b6c0 con 0x7f0d54156e20 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60542/0 (socket says 172.21.15.46:60542) 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 -- 172.21.15.46:0/2352727485 learned_addr learned my addr 172.21.15.46:0/2352727485 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d5a922700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d54156e20 0x7f0d5416ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d5a121700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d54067590 0x7f0d540c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 -- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d54156e20 msgr2=0x7f0d5416ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d54156e20 0x7f0d5416ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 -- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d54067590 msgr2=0x7f0d540c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:41.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d54067590 0x7f0d540c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 -- 172.21.15.46:0/2352727485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c009a20 con 0x7f0d540aaf30 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d59920700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400ec50 tx=0x7f0d440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400c850 con 0x7f0d540aaf30 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4400c9b0 con 0x7f0d540aaf30 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400cd70 con 0x7f0d540aaf30 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5415b9a0 con 0x7f0d540aaf30 2023-10-06T12:50:41.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5415bc80 con 0x7f0d540aaf30 2023-10-06T12:50:41.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.889+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d54064590 con 0x7f0d540aaf30 2023-10-06T12:50:41.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.893+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0d44013680 con 0x7f0d540aaf30 2023-10-06T12:50:41.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.893+0000 7f0d4a7fc700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 0x7f0d38062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:41.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.893+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f0d4407d650 con 0x7f0d540aaf30 2023-10-06T12:50:41.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.893+0000 7f0d5a121700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 0x7f0d38062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:41.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.897+0000 7f0d5a121700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 0x7f0d38062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007fd0 tx=0x7f0d50009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:41.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:41.897+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d4404d830 con 0x7f0d540aaf30 2023-10-06T12:50:42.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.069+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d540ad600 con 0x7f0d540aaf30 2023-10-06T12:50:42.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.073+0000 7f0d4a7fc700 1 -- 172.21.15.46:0/2352727485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17713 (secure 0 0 0) 0x7f0d44050e50 con 0x7f0d540aaf30 2023-10-06T12:50:42.080 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:42.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":183,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:41.751873+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 msgr2=0x7f0d38062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 0x7f0d38062b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0d50007fd0 tx=0x7f0d50009040 comp rx=0 tx=0).stop 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 msgr2=0x7f0d540b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400ec50 tx=0x7f0d440091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 shutdown_connections 2023-10-06T12:50:42.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d54156e20 0x7f0d5416ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d540aaf30 0x7f0d540b0ba0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0d38060670 0x7f0d38062b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 --2- 172.21.15.46:0/2352727485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d54067590 0x7f0d540c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 >> 172.21.15.46:0/2352727485 conn(0x7f0d54062ef0 msgr2=0x7f0d541560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 shutdown_connections 2023-10-06T12:50:42.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.077+0000 7f0d5b123700 1 -- 172.21.15.46:0/2352727485 wait complete. 2023-10-06T12:50:42.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T12:50:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.289+0000 7f837c7c5700 1 Processor -- start 2023-10-06T12:50:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.289+0000 7f837c7c5700 1 -- start start 2023-10-06T12:50:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.289+0000 7f837c7c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83740b23c0 0x7f83740b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837c7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 0x7f83740cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837c7c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83740cc510 con 0x7f83740ccb40 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837c7c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83740cb900 con 0x7f83740b23c0 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837c7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740cbbe0 con 0x7f83740cae40 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837ad62700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83740ccb40 0x7f837415cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58798/0 (socket says 172.21.15.46:58798) 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837ad62700 1 -- 172.21.15.46:0/918504851 learned_addr learned my addr 172.21.15.46:0/918504851 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837a561700 1 --2- 172.21.15.46:0/918504851 >> v2:172.21.15.132:3300/0 conn(0x7f83740b23c0 0x7f83740b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 --2- 172.21.15.46:0/918504851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 0x7f83740cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837955f700 1 -- 172.21.15.46:0/918504851 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683954027 0 0) 0x7f83740cc510 con 0x7f83740ccb40 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837955f700 1 -- 172.21.15.46:0/918504851 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8360003570 con 0x7f83740ccb40 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 -- 172.21.15.46:0/918504851 >> v2:172.21.15.132:3300/0 conn(0x7f83740b23c0 msgr2=0x7f83740b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 --2- 172.21.15.46:0/918504851 >> v2:172.21.15.132:3300/0 conn(0x7f83740b23c0 0x7f83740b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 -- 172.21.15.46:0/918504851 >> v1:172.21.15.46:6789/0 conn(0x7f83740ccb40 legacy=0x7f837415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 -- 172.21.15.46:0/918504851 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740cbec0 con 0x7f83740cae40 2023-10-06T12:50:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f8379d60700 1 --2- 172.21.15.46:0/918504851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 0x7f83740cb290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f83640098e0 tx=0x7f83640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655c3034f952be89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.293+0000 7f837955f700 1 -- 172.21.15.46:0/918504851 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011630 con 0x7f83740cae40 2023-10-06T12:50:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837955f700 1 -- 172.21.15.46:0/918504851 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364011790 con 0x7f83740cae40 2023-10-06T12:50:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837955f700 1 -- 172.21.15.46:0/918504851 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011b30 con 0x7f83740cae40 2023-10-06T12:50:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 -- 172.21.15.46:0/918504851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 msgr2=0x7f83740cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 --2- 172.21.15.46:0/918504851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 0x7f83740cb290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f83640098e0 tx=0x7f83640091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 -- 172.21.15.46:0/918504851 shutdown_connections 2023-10-06T12:50:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 --2- 172.21.15.46:0/918504851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740cae40 0x7f83740cb290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 --2- 172.21.15.46:0/918504851 >> v2:172.21.15.132:3300/0 conn(0x7f83740b23c0 0x7f83740b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 -- 172.21.15.46:0/918504851 >> 172.21.15.46:0/918504851 conn(0x7f8374099480 msgr2=0x7f837406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 -- 172.21.15.46:0/918504851 shutdown_connections 2023-10-06T12:50:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.297+0000 7f837c7c5700 1 -- 172.21.15.46:0/918504851 wait complete. 2023-10-06T12:50:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 Processor -- start 2023-10-06T12:50:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 -- start start 2023-10-06T12:50:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83740cae40 0x7f83740c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740ccb40 0x7f83740d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b3510 con 0x7f83740b23c0 2023-10-06T12:50:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b37f0 con 0x7f83740cae40 2023-10-06T12:50:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837c7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b3ad0 con 0x7f83740ccb40 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60556/0 (socket says 172.21.15.46:60556) 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 -- 172.21.15.46:0/3362739159 learned_addr learned my addr 172.21.15.46:0/3362739159 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f8379d60700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83740cae40 0x7f83740c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837ad62700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740ccb40 0x7f83740d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 -- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740ccb40 msgr2=0x7f83740d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740ccb40 0x7f83740d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 -- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83740cae40 msgr2=0x7f83740c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83740cae40 0x7f83740c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.301+0000 7f837a561700 1 -- 172.21.15.46:0/3362739159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364009420 con 0x7f83740b23c0 2023-10-06T12:50:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f837a561700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f837000ea00 tx=0x7f8370009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000c6a0 con 0x7f83740b23c0 2023-10-06T12:50:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000c800 con 0x7f83740b23c0 2023-10-06T12:50:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83740b3db0 con 0x7f83740b23c0 2023-10-06T12:50:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000cb80 con 0x7f83740b23c0 2023-10-06T12:50:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83740c9190 con 0x7f83740b23c0 2023-10-06T12:50:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.305+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83740660f0 con 0x7f83740b23c0 2023-10-06T12:50:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.309+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8370013470 con 0x7f83740b23c0 2023-10-06T12:50:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.309+0000 7f836a7fc700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 0x7f8358062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.309+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f837007d170 con 0x7f83740b23c0 2023-10-06T12:50:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.309+0000 7f8379d60700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 0x7f8358062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.313+0000 7f8379d60700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 0x7f8358062b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83640091e0 tx=0x7f8364017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.317+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f837004d2d0 con 0x7f83740b23c0 2023-10-06T12:50:42.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:42.533+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83740cc1a0 con 0x7f83740b23c0 2023-10-06T12:50:43.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.321+0000 7f836a7fc700 1 -- 172.21.15.46:0/3362739159 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83700508f0 con 0x7f83740b23c0 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.325+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 msgr2=0x7f8358062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.325+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 0x7f8358062b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83640091e0 tx=0x7f8364017000 comp rx=0 tx=0).stop 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.325+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 msgr2=0x7f8374150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.325+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f837000ea00 tx=0x7f8370009120 comp rx=0 tx=0).stop 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 shutdown_connections 2023-10-06T12:50:43.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83740ccb40 0x7f83740d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f83580606a0 0x7f8358062b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:43.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83740cae40 0x7f83740c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:43.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 --2- 172.21.15.46:0/3362739159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83740b23c0 0x7f8374150d20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:43.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 >> 172.21.15.46:0/3362739159 conn(0x7f8374099480 msgr2=0x7f83740b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:43.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 shutdown_connections 2023-10-06T12:50:43.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:43.329+0000 7f837c7c5700 1 -- 172.21.15.46:0/3362739159 wait complete. 2023-10-06T12:50:43.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T12:50:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.133+0000 7f33453cc700 1 Processor -- start 2023-10-06T12:50:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.133+0000 7f33453cc700 1 -- start start 2023-10-06T12:50:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.133+0000 7f33453cc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f33400c97e0 0x7f33400c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.133+0000 7f33453cc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400c2220 0x7f33400c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.133+0000 7f33453cc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400c1d90 con 0x7f33400c6250 2023-10-06T12:50:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400c1460 con 0x7f33400c2220 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400c1740 con 0x7f33400c97e0 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400c2220 0x7f33400c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33400c2220 0x7f33400c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47256/0 (socket says 172.21.15.46:47256) 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 -- 172.21.15.46:0/3189046526 learned_addr learned my addr 172.21.15.46:0/3189046526 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f333ffff700 1 --2- 172.21.15.46:0/3189046526 >> v2:172.21.15.132:3300/0 conn(0x7f33400c97e0 0x7f33400c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 -- 172.21.15.46:0/3189046526 >> v2:172.21.15.132:3300/0 conn(0x7f33400c97e0 msgr2=0x7f33400c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 --2- 172.21.15.46:0/3189046526 >> v2:172.21.15.132:3300/0 conn(0x7f33400c97e0 0x7f33400c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 -- 172.21.15.46:0/3189046526 >> v1:172.21.15.132:6789/0 conn(0x7f33400c6250 legacy=0x7f334006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 -- 172.21.15.46:0/3189046526 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400c1a20 con 0x7f33400c2220 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f3344bcb700 1 --2- 172.21.15.46:0/3189046526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c2220 0x7f33400c2680 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f33400cc8f0 tx=0x7f3338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de253ab4c3d10bad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f333f7fe700 1 -- 172.21.15.46:0/3189046526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338009e70 con 0x7f33400c2220 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f333f7fe700 1 -- 172.21.15.46:0/3189046526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800b860 con 0x7f33400c2220 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f333f7fe700 1 -- 172.21.15.46:0/3189046526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800bc00 con 0x7f33400c2220 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 -- 172.21.15.46:0/3189046526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c2220 msgr2=0x7f33400c2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 --2- 172.21.15.46:0/3189046526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c2220 0x7f33400c2680 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f33400cc8f0 tx=0x7f3338009250 comp rx=0 tx=0).stop 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 -- 172.21.15.46:0/3189046526 shutdown_connections 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 --2- 172.21.15.46:0/3189046526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c2220 0x7f33400c2680 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 --2- 172.21.15.46:0/3189046526 >> v2:172.21.15.132:3300/0 conn(0x7f33400c97e0 0x7f33400c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.137+0000 7f33453cc700 1 -- 172.21.15.46:0/3189046526 >> 172.21.15.46:0/3189046526 conn(0x7f33400cbc10 msgr2=0x7f33400cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- 172.21.15.46:0/3189046526 shutdown_connections 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- 172.21.15.46:0/3189046526 wait complete. 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 Processor -- start 2023-10-06T12:50:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- start start 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33400c2220 0x7f334015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33400c97e0 0x7f33400c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334015ef20 con 0x7f33400c6250 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334015f200 con 0x7f33400c2220 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.141+0000 7f33453cc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334015f4e0 con 0x7f33400c97e0 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47266/0 (socket says 172.21.15.46:47266) 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 -- 172.21.15.46:0/1515946377 learned_addr learned my addr 172.21.15.46:0/1515946377 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f3344bcb700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33400c97e0 0x7f33400c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f333ffff700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33400c2220 0x7f334015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 -- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33400c97e0 msgr2=0x7f33400c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33400c97e0 0x7f33400c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 -- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33400c2220 msgr2=0x7f334015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33400c2220 0x7f334015ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 -- 172.21.15.46:0/1515946377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33380094f0 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33377fe700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f332800c4e0 tx=0x7f332800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328011590 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328011710 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334015f7c0 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328011ad0 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.145+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334015ebd0 con 0x7f33400c6250 2023-10-06T12:50:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.149+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334015fcc0 con 0x7f33400c6250 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.149+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f332801b3b0 con 0x7f33400c6250 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.149+0000 7f333cff9700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.149+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f332807c550 con 0x7f33400c6250 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.149+0000 7f333ffff700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.153+0000 7f333ffff700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 0x7f3320062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f333000f570 tx=0x7f333000b040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.157+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f332804c690 con 0x7f33400c6250 2023-10-06T12:50:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:44.369+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3340064590 con 0x7f33400c6250 2023-10-06T12:50:45.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.349+0000 7f333cff9700 1 -- 172.21.15.46:0/1515946377 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3328017020 con 0x7f33400c6250 2023-10-06T12:50:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 msgr2=0x7f3320062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 0x7f3320062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f333000f570 tx=0x7f333000b040 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 msgr2=0x7f33400c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f332800c4e0 tx=0x7f332800c7f0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 shutdown_connections 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33400c97e0 0x7f33400c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33400c6250 0x7f33400c2cc0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3320060670 0x7f3320062b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 --2- 172.21.15.46:0/1515946377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33400c2220 0x7f334015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 >> 172.21.15.46:0/1515946377 conn(0x7f33400cbc10 msgr2=0x7f3340157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 shutdown_connections 2023-10-06T12:50:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.353+0000 7f33453cc700 1 -- 172.21.15.46:0/1515946377 wait complete. 2023-10-06T12:50:45.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.829+0000 7f24cc864700 1 Processor -- start 2023-10-06T12:50:45.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 -- start start 2023-10-06T12:50:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f24c8158dd0 0x7f24c8067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24c81591a0 0x7f24c8067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c8068bf0 0x7f24c8069070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c80699c0 con 0x7f24c81591a0 2023-10-06T12:50:45.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c815db50 con 0x7f24c8158dd0 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24cc864700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815d6c0 con 0x7f24c8068bf0 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24c81591a0 0x7f24c8067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24c81591a0 0x7f24c8067590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47280/0 (socket says 172.21.15.46:47280) 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 -- 172.21.15.46:0/3556233971 learned_addr learned my addr 172.21.15.46:0/3556233971 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24c759e700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c8068bf0 0x7f24c8069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24c6d9d700 1 --2- 172.21.15.46:0/3556233971 >> v2:172.21.15.132:3300/0 conn(0x7f24c8158dd0 0x7f24c8067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 -- 172.21.15.46:0/3556233971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c8068bf0 msgr2=0x7f24c8069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:45.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c8068bf0 0x7f24c8069070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 -- 172.21.15.46:0/3556233971 >> v2:172.21.15.132:3300/0 conn(0x7f24c8158dd0 msgr2=0x7f24c8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 --2- 172.21.15.46:0/3556233971 >> v2:172.21.15.132:3300/0 conn(0x7f24c8158dd0 0x7f24c8067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 -- 172.21.15.46:0/3556233971 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c815cd90 con 0x7f24c81591a0 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.833+0000 7f24bffff700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c81591a0 0x7f24c8067590 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f24c806ab40 tx=0x7f24b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad7185e53a8d7c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24c659c700 1 -- 172.21.15.46:0/3556233971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000b640 con 0x7f24c81591a0 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24c659c700 1 -- 172.21.15.46:0/3556233971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000b7a0 con 0x7f24c81591a0 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24c659c700 1 -- 172.21.15.46:0/3556233971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000bb40 con 0x7f24c81591a0 2023-10-06T12:50:45.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 -- 172.21.15.46:0/3556233971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c81591a0 msgr2=0x7f24c8067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c81591a0 0x7f24c8067590 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f24c806ab40 tx=0x7f24b0009250 comp rx=0 tx=0).stop 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 -- 172.21.15.46:0/3556233971 shutdown_connections 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c8068bf0 0x7f24c8069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 --2- 172.21.15.46:0/3556233971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c81591a0 0x7f24c8067590 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 --2- 172.21.15.46:0/3556233971 >> v2:172.21.15.132:3300/0 conn(0x7f24c8158dd0 0x7f24c8067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 -- 172.21.15.46:0/3556233971 >> 172.21.15.46:0/3556233971 conn(0x7f24c8062ef0 msgr2=0x7f24c80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 -- 172.21.15.46:0/3556233971 shutdown_connections 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 -- 172.21.15.46:0/3556233971 wait complete. 2023-10-06T12:50:45.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.837+0000 7f24cc864700 1 Processor -- start 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- start start 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c8158dd0 0x7f24c80c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c81591a0 0x7f24c80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8169830 con 0x7f24c8068bf0 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80c2e80 con 0x7f24c8158dd0 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80c4650 con 0x7f24c81591a0 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47292/0 (socket says 172.21.15.46:47292) 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 -- 172.21.15.46:0/936208728 learned_addr learned my addr 172.21.15.46:0/936208728 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24bffff700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c8158dd0 0x7f24c80c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c759e700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c81591a0 0x7f24c80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 -- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c81591a0 msgr2=0x7f24c80c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c81591a0 0x7f24c80c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 -- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c8158dd0 msgr2=0x7f24c80c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c8158dd0 0x7f24c80c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 -- 172.21.15.46:0/936208728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00094f0 con 0x7f24c8068bf0 2023-10-06T12:50:45.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24c6d9d700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ec80 tx=0x7f24b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800c850 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800c9b0 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c80c4930 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cd50 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.841+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c80c4ef0 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.845+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c8064590 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.845+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f24b8013680 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.849+0000 7f24beffd700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 0x7f24a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.849+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f24b807cff0 con 0x7f24c8068bf0 2023-10-06T12:50:45.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.849+0000 7f24bffff700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 0x7f24a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:45.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.849+0000 7f24bffff700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 0x7f24a8062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24c806ab40 tx=0x7f24b0017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:45.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:45.853+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f24b80acbe0 con 0x7f24c8068bf0 2023-10-06T12:50:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.057+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24c815d070 con 0x7f24c8068bf0 2023-10-06T12:50:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.057+0000 7f24beffd700 1 -- 172.21.15.46:0/936208728 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f24b804d0d0 con 0x7f24c8068bf0 2023-10-06T12:50:46.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:46.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:50:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 msgr2=0x7f24a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 0x7f24a8062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f24c806ab40 tx=0x7f24b0017000 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 msgr2=0x7f24c815a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ec80 tx=0x7f24b80091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 shutdown_connections 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c81591a0 0x7f24c80c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f24a8060670 0x7f24a8062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c8158dd0 0x7f24c80c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 --2- 172.21.15.46:0/936208728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24c8068bf0 0x7f24c815a920 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 >> 172.21.15.46:0/936208728 conn(0x7f24c8062ef0 msgr2=0x7f24c80cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 shutdown_connections 2023-10-06T12:50:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.061+0000 7f24cc864700 1 -- 172.21.15.46:0/936208728 wait complete. 2023-10-06T12:50:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:50:46.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:50:46.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.245+0000 7f3aec4cd700 1 Processor -- start 2023-10-06T12:50:46.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.245+0000 7f3aec4cd700 1 -- start start 2023-10-06T12:50:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40c5ff0 0x7f3ae40ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae40ccb50 0x7f3ae40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40cc150 con 0x7f3ae415c770 2023-10-06T12:50:46.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae40cfbb0 con 0x7f3ae40ccb50 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40cdbf0 con 0x7f3ae40c5ff0 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aea269700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ae415c770 0x7f3ae40cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38050/0 (socket says 172.21.15.46:38050) 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aea269700 1 -- 172.21.15.46:0/1755138030 learned_addr learned my addr 172.21.15.46:0/1755138030 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/1755138030 >> v2:172.21.15.46:3300/0 conn(0x7f3ae40ccb50 0x7f3ae40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9a68700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40c5ff0 0x7f3ae40ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9267700 1 -- 172.21.15.46:0/1755138030 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925363657 0 0) 0x7f3ae40cc150 con 0x7f3ae415c770 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9267700 1 -- 172.21.15.46:0/1755138030 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad0003570 con 0x7f3ae415c770 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/1755138030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40c5ff0 msgr2=0x7f3ae40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40c5ff0 0x7f3ae40ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/1755138030 >> v1:172.21.15.46:6789/0 conn(0x7f3ae415c770 legacy=0x7f3ae40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/1755138030 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40cb540 con 0x7f3ae40ccb50 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae40ccb50 0x7f3ae40cea30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009e60 tx=0x7f3adc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2758008067b2924d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9267700 1 -- 172.21.15.46:0/1755138030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00e070 con 0x7f3ae40ccb50 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9267700 1 -- 172.21.15.46:0/1755138030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc00bab0 con 0x7f3ae40ccb50 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3ae9267700 1 -- 172.21.15.46:0/1755138030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00be70 con 0x7f3ae40ccb50 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 -- 172.21.15.46:0/1755138030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae40ccb50 msgr2=0x7f3ae40cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.249+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae40ccb50 0x7f3ae40cea30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009e60 tx=0x7f3adc009780 comp rx=0 tx=0).stop 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 -- 172.21.15.46:0/1755138030 shutdown_connections 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae40ccb50 0x7f3ae40cea30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/1755138030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40c5ff0 0x7f3ae40ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 -- 172.21.15.46:0/1755138030 >> 172.21.15.46:0/1755138030 conn(0x7f3ae4099480 msgr2=0x7f3ae406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:46.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 -- 172.21.15.46:0/1755138030 shutdown_connections 2023-10-06T12:50:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 -- 172.21.15.46:0/1755138030 wait complete. 2023-10-06T12:50:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 Processor -- start 2023-10-06T12:50:46.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.253+0000 7f3aec4cd700 1 -- start start 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae40c5ff0 0x7f3ae4158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40ccb50 0x7f3ae409c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae409cd40 con 0x7f3ae415c770 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae409d010 con 0x7f3ae40c5ff0 2023-10-06T12:50:46.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae409d2f0 con 0x7f3ae40ccb50 2023-10-06T12:50:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47310/0 (socket says 172.21.15.46:47310) 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/2359768158 learned_addr learned my addr 172.21.15.46:0/2359768158 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aea269700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae40c5ff0 0x7f3ae4158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3ae9a68700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40ccb50 0x7f3ae409c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40ccb50 msgr2=0x7f3ae409c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40ccb50 0x7f3ae409c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae40c5ff0 msgr2=0x7f3ae4158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae40c5ff0 0x7f3ae4158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 -- 172.21.15.46:0/2359768158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc009a20 con 0x7f3ae415c770 2023-10-06T12:50:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aeaa6a700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4064b20 tx=0x7f3adc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc027470 con 0x7f3ae415c770 2023-10-06T12:50:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc0275f0 con 0x7f3ae415c770 2023-10-06T12:50:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0279b0 con 0x7f3ae415c770 2023-10-06T12:50:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae409d5d0 con 0x7f3ae415c770 2023-10-06T12:50:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.257+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae4158a60 con 0x7f3ae415c770 2023-10-06T12:50:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.261+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae409ddd0 con 0x7f3ae415c770 2023-10-06T12:50:46.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.261+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3adc024070 con 0x7f3ae415c770 2023-10-06T12:50:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.261+0000 7f3ada7fc700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 0x7f3ac8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.261+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f3adc07f5f0 con 0x7f3ae415c770 2023-10-06T12:50:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.261+0000 7f3aea269700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 0x7f3ac8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.265+0000 7f3aea269700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 0x7f3ac8062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001360 tx=0x7f3ae0009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.269+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3ae409ddd0 con 0x7f3ae415c770 2023-10-06T12:50:46.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.449+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ae409ddd0 con 0x7f3ae415c770 2023-10-06T12:50:46.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3ada7fc700 1 -- 172.21.15.46:0/2359768158 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17713 (secure 0 0 0) 0x7f3ae409ddd0 con 0x7f3ae415c770 2023-10-06T12:50:46.459 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:46.459 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":183,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:41.751873+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:50:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 msgr2=0x7f3ac8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 0x7f3ac8062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0001360 tx=0x7f3ae0009040 comp rx=0 tx=0).stop 2023-10-06T12:50:46.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 msgr2=0x7f3ae409c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4064b20 tx=0x7f3adc019000 comp rx=0 tx=0).stop 2023-10-06T12:50:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 shutdown_connections 2023-10-06T12:50:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae415c770 0x7f3ae409c800 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40ccb50 0x7f3ae409c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac8060670 0x7f3ac8062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 --2- 172.21.15.46:0/2359768158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae40c5ff0 0x7f3ae4158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 >> 172.21.15.46:0/2359768158 conn(0x7f3ae4099480 msgr2=0x7f3ae415a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:46.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 shutdown_connections 2023-10-06T12:50:46.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.453+0000 7f3aec4cd700 1 -- 172.21.15.46:0/2359768158 wait complete. 2023-10-06T12:50:46.483 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T12:50:46.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:50:46.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.737+0000 7f0b6c2f0700 1 Processor -- start 2023-10-06T12:50:46.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.737+0000 7f0b6c2f0700 1 -- start start 2023-10-06T12:50:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6c2f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b6415c770 0x7f0b640cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6c2f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640ccb50 0x7f0b640ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6c2f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b640cc150 con 0x7f0b640cea30 2023-10-06T12:50:46.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6c2f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b640cfbb0 con 0x7f0b6415c770 2023-10-06T12:50:46.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6c2f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640cdbf0 con 0x7f0b640ccb50 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6a88d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0b640cea30 0x7f0b640c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38064/0 (socket says 172.21.15.46:38064) 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6a88d700 1 -- 172.21.15.46:0/947705829 learned_addr learned my addr 172.21.15.46:0/947705829 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.741+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/947705829 >> v2:172.21.15.132:3300/0 conn(0x7f0b6415c770 0x7f0b640cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6988b700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640ccb50 0x7f0b640ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6908a700 1 -- 172.21.15.46:0/947705829 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113059503 0 0) 0x7f0b640cc150 con 0x7f0b640cea30 2023-10-06T12:50:46.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6908a700 1 -- 172.21.15.46:0/947705829 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b50003570 con 0x7f0b640cea30 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6a08c700 1 -- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640ccb50 msgr2=0x7f0b640ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640ccb50 0x7f0b640ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6a08c700 1 -- 172.21.15.46:0/947705829 >> v1:172.21.15.46:6789/0 conn(0x7f0b640cea30 legacy=0x7f0b640c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6a08c700 1 -- 172.21.15.46:0/947705829 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b640cee70 con 0x7f0b6415c770 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b6415c770 0x7f0b640cad40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098d0 tx=0x7f0b60009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f196949e6397db43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6908a700 1 -- 172.21.15.46:0/947705829 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60005c10 con 0x7f0b6415c770 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6908a700 1 -- 172.21.15.46:0/947705829 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60005d70 con 0x7f0b6415c770 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6908a700 1 -- 172.21.15.46:0/947705829 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60006280 con 0x7f0b6415c770 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b6415c770 msgr2=0x7f0b640cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b6415c770 0x7f0b640cad40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098d0 tx=0x7f0b60009120 comp rx=0 tx=0).stop 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/947705829 shutdown_connections 2023-10-06T12:50:46.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640ccb50 0x7f0b640ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/947705829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b6415c770 0x7f0b640cad40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/947705829 >> 172.21.15.46:0/947705829 conn(0x7f0b64099480 msgr2=0x7f0b6406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/947705829 shutdown_connections 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.745+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/947705829 wait complete. 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 Processor -- start 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 -- start start 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b640ccb50 0x7f0b64151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640cea30 0x7f0b6415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640b4290 con 0x7f0b6415c770 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640b3680 con 0x7f0b640ccb50 2023-10-06T12:50:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6c2f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640b3960 con 0x7f0b640cea30 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6a88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6a88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47314/0 (socket says 172.21.15.46:47314) 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6a88d700 1 -- 172.21.15.46:0/1624454183 learned_addr learned my addr 172.21.15.46:0/1624454183 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6988b700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640cea30 0x7f0b6415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.749+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b640ccb50 0x7f0b64151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 -- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640cea30 msgr2=0x7f0b6415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640cea30 0x7f0b6415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 -- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b640ccb50 msgr2=0x7f0b64151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b640ccb50 0x7f0b64151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 -- 172.21.15.46:0/1624454183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600093c0 con 0x7f0b6415c770 2023-10-06T12:50:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6a88d700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c001460 tx=0x7f0b5c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00a890 con 0x7f0b6415c770 2023-10-06T12:50:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c00a9f0 con 0x7f0b6415c770 2023-10-06T12:50:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00ad70 con 0x7f0b6415c770 2023-10-06T12:50:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b640b3c40 con 0x7f0b6415c770 2023-10-06T12:50:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b64152430 con 0x7f0b6415c770 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.753+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b640699f0 con 0x7f0b6415c770 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.757+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0b5c007300 con 0x7f0b6415c770 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.757+0000 7f0b5a7fc700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 0x7f0b48062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.757+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f0b5c07d0e0 con 0x7f0b6415c770 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.757+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 0x7f0b48062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.761+0000 7f0b6a08c700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 0x7f0b48062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009340 tx=0x7f0b60016000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.765+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0b640699f0 con 0x7f0b6415c770 2023-10-06T12:50:46.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.933+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b640b5e80 con 0x7f0b48060670 2023-10-06T12:50:47.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.953+0000 7f0b5a7fc700 1 -- 172.21.15.46:0/1624454183 <== mgr.9666 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308864 (secure 0 0 0) 0x7f0b640b5e80 con 0x7f0b48060670 2023-10-06T12:50:47.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:47.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 msgr2=0x7f0b48062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 0x7f0b48062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009340 tx=0x7f0b60016000 comp rx=0 tx=0).stop 2023-10-06T12:50:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 msgr2=0x7f0b641540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c001460 tx=0x7f0b5c001ae0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 shutdown_connections 2023-10-06T12:50:47.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b6415c770 0x7f0b641540e0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b640cea30 0x7f0b6415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0b48060670 0x7f0b48062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 --2- 172.21.15.46:0/1624454183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b640ccb50 0x7f0b64151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 >> 172.21.15.46:0/1624454183 conn(0x7f0b64099480 msgr2=0x7f0b64154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 shutdown_connections 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:46.961+0000 7f0b6c2f0700 1 -- 172.21.15.46:0/1624454183 wait complete. 2023-10-06T12:50:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:50:47.293 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T12:50:45.787543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2471144,"kb_used_data":19112,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752503576,"statfs":{"total":773094113280,"available":770563661824,"internally_reserved":0,"allocated":19570688,"data_stored":10900924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001871"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782893+0000","last_change":"2023-10-06T12:50:40.778335+0000","last_active":"2023-10-06T12:50:40.782893+0000","last_peered":"2023-10-06T12:50:40.782893+0000","last_clean":"2023-10-06T12:50:40.782893+0000","last_became_active":"2023-10-06T12:50:40.778062+0000","last_became_peered":"2023-10-06T12:50:40.778062+0000","last_unstale":"2023-10-06T12:50:40.782893+0000","last_undegraded":"2023-10-06T12:50:40.782893+0000","last_fullsized":"2023-10-06T12:50:40.782893+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:01.262267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786847+0000","last_change":"2023-10-06T12:50:40.782280+0000","last_active":"2023-10-06T12:50:40.786847+0000","last_peered":"2023-10-06T12:50:40.786847+0000","last_clean":"2023-10-06T12:50:40.786847+0000","last_became_active":"2023-10-06T12:50:40.781975+0000","last_became_peered":"2023-10-06T12:50:40.781975+0000","last_unstale":"2023-10-06T12:50:40.786847+0000","last_undegraded":"2023-10-06T12:50:40.786847+0000","last_fullsized":"2023-10-06T12:50:40.786847+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787332+0000","last_change":"2023-10-06T12:50:40.784263+0000","last_active":"2023-10-06T12:50:40.787332+0000","last_peered":"2023-10-06T12:50:40.787332+0000","last_clean":"2023-10-06T12:50:40.787332+0000","last_became_active":"2023-10-06T12:50:40.783981+0000","last_became_peered":"2023-10-06T12:50:40.783981+0000","last_unstale":"2023-10-06T12:50:40.787332+0000","last_undegraded":"2023-10-06T12:50:40.787332+0000","last_fullsized":"2023-10-06T12:50:40.787332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:12:08.047276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780877+0000","last_change":"2023-10-06T12:50:40.776723+0000","last_active":"2023-10-06T12:50:40.780877+0000","last_peered":"2023-10-06T12:50:40.780877+0000","last_clean":"2023-10-06T12:50:40.780877+0000","last_became_active":"2023-10-06T12:50:40.776436+0000","last_became_peered":"2023-10-06T12:50:40.776436+0000","last_unstale":"2023-10-06T12:50:40.780877+0000","last_undegraded":"2023-10-06T12:50:40.780877+0000","last_fullsized":"2023-10-06T12:50:40.780877+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:10.144527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778358+0000","last_change":"2023-10-06T12:50:40.772646+0000","last_active":"2023-10-06T12:50:40.778358+0000","last_peered":"2023-10-06T12:50:40.778358+0000","last_clean":"2023-10-06T12:50:40.778358+0000","last_became_active":"2023-10-06T12:50:40.772150+0000","last_became_peered":"2023-10-06T12:50:40.772150+0000","last_unstale":"2023-10-06T12:50:40.778358+0000","last_undegraded":"2023-10-06T12:50:40.778358+0000","last_fullsized":"2023-10-06T12:50:40.778358+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:46.651857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783771+0000","last_change":"2023-10-06T12:50:40.775511+0000","last_active":"2023-10-06T12:50:40.783771+0000","last_peered":"2023-10-06T12:50:40.783771+0000","last_clean":"2023-10-06T12:50:40.783771+0000","last_became_active":"2023-10-06T12:50:40.775200+0000","last_became_peered":"2023-10-06T12:50:40.775200+0000","last_unstale":"2023-10-06T12:50:40.783771+0000","last_undegraded":"2023-10-06T12:50:40.783771+0000","last_fullsized":"2023-10-06T12:50:40.783771+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:21.013742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788174+0000","last_change":"2023-10-06T12:50:40.784220+0000","last_active":"2023-10-06T12:50:40.788174+0000","last_peered":"2023-10-06T12:50:40.788174+0000","last_clean":"2023-10-06T12:50:40.788174+0000","last_became_active":"2023-10-06T12:50:40.783922+0000","last_became_peered":"2023-10-06T12:50:40.783922+0000","last_unstale":"2023-10-06T12:50:40.788174+0000","last_undegraded":"2023-10-06T12:50:40.788174+0000","last_fullsized":"2023-10-06T12:50:40.788174+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:04.673251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787085+0000","last_change":"2023-10-06T12:50:40.777423+0000","last_active":"2023-10-06T12:50:40.787085+0000","last_peered":"2023-10-06T12:50:40.787085+0000","last_clean":"2023-10-06T12:50:40.787085+0000","last_became_active":"2023-10-06T12:50:40.777128+0000","last_became_peered":"2023-10-06T12:50:40.777128+0000","last_unstale":"2023-10-06T12:50:40.787085+0000","last_undegraded":"2023-10-06T12:50:40.787085+0000","last_fullsized":"2023-10-06T12:50:40.787085+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:51.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787707+0000","last_change":"2023-10-06T12:50:40.783209+0000","last_active":"2023-10-06T12:50:40.787707+0000","last_peered":"2023-10-06T12:50:40.787707+0000","last_clean":"2023-10-06T12:50:40.787707+0000","last_became_active":"2023-10-06T12:50:40.782919+0000","last_became_peered":"2023-10-06T12:50:40.782919+0000","last_unstale":"2023-10-06T12:50:40.787707+0000","last_undegraded":"2023-10-06T12:50:40.787707+0000","last_fullsized":"2023-10-06T12:50:40.787707+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:55.492708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787127+0000","last_change":"2023-10-06T12:50:40.777737+0000","last_active":"2023-10-06T12:50:40.787127+0000","last_peered":"2023-10-06T12:50:40.787127+0000","last_clean":"2023-10-06T12:50:40.787127+0000","last_became_active":"2023-10-06T12:50:40.777216+0000","last_became_peered":"2023-10-06T12:50:40.777216+0000","last_unstale":"2023-10-06T12:50:40.787127+0000","last_undegraded":"2023-10-06T12:50:40.787127+0000","last_fullsized":"2023-10-06T12:50:40.787127+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780082+0000","last_change":"2023-10-06T12:50:40.776224+0000","last_active":"2023-10-06T12:50:40.780082+0000","last_peered":"2023-10-06T12:50:40.780082+0000","last_clean":"2023-10-06T12:50:40.780082+0000","last_became_active":"2023-10-06T12:50:40.775933+0000","last_became_peered":"2023-10-06T12:50:40.775933+0000","last_unstale":"2023-10-06T12:50:40.780082+0000","last_undegraded":"2023-10-06T12:50:40.780082+0000","last_fullsized":"2023-10-06T12:50:40.780082+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:03.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784227+0000","last_change":"2023-10-06T12:50:40.775841+0000","last_active":"2023-10-06T12:50:40.784227+0000","last_peered":"2023-10-06T12:50:40.784227+0000","last_clean":"2023-10-06T12:50:40.784227+0000","last_became_active":"2023-10-06T12:50:40.775186+0000","last_became_peered":"2023-10-06T12:50:40.775186+0000","last_unstale":"2023-10-06T12:50:40.784227+0000","last_undegraded":"2023-10-06T12:50:40.784227+0000","last_fullsized":"2023-10-06T12:50:40.784227+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:54.589070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.777761+0000","last_change":"2023-10-06T12:50:40.773781+0000","last_active":"2023-10-06T12:50:40.777761+0000","last_peered":"2023-10-06T12:50:40.777761+0000","last_clean":"2023-10-06T12:50:40.777761+0000","last_became_active":"2023-10-06T12:50:40.773449+0000","last_became_peered":"2023-10-06T12:50:40.773449+0000","last_unstale":"2023-10-06T12:50:40.777761+0000","last_undegraded":"2023-10-06T12:50:40.777761+0000","last_fullsized":"2023-10-06T12:50:40.777761+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:56.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788121+0000","last_change":"2023-10-06T12:50:40.782510+0000","last_active":"2023-10-06T12:50:40.788121+0000","last_peered":"2023-10-06T12:50:40.788121+0000","last_clean":"2023-10-06T12:50:40.788121+0000","last_became_active":"2023-10-06T12:50:40.782167+0000","last_became_peered":"2023-10-06T12:50:40.782167+0000","last_unstale":"2023-10-06T12:50:40.788121+0000","last_undegraded":"2023-10-06T12:50:40.788121+0000","last_fullsized":"2023-10-06T12:50:40.788121+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:23.655289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780305+0000","last_change":"2023-10-06T12:50:40.778146+0000","last_active":"2023-10-06T12:50:40.780305+0000","last_peered":"2023-10-06T12:50:40.780305+0000","last_clean":"2023-10-06T12:50:40.780305+0000","last_became_active":"2023-10-06T12:50:40.777837+0000","last_became_peered":"2023-10-06T12:50:40.777837+0000","last_unstale":"2023-10-06T12:50:40.780305+0000","last_undegraded":"2023-10-06T12:50:40.780305+0000","last_fullsized":"2023-10-06T12:50:40.780305+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:51:17.261560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781264+0000","last_change":"2023-10-06T12:50:40.776761+0000","last_active":"2023-10-06T12:50:40.781264+0000","last_peered":"2023-10-06T12:50:40.781264+0000","last_clean":"2023-10-06T12:50:40.781264+0000","last_became_active":"2023-10-06T12:50:40.776458+0000","last_became_peered":"2023-10-06T12:50:40.776458+0000","last_unstale":"2023-10-06T12:50:40.781264+0000","last_undegraded":"2023-10-06T12:50:40.781264+0000","last_fullsized":"2023-10-06T12:50:40.781264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:56.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787157+0000","last_change":"2023-10-06T12:50:40.777985+0000","last_active":"2023-10-06T12:50:40.787157+0000","last_peered":"2023-10-06T12:50:40.787157+0000","last_clean":"2023-10-06T12:50:40.787157+0000","last_became_active":"2023-10-06T12:50:40.777628+0000","last_became_peered":"2023-10-06T12:50:40.777628+0000","last_unstale":"2023-10-06T12:50:40.787157+0000","last_undegraded":"2023-10-06T12:50:40.787157+0000","last_fullsized":"2023-10-06T12:50:40.787157+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:22:13.682136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783869+0000","last_change":"2023-10-06T12:50:40.779890+0000","last_active":"2023-10-06T12:50:40.783869+0000","last_peered":"2023-10-06T12:50:40.783869+0000","last_clean":"2023-10-06T12:50:40.783869+0000","last_became_active":"2023-10-06T12:50:40.777022+0000","last_became_peered":"2023-10-06T12:50:40.777022+0000","last_unstale":"2023-10-06T12:50:40.783869+0000","last_undegraded":"2023-10-06T12:50:40.783869+0000","last_fullsized":"2023-10-06T12:50:40.783869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:16.774125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784780+0000","last_change":"2023-10-06T12:50:40.778707+0000","last_active":"2023-10-06T12:50:40.784780+0000","last_peered":"2023-10-06T12:50:40.784780+0000","last_clean":"2023-10-06T12:50:40.784780+0000","last_became_active":"2023-10-06T12:50:40.778236+0000","last_became_peered":"2023-10-06T12:50:40.778236+0000","last_unstale":"2023-10-06T12:50:40.784780+0000","last_undegraded":"2023-10-06T12:50:40.784780+0000","last_fullsized":"2023-10-06T12:50:40.784780+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:19:33.126754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782866+0000","last_change":"2023-10-06T12:50:40.780014+0000","last_active":"2023-10-06T12:50:40.782866+0000","last_peered":"2023-10-06T12:50:40.782866+0000","last_clean":"2023-10-06T12:50:40.782866+0000","last_became_active":"2023-10-06T12:50:40.779620+0000","last_became_peered":"2023-10-06T12:50:40.779620+0000","last_unstale":"2023-10-06T12:50:40.782866+0000","last_undegraded":"2023-10-06T12:50:40.782866+0000","last_fullsized":"2023-10-06T12:50:40.782866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:23.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787173+0000","last_change":"2023-10-06T12:50:40.778720+0000","last_active":"2023-10-06T12:50:40.787173+0000","last_peered":"2023-10-06T12:50:40.787173+0000","last_clean":"2023-10-06T12:50:40.787173+0000","last_became_active":"2023-10-06T12:50:40.778411+0000","last_became_peered":"2023-10-06T12:50:40.778411+0000","last_unstale":"2023-10-06T12:50:40.787173+0000","last_undegraded":"2023-10-06T12:50:40.787173+0000","last_fullsized":"2023-10-06T12:50:40.787173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:54.960186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786788+0000","last_change":"2023-10-06T12:50:40.781834+0000","last_active":"2023-10-06T12:50:40.786788+0000","last_peered":"2023-10-06T12:50:40.786788+0000","last_clean":"2023-10-06T12:50:40.786788+0000","last_became_active":"2023-10-06T12:50:40.781535+0000","last_became_peered":"2023-10-06T12:50:40.781535+0000","last_unstale":"2023-10-06T12:50:40.786788+0000","last_undegraded":"2023-10-06T12:50:40.786788+0000","last_fullsized":"2023-10-06T12:50:40.786788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:17.097688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.779397+0000","last_change":"2023-10-06T12:50:40.777085+0000","last_active":"2023-10-06T12:50:40.779397+0000","last_peered":"2023-10-06T12:50:40.779397+0000","last_clean":"2023-10-06T12:50:40.779397+0000","last_became_active":"2023-10-06T12:50:40.776810+0000","last_became_peered":"2023-10-06T12:50:40.776810+0000","last_unstale":"2023-10-06T12:50:40.779397+0000","last_undegraded":"2023-10-06T12:50:40.779397+0000","last_fullsized":"2023-10-06T12:50:40.779397+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:43:54.696710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781245+0000","last_change":"2023-10-06T12:50:40.774098+0000","last_active":"2023-10-06T12:50:40.781245+0000","last_peered":"2023-10-06T12:50:40.781245+0000","last_clean":"2023-10-06T12:50:40.781245+0000","last_became_active":"2023-10-06T12:50:40.773818+0000","last_became_peered":"2023-10-06T12:50:40.773818+0000","last_unstale":"2023-10-06T12:50:40.781245+0000","last_undegraded":"2023-10-06T12:50:40.781245+0000","last_fullsized":"2023-10-06T12:50:40.781245+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:34.296097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782123+0000","last_change":"2023-10-06T12:50:40.776905+0000","last_active":"2023-10-06T12:50:40.782123+0000","last_peered":"2023-10-06T12:50:40.782123+0000","last_clean":"2023-10-06T12:50:40.782123+0000","last_became_active":"2023-10-06T12:50:40.776601+0000","last_became_peered":"2023-10-06T12:50:40.776601+0000","last_unstale":"2023-10-06T12:50:40.782123+0000","last_undegraded":"2023-10-06T12:50:40.782123+0000","last_fullsized":"2023-10-06T12:50:40.782123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:34.296097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787208+0000","last_change":"2023-10-06T12:50:40.780478+0000","last_active":"2023-10-06T12:50:40.787208+0000","last_peered":"2023-10-06T12:50:40.787208+0000","last_clean":"2023-10-06T12:50:40.787208+0000","last_became_active":"2023-10-06T12:50:40.779240+0000","last_became_peered":"2023-10-06T12:50:40.779240+0000","last_unstale":"2023-10-06T12:50:40.787208+0000","last_undegraded":"2023-10-06T12:50:40.787208+0000","last_fullsized":"2023-10-06T12:50:40.787208+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:04.761816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786967+0000","last_change":"2023-10-06T12:50:40.781962+0000","last_active":"2023-10-06T12:50:40.786967+0000","last_peered":"2023-10-06T12:50:40.786967+0000","last_clean":"2023-10-06T12:50:40.786967+0000","last_became_active":"2023-10-06T12:50:40.781663+0000","last_became_peered":"2023-10-06T12:50:40.781663+0000","last_unstale":"2023-10-06T12:50:40.786967+0000","last_undegraded":"2023-10-06T12:50:40.786967+0000","last_fullsized":"2023-10-06T12:50:40.786967+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:51.651199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788289+0000","last_change":"2023-10-06T12:50:40.780354+0000","last_active":"2023-10-06T12:50:40.788289+0000","last_peered":"2023-10-06T12:50:40.788289+0000","last_clean":"2023-10-06T12:50:40.788289+0000","last_became_active":"2023-10-06T12:50:40.779798+0000","last_became_peered":"2023-10-06T12:50:40.779798+0000","last_unstale":"2023-10-06T12:50:40.788289+0000","last_undegraded":"2023-10-06T12:50:40.788289+0000","last_fullsized":"2023-10-06T12:50:40.788289+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:10.144527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786647+0000","last_change":"2023-10-06T12:50:40.779059+0000","last_active":"2023-10-06T12:50:40.786647+0000","last_peered":"2023-10-06T12:50:40.786647+0000","last_clean":"2023-10-06T12:50:40.786647+0000","last_became_active":"2023-10-06T12:50:40.778788+0000","last_became_peered":"2023-10-06T12:50:40.778788+0000","last_unstale":"2023-10-06T12:50:40.786647+0000","last_undegraded":"2023-10-06T12:50:40.786647+0000","last_fullsized":"2023-10-06T12:50:40.786647+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:29:18.444148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783512+0000","last_change":"2023-10-06T12:50:40.781633+0000","last_active":"2023-10-06T12:50:40.783512+0000","last_peered":"2023-10-06T12:50:40.783512+0000","last_clean":"2023-10-06T12:50:40.783512+0000","last_became_active":"2023-10-06T12:50:40.781346+0000","last_became_peered":"2023-10-06T12:50:40.781346+0000","last_unstale":"2023-10-06T12:50:40.783512+0000","last_undegraded":"2023-10-06T12:50:40.783512+0000","last_fullsized":"2023-10-06T12:50:40.783512+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:41.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784789+0000","last_change":"2023-10-06T12:50:40.778927+0000","last_active":"2023-10-06T12:50:40.784789+0000","last_peered":"2023-10-06T12:50:40.784789+0000","last_clean":"2023-10-06T12:50:40.784789+0000","last_became_active":"2023-10-06T12:50:40.778552+0000","last_became_peered":"2023-10-06T12:50:40.778552+0000","last_unstale":"2023-10-06T12:50:40.784789+0000","last_undegraded":"2023-10-06T12:50:40.784789+0000","last_fullsized":"2023-10-06T12:50:40.784789+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:22.757377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788111+0000","last_change":"2023-10-06T12:50:40.784137+0000","last_active":"2023-10-06T12:50:40.788111+0000","last_peered":"2023-10-06T12:50:40.788111+0000","last_clean":"2023-10-06T12:50:40.788111+0000","last_became_active":"2023-10-06T12:50:40.783849+0000","last_became_peered":"2023-10-06T12:50:40.783849+0000","last_unstale":"2023-10-06T12:50:40.788111+0000","last_undegraded":"2023-10-06T12:50:40.788111+0000","last_fullsized":"2023-10-06T12:50:40.788111+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:05.558031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786940+0000","last_change":"2023-10-06T12:50:40.780586+0000","last_active":"2023-10-06T12:50:40.786940+0000","last_peered":"2023-10-06T12:50:40.786940+0000","last_clean":"2023-10-06T12:50:40.786940+0000","last_became_active":"2023-10-06T12:50:40.776048+0000","last_became_peered":"2023-10-06T12:50:40.776048+0000","last_unstale":"2023-10-06T12:50:40.786940+0000","last_undegraded":"2023-10-06T12:50:40.786940+0000","last_fullsized":"2023-10-06T12:50:40.786940+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:04.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"182'4","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.316091+0000","last_change":"2023-10-06T12:50:40.287857+0000","last_active":"2023-10-06T12:50:41.316091+0000","last_peered":"2023-10-06T12:50:41.316091+0000","last_clean":"2023-10-06T12:50:41.316091+0000","last_became_active":"2023-10-06T12:50:40.287578+0000","last_became_peered":"2023-10-06T12:50:40.287578+0000","last_unstale":"2023-10-06T12:50:41.316091+0000","last_undegraded":"2023-10-06T12:50:41.316091+0000","last_fullsized":"2023-10-06T12:50:41.316091+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.662922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780840+0000","last_change":"2023-10-06T12:50:40.777280+0000","last_active":"2023-10-06T12:50:40.780840+0000","last_peered":"2023-10-06T12:50:40.780840+0000","last_clean":"2023-10-06T12:50:40.780840+0000","last_became_active":"2023-10-06T12:50:40.777010+0000","last_became_peered":"2023-10-06T12:50:40.777010+0000","last_unstale":"2023-10-06T12:50:40.780840+0000","last_undegraded":"2023-10-06T12:50:40.780840+0000","last_fullsized":"2023-10-06T12:50:40.780840+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:17.816213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315724+0000","last_change":"2023-10-06T12:50:40.298134+0000","last_active":"2023-10-06T12:50:41.315724+0000","last_peered":"2023-10-06T12:50:41.315724+0000","last_clean":"2023-10-06T12:50:41.315724+0000","last_became_active":"2023-10-06T12:50:40.297844+0000","last_became_peered":"2023-10-06T12:50:40.297844+0000","last_unstale":"2023-10-06T12:50:41.315724+0000","last_undegraded":"2023-10-06T12:50:41.315724+0000","last_fullsized":"2023-10-06T12:50:41.315724+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:10.127709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778588+0000","last_change":"2023-10-06T12:50:40.776226+0000","last_active":"2023-10-06T12:50:40.778588+0000","last_peered":"2023-10-06T12:50:40.778588+0000","last_clean":"2023-10-06T12:50:40.778588+0000","last_became_active":"2023-10-06T12:50:40.775922+0000","last_became_peered":"2023-10-06T12:50:40.775922+0000","last_unstale":"2023-10-06T12:50:40.778588+0000","last_undegraded":"2023-10-06T12:50:40.778588+0000","last_fullsized":"2023-10-06T12:50:40.778588+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:23.681463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.310625+0000","last_change":"2023-10-06T12:50:40.297474+0000","last_active":"2023-10-06T12:50:41.310625+0000","last_peered":"2023-10-06T12:50:41.310625+0000","last_clean":"2023-10-06T12:50:41.310625+0000","last_became_active":"2023-10-06T12:50:40.297156+0000","last_became_peered":"2023-10-06T12:50:40.297156+0000","last_unstale":"2023-10-06T12:50:41.310625+0000","last_undegraded":"2023-10-06T12:50:41.310625+0000","last_fullsized":"2023-10-06T12:50:41.310625+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:36.044092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784188+0000","last_change":"2023-10-06T12:50:40.775470+0000","last_active":"2023-10-06T12:50:40.784188+0000","last_peered":"2023-10-06T12:50:40.784188+0000","last_clean":"2023-10-06T12:50:40.784188+0000","last_became_active":"2023-10-06T12:50:40.775120+0000","last_became_peered":"2023-10-06T12:50:40.775120+0000","last_unstale":"2023-10-06T12:50:40.784188+0000","last_undegraded":"2023-10-06T12:50:40.784188+0000","last_fullsized":"2023-10-06T12:50:40.784188+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:59:13.625755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.309389+0000","last_change":"2023-10-06T12:50:40.295385+0000","last_active":"2023-10-06T12:50:41.309389+0000","last_peered":"2023-10-06T12:50:41.309389+0000","last_clean":"2023-10-06T12:50:41.309389+0000","last_became_active":"2023-10-06T12:50:40.295057+0000","last_became_peered":"2023-10-06T12:50:40.295057+0000","last_unstale":"2023-10-06T12:50:41.309389+0000","last_undegraded":"2023-10-06T12:50:41.309389+0000","last_fullsized":"2023-10-06T12:50:41.309389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:02.099185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787738+0000","last_change":"2023-10-06T12:50:40.783444+0000","last_active":"2023-10-06T12:50:40.787738+0000","last_peered":"2023-10-06T12:50:40.787738+0000","last_clean":"2023-10-06T12:50:40.787738+0000","last_became_active":"2023-10-06T12:50:40.780663+0000","last_became_peered":"2023-10-06T12:50:40.780663+0000","last_unstale":"2023-10-06T12:50:40.787738+0000","last_undegraded":"2023-10-06T12:50:40.787738+0000","last_fullsized":"2023-10-06T12:50:40.787738+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:57.388724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315299+0000","last_change":"2023-10-06T12:50:40.299371+0000","last_active":"2023-10-06T12:50:41.315299+0000","last_peered":"2023-10-06T12:50:41.315299+0000","last_clean":"2023-10-06T12:50:41.315299+0000","last_became_active":"2023-10-06T12:50:40.299059+0000","last_became_peered":"2023-10-06T12:50:40.299059+0000","last_unstale":"2023-10-06T12:50:41.315299+0000","last_undegraded":"2023-10-06T12:50:41.315299+0000","last_fullsized":"2023-10-06T12:50:41.315299+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:13.496718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781463+0000","last_change":"2023-10-06T12:50:40.780056+0000","last_active":"2023-10-06T12:50:40.781463+0000","last_peered":"2023-10-06T12:50:40.781463+0000","last_clean":"2023-10-06T12:50:40.781463+0000","last_became_active":"2023-10-06T12:50:40.779712+0000","last_became_peered":"2023-10-06T12:50:40.779712+0000","last_unstale":"2023-10-06T12:50:40.781463+0000","last_undegraded":"2023-10-06T12:50:40.781463+0000","last_fullsized":"2023-10-06T12:50:40.781463+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314973+0000","last_change":"2023-10-06T12:50:40.292623+0000","last_active":"2023-10-06T12:50:41.314973+0000","last_peered":"2023-10-06T12:50:41.314973+0000","last_clean":"2023-10-06T12:50:41.314973+0000","last_became_active":"2023-10-06T12:50:40.291297+0000","last_became_peered":"2023-10-06T12:50:40.291297+0000","last_unstale":"2023-10-06T12:50:41.314973+0000","last_undegraded":"2023-10-06T12:50:41.314973+0000","last_fullsized":"2023-10-06T12:50:41.314973+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:56.537480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780587+0000","last_change":"2023-10-06T12:50:40.777001+0000","last_active":"2023-10-06T12:50:40.780587+0000","last_peered":"2023-10-06T12:50:40.780587+0000","last_clean":"2023-10-06T12:50:40.780587+0000","last_became_active":"2023-10-06T12:50:40.776681+0000","last_became_peered":"2023-10-06T12:50:40.776681+0000","last_unstale":"2023-10-06T12:50:40.780587+0000","last_undegraded":"2023-10-06T12:50:40.780587+0000","last_fullsized":"2023-10-06T12:50:40.780587+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:04.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766443+0000","last_change":"2023-10-06T12:50:40.299038+0000","last_active":"2023-10-06T12:50:40.766443+0000","last_peered":"2023-10-06T12:50:40.766443+0000","last_clean":"2023-10-06T12:50:40.766443+0000","last_became_active":"2023-10-06T12:50:40.298763+0000","last_became_peered":"2023-10-06T12:50:40.298763+0000","last_unstale":"2023-10-06T12:50:40.766443+0000","last_undegraded":"2023-10-06T12:50:40.766443+0000","last_fullsized":"2023-10-06T12:50:40.766443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:23.394579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.773231+0000","last_change":"2023-10-06T12:50:40.767553+0000","last_active":"2023-10-06T12:50:40.773231+0000","last_peered":"2023-10-06T12:50:40.773231+0000","last_clean":"2023-10-06T12:50:40.773231+0000","last_became_active":"2023-10-06T12:50:40.767227+0000","last_became_peered":"2023-10-06T12:50:40.767227+0000","last_unstale":"2023-10-06T12:50:40.773231+0000","last_undegraded":"2023-10-06T12:50:40.773231+0000","last_fullsized":"2023-10-06T12:50:40.773231+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:17.816213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770164+0000","last_change":"2023-10-06T12:50:40.296743+0000","last_active":"2023-10-06T12:50:40.770164+0000","last_peered":"2023-10-06T12:50:40.770164+0000","last_clean":"2023-10-06T12:50:40.770164+0000","last_became_active":"2023-10-06T12:50:40.296234+0000","last_became_peered":"2023-10-06T12:50:40.296234+0000","last_unstale":"2023-10-06T12:50:40.770164+0000","last_undegraded":"2023-10-06T12:50:40.770164+0000","last_fullsized":"2023-10-06T12:50:40.770164+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:13:58.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788211+0000","last_change":"2023-10-06T12:50:40.785098+0000","last_active":"2023-10-06T12:50:40.788211+0000","last_peered":"2023-10-06T12:50:40.788211+0000","last_clean":"2023-10-06T12:50:40.788211+0000","last_became_active":"2023-10-06T12:50:40.784020+0000","last_became_peered":"2023-10-06T12:50:40.784020+0000","last_unstale":"2023-10-06T12:50:40.788211+0000","last_undegraded":"2023-10-06T12:50:40.788211+0000","last_fullsized":"2023-10-06T12:50:40.788211+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:25:07.224298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.313890+0000","last_change":"2023-10-06T12:50:40.301698+0000","last_active":"2023-10-06T12:50:41.313890+0000","last_peered":"2023-10-06T12:50:41.313890+0000","last_clean":"2023-10-06T12:50:41.313890+0000","last_became_active":"2023-10-06T12:50:40.301185+0000","last_became_peered":"2023-10-06T12:50:40.301185+0000","last_unstale":"2023-10-06T12:50:41.313890+0000","last_undegraded":"2023-10-06T12:50:41.313890+0000","last_fullsized":"2023-10-06T12:50:41.313890+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:22.508131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786818+0000","last_change":"2023-10-06T12:50:40.782490+0000","last_active":"2023-10-06T12:50:40.786818+0000","last_peered":"2023-10-06T12:50:40.786818+0000","last_clean":"2023-10-06T12:50:40.786818+0000","last_became_active":"2023-10-06T12:50:40.782194+0000","last_became_peered":"2023-10-06T12:50:40.782194+0000","last_unstale":"2023-10-06T12:50:40.786818+0000","last_undegraded":"2023-10-06T12:50:40.786818+0000","last_fullsized":"2023-10-06T12:50:40.786818+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:31.559045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315371+0000","last_change":"2023-10-06T12:50:40.299353+0000","last_active":"2023-10-06T12:50:41.315371+0000","last_peered":"2023-10-06T12:50:41.315371+0000","last_clean":"2023-10-06T12:50:41.315371+0000","last_became_active":"2023-10-06T12:50:40.298708+0000","last_became_peered":"2023-10-06T12:50:40.298708+0000","last_unstale":"2023-10-06T12:50:41.315371+0000","last_undegraded":"2023-10-06T12:50:41.315371+0000","last_fullsized":"2023-10-06T12:50:41.315371+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:56.515951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.772550+0000","last_change":"2023-10-06T12:50:40.768348+0000","last_active":"2023-10-06T12:50:40.772550+0000","last_peered":"2023-10-06T12:50:40.772550+0000","last_clean":"2023-10-06T12:50:40.772550+0000","last_became_active":"2023-10-06T12:50:40.768040+0000","last_became_peered":"2023-10-06T12:50:40.768040+0000","last_unstale":"2023-10-06T12:50:40.772550+0000","last_undegraded":"2023-10-06T12:50:40.772550+0000","last_fullsized":"2023-10-06T12:50:40.772550+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:44.895817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314329+0000","last_change":"2023-10-06T12:50:40.288872+0000","last_active":"2023-10-06T12:50:41.314329+0000","last_peered":"2023-10-06T12:50:41.314329+0000","last_clean":"2023-10-06T12:50:41.314329+0000","last_became_active":"2023-10-06T12:50:40.288257+0000","last_became_peered":"2023-10-06T12:50:40.288257+0000","last_unstale":"2023-10-06T12:50:41.314329+0000","last_undegraded":"2023-10-06T12:50:41.314329+0000","last_fullsized":"2023-10-06T12:50:41.314329+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:35.394664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787344+0000","last_change":"2023-10-06T12:50:40.781308+0000","last_active":"2023-10-06T12:50:40.787344+0000","last_peered":"2023-10-06T12:50:40.787344+0000","last_clean":"2023-10-06T12:50:40.787344+0000","last_became_active":"2023-10-06T12:50:40.780957+0000","last_became_peered":"2023-10-06T12:50:40.780957+0000","last_unstale":"2023-10-06T12:50:40.787344+0000","last_undegraded":"2023-10-06T12:50:40.787344+0000","last_fullsized":"2023-10-06T12:50:40.787344+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:18.424675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770900+0000","last_change":"2023-10-06T12:50:40.300356+0000","last_active":"2023-10-06T12:50:40.770900+0000","last_peered":"2023-10-06T12:50:40.770900+0000","last_clean":"2023-10-06T12:50:40.770900+0000","last_became_active":"2023-10-06T12:50:40.299966+0000","last_became_peered":"2023-10-06T12:50:40.299966+0000","last_unstale":"2023-10-06T12:50:40.770900+0000","last_undegraded":"2023-10-06T12:50:40.770900+0000","last_fullsized":"2023-10-06T12:50:40.770900+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:33:54.397929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788270+0000","last_change":"2023-10-06T12:50:40.780155+0000","last_active":"2023-10-06T12:50:40.788270+0000","last_peered":"2023-10-06T12:50:40.788270+0000","last_clean":"2023-10-06T12:50:40.788270+0000","last_became_active":"2023-10-06T12:50:40.779416+0000","last_became_peered":"2023-10-06T12:50:40.779416+0000","last_unstale":"2023-10-06T12:50:40.788270+0000","last_undegraded":"2023-10-06T12:50:40.788270+0000","last_fullsized":"2023-10-06T12:50:40.788270+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:02.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315170+0000","last_change":"2023-10-06T12:50:40.291256+0000","last_active":"2023-10-06T12:50:41.315170+0000","last_peered":"2023-10-06T12:50:41.315170+0000","last_clean":"2023-10-06T12:50:41.315170+0000","last_became_active":"2023-10-06T12:50:40.290838+0000","last_became_peered":"2023-10-06T12:50:40.290838+0000","last_unstale":"2023-10-06T12:50:41.315170+0000","last_undegraded":"2023-10-06T12:50:41.315170+0000","last_fullsized":"2023-10-06T12:50:41.315170+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:53.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784462+0000","last_change":"2023-10-06T12:50:40.778770+0000","last_active":"2023-10-06T12:50:40.784462+0000","last_peered":"2023-10-06T12:50:40.784462+0000","last_clean":"2023-10-06T12:50:40.784462+0000","last_became_active":"2023-10-06T12:50:40.776268+0000","last_became_peered":"2023-10-06T12:50:40.776268+0000","last_unstale":"2023-10-06T12:50:40.784462+0000","last_undegraded":"2023-10-06T12:50:40.784462+0000","last_fullsized":"2023-10-06T12:50:40.784462+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:20:55.856953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763801+0000","last_change":"2023-10-06T12:50:40.288657+0000","last_active":"2023-10-06T12:50:40.763801+0000","last_peered":"2023-10-06T12:50:40.763801+0000","last_clean":"2023-10-06T12:50:40.763801+0000","last_became_active":"2023-10-06T12:50:40.288299+0000","last_became_peered":"2023-10-06T12:50:40.288299+0000","last_unstale":"2023-10-06T12:50:40.763801+0000","last_undegraded":"2023-10-06T12:50:40.763801+0000","last_fullsized":"2023-10-06T12:50:40.763801+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:45.064666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778509+0000","last_change":"2023-10-06T12:50:40.774871+0000","last_active":"2023-10-06T12:50:40.778509+0000","last_peered":"2023-10-06T12:50:40.778509+0000","last_clean":"2023-10-06T12:50:40.778509+0000","last_became_active":"2023-10-06T12:50:40.774591+0000","last_became_peered":"2023-10-06T12:50:40.774591+0000","last_unstale":"2023-10-06T12:50:40.778509+0000","last_undegraded":"2023-10-06T12:50:40.778509+0000","last_fullsized":"2023-10-06T12:50:40.778509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:54.108194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766535+0000","last_change":"2023-10-06T12:50:40.295842+0000","last_active":"2023-10-06T12:50:40.766535+0000","last_peered":"2023-10-06T12:50:40.766535+0000","last_clean":"2023-10-06T12:50:40.766535+0000","last_became_active":"2023-10-06T12:50:40.295234+0000","last_became_peered":"2023-10-06T12:50:40.295234+0000","last_unstale":"2023-10-06T12:50:40.766535+0000","last_undegraded":"2023-10-06T12:50:40.766535+0000","last_fullsized":"2023-10-06T12:50:40.766535+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:15.759155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786930+0000","last_change":"2023-10-06T12:50:40.784087+0000","last_active":"2023-10-06T12:50:40.786930+0000","last_peered":"2023-10-06T12:50:40.786930+0000","last_clean":"2023-10-06T12:50:40.786930+0000","last_became_active":"2023-10-06T12:50:40.783800+0000","last_became_peered":"2023-10-06T12:50:40.783800+0000","last_unstale":"2023-10-06T12:50:40.786930+0000","last_undegraded":"2023-10-06T12:50:40.786930+0000","last_fullsized":"2023-10-06T12:50:40.786930+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:38.677892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766706+0000","last_change":"2023-10-06T12:50:40.290848+0000","last_active":"2023-10-06T12:50:40.766706+0000","last_peered":"2023-10-06T12:50:40.766706+0000","last_clean":"2023-10-06T12:50:40.766706+0000","last_became_active":"2023-10-06T12:50:40.290513+0000","last_became_peered":"2023-10-06T12:50:40.290513+0000","last_unstale":"2023-10-06T12:50:40.766706+0000","last_undegraded":"2023-10-06T12:50:40.766706+0000","last_fullsized":"2023-10-06T12:50:40.766706+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:02.099185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784763+0000","last_change":"2023-10-06T12:50:40.778422+0000","last_active":"2023-10-06T12:50:40.784763+0000","last_peered":"2023-10-06T12:50:40.784763+0000","last_clean":"2023-10-06T12:50:40.784763+0000","last_became_active":"2023-10-06T12:50:40.778133+0000","last_became_peered":"2023-10-06T12:50:40.778133+0000","last_unstale":"2023-10-06T12:50:40.784763+0000","last_undegraded":"2023-10-06T12:50:40.784763+0000","last_fullsized":"2023-10-06T12:50:40.784763+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:52:44.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314963+0000","last_change":"2023-10-06T12:50:40.285393+0000","last_active":"2023-10-06T12:50:41.314963+0000","last_peered":"2023-10-06T12:50:41.314963+0000","last_clean":"2023-10-06T12:50:41.314963+0000","last_became_active":"2023-10-06T12:50:40.285070+0000","last_became_peered":"2023-10-06T12:50:40.285070+0000","last_unstale":"2023-10-06T12:50:41.314963+0000","last_undegraded":"2023-10-06T12:50:41.314963+0000","last_fullsized":"2023-10-06T12:50:41.314963+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:25.037833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787796+0000","last_change":"2023-10-06T12:50:40.782278+0000","last_active":"2023-10-06T12:50:40.787796+0000","last_peered":"2023-10-06T12:50:40.787796+0000","last_clean":"2023-10-06T12:50:40.787796+0000","last_became_active":"2023-10-06T12:50:40.781973+0000","last_became_peered":"2023-10-06T12:50:40.781973+0000","last_unstale":"2023-10-06T12:50:40.787796+0000","last_undegraded":"2023-10-06T12:50:40.787796+0000","last_fullsized":"2023-10-06T12:50:40.787796+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:34.886992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.764679+0000","last_change":"2023-10-06T12:50:40.299285+0000","last_active":"2023-10-06T12:50:40.764679+0000","last_peered":"2023-10-06T12:50:40.764679+0000","last_clean":"2023-10-06T12:50:40.764679+0000","last_became_active":"2023-10-06T12:50:40.298903+0000","last_became_peered":"2023-10-06T12:50:40.298903+0000","last_unstale":"2023-10-06T12:50:40.764679+0000","last_undegraded":"2023-10-06T12:50:40.764679+0000","last_fullsized":"2023-10-06T12:50:40.764679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:03.443330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788978+0000","last_change":"2023-10-06T12:50:40.781454+0000","last_active":"2023-10-06T12:50:40.788978+0000","last_peered":"2023-10-06T12:50:40.788978+0000","last_clean":"2023-10-06T12:50:40.788978+0000","last_became_active":"2023-10-06T12:50:40.781107+0000","last_became_peered":"2023-10-06T12:50:40.781107+0000","last_unstale":"2023-10-06T12:50:40.788978+0000","last_undegraded":"2023-10-06T12:50:40.788978+0000","last_fullsized":"2023-10-06T12:50:40.788978+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:46.760142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763993+0000","last_change":"2023-10-06T12:50:40.290074+0000","last_active":"2023-10-06T12:50:40.763993+0000","last_peered":"2023-10-06T12:50:40.763993+0000","last_clean":"2023-10-06T12:50:40.763993+0000","last_became_active":"2023-10-06T12:50:40.289751+0000","last_became_peered":"2023-10-06T12:50:40.289751+0000","last_unstale":"2023-10-06T12:50:40.763993+0000","last_undegraded":"2023-10-06T12:50:40.763993+0000","last_fullsized":"2023-10-06T12:50:40.763993+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:52.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787766+0000","last_change":"2023-10-06T12:50:40.783197+0000","last_active":"2023-10-06T12:50:40.787766+0000","last_peered":"2023-10-06T12:50:40.787766+0000","last_clean":"2023-10-06T12:50:40.787766+0000","last_became_active":"2023-10-06T12:50:40.782606+0000","last_became_peered":"2023-10-06T12:50:40.782606+0000","last_unstale":"2023-10-06T12:50:40.787766+0000","last_undegraded":"2023-10-06T12:50:40.787766+0000","last_fullsized":"2023-10-06T12:50:40.787766+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:32:51.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.765793+0000","last_change":"2023-10-06T12:50:40.295945+0000","last_active":"2023-10-06T12:50:40.765793+0000","last_peered":"2023-10-06T12:50:40.765793+0000","last_clean":"2023-10-06T12:50:40.765793+0000","last_became_active":"2023-10-06T12:50:40.295659+0000","last_became_peered":"2023-10-06T12:50:40.295659+0000","last_unstale":"2023-10-06T12:50:40.765793+0000","last_undegraded":"2023-10-06T12:50:40.765793+0000","last_fullsized":"2023-10-06T12:50:40.765793+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:39.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.785785+0000","last_change":"2023-10-06T12:50:40.780760+0000","last_active":"2023-10-06T12:50:40.785785+0000","last_peered":"2023-10-06T12:50:40.785785+0000","last_clean":"2023-10-06T12:50:40.785785+0000","last_became_active":"2023-10-06T12:50:40.780384+0000","last_became_peered":"2023-10-06T12:50:40.780384+0000","last_unstale":"2023-10-06T12:50:40.785785+0000","last_undegraded":"2023-10-06T12:50:40.785785+0000","last_fullsized":"2023-10-06T12:50:40.785785+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:02.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.761290+0000","last_change":"2023-10-06T12:50:40.280675+0000","last_active":"2023-10-06T12:50:40.761290+0000","last_peered":"2023-10-06T12:50:40.761290+0000","last_clean":"2023-10-06T12:50:40.761290+0000","last_became_active":"2023-10-06T12:50:40.280017+0000","last_became_peered":"2023-10-06T12:50:40.280017+0000","last_unstale":"2023-10-06T12:50:40.761290+0000","last_undegraded":"2023-10-06T12:50:40.761290+0000","last_fullsized":"2023-10-06T12:50:40.761290+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:53.574100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784296+0000","last_change":"2023-10-06T12:50:40.776901+0000","last_active":"2023-10-06T12:50:40.784296+0000","last_peered":"2023-10-06T12:50:40.784296+0000","last_clean":"2023-10-06T12:50:40.784296+0000","last_became_active":"2023-10-06T12:50:40.776620+0000","last_became_peered":"2023-10-06T12:50:40.776620+0000","last_unstale":"2023-10-06T12:50:40.784296+0000","last_undegraded":"2023-10-06T12:50:40.784296+0000","last_fullsized":"2023-10-06T12:50:40.784296+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:45.504866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766443+0000","last_change":"2023-10-06T12:50:40.299167+0000","last_active":"2023-10-06T12:50:40.766443+0000","last_peered":"2023-10-06T12:50:40.766443+0000","last_clean":"2023-10-06T12:50:40.766443+0000","last_became_active":"2023-10-06T12:50:40.298874+0000","last_became_peered":"2023-10-06T12:50:40.298874+0000","last_unstale":"2023-10-06T12:50:40.766443+0000","last_undegraded":"2023-10-06T12:50:40.766443+0000","last_fullsized":"2023-10-06T12:50:40.766443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:46.170536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787135+0000","last_change":"2023-10-06T12:50:40.779002+0000","last_active":"2023-10-06T12:50:40.787135+0000","last_peered":"2023-10-06T12:50:40.787135+0000","last_clean":"2023-10-06T12:50:40.787135+0000","last_became_active":"2023-10-06T12:50:40.778709+0000","last_became_peered":"2023-10-06T12:50:40.778709+0000","last_unstale":"2023-10-06T12:50:40.787135+0000","last_undegraded":"2023-10-06T12:50:40.787135+0000","last_fullsized":"2023-10-06T12:50:40.787135+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:17.165460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.761052+0000","last_change":"2023-10-06T12:50:40.297409+0000","last_active":"2023-10-06T12:50:40.761052+0000","last_peered":"2023-10-06T12:50:40.761052+0000","last_clean":"2023-10-06T12:50:40.761052+0000","last_became_active":"2023-10-06T12:50:40.297094+0000","last_became_peered":"2023-10-06T12:50:40.297094+0000","last_unstale":"2023-10-06T12:50:40.761052+0000","last_undegraded":"2023-10-06T12:50:40.761052+0000","last_fullsized":"2023-10-06T12:50:40.761052+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:31.447429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787088+0000","last_change":"2023-10-06T12:50:40.781121+0000","last_active":"2023-10-06T12:50:40.787088+0000","last_peered":"2023-10-06T12:50:40.787088+0000","last_clean":"2023-10-06T12:50:40.787088+0000","last_became_active":"2023-10-06T12:50:40.780806+0000","last_became_peered":"2023-10-06T12:50:40.780806+0000","last_unstale":"2023-10-06T12:50:40.787088+0000","last_undegraded":"2023-10-06T12:50:40.787088+0000","last_fullsized":"2023-10-06T12:50:40.787088+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:04.673251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.756532+0000","last_change":"2023-10-06T12:50:40.290805+0000","last_active":"2023-10-06T12:50:40.756532+0000","last_peered":"2023-10-06T12:50:40.756532+0000","last_clean":"2023-10-06T12:50:40.756532+0000","last_became_active":"2023-10-06T12:50:40.290457+0000","last_became_peered":"2023-10-06T12:50:40.290457+0000","last_unstale":"2023-10-06T12:50:40.756532+0000","last_undegraded":"2023-10-06T12:50:40.756532+0000","last_fullsized":"2023-10-06T12:50:40.756532+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:25.206614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786881+0000","last_change":"2023-10-06T12:50:40.779184+0000","last_active":"2023-10-06T12:50:40.786881+0000","last_peered":"2023-10-06T12:50:40.786881+0000","last_clean":"2023-10-06T12:50:40.786881+0000","last_became_active":"2023-10-06T12:50:40.778347+0000","last_became_peered":"2023-10-06T12:50:40.778347+0000","last_unstale":"2023-10-06T12:50:40.786881+0000","last_undegraded":"2023-10-06T12:50:40.786881+0000","last_fullsized":"2023-10-06T12:50:40.786881+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:46.760142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.764564+0000","last_change":"2023-10-06T12:50:40.301356+0000","last_active":"2023-10-06T12:50:40.764564+0000","last_peered":"2023-10-06T12:50:40.764564+0000","last_clean":"2023-10-06T12:50:40.764564+0000","last_became_active":"2023-10-06T12:50:40.301060+0000","last_became_peered":"2023-10-06T12:50:40.301060+0000","last_unstale":"2023-10-06T12:50:40.764564+0000","last_undegraded":"2023-10-06T12:50:40.764564+0000","last_fullsized":"2023-10-06T12:50:40.764564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:45.745172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"172'326","reported_seq":1042,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766958+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:50:40.766958+0000","last_peered":"2023-10-06T12:50:40.766958+0000","last_clean":"2023-10-06T12:50:40.766958+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:50:40.766958+0000","last_undegraded":"2023-10-06T12:50:40.766958+0000","last_fullsized":"2023-10-06T12:50:40.766958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.760794+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:50:40.760794+0000","last_peered":"2023-10-06T12:50:40.760794+0000","last_clean":"2023-10-06T12:50:40.760794+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:50:40.760794+0000","last_undegraded":"2023-10-06T12:50:40.760794+0000","last_fullsized":"2023-10-06T12:50:40.760794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.777508+0000","last_change":"2023-10-06T12:50:40.775783+0000","last_active":"2023-10-06T12:50:40.777508+0000","last_peered":"2023-10-06T12:50:40.777508+0000","last_clean":"2023-10-06T12:50:40.777508+0000","last_became_active":"2023-10-06T12:50:40.775512+0000","last_became_peered":"2023-10-06T12:50:40.775512+0000","last_unstale":"2023-10-06T12:50:40.777508+0000","last_undegraded":"2023-10-06T12:50:40.777508+0000","last_fullsized":"2023-10-06T12:50:40.777508+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:01.262267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"182'3","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.310875+0000","last_change":"2023-10-06T12:50:40.285774+0000","last_active":"2023-10-06T12:50:41.310875+0000","last_peered":"2023-10-06T12:50:41.310875+0000","last_clean":"2023-10-06T12:50:41.310875+0000","last_became_active":"2023-10-06T12:50:40.285452+0000","last_became_peered":"2023-10-06T12:50:40.285452+0000","last_unstale":"2023-10-06T12:50:41.310875+0000","last_undegraded":"2023-10-06T12:50:41.310875+0000","last_fullsized":"2023-10-06T12:50:41.310875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:19.998772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766873+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:50:40.766873+0000","last_peered":"2023-10-06T12:50:40.766873+0000","last_clean":"2023-10-06T12:50:40.766873+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:50:40.766873+0000","last_undegraded":"2023-10-06T12:50:40.766873+0000","last_fullsized":"2023-10-06T12:50:40.766873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788947+0000","last_change":"2023-10-06T12:50:40.783276+0000","last_active":"2023-10-06T12:50:40.788947+0000","last_peered":"2023-10-06T12:50:40.788947+0000","last_clean":"2023-10-06T12:50:40.788947+0000","last_became_active":"2023-10-06T12:50:40.782946+0000","last_became_peered":"2023-10-06T12:50:40.782946+0000","last_unstale":"2023-10-06T12:50:40.788947+0000","last_undegraded":"2023-10-06T12:50:40.788947+0000","last_fullsized":"2023-10-06T12:50:40.788947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:32:51.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"182'6","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.317781+0000","last_change":"2023-10-06T12:50:40.299914+0000","last_active":"2023-10-06T12:50:41.317781+0000","last_peered":"2023-10-06T12:50:41.317781+0000","last_clean":"2023-10-06T12:50:41.317781+0000","last_became_active":"2023-10-06T12:50:40.299632+0000","last_became_peered":"2023-10-06T12:50:40.299632+0000","last_unstale":"2023-10-06T12:50:41.317781+0000","last_undegraded":"2023-10-06T12:50:41.317781+0000","last_fullsized":"2023-10-06T12:50:41.317781+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:03.746846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766609+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:50:40.766609+0000","last_peered":"2023-10-06T12:50:40.766609+0000","last_clean":"2023-10-06T12:50:40.766609+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:50:40.766609+0000","last_undegraded":"2023-10-06T12:50:40.766609+0000","last_fullsized":"2023-10-06T12:50:40.766609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786958+0000","last_change":"2023-10-06T12:50:40.780781+0000","last_active":"2023-10-06T12:50:40.786958+0000","last_peered":"2023-10-06T12:50:40.786958+0000","last_clean":"2023-10-06T12:50:40.786958+0000","last_became_active":"2023-10-06T12:50:40.775160+0000","last_became_peered":"2023-10-06T12:50:40.775160+0000","last_unstale":"2023-10-06T12:50:40.786958+0000","last_undegraded":"2023-10-06T12:50:40.786958+0000","last_fullsized":"2023-10-06T12:50:40.786958+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:54:25.796751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.765278+0000","last_change":"2023-10-06T12:50:40.301828+0000","last_active":"2023-10-06T12:50:40.765278+0000","last_peered":"2023-10-06T12:50:40.765278+0000","last_clean":"2023-10-06T12:50:40.765278+0000","last_became_active":"2023-10-06T12:50:40.301514+0000","last_became_peered":"2023-10-06T12:50:40.301514+0000","last_unstale":"2023-10-06T12:50:40.765278+0000","last_undegraded":"2023-10-06T12:50:40.765278+0000","last_fullsized":"2023-10-06T12:50:40.765278+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:53.574100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763786+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:50:40.763786+0000","last_peered":"2023-10-06T12:50:40.763786+0000","last_clean":"2023-10-06T12:50:40.763786+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:50:40.763786+0000","last_undegraded":"2023-10-06T12:50:40.763786+0000","last_fullsized":"2023-10-06T12:50:40.763786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.779247+0000","last_change":"2023-10-06T12:50:40.774244+0000","last_active":"2023-10-06T12:50:40.779247+0000","last_peered":"2023-10-06T12:50:40.779247+0000","last_clean":"2023-10-06T12:50:40.779247+0000","last_became_active":"2023-10-06T12:50:40.773905+0000","last_became_peered":"2023-10-06T12:50:40.773905+0000","last_unstale":"2023-10-06T12:50:40.779247+0000","last_undegraded":"2023-10-06T12:50:40.779247+0000","last_fullsized":"2023-10-06T12:50:40.779247+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.354187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.760725+0000","last_change":"2023-10-06T12:50:40.288773+0000","last_active":"2023-10-06T12:50:40.760725+0000","last_peered":"2023-10-06T12:50:40.760725+0000","last_clean":"2023-10-06T12:50:40.760725+0000","last_became_active":"2023-10-06T12:50:40.288409+0000","last_became_peered":"2023-10-06T12:50:40.288409+0000","last_unstale":"2023-10-06T12:50:40.760725+0000","last_undegraded":"2023-10-06T12:50:40.760725+0000","last_fullsized":"2023-10-06T12:50:40.760725+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:13.351610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.769705+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:50:40.769705+0000","last_peered":"2023-10-06T12:50:40.769705+0000","last_clean":"2023-10-06T12:50:40.769705+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:50:40.769705+0000","last_undegraded":"2023-10-06T12:50:40.769705+0000","last_fullsized":"2023-10-06T12:50:40.769705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780595+0000","last_change":"2023-10-06T12:50:40.774808+0000","last_active":"2023-10-06T12:50:40.780595+0000","last_peered":"2023-10-06T12:50:40.780595+0000","last_clean":"2023-10-06T12:50:40.780595+0000","last_became_active":"2023-10-06T12:50:40.774526+0000","last_became_peered":"2023-10-06T12:50:40.774526+0000","last_unstale":"2023-10-06T12:50:40.780595+0000","last_undegraded":"2023-10-06T12:50:40.780595+0000","last_fullsized":"2023-10-06T12:50:40.780595+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:45.504866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"182'8","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.316168+0000","last_change":"2023-10-06T12:50:40.296552+0000","last_active":"2023-10-06T12:50:41.316168+0000","last_peered":"2023-10-06T12:50:41.316168+0000","last_clean":"2023-10-06T12:50:41.316168+0000","last_became_active":"2023-10-06T12:50:40.295853+0000","last_became_peered":"2023-10-06T12:50:40.295853+0000","last_unstale":"2023-10-06T12:50:41.316168+0000","last_undegraded":"2023-10-06T12:50:41.316168+0000","last_fullsized":"2023-10-06T12:50:41.316168+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:29.373888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770290+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:50:40.770290+0000","last_peered":"2023-10-06T12:50:40.770290+0000","last_clean":"2023-10-06T12:50:40.770290+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:50:40.770290+0000","last_undegraded":"2023-10-06T12:50:40.770290+0000","last_fullsized":"2023-10-06T12:50:40.770290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788119+0000","last_change":"2023-10-06T12:50:40.784137+0000","last_active":"2023-10-06T12:50:40.788119+0000","last_peered":"2023-10-06T12:50:40.788119+0000","last_clean":"2023-10-06T12:50:40.788119+0000","last_became_active":"2023-10-06T12:50:40.783861+0000","last_became_peered":"2023-10-06T12:50:40.783861+0000","last_unstale":"2023-10-06T12:50:40.788119+0000","last_undegraded":"2023-10-06T12:50:40.788119+0000","last_fullsized":"2023-10-06T12:50:40.788119+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:45.076521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315436+0000","last_change":"2023-10-06T12:50:40.285992+0000","last_active":"2023-10-06T12:50:41.315436+0000","last_peered":"2023-10-06T12:50:41.315436+0000","last_clean":"2023-10-06T12:50:41.315436+0000","last_became_active":"2023-10-06T12:50:40.285679+0000","last_became_peered":"2023-10-06T12:50:40.285679+0000","last_unstale":"2023-10-06T12:50:41.315436+0000","last_undegraded":"2023-10-06T12:50:41.315436+0000","last_fullsized":"2023-10-06T12:50:41.315436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:28:39.180568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.771402+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:50:40.771402+0000","last_peered":"2023-10-06T12:50:40.771402+0000","last_clean":"2023-10-06T12:50:40.771402+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:50:40.771402+0000","last_undegraded":"2023-10-06T12:50:40.771402+0000","last_fullsized":"2023-10-06T12:50:40.771402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787453+0000","last_change":"2023-10-06T12:50:40.782198+0000","last_active":"2023-10-06T12:50:40.787453+0000","last_peered":"2023-10-06T12:50:40.787453+0000","last_clean":"2023-10-06T12:50:40.787453+0000","last_became_active":"2023-10-06T12:50:40.781890+0000","last_became_peered":"2023-10-06T12:50:40.781890+0000","last_unstale":"2023-10-06T12:50:40.787453+0000","last_undegraded":"2023-10-06T12:50:40.787453+0000","last_fullsized":"2023-10-06T12:50:40.787453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:32.586422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766535+0000","last_change":"2023-10-06T12:50:40.297347+0000","last_active":"2023-10-06T12:50:40.766535+0000","last_peered":"2023-10-06T12:50:40.766535+0000","last_clean":"2023-10-06T12:50:40.766535+0000","last_became_active":"2023-10-06T12:50:40.293329+0000","last_became_peered":"2023-10-06T12:50:40.293329+0000","last_unstale":"2023-10-06T12:50:40.766535+0000","last_undegraded":"2023-10-06T12:50:40.766535+0000","last_fullsized":"2023-10-06T12:50:40.766535+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:44.489896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.769875+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:50:40.769875+0000","last_peered":"2023-10-06T12:50:40.769875+0000","last_clean":"2023-10-06T12:50:40.769875+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:50:40.769875+0000","last_undegraded":"2023-10-06T12:50:40.769875+0000","last_fullsized":"2023-10-06T12:50:40.769875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067640,"statfs":{"total":96636764160,"available":96325263360,"internally_reserved":0,"allocated":2883584,"data_stored":1804815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1799999999999999,"15min":1.169},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.56299999999999994},"max":{"1min":1.5980000000000001,"5min":1.758,"15min":1.758},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.143,"5min":1.1679999999999999,"15min":1.1559999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.8660000000000001},"last":1.077}]},{"osd":1,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.379,"15min":1.379},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.72099999999999997},"max":{"1min":1.8029999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.357},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2789999999999999,"15min":1.2929999999999999},"min":{"1min":0.82999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.76},"last":1.395}]},{"osd":2,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.56299999999999994},"max":{"1min":1.5209999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.431},{"interface":"front","average":{"1min":1.246,"5min":1.2629999999999999,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.92300000000000004,"15min":0.78400000000000003},"max":{"1min":1.621,"5min":1.7150000000000001,"15min":1.8360000000000001},"last":1.248}]},{"osd":3,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.78000000000000003},"max":{"1min":1.694,"5min":1.9019999999999999,"15min":2.105},"last":1.657},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.351,"15min":1.343},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.81999999999999995},"max":{"1min":1.71,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.6279999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.486,"5min":1.667,"15min":1.681},"last":1.29},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.649,"5min":2.0939999999999999,"15min":2.2989999999999999},"last":1.776}]},{"osd":5,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3260000000000001,"15min":1.3400000000000001},"min":{"1min":0.93100000000000005,"5min":0.874,"15min":0.59499999999999997},"max":{"1min":1.657,"5min":2.036,"15min":2.036},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6419999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.385,"15min":1.391},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.7490000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.496},{"interface":"front","average":{"1min":1.365,"5min":1.466,"15min":1.466},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.8360000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575950,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315908,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055932,"statfs":{"total":96636764160,"available":96313274368,"internally_reserved":0,"allocated":2289664,"data_stored":1214447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2969999999999999,"15min":1.3100000000000001},"min":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5569999999999999,"5min":1.857,"15min":1.857},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.3160000000000001},"min":{"1min":1.034,"5min":0.79700000000000004,"15min":0.74399999999999999},"max":{"1min":1.4510000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3879999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1990000000000001,"15min":1.1890000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.363,"5min":1.597,"15min":1.7789999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2150000000000001,"15min":1.2050000000000001},"min":{"1min":0.89000000000000001,"5min":0.877,"15min":0.81399999999999995},"max":{"1min":1.623,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.5229999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.208},"min":{"1min":0.92300000000000004,"5min":0.79200000000000004,"15min":0.68500000000000005},"max":{"1min":1.536,"5min":1.724,"15min":1.839},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.88100000000000001,"15min":0.77500000000000002},"max":{"1min":1.526,"5min":1.5669999999999999,"15min":1.7290000000000001},"last":1.2290000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3100000000000001,"15min":1.323},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.68600000000000005},"max":{"1min":1.5580000000000001,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.411},{"interface":"front","average":{"1min":1.274,"5min":1.3240000000000001,"15min":1.339},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.6719999999999999,"15min":1.8879999999999999},"last":1.4399999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.89500000000000002,"5min":0.77300000000000002,"15min":0.69599999999999995},"max":{"1min":1.47,"5min":1.998,"15min":2.1459999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":0.85099999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5109999999999999,"5min":1.873,"15min":1.911},"last":1.1619999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2270000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.81399999999999995,"15min":0.77200000000000002},"max":{"1min":1.633,"5min":1.778,"15min":1.8600000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.397,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":1.115,"5min":0.875,"15min":0.75900000000000001},"max":{"1min":1.6850000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.268}]},{"osd":7,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":1.242,"5min":0.93799999999999994,"15min":0.63200000000000001},"max":{"1min":1.5980000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.498},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4490000000000001,"15min":1.464},"min":{"1min":1.0309999999999999,"5min":0.86699999999999999,"15min":0.81000000000000005},"max":{"1min":1.72,"5min":2.0609999999999999,"15min":2.2170000000000001},"last":1.355}]}]},{"osd":5,"up_from":13,"seq":55834575950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311820,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060020,"statfs":{"total":96636764160,"available":96317460480,"internally_reserved":0,"allocated":2297856,"data_stored":1214571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.276,"15min":1.292},"min":{"1min":1.091,"5min":0.751,"15min":0.751},"max":{"1min":1.663,"5min":1.6919999999999999,"15min":1.7889999999999999},"last":1.365},{"interface":"front","average":{"1min":1.365,"5min":1.3620000000000001,"15min":1.373},"min":{"1min":1.099,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.556,"5min":1.7929999999999999,"15min":2.657},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.173},"min":{"1min":0.86699999999999999,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.492,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.94799999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.569,"5min":1.716,"15min":1.732},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.228},"min":{"1min":0.96899999999999997,"5min":0.77000000000000002,"15min":0.74399999999999999},"max":{"1min":1.466,"5min":1.7549999999999999,"15min":2.754},"last":1.103},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2909999999999999,"15min":1.2849999999999999},"min":{"1min":1.125,"5min":0.89400000000000002,"15min":0.85699999999999998},"max":{"1min":1.4279999999999999,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.292},"min":{"1min":0.997,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.4790000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3540000000000001,"15min":1.3620000000000001},"min":{"1min":1.111,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.6559999999999999,"5min":1.845,"15min":2.9100000000000001},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.248,"15min":1.2669999999999999},"min":{"1min":0.85299999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5429999999999999,"5min":1.698,"15min":4.0899999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.3660000000000001,"15min":1.3959999999999999},"min":{"1min":1.2110000000000001,"5min":0.71899999999999997,"15min":0.67500000000000004},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":2.2069999999999999},"last":1.5029999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2709999999999999,"15min":1.3},"min":{"1min":0.92600000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5589999999999999,"5min":1.915,"15min":2.8300000000000001},"last":1.472},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.357},"min":{"1min":0.95799999999999996,"5min":0.76100000000000001,"15min":0.67000000000000004},"max":{"1min":1.649,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.327}]},{"osd":7,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3240000000000001,"15min":1.363},"min":{"1min":0.93999999999999995,"5min":0.877,"15min":0.76000000000000001},"max":{"1min":1.74,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.548},{"interface":"front","average":{"1min":1.345,"5min":1.3999999999999999,"15min":1.4330000000000001},"min":{"1min":1.0800000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7669999999999999,"5min":1.968,"15min":3.0049999999999999},"last":1.536}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311828,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060012,"statfs":{"total":96636764160,"available":96317452288,"internally_reserved":0,"allocated":2306048,"data_stored":1215183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.153,"15min":1.1919999999999999},"min":{"1min":0.84699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5760000000000001,"5min":1.851,"15min":1.851},"last":1.46},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1899999999999999,"15min":1.216},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6559999999999999,"5min":1.7529999999999999,"15min":2.1680000000000001},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4119999999999999},"min":{"1min":0.94299999999999995,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6319999999999999,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.333,"15min":1.321},"min":{"1min":1.046,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6160000000000001,"5min":1.948,"15min":1.948},"last":1.4159999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.2629999999999999},"min":{"1min":0.90900000000000003,"5min":0.77500000000000002,"15min":0.76500000000000001},"max":{"1min":1.6779999999999999,"5min":2.024,"15min":2.024},"last":1.651},{"interface":"front","average":{"1min":1.321,"5min":1.29,"15min":1.3029999999999999},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.6950000000000001,"5min":1.917,"15min":2.0630000000000002},"last":1.4370000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2749999999999999,"15min":1.2669999999999999},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.5329999999999999,"5min":1.7430000000000001,"15min":1.768},"last":1.508},{"interface":"front","average":{"1min":1.319,"5min":1.347,"15min":1.3420000000000001},"min":{"1min":1.0129999999999999,"5min":0.94399999999999995,"15min":0.86699999999999999},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.24},"min":{"1min":0.93700000000000006,"5min":0.73199999999999998,"15min":0.71399999999999997},"max":{"1min":1.6259999999999999,"5min":2.012,"15min":2.3220000000000001},"last":0.87},{"interface":"front","average":{"1min":1.212,"5min":1.2470000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.77700000000000002,"15min":0.61899999999999999},"max":{"1min":1.6859999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2849999999999999,"15min":1.2769999999999999},"min":{"1min":0.91600000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7789999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4279999999999999,"15min":1.403},"min":{"1min":0.86699999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.75,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.2230000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3779999999999999,"15min":1.369},"min":{"1min":1.018,"5min":0.81100000000000005,"15min":0.70999999999999996},"max":{"1min":1.944,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4450000000000001,"15min":1.444},"min":{"1min":1.1000000000000001,"5min":1.01,"15min":0.59599999999999997},"max":{"1min":1.7030000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.5660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575952,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299544,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072296,"statfs":{"total":96636764160,"available":96330031104,"internally_reserved":0,"allocated":2310144,"data_stored":1216559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92800000000000005,"15min":0.92600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56999999999999995,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.478,"15min":1.605},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99199999999999999,"15min":1.0269999999999999},"min":{"1min":0.66500000000000004,"5min":0.63,"15min":0.56699999999999995},"max":{"1min":1.431,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.464}]},{"osd":1,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.98899999999999999},"min":{"1min":0.69799999999999995,"5min":0.54000000000000004,"15min":0.53100000000000003},"max":{"1min":1.464,"5min":1.5780000000000001,"15min":1.639},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.105,"15min":1.119},"min":{"1min":0.88100000000000001,"5min":0.68000000000000005,"15min":0.55400000000000005},"max":{"1min":1.5700000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.099}]},{"osd":2,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.84099999999999997,"5min":0.68799999999999994,"15min":0.60399999999999998},"max":{"1min":1.4510000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.212,"15min":1.2390000000000001},"min":{"1min":1.0049999999999999,"5min":0.747,"15min":0.63900000000000001},"max":{"1min":1.5740000000000001,"5min":1.738,"15min":1.768},"last":1.2649999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.23,"15min":1.264},"min":{"1min":0.95099999999999996,"5min":0.90400000000000003,"15min":0.76600000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":2.008},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.363,"15min":1.385},"min":{"1min":0.999,"5min":0.95599999999999996,"15min":0.94299999999999995},"max":{"1min":1.6779999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.163}]},{"osd":5,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.756,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.1679999999999999,"5min":0.97899999999999998,"15min":0.64200000000000002},"max":{"1min":1.8169999999999999,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.4730000000000001,"15min":1.4930000000000001},"min":{"1min":1.2589999999999999,"5min":1.0129999999999999,"15min":0.72199999999999998},"max":{"1min":1.911,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4179999999999999,"15min":1.4119999999999999},"min":{"1min":1.056,"5min":0.995,"15min":0.84099999999999997},"max":{"1min":1.6759999999999999,"5min":1.9159999999999999,"15min":1.929},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.454,"15min":1.47},"min":{"1min":1.022,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.726,"5min":2.085,"15min":2.0899999999999999},"last":1.708},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.528,"15min":1.5429999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.9079999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.653}]}]},{"osd":2,"up_from":13,"seq":55834575950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311812,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060028,"statfs":{"total":96636764160,"available":96317468672,"internally_reserved":0,"allocated":2289664,"data_stored":1214447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.998,"15min":0.98799999999999999},"min":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.665,"15min":1.665},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.93300000000000005,"5min":0.77400000000000002,"15min":0.63800000000000001},"max":{"1min":1.415,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.081}]},{"osd":1,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91900000000000004,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0820000000000001,"5min":1.577,"15min":1.728},"last":1.234},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0489999999999999,"15min":1.0569999999999999},"min":{"1min":0.79300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5,"5min":1.6040000000000001,"15min":1.8200000000000001},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.27},"min":{"1min":0.97899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.718,"5min":1.804,"15min":1.962},"last":1.55},{"interface":"front","average":{"1min":1.403,"5min":1.3859999999999999,"15min":1.3660000000000001},"min":{"1min":1.0389999999999999,"5min":0.64400000000000002,"15min":0.54600000000000004},"max":{"1min":1.7789999999999999,"5min":1.806,"15min":1.9510000000000001},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.98799999999999999,"5min":0.84999999999999998,"15min":0.82099999999999995},"max":{"1min":1.696,"5min":1.696,"15min":1.897},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.339,"15min":1.3500000000000001},"min":{"1min":1.012,"5min":1.006,"15min":0.95699999999999996},"max":{"1min":1.5880000000000001,"5min":1.716,"15min":1.8560000000000001},"last":1.292}]},{"osd":5,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.351},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":1.1379999999999999,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.7070000000000001,"5min":1.764,"15min":2.0619999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4119999999999999,"15min":1.423},"min":{"1min":1.1839999999999999,"5min":1.0049999999999999,"15min":0.88900000000000001},"max":{"1min":1.6899999999999999,"5min":1.847,"15min":2.0699999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":1.133,"5min":1.119,"15min":1.111},"max":{"1min":2.0609999999999999,"5min":2.206,"15min":2.206},"last":1.139}]},{"osd":7,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4019999999999999,"15min":1.4350000000000001},"min":{"1min":1.1080000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.837,"5min":1.8959999999999999,"15min":1.9970000000000001},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.5049999999999999,"15min":1.5229999999999999},"min":{"1min":1.167,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7370000000000001,"5min":1.9890000000000001,"15min":2.177},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834575950,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311824,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060016,"statfs":{"total":96636764160,"available":96317456384,"internally_reserved":0,"allocated":2301952,"data_stored":1215983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94899999999999995,"15min":0.94599999999999995},"min":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.48799999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.036,"5min":1.0389999999999999,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.627},"max":{"1min":1.498,"5min":1.6539999999999999,"15min":1.9099999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":0.98699999999999999},"min":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.56499999999999995},"max":{"1min":1.573,"5min":1.585,"15min":1.6970000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.117,"15min":1.131},"min":{"1min":0.86599999999999999,"5min":0.79700000000000004,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.6619999999999999,"15min":1.6739999999999999},"last":1.0389999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.232,"15min":1.2749999999999999},"min":{"1min":0.95799999999999996,"5min":0.85099999999999998,"15min":0.81299999999999994},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":1.9399999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.157,"15min":1.1639999999999999},"min":{"1min":0.82399999999999995,"5min":0.74099999999999999,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":1.6140000000000001,"15min":1.7609999999999999},"last":1.2130000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3029999999999999,"15min":1.276},"min":{"1min":1.075,"5min":1.036,"15min":0.84799999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.9179999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4279999999999999,"15min":1.4390000000000001},"min":{"1min":1.1639999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.675,"5min":2.093,"15min":2.093},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.407,"15min":1.401},"min":{"1min":1.1240000000000001,"5min":1.0900000000000001,"15min":0.81599999999999995},"max":{"1min":1.7649999999999999,"5min":2.052,"15min":2.052},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4139999999999999,"15min":1.397},"min":{"1min":1.103,"5min":0.93100000000000005,"15min":0.91600000000000004},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.633}]},{"osd":6,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5,"15min":1.524},"min":{"1min":1.2010000000000001,"5min":1.1359999999999999,"15min":0.996},"max":{"1min":1.6779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.472,"15min":1.484},"min":{"1min":1.1830000000000001,"5min":1.157,"15min":1.0289999999999999},"max":{"1min":1.867,"5min":2.0430000000000001,"15min":2.137},"last":1.573}]},{"osd":7,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5900000000000001,"15min":1.585},"min":{"1min":1.4399999999999999,"5min":1.228,"15min":1.0880000000000001},"max":{"1min":1.9279999999999999,"5min":2.1389999999999998,"15min":2.2480000000000002},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.556,"15min":1.526},"min":{"1min":1.335,"5min":1.161,"15min":0.99299999999999999},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.381}]}]},{"osd":0,"up_from":13,"seq":55834575951,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304208,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067632,"statfs":{"total":96636764160,"available":96325255168,"internally_reserved":0,"allocated":2891776,"data_stored":1804919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.71699999999999997,"5min":0.60499999999999998,"15min":0.59699999999999998},"max":{"1min":1.3200000000000001,"5min":1.4450000000000001,"15min":1.724},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.84499999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.84799999999999998}]},{"osd":2,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0349999999999999,"15min":1.038},"min":{"1min":0.84199999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.343,"5min":1.708,"15min":1.7370000000000001},"last":1.006},{"interface":"front","average":{"1min":1.101,"5min":1.0680000000000001,"15min":1.093},"min":{"1min":0.86099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.5269999999999999,"5min":1.528,"15min":2.028},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.137,"15min":1.125},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.61099999999999999},"max":{"1min":1.5840000000000001,"5min":1.724,"15min":2.004},"last":1.069},{"interface":"front","average":{"1min":1.383,"5min":1.3660000000000001,"15min":1.375},"min":{"1min":1.1859999999999999,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.6140000000000001,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.411}]},{"osd":4,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.347,"15min":1.3500000000000001},"min":{"1min":1.1830000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.762,"5min":1.823,"15min":2.0880000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.367,"15min":1.363},"min":{"1min":0.98699999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.637,"5min":1.9850000000000001,"15min":2.387},"last":1.663}]},{"osd":5,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4159999999999999,"15min":1.409},"min":{"1min":1.069,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.742,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3879999999999999,"15min":1.4259999999999999},"min":{"1min":1.1240000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6739999999999999,"5min":1.9510000000000001,"15min":2.23},"last":1.524}]},{"osd":6,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.409,"15min":1.4159999999999999},"min":{"1min":1.1699999999999999,"5min":0.749,"15min":0.749},"max":{"1min":1.8939999999999999,"5min":1.9770000000000001,"15min":2.0910000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.181,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.9299999999999999,"5min":1.9390000000000001,"15min":2.7530000000000001},"last":1.508}]},{"osd":7,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5009999999999999,"15min":1.5},"min":{"1min":1.2589999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.867,"5min":2.177,"15min":2.177},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.472,"15min":1.492},"min":{"1min":1.2390000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.869,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.577}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:50:47.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:50:47.625 INFO:tasks.ceph.mds.c.smithi046.stderr:2023-10-06T12:50:47.236+0000 7f744d3c5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:50:47.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 Processor -- start 2023-10-06T12:50:47.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 -- start start 2023-10-06T12:50:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89ec0697c0 0x7f89ec0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec0cf100 con 0x7f89ec0af000 2023-10-06T12:50:47.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ec0ce4f0 con 0x7f89ec0cb4c0 2023-10-06T12:50:47.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.792+0000 7f89f2dc3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ec0ce300 con 0x7f89ec0697c0 2023-10-06T12:50:47.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f1dc1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89ec0af000 0x7f89ec0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38078/0 (socket says 172.21.15.46:38078) 2023-10-06T12:50:47.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f1dc1700 1 -- 172.21.15.46:0/2787337496 learned_addr learned my addr 172.21.15.46:0/2787337496 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:47.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f25c2700 1 --2- 172.21.15.46:0/2787337496 >> v2:172.21.15.46:3300/0 conn(0x7f89ec0697c0 0x7f89ec0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:47.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141525632 0 0) 0x7f89ec0cf100 con 0x7f89ec0af000 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d8003570 con 0x7f89ec0af000 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653248128 0 0) 0x7f89ec0ce4f0 con 0x7f89ec0cb4c0 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89ec0cf100 con 0x7f89ec0cb4c0 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f25c2700 1 -- 172.21.15.46:0/2787337496 >> v1:172.21.15.132:6789/0 conn(0x7f89ec0cb4c0 legacy=0x7f89ec0690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f25c2700 1 -- 172.21.15.46:0/2787337496 >> v1:172.21.15.46:6789/0 conn(0x7f89ec0af000 legacy=0x7f89ec0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f25c2700 1 -- 172.21.15.46:0/2787337496 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0ce7d0 con 0x7f89ec0697c0 2023-10-06T12:50:47.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f25c2700 1 --2- 172.21.15.46:0/2787337496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec0cf590 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009e60 tx=0x7f89e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50b2a5b0be99633 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400e070 con 0x7f89ec0697c0 2023-10-06T12:50:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e400bab0 con 0x7f89ec0697c0 2023-10-06T12:50:47.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f0dbf700 1 -- 172.21.15.46:0/2787337496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400be70 con 0x7f89ec0697c0 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f2dc3700 1 -- 172.21.15.46:0/2787337496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 msgr2=0x7f89ec0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/2787337496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec0cf590 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009e60 tx=0x7f89e4009780 comp rx=0 tx=0).stop 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f2dc3700 1 -- 172.21.15.46:0/2787337496 shutdown_connections 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/2787337496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec0cf590 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.796+0000 7f89f2dc3700 1 -- 172.21.15.46:0/2787337496 >> 172.21.15.46:0/2787337496 conn(0x7f89ec0683b0 msgr2=0x7f89ec0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:47.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.800+0000 7f89f2dc3700 1 -- 172.21.15.46:0/2787337496 shutdown_connections 2023-10-06T12:50:47.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.800+0000 7f89f2dc3700 1 -- 172.21.15.46:0/2787337496 wait complete. 2023-10-06T12:50:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.800+0000 7f89f2dc3700 1 Processor -- start 2023-10-06T12:50:47.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.800+0000 7f89f2dc3700 1 -- start start 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89ec0af000 0x7f89ec15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89ec0cb4c0 0x7f89ec15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0b31d0 con 0x7f89ec0697c0 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0c7250 con 0x7f89ec0cb4c0 2023-10-06T12:50:47.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ec0c7530 con 0x7f89ec0af000 2023-10-06T12:50:47.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47332/0 (socket says 172.21.15.46:47332) 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 -- 172.21.15.46:0/3476418836 learned_addr learned my addr 172.21.15.46:0/3476418836 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f25c2700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89ec0cb4c0 0x7f89ec15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f15c0700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89ec0af000 0x7f89ec15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 -- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89ec0af000 msgr2=0x7f89ec15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89ec0af000 0x7f89ec15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 -- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89ec0cb4c0 msgr2=0x7f89ec15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:47.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89ec0cb4c0 0x7f89ec15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 -- 172.21.15.46:0/3476418836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009a20 con 0x7f89ec0697c0 2023-10-06T12:50:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f1dc1700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c790 tx=0x7f89e8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8005b30 con 0x7f89ec0697c0 2023-10-06T12:50:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8005c90 con 0x7f89ec0697c0 2023-10-06T12:50:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ec0c7810 con 0x7f89ec0697c0 2023-10-06T12:50:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.804+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8007300 con 0x7f89ec0697c0 2023-10-06T12:50:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.808+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec15c4c0 con 0x7f89ec0697c0 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.808+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ec064590 con 0x7f89ec0697c0 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.808+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f89e801b3b0 con 0x7f89ec0697c0 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.808+0000 7f89e1ffb700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 0x7f89d0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.808+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f89e807c350 con 0x7f89ec0697c0 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.812+0000 7f89f15c0700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 0x7f89d0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.812+0000 7f89f15c0700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 0x7f89d0062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89ec16d520 tx=0x7f89dc009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:47.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.816+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f89e804fea0 con 0x7f89ec0697c0 2023-10-06T12:50:47.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:47.992+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89ec0b3310 con 0x7f89d0060670 2023-10-06T12:50:48.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.008+0000 7f89e1ffb700 1 -- 172.21.15.46:0/3476418836 <== mgr.9666 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308864 (secure 0 0 0) 0x7f89dc013420 con 0x7f89d0060670 2023-10-06T12:50:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 msgr2=0x7f89d0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 0x7f89d0062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89ec16d520 tx=0x7f89dc009040 comp rx=0 tx=0).stop 2023-10-06T12:50:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 msgr2=0x7f89ec15bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c790 tx=0x7f89e8007ea0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 shutdown_connections 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89d0060670 0x7f89d0062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89ec0cb4c0 0x7f89ec15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89ec0af000 0x7f89ec15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 --2- 172.21.15.46:0/3476418836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89ec0697c0 0x7f89ec15bf80 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 >> 172.21.15.46:0/3476418836 conn(0x7f89ec0683b0 msgr2=0x7f89ec151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 shutdown_connections 2023-10-06T12:50:48.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.016+0000 7f89f2dc3700 1 -- 172.21.15.46:0/3476418836 wait complete. 2023-10-06T12:50:48.027 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:50:48.044 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T12:50:47.788284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2471144,"kb_used_data":19112,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752503576,"statfs":{"total":773094113280,"available":770563661824,"internally_reserved":0,"allocated":19570688,"data_stored":10900924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.521239"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782893+0000","last_change":"2023-10-06T12:50:40.778335+0000","last_active":"2023-10-06T12:50:40.782893+0000","last_peered":"2023-10-06T12:50:40.782893+0000","last_clean":"2023-10-06T12:50:40.782893+0000","last_became_active":"2023-10-06T12:50:40.778062+0000","last_became_peered":"2023-10-06T12:50:40.778062+0000","last_unstale":"2023-10-06T12:50:40.782893+0000","last_undegraded":"2023-10-06T12:50:40.782893+0000","last_fullsized":"2023-10-06T12:50:40.782893+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:01.262267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786847+0000","last_change":"2023-10-06T12:50:40.782280+0000","last_active":"2023-10-06T12:50:40.786847+0000","last_peered":"2023-10-06T12:50:40.786847+0000","last_clean":"2023-10-06T12:50:40.786847+0000","last_became_active":"2023-10-06T12:50:40.781975+0000","last_became_peered":"2023-10-06T12:50:40.781975+0000","last_unstale":"2023-10-06T12:50:40.786847+0000","last_undegraded":"2023-10-06T12:50:40.786847+0000","last_fullsized":"2023-10-06T12:50:40.786847+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787332+0000","last_change":"2023-10-06T12:50:40.784263+0000","last_active":"2023-10-06T12:50:40.787332+0000","last_peered":"2023-10-06T12:50:40.787332+0000","last_clean":"2023-10-06T12:50:40.787332+0000","last_became_active":"2023-10-06T12:50:40.783981+0000","last_became_peered":"2023-10-06T12:50:40.783981+0000","last_unstale":"2023-10-06T12:50:40.787332+0000","last_undegraded":"2023-10-06T12:50:40.787332+0000","last_fullsized":"2023-10-06T12:50:40.787332+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:12:08.047276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780877+0000","last_change":"2023-10-06T12:50:40.776723+0000","last_active":"2023-10-06T12:50:40.780877+0000","last_peered":"2023-10-06T12:50:40.780877+0000","last_clean":"2023-10-06T12:50:40.780877+0000","last_became_active":"2023-10-06T12:50:40.776436+0000","last_became_peered":"2023-10-06T12:50:40.776436+0000","last_unstale":"2023-10-06T12:50:40.780877+0000","last_undegraded":"2023-10-06T12:50:40.780877+0000","last_fullsized":"2023-10-06T12:50:40.780877+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:10.144527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778358+0000","last_change":"2023-10-06T12:50:40.772646+0000","last_active":"2023-10-06T12:50:40.778358+0000","last_peered":"2023-10-06T12:50:40.778358+0000","last_clean":"2023-10-06T12:50:40.778358+0000","last_became_active":"2023-10-06T12:50:40.772150+0000","last_became_peered":"2023-10-06T12:50:40.772150+0000","last_unstale":"2023-10-06T12:50:40.778358+0000","last_undegraded":"2023-10-06T12:50:40.778358+0000","last_fullsized":"2023-10-06T12:50:40.778358+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:46.651857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783771+0000","last_change":"2023-10-06T12:50:40.775511+0000","last_active":"2023-10-06T12:50:40.783771+0000","last_peered":"2023-10-06T12:50:40.783771+0000","last_clean":"2023-10-06T12:50:40.783771+0000","last_became_active":"2023-10-06T12:50:40.775200+0000","last_became_peered":"2023-10-06T12:50:40.775200+0000","last_unstale":"2023-10-06T12:50:40.783771+0000","last_undegraded":"2023-10-06T12:50:40.783771+0000","last_fullsized":"2023-10-06T12:50:40.783771+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:21.013742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788174+0000","last_change":"2023-10-06T12:50:40.784220+0000","last_active":"2023-10-06T12:50:40.788174+0000","last_peered":"2023-10-06T12:50:40.788174+0000","last_clean":"2023-10-06T12:50:40.788174+0000","last_became_active":"2023-10-06T12:50:40.783922+0000","last_became_peered":"2023-10-06T12:50:40.783922+0000","last_unstale":"2023-10-06T12:50:40.788174+0000","last_undegraded":"2023-10-06T12:50:40.788174+0000","last_fullsized":"2023-10-06T12:50:40.788174+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:04.673251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787085+0000","last_change":"2023-10-06T12:50:40.777423+0000","last_active":"2023-10-06T12:50:40.787085+0000","last_peered":"2023-10-06T12:50:40.787085+0000","last_clean":"2023-10-06T12:50:40.787085+0000","last_became_active":"2023-10-06T12:50:40.777128+0000","last_became_peered":"2023-10-06T12:50:40.777128+0000","last_unstale":"2023-10-06T12:50:40.787085+0000","last_undegraded":"2023-10-06T12:50:40.787085+0000","last_fullsized":"2023-10-06T12:50:40.787085+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:03:51.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787707+0000","last_change":"2023-10-06T12:50:40.783209+0000","last_active":"2023-10-06T12:50:40.787707+0000","last_peered":"2023-10-06T12:50:40.787707+0000","last_clean":"2023-10-06T12:50:40.787707+0000","last_became_active":"2023-10-06T12:50:40.782919+0000","last_became_peered":"2023-10-06T12:50:40.782919+0000","last_unstale":"2023-10-06T12:50:40.787707+0000","last_undegraded":"2023-10-06T12:50:40.787707+0000","last_fullsized":"2023-10-06T12:50:40.787707+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:55.492708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787127+0000","last_change":"2023-10-06T12:50:40.777737+0000","last_active":"2023-10-06T12:50:40.787127+0000","last_peered":"2023-10-06T12:50:40.787127+0000","last_clean":"2023-10-06T12:50:40.787127+0000","last_became_active":"2023-10-06T12:50:40.777216+0000","last_became_peered":"2023-10-06T12:50:40.777216+0000","last_unstale":"2023-10-06T12:50:40.787127+0000","last_undegraded":"2023-10-06T12:50:40.787127+0000","last_fullsized":"2023-10-06T12:50:40.787127+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780082+0000","last_change":"2023-10-06T12:50:40.776224+0000","last_active":"2023-10-06T12:50:40.780082+0000","last_peered":"2023-10-06T12:50:40.780082+0000","last_clean":"2023-10-06T12:50:40.780082+0000","last_became_active":"2023-10-06T12:50:40.775933+0000","last_became_peered":"2023-10-06T12:50:40.775933+0000","last_unstale":"2023-10-06T12:50:40.780082+0000","last_undegraded":"2023-10-06T12:50:40.780082+0000","last_fullsized":"2023-10-06T12:50:40.780082+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:03.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784227+0000","last_change":"2023-10-06T12:50:40.775841+0000","last_active":"2023-10-06T12:50:40.784227+0000","last_peered":"2023-10-06T12:50:40.784227+0000","last_clean":"2023-10-06T12:50:40.784227+0000","last_became_active":"2023-10-06T12:50:40.775186+0000","last_became_peered":"2023-10-06T12:50:40.775186+0000","last_unstale":"2023-10-06T12:50:40.784227+0000","last_undegraded":"2023-10-06T12:50:40.784227+0000","last_fullsized":"2023-10-06T12:50:40.784227+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:54.589070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.777761+0000","last_change":"2023-10-06T12:50:40.773781+0000","last_active":"2023-10-06T12:50:40.777761+0000","last_peered":"2023-10-06T12:50:40.777761+0000","last_clean":"2023-10-06T12:50:40.777761+0000","last_became_active":"2023-10-06T12:50:40.773449+0000","last_became_peered":"2023-10-06T12:50:40.773449+0000","last_unstale":"2023-10-06T12:50:40.777761+0000","last_undegraded":"2023-10-06T12:50:40.777761+0000","last_fullsized":"2023-10-06T12:50:40.777761+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:56.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788121+0000","last_change":"2023-10-06T12:50:40.782510+0000","last_active":"2023-10-06T12:50:40.788121+0000","last_peered":"2023-10-06T12:50:40.788121+0000","last_clean":"2023-10-06T12:50:40.788121+0000","last_became_active":"2023-10-06T12:50:40.782167+0000","last_became_peered":"2023-10-06T12:50:40.782167+0000","last_unstale":"2023-10-06T12:50:40.788121+0000","last_undegraded":"2023-10-06T12:50:40.788121+0000","last_fullsized":"2023-10-06T12:50:40.788121+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:23.655289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780305+0000","last_change":"2023-10-06T12:50:40.778146+0000","last_active":"2023-10-06T12:50:40.780305+0000","last_peered":"2023-10-06T12:50:40.780305+0000","last_clean":"2023-10-06T12:50:40.780305+0000","last_became_active":"2023-10-06T12:50:40.777837+0000","last_became_peered":"2023-10-06T12:50:40.777837+0000","last_unstale":"2023-10-06T12:50:40.780305+0000","last_undegraded":"2023-10-06T12:50:40.780305+0000","last_fullsized":"2023-10-06T12:50:40.780305+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:51:17.261560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781264+0000","last_change":"2023-10-06T12:50:40.776761+0000","last_active":"2023-10-06T12:50:40.781264+0000","last_peered":"2023-10-06T12:50:40.781264+0000","last_clean":"2023-10-06T12:50:40.781264+0000","last_became_active":"2023-10-06T12:50:40.776458+0000","last_became_peered":"2023-10-06T12:50:40.776458+0000","last_unstale":"2023-10-06T12:50:40.781264+0000","last_undegraded":"2023-10-06T12:50:40.781264+0000","last_fullsized":"2023-10-06T12:50:40.781264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:58:56.899974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787157+0000","last_change":"2023-10-06T12:50:40.777985+0000","last_active":"2023-10-06T12:50:40.787157+0000","last_peered":"2023-10-06T12:50:40.787157+0000","last_clean":"2023-10-06T12:50:40.787157+0000","last_became_active":"2023-10-06T12:50:40.777628+0000","last_became_peered":"2023-10-06T12:50:40.777628+0000","last_unstale":"2023-10-06T12:50:40.787157+0000","last_undegraded":"2023-10-06T12:50:40.787157+0000","last_fullsized":"2023-10-06T12:50:40.787157+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:22:13.682136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783869+0000","last_change":"2023-10-06T12:50:40.779890+0000","last_active":"2023-10-06T12:50:40.783869+0000","last_peered":"2023-10-06T12:50:40.783869+0000","last_clean":"2023-10-06T12:50:40.783869+0000","last_became_active":"2023-10-06T12:50:40.777022+0000","last_became_peered":"2023-10-06T12:50:40.777022+0000","last_unstale":"2023-10-06T12:50:40.783869+0000","last_undegraded":"2023-10-06T12:50:40.783869+0000","last_fullsized":"2023-10-06T12:50:40.783869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:16.774125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784780+0000","last_change":"2023-10-06T12:50:40.778707+0000","last_active":"2023-10-06T12:50:40.784780+0000","last_peered":"2023-10-06T12:50:40.784780+0000","last_clean":"2023-10-06T12:50:40.784780+0000","last_became_active":"2023-10-06T12:50:40.778236+0000","last_became_peered":"2023-10-06T12:50:40.778236+0000","last_unstale":"2023-10-06T12:50:40.784780+0000","last_undegraded":"2023-10-06T12:50:40.784780+0000","last_fullsized":"2023-10-06T12:50:40.784780+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:19:33.126754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782866+0000","last_change":"2023-10-06T12:50:40.780014+0000","last_active":"2023-10-06T12:50:40.782866+0000","last_peered":"2023-10-06T12:50:40.782866+0000","last_clean":"2023-10-06T12:50:40.782866+0000","last_became_active":"2023-10-06T12:50:40.779620+0000","last_became_peered":"2023-10-06T12:50:40.779620+0000","last_unstale":"2023-10-06T12:50:40.782866+0000","last_undegraded":"2023-10-06T12:50:40.782866+0000","last_fullsized":"2023-10-06T12:50:40.782866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:23.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787173+0000","last_change":"2023-10-06T12:50:40.778720+0000","last_active":"2023-10-06T12:50:40.787173+0000","last_peered":"2023-10-06T12:50:40.787173+0000","last_clean":"2023-10-06T12:50:40.787173+0000","last_became_active":"2023-10-06T12:50:40.778411+0000","last_became_peered":"2023-10-06T12:50:40.778411+0000","last_unstale":"2023-10-06T12:50:40.787173+0000","last_undegraded":"2023-10-06T12:50:40.787173+0000","last_fullsized":"2023-10-06T12:50:40.787173+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:47:54.960186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786788+0000","last_change":"2023-10-06T12:50:40.781834+0000","last_active":"2023-10-06T12:50:40.786788+0000","last_peered":"2023-10-06T12:50:40.786788+0000","last_clean":"2023-10-06T12:50:40.786788+0000","last_became_active":"2023-10-06T12:50:40.781535+0000","last_became_peered":"2023-10-06T12:50:40.781535+0000","last_unstale":"2023-10-06T12:50:40.786788+0000","last_undegraded":"2023-10-06T12:50:40.786788+0000","last_fullsized":"2023-10-06T12:50:40.786788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:38:17.097688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.779397+0000","last_change":"2023-10-06T12:50:40.777085+0000","last_active":"2023-10-06T12:50:40.779397+0000","last_peered":"2023-10-06T12:50:40.779397+0000","last_clean":"2023-10-06T12:50:40.779397+0000","last_became_active":"2023-10-06T12:50:40.776810+0000","last_became_peered":"2023-10-06T12:50:40.776810+0000","last_unstale":"2023-10-06T12:50:40.779397+0000","last_undegraded":"2023-10-06T12:50:40.779397+0000","last_fullsized":"2023-10-06T12:50:40.779397+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:43:54.696710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781245+0000","last_change":"2023-10-06T12:50:40.774098+0000","last_active":"2023-10-06T12:50:40.781245+0000","last_peered":"2023-10-06T12:50:40.781245+0000","last_clean":"2023-10-06T12:50:40.781245+0000","last_became_active":"2023-10-06T12:50:40.773818+0000","last_became_peered":"2023-10-06T12:50:40.773818+0000","last_unstale":"2023-10-06T12:50:40.781245+0000","last_undegraded":"2023-10-06T12:50:40.781245+0000","last_fullsized":"2023-10-06T12:50:40.781245+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:34.296097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.782123+0000","last_change":"2023-10-06T12:50:40.776905+0000","last_active":"2023-10-06T12:50:40.782123+0000","last_peered":"2023-10-06T12:50:40.782123+0000","last_clean":"2023-10-06T12:50:40.782123+0000","last_became_active":"2023-10-06T12:50:40.776601+0000","last_became_peered":"2023-10-06T12:50:40.776601+0000","last_unstale":"2023-10-06T12:50:40.782123+0000","last_undegraded":"2023-10-06T12:50:40.782123+0000","last_fullsized":"2023-10-06T12:50:40.782123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:34:34.296097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787208+0000","last_change":"2023-10-06T12:50:40.780478+0000","last_active":"2023-10-06T12:50:40.787208+0000","last_peered":"2023-10-06T12:50:40.787208+0000","last_clean":"2023-10-06T12:50:40.787208+0000","last_became_active":"2023-10-06T12:50:40.779240+0000","last_became_peered":"2023-10-06T12:50:40.779240+0000","last_unstale":"2023-10-06T12:50:40.787208+0000","last_undegraded":"2023-10-06T12:50:40.787208+0000","last_fullsized":"2023-10-06T12:50:40.787208+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:04.761816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786967+0000","last_change":"2023-10-06T12:50:40.781962+0000","last_active":"2023-10-06T12:50:40.786967+0000","last_peered":"2023-10-06T12:50:40.786967+0000","last_clean":"2023-10-06T12:50:40.786967+0000","last_became_active":"2023-10-06T12:50:40.781663+0000","last_became_peered":"2023-10-06T12:50:40.781663+0000","last_unstale":"2023-10-06T12:50:40.786967+0000","last_undegraded":"2023-10-06T12:50:40.786967+0000","last_fullsized":"2023-10-06T12:50:40.786967+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:11:51.651199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788289+0000","last_change":"2023-10-06T12:50:40.780354+0000","last_active":"2023-10-06T12:50:40.788289+0000","last_peered":"2023-10-06T12:50:40.788289+0000","last_clean":"2023-10-06T12:50:40.788289+0000","last_became_active":"2023-10-06T12:50:40.779798+0000","last_became_peered":"2023-10-06T12:50:40.779798+0000","last_unstale":"2023-10-06T12:50:40.788289+0000","last_undegraded":"2023-10-06T12:50:40.788289+0000","last_fullsized":"2023-10-06T12:50:40.788289+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:10.144527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786647+0000","last_change":"2023-10-06T12:50:40.779059+0000","last_active":"2023-10-06T12:50:40.786647+0000","last_peered":"2023-10-06T12:50:40.786647+0000","last_clean":"2023-10-06T12:50:40.786647+0000","last_became_active":"2023-10-06T12:50:40.778788+0000","last_became_peered":"2023-10-06T12:50:40.778788+0000","last_unstale":"2023-10-06T12:50:40.786647+0000","last_undegraded":"2023-10-06T12:50:40.786647+0000","last_fullsized":"2023-10-06T12:50:40.786647+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:29:18.444148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.783512+0000","last_change":"2023-10-06T12:50:40.781633+0000","last_active":"2023-10-06T12:50:40.783512+0000","last_peered":"2023-10-06T12:50:40.783512+0000","last_clean":"2023-10-06T12:50:40.783512+0000","last_became_active":"2023-10-06T12:50:40.781346+0000","last_became_peered":"2023-10-06T12:50:40.781346+0000","last_unstale":"2023-10-06T12:50:40.783512+0000","last_undegraded":"2023-10-06T12:50:40.783512+0000","last_fullsized":"2023-10-06T12:50:40.783512+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:41.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784789+0000","last_change":"2023-10-06T12:50:40.778927+0000","last_active":"2023-10-06T12:50:40.784789+0000","last_peered":"2023-10-06T12:50:40.784789+0000","last_clean":"2023-10-06T12:50:40.784789+0000","last_became_active":"2023-10-06T12:50:40.778552+0000","last_became_peered":"2023-10-06T12:50:40.778552+0000","last_unstale":"2023-10-06T12:50:40.784789+0000","last_undegraded":"2023-10-06T12:50:40.784789+0000","last_fullsized":"2023-10-06T12:50:40.784789+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:22.757377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788111+0000","last_change":"2023-10-06T12:50:40.784137+0000","last_active":"2023-10-06T12:50:40.788111+0000","last_peered":"2023-10-06T12:50:40.788111+0000","last_clean":"2023-10-06T12:50:40.788111+0000","last_became_active":"2023-10-06T12:50:40.783849+0000","last_became_peered":"2023-10-06T12:50:40.783849+0000","last_unstale":"2023-10-06T12:50:40.788111+0000","last_undegraded":"2023-10-06T12:50:40.788111+0000","last_fullsized":"2023-10-06T12:50:40.788111+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:05.558031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786940+0000","last_change":"2023-10-06T12:50:40.780586+0000","last_active":"2023-10-06T12:50:40.786940+0000","last_peered":"2023-10-06T12:50:40.786940+0000","last_clean":"2023-10-06T12:50:40.786940+0000","last_became_active":"2023-10-06T12:50:40.776048+0000","last_became_peered":"2023-10-06T12:50:40.776048+0000","last_unstale":"2023-10-06T12:50:40.786940+0000","last_undegraded":"2023-10-06T12:50:40.786940+0000","last_fullsized":"2023-10-06T12:50:40.786940+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:04.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"182'4","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.316091+0000","last_change":"2023-10-06T12:50:40.287857+0000","last_active":"2023-10-06T12:50:41.316091+0000","last_peered":"2023-10-06T12:50:41.316091+0000","last_clean":"2023-10-06T12:50:41.316091+0000","last_became_active":"2023-10-06T12:50:40.287578+0000","last_became_peered":"2023-10-06T12:50:40.287578+0000","last_unstale":"2023-10-06T12:50:41.316091+0000","last_undegraded":"2023-10-06T12:50:41.316091+0000","last_fullsized":"2023-10-06T12:50:41.316091+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:37.662922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780840+0000","last_change":"2023-10-06T12:50:40.777280+0000","last_active":"2023-10-06T12:50:40.780840+0000","last_peered":"2023-10-06T12:50:40.780840+0000","last_clean":"2023-10-06T12:50:40.780840+0000","last_became_active":"2023-10-06T12:50:40.777010+0000","last_became_peered":"2023-10-06T12:50:40.777010+0000","last_unstale":"2023-10-06T12:50:40.780840+0000","last_undegraded":"2023-10-06T12:50:40.780840+0000","last_fullsized":"2023-10-06T12:50:40.780840+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:17.816213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315724+0000","last_change":"2023-10-06T12:50:40.298134+0000","last_active":"2023-10-06T12:50:41.315724+0000","last_peered":"2023-10-06T12:50:41.315724+0000","last_clean":"2023-10-06T12:50:41.315724+0000","last_became_active":"2023-10-06T12:50:40.297844+0000","last_became_peered":"2023-10-06T12:50:40.297844+0000","last_unstale":"2023-10-06T12:50:41.315724+0000","last_undegraded":"2023-10-06T12:50:41.315724+0000","last_fullsized":"2023-10-06T12:50:41.315724+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:30:10.127709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778588+0000","last_change":"2023-10-06T12:50:40.776226+0000","last_active":"2023-10-06T12:50:40.778588+0000","last_peered":"2023-10-06T12:50:40.778588+0000","last_clean":"2023-10-06T12:50:40.778588+0000","last_became_active":"2023-10-06T12:50:40.775922+0000","last_became_peered":"2023-10-06T12:50:40.775922+0000","last_unstale":"2023-10-06T12:50:40.778588+0000","last_undegraded":"2023-10-06T12:50:40.778588+0000","last_fullsized":"2023-10-06T12:50:40.778588+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:05:23.681463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.310625+0000","last_change":"2023-10-06T12:50:40.297474+0000","last_active":"2023-10-06T12:50:41.310625+0000","last_peered":"2023-10-06T12:50:41.310625+0000","last_clean":"2023-10-06T12:50:41.310625+0000","last_became_active":"2023-10-06T12:50:40.297156+0000","last_became_peered":"2023-10-06T12:50:40.297156+0000","last_unstale":"2023-10-06T12:50:41.310625+0000","last_undegraded":"2023-10-06T12:50:41.310625+0000","last_fullsized":"2023-10-06T12:50:41.310625+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:54:36.044092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784188+0000","last_change":"2023-10-06T12:50:40.775470+0000","last_active":"2023-10-06T12:50:40.784188+0000","last_peered":"2023-10-06T12:50:40.784188+0000","last_clean":"2023-10-06T12:50:40.784188+0000","last_became_active":"2023-10-06T12:50:40.775120+0000","last_became_peered":"2023-10-06T12:50:40.775120+0000","last_unstale":"2023-10-06T12:50:40.784188+0000","last_undegraded":"2023-10-06T12:50:40.784188+0000","last_fullsized":"2023-10-06T12:50:40.784188+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:59:13.625755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.309389+0000","last_change":"2023-10-06T12:50:40.295385+0000","last_active":"2023-10-06T12:50:41.309389+0000","last_peered":"2023-10-06T12:50:41.309389+0000","last_clean":"2023-10-06T12:50:41.309389+0000","last_became_active":"2023-10-06T12:50:40.295057+0000","last_became_peered":"2023-10-06T12:50:40.295057+0000","last_unstale":"2023-10-06T12:50:41.309389+0000","last_undegraded":"2023-10-06T12:50:41.309389+0000","last_fullsized":"2023-10-06T12:50:41.309389+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:02.099185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787738+0000","last_change":"2023-10-06T12:50:40.783444+0000","last_active":"2023-10-06T12:50:40.787738+0000","last_peered":"2023-10-06T12:50:40.787738+0000","last_clean":"2023-10-06T12:50:40.787738+0000","last_became_active":"2023-10-06T12:50:40.780663+0000","last_became_peered":"2023-10-06T12:50:40.780663+0000","last_unstale":"2023-10-06T12:50:40.787738+0000","last_undegraded":"2023-10-06T12:50:40.787738+0000","last_fullsized":"2023-10-06T12:50:40.787738+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:57.388724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315299+0000","last_change":"2023-10-06T12:50:40.299371+0000","last_active":"2023-10-06T12:50:41.315299+0000","last_peered":"2023-10-06T12:50:41.315299+0000","last_clean":"2023-10-06T12:50:41.315299+0000","last_became_active":"2023-10-06T12:50:40.299059+0000","last_became_peered":"2023-10-06T12:50:40.299059+0000","last_unstale":"2023-10-06T12:50:41.315299+0000","last_undegraded":"2023-10-06T12:50:41.315299+0000","last_fullsized":"2023-10-06T12:50:41.315299+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:07:13.496718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.781463+0000","last_change":"2023-10-06T12:50:40.780056+0000","last_active":"2023-10-06T12:50:40.781463+0000","last_peered":"2023-10-06T12:50:40.781463+0000","last_clean":"2023-10-06T12:50:40.781463+0000","last_became_active":"2023-10-06T12:50:40.779712+0000","last_became_peered":"2023-10-06T12:50:40.779712+0000","last_unstale":"2023-10-06T12:50:40.781463+0000","last_undegraded":"2023-10-06T12:50:40.781463+0000","last_fullsized":"2023-10-06T12:50:40.781463+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:19.938433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314973+0000","last_change":"2023-10-06T12:50:40.292623+0000","last_active":"2023-10-06T12:50:41.314973+0000","last_peered":"2023-10-06T12:50:41.314973+0000","last_clean":"2023-10-06T12:50:41.314973+0000","last_became_active":"2023-10-06T12:50:40.291297+0000","last_became_peered":"2023-10-06T12:50:40.291297+0000","last_unstale":"2023-10-06T12:50:41.314973+0000","last_undegraded":"2023-10-06T12:50:41.314973+0000","last_fullsized":"2023-10-06T12:50:41.314973+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:44:56.537480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780587+0000","last_change":"2023-10-06T12:50:40.777001+0000","last_active":"2023-10-06T12:50:40.780587+0000","last_peered":"2023-10-06T12:50:40.780587+0000","last_clean":"2023-10-06T12:50:40.780587+0000","last_became_active":"2023-10-06T12:50:40.776681+0000","last_became_peered":"2023-10-06T12:50:40.776681+0000","last_unstale":"2023-10-06T12:50:40.780587+0000","last_undegraded":"2023-10-06T12:50:40.780587+0000","last_fullsized":"2023-10-06T12:50:40.780587+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:04.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766443+0000","last_change":"2023-10-06T12:50:40.299038+0000","last_active":"2023-10-06T12:50:40.766443+0000","last_peered":"2023-10-06T12:50:40.766443+0000","last_clean":"2023-10-06T12:50:40.766443+0000","last_became_active":"2023-10-06T12:50:40.298763+0000","last_became_peered":"2023-10-06T12:50:40.298763+0000","last_unstale":"2023-10-06T12:50:40.766443+0000","last_undegraded":"2023-10-06T12:50:40.766443+0000","last_fullsized":"2023-10-06T12:50:40.766443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:54:23.394579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.773231+0000","last_change":"2023-10-06T12:50:40.767553+0000","last_active":"2023-10-06T12:50:40.773231+0000","last_peered":"2023-10-06T12:50:40.773231+0000","last_clean":"2023-10-06T12:50:40.773231+0000","last_became_active":"2023-10-06T12:50:40.767227+0000","last_became_peered":"2023-10-06T12:50:40.767227+0000","last_unstale":"2023-10-06T12:50:40.773231+0000","last_undegraded":"2023-10-06T12:50:40.773231+0000","last_fullsized":"2023-10-06T12:50:40.773231+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:17.816213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770164+0000","last_change":"2023-10-06T12:50:40.296743+0000","last_active":"2023-10-06T12:50:40.770164+0000","last_peered":"2023-10-06T12:50:40.770164+0000","last_clean":"2023-10-06T12:50:40.770164+0000","last_became_active":"2023-10-06T12:50:40.296234+0000","last_became_peered":"2023-10-06T12:50:40.296234+0000","last_unstale":"2023-10-06T12:50:40.770164+0000","last_undegraded":"2023-10-06T12:50:40.770164+0000","last_fullsized":"2023-10-06T12:50:40.770164+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:13:58.004423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788211+0000","last_change":"2023-10-06T12:50:40.785098+0000","last_active":"2023-10-06T12:50:40.788211+0000","last_peered":"2023-10-06T12:50:40.788211+0000","last_clean":"2023-10-06T12:50:40.788211+0000","last_became_active":"2023-10-06T12:50:40.784020+0000","last_became_peered":"2023-10-06T12:50:40.784020+0000","last_unstale":"2023-10-06T12:50:40.788211+0000","last_undegraded":"2023-10-06T12:50:40.788211+0000","last_fullsized":"2023-10-06T12:50:40.788211+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:25:07.224298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.313890+0000","last_change":"2023-10-06T12:50:40.301698+0000","last_active":"2023-10-06T12:50:41.313890+0000","last_peered":"2023-10-06T12:50:41.313890+0000","last_clean":"2023-10-06T12:50:41.313890+0000","last_became_active":"2023-10-06T12:50:40.301185+0000","last_became_peered":"2023-10-06T12:50:40.301185+0000","last_unstale":"2023-10-06T12:50:41.313890+0000","last_undegraded":"2023-10-06T12:50:41.313890+0000","last_fullsized":"2023-10-06T12:50:41.313890+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:22.508131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786818+0000","last_change":"2023-10-06T12:50:40.782490+0000","last_active":"2023-10-06T12:50:40.786818+0000","last_peered":"2023-10-06T12:50:40.786818+0000","last_clean":"2023-10-06T12:50:40.786818+0000","last_became_active":"2023-10-06T12:50:40.782194+0000","last_became_peered":"2023-10-06T12:50:40.782194+0000","last_unstale":"2023-10-06T12:50:40.786818+0000","last_undegraded":"2023-10-06T12:50:40.786818+0000","last_fullsized":"2023-10-06T12:50:40.786818+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:31.559045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315371+0000","last_change":"2023-10-06T12:50:40.299353+0000","last_active":"2023-10-06T12:50:41.315371+0000","last_peered":"2023-10-06T12:50:41.315371+0000","last_clean":"2023-10-06T12:50:41.315371+0000","last_became_active":"2023-10-06T12:50:40.298708+0000","last_became_peered":"2023-10-06T12:50:40.298708+0000","last_unstale":"2023-10-06T12:50:41.315371+0000","last_undegraded":"2023-10-06T12:50:41.315371+0000","last_fullsized":"2023-10-06T12:50:41.315371+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:56.515951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.772550+0000","last_change":"2023-10-06T12:50:40.768348+0000","last_active":"2023-10-06T12:50:40.772550+0000","last_peered":"2023-10-06T12:50:40.772550+0000","last_clean":"2023-10-06T12:50:40.772550+0000","last_became_active":"2023-10-06T12:50:40.768040+0000","last_became_peered":"2023-10-06T12:50:40.768040+0000","last_unstale":"2023-10-06T12:50:40.772550+0000","last_undegraded":"2023-10-06T12:50:40.772550+0000","last_fullsized":"2023-10-06T12:50:40.772550+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:44.895817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"182'1","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314329+0000","last_change":"2023-10-06T12:50:40.288872+0000","last_active":"2023-10-06T12:50:41.314329+0000","last_peered":"2023-10-06T12:50:41.314329+0000","last_clean":"2023-10-06T12:50:41.314329+0000","last_became_active":"2023-10-06T12:50:40.288257+0000","last_became_peered":"2023-10-06T12:50:40.288257+0000","last_unstale":"2023-10-06T12:50:41.314329+0000","last_undegraded":"2023-10-06T12:50:41.314329+0000","last_fullsized":"2023-10-06T12:50:41.314329+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:59:35.394664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787344+0000","last_change":"2023-10-06T12:50:40.781308+0000","last_active":"2023-10-06T12:50:40.787344+0000","last_peered":"2023-10-06T12:50:40.787344+0000","last_clean":"2023-10-06T12:50:40.787344+0000","last_became_active":"2023-10-06T12:50:40.780957+0000","last_became_peered":"2023-10-06T12:50:40.780957+0000","last_unstale":"2023-10-06T12:50:40.787344+0000","last_undegraded":"2023-10-06T12:50:40.787344+0000","last_fullsized":"2023-10-06T12:50:40.787344+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:22:18.424675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770900+0000","last_change":"2023-10-06T12:50:40.300356+0000","last_active":"2023-10-06T12:50:40.770900+0000","last_peered":"2023-10-06T12:50:40.770900+0000","last_clean":"2023-10-06T12:50:40.770900+0000","last_became_active":"2023-10-06T12:50:40.299966+0000","last_became_peered":"2023-10-06T12:50:40.299966+0000","last_unstale":"2023-10-06T12:50:40.770900+0000","last_undegraded":"2023-10-06T12:50:40.770900+0000","last_fullsized":"2023-10-06T12:50:40.770900+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:33:54.397929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788270+0000","last_change":"2023-10-06T12:50:40.780155+0000","last_active":"2023-10-06T12:50:40.788270+0000","last_peered":"2023-10-06T12:50:40.788270+0000","last_clean":"2023-10-06T12:50:40.788270+0000","last_became_active":"2023-10-06T12:50:40.779416+0000","last_became_peered":"2023-10-06T12:50:40.779416+0000","last_unstale":"2023-10-06T12:50:40.788270+0000","last_undegraded":"2023-10-06T12:50:40.788270+0000","last_fullsized":"2023-10-06T12:50:40.788270+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:02.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315170+0000","last_change":"2023-10-06T12:50:40.291256+0000","last_active":"2023-10-06T12:50:41.315170+0000","last_peered":"2023-10-06T12:50:41.315170+0000","last_clean":"2023-10-06T12:50:41.315170+0000","last_became_active":"2023-10-06T12:50:40.290838+0000","last_became_peered":"2023-10-06T12:50:40.290838+0000","last_unstale":"2023-10-06T12:50:41.315170+0000","last_undegraded":"2023-10-06T12:50:41.315170+0000","last_fullsized":"2023-10-06T12:50:41.315170+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:53.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784462+0000","last_change":"2023-10-06T12:50:40.778770+0000","last_active":"2023-10-06T12:50:40.784462+0000","last_peered":"2023-10-06T12:50:40.784462+0000","last_clean":"2023-10-06T12:50:40.784462+0000","last_became_active":"2023-10-06T12:50:40.776268+0000","last_became_peered":"2023-10-06T12:50:40.776268+0000","last_unstale":"2023-10-06T12:50:40.784462+0000","last_undegraded":"2023-10-06T12:50:40.784462+0000","last_fullsized":"2023-10-06T12:50:40.784462+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:20:55.856953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763801+0000","last_change":"2023-10-06T12:50:40.288657+0000","last_active":"2023-10-06T12:50:40.763801+0000","last_peered":"2023-10-06T12:50:40.763801+0000","last_clean":"2023-10-06T12:50:40.763801+0000","last_became_active":"2023-10-06T12:50:40.288299+0000","last_became_peered":"2023-10-06T12:50:40.288299+0000","last_unstale":"2023-10-06T12:50:40.763801+0000","last_undegraded":"2023-10-06T12:50:40.763801+0000","last_fullsized":"2023-10-06T12:50:40.763801+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:35:45.064666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.778509+0000","last_change":"2023-10-06T12:50:40.774871+0000","last_active":"2023-10-06T12:50:40.778509+0000","last_peered":"2023-10-06T12:50:40.778509+0000","last_clean":"2023-10-06T12:50:40.778509+0000","last_became_active":"2023-10-06T12:50:40.774591+0000","last_became_peered":"2023-10-06T12:50:40.774591+0000","last_unstale":"2023-10-06T12:50:40.778509+0000","last_undegraded":"2023-10-06T12:50:40.778509+0000","last_fullsized":"2023-10-06T12:50:40.778509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:47:54.108194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766535+0000","last_change":"2023-10-06T12:50:40.295842+0000","last_active":"2023-10-06T12:50:40.766535+0000","last_peered":"2023-10-06T12:50:40.766535+0000","last_clean":"2023-10-06T12:50:40.766535+0000","last_became_active":"2023-10-06T12:50:40.295234+0000","last_became_peered":"2023-10-06T12:50:40.295234+0000","last_unstale":"2023-10-06T12:50:40.766535+0000","last_undegraded":"2023-10-06T12:50:40.766535+0000","last_fullsized":"2023-10-06T12:50:40.766535+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:15.759155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786930+0000","last_change":"2023-10-06T12:50:40.784087+0000","last_active":"2023-10-06T12:50:40.786930+0000","last_peered":"2023-10-06T12:50:40.786930+0000","last_clean":"2023-10-06T12:50:40.786930+0000","last_became_active":"2023-10-06T12:50:40.783800+0000","last_became_peered":"2023-10-06T12:50:40.783800+0000","last_unstale":"2023-10-06T12:50:40.786930+0000","last_undegraded":"2023-10-06T12:50:40.786930+0000","last_fullsized":"2023-10-06T12:50:40.786930+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:45:38.677892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766706+0000","last_change":"2023-10-06T12:50:40.290848+0000","last_active":"2023-10-06T12:50:40.766706+0000","last_peered":"2023-10-06T12:50:40.766706+0000","last_clean":"2023-10-06T12:50:40.766706+0000","last_became_active":"2023-10-06T12:50:40.290513+0000","last_became_peered":"2023-10-06T12:50:40.290513+0000","last_unstale":"2023-10-06T12:50:40.766706+0000","last_undegraded":"2023-10-06T12:50:40.766706+0000","last_fullsized":"2023-10-06T12:50:40.766706+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:07:02.099185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784763+0000","last_change":"2023-10-06T12:50:40.778422+0000","last_active":"2023-10-06T12:50:40.784763+0000","last_peered":"2023-10-06T12:50:40.784763+0000","last_clean":"2023-10-06T12:50:40.784763+0000","last_became_active":"2023-10-06T12:50:40.778133+0000","last_became_peered":"2023-10-06T12:50:40.778133+0000","last_unstale":"2023-10-06T12:50:40.784763+0000","last_undegraded":"2023-10-06T12:50:40.784763+0000","last_fullsized":"2023-10-06T12:50:40.784763+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:52:44.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.314963+0000","last_change":"2023-10-06T12:50:40.285393+0000","last_active":"2023-10-06T12:50:41.314963+0000","last_peered":"2023-10-06T12:50:41.314963+0000","last_clean":"2023-10-06T12:50:41.314963+0000","last_became_active":"2023-10-06T12:50:40.285070+0000","last_became_peered":"2023-10-06T12:50:40.285070+0000","last_unstale":"2023-10-06T12:50:41.314963+0000","last_undegraded":"2023-10-06T12:50:41.314963+0000","last_fullsized":"2023-10-06T12:50:41.314963+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:25.037833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787796+0000","last_change":"2023-10-06T12:50:40.782278+0000","last_active":"2023-10-06T12:50:40.787796+0000","last_peered":"2023-10-06T12:50:40.787796+0000","last_clean":"2023-10-06T12:50:40.787796+0000","last_became_active":"2023-10-06T12:50:40.781973+0000","last_became_peered":"2023-10-06T12:50:40.781973+0000","last_unstale":"2023-10-06T12:50:40.787796+0000","last_undegraded":"2023-10-06T12:50:40.787796+0000","last_fullsized":"2023-10-06T12:50:40.787796+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:34.886992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.764679+0000","last_change":"2023-10-06T12:50:40.299285+0000","last_active":"2023-10-06T12:50:40.764679+0000","last_peered":"2023-10-06T12:50:40.764679+0000","last_clean":"2023-10-06T12:50:40.764679+0000","last_became_active":"2023-10-06T12:50:40.298903+0000","last_became_peered":"2023-10-06T12:50:40.298903+0000","last_unstale":"2023-10-06T12:50:40.764679+0000","last_undegraded":"2023-10-06T12:50:40.764679+0000","last_fullsized":"2023-10-06T12:50:40.764679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:03.443330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788978+0000","last_change":"2023-10-06T12:50:40.781454+0000","last_active":"2023-10-06T12:50:40.788978+0000","last_peered":"2023-10-06T12:50:40.788978+0000","last_clean":"2023-10-06T12:50:40.788978+0000","last_became_active":"2023-10-06T12:50:40.781107+0000","last_became_peered":"2023-10-06T12:50:40.781107+0000","last_unstale":"2023-10-06T12:50:40.788978+0000","last_undegraded":"2023-10-06T12:50:40.788978+0000","last_fullsized":"2023-10-06T12:50:40.788978+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:46.760142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763993+0000","last_change":"2023-10-06T12:50:40.290074+0000","last_active":"2023-10-06T12:50:40.763993+0000","last_peered":"2023-10-06T12:50:40.763993+0000","last_clean":"2023-10-06T12:50:40.763993+0000","last_became_active":"2023-10-06T12:50:40.289751+0000","last_became_peered":"2023-10-06T12:50:40.289751+0000","last_unstale":"2023-10-06T12:50:40.763993+0000","last_undegraded":"2023-10-06T12:50:40.763993+0000","last_fullsized":"2023-10-06T12:50:40.763993+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:31:52.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787766+0000","last_change":"2023-10-06T12:50:40.783197+0000","last_active":"2023-10-06T12:50:40.787766+0000","last_peered":"2023-10-06T12:50:40.787766+0000","last_clean":"2023-10-06T12:50:40.787766+0000","last_became_active":"2023-10-06T12:50:40.782606+0000","last_became_peered":"2023-10-06T12:50:40.782606+0000","last_unstale":"2023-10-06T12:50:40.787766+0000","last_undegraded":"2023-10-06T12:50:40.787766+0000","last_fullsized":"2023-10-06T12:50:40.787766+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:32:51.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.765793+0000","last_change":"2023-10-06T12:50:40.295945+0000","last_active":"2023-10-06T12:50:40.765793+0000","last_peered":"2023-10-06T12:50:40.765793+0000","last_clean":"2023-10-06T12:50:40.765793+0000","last_became_active":"2023-10-06T12:50:40.295659+0000","last_became_peered":"2023-10-06T12:50:40.295659+0000","last_unstale":"2023-10-06T12:50:40.765793+0000","last_undegraded":"2023-10-06T12:50:40.765793+0000","last_fullsized":"2023-10-06T12:50:40.765793+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:18:39.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.785785+0000","last_change":"2023-10-06T12:50:40.780760+0000","last_active":"2023-10-06T12:50:40.785785+0000","last_peered":"2023-10-06T12:50:40.785785+0000","last_clean":"2023-10-06T12:50:40.785785+0000","last_became_active":"2023-10-06T12:50:40.780384+0000","last_became_peered":"2023-10-06T12:50:40.780384+0000","last_unstale":"2023-10-06T12:50:40.785785+0000","last_undegraded":"2023-10-06T12:50:40.785785+0000","last_fullsized":"2023-10-06T12:50:40.785785+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:16:02.271089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.761290+0000","last_change":"2023-10-06T12:50:40.280675+0000","last_active":"2023-10-06T12:50:40.761290+0000","last_peered":"2023-10-06T12:50:40.761290+0000","last_clean":"2023-10-06T12:50:40.761290+0000","last_became_active":"2023-10-06T12:50:40.280017+0000","last_became_peered":"2023-10-06T12:50:40.280017+0000","last_unstale":"2023-10-06T12:50:40.761290+0000","last_undegraded":"2023-10-06T12:50:40.761290+0000","last_fullsized":"2023-10-06T12:50:40.761290+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:53.574100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.784296+0000","last_change":"2023-10-06T12:50:40.776901+0000","last_active":"2023-10-06T12:50:40.784296+0000","last_peered":"2023-10-06T12:50:40.784296+0000","last_clean":"2023-10-06T12:50:40.784296+0000","last_became_active":"2023-10-06T12:50:40.776620+0000","last_became_peered":"2023-10-06T12:50:40.776620+0000","last_unstale":"2023-10-06T12:50:40.784296+0000","last_undegraded":"2023-10-06T12:50:40.784296+0000","last_fullsized":"2023-10-06T12:50:40.784296+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:45.504866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766443+0000","last_change":"2023-10-06T12:50:40.299167+0000","last_active":"2023-10-06T12:50:40.766443+0000","last_peered":"2023-10-06T12:50:40.766443+0000","last_clean":"2023-10-06T12:50:40.766443+0000","last_became_active":"2023-10-06T12:50:40.298874+0000","last_became_peered":"2023-10-06T12:50:40.298874+0000","last_unstale":"2023-10-06T12:50:40.766443+0000","last_undegraded":"2023-10-06T12:50:40.766443+0000","last_fullsized":"2023-10-06T12:50:40.766443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:46.170536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787135+0000","last_change":"2023-10-06T12:50:40.779002+0000","last_active":"2023-10-06T12:50:40.787135+0000","last_peered":"2023-10-06T12:50:40.787135+0000","last_clean":"2023-10-06T12:50:40.787135+0000","last_became_active":"2023-10-06T12:50:40.778709+0000","last_became_peered":"2023-10-06T12:50:40.778709+0000","last_unstale":"2023-10-06T12:50:40.787135+0000","last_undegraded":"2023-10-06T12:50:40.787135+0000","last_fullsized":"2023-10-06T12:50:40.787135+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:04:17.165460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.761052+0000","last_change":"2023-10-06T12:50:40.297409+0000","last_active":"2023-10-06T12:50:40.761052+0000","last_peered":"2023-10-06T12:50:40.761052+0000","last_clean":"2023-10-06T12:50:40.761052+0000","last_became_active":"2023-10-06T12:50:40.297094+0000","last_became_peered":"2023-10-06T12:50:40.297094+0000","last_unstale":"2023-10-06T12:50:40.761052+0000","last_undegraded":"2023-10-06T12:50:40.761052+0000","last_fullsized":"2023-10-06T12:50:40.761052+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:54:31.447429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787088+0000","last_change":"2023-10-06T12:50:40.781121+0000","last_active":"2023-10-06T12:50:40.787088+0000","last_peered":"2023-10-06T12:50:40.787088+0000","last_clean":"2023-10-06T12:50:40.787088+0000","last_became_active":"2023-10-06T12:50:40.780806+0000","last_became_peered":"2023-10-06T12:50:40.780806+0000","last_unstale":"2023-10-06T12:50:40.787088+0000","last_undegraded":"2023-10-06T12:50:40.787088+0000","last_fullsized":"2023-10-06T12:50:40.787088+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:33:04.673251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.756532+0000","last_change":"2023-10-06T12:50:40.290805+0000","last_active":"2023-10-06T12:50:40.756532+0000","last_peered":"2023-10-06T12:50:40.756532+0000","last_clean":"2023-10-06T12:50:40.756532+0000","last_became_active":"2023-10-06T12:50:40.290457+0000","last_became_peered":"2023-10-06T12:50:40.290457+0000","last_unstale":"2023-10-06T12:50:40.756532+0000","last_undegraded":"2023-10-06T12:50:40.756532+0000","last_fullsized":"2023-10-06T12:50:40.756532+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:25.206614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786881+0000","last_change":"2023-10-06T12:50:40.779184+0000","last_active":"2023-10-06T12:50:40.786881+0000","last_peered":"2023-10-06T12:50:40.786881+0000","last_clean":"2023-10-06T12:50:40.786881+0000","last_became_active":"2023-10-06T12:50:40.778347+0000","last_became_peered":"2023-10-06T12:50:40.778347+0000","last_unstale":"2023-10-06T12:50:40.786881+0000","last_undegraded":"2023-10-06T12:50:40.786881+0000","last_fullsized":"2023-10-06T12:50:40.786881+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:46.760142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.764564+0000","last_change":"2023-10-06T12:50:40.301356+0000","last_active":"2023-10-06T12:50:40.764564+0000","last_peered":"2023-10-06T12:50:40.764564+0000","last_clean":"2023-10-06T12:50:40.764564+0000","last_became_active":"2023-10-06T12:50:40.301060+0000","last_became_peered":"2023-10-06T12:50:40.301060+0000","last_unstale":"2023-10-06T12:50:40.764564+0000","last_undegraded":"2023-10-06T12:50:40.764564+0000","last_fullsized":"2023-10-06T12:50:40.764564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:45.745172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"172'326","reported_seq":1042,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766958+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:50:40.766958+0000","last_peered":"2023-10-06T12:50:40.766958+0000","last_clean":"2023-10-06T12:50:40.766958+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:50:40.766958+0000","last_undegraded":"2023-10-06T12:50:40.766958+0000","last_fullsized":"2023-10-06T12:50:40.766958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.760794+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:50:40.760794+0000","last_peered":"2023-10-06T12:50:40.760794+0000","last_clean":"2023-10-06T12:50:40.760794+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:50:40.760794+0000","last_undegraded":"2023-10-06T12:50:40.760794+0000","last_fullsized":"2023-10-06T12:50:40.760794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.777508+0000","last_change":"2023-10-06T12:50:40.775783+0000","last_active":"2023-10-06T12:50:40.777508+0000","last_peered":"2023-10-06T12:50:40.777508+0000","last_clean":"2023-10-06T12:50:40.777508+0000","last_became_active":"2023-10-06T12:50:40.775512+0000","last_became_peered":"2023-10-06T12:50:40.775512+0000","last_unstale":"2023-10-06T12:50:40.777508+0000","last_undegraded":"2023-10-06T12:50:40.777508+0000","last_fullsized":"2023-10-06T12:50:40.777508+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:09:01.262267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"182'3","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.310875+0000","last_change":"2023-10-06T12:50:40.285774+0000","last_active":"2023-10-06T12:50:41.310875+0000","last_peered":"2023-10-06T12:50:41.310875+0000","last_clean":"2023-10-06T12:50:41.310875+0000","last_became_active":"2023-10-06T12:50:40.285452+0000","last_became_peered":"2023-10-06T12:50:40.285452+0000","last_unstale":"2023-10-06T12:50:41.310875+0000","last_undegraded":"2023-10-06T12:50:41.310875+0000","last_fullsized":"2023-10-06T12:50:41.310875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:36:19.998772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766873+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:50:40.766873+0000","last_peered":"2023-10-06T12:50:40.766873+0000","last_clean":"2023-10-06T12:50:40.766873+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:50:40.766873+0000","last_undegraded":"2023-10-06T12:50:40.766873+0000","last_fullsized":"2023-10-06T12:50:40.766873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788947+0000","last_change":"2023-10-06T12:50:40.783276+0000","last_active":"2023-10-06T12:50:40.788947+0000","last_peered":"2023-10-06T12:50:40.788947+0000","last_clean":"2023-10-06T12:50:40.788947+0000","last_became_active":"2023-10-06T12:50:40.782946+0000","last_became_peered":"2023-10-06T12:50:40.782946+0000","last_unstale":"2023-10-06T12:50:40.788947+0000","last_undegraded":"2023-10-06T12:50:40.788947+0000","last_fullsized":"2023-10-06T12:50:40.788947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:32:51.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"182'6","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.317781+0000","last_change":"2023-10-06T12:50:40.299914+0000","last_active":"2023-10-06T12:50:41.317781+0000","last_peered":"2023-10-06T12:50:41.317781+0000","last_clean":"2023-10-06T12:50:41.317781+0000","last_became_active":"2023-10-06T12:50:40.299632+0000","last_became_peered":"2023-10-06T12:50:40.299632+0000","last_unstale":"2023-10-06T12:50:41.317781+0000","last_undegraded":"2023-10-06T12:50:41.317781+0000","last_fullsized":"2023-10-06T12:50:41.317781+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:03.746846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766609+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:50:40.766609+0000","last_peered":"2023-10-06T12:50:40.766609+0000","last_clean":"2023-10-06T12:50:40.766609+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:50:40.766609+0000","last_undegraded":"2023-10-06T12:50:40.766609+0000","last_fullsized":"2023-10-06T12:50:40.766609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.786958+0000","last_change":"2023-10-06T12:50:40.780781+0000","last_active":"2023-10-06T12:50:40.786958+0000","last_peered":"2023-10-06T12:50:40.786958+0000","last_clean":"2023-10-06T12:50:40.786958+0000","last_became_active":"2023-10-06T12:50:40.775160+0000","last_became_peered":"2023-10-06T12:50:40.775160+0000","last_unstale":"2023-10-06T12:50:40.786958+0000","last_undegraded":"2023-10-06T12:50:40.786958+0000","last_fullsized":"2023-10-06T12:50:40.786958+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:54:25.796751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.765278+0000","last_change":"2023-10-06T12:50:40.301828+0000","last_active":"2023-10-06T12:50:40.765278+0000","last_peered":"2023-10-06T12:50:40.765278+0000","last_clean":"2023-10-06T12:50:40.765278+0000","last_became_active":"2023-10-06T12:50:40.301514+0000","last_became_peered":"2023-10-06T12:50:40.301514+0000","last_unstale":"2023-10-06T12:50:40.765278+0000","last_undegraded":"2023-10-06T12:50:40.765278+0000","last_fullsized":"2023-10-06T12:50:40.765278+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:53.574100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.763786+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:50:40.763786+0000","last_peered":"2023-10-06T12:50:40.763786+0000","last_clean":"2023-10-06T12:50:40.763786+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:50:40.763786+0000","last_undegraded":"2023-10-06T12:50:40.763786+0000","last_fullsized":"2023-10-06T12:50:40.763786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.779247+0000","last_change":"2023-10-06T12:50:40.774244+0000","last_active":"2023-10-06T12:50:40.779247+0000","last_peered":"2023-10-06T12:50:40.779247+0000","last_clean":"2023-10-06T12:50:40.779247+0000","last_became_active":"2023-10-06T12:50:40.773905+0000","last_became_peered":"2023-10-06T12:50:40.773905+0000","last_unstale":"2023-10-06T12:50:40.779247+0000","last_undegraded":"2023-10-06T12:50:40.779247+0000","last_fullsized":"2023-10-06T12:50:40.779247+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.354187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.760725+0000","last_change":"2023-10-06T12:50:40.288773+0000","last_active":"2023-10-06T12:50:40.760725+0000","last_peered":"2023-10-06T12:50:40.760725+0000","last_clean":"2023-10-06T12:50:40.760725+0000","last_became_active":"2023-10-06T12:50:40.288409+0000","last_became_peered":"2023-10-06T12:50:40.288409+0000","last_unstale":"2023-10-06T12:50:40.760725+0000","last_undegraded":"2023-10-06T12:50:40.760725+0000","last_fullsized":"2023-10-06T12:50:40.760725+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:13.351610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.769705+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:50:40.769705+0000","last_peered":"2023-10-06T12:50:40.769705+0000","last_clean":"2023-10-06T12:50:40.769705+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:50:40.769705+0000","last_undegraded":"2023-10-06T12:50:40.769705+0000","last_fullsized":"2023-10-06T12:50:40.769705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.780595+0000","last_change":"2023-10-06T12:50:40.774808+0000","last_active":"2023-10-06T12:50:40.780595+0000","last_peered":"2023-10-06T12:50:40.780595+0000","last_clean":"2023-10-06T12:50:40.780595+0000","last_became_active":"2023-10-06T12:50:40.774526+0000","last_became_peered":"2023-10-06T12:50:40.774526+0000","last_unstale":"2023-10-06T12:50:40.780595+0000","last_undegraded":"2023-10-06T12:50:40.780595+0000","last_fullsized":"2023-10-06T12:50:40.780595+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:45.504866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"182'8","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.316168+0000","last_change":"2023-10-06T12:50:40.296552+0000","last_active":"2023-10-06T12:50:41.316168+0000","last_peered":"2023-10-06T12:50:41.316168+0000","last_clean":"2023-10-06T12:50:41.316168+0000","last_became_active":"2023-10-06T12:50:40.295853+0000","last_became_peered":"2023-10-06T12:50:40.295853+0000","last_unstale":"2023-10-06T12:50:41.316168+0000","last_undegraded":"2023-10-06T12:50:41.316168+0000","last_fullsized":"2023-10-06T12:50:41.316168+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:24:29.373888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.770290+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:50:40.770290+0000","last_peered":"2023-10-06T12:50:40.770290+0000","last_clean":"2023-10-06T12:50:40.770290+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:50:40.770290+0000","last_undegraded":"2023-10-06T12:50:40.770290+0000","last_fullsized":"2023-10-06T12:50:40.770290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.788119+0000","last_change":"2023-10-06T12:50:40.784137+0000","last_active":"2023-10-06T12:50:40.788119+0000","last_peered":"2023-10-06T12:50:40.788119+0000","last_clean":"2023-10-06T12:50:40.788119+0000","last_became_active":"2023-10-06T12:50:40.783861+0000","last_became_peered":"2023-10-06T12:50:40.783861+0000","last_unstale":"2023-10-06T12:50:40.788119+0000","last_undegraded":"2023-10-06T12:50:40.788119+0000","last_fullsized":"2023-10-06T12:50:40.788119+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:45.076521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"182'2","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:41.315436+0000","last_change":"2023-10-06T12:50:40.285992+0000","last_active":"2023-10-06T12:50:41.315436+0000","last_peered":"2023-10-06T12:50:41.315436+0000","last_clean":"2023-10-06T12:50:41.315436+0000","last_became_active":"2023-10-06T12:50:40.285679+0000","last_became_peered":"2023-10-06T12:50:40.285679+0000","last_unstale":"2023-10-06T12:50:41.315436+0000","last_undegraded":"2023-10-06T12:50:41.315436+0000","last_fullsized":"2023-10-06T12:50:41.315436+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:28:39.180568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.771402+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:50:40.771402+0000","last_peered":"2023-10-06T12:50:40.771402+0000","last_clean":"2023-10-06T12:50:40.771402+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:50:40.771402+0000","last_undegraded":"2023-10-06T12:50:40.771402+0000","last_fullsized":"2023-10-06T12:50:40.771402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.787453+0000","last_change":"2023-10-06T12:50:40.782198+0000","last_active":"2023-10-06T12:50:40.787453+0000","last_peered":"2023-10-06T12:50:40.787453+0000","last_clean":"2023-10-06T12:50:40.787453+0000","last_became_active":"2023-10-06T12:50:40.781890+0000","last_became_peered":"2023-10-06T12:50:40.781890+0000","last_unstale":"2023-10-06T12:50:40.787453+0000","last_undegraded":"2023-10-06T12:50:40.787453+0000","last_fullsized":"2023-10-06T12:50:40.787453+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:40.258108+0000","last_clean_scrub_stamp":"2023-10-06T12:50:40.258108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:32:32.586422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.766535+0000","last_change":"2023-10-06T12:50:40.297347+0000","last_active":"2023-10-06T12:50:40.766535+0000","last_peered":"2023-10-06T12:50:40.766535+0000","last_clean":"2023-10-06T12:50:40.766535+0000","last_became_active":"2023-10-06T12:50:40.293329+0000","last_became_peered":"2023-10-06T12:50:40.293329+0000","last_unstale":"2023-10-06T12:50:40.766535+0000","last_undegraded":"2023-10-06T12:50:40.766535+0000","last_fullsized":"2023-10-06T12:50:40.766535+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:50:39.243138+0000","last_clean_scrub_stamp":"2023-10-06T12:50:39.243138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:44.489896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":149,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-06T12:50:40.769875+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:50:40.769875+0000","last_peered":"2023-10-06T12:50:40.769875+0000","last_clean":"2023-10-06T12:50:40.769875+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:50:40.769875+0000","last_undegraded":"2023-10-06T12:50:40.769875+0000","last_fullsized":"2023-10-06T12:50:40.769875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304200,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067640,"statfs":{"total":96636764160,"available":96325263360,"internally_reserved":0,"allocated":2883584,"data_stored":1804815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1799999999999999,"15min":1.169},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.56299999999999994},"max":{"1min":1.5980000000000001,"5min":1.758,"15min":1.758},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.143,"5min":1.1679999999999999,"15min":1.1559999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.8660000000000001},"last":1.077}]},{"osd":1,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.379,"15min":1.379},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.72099999999999997},"max":{"1min":1.8029999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.357},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2789999999999999,"15min":1.2929999999999999},"min":{"1min":0.82999999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.76},"last":1.395}]},{"osd":2,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.56299999999999994},"max":{"1min":1.5209999999999999,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.431},{"interface":"front","average":{"1min":1.246,"5min":1.2629999999999999,"15min":1.272},"min":{"1min":0.96199999999999997,"5min":0.92300000000000004,"15min":0.78400000000000003},"max":{"1min":1.621,"5min":1.7150000000000001,"15min":1.8360000000000001},"last":1.248}]},{"osd":3,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.78000000000000003},"max":{"1min":1.694,"5min":1.9019999999999999,"15min":2.105},"last":1.657},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.351,"15min":1.343},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.81999999999999995},"max":{"1min":1.71,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.6279999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1000000000000001,"15min":1.1080000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.486,"5min":1.667,"15min":1.681},"last":1.29},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.649,"5min":2.0939999999999999,"15min":2.2989999999999999},"last":1.776}]},{"osd":5,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3260000000000001,"15min":1.3400000000000001},"min":{"1min":0.93100000000000005,"5min":0.874,"15min":0.59499999999999997},"max":{"1min":1.657,"5min":2.036,"15min":2.036},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.6419999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:50:22 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.385,"15min":1.391},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.7490000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.496},{"interface":"front","average":{"1min":1.365,"5min":1.466,"15min":1.466},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.8360000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575950,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315908,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055932,"statfs":{"total":96636764160,"available":96313274368,"internally_reserved":0,"allocated":2289664,"data_stored":1214447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2969999999999999,"15min":1.3100000000000001},"min":{"1min":0.77000000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5569999999999999,"5min":1.857,"15min":1.857},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.3160000000000001},"min":{"1min":1.034,"5min":0.79700000000000004,"15min":0.74399999999999999},"max":{"1min":1.4510000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3879999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1990000000000001,"15min":1.1890000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.363,"5min":1.597,"15min":1.7789999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2150000000000001,"15min":1.2050000000000001},"min":{"1min":0.89000000000000001,"5min":0.877,"15min":0.81399999999999995},"max":{"1min":1.623,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.5229999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.198,"15min":1.208},"min":{"1min":0.92300000000000004,"5min":0.79200000000000004,"15min":0.68500000000000005},"max":{"1min":1.536,"5min":1.724,"15min":1.839},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":1.024,"5min":0.88100000000000001,"15min":0.77500000000000002},"max":{"1min":1.526,"5min":1.5669999999999999,"15min":1.7290000000000001},"last":1.2290000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3100000000000001,"15min":1.323},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.68600000000000005},"max":{"1min":1.5580000000000001,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.411},{"interface":"front","average":{"1min":1.274,"5min":1.3240000000000001,"15min":1.339},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":1.6719999999999999,"15min":1.8879999999999999},"last":1.4399999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.89500000000000002,"5min":0.77300000000000002,"15min":0.69599999999999995},"max":{"1min":1.47,"5min":1.998,"15min":2.1459999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.2749999999999999,"15min":1.2969999999999999},"min":{"1min":0.85099999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.5109999999999999,"5min":1.873,"15min":1.911},"last":1.1619999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2270000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.81399999999999995,"15min":0.77200000000000002},"max":{"1min":1.633,"5min":1.778,"15min":1.8600000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.397,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":1.115,"5min":0.875,"15min":0.75900000000000001},"max":{"1min":1.6850000000000001,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.268}]},{"osd":7,"last update":"Fri Oct 6 12:50:06 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4690000000000001,"15min":1.4730000000000001},"min":{"1min":1.242,"5min":0.93799999999999994,"15min":0.63200000000000001},"max":{"1min":1.5980000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.498},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4490000000000001,"15min":1.464},"min":{"1min":1.0309999999999999,"5min":0.86699999999999999,"15min":0.81000000000000005},"max":{"1min":1.72,"5min":2.0609999999999999,"15min":2.2170000000000001},"last":1.355}]}]},{"osd":5,"up_from":13,"seq":55834575950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311820,"kb_used_data":2244,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060020,"statfs":{"total":96636764160,"available":96317460480,"internally_reserved":0,"allocated":2297856,"data_stored":1214571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.276,"15min":1.292},"min":{"1min":1.091,"5min":0.751,"15min":0.751},"max":{"1min":1.663,"5min":1.6919999999999999,"15min":1.7889999999999999},"last":1.365},{"interface":"front","average":{"1min":1.365,"5min":1.3620000000000001,"15min":1.373},"min":{"1min":1.099,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.556,"5min":1.7929999999999999,"15min":2.657},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.163,"15min":1.173},"min":{"1min":0.86699999999999999,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.492,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.214,"15min":1.23},"min":{"1min":0.94799999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.569,"5min":1.716,"15min":1.732},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1930000000000001,"15min":1.228},"min":{"1min":0.96899999999999997,"5min":0.77000000000000002,"15min":0.74399999999999999},"max":{"1min":1.466,"5min":1.7549999999999999,"15min":2.754},"last":1.103},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2909999999999999,"15min":1.2849999999999999},"min":{"1min":1.125,"5min":0.89400000000000002,"15min":0.85699999999999998},"max":{"1min":1.4279999999999999,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.292},"min":{"1min":0.997,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.4790000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.3540000000000001,"15min":1.3620000000000001},"min":{"1min":1.111,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.6559999999999999,"5min":1.845,"15min":2.9100000000000001},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.248,"15min":1.2669999999999999},"min":{"1min":0.85299999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5429999999999999,"5min":1.698,"15min":4.0899999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.3660000000000001,"15min":1.3959999999999999},"min":{"1min":1.2110000000000001,"5min":0.71899999999999997,"15min":0.67500000000000004},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":2.2069999999999999},"last":1.5029999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2709999999999999,"15min":1.3},"min":{"1min":0.92600000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.5589999999999999,"5min":1.915,"15min":2.8300000000000001},"last":1.472},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.357},"min":{"1min":0.95799999999999996,"5min":0.76100000000000001,"15min":0.67000000000000004},"max":{"1min":1.649,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.327}]},{"osd":7,"last update":"Fri Oct 6 12:49:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3240000000000001,"15min":1.363},"min":{"1min":0.93999999999999995,"5min":0.877,"15min":0.76000000000000001},"max":{"1min":1.74,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.548},{"interface":"front","average":{"1min":1.345,"5min":1.3999999999999999,"15min":1.4330000000000001},"min":{"1min":1.0800000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7669999999999999,"5min":1.968,"15min":3.0049999999999999},"last":1.536}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311828,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060012,"statfs":{"total":96636764160,"available":96317452288,"internally_reserved":0,"allocated":2306048,"data_stored":1215183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.153,"15min":1.1919999999999999},"min":{"1min":0.84699999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5760000000000001,"5min":1.851,"15min":1.851},"last":1.46},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1899999999999999,"15min":1.216},"min":{"1min":0.82099999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6559999999999999,"5min":1.7529999999999999,"15min":2.1680000000000001},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4119999999999999},"min":{"1min":0.94299999999999995,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6319999999999999,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.333,"15min":1.321},"min":{"1min":1.046,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6160000000000001,"5min":1.948,"15min":1.948},"last":1.4159999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.2629999999999999},"min":{"1min":0.90900000000000003,"5min":0.77500000000000002,"15min":0.76500000000000001},"max":{"1min":1.6779999999999999,"5min":2.024,"15min":2.024},"last":1.651},{"interface":"front","average":{"1min":1.321,"5min":1.29,"15min":1.3029999999999999},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.6950000000000001,"5min":1.917,"15min":2.0630000000000002},"last":1.4370000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2749999999999999,"15min":1.2669999999999999},"min":{"1min":0.98299999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.5329999999999999,"5min":1.7430000000000001,"15min":1.768},"last":1.508},{"interface":"front","average":{"1min":1.319,"5min":1.347,"15min":1.3420000000000001},"min":{"1min":1.0129999999999999,"5min":0.94399999999999995,"15min":0.86699999999999999},"max":{"1min":1.581,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.24},"min":{"1min":0.93700000000000006,"5min":0.73199999999999998,"15min":0.71399999999999997},"max":{"1min":1.6259999999999999,"5min":2.012,"15min":2.3220000000000001},"last":0.87},{"interface":"front","average":{"1min":1.212,"5min":1.2470000000000001,"15min":1.2350000000000001},"min":{"1min":0.85199999999999998,"5min":0.77700000000000002,"15min":0.61899999999999999},"max":{"1min":1.6859999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2849999999999999,"15min":1.2769999999999999},"min":{"1min":0.91600000000000004,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7789999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4279999999999999,"15min":1.403},"min":{"1min":0.86699999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.75,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.2230000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:50:15 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3779999999999999,"15min":1.369},"min":{"1min":1.018,"5min":0.81100000000000005,"15min":0.70999999999999996},"max":{"1min":1.944,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4450000000000001,"15min":1.444},"min":{"1min":1.1000000000000001,"5min":1.01,"15min":0.59599999999999997},"max":{"1min":1.7030000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.5660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575952,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299544,"kb_used_data":2256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072296,"statfs":{"total":96636764160,"available":96330031104,"internally_reserved":0,"allocated":2310144,"data_stored":1216559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.92800000000000005,"15min":0.92600000000000005},"min":{"1min":0.60399999999999998,"5min":0.56999999999999995,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.478,"15min":1.605},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99199999999999999,"15min":1.0269999999999999},"min":{"1min":0.66500000000000004,"5min":0.63,"15min":0.56699999999999995},"max":{"1min":1.431,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.464}]},{"osd":1,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95999999999999996,"15min":0.98899999999999999},"min":{"1min":0.69799999999999995,"5min":0.54000000000000004,"15min":0.53100000000000003},"max":{"1min":1.464,"5min":1.5780000000000001,"15min":1.639},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.105,"15min":1.119},"min":{"1min":0.88100000000000001,"5min":0.68000000000000005,"15min":0.55400000000000005},"max":{"1min":1.5700000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.099}]},{"osd":2,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1519999999999999,"15min":1.161},"min":{"1min":0.84099999999999997,"5min":0.68799999999999994,"15min":0.60399999999999998},"max":{"1min":1.4510000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.212,"15min":1.2390000000000001},"min":{"1min":1.0049999999999999,"5min":0.747,"15min":0.63900000000000001},"max":{"1min":1.5740000000000001,"5min":1.738,"15min":1.768},"last":1.2649999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.23,"15min":1.264},"min":{"1min":0.95099999999999996,"5min":0.90400000000000003,"15min":0.76600000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":2.008},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.363,"15min":1.385},"min":{"1min":0.999,"5min":0.95599999999999996,"15min":0.94299999999999995},"max":{"1min":1.6779999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.163}]},{"osd":5,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.756,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.1679999999999999,"5min":0.97899999999999998,"15min":0.64200000000000002},"max":{"1min":1.8169999999999999,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.4730000000000001,"15min":1.4930000000000001},"min":{"1min":1.2589999999999999,"5min":1.0129999999999999,"15min":0.72199999999999998},"max":{"1min":1.911,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4179999999999999,"15min":1.4119999999999999},"min":{"1min":1.056,"5min":0.995,"15min":0.84099999999999997},"max":{"1min":1.6759999999999999,"5min":1.9159999999999999,"15min":1.929},"last":1.6140000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:50:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.454,"15min":1.47},"min":{"1min":1.022,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.726,"5min":2.085,"15min":2.0899999999999999},"last":1.708},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.528,"15min":1.5429999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"max":{"1min":1.9079999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.653}]}]},{"osd":2,"up_from":13,"seq":55834575950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311812,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060028,"statfs":{"total":96636764160,"available":96317468672,"internally_reserved":0,"allocated":2289664,"data_stored":1214447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.998,"15min":0.98799999999999999},"min":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.665,"15min":1.665},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.93300000000000005,"5min":0.77400000000000002,"15min":0.63800000000000001},"max":{"1min":1.415,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.081}]},{"osd":1,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.91900000000000004,"15min":0.96399999999999997},"min":{"1min":0.73799999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0820000000000001,"5min":1.577,"15min":1.728},"last":1.234},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0489999999999999,"15min":1.0569999999999999},"min":{"1min":0.79300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5,"5min":1.6040000000000001,"15min":1.8200000000000001},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.27},"min":{"1min":0.97899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.718,"5min":1.804,"15min":1.962},"last":1.55},{"interface":"front","average":{"1min":1.403,"5min":1.3859999999999999,"15min":1.3660000000000001},"min":{"1min":1.0389999999999999,"5min":0.64400000000000002,"15min":0.54600000000000004},"max":{"1min":1.7789999999999999,"5min":1.806,"15min":1.9510000000000001},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2509999999999999,"15min":1.2589999999999999},"min":{"1min":0.98799999999999999,"5min":0.84999999999999998,"15min":0.82099999999999995},"max":{"1min":1.696,"5min":1.696,"15min":1.897},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.339,"15min":1.3500000000000001},"min":{"1min":1.012,"5min":1.006,"15min":0.95699999999999996},"max":{"1min":1.5880000000000001,"5min":1.716,"15min":1.8560000000000001},"last":1.292}]},{"osd":5,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":1.022,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.351},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3700000000000001,"15min":1.4039999999999999},"min":{"1min":1.1379999999999999,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.7070000000000001,"5min":1.764,"15min":2.0619999999999998},"last":1.2}]},{"osd":6,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4119999999999999,"15min":1.423},"min":{"1min":1.1839999999999999,"5min":1.0049999999999999,"15min":0.88900000000000001},"max":{"1min":1.6899999999999999,"5min":1.847,"15min":2.0699999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":1.133,"5min":1.119,"15min":1.111},"max":{"1min":2.0609999999999999,"5min":2.206,"15min":2.206},"last":1.139}]},{"osd":7,"last update":"Fri Oct 6 12:49:56 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4019999999999999,"15min":1.4350000000000001},"min":{"1min":1.1080000000000001,"5min":0.753,"15min":0.753},"max":{"1min":1.837,"5min":1.8959999999999999,"15min":1.9970000000000001},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.5049999999999999,"15min":1.5229999999999999},"min":{"1min":1.167,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7370000000000001,"5min":1.9890000000000001,"15min":2.177},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834575950,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311824,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060016,"statfs":{"total":96636764160,"available":96317456384,"internally_reserved":0,"allocated":2301952,"data_stored":1215983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94899999999999995,"15min":0.94599999999999995},"min":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.48799999999999999},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.036,"5min":1.0389999999999999,"15min":1.028},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.627},"max":{"1min":1.498,"5min":1.6539999999999999,"15min":1.9099999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0209999999999999,"15min":0.98699999999999999},"min":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.56499999999999995},"max":{"1min":1.573,"5min":1.585,"15min":1.6970000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.117,"15min":1.131},"min":{"1min":0.86599999999999999,"5min":0.79700000000000004,"15min":0.63700000000000001},"max":{"1min":1.2969999999999999,"5min":1.6619999999999999,"15min":1.6739999999999999},"last":1.0389999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.232,"15min":1.2749999999999999},"min":{"1min":0.95799999999999996,"5min":0.85099999999999998,"15min":0.81299999999999994},"max":{"1min":1.595,"5min":1.7490000000000001,"15min":1.9399999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.157,"15min":1.1639999999999999},"min":{"1min":0.82399999999999995,"5min":0.74099999999999999,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":1.6140000000000001,"15min":1.7609999999999999},"last":1.2130000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3029999999999999,"15min":1.276},"min":{"1min":1.075,"5min":1.036,"15min":0.84799999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.9179999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4279999999999999,"15min":1.4390000000000001},"min":{"1min":1.1639999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":1.675,"5min":2.093,"15min":2.093},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.407,"15min":1.401},"min":{"1min":1.1240000000000001,"5min":1.0900000000000001,"15min":0.81599999999999995},"max":{"1min":1.7649999999999999,"5min":2.052,"15min":2.052},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4139999999999999,"15min":1.397},"min":{"1min":1.103,"5min":0.93100000000000005,"15min":0.91600000000000004},"max":{"1min":1.8109999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.633}]},{"osd":6,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.5,"15min":1.524},"min":{"1min":1.2010000000000001,"5min":1.1359999999999999,"15min":0.996},"max":{"1min":1.6779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.472,"15min":1.484},"min":{"1min":1.1830000000000001,"5min":1.157,"15min":1.0289999999999999},"max":{"1min":1.867,"5min":2.0430000000000001,"15min":2.137},"last":1.573}]},{"osd":7,"last update":"Fri Oct 6 12:50:11 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5900000000000001,"15min":1.585},"min":{"1min":1.4399999999999999,"5min":1.228,"15min":1.0880000000000001},"max":{"1min":1.9279999999999999,"5min":2.1389999999999998,"15min":2.2480000000000002},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.556,"15min":1.526},"min":{"1min":1.335,"5min":1.161,"15min":0.99299999999999999},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.381}]}]},{"osd":0,"up_from":13,"seq":55834575951,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304208,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067632,"statfs":{"total":96636764160,"available":96325255168,"internally_reserved":0,"allocated":2891776,"data_stored":1804919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90500000000000003,"15min":0.92700000000000005},"min":{"1min":0.71699999999999997,"5min":0.60499999999999998,"15min":0.59699999999999998},"max":{"1min":1.3200000000000001,"5min":1.4450000000000001,"15min":1.724},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.97199999999999998,"15min":0.98799999999999999},"min":{"1min":0.84499999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.84799999999999998}]},{"osd":2,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0349999999999999,"15min":1.038},"min":{"1min":0.84199999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.343,"5min":1.708,"15min":1.7370000000000001},"last":1.006},{"interface":"front","average":{"1min":1.101,"5min":1.0680000000000001,"15min":1.093},"min":{"1min":0.86099999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.5269999999999999,"5min":1.528,"15min":2.028},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.137,"15min":1.125},"min":{"1min":0.81799999999999995,"5min":0.71199999999999997,"15min":0.61099999999999999},"max":{"1min":1.5840000000000001,"5min":1.724,"15min":2.004},"last":1.069},{"interface":"front","average":{"1min":1.383,"5min":1.3660000000000001,"15min":1.375},"min":{"1min":1.1859999999999999,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.6140000000000001,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.411}]},{"osd":4,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.347,"15min":1.3500000000000001},"min":{"1min":1.1830000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.762,"5min":1.823,"15min":2.0880000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.367,"15min":1.363},"min":{"1min":0.98699999999999999,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.637,"5min":1.9850000000000001,"15min":2.387},"last":1.663}]},{"osd":5,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4159999999999999,"15min":1.409},"min":{"1min":1.069,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.742,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.3879999999999999,"15min":1.4259999999999999},"min":{"1min":1.1240000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6739999999999999,"5min":1.9510000000000001,"15min":2.23},"last":1.524}]},{"osd":6,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.409,"15min":1.4159999999999999},"min":{"1min":1.1699999999999999,"5min":0.749,"15min":0.749},"max":{"1min":1.8939999999999999,"5min":1.9770000000000001,"15min":2.0910000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.4690000000000001,"15min":1.482},"min":{"1min":1.181,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.9299999999999999,"5min":1.9390000000000001,"15min":2.7530000000000001},"last":1.508}]},{"osd":7,"last update":"Fri Oct 6 12:50:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.5009999999999999,"15min":1.5},"min":{"1min":1.2589999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.867,"5min":2.177,"15min":2.177},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.472,"15min":1.492},"min":{"1min":1.2390000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.869,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.577}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:50:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T12:50:48.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T12:50:48.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.452+0000 7fa9ec8e3700 1 Processor -- start 2023-10-06T12:50:48.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 -- start start 2023-10-06T12:50:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e81590b0 0x7fa9e80cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9e8068020 0x7fa9e80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e80cc8b0 con 0x7fa9e80b2420 2023-10-06T12:50:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e80cbca0 con 0x7fa9e81590b0 2023-10-06T12:50:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9ec8e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e80cbf80 con 0x7fa9e8068020 2023-10-06T12:50:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9e77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e81590b0 0x7fa9e80cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.456+0000 7fa9e77fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9e81590b0 0x7fa9e80cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47334/0 (socket says 172.21.15.46:47334) 2023-10-06T12:50:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 -- 172.21.15.46:0/357759739 learned_addr learned my addr 172.21.15.46:0/357759739 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:48.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e7fff700 1 --2- 172.21.15.46:0/357759739 >> v2:172.21.15.132:3300/0 conn(0x7fa9e8068020 0x7fa9e80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 -- 172.21.15.46:0/357759739 >> v2:172.21.15.132:3300/0 conn(0x7fa9e8068020 msgr2=0x7fa9e80b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 --2- 172.21.15.46:0/357759739 >> v2:172.21.15.132:3300/0 conn(0x7fa9e8068020 0x7fa9e80b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 -- 172.21.15.46:0/357759739 >> v1:172.21.15.132:6789/0 conn(0x7fa9e80b2420 legacy=0x7fa9e8067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 -- 172.21.15.46:0/357759739 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80cc260 con 0x7fa9e81590b0 2023-10-06T12:50:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e77fe700 1 --2- 172.21.15.46:0/357759739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e81590b0 0x7fa9e80cb600 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80d0fc0 tx=0x7fa9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf09cbd93c4a25d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e67fc700 1 -- 172.21.15.46:0/357759739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8009e70 con 0x7fa9e81590b0 2023-10-06T12:50:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e67fc700 1 -- 172.21.15.46:0/357759739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d800b860 con 0x7fa9e81590b0 2023-10-06T12:50:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9e67fc700 1 -- 172.21.15.46:0/357759739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800bc00 con 0x7fa9e81590b0 2023-10-06T12:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/357759739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e81590b0 msgr2=0x7fa9e80cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/357759739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e81590b0 0x7fa9e80cb600 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80d0fc0 tx=0x7fa9d8009250 comp rx=0 tx=0).stop 2023-10-06T12:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/357759739 shutdown_connections 2023-10-06T12:50:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/357759739 >> v2:172.21.15.132:3300/0 conn(0x7fa9e8068020 0x7fa9e80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/357759739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e81590b0 0x7fa9e80cb600 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/357759739 >> 172.21.15.46:0/357759739 conn(0x7fa9e80cfff0 msgr2=0x7fa9e80d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/357759739 shutdown_connections 2023-10-06T12:50:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.460+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/357759739 wait complete. 2023-10-06T12:50:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 Processor -- start 2023-10-06T12:50:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 -- start start 2023-10-06T12:50:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e8068020 0x7fa9e80c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e81590b0 0x7fa9e80c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e80c94b0 con 0x7fa9e80b2420 2023-10-06T12:50:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e80c9790 con 0x7fa9e81590b0 2023-10-06T12:50:48.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9ec8e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e80c9a70 con 0x7fa9e8068020 2023-10-06T12:50:48.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9e6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9e6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47340/0 (socket says 172.21.15.46:47340) 2023-10-06T12:50:48.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9e6ffd700 1 -- 172.21.15.46:0/214535065 learned_addr learned my addr 172.21.15.46:0/214535065 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9e7fff700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e81590b0 0x7fa9e80c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.464+0000 7fa9e77fe700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e8068020 0x7fa9e80c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 -- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e8068020 msgr2=0x7fa9e80c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e8068020 0x7fa9e80c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 -- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e81590b0 msgr2=0x7fa9e80c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e81590b0 0x7fa9e80c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 -- 172.21.15.46:0/214535065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80094f0 con 0x7fa9e80b2420 2023-10-06T12:50:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9e6ffd700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c590 tx=0x7fa9dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc011590 con 0x7fa9e80b2420 2023-10-06T12:50:48.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc011710 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc011ad0 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e80c9d50 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e80c36a0 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.468+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e8064590 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.472+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa9dc01b3b0 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.472+0000 7fa9cf7fe700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 0x7fa9d4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.472+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fa9dc07c0c0 con 0x7fa9e80b2420 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.472+0000 7fa9e77fe700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 0x7fa9d4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.472+0000 7fa9e77fe700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 0x7fa9d4062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80d0fc0 tx=0x7fa9d8017000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.480+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa9dc04c220 con 0x7fa9e80b2420 2023-10-06T12:50:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.708+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa9e80cc540 con 0x7fa9e80b2420 2023-10-06T12:50:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.716+0000 7fa9cf7fe700 1 -- 172.21.15.46:0/214535065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa9dc0170e0 con 0x7fa9e80b2420 2023-10-06T12:50:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 msgr2=0x7fa9d4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 0x7fa9d4062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80d0fc0 tx=0x7fa9d8017000 comp rx=0 tx=0).stop 2023-10-06T12:50:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 msgr2=0x7fa9e80c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00c590 tx=0x7fa9dc00c8a0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 shutdown_connections 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa9d4060670 0x7fa9d4062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9e81590b0 0x7fa9e80c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9e80b2420 0x7fa9e80c8960 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 --2- 172.21.15.46:0/214535065 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9e8068020 0x7fa9e80c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 >> 172.21.15.46:0/214535065 conn(0x7fa9e80cfff0 msgr2=0x7fa9e80b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 shutdown_connections 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.720+0000 7fa9ec8e3700 1 -- 172.21.15.46:0/214535065 wait complete. 2023-10-06T12:50:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T12:50:48.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:50:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 Processor -- start 2023-10-06T12:50:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 -- start start 2023-10-06T12:50:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff088158ea0 0x7ff088069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff088159dd0 con 0x7ff0880b5d00 2023-10-06T12:50:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08815a5d0 con 0x7ff088159980 2023-10-06T12:50:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff08d9c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880ce950 con 0x7ff088158ea0 2023-10-06T12:50:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff0877fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ff088159980 0x7ff088162c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35782/0 (socket says 172.21.15.46:35782) 2023-10-06T12:50:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff0877fe700 1 -- 172.21.15.46:0/3573801584 learned_addr learned my addr 172.21.15.46:0/3573801584 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff0867fc700 1 --1- 172.21.15.46:0/3573801584 >> v1:172.21.15.46:6789/0 conn(0x7ff0880b5d00 0x7ff088159270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38094/0 (socket says 172.21.15.46:38094) 2023-10-06T12:50:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.900+0000 7ff086ffd700 1 --2- 172.21.15.46:0/3573801584 >> v2:172.21.15.46:3300/0 conn(0x7ff088158ea0 0x7ff088069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812872104 0 0) 0x7ff08815a5d0 con 0x7ff088159980 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff070003570 con 0x7ff088159980 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff086ffd700 1 -- 172.21.15.46:0/3573801584 >> v1:172.21.15.132:6789/0 conn(0x7ff088159980 legacy=0x7ff088162c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff086ffd700 1 -- 172.21.15.46:0/3573801584 >> v1:172.21.15.46:6789/0 conn(0x7ff0880b5d00 legacy=0x7ff088159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff086ffd700 1 -- 172.21.15.46:0/3573801584 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880cfdf0 con 0x7ff088158ea0 2023-10-06T12:50:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987875742 0 0) 0x7ff07c001110 con 0x7ff0880b5d00 2023-10-06T12:50:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff086ffd700 1 --2- 172.21.15.46:0/3573801584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff088158ea0 0x7ff088069060 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098e0 tx=0x7ff0780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f182aa77aec709 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800b640 con 0x7ff088158ea0 2023-10-06T12:50:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800b7a0 con 0x7ff088158ea0 2023-10-06T12:50:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff085ffb700 1 -- 172.21.15.46:0/3573801584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800bb40 con 0x7ff088158ea0 2023-10-06T12:50:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 -- 172.21.15.46:0/3573801584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff088158ea0 msgr2=0x7ff088069060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/3573801584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff088158ea0 0x7ff088069060 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098e0 tx=0x7ff0780091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 -- 172.21.15.46:0/3573801584 shutdown_connections 2023-10-06T12:50:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/3573801584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff088158ea0 0x7ff088069060 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 -- 172.21.15.46:0/3573801584 >> 172.21.15.46:0/3573801584 conn(0x7ff088068390 msgr2=0x7ff0880b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 -- 172.21.15.46:0/3573801584 shutdown_connections 2023-10-06T12:50:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.904+0000 7ff08d9c5700 1 -- 172.21.15.46:0/3573801584 wait complete. 2023-10-06T12:50:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.908+0000 7ff08d9c5700 1 Processor -- start 2023-10-06T12:50:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 -- start start 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff088158ea0 0x7ff08815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088159980 0x7ff088149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08815e200 con 0x7ff0880b5d00 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088147390 con 0x7ff088159980 2023-10-06T12:50:48.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08815dc40 con 0x7ff088158ea0 2023-10-06T12:50:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47350/0 (socket says 172.21.15.46:47350) 2023-10-06T12:50:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 -- 172.21.15.46:0/2207144103 learned_addr learned my addr 172.21.15.46:0/2207144103 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff0867fc700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff088158ea0 0x7ff08815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff0877fe700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088159980 0x7ff088149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 -- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff088158ea0 msgr2=0x7ff08815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff088158ea0 0x7ff08815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 -- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088159980 msgr2=0x7ff088149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088159980 0x7ff088149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 -- 172.21.15.46:0/2207144103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078009420 con 0x7ff0880b5d00 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff086ffd700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff078009f80 tx=0x7ff078017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07801f070 con 0x7ff0880b5d00 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800b910 con 0x7ff0880b5d00 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078011420 con 0x7ff0880b5d00 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.912+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08815df20 con 0x7ff0880b5d00 2023-10-06T12:50:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.916+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088149d20 con 0x7ff0880b5d00 2023-10-06T12:50:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.916+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088064590 con 0x7ff0880b5d00 2023-10-06T12:50:48.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.916+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff0780117d0 con 0x7ff0880b5d00 2023-10-06T12:50:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.920+0000 7ff06effd700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 0x7ff074062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:48.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.920+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7ff07807e160 con 0x7ff0880b5d00 2023-10-06T12:50:48.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.920+0000 7ff0867fc700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 0x7ff074062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:48.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.920+0000 7ff0867fc700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 0x7ff074062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0880ce170 tx=0x7ff07c007a30 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:48.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:48.928+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff07804e2c0 con 0x7ff0880b5d00 2023-10-06T12:50:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.132+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff08815a010 con 0x7ff0880b5d00 2023-10-06T12:50:49.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff06effd700 1 -- 172.21.15.46:0/2207144103 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7ff0780518e0 con 0x7ff0880b5d00 2023-10-06T12:50:49.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:49.215 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 msgr2=0x7ff074062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 0x7ff074062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff0880ce170 tx=0x7ff07c007a30 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 msgr2=0x7ff0880c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff078009f80 tx=0x7ff078017000 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 shutdown_connections 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff074060670 0x7ff074062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff088159980 0x7ff088149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff088158ea0 0x7ff08815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 --2- 172.21.15.46:0/2207144103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0880b5d00 0x7ff0880c7cb0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 >> 172.21.15.46:0/2207144103 conn(0x7ff088068390 msgr2=0x7ff088157940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 shutdown_connections 2023-10-06T12:50:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.136+0000 7ff08d9c5700 1 -- 172.21.15.46:0/2207144103 wait complete. 2023-10-06T12:50:49.218 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:50:49.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T12:50:40.722486+0000', 'modified': '2023-10-06T12:50:45.339830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9619': {'gid': 9619, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/192039016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 192039016}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 192039016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T12:50:49.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T12:50:49.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T12:50:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.784+0000 7f00484f0700 1 Processor -- start 2023-10-06T12:50:49.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.784+0000 7f00484f0700 1 -- start start 2023-10-06T12:50:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f00484f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f004006e8a0 0x7f004006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f00484f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004006d460 con 0x7f00400ad6c0 2023-10-06T12:50:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f00484f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00400c7000 con 0x7f004015ab60 2023-10-06T12:50:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f00484f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004006ed40 con 0x7f004006e8a0 2023-10-06T12:50:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f0046a8d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00400ad6c0 0x7f00400adaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38108/0 (socket says 172.21.15.46:38108) 2023-10-06T12:50:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f0046a8d700 1 -- 172.21.15.46:0/2197739714 learned_addr learned my addr 172.21.15.46:0/2197739714 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2197739714 >> v2:172.21.15.132:3300/0 conn(0x7f004006e8a0 0x7f004006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732336112 0 0) 0x7f004006d460 con 0x7f00400ad6c0 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c003570 con 0x7f00400ad6c0 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915687045 0 0) 0x7f00400c7000 con 0x7f004015ab60 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f004006d460 con 0x7f004015ab60 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314565965 0 0) 0x7f002c003570 con 0x7f00400ad6c0 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 >> v2:172.21.15.132:3300/0 conn(0x7f004006e8a0 msgr2=0x7f004006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 --2- 172.21.15.46:0/2197739714 >> v2:172.21.15.132:3300/0 conn(0x7f004006e8a0 0x7f004006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 >> v1:172.21.15.132:6789/0 conn(0x7f004015ab60 legacy=0x7f004015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400c72c0 con 0x7f00400ad6c0 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f00380017f0 con 0x7f00400ad6c0 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 >> v1:172.21.15.46:6789/0 conn(0x7f00400ad6c0 legacy=0x7f00400adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c003bf0 0x7f002c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 0x7f002c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f002c0051c0 0x7f002c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004006ed40 con 0x7f002c004810 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00c160 con 0x7f002c003bf0 2023-10-06T12:50:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00c320 con 0x7f002c0051c0 2023-10-06T12:50:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 0x7f002c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.788+0000 7f004628c700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c003bf0 0x7f002c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0046a8d700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f002c0051c0 0x7f002c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 -- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f002c0051c0 msgr2=0x7f002c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f002c0051c0 0x7f002c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 -- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c003bf0 msgr2=0x7f002c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c003bf0 0x7f002c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 -- 172.21.15.46:0/2197739714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c003570 con 0x7f002c004810 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0046a8d700 1 -- 172.21.15.46:0/2197739714 reap_dead start 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 0x7f002c004c80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0030006f00 tx=0x7f00300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54385b2faef50737 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000e710 con 0x7f002c004810 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003000e870 con 0x7f002c004810 2023-10-06T12:50:49.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f004528a700 1 -- 172.21.15.46:0/2197739714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000ec30 con 0x7f002c004810 2023-10-06T12:50:49.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 -- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 msgr2=0x7f002c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 0x7f002c004c80 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0030006f00 tx=0x7f00300091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 -- 172.21.15.46:0/2197739714 shutdown_connections 2023-10-06T12:50:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 --2- 172.21.15.46:0/2197739714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f002c004810 0x7f002c004c80 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 -- 172.21.15.46:0/2197739714 >> 172.21.15.46:0/2197739714 conn(0x7f00400cbe30 msgr2=0x7f00400cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 -- 172.21.15.46:0/2197739714 shutdown_connections 2023-10-06T12:50:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.792+0000 7f00484f0700 1 -- 172.21.15.46:0/2197739714 wait complete. 2023-10-06T12:50:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.796+0000 7f00484f0700 1 Processor -- start 2023-10-06T12:50:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.796+0000 7f00484f0700 1 -- start start 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 0x7f0040169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f004016a470 0x7f004016bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004016a230 con 0x7f00400c4750 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c9aa0 con 0x7f002c004810 2023-10-06T12:50:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00400c9d80 con 0x7f004016a470 2023-10-06T12:50:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47364/0 (socket says 172.21.15.46:47364) 2023-10-06T12:50:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 -- 172.21.15.46:0/256228443 learned_addr learned my addr 172.21.15.46:0/256228443 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f004628c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 0x7f0040169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0045a8b700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f004016a470 0x7f004016bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 -- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f004016a470 msgr2=0x7f004016bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f004016a470 0x7f004016bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 -- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 msgr2=0x7f0040169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 0x7f0040169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 -- 172.21.15.46:0/256228443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400ca060 con 0x7f00400c4750 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f004628c700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 0x7f0040169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f0046a8d700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f00400c8310 tx=0x7f00380013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00367fc700 1 -- 172.21.15.46:0/256228443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800eb60 con 0x7f00400c4750 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00367fc700 1 -- 172.21.15.46:0/256228443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003800ecc0 con 0x7f00400c4750 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00367fc700 1 -- 172.21.15.46:0/256228443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038013420 con 0x7f00400c4750 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00400ca340 con 0x7f00400c4750 2023-10-06T12:50:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.800+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004016c4a0 con 0x7f00400c4750 2023-10-06T12:50:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.804+0000 7f00367fc700 1 -- 172.21.15.46:0/256228443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f00380137d0 con 0x7f00400c4750 2023-10-06T12:50:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.804+0000 7f00367fc700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 0x7f0024062c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.804+0000 7f00367fc700 1 -- 172.21.15.46:0/256228443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f003807d0d0 con 0x7f00400c4750 2023-10-06T12:50:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004628c700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 0x7f0024062c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004628c700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 0x7f0024062c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00401508e0 tx=0x7f003c007d80 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 Processor -- start 2023-10-06T12:50:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 -- start start 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x120fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x12100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x113df30 con 0xf92ae0 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1139ed0 con 0xf92710 2023-10-06T12:50:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x11000b0 con 0xfdd790 2023-10-06T12:50:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004628c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004628c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47366/0 (socket says 172.21.15.46:47366) 2023-10-06T12:50:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f004628c700 1 -- 172.21.15.46:0/136096434 learned_addr learned my addr 172.21.15.46:0/136096434 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f0046a8d700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x12100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.808+0000 7f0045a8b700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x120fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 -- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 msgr2=0x12100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x12100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 -- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 msgr2=0x120fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x120fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 -- 172.21.15.46:0/136096434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfde9e0 con 0xf92ae0 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f0045a8b700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x120fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004628c700 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f003c012d50 tx=0x7f003c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ae16f350150a01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f00237fe700 1 -- 172.21.15.46:0/136096434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c015e70 con 0xf92ae0 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f00237fe700 1 -- 172.21.15.46:0/136096434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c012e50 con 0xf92ae0 2023-10-06T12:50:49.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f00237fe700 1 -- 172.21.15.46:0/136096434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c021640 con 0xf92ae0 2023-10-06T12:50:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- 172.21.15.46:0/136096434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 msgr2=0x120f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f003c012d50 tx=0x7f003c018000 comp rx=0 tx=0).stop 2023-10-06T12:50:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- 172.21.15.46:0/136096434 shutdown_connections 2023-10-06T12:50:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x12100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x120fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 --2- 172.21.15.46:0/136096434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x120f630 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- 172.21.15.46:0/136096434 >> 172.21.15.46:0/136096434 conn(0xf92eb0 msgr2=0x11cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- 172.21.15.46:0/136096434 shutdown_connections 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- 172.21.15.46:0/136096434 wait complete. 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 -- start start 2023-10-06T12:50:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.812+0000 7f004a7c1740 1 Processor -- start 2023-10-06T12:50:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x1213c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x11c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x10ffd50 con 0xf92ae0 2023-10-06T12:50:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x10d8970 con 0xf92710 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x10e3510 con 0xfdd790 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 10 client.0 _ll_register_callbacks cb 0x10dcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47368/0 (socket says 172.21.15.46:47368) 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 -- 172.21.15.46:0/180685984 learned_addr learned my addr 172.21.15.46:0/180685984 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004628c700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x1213c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0046a8d700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x11c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 -- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 msgr2=0x11c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x11c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 -- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 msgr2=0x1213c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x1213c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 -- 172.21.15.46:0/180685984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004006d460 con 0xf92ae0 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0045a8b700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c130 tx=0x7f003001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b32c2bc76536f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0020ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:50:49.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030022070 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x10d8cc0 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f0030013680 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0030006c90 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 10 client.10014 fetch_fsmap learned FSMap version 94 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x1108370 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.816+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x10d8cc0 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0030011070 con 0xf92ae0 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 10 client.10014 fetch_fsmap finished waiting for FSMap version 94 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 10 client.10014 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 20 client.10014 populate_metadata read hostname 'smithi046' 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11cb3c0 0x121ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 4 client.10014 mds_command: new command op to 9619 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T12:50:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 --> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x10dc3e0 con 0x11cb3c0 2023-10-06T12:50:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004628c700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11cb3c0 0x121ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f0020ff9700 10 client.10014 ms_handle_connect on v2:172.21.15.46:6834/192039016 2023-10-06T12:50:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004628c700 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11cb3c0 0x121ede0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e17193bc4d0b05 server_cookie=74ea4792ae5aa9aa in_seq=0 out_seq=0 2023-10-06T12:50:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f0020ff9700 1 -- 172.21.15.46:0/180685984 <== mds.0 v2:172.21.15.46:6834/192039016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x10dc3e0 con 0x11cb3c0 2023-10-06T12:50:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f0020ff9700 10 client.10014 handle_command_reply: tid=0 2023-10-06T12:50:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 client.10014 shutdown 2023-10-06T12:50:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 msgr2=0x1214180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c130 tx=0x7f003001a000 comp rx=0 tx=0).stop 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 shutdown_connections 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xfdd790 0x11c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92ae0 0x1214180 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11cb3c0 0x121ede0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 --2- 172.21.15.46:0/180685984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xf92710 0x1213c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 >> 172.21.15.46:0/180685984 conn(0xf92eb0 msgr2=0x11f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f0020ff9700 0 client.10014 ms_handle_reset on v2:172.21.15.46:6834/192039016 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 shutdown_connections 2023-10-06T12:50:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.820+0000 7f004a7c1740 1 -- 172.21.15.46:0/180685984 wait complete. 2023-10-06T12:50:49.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.824+0000 7f004a7c1740 20 client.10014 trim_cache size 0 max 16384 2023-10-06T12:50:49.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 Processor -- start 2023-10-06T12:50:49.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 -- start start 2023-10-06T12:50:49.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x11c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xfdd790 0x11eee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x110e610 con 0xf92710 2023-10-06T12:50:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x110f010 con 0xfdd790 2023-10-06T12:50:49.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.840+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x100f650 con 0xf92ae0 2023-10-06T12:50:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47376/0 (socket says 172.21.15.46:47376) 2023-10-06T12:50:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 -- 172.21.15.46:0/2911985085 learned_addr learned my addr 172.21.15.46:0/2911985085 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:49.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f0046a8d700 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xfdd790 0x11eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x11c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 -- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 msgr2=0x11c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x11c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 -- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xfdd790 msgr2=0x11eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xfdd790 0x11eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 -- 172.21.15.46:0/2911985085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x10eed80 con 0xf92710 2023-10-06T12:50:49.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004628c700 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x1137b50 tx=0x7f003c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53d523a24d1300b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f0023fff700 1 -- 172.21.15.46:0/2911985085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c015e70 con 0xf92710 2023-10-06T12:50:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f0023fff700 1 -- 172.21.15.46:0/2911985085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c032040 con 0xf92710 2023-10-06T12:50:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f0023fff700 1 -- 172.21.15.46:0/2911985085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c001030 con 0xf92710 2023-10-06T12:50:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- 172.21.15.46:0/2911985085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 msgr2=0x11c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x1137b50 tx=0x7f003c01c000 comp rx=0 tx=0).stop 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- 172.21.15.46:0/2911985085 shutdown_connections 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0xfdd790 0x11eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x11c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2911985085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xf92710 0x11c46b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- 172.21.15.46:0/2911985085 >> 172.21.15.46:0/2911985085 conn(0xf92eb0 msgr2=0x11cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- 172.21.15.46:0/2911985085 shutdown_connections 2023-10-06T12:50:49.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- 172.21.15.46:0/2911985085 wait complete. 2023-10-06T12:50:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.844+0000 7f004a7c1740 1 -- start start 2023-10-06T12:50:49.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 Processor -- start 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x12693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 0x11ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1138d60 con 0xfdd790 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x113a770 con 0x11ede50 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x10eb5a0 con 0xf92ae0 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 10 client.0 _ll_register_callbacks cb 0x1105110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47378/0 (socket says 172.21.15.46:47378) 2023-10-06T12:50:49.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 -- 172.21.15.46:0/2291145315 learned_addr learned my addr 172.21.15.46:0/2291145315 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0046a8d700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 0x11ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004628c700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x12693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 -- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 msgr2=0x12693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x12693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 -- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 msgr2=0x11ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 0x11ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 -- 172.21.15.46:0/2291145315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xfde9e0 con 0xfdd790 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0046a8d700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 0x11ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0045a8b700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0030005d60 tx=0x7f0030027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63edd6019aa507 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0020ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030022070 con 0xfdd790 2023-10-06T12:50:49.859 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f0030013680 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.848+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f003c012e50 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003000e620 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 10 client.10020 fetch_fsmap learned FSMap version 94 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x10e3130 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c012e50 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f003002f6c0 con 0xfdd790 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 10 client.10020 fetch_fsmap finished waiting for FSMap version 94 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 10 client.10020 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 20 client.10020 populate_metadata read hostname 'smithi046' 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11eebb0 0x11eef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 4 client.10020 mds_command: new command op to 9619 tid=0[{"prefix": "status"}] 2023-10-06T12:50:49.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 --> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] -- command(tid 0: {"prefix": "status"}) v1 -- 0x110e410 con 0x11eebb0 2023-10-06T12:50:49.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004628c700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11eebb0 0x11eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004628c700 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11eebb0 0x11eef80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8d2beaa84b565 server_cookie=96e51a38daa725cd in_seq=0 out_seq=0 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 10 client.10020 ms_handle_connect on v2:172.21.15.46:6834/192039016 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 1 -- 172.21.15.46:0/2291145315 <== mds.0 v2:172.21.15.46:6834/192039016 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x110e410 con 0x11eebb0 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 10 client.10020 handle_command_reply: tid=0 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 client.10020 shutdown 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 msgr2=0x1269910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0030005d60 tx=0x7f0030027000 comp rx=0 tx=0).stop 2023-10-06T12:50:49.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 shutdown_connections 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:6834/192039016,v1:172.21.15.46:6835/192039016] conn(0x11eebb0 0x11eef80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x11ede50 0x11ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0xfdd790 0x1269910 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 --2- 172.21.15.46:0/2291145315 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0xf92ae0 0x12693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f0020ff9700 0 client.10020 ms_handle_reset on v2:172.21.15.46:6834/192039016 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 >> 172.21.15.46:0/2291145315 conn(0xf92eb0 msgr2=0x11f3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 shutdown_connections 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 1 -- 172.21.15.46:0/2291145315 wait complete. 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.852+0000 7f004a7c1740 20 client.10020 trim_cache size 0 max 16384 2023-10-06T12:50:49.863 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9619, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 8.5650366850000008, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 94, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 183, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 183, 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 14.347505483000001 2023-10-06T12:50:49.864 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:50:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 msgr2=0x7f0024062c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 0x7f0024062c20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00401508e0 tx=0x7f003c007d80 comp rx=0 tx=0).stop 2023-10-06T12:50:49.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 msgr2=0x7f00400c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f00400c8310 tx=0x7f00380013b0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 shutdown_connections 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00400c4750 0x7f00400c1760 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f004016a470 0x7f004016bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0024060770 0x7f0024062c20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 --2- 172.21.15.46:0/256228443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f002c004810 0x7f0040169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 >> 172.21.15.46:0/256228443 conn(0x7f00400cbe30 msgr2=0x7f0040069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 shutdown_connections 2023-10-06T12:50:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:49.856+0000 7f00484f0700 1 -- 172.21.15.46:0/256228443 wait complete. 2023-10-06T12:50:49.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T12:50:50.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 Processor -- start 2023-10-06T12:50:50.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 -- start start 2023-10-06T12:50:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d980d0aa0 0x7f6d98159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d980b5410 0x7f6d980b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d980b4f80 con 0x7f6d98159050 2023-10-06T12:50:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d980b40c0 con 0x7f6d980b5410 2023-10-06T12:50:50.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.052+0000 7f6d9eaa5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980b4370 con 0x7f6d980d0aa0 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d980b5410 0x7f6d980b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d980d0aa0 0x7f6d98159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9c841700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6d98159050 0x7f6d980b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:35802/0 (socket says 172.21.15.46:35802) 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d980b5410 0x7f6d980b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47388/0 (socket says 172.21.15.46:47388) 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9c841700 1 -- 172.21.15.46:0/3723080985 learned_addr learned my addr 172.21.15.46:0/3723080985 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:50.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 -- 172.21.15.46:0/3723080985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d980d0aa0 msgr2=0x7f6d98159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:50.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 --2- 172.21.15.46:0/3723080985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d980d0aa0 0x7f6d98159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 -- 172.21.15.46:0/3723080985 >> v1:172.21.15.132:6789/0 conn(0x7f6d98159050 legacy=0x7f6d980b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 -- 172.21.15.46:0/3723080985 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d980b4650 con 0x7f6d980b5410 2023-10-06T12:50:50.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9d042700 1 --2- 172.21.15.46:0/3723080985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d980b1fc0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e60 tx=0x7f6d90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2bc5e548ef09d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d977fe700 1 -- 172.21.15.46:0/3723080985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000e070 con 0x7f6d980b5410 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d977fe700 1 -- 172.21.15.46:0/3723080985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9000ba60 con 0x7f6d980b5410 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d977fe700 1 -- 172.21.15.46:0/3723080985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000be20 con 0x7f6d980b5410 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/3723080985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 msgr2=0x7f6d980b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/3723080985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d980b1fc0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e60 tx=0x7f6d90009780 comp rx=0 tx=0).stop 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/3723080985 shutdown_connections 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/3723080985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d980b1fc0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/3723080985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d980d0aa0 0x7f6d98159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/3723080985 >> 172.21.15.46:0/3723080985 conn(0x7f6d9806e780 msgr2=0x7f6d9806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/3723080985 shutdown_connections 2023-10-06T12:50:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.056+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/3723080985 wait complete. 2023-10-06T12:50:50.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 Processor -- start 2023-10-06T12:50:50.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 -- start start 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 0x7f6d980c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d98159050 0x7f6d981503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9816dc20 con 0x7f6d980b5410 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981513c0 con 0x7f6d980d0aa0 2023-10-06T12:50:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9eaa5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981516a0 con 0x7f6d98159050 2023-10-06T12:50:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9c841700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9c841700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47390/0 (socket says 172.21.15.46:47390) 2023-10-06T12:50:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9c841700 1 -- 172.21.15.46:0/949029127 learned_addr learned my addr 172.21.15.46:0/949029127 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d97fff700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 0x7f6d980c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.060+0000 7f6d9d042700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d98159050 0x7f6d981503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 -- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d98159050 msgr2=0x7f6d981503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d98159050 0x7f6d981503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 -- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 msgr2=0x7f6d980c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 0x7f6d980c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 -- 172.21.15.46:0/949029127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90009a20 con 0x7f6d980b5410 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d97fff700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 0x7f6d980c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9c841700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c810 tx=0x7f6d8c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c005b30 con 0x7f6d980b5410 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c005cb0 con 0x7f6d980b5410 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d98151980 con 0x7f6d980b5410 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c007320 con 0x7f6d980b5410 2023-10-06T12:50:50.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980c5670 con 0x7f6d980b5410 2023-10-06T12:50:50.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.064+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d980cf5b0 con 0x7f6d980b5410 2023-10-06T12:50:50.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.068+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6d8c01b3b0 con 0x7f6d980b5410 2023-10-06T12:50:50.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.068+0000 7f6d94ff9700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 0x7f6d7c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:50.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.068+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f6d8c07c1e0 con 0x7f6d980b5410 2023-10-06T12:50:50.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.068+0000 7f6d97fff700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 0x7f6d7c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:50.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.068+0000 7f6d97fff700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 0x7f6d7c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d9816ce50 tx=0x7f6d840091a0 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.072+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6d980cf5b0 con 0x7f6d980b5410 2023-10-06T12:50:50.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.276+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6d980cf5b0 con 0x7f6d980b5410 2023-10-06T12:50:50.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.276+0000 7f6d94ff9700 1 -- 172.21.15.46:0/949029127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6d980cf5b0 con 0x7f6d980b5410 2023-10-06T12:50:50.285 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T12:50:50.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 msgr2=0x7f6d7c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 0x7f6d7c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d9816ce50 tx=0x7f6d840091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 msgr2=0x7f6d981599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:50.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c810 tx=0x7f6d8c007ef0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 shutdown_connections 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d98159050 0x7f6d981503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d7c060670 0x7f6d7c062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d980d0aa0 0x7f6d980c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 --2- 172.21.15.46:0/949029127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d980b5410 0x7f6d981599c0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 >> 172.21.15.46:0/949029127 conn(0x7f6d9806e780 msgr2=0x7f6d98153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 shutdown_connections 2023-10-06T12:50:50.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:50.280+0000 7f6d9eaa5700 1 -- 172.21.15.46:0/949029127 wait complete. 2023-10-06T12:50:50.308 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T12:50:50.308 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T12:50:50.309 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:50:50.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.309 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T12:50:50.327 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 36880sec preferred_lft 36880sec 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.328 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T12:50:50.329 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T12:50:50.330 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:50:50.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.332 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T12:50:50.415 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T12:50:50.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.418 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T12:50:50.486 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T12:50:50.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T12:50:50.487 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T12:50:50.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.624 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:50:50.639 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T12:50:50.639 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T12:50:50.640 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:50:50.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:50.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T12:50:50.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:50:50.718 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T12:50:50.778 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:50:50.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:50:50.779 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:50:50.838 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T12:50:50.838 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T12:50:50.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:50:50.964 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:50:50.964 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:50:50.964 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:50:50.992 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:50:50.992 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:50:50.992 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:50:50.992 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:50:50.993 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:50:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:50:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:50:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:50:51.009 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:50:51.010 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:50:51.010 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T12:50:51.010 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T12:50:51.010 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T12:50:51.015 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T12:50:51.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:50:51.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:50:51.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T12:50:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.252+0000 7f3ae446e700 1 Processor -- start 2023-10-06T12:50:51.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.252+0000 7f3ae446e700 1 -- start start 2023-10-06T12:50:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae446e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc0c2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae446e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 0x7f3adc0c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae446e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc06dd30 con 0x7f3adc159260 2023-10-06T12:50:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae446e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc06ce80 con 0x7f3adc155850 2023-10-06T12:50:51.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae446e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc06d120 con 0x7f3adc155480 2023-10-06T12:50:51.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae1a09700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 0x7f3adc0c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc0c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae1a09700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 0x7f3adc0c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47396/0 (socket says 172.21.15.46:47396) 2023-10-06T12:50:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae1a09700 1 -- 172.21.15.46:0/2618761309 learned_addr learned my addr 172.21.15.46:0/2618761309 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 -- 172.21.15.46:0/2618761309 >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 msgr2=0x7f3adc0c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 --2- 172.21.15.46:0/2618761309 >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 0x7f3adc0c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 -- 172.21.15.46:0/2618761309 >> v1:172.21.15.132:6789/0 conn(0x7f3adc159260 legacy=0x7f3adc1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 -- 172.21.15.46:0/2618761309 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc06d400 con 0x7f3adc155480 2023-10-06T12:50:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae220a700 1 --2- 172.21.15.46:0/2618761309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc0c2300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800dcf0 tx=0x7f3ad800d5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c474062628b292fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.256+0000 7f3ae1208700 1 -- 172.21.15.46:0/2618761309 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8012070 con 0x7f3adc155480 2023-10-06T12:50:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae1208700 1 -- 172.21.15.46:0/2618761309 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8005a00 con 0x7f3adc155480 2023-10-06T12:50:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae1208700 1 -- 172.21.15.46:0/2618761309 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8005da0 con 0x7f3adc155480 2023-10-06T12:50:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 -- 172.21.15.46:0/2618761309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 msgr2=0x7f3adc0c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2618761309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc0c2300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800dcf0 tx=0x7f3ad800d5e0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 -- 172.21.15.46:0/2618761309 shutdown_connections 2023-10-06T12:50:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2618761309 >> v2:172.21.15.46:3300/0 conn(0x7f3adc155850 0x7f3adc0c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2618761309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc0c2300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 -- 172.21.15.46:0/2618761309 >> 172.21.15.46:0/2618761309 conn(0x7f3adc0cbe30 msgr2=0x7f3adc0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 -- 172.21.15.46:0/2618761309 shutdown_connections 2023-10-06T12:50:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 -- 172.21.15.46:0/2618761309 wait complete. 2023-10-06T12:50:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.260+0000 7f3ae446e700 1 Processor -- start 2023-10-06T12:50:51.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 -- start start 2023-10-06T12:50:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3adc159260 0x7f3adc157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc14f2e0 con 0x7f3adc155850 2023-10-06T12:50:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc156cd0 con 0x7f3adc159260 2023-10-06T12:50:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae446e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc166820 con 0x7f3adc155480 2023-10-06T12:50:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47400/0 (socket says 172.21.15.46:47400) 2023-10-06T12:50:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae220a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 -- 172.21.15.46:0/2850083234 learned_addr learned my addr 172.21.15.46:0/2850083234 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae2a0b700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3adc159260 0x7f3adc157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 -- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 msgr2=0x7f3adc162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc162ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 -- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3adc159260 msgr2=0x7f3adc157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3adc159260 0x7f3adc157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 -- 172.21.15.46:0/2850083234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad800d860 con 0x7f3adc155850 2023-10-06T12:50:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.264+0000 7f3ae1a09700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00ea50 tx=0x7f3acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00c640 con 0x7f3adc155850 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc1560c0 con 0x7f3adc155850 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc00c7a0 con 0x7f3adc155850 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00cb20 con 0x7f3adc155850 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc156680 con 0x7f3adc155850 2023-10-06T12:50:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc063f90 con 0x7f3adc155850 2023-10-06T12:50:51.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.268+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3acc013470 con 0x7f3adc155850 2023-10-06T12:50:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.272+0000 7f3ad27fc700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 0x7f3ac0062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.272+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f3acc07cf60 con 0x7f3adc155850 2023-10-06T12:50:51.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.272+0000 7f3ae220a700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 0x7f3ac0062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.272+0000 7f3ae220a700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 0x7f3ac0062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d620 tx=0x7f3ad801b000 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.276+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3acc04d020 con 0x7f3adc155850 2023-10-06T12:50:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.468+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3adc156e10 con 0x7f3ac0060670 2023-10-06T12:50:51.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.472+0000 7f3ad27fc700 1 -- 172.21.15.46:0/2850083234 <== mgr.9666 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3adc156e10 con 0x7f3ac0060670 2023-10-06T12:50:51.479 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:50:51.480 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:50:51.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T12:50:51.480 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:50:51.480 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:50:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 msgr2=0x7f3ac0062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 0x7f3ac0062b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d620 tx=0x7f3ad801b000 comp rx=0 tx=0).stop 2023-10-06T12:50:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 msgr2=0x7f3adc1662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00ea50 tx=0x7f3acc009250 comp rx=0 tx=0).stop 2023-10-06T12:50:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 shutdown_connections 2023-10-06T12:50:51.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3ac0060670 0x7f3ac0062b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3adc159260 0x7f3adc157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3adc155850 0x7f3adc1662e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 --2- 172.21.15.46:0/2850083234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3adc155480 0x7f3adc162ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 >> 172.21.15.46:0/2850083234 conn(0x7f3adc0cbe30 msgr2=0x7f3adc06a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 shutdown_connections 2023-10-06T12:50:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.476+0000 7f3ae446e700 1 -- 172.21.15.46:0/2850083234 wait complete. 2023-10-06T12:50:51.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T12:50:51.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.664+0000 7f5954d45700 1 Processor -- start 2023-10-06T12:50:51.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.664+0000 7f5954d45700 1 -- start start 2023-10-06T12:50:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5954d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5954d45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500af6f0 con 0x7f59500aee00 2023-10-06T12:50:51.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5954d45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500b43e0 con 0x7f5950159550 2023-10-06T12:50:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5954d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500b04b0 con 0x7f59500b2360 2023-10-06T12:50:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5947fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f59500aee00 0x7f5950158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38110/0 (socket says 172.21.15.46:38110) 2023-10-06T12:50:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f5947fff700 1 -- 172.21.15.46:0/1617819385 learned_addr learned my addr 172.21.15.46:0/1617819385 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:51.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f594e59c700 1 --2- 172.21.15.46:0/1617819385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.668+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907718355 0 0) 0x7f59500af6f0 con 0x7f59500aee00 2023-10-06T12:50:51.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f593c003570 con 0x7f59500aee00 2023-10-06T12:50:51.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288186531 0 0) 0x7f59500b43e0 con 0x7f5950159550 2023-10-06T12:50:51.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59500af6f0 con 0x7f5950159550 2023-10-06T12:50:51.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594e59c700 1 -- 172.21.15.46:0/1617819385 >> v1:172.21.15.132:6789/0 conn(0x7f5950159550 legacy=0x7f59500b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594e59c700 1 -- 172.21.15.46:0/1617819385 >> v1:172.21.15.46:6789/0 conn(0x7f59500aee00 legacy=0x7f5950158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594e59c700 1 -- 172.21.15.46:0/1617819385 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500b0790 con 0x7f59500b2360 2023-10-06T12:50:51.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594e59c700 1 --2- 172.21.15.46:0/1617819385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500b2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5940009940 tx=0x7f59400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9be1a3ff87ec71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011630 con 0x7f59500b2360 2023-10-06T12:50:51.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940011790 con 0x7f59500b2360 2023-10-06T12:50:51.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f594dd9b700 1 -- 172.21.15.46:0/1617819385 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011b30 con 0x7f59500b2360 2023-10-06T12:50:51.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 -- 172.21.15.46:0/1617819385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 msgr2=0x7f59500b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 --2- 172.21.15.46:0/1617819385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500b2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5940009940 tx=0x7f59400091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 -- 172.21.15.46:0/1617819385 shutdown_connections 2023-10-06T12:50:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 --2- 172.21.15.46:0/1617819385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500b2730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 -- 172.21.15.46:0/1617819385 >> 172.21.15.46:0/1617819385 conn(0x7f595006e780 msgr2=0x7f595006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 -- 172.21.15.46:0/1617819385 shutdown_connections 2023-10-06T12:50:51.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.672+0000 7f5954d45700 1 -- 172.21.15.46:0/1617819385 wait complete. 2023-10-06T12:50:51.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 Processor -- start 2023-10-06T12:50:51.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 -- start start 2023-10-06T12:50:51.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5950159550 0x7f59500c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500ccf70 con 0x7f59500aee00 2023-10-06T12:50:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015ad70 con 0x7f5950159550 2023-10-06T12:50:51.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5954d45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500cc9b0 con 0x7f59500b2360 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f594e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f594e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47408/0 (socket says 172.21.15.46:47408) 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f594e59c700 1 -- 172.21.15.46:0/836400955 learned_addr learned my addr 172.21.15.46:0/836400955 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f594ed9d700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5950159550 0x7f59500c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.676+0000 7f5947fff700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 -- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 msgr2=0x7f59500cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 -- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5950159550 msgr2=0x7f59500c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:51.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5950159550 0x7f59500c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:51.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 -- 172.21.15.46:0/836400955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940009420 con 0x7f59500aee00 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f594e59c700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f59400093f0 tx=0x7f5940017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594001f070 con 0x7f59500aee00 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940006b10 con 0x7f59500aee00 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011900 con 0x7f59500aee00 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500ccc90 con 0x7f59500aee00 2023-10-06T12:50:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5950155b70 con 0x7f59500aee00 2023-10-06T12:50:51.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.680+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950088840 con 0x7f59500aee00 2023-10-06T12:50:51.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.684+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 100) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5940023070 con 0x7f59500aee00 2023-10-06T12:50:51.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.684+0000 7f59467fc700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 0x7f5930062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.684+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f594007d9d0 con 0x7f59500aee00 2023-10-06T12:50:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.688+0000 7f5947fff700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 0x7f5930062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.688+0000 7f5947fff700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 0x7f5930062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59500c90c0 tx=0x7f5938009040 comp rx=0 tx=0).ready entity=mgr.9666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:51.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.692+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f594004db40 con 0x7f59500aee00 2023-10-06T12:50:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:51.904+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f59500b0a70 con 0x7f59500aee00 2023-10-06T12:50:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.860+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 101) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f594004df10 con 0x7f59500aee00 2023-10-06T12:50:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.860+0000 7f59467fc700 1 -- 172.21.15.46:0/836400955 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v101) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5940060850 con 0x7f59500aee00 2023-10-06T12:50:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 msgr2=0x7f5930062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 0x7f5930062b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59500c90c0 tx=0x7f5938009040 comp rx=0 tx=0).stop 2023-10-06T12:50:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 msgr2=0x7f5950155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f59400093f0 tx=0x7f5940017000 comp rx=0 tx=0).stop 2023-10-06T12:50:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 shutdown_connections 2023-10-06T12:50:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f59300606a0 0x7f5930062b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5950159550 0x7f59500c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59500b2360 0x7f59500cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 --2- 172.21.15.46:0/836400955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59500aee00 0x7f5950155630 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 >> 172.21.15.46:0/836400955 conn(0x7f595006e780 msgr2=0x7f59500b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 shutdown_connections 2023-10-06T12:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:52.864+0000 7f5954d45700 1 -- 172.21.15.46:0/836400955 wait complete. 2023-10-06T12:50:52.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T12:50:53.045 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:53.041+0000 7f79289ce000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:50:53.071 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:53.064+0000 7ff5f8ceb000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:50:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.100+0000 7fa536a4c700 1 Processor -- start 2023-10-06T12:50:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.100+0000 7fa536a4c700 1 -- start start 2023-10-06T12:50:53.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.100+0000 7fa536a4c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5300adbe0 0x7fa5300d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.100+0000 7fa536a4c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5300b0dc0 0x7fa5300b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5300b1700 con 0x7fa5300d0d60 2023-10-06T12:50:53.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa530159740 con 0x7fa5300b0dc0 2023-10-06T12:50:53.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5300afd20 con 0x7fa5300adbe0 2023-10-06T12:50:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5300adbe0 0x7fa5300d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa53624b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5300b0dc0 0x7fa5300b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5300adbe0 0x7fa5300d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43192/0 (socket says 172.21.15.46:43192) 2023-10-06T12:50:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 -- 172.21.15.46:0/154664211 learned_addr learned my addr 172.21.15.46:0/154664211 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535249700 1 --1- 172.21.15.46:0/154664211 >> v1:172.21.15.46:6789/0 conn(0x7fa5300d0d60 0x7fa5300b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51200/0 (socket says 172.21.15.46:51200) 2023-10-06T12:50:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 -- 172.21.15.46:0/154664211 >> v2:172.21.15.46:3300/0 conn(0x7fa5300b0dc0 msgr2=0x7fa5300b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 --2- 172.21.15.46:0/154664211 >> v2:172.21.15.46:3300/0 conn(0x7fa5300b0dc0 0x7fa5300b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 -- 172.21.15.46:0/154664211 >> v1:172.21.15.46:6789/0 conn(0x7fa5300d0d60 legacy=0x7fa5300b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 -- 172.21.15.46:0/154664211 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300b0000 con 0x7fa5300adbe0 2023-10-06T12:50:53.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa535a4a700 1 --2- 172.21.15.46:0/154664211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300adbe0 0x7fa5300d0820 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa53006a0c0 tx=0x7fa52c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05dfb2f12b66aaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa534a48700 1 -- 172.21.15.46:0/154664211 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c009e70 con 0x7fa5300adbe0 2023-10-06T12:50:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa534a48700 1 -- 172.21.15.46:0/154664211 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00b860 con 0x7fa5300adbe0 2023-10-06T12:50:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa534a48700 1 -- 172.21.15.46:0/154664211 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00bc00 con 0x7fa5300adbe0 2023-10-06T12:50:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- 172.21.15.46:0/154664211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300adbe0 msgr2=0x7fa5300d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 --2- 172.21.15.46:0/154664211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300adbe0 0x7fa5300d0820 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa53006a0c0 tx=0x7fa52c009250 comp rx=0 tx=0).stop 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- 172.21.15.46:0/154664211 shutdown_connections 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 --2- 172.21.15.46:0/154664211 >> v2:172.21.15.46:3300/0 conn(0x7fa5300b0dc0 0x7fa5300b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 --2- 172.21.15.46:0/154664211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300adbe0 0x7fa5300d0820 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.104+0000 7fa536a4c700 1 -- 172.21.15.46:0/154664211 >> 172.21.15.46:0/154664211 conn(0x7fa53006ef60 msgr2=0x7fa5300d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- 172.21.15.46:0/154664211 shutdown_connections 2023-10-06T12:50:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- 172.21.15.46:0/154664211 wait complete. 2023-10-06T12:50:53.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 Processor -- start 2023-10-06T12:50:53.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- start start 2023-10-06T12:50:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53015ca00 con 0x7fa5300b0dc0 2023-10-06T12:50:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53009bf70 con 0x7fa5300d0d60 2023-10-06T12:50:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.108+0000 7fa536a4c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa530155f80 con 0x7fa5300adbe0 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535249700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535a4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535a4a700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55426/0 (socket says 172.21.15.46:55426) 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:43200/0 (socket says 172.21.15.46:43200) 2023-10-06T12:50:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535249700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37604/0 (socket says 172.21.15.46:37604) 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535a4a700 1 -- 172.21.15.46:0/920502508 learned_addr learned my addr 172.21.15.46:0/920502508 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 -- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 msgr2=0x7fa53009ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 -- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 msgr2=0x7fa53015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 -- 172.21.15.46:0/920502508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0094f0 con 0x7fa5300d0d60 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535a4a700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa535249700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:50:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa53624b700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa528009e50 tx=0x7fa52800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800caa0 con 0x7fa5300d0d60 2023-10-06T12:50:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52800f040 con 0x7fa5300d0d60 2023-10-06T12:50:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53009b360 con 0x7fa5300d0d60 2023-10-06T12:50:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52800ce50 con 0x7fa5300d0d60 2023-10-06T12:50:53.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.112+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53009b920 con 0x7fa5300d0d60 2023-10-06T12:50:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.116+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa530064590 con 0x7fa5300d0d60 2023-10-06T12:50:53.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.116+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 101) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fa528014690 con 0x7fa5300d0d60 2023-10-06T12:50:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.120+0000 7fa525ffb700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 0x7fa514062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.120+0000 7fa535a4a700 1 -- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 msgr2=0x7fa514062ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.120+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fa52807da00 con 0x7fa5300d0d60 2023-10-06T12:50:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.120+0000 7fa535a4a700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 0x7fa514062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:53.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.128+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa52804d9b0 con 0x7fa5300d0d60 2023-10-06T12:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.296+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fa53009c0b0 con 0x7fa5300d0d60 2023-10-06T12:50:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.308+0000 7fa525ffb700 1 -- 172.21.15.46:0/920502508 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5280180d0 con 0x7fa5300d0d60 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.312+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 msgr2=0x7fa514062ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.312+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 0x7fa514062ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.312+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 msgr2=0x7fa530155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.312+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa528009e50 tx=0x7fa52800b040 comp rx=0 tx=0).stop 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 shutdown_connections 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa5140606f0 0x7fa514062ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5300d0d60 0x7fa530155a40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5300b0dc0 0x7fa53015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 --2- 172.21.15.46:0/920502508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5300adbe0 0x7fa53009ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 >> 172.21.15.46:0/920502508 conn(0x7fa53006ef60 msgr2=0x7fa530068200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 shutdown_connections 2023-10-06T12:50:53.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.316+0000 7fa536a4c700 1 -- 172.21.15.46:0/920502508 wait complete. 2023-10-06T12:50:53.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T12:50:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.504+0000 7fc11948e700 1 Processor -- start 2023-10-06T12:50:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.504+0000 7fc11948e700 1 -- start start 2023-10-06T12:50:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc11948e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc11948e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc1140b0dc0 0x7fc1140b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc11948e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1140afd20 con 0x7fc1140d0d60 2023-10-06T12:50:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc11948e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc114159740 con 0x7fc1140b0dc0 2023-10-06T12:50:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc11948e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140b1760 con 0x7fc1140adbe0 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc113fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc113fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55432/0 (socket says 172.21.15.46:55432) 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc113fff700 1 -- 172.21.15.46:0/3292669917 learned_addr learned my addr 172.21.15.46:0/3292669917 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.508+0000 7fc118c8d700 1 --2- 172.21.15.46:0/3292669917 >> v2:172.21.15.132:3300/0 conn(0x7fc1140b0dc0 0x7fc1140b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc113fff700 1 -- 172.21.15.46:0/3292669917 >> v2:172.21.15.132:3300/0 conn(0x7fc1140b0dc0 msgr2=0x7fc1140b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc113fff700 1 --2- 172.21.15.46:0/3292669917 >> v2:172.21.15.132:3300/0 conn(0x7fc1140b0dc0 0x7fc1140b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc113fff700 1 -- 172.21.15.46:0/3292669917 >> v1:172.21.15.132:6789/0 conn(0x7fc1140d0d60 legacy=0x7fc1140b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc113fff700 1 -- 172.21.15.46:0/3292669917 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140b0000 con 0x7fc1140adbe0 2023-10-06T12:50:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc113fff700 1 --2- 172.21.15.46:0/3292669917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc1040099b0 tx=0x7fc104009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b95084a3c5290eaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc112ffd700 1 -- 172.21.15.46:0/3292669917 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104009ea0 con 0x7fc1140adbe0 2023-10-06T12:50:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc112ffd700 1 -- 172.21.15.46:0/3292669917 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400b860 con 0x7fc1140adbe0 2023-10-06T12:50:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc112ffd700 1 -- 172.21.15.46:0/3292669917 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400bc00 con 0x7fc1140adbe0 2023-10-06T12:50:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 -- 172.21.15.46:0/3292669917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 msgr2=0x7fc1140d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 --2- 172.21.15.46:0/3292669917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc1040099b0 tx=0x7fc104009250 comp rx=0 tx=0).stop 2023-10-06T12:50:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 -- 172.21.15.46:0/3292669917 shutdown_connections 2023-10-06T12:50:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 --2- 172.21.15.46:0/3292669917 >> v2:172.21.15.132:3300/0 conn(0x7fc1140b0dc0 0x7fc1140b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 --2- 172.21.15.46:0/3292669917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc1140d0820 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 -- 172.21.15.46:0/3292669917 >> 172.21.15.46:0/3292669917 conn(0x7fc11406ef60 msgr2=0x7fc1140d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 -- 172.21.15.46:0/3292669917 shutdown_connections 2023-10-06T12:50:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.512+0000 7fc11948e700 1 -- 172.21.15.46:0/3292669917 wait complete. 2023-10-06T12:50:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 Processor -- start 2023-10-06T12:50:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 -- start start 2023-10-06T12:50:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1140b0dc0 0x7fc11414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1140d0d60 0x7fc11414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11415cb90 con 0x7fc1140b0dc0 2023-10-06T12:50:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1141567c0 con 0x7fc1140d0d60 2023-10-06T12:50:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.516+0000 7fc11948e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11415ae80 con 0x7fc1140adbe0 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc118c8d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1140d0d60 0x7fc11414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc1137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1140b0dc0 0x7fc11414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55440/0 (socket says 172.21.15.46:55440) 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 -- 172.21.15.46:0/415845400 learned_addr learned my addr 172.21.15.46:0/415845400 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 -- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1140d0d60 msgr2=0x7fc11414fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1140d0d60 0x7fc11414fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 -- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1140b0dc0 msgr2=0x7fc11414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1140b0dc0 0x7fc11414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 -- 172.21.15.46:0/415845400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040094f0 con 0x7fc1140adbe0 2023-10-06T12:50:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc113fff700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b3c0 tx=0x7fc104017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104000d10 con 0x7fc1140adbe0 2023-10-06T12:50:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400bbe0 con 0x7fc1140adbe0 2023-10-06T12:50:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc114155650 con 0x7fc1140adbe0 2023-10-06T12:50:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104011690 con 0x7fc1140adbe0 2023-10-06T12:50:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.520+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11415a960 con 0x7fc1140adbe0 2023-10-06T12:50:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.524+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11415ab80 con 0x7fc1140adbe0 2023-10-06T12:50:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.524+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 101) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fc104046ae0 con 0x7fc1140adbe0 2023-10-06T12:50:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.528+0000 7fc0fbfff700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 0x7fc100062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.528+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fc10407e790 con 0x7fc1140adbe0 2023-10-06T12:50:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.528+0000 7fc1137fe700 1 -- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 msgr2=0x7fc100062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.528+0000 7fc1137fe700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 0x7fc100062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.532+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc10404e8d0 con 0x7fc1140adbe0 2023-10-06T12:50:53.576 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:53.573+0000 7f79289ce000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:50:53.647 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:53.640+0000 7ff5f8ceb000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:50:53.648 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:53.645+0000 7f79289ce000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:50:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.704+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc114064590 con 0x7fc1140adbe0 2023-10-06T12:50:53.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.716+0000 7fc0fbfff700 1 -- 172.21.15.46:0/415845400 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc104051ef0 con 0x7fc1140adbe0 2023-10-06T12:50:53.724 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:53.716+0000 7ff5f8ceb000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 msgr2=0x7fc100062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 0x7fc100062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 msgr2=0x7fc114156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b3c0 tx=0x7fc104017000 comp rx=0 tx=0).stop 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 shutdown_connections 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc1000606c0 0x7fc100062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1140d0d60 0x7fc11414fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1140b0dc0 0x7fc11414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 --2- 172.21.15.46:0/415845400 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140adbe0 0x7fc114156280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 >> 172.21.15.46:0/415845400 conn(0x7fc11406ef60 msgr2=0x7fc1140b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 shutdown_connections 2023-10-06T12:50:53.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.720+0000 7fc11948e700 1 -- 172.21.15.46:0/415845400 wait complete. 2023-10-06T12:50:53.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:53.749 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T12:50:53.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T12:50:53.784 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:53.781+0000 7f79289ce000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:53.883 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:53.876+0000 7ff5f8ceb000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.980+0000 7fd51700c700 1 Processor -- start 2023-10-06T12:50:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.980+0000 7fd51700c700 1 -- start start 2023-10-06T12:50:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51700c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd510067760 0x7fd5101572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51700c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100adde0 con 0x7fd510069910 2023-10-06T12:50:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51700c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd510158450 con 0x7fd510156f20 2023-10-06T12:50:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51700c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd510068b50 con 0x7fd510067760 2023-10-06T12:50:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515809700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd510067760 0x7fd5101572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51680b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd510069910 0x7fd510157830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51206/0 (socket says 172.21.15.46:51206) 2023-10-06T12:50:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd51680b700 1 -- 172.21.15.46:0/1177698218 learned_addr learned my addr 172.21.15.46:0/1177698218 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:50:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515809700 1 -- 172.21.15.46:0/1177698218 >> v1:172.21.15.132:6789/0 conn(0x7fd510156f20 legacy=0x7fd510067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515809700 1 -- 172.21.15.46:0/1177698218 >> v1:172.21.15.46:6789/0 conn(0x7fd510069910 legacy=0x7fd510157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515809700 1 -- 172.21.15.46:0/1177698218 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510068e30 con 0x7fd510067760 2023-10-06T12:50:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515809700 1 --2- 172.21.15.46:0/1177698218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510067760 0x7fd5101572f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd5000098e0 tx=0x7fd5000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50daedf767445b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515008700 1 -- 172.21.15.46:0/1177698218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500009e80 con 0x7fd510067760 2023-10-06T12:50:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515008700 1 -- 172.21.15.46:0/1177698218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000b860 con 0x7fd510067760 2023-10-06T12:50:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.984+0000 7fd515008700 1 -- 172.21.15.46:0/1177698218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000bbe0 con 0x7fd510067760 2023-10-06T12:50:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- 172.21.15.46:0/1177698218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510067760 msgr2=0x7fd5101572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:53.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 --2- 172.21.15.46:0/1177698218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510067760 0x7fd5101572f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd5000098e0 tx=0x7fd5000091a0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- 172.21.15.46:0/1177698218 shutdown_connections 2023-10-06T12:50:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 --2- 172.21.15.46:0/1177698218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510067760 0x7fd5101572f0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- 172.21.15.46:0/1177698218 >> 172.21.15.46:0/1177698218 conn(0x7fd5100cbbb0 msgr2=0x7fd5100cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- 172.21.15.46:0/1177698218 shutdown_connections 2023-10-06T12:50:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- 172.21.15.46:0/1177698218 wait complete. 2023-10-06T12:50:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 Processor -- start 2023-10-06T12:50:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.988+0000 7fd51700c700 1 -- start start 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 0x7fd5100ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd510069910 0x7fd5100c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510169a50 con 0x7fd510156f20 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510169d30 con 0x7fd510067760 2023-10-06T12:50:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51016a010 con 0x7fd510069910 2023-10-06T12:50:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37622/0 (socket says 172.21.15.46:37622) 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 -- 172.21.15.46:0/3511671778 learned_addr learned my addr 172.21.15.46:0/3511671778 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd515809700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd510069910 0x7fd5100c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 0x7fd5100ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd510069910 msgr2=0x7fd5100c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd510069910 0x7fd5100c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 msgr2=0x7fd5100ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 0x7fd5100ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500009420 con 0x7fd510156f20 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 0x7fd5100ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51680b700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cc70 tx=0x7fd508001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800a8e0 con 0x7fd510156f20 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50800aa40 con 0x7fd510156f20 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51016a2f0 con 0x7fd510156f20 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800ae00 con 0x7fd510156f20 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.992+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd510169720 con 0x7fd510156f20 2023-10-06T12:50:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.996+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51016a4e0 con 0x7fd510156f20 2023-10-06T12:50:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:53.996+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 101) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fd508007300 con 0x7fd510156f20 2023-10-06T12:50:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.000+0000 7fd5067fc700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.000+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fd50807cec0 con 0x7fd510156f20 2023-10-06T12:50:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.000+0000 7fd51600a700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.000+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:50:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.004+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd50804d0a0 con 0x7fd510156f20 2023-10-06T12:50:54.035 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.029+0000 7f79289ce000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:50:54.107 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.101+0000 7f79289ce000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:54.183 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:54.176+0000 7ff5f8ceb000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:50:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.196+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5100adde0 con 0x7fd4f40606c0 2023-10-06T12:50:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.204+0000 7fd51600a700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.204+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:50:54.262 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:54.256+0000 7ff5f8ceb000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:54.310 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.305+0000 7f79289ce000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:50:54.390 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.385+0000 7f79289ce000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:50:54.469 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.465+0000 7f79289ce000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:50:54.492 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:54.484+0000 7ff5f8ceb000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:50:54.579 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:54.572+0000 7ff5f8ceb000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:50:54.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.608+0000 7fd51600a700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:54.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:54.608+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T12:50:54.668 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:54.660+0000 7ff5f8ceb000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:50:54.842 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.837+0000 7f79289ce000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:50:54.995 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:54.989+0000 7f79289ce000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:50:55.078 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:55.072+0000 7ff5f8ceb000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:50:55.248 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:55.240+0000 7ff5f8ceb000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:50:55.328 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.325+0000 7f79289ce000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:50:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:55.412+0000 7fd51600a700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:55.412+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T12:50:55.544 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.541+0000 7f79289ce000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:55.627 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.621+0000 7f79289ce000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:50:55.632 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:55.624+0000 7ff5f8ceb000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:50:55.699 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.693+0000 7f79289ce000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:50:55.777 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.773+0000 7f79289ce000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:50:55.852 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:55.844+0000 7ff5f8ceb000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:50:55.942 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:55.936+0000 7ff5f8ceb000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:50:55.989 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:55.985+0000 7f79289ce000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:56.017 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.008+0000 7ff5f8ceb000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:50:56.060 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:56.057+0000 7f79289ce000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:50:56.093 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.084+0000 7ff5f8ceb000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:50:56.144 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:56.141+0000 7f79289ce000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:50:56.312 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.304+0000 7ff5f8ceb000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:50:56.329 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:56.325+0000 7f79289ce000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:50:56.390 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.384+0000 7ff5f8ceb000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:50:56.410 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:56.405+0000 7f79289ce000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:50:56.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.468+0000 7ff5f8ceb000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:50:56.674 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.668+0000 7ff5f8ceb000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:50:56.763 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:56.756+0000 7ff5f8ceb000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:50:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.016+0000 7fd51600a700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:50:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.016+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T12:50:57.106 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:50:57.101+0000 7f79289ce000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:50:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.168+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 102) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fd50804d470 con 0x7fd510156f20 2023-10-06T12:50:57.495 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.488+0000 7ff5f8ceb000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:50:57.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.504+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 103) v1 ==== 79687+0+0 (secure 0 0 0) 0x7fd50807b020 con 0x7fd510156f20 2023-10-06T12:50:57.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.504+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:50:57.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:57.504+0000 7fd5067fc700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:57.685 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.676+0000 7ff5bab25700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.686 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.676+0000 7ff5bab25700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.686 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.676+0000 7ff5bab25700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.686 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.676+0000 7ff5bab25700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.686 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.676+0000 7ff5bab25700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.680+0000 7ff5b9322700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.680+0000 7ff5b9322700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.680+0000 7ff5b9322700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.680+0000 7ff5b9322700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:57.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:50:57.680+0000 7ff5b9322700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:50:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.516+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 104) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fd50805ecb0 con 0x7fd510156f20 2023-10-06T12:50:58.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.516+0000 7fd5067fc700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.516+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5100adde0 con 0x7fd4f40606c0 2023-10-06T12:50:58.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.516+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.520+0000 7fd51600a700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007d50 tx=0x7fd50c001180 comp rx=0 tx=0).ready entity=mgr.10050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:58.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.608+0000 7fd5067fc700 1 -- 172.21.15.46:0/3511671778 <== mgr.10050 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd5100adde0 con 0x7fd4f40606c0 2023-10-06T12:50:58.615 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T12:50:58.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 msgr2=0x7fd4f4062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007d50 tx=0x7fd50c001180 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 msgr2=0x7fd5100b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cc70 tx=0x7fd508001a20 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 shutdown_connections 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd510156f20 0x7fd5100b08e0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd510069910 0x7fd5100c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd4f40606c0 0x7fd4f4062af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 --2- 172.21.15.46:0/3511671778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd510067760 0x7fd5100ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 >> 172.21.15.46:0/3511671778 conn(0x7fd5100cbbb0 msgr2=0x7fd5100ab020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:58.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 shutdown_connections 2023-10-06T12:50:58.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.612+0000 7fd51700c700 1 -- 172.21.15.46:0/3511671778 wait complete. 2023-10-06T12:50:58.644 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T12:50:58.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-06T12:50:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.808+0000 7fd6dc7c5700 1 Processor -- start 2023-10-06T12:50:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.812+0000 7fd6dc7c5700 1 -- start start 2023-10-06T12:50:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dc7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dc7c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6d40cd250 0x7fd6d40cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dc7c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40cc510 con 0x7fd6d40cae40 2023-10-06T12:50:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dc7c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40cb900 con 0x7fd6d40cd250 2023-10-06T12:50:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dc7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40cbbe0 con 0x7fd6d40b23c0 2023-10-06T12:50:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55580/0 (socket says 172.21.15.46:55580) 2023-10-06T12:50:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 -- 172.21.15.46:0/4214162728 learned_addr learned my addr 172.21.15.46:0/4214162728 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:58.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6dad62700 1 --2- 172.21.15.46:0/4214162728 >> v2:172.21.15.132:3300/0 conn(0x7fd6d40cd250 0x7fd6d40cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 -- 172.21.15.46:0/4214162728 >> v2:172.21.15.132:3300/0 conn(0x7fd6d40cd250 msgr2=0x7fd6d40cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 --2- 172.21.15.46:0/4214162728 >> v2:172.21.15.132:3300/0 conn(0x7fd6d40cd250 0x7fd6d40cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 -- 172.21.15.46:0/4214162728 >> v1:172.21.15.132:6789/0 conn(0x7fd6d40cae40 legacy=0x7fd6d40ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 -- 172.21.15.46:0/4214162728 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40cbec0 con 0x7fd6d40b23c0 2023-10-06T12:50:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6da561700 1 --2- 172.21.15.46:0/4214162728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009f70 tx=0x7fd6d0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff2dd57637b2796 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6d955f700 1 -- 172.21.15.46:0/4214162728 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000e070 con 0x7fd6d40b23c0 2023-10-06T12:50:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6d955f700 1 -- 172.21.15.46:0/4214162728 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000ba20 con 0x7fd6d40b23c0 2023-10-06T12:50:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.816+0000 7fd6d955f700 1 -- 172.21.15.46:0/4214162728 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00092e0 con 0x7fd6d40b23c0 2023-10-06T12:50:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/4214162728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 msgr2=0x7fd6d40b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/4214162728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009f70 tx=0x7fd6d0009840 comp rx=0 tx=0).stop 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/4214162728 shutdown_connections 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/4214162728 >> v2:172.21.15.132:3300/0 conn(0x7fd6d40cd250 0x7fd6d40cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/4214162728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40b23c0 0x7fd6d40b2790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/4214162728 >> 172.21.15.46:0/4214162728 conn(0x7fd6d4099480 msgr2=0x7fd6d406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/4214162728 shutdown_connections 2023-10-06T12:50:58.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.820+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/4214162728 wait complete. 2023-10-06T12:50:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 Processor -- start 2023-10-06T12:50:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 -- start start 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d40b23c0 0x7fd6d4169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d40cd250 0x7fd6d409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d409a430 con 0x7fd6d40cd250 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d409a6b0 con 0x7fd6d40b23c0 2023-10-06T12:50:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dc7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d409a990 con 0x7fd6d40cae40 2023-10-06T12:50:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6dad62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d40cd250 0x7fd6d409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:55588/0 (socket says 172.21.15.46:55588) 2023-10-06T12:50:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 -- 172.21.15.46:0/3829081507 learned_addr learned my addr 172.21.15.46:0/3829081507 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6da561700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d40b23c0 0x7fd6d4169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 -- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d40b23c0 msgr2=0x7fd6d4169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d40b23c0 0x7fd6d4169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 -- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d40cd250 msgr2=0x7fd6d409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d40cd250 0x7fd6d409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:58.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.824+0000 7fd6d9d60700 1 -- 172.21.15.46:0/3829081507 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009ae0 con 0x7fd6d40cae40 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6d9d60700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c590 tx=0x7fd6c400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4011590 con 0x7fd6d40cae40 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4011710 con 0x7fd6d40cae40 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4011ad0 con 0x7fd6d40cae40 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d409ac70 con 0x7fd6d40cae40 2023-10-06T12:50:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d409b230 con 0x7fd6d40cae40 2023-10-06T12:50:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d409b450 con 0x7fd6d40cae40 2023-10-06T12:50:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.828+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 104) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fd6c401b3b0 con 0x7fd6d40cae40 2023-10-06T12:50:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.832+0000 7fd6ca7fc700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 0x7fd6b80628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.832+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7fd6c407bef0 con 0x7fd6d40cae40 2023-10-06T12:50:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.832+0000 7fd6da561700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 0x7fd6b80628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.832+0000 7fd6da561700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 0x7fd6b80628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009880 tx=0x7fd6d0018000 comp rx=0 tx=0).ready entity=mgr.10050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:58.840+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd6c404c1e0 con 0x7fd6d40cae40 2023-10-06T12:50:59.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.032+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d409aad0 con 0x7fd6b80603f0 2023-10-06T12:50:59.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.116+0000 7fd6ca7fc700 1 -- 172.21.15.46:0/3829081507 <== mgr.10050 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fd6d409aad0 con 0x7fd6b80603f0 2023-10-06T12:50:59.126 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path snap_test_dir1 2023-10-06T12:50:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 msgr2=0x7fd6b80628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 0x7fd6b80628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009880 tx=0x7fd6d0018000 comp rx=0 tx=0).stop 2023-10-06T12:50:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 msgr2=0x7fd6d4099ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c590 tx=0x7fd6c400c8a0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 shutdown_connections 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6d40cd250 0x7fd6d409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6d40cae40 0x7fd6d4099ef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd6b80603f0 0x7fd6b80628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 --2- 172.21.15.46:0/3829081507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6d40b23c0 0x7fd6d4169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 >> 172.21.15.46:0/3829081507 conn(0x7fd6d4099480 msgr2=0x7fd6d40691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 shutdown_connections 2023-10-06T12:50:59.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.120+0000 7fd6dc7c5700 1 -- 172.21.15.46:0/3829081507 wait complete. 2023-10-06T12:50:59.152 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-06T12:50:59.153 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-12_51_00 in ~1s... 2023-10-06T12:50:59.153 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-06T12:50:59.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T12:50:59.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.320+0000 7f38b3ef0700 1 Processor -- start 2023-10-06T12:50:59.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.320+0000 7f38b3ef0700 1 -- start start 2023-10-06T12:50:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b3ef0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f38ac146ea0 0x7f38ac06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b3ef0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38ac0c23a0 0x7f38ac0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b3ef0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ac0c1800 con 0x7f38ac161e50 2023-10-06T12:50:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b3ef0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac0c1580 con 0x7f38ac0c23a0 2023-10-06T12:50:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b3ef0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ac0c6ce0 con 0x7f38ac146ea0 2023-10-06T12:50:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b248d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38ac0c23a0 0x7f38ac0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b248d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38ac0c23a0 0x7f38ac0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37768/0 (socket says 172.21.15.46:37768) 2023-10-06T12:50:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b248d700 1 -- 172.21.15.46:0/2112388784 learned_addr learned my addr 172.21.15.46:0/2112388784 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:59.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b1c8c700 1 --2- 172.21.15.46:0/2112388784 >> v2:172.21.15.132:3300/0 conn(0x7f38ac146ea0 0x7f38ac06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.324+0000 7f38b0c8a700 1 -- 172.21.15.46:0/2112388784 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317514643 0 0) 0x7f38ac0c1800 con 0x7f38ac161e50 2023-10-06T12:50:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b248d700 1 -- 172.21.15.46:0/2112388784 >> v2:172.21.15.132:3300/0 conn(0x7f38ac146ea0 msgr2=0x7f38ac06cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:50:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b248d700 1 --2- 172.21.15.46:0/2112388784 >> v2:172.21.15.132:3300/0 conn(0x7f38ac146ea0 0x7f38ac06cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b248d700 1 -- 172.21.15.46:0/2112388784 >> v1:172.21.15.132:6789/0 conn(0x7f38ac161e50 legacy=0x7f38ac0c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b248d700 1 -- 172.21.15.46:0/2112388784 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac0c6460 con 0x7f38ac0c23a0 2023-10-06T12:50:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b248d700 1 --2- 172.21.15.46:0/2112388784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0caa00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f38ac06c340 tx=0x7f38a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadab7c8af486c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b0c8a700 1 -- 172.21.15.46:0/2112388784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a400e070 con 0x7f38ac0c23a0 2023-10-06T12:50:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b0c8a700 1 -- 172.21.15.46:0/2112388784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a400ba20 con 0x7f38ac0c23a0 2023-10-06T12:50:59.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b0c8a700 1 -- 172.21.15.46:0/2112388784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40092e0 con 0x7f38ac0c23a0 2023-10-06T12:50:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2112388784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac0c23a0 msgr2=0x7f38ac0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2112388784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0caa00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f38ac06c340 tx=0x7f38a4009840 comp rx=0 tx=0).stop 2023-10-06T12:50:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2112388784 shutdown_connections 2023-10-06T12:50:59.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2112388784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0caa00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2112388784 >> v2:172.21.15.132:3300/0 conn(0x7f38ac146ea0 0x7f38ac06cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2112388784 >> 172.21.15.46:0/2112388784 conn(0x7f38ac0cbe20 msgr2=0x7f38ac0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2112388784 shutdown_connections 2023-10-06T12:50:59.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.328+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2112388784 wait complete. 2023-10-06T12:50:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 Processor -- start 2023-10-06T12:50:59.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 -- start start 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ac146ea0 0x7f38ac15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0b0af0 con 0x7f38ac161e50 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0c3650 con 0x7f38ac0c23a0 2023-10-06T12:50:59.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b3ef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac0b0810 con 0x7f38ac146ea0 2023-10-06T12:50:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b248d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b248d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37772/0 (socket says 172.21.15.46:37772) 2023-10-06T12:50:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b248d700 1 -- 172.21.15.46:0/2629129453 learned_addr learned my addr 172.21.15.46:0/2629129453 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:50:59.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b1c8c700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.332+0000 7f38b148b700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ac146ea0 0x7f38ac15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 -- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ac146ea0 msgr2=0x7f38ac15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ac146ea0 0x7f38ac15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 -- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 msgr2=0x7f38ac0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 -- 172.21.15.46:0/2629129453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4009ae0 con 0x7f38ac161e50 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b1c8c700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:50:59.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b248d700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009ab0 tx=0x7f38a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4008be0 con 0x7f38ac161e50 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4008d40 con 0x7f38ac161e50 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac0c3000 con 0x7f38ac161e50 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40266b0 con 0x7f38ac161e50 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac0ca610 con 0x7f38ac161e50 2023-10-06T12:50:59.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.336+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac0c37e0 con 0x7f38ac161e50 2023-10-06T12:50:59.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.340+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 104) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f38a4003040 con 0x7f38ac161e50 2023-10-06T12:50:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.340+0000 7f38a1ffb700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 0x7f38900628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:50:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.340+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f38a407e9f0 con 0x7f38ac161e50 2023-10-06T12:50:59.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.340+0000 7f38b1c8c700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 0x7f38900628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:50:59.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.340+0000 7f38b1c8c700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 0x7f38900628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009d80 tx=0x7f38a8008040 comp rx=0 tx=0).ready entity=mgr.10050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:50:59.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.348+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f38a404ece0 con 0x7f38ac161e50 2023-10-06T12:50:59.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.528+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 105) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f38a4080d50 con 0x7f38ac161e50 2023-10-06T12:50:59.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.548+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38ac0c6720 con 0x7f38900603f0 2023-10-06T12:50:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.560+0000 7f38a1ffb700 1 -- 172.21.15.46:0/2629129453 <== mgr.10050 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f38ac0c6720 con 0x7f38900603f0 2023-10-06T12:50:59.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:50:59.569 INFO:teuthology.orchestra.run.smithi046.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T12:50:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 msgr2=0x7f38900628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 0x7f38900628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009d80 tx=0x7f38a8008040 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 msgr2=0x7f38ac0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009ab0 tx=0x7f38a4018000 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 shutdown_connections 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38ac161e50 0x7f38ac0afb90 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ac146ea0 0x7f38ac15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f38900603f0 0x7f38900628a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 --2- 172.21.15.46:0/2629129453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ac0c23a0 0x7f38ac0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 >> 172.21.15.46:0/2629129453 conn(0x7f38ac0cbe20 msgr2=0x7f38ac06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:50:59.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 shutdown_connections 2023-10-06T12:50:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:50:59.564+0000 7f38b3ef0700 1 -- 172.21.15.46:0/2629129453 wait complete. 2023-10-06T12:50:59.594 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T12:50:59.594 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-06T12:50:59.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:50:59.594 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:50:59.615 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T12:50:59.615 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:50:59.615 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:00.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:51:00.136+0000 7ff5ab306700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:51:00.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:51:00.136+0000 7ff5ab306700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:51:00.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:51:00.136+0000 7ff5ab306700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:51:00.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:51:00.136+0000 7ff5ab306700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:51:00.145 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:51:00.136+0000 7ff5ab306700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:51:00.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:00.616 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:00.640 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:00.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:00.642 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:00.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:00.642 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-12_51_00_UTC'] 2023-10-06T12:51:00.642 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696596659.1536398, now = 1696596660.6425343, timo = 1 2023-10-06T12:51:01.642 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-06-12_51_00 in ~61s... 2023-10-06T12:51:01.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:01.643 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:01.665 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:01.667 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:01.667 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:01.667 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:02.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:02.668 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:02.692 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:02.693 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:02.693 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:02.693 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:03.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:03.694 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:03.718 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:03.720 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:03.720 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:03.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:04.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:04.721 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:04.742 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:04.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:04.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:04.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:05.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:05.745 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:05.769 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:05.770 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:05.770 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:05.770 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:06.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:51:06.753 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:51:06.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:06.771 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:06.814 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:06.815 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:06.816 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:06.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:07.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:07.817 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:07.837 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:07.839 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:07.839 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:07.839 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:08.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:09.042 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:09.066 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:09.068 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:09.068 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:09.068 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:10.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:10.096 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:10.120 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:10.121 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:10.121 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:10.121 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:11.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:11.155 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:11.178 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:11.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:11.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:11.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:12.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:12.214 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:12.239 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:12.240 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:12.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:12.240 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:13.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:13.281 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:13.306 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:13.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:13.450 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:13.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:14.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:14.451 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:14.474 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:14.476 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:14.679 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:14.679 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:15.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:15.802 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:15.824 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:15.957 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:15.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:15.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:16.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:16.964 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:16.987 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:17.089 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:17.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:17.380 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:18.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:18.469 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:18.491 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:18.623 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:18.623 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:18.623 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:19.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:19.707 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:19.732 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:19.869 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:19.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:19.872 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:20.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:20.942 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:20.967 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:20.968 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:20.969 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:20.969 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:21.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:21.970 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:21.991 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:22.033 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:22.033 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:22.033 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:23.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:23.034 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:23.056 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:23.058 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:23.058 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:23.058 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:24.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:24.065 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:24.089 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:24.195 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:24.195 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:24.195 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:25.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:25.196 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:25.218 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:25.220 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:25.220 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:25.220 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:26.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:26.221 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:26.245 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:26.246 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:26.247 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:26.247 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:27.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:27.248 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:27.273 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:27.274 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:27.274 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:27.275 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:28.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:28.276 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:28.297 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:28.298 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:28.299 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:28.299 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:29.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:29.300 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:29.323 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:29.325 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:29.325 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:29.325 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:30.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:30.326 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:30.349 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:30.350 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:30.351 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:30.351 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:31.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:31.351 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:31.373 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:31.375 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:31.375 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:31.375 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:32.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:32.376 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:32.399 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:32.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:32.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:32.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:33.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:33.402 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:33.425 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:33.426 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:33.426 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:34.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:34.427 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:34.450 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:34.451 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:34.451 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:34.452 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:35.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:35.452 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:35.476 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:35.478 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:35.478 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:35.478 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:36.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:36.480 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:36.503 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:36.504 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:36.505 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:36.505 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:36.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:51:36.873 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:51:37.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:37.506 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:37.529 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:37.530 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:37.530 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:37.530 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:38.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:38.532 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:38.554 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:38.556 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:38.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:38.556 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:39.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:39.556 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:39.579 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:39.581 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:39.581 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:39.581 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:40.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:40.582 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:40.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:40.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:40.606 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:41.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:41.607 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:41.631 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:41.633 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:41.633 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:41.633 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:42.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:42.634 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:42.655 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:42.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:42.656 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:42.656 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:43.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:43.657 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:43.681 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:43.682 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:43.683 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:43.683 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:44.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:44.684 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:44.706 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:44.707 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:44.707 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:44.707 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:45.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:45.709 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:45.733 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:45.734 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:45.734 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:45.734 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:46.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:46.736 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:46.762 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:46.762 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:46.762 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:47.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:47.763 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:47.786 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:47.788 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:47.788 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:47.788 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:48.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:48.789 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:48.811 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:48.813 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:48.813 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:48.813 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:49.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:49.815 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:49.837 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:49.838 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:49.838 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:49.838 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:50.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:50.839 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:50.863 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:50.864 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:50.864 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:50.864 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:51.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:51.865 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:51.888 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:51.890 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:51.890 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:51.890 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:52.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:52.891 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:52.916 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:52.917 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:52.917 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:52.917 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:53.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:53.918 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:53.941 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:53.943 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:53.943 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:53.943 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:54.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:54.944 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:54.966 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:54.968 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:54.968 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:54.968 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:55.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:55.969 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:55.991 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:55.992 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:55.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:55.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:56.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:56.993 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:57.016 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:57.018 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:57.018 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:57.018 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:58.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:58.019 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:58.042 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:58.043 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:58.043 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:58.043 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:51:59.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:51:59.045 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:51:59.070 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:51:59.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:51:59.072 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:51:59.072 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:52:00.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:00.073 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:52:00.097 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_51_00_UTC 2023-10-06T12:52:00.098 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:52:00.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T12:52:00.099 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T12:52:01.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:01.100 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:52:01.123 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_52_00_UTC 2023-10-06T12:52:01.124 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-12_52_00_UTC'} 2023-10-06T12:52:01.125 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-12_52_00_UTC'} 2023-10-06T12:52:01.125 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-06-12_51_00_UTC'} 2023-10-06T12:52:01.125 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-06-12_51_00_UTC'] 2023-10-06T12:52:01.125 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696596659.1536398, now = 1696596721.1253157, timo = 61 2023-10-06T12:52:02.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T12:52:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.296+0000 7f9ccc26e700 1 Processor -- start 2023-10-06T12:52:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.296+0000 7f9ccc26e700 1 -- start start 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9cc41596a0 0x7f9cc40c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9cc40c6780 0x7f9cc40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc40c5db0 con 0x7f9cc41592d0 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc40c5ab0 con 0x7f9cc41596a0 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc40d0aa0 con 0x7f9cc40c6780 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cca00a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9cc41592d0 0x7f9cc40acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52778/0 (socket says 172.21.15.46:52778) 2023-10-06T12:52:02.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cca00a700 1 -- 172.21.15.46:0/808805719 learned_addr learned my addr 172.21.15.46:0/808805719 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 --2- 172.21.15.46:0/808805719 >> v2:172.21.15.46:3300/0 conn(0x7f9cc41596a0 0x7f9cc40c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cca80b700 1 --2- 172.21.15.46:0/808805719 >> v2:172.21.15.132:3300/0 conn(0x7f9cc40c6780 0x7f9cc40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9008700 1 -- 172.21.15.46:0/808805719 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365603067 0 0) 0x7f9cc40c5db0 con 0x7f9cc41592d0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 -- 172.21.15.46:0/808805719 >> v2:172.21.15.132:3300/0 conn(0x7f9cc40c6780 msgr2=0x7f9cc40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 --2- 172.21.15.46:0/808805719 >> v2:172.21.15.132:3300/0 conn(0x7f9cc40c6780 0x7f9cc40ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 -- 172.21.15.46:0/808805719 >> v1:172.21.15.46:6789/0 conn(0x7f9cc41592d0 legacy=0x7f9cc40acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 -- 172.21.15.46:0/808805719 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40d0d80 con 0x7f9cc41596a0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9809700 1 --2- 172.21.15.46:0/808805719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41596a0 0x7f9cc40c6240 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40881e0 tx=0x7f9cb4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaec0e48d3c7370 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9008700 1 -- 172.21.15.46:0/808805719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4005c10 con 0x7f9cc41596a0 2023-10-06T12:52:02.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9008700 1 -- 172.21.15.46:0/808805719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb4005d70 con 0x7f9cc41596a0 2023-10-06T12:52:02.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9cc9008700 1 -- 172.21.15.46:0/808805719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4006260 con 0x7f9cc41596a0 2023-10-06T12:52:02.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 -- 172.21.15.46:0/808805719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41596a0 msgr2=0x7f9cc40c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.300+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/808805719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41596a0 0x7f9cc40c6240 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40881e0 tx=0x7f9cb4009120 comp rx=0 tx=0).stop 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- 172.21.15.46:0/808805719 shutdown_connections 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/808805719 >> v2:172.21.15.132:3300/0 conn(0x7f9cc40c6780 0x7f9cc40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/808805719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41596a0 0x7f9cc40c6240 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- 172.21.15.46:0/808805719 >> 172.21.15.46:0/808805719 conn(0x7f9cc406ef60 msgr2=0x7f9cc40d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- 172.21.15.46:0/808805719 shutdown_connections 2023-10-06T12:52:02.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- 172.21.15.46:0/808805719 wait complete. 2023-10-06T12:52:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 Processor -- start 2023-10-06T12:52:02.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- start start 2023-10-06T12:52:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc40c6780 0x7f9cc4158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cc41596a0 0x7f9cc4157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc414f970 con 0x7f9cc41592d0 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40b3ab0 con 0x7f9cc40c6780 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.304+0000 7f9ccc26e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40b3d90 con 0x7f9cc41596a0 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39456/0 (socket says 172.21.15.46:39456) 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 -- 172.21.15.46:0/2309213476 learned_addr learned my addr 172.21.15.46:0/2309213476 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cca80b700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cc41596a0 0x7f9cc4157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cca00a700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc40c6780 0x7f9cc4158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 -- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cc41596a0 msgr2=0x7f9cc4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cc41596a0 0x7f9cc4157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 -- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc40c6780 msgr2=0x7f9cc4158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc40c6780 0x7f9cc4158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 -- 172.21.15.46:0/2309213476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40093c0 con 0x7f9cc41592d0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cc9809700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009f40 tx=0x7f9cb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb401d800 con 0x7f9cc41592d0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb401d980 con 0x7f9cc41592d0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb401dd40 con 0x7f9cc41592d0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc40b4070 con 0x7f9cc41592d0 2023-10-06T12:52:02.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.308+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc414a720 con 0x7f9cc41592d0 2023-10-06T12:52:02.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.312+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc40b4570 con 0x7f9cc41592d0 2023-10-06T12:52:02.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.312+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 106) v1 ==== 80192+0+0 (secure 0 0 0) 0x7f9cb401b690 con 0x7f9cc41592d0 2023-10-06T12:52:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.312+0000 7f9cba7fc700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 0x7f9ca8062c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.312+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f9cb407ed70 con 0x7f9cc41592d0 2023-10-06T12:52:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.312+0000 7f9cca00a700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 0x7f9ca8062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.316+0000 7f9cca00a700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 0x7f9ca8062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00029c0 tx=0x7f9cc0009040 comp rx=0 tx=0).ready entity=mgr.10050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:02.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.320+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9cb404eb50 con 0x7f9cc41592d0 2023-10-06T12:52:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.512+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cc40b3c20 con 0x7f9ca80607e0 2023-10-06T12:52:02.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9cba7fc700 1 -- 172.21.15.46:0/2309213476 <== mgr.10050 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9cc40b3c20 con 0x7f9ca80607e0 2023-10-06T12:52:02.623 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule removed for path snap_test_dir1 2023-10-06T12:52:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 msgr2=0x7f9ca8062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 0x7f9ca8062c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00029c0 tx=0x7f9cc0009040 comp rx=0 tx=0).stop 2023-10-06T12:52:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 msgr2=0x7f9cc414a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009f40 tx=0x7f9cb4016000 comp rx=0 tx=0).stop 2023-10-06T12:52:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 shutdown_connections 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.616+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cc41596a0 0x7f9cc4157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cc41592d0 0x7f9cc414a1e0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9ca80607e0 0x7f9ca8062c90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 --2- 172.21.15.46:0/2309213476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cc40c6780 0x7f9cc4158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 >> 172.21.15.46:0/2309213476 conn(0x7f9cc406ef60 msgr2=0x7f9cc4155c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 shutdown_connections 2023-10-06T12:52:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:02.620+0000 7f9ccc26e700 1 -- 172.21.15.46:0/2309213476 wait complete. 2023-10-06T12:52:02.648 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T12:52:02.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:02.649 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T12:52:02.668 INFO:teuthology.orchestra.run.smithi046.stdout:scheduled-2023-10-06-12_52_00_UTC 2023-10-06T12:52:02.669 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-12_52_00_UTC 2023-10-06T12:52:02.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:02.669 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-12_52_00_UTC) 2023-10-06T12:52:02.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:02.745 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T12:52:02.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T12:52:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.020+0000 7f320ea78700 1 Processor -- start 2023-10-06T12:52:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.020+0000 7f320ea78700 1 -- start start 2023-10-06T12:52:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320ea78700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080b23f0 0x7f32080cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320ea78700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32080ccc00 0x7f32080ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320ea78700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080cb6e0 con 0x7f32080b27c0 2023-10-06T12:52:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320ea78700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080cc780 con 0x7f32080b23f0 2023-10-06T12:52:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320ea78700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080cb9c0 con 0x7f32080ccc00 2023-10-06T12:52:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080b23f0 0x7f32080cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32080b23f0 0x7f32080cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39458/0 (socket says 172.21.15.46:39458) 2023-10-06T12:52:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 -- 172.21.15.46:0/4105777141 learned_addr learned my addr 172.21.15.46:0/4105777141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f3207fff700 1 --2- 172.21.15.46:0/4105777141 >> v2:172.21.15.132:3300/0 conn(0x7f32080ccc00 0x7f32080ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 -- 172.21.15.46:0/4105777141 >> v2:172.21.15.132:3300/0 conn(0x7f32080ccc00 msgr2=0x7f32080ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 --2- 172.21.15.46:0/4105777141 >> v2:172.21.15.132:3300/0 conn(0x7f32080ccc00 0x7f32080ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 -- 172.21.15.46:0/4105777141 >> v1:172.21.15.132:6789/0 conn(0x7f32080b27c0 legacy=0x7f320815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 -- 172.21.15.46:0/4105777141 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080cbca0 con 0x7f32080b23f0 2023-10-06T12:52:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.024+0000 7f320c814700 1 --2- 172.21.15.46:0/4105777141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f32080cad90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f320815cdb0 tx=0x7f31fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ed9502c7f10adf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f32077fe700 1 -- 172.21.15.46:0/4105777141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc009e60 con 0x7f32080b23f0 2023-10-06T12:52:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f32077fe700 1 -- 172.21.15.46:0/4105777141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00b860 con 0x7f32080b23f0 2023-10-06T12:52:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f32077fe700 1 -- 172.21.15.46:0/4105777141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00bc00 con 0x7f32080b23f0 2023-10-06T12:52:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 -- 172.21.15.46:0/4105777141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 msgr2=0x7f32080cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 --2- 172.21.15.46:0/4105777141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f32080cad90 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f320815cdb0 tx=0x7f31fc009250 comp rx=0 tx=0).stop 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 -- 172.21.15.46:0/4105777141 shutdown_connections 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 --2- 172.21.15.46:0/4105777141 >> v2:172.21.15.132:3300/0 conn(0x7f32080ccc00 0x7f32080ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 --2- 172.21.15.46:0/4105777141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f32080cad90 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 -- 172.21.15.46:0/4105777141 >> 172.21.15.46:0/4105777141 conn(0x7f3208099440 msgr2=0x7f320806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 -- 172.21.15.46:0/4105777141 shutdown_connections 2023-10-06T12:52:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.028+0000 7f320ea78700 1 -- 172.21.15.46:0/4105777141 wait complete. 2023-10-06T12:52:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 Processor -- start 2023-10-06T12:52:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 -- start start 2023-10-06T12:52:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080b27c0 0x7f3208069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 0x7f320809d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32081598e0 con 0x7f32080b23f0 2023-10-06T12:52:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208169b20 con 0x7f32080b27c0 2023-10-06T12:52:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320ea78700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3208169e00 con 0x7f32080ccc00 2023-10-06T12:52:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320c814700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320c814700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39466/0 (socket says 172.21.15.46:39466) 2023-10-06T12:52:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.032+0000 7f320c814700 1 -- 172.21.15.46:0/375055502 learned_addr learned my addr 172.21.15.46:0/375055502 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f3207fff700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080b27c0 0x7f3208069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320d015700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 0x7f320809d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 -- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 msgr2=0x7f320809d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 0x7f320809d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 -- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080b27c0 msgr2=0x7f3208069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080b27c0 0x7f3208069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 -- 172.21.15.46:0/375055502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc009490 con 0x7f32080b23f0 2023-10-06T12:52:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320d015700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 0x7f320809d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320c814700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3208069e30 tx=0x7f31fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00bbe0 con 0x7f32080b23f0 2023-10-06T12:52:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc011420 con 0x7f32080b23f0 2023-10-06T12:52:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320816a0e0 con 0x7f32080b23f0 2023-10-06T12:52:03.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0117e0 con 0x7f32080b23f0 2023-10-06T12:52:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.036+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3208155d40 con 0x7f32080b23f0 2023-10-06T12:52:03.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.040+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32080660f0 con 0x7f32080b23f0 2023-10-06T12:52:03.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.040+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 106) v1 ==== 80192+0+0 (secure 0 0 0) 0x7f31fc011bd0 con 0x7f32080b23f0 2023-10-06T12:52:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.044+0000 7f3204ff9700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 0x7f31ec062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.044+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f31fc07e4c0 con 0x7f32080b23f0 2023-10-06T12:52:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.044+0000 7f3207fff700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 0x7f31ec062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.044+0000 7f3207fff700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 0x7f31ec062d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3208156600 tx=0x7f31f40091a0 comp rx=0 tx=0).ready entity=mgr.10050 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:03.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.048+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f31fc04e3a0 con 0x7f32080b23f0 2023-10-06T12:52:03.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.260+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f320804b5c0 con 0x7f32080b23f0 2023-10-06T12:52:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.976+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f31fc04e770 con 0x7f32080b23f0 2023-10-06T12:52:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.976+0000 7f3204ff9700 1 -- 172.21.15.46:0/375055502 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v107) v1 ==== 93+0+0 (secure 0 0 0) 0x7f31fc051e30 con 0x7f32080b23f0 2023-10-06T12:52:03.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 msgr2=0x7f31ec062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 0x7f31ec062d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3208156600 tx=0x7f31f40091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 msgr2=0x7f3208155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3208069e30 tx=0x7f31fc017000 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 shutdown_connections 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32080ccc00 0x7f320809d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f31ec0608b0 0x7f31ec062d60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32080b27c0 0x7f3208069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 --2- 172.21.15.46:0/375055502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32080b23f0 0x7f3208155800 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 >> 172.21.15.46:0/375055502 conn(0x7f3208099440 msgr2=0x7f32080cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 shutdown_connections 2023-10-06T12:52:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:03.980+0000 7f320ea78700 1 -- 172.21.15.46:0/375055502 wait complete. 2023-10-06T12:52:04.006 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:52:04.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:04.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:04.087 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:52:04.087 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T12:52:04.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:04.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:04.166 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:04.159+0000 7f1a77968000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:04.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:52:04.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:04.183 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T12:52:04.183 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:04.176+0000 7f5e4aee9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:04.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:04.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.400+0000 7f7f9e816700 1 Processor -- start 2023-10-06T12:52:04.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 -- start start 2023-10-06T12:52:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7f981591a0 0x7f7f98069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9815d6c0 con 0x7f7f98158dd0 2023-10-06T12:52:04.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f980699c0 con 0x7f7f981591a0 2023-10-06T12:52:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.404+0000 7f7f9e816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9815db50 con 0x7f7f98067760 2023-10-06T12:52:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:51250/0 (socket says 172.21.15.46:51250) 2023-10-06T12:52:04.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 -- 172.21.15.46:0/158651658 learned_addr learned my addr 172.21.15.46:0/158651658 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9e015700 1 --2- 172.21.15.46:0/158651658 >> v2:172.21.15.132:3300/0 conn(0x7f7f981591a0 0x7f7f98069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 -- 172.21.15.46:0/158651658 >> v2:172.21.15.132:3300/0 conn(0x7f7f981591a0 msgr2=0x7f7f98069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 --2- 172.21.15.46:0/158651658 >> v2:172.21.15.132:3300/0 conn(0x7f7f981591a0 0x7f7f98069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 -- 172.21.15.46:0/158651658 >> v1:172.21.15.132:6789/0 conn(0x7f7f98158dd0 legacy=0x7f7f98067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 -- 172.21.15.46:0/158651658 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9815cd90 con 0x7f7f98067760 2023-10-06T12:52:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9d013700 1 --2- 172.21.15.46:0/158651658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f880099b0 tx=0x7f7f88009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84f6e5b4c7237bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:04.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9c812700 1 -- 172.21.15.46:0/158651658 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88009ea0 con 0x7f7f98067760 2023-10-06T12:52:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9c812700 1 -- 172.21.15.46:0/158651658 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8800b860 con 0x7f7f98067760 2023-10-06T12:52:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9c812700 1 -- 172.21.15.46:0/158651658 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800bc00 con 0x7f7f98067760 2023-10-06T12:52:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9e816700 1 -- 172.21.15.46:0/158651658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 msgr2=0x7f7f98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.408+0000 7f7f9e816700 1 --2- 172.21.15.46:0/158651658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7f880099b0 tx=0x7f7f88009250 comp rx=0 tx=0).stop 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- 172.21.15.46:0/158651658 shutdown_connections 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 --2- 172.21.15.46:0/158651658 >> v2:172.21.15.132:3300/0 conn(0x7f7f981591a0 0x7f7f98069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 --2- 172.21.15.46:0/158651658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f98068bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- 172.21.15.46:0/158651658 >> 172.21.15.46:0/158651658 conn(0x7f7f98062ef0 msgr2=0x7f7f980cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- 172.21.15.46:0/158651658 shutdown_connections 2023-10-06T12:52:04.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- 172.21.15.46:0/158651658 wait complete. 2023-10-06T12:52:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 Processor -- start 2023-10-06T12:52:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- start start 2023-10-06T12:52:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f980a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98158dd0 0x7f7f980abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980aaad0 con 0x7f7f981591a0 2023-10-06T12:52:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980a9ec0 con 0x7f7f98158dd0 2023-10-06T12:52:04.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.412+0000 7f7f9e816700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f980aa1a0 con 0x7f7f98067760 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39052/0 (socket says 172.21.15.46:39052) 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 -- 172.21.15.46:0/1248529195 learned_addr learned my addr 172.21.15.46:0/1248529195 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9d013700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98158dd0 0x7f7f980abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9d814700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f980a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 msgr2=0x7f7f980a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f980a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98158dd0 msgr2=0x7f7f980abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98158dd0 0x7f7f980abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 -- 172.21.15.46:0/1248529195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880094f0 con 0x7f7f981591a0 2023-10-06T12:52:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e015700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009df0 tx=0x7f7f9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000caa0 con 0x7f7f981591a0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9000f040 con 0x7f7f981591a0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f980aa480 con 0x7f7f981591a0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000ce70 con 0x7f7f981591a0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.416+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f980ac010 con 0x7f7f981591a0 2023-10-06T12:52:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f98064590 con 0x7f7f981591a0 2023-10-06T12:52:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f7f90014690 con 0x7f7f981591a0 2023-10-06T12:52:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f8dffb700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 0x7f7f7c062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f7f9007d990 con 0x7f7f981591a0 2023-10-06T12:52:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f9d814700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 msgr2=0x7f7f7c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.420+0000 7f7f9d814700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 0x7f7f7c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.428+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f9004d660 con 0x7f7f981591a0 2023-10-06T12:52:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.624+0000 7f7f9d814700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 msgr2=0x7f7f7c062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:04.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.624+0000 7f7f9d814700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 0x7f7f7c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:04.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.636+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f980aac10 con 0x7f7f981591a0 2023-10-06T12:52:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.636+0000 7f7f8dffb700 1 -- 172.21.15.46:0/1248529195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f7f90050c80 con 0x7f7f981591a0 2023-10-06T12:52:04.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:04.645 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 msgr2=0x7f7f7c062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 0x7f7f7c062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 msgr2=0x7f7f980adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009df0 tx=0x7f7f9000b040 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 shutdown_connections 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f981591a0 0x7f7f980adb50 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7f7c060790 0x7f7f7c062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f98158dd0 0x7f7f980abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 --2- 172.21.15.46:0/1248529195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f98067760 0x7f7f980a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 >> 172.21.15.46:0/1248529195 conn(0x7f7f98062ef0 msgr2=0x7f7f980b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 shutdown_connections 2023-10-06T12:52:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.640+0000 7f7f9e816700 1 -- 172.21.15.46:0/1248529195 wait complete. 2023-10-06T12:52:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:52:04.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:04.700 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:04.695+0000 7f1a77968000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:04.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:04.752+0000 7f5e4aee9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:04.768 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:04.763+0000 7f1a77968000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 Processor -- start 2023-10-06T12:52:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 -- start start 2023-10-06T12:52:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6914155c70 0x7f691406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69140c6150 0x7f69140c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140c74d0 con 0x7f6914067050 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140c68c0 con 0x7f6914155c70 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6919e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140c6ba0 con 0x7f69140c6150 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.816+0000 7f6918e8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6914155c70 0x7f691406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6914155c70 0x7f691406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39066/0 (socket says 172.21.15.46:39066) 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 -- 172.21.15.46:0/3588439161 learned_addr learned my addr 172.21.15.46:0/3588439161 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6913fff700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69140c6150 0x7f69140c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 -- 172.21.15.46:0/3588439161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69140c6150 msgr2=0x7f69140c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69140c6150 0x7f69140c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 -- 172.21.15.46:0/3588439161 >> v1:172.21.15.46:6789/0 conn(0x7f6914067050 legacy=0x7f69140abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 -- 172.21.15.46:0/3588439161 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140c6e80 con 0x7f6914155c70 2023-10-06T12:52:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6918e8c700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6914155c70 0x7f691406da60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f690800f490 tx=0x7f690800dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6882d187270a5978 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f69137fe700 1 -- 172.21.15.46:0/3588439161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908012070 con 0x7f6914155c70 2023-10-06T12:52:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f69137fe700 1 -- 172.21.15.46:0/3588439161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690800d0d0 con 0x7f6914155c70 2023-10-06T12:52:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f69137fe700 1 -- 172.21.15.46:0/3588439161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690800d490 con 0x7f6914155c70 2023-10-06T12:52:04.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 -- 172.21.15.46:0/3588439161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6914155c70 msgr2=0x7f691406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6914155c70 0x7f691406da60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f690800f490 tx=0x7f690800dc50 comp rx=0 tx=0).stop 2023-10-06T12:52:04.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 -- 172.21.15.46:0/3588439161 shutdown_connections 2023-10-06T12:52:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69140c6150 0x7f69140c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 --2- 172.21.15.46:0/3588439161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6914155c70 0x7f691406da60 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.820+0000 7f6919e8e700 1 -- 172.21.15.46:0/3588439161 >> 172.21.15.46:0/3588439161 conn(0x7f6914062ef0 msgr2=0x7f69140cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.824+0000 7f6919e8e700 1 -- 172.21.15.46:0/3588439161 shutdown_connections 2023-10-06T12:52:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.824+0000 7f6919e8e700 1 -- 172.21.15.46:0/3588439161 wait complete. 2023-10-06T12:52:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.824+0000 7f6919e8e700 1 Processor -- start 2023-10-06T12:52:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.824+0000 7f6919e8e700 1 -- start start 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6914067050 0x7f69140ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6914155c70 0x7f69140c1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140afc30 con 0x7f69140c6150 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140af020 con 0x7f6914067050 2023-10-06T12:52:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140af300 con 0x7f6914155c70 2023-10-06T12:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6918e8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6914067050 0x7f69140ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39068/0 (socket says 172.21.15.46:39068) 2023-10-06T12:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 -- 172.21.15.46:0/290860462 learned_addr learned my addr 172.21.15.46:0/290860462 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:04.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f691968d700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6914155c70 0x7f69140c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:04.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 -- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6914155c70 msgr2=0x7f69140c1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6914155c70 0x7f69140c1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 -- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6914067050 msgr2=0x7f69140ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6914067050 0x7f69140ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 -- 172.21.15.46:0/290860462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690800f050 con 0x7f69140c6150 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6913fff700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f690000ec80 tx=0x7f69000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000c850 con 0x7f69140c6150 2023-10-06T12:52:04.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690000c9d0 con 0x7f69140c6150 2023-10-06T12:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140af5e0 con 0x7f69140c6150 2023-10-06T12:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000cd90 con 0x7f69140c6150 2023-10-06T12:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140b00c0 con 0x7f69140c6150 2023-10-06T12:52:04.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.828+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69140afde0 con 0x7f69140c6150 2023-10-06T12:52:04.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f6900013680 con 0x7f69140c6150 2023-10-06T12:52:04.840 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f5e4aee9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f6910ff9700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 0x7f68f8062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f6918e8c700 1 -- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 msgr2=0x7f68f8062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f690007d350 con 0x7f69140c6150 2023-10-06T12:52:04.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.832+0000 7f6918e8c700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 0x7f68f8062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:04.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:04.840+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f69140afde0 con 0x7f69140c6150 2023-10-06T12:52:04.904 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:04.899+0000 7f1a77968000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:04.990 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:04.984+0000 7f5e4aee9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:05.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.012+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69140b0340 con 0x7f69140c6150 2023-10-06T12:52:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.012+0000 7f6910ff9700 1 -- 172.21.15.46:0/290860462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+18202 (secure 0 0 0) 0x7f690004d130 con 0x7f69140c6150 2023-10-06T12:52:05.022 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:05.022 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":184,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:50:57.495689+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:6832/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3422943682":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3376719297":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/2850989399":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3444801103":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/300086595":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/414605609":"2023-10-07T12:50:57.495606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 msgr2=0x7f68f8062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 0x7f68f8062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 msgr2=0x7f69140c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f690000ec80 tx=0x7f69000091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 shutdown_connections 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6914155c70 0x7f69140c1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69140c6150 0x7f69140c0a10 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f68f80607c0 0x7f68f8062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 --2- 172.21.15.46:0/290860462 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6914067050 0x7f69140ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 >> 172.21.15.46:0/290860462 conn(0x7f6914062ef0 msgr2=0x7f69140cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 shutdown_connections 2023-10-06T12:52:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.016+0000 7f6919e8e700 1 -- 172.21.15.46:0/290860462 wait complete. 2023-10-06T12:52:05.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T12:50:38.842191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T12:50:39.812688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T12:52:05.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:52:05.177 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:05.171+0000 7f1a77968000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.208+0000 7f43f8d63700 1 Processor -- start 2023-10-06T12:52:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.212+0000 7f43f8d63700 1 -- start start 2023-10-06T12:52:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.212+0000 7f43f8d63700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43f40c91f0 0x7f43f40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.212+0000 7f43f8d63700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f40cbfd0 con 0x7f43f40b0970 2023-10-06T12:52:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.212+0000 7f43f8d63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f40cb120 con 0x7f43f40c8c80 2023-10-06T12:52:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.212+0000 7f43f8d63700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f40cb3c0 con 0x7f43f40c91f0 2023-10-06T12:52:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f37fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f43f40b0970 0x7f43f40b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36928/0 (socket says 172.21.15.46:36928) 2023-10-06T12:52:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f3fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43f40c91f0 0x7f43f40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f37fe700 1 -- 172.21.15.46:0/1348482693 learned_addr learned my addr 172.21.15.46:0/1348482693 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f3fff700 1 -- 172.21.15.46:0/1348482693 >> v1:172.21.15.132:6789/0 conn(0x7f43f40c8c80 legacy=0x7f43f40ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f27fc700 1 -- 172.21.15.46:0/1348482693 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010320668 0 0) 0x7f43f40cb120 con 0x7f43f40c8c80 2023-10-06T12:52:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f3fff700 1 -- 172.21.15.46:0/1348482693 >> v1:172.21.15.46:6789/0 conn(0x7f43f40b0970 legacy=0x7f43f40b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f3fff700 1 -- 172.21.15.46:0/1348482693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f40cb6a0 con 0x7f43f40c91f0 2023-10-06T12:52:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f3fff700 1 --2- 172.21.15.46:0/1348482693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c91f0 0x7f43f40cc460 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009e60 tx=0x7f43ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f01cb5341cac64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f27fc700 1 -- 172.21.15.46:0/1348482693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00e070 con 0x7f43f40c91f0 2023-10-06T12:52:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f27fc700 1 -- 172.21.15.46:0/1348482693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec00bb30 con 0x7f43f40c91f0 2023-10-06T12:52:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f27fc700 1 -- 172.21.15.46:0/1348482693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00bed0 con 0x7f43f40c91f0 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 -- 172.21.15.46:0/1348482693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c91f0 msgr2=0x7f43f40cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 --2- 172.21.15.46:0/1348482693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c91f0 0x7f43f40cc460 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009e60 tx=0x7f43ec009780 comp rx=0 tx=0).stop 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 -- 172.21.15.46:0/1348482693 shutdown_connections 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 --2- 172.21.15.46:0/1348482693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c91f0 0x7f43f40cc460 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 -- 172.21.15.46:0/1348482693 >> 172.21.15.46:0/1348482693 conn(0x7f43f40ccd50 msgr2=0x7f43f40cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 -- 172.21.15.46:0/1348482693 shutdown_connections 2023-10-06T12:52:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.216+0000 7f43f8d63700 1 -- 172.21.15.46:0/1348482693 wait complete. 2023-10-06T12:52:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.220+0000 7f43f8d63700 1 Processor -- start 2023-10-06T12:52:05.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.220+0000 7f43f8d63700 1 -- start start 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f40b0970 0x7f43f4168d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43f40c91f0 0x7f43f40a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f40a9840 con 0x7f43f40c8c80 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4169550 con 0x7f43f40b0970 2023-10-06T12:52:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f4169830 con 0x7f43f40c91f0 2023-10-06T12:52:05.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39078/0 (socket says 172.21.15.46:39078) 2023-10-06T12:52:05.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 -- 172.21.15.46:0/445560851 learned_addr learned my addr 172.21.15.46:0/445560851 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f37fe700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f40b0970 0x7f43f4168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f3fff700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43f40c91f0 0x7f43f40a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 -- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43f40c91f0 msgr2=0x7f43f40a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43f40c91f0 0x7f43f40a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 -- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f40b0970 msgr2=0x7f43f4168d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f40b0970 0x7f43f4168d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 -- 172.21.15.46:0/445560851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec009a20 con 0x7f43f40c8c80 2023-10-06T12:52:05.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f2ffd700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c5d0 tx=0x7f43e8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8011590 con 0x7f43f40c8c80 2023-10-06T12:52:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8011710 con 0x7f43f40c8c80 2023-10-06T12:52:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4169b10 con 0x7f43f40c8c80 2023-10-06T12:52:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8011ad0 con 0x7f43f40c8c80 2023-10-06T12:52:05.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.224+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f415d270 con 0x7f43f40c8c80 2023-10-06T12:52:05.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f40a99b0 con 0x7f43f40c8c80 2023-10-06T12:52:05.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f43e801b3b0 con 0x7f43f40c8c80 2023-10-06T12:52:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43db7fe700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 0x7f43e0062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f43e807c510 con 0x7f43f40c8c80 2023-10-06T12:52:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43f37fe700 1 -- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 msgr2=0x7f43e0062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.228+0000 7f43f37fe700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 0x7f43e0062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:05.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.236+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f43f40a99b0 con 0x7f43f40c8c80 2023-10-06T12:52:05.252 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:05.247+0000 7f1a77968000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:05.286 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:05.280+0000 7f5e4aee9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:05.366 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:05.356+0000 7f5e4aee9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:05.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.404+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43f40a9980 con 0x7f43f40c8c80 2023-10-06T12:52:05.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.404+0000 7f43db7fe700 1 -- 172.21.15.46:0/445560851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43e80170a0 con 0x7f43f40c8c80 2023-10-06T12:52:05.413 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:52:05.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 msgr2=0x7f43e0062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 0x7f43e0062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 msgr2=0x7f43f415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c5d0 tx=0x7f43e8007d70 comp rx=0 tx=0).stop 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 shutdown_connections 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43f40c91f0 0x7f43f40a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43f40c8c80 0x7f43f415cd30 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f43e0060790 0x7f43e0062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 --2- 172.21.15.46:0/445560851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43f40b0970 0x7f43f4168d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 >> 172.21.15.46:0/445560851 conn(0x7f43f40ccd50 msgr2=0x7f43f4067050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 shutdown_connections 2023-10-06T12:52:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.408+0000 7f43f8d63700 1 -- 172.21.15.46:0/445560851 wait complete. 2023-10-06T12:52:05.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:52:05.477 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:05.471+0000 7f1a77968000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:05.559 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:05.551+0000 7f1a77968000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:05.595 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:05.588+0000 7f5e4aee9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.592+0000 7f75ceef0700 1 Processor -- start 2023-10-06T12:52:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.592+0000 7f75ceef0700 1 -- start start 2023-10-06T12:52:05.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ceef0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75c80c6470 0x7f75c80ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ceef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ceef0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c80c1bf0 con 0x7f75c806e8a0 2023-10-06T12:52:05.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ceef0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c80abbe0 con 0x7f75c80c6470 2023-10-06T12:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ceef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c806cf80 con 0x7f75c806ed10 2023-10-06T12:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75c80c6470 0x7f75c80ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75c80c6470 0x7f75c80ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39090/0 (socket says 172.21.15.46:39090) 2023-10-06T12:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1212906349 learned_addr learned my addr 172.21.15.46:0/1212906349 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:05.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75cd48d700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1212906349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 msgr2=0x7f75c80c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75c77fe700 1 -- 172.21.15.46:0/1212906349 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092225885 0 0) 0x7f75c80c1bf0 con 0x7f75c806e8a0 2023-10-06T12:52:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1212906349 >> v1:172.21.15.132:6789/0 conn(0x7f75c806e8a0 legacy=0x7f75c80c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.596+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1212906349 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c806d260 con 0x7f75c80c6470 2023-10-06T12:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ccc8c700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c80c6470 0x7f75c80ab6a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0099b0 tx=0x7f75bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7ca620f10d528c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75c77fe700 1 -- 172.21.15.46:0/1212906349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc009ea0 con 0x7f75c80c6470 2023-10-06T12:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75c77fe700 1 -- 172.21.15.46:0/1212906349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc00b860 con 0x7f75c80c6470 2023-10-06T12:52:05.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75c77fe700 1 -- 172.21.15.46:0/1212906349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00bc00 con 0x7f75c80c6470 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 -- 172.21.15.46:0/1212906349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c80c6470 msgr2=0x7f75c80ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c80c6470 0x7f75c80ab6a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0099b0 tx=0x7f75bc009250 comp rx=0 tx=0).stop 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 -- 172.21.15.46:0/1212906349 shutdown_connections 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1212906349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c80c6470 0x7f75c80ab6a0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 -- 172.21.15.46:0/1212906349 >> 172.21.15.46:0/1212906349 conn(0x7f75c80cbe30 msgr2=0x7f75c80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 -- 172.21.15.46:0/1212906349 shutdown_connections 2023-10-06T12:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.600+0000 7f75ceef0700 1 -- 172.21.15.46:0/1212906349 wait complete. 2023-10-06T12:52:05.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 Processor -- start 2023-10-06T12:52:05.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- start start 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c80c6470 0x7f75c815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80c33a0 con 0x7f75c806e8a0 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80b0030 con 0x7f75c80c6470 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80b0310 con 0x7f75c806ed10 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39094/0 (socket says 172.21.15.46:39094) 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1874626736 learned_addr learned my addr 172.21.15.46:0/1874626736 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75c7fff700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75cd48d700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c80c6470 0x7f75c815c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 msgr2=0x7f75c80af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c80c6470 msgr2=0x7f75c815c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c80c6470 0x7f75c815c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 -- 172.21.15.46:0/1874626736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0094f0 con 0x7f75c806e8a0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ccc8c700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f75c806bf10 tx=0x7f75bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00bbe0 con 0x7f75c806e8a0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc011420 con 0x7f75c806e8a0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0117c0 con 0x7f75c806e8a0 2023-10-06T12:52:05.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c80b05f0 con 0x7f75c806e8a0 2023-10-06T12:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.604+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c80ca9f0 con 0x7f75c806e8a0 2023-10-06T12:52:05.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.608+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c80660f0 con 0x7f75c806e8a0 2023-10-06T12:52:05.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.608+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f75bc011bb0 con 0x7f75c806e8a0 2023-10-06T12:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.612+0000 7f75c4ff9700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 0x7f75ac062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.612+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6946+0+0 (secure 0 0 0) 0x7f75bc07e560 con 0x7f75c806e8a0 2023-10-06T12:52:05.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.612+0000 7f75c7fff700 1 -- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 msgr2=0x7f75ac062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:05.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.612+0000 7f75c7fff700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 0x7f75ac062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:05.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.616+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f75bc04e3c0 con 0x7f75c806e8a0 2023-10-06T12:52:05.647 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:05.639+0000 7f1a77968000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:05.687 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:05.680+0000 7f5e4aee9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:05.775 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:05.768+0000 7f5e4aee9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.792+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75c804b5c0 con 0x7f75c806e8a0 2023-10-06T12:52:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.792+0000 7f75c4ff9700 1 -- 172.21.15.46:0/1874626736 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v184) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75bc011e60 con 0x7f75c806e8a0 2023-10-06T12:52:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 msgr2=0x7f75ac062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 0x7f75ac062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 msgr2=0x7f75c80c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f75c806bf10 tx=0x7f75bc017000 comp rx=0 tx=0).stop 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 shutdown_connections 2023-10-06T12:52:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f75ac060790 0x7f75ac062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c80c6470 0x7f75c815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c806ed10 0x7f75c80af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 --2- 172.21.15.46:0/1874626736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75c806e8a0 0x7f75c80c22b0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 >> 172.21.15.46:0/1874626736 conn(0x7f75c80cbe30 msgr2=0x7f75c80addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 shutdown_connections 2023-10-06T12:52:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.796+0000 7f75ceef0700 1 -- 172.21.15.46:0/1874626736 wait complete. 2023-10-06T12:52:05.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T12:52:05.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.976+0000 7f54a8cf0700 1 Processor -- start 2023-10-06T12:52:05.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.976+0000 7f54a8cf0700 1 -- start start 2023-10-06T12:52:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a8cf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f54a4161fc0 0x7f54a40caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a8cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 0x7f54a406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a8cf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a40c1980 con 0x7f54a4146eb0 2023-10-06T12:52:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a8cf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a40c2520 con 0x7f54a4161fc0 2023-10-06T12:52:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a8cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40cb1e0 con 0x7f54a4147280 2023-10-06T12:52:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a259c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f54a4146eb0 0x7f54a40c1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:51994/0 (socket says 172.21.15.46:51994) 2023-10-06T12:52:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a259c700 1 -- 172.21.15.46:0/1885010364 learned_addr learned my addr 172.21.15.46:0/1885010364 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a2d9d700 1 --2- 172.21.15.46:0/1885010364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 0x7f54a406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.980+0000 7f54a1d9b700 1 --2- 172.21.15.46:0/1885010364 >> v2:172.21.15.132:3300/0 conn(0x7f54a4161fc0 0x7f54a40caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.984+0000 7f54a2d9d700 1 -- 172.21.15.46:0/1885010364 >> v2:172.21.15.132:3300/0 conn(0x7f54a4161fc0 msgr2=0x7f54a40caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.984+0000 7f54a2d9d700 1 --2- 172.21.15.46:0/1885010364 >> v2:172.21.15.132:3300/0 conn(0x7f54a4161fc0 0x7f54a40caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.984+0000 7f54a2d9d700 1 -- 172.21.15.46:0/1885010364 >> v1:172.21.15.132:6789/0 conn(0x7f54a4146eb0 legacy=0x7f54a40c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.984+0000 7f54a2d9d700 1 -- 172.21.15.46:0/1885010364 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40c0d70 con 0x7f54a4147280 2023-10-06T12:52:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.984+0000 7f54a2d9d700 1 --2- 172.21.15.46:0/1885010364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 0x7f54a406cc30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f549c009e60 tx=0x7f549c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63a6ad51191af46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a159a700 1 -- 172.21.15.46:0/1885010364 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00e070 con 0x7f54a4147280 2023-10-06T12:52:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a159a700 1 -- 172.21.15.46:0/1885010364 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00bc30 con 0x7f54a4147280 2023-10-06T12:52:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a159a700 1 -- 172.21.15.46:0/1885010364 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c012430 con 0x7f54a4147280 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1885010364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 msgr2=0x7f54a406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1885010364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 0x7f54a406cc30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f549c009e60 tx=0x7f549c009780 comp rx=0 tx=0).stop 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1885010364 shutdown_connections 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1885010364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4147280 0x7f54a406cc30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1885010364 >> v2:172.21.15.132:3300/0 conn(0x7f54a4161fc0 0x7f54a40caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1885010364 >> 172.21.15.46:0/1885010364 conn(0x7f54a40cbe30 msgr2=0x7f54a40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1885010364 shutdown_connections 2023-10-06T12:52:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.992+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1885010364 wait complete. 2023-10-06T12:52:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 Processor -- start 2023-10-06T12:52:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 -- start start 2023-10-06T12:52:06.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4147280 0x7f54a40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54a4161fc0 0x7f54a4160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a40af110 con 0x7f54a4147280 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a416fe20 con 0x7f54a4161fc0 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a8cf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a406a940 con 0x7f54a4146eb0 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:05.996+0000 7f54a259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:51314/0 (socket says 172.21.15.46:51314) 2023-10-06T12:52:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 -- 172.21.15.46:0/1975655369 learned_addr learned my addr 172.21.15.46:0/1975655369 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 -- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54a4161fc0 msgr2=0x7f54a4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54a4161fc0 0x7f54a4160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 -- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4147280 msgr2=0x7f54a40aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4147280 0x7f54a40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.000+0000 7f54a259c700 1 -- 172.21.15.46:0/1975655369 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c009a20 con 0x7f54a4146eb0 2023-10-06T12:52:06.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.004+0000 7f54a259c700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7c0 tx=0x7f5494007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.004+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494005b30 con 0x7f54a4146eb0 2023-10-06T12:52:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.004+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494005cb0 con 0x7f54a4146eb0 2023-10-06T12:52:06.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.004+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494004a80 con 0x7f54a4146eb0 2023-10-06T12:52:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.008+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a416f210 con 0x7f54a4146eb0 2023-10-06T12:52:06.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.008+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a416f7d0 con 0x7f54a4146eb0 2023-10-06T12:52:06.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.008+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a40660f0 con 0x7f54a4146eb0 2023-10-06T12:52:06.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.012+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f54940070f0 con 0x7f54a4146eb0 2023-10-06T12:52:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.012+0000 7f54927fc700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 0x7f5484062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.012+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f549407c5a0 con 0x7f54a4146eb0 2023-10-06T12:52:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.012+0000 7f54a1d9b700 1 -- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 msgr2=0x7f5484062dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:06.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.012+0000 7f54a1d9b700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 0x7f5484062dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.020+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f549404c450 con 0x7f54a4146eb0 2023-10-06T12:52:06.073 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.067+0000 7f1a77968000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:06.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.188+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54a40c1050 con 0x7f54a4146eb0 2023-10-06T12:52:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.188+0000 7f54927fc700 1 -- 172.21.15.46:0/1975655369 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54940180a0 con 0x7f54a4146eb0 2023-10-06T12:52:06.197 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 msgr2=0x7f5484062dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 0x7f5484062dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 msgr2=0x7f54a40c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f549400c7c0 tx=0x7f5494007f70 comp rx=0 tx=0).stop 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 shutdown_connections 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5484060910 0x7f5484062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54a4161fc0 0x7f54a4160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54a4147280 0x7f54a40aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 --2- 172.21.15.46:0/1975655369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54a4146eb0 0x7f54a40c2ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 >> 172.21.15.46:0/1975655369 conn(0x7f54a40cbe30 msgr2=0x7f54a415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 shutdown_connections 2023-10-06T12:52:06.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.192+0000 7f54a8cf0700 1 -- 172.21.15.46:0/1975655369 wait complete. 2023-10-06T12:52:06.219 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:06.212+0000 7f5e4aee9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:06.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T12:52:06.235 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.227+0000 7f1a77968000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 Processor -- start 2023-10-06T12:52:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 -- start start 2023-10-06T12:52:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9cd00adbe0 0x7f9cd00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9cd00d0d60 0x7f9cd00b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00b3630 0x7f9cd00b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd0159740 con 0x7f9cd00d0d60 2023-10-06T12:52:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00afd20 con 0x7f9cd00adbe0 2023-10-06T12:52:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cd488e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00b0000 con 0x7f9cd00b3630 2023-10-06T12:52:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9cd00d0d60 0x7f9cd00b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9cd00d0d60 0x7f9cd00b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39108/0 (socket says 172.21.15.46:39108) 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 -- 172.21.15.46:0/1242696226 learned_addr learned my addr 172.21.15.46:0/1242696226 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9ccffff700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00b3630 0x7f9cd00b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9ccf7fe700 1 --2- 172.21.15.46:0/1242696226 >> v2:172.21.15.132:3300/0 conn(0x7f9cd00adbe0 0x7f9cd00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 -- 172.21.15.46:0/1242696226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00b3630 msgr2=0x7f9cd00b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00b3630 0x7f9cd00b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 -- 172.21.15.46:0/1242696226 >> v2:172.21.15.132:3300/0 conn(0x7f9cd00adbe0 msgr2=0x7f9cd00d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1242696226 >> v2:172.21.15.132:3300/0 conn(0x7f9cd00adbe0 0x7f9cd00d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.372+0000 7f9cceffd700 1 -- 172.21.15.46:0/1242696226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00b02e0 con 0x7f9cd00d0d60 2023-10-06T12:52:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00d0d60 0x7f9cd00b30f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006a0c0 tx=0x7f9cc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76009de86d26eb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cce7fc700 1 -- 172.21.15.46:0/1242696226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4009e70 con 0x7f9cd00d0d60 2023-10-06T12:52:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cce7fc700 1 -- 172.21.15.46:0/1242696226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400b860 con 0x7f9cd00d0d60 2023-10-06T12:52:06.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cce7fc700 1 -- 172.21.15.46:0/1242696226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400bc00 con 0x7f9cd00d0d60 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 -- 172.21.15.46:0/1242696226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00d0d60 msgr2=0x7f9cd00b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00d0d60 0x7f9cd00b30f0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9cd006a0c0 tx=0x7f9cc4009250 comp rx=0 tx=0).stop 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 -- 172.21.15.46:0/1242696226 shutdown_connections 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00b3630 0x7f9cd00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1242696226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00d0d60 0x7f9cd00b30f0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1242696226 >> v2:172.21.15.132:3300/0 conn(0x7f9cd00adbe0 0x7f9cd00d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 -- 172.21.15.46:0/1242696226 >> 172.21.15.46:0/1242696226 conn(0x7f9cd006ef60 msgr2=0x7f9cd00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 -- 172.21.15.46:0/1242696226 shutdown_connections 2023-10-06T12:52:06.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.376+0000 7f9cd488e700 1 -- 172.21.15.46:0/1242696226 wait complete. 2023-10-06T12:52:06.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 Processor -- start 2023-10-06T12:52:06.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 -- start start 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cd00adbe0 0x7f9cd0145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00d0d60 0x7f9cd0099db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd009a2f0 con 0x7f9cd00b3630 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd015a2b0 con 0x7f9cd00adbe0 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cd488e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd015b100 con 0x7f9cd00d0d60 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39112/0 (socket says 172.21.15.46:39112) 2023-10-06T12:52:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 -- 172.21.15.46:0/1939611519 learned_addr learned my addr 172.21.15.46:0/1939611519 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:06.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9ccffff700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00d0d60 0x7f9cd0099db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9ccf7fe700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cd00adbe0 0x7f9cd0145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 -- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00d0d60 msgr2=0x7f9cd0099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00d0d60 0x7f9cd0099db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 -- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cd00adbe0 msgr2=0x7f9cd0145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cd00adbe0 0x7f9cd0145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 -- 172.21.15.46:0/1939611519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40094f0 con 0x7f9cd00b3630 2023-10-06T12:52:06.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cceffd700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0088270 tx=0x7f9cc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4000d10 con 0x7f9cd00b3630 2023-10-06T12:52:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.380+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400bbe0 con 0x7f9cd00b3630 2023-10-06T12:52:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.384+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd01599c0 con 0x7f9cd00b3630 2023-10-06T12:52:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.384+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4011670 con 0x7f9cd00b3630 2023-10-06T12:52:06.391 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.384+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0159f40 con 0x7f9cd00b3630 2023-10-06T12:52:06.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.384+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd015a440 con 0x7f9cd00b3630 2023-10-06T12:52:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.384+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f9cc4011aa0 con 0x7f9cd00b3630 2023-10-06T12:52:06.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.388+0000 7f9cb77fe700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 0x7f9cbc062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.388+0000 7f9ccf7fe700 1 -- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 msgr2=0x7f9cbc062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:06.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.388+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f9cc407e5b0 con 0x7f9cd00b3630 2023-10-06T12:52:06.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.388+0000 7f9ccf7fe700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 0x7f9cbc062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:06.397 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:06.388+0000 7f5e4aee9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.396+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9cc404e360 con 0x7f9cd00b3630 2023-10-06T12:52:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.568+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cd0064590 con 0x7f9cd00b3630 2023-10-06T12:52:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.568+0000 7f9cb77fe700 1 -- 172.21.15.46:0/1939611519 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cc4051980 con 0x7f9cd00b3630 2023-10-06T12:52:06.576 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T12:52:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 msgr2=0x7f9cbc062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 0x7f9cbc062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 msgr2=0x7f9cd015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0088270 tx=0x7f9cc4017000 comp rx=0 tx=0).stop 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 shutdown_connections 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cd00d0d60 0x7f9cd0099db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cd00b3630 0x7f9cd015aa70 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9cbc060790 0x7f9cbc062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 --2- 172.21.15.46:0/1939611519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cd00adbe0 0x7f9cd0145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 >> 172.21.15.46:0/1939611519 conn(0x7f9cd006ef60 msgr2=0x7f9cd0157620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 shutdown_connections 2023-10-06T12:52:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.572+0000 7f9cd488e700 1 -- 172.21.15.46:0/1939611519 wait complete. 2023-10-06T12:52:06.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:06.608 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.603+0000 7f1a77968000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:06.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 Processor -- start 2023-10-06T12:52:06.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 -- start start 2023-10-06T12:52:06.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f04240d0aa0 0x7f0424159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04240b5830 con 0x7f04240b5410 2023-10-06T12:52:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04240b4f80 con 0x7f0424159050 2023-10-06T12:52:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042c585700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04240b40c0 con 0x7f04240d0aa0 2023-10-06T12:52:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042ab22700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f04240b5410 0x7f04240b1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36938/0 (socket says 172.21.15.46:36938) 2023-10-06T12:52:06.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042ab22700 1 -- 172.21.15.46:0/94484462 learned_addr learned my addr 172.21.15.46:0/94484462 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f0429b20700 1 --2- 172.21.15.46:0/94484462 >> v2:172.21.15.46:3300/0 conn(0x7f04240d0aa0 0x7f0424159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30127360 0 0) 0x7f04240b5830 con 0x7f04240b5410 2023-10-06T12:52:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0410003570 con 0x7f04240b5410 2023-10-06T12:52:06.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f0429b20700 1 -- 172.21.15.46:0/94484462 >> v1:172.21.15.132:6789/0 conn(0x7f0424159050 legacy=0x7f04240b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620195991 0 0) 0x7f04240b4f80 con 0x7f0424159050 2023-10-06T12:52:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f0429b20700 1 -- 172.21.15.46:0/94484462 >> v1:172.21.15.46:6789/0 conn(0x7f04240b5410 legacy=0x7f04240b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.756+0000 7f0429b20700 1 -- 172.21.15.46:0/94484462 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240b4370 con 0x7f04240d0aa0 2023-10-06T12:52:06.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f0429b20700 1 --2- 172.21.15.46:0/94484462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f0424159420 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f04140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59612d649052eed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400b640 con 0x7f04240d0aa0 2023-10-06T12:52:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400b7a0 con 0x7f04240d0aa0 2023-10-06T12:52:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042931f700 1 -- 172.21.15.46:0/94484462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400bb40 con 0x7f04240d0aa0 2023-10-06T12:52:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 -- 172.21.15.46:0/94484462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 msgr2=0x7f0424159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 --2- 172.21.15.46:0/94484462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f0424159420 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f04140098e0 tx=0x7f04140091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 -- 172.21.15.46:0/94484462 shutdown_connections 2023-10-06T12:52:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 --2- 172.21.15.46:0/94484462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f0424159420 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 -- 172.21.15.46:0/94484462 >> 172.21.15.46:0/94484462 conn(0x7f042406e780 msgr2=0x7f042406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 -- 172.21.15.46:0/94484462 shutdown_connections 2023-10-06T12:52:06.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.760+0000 7f042c585700 1 -- 172.21.15.46:0/94484462 wait complete. 2023-10-06T12:52:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 Processor -- start 2023-10-06T12:52:06.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 -- start start 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04240b5410 0x7f04240cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 0x7f04241554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04240c9ec0 con 0x7f04240d0aa0 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04240c9620 con 0x7f0424159050 2023-10-06T12:52:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042c585700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04240c9900 con 0x7f04240b5410 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39128/0 (socket says 172.21.15.46:39128) 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 -- 172.21.15.46:0/3952924117 learned_addr learned my addr 172.21.15.46:0/3952924117 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042ab22700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 0x7f04241554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042a321700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04240b5410 0x7f04240cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:06.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04240b5410 msgr2=0x7f04240cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04240b5410 0x7f04240cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 msgr2=0x7f04241554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 0x7f04241554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 -- 172.21.15.46:0/3952924117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009420 con 0x7f04240d0aa0 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f042ab22700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 0x7f04241554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:06.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f0429b20700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04240892a0 tx=0x7f0414017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.764+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041401f070 con 0x7f04240d0aa0 2023-10-06T12:52:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414000bc0 con 0x7f04240d0aa0 2023-10-06T12:52:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04240c9be0 con 0x7f04240d0aa0 2023-10-06T12:52:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414025420 con 0x7f04240d0aa0 2023-10-06T12:52:06.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04240c92d0 con 0x7f04240d0aa0 2023-10-06T12:52:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0424088820 con 0x7f04240d0aa0 2023-10-06T12:52:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.768+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f0414023070 con 0x7f04240d0aa0 2023-10-06T12:52:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.772+0000 7f041a7fc700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 0x7f0408062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.772+0000 7f042a321700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 msgr2=0x7f0408062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.772+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f041407e490 con 0x7f04240d0aa0 2023-10-06T12:52:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.772+0000 7f042a321700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 0x7f0408062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.776+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f041404e240 con 0x7f04240d0aa0 2023-10-06T12:52:06.786 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:06.780+0000 7f5e4aee9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:06.816 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.811+0000 7f1a77968000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:06.909 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.903+0000 7f1a77968000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.976+0000 7f042a321700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 msgr2=0x7f0408062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:06.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.976+0000 7f042a321700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 0x7f0408062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:06.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.980+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04240b4650 con 0x7f04240d0aa0 2023-10-06T12:52:06.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.980+0000 7f041a7fc700 1 -- 172.21.15.46:0/3952924117 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f0414051860 con 0x7f04240d0aa0 2023-10-06T12:52:06.987 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:06.987 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:52:06.988 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:06.979+0000 7f1a77968000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:06.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 msgr2=0x7f0408062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 0x7f0408062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 msgr2=0x7f04240cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04240892a0 tx=0x7f0414017000 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 shutdown_connections 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0408060790 0x7f0408062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0424159050 0x7f04241554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04240d0aa0 0x7f04240cfd50 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 --2- 172.21.15.46:0/3952924117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04240b5410 0x7f04240cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 >> 172.21.15.46:0/3952924117 conn(0x7f042406e780 msgr2=0x7f0424151630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 shutdown_connections 2023-10-06T12:52:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:06.984+0000 7f042c585700 1 -- 172.21.15.46:0/3952924117 wait complete. 2023-10-06T12:52:06.992 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:52:07.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:07.021 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.012+0000 7f5e4aee9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:07.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:52:07.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:52:07.067 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.059+0000 7f1a77968000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:07.118 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.112+0000 7f5e4aee9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:07.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.160+0000 7f585f62e700 1 Processor -- start 2023-10-06T12:52:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 -- start start 2023-10-06T12:52:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5858158e70 0x7f58580b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5858159240 0x7f58580cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585815dd50 con 0x7f58580c8650 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580ca5e0 con 0x7f5858158e70 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585f62e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585815d530 con 0x7f5858159240 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.164+0000 7f585d3ca700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5858158e70 0x7f58580b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585dbcb700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5858159240 0x7f58580cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585cbc9700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f58580c8650 0x7f585815ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52018/0 (socket says 172.21.15.46:52018) 2023-10-06T12:52:07.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585cbc9700 1 -- 172.21.15.46:0/2599149006 learned_addr learned my addr 172.21.15.46:0/2599149006 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585d3ca700 1 -- 172.21.15.46:0/2599149006 >> v2:172.21.15.132:3300/0 conn(0x7f5858159240 msgr2=0x7f58580cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585d3ca700 1 --2- 172.21.15.46:0/2599149006 >> v2:172.21.15.132:3300/0 conn(0x7f5858159240 0x7f58580cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585d3ca700 1 -- 172.21.15.46:0/2599149006 >> v1:172.21.15.132:6789/0 conn(0x7f58580c8650 legacy=0x7f585815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585d3ca700 1 -- 172.21.15.46:0/2599149006 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858159670 con 0x7f5858158e70 2023-10-06T12:52:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585d3ca700 1 --2- 172.21.15.46:0/2599149006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858158e70 0x7f58580b0da0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f58580b0af0 tx=0x7f58540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925534c1e6a2e7b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f584ffff700 1 -- 172.21.15.46:0/2599149006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854011630 con 0x7f5858158e70 2023-10-06T12:52:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f584ffff700 1 -- 172.21.15.46:0/2599149006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5854011790 con 0x7f5858158e70 2023-10-06T12:52:07.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f584ffff700 1 -- 172.21.15.46:0/2599149006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854011b30 con 0x7f5858158e70 2023-10-06T12:52:07.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 -- 172.21.15.46:0/2599149006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858158e70 msgr2=0x7f58580b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 --2- 172.21.15.46:0/2599149006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858158e70 0x7f58580b0da0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f58580b0af0 tx=0x7f58540091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 -- 172.21.15.46:0/2599149006 shutdown_connections 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 --2- 172.21.15.46:0/2599149006 >> v2:172.21.15.132:3300/0 conn(0x7f5858159240 0x7f58580cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 --2- 172.21.15.46:0/2599149006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5858158e70 0x7f58580b0da0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 -- 172.21.15.46:0/2599149006 >> 172.21.15.46:0/2599149006 conn(0x7f5858062d50 msgr2=0x7f5858063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 -- 172.21.15.46:0/2599149006 shutdown_connections 2023-10-06T12:52:07.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.168+0000 7f585f62e700 1 -- 172.21.15.46:0/2599149006 wait complete. 2023-10-06T12:52:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 Processor -- start 2023-10-06T12:52:07.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 -- start start 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5858158e70 0x7f585816f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5858159240 0x7f58581714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858159890 con 0x7f58580c8650 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585815a130 con 0x7f5858158e70 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585f62e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585815ba80 con 0x7f5858159240 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39146/0 (socket says 172.21.15.46:39146) 2023-10-06T12:52:07.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 -- 172.21.15.46:0/1192878059 learned_addr learned my addr 172.21.15.46:0/1192878059 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585dbcb700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5858159240 0x7f58581714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585cbc9700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5858158e70 0x7f585816f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 -- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5858159240 msgr2=0x7f58581714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5858159240 0x7f58581714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 -- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5858158e70 msgr2=0x7f585816f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5858158e70 0x7f585816f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.172+0000 7f585d3ca700 1 -- 172.21.15.46:0/1192878059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5854009420 con 0x7f58580c8650 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f585d3ca700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f585806e820 tx=0x7f5854017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:07.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854011b10 con 0x7f58580c8650 2023-10-06T12:52:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58540253f0 con 0x7f58580c8650 2023-10-06T12:52:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585816f5f0 con 0x7f58580c8650 2023-10-06T12:52:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854025790 con 0x7f58580c8650 2023-10-06T12:52:07.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5858159ae0 con 0x7f58580c8650 2023-10-06T12:52:07.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585806dda0 con 0x7f58580c8650 2023-10-06T12:52:07.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.176+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f5854025b80 con 0x7f58580c8650 2023-10-06T12:52:07.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.180+0000 7f584d7fa700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 0x7f583c062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.180+0000 7f585cbc9700 1 -- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 msgr2=0x7f583c062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:07.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.180+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f585407e940 con 0x7f58580c8650 2023-10-06T12:52:07.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.180+0000 7f585cbc9700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 0x7f583c062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:07.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.188+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f585404e6f0 con 0x7f58580c8650 2023-10-06T12:52:07.200 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.196+0000 7f5e4aee9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:07.280 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.275+0000 7f1a77968000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:07.286 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.280+0000 7f5e4aee9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:07.356 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.351+0000 7f1a77968000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.352+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f585815d790 con 0x7f58580c8650 2023-10-06T12:52:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f584d7fa700 1 -- 172.21.15.46:0/1192878059 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18314 (secure 0 0 0) 0x7f5854015090 con 0x7f58580c8650 2023-10-06T12:52:07.363 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:07.364 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":185,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:05.974313+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":185,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:6832/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3422943682":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3376719297":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/2850989399":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3444801103":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/300086595":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/414605609":"2023-10-07T12:50:57.495606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 msgr2=0x7f583c062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 0x7f583c062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 msgr2=0x7f58580c2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f585806e820 tx=0x7f5854017000 comp rx=0 tx=0).stop 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 shutdown_connections 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5858159240 0x7f58581714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f583c0607c0 0x7f583c062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5858158e70 0x7f585816f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 --2- 172.21.15.46:0/1192878059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58580c8650 0x7f58580c2b80 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.356+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 >> 172.21.15.46:0/1192878059 conn(0x7f5858062d50 msgr2=0x7f5858156da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.360+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 shutdown_connections 2023-10-06T12:52:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.360+0000 7f585f62e700 1 -- 172.21.15.46:0/1192878059 wait complete. 2023-10-06T12:52:07.387 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:52:07.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:07.443 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.435+0000 7f1a77968000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:07.526 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.516+0000 7f5e4aee9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:07.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.580+0000 7f2c34c6e700 1 Processor -- start 2023-10-06T12:52:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.580+0000 7f2c34c6e700 1 -- start start 2023-10-06T12:52:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c34c6e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c3015ab60 0x7f2c3006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c34c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015af30 0x7f2c3006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c34c6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3006d460 con 0x7f2c300ad6c0 2023-10-06T12:52:07.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c34c6e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300c7000 con 0x7f2c3015ab60 2023-10-06T12:52:07.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c34c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300c72c0 con 0x7f2c3015af30 2023-10-06T12:52:07.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2ed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c300ad6c0 0x7f2c300adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36946/0 (socket says 172.21.15.46:36946) 2023-10-06T12:52:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2ed9d700 1 -- 172.21.15.46:0/4099300778 learned_addr learned my addr 172.21.15.46:0/4099300778 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4099300778 >> v2:172.21.15.46:3300/0 conn(0x7f2c3015ab60 0x7f2c3006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2dd9b700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015af30 0x7f2c3006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2d59a700 1 -- 172.21.15.46:0/4099300778 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339397462 0 0) 0x7f2c3006d460 con 0x7f2c300ad6c0 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2d59a700 1 -- 172.21.15.46:0/4099300778 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c18003570 con 0x7f2c300ad6c0 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4099300778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015af30 msgr2=0x7f2c3006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015af30 0x7f2c3006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4099300778 >> v1:172.21.15.46:6789/0 conn(0x7f2c300ad6c0 legacy=0x7f2c300adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4099300778 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300c75a0 con 0x7f2c3015ab60 2023-10-06T12:52:07.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3015ab60 0x7f2c3006e8a0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098d0 tx=0x7f2c200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c4a0ce5d223c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2d59a700 1 -- 172.21.15.46:0/4099300778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011630 con 0x7f2c3015ab60 2023-10-06T12:52:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2d59a700 1 -- 172.21.15.46:0/4099300778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011790 con 0x7f2c3015ab60 2023-10-06T12:52:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.584+0000 7f2c2d59a700 1 -- 172.21.15.46:0/4099300778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011b30 con 0x7f2c3015ab60 2023-10-06T12:52:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4099300778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3015ab60 msgr2=0x7f2c3006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3015ab60 0x7f2c3006e8a0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098d0 tx=0x7f2c200091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4099300778 shutdown_connections 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015af30 0x7f2c3006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4099300778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3015ab60 0x7f2c3006e8a0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4099300778 >> 172.21.15.46:0/4099300778 conn(0x7f2c300cbe30 msgr2=0x7f2c300cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4099300778 shutdown_connections 2023-10-06T12:52:07.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4099300778 wait complete. 2023-10-06T12:52:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.588+0000 7f2c34c6e700 1 Processor -- start 2023-10-06T12:52:07.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- start start 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 0x7f2c301590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c3015af30 0x7f2c30147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301556e0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3016ce00 con 0x7f2c3015af30 2023-10-06T12:52:07.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3016d0e0 con 0x7f2c3015ab60 2023-10-06T12:52:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39166/0 (socket says 172.21.15.46:39166) 2023-10-06T12:52:07.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4011981383 learned_addr learned my addr 172.21.15.46:0/4011981383 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2dd9b700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 0x7f2c301590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2ed9d700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c3015af30 0x7f2c30147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 msgr2=0x7f2c301590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 0x7f2c301590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c3015af30 msgr2=0x7f2c30147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c3015af30 0x7f2c30147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 -- 172.21.15.46:0/4011981383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200093c0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2dd9b700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 0x7f2c301590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:07.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c2e59c700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006bf10 tx=0x7f2c20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011b10 con 0x7f2c300ad6c0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20006af0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3016d3c0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20025620 con 0x7f2c300ad6c0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.592+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c301692a0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.596+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c30063f90 con 0x7f2c300ad6c0 2023-10-06T12:52:07.604 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.596+0000 7f5e4aee9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:07.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.596+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f2c20025a50 con 0x7f2c300ad6c0 2023-10-06T12:52:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.600+0000 7f2c1e7fc700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 0x7f2c10062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.600+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f2c2007e790 con 0x7f2c300ad6c0 2023-10-06T12:52:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.600+0000 7f2c2dd9b700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 msgr2=0x7f2c10062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:07.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.600+0000 7f2c2dd9b700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 0x7f2c10062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:07.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.604+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2c2004e5e0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.629 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.623+0000 7f1a77968000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:07.699 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.692+0000 7f5e4aee9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:07.710 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:07.703+0000 7f1a77968000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:07.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.804+0000 7f2c2dd9b700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 msgr2=0x7f2c10062c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:07.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.804+0000 7f2c2dd9b700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 0x7f2c10062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.824+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c3004b5c0 con 0x7f2c300ad6c0 2023-10-06T12:52:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c1e7fc700 1 -- 172.21.15.46:0/4011981383 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f2c20051c00 con 0x7f2c300ad6c0 2023-10-06T12:52:07.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:07.835 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:52:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 msgr2=0x7f2c10062c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 0x7f2c10062c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 msgr2=0x7f2c30154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006bf10 tx=0x7f2c20017000 comp rx=0 tx=0).stop 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 shutdown_connections 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c100607c0 0x7f2c10062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c3015af30 0x7f2c30147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c3015ab60 0x7f2c301590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 --2- 172.21.15.46:0/4011981383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300ad6c0 0x7f2c30154d40 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 >> 172.21.15.46:0/4011981383 conn(0x7f2c300cbe30 msgr2=0x7f2c300aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.828+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 shutdown_connections 2023-10-06T12:52:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:07.832+0000 7f2c34c6e700 1 -- 172.21.15.46:0/4011981383 wait complete. 2023-10-06T12:52:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:52:07.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:07.930 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:07.928+0000 7f5e4aee9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:08.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.008+0000 7f8186ce3700 1 Processor -- start 2023-10-06T12:52:08.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8186ce3700 1 -- start start 2023-10-06T12:52:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8186ce3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81800b1300 0x7f81800ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8186ce3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800ae340 con 0x7f81800d07b0 2023-10-06T12:52:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8186ce3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800b09a0 con 0x7f8180159290 2023-10-06T12:52:08.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8186ce3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8180159660 con 0x7f81800b1300 2023-10-06T12:52:08.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81854e0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f81800d07b0 0x7f81800b0bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36958/0 (socket says 172.21.15.46:36958) 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81854e0700 1 -- 172.21.15.46:0/467797211 learned_addr learned my addr 172.21.15.46:0/467797211 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81864e2700 1 --2- 172.21.15.46:0/467797211 >> v2:172.21.15.46:3300/0 conn(0x7f81800b1300 0x7f81800ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770349826 0 0) 0x7f81800ae340 con 0x7f81800d07b0 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c003570 con 0x7f81800d07b0 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715087057 0 0) 0x7f81800b09a0 con 0x7f8180159290 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81864e2700 1 -- 172.21.15.46:0/467797211 >> v1:172.21.15.132:6789/0 conn(0x7f8180159290 legacy=0x7f81800b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81864e2700 1 -- 172.21.15.46:0/467797211 >> v1:172.21.15.46:6789/0 conn(0x7f81800d07b0 legacy=0x7f81800b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.012+0000 7f81864e2700 1 -- 172.21.15.46:0/467797211 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800b06c0 con 0x7f81800b1300 2023-10-06T12:52:08.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f81864e2700 1 --2- 172.21.15.46:0/467797211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800b1300 0x7f81800ae7d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f8178009e60 tx=0x7f8178009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303da450fce7a732 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.023 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f5e4aee9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800e070 con 0x7f81800b1300 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817800b8e0 con 0x7f81800b1300 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8184cdf700 1 -- 172.21.15.46:0/467797211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170001470 con 0x7f81800b1300 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 -- 172.21.15.46:0/467797211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800b1300 msgr2=0x7f81800ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 --2- 172.21.15.46:0/467797211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800b1300 0x7f81800ae7d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f8178009e60 tx=0x7f8178009780 comp rx=0 tx=0).stop 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 -- 172.21.15.46:0/467797211 shutdown_connections 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 --2- 172.21.15.46:0/467797211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800b1300 0x7f81800ae7d0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 -- 172.21.15.46:0/467797211 >> 172.21.15.46:0/467797211 conn(0x7f818006ef60 msgr2=0x7f81800d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 -- 172.21.15.46:0/467797211 shutdown_connections 2023-10-06T12:52:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.016+0000 7f8186ce3700 1 -- 172.21.15.46:0/467797211 wait complete. 2023-10-06T12:52:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 Processor -- start 2023-10-06T12:52:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 -- start start 2023-10-06T12:52:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81800b1300 0x7f8180146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8180159290 0x7f8180067a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818009d9f0 con 0x7f81800d07b0 2023-10-06T12:52:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818009cde0 con 0x7f81800b1300 2023-10-06T12:52:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f8186ce3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818009d0c0 con 0x7f8180159290 2023-10-06T12:52:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f81854e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f81854e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39188/0 (socket says 172.21.15.46:39188) 2023-10-06T12:52:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f81854e0700 1 -- 172.21.15.46:0/235689414 learned_addr learned my addr 172.21.15.46:0/235689414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.020+0000 7f81864e2700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8180159290 0x7f8180067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8185ce1700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81800b1300 0x7f8180146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8180159290 msgr2=0x7f8180067a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8180159290 0x7f8180067a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81800b1300 msgr2=0x7f8180146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81800b1300 0x7f8180146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 -- 172.21.15.46:0/235689414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178009a20 con 0x7f81800d07b0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f81854e0700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f81700015a0 tx=0x7f8170001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170007360 con 0x7f81800d07b0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81700074e0 con 0x7f81800d07b0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170004830 con 0x7f81800d07b0 2023-10-06T12:52:08.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818009d3a0 con 0x7f81800d07b0 2023-10-06T12:52:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818009ca80 con 0x7f81800d07b0 2023-10-06T12:52:08.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.024+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818009db60 con 0x7f81800d07b0 2023-10-06T12:52:08.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.028+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7f8170004bc0 con 0x7f81800d07b0 2023-10-06T12:52:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.028+0000 7f8175ffb700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 0x7f8164062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.028+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7f817007c530 con 0x7f81800d07b0 2023-10-06T12:52:08.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.028+0000 7f8185ce1700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 msgr2=0x7f8164062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:08.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.028+0000 7f8185ce1700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 0x7f8164062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:08.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.036+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f817004c360 con 0x7f81800d07b0 2023-10-06T12:52:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.232+0000 7f8185ce1700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 msgr2=0x7f8164062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.232+0000 7f8185ce1700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 0x7f8164062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.244+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8180064590 con 0x7f81800d07b0 2023-10-06T12:52:08.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.244+0000 7f8175ffb700 1 -- 172.21.15.46:0/235689414 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4598 (secure 0 0 0) 0x7f817004f980 con 0x7f81800d07b0 2023-10-06T12:52:08.252 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:08.253 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:50:40.722486+0000","modified":"2023-10-06T12:50:45.339830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9619":{"gid":9619,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/192039016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":192039016},{"type":"v1","addr":"172.21.15.46:6835","nonce":192039016}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 msgr2=0x7f8164062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 0x7f8164062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 msgr2=0x7f818009c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f81700015a0 tx=0x7f8170001bf0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 shutdown_connections 2023-10-06T12:52:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8180159290 0x7f8180067a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81800d07b0 0x7f818009c540 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8164060790 0x7f8164062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 --2- 172.21.15.46:0/235689414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81800b1300 0x7f8180146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 >> 172.21.15.46:0/235689414 conn(0x7f818006ef60 msgr2=0x7f81800b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 shutdown_connections 2023-10-06T12:52:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.248+0000 7f8186ce3700 1 -- 172.21.15.46:0/235689414 wait complete. 2023-10-06T12:52:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2023-10-06T12:52:08.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:08.418 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:08.411+0000 7f1a77968000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:08.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.432+0000 7fd91f59e700 1 Processor -- start 2023-10-06T12:52:08.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.432+0000 7fd91f59e700 1 -- start start 2023-10-06T12:52:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9200af000 0x7fd9200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200cf100 con 0x7fd9200af3d0 2023-10-06T12:52:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200ce4f0 con 0x7fd9200cb4c0 2023-10-06T12:52:08.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200ce300 con 0x7fd9200af000 2023-10-06T12:52:08.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91e59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9200af000 0x7fd9200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91ed9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd9200af3d0 0x7fd9200690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36960/0 (socket says 172.21.15.46:36960) 2023-10-06T12:52:08.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd917fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd9200cb4c0 0x7fd9200cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52038/0 (socket says 172.21.15.46:52038) 2023-10-06T12:52:08.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91ed9d700 1 -- 172.21.15.46:0/1132200746 learned_addr learned my addr 172.21.15.46:0/1132200746 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:08.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91e59c700 1 -- 172.21.15.46:0/1132200746 >> v1:172.21.15.132:6789/0 conn(0x7fd9200cb4c0 legacy=0x7fd9200cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91e59c700 1 -- 172.21.15.46:0/1132200746 >> v1:172.21.15.46:6789/0 conn(0x7fd9200af3d0 legacy=0x7fd9200690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91e59c700 1 -- 172.21.15.46:0/1132200746 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200ce7d0 con 0x7fd9200af000 2023-10-06T12:52:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91e59c700 1 --2- 172.21.15.46:0/1132200746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af000 0x7fd9200cf590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd9100098e0 tx=0x7fd9100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9c2462b5c1f01d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91dd9b700 1 -- 172.21.15.46:0/1132200746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910009e80 con 0x7fd9200af000 2023-10-06T12:52:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91dd9b700 1 -- 172.21.15.46:0/1132200746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91000b860 con 0x7fd9200af000 2023-10-06T12:52:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.436+0000 7fd91dd9b700 1 -- 172.21.15.46:0/1132200746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000bbe0 con 0x7fd9200af000 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- 172.21.15.46:0/1132200746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af000 msgr2=0x7fd9200cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 --2- 172.21.15.46:0/1132200746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af000 0x7fd9200cf590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd9100098e0 tx=0x7fd9100091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- 172.21.15.46:0/1132200746 shutdown_connections 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 --2- 172.21.15.46:0/1132200746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af000 0x7fd9200cf590 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- 172.21.15.46:0/1132200746 >> 172.21.15.46:0/1132200746 conn(0x7fd9200683b0 msgr2=0x7fd9200d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- 172.21.15.46:0/1132200746 shutdown_connections 2023-10-06T12:52:08.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- 172.21.15.46:0/1132200746 wait complete. 2023-10-06T12:52:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 Processor -- start 2023-10-06T12:52:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.440+0000 7fd91f59e700 1 -- start start 2023-10-06T12:52:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9200af000 0x7fd92014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9200cb4c0 0x7fd9200b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920150ce0 con 0x7fd9200af3d0 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920150160 con 0x7fd9200af000 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd920150440 con 0x7fd9200cb4c0 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39202/0 (socket says 172.21.15.46:39202) 2023-10-06T12:52:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 -- 172.21.15.46:0/3635175216 learned_addr learned my addr 172.21.15.46:0/3635175216 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91ed9d700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9200cb4c0 0x7fd9200b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91e59c700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9200af000 0x7fd92014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 -- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9200cb4c0 msgr2=0x7fd9200b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9200cb4c0 0x7fd9200b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 -- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9200af000 msgr2=0x7fd92014ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9200af000 0x7fd92014ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 -- 172.21.15.46:0/3635175216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009420 con 0x7fd9200af3d0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd917fff700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fd908001eb0 tx=0x7fd908007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908005a70 con 0x7fd9200af3d0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908005bd0 con 0x7fd9200af3d0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080070d0 con 0x7fd9200af3d0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd920150720 con 0x7fd9200af3d0 2023-10-06T12:52:08.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.444+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92014fda0 con 0x7fd9200af3d0 2023-10-06T12:52:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.448+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92015ac20 con 0x7fd9200af3d0 2023-10-06T12:52:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.448+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd908007480 con 0x7fd9200af3d0 2023-10-06T12:52:08.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.452+0000 7fd9167fc700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 0x7fd900062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.452+0000 7fd91e59c700 1 -- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 msgr2=0x7fd900062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:08.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.452+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7fd90807c5d0 con 0x7fd9200af3d0 2023-10-06T12:52:08.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.452+0000 7fd91e59c700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 0x7fd900062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:08.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.456+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd90804c380 con 0x7fd9200af3d0 2023-10-06T12:52:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.628+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd920064590 con 0x7fd9200af3d0 2023-10-06T12:52:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.628+0000 7fd9167fc700 1 -- 172.21.15.46:0/3635175216 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+18314 (secure 0 0 0) 0x7fd90804f9a0 con 0x7fd9200af3d0 2023-10-06T12:52:08.636 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:08.637 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":185,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:05.974313+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:50:38.842191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T12:50:39.812688+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":185,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:6832/59347":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3422943682":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3376719297":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/2850989399":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/3444801103":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/300086595":"2023-10-07T12:50:57.495606+0000","172.21.15.46:0/414605609":"2023-10-07T12:50:57.495606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 msgr2=0x7fd900062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 0x7fd900062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 msgr2=0x7fd92014f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fd908001eb0 tx=0x7fd908007f40 comp rx=0 tx=0).stop 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 shutdown_connections 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9200cb4c0 0x7fd9200b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9200af3d0 0x7fd92014f860 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd900060790 0x7fd900062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 --2- 172.21.15.46:0/3635175216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9200af000 0x7fd92014ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 >> 172.21.15.46:0/3635175216 conn(0x7fd9200683b0 msgr2=0x7fd9200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 shutdown_connections 2023-10-06T12:52:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.632+0000 7fd91f59e700 1 -- 172.21.15.46:0/3635175216 wait complete. 2023-10-06T12:52:08.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T12:52:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 Processor -- start 2023-10-06T12:52:08.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 -- start start 2023-10-06T12:52:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7ac067760 0x7fd7ac0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 0x7fd7ac0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac0c8480 con 0x7fd7ac0bedd0 2023-10-06T12:52:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ac069660 con 0x7fd7ac067760 2023-10-06T12:52:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.808+0000 7fd7b288e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac068ac0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b188c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7ac0bedd0 0x7fd7ac067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36976/0 (socket says 172.21.15.46:36976) 2023-10-06T12:52:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b188c700 1 -- 172.21.15.46:0/1368669058 learned_addr learned my addr 172.21.15.46:0/1368669058 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 --2- 172.21.15.46:0/1368669058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 0x7fd7ac0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b108b700 1 --2- 172.21.15.46:0/1368669058 >> v2:172.21.15.132:3300/0 conn(0x7fd7ac067760 0x7fd7ac0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b088a700 1 -- 172.21.15.46:0/1368669058 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995863145 0 0) 0x7fd7a8001110 con 0x7fd7ac0bedd0 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b088a700 1 -- 172.21.15.46:0/1368669058 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd798003570 con 0x7fd7ac0bedd0 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 -- 172.21.15.46:0/1368669058 >> v2:172.21.15.132:3300/0 conn(0x7fd7ac067760 msgr2=0x7fd7ac0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 --2- 172.21.15.46:0/1368669058 >> v2:172.21.15.132:3300/0 conn(0x7fd7ac067760 0x7fd7ac0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 -- 172.21.15.46:0/1368669058 >> v1:172.21.15.46:6789/0 conn(0x7fd7ac0bedd0 legacy=0x7fd7ac067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 -- 172.21.15.46:0/1368669058 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac068d30 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b208d700 1 --2- 172.21.15.46:0/1368669058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 0x7fd7ac0c7f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009e60 tx=0x7fd7a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe950df15d74195 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b088a700 1 -- 172.21.15.46:0/1368669058 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400e070 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b088a700 1 -- 172.21.15.46:0/1368669058 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400bab0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b088a700 1 -- 172.21.15.46:0/1368669058 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a400be70 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 -- 172.21.15.46:0/1368669058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 msgr2=0x7fd7ac0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 --2- 172.21.15.46:0/1368669058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 0x7fd7ac0c7f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009e60 tx=0x7fd7a4009780 comp rx=0 tx=0).stop 2023-10-06T12:52:08.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 -- 172.21.15.46:0/1368669058 shutdown_connections 2023-10-06T12:52:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 --2- 172.21.15.46:0/1368669058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bf1a0 0x7fd7ac0c7f40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 --2- 172.21.15.46:0/1368669058 >> v2:172.21.15.132:3300/0 conn(0x7fd7ac067760 0x7fd7ac0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 -- 172.21.15.46:0/1368669058 >> 172.21.15.46:0/1368669058 conn(0x7fd7ac062ef0 msgr2=0x7fd7ac0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 -- 172.21.15.46:0/1368669058 shutdown_connections 2023-10-06T12:52:08.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.816+0000 7fd7b288e700 1 -- 172.21.15.46:0/1368669058 wait complete. 2023-10-06T12:52:08.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.820+0000 7fd7b288e700 1 Processor -- start 2023-10-06T12:52:08.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.820+0000 7fd7b288e700 1 -- start start 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ac067760 0x7fd7ac0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bedd0 0x7fd7ac14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0c66c0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0c50e0 con 0x7fd7ac067760 2023-10-06T12:52:08.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac0c53c0 con 0x7fd7ac0bedd0 2023-10-06T12:52:08.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39208/0 (socket says 172.21.15.46:39208) 2023-10-06T12:52:08.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 -- 172.21.15.46:0/2704165888 learned_addr learned my addr 172.21.15.46:0/2704165888 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b108b700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bedd0 0x7fd7ac14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b188c700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ac067760 0x7fd7ac0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 -- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bedd0 msgr2=0x7fd7ac14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bedd0 0x7fd7ac14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 -- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ac067760 msgr2=0x7fd7ac0c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ac067760 0x7fd7ac0c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 -- 172.21.15.46:0/2704165888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4009a20 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b208d700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4013000 tx=0x7fd7a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4027470 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40275f0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac0c56a0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40279b0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.824+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac0c5c60 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.828+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac0c5e50 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.828+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 107) v1 ==== 80175+0+0 (secure 0 0 0) 0x7fd7a4024070 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.828+0000 7fd7a1ffb700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 0x7fd790062c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:08.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.832+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7002+0+0 (secure 0 0 0) 0x7fd7a407fb20 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.832+0000 7fd7b188c700 1 -- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 msgr2=0x7fd790062c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:08.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.832+0000 7fd7b188c700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 0x7fd790062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:08.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.836+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7a404f8d0 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.844 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.840+0000 7f5e4aee9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.860+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 108) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fd7a4003040 con 0x7fd7ac0bf1a0 2023-10-06T12:52:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.860+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 msgr2=0x7fd790062c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:08.860+0000 7fd7a1ffb700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 0x7fd790062c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e1858a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e1858a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e1858a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e1858a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e1858a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.957 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e13d81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.958 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e13d81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.958 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e13d81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.958 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e13d81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:08.958 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:08.952+0000 7f5e13d81700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.084+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7ac064590 con 0x7fd7ac0bf1a0 2023-10-06T12:52:09.089 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T12:52:09.084+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T12:52:09.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.096+0000 7fd7a1ffb700 1 -- 172.21.15.46:0/2704165888 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd7a4055600 con 0x7fd7ac0bf1a0 2023-10-06T12:52:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 msgr2=0x7fd7ac15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4013000 tx=0x7fd7a4019000 comp rx=0 tx=0).stop 2023-10-06T12:52:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 shutdown_connections 2023-10-06T12:52:09.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7ac0bf1a0 0x7fd7ac15e510 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7ac0bedd0 0x7fd7ac14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd790060790 0x7fd790062c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 --2- 172.21.15.46:0/2704165888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7ac067760 0x7fd7ac0c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 >> 172.21.15.46:0/2704165888 conn(0x7fd7ac062ef0 msgr2=0x7fd7ac0cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 shutdown_connections 2023-10-06T12:52:09.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.100+0000 7fd7b288e700 1 -- 172.21.15.46:0/2704165888 wait complete. 2023-10-06T12:52:09.106 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T12:52:09.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T12:52:09.167 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:52:09.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.296+0000 7fb44fdc5700 1 Processor -- start 2023-10-06T12:52:09.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.296+0000 7fb44fdc5700 1 -- start start 2023-10-06T12:52:09.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44fdc5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb448146eb0 0x7fb4480c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:09.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44fdc5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4480c1980 con 0x7fb44806d340 2023-10-06T12:52:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44fdc5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4480c0d70 con 0x7fb4480c2350 2023-10-06T12:52:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44fdc5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4480c1050 con 0x7fb448146eb0 2023-10-06T12:52:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44e362700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb44806d340 0x7fb4480caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36994/0 (socket says 172.21.15.46:36994) 2023-10-06T12:52:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44e362700 1 -- 172.21.15.46:0/1231134446 learned_addr learned my addr 172.21.15.46:0/1231134446 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:09.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44db61700 1 --2- 172.21.15.46:0/1231134446 >> v2:172.21.15.132:3300/0 conn(0x7fb448146eb0 0x7fb4480c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859674488 0 0) 0x7fb4480c1980 con 0x7fb44806d340 2023-10-06T12:52:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb434003570 con 0x7fb44806d340 2023-10-06T12:52:09.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44db61700 1 -- 172.21.15.46:0/1231134446 >> v1:172.21.15.132:6789/0 conn(0x7fb4480c2350 legacy=0x7fb44806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3460792936 0 0) 0x7fb434003570 con 0x7fb44806d340 2023-10-06T12:52:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44db61700 1 -- 172.21.15.46:0/1231134446 >> v1:172.21.15.46:6789/0 conn(0x7fb44806d340 legacy=0x7fb4480caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44db61700 1 -- 172.21.15.46:0/1231134446 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480c1330 con 0x7fb448146eb0 2023-10-06T12:52:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44db61700 1 --2- 172.21.15.46:0/1231134446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb448146eb0 0x7fb4480c1e10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098e0 tx=0x7fb4440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c20a388d51fc5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:09.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400017f0 con 0x7fb448146eb0 2023-10-06T12:52:09.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.300+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444011420 con 0x7fb448146eb0 2023-10-06T12:52:09.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44cb5f700 1 -- 172.21.15.46:0/1231134446 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4440117c0 con 0x7fb448146eb0 2023-10-06T12:52:09.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- 172.21.15.46:0/1231134446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb448146eb0 msgr2=0x7fb4480c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/1231134446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb448146eb0 0x7fb4480c1e10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb4440098e0 tx=0x7fb4440091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- 172.21.15.46:0/1231134446 shutdown_connections 2023-10-06T12:52:09.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/1231134446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb448146eb0 0x7fb4480c1e10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- 172.21.15.46:0/1231134446 >> 172.21.15.46:0/1231134446 conn(0x7fb4480cbe30 msgr2=0x7fb4480cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:09.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- 172.21.15.46:0/1231134446 shutdown_connections 2023-10-06T12:52:09.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- 172.21.15.46:0/1231134446 wait complete. 2023-10-06T12:52:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 Processor -- start 2023-10-06T12:52:09.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.304+0000 7fb44fdc5700 1 -- start start 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 0x7fb44815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4480c2350 0x7fb44815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44815fdc0 con 0x7fb448146eb0 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44816edb0 con 0x7fb4480c2350 2023-10-06T12:52:09.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb448162650 con 0x7fb44806d340 2023-10-06T12:52:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39358/0 (socket says 172.21.15.46:39358) 2023-10-06T12:52:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 -- 172.21.15.46:0/3057104451 learned_addr learned my addr 172.21.15.46:0/3057104451 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44db61700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 0x7fb44815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:09.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44d360700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4480c2350 0x7fb44815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 -- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 msgr2=0x7fb44815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 0x7fb44815d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 -- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4480c2350 msgr2=0x7fb44815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4480c2350 0x7fb44815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 -- 172.21.15.46:0/3057104451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb444009420 con 0x7fb448146eb0 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44db61700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 0x7fb44815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:09.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44e362700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb44000cc40 tx=0x7fb440001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440004dc0 con 0x7fb448146eb0 2023-10-06T12:52:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44000a660 con 0x7fb448146eb0 2023-10-06T12:52:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000aa20 con 0x7fb448146eb0 2023-10-06T12:52:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.308+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44815f520 con 0x7fb448146eb0 2023-10-06T12:52:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.312+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44815fae0 con 0x7fb448146eb0 2023-10-06T12:52:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.312+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4480660f0 con 0x7fb448146eb0 2023-10-06T12:52:09.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.312+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 108) v1 ==== 79670+0+0 (secure 0 0 0) 0x7fb440007150 con 0x7fb448146eb0 2023-10-06T12:52:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.316+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 7404+0+0 (secure 0 0 0) 0x7fb44007c1d0 con 0x7fb448146eb0 2023-10-06T12:52:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.324+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb44004c470 con 0x7fb448146eb0 2023-10-06T12:52:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.540+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb448162790 con 0x7fb448146eb0 2023-10-06T12:52:09.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.872+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 109) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fb44004bf20 con 0x7fb448146eb0 2023-10-06T12:52:09.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.872+0000 7fb43dffb700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 0x7fb42c068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.872+0000 7fb44db61700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 0x7fb42c068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:09.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:09.876+0000 7fb44db61700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 0x7fb42c068f60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4440091e0 tx=0x7fb444016000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:10.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.100+0000 7fb43dffb700 1 -- 172.21.15.46:0/3057104451 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb440078030 con 0x7fb448146eb0 2023-10-06T12:52:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 msgr2=0x7fb42c068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 0x7fb42c068f60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb4440091e0 tx=0x7fb444016000 comp rx=0 tx=0).stop 2023-10-06T12:52:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 msgr2=0x7fb44816e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb44000cc40 tx=0x7fb440001640 comp rx=0 tx=0).stop 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 shutdown_connections 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb448146eb0 0x7fb44816e870 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb42c066b80 0x7fb42c068f60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4480c2350 0x7fb44815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 --2- 172.21.15.46:0/3057104451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb44806d340 0x7fb44815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 >> 172.21.15.46:0/3057104451 conn(0x7fb4480cbe30 msgr2=0x7fb44806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 shutdown_connections 2023-10-06T12:52:10.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.104+0000 7fb44fdc5700 1 -- 172.21.15.46:0/3057104451 wait complete. 2023-10-06T12:52:10.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T12:52:10.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.344+0000 7fce77345700 1 Processor -- start 2023-10-06T12:52:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.344+0000 7fce77345700 1 -- start start 2023-10-06T12:52:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce77345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce700d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce77345700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700ce2c0 con 0x7fce70159130 2023-10-06T12:52:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce77345700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700af570 con 0x7fce70159500 2023-10-06T12:52:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce77345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700afb30 con 0x7fce700698a0 2023-10-06T12:52:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce750e1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce70159130 0x7fce700aed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37010/0 (socket says 172.21.15.46:37010) 2023-10-06T12:52:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce750e1700 1 -- 172.21.15.46:0/1310426225 learned_addr learned my addr 172.21.15.46:0/1310426225 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.348+0000 7fce758e2700 1 --2- 172.21.15.46:0/1310426225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce700d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802514486 0 0) 0x7fce700ce2c0 con 0x7fce70159130 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c003570 con 0x7fce70159130 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575509807 0 0) 0x7fce700af570 con 0x7fce70159500 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce700ce2c0 con 0x7fce70159500 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce758e2700 1 -- 172.21.15.46:0/1310426225 >> v1:172.21.15.132:6789/0 conn(0x7fce70159500 legacy=0x7fce70069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce758e2700 1 -- 172.21.15.46:0/1310426225 >> v1:172.21.15.46:6789/0 conn(0x7fce70159130 legacy=0x7fce700aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006379497 0 0) 0x7fce5c003570 con 0x7fce70159130 2023-10-06T12:52:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce758e2700 1 -- 172.21.15.46:0/1310426225 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700af850 con 0x7fce700698a0 2023-10-06T12:52:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce758e2700 1 --2- 172.21.15.46:0/1310426225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce700d0d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e90 tx=0x7fce68009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d060b529b8b2ed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e070 con 0x7fce700698a0 2023-10-06T12:52:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800b870 con 0x7fce700698a0 2023-10-06T12:52:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce67fff700 1 -- 172.21.15.46:0/1310426225 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c001470 con 0x7fce700698a0 2023-10-06T12:52:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 -- 172.21.15.46:0/1310426225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 msgr2=0x7fce700d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 --2- 172.21.15.46:0/1310426225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce700d0d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e90 tx=0x7fce68009780 comp rx=0 tx=0).stop 2023-10-06T12:52:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 -- 172.21.15.46:0/1310426225 shutdown_connections 2023-10-06T12:52:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 --2- 172.21.15.46:0/1310426225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce700d0d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 -- 172.21.15.46:0/1310426225 >> 172.21.15.46:0/1310426225 conn(0x7fce70068510 msgr2=0x7fce700d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 -- 172.21.15.46:0/1310426225 shutdown_connections 2023-10-06T12:52:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.352+0000 7fce77345700 1 -- 172.21.15.46:0/1310426225 wait complete. 2023-10-06T12:52:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.356+0000 7fce77345700 1 Processor -- start 2023-10-06T12:52:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.356+0000 7fce77345700 1 -- start start 2023-10-06T12:52:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce7016cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce70159500 0x7fce700cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70154a70 con 0x7fce70159130 2023-10-06T12:52:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7015a1c0 con 0x7fce70159500 2023-10-06T12:52:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700c7fc0 con 0x7fce700698a0 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39374/0 (socket says 172.21.15.46:39374) 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 -- 172.21.15.46:0/1585774109 learned_addr learned my addr 172.21.15.46:0/1585774109 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce750e1700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce7016cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce758e2700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce70159500 0x7fce700cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 -- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 msgr2=0x7fce7016cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce7016cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 -- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce70159500 msgr2=0x7fce700cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce70159500 0x7fce700cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 -- 172.21.15.46:0/1585774109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009a20 con 0x7fce70159130 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce750e1700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce7016cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce748e0700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c7c0 tx=0x7fce60007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60010510 con 0x7fce70159130 2023-10-06T12:52:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60010670 con 0x7fce70159130 2023-10-06T12:52:10.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60010a30 con 0x7fce70159130 2023-10-06T12:52:10.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce70154ea0 con 0x7fce70159130 2023-10-06T12:52:10.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.360+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700c7a00 con 0x7fce70159130 2023-10-06T12:52:10.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.364+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce700c81e0 con 0x7fce70159130 2023-10-06T12:52:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.364+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 109) v1 ==== 79797+0+0 (secure 0 0 0) 0x7fce6001b400 con 0x7fce70159130 2023-10-06T12:52:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.368+0000 7fce657fa700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 0x7fce54062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:10.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.368+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 7404+0+0 (secure 0 0 0) 0x7fce6007c9b0 con 0x7fce70159130 2023-10-06T12:52:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.368+0000 7fce750e1700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 0x7fce54062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:10.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.368+0000 7fce750e1700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 0x7fce54062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0011e0 tx=0x7fce6c000bc0 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.372+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fce6004cae0 con 0x7fce70159130 2023-10-06T12:52:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:10.544+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce7004b5c0 con 0x7fce70159130 2023-10-06T12:52:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.104+0000 7fce657fa700 1 -- 172.21.15.46:0/1585774109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v188) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce60050100 con 0x7fce70159130 2023-10-06T12:52:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 msgr2=0x7fce54062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 0x7fce54062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0011e0 tx=0x7fce6c000bc0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 msgr2=0x7fce700ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c7c0 tx=0x7fce60007e10 comp rx=0 tx=0).stop 2023-10-06T12:52:11.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 shutdown_connections 2023-10-06T12:52:11.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fce540603a0 0x7fce54062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce70159500 0x7fce700cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce70159130 0x7fce700ccac0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 --2- 172.21.15.46:0/1585774109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce700698a0 0x7fce7016cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 >> 172.21.15.46:0/1585774109 conn(0x7fce70068510 msgr2=0x7fce700b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:11.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 shutdown_connections 2023-10-06T12:52:11.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.112+0000 7fce77345700 1 -- 172.21.15.46:0/1585774109 wait complete. 2023-10-06T12:52:11.118 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T12:52:11.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T12:52:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.304+0000 7f4bc0545700 1 Processor -- start 2023-10-06T12:52:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.304+0000 7f4bc0545700 1 -- start start 2023-10-06T12:52:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bc0545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bc0545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bb8156bf0 0x7f4bb8157010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bc0545700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80c6870 con 0x7f4bb8153c40 2023-10-06T12:52:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bc0545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb806e690 con 0x7f4bb8156bf0 2023-10-06T12:52:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bc0545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80c7d10 con 0x7f4bb8153870 2023-10-06T12:52:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbdae0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4bb8153c40 0x7f4bb81564e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37026/0 (socket says 172.21.15.46:37026) 2023-10-06T12:52:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbdae0700 1 -- 172.21.15.46:0/2689705259 learned_addr learned my addr 172.21.15.46:0/2689705259 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/2689705259 >> v2:172.21.15.46:3300/0 conn(0x7f4bb8156bf0 0x7f4bb8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbe2e1700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbeae2700 1 -- 172.21.15.46:0/2689705259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 msgr2=0x7f4bb806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbeae2700 1 -- 172.21.15.46:0/2689705259 >> v1:172.21.15.46:6789/0 conn(0x7f4bb8153c40 legacy=0x7f4bb81564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbeae2700 1 -- 172.21.15.46:0/2689705259 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb806e8a0 con 0x7f4bb8156bf0 2023-10-06T12:52:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.308+0000 7f4bbd2df700 1 -- 172.21.15.46:0/2689705259 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425326955 0 0) 0x7f4bb80c6870 con 0x7f4bb8153c40 2023-10-06T12:52:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb8157010 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009e60 tx=0x7f4bb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e801bc75f4b4af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bbd2df700 1 -- 172.21.15.46:0/2689705259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000e070 con 0x7f4bb8156bf0 2023-10-06T12:52:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bbd2df700 1 -- 172.21.15.46:0/2689705259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000bb60 con 0x7f4bb8156bf0 2023-10-06T12:52:11.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bbd2df700 1 -- 172.21.15.46:0/2689705259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0012430 con 0x7f4bb8156bf0 2023-10-06T12:52:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 -- 172.21.15.46:0/2689705259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 msgr2=0x7f4bb8157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb8157010 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009e60 tx=0x7f4bb0009780 comp rx=0 tx=0).stop 2023-10-06T12:52:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 -- 172.21.15.46:0/2689705259 shutdown_connections 2023-10-06T12:52:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb8157010 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 --2- 172.21.15.46:0/2689705259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 -- 172.21.15.46:0/2689705259 >> 172.21.15.46:0/2689705259 conn(0x7f4bb80cbe20 msgr2=0x7f4bb80cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 -- 172.21.15.46:0/2689705259 shutdown_connections 2023-10-06T12:52:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.312+0000 7f4bc0545700 1 -- 172.21.15.46:0/2689705259 wait complete. 2023-10-06T12:52:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 Processor -- start 2023-10-06T12:52:11.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 -- start start 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb80b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 0x7f4bb80c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80aebc0 con 0x7f4bb8156bf0 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80aeea0 con 0x7f4bb8153c40 2023-10-06T12:52:11.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bc0545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80c5770 con 0x7f4bb8153870 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bbeae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bbeae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39400/0 (socket says 172.21.15.46:39400) 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.316+0000 7f4bbeae2700 1 -- 172.21.15.46:0/3799127127 learned_addr learned my addr 172.21.15.46:0/3799127127 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbe2e1700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb80b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbdae0700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 0x7f4bb80c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 -- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 msgr2=0x7f4bb80b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb80b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 -- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 msgr2=0x7f4bb80c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 0x7f4bb80c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 -- 172.21.15.46:0/3799127127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009a20 con 0x7f4bb8156bf0 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbe2e1700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbdae0700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 0x7f4bb80c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bbeae2700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0012a50 tx=0x7f4bb0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0012e60 con 0x7f4bb8156bf0 2023-10-06T12:52:11.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb001d630 con 0x7f4bb8156bf0 2023-10-06T12:52:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb001d9f0 con 0x7f4bb8156bf0 2023-10-06T12:52:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80c59f0 con 0x7f4bb8156bf0 2023-10-06T12:52:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80c5fb0 con 0x7f4bb8156bf0 2023-10-06T12:52:11.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.320+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb806b890 con 0x7f4bb8156bf0 2023-10-06T12:52:11.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.324+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4bb0024070 con 0x7f4bb8156bf0 2023-10-06T12:52:11.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.324+0000 7f4bae7fc700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 0x7f4b9c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:11.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.324+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6966+0+0 (secure 0 0 0) 0x7f4bb007f740 con 0x7f4bb8156bf0 2023-10-06T12:52:11.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.324+0000 7f4bbe2e1700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 0x7f4b9c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:11.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.324+0000 7f4bbe2e1700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 0x7f4b9c062b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006f00 tx=0x7f4bb4008040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.332+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4bb004f690 con 0x7f4bb8156bf0 2023-10-06T12:52:11.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:11.500+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4bb80aefe0 con 0x7f4bb8156bf0 2023-10-06T12:52:12.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.124+0000 7f4bae7fc700 1 -- 172.21.15.46:0/3799127127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v189) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4bb0052cb0 con 0x7f4bb8156bf0 2023-10-06T12:52:12.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.128+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 msgr2=0x7f4b9c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.128+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 0x7f4b9c062b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006f00 tx=0x7f4bb4008040 comp rx=0 tx=0).stop 2023-10-06T12:52:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.128+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 msgr2=0x7f4bb81523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.128+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0012a50 tx=0x7f4bb0019000 comp rx=0 tx=0).stop 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 shutdown_connections 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bb8156bf0 0x7f4bb81523b0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b9c0606a0 0x7f4b9c062b50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bb8153c40 0x7f4bb80c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 --2- 172.21.15.46:0/3799127127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bb8153870 0x7f4bb80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 >> 172.21.15.46:0/3799127127 conn(0x7f4bb80cbe20 msgr2=0x7f4bb80cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 shutdown_connections 2023-10-06T12:52:12.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.132+0000 7f4bc0545700 1 -- 172.21.15.46:0/3799127127 wait complete. 2023-10-06T12:52:12.137 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T12:52:12.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T12:52:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.320+0000 7f764f59e700 1 Processor -- start 2023-10-06T12:52:12.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.320+0000 7f764f59e700 1 -- start start 2023-10-06T12:52:12.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f76500c7630 0x7f76500ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f765015f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765006e0d0 con 0x7f76500c6250 2023-10-06T12:52:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76500c0d60 con 0x7f76500c7630 2023-10-06T12:52:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500c7a80 con 0x7f76500ca6d0 2023-10-06T12:52:12.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764e59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f76500c6250 0x7f76500c9a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37034/0 (socket says 172.21.15.46:37034) 2023-10-06T12:52:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764e59c700 1 -- 172.21.15.46:0/3536622746 learned_addr learned my addr 172.21.15.46:0/3536622746 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f7645d9b700 1 --2- 172.21.15.46:0/3536622746 >> v2:172.21.15.46:3300/0 conn(0x7f76500c7630 0x7f76500ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764ed9d700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f765015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299253700 0 0) 0x7f765006e0d0 con 0x7f76500c6250 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f763c003570 con 0x7f76500c6250 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981729103 0 0) 0x7f763c003570 con 0x7f76500c6250 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 msgr2=0x7f765015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f765015f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 >> v2:172.21.15.46:3300/0 conn(0x7f76500c7630 msgr2=0x7f76500ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:12.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 --2- 172.21.15.46:0/3536622746 >> v2:172.21.15.46:3300/0 conn(0x7f76500c7630 0x7f76500ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765006e3b0 con 0x7f76500c6250 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f7640001810 con 0x7f76500c6250 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f7645d9b700 1 --2- 172.21.15.46:0/3536622746 >> v2:172.21.15.46:3300/0 conn(0x7f76500c7630 0x7f76500ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 >> v1:172.21.15.46:6789/0 conn(0x7f76500c6250 legacy=0x7f76500c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c003a90 0x7f763c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f763c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f763c004d50 0x7f763c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500c0d60 con 0x7f763c004d50 2023-10-06T12:52:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500c7a80 con 0x7f763c003a90 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c00ba90 con 0x7f76500ca6d0 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.324+0000 7f764ed9d700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f763c004d50 0x7f763c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764e59c700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c003a90 0x7f763c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f7645d9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f763c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 -- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 msgr2=0x7f763c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f763c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 -- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c003a90 msgr2=0x7f763c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c003a90 0x7f763c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 -- 172.21.15.46:0/3536622746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c003570 con 0x7f763c004d50 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f7645d9b700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f76500ca6d0 0x7f763c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f763c004d50 0x7f763c0051c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f7648009c00 tx=0x7f76480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4379bd8acd62fd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764800e6b0 con 0x7f763c004d50 2023-10-06T12:52:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764800e810 con 0x7f763c004d50 2023-10-06T12:52:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764dd9b700 1 -- 172.21.15.46:0/3536622746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764800ebd0 con 0x7f763c004d50 2023-10-06T12:52:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 -- 172.21.15.46:0/3536622746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f763c004d50 msgr2=0x7f763c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 --2- 172.21.15.46:0/3536622746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f763c004d50 0x7f763c0051c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f7648009c00 tx=0x7f76480091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764ed9d700 1 -- 172.21.15.46:0/3536622746 reap_dead start 2023-10-06T12:52:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 -- 172.21.15.46:0/3536622746 shutdown_connections 2023-10-06T12:52:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 -- 172.21.15.46:0/3536622746 >> 172.21.15.46:0/3536622746 conn(0x7f76500cbc10 msgr2=0x7f76500cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 -- 172.21.15.46:0/3536622746 shutdown_connections 2023-10-06T12:52:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.328+0000 7f764f59e700 1 -- 172.21.15.46:0/3536622746 wait complete. 2023-10-06T12:52:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 Processor -- start 2023-10-06T12:52:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 -- start start 2023-10-06T12:52:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c004d50 0x7f7650157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7650147010 0x7f7650147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650146910 con 0x7f76500ae9b0 2023-10-06T12:52:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500aef70 con 0x7f763c004d50 2023-10-06T12:52:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500af810 con 0x7f7650147010 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f7645d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f7645d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39418/0 (socket says 172.21.15.46:39418) 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f7645d9b700 1 -- 172.21.15.46:0/1355457423 learned_addr learned my addr 172.21.15.46:0/1355457423 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764ed9d700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7650147010 0x7f7650147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.332+0000 7f764e59c700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c004d50 0x7f7650157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 -- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7650147010 msgr2=0x7f7650147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7650147010 0x7f7650147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 -- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c004d50 msgr2=0x7f7650157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c004d50 0x7f7650157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 -- 172.21.15.46:0/1355457423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76501479a0 con 0x7f76500ae9b0 2023-10-06T12:52:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7645d9b700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7638010cd0 tx=0x7f7638008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800e940 con 0x7f76500ae9b0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763800eaa0 con 0x7f76500ae9b0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800ee40 con 0x7f76500ae9b0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650158640 con 0x7f76500ae9b0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500af1c0 con 0x7f76500ae9b0 2023-10-06T12:52:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.336+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650064590 con 0x7f76500ae9b0 2023-10-06T12:52:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.340+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f763801c610 con 0x7f76500ae9b0 2023-10-06T12:52:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.340+0000 7f7646ffd700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 0x7f7630062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.340+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f763807d230 con 0x7f76500ae9b0 2023-10-06T12:52:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.340+0000 7f764e59c700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 0x7f7630062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.340+0000 7f764e59c700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 0x7f7630062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7650159890 tx=0x7f764000c040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.344+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f763804d320 con 0x7f76500ae9b0 2023-10-06T12:52:12.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.516+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f76500af950 con 0x7f76500ae9b0 2023-10-06T12:52:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.524+0000 7f7646ffd700 1 -- 172.21.15.46:0/1355457423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7638050940 con 0x7f76500ae9b0 2023-10-06T12:52:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 msgr2=0x7f7630062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 0x7f7630062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7650159890 tx=0x7f764000c040 comp rx=0 tx=0).stop 2023-10-06T12:52:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 msgr2=0x7f76501429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7638010cd0 tx=0x7f7638008040 comp rx=0 tx=0).stop 2023-10-06T12:52:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 shutdown_connections 2023-10-06T12:52:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7650147010 0x7f7650147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76500ae9b0 0x7f76501429b0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7630060670 0x7f7630062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 --2- 172.21.15.46:0/1355457423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f763c004d50 0x7f7650157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 >> 172.21.15.46:0/1355457423 conn(0x7f76500cbc10 msgr2=0x7f76500caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 shutdown_connections 2023-10-06T12:52:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.528+0000 7f764f59e700 1 -- 172.21.15.46:0/1355457423 wait complete. 2023-10-06T12:52:12.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T12:52:12.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.716+0000 7fdb2bb45700 1 Processor -- start 2023-10-06T12:52:12.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.716+0000 7fdb2bb45700 1 -- start start 2023-10-06T12:52:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2bb45700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb240c6470 0x7fdb240ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2bb45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2bb45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb240c1bf0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2bb45700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb240abbe0 con 0x7fdb240c6470 2023-10-06T12:52:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2bb45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2406cf80 con 0x7fdb2406ed10 2023-10-06T12:52:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb290e0700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdb2406e8a0 0x7fdb240c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37036/0 (socket says 172.21.15.46:37036) 2023-10-06T12:52:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb290e0700 1 -- 172.21.15.46:0/4208727039 learned_addr learned my addr 172.21.15.46:0/4208727039 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 --2- 172.21.15.46:0/4208727039 >> v2:172.21.15.46:3300/0 conn(0x7fdb240c6470 0x7fdb240ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb2a0e2700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 -- 172.21.15.46:0/4208727039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 msgr2=0x7fdb240c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb288df700 1 -- 172.21.15.46:0/4208727039 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652138088 0 0) 0x7fdb240c1bf0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 -- 172.21.15.46:0/4208727039 >> v1:172.21.15.46:6789/0 conn(0x7fdb2406e8a0 legacy=0x7fdb240c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 -- 172.21.15.46:0/4208727039 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2406d260 con 0x7fdb240c6470 2023-10-06T12:52:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb298e1700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb240c6470 0x7fdb240ab6a0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fdb200098e0 tx=0x7fdb200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4351db6bd8f2bb39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb288df700 1 -- 172.21.15.46:0/4208727039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20009e80 con 0x7fdb240c6470 2023-10-06T12:52:12.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb288df700 1 -- 172.21.15.46:0/4208727039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000b860 con 0x7fdb240c6470 2023-10-06T12:52:12.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.720+0000 7fdb288df700 1 -- 172.21.15.46:0/4208727039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000bbe0 con 0x7fdb240c6470 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 -- 172.21.15.46:0/4208727039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb240c6470 msgr2=0x7fdb240ab6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb240c6470 0x7fdb240ab6a0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fdb200098e0 tx=0x7fdb200091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 -- 172.21.15.46:0/4208727039 shutdown_connections 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/4208727039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb240c6470 0x7fdb240ab6a0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 -- 172.21.15.46:0/4208727039 >> 172.21.15.46:0/4208727039 conn(0x7fdb240cbe30 msgr2=0x7fdb240cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 -- 172.21.15.46:0/4208727039 shutdown_connections 2023-10-06T12:52:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.724+0000 7fdb2bb45700 1 -- 172.21.15.46:0/4208727039 wait complete. 2023-10-06T12:52:12.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 Processor -- start 2023-10-06T12:52:12.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 -- start start 2023-10-06T12:52:12.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb240c6470 0x7fdb2415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2406aa40 con 0x7fdb2406e8a0 2023-10-06T12:52:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2413e2b0 con 0x7fdb240c6470 2023-10-06T12:52:12.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2bb45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb240c3ea0 con 0x7fdb2406ed10 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39444/0 (socket says 172.21.15.46:39444) 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 -- 172.21.15.46:0/2148478669 learned_addr learned my addr 172.21.15.46:0/2148478669 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb290e0700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb2a0e2700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb240c6470 0x7fdb2415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 -- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 msgr2=0x7fdb240cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 -- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb240c6470 msgr2=0x7fdb2415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb240c6470 0x7fdb2415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.728+0000 7fdb298e1700 1 -- 172.21.15.46:0/2148478669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20009420 con 0x7fdb2406e8a0 2023-10-06T12:52:12.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb298e1700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406bf10 tx=0x7fdb20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000b9d0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2000bb30 con 0x7fdb2406e8a0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb240c3600 con 0x7fdb2406e8a0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20026660 con 0x7fdb2406e8a0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb240aede0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb240aec20 con 0x7fdb2406e8a0 2023-10-06T12:52:12.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.732+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fdb20023030 con 0x7fdb2406e8a0 2023-10-06T12:52:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.736+0000 7fdb19ffb700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 0x7fdb08062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.736+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fdb2007e5e0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.736+0000 7fdb290e0700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 0x7fdb08062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:12.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.736+0000 7fdb290e0700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 0x7fdb08062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406a390 tx=0x7fdb14009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:12.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.744+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdb2004e6d0 con 0x7fdb2406e8a0 2023-10-06T12:52:12.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.924+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fdb240aefd0 con 0x7fdb2406e8a0 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.928+0000 7fdb19ffb700 1 -- 172.21.15.46:0/2148478669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdb20051cf0 con 0x7fdb2406e8a0 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 msgr2=0x7fdb08062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 0x7fdb08062b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406a390 tx=0x7fdb14009040 comp rx=0 tx=0).stop 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 msgr2=0x7fdb2413dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fdb2406bf10 tx=0x7fdb20017000 comp rx=0 tx=0).stop 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 shutdown_connections 2023-10-06T12:52:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fdb08060670 0x7fdb08062b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb240c6470 0x7fdb2415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb2406ed10 0x7fdb240cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 --2- 172.21.15.46:0/2148478669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb2406e8a0 0x7fdb2413dd70 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 >> 172.21.15.46:0/2148478669 conn(0x7fdb240cbe30 msgr2=0x7fdb240addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 shutdown_connections 2023-10-06T12:52:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:12.932+0000 7fdb2bb45700 1 -- 172.21.15.46:0/2148478669 wait complete. 2023-10-06T12:52:13.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T12:52:13.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.620+0000 7ff56c8e3700 1 Processor -- start 2023-10-06T12:52:13.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.620+0000 7ff56c8e3700 1 -- start start 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff56c8e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5680c6250 0x7ff5680c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff56c8e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff5680c9fc0 0x7ff5680ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff56c8e3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680cad90 con 0x7ff5680ca950 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff56c8e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5680c0d60 con 0x7ff5680c6250 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff56c8e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56806e0d0 con 0x7ff5680c9fc0 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5680c6250 0x7ff5680c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5680c6250 0x7ff5680c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50038/0 (socket says 172.21.15.46:50038) 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 -- 172.21.15.46:0/2863537355 learned_addr learned my addr 172.21.15.46:0/2863537355 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff566ffd700 1 --2- 172.21.15.46:0/2863537355 >> v2:172.21.15.132:3300/0 conn(0x7ff5680c9fc0 0x7ff5680ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 -- 172.21.15.46:0/2863537355 >> v2:172.21.15.132:3300/0 conn(0x7ff5680c9fc0 msgr2=0x7ff5680ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 --2- 172.21.15.46:0/2863537355 >> v2:172.21.15.132:3300/0 conn(0x7ff5680c9fc0 0x7ff5680ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 -- 172.21.15.46:0/2863537355 >> v1:172.21.15.132:6789/0 conn(0x7ff5680ca950 legacy=0x7ff5680c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 -- 172.21.15.46:0/2863537355 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56806e3b0 con 0x7ff5680c6250 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5677fe700 1 --2- 172.21.15.46:0/2863537355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680c9a80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff56806c6e0 tx=0x7ff558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d8487b7b9dd9fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5667fc700 1 -- 172.21.15.46:0/2863537355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558009e70 con 0x7ff5680c6250 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5667fc700 1 -- 172.21.15.46:0/2863537355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55800b860 con 0x7ff5680c6250 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.624+0000 7ff5667fc700 1 -- 172.21.15.46:0/2863537355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800bc00 con 0x7ff5680c6250 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- 172.21.15.46:0/2863537355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 msgr2=0x7ff5680c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/2863537355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680c9a80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff56806c6e0 tx=0x7ff558009250 comp rx=0 tx=0).stop 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- 172.21.15.46:0/2863537355 shutdown_connections 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/2863537355 >> v2:172.21.15.132:3300/0 conn(0x7ff5680c9fc0 0x7ff5680ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/2863537355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680c9a80 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- 172.21.15.46:0/2863537355 >> 172.21.15.46:0/2863537355 conn(0x7ff5680cbc10 msgr2=0x7ff5680cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- 172.21.15.46:0/2863537355 shutdown_connections 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- 172.21.15.46:0/2863537355 wait complete. 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 Processor -- start 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.628+0000 7ff56c8e3700 1 -- start start 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5680c9fc0 0x7ff5680c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5680ca950 0x7ff5680ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c5060 con 0x7ff5680c6250 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c41d0 con 0x7ff5680ca950 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c4450 con 0x7ff5680c9fc0 2023-10-06T12:52:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50042/0 (socket says 172.21.15.46:50042) 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 -- 172.21.15.46:0/3168552598 learned_addr learned my addr 172.21.15.46:0/3168552598 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff566ffd700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5680c9fc0 0x7ff5680c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff567fff700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5680ca950 0x7ff5680ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 -- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5680c9fc0 msgr2=0x7ff5680c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5680c9fc0 0x7ff5680c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 -- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5680ca950 msgr2=0x7ff5680ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5680ca950 0x7ff5680ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 -- 172.21.15.46:0/3168552598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580094f0 con 0x7ff5680c6250 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff5677fe700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff558006fd0 tx=0x7ff558017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011420 con 0x7ff5680c6250 2023-10-06T12:52:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5580115a0 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680c4730 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011960 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.632+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5680c4cf0 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.636+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56806a390 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.636+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff5580253f0 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.636+0000 7ff54f7fe700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 0x7ff554062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.636+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7ff55807e640 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.636+0000 7ff566ffd700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 0x7ff554062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.640+0000 7ff566ffd700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 0x7ff554062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5680c9db0 tx=0x7ff55c008040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.644+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff56806a390 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.816+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff56806a390 con 0x7ff5680c6250 2023-10-06T12:52:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.824+0000 7ff54f7fe700 1 -- 172.21.15.46:0/3168552598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff56806a390 con 0x7ff5680c6250 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 msgr2=0x7ff554062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 0x7ff554062b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff5680c9db0 tx=0x7ff55c008040 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 msgr2=0x7ff5680ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff558006fd0 tx=0x7ff558017000 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 shutdown_connections 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff554060670 0x7ff554062b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5680ca950 0x7ff5680ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff5680c9fc0 0x7ff5680c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 --2- 172.21.15.46:0/3168552598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5680c6250 0x7ff5680ad100 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 >> 172.21.15.46:0/3168552598 conn(0x7ff5680cbc10 msgr2=0x7ff5680c72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 shutdown_connections 2023-10-06T12:52:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:13.828+0000 7ff56c8e3700 1 -- 172.21.15.46:0/3168552598 wait complete. 2023-10-06T12:52:13.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-06T12:52:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.140+0000 7f186d02e700 1 Processor -- start 2023-10-06T12:52:14.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.140+0000 7f186d02e700 1 -- start start 2023-10-06T12:52:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f186806e8a0 0x7f186806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186806d460 con 0x7f186815ab60 2023-10-06T12:52:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680c7000 con 0x7f18680ad6c0 2023-10-06T12:52:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186806ed40 con 0x7f186806e8a0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f1866d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f186815ab60 0x7f186815af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42708/0 (socket says 172.21.15.46:42708) 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f1866d9d700 1 -- 172.21.15.46:0/311642595 learned_addr learned my addr 172.21.15.46:0/311642595 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186659c700 1 --2- 172.21.15.46:0/311642595 >> v2:172.21.15.46:3300/0 conn(0x7f186806e8a0 0x7f186806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186659c700 1 -- 172.21.15.46:0/311642595 >> v1:172.21.15.132:6789/0 conn(0x7f18680ad6c0 legacy=0x7f18680adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186659c700 1 -- 172.21.15.46:0/311642595 >> v1:172.21.15.46:6789/0 conn(0x7f186815ab60 legacy=0x7f186815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186659c700 1 -- 172.21.15.46:0/311642595 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680c72c0 con 0x7f186806e8a0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186659c700 1 --2- 172.21.15.46:0/311642595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186806e8a0 0x7f186806cc30 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f185c0098e0 tx=0x7f185c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c0d26749698b80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f1865d9b700 1 -- 172.21.15.46:0/311642595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c009e80 con 0x7f186806e8a0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f1865d9b700 1 -- 172.21.15.46:0/311642595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00b820 con 0x7f186806e8a0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f1865d9b700 1 -- 172.21.15.46:0/311642595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00bba0 con 0x7f186806e8a0 2023-10-06T12:52:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- 172.21.15.46:0/311642595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186806e8a0 msgr2=0x7f186806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 --2- 172.21.15.46:0/311642595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186806e8a0 0x7f186806cc30 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f185c0098e0 tx=0x7f185c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- 172.21.15.46:0/311642595 shutdown_connections 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 --2- 172.21.15.46:0/311642595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186806e8a0 0x7f186806cc30 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- 172.21.15.46:0/311642595 >> 172.21.15.46:0/311642595 conn(0x7f18680cbe30 msgr2=0x7f18680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- 172.21.15.46:0/311642595 shutdown_connections 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.144+0000 7f186d02e700 1 -- 172.21.15.46:0/311642595 wait complete. 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 Processor -- start 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 -- start start 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f186806e8a0 0x7f18680b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18680ad6c0 0x7f18680b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18680b13c0 con 0x7f186815ab60 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18680c3e50 con 0x7f18680ad6c0 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.148+0000 7f186d02e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18680c38b0 con 0x7f186806e8a0 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50058/0 (socket says 172.21.15.46:50058) 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 -- 172.21.15.46:0/1297559771 learned_addr learned my addr 172.21.15.46:0/1297559771 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f1866d9d700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f186806e8a0 0x7f18680b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186659c700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18680ad6c0 0x7f18680b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 -- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f186806e8a0 msgr2=0x7f18680b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f186806e8a0 0x7f18680b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 -- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18680ad6c0 msgr2=0x7f18680b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18680ad6c0 0x7f18680b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 -- 172.21.15.46:0/1297559771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009420 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186759e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f186000c930 tx=0x7f18600026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860012630 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18600127d0 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860012b90 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18680c3b70 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.152+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186816d190 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.156+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186816cd10 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.156+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f186001a3b0 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.156+0000 7f184effd700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 0x7f1854062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.156+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f186007c9b0 con 0x7f186815ab60 2023-10-06T12:52:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.156+0000 7f1866d9d700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 0x7f1854062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.160+0000 7f1866d9d700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 0x7f1854062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18680af520 tx=0x7f1858009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.164+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f186816cd10 con 0x7f186815ab60 2023-10-06T12:52:14.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.392+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f186816cd10 con 0x7f186815ab60 2023-10-06T12:52:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.532+0000 7f184effd700 1 -- 172.21.15.46:0/1297559771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3029) v1 ==== 163+0+0 (secure 0 0 0) 0x7f186816cd10 con 0x7f186815ab60 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 msgr2=0x7f1854062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 0x7f1854062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18680af520 tx=0x7f1858009040 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 msgr2=0x7f18680c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f186000c930 tx=0x7f18600026e0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 shutdown_connections 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f186815ab60 0x7f18680c0db0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1854060670 0x7f1854062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f18680ad6c0 0x7f18680b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 --2- 172.21.15.46:0/1297559771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f186806e8a0 0x7f18680b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 >> 172.21.15.46:0/1297559771 conn(0x7f18680cbe30 msgr2=0x7f18680c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.536+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 shutdown_connections 2023-10-06T12:52:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.540+0000 7f186d02e700 1 -- 172.21.15.46:0/1297559771 wait complete. 2023-10-06T12:52:14.564 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-06T12:52:14.564 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-06T12:52:14.564 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T12:52:14.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T12:52:14.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 Processor -- start 2023-10-06T12:52:14.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 -- start start 2023-10-06T12:52:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 0x7fd69c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c06d460 con 0x7fd69c0ad6c0 2023-10-06T12:52:14.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c0c7000 con 0x7fd69c15af30 2023-10-06T12:52:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd6a0f45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0c72c0 con 0x7fd69c15ab60 2023-10-06T12:52:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd69a59c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 0x7fd69c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd69ad9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd69c0ad6c0 0x7fd69c0adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42718/0 (socket says 172.21.15.46:42718) 2023-10-06T12:52:14.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.728+0000 7fd69ad9d700 1 -- 172.21.15.46:0/110876534 learned_addr learned my addr 172.21.15.46:0/110876534 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69a59c700 1 -- 172.21.15.46:0/110876534 >> v1:172.21.15.132:6789/0 conn(0x7fd69c15af30 legacy=0x7fd69c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69a59c700 1 -- 172.21.15.46:0/110876534 >> v1:172.21.15.46:6789/0 conn(0x7fd69c0ad6c0 legacy=0x7fd69c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69a59c700 1 -- 172.21.15.46:0/110876534 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0c75a0 con 0x7fd69c15ab60 2023-10-06T12:52:14.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69a59c700 1 --2- 172.21.15.46:0/110876534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 0x7fd69c06e8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009900 tx=0x7fd68c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec751ad846c251f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69959a700 1 -- 172.21.15.46:0/110876534 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c009ea0 con 0x7fd69c15ab60 2023-10-06T12:52:14.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69959a700 1 -- 172.21.15.46:0/110876534 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c005c10 con 0x7fd69c15ab60 2023-10-06T12:52:14.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd69959a700 1 -- 172.21.15.46:0/110876534 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00b610 con 0x7fd69c15ab60 2023-10-06T12:52:14.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 -- 172.21.15.46:0/110876534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 msgr2=0x7fd69c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/110876534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 0x7fd69c06e8a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009900 tx=0x7fd68c009120 comp rx=0 tx=0).stop 2023-10-06T12:52:14.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 -- 172.21.15.46:0/110876534 shutdown_connections 2023-10-06T12:52:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/110876534 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c15ab60 0x7fd69c06e8a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 -- 172.21.15.46:0/110876534 >> 172.21.15.46:0/110876534 conn(0x7fd69c0cbe30 msgr2=0x7fd69c0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 -- 172.21.15.46:0/110876534 shutdown_connections 2023-10-06T12:52:14.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.732+0000 7fd6a0f45700 1 -- 172.21.15.46:0/110876534 wait complete. 2023-10-06T12:52:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 Processor -- start 2023-10-06T12:52:14.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 -- start start 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 0x7fd69c150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 0x7fd69c0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 0x7fd69c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0af590 con 0x7fd69c15af30 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c150fd0 con 0x7fd69c15ab60 2023-10-06T12:52:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd6a0f45700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c169a00 con 0x7fd69c0ad6c0 2023-10-06T12:52:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd69ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 0x7fd69c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd69ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 0x7fd69c0ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50064/0 (socket says 172.21.15.46:50064) 2023-10-06T12:52:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd69ad9d700 1 -- 172.21.15.46:0/899606554 learned_addr learned my addr 172.21.15.46:0/899606554 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd69a59c700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 0x7fd69c150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 0x7fd69c0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 -- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 msgr2=0x7fd69c150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 0x7fd69c150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 -- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 msgr2=0x7fd69c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 0x7fd69c0ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd699d9b700 1 -- 172.21.15.46:0/899606554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c0093c0 con 0x7fd69c15ab60 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.736+0000 7fd69a59c700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 0x7fd69c150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd699d9b700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 0x7fd69c0ca2b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd69000c5e0 tx=0x7fd690007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690016070 con 0x7fd69c15ab60 2023-10-06T12:52:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd690006820 con 0x7fd69c15ab60 2023-10-06T12:52:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690006c00 con 0x7fd69c15ab60 2023-10-06T12:52:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0c3450 con 0x7fd69c15ab60 2023-10-06T12:52:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c0cb320 con 0x7fd69c15ab60 2023-10-06T12:52:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.740+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c0af700 con 0x7fd69c15ab60 2023-10-06T12:52:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.744+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd690007000 con 0x7fd69c15ab60 2023-10-06T12:52:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.744+0000 7fd68a7fc700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 0x7fd67c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.744+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fd69007c450 con 0x7fd69c15ab60 2023-10-06T12:52:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.744+0000 7fd69a59c700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 0x7fd67c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.744+0000 7fd69a59c700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 0x7fd67c062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd68c003080 tx=0x7fd68c015000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.752+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd69c0af700 con 0x7fd69c15ab60 2023-10-06T12:52:14.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:14.988+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fd69c0af6d0 con 0x7fd69c15ab60 2023-10-06T12:52:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.539+0000 7fd68a7fc700 1 -- 172.21.15.46:0/899606554 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3030) v1 ==== 158+0+0 (secure 0 0 0) 0x7fd69004c540 con 0x7fd69c15ab60 2023-10-06T12:52:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 msgr2=0x7fd67c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 0x7fd67c062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd68c003080 tx=0x7fd68c015000 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 msgr2=0x7fd69c0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 0x7fd69c0ca2b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd69000c5e0 tx=0x7fd690007ec0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 shutdown_connections 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd69c15af30 0x7fd69c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd67c060670 0x7fd67c062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd69c15ab60 0x7fd69c0ca2b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 --2- 172.21.15.46:0/899606554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd69c0ad6c0 0x7fd69c150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 >> 172.21.15.46:0/899606554 conn(0x7fd69c0cbe30 msgr2=0x7fd69c0c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 shutdown_connections 2023-10-06T12:52:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.543+0000 7fd6a0f45700 1 -- 172.21.15.46:0/899606554 wait complete. 2023-10-06T12:52:15.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T12:52:15.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.743+0000 7f6aeacf0700 1 Processor -- start 2023-10-06T12:52:15.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.743+0000 7f6aeacf0700 1 -- start start 2023-10-06T12:52:15.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6aeacf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ae4159d90 0x7f6ae415a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6aeacf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6aeacf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae40c6470 con 0x7f6ae4153550 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6aeacf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae40c1110 con 0x7f6ae4159d90 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6aeacf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c6750 con 0x7f6ae406e290 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae8a8c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ae4159d90 0x7f6ae415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae3fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae928d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f6ae4153550 0x7f6ae41539b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:46966/0 (socket says 172.21.15.46:46966) 2023-10-06T12:52:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae928d700 1 -- 172.21.15.46:0/1110743362 learned_addr learned my addr 172.21.15.46:0/1110743362 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:15.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae8a8c700 1 -- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 msgr2=0x7f6ae40c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:15.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae8a8c700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae8a8c700 1 -- 172.21.15.46:0/1110743362 >> v1:172.21.15.132:6789/0 conn(0x7f6ae4153550 legacy=0x7f6ae41539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae8a8c700 1 -- 172.21.15.46:0/1110743362 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae415a9c0 con 0x7f6ae4159d90 2023-10-06T12:52:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae3fff700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.747+0000 7f6ae37fe700 1 -- 172.21.15.46:0/1110743362 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771386168 0 0) 0x7f6ae40c6470 con 0x7f6ae4153550 2023-10-06T12:52:15.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6ae8a8c700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae415a160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098e0 tx=0x7f6ad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e50b10a8e05508 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:15.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6ae37fe700 1 -- 172.21.15.46:0/1110743362 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800b640 con 0x7f6ae4159d90 2023-10-06T12:52:15.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6ae37fe700 1 -- 172.21.15.46:0/1110743362 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad800b7a0 con 0x7f6ae4159d90 2023-10-06T12:52:15.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6ae37fe700 1 -- 172.21.15.46:0/1110743362 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800bb40 con 0x7f6ae4159d90 2023-10-06T12:52:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 -- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 msgr2=0x7f6ae415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae415a160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80098e0 tx=0x7f6ad80091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 -- 172.21.15.46:0/1110743362 shutdown_connections 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/1110743362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae415a160 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 -- 172.21.15.46:0/1110743362 >> 172.21.15.46:0/1110743362 conn(0x7f6ae40cbe30 msgr2=0x7f6ae40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 -- 172.21.15.46:0/1110743362 shutdown_connections 2023-10-06T12:52:15.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.751+0000 7f6aeacf0700 1 -- 172.21.15.46:0/1110743362 wait complete. 2023-10-06T12:52:15.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 Processor -- start 2023-10-06T12:52:15.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 -- start start 2023-10-06T12:52:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae40ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40ae3d0 con 0x7f6ae4153550 2023-10-06T12:52:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40ada00 con 0x7f6ae4159d90 2023-10-06T12:52:15.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.755+0000 7f6aeacf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40acb50 con 0x7f6ae406e290 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae8a8c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50072/0 (socket says 172.21.15.46:50072) 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae928d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae40ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 -- 172.21.15.46:0/781579273 learned_addr learned my addr 172.21.15.46:0/781579273 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 -- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 msgr2=0x7f6ae40afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 -- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 msgr2=0x7f6ae40ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae40ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 -- 172.21.15.46:0/781579273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0009420 con 0x7f6ae4153550 2023-10-06T12:52:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae928d700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae40ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:15.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae8a8c700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae3fff700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000ec80 tx=0x7f6ad00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000c920 con 0x7f6ae4153550 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad000ca80 con 0x7f6ae4153550 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad8009420 con 0x7f6ae4153550 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000ce00 con 0x7f6ae4153550 2023-10-06T12:52:15.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.759+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae40ad150 con 0x7f6ae4153550 2023-10-06T12:52:15.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.763+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae40adb90 con 0x7f6ae4153550 2023-10-06T12:52:15.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.763+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6ad0013680 con 0x7f6ae4153550 2023-10-06T12:52:15.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.767+0000 7f6ae0ff9700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 0x7f6ac8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:15.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.767+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f6ad007d060 con 0x7f6ae4153550 2023-10-06T12:52:15.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.767+0000 7f6ae8a8c700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 0x7f6ac8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:15.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.767+0000 7f6ae8a8c700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 0x7f6ac8062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80093f0 tx=0x7f6ad8017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:15.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.775+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6ad00acbe0 con 0x7f6ae4153550 2023-10-06T12:52:15.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.947+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6ae40660f0 con 0x7f6ae4153550 2023-10-06T12:52:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6ae0ff9700 1 -- 172.21.15.46:0/781579273 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ad004d150 con 0x7f6ae4153550 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 msgr2=0x7f6ac8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 0x7f6ac8062b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80093f0 tx=0x7f6ad8017000 comp rx=0 tx=0).stop 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 msgr2=0x7f6ae40ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000ec80 tx=0x7f6ad00091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 shutdown_connections 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6ac80606a0 0x7f6ac8062b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ae4159d90 0x7f6ae40ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae4153550 0x7f6ae40ade90 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 --2- 172.21.15.46:0/781579273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ae406e290 0x7f6ae40afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 >> 172.21.15.46:0/781579273 conn(0x7f6ae40cbe30 msgr2=0x7f6ae4152970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 shutdown_connections 2023-10-06T12:52:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:15.959+0000 7f6aeacf0700 1 -- 172.21.15.46:0/781579273 wait complete. 2023-10-06T12:52:15.986 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:52:15.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:16.027 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:16.070 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:52:16.070 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T12:52:16.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.231+0000 7fb406e6b700 1 Processor -- start 2023-10-06T12:52:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb406e6b700 1 -- start start 2023-10-06T12:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb406e6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 0x7fb4001683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb406e6b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb400069660 con 0x7fb4000bedd0 2023-10-06T12:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb406e6b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb400068ac0 con 0x7fb400067760 2023-10-06T12:52:16.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb406e6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb400068d30 con 0x7fb4000bf1a0 2023-10-06T12:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb405e69700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4000bedd0 0x7fb400067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42726/0 (socket says 172.21.15.46:42726) 2023-10-06T12:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb405e69700 1 -- 172.21.15.46:0/253051117 learned_addr learned my addr 172.21.15.46:0/253051117 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:16.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.235+0000 7fb40666a700 1 --2- 172.21.15.46:0/253051117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 0x7fb4001683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292036548 0 0) 0x7fb400068ac0 con 0x7fb400067760 2023-10-06T12:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ec003570 con 0x7fb400067760 2023-10-06T12:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651801636 0 0) 0x7fb400069660 con 0x7fb4000bedd0 2023-10-06T12:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb400068ac0 con 0x7fb4000bedd0 2023-10-06T12:52:16.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb40666a700 1 -- 172.21.15.46:0/253051117 >> v1:172.21.15.132:6789/0 conn(0x7fb400067760 legacy=0x7fb4000c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb40666a700 1 -- 172.21.15.46:0/253051117 >> v1:172.21.15.46:6789/0 conn(0x7fb4000bedd0 legacy=0x7fb400067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb40666a700 1 -- 172.21.15.46:0/253051117 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400069010 con 0x7fb4000bf1a0 2023-10-06T12:52:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb40666a700 1 --2- 172.21.15.46:0/253051117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 0x7fb4001683d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e90 tx=0x7fb3f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9e4a10d9ab90c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800e070 con 0x7fb4000bf1a0 2023-10-06T12:52:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f800bb80 con 0x7fb4000bf1a0 2023-10-06T12:52:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb404e67700 1 -- 172.21.15.46:0/253051117 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8012430 con 0x7fb4000bf1a0 2023-10-06T12:52:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 -- 172.21.15.46:0/253051117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 msgr2=0x7fb4001683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 --2- 172.21.15.46:0/253051117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 0x7fb4001683d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009e90 tx=0x7fb3f8009780 comp rx=0 tx=0).stop 2023-10-06T12:52:16.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 -- 172.21.15.46:0/253051117 shutdown_connections 2023-10-06T12:52:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 --2- 172.21.15.46:0/253051117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4000bf1a0 0x7fb4001683d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 -- 172.21.15.46:0/253051117 >> 172.21.15.46:0/253051117 conn(0x7fb400062ef0 msgr2=0x7fb4000cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 -- 172.21.15.46:0/253051117 shutdown_connections 2023-10-06T12:52:16.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.239+0000 7fb406e6b700 1 -- 172.21.15.46:0/253051117 wait complete. 2023-10-06T12:52:16.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 Processor -- start 2023-10-06T12:52:16.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 -- start start 2023-10-06T12:52:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb400067760 0x7fb40015db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4000bf1a0 0x7fb400150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015f430 con 0x7fb4000bedd0 2023-10-06T12:52:16.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015e390 con 0x7fb4000bf1a0 2023-10-06T12:52:16.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb406e6b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015e670 con 0x7fb400067760 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50078/0 (socket says 172.21.15.46:50078) 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 -- 172.21.15.46:0/459826806 learned_addr learned my addr 172.21.15.46:0/459826806 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb40666a700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4000bf1a0 0x7fb400150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405e69700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb400067760 0x7fb40015db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 -- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb400067760 msgr2=0x7fb40015db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb400067760 0x7fb40015db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 -- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4000bf1a0 msgr2=0x7fb400150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4000bf1a0 0x7fb400150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.243+0000 7fb405668700 1 -- 172.21.15.46:0/459826806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8009a20 con 0x7fb4000bedd0 2023-10-06T12:52:16.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb405668700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c790 tx=0x7fb3f0007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0010510 con 0x7fb4000bedd0 2023-10-06T12:52:16.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0010670 con 0x7fb4000bedd0 2023-10-06T12:52:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40015e950 con 0x7fb4000bedd0 2023-10-06T12:52:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00109f0 con 0x7fb4000bedd0 2023-10-06T12:52:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40015ef10 con 0x7fb4000bedd0 2023-10-06T12:52:16.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.247+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40015f130 con 0x7fb4000bedd0 2023-10-06T12:52:16.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.251+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb3f001b400 con 0x7fb4000bedd0 2023-10-06T12:52:16.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.251+0000 7fb3f5ffb700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 0x7fb3e4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.251+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fb3f007c9b0 con 0x7fb4000bedd0 2023-10-06T12:52:16.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.251+0000 7fb405e69700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 0x7fb3e4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.251+0000 7fb405e69700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 0x7fb3e4062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0058d0 tx=0x7fb3fc009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.259+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb3f004cb40 con 0x7fb4000bedd0 2023-10-06T12:52:16.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.467+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb400064590 con 0x7fb4000bedd0 2023-10-06T12:52:16.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.467+0000 7fb3f5ffb700 1 -- 172.21.15.46:0/459826806 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3530 (secure 0 0 0) 0x7fb3f0050160 con 0x7fb4000bedd0 2023-10-06T12:52:16.472 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:16.472 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8267,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2918726259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2918726259},{"type":"v1","addr":"172.21.15.46:6837","nonce":2918726259}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/652735998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":652735998},{"type":"v1","addr":"172.21.15.132:6835","nonce":652735998}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/353353386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":353353386},{"type":"v1","addr":"172.21.15.132:6833","nonce":353353386}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":10281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3503217535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3503217535},{"type":"v1","addr":"172.21.15.46:6835","nonce":3503217535}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[]} 2023-10-06T12:52:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 msgr2=0x7fb3e4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 0x7fb3e4062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0058d0 tx=0x7fb3fc009040 comp rx=0 tx=0).stop 2023-10-06T12:52:16.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 msgr2=0x7fb4000bca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c790 tx=0x7fb3f0007e30 comp rx=0 tx=0).stop 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 shutdown_connections 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb3e4060670 0x7fb3e4062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4000bf1a0 0x7fb400150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4000bedd0 0x7fb4000bca30 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 --2- 172.21.15.46:0/459826806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb400067760 0x7fb40015db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 >> 172.21.15.46:0/459826806 conn(0x7fb400062ef0 msgr2=0x7fb40015d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 shutdown_connections 2023-10-06T12:52:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.471+0000 7fb406e6b700 1 -- 172.21.15.46:0/459826806 wait complete. 2023-10-06T12:52:16.478 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 96 2023-10-06T12:52:16.495 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T12:52:16.496 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T12:52:16.496 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T12:52:16.496 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:52:16.496 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T12:52:16.496 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T12:52:16.496 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T12:52:16.496 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:52:16.497 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T12:52:16.497 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T12:52:16.497 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T12:52:16.497 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:52:16.497 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T12:52:16.497 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T12:52:16.498 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T12:52:16.498 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:52:16.514 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T12:52:16.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T12:52:16.517 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T12:52:16.517 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T12:52:16.518 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T12:52:16.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T12:52:16.520 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T12:52:16.520 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T12:52:16.569 INFO:tasks.ceph.mds.b:Started 2023-10-06T12:52:16.569 DEBUG:teuthology.parallel:result is None 2023-10-06T12:52:16.571 INFO:tasks.ceph.mds.c:Started 2023-10-06T12:52:16.572 DEBUG:teuthology.parallel:result is None 2023-10-06T12:52:16.576 INFO:tasks.ceph.mds.d:Started 2023-10-06T12:52:16.576 INFO:tasks.ceph.mds.a:Started 2023-10-06T12:52:16.576 DEBUG:teuthology.parallel:result is None 2023-10-06T12:52:16.576 DEBUG:teuthology.parallel:result is None 2023-10-06T12:52:16.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T12:52:16.673 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:52:16.678 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2023-10-06T12:52:16.678 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2023-10-06T12:52:16.683 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2023-10-06T12:52:16.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.727+0000 7f45fb40c700 1 Processor -- start 2023-10-06T12:52:16.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.727+0000 7f45fb40c700 1 -- start start 2023-10-06T12:52:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fb40c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45f40af000 0x7f45f40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fb40c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f40690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fb40c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40cf100 con 0x7f45f40cb4c0 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fb40c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f40ce300 con 0x7f45f40af000 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fb40c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40ce4f0 con 0x7f45f40af3d0 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45f40af000 0x7f45f40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45f40af000 0x7f45f40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50104/0 (socket says 172.21.15.46:50104) 2023-10-06T12:52:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 -- 172.21.15.46:0/2840253334 learned_addr learned my addr 172.21.15.46:0/2840253334 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fac0b700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f40690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 -- 172.21.15.46:0/2840253334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 msgr2=0x7f45f40690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f40690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 -- 172.21.15.46:0/2840253334 >> v1:172.21.15.132:6789/0 conn(0x7f45f40cb4c0 legacy=0x7f45f40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fa40a700 1 -- 172.21.15.46:0/2840253334 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40ce7d0 con 0x7f45f40af000 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.731+0000 7f45fac0b700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f40690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fa40a700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40af000 0x7f45f40cf590 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f45f00099b0 tx=0x7f45f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee254d0ded73e96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45f9408700 1 -- 172.21.15.46:0/2840253334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000b860 con 0x7f45f40af000 2023-10-06T12:52:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45f9408700 1 -- 172.21.15.46:0/2840253334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000b9c0 con 0x7f45f40af000 2023-10-06T12:52:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45f9408700 1 -- 172.21.15.46:0/2840253334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000bd60 con 0x7f45f40af000 2023-10-06T12:52:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 -- 172.21.15.46:0/2840253334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40af000 msgr2=0x7f45f40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40af000 0x7f45f40cf590 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f45f00099b0 tx=0x7f45f0009250 comp rx=0 tx=0).stop 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 -- 172.21.15.46:0/2840253334 shutdown_connections 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f40690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 --2- 172.21.15.46:0/2840253334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40af000 0x7f45f40cf590 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 -- 172.21.15.46:0/2840253334 >> 172.21.15.46:0/2840253334 conn(0x7f45f40683b0 msgr2=0x7f45f40d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 -- 172.21.15.46:0/2840253334 shutdown_connections 2023-10-06T12:52:16.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.735+0000 7f45fb40c700 1 -- 172.21.15.46:0/2840253334 wait complete. 2023-10-06T12:52:16.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 Processor -- start 2023-10-06T12:52:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- start start 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 0x7f45f40c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f4171b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4172060 con 0x7f45f40cb4c0 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40c87e0 con 0x7f45f40af000 2023-10-06T12:52:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415ae30 con 0x7f45f40af3d0 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50106/0 (socket says 172.21.15.46:50106) 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 -- 172.21.15.46:0/4287524341 learned_addr learned my addr 172.21.15.46:0/4287524341 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fa40a700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 0x7f45f40c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45f9c09700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f4171b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 -- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 msgr2=0x7f45f4171b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f4171b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 -- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 msgr2=0x7f45f40c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 0x7f45f40c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 -- 172.21.15.46:0/4287524341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00094f0 con 0x7f45f40cb4c0 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fa40a700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 0x7f45f40c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fac0b700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00a380 tx=0x7f45ec007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00ae10 con 0x7f45f40cb4c0 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec00d650 con 0x7f45f40cb4c0 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f416daf0 con 0x7f45f40cb4c0 2023-10-06T12:52:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00da10 con 0x7f45f40cb4c0 2023-10-06T12:52:16.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.739+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f40c8500 con 0x7f45f40cb4c0 2023-10-06T12:52:16.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.743+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f4064590 con 0x7f45f40cb4c0 2023-10-06T12:52:16.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.743+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f45ec01f440 con 0x7f45f40cb4c0 2023-10-06T12:52:16.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.743+0000 7f45ea7fc700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 0x7f45d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.743+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f45ec07d6a0 con 0x7f45f40cb4c0 2023-10-06T12:52:16.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.743+0000 7f45fa40a700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 0x7f45d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:16.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.747+0000 7f45fa40a700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 0x7f45d8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45f00099b0 tx=0x7f45f0017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:16.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.751+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f45ec04d790 con 0x7f45f40cb4c0 2023-10-06T12:52:16.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:16.935+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45f415afa0 con 0x7f45f40cb4c0 2023-10-06T12:52:17.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.559+0000 7f45ea7fc700 1 -- 172.21.15.46:0/4287524341 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v191) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45ec050db0 con 0x7f45f40cb4c0 2023-10-06T12:52:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 msgr2=0x7f45d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 0x7f45d8062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45f00099b0 tx=0x7f45f0017000 comp rx=0 tx=0).stop 2023-10-06T12:52:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 msgr2=0x7f45f41500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00a380 tx=0x7f45ec007900 comp rx=0 tx=0).stop 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 shutdown_connections 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40cb4c0 0x7f45f41500f0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45f40af3d0 0x7f45f4171b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f45d8060670 0x7f45d8062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 --2- 172.21.15.46:0/4287524341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45f40af000 0x7f45f40c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 >> 172.21.15.46:0/4287524341 conn(0x7f45f40683b0 msgr2=0x7f45f4158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:17.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 shutdown_connections 2023-10-06T12:52:17.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.563+0000 7f45fb40c700 1 -- 172.21.15.46:0/4287524341 wait complete. 2023-10-06T12:52:17.569 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-10-06T12:52:17.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T12:52:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 Processor -- start 2023-10-06T12:52:17.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 -- start start 2023-10-06T12:52:17.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b20156c80 0x7f4b200ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200c8500 con 0x7f4b2015cde0 2023-10-06T12:52:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200c6470 con 0x7f4b20156c80 2023-10-06T12:52:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.743+0000 7f4b262f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200c7760 con 0x7f4b200c7b50 2023-10-06T12:52:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b20156c80 0x7f4b200ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b17fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b17fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:34878/0 (socket says 172.21.15.46:34878) 2023-10-06T12:52:17.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b17fff700 1 -- 172.21.15.46:0/1897540209 learned_addr learned my addr 172.21.15.46:0/1897540209 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:17.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b2488d700 1 --1- 172.21.15.46:0/1897540209 >> v1:172.21.15.46:6789/0 conn(0x7f4b2015cde0 0x7f4b2015d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42732/0 (socket says 172.21.15.46:42732) 2023-10-06T12:52:17.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1f7fe700 1 -- 172.21.15.46:0/1897540209 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657089347 0 0) 0x7f4b200c8500 con 0x7f4b2015cde0 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1f7fe700 1 -- 172.21.15.46:0/1897540209 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c003570 con 0x7f4b2015cde0 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 -- 172.21.15.46:0/1897540209 >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 msgr2=0x7f4b200c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 --2- 172.21.15.46:0/1897540209 >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 -- 172.21.15.46:0/1897540209 >> v1:172.21.15.46:6789/0 conn(0x7f4b2015cde0 legacy=0x7f4b2015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 -- 172.21.15.46:0/1897540209 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200ca710 con 0x7f4b20156c80 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b17fff700 1 --2- 172.21.15.46:0/1897540209 >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:17.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1ffff700 1 --2- 172.21.15.46:0/1897540209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b200ca1d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4b2006bac0 tx=0x7f4b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf0ab3c1b3287d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:17.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1f7fe700 1 -- 172.21.15.46:0/1897540209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10011630 con 0x7f4b20156c80 2023-10-06T12:52:17.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1f7fe700 1 -- 172.21.15.46:0/1897540209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b10011790 con 0x7f4b20156c80 2023-10-06T12:52:17.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b1f7fe700 1 -- 172.21.15.46:0/1897540209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10011b10 con 0x7f4b20156c80 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 -- 172.21.15.46:0/1897540209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 msgr2=0x7f4b200ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 --2- 172.21.15.46:0/1897540209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b200ca1d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4b2006bac0 tx=0x7f4b100091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 -- 172.21.15.46:0/1897540209 shutdown_connections 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 --2- 172.21.15.46:0/1897540209 >> v2:172.21.15.132:3300/0 conn(0x7f4b200c7b50 0x7f4b200c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 --2- 172.21.15.46:0/1897540209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b200ca1d0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 -- 172.21.15.46:0/1897540209 >> 172.21.15.46:0/1897540209 conn(0x7f4b200cbe30 msgr2=0x7f4b200cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.747+0000 7f4b262f0700 1 -- 172.21.15.46:0/1897540209 shutdown_connections 2023-10-06T12:52:17.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.751+0000 7f4b262f0700 1 -- 172.21.15.46:0/1897540209 wait complete. 2023-10-06T12:52:17.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.751+0000 7f4b262f0700 1 Processor -- start 2023-10-06T12:52:17.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.751+0000 7f4b262f0700 1 -- start start 2023-10-06T12:52:17.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200c7b50 0x7f4b200aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2015cde0 0x7f4b20150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200af110 con 0x7f4b20156c80 2023-10-06T12:52:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200c46a0 con 0x7f4b2015cde0 2023-10-06T12:52:17.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2016d180 con 0x7f4b200c7b50 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50116/0 (socket says 172.21.15.46:50116) 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 -- 172.21.15.46:0/3147875502 learned_addr learned my addr 172.21.15.46:0/3147875502 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b1ffff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200c7b50 0x7f4b200aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b2488d700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2015cde0 0x7f4b20150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 -- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200c7b50 msgr2=0x7f4b200aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200c7b50 0x7f4b200aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 -- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2015cde0 msgr2=0x7f4b20150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2015cde0 0x7f4b20150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 -- 172.21.15.46:0/3147875502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080094f0 con 0x7f4b20156c80 2023-10-06T12:52:17.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b17fff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800ec50 tx=0x7f4b08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08009730 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0800c9d0 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b100093c0 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800cd50 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.755+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b200c4900 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.759+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20063f90 con 0x7f4b20156c80 2023-10-06T12:52:17.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.759+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4b08013680 con 0x7f4b20156c80 2023-10-06T12:52:17.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.759+0000 7f4b1cff9700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 0x7f4b00062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:17.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.763+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f4b0807cfe0 con 0x7f4b20156c80 2023-10-06T12:52:17.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.763+0000 7f4b1ffff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 0x7f4b00062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.763+0000 7f4b1ffff700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 0x7f4b00062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009390 tx=0x7f4b10017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:17.767+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b0804d430 con 0x7f4b20156c80 2023-10-06T12:52:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.007+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4b2004b5c0 con 0x7f4b20156c80 2023-10-06T12:52:18.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.007+0000 7f4b1cff9700 1 -- 172.21.15.46:0/3147875502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v49) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4b08050a50 con 0x7f4b20156c80 2023-10-06T12:52:18.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lXkf5IxAAQOQtmYeRZFHF1BGTDn6yPQ==", 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.963 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lgblLNhAA+EO+H+fKGo3ViDStVbWY/g==", 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.964 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.965 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2023-10-06T12:52:18.965 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lxDpjLhAAPtQV37GmCPaSo8Z8D7tRnw==", 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.966 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9leFnwBBAAJuhhx9WFhX8NdlK/ONIkng==", 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.967 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC/7B9lGU5qMxAAuMMwA23pnvkH1pd14TMZMQ==", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDD7B9l+DHbAxAAtUKooT6RG5VdsFlrsCgYlw==", 2023-10-06T12:52:18.968 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDG7B9lKemkEBAAjfwmzzNRQvUZeuivcO/9iw==", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.969 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDJ7B9lXq5dHBAAfdqe1KGWhYPxsmXqWlVrDA==", 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.970 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDP7B9lupTULhAAG4+bqZ/TxEyvHBHQGORZUw==", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDS7B9ljCA3OxAAXg5aXAnglt3hq5+R6jcizQ==", 2023-10-06T12:52:18.971 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDW7B9lsdvwChAAGkKwyg9yTW6T/y8JFzY3Ag==", 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.972 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQDZ7B9lBabfGBAAqp64lRRI/CCWuONjbKiBaQ==", 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lGHMoCxAAS2SiNUGBzkiqdYnLmKSSkg==", 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2023-10-06T12:52:18.974 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9ll83HFhAAwSeNtvTxB/4LJyp2f1qFQQ==", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lnJfoEBAAXL2PVkd84NicQ5lZgWxLMQ==", 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC+7B9lYfNmHBAAVtNC3atKmrm1hlJsIY66Fw==", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-10-06T12:52:18.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC87B9lKqGANxAArk8jJqomcYEc55VBiifd2Q==", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9lL34UHhAAOzwumXPO6Deh4BKF4+26dQ==", 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:18.978 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC97B9l3sT0FxAA/eqJJOgFr2cZZovNI971Jw==", 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-10-06T12:52:18.979 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:52:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 msgr2=0x7f4b00062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 0x7f4b00062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009390 tx=0x7f4b10017000 comp rx=0 tx=0).stop 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 msgr2=0x7f4b2014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800ec50 tx=0x7f4b08009250 comp rx=0 tx=0).stop 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 shutdown_connections 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b00060670 0x7f4b00062b20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2015cde0 0x7f4b20150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b20156c80 0x7f4b2014fd30 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 --2- 172.21.15.46:0/3147875502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b200c7b50 0x7f4b200aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 >> 172.21.15.46:0/3147875502 conn(0x7f4b200cbe30 msgr2=0x7f4b200aaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 shutdown_connections 2023-10-06T12:52:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:18.011+0000 7f4b262f0700 1 -- 172.21.15.46:0/3147875502 wait complete. 2023-10-06T12:52:20.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.639+0000 7f0646ee3700 1 Processor -- start 2023-10-06T12:52:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.639+0000 7f0646ee3700 1 -- start start 2023-10-06T12:52:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0646ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f0640067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0646ee3700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064015d6c0 con 0x7f0640067590 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0646ee3700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064015cd90 con 0x7f06401591a0 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0646ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06400699c0 con 0x7f0640158dd0 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f06466e2700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0640067590 0x7f064006c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42740/0 (socket says 172.21.15.46:42740) 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f06466e2700 1 -- 172.21.15.46:0/624798509 learned_addr learned my addr 172.21.15.46:0/624798509 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0645ee1700 1 --2- 172.21.15.46:0/624798509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f0640067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292562935 0 0) 0x7f064015d6c0 con 0x7f0640067590 2023-10-06T12:52:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f062c003570 con 0x7f0640067590 2023-10-06T12:52:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740794346 0 0) 0x7f064015cd90 con 0x7f06401591a0 2023-10-06T12:52:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f064015d6c0 con 0x7f06401591a0 2023-10-06T12:52:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0645ee1700 1 -- 172.21.15.46:0/624798509 >> v1:172.21.15.132:6789/0 conn(0x7f06401591a0 legacy=0x7f0640068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0645ee1700 1 -- 172.21.15.46:0/624798509 >> v1:172.21.15.46:6789/0 conn(0x7f0640067590 legacy=0x7f064006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.643+0000 7f0645ee1700 1 -- 172.21.15.46:0/624798509 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064015d070 con 0x7f0640158dd0 2023-10-06T12:52:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0645ee1700 1 --2- 172.21.15.46:0/624798509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f0640067050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f063c009e90 tx=0x7f063c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967865bc9adf43e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00e070 con 0x7f0640158dd0 2023-10-06T12:52:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c00bb80 con 0x7f0640158dd0 2023-10-06T12:52:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0644edf700 1 -- 172.21.15.46:0/624798509 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c012430 con 0x7f0640158dd0 2023-10-06T12:52:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 -- 172.21.15.46:0/624798509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 msgr2=0x7f0640067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 --2- 172.21.15.46:0/624798509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f0640067050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f063c009e90 tx=0x7f063c009780 comp rx=0 tx=0).stop 2023-10-06T12:52:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 -- 172.21.15.46:0/624798509 shutdown_connections 2023-10-06T12:52:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 --2- 172.21.15.46:0/624798509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f0640067050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 -- 172.21.15.46:0/624798509 >> 172.21.15.46:0/624798509 conn(0x7f0640062ef0 msgr2=0x7f06400cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 -- 172.21.15.46:0/624798509 shutdown_connections 2023-10-06T12:52:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.647+0000 7f0646ee3700 1 -- 172.21.15.46:0/624798509 wait complete. 2023-10-06T12:52:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 Processor -- start 2023-10-06T12:52:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 -- start start 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640067590 0x7f064015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f064014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640158270 con 0x7f06401591a0 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064015a7c0 con 0x7f0640067590 2023-10-06T12:52:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0646ee3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064014d580 con 0x7f0640158dd0 2023-10-06T12:52:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50124/0 (socket says 172.21.15.46:50124) 2023-10-06T12:52:20.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 -- 172.21.15.46:0/833788066 learned_addr learned my addr 172.21.15.46:0/833788066 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06456e0700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f064014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0645ee1700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640067590 0x7f064015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 -- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 msgr2=0x7f064014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f064014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 -- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640067590 msgr2=0x7f064015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640067590 0x7f064015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 -- 172.21.15.46:0/833788066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c009a20 con 0x7f06401591a0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f06466e2700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f063800c790 tx=0x7f0638007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638005c00 con 0x7f06401591a0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.651+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638005d60 con 0x7f06401591a0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0640159bb0 con 0x7f06401591a0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638007320 con 0x7f06401591a0 2023-10-06T12:52:20.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064015a170 con 0x7f06401591a0 2023-10-06T12:52:20.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064015a930 con 0x7f06401591a0 2023-10-06T12:52:20.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0638007690 con 0x7f06401591a0 2023-10-06T12:52:20.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0635ffb700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 0x7f0624062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.655+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f063807bfa0 con 0x7f06401591a0 2023-10-06T12:52:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.659+0000 7f0645ee1700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 0x7f0624062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:20.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.659+0000 7f0645ee1700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 0x7f0624062b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f063c009de0 tx=0x7f063c019000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.663+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f063804c390 con 0x7f06401591a0 2023-10-06T12:52:20.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.839+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0640064590 con 0x7f06401591a0 2023-10-06T12:52:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.839+0000 7f0635ffb700 1 -- 172.21.15.46:0/833788066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+14538 (secure 0 0 0) 0x7f0638017020 con 0x7f06401591a0 2023-10-06T12:52:20.846 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:20.846 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":193,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:17.566867+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 msgr2=0x7f0624062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 0x7f0624062b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f063c009de0 tx=0x7f063c019000 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 msgr2=0x7f06400c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f063800c790 tx=0x7f0638007dd0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 shutdown_connections 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06401591a0 0x7f06400c0a10 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0640158dd0 0x7f064014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06240606a0 0x7f0624062b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 --2- 172.21.15.46:0/833788066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640067590 0x7f064015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 >> 172.21.15.46:0/833788066 conn(0x7f0640062ef0 msgr2=0x7f064006e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 shutdown_connections 2023-10-06T12:52:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:20.847+0000 7f0646ee3700 1 -- 172.21.15.46:0/833788066 wait complete. 2023-10-06T12:52:20.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T12:52:20.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:52:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.035+0000 7f8cd02f0700 1 Processor -- start 2023-10-06T12:52:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.035+0000 7f8cd02f0700 1 -- start start 2023-10-06T12:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.035+0000 7f8cd02f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cc80b2790 0x7f8cc80ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cd02f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8cc80cd080 0x7f8cc80c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cd02f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80cc510 con 0x7f8cc80b23c0 2023-10-06T12:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cd02f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80cd500 con 0x7f8cc80b2790 2023-10-06T12:52:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cd02f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80b2be0 con 0x7f8cc80cd080 2023-10-06T12:52:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cc80b2790 0x7f8cc80ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cce88d700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8cc80cd080 0x7f8cc80c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cc80b2790 0x7f8cc80ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50132/0 (socket says 172.21.15.46:50132) 2023-10-06T12:52:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8cce08c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8cc80b23c0 0x7f8cc80cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:47022/0 (socket says 172.21.15.46:47022) 2023-10-06T12:52:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 -- 172.21.15.46:0/1661573210 learned_addr learned my addr 172.21.15.46:0/1661573210 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 -- 172.21.15.46:0/1661573210 >> v2:172.21.15.132:3300/0 conn(0x7f8cc80cd080 msgr2=0x7f8cc80c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 --2- 172.21.15.46:0/1661573210 >> v2:172.21.15.132:3300/0 conn(0x7f8cc80cd080 0x7f8cc80c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 -- 172.21.15.46:0/1661573210 >> v1:172.21.15.132:6789/0 conn(0x7f8cc80b23c0 legacy=0x7f8cc80cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 -- 172.21.15.46:0/1661573210 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80cb900 con 0x7f8cc80b2790 2023-10-06T12:52:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd88b700 1 --2- 172.21.15.46:0/1661573210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80b2790 0x7f8cc80ccb40 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f8cc81599f0 tx=0x7f8cb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7c2489189767b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd08a700 1 -- 172.21.15.46:0/1661573210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8009e70 con 0x7f8cc80b2790 2023-10-06T12:52:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd08a700 1 -- 172.21.15.46:0/1661573210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800b860 con 0x7f8cc80b2790 2023-10-06T12:52:21.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.039+0000 7f8ccd08a700 1 -- 172.21.15.46:0/1661573210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800bc00 con 0x7f8cc80b2790 2023-10-06T12:52:21.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 -- 172.21.15.46:0/1661573210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80b2790 msgr2=0x7f8cc80ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:21.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/1661573210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80b2790 0x7f8cc80ccb40 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f8cc81599f0 tx=0x7f8cb8009250 comp rx=0 tx=0).stop 2023-10-06T12:52:21.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 -- 172.21.15.46:0/1661573210 shutdown_connections 2023-10-06T12:52:21.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/1661573210 >> v2:172.21.15.132:3300/0 conn(0x7f8cc80cd080 0x7f8cc80c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:21.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/1661573210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80b2790 0x7f8cc80ccb40 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:21.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 -- 172.21.15.46:0/1661573210 >> 172.21.15.46:0/1661573210 conn(0x7f8cc8099480 msgr2=0x7f8cc806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:21.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 -- 172.21.15.46:0/1661573210 shutdown_connections 2023-10-06T12:52:21.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.043+0000 7f8cd02f0700 1 -- 172.21.15.46:0/1661573210 wait complete. 2023-10-06T12:52:21.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 Processor -- start 2023-10-06T12:52:21.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 -- start start 2023-10-06T12:52:21.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cc80b23c0 0x7f8cc816a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 0x7f8cc80b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80b4060 con 0x7f8cc80cd080 2023-10-06T12:52:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80b3080 con 0x7f8cc80b23c0 2023-10-06T12:52:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cd02f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80b5030 con 0x7f8cc80b2790 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50144/0 (socket says 172.21.15.46:50144) 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 -- 172.21.15.46:0/245266459 learned_addr learned my addr 172.21.15.46:0/245266459 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8ccd88b700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 0x7f8cc80b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce08c700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cc80b23c0 0x7f8cc816a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 -- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 msgr2=0x7f8cc80b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 0x7f8cc80b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 -- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cc80b23c0 msgr2=0x7f8cc816a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:21.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cc80b23c0 0x7f8cc816a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:21.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 -- 172.21.15.46:0/245266459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80094f0 con 0x7f8cc80cd080 2023-10-06T12:52:21.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8ccd88b700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 0x7f8cc80b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:21.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.047+0000 7f8cce88d700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009e90 tx=0x7f8cc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000cb70 con 0x7f8cc80cd080 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000f040 con 0x7f8cc80cd080 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80b3320 con 0x7f8cc80cd080 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0014480 con 0x7f8cc80cd080 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc8156400 con 0x7f8cc80cd080 2023-10-06T12:52:21.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc80b3800 con 0x7f8cc80cd080 2023-10-06T12:52:21.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.051+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8cc0014830 con 0x7f8cc80cd080 2023-10-06T12:52:21.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.055+0000 7f8cbe7fc700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 0x7f8cac062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:21.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.055+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f8cc007d490 con 0x7f8cc80cd080 2023-10-06T12:52:21.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.055+0000 7f8cce08c700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 0x7f8cac062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:21.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.055+0000 7f8cce08c700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 0x7f8cac062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4007fd0 tx=0x7f8cc4007f00 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:21.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.063+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8cc004d770 con 0x7f8cc80cd080 2023-10-06T12:52:21.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.235+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cc80660f0 con 0x7f8cc80cd080 2023-10-06T12:52:23.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.235+0000 7f8cbe7fc700 1 -- 172.21.15.46:0/245266459 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cc0050d90 con 0x7f8cc80cd080 2023-10-06T12:52:23.216 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:52:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 msgr2=0x7f8cac062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 0x7f8cac062b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4007fd0 tx=0x7f8cc4007f00 comp rx=0 tx=0).stop 2023-10-06T12:52:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 msgr2=0x7f8cc8155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009e90 tx=0x7f8cc000b040 comp rx=0 tx=0).stop 2023-10-06T12:52:23.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 shutdown_connections 2023-10-06T12:52:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc80cd080 0x7f8cc8155940 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:23.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8cc80b2790 0x7f8cc80b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:23.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8cac060670 0x7f8cac062b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:23.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 --2- 172.21.15.46:0/245266459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cc80b23c0 0x7f8cc816a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:23.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 >> 172.21.15.46:0/245266459 conn(0x7f8cc8099480 msgr2=0x7f8cc8069430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:23.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 shutdown_connections 2023-10-06T12:52:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:21.239+0000 7f8cd02f0700 1 -- 172.21.15.46:0/245266459 wait complete. 2023-10-06T12:52:23.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:52:23.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.391+0000 7f0779785700 1 Processor -- start 2023-10-06T12:52:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.391+0000 7f0779785700 1 -- start start 2023-10-06T12:52:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0779785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07740ca430 0x7f07740c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0779785700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07740c1630 con 0x7f07740c7d80 2023-10-06T12:52:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0779785700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07740c0d00 con 0x7f07740a9840 2023-10-06T12:52:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0779785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07740c0fe0 con 0x7f07740ca430 2023-10-06T12:52:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0772ffd700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f07740c7d80 0x7f07740c6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59368/0 (socket says 172.21.15.46:59368) 2023-10-06T12:52:24.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f0772ffd700 1 -- 172.21.15.46:0/537458529 learned_addr learned my addr 172.21.15.46:0/537458529 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.395+0000 7f076bfff700 1 --2- 172.21.15.46:0/537458529 >> v2:172.21.15.132:3300/0 conn(0x7f07740ca430 0x7f07740c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f07727fc700 1 -- 172.21.15.46:0/537458529 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165167644 0 0) 0x7f07740c1630 con 0x7f07740c7d80 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f07727fc700 1 -- 172.21.15.46:0/537458529 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0760003570 con 0x7f07740c7d80 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f076bfff700 1 -- 172.21.15.46:0/537458529 >> v1:172.21.15.132:6789/0 conn(0x7f07740a9840 legacy=0x7f07740c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f076bfff700 1 -- 172.21.15.46:0/537458529 >> v1:172.21.15.46:6789/0 conn(0x7f07740c7d80 legacy=0x7f07740c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f076bfff700 1 -- 172.21.15.46:0/537458529 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740c12c0 con 0x7f07740ca430 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f076bfff700 1 --2- 172.21.15.46:0/537458529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740ca430 0x7f07740c8150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f075c0098e0 tx=0x7f075c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f33195c8af96cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f07727fc700 1 -- 172.21.15.46:0/537458529 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00b640 con 0x7f07740ca430 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f07727fc700 1 -- 172.21.15.46:0/537458529 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00b7a0 con 0x7f07740ca430 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f07727fc700 1 -- 172.21.15.46:0/537458529 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00bb40 con 0x7f07740ca430 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 -- 172.21.15.46:0/537458529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740ca430 msgr2=0x7f07740c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 --2- 172.21.15.46:0/537458529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740ca430 0x7f07740c8150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f075c0098e0 tx=0x7f075c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 -- 172.21.15.46:0/537458529 shutdown_connections 2023-10-06T12:52:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 --2- 172.21.15.46:0/537458529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740ca430 0x7f07740c8150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 -- 172.21.15.46:0/537458529 >> 172.21.15.46:0/537458529 conn(0x7f07740cbe30 msgr2=0x7f07740cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 -- 172.21.15.46:0/537458529 shutdown_connections 2023-10-06T12:52:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.399+0000 7f0779785700 1 -- 172.21.15.46:0/537458529 wait complete. 2023-10-06T12:52:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.403+0000 7f0779785700 1 Processor -- start 2023-10-06T12:52:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.403+0000 7f0779785700 1 -- start start 2023-10-06T12:52:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.403+0000 7f0779785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 0x7f0774140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07740c7d80 0x7f0774068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774158fd0 con 0x7f07740ca430 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07741577a0 con 0x7f07740a9840 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0774157a80 con 0x7f07740c7d80 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35672/0 (socket says 172.21.15.46:35672) 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 -- 172.21.15.46:0/3550657550 learned_addr learned my addr 172.21.15.46:0/3550657550 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0772ffd700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 0x7f0774140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f076bfff700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07740c7d80 0x7f0774068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 -- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07740c7d80 msgr2=0x7f0774068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07740c7d80 0x7f0774068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 -- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 msgr2=0x7f0774140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 0x7f0774140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 -- 172.21.15.46:0/3550657550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009420 con 0x7f07740ca430 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0772ffd700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 0x7f0774140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f07737fe700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c8e0 tx=0x7f076c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00ce80 con 0x7f07740ca430 2023-10-06T12:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c00a7e0 con 0x7f07740ca430 2023-10-06T12:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0774157d60 con 0x7f07740ca430 2023-10-06T12:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c00ab80 con 0x7f07740ca430 2023-10-06T12:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.407+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0774154eb0 con 0x7f07740ca430 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.411+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0774158d90 con 0x7f07740ca430 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.411+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f076c012420 con 0x7f07740ca430 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.411+0000 7f076affd700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 0x7f0754062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.411+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f076c07cca0 con 0x7f07740ca430 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.411+0000 7f0772ffd700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 0x7f0754062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.415+0000 7f0772ffd700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 0x7f0754062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07640058d0 tx=0x7f0764009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.419+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f076c04d130 con 0x7f07740ca430 2023-10-06T12:52:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.587+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07740660f0 con 0x7f07740ca430 2023-10-06T12:52:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.587+0000 7f076affd700 1 -- 172.21.15.46:0/3550657550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v193) v1 ==== 92+0+10 (secure 0 0 0) 0x7f076c050750 con 0x7f07740ca430 2023-10-06T12:52:24.376 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:52:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 msgr2=0x7f0754062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 0x7f0754062b20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07640058d0 tx=0x7f0764009040 comp rx=0 tx=0).stop 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 msgr2=0x7f0774069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c8e0 tx=0x7f076c001710 comp rx=0 tx=0).stop 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 shutdown_connections 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07740ca430 0x7f0774069130 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07740c7d80 0x7f0774068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0754060670 0x7f0754062b20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 --2- 172.21.15.46:0/3550657550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07740a9840 0x7f0774140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 >> 172.21.15.46:0/3550657550 conn(0x7f07740cbe30 msgr2=0x7f077406da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 shutdown_connections 2023-10-06T12:52:24.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:23.591+0000 7f0779785700 1 -- 172.21.15.46:0/3550657550 wait complete. 2023-10-06T12:52:24.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T12:52:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.591+0000 7f804a60c700 1 Processor -- start 2023-10-06T12:52:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.591+0000 7f804a60c700 1 -- start start 2023-10-06T12:52:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f804a60c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f80440697c0 0x7f80440cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f804a60c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440cf100 con 0x7f80440cb4c0 2023-10-06T12:52:24.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f804a60c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440ce4f0 con 0x7f80440af000 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f804a60c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440ce300 con 0x7f80440697c0 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f8048e09700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f80440cb4c0 0x7f80440690b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59372/0 (socket says 172.21.15.46:59372) 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f8048e09700 1 -- 172.21.15.46:0/3279984240 learned_addr learned my addr 172.21.15.46:0/3279984240 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.595+0000 7f8049e0b700 1 --2- 172.21.15.46:0/3279984240 >> v2:172.21.15.132:3300/0 conn(0x7f80440697c0 0x7f80440cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652655887 0 0) 0x7f80440cf100 con 0x7f80440cb4c0 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8030003570 con 0x7f80440cb4c0 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176520120 0 0) 0x7f80440ce4f0 con 0x7f80440af000 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80440cf100 con 0x7f80440af000 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f8049e0b700 1 -- 172.21.15.46:0/3279984240 >> v1:172.21.15.132:6789/0 conn(0x7f80440af000 legacy=0x7f80440cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f8049e0b700 1 -- 172.21.15.46:0/3279984240 >> v1:172.21.15.46:6789/0 conn(0x7f80440cb4c0 legacy=0x7f80440690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f8049e0b700 1 -- 172.21.15.46:0/3279984240 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440ce7d0 con 0x7f80440697c0 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f8049e0b700 1 --2- 172.21.15.46:0/3279984240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440697c0 0x7f80440cf590 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553be4fbc97e82df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c011630 con 0x7f80440697c0 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c011790 con 0x7f80440697c0 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f803bfff700 1 -- 172.21.15.46:0/3279984240 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c011b30 con 0x7f80440697c0 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 -- 172.21.15.46:0/3279984240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440697c0 msgr2=0x7f80440cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 --2- 172.21.15.46:0/3279984240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440697c0 0x7f80440cf590 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 -- 172.21.15.46:0/3279984240 shutdown_connections 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 --2- 172.21.15.46:0/3279984240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440697c0 0x7f80440cf590 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 -- 172.21.15.46:0/3279984240 >> 172.21.15.46:0/3279984240 conn(0x7f80440683b0 msgr2=0x7f80440d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 -- 172.21.15.46:0/3279984240 shutdown_connections 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.599+0000 7f804a60c700 1 -- 172.21.15.46:0/3279984240 wait complete. 2023-10-06T12:52:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 Processor -- start 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 -- start start 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80440697c0 0x7f8044156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440af000 0x7f8044159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c84d0 con 0x7f80440cb4c0 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415a1b0 con 0x7f80440af000 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.603+0000 7f804a60c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804415a490 con 0x7f80440697c0 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35678/0 (socket says 172.21.15.46:35678) 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 -- 172.21.15.46:0/891181276 learned_addr learned my addr 172.21.15.46:0/891181276 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8048e09700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440af000 0x7f8044159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f804960a700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80440697c0 0x7f8044156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 -- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80440697c0 msgr2=0x7f8044156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80440697c0 0x7f8044156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 -- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440af000 msgr2=0x7f8044159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440af000 0x7f8044159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 -- 172.21.15.46:0/891181276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009420 con 0x7f80440cb4c0 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f8049e0b700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f803c013000 tx=0x7f803c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c01f070 con 0x7f80440cb4c0 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c002f70 con 0x7f80440cb4c0 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c023640 con 0x7f80440cb4c0 2023-10-06T12:52:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804415a770 con 0x7f80440cb4c0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.607+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044156fb0 con 0x7f80440cb4c0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.611+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044064590 con 0x7f80440cb4c0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.611+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f803c02c3b0 con 0x7f80440cb4c0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.611+0000 7f80397fa700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 0x7f8028062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.611+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f803c07de30 con 0x7f80440cb4c0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.615+0000 7f804960a700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 0x7f8028062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.615+0000 7f804960a700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 0x7f8028062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8040004140 tx=0x7f8040007f90 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.619+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f803c04e260 con 0x7f80440cb4c0 2023-10-06T12:52:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.803+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f80440ceab0 con 0x7f80440cb4c0 2023-10-06T12:52:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.803+0000 7f80397fa700 1 -- 172.21.15.46:0/891181276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v193) v1 ==== 84+0+126 (secure 0 0 0) 0x7f803c015090 con 0x7f80440cb4c0 2023-10-06T12:52:24.871 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 2, 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": "rbd" 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:24.872 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 msgr2=0x7f8028062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 0x7f8028062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8040004140 tx=0x7f8040007f90 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 msgr2=0x7f80440c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f803c013000 tx=0x7f803c017000 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 shutdown_connections 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80440cb4c0 0x7f80440c5d90 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8028060670 0x7f8028062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440af000 0x7f8044159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 --2- 172.21.15.46:0/891181276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80440697c0 0x7f8044156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 >> 172.21.15.46:0/891181276 conn(0x7f80440683b0 msgr2=0x7f80440b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:24.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 shutdown_connections 2023-10-06T12:52:24.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:24.807+0000 7f804a60c700 1 -- 172.21.15.46:0/891181276 wait complete. 2023-10-06T12:52:24.875 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T12:52:24.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T12:52:25.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.043+0000 7f070604c700 1 Processor -- start 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.043+0000 7f070604c700 1 -- start start 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0700158de0 0x7f07000b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 0x7f0700069060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 0x7f070015cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0700069a20 con 0x7f0700158de0 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07000b5eb0 con 0x7f07000695a0 2023-10-06T12:52:25.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070604c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cef30 con 0x7f07001591b0 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0700158de0 0x7f07000b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0700158de0 0x7f07000b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35684/0 (socket says 172.21.15.46:35684) 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 -- 172.21.15.46:0/155300177 learned_addr learned my addr 172.21.15.46:0/155300177 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070584b700 1 --2- 172.21.15.46:0/155300177 >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 0x7f070015cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f0704849700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 0x7f0700069060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 -- 172.21.15.46:0/155300177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 msgr2=0x7f0700069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 0x7f0700069060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 -- 172.21.15.46:0/155300177 >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 msgr2=0x7f070015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 --2- 172.21.15.46:0/155300177 >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 0x7f070015cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 -- 172.21.15.46:0/155300177 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000cf4f0 con 0x7f0700158de0 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070584b700 1 --2- 172.21.15.46:0/155300177 >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 0x7f070015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f0704849700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 0x7f0700069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f070504a700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0700158de0 0x7f07000b5970 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f07000b26f0 tx=0x7f06fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef7e924aff9b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f06f7fff700 1 -- 172.21.15.46:0/155300177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00b860 con 0x7f0700158de0 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f06f7fff700 1 -- 172.21.15.46:0/155300177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc00b9c0 con 0x7f0700158de0 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.047+0000 7f06f7fff700 1 -- 172.21.15.46:0/155300177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc00bd60 con 0x7f0700158de0 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- 172.21.15.46:0/155300177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0700158de0 msgr2=0x7f07000b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0700158de0 0x7f07000b5970 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f07000b26f0 tx=0x7f06fc009250 comp rx=0 tx=0).stop 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- 172.21.15.46:0/155300177 shutdown_connections 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 --2- 172.21.15.46:0/155300177 >> v2:172.21.15.132:3300/0 conn(0x7f07000695a0 0x7f070015cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07001591b0 0x7f0700069060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 --2- 172.21.15.46:0/155300177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0700158de0 0x7f07000b5970 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- 172.21.15.46:0/155300177 >> 172.21.15.46:0/155300177 conn(0x7f0700068390 msgr2=0x7f07000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- 172.21.15.46:0/155300177 shutdown_connections 2023-10-06T12:52:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- 172.21.15.46:0/155300177 wait complete. 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 Processor -- start 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.051+0000 7f070604c700 1 -- start start 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07000695a0 0x7f07000aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0700158de0 0x7f07000b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070015b720 con 0x7f07001591b0 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700169680 con 0x7f07000695a0 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000c63a0 con 0x7f0700158de0 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35698/0 (socket says 172.21.15.46:35698) 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 -- 172.21.15.46:0/1027748728 learned_addr learned my addr 172.21.15.46:0/1027748728 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070504a700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07000695a0 0x7f07000aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f0704849700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0700158de0 0x7f07000b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 -- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0700158de0 msgr2=0x7f07000b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0700158de0 0x7f07000b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 -- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07000695a0 msgr2=0x7f07000aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07000695a0 0x7f07000aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 -- 172.21.15.46:0/1027748728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0094f0 con 0x7f07001591b0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070584b700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f06f800d180 tx=0x7f06f8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800dd70 con 0x7f07001591b0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800f040 con 0x7f07001591b0 2023-10-06T12:52:25.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000c6680 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8015690 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.055+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000af3d0 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.059+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0700064590 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.059+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f06f80143a0 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.063+0000 7f06f57fa700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 0x7f06e4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.063+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f06f807cd80 con 0x7f07001591b0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.063+0000 7f070504a700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 0x7f06e4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.063+0000 7f070504a700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 0x7f06e4062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07000b26f0 tx=0x7f06fc017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.071+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f06f804d060 con 0x7f07001591b0 2023-10-06T12:52:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.243+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f07000ccac0 con 0x7f07001591b0 2023-10-06T12:52:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.963+0000 7f06f57fa700 1 -- 172.21.15.46:0/1027748728 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v194) v1 ==== 138+0+0 (secure 0 0 0) 0x7f06f8050680 con 0x7f07001591b0 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.967+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 msgr2=0x7f06e4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.967+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 0x7f06e4062b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f07000b26f0 tx=0x7f06fc017000 comp rx=0 tx=0).stop 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.967+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 msgr2=0x7f07000b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.967+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f06f800d180 tx=0x7f06f8009a50 comp rx=0 tx=0).stop 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 shutdown_connections 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07001591b0 0x7f07000b1240 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0700158de0 0x7f07000b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f06e40606a0 0x7f06e4062b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 --2- 172.21.15.46:0/1027748728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07000695a0 0x7f07000aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.971+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 >> 172.21.15.46:0/1027748728 conn(0x7f0700068390 msgr2=0x7f0700155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:26.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.975+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 shutdown_connections 2023-10-06T12:52:26.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:25.975+0000 7f070604c700 1 -- 172.21.15.46:0/1027748728 wait complete. 2023-10-06T12:52:26.038 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-10-06T12:52:26.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T12:52:26.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.215+0000 7f89f5785700 1 Processor -- start 2023-10-06T12:52:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.215+0000 7f89f5785700 1 -- start start 2023-10-06T12:52:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89f0158da0 0x7f89f015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 0x7f89f00690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f00caeb0 0x7f89f00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f0069610 con 0x7f89f0158da0 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00cb850 con 0x7f89f00d0af0 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89f5785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f015d3c0 con 0x7f89f00caeb0 2023-10-06T12:52:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89f0158da0 0x7f89f015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89ee7fc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 0x7f89f00690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89f0158da0 0x7f89f015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35710/0 (socket says 172.21.15.46:35710) 2023-10-06T12:52:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 -- 172.21.15.46:0/592270586 learned_addr learned my addr 172.21.15.46:0/592270586 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89ef7fe700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f00caeb0 0x7f89f00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 -- 172.21.15.46:0/592270586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f00caeb0 msgr2=0x7f89f00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f00caeb0 0x7f89f00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 -- 172.21.15.46:0/592270586 >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 msgr2=0x7f89f00690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 --2- 172.21.15.46:0/592270586 >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 0x7f89f00690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89eeffd700 1 -- 172.21.15.46:0/592270586 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00d0f40 con 0x7f89f0158da0 2023-10-06T12:52:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.219+0000 7f89ee7fc700 1 --2- 172.21.15.46:0/592270586 >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 0x7f89f00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89eeffd700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f0158da0 0x7f89f015d850 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f89f00b2bd0 tx=0x7f89e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e27a3b5003b51e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89edffb700 1 -- 172.21.15.46:0/592270586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000b640 con 0x7f89f0158da0 2023-10-06T12:52:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89edffb700 1 -- 172.21.15.46:0/592270586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000b7a0 con 0x7f89f0158da0 2023-10-06T12:52:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89edffb700 1 -- 172.21.15.46:0/592270586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000bb60 con 0x7f89f0158da0 2023-10-06T12:52:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 -- 172.21.15.46:0/592270586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f0158da0 msgr2=0x7f89f015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f0158da0 0x7f89f015d850 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f89f00b2bd0 tx=0x7f89e0009250 comp rx=0 tx=0).stop 2023-10-06T12:52:26.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 -- 172.21.15.46:0/592270586 shutdown_connections 2023-10-06T12:52:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 --2- 172.21.15.46:0/592270586 >> v2:172.21.15.132:3300/0 conn(0x7f89f00d0af0 0x7f89f00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f00caeb0 0x7f89f00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 --2- 172.21.15.46:0/592270586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f0158da0 0x7f89f015d850 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 -- 172.21.15.46:0/592270586 >> 172.21.15.46:0/592270586 conn(0x7f89f00683f0 msgr2=0x7f89f00d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 -- 172.21.15.46:0/592270586 shutdown_connections 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.223+0000 7f89f5785700 1 -- 172.21.15.46:0/592270586 wait complete. 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.227+0000 7f89f5785700 1 Processor -- start 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.227+0000 7f89f5785700 1 -- start start 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89f00d0af0 0x7f89f015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f0158da0 0x7f89f00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f015cc70 con 0x7f89f00caeb0 2023-10-06T12:52:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c4fb0 con 0x7f89f00d0af0 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c5290 con 0x7f89f0158da0 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35718/0 (socket says 172.21.15.46:35718) 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 -- 172.21.15.46:0/3017467758 learned_addr learned my addr 172.21.15.46:0/3017467758 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89ee7fc700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89f00d0af0 0x7f89f015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89ef7fe700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f0158da0 0x7f89f00c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 -- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f0158da0 msgr2=0x7f89f00c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f0158da0 0x7f89f00c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 -- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89f00d0af0 msgr2=0x7f89f015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89f00d0af0 0x7f89f015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 -- 172.21.15.46:0/3017467758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009420 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89eeffd700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f89f00884e0 tx=0x7f89e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000b910 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000ba70 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e00094f0 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0011630 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.231+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00c5850 con 0x7f89f00caeb0 2023-10-06T12:52:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.235+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f0064590 con 0x7f89f00caeb0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.235+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f89e00119a0 con 0x7f89f00caeb0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.239+0000 7f89d6ffd700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 0x7f89dc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.239+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f89e007e6f0 con 0x7f89f00caeb0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.239+0000 7f89ee7fc700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 0x7f89dc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.239+0000 7f89ee7fc700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 0x7f89dc062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89f01562d0 tx=0x7f89e40091a0 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.243+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f89e004e9f0 con 0x7f89f00caeb0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.415+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f89f00880e0 con 0x7f89f00caeb0 2023-10-06T12:52:27.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:26.979+0000 7f89d6ffd700 1 -- 172.21.15.46:0/3017467758 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v195) v1 ==== 170+0+0 (secure 0 0 0) 0x7f89e0052010 con 0x7f89f00caeb0 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 msgr2=0x7f89dc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 0x7f89dc062b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89f01562d0 tx=0x7f89e40091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 msgr2=0x7f89f00b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f89f00884e0 tx=0x7f89e0017000 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 shutdown_connections 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89f0158da0 0x7f89f00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f89dc060670 0x7f89dc062b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89f00d0af0 0x7f89f015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 --2- 172.21.15.46:0/3017467758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f00caeb0 0x7f89f00b3bc0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.003+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 >> 172.21.15.46:0/3017467758 conn(0x7f89f00683f0 msgr2=0x7f89f00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:27.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.007+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 shutdown_connections 2023-10-06T12:52:27.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.007+0000 7f89f5785700 1 -- 172.21.15.46:0/3017467758 wait complete. 2023-10-06T12:52:27.130 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-10-06T12:52:27.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T12:52:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.303+0000 7f1a2f59e700 1 Processor -- start 2023-10-06T12:52:27.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.303+0000 7f1a2f59e700 1 -- start start 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2f59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a30154880 0x7f1a300c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a300c6cf0 con 0x7f1a300c7f20 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2f59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a300c2a30 con 0x7f1a30154880 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a300c83e0 con 0x7f1a30154c50 2023-10-06T12:52:27.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2dd9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2dd9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:34688/0 (socket says 172.21.15.46:34688) 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2e59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a30154880 0x7f1a300c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2ed9d700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1a300c7f20 0x7f1a3015aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:40950/0 (socket says 172.21.15.46:40950) 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2dd9b700 1 -- 172.21.15.46:0/1665916329 learned_addr learned my addr 172.21.15.46:0/1665916329 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2e59c700 1 -- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 msgr2=0x7f1a3006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2e59c700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2e59c700 1 -- 172.21.15.46:0/1665916329 >> v1:172.21.15.132:6789/0 conn(0x7f1a300c7f20 legacy=0x7f1a3015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2e59c700 1 -- 172.21.15.46:0/1665916329 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3006e5d0 con 0x7f1a30154880 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2dd9b700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.307+0000 7f1a2d59a700 1 -- 172.21.15.46:0/1665916329 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150512789 0 0) 0x7f1a300c6cf0 con 0x7f1a300c7f20 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2e59c700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154880 0x7f1a300c7180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098e0 tx=0x7f1a200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c68fdf8f0657ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:27.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2d59a700 1 -- 172.21.15.46:0/1665916329 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000b640 con 0x7f1a30154880 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2d59a700 1 -- 172.21.15.46:0/1665916329 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2000b7a0 con 0x7f1a30154880 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2d59a700 1 -- 172.21.15.46:0/1665916329 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000bb40 con 0x7f1a30154880 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 -- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154880 msgr2=0x7f1a300c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154880 0x7f1a300c7180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098e0 tx=0x7f1a200091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 -- 172.21.15.46:0/1665916329 shutdown_connections 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154c50 0x7f1a3006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/1665916329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154880 0x7f1a300c7180 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 -- 172.21.15.46:0/1665916329 >> 172.21.15.46:0/1665916329 conn(0x7f1a300cbc10 msgr2=0x7f1a300cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 -- 172.21.15.46:0/1665916329 shutdown_connections 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.311+0000 7f1a2f59e700 1 -- 172.21.15.46:0/1665916329 wait complete. 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 Processor -- start 2023-10-06T12:52:27.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 -- start start 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154880 0x7f1a30155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 0x7f1a300c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3015a2c0 con 0x7f1a300c7f20 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30155590 con 0x7f1a30154c50 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a301599d0 con 0x7f1a30154880 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35720/0 (socket says 172.21.15.46:35720) 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 -- 172.21.15.46:0/504174879 learned_addr learned my addr 172.21.15.46:0/504174879 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2dd9b700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154880 0x7f1a30155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2ed9d700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 0x7f1a300c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 -- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154880 msgr2=0x7f1a30155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154880 0x7f1a30155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 -- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 msgr2=0x7f1a300c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 0x7f1a300c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 -- 172.21.15.46:0/504174879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240094f0 con 0x7f1a300c7f20 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2ed9d700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 0x7f1a300c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.315+0000 7f1a2e59c700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f1a200093f0 tx=0x7f1a20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:27.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2001bd70 con 0x7f1a300c7f20 2023-10-06T12:52:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20011420 con 0x7f1a300c7f20 2023-10-06T12:52:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a200117e0 con 0x7f1a300c7f20 2023-10-06T12:52:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a20009420 con 0x7f1a300c7f20 2023-10-06T12:52:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a30159c70 con 0x7f1a300c7f20 2023-10-06T12:52:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.319+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a30064590 con 0x7f1a300c7f20 2023-10-06T12:52:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.323+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f1a20011c10 con 0x7f1a300c7f20 2023-10-06T12:52:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.323+0000 7f1a1e7fc700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 0x7f1a10062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.323+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f1a2007e9b0 con 0x7f1a300c7f20 2023-10-06T12:52:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.323+0000 7f1a2dd9b700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 0x7f1a10062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.327+0000 7f1a2dd9b700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 0x7f1a10062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a3015b530 tx=0x7f1a24009250 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.331+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1a2004eb30 con 0x7f1a300c7f20 2023-10-06T12:52:27.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:27.539+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1a3006e8b0 con 0x7f1a300c7f20 2023-10-06T12:52:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.019+0000 7f1a1e7fc700 1 -- 172.21.15.46:0/504174879 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1a20052150 con 0x7f1a300c7f20 2023-10-06T12:52:28.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 msgr2=0x7f1a10062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 0x7f1a10062b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a3015b530 tx=0x7f1a24009250 comp rx=0 tx=0).stop 2023-10-06T12:52:28.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 msgr2=0x7f1a300b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f1a200093f0 tx=0x7f1a20017000 comp rx=0 tx=0).stop 2023-10-06T12:52:28.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 shutdown_connections 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f1a100606a0 0x7f1a10062b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a30154c50 0x7f1a300c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a30154880 0x7f1a30155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 --2- 172.21.15.46:0/504174879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a300c7f20 0x7f1a300b0500 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 >> 172.21.15.46:0/504174879 conn(0x7f1a300cbc10 msgr2=0x7f1a3014f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 shutdown_connections 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.031+0000 7f1a2f59e700 1 -- 172.21.15.46:0/504174879 wait complete. 2023-10-06T12:52:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T12:52:28.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:28.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.419+0000 7f2cf0923700 1 Processor -- start 2023-10-06T12:52:28.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.419+0000 7f2cf0923700 1 -- start start 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cf0923700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cec067760 0x7f2cec0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cf0923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cec0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cf0923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0c8480 con 0x7f2cec0bedd0 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cf0923700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec069660 con 0x7f2cec067760 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cf0923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec068ac0 con 0x7f2cec0bf1a0 2023-10-06T12:52:28.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cec0bedd0 0x7f2cec067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59374/0 (socket says 172.21.15.46:59374) 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/3054958218 learned_addr learned my addr 172.21.15.46:0/3054958218 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cebfff700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cec0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540296658 0 0) 0x7f2cdc001110 con 0x7f2cec0bedd0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/3054958218 >> v2:172.21.15.132:3300/0 conn(0x7f2cec067760 0x7f2cec0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd4003570 con 0x7f2cec0bedd0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176826882 0 0) 0x7f2cd4003570 con 0x7f2cec0bedd0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 msgr2=0x7f2cec0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cec0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 >> v2:172.21.15.132:3300/0 conn(0x7f2cec067760 msgr2=0x7f2cec0c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 --2- 172.21.15.46:0/3054958218 >> v2:172.21.15.132:3300/0 conn(0x7f2cec067760 0x7f2cec0c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec068d30 con 0x7f2cec0bedd0 2023-10-06T12:52:28.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f2cdc001600 con 0x7f2cec0bedd0 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/3054958218 >> v2:172.21.15.132:3300/0 conn(0x7f2cec067760 0x7f2cec0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 >> v1:172.21.15.46:6789/0 conn(0x7f2cec0bedd0 legacy=0x7f2cec067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cd4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec069660 con 0x7f2cd4003a60 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec068ac0 con 0x7f2cd4004d50 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd400bae0 con 0x7f2cec0bf1a0 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cd4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2cebfff700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 msgr2=0x7f2cd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0bf1a0 0x7f2cd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd4004d50 msgr2=0x7f2cd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.423+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/3054958218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4003570 con 0x7f2cd4003a60 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cd4003e70 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c5f0 tx=0x7f2cdc007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d39ed471740175 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00cdf0 con 0x7f2cd4003a60 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc005920 con 0x7f2cd4003a60 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cea7fc700 1 -- 172.21.15.46:0/3054958218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc005ce0 con 0x7f2cd4003a60 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 -- 172.21.15.46:0/3054958218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 msgr2=0x7f2cd4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 --2- 172.21.15.46:0/3054958218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cd4003e70 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c5f0 tx=0x7f2cdc007a80 comp rx=0 tx=0).stop 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cebfff700 1 -- 172.21.15.46:0/3054958218 reap_dead start 2023-10-06T12:52:28.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 -- 172.21.15.46:0/3054958218 shutdown_connections 2023-10-06T12:52:28.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 -- 172.21.15.46:0/3054958218 >> 172.21.15.46:0/3054958218 conn(0x7f2cec062ef0 msgr2=0x7f2cec0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 -- 172.21.15.46:0/3054958218 shutdown_connections 2023-10-06T12:52:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.427+0000 7f2cf0923700 1 -- 172.21.15.46:0/3054958218 wait complete. 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 Processor -- start 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 -- start start 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cec15ebf0 0x7f2cec15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 0x7f2cec0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c9970 con 0x7f2cd4003a60 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c9c50 con 0x7f2cec15ebf0 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cf0923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c9f30 con 0x7f2cec0caa10 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35742/0 (socket says 172.21.15.46:35742) 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/2727679449 learned_addr learned my addr 172.21.15.46:0/2727679449 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cec15ebf0 0x7f2cec15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.431+0000 7f2cebfff700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 0x7f2cec0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 msgr2=0x7f2cec0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 0x7f2cec0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cec15ebf0 msgr2=0x7f2cec15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cec15ebf0 0x7f2cec15f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 -- 172.21.15.46:0/2727679449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0ca210 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cebfff700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 0x7f2cec0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2ceb7fe700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00cbe0 tx=0x7f2cdc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0072e0 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc007460 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec0ca4f0 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc007820 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec0ba910 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.435+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec064590 con 0x7f2cd4003a60 2023-10-06T12:52:28.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.439+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2cdc019030 con 0x7f2cd4003a60 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.439+0000 7f2cd37fe700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 0x7f2cd8062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.439+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f2cdc07e490 con 0x7f2cd4003a60 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.439+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 0x7f2cd8062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.439+0000 7f2ceaffd700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 0x7f2cd8062b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cec15db80 tx=0x7f2ce0008040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.447+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2cdc04e530 con 0x7f2cd4003a60 2023-10-06T12:52:28.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.619+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cec04b5c0 con 0x7f2cd4003a60 2023-10-06T12:52:28.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.619+0000 7f2cd37fe700 1 -- 172.21.15.46:0/2727679449 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17771 (secure 0 0 0) 0x7f2cdc01ae60 con 0x7f2cd4003a60 2023-10-06T12:52:28.625 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:28.625 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":196,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:27.541637+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 msgr2=0x7f2cd8062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 0x7f2cd8062b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cec15db80 tx=0x7f2ce0008040 comp rx=0 tx=0).stop 2023-10-06T12:52:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 msgr2=0x7f2cec15a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00cbe0 tx=0x7f2cdc015000 comp rx=0 tx=0).stop 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 shutdown_connections 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cec0caa10 0x7f2cec0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2cd80606a0 0x7f2cd8062b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cec15ebf0 0x7f2cec15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 --2- 172.21.15.46:0/2727679449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd4003a60 0x7f2cec15a4b0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 >> 172.21.15.46:0/2727679449 conn(0x7f2cec062ef0 msgr2=0x7f2cec15adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 shutdown_connections 2023-10-06T12:52:28.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.623+0000 7f2cf0923700 1 -- 172.21.15.46:0/2727679449 wait complete. 2023-10-06T12:52:28.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.915+0000 7f8b2b59e700 1 Processor -- start 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.915+0000 7f8b2b59e700 1 -- start start 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2b59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b2c0bedd0 0x7f8b2c067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2b59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8b2c0bf1a0 0x7f8b2c067590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2b59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c069660 con 0x7f8b2c0c7a00 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2b59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c068ac0 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2b59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c068d30 con 0x7f8b2c0bf1a0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b2c0bedd0 0x7f8b2c067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b2c0bedd0 0x7f8b2c067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35748/0 (socket says 172.21.15.46:35748) 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 -- 172.21.15.46:0/3901798062 learned_addr learned my addr 172.21.15.46:0/3901798062 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b21d9b700 1 --2- 172.21.15.46:0/3901798062 >> v2:172.21.15.132:3300/0 conn(0x7f8b2c0bf1a0 0x7f8b2c067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 -- 172.21.15.46:0/3901798062 >> v2:172.21.15.132:3300/0 conn(0x7f8b2c0bf1a0 msgr2=0x7f8b2c067590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 --2- 172.21.15.46:0/3901798062 >> v2:172.21.15.132:3300/0 conn(0x7f8b2c0bf1a0 0x7f8b2c067590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 -- 172.21.15.46:0/3901798062 >> v1:172.21.15.132:6789/0 conn(0x7f8b2c0c7a00 legacy=0x7f8b2c0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 -- 172.21.15.46:0/3901798062 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c069010 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b2a59c700 1 --2- 172.21.15.46:0/3901798062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0bedd0 0x7f8b2c067050 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0cc5e0 tx=0x7f8b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dcbb393d9df61a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b29d9b700 1 -- 172.21.15.46:0/3901798062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c009e70 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b29d9b700 1 -- 172.21.15.46:0/3901798062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c00b860 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.919+0000 7f8b29d9b700 1 -- 172.21.15.46:0/3901798062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00bc00 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- 172.21.15.46:0/3901798062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0bedd0 msgr2=0x7f8b2c067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/3901798062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0bedd0 0x7f8b2c067050 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0cc5e0 tx=0x7f8b1c009250 comp rx=0 tx=0).stop 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- 172.21.15.46:0/3901798062 shutdown_connections 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/3901798062 >> v2:172.21.15.132:3300/0 conn(0x7f8b2c0bf1a0 0x7f8b2c067590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/3901798062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0bedd0 0x7f8b2c067050 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- 172.21.15.46:0/3901798062 >> 172.21.15.46:0/3901798062 conn(0x7f8b2c062ef0 msgr2=0x7f8b2c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- 172.21.15.46:0/3901798062 shutdown_connections 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- 172.21.15.46:0/3901798062 wait complete. 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 Processor -- start 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.923+0000 7f8b2b59e700 1 -- start start 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b2c0bedd0 0x7f8b2c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 0x7f8b2c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c160820 con 0x7f8b2c0c7a00 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0ae960 con 0x7f8b2c0bf1a0 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c15ee90 con 0x7f8b2c0bedd0 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35760/0 (socket says 172.21.15.46:35760) 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 -- 172.21.15.46:0/1573297323 learned_addr learned my addr 172.21.15.46:0/1573297323 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2a59c700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b2c0bedd0 0x7f8b2c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b21d9b700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 0x7f8b2c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 -- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b2c0bedd0 msgr2=0x7f8b2c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b2c0bedd0 0x7f8b2c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 -- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 msgr2=0x7f8b2c0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 0x7f8b2c0b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 -- 172.21.15.46:0/1573297323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0094f0 con 0x7f8b2c0c7a00 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b21d9b700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 0x7f8b2c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2ad9d700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c910 tx=0x7f8b240016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2400ce50 con 0x7f8b2c0c7a00 2023-10-06T12:52:28.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2400a7e0 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2400aba0 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c15ff80 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.927+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c160310 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.931+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c064590 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.931+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8b24003030 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.931+0000 7f8b22ffd700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 0x7f8b10062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.935+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f8b2407cdc0 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.935+0000 7f8b2a59c700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 0x7f8b10062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.935+0000 7f8b2a59c700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 0x7f8b10062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0cc5e0 tx=0x7f8b1c017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:28.939+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b2404ce60 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.111+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b2c04b5c0 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b22ffd700 1 -- 172.21.15.46:0/1573297323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+17758 (secure 0 0 0) 0x7f8b24050480 con 0x7f8b2c0c7a00 2023-10-06T12:52:29.165 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":197,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:28.983615+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 msgr2=0x7f8b10062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 0x7f8b10062b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0cc5e0 tx=0x7f8b1c017000 comp rx=0 tx=0).stop 2023-10-06T12:52:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 msgr2=0x7f8b2c15e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400c910 tx=0x7f8b240016d0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 shutdown_connections 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b2c0c7a00 0x7f8b2c15e030 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8b100606a0 0x7f8b10062b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b2c0bf1a0 0x7f8b2c0b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 --2- 172.21.15.46:0/1573297323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b2c0bedd0 0x7f8b2c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.115+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 >> 172.21.15.46:0/1573297323 conn(0x7f8b2c062ef0 msgr2=0x7f8b2c15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.119+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 shutdown_connections 2023-10-06T12:52:29.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.119+0000 7f8b2b59e700 1 -- 172.21.15.46:0/1573297323 wait complete. 2023-10-06T12:52:29.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T12:52:29.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.339+0000 7f2c7042e700 1 Processor -- start 2023-10-06T12:52:29.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.339+0000 7f2c7042e700 1 -- start start 2023-10-06T12:52:29.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c7042e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2c68161fc0 0x7f2c680caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c7042e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680c1980 con 0x7f2c68146eb0 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c7042e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680c2520 con 0x7f2c68147280 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c7042e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c680cb1e0 con 0x7f2c68161fc0 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6e1ca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c68146eb0 0x7f2c680c1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59380/0 (socket says 172.21.15.46:59380) 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6e1ca700 1 -- 172.21.15.46:0/3297048512 learned_addr learned my addr 172.21.15.46:0/3297048512 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/3297048512 >> v2:172.21.15.132:3300/0 conn(0x7f2c68161fc0 0x7f2c680caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751647304 0 0) 0x7f2c680c1980 con 0x7f2c68146eb0 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c54003570 con 0x7f2c68146eb0 2023-10-06T12:52:29.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808976854 0 0) 0x7f2c680c2520 con 0x7f2c68147280 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c680c1980 con 0x7f2c68147280 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d9c9700 1 -- 172.21.15.46:0/3297048512 >> v1:172.21.15.132:6789/0 conn(0x7f2c68147280 legacy=0x7f2c6806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d9c9700 1 -- 172.21.15.46:0/3297048512 >> v1:172.21.15.46:6789/0 conn(0x7f2c68146eb0 legacy=0x7f2c680c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d9c9700 1 -- 172.21.15.46:0/3297048512 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680c0d70 con 0x7f2c68161fc0 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.343+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/3297048512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68161fc0 0x7f2c680caca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2c580098e0 tx=0x7f2c580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453a035bb27e188e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011630 con 0x7f2c68161fc0 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58011790 con 0x7f2c68161fc0 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c6d1c8700 1 -- 172.21.15.46:0/3297048512 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011b30 con 0x7f2c68161fc0 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 -- 172.21.15.46:0/3297048512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68161fc0 msgr2=0x7f2c680caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 --2- 172.21.15.46:0/3297048512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68161fc0 0x7f2c680caca0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2c580098e0 tx=0x7f2c580091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 -- 172.21.15.46:0/3297048512 shutdown_connections 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 --2- 172.21.15.46:0/3297048512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68161fc0 0x7f2c680caca0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 -- 172.21.15.46:0/3297048512 >> 172.21.15.46:0/3297048512 conn(0x7f2c680cbe30 msgr2=0x7f2c680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 -- 172.21.15.46:0/3297048512 shutdown_connections 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 -- 172.21.15.46:0/3297048512 wait complete. 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.347+0000 7f2c7042e700 1 Processor -- start 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 -- start start 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 0x7f2c6816e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c68161fc0 0x7f2c680afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680af0a0 con 0x7f2c68146eb0 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680b1940 con 0x7f2c68147280 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6816ef70 con 0x7f2c68161fc0 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35774/0 (socket says 172.21.15.46:35774) 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 -- 172.21.15.46:0/4060368313 learned_addr learned my addr 172.21.15.46:0/4060368313 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e9cb700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c68161fc0 0x7f2c680afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 0x7f2c6816e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 -- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c68161fc0 msgr2=0x7f2c680afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c68161fc0 0x7f2c680afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 -- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 msgr2=0x7f2c6816e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 0x7f2c6816e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 -- 172.21.15.46:0/4060368313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58009420 con 0x7f2c68146eb0 2023-10-06T12:52:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 0x7f2c6816e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c6e1ca700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c7c0 tx=0x7f2c64007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:29.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64005b30 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64005c90 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c680af2f0 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.351+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007300 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.355+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68160860 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.355+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c680660f0 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.355+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2c6401b3b0 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.359+0000 7f2c5e7fc700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 0x7f2c4c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.359+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f2c6407c340 con 0x7f2c68146eb0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.359+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 0x7f2c4c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.359+0000 7f2c6d9c9700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 0x7f2c4c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c580091e0 tx=0x7f2c58017000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.367+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2c6404fe70 con 0x7f2c68146eb0 2023-10-06T12:52:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:29.575+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2c680c1050 con 0x7f2c68146eb0 2023-10-06T12:52:30.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.019+0000 7f2c5e7fc700 1 -- 172.21.15.46:0/4060368313 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2c6404fa00 con 0x7f2c68146eb0 2023-10-06T12:52:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 msgr2=0x7f2c4c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 0x7f2c4c062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c580091e0 tx=0x7f2c58017000 comp rx=0 tx=0).stop 2023-10-06T12:52:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 msgr2=0x7f2c6815c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c7c0 tx=0x7f2c64007ea0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 shutdown_connections 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2c68161fc0 0x7f2c680afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2c4c060670 0x7f2c4c062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c68147280 0x7f2c6816e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 --2- 172.21.15.46:0/4060368313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c68146eb0 0x7f2c6815c8b0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 >> 172.21.15.46:0/4060368313 conn(0x7f2c680cbe30 msgr2=0x7f2c68145d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 shutdown_connections 2023-10-06T12:52:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.023+0000 7f2c7042e700 1 -- 172.21.15.46:0/4060368313 wait complete. 2023-10-06T12:52:30.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T12:52:30.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.231+0000 7f6a9f59e700 1 Processor -- start 2023-10-06T12:52:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.231+0000 7f6a9f59e700 1 -- start start 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6aa00adbe0 0x7f6aa00d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00afd20 con 0x7f6aa00b0dc0 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00b0000 con 0x7f6aa00d0d60 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa0159740 con 0x7f6aa00adbe0 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6aa00adbe0 0x7f6aa00d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6aa00adbe0 0x7f6aa00d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35780/0 (socket says 172.21.15.46:35780) 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1693187965 learned_addr learned my addr 172.21.15.46:0/1693187965 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1693187965 >> v1:172.21.15.132:6789/0 conn(0x7f6aa00d0d60 legacy=0x7f6aa00b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1693187965 >> v1:172.21.15.46:6789/0 conn(0x7f6aa00b0dc0 legacy=0x7f6aa00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1693187965 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00b02e0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9e59c700 1 --2- 172.21.15.46:0/1693187965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa00d0820 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6a900099b0 tx=0x7f6a90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86489756f66f996f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9d59a700 1 -- 172.21.15.46:0/1693187965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000b860 con 0x7f6aa00adbe0 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9d59a700 1 -- 172.21.15.46:0/1693187965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9000b9c0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9d59a700 1 -- 172.21.15.46:0/1693187965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9000bd60 con 0x7f6aa00adbe0 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1693187965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 msgr2=0x7f6aa00d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.235+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1693187965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa00d0820 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f6a900099b0 tx=0x7f6a90009250 comp rx=0 tx=0).stop 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1693187965 shutdown_connections 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1693187965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa00d0820 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1693187965 >> 172.21.15.46:0/1693187965 conn(0x7f6aa006ef60 msgr2=0x7f6aa00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1693187965 shutdown_connections 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1693187965 wait complete. 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 Processor -- start 2023-10-06T12:52:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- start start 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa00b0dc0 0x7f6aa015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa00d0d60 0x7f6aa016d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa016dfd0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0159da0 con 0x7f6aa00d0d60 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa016da10 con 0x7f6aa00b0dc0 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35792/0 (socket says 172.21.15.46:35792) 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.239+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1162519644 learned_addr learned my addr 172.21.15.46:0/1162519644 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9dd9b700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa00b0dc0 0x7f6aa015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9ed9d700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa00d0d60 0x7f6aa016d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa00b0dc0 msgr2=0x7f6aa015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa00b0dc0 0x7f6aa015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa00d0d60 msgr2=0x7f6aa016d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa00d0d60 0x7f6aa016d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 -- 172.21.15.46:0/1162519644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900094f0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9e59c700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0088270 tx=0x7f6a90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90011ea0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9001a630 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9001a9d0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa016dcf0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa0151ea0 con 0x7f6aa00adbe0 2023-10-06T12:52:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.243+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa0152090 con 0x7f6aa00adbe0 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.247+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6a9002d440 con 0x7f6aa00adbe0 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.247+0000 7f6a8e7fc700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 0x7f6a80062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.247+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f6a9007e850 con 0x7f6aa00adbe0 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.247+0000 7f6a9dd9b700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 0x7f6a80062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.247+0000 7f6a9dd9b700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 0x7f6a80062b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015ed50 tx=0x7f6a94009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.255+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6aa0152090 con 0x7f6aa00adbe0 2023-10-06T12:52:30.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:30.459+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6aa0159890 con 0x7f6aa00adbe0 2023-10-06T12:52:31.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.023+0000 7f6a8e7fc700 1 -- 172.21.15.46:0/1162519644 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6aa0159890 con 0x7f6aa00adbe0 2023-10-06T12:52:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 msgr2=0x7f6a80062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 0x7f6a80062b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015ed50 tx=0x7f6a94009040 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 msgr2=0x7f6aa015e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0088270 tx=0x7f6a90017000 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 shutdown_connections 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6a800606a0 0x7f6a80062b50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa00d0d60 0x7f6aa016d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa00b0dc0 0x7f6aa015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 --2- 172.21.15.46:0/1162519644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00adbe0 0x7f6aa015e1f0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 >> 172.21.15.46:0/1162519644 conn(0x7f6aa006ef60 msgr2=0x7f6aa00b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 shutdown_connections 2023-10-06T12:52:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:31.027+0000 7f6a9f59e700 1 -- 172.21.15.46:0/1162519644 wait complete. 2023-10-06T12:52:32.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:32.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 Processor -- start 2023-10-06T12:52:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 -- start start 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2180aaf30 0x7fd218067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 0x7fd218068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180679f0 con 0x7fd218156e20 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180acd60 con 0x7fd2180aaf30 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21f08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180ad040 con 0x7fd218067590 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21e08c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd218156e20 0x7fd21806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59388/0 (socket says 172.21.15.46:59388) 2023-10-06T12:52:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21e08c700 1 -- 172.21.15.46:0/3997628354 learned_addr learned my addr 172.21.15.46:0/3997628354 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21d88b700 1 --2- 172.21.15.46:0/3997628354 >> v2:172.21.15.46:3300/0 conn(0x7fd2180aaf30 0x7fd218067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 0x7fd218068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629995176 0 0) 0x7fd214001490 con 0x7fd218156e20 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.543+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd204003570 con 0x7fd218156e20 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409153562 0 0) 0x7fd204003570 con 0x7fd218156e20 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 msgr2=0x7fd218068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 0x7fd218068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 >> v2:172.21.15.46:3300/0 conn(0x7fd2180aaf30 msgr2=0x7fd218067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 --2- 172.21.15.46:0/3997628354 >> v2:172.21.15.46:3300/0 conn(0x7fd2180aaf30 0x7fd218067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180ad320 con 0x7fd218156e20 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7fd2140019a0 con 0x7fd218156e20 2023-10-06T12:52:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 >> v1:172.21.15.46:6789/0 conn(0x7fd218156e20 legacy=0x7fd21806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204003a60 0x7fd204003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 0x7fd204004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd204004d50 0x7fd2040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180acd60 con 0x7fd204004d50 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180ad040 con 0x7fd204003a60 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20400bb10 con 0x7fd218067590 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d88b700 1 --2- 172.21.15.46:0/3997628354 >> v2:172.21.15.46:3300/0 conn(0x7fd2180aaf30 0x7fd218067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd204004d50 0x7fd2040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204003a60 0x7fd204003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 -- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 msgr2=0x7fd204004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd218067590 0x7fd204004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 -- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204003a60 msgr2=0x7fd204003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204003a60 0x7fd204003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 -- 172.21.15.46:0/3997628354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204003570 con 0x7fd204004d50 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204003a60 0x7fd204003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd204004d50 0x7fd2040051c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bb90 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9408d5a1cfe20a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000d5d0 con 0x7fd204004d50 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000d730 con 0x7fd204004d50 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21d08a700 1 -- 172.21.15.46:0/3997628354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000daf0 con 0x7fd204004d50 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 -- 172.21.15.46:0/3997628354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd204004d50 msgr2=0x7fd2040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3997628354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd204004d50 0x7fd2040051c0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd21000bb90 tx=0x7fd210008040 comp rx=0 tx=0).stop 2023-10-06T12:52:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21e88d700 1 -- 172.21.15.46:0/3997628354 reap_dead start 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 -- 172.21.15.46:0/3997628354 shutdown_connections 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 -- 172.21.15.46:0/3997628354 >> 172.21.15.46:0/3997628354 conn(0x7fd218062ef0 msgr2=0x7fd2180cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 -- 172.21.15.46:0/3997628354 shutdown_connections 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.547+0000 7fd21f08e700 1 -- 172.21.15.46:0/3997628354 wait complete. 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.551+0000 7fd21f08e700 1 Processor -- start 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.551+0000 7fd21f08e700 1 -- start start 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 0x7fd2180b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2180b0b50 0x7fd218154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218154f20 con 0x7fd218154ac0 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180c5020 con 0x7fd204004d50 2023-10-06T12:52:32.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180c5300 con 0x7fd2180b0b50 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35810/0 (socket says 172.21.15.46:35810) 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 -- 172.21.15.46:0/3044592928 learned_addr learned my addr 172.21.15.46:0/3044592928 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21d88b700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2180b0b50 0x7fd218154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 0x7fd2180b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 -- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2180b0b50 msgr2=0x7fd218154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2180b0b50 0x7fd218154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 -- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 msgr2=0x7fd2180b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 0x7fd2180b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 -- 172.21.15.46:0/3044592928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180c55e0 con 0x7fd218154ac0 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 0x7fd2180b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:32.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21e88d700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c010 tx=0x7fd21001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000d5d0 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21000de60 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2180c58c0 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210014e80 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.555+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2180c4810 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.559+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2180a8200 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.559+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd21002e3f0 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.559+0000 7fd20e7fc700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 0x7fd1fc062b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.559+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fd210000e40 con 0x7fd218154ac0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.559+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 0x7fd1fc062b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:32.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.563+0000 7fd21e08c700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 0x7fd1fc062b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd214007c90 tx=0x7fd214009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:32.707 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-10-06T12:52:32.563+0000 7f6e3c95b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:52:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.567+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2180a8200 con 0x7fd218154ac0 2023-10-06T12:52:32.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.775+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2180a8200 con 0x7fd218154ac0 2023-10-06T12:52:32.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.775+0000 7fd20e7fc700 1 -- 172.21.15.46:0/3044592928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7fd2180a8200 con 0x7fd218154ac0 2023-10-06T12:52:32.784 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:32.784 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:32.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 msgr2=0x7fd1fc062b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 0x7fd1fc062b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd214007c90 tx=0x7fd214009040 comp rx=0 tx=0).stop 2023-10-06T12:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 msgr2=0x7fd2180c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c010 tx=0x7fd21001b000 comp rx=0 tx=0).stop 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 shutdown_connections 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd218154ac0 0x7fd2180c42d0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2180b0b50 0x7fd218154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd1fc060660 0x7fd1fc062b10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 --2- 172.21.15.46:0/3044592928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd204004d50 0x7fd2180b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 >> 172.21.15.46:0/3044592928 conn(0x7fd218062ef0 msgr2=0x7fd2180cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 shutdown_connections 2023-10-06T12:52:33.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:32.779+0000 7fd21f08e700 1 -- 172.21.15.46:0/3044592928 wait complete. 2023-10-06T12:52:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:33.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:33.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.771+0000 7fd7f759e700 1 Processor -- start 2023-10-06T12:52:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.771+0000 7fd7f759e700 1 -- start start 2023-10-06T12:52:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f759e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7f8156e20 0x7fd7f806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f759e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd7f8067760 0x7fd7f8068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f759e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f806d790 con 0x7fd7f80aaf30 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f759e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80acd60 con 0x7fd7f8156e20 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f759e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80ad040 con 0x7fd7f8067760 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd7f8156e20 0x7fd7f806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7edd9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7f80aaf30 0x7fd7f8067050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36236/0 (socket says 172.21.15.46:36236) 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7edd9b700 1 -- 172.21.15.46:0/3051015240 learned_addr learned my addr 172.21.15.46:0/3051015240 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f6d9d700 1 --2- 172.21.15.46:0/3051015240 >> v2:172.21.15.132:3300/0 conn(0x7fd7f8067760 0x7fd7f8068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f5d9b700 1 -- 172.21.15.46:0/3051015240 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207463625 0 0) 0x7fd7f806d790 con 0x7fd7f80aaf30 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f5d9b700 1 -- 172.21.15.46:0/3051015240 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e4003570 con 0x7fd7f80aaf30 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 -- 172.21.15.46:0/3051015240 >> v2:172.21.15.132:3300/0 conn(0x7fd7f8067760 msgr2=0x7fd7f8068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 --2- 172.21.15.46:0/3051015240 >> v2:172.21.15.132:3300/0 conn(0x7fd7f8067760 0x7fd7f8068a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 -- 172.21.15.46:0/3051015240 >> v1:172.21.15.46:6789/0 conn(0x7fd7f80aaf30 legacy=0x7fd7f8067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 -- 172.21.15.46:0/3051015240 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80ad320 con 0x7fd7f8156e20 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f659c700 1 --2- 172.21.15.46:0/3051015240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f806d250 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80c5f00 tx=0x7fd7e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4e3fd18d47abea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f5d9b700 1 -- 172.21.15.46:0/3051015240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8011630 con 0x7fd7f8156e20 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f5d9b700 1 -- 172.21.15.46:0/3051015240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8011790 con 0x7fd7f8156e20 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.775+0000 7fd7f5d9b700 1 -- 172.21.15.46:0/3051015240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8011b30 con 0x7fd7f8156e20 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- 172.21.15.46:0/3051015240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 msgr2=0x7fd7f806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 --2- 172.21.15.46:0/3051015240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f806d250 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80c5f00 tx=0x7fd7e80091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- 172.21.15.46:0/3051015240 shutdown_connections 2023-10-06T12:52:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 --2- 172.21.15.46:0/3051015240 >> v2:172.21.15.132:3300/0 conn(0x7fd7f8067760 0x7fd7f8068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 --2- 172.21.15.46:0/3051015240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f806d250 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- 172.21.15.46:0/3051015240 >> 172.21.15.46:0/3051015240 conn(0x7fd7f8062ef0 msgr2=0x7fd7f80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- 172.21.15.46:0/3051015240 shutdown_connections 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- 172.21.15.46:0/3051015240 wait complete. 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 Processor -- start 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.779+0000 7fd7f759e700 1 -- start start 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7f8067760 0x7fd7f8168d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f80aaf30 0x7fd7f816cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8169620 con 0x7fd7f8156e20 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8169900 con 0x7fd7f80aaf30 2023-10-06T12:52:33.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8169be0 con 0x7fd7f8067760 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50706/0 (socket says 172.21.15.46:50706) 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 -- 172.21.15.46:0/328391858 learned_addr learned my addr 172.21.15.46:0/328391858 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7edd9b700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f80aaf30 0x7fd7f816cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f659c700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7f8067760 0x7fd7f8168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 -- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7f8067760 msgr2=0x7fd7f8168d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7f8067760 0x7fd7f8168d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 -- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f80aaf30 msgr2=0x7fd7f816cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f80aaf30 0x7fd7f816cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 -- 172.21.15.46:0/328391858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009420 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f6d9d700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000ea50 tx=0x7fd7f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000ee90 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f000c850 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f816c320 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000cc10 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.783+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81692a0 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.787+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8064590 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.787+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd7f0013470 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.791+0000 7fd7eeffd700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 0x7fd7d8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.791+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fd7f007cf00 con 0x7fd7f8156e20 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.791+0000 7fd7f659c700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 0x7fd7d8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:33.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.791+0000 7fd7f659c700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 0x7fd7d8062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80030c0 tx=0x7fd7e8013000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:33.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.795+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7f004cfa0 con 0x7fd7f8156e20 2023-10-06T12:52:33.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.967+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f80ad600 con 0x7fd7f8156e20 2023-10-06T12:52:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.971+0000 7fd7eeffd700 1 -- 172.21.15.46:0/328391858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+17758 (secure 0 0 0) 0x7fd7f00505c0 con 0x7fd7f8156e20 2023-10-06T12:52:33.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:33.977 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":197,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:28.983615+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 msgr2=0x7fd7d8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 0x7fd7d8062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80030c0 tx=0x7fd7e8013000 comp rx=0 tx=0).stop 2023-10-06T12:52:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 msgr2=0x7fd7f80a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000ea50 tx=0x7fd7f00091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 shutdown_connections 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7f8156e20 0x7fd7f80a9a80 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd7d8060670 0x7fd7d8062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7f80aaf30 0x7fd7f816cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 --2- 172.21.15.46:0/328391858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7f8067760 0x7fd7f8168d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 >> 172.21.15.46:0/328391858 conn(0x7fd7f8062ef0 msgr2=0x7fd7f80c4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 shutdown_connections 2023-10-06T12:52:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:33.975+0000 7fd7f759e700 1 -- 172.21.15.46:0/328391858 wait complete. 2023-10-06T12:52:34.001 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T12:52:34.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:52:34.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.819+0000 7fd884e8e700 1 Processor -- start 2023-10-06T12:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.819+0000 7fd884e8e700 1 -- start start 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd884e8e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8800aaf30 0x7fd880067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd884e8e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd880067590 0x7fd880068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd884e8e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8800679f0 con 0x7fd880156e20 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd884e8e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800acd60 con 0x7fd880067590 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd884e8e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800ad040 con 0x7fd8800aaf30 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd880067590 0x7fd880068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd880156e20 0x7fd88006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36250/0 (socket says 172.21.15.46:36250) 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87f7fe700 1 -- 172.21.15.46:0/3820789829 learned_addr learned my addr 172.21.15.46:0/3820789829 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd880067590 0x7fd880068a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50708/0 (socket says 172.21.15.46:50708) 2023-10-06T12:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd876ffd700 1 --2- 172.21.15.46:0/3820789829 >> v2:172.21.15.132:3300/0 conn(0x7fd8800aaf30 0x7fd880067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 -- 172.21.15.46:0/3820789829 >> v2:172.21.15.132:3300/0 conn(0x7fd8800aaf30 msgr2=0x7fd880067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 --2- 172.21.15.46:0/3820789829 >> v2:172.21.15.132:3300/0 conn(0x7fd8800aaf30 0x7fd880067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 -- 172.21.15.46:0/3820789829 >> v1:172.21.15.46:6789/0 conn(0x7fd880156e20 legacy=0x7fd88006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 -- 172.21.15.46:0/3820789829 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800ad320 con 0x7fd880067590 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87ffff700 1 --2- 172.21.15.46:0/3820789829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880067590 0x7fd880068a20 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd8800c2b50 tx=0x7fd8780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a082dc417ac36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87effd700 1 -- 172.21.15.46:0/3820789829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011630 con 0x7fd880067590 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87effd700 1 -- 172.21.15.46:0/3820789829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878011790 con 0x7fd880067590 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.823+0000 7fd87effd700 1 -- 172.21.15.46:0/3820789829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011b30 con 0x7fd880067590 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 -- 172.21.15.46:0/3820789829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880067590 msgr2=0x7fd880068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 --2- 172.21.15.46:0/3820789829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880067590 0x7fd880068a20 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd8800c2b50 tx=0x7fd8780091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 -- 172.21.15.46:0/3820789829 shutdown_connections 2023-10-06T12:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 --2- 172.21.15.46:0/3820789829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880067590 0x7fd880068a20 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 --2- 172.21.15.46:0/3820789829 >> v2:172.21.15.132:3300/0 conn(0x7fd8800aaf30 0x7fd880067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 -- 172.21.15.46:0/3820789829 >> 172.21.15.46:0/3820789829 conn(0x7fd880062ef0 msgr2=0x7fd8800cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 -- 172.21.15.46:0/3820789829 shutdown_connections 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 -- 172.21.15.46:0/3820789829 wait complete. 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.827+0000 7fd884e8e700 1 Processor -- start 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 -- start start 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 0x7fd88015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8800aaf30 0x7fd880166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 0x7fd8800c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801580e0 con 0x7fd880156e20 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88015c800 con 0x7fd880067590 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd884e8e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd880157560 con 0x7fd8800aaf30 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 0x7fd88015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 0x7fd88015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54292/0 (socket says 172.21.15.46:54292) 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd87f7fe700 1 -- 172.21.15.46:0/1713209824 learned_addr learned my addr 172.21.15.46:0/1713209824 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd87ffff700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 0x7fd8800c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.831+0000 7fd876ffd700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8800aaf30 0x7fd880166ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 -- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8800aaf30 msgr2=0x7fd880166ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8800aaf30 0x7fd880166ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.862 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 -- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 msgr2=0x7fd88015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 0x7fd88015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 -- 172.21.15.46:0/1713209824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009420 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.835+0000 7fd87ffff700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 0x7fd8800c4870 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd878016000 tx=0x7fd878017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011b10 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780253f0 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780257b0 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880157840 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880157e00 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.839+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880064590 con 0x7fd880156e20 2023-10-06T12:52:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.843+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fd8780193b0 con 0x7fd880156e20 2023-10-06T12:52:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.843+0000 7fd877fff700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 0x7fd860062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.843+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fd87807e900 con 0x7fd880156e20 2023-10-06T12:52:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.843+0000 7fd87f7fe700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 0x7fd860062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.843+0000 7fd87f7fe700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 0x7fd860062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:34.851+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd87804e9a0 con 0x7fd880156e20 2023-10-06T12:52:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.027+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8800ad600 con 0x7fd860060670 2023-10-06T12:52:35.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.047+0000 7fd877fff700 1 -- 172.21.15.46:0/1713209824 <== mgr.10158 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308891 (secure 0 0 0) 0x7fd8800ad600 con 0x7fd860060670 2023-10-06T12:52:35.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 msgr2=0x7fd860062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 0x7fd860062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).stop 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 msgr2=0x7fd8800c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 0x7fd8800c4870 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd878016000 tx=0x7fd878017000 comp rx=0 tx=0).stop 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 shutdown_connections 2023-10-06T12:52:35.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd880156e20 0x7fd8800c4870 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8800aaf30 0x7fd880166ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd860060670 0x7fd860062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 --2- 172.21.15.46:0/1713209824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd880067590 0x7fd88015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 >> 172.21.15.46:0/1713209824 conn(0x7fd880062ef0 msgr2=0x7fd880068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 shutdown_connections 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.055+0000 7fd884e8e700 1 -- 172.21.15.46:0/1713209824 wait complete. 2023-10-06T12:52:35.219 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:52:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T12:52:34.883751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":624,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456168,"kb_used_data":20520,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752518552,"statfs":{"total":773094113280,"available":770578997248,"internally_reserved":0,"allocated":21012480,"data_stored":11697644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.905503"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016956+0000","last_change":"2023-10-06T12:52:28.011464+0000","last_active":"2023-10-06T12:52:28.016956+0000","last_peered":"2023-10-06T12:52:28.016956+0000","last_clean":"2023-10-06T12:52:28.016956+0000","last_became_active":"2023-10-06T12:52:28.011035+0000","last_became_peered":"2023-10-06T12:52:28.011035+0000","last_unstale":"2023-10-06T12:52:28.016956+0000","last_undegraded":"2023-10-06T12:52:28.016956+0000","last_fullsized":"2023-10-06T12:52:28.016956+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:38.275526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012713+0000","last_change":"2023-10-06T12:52:28.008738+0000","last_active":"2023-10-06T12:52:28.012713+0000","last_peered":"2023-10-06T12:52:28.012713+0000","last_clean":"2023-10-06T12:52:28.012713+0000","last_became_active":"2023-10-06T12:52:28.008427+0000","last_became_peered":"2023-10-06T12:52:28.008427+0000","last_unstale":"2023-10-06T12:52:28.012713+0000","last_undegraded":"2023-10-06T12:52:28.012713+0000","last_fullsized":"2023-10-06T12:52:28.012713+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:24.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023148+0000","last_change":"2023-10-06T12:52:28.022490+0000","last_active":"2023-10-06T12:52:28.023148+0000","last_peered":"2023-10-06T12:52:28.023148+0000","last_clean":"2023-10-06T12:52:28.023148+0000","last_became_active":"2023-10-06T12:52:28.022155+0000","last_became_peered":"2023-10-06T12:52:28.022155+0000","last_unstale":"2023-10-06T12:52:28.023148+0000","last_undegraded":"2023-10-06T12:52:28.023148+0000","last_fullsized":"2023-10-06T12:52:28.023148+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:18.478057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012276+0000","last_change":"2023-10-06T12:52:28.004727+0000","last_active":"2023-10-06T12:52:28.012276+0000","last_peered":"2023-10-06T12:52:28.012276+0000","last_clean":"2023-10-06T12:52:28.012276+0000","last_became_active":"2023-10-06T12:52:28.003708+0000","last_became_peered":"2023-10-06T12:52:28.003708+0000","last_unstale":"2023-10-06T12:52:28.012276+0000","last_undegraded":"2023-10-06T12:52:28.012276+0000","last_fullsized":"2023-10-06T12:52:28.012276+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:18.319135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.004458+0000","last_change":"2023-10-06T12:52:27.996748+0000","last_active":"2023-10-06T12:52:28.004458+0000","last_peered":"2023-10-06T12:52:28.004458+0000","last_clean":"2023-10-06T12:52:28.004458+0000","last_became_active":"2023-10-06T12:52:27.996425+0000","last_became_peered":"2023-10-06T12:52:27.996425+0000","last_unstale":"2023-10-06T12:52:28.004458+0000","last_undegraded":"2023-10-06T12:52:28.004458+0000","last_fullsized":"2023-10-06T12:52:28.004458+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:37.531390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.014019+0000","last_change":"2023-10-06T12:52:28.011794+0000","last_active":"2023-10-06T12:52:28.014019+0000","last_peered":"2023-10-06T12:52:28.014019+0000","last_clean":"2023-10-06T12:52:28.014019+0000","last_became_active":"2023-10-06T12:52:28.011430+0000","last_became_peered":"2023-10-06T12:52:28.011430+0000","last_unstale":"2023-10-06T12:52:28.014019+0000","last_undegraded":"2023-10-06T12:52:28.014019+0000","last_fullsized":"2023-10-06T12:52:28.014019+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.968731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021491+0000","last_change":"2023-10-06T12:52:28.015401+0000","last_active":"2023-10-06T12:52:28.021491+0000","last_peered":"2023-10-06T12:52:28.021491+0000","last_clean":"2023-10-06T12:52:28.021491+0000","last_became_active":"2023-10-06T12:52:28.015093+0000","last_became_peered":"2023-10-06T12:52:28.015093+0000","last_unstale":"2023-10-06T12:52:28.021491+0000","last_undegraded":"2023-10-06T12:52:28.021491+0000","last_fullsized":"2023-10-06T12:52:28.021491+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:48.150210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012346+0000","last_change":"2023-10-06T12:52:28.009183+0000","last_active":"2023-10-06T12:52:28.012346+0000","last_peered":"2023-10-06T12:52:28.012346+0000","last_clean":"2023-10-06T12:52:28.012346+0000","last_became_active":"2023-10-06T12:52:28.008689+0000","last_became_peered":"2023-10-06T12:52:28.008689+0000","last_unstale":"2023-10-06T12:52:28.012346+0000","last_undegraded":"2023-10-06T12:52:28.012346+0000","last_fullsized":"2023-10-06T12:52:28.012346+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.009259+0000","last_change":"2023-10-06T12:52:28.005784+0000","last_active":"2023-10-06T12:52:28.009259+0000","last_peered":"2023-10-06T12:52:28.009259+0000","last_clean":"2023-10-06T12:52:28.009259+0000","last_became_active":"2023-10-06T12:52:28.005497+0000","last_became_peered":"2023-10-06T12:52:28.005497+0000","last_unstale":"2023-10-06T12:52:28.009259+0000","last_undegraded":"2023-10-06T12:52:28.009259+0000","last_fullsized":"2023-10-06T12:52:28.009259+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:40:14.695497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007678+0000","last_change":"2023-10-06T12:52:28.002130+0000","last_active":"2023-10-06T12:52:28.007678+0000","last_peered":"2023-10-06T12:52:28.007678+0000","last_clean":"2023-10-06T12:52:28.007678+0000","last_became_active":"2023-10-06T12:52:28.001818+0000","last_became_peered":"2023-10-06T12:52:28.001818+0000","last_unstale":"2023-10-06T12:52:28.007678+0000","last_undegraded":"2023-10-06T12:52:28.007678+0000","last_fullsized":"2023-10-06T12:52:28.007678+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:02.252564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010942+0000","last_change":"2023-10-06T12:52:28.006710+0000","last_active":"2023-10-06T12:52:28.010942+0000","last_peered":"2023-10-06T12:52:28.010942+0000","last_clean":"2023-10-06T12:52:28.010942+0000","last_became_active":"2023-10-06T12:52:28.006373+0000","last_became_peered":"2023-10-06T12:52:28.006373+0000","last_unstale":"2023-10-06T12:52:28.010942+0000","last_undegraded":"2023-10-06T12:52:28.010942+0000","last_fullsized":"2023-10-06T12:52:28.010942+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:38.406215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017693+0000","last_change":"2023-10-06T12:52:28.014087+0000","last_active":"2023-10-06T12:52:28.017693+0000","last_peered":"2023-10-06T12:52:28.017693+0000","last_clean":"2023-10-06T12:52:28.017693+0000","last_became_active":"2023-10-06T12:52:28.013702+0000","last_became_peered":"2023-10-06T12:52:28.013702+0000","last_unstale":"2023-10-06T12:52:28.017693+0000","last_undegraded":"2023-10-06T12:52:28.017693+0000","last_fullsized":"2023-10-06T12:52:28.017693+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:59:11.161220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016865+0000","last_change":"2023-10-06T12:52:28.010794+0000","last_active":"2023-10-06T12:52:28.016865+0000","last_peered":"2023-10-06T12:52:28.016865+0000","last_clean":"2023-10-06T12:52:28.016865+0000","last_became_active":"2023-10-06T12:52:28.010511+0000","last_became_peered":"2023-10-06T12:52:28.010511+0000","last_unstale":"2023-10-06T12:52:28.016865+0000","last_undegraded":"2023-10-06T12:52:28.016865+0000","last_fullsized":"2023-10-06T12:52:28.016865+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.034573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021960+0000","last_change":"2023-10-06T12:52:28.018712+0000","last_active":"2023-10-06T12:52:28.021960+0000","last_peered":"2023-10-06T12:52:28.021960+0000","last_clean":"2023-10-06T12:52:28.021960+0000","last_became_active":"2023-10-06T12:52:28.018402+0000","last_became_peered":"2023-10-06T12:52:28.018402+0000","last_unstale":"2023-10-06T12:52:28.021960+0000","last_undegraded":"2023-10-06T12:52:28.021960+0000","last_fullsized":"2023-10-06T12:52:28.021960+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.018526+0000","last_change":"2023-10-06T12:52:28.016694+0000","last_active":"2023-10-06T12:52:28.018526+0000","last_peered":"2023-10-06T12:52:28.018526+0000","last_clean":"2023-10-06T12:52:28.018526+0000","last_became_active":"2023-10-06T12:52:28.015994+0000","last_became_peered":"2023-10-06T12:52:28.015994+0000","last_unstale":"2023-10-06T12:52:28.018526+0000","last_undegraded":"2023-10-06T12:52:28.018526+0000","last_fullsized":"2023-10-06T12:52:28.018526+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:39.665430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008109+0000","last_change":"2023-10-06T12:52:27.997222+0000","last_active":"2023-10-06T12:52:28.008109+0000","last_peered":"2023-10-06T12:52:28.008109+0000","last_clean":"2023-10-06T12:52:28.008109+0000","last_became_active":"2023-10-06T12:52:27.996744+0000","last_became_peered":"2023-10-06T12:52:27.996744+0000","last_unstale":"2023-10-06T12:52:28.008109+0000","last_undegraded":"2023-10-06T12:52:28.008109+0000","last_fullsized":"2023-10-06T12:52:28.008109+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:17:40.904397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013713+0000","last_change":"2023-10-06T12:52:28.011601+0000","last_active":"2023-10-06T12:52:28.013713+0000","last_peered":"2023-10-06T12:52:28.013713+0000","last_clean":"2023-10-06T12:52:28.013713+0000","last_became_active":"2023-10-06T12:52:28.011154+0000","last_became_peered":"2023-10-06T12:52:28.011154+0000","last_unstale":"2023-10-06T12:52:28.013713+0000","last_undegraded":"2023-10-06T12:52:28.013713+0000","last_fullsized":"2023-10-06T12:52:28.013713+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:41:20.901664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015747+0000","last_change":"2023-10-06T12:52:28.005775+0000","last_active":"2023-10-06T12:52:28.015747+0000","last_peered":"2023-10-06T12:52:28.015747+0000","last_clean":"2023-10-06T12:52:28.015747+0000","last_became_active":"2023-10-06T12:52:28.005490+0000","last_became_peered":"2023-10-06T12:52:28.005490+0000","last_unstale":"2023-10-06T12:52:28.015747+0000","last_undegraded":"2023-10-06T12:52:28.015747+0000","last_fullsized":"2023-10-06T12:52:28.015747+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:17:40.904397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.025963+0000","last_change":"2023-10-06T12:52:28.025040+0000","last_active":"2023-10-06T12:52:28.025963+0000","last_peered":"2023-10-06T12:52:28.025963+0000","last_clean":"2023-10-06T12:52:28.025963+0000","last_became_active":"2023-10-06T12:52:28.024822+0000","last_became_peered":"2023-10-06T12:52:28.024822+0000","last_unstale":"2023-10-06T12:52:28.025963+0000","last_undegraded":"2023-10-06T12:52:28.025963+0000","last_fullsized":"2023-10-06T12:52:28.025963+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:37.531390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012426+0000","last_change":"2023-10-06T12:52:28.010770+0000","last_active":"2023-10-06T12:52:28.012426+0000","last_peered":"2023-10-06T12:52:28.012426+0000","last_clean":"2023-10-06T12:52:28.012426+0000","last_became_active":"2023-10-06T12:52:28.010467+0000","last_became_peered":"2023-10-06T12:52:28.010467+0000","last_unstale":"2023-10-06T12:52:28.012426+0000","last_undegraded":"2023-10-06T12:52:28.012426+0000","last_fullsized":"2023-10-06T12:52:28.012426+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:29:08.634324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012753+0000","last_change":"2023-10-06T12:52:28.007917+0000","last_active":"2023-10-06T12:52:28.012753+0000","last_peered":"2023-10-06T12:52:28.012753+0000","last_clean":"2023-10-06T12:52:28.012753+0000","last_became_active":"2023-10-06T12:52:28.007499+0000","last_became_peered":"2023-10-06T12:52:28.007499+0000","last_unstale":"2023-10-06T12:52:28.012753+0000","last_undegraded":"2023-10-06T12:52:28.012753+0000","last_fullsized":"2023-10-06T12:52:28.012753+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:35.009453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013012+0000","last_change":"2023-10-06T12:52:28.008709+0000","last_active":"2023-10-06T12:52:28.013012+0000","last_peered":"2023-10-06T12:52:28.013012+0000","last_clean":"2023-10-06T12:52:28.013012+0000","last_became_active":"2023-10-06T12:52:28.008295+0000","last_became_peered":"2023-10-06T12:52:28.008295+0000","last_unstale":"2023-10-06T12:52:28.013012+0000","last_undegraded":"2023-10-06T12:52:28.013012+0000","last_fullsized":"2023-10-06T12:52:28.013012+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.297245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.018042+0000","last_change":"2023-10-06T12:52:28.014967+0000","last_active":"2023-10-06T12:52:28.018042+0000","last_peered":"2023-10-06T12:52:28.018042+0000","last_clean":"2023-10-06T12:52:28.018042+0000","last_became_active":"2023-10-06T12:52:28.011920+0000","last_became_peered":"2023-10-06T12:52:28.011920+0000","last_unstale":"2023-10-06T12:52:28.018042+0000","last_undegraded":"2023-10-06T12:52:28.018042+0000","last_fullsized":"2023-10-06T12:52:28.018042+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:02.453956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015994+0000","last_change":"2023-10-06T12:52:28.011068+0000","last_active":"2023-10-06T12:52:28.015994+0000","last_peered":"2023-10-06T12:52:28.015994+0000","last_clean":"2023-10-06T12:52:28.015994+0000","last_became_active":"2023-10-06T12:52:28.010749+0000","last_became_peered":"2023-10-06T12:52:28.010749+0000","last_unstale":"2023-10-06T12:52:28.015994+0000","last_undegraded":"2023-10-06T12:52:28.015994+0000","last_fullsized":"2023-10-06T12:52:28.015994+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:48.150210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.024831+0000","last_change":"2023-10-06T12:52:28.021964+0000","last_active":"2023-10-06T12:52:28.024831+0000","last_peered":"2023-10-06T12:52:28.024831+0000","last_clean":"2023-10-06T12:52:28.024831+0000","last_became_active":"2023-10-06T12:52:28.018176+0000","last_became_peered":"2023-10-06T12:52:28.018176+0000","last_unstale":"2023-10-06T12:52:28.024831+0000","last_undegraded":"2023-10-06T12:52:28.024831+0000","last_fullsized":"2023-10-06T12:52:28.024831+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008020+0000","last_change":"2023-10-06T12:52:28.001401+0000","last_active":"2023-10-06T12:52:28.008020+0000","last_peered":"2023-10-06T12:52:28.008020+0000","last_clean":"2023-10-06T12:52:28.008020+0000","last_became_active":"2023-10-06T12:52:28.000740+0000","last_became_peered":"2023-10-06T12:52:28.000740+0000","last_unstale":"2023-10-06T12:52:28.008020+0000","last_undegraded":"2023-10-06T12:52:28.008020+0000","last_fullsized":"2023-10-06T12:52:28.008020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:42.126935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023908+0000","last_change":"2023-10-06T12:52:28.022816+0000","last_active":"2023-10-06T12:52:28.023908+0000","last_peered":"2023-10-06T12:52:28.023908+0000","last_clean":"2023-10-06T12:52:28.023908+0000","last_became_active":"2023-10-06T12:52:28.022513+0000","last_became_peered":"2023-10-06T12:52:28.022513+0000","last_unstale":"2023-10-06T12:52:28.023908+0000","last_undegraded":"2023-10-06T12:52:28.023908+0000","last_fullsized":"2023-10-06T12:52:28.023908+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:39.150778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007771+0000","last_change":"2023-10-06T12:52:27.997436+0000","last_active":"2023-10-06T12:52:28.007771+0000","last_peered":"2023-10-06T12:52:28.007771+0000","last_clean":"2023-10-06T12:52:28.007771+0000","last_became_active":"2023-10-06T12:52:27.997101+0000","last_became_peered":"2023-10-06T12:52:27.997101+0000","last_unstale":"2023-10-06T12:52:28.007771+0000","last_undegraded":"2023-10-06T12:52:28.007771+0000","last_fullsized":"2023-10-06T12:52:28.007771+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:27.981585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015581+0000","last_change":"2023-10-06T12:52:28.011098+0000","last_active":"2023-10-06T12:52:28.015581+0000","last_peered":"2023-10-06T12:52:28.015581+0000","last_clean":"2023-10-06T12:52:28.015581+0000","last_became_active":"2023-10-06T12:52:28.009726+0000","last_became_peered":"2023-10-06T12:52:28.009726+0000","last_unstale":"2023-10-06T12:52:28.015581+0000","last_undegraded":"2023-10-06T12:52:28.015581+0000","last_fullsized":"2023-10-06T12:52:28.015581+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:44.832554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017040+0000","last_change":"2023-10-06T12:52:28.015479+0000","last_active":"2023-10-06T12:52:28.017040+0000","last_peered":"2023-10-06T12:52:28.017040+0000","last_clean":"2023-10-06T12:52:28.017040+0000","last_became_active":"2023-10-06T12:52:28.012607+0000","last_became_peered":"2023-10-06T12:52:28.012607+0000","last_unstale":"2023-10-06T12:52:28.017040+0000","last_undegraded":"2023-10-06T12:52:28.017040+0000","last_fullsized":"2023-10-06T12:52:28.017040+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:53.868689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023944+0000","last_change":"2023-10-06T12:52:28.020348+0000","last_active":"2023-10-06T12:52:28.023944+0000","last_peered":"2023-10-06T12:52:28.023944+0000","last_clean":"2023-10-06T12:52:28.023944+0000","last_became_active":"2023-10-06T12:52:28.018619+0000","last_became_peered":"2023-10-06T12:52:28.018619+0000","last_unstale":"2023-10-06T12:52:28.023944+0000","last_undegraded":"2023-10-06T12:52:28.023944+0000","last_fullsized":"2023-10-06T12:52:28.023944+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:39.150778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.004492+0000","last_change":"2023-10-06T12:52:27.994799+0000","last_active":"2023-10-06T12:52:28.004492+0000","last_peered":"2023-10-06T12:52:28.004492+0000","last_clean":"2023-10-06T12:52:28.004492+0000","last_became_active":"2023-10-06T12:52:27.994433+0000","last_became_peered":"2023-10-06T12:52:27.994433+0000","last_unstale":"2023-10-06T12:52:28.004492+0000","last_undegraded":"2023-10-06T12:52:28.004492+0000","last_fullsized":"2023-10-06T12:52:28.004492+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:16.163073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017871+0000","last_change":"2023-10-06T12:52:28.012749+0000","last_active":"2023-10-06T12:52:28.017871+0000","last_peered":"2023-10-06T12:52:28.017871+0000","last_clean":"2023-10-06T12:52:28.017871+0000","last_became_active":"2023-10-06T12:52:28.012021+0000","last_became_peered":"2023-10-06T12:52:28.012021+0000","last_unstale":"2023-10-06T12:52:28.017871+0000","last_undegraded":"2023-10-06T12:52:28.017871+0000","last_fullsized":"2023-10-06T12:52:28.017871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.989163+0000","last_change":"2023-10-06T12:52:27.008937+0000","last_active":"2023-10-06T12:52:27.989163+0000","last_peered":"2023-10-06T12:52:27.989163+0000","last_clean":"2023-10-06T12:52:27.989163+0000","last_became_active":"2023-10-06T12:52:27.008653+0000","last_became_peered":"2023-10-06T12:52:27.008653+0000","last_unstale":"2023-10-06T12:52:27.989163+0000","last_undegraded":"2023-10-06T12:52:27.989163+0000","last_fullsized":"2023-10-06T12:52:27.989163+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:50.245593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007774+0000","last_change":"2023-10-06T12:52:28.003244+0000","last_active":"2023-10-06T12:52:28.007774+0000","last_peered":"2023-10-06T12:52:28.007774+0000","last_clean":"2023-10-06T12:52:28.007774+0000","last_became_active":"2023-10-06T12:52:28.002941+0000","last_became_peered":"2023-10-06T12:52:28.002941+0000","last_unstale":"2023-10-06T12:52:28.007774+0000","last_undegraded":"2023-10-06T12:52:28.007774+0000","last_fullsized":"2023-10-06T12:52:28.007774+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:17.957852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998316+0000","last_change":"2023-10-06T12:52:27.011668+0000","last_active":"2023-10-06T12:52:27.998316+0000","last_peered":"2023-10-06T12:52:27.998316+0000","last_clean":"2023-10-06T12:52:27.998316+0000","last_became_active":"2023-10-06T12:52:27.006682+0000","last_became_peered":"2023-10-06T12:52:27.006682+0000","last_unstale":"2023-10-06T12:52:27.998316+0000","last_undegraded":"2023-10-06T12:52:27.998316+0000","last_fullsized":"2023-10-06T12:52:27.998316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:13.836962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010548+0000","last_change":"2023-10-06T12:52:28.003701+0000","last_active":"2023-10-06T12:52:28.010548+0000","last_peered":"2023-10-06T12:52:28.010548+0000","last_clean":"2023-10-06T12:52:28.010548+0000","last_became_active":"2023-10-06T12:52:28.003402+0000","last_became_peered":"2023-10-06T12:52:28.003402+0000","last_unstale":"2023-10-06T12:52:28.010548+0000","last_undegraded":"2023-10-06T12:52:28.010548+0000","last_fullsized":"2023-10-06T12:52:28.010548+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:27.981585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"196'3","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033069+0000","last_change":"2023-10-06T12:52:26.999796+0000","last_active":"2023-10-06T12:52:28.033069+0000","last_peered":"2023-10-06T12:52:28.033069+0000","last_clean":"2023-10-06T12:52:28.033069+0000","last_became_active":"2023-10-06T12:52:26.999486+0000","last_became_peered":"2023-10-06T12:52:26.999486+0000","last_unstale":"2023-10-06T12:52:28.033069+0000","last_undegraded":"2023-10-06T12:52:28.033069+0000","last_fullsized":"2023-10-06T12:52:28.033069+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:56.790454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012790+0000","last_change":"2023-10-06T12:52:28.008114+0000","last_active":"2023-10-06T12:52:28.012790+0000","last_peered":"2023-10-06T12:52:28.012790+0000","last_clean":"2023-10-06T12:52:28.012790+0000","last_became_active":"2023-10-06T12:52:28.007784+0000","last_became_peered":"2023-10-06T12:52:28.007784+0000","last_unstale":"2023-10-06T12:52:28.012790+0000","last_undegraded":"2023-10-06T12:52:28.012790+0000","last_fullsized":"2023-10-06T12:52:28.012790+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:57.287695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"196'6","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.037986+0000","last_change":"2023-10-06T12:52:26.998830+0000","last_active":"2023-10-06T12:52:28.037986+0000","last_peered":"2023-10-06T12:52:28.037986+0000","last_clean":"2023-10-06T12:52:28.037986+0000","last_became_active":"2023-10-06T12:52:26.998528+0000","last_became_peered":"2023-10-06T12:52:26.998528+0000","last_unstale":"2023-10-06T12:52:28.037986+0000","last_undegraded":"2023-10-06T12:52:28.037986+0000","last_fullsized":"2023-10-06T12:52:28.037986+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:37.259985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.020883+0000","last_change":"2023-10-06T12:52:28.015945+0000","last_active":"2023-10-06T12:52:28.020883+0000","last_peered":"2023-10-06T12:52:28.020883+0000","last_clean":"2023-10-06T12:52:28.020883+0000","last_became_active":"2023-10-06T12:52:28.015654+0000","last_became_peered":"2023-10-06T12:52:28.015654+0000","last_unstale":"2023-10-06T12:52:28.020883+0000","last_undegraded":"2023-10-06T12:52:28.020883+0000","last_fullsized":"2023-10-06T12:52:28.020883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.297245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.035282+0000","last_change":"2023-10-06T12:52:27.009426+0000","last_active":"2023-10-06T12:52:28.035282+0000","last_peered":"2023-10-06T12:52:28.035282+0000","last_clean":"2023-10-06T12:52:28.035282+0000","last_became_active":"2023-10-06T12:52:27.009071+0000","last_became_peered":"2023-10-06T12:52:27.009071+0000","last_unstale":"2023-10-06T12:52:28.035282+0000","last_undegraded":"2023-10-06T12:52:28.035282+0000","last_fullsized":"2023-10-06T12:52:28.035282+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:45.701372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012457+0000","last_change":"2023-10-06T12:52:28.010409+0000","last_active":"2023-10-06T12:52:28.012457+0000","last_peered":"2023-10-06T12:52:28.012457+0000","last_clean":"2023-10-06T12:52:28.012457+0000","last_became_active":"2023-10-06T12:52:28.010074+0000","last_became_peered":"2023-10-06T12:52:28.010074+0000","last_unstale":"2023-10-06T12:52:28.012457+0000","last_undegraded":"2023-10-06T12:52:28.012457+0000","last_fullsized":"2023-10-06T12:52:28.012457+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:46.487911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000567+0000","last_change":"2023-10-06T12:52:27.011205+0000","last_active":"2023-10-06T12:52:28.000567+0000","last_peered":"2023-10-06T12:52:28.000567+0000","last_clean":"2023-10-06T12:52:28.000567+0000","last_became_active":"2023-10-06T12:52:27.008935+0000","last_became_peered":"2023-10-06T12:52:27.008935+0000","last_unstale":"2023-10-06T12:52:28.000567+0000","last_undegraded":"2023-10-06T12:52:28.000567+0000","last_fullsized":"2023-10-06T12:52:28.000567+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:31.450674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.014730+0000","last_change":"2023-10-06T12:52:28.004902+0000","last_active":"2023-10-06T12:52:28.014730+0000","last_peered":"2023-10-06T12:52:28.014730+0000","last_clean":"2023-10-06T12:52:28.014730+0000","last_became_active":"2023-10-06T12:52:28.004166+0000","last_became_peered":"2023-10-06T12:52:28.004166+0000","last_unstale":"2023-10-06T12:52:28.014730+0000","last_undegraded":"2023-10-06T12:52:28.014730+0000","last_fullsized":"2023-10-06T12:52:28.014730+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:55.552335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000298+0000","last_change":"2023-10-06T12:52:27.003933+0000","last_active":"2023-10-06T12:52:28.000298+0000","last_peered":"2023-10-06T12:52:28.000298+0000","last_clean":"2023-10-06T12:52:28.000298+0000","last_became_active":"2023-10-06T12:52:27.003656+0000","last_became_peered":"2023-10-06T12:52:27.003656+0000","last_unstale":"2023-10-06T12:52:28.000298+0000","last_undegraded":"2023-10-06T12:52:28.000298+0000","last_fullsized":"2023-10-06T12:52:28.000298+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:38.856122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017554+0000","last_change":"2023-10-06T12:52:28.014462+0000","last_active":"2023-10-06T12:52:28.017554+0000","last_peered":"2023-10-06T12:52:28.017554+0000","last_clean":"2023-10-06T12:52:28.017554+0000","last_became_active":"2023-10-06T12:52:28.014140+0000","last_became_peered":"2023-10-06T12:52:28.014140+0000","last_unstale":"2023-10-06T12:52:28.017554+0000","last_undegraded":"2023-10-06T12:52:28.017554+0000","last_fullsized":"2023-10-06T12:52:28.017554+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:41:20.901664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"196'8","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.034909+0000","last_change":"2023-10-06T12:52:27.000573+0000","last_active":"2023-10-06T12:52:28.034909+0000","last_peered":"2023-10-06T12:52:28.034909+0000","last_clean":"2023-10-06T12:52:28.034909+0000","last_became_active":"2023-10-06T12:52:27.000257+0000","last_became_peered":"2023-10-06T12:52:27.000257+0000","last_unstale":"2023-10-06T12:52:28.034909+0000","last_undegraded":"2023-10-06T12:52:28.034909+0000","last_fullsized":"2023-10-06T12:52:28.034909+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:01.237024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.024050+0000","last_change":"2023-10-06T12:52:28.021092+0000","last_active":"2023-10-06T12:52:28.024050+0000","last_peered":"2023-10-06T12:52:28.024050+0000","last_clean":"2023-10-06T12:52:28.024050+0000","last_became_active":"2023-10-06T12:52:28.020756+0000","last_became_peered":"2023-10-06T12:52:28.020756+0000","last_unstale":"2023-10-06T12:52:28.024050+0000","last_undegraded":"2023-10-06T12:52:28.024050+0000","last_fullsized":"2023-10-06T12:52:28.024050+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033874+0000","last_change":"2023-10-06T12:52:27.000614+0000","last_active":"2023-10-06T12:52:28.033874+0000","last_peered":"2023-10-06T12:52:28.033874+0000","last_clean":"2023-10-06T12:52:28.033874+0000","last_became_active":"2023-10-06T12:52:27.000316+0000","last_became_peered":"2023-10-06T12:52:27.000316+0000","last_unstale":"2023-10-06T12:52:28.033874+0000","last_undegraded":"2023-10-06T12:52:28.033874+0000","last_fullsized":"2023-10-06T12:52:28.033874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:34.456077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008002+0000","last_change":"2023-10-06T12:52:28.003733+0000","last_active":"2023-10-06T12:52:28.008002+0000","last_peered":"2023-10-06T12:52:28.008002+0000","last_clean":"2023-10-06T12:52:28.008002+0000","last_became_active":"2023-10-06T12:52:28.003293+0000","last_became_peered":"2023-10-06T12:52:28.003293+0000","last_unstale":"2023-10-06T12:52:28.008002+0000","last_undegraded":"2023-10-06T12:52:28.008002+0000","last_fullsized":"2023-10-06T12:52:28.008002+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:27.425021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032222+0000","last_change":"2023-10-06T12:52:27.004632+0000","last_active":"2023-10-06T12:52:28.032222+0000","last_peered":"2023-10-06T12:52:28.032222+0000","last_clean":"2023-10-06T12:52:28.032222+0000","last_became_active":"2023-10-06T12:52:27.004372+0000","last_became_peered":"2023-10-06T12:52:27.004372+0000","last_unstale":"2023-10-06T12:52:28.032222+0000","last_undegraded":"2023-10-06T12:52:28.032222+0000","last_fullsized":"2023-10-06T12:52:28.032222+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:01.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008038+0000","last_change":"2023-10-06T12:52:28.001659+0000","last_active":"2023-10-06T12:52:28.008038+0000","last_peered":"2023-10-06T12:52:28.008038+0000","last_clean":"2023-10-06T12:52:28.008038+0000","last_became_active":"2023-10-06T12:52:28.001139+0000","last_became_peered":"2023-10-06T12:52:28.001139+0000","last_unstale":"2023-10-06T12:52:28.008038+0000","last_undegraded":"2023-10-06T12:52:28.008038+0000","last_fullsized":"2023-10-06T12:52:28.008038+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:49.292946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033237+0000","last_change":"2023-10-06T12:52:27.003853+0000","last_active":"2023-10-06T12:52:28.033237+0000","last_peered":"2023-10-06T12:52:28.033237+0000","last_clean":"2023-10-06T12:52:28.033237+0000","last_became_active":"2023-10-06T12:52:27.003500+0000","last_became_peered":"2023-10-06T12:52:27.003500+0000","last_unstale":"2023-10-06T12:52:28.033237+0000","last_undegraded":"2023-10-06T12:52:28.033237+0000","last_fullsized":"2023-10-06T12:52:28.033237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:42.920365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017767+0000","last_change":"2023-10-06T12:52:28.011711+0000","last_active":"2023-10-06T12:52:28.017767+0000","last_peered":"2023-10-06T12:52:28.017767+0000","last_clean":"2023-10-06T12:52:28.017767+0000","last_became_active":"2023-10-06T12:52:28.010565+0000","last_became_peered":"2023-10-06T12:52:28.010565+0000","last_unstale":"2023-10-06T12:52:28.017767+0000","last_undegraded":"2023-10-06T12:52:28.017767+0000","last_fullsized":"2023-10-06T12:52:28.017767+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:16.856488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.991699+0000","last_change":"2023-10-06T12:52:26.998580+0000","last_active":"2023-10-06T12:52:27.991699+0000","last_peered":"2023-10-06T12:52:27.991699+0000","last_clean":"2023-10-06T12:52:27.991699+0000","last_became_active":"2023-10-06T12:52:26.998269+0000","last_became_peered":"2023-10-06T12:52:26.998269+0000","last_unstale":"2023-10-06T12:52:27.991699+0000","last_undegraded":"2023-10-06T12:52:27.991699+0000","last_fullsized":"2023-10-06T12:52:27.991699+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:29:07.618783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015975+0000","last_change":"2023-10-06T12:52:28.010398+0000","last_active":"2023-10-06T12:52:28.015975+0000","last_peered":"2023-10-06T12:52:28.015975+0000","last_clean":"2023-10-06T12:52:28.015975+0000","last_became_active":"2023-10-06T12:52:28.010074+0000","last_became_peered":"2023-10-06T12:52:28.010074+0000","last_unstale":"2023-10-06T12:52:28.015975+0000","last_undegraded":"2023-10-06T12:52:28.015975+0000","last_fullsized":"2023-10-06T12:52:28.015975+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:58.432758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033363+0000","last_change":"2023-10-06T12:52:27.003914+0000","last_active":"2023-10-06T12:52:28.033363+0000","last_peered":"2023-10-06T12:52:28.033363+0000","last_clean":"2023-10-06T12:52:28.033363+0000","last_became_active":"2023-10-06T12:52:27.003550+0000","last_became_peered":"2023-10-06T12:52:27.003550+0000","last_unstale":"2023-10-06T12:52:28.033363+0000","last_undegraded":"2023-10-06T12:52:28.033363+0000","last_fullsized":"2023-10-06T12:52:28.033363+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:50.245593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012235+0000","last_change":"2023-10-06T12:52:28.006006+0000","last_active":"2023-10-06T12:52:28.012235+0000","last_peered":"2023-10-06T12:52:28.012235+0000","last_clean":"2023-10-06T12:52:28.012235+0000","last_became_active":"2023-10-06T12:52:28.005722+0000","last_became_peered":"2023-10-06T12:52:28.005722+0000","last_unstale":"2023-10-06T12:52:28.012235+0000","last_undegraded":"2023-10-06T12:52:28.012235+0000","last_fullsized":"2023-10-06T12:52:28.012235+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.057705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000969+0000","last_change":"2023-10-06T12:52:27.002243+0000","last_active":"2023-10-06T12:52:28.000969+0000","last_peered":"2023-10-06T12:52:28.000969+0000","last_clean":"2023-10-06T12:52:28.000969+0000","last_became_active":"2023-10-06T12:52:27.001864+0000","last_became_peered":"2023-10-06T12:52:27.001864+0000","last_unstale":"2023-10-06T12:52:28.000969+0000","last_undegraded":"2023-10-06T12:52:28.000969+0000","last_fullsized":"2023-10-06T12:52:28.000969+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:14.660171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012556+0000","last_change":"2023-10-06T12:52:28.003305+0000","last_active":"2023-10-06T12:52:28.012556+0000","last_peered":"2023-10-06T12:52:28.012556+0000","last_clean":"2023-10-06T12:52:28.012556+0000","last_became_active":"2023-10-06T12:52:28.002941+0000","last_became_peered":"2023-10-06T12:52:28.002941+0000","last_unstale":"2023-10-06T12:52:28.012556+0000","last_undegraded":"2023-10-06T12:52:28.012556+0000","last_fullsized":"2023-10-06T12:52:28.012556+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:26.641824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005825+0000","last_change":"2023-10-06T12:52:27.013158+0000","last_active":"2023-10-06T12:52:28.005825+0000","last_peered":"2023-10-06T12:52:28.005825+0000","last_clean":"2023-10-06T12:52:28.005825+0000","last_became_active":"2023-10-06T12:52:27.012837+0000","last_became_peered":"2023-10-06T12:52:27.012837+0000","last_unstale":"2023-10-06T12:52:28.005825+0000","last_undegraded":"2023-10-06T12:52:28.005825+0000","last_fullsized":"2023-10-06T12:52:28.005825+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:11.131639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.029293+0000","last_change":"2023-10-06T12:52:28.021734+0000","last_active":"2023-10-06T12:52:28.029293+0000","last_peered":"2023-10-06T12:52:28.029293+0000","last_clean":"2023-10-06T12:52:28.029293+0000","last_became_active":"2023-10-06T12:52:28.021428+0000","last_became_peered":"2023-10-06T12:52:28.021428+0000","last_unstale":"2023-10-06T12:52:28.029293+0000","last_undegraded":"2023-10-06T12:52:28.029293+0000","last_fullsized":"2023-10-06T12:52:28.029293+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:35.471617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998414+0000","last_change":"2023-10-06T12:52:26.995918+0000","last_active":"2023-10-06T12:52:27.998414+0000","last_peered":"2023-10-06T12:52:27.998414+0000","last_clean":"2023-10-06T12:52:27.998414+0000","last_became_active":"2023-10-06T12:52:26.995600+0000","last_became_peered":"2023-10-06T12:52:26.995600+0000","last_unstale":"2023-10-06T12:52:27.998414+0000","last_undegraded":"2023-10-06T12:52:27.998414+0000","last_fullsized":"2023-10-06T12:52:27.998414+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:42:19.213954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.020501+0000","last_change":"2023-10-06T12:52:28.017455+0000","last_active":"2023-10-06T12:52:28.020501+0000","last_peered":"2023-10-06T12:52:28.020501+0000","last_clean":"2023-10-06T12:52:28.020501+0000","last_became_active":"2023-10-06T12:52:28.017156+0000","last_became_peered":"2023-10-06T12:52:28.017156+0000","last_unstale":"2023-10-06T12:52:28.020501+0000","last_undegraded":"2023-10-06T12:52:28.020501+0000","last_fullsized":"2023-10-06T12:52:28.020501+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:24.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032392+0000","last_change":"2023-10-06T12:52:26.999491+0000","last_active":"2023-10-06T12:52:28.032392+0000","last_peered":"2023-10-06T12:52:28.032392+0000","last_clean":"2023-10-06T12:52:28.032392+0000","last_became_active":"2023-10-06T12:52:26.999166+0000","last_became_peered":"2023-10-06T12:52:26.999166+0000","last_unstale":"2023-10-06T12:52:28.032392+0000","last_undegraded":"2023-10-06T12:52:28.032392+0000","last_fullsized":"2023-10-06T12:52:28.032392+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:14.411876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.027200+0000","last_change":"2023-10-06T12:52:28.025077+0000","last_active":"2023-10-06T12:52:28.027200+0000","last_peered":"2023-10-06T12:52:28.027200+0000","last_clean":"2023-10-06T12:52:28.027200+0000","last_became_active":"2023-10-06T12:52:28.023626+0000","last_became_peered":"2023-10-06T12:52:28.023626+0000","last_unstale":"2023-10-06T12:52:28.027200+0000","last_undegraded":"2023-10-06T12:52:28.027200+0000","last_fullsized":"2023-10-06T12:52:28.027200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T12:52:29.153485+0000","last_change":"2023-10-06T12:52:27.014697+0000","last_active":"2023-10-06T12:52:29.153485+0000","last_peered":"2023-10-06T12:52:29.153485+0000","last_clean":"2023-10-06T12:52:29.153485+0000","last_became_active":"2023-10-06T12:52:27.014377+0000","last_became_peered":"2023-10-06T12:52:27.014377+0000","last_unstale":"2023-10-06T12:52:29.153485+0000","last_undegraded":"2023-10-06T12:52:29.153485+0000","last_fullsized":"2023-10-06T12:52:29.153485+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:06.003673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015960+0000","last_change":"2023-10-06T12:52:28.010792+0000","last_active":"2023-10-06T12:52:28.015960+0000","last_peered":"2023-10-06T12:52:28.015960+0000","last_clean":"2023-10-06T12:52:28.015960+0000","last_became_active":"2023-10-06T12:52:28.010475+0000","last_became_peered":"2023-10-06T12:52:28.010475+0000","last_unstale":"2023-10-06T12:52:28.015960+0000","last_undegraded":"2023-10-06T12:52:28.015960+0000","last_fullsized":"2023-10-06T12:52:28.015960+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.990065+0000","last_change":"2023-10-06T12:52:27.004657+0000","last_active":"2023-10-06T12:52:27.990065+0000","last_peered":"2023-10-06T12:52:27.990065+0000","last_clean":"2023-10-06T12:52:27.990065+0000","last_became_active":"2023-10-06T12:52:27.004224+0000","last_became_peered":"2023-10-06T12:52:27.004224+0000","last_unstale":"2023-10-06T12:52:27.990065+0000","last_undegraded":"2023-10-06T12:52:27.990065+0000","last_fullsized":"2023-10-06T12:52:27.990065+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:37.259985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023991+0000","last_change":"2023-10-06T12:52:28.019806+0000","last_active":"2023-10-06T12:52:28.023991+0000","last_peered":"2023-10-06T12:52:28.023991+0000","last_clean":"2023-10-06T12:52:28.023991+0000","last_became_active":"2023-10-06T12:52:28.017823+0000","last_became_peered":"2023-10-06T12:52:28.017823+0000","last_unstale":"2023-10-06T12:52:28.023991+0000","last_undegraded":"2023-10-06T12:52:28.023991+0000","last_fullsized":"2023-10-06T12:52:28.023991+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:39.665430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.990085+0000","last_change":"2023-10-06T12:52:27.014559+0000","last_active":"2023-10-06T12:52:27.990085+0000","last_peered":"2023-10-06T12:52:27.990085+0000","last_clean":"2023-10-06T12:52:27.990085+0000","last_became_active":"2023-10-06T12:52:27.014225+0000","last_became_peered":"2023-10-06T12:52:27.014225+0000","last_unstale":"2023-10-06T12:52:27.990085+0000","last_undegraded":"2023-10-06T12:52:27.990085+0000","last_fullsized":"2023-10-06T12:52:27.990085+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:38.649889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.009599+0000","last_change":"2023-10-06T12:52:28.006080+0000","last_active":"2023-10-06T12:52:28.009599+0000","last_peered":"2023-10-06T12:52:28.009599+0000","last_clean":"2023-10-06T12:52:28.009599+0000","last_became_active":"2023-10-06T12:52:28.005789+0000","last_became_peered":"2023-10-06T12:52:28.005789+0000","last_unstale":"2023-10-06T12:52:28.009599+0000","last_undegraded":"2023-10-06T12:52:28.009599+0000","last_fullsized":"2023-10-06T12:52:28.009599+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:53.877552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999546+0000","last_change":"2023-10-06T12:52:27.005220+0000","last_active":"2023-10-06T12:52:27.999546+0000","last_peered":"2023-10-06T12:52:27.999546+0000","last_clean":"2023-10-06T12:52:27.999546+0000","last_became_active":"2023-10-06T12:52:27.004840+0000","last_became_peered":"2023-10-06T12:52:27.004840+0000","last_unstale":"2023-10-06T12:52:27.999546+0000","last_undegraded":"2023-10-06T12:52:27.999546+0000","last_fullsized":"2023-10-06T12:52:27.999546+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:57.417218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016038+0000","last_change":"2023-10-06T12:52:28.008930+0000","last_active":"2023-10-06T12:52:28.016038+0000","last_peered":"2023-10-06T12:52:28.016038+0000","last_clean":"2023-10-06T12:52:28.016038+0000","last_became_active":"2023-10-06T12:52:28.008646+0000","last_became_peered":"2023-10-06T12:52:28.008646+0000","last_unstale":"2023-10-06T12:52:28.016038+0000","last_undegraded":"2023-10-06T12:52:28.016038+0000","last_fullsized":"2023-10-06T12:52:28.016038+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:46.487911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999375+0000","last_change":"2023-10-06T12:52:27.009451+0000","last_active":"2023-10-06T12:52:27.999375+0000","last_peered":"2023-10-06T12:52:27.999375+0000","last_clean":"2023-10-06T12:52:27.999375+0000","last_became_active":"2023-10-06T12:52:27.009130+0000","last_became_peered":"2023-10-06T12:52:27.009130+0000","last_unstale":"2023-10-06T12:52:27.999375+0000","last_undegraded":"2023-10-06T12:52:27.999375+0000","last_fullsized":"2023-10-06T12:52:27.999375+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:15.840947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017705+0000","last_change":"2023-10-06T12:52:28.012383+0000","last_active":"2023-10-06T12:52:28.017705+0000","last_peered":"2023-10-06T12:52:28.017705+0000","last_clean":"2023-10-06T12:52:28.017705+0000","last_became_active":"2023-10-06T12:52:28.011392+0000","last_became_peered":"2023-10-06T12:52:28.011392+0000","last_unstale":"2023-10-06T12:52:28.017705+0000","last_undegraded":"2023-10-06T12:52:28.017705+0000","last_fullsized":"2023-10-06T12:52:28.017705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005718+0000","last_change":"2023-10-06T12:52:27.002635+0000","last_active":"2023-10-06T12:52:28.005718+0000","last_peered":"2023-10-06T12:52:28.005718+0000","last_clean":"2023-10-06T12:52:28.005718+0000","last_became_active":"2023-10-06T12:52:27.002224+0000","last_became_peered":"2023-10-06T12:52:27.002224+0000","last_unstale":"2023-10-06T12:52:28.005718+0000","last_undegraded":"2023-10-06T12:52:28.005718+0000","last_fullsized":"2023-10-06T12:52:28.005718+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.609227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015929+0000","last_change":"2023-10-06T12:52:28.011888+0000","last_active":"2023-10-06T12:52:28.015929+0000","last_peered":"2023-10-06T12:52:28.015929+0000","last_clean":"2023-10-06T12:52:28.015929+0000","last_became_active":"2023-10-06T12:52:28.011518+0000","last_became_peered":"2023-10-06T12:52:28.011518+0000","last_unstale":"2023-10-06T12:52:28.015929+0000","last_undegraded":"2023-10-06T12:52:28.015929+0000","last_fullsized":"2023-10-06T12:52:28.015929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:52.220289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998527+0000","last_change":"2023-10-06T12:52:27.005454+0000","last_active":"2023-10-06T12:52:27.998527+0000","last_peered":"2023-10-06T12:52:27.998527+0000","last_clean":"2023-10-06T12:52:27.998527+0000","last_became_active":"2023-10-06T12:52:27.005138+0000","last_became_peered":"2023-10-06T12:52:27.005138+0000","last_unstale":"2023-10-06T12:52:27.998527+0000","last_undegraded":"2023-10-06T12:52:27.998527+0000","last_fullsized":"2023-10-06T12:52:27.998527+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:27.092380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012567+0000","last_change":"2023-10-06T12:52:28.010210+0000","last_active":"2023-10-06T12:52:28.012567+0000","last_peered":"2023-10-06T12:52:28.012567+0000","last_clean":"2023-10-06T12:52:28.012567+0000","last_became_active":"2023-10-06T12:52:28.009857+0000","last_became_peered":"2023-10-06T12:52:28.009857+0000","last_unstale":"2023-10-06T12:52:28.012567+0000","last_undegraded":"2023-10-06T12:52:28.012567+0000","last_fullsized":"2023-10-06T12:52:28.012567+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:53.941098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000066+0000","last_change":"2023-10-06T12:52:26.999893+0000","last_active":"2023-10-06T12:52:28.000066+0000","last_peered":"2023-10-06T12:52:28.000066+0000","last_clean":"2023-10-06T12:52:28.000066+0000","last_became_active":"2023-10-06T12:52:26.999589+0000","last_became_peered":"2023-10-06T12:52:26.999589+0000","last_unstale":"2023-10-06T12:52:28.000066+0000","last_undegraded":"2023-10-06T12:52:28.000066+0000","last_fullsized":"2023-10-06T12:52:28.000066+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:52.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"186'348","reported_seq":1131,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999435+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:27.999435+0000","last_peered":"2023-10-06T12:52:27.999435+0000","last_clean":"2023-10-06T12:52:27.999435+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:27.999435+0000","last_undegraded":"2023-10-06T12:52:27.999435+0000","last_fullsized":"2023-10-06T12:52:27.999435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":181,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.991029+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:27.991029+0000","last_peered":"2023-10-06T12:52:27.991029+0000","last_clean":"2023-10-06T12:52:27.991029+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:27.991029+0000","last_undegraded":"2023-10-06T12:52:27.991029+0000","last_fullsized":"2023-10-06T12:52:27.991029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.003741+0000","last_change":"2023-10-06T12:52:27.997760+0000","last_active":"2023-10-06T12:52:28.003741+0000","last_peered":"2023-10-06T12:52:28.003741+0000","last_clean":"2023-10-06T12:52:28.003741+0000","last_became_active":"2023-10-06T12:52:27.997443+0000","last_became_peered":"2023-10-06T12:52:27.997443+0000","last_unstale":"2023-10-06T12:52:28.003741+0000","last_undegraded":"2023-10-06T12:52:28.003741+0000","last_fullsized":"2023-10-06T12:52:28.003741+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:16.163073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.030604+0000","last_change":"2023-10-06T12:52:27.000987+0000","last_active":"2023-10-06T12:52:28.030604+0000","last_peered":"2023-10-06T12:52:28.030604+0000","last_clean":"2023-10-06T12:52:28.030604+0000","last_became_active":"2023-10-06T12:52:27.000617+0000","last_became_peered":"2023-10-06T12:52:27.000617+0000","last_unstale":"2023-10-06T12:52:28.030604+0000","last_undegraded":"2023-10-06T12:52:28.030604+0000","last_fullsized":"2023-10-06T12:52:28.030604+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:47.134670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999341+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:27.999341+0000","last_peered":"2023-10-06T12:52:27.999341+0000","last_clean":"2023-10-06T12:52:27.999341+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:27.999341+0000","last_undegraded":"2023-10-06T12:52:27.999341+0000","last_fullsized":"2023-10-06T12:52:27.999341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021593+0000","last_change":"2023-10-06T12:52:28.019292+0000","last_active":"2023-10-06T12:52:28.021593+0000","last_peered":"2023-10-06T12:52:28.021593+0000","last_clean":"2023-10-06T12:52:28.021593+0000","last_became_active":"2023-10-06T12:52:28.018945+0000","last_became_peered":"2023-10-06T12:52:28.018945+0000","last_unstale":"2023-10-06T12:52:28.021593+0000","last_undegraded":"2023-10-06T12:52:28.021593+0000","last_fullsized":"2023-10-06T12:52:28.021593+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032980+0000","last_change":"2023-10-06T12:52:27.008216+0000","last_active":"2023-10-06T12:52:28.032980+0000","last_peered":"2023-10-06T12:52:28.032980+0000","last_clean":"2023-10-06T12:52:28.032980+0000","last_became_active":"2023-10-06T12:52:27.007775+0000","last_became_peered":"2023-10-06T12:52:27.007775+0000","last_unstale":"2023-10-06T12:52:28.032980+0000","last_undegraded":"2023-10-06T12:52:28.032980+0000","last_fullsized":"2023-10-06T12:52:28.032980+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:37.367512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005908+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:28.005908+0000","last_peered":"2023-10-06T12:52:28.005908+0000","last_clean":"2023-10-06T12:52:28.005908+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:28.005908+0000","last_undegraded":"2023-10-06T12:52:28.005908+0000","last_fullsized":"2023-10-06T12:52:28.005908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017728+0000","last_change":"2023-10-06T12:52:28.012887+0000","last_active":"2023-10-06T12:52:28.017728+0000","last_peered":"2023-10-06T12:52:28.017728+0000","last_clean":"2023-10-06T12:52:28.017728+0000","last_became_active":"2023-10-06T12:52:28.011870+0000","last_became_peered":"2023-10-06T12:52:28.011870+0000","last_unstale":"2023-10-06T12:52:28.017728+0000","last_undegraded":"2023-10-06T12:52:28.017728+0000","last_fullsized":"2023-10-06T12:52:28.017728+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:11.257569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.993415+0000","last_change":"2023-10-06T12:52:27.002628+0000","last_active":"2023-10-06T12:52:27.993415+0000","last_peered":"2023-10-06T12:52:27.993415+0000","last_clean":"2023-10-06T12:52:27.993415+0000","last_became_active":"2023-10-06T12:52:27.002333+0000","last_became_peered":"2023-10-06T12:52:27.002333+0000","last_unstale":"2023-10-06T12:52:27.993415+0000","last_undegraded":"2023-10-06T12:52:27.993415+0000","last_fullsized":"2023-10-06T12:52:27.993415+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:38.649889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":181,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.992142+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:27.992142+0000","last_peered":"2023-10-06T12:52:27.992142+0000","last_clean":"2023-10-06T12:52:27.992142+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:27.992142+0000","last_undegraded":"2023-10-06T12:52:27.992142+0000","last_fullsized":"2023-10-06T12:52:27.992142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017939+0000","last_change":"2023-10-06T12:52:28.014605+0000","last_active":"2023-10-06T12:52:28.017939+0000","last_peered":"2023-10-06T12:52:28.017939+0000","last_clean":"2023-10-06T12:52:28.017939+0000","last_became_active":"2023-10-06T12:52:28.014325+0000","last_became_peered":"2023-10-06T12:52:28.014325+0000","last_unstale":"2023-10-06T12:52:28.017939+0000","last_undegraded":"2023-10-06T12:52:28.017939+0000","last_fullsized":"2023-10-06T12:52:28.017939+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:03:30.683577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"196'4","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033105+0000","last_change":"2023-10-06T12:52:27.001679+0000","last_active":"2023-10-06T12:52:28.033105+0000","last_peered":"2023-10-06T12:52:28.033105+0000","last_clean":"2023-10-06T12:52:28.033105+0000","last_became_active":"2023-10-06T12:52:27.001355+0000","last_became_peered":"2023-10-06T12:52:27.001355+0000","last_unstale":"2023-10-06T12:52:28.033105+0000","last_undegraded":"2023-10-06T12:52:28.033105+0000","last_fullsized":"2023-10-06T12:52:28.033105+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:28.092062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.997436+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:27.997436+0000","last_peered":"2023-10-06T12:52:27.997436+0000","last_clean":"2023-10-06T12:52:27.997436+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:27.997436+0000","last_undegraded":"2023-10-06T12:52:27.997436+0000","last_fullsized":"2023-10-06T12:52:27.997436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021938+0000","last_change":"2023-10-06T12:52:28.019332+0000","last_active":"2023-10-06T12:52:28.021938+0000","last_peered":"2023-10-06T12:52:28.021938+0000","last_clean":"2023-10-06T12:52:28.021938+0000","last_became_active":"2023-10-06T12:52:28.019016+0000","last_became_peered":"2023-10-06T12:52:28.019016+0000","last_unstale":"2023-10-06T12:52:28.021938+0000","last_undegraded":"2023-10-06T12:52:28.021938+0000","last_fullsized":"2023-10-06T12:52:28.021938+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:38.275526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.031621+0000","last_change":"2023-10-06T12:52:27.002052+0000","last_active":"2023-10-06T12:52:28.031621+0000","last_peered":"2023-10-06T12:52:28.031621+0000","last_clean":"2023-10-06T12:52:28.031621+0000","last_became_active":"2023-10-06T12:52:27.000367+0000","last_became_peered":"2023-10-06T12:52:27.000367+0000","last_unstale":"2023-10-06T12:52:28.031621+0000","last_undegraded":"2023-10-06T12:52:28.031621+0000","last_fullsized":"2023-10-06T12:52:28.031621+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:23.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.001376+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:28.001376+0000","last_peered":"2023-10-06T12:52:28.001376+0000","last_clean":"2023-10-06T12:52:28.001376+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:28.001376+0000","last_undegraded":"2023-10-06T12:52:28.001376+0000","last_fullsized":"2023-10-06T12:52:28.001376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010588+0000","last_change":"2023-10-06T12:52:28.006040+0000","last_active":"2023-10-06T12:52:28.010588+0000","last_peered":"2023-10-06T12:52:28.010588+0000","last_clean":"2023-10-06T12:52:28.010588+0000","last_became_active":"2023-10-06T12:52:28.005698+0000","last_became_peered":"2023-10-06T12:52:28.005698+0000","last_unstale":"2023-10-06T12:52:28.010588+0000","last_undegraded":"2023-10-06T12:52:28.010588+0000","last_fullsized":"2023-10-06T12:52:28.010588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:09.341415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.027829+0000","last_change":"2023-10-06T12:52:26.998978+0000","last_active":"2023-10-06T12:52:28.027829+0000","last_peered":"2023-10-06T12:52:28.027829+0000","last_clean":"2023-10-06T12:52:28.027829+0000","last_became_active":"2023-10-06T12:52:26.998528+0000","last_became_peered":"2023-10-06T12:52:26.998528+0000","last_unstale":"2023-10-06T12:52:28.027829+0000","last_undegraded":"2023-10-06T12:52:28.027829+0000","last_fullsized":"2023-10-06T12:52:28.027829+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:26.409480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998414+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:27.998414+0000","last_peered":"2023-10-06T12:52:27.998414+0000","last_clean":"2023-10-06T12:52:27.998414+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:27.998414+0000","last_undegraded":"2023-10-06T12:52:27.998414+0000","last_fullsized":"2023-10-06T12:52:27.998414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013706+0000","last_change":"2023-10-06T12:52:28.007813+0000","last_active":"2023-10-06T12:52:28.013706+0000","last_peered":"2023-10-06T12:52:28.013706+0000","last_clean":"2023-10-06T12:52:28.013706+0000","last_became_active":"2023-10-06T12:52:28.007473+0000","last_became_peered":"2023-10-06T12:52:28.007473+0000","last_unstale":"2023-10-06T12:52:28.013706+0000","last_undegraded":"2023-10-06T12:52:28.013706+0000","last_fullsized":"2023-10-06T12:52:28.013706+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:16.299803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032706+0000","last_change":"2023-10-06T12:52:27.008092+0000","last_active":"2023-10-06T12:52:28.032706+0000","last_peered":"2023-10-06T12:52:28.032706+0000","last_clean":"2023-10-06T12:52:28.032706+0000","last_became_active":"2023-10-06T12:52:27.007681+0000","last_became_peered":"2023-10-06T12:52:27.007681+0000","last_unstale":"2023-10-06T12:52:28.032706+0000","last_undegraded":"2023-10-06T12:52:28.032706+0000","last_fullsized":"2023-10-06T12:52:28.032706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:28.210983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999111+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:27.999111+0000","last_peered":"2023-10-06T12:52:27.999111+0000","last_clean":"2023-10-06T12:52:27.999111+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:27.999111+0000","last_undegraded":"2023-10-06T12:52:27.999111+0000","last_fullsized":"2023-10-06T12:52:27.999111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304376,"kb_used_data":2992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067464,"statfs":{"total":96636764160,"available":96325083136,"internally_reserved":0,"allocated":3063808,"data_stored":1904405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":1.381},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.4870000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.081}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.788},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.8680000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.7569999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":2.1779999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.998},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.534}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":2.2839999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299704,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072136,"statfs":{"total":96636764160,"available":96329867264,"internally_reserved":0,"allocated":2473984,"data_stored":1314107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.91300000000000003}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.214}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.355},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.169}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.597}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":1.387},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.4670000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312004,"kb_used_data":2428,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059836,"statfs":{"total":96636764160,"available":96317272064,"internally_reserved":0,"allocated":2486272,"data_stored":1315595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":1.198},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":1.478}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":1.429},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.153}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.393},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.087}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.236}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.3560000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312004,"kb_used_data":2428,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059836,"statfs":{"total":96636764160,"available":96317272064,"internally_reserved":0,"allocated":2486272,"data_stored":1315607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.163},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":1.5449999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.395},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.1930000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.357},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.1299999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.274}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":1.48},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.2310000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.319},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4550000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575973,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299708,"kb_used_data":2420,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072132,"statfs":{"total":96636764160,"available":96329863168,"internally_reserved":0,"allocated":2478080,"data_stored":1314197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.0069999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.071},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.6140000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.095},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.141}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.51},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.552}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.3069999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311992,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059848,"statfs":{"total":96636764160,"available":96317284352,"internally_reserved":0,"allocated":2473984,"data_stored":1314591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.139}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":1.1839999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":1.431}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":1.008},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":1.325},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":1.3}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":1.25},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":1.534}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.228},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":1.4590000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311988,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059852,"statfs":{"total":96636764160,"available":96317288448,"internally_reserved":0,"allocated":2469888,"data_stored":1314037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.1220000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.28}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.365}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.25},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.657}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.444},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3400000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.419},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.6990000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.1679999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575972,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304392,"kb_used_data":3008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067448,"statfs":{"total":96636764160,"available":96325066752,"internally_reserved":0,"allocated":3080192,"data_stored":1905105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":0.627}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.113}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":0.875}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.296}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.157},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:52:35.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:52:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 Processor -- start 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 -- start start 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5de00c5ff0 0x7f5de00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de00cc150 con 0x7f5de00ccb50 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de00cb540 con 0x7f5de015c770 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.487+0000 7f5de866e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de00cfbb0 con 0x7f5de00c5ff0 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5c09700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5de00c5ff0 0x7f5de00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de6c0b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5de00ccb50 0x7f5de0162e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36254/0 (socket says 172.21.15.46:36254) 2023-10-06T12:52:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de6c0b700 1 -- 172.21.15.46:0/1225874989 learned_addr learned my addr 172.21.15.46:0/1225874989 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5408700 1 -- 172.21.15.46:0/1225874989 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371318395 0 0) 0x7f5de00cc150 con 0x7f5de00ccb50 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5408700 1 -- 172.21.15.46:0/1225874989 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dcc003570 con 0x7f5de00ccb50 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5c09700 1 -- 172.21.15.46:0/1225874989 >> v1:172.21.15.132:6789/0 conn(0x7f5de015c770 legacy=0x7f5de00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5c09700 1 -- 172.21.15.46:0/1225874989 >> v1:172.21.15.46:6789/0 conn(0x7f5de00ccb50 legacy=0x7f5de0162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5c09700 1 -- 172.21.15.46:0/1225874989 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de00cb820 con 0x7f5de00c5ff0 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5c09700 1 --2- 172.21.15.46:0/1225874989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de00c5ff0 0x7f5de00ce4f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098e0 tx=0x7f5dd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917e1c6b971dcfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5408700 1 -- 172.21.15.46:0/1225874989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000b640 con 0x7f5de00c5ff0 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5408700 1 -- 172.21.15.46:0/1225874989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000b7a0 con 0x7f5de00c5ff0 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de5408700 1 -- 172.21.15.46:0/1225874989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000bb40 con 0x7f5de00c5ff0 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 -- 172.21.15.46:0/1225874989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de00c5ff0 msgr2=0x7f5de00ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 --2- 172.21.15.46:0/1225874989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de00c5ff0 0x7f5de00ce4f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098e0 tx=0x7f5dd00091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 -- 172.21.15.46:0/1225874989 shutdown_connections 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 --2- 172.21.15.46:0/1225874989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de00c5ff0 0x7f5de00ce4f0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 -- 172.21.15.46:0/1225874989 >> 172.21.15.46:0/1225874989 conn(0x7f5de0099480 msgr2=0x7f5de006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 -- 172.21.15.46:0/1225874989 shutdown_connections 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.491+0000 7f5de866e700 1 -- 172.21.15.46:0/1225874989 wait complete. 2023-10-06T12:52:35.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 Processor -- start 2023-10-06T12:52:35.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 -- start start 2023-10-06T12:52:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de00c5ff0 0x7f5de009b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 0x7f5de00b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:35.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 0x7f5de009c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009de90 con 0x7f5de015c770 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009d680 con 0x7f5de00ccb50 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.495+0000 7f5de866e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de009be60 con 0x7f5de00c5ff0 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de5c09700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 0x7f5de00b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 0x7f5de009c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de5c09700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 0x7f5de00b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54294/0 (socket says 172.21.15.46:54294) 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de5c09700 1 -- 172.21.15.46:0/1599747669 learned_addr learned my addr 172.21.15.46:0/1599747669 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:35.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de640a700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de00c5ff0 0x7f5de009b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 -- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de00c5ff0 msgr2=0x7f5de009b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de00c5ff0 0x7f5de009b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 -- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 msgr2=0x7f5de00b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 0x7f5de00b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 -- 172.21.15.46:0/1599747669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009420 con 0x7f5de015c770 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de6c0b700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 0x7f5de009c2f0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800cbf0 tx=0x7f5dd8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800a8d0 con 0x7f5de015c770 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800aa30 con 0x7f5de015c770 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de009d8d0 con 0x7f5de015c770 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800adb0 con 0x7f5de015c770 2023-10-06T12:52:35.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.499+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de009baf0 con 0x7f5de015c770 2023-10-06T12:52:35.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.503+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de00c6c70 con 0x7f5de015c770 2023-10-06T12:52:35.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.503+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5dd8007300 con 0x7f5de015c770 2023-10-06T12:52:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.507+0000 7f5dd67fc700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 0x7f5dc4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.507+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f5dd807d120 con 0x7f5de015c770 2023-10-06T12:52:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.507+0000 7f5de640a700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 0x7f5dc4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.507+0000 7f5de640a700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 0x7f5dc4062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc003890 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:35.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.511+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5de00c6c70 con 0x7f5de015c770 2023-10-06T12:52:35.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.679+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5de00c6c70 con 0x7f5dc4060670 2023-10-06T12:52:35.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.695+0000 7f5dd67fc700 1 -- 172.21.15.46:0/1599747669 <== mgr.10158 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308891 (secure 0 0 0) 0x7f5de00c6c70 con 0x7f5dc4060670 2023-10-06T12:52:35.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 msgr2=0x7f5dc4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 0x7f5dc4062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc003890 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2023-10-06T12:52:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 msgr2=0x7f5de009c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:35.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 0x7f5de009c2f0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800cbf0 tx=0x7f5dd8001a20 comp rx=0 tx=0).stop 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 shutdown_connections 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de015c770 0x7f5de009c2f0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5dc4060670 0x7f5dc4062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5de00ccb50 0x7f5de00b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 --2- 172.21.15.46:0/1599747669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5de00c5ff0 0x7f5de009b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 >> 172.21.15.46:0/1599747669 conn(0x7f5de0099480 msgr2=0x7f5de00687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 shutdown_connections 2023-10-06T12:52:35.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:35.703+0000 7f5de866e700 1 -- 172.21.15.46:0/1599747669 wait complete. 2023-10-06T12:52:35.711 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:52:35.728 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T12:52:34.883751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":624,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456168,"kb_used_data":20520,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752518552,"statfs":{"total":773094113280,"available":770578997248,"internally_reserved":0,"allocated":21012480,"data_stored":11697644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.905503"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016956+0000","last_change":"2023-10-06T12:52:28.011464+0000","last_active":"2023-10-06T12:52:28.016956+0000","last_peered":"2023-10-06T12:52:28.016956+0000","last_clean":"2023-10-06T12:52:28.016956+0000","last_became_active":"2023-10-06T12:52:28.011035+0000","last_became_peered":"2023-10-06T12:52:28.011035+0000","last_unstale":"2023-10-06T12:52:28.016956+0000","last_undegraded":"2023-10-06T12:52:28.016956+0000","last_fullsized":"2023-10-06T12:52:28.016956+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:38.275526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012713+0000","last_change":"2023-10-06T12:52:28.008738+0000","last_active":"2023-10-06T12:52:28.012713+0000","last_peered":"2023-10-06T12:52:28.012713+0000","last_clean":"2023-10-06T12:52:28.012713+0000","last_became_active":"2023-10-06T12:52:28.008427+0000","last_became_peered":"2023-10-06T12:52:28.008427+0000","last_unstale":"2023-10-06T12:52:28.012713+0000","last_undegraded":"2023-10-06T12:52:28.012713+0000","last_fullsized":"2023-10-06T12:52:28.012713+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:24.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023148+0000","last_change":"2023-10-06T12:52:28.022490+0000","last_active":"2023-10-06T12:52:28.023148+0000","last_peered":"2023-10-06T12:52:28.023148+0000","last_clean":"2023-10-06T12:52:28.023148+0000","last_became_active":"2023-10-06T12:52:28.022155+0000","last_became_peered":"2023-10-06T12:52:28.022155+0000","last_unstale":"2023-10-06T12:52:28.023148+0000","last_undegraded":"2023-10-06T12:52:28.023148+0000","last_fullsized":"2023-10-06T12:52:28.023148+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:49:18.478057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012276+0000","last_change":"2023-10-06T12:52:28.004727+0000","last_active":"2023-10-06T12:52:28.012276+0000","last_peered":"2023-10-06T12:52:28.012276+0000","last_clean":"2023-10-06T12:52:28.012276+0000","last_became_active":"2023-10-06T12:52:28.003708+0000","last_became_peered":"2023-10-06T12:52:28.003708+0000","last_unstale":"2023-10-06T12:52:28.012276+0000","last_undegraded":"2023-10-06T12:52:28.012276+0000","last_fullsized":"2023-10-06T12:52:28.012276+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:18.319135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.004458+0000","last_change":"2023-10-06T12:52:27.996748+0000","last_active":"2023-10-06T12:52:28.004458+0000","last_peered":"2023-10-06T12:52:28.004458+0000","last_clean":"2023-10-06T12:52:28.004458+0000","last_became_active":"2023-10-06T12:52:27.996425+0000","last_became_peered":"2023-10-06T12:52:27.996425+0000","last_unstale":"2023-10-06T12:52:28.004458+0000","last_undegraded":"2023-10-06T12:52:28.004458+0000","last_fullsized":"2023-10-06T12:52:28.004458+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:37.531390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.014019+0000","last_change":"2023-10-06T12:52:28.011794+0000","last_active":"2023-10-06T12:52:28.014019+0000","last_peered":"2023-10-06T12:52:28.014019+0000","last_clean":"2023-10-06T12:52:28.014019+0000","last_became_active":"2023-10-06T12:52:28.011430+0000","last_became_peered":"2023-10-06T12:52:28.011430+0000","last_unstale":"2023-10-06T12:52:28.014019+0000","last_undegraded":"2023-10-06T12:52:28.014019+0000","last_fullsized":"2023-10-06T12:52:28.014019+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.968731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021491+0000","last_change":"2023-10-06T12:52:28.015401+0000","last_active":"2023-10-06T12:52:28.021491+0000","last_peered":"2023-10-06T12:52:28.021491+0000","last_clean":"2023-10-06T12:52:28.021491+0000","last_became_active":"2023-10-06T12:52:28.015093+0000","last_became_peered":"2023-10-06T12:52:28.015093+0000","last_unstale":"2023-10-06T12:52:28.021491+0000","last_undegraded":"2023-10-06T12:52:28.021491+0000","last_fullsized":"2023-10-06T12:52:28.021491+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:48.150210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012346+0000","last_change":"2023-10-06T12:52:28.009183+0000","last_active":"2023-10-06T12:52:28.012346+0000","last_peered":"2023-10-06T12:52:28.012346+0000","last_clean":"2023-10-06T12:52:28.012346+0000","last_became_active":"2023-10-06T12:52:28.008689+0000","last_became_peered":"2023-10-06T12:52:28.008689+0000","last_unstale":"2023-10-06T12:52:28.012346+0000","last_undegraded":"2023-10-06T12:52:28.012346+0000","last_fullsized":"2023-10-06T12:52:28.012346+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.009259+0000","last_change":"2023-10-06T12:52:28.005784+0000","last_active":"2023-10-06T12:52:28.009259+0000","last_peered":"2023-10-06T12:52:28.009259+0000","last_clean":"2023-10-06T12:52:28.009259+0000","last_became_active":"2023-10-06T12:52:28.005497+0000","last_became_peered":"2023-10-06T12:52:28.005497+0000","last_unstale":"2023-10-06T12:52:28.009259+0000","last_undegraded":"2023-10-06T12:52:28.009259+0000","last_fullsized":"2023-10-06T12:52:28.009259+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:40:14.695497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007678+0000","last_change":"2023-10-06T12:52:28.002130+0000","last_active":"2023-10-06T12:52:28.007678+0000","last_peered":"2023-10-06T12:52:28.007678+0000","last_clean":"2023-10-06T12:52:28.007678+0000","last_became_active":"2023-10-06T12:52:28.001818+0000","last_became_peered":"2023-10-06T12:52:28.001818+0000","last_unstale":"2023-10-06T12:52:28.007678+0000","last_undegraded":"2023-10-06T12:52:28.007678+0000","last_fullsized":"2023-10-06T12:52:28.007678+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:02.252564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010942+0000","last_change":"2023-10-06T12:52:28.006710+0000","last_active":"2023-10-06T12:52:28.010942+0000","last_peered":"2023-10-06T12:52:28.010942+0000","last_clean":"2023-10-06T12:52:28.010942+0000","last_became_active":"2023-10-06T12:52:28.006373+0000","last_became_peered":"2023-10-06T12:52:28.006373+0000","last_unstale":"2023-10-06T12:52:28.010942+0000","last_undegraded":"2023-10-06T12:52:28.010942+0000","last_fullsized":"2023-10-06T12:52:28.010942+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:20:38.406215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017693+0000","last_change":"2023-10-06T12:52:28.014087+0000","last_active":"2023-10-06T12:52:28.017693+0000","last_peered":"2023-10-06T12:52:28.017693+0000","last_clean":"2023-10-06T12:52:28.017693+0000","last_became_active":"2023-10-06T12:52:28.013702+0000","last_became_peered":"2023-10-06T12:52:28.013702+0000","last_unstale":"2023-10-06T12:52:28.017693+0000","last_undegraded":"2023-10-06T12:52:28.017693+0000","last_fullsized":"2023-10-06T12:52:28.017693+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:59:11.161220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016865+0000","last_change":"2023-10-06T12:52:28.010794+0000","last_active":"2023-10-06T12:52:28.016865+0000","last_peered":"2023-10-06T12:52:28.016865+0000","last_clean":"2023-10-06T12:52:28.016865+0000","last_became_active":"2023-10-06T12:52:28.010511+0000","last_became_peered":"2023-10-06T12:52:28.010511+0000","last_unstale":"2023-10-06T12:52:28.016865+0000","last_undegraded":"2023-10-06T12:52:28.016865+0000","last_fullsized":"2023-10-06T12:52:28.016865+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:52.034573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021960+0000","last_change":"2023-10-06T12:52:28.018712+0000","last_active":"2023-10-06T12:52:28.021960+0000","last_peered":"2023-10-06T12:52:28.021960+0000","last_clean":"2023-10-06T12:52:28.021960+0000","last_became_active":"2023-10-06T12:52:28.018402+0000","last_became_peered":"2023-10-06T12:52:28.018402+0000","last_unstale":"2023-10-06T12:52:28.021960+0000","last_undegraded":"2023-10-06T12:52:28.021960+0000","last_fullsized":"2023-10-06T12:52:28.021960+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.018526+0000","last_change":"2023-10-06T12:52:28.016694+0000","last_active":"2023-10-06T12:52:28.018526+0000","last_peered":"2023-10-06T12:52:28.018526+0000","last_clean":"2023-10-06T12:52:28.018526+0000","last_became_active":"2023-10-06T12:52:28.015994+0000","last_became_peered":"2023-10-06T12:52:28.015994+0000","last_unstale":"2023-10-06T12:52:28.018526+0000","last_undegraded":"2023-10-06T12:52:28.018526+0000","last_fullsized":"2023-10-06T12:52:28.018526+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:39.665430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008109+0000","last_change":"2023-10-06T12:52:27.997222+0000","last_active":"2023-10-06T12:52:28.008109+0000","last_peered":"2023-10-06T12:52:28.008109+0000","last_clean":"2023-10-06T12:52:28.008109+0000","last_became_active":"2023-10-06T12:52:27.996744+0000","last_became_peered":"2023-10-06T12:52:27.996744+0000","last_unstale":"2023-10-06T12:52:28.008109+0000","last_undegraded":"2023-10-06T12:52:28.008109+0000","last_fullsized":"2023-10-06T12:52:28.008109+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:17:40.904397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013713+0000","last_change":"2023-10-06T12:52:28.011601+0000","last_active":"2023-10-06T12:52:28.013713+0000","last_peered":"2023-10-06T12:52:28.013713+0000","last_clean":"2023-10-06T12:52:28.013713+0000","last_became_active":"2023-10-06T12:52:28.011154+0000","last_became_peered":"2023-10-06T12:52:28.011154+0000","last_unstale":"2023-10-06T12:52:28.013713+0000","last_undegraded":"2023-10-06T12:52:28.013713+0000","last_fullsized":"2023-10-06T12:52:28.013713+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:41:20.901664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015747+0000","last_change":"2023-10-06T12:52:28.005775+0000","last_active":"2023-10-06T12:52:28.015747+0000","last_peered":"2023-10-06T12:52:28.015747+0000","last_clean":"2023-10-06T12:52:28.015747+0000","last_became_active":"2023-10-06T12:52:28.005490+0000","last_became_peered":"2023-10-06T12:52:28.005490+0000","last_unstale":"2023-10-06T12:52:28.015747+0000","last_undegraded":"2023-10-06T12:52:28.015747+0000","last_fullsized":"2023-10-06T12:52:28.015747+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:17:40.904397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.025963+0000","last_change":"2023-10-06T12:52:28.025040+0000","last_active":"2023-10-06T12:52:28.025963+0000","last_peered":"2023-10-06T12:52:28.025963+0000","last_clean":"2023-10-06T12:52:28.025963+0000","last_became_active":"2023-10-06T12:52:28.024822+0000","last_became_peered":"2023-10-06T12:52:28.024822+0000","last_unstale":"2023-10-06T12:52:28.025963+0000","last_undegraded":"2023-10-06T12:52:28.025963+0000","last_fullsized":"2023-10-06T12:52:28.025963+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:56:37.531390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012426+0000","last_change":"2023-10-06T12:52:28.010770+0000","last_active":"2023-10-06T12:52:28.012426+0000","last_peered":"2023-10-06T12:52:28.012426+0000","last_clean":"2023-10-06T12:52:28.012426+0000","last_became_active":"2023-10-06T12:52:28.010467+0000","last_became_peered":"2023-10-06T12:52:28.010467+0000","last_unstale":"2023-10-06T12:52:28.012426+0000","last_undegraded":"2023-10-06T12:52:28.012426+0000","last_fullsized":"2023-10-06T12:52:28.012426+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:29:08.634324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012753+0000","last_change":"2023-10-06T12:52:28.007917+0000","last_active":"2023-10-06T12:52:28.012753+0000","last_peered":"2023-10-06T12:52:28.012753+0000","last_clean":"2023-10-06T12:52:28.012753+0000","last_became_active":"2023-10-06T12:52:28.007499+0000","last_became_peered":"2023-10-06T12:52:28.007499+0000","last_unstale":"2023-10-06T12:52:28.012753+0000","last_undegraded":"2023-10-06T12:52:28.012753+0000","last_fullsized":"2023-10-06T12:52:28.012753+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:35.009453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013012+0000","last_change":"2023-10-06T12:52:28.008709+0000","last_active":"2023-10-06T12:52:28.013012+0000","last_peered":"2023-10-06T12:52:28.013012+0000","last_clean":"2023-10-06T12:52:28.013012+0000","last_became_active":"2023-10-06T12:52:28.008295+0000","last_became_peered":"2023-10-06T12:52:28.008295+0000","last_unstale":"2023-10-06T12:52:28.013012+0000","last_undegraded":"2023-10-06T12:52:28.013012+0000","last_fullsized":"2023-10-06T12:52:28.013012+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.297245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.018042+0000","last_change":"2023-10-06T12:52:28.014967+0000","last_active":"2023-10-06T12:52:28.018042+0000","last_peered":"2023-10-06T12:52:28.018042+0000","last_clean":"2023-10-06T12:52:28.018042+0000","last_became_active":"2023-10-06T12:52:28.011920+0000","last_became_peered":"2023-10-06T12:52:28.011920+0000","last_unstale":"2023-10-06T12:52:28.018042+0000","last_undegraded":"2023-10-06T12:52:28.018042+0000","last_fullsized":"2023-10-06T12:52:28.018042+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:02.453956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015994+0000","last_change":"2023-10-06T12:52:28.011068+0000","last_active":"2023-10-06T12:52:28.015994+0000","last_peered":"2023-10-06T12:52:28.015994+0000","last_clean":"2023-10-06T12:52:28.015994+0000","last_became_active":"2023-10-06T12:52:28.010749+0000","last_became_peered":"2023-10-06T12:52:28.010749+0000","last_unstale":"2023-10-06T12:52:28.015994+0000","last_undegraded":"2023-10-06T12:52:28.015994+0000","last_fullsized":"2023-10-06T12:52:28.015994+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:48.150210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.024831+0000","last_change":"2023-10-06T12:52:28.021964+0000","last_active":"2023-10-06T12:52:28.024831+0000","last_peered":"2023-10-06T12:52:28.024831+0000","last_clean":"2023-10-06T12:52:28.024831+0000","last_became_active":"2023-10-06T12:52:28.018176+0000","last_became_peered":"2023-10-06T12:52:28.018176+0000","last_unstale":"2023-10-06T12:52:28.024831+0000","last_undegraded":"2023-10-06T12:52:28.024831+0000","last_fullsized":"2023-10-06T12:52:28.024831+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008020+0000","last_change":"2023-10-06T12:52:28.001401+0000","last_active":"2023-10-06T12:52:28.008020+0000","last_peered":"2023-10-06T12:52:28.008020+0000","last_clean":"2023-10-06T12:52:28.008020+0000","last_became_active":"2023-10-06T12:52:28.000740+0000","last_became_peered":"2023-10-06T12:52:28.000740+0000","last_unstale":"2023-10-06T12:52:28.008020+0000","last_undegraded":"2023-10-06T12:52:28.008020+0000","last_fullsized":"2023-10-06T12:52:28.008020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:42.126935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023908+0000","last_change":"2023-10-06T12:52:28.022816+0000","last_active":"2023-10-06T12:52:28.023908+0000","last_peered":"2023-10-06T12:52:28.023908+0000","last_clean":"2023-10-06T12:52:28.023908+0000","last_became_active":"2023-10-06T12:52:28.022513+0000","last_became_peered":"2023-10-06T12:52:28.022513+0000","last_unstale":"2023-10-06T12:52:28.023908+0000","last_undegraded":"2023-10-06T12:52:28.023908+0000","last_fullsized":"2023-10-06T12:52:28.023908+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:39.150778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007771+0000","last_change":"2023-10-06T12:52:27.997436+0000","last_active":"2023-10-06T12:52:28.007771+0000","last_peered":"2023-10-06T12:52:28.007771+0000","last_clean":"2023-10-06T12:52:28.007771+0000","last_became_active":"2023-10-06T12:52:27.997101+0000","last_became_peered":"2023-10-06T12:52:27.997101+0000","last_unstale":"2023-10-06T12:52:28.007771+0000","last_undegraded":"2023-10-06T12:52:28.007771+0000","last_fullsized":"2023-10-06T12:52:28.007771+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:27.981585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015581+0000","last_change":"2023-10-06T12:52:28.011098+0000","last_active":"2023-10-06T12:52:28.015581+0000","last_peered":"2023-10-06T12:52:28.015581+0000","last_clean":"2023-10-06T12:52:28.015581+0000","last_became_active":"2023-10-06T12:52:28.009726+0000","last_became_peered":"2023-10-06T12:52:28.009726+0000","last_unstale":"2023-10-06T12:52:28.015581+0000","last_undegraded":"2023-10-06T12:52:28.015581+0000","last_fullsized":"2023-10-06T12:52:28.015581+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:44.832554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017040+0000","last_change":"2023-10-06T12:52:28.015479+0000","last_active":"2023-10-06T12:52:28.017040+0000","last_peered":"2023-10-06T12:52:28.017040+0000","last_clean":"2023-10-06T12:52:28.017040+0000","last_became_active":"2023-10-06T12:52:28.012607+0000","last_became_peered":"2023-10-06T12:52:28.012607+0000","last_unstale":"2023-10-06T12:52:28.017040+0000","last_undegraded":"2023-10-06T12:52:28.017040+0000","last_fullsized":"2023-10-06T12:52:28.017040+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:53.868689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023944+0000","last_change":"2023-10-06T12:52:28.020348+0000","last_active":"2023-10-06T12:52:28.023944+0000","last_peered":"2023-10-06T12:52:28.023944+0000","last_clean":"2023-10-06T12:52:28.023944+0000","last_became_active":"2023-10-06T12:52:28.018619+0000","last_became_peered":"2023-10-06T12:52:28.018619+0000","last_unstale":"2023-10-06T12:52:28.023944+0000","last_undegraded":"2023-10-06T12:52:28.023944+0000","last_fullsized":"2023-10-06T12:52:28.023944+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:39.150778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.004492+0000","last_change":"2023-10-06T12:52:27.994799+0000","last_active":"2023-10-06T12:52:28.004492+0000","last_peered":"2023-10-06T12:52:28.004492+0000","last_clean":"2023-10-06T12:52:28.004492+0000","last_became_active":"2023-10-06T12:52:27.994433+0000","last_became_peered":"2023-10-06T12:52:27.994433+0000","last_unstale":"2023-10-06T12:52:28.004492+0000","last_undegraded":"2023-10-06T12:52:28.004492+0000","last_fullsized":"2023-10-06T12:52:28.004492+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:16.163073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017871+0000","last_change":"2023-10-06T12:52:28.012749+0000","last_active":"2023-10-06T12:52:28.017871+0000","last_peered":"2023-10-06T12:52:28.017871+0000","last_clean":"2023-10-06T12:52:28.017871+0000","last_became_active":"2023-10-06T12:52:28.012021+0000","last_became_peered":"2023-10-06T12:52:28.012021+0000","last_unstale":"2023-10-06T12:52:28.017871+0000","last_undegraded":"2023-10-06T12:52:28.017871+0000","last_fullsized":"2023-10-06T12:52:28.017871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.989163+0000","last_change":"2023-10-06T12:52:27.008937+0000","last_active":"2023-10-06T12:52:27.989163+0000","last_peered":"2023-10-06T12:52:27.989163+0000","last_clean":"2023-10-06T12:52:27.989163+0000","last_became_active":"2023-10-06T12:52:27.008653+0000","last_became_peered":"2023-10-06T12:52:27.008653+0000","last_unstale":"2023-10-06T12:52:27.989163+0000","last_undegraded":"2023-10-06T12:52:27.989163+0000","last_fullsized":"2023-10-06T12:52:27.989163+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:50.245593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.007774+0000","last_change":"2023-10-06T12:52:28.003244+0000","last_active":"2023-10-06T12:52:28.007774+0000","last_peered":"2023-10-06T12:52:28.007774+0000","last_clean":"2023-10-06T12:52:28.007774+0000","last_became_active":"2023-10-06T12:52:28.002941+0000","last_became_peered":"2023-10-06T12:52:28.002941+0000","last_unstale":"2023-10-06T12:52:28.007774+0000","last_undegraded":"2023-10-06T12:52:28.007774+0000","last_fullsized":"2023-10-06T12:52:28.007774+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:08:17.957852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998316+0000","last_change":"2023-10-06T12:52:27.011668+0000","last_active":"2023-10-06T12:52:27.998316+0000","last_peered":"2023-10-06T12:52:27.998316+0000","last_clean":"2023-10-06T12:52:27.998316+0000","last_became_active":"2023-10-06T12:52:27.006682+0000","last_became_peered":"2023-10-06T12:52:27.006682+0000","last_unstale":"2023-10-06T12:52:27.998316+0000","last_undegraded":"2023-10-06T12:52:27.998316+0000","last_fullsized":"2023-10-06T12:52:27.998316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:20:13.836962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010548+0000","last_change":"2023-10-06T12:52:28.003701+0000","last_active":"2023-10-06T12:52:28.010548+0000","last_peered":"2023-10-06T12:52:28.010548+0000","last_clean":"2023-10-06T12:52:28.010548+0000","last_became_active":"2023-10-06T12:52:28.003402+0000","last_became_peered":"2023-10-06T12:52:28.003402+0000","last_unstale":"2023-10-06T12:52:28.010548+0000","last_undegraded":"2023-10-06T12:52:28.010548+0000","last_fullsized":"2023-10-06T12:52:28.010548+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:47:27.981585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"196'3","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033069+0000","last_change":"2023-10-06T12:52:26.999796+0000","last_active":"2023-10-06T12:52:28.033069+0000","last_peered":"2023-10-06T12:52:28.033069+0000","last_clean":"2023-10-06T12:52:28.033069+0000","last_became_active":"2023-10-06T12:52:26.999486+0000","last_became_peered":"2023-10-06T12:52:26.999486+0000","last_unstale":"2023-10-06T12:52:28.033069+0000","last_undegraded":"2023-10-06T12:52:28.033069+0000","last_fullsized":"2023-10-06T12:52:28.033069+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:00:56.790454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012790+0000","last_change":"2023-10-06T12:52:28.008114+0000","last_active":"2023-10-06T12:52:28.012790+0000","last_peered":"2023-10-06T12:52:28.012790+0000","last_clean":"2023-10-06T12:52:28.012790+0000","last_became_active":"2023-10-06T12:52:28.007784+0000","last_became_peered":"2023-10-06T12:52:28.007784+0000","last_unstale":"2023-10-06T12:52:28.012790+0000","last_undegraded":"2023-10-06T12:52:28.012790+0000","last_fullsized":"2023-10-06T12:52:28.012790+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:48:57.287695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"196'6","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.037986+0000","last_change":"2023-10-06T12:52:26.998830+0000","last_active":"2023-10-06T12:52:28.037986+0000","last_peered":"2023-10-06T12:52:28.037986+0000","last_clean":"2023-10-06T12:52:28.037986+0000","last_became_active":"2023-10-06T12:52:26.998528+0000","last_became_peered":"2023-10-06T12:52:26.998528+0000","last_unstale":"2023-10-06T12:52:28.037986+0000","last_undegraded":"2023-10-06T12:52:28.037986+0000","last_fullsized":"2023-10-06T12:52:28.037986+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:37.259985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.020883+0000","last_change":"2023-10-06T12:52:28.015945+0000","last_active":"2023-10-06T12:52:28.020883+0000","last_peered":"2023-10-06T12:52:28.020883+0000","last_clean":"2023-10-06T12:52:28.020883+0000","last_became_active":"2023-10-06T12:52:28.015654+0000","last_became_peered":"2023-10-06T12:52:28.015654+0000","last_unstale":"2023-10-06T12:52:28.020883+0000","last_undegraded":"2023-10-06T12:52:28.020883+0000","last_fullsized":"2023-10-06T12:52:28.020883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:58:11.297245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.035282+0000","last_change":"2023-10-06T12:52:27.009426+0000","last_active":"2023-10-06T12:52:28.035282+0000","last_peered":"2023-10-06T12:52:28.035282+0000","last_clean":"2023-10-06T12:52:28.035282+0000","last_became_active":"2023-10-06T12:52:27.009071+0000","last_became_peered":"2023-10-06T12:52:27.009071+0000","last_unstale":"2023-10-06T12:52:28.035282+0000","last_undegraded":"2023-10-06T12:52:28.035282+0000","last_fullsized":"2023-10-06T12:52:28.035282+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:45.701372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012457+0000","last_change":"2023-10-06T12:52:28.010409+0000","last_active":"2023-10-06T12:52:28.012457+0000","last_peered":"2023-10-06T12:52:28.012457+0000","last_clean":"2023-10-06T12:52:28.012457+0000","last_became_active":"2023-10-06T12:52:28.010074+0000","last_became_peered":"2023-10-06T12:52:28.010074+0000","last_unstale":"2023-10-06T12:52:28.012457+0000","last_undegraded":"2023-10-06T12:52:28.012457+0000","last_fullsized":"2023-10-06T12:52:28.012457+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:46.487911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000567+0000","last_change":"2023-10-06T12:52:27.011205+0000","last_active":"2023-10-06T12:52:28.000567+0000","last_peered":"2023-10-06T12:52:28.000567+0000","last_clean":"2023-10-06T12:52:28.000567+0000","last_became_active":"2023-10-06T12:52:27.008935+0000","last_became_peered":"2023-10-06T12:52:27.008935+0000","last_unstale":"2023-10-06T12:52:28.000567+0000","last_undegraded":"2023-10-06T12:52:28.000567+0000","last_fullsized":"2023-10-06T12:52:28.000567+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:31.450674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.014730+0000","last_change":"2023-10-06T12:52:28.004902+0000","last_active":"2023-10-06T12:52:28.014730+0000","last_peered":"2023-10-06T12:52:28.014730+0000","last_clean":"2023-10-06T12:52:28.014730+0000","last_became_active":"2023-10-06T12:52:28.004166+0000","last_became_peered":"2023-10-06T12:52:28.004166+0000","last_unstale":"2023-10-06T12:52:28.014730+0000","last_undegraded":"2023-10-06T12:52:28.014730+0000","last_fullsized":"2023-10-06T12:52:28.014730+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:55.552335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000298+0000","last_change":"2023-10-06T12:52:27.003933+0000","last_active":"2023-10-06T12:52:28.000298+0000","last_peered":"2023-10-06T12:52:28.000298+0000","last_clean":"2023-10-06T12:52:28.000298+0000","last_became_active":"2023-10-06T12:52:27.003656+0000","last_became_peered":"2023-10-06T12:52:27.003656+0000","last_unstale":"2023-10-06T12:52:28.000298+0000","last_undegraded":"2023-10-06T12:52:28.000298+0000","last_fullsized":"2023-10-06T12:52:28.000298+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:10:38.856122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017554+0000","last_change":"2023-10-06T12:52:28.014462+0000","last_active":"2023-10-06T12:52:28.017554+0000","last_peered":"2023-10-06T12:52:28.017554+0000","last_clean":"2023-10-06T12:52:28.017554+0000","last_became_active":"2023-10-06T12:52:28.014140+0000","last_became_peered":"2023-10-06T12:52:28.014140+0000","last_unstale":"2023-10-06T12:52:28.017554+0000","last_undegraded":"2023-10-06T12:52:28.017554+0000","last_fullsized":"2023-10-06T12:52:28.017554+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:41:20.901664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"196'8","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.034909+0000","last_change":"2023-10-06T12:52:27.000573+0000","last_active":"2023-10-06T12:52:28.034909+0000","last_peered":"2023-10-06T12:52:28.034909+0000","last_clean":"2023-10-06T12:52:28.034909+0000","last_became_active":"2023-10-06T12:52:27.000257+0000","last_became_peered":"2023-10-06T12:52:27.000257+0000","last_unstale":"2023-10-06T12:52:28.034909+0000","last_undegraded":"2023-10-06T12:52:28.034909+0000","last_fullsized":"2023-10-06T12:52:28.034909+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:01.237024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.024050+0000","last_change":"2023-10-06T12:52:28.021092+0000","last_active":"2023-10-06T12:52:28.024050+0000","last_peered":"2023-10-06T12:52:28.024050+0000","last_clean":"2023-10-06T12:52:28.024050+0000","last_became_active":"2023-10-06T12:52:28.020756+0000","last_became_peered":"2023-10-06T12:52:28.020756+0000","last_unstale":"2023-10-06T12:52:28.024050+0000","last_undegraded":"2023-10-06T12:52:28.024050+0000","last_fullsized":"2023-10-06T12:52:28.024050+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033874+0000","last_change":"2023-10-06T12:52:27.000614+0000","last_active":"2023-10-06T12:52:28.033874+0000","last_peered":"2023-10-06T12:52:28.033874+0000","last_clean":"2023-10-06T12:52:28.033874+0000","last_became_active":"2023-10-06T12:52:27.000316+0000","last_became_peered":"2023-10-06T12:52:27.000316+0000","last_unstale":"2023-10-06T12:52:28.033874+0000","last_undegraded":"2023-10-06T12:52:28.033874+0000","last_fullsized":"2023-10-06T12:52:28.033874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:34.456077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008002+0000","last_change":"2023-10-06T12:52:28.003733+0000","last_active":"2023-10-06T12:52:28.008002+0000","last_peered":"2023-10-06T12:52:28.008002+0000","last_clean":"2023-10-06T12:52:28.008002+0000","last_became_active":"2023-10-06T12:52:28.003293+0000","last_became_peered":"2023-10-06T12:52:28.003293+0000","last_unstale":"2023-10-06T12:52:28.008002+0000","last_undegraded":"2023-10-06T12:52:28.008002+0000","last_fullsized":"2023-10-06T12:52:28.008002+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:27.425021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032222+0000","last_change":"2023-10-06T12:52:27.004632+0000","last_active":"2023-10-06T12:52:28.032222+0000","last_peered":"2023-10-06T12:52:28.032222+0000","last_clean":"2023-10-06T12:52:28.032222+0000","last_became_active":"2023-10-06T12:52:27.004372+0000","last_became_peered":"2023-10-06T12:52:27.004372+0000","last_unstale":"2023-10-06T12:52:28.032222+0000","last_undegraded":"2023-10-06T12:52:28.032222+0000","last_fullsized":"2023-10-06T12:52:28.032222+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:01.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.008038+0000","last_change":"2023-10-06T12:52:28.001659+0000","last_active":"2023-10-06T12:52:28.008038+0000","last_peered":"2023-10-06T12:52:28.008038+0000","last_clean":"2023-10-06T12:52:28.008038+0000","last_became_active":"2023-10-06T12:52:28.001139+0000","last_became_peered":"2023-10-06T12:52:28.001139+0000","last_unstale":"2023-10-06T12:52:28.008038+0000","last_undegraded":"2023-10-06T12:52:28.008038+0000","last_fullsized":"2023-10-06T12:52:28.008038+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:37:49.292946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033237+0000","last_change":"2023-10-06T12:52:27.003853+0000","last_active":"2023-10-06T12:52:28.033237+0000","last_peered":"2023-10-06T12:52:28.033237+0000","last_clean":"2023-10-06T12:52:28.033237+0000","last_became_active":"2023-10-06T12:52:27.003500+0000","last_became_peered":"2023-10-06T12:52:27.003500+0000","last_unstale":"2023-10-06T12:52:28.033237+0000","last_undegraded":"2023-10-06T12:52:28.033237+0000","last_fullsized":"2023-10-06T12:52:28.033237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:42.920365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017767+0000","last_change":"2023-10-06T12:52:28.011711+0000","last_active":"2023-10-06T12:52:28.017767+0000","last_peered":"2023-10-06T12:52:28.017767+0000","last_clean":"2023-10-06T12:52:28.017767+0000","last_became_active":"2023-10-06T12:52:28.010565+0000","last_became_peered":"2023-10-06T12:52:28.010565+0000","last_unstale":"2023-10-06T12:52:28.017767+0000","last_undegraded":"2023-10-06T12:52:28.017767+0000","last_fullsized":"2023-10-06T12:52:28.017767+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:16.856488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.991699+0000","last_change":"2023-10-06T12:52:26.998580+0000","last_active":"2023-10-06T12:52:27.991699+0000","last_peered":"2023-10-06T12:52:27.991699+0000","last_clean":"2023-10-06T12:52:27.991699+0000","last_became_active":"2023-10-06T12:52:26.998269+0000","last_became_peered":"2023-10-06T12:52:26.998269+0000","last_unstale":"2023-10-06T12:52:27.991699+0000","last_undegraded":"2023-10-06T12:52:27.991699+0000","last_fullsized":"2023-10-06T12:52:27.991699+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:29:07.618783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015975+0000","last_change":"2023-10-06T12:52:28.010398+0000","last_active":"2023-10-06T12:52:28.015975+0000","last_peered":"2023-10-06T12:52:28.015975+0000","last_clean":"2023-10-06T12:52:28.015975+0000","last_became_active":"2023-10-06T12:52:28.010074+0000","last_became_peered":"2023-10-06T12:52:28.010074+0000","last_unstale":"2023-10-06T12:52:28.015975+0000","last_undegraded":"2023-10-06T12:52:28.015975+0000","last_fullsized":"2023-10-06T12:52:28.015975+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:58.432758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033363+0000","last_change":"2023-10-06T12:52:27.003914+0000","last_active":"2023-10-06T12:52:28.033363+0000","last_peered":"2023-10-06T12:52:28.033363+0000","last_clean":"2023-10-06T12:52:28.033363+0000","last_became_active":"2023-10-06T12:52:27.003550+0000","last_became_peered":"2023-10-06T12:52:27.003550+0000","last_unstale":"2023-10-06T12:52:28.033363+0000","last_undegraded":"2023-10-06T12:52:28.033363+0000","last_fullsized":"2023-10-06T12:52:28.033363+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:50.245593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012235+0000","last_change":"2023-10-06T12:52:28.006006+0000","last_active":"2023-10-06T12:52:28.012235+0000","last_peered":"2023-10-06T12:52:28.012235+0000","last_clean":"2023-10-06T12:52:28.012235+0000","last_became_active":"2023-10-06T12:52:28.005722+0000","last_became_peered":"2023-10-06T12:52:28.005722+0000","last_unstale":"2023-10-06T12:52:28.012235+0000","last_undegraded":"2023-10-06T12:52:28.012235+0000","last_fullsized":"2023-10-06T12:52:28.012235+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:48:42.057705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000969+0000","last_change":"2023-10-06T12:52:27.002243+0000","last_active":"2023-10-06T12:52:28.000969+0000","last_peered":"2023-10-06T12:52:28.000969+0000","last_clean":"2023-10-06T12:52:28.000969+0000","last_became_active":"2023-10-06T12:52:27.001864+0000","last_became_peered":"2023-10-06T12:52:27.001864+0000","last_unstale":"2023-10-06T12:52:28.000969+0000","last_undegraded":"2023-10-06T12:52:28.000969+0000","last_fullsized":"2023-10-06T12:52:28.000969+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:14.660171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012556+0000","last_change":"2023-10-06T12:52:28.003305+0000","last_active":"2023-10-06T12:52:28.012556+0000","last_peered":"2023-10-06T12:52:28.012556+0000","last_clean":"2023-10-06T12:52:28.012556+0000","last_became_active":"2023-10-06T12:52:28.002941+0000","last_became_peered":"2023-10-06T12:52:28.002941+0000","last_unstale":"2023-10-06T12:52:28.012556+0000","last_undegraded":"2023-10-06T12:52:28.012556+0000","last_fullsized":"2023-10-06T12:52:28.012556+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:26:26.641824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005825+0000","last_change":"2023-10-06T12:52:27.013158+0000","last_active":"2023-10-06T12:52:28.005825+0000","last_peered":"2023-10-06T12:52:28.005825+0000","last_clean":"2023-10-06T12:52:28.005825+0000","last_became_active":"2023-10-06T12:52:27.012837+0000","last_became_peered":"2023-10-06T12:52:27.012837+0000","last_unstale":"2023-10-06T12:52:28.005825+0000","last_undegraded":"2023-10-06T12:52:28.005825+0000","last_fullsized":"2023-10-06T12:52:28.005825+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T00:20:11.131639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.029293+0000","last_change":"2023-10-06T12:52:28.021734+0000","last_active":"2023-10-06T12:52:28.029293+0000","last_peered":"2023-10-06T12:52:28.029293+0000","last_clean":"2023-10-06T12:52:28.029293+0000","last_became_active":"2023-10-06T12:52:28.021428+0000","last_became_peered":"2023-10-06T12:52:28.021428+0000","last_unstale":"2023-10-06T12:52:28.029293+0000","last_undegraded":"2023-10-06T12:52:28.029293+0000","last_fullsized":"2023-10-06T12:52:28.029293+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:46:35.471617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998414+0000","last_change":"2023-10-06T12:52:26.995918+0000","last_active":"2023-10-06T12:52:27.998414+0000","last_peered":"2023-10-06T12:52:27.998414+0000","last_clean":"2023-10-06T12:52:27.998414+0000","last_became_active":"2023-10-06T12:52:26.995600+0000","last_became_peered":"2023-10-06T12:52:26.995600+0000","last_unstale":"2023-10-06T12:52:27.998414+0000","last_undegraded":"2023-10-06T12:52:27.998414+0000","last_fullsized":"2023-10-06T12:52:27.998414+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:42:19.213954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.020501+0000","last_change":"2023-10-06T12:52:28.017455+0000","last_active":"2023-10-06T12:52:28.020501+0000","last_peered":"2023-10-06T12:52:28.020501+0000","last_clean":"2023-10-06T12:52:28.020501+0000","last_became_active":"2023-10-06T12:52:28.017156+0000","last_became_peered":"2023-10-06T12:52:28.017156+0000","last_unstale":"2023-10-06T12:52:28.020501+0000","last_undegraded":"2023-10-06T12:52:28.020501+0000","last_fullsized":"2023-10-06T12:52:28.020501+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:24.346419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032392+0000","last_change":"2023-10-06T12:52:26.999491+0000","last_active":"2023-10-06T12:52:28.032392+0000","last_peered":"2023-10-06T12:52:28.032392+0000","last_clean":"2023-10-06T12:52:28.032392+0000","last_became_active":"2023-10-06T12:52:26.999166+0000","last_became_peered":"2023-10-06T12:52:26.999166+0000","last_unstale":"2023-10-06T12:52:28.032392+0000","last_undegraded":"2023-10-06T12:52:28.032392+0000","last_fullsized":"2023-10-06T12:52:28.032392+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:14.411876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.027200+0000","last_change":"2023-10-06T12:52:28.025077+0000","last_active":"2023-10-06T12:52:28.027200+0000","last_peered":"2023-10-06T12:52:28.027200+0000","last_clean":"2023-10-06T12:52:28.027200+0000","last_became_active":"2023-10-06T12:52:28.023626+0000","last_became_peered":"2023-10-06T12:52:28.023626+0000","last_unstale":"2023-10-06T12:52:28.027200+0000","last_undegraded":"2023-10-06T12:52:28.027200+0000","last_fullsized":"2023-10-06T12:52:28.027200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:46.716912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"197'1","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T12:52:29.153485+0000","last_change":"2023-10-06T12:52:27.014697+0000","last_active":"2023-10-06T12:52:29.153485+0000","last_peered":"2023-10-06T12:52:29.153485+0000","last_clean":"2023-10-06T12:52:29.153485+0000","last_became_active":"2023-10-06T12:52:27.014377+0000","last_became_peered":"2023-10-06T12:52:27.014377+0000","last_unstale":"2023-10-06T12:52:29.153485+0000","last_undegraded":"2023-10-06T12:52:29.153485+0000","last_fullsized":"2023-10-06T12:52:29.153485+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:42:06.003673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015960+0000","last_change":"2023-10-06T12:52:28.010792+0000","last_active":"2023-10-06T12:52:28.015960+0000","last_peered":"2023-10-06T12:52:28.015960+0000","last_clean":"2023-10-06T12:52:28.015960+0000","last_became_active":"2023-10-06T12:52:28.010475+0000","last_became_peered":"2023-10-06T12:52:28.010475+0000","last_unstale":"2023-10-06T12:52:28.015960+0000","last_undegraded":"2023-10-06T12:52:28.015960+0000","last_fullsized":"2023-10-06T12:52:28.015960+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.990065+0000","last_change":"2023-10-06T12:52:27.004657+0000","last_active":"2023-10-06T12:52:27.990065+0000","last_peered":"2023-10-06T12:52:27.990065+0000","last_clean":"2023-10-06T12:52:27.990065+0000","last_became_active":"2023-10-06T12:52:27.004224+0000","last_became_peered":"2023-10-06T12:52:27.004224+0000","last_unstale":"2023-10-06T12:52:27.990065+0000","last_undegraded":"2023-10-06T12:52:27.990065+0000","last_fullsized":"2023-10-06T12:52:27.990065+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:37.259985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.023991+0000","last_change":"2023-10-06T12:52:28.019806+0000","last_active":"2023-10-06T12:52:28.023991+0000","last_peered":"2023-10-06T12:52:28.023991+0000","last_clean":"2023-10-06T12:52:28.023991+0000","last_became_active":"2023-10-06T12:52:28.017823+0000","last_became_peered":"2023-10-06T12:52:28.017823+0000","last_unstale":"2023-10-06T12:52:28.023991+0000","last_undegraded":"2023-10-06T12:52:28.023991+0000","last_fullsized":"2023-10-06T12:52:28.023991+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:39.665430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.990085+0000","last_change":"2023-10-06T12:52:27.014559+0000","last_active":"2023-10-06T12:52:27.990085+0000","last_peered":"2023-10-06T12:52:27.990085+0000","last_clean":"2023-10-06T12:52:27.990085+0000","last_became_active":"2023-10-06T12:52:27.014225+0000","last_became_peered":"2023-10-06T12:52:27.014225+0000","last_unstale":"2023-10-06T12:52:27.990085+0000","last_undegraded":"2023-10-06T12:52:27.990085+0000","last_fullsized":"2023-10-06T12:52:27.990085+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:38.649889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.009599+0000","last_change":"2023-10-06T12:52:28.006080+0000","last_active":"2023-10-06T12:52:28.009599+0000","last_peered":"2023-10-06T12:52:28.009599+0000","last_clean":"2023-10-06T12:52:28.009599+0000","last_became_active":"2023-10-06T12:52:28.005789+0000","last_became_peered":"2023-10-06T12:52:28.005789+0000","last_unstale":"2023-10-06T12:52:28.009599+0000","last_undegraded":"2023-10-06T12:52:28.009599+0000","last_fullsized":"2023-10-06T12:52:28.009599+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:53.877552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999546+0000","last_change":"2023-10-06T12:52:27.005220+0000","last_active":"2023-10-06T12:52:27.999546+0000","last_peered":"2023-10-06T12:52:27.999546+0000","last_clean":"2023-10-06T12:52:27.999546+0000","last_became_active":"2023-10-06T12:52:27.004840+0000","last_became_peered":"2023-10-06T12:52:27.004840+0000","last_unstale":"2023-10-06T12:52:27.999546+0000","last_undegraded":"2023-10-06T12:52:27.999546+0000","last_fullsized":"2023-10-06T12:52:27.999546+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:51:57.417218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.016038+0000","last_change":"2023-10-06T12:52:28.008930+0000","last_active":"2023-10-06T12:52:28.016038+0000","last_peered":"2023-10-06T12:52:28.016038+0000","last_clean":"2023-10-06T12:52:28.016038+0000","last_became_active":"2023-10-06T12:52:28.008646+0000","last_became_peered":"2023-10-06T12:52:28.008646+0000","last_unstale":"2023-10-06T12:52:28.016038+0000","last_undegraded":"2023-10-06T12:52:28.016038+0000","last_fullsized":"2023-10-06T12:52:28.016038+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:46.487911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999375+0000","last_change":"2023-10-06T12:52:27.009451+0000","last_active":"2023-10-06T12:52:27.999375+0000","last_peered":"2023-10-06T12:52:27.999375+0000","last_clean":"2023-10-06T12:52:27.999375+0000","last_became_active":"2023-10-06T12:52:27.009130+0000","last_became_peered":"2023-10-06T12:52:27.009130+0000","last_unstale":"2023-10-06T12:52:27.999375+0000","last_undegraded":"2023-10-06T12:52:27.999375+0000","last_fullsized":"2023-10-06T12:52:27.999375+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:21:15.840947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017705+0000","last_change":"2023-10-06T12:52:28.012383+0000","last_active":"2023-10-06T12:52:28.017705+0000","last_peered":"2023-10-06T12:52:28.017705+0000","last_clean":"2023-10-06T12:52:28.017705+0000","last_became_active":"2023-10-06T12:52:28.011392+0000","last_became_peered":"2023-10-06T12:52:28.011392+0000","last_unstale":"2023-10-06T12:52:28.017705+0000","last_undegraded":"2023-10-06T12:52:28.017705+0000","last_fullsized":"2023-10-06T12:52:28.017705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:16:51.261133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005718+0000","last_change":"2023-10-06T12:52:27.002635+0000","last_active":"2023-10-06T12:52:28.005718+0000","last_peered":"2023-10-06T12:52:28.005718+0000","last_clean":"2023-10-06T12:52:28.005718+0000","last_became_active":"2023-10-06T12:52:27.002224+0000","last_became_peered":"2023-10-06T12:52:27.002224+0000","last_unstale":"2023-10-06T12:52:28.005718+0000","last_undegraded":"2023-10-06T12:52:28.005718+0000","last_fullsized":"2023-10-06T12:52:28.005718+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.609227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.015929+0000","last_change":"2023-10-06T12:52:28.011888+0000","last_active":"2023-10-06T12:52:28.015929+0000","last_peered":"2023-10-06T12:52:28.015929+0000","last_clean":"2023-10-06T12:52:28.015929+0000","last_became_active":"2023-10-06T12:52:28.011518+0000","last_became_peered":"2023-10-06T12:52:28.011518+0000","last_unstale":"2023-10-06T12:52:28.015929+0000","last_undegraded":"2023-10-06T12:52:28.015929+0000","last_fullsized":"2023-10-06T12:52:28.015929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:19:52.220289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998527+0000","last_change":"2023-10-06T12:52:27.005454+0000","last_active":"2023-10-06T12:52:27.998527+0000","last_peered":"2023-10-06T12:52:27.998527+0000","last_clean":"2023-10-06T12:52:27.998527+0000","last_became_active":"2023-10-06T12:52:27.005138+0000","last_became_peered":"2023-10-06T12:52:27.005138+0000","last_unstale":"2023-10-06T12:52:27.998527+0000","last_undegraded":"2023-10-06T12:52:27.998527+0000","last_fullsized":"2023-10-06T12:52:27.998527+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:55:27.092380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.012567+0000","last_change":"2023-10-06T12:52:28.010210+0000","last_active":"2023-10-06T12:52:28.012567+0000","last_peered":"2023-10-06T12:52:28.012567+0000","last_clean":"2023-10-06T12:52:28.012567+0000","last_became_active":"2023-10-06T12:52:28.009857+0000","last_became_peered":"2023-10-06T12:52:28.009857+0000","last_unstale":"2023-10-06T12:52:28.012567+0000","last_undegraded":"2023-10-06T12:52:28.012567+0000","last_fullsized":"2023-10-06T12:52:28.012567+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:25:53.941098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.000066+0000","last_change":"2023-10-06T12:52:26.999893+0000","last_active":"2023-10-06T12:52:28.000066+0000","last_peered":"2023-10-06T12:52:28.000066+0000","last_clean":"2023-10-06T12:52:28.000066+0000","last_became_active":"2023-10-06T12:52:26.999589+0000","last_became_peered":"2023-10-06T12:52:26.999589+0000","last_unstale":"2023-10-06T12:52:28.000066+0000","last_undegraded":"2023-10-06T12:52:28.000066+0000","last_fullsized":"2023-10-06T12:52:28.000066+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:52.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"186'348","reported_seq":1131,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999435+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:27.999435+0000","last_peered":"2023-10-06T12:52:27.999435+0000","last_clean":"2023-10-06T12:52:27.999435+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:27.999435+0000","last_undegraded":"2023-10-06T12:52:27.999435+0000","last_fullsized":"2023-10-06T12:52:27.999435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":181,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.991029+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:27.991029+0000","last_peered":"2023-10-06T12:52:27.991029+0000","last_clean":"2023-10-06T12:52:27.991029+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:27.991029+0000","last_undegraded":"2023-10-06T12:52:27.991029+0000","last_fullsized":"2023-10-06T12:52:27.991029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.003741+0000","last_change":"2023-10-06T12:52:27.997760+0000","last_active":"2023-10-06T12:52:28.003741+0000","last_peered":"2023-10-06T12:52:28.003741+0000","last_clean":"2023-10-06T12:52:28.003741+0000","last_became_active":"2023-10-06T12:52:27.997443+0000","last_became_peered":"2023-10-06T12:52:27.997443+0000","last_unstale":"2023-10-06T12:52:28.003741+0000","last_undegraded":"2023-10-06T12:52:28.003741+0000","last_fullsized":"2023-10-06T12:52:28.003741+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:16.163073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.030604+0000","last_change":"2023-10-06T12:52:27.000987+0000","last_active":"2023-10-06T12:52:28.030604+0000","last_peered":"2023-10-06T12:52:28.030604+0000","last_clean":"2023-10-06T12:52:28.030604+0000","last_became_active":"2023-10-06T12:52:27.000617+0000","last_became_peered":"2023-10-06T12:52:27.000617+0000","last_unstale":"2023-10-06T12:52:28.030604+0000","last_undegraded":"2023-10-06T12:52:28.030604+0000","last_fullsized":"2023-10-06T12:52:28.030604+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:47.134670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999341+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:27.999341+0000","last_peered":"2023-10-06T12:52:27.999341+0000","last_clean":"2023-10-06T12:52:27.999341+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:27.999341+0000","last_undegraded":"2023-10-06T12:52:27.999341+0000","last_fullsized":"2023-10-06T12:52:27.999341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021593+0000","last_change":"2023-10-06T12:52:28.019292+0000","last_active":"2023-10-06T12:52:28.021593+0000","last_peered":"2023-10-06T12:52:28.021593+0000","last_clean":"2023-10-06T12:52:28.021593+0000","last_became_active":"2023-10-06T12:52:28.018945+0000","last_became_peered":"2023-10-06T12:52:28.018945+0000","last_unstale":"2023-10-06T12:52:28.021593+0000","last_undegraded":"2023-10-06T12:52:28.021593+0000","last_fullsized":"2023-10-06T12:52:28.021593+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:29.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032980+0000","last_change":"2023-10-06T12:52:27.008216+0000","last_active":"2023-10-06T12:52:28.032980+0000","last_peered":"2023-10-06T12:52:28.032980+0000","last_clean":"2023-10-06T12:52:28.032980+0000","last_became_active":"2023-10-06T12:52:27.007775+0000","last_became_peered":"2023-10-06T12:52:27.007775+0000","last_unstale":"2023-10-06T12:52:28.032980+0000","last_undegraded":"2023-10-06T12:52:28.032980+0000","last_fullsized":"2023-10-06T12:52:28.032980+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:02:37.367512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.005908+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:28.005908+0000","last_peered":"2023-10-06T12:52:28.005908+0000","last_clean":"2023-10-06T12:52:28.005908+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:28.005908+0000","last_undegraded":"2023-10-06T12:52:28.005908+0000","last_fullsized":"2023-10-06T12:52:28.005908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017728+0000","last_change":"2023-10-06T12:52:28.012887+0000","last_active":"2023-10-06T12:52:28.017728+0000","last_peered":"2023-10-06T12:52:28.017728+0000","last_clean":"2023-10-06T12:52:28.017728+0000","last_became_active":"2023-10-06T12:52:28.011870+0000","last_became_peered":"2023-10-06T12:52:28.011870+0000","last_unstale":"2023-10-06T12:52:28.017728+0000","last_undegraded":"2023-10-06T12:52:28.017728+0000","last_fullsized":"2023-10-06T12:52:28.017728+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:11.257569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.993415+0000","last_change":"2023-10-06T12:52:27.002628+0000","last_active":"2023-10-06T12:52:27.993415+0000","last_peered":"2023-10-06T12:52:27.993415+0000","last_clean":"2023-10-06T12:52:27.993415+0000","last_became_active":"2023-10-06T12:52:27.002333+0000","last_became_peered":"2023-10-06T12:52:27.002333+0000","last_unstale":"2023-10-06T12:52:27.993415+0000","last_undegraded":"2023-10-06T12:52:27.993415+0000","last_fullsized":"2023-10-06T12:52:27.993415+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:48:38.649889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":181,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.992142+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:27.992142+0000","last_peered":"2023-10-06T12:52:27.992142+0000","last_clean":"2023-10-06T12:52:27.992142+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:27.992142+0000","last_undegraded":"2023-10-06T12:52:27.992142+0000","last_fullsized":"2023-10-06T12:52:27.992142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.017939+0000","last_change":"2023-10-06T12:52:28.014605+0000","last_active":"2023-10-06T12:52:28.017939+0000","last_peered":"2023-10-06T12:52:28.017939+0000","last_clean":"2023-10-06T12:52:28.017939+0000","last_became_active":"2023-10-06T12:52:28.014325+0000","last_became_peered":"2023-10-06T12:52:28.014325+0000","last_unstale":"2023-10-06T12:52:28.017939+0000","last_undegraded":"2023-10-06T12:52:28.017939+0000","last_fullsized":"2023-10-06T12:52:28.017939+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:03:30.683577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"196'4","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.033105+0000","last_change":"2023-10-06T12:52:27.001679+0000","last_active":"2023-10-06T12:52:28.033105+0000","last_peered":"2023-10-06T12:52:28.033105+0000","last_clean":"2023-10-06T12:52:28.033105+0000","last_became_active":"2023-10-06T12:52:27.001355+0000","last_became_peered":"2023-10-06T12:52:27.001355+0000","last_unstale":"2023-10-06T12:52:28.033105+0000","last_undegraded":"2023-10-06T12:52:28.033105+0000","last_fullsized":"2023-10-06T12:52:28.033105+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:28.092062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.997436+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:27.997436+0000","last_peered":"2023-10-06T12:52:27.997436+0000","last_clean":"2023-10-06T12:52:27.997436+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:27.997436+0000","last_undegraded":"2023-10-06T12:52:27.997436+0000","last_fullsized":"2023-10-06T12:52:27.997436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.021938+0000","last_change":"2023-10-06T12:52:28.019332+0000","last_active":"2023-10-06T12:52:28.021938+0000","last_peered":"2023-10-06T12:52:28.021938+0000","last_clean":"2023-10-06T12:52:28.021938+0000","last_became_active":"2023-10-06T12:52:28.019016+0000","last_became_peered":"2023-10-06T12:52:28.019016+0000","last_unstale":"2023-10-06T12:52:28.021938+0000","last_undegraded":"2023-10-06T12:52:28.021938+0000","last_fullsized":"2023-10-06T12:52:28.021938+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:31:38.275526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.031621+0000","last_change":"2023-10-06T12:52:27.002052+0000","last_active":"2023-10-06T12:52:28.031621+0000","last_peered":"2023-10-06T12:52:28.031621+0000","last_clean":"2023-10-06T12:52:28.031621+0000","last_became_active":"2023-10-06T12:52:27.000367+0000","last_became_peered":"2023-10-06T12:52:27.000367+0000","last_unstale":"2023-10-06T12:52:28.031621+0000","last_undegraded":"2023-10-06T12:52:28.031621+0000","last_fullsized":"2023-10-06T12:52:28.031621+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:29:23.330879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.001376+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:28.001376+0000","last_peered":"2023-10-06T12:52:28.001376+0000","last_clean":"2023-10-06T12:52:28.001376+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:28.001376+0000","last_undegraded":"2023-10-06T12:52:28.001376+0000","last_fullsized":"2023-10-06T12:52:28.001376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.010588+0000","last_change":"2023-10-06T12:52:28.006040+0000","last_active":"2023-10-06T12:52:28.010588+0000","last_peered":"2023-10-06T12:52:28.010588+0000","last_clean":"2023-10-06T12:52:28.010588+0000","last_became_active":"2023-10-06T12:52:28.005698+0000","last_became_peered":"2023-10-06T12:52:28.005698+0000","last_unstale":"2023-10-06T12:52:28.010588+0000","last_undegraded":"2023-10-06T12:52:28.010588+0000","last_fullsized":"2023-10-06T12:52:28.010588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:09.341415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.027829+0000","last_change":"2023-10-06T12:52:26.998978+0000","last_active":"2023-10-06T12:52:28.027829+0000","last_peered":"2023-10-06T12:52:28.027829+0000","last_clean":"2023-10-06T12:52:28.027829+0000","last_became_active":"2023-10-06T12:52:26.998528+0000","last_became_peered":"2023-10-06T12:52:26.998528+0000","last_unstale":"2023-10-06T12:52:28.027829+0000","last_undegraded":"2023-10-06T12:52:28.027829+0000","last_fullsized":"2023-10-06T12:52:28.027829+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:25:26.409480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.998414+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:27.998414+0000","last_peered":"2023-10-06T12:52:27.998414+0000","last_clean":"2023-10-06T12:52:27.998414+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:27.998414+0000","last_undegraded":"2023-10-06T12:52:27.998414+0000","last_fullsized":"2023-10-06T12:52:27.998414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.013706+0000","last_change":"2023-10-06T12:52:28.007813+0000","last_active":"2023-10-06T12:52:28.013706+0000","last_peered":"2023-10-06T12:52:28.013706+0000","last_clean":"2023-10-06T12:52:28.013706+0000","last_became_active":"2023-10-06T12:52:28.007473+0000","last_became_peered":"2023-10-06T12:52:28.007473+0000","last_unstale":"2023-10-06T12:52:28.013706+0000","last_undegraded":"2023-10-06T12:52:28.013706+0000","last_fullsized":"2023-10-06T12:52:28.013706+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:26.962837+0000","last_clean_scrub_stamp":"2023-10-06T12:52:26.962837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:08:16.299803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:28.032706+0000","last_change":"2023-10-06T12:52:27.008092+0000","last_active":"2023-10-06T12:52:28.032706+0000","last_peered":"2023-10-06T12:52:28.032706+0000","last_clean":"2023-10-06T12:52:28.032706+0000","last_became_active":"2023-10-06T12:52:27.007681+0000","last_became_peered":"2023-10-06T12:52:27.007681+0000","last_unstale":"2023-10-06T12:52:28.032706+0000","last_undegraded":"2023-10-06T12:52:28.032706+0000","last_fullsized":"2023-10-06T12:52:28.032706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:52:25.947297+0000","last_clean_scrub_stamp":"2023-10-06T12:52:25.947297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:28.210983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-06T12:52:27.999111+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:27.999111+0000","last_peered":"2023-10-06T12:52:27.999111+0000","last_clean":"2023-10-06T12:52:27.999111+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:27.999111+0000","last_undegraded":"2023-10-06T12:52:27.999111+0000","last_fullsized":"2023-10-06T12:52:27.999111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304376,"kb_used_data":2992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067464,"statfs":{"total":96636764160,"available":96325083136,"internally_reserved":0,"allocated":3063808,"data_stored":1904405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":1.381},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.4870000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.081}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.788},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.8680000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.7569999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":2.1779999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.998},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.534}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":2.2839999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299704,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072136,"statfs":{"total":96636764160,"available":96329867264,"internally_reserved":0,"allocated":2473984,"data_stored":1314107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.91300000000000003}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.2589999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.214}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.355},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.169}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.597}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":1.387},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.4670000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312004,"kb_used_data":2428,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059836,"statfs":{"total":96636764160,"available":96317272064,"internally_reserved":0,"allocated":2486272,"data_stored":1315595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":1.198},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":1.478}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":1.429},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.153}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.393},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.087}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.236}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.3560000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312004,"kb_used_data":2428,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059836,"statfs":{"total":96636764160,"available":96317272064,"internally_reserved":0,"allocated":2486272,"data_stored":1315607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.163},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":1.5449999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.395},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.1930000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.357},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.1299999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.274}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":1.48},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.2310000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.319},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4550000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575973,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299708,"kb_used_data":2420,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072132,"statfs":{"total":96636764160,"available":96329863168,"internally_reserved":0,"allocated":2478080,"data_stored":1314197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.0069999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.071},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.6140000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.095},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.141}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.51},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.552}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.3069999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311992,"kb_used_data":2416,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059848,"statfs":{"total":96636764160,"available":96317284352,"internally_reserved":0,"allocated":2473984,"data_stored":1314591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.139}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":1.1839999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":1.431}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":1.008},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":1.3720000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":1.325},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":1.3}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":1.25},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":1.534}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.228},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":1.4590000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575971,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311988,"kb_used_data":2412,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059852,"statfs":{"total":96636764160,"available":96317288448,"internally_reserved":0,"allocated":2469888,"data_stored":1314037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.1220000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.28}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.365}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.25},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.657}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.444},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.3400000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.419},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.6990000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.1679999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575972,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304392,"kb_used_data":3008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067448,"statfs":{"total":96636764160,"available":96325066752,"internally_reserved":0,"allocated":3080192,"data_stored":1905105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":0.627}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.3979999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.113}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":0.875}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.296}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.157},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:52:35.848 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T12:52:35.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T12:52:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.027+0000 7fa12228e700 1 Processor -- start 2023-10-06T12:52:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.027+0000 7fa12228e700 1 -- start start 2023-10-06T12:52:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa11c1590b0 0x7fa11c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa11c0b2420 0x7fa11c067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0cc8b0 con 0x7fa11c067e50 2023-10-06T12:52:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c068280 con 0x7fa11c1590b0 2023-10-06T12:52:36.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0cbca0 con 0x7fa11c0b2420 2023-10-06T12:52:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa11c1590b0 0x7fa11c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa11c1590b0 0x7fa11c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50744/0 (socket says 172.21.15.46:50744) 2023-10-06T12:52:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 -- 172.21.15.46:0/2576612462 learned_addr learned my addr 172.21.15.46:0/2576612462 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa120a8b700 1 --2- 172.21.15.46:0/2576612462 >> v2:172.21.15.132:3300/0 conn(0x7fa11c0b2420 0x7fa11c067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 -- 172.21.15.46:0/2576612462 >> v2:172.21.15.132:3300/0 conn(0x7fa11c0b2420 msgr2=0x7fa11c067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 --2- 172.21.15.46:0/2576612462 >> v2:172.21.15.132:3300/0 conn(0x7fa11c0b2420 0x7fa11c067910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 -- 172.21.15.46:0/2576612462 >> v1:172.21.15.132:6789/0 conn(0x7fa11c067e50 legacy=0x7fa11c0691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 -- 172.21.15.46:0/2576612462 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0cbf80 con 0x7fa11c1590b0 2023-10-06T12:52:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12128c700 1 --2- 172.21.15.46:0/2576612462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c0cb600 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fa11c06a210 tx=0x7fa118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d25196188b7f2e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa113fff700 1 -- 172.21.15.46:0/2576612462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118009e70 con 0x7fa11c1590b0 2023-10-06T12:52:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa113fff700 1 -- 172.21.15.46:0/2576612462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800b860 con 0x7fa11c1590b0 2023-10-06T12:52:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa113fff700 1 -- 172.21.15.46:0/2576612462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800bc00 con 0x7fa11c1590b0 2023-10-06T12:52:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 -- 172.21.15.46:0/2576612462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 msgr2=0x7fa11c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.031+0000 7fa12228e700 1 --2- 172.21.15.46:0/2576612462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c0cb600 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fa11c06a210 tx=0x7fa118009250 comp rx=0 tx=0).stop 2023-10-06T12:52:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 -- 172.21.15.46:0/2576612462 shutdown_connections 2023-10-06T12:52:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 --2- 172.21.15.46:0/2576612462 >> v2:172.21.15.132:3300/0 conn(0x7fa11c0b2420 0x7fa11c067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 --2- 172.21.15.46:0/2576612462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c0cb600 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 -- 172.21.15.46:0/2576612462 >> 172.21.15.46:0/2576612462 conn(0x7fa11c0cfff0 msgr2=0x7fa11c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 -- 172.21.15.46:0/2576612462 shutdown_connections 2023-10-06T12:52:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 -- 172.21.15.46:0/2576612462 wait complete. 2023-10-06T12:52:36.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.035+0000 7fa12228e700 1 Processor -- start 2023-10-06T12:52:36.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 -- start start 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 0x7fa11c150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11c0b2420 0x7fa11c0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c76e0 con 0x7fa11c1590b0 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15a110 con 0x7fa11c067e50 2023-10-06T12:52:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12228e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15a3f0 con 0x7fa11c0b2420 2023-10-06T12:52:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa121a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 0x7fa11c150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa121a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c14fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50754/0 (socket says 172.21.15.46:50754) 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa121a8d700 1 -- 172.21.15.46:0/2024590253 learned_addr learned my addr 172.21.15.46:0/2024590253 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa120a8b700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11c0b2420 0x7fa11c0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 -- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11c0b2420 msgr2=0x7fa11c0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11c0b2420 0x7fa11c0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 -- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 msgr2=0x7fa11c14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.039+0000 7fa12128c700 1 -- 172.21.15.46:0/2024590253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180094f0 con 0x7fa11c067e50 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa12128c700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 0x7fa11c150440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa118013000 tx=0x7fa118017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800bbe0 con 0x7fa11c067e50 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118000c90 con 0x7fa11c067e50 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118011650 con 0x7fa11c067e50 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c15c250 con 0x7fa11c067e50 2023-10-06T12:52:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c0c7e90 con 0x7fa11c067e50 2023-10-06T12:52:36.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.043+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c064590 con 0x7fa11c067e50 2023-10-06T12:52:36.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.047+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa118011a40 con 0x7fa11c067e50 2023-10-06T12:52:36.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.047+0000 7fa1117fa700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 0x7fa100062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.047+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fa11807ebc0 con 0x7fa11c067e50 2023-10-06T12:52:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.047+0000 7fa120a8b700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 0x7fa100062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.051+0000 7fa120a8b700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 0x7fa100062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cef00 tx=0x7fa10c0091a0 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.055+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa1180526f0 con 0x7fa11c067e50 2023-10-06T12:52:36.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.283+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa11c0cc260 con 0x7fa11c067e50 2023-10-06T12:52:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.299+0000 7fa1117fa700 1 -- 172.21.15.46:0/2024590253 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v50) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa118052280 con 0x7fa11c067e50 2023-10-06T12:52:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.299+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 msgr2=0x7fa100062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.299+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 0x7fa100062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cef00 tx=0x7fa10c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.299+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 msgr2=0x7fa11c150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.299+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 0x7fa11c150440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa118013000 tx=0x7fa118017000 comp rx=0 tx=0).stop 2023-10-06T12:52:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 shutdown_connections 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa11c1590b0 0x7fa11c14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11c0b2420 0x7fa11c0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa100060670 0x7fa100062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 --2- 172.21.15.46:0/2024590253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa11c067e50 0x7fa11c150440 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 >> 172.21.15.46:0/2024590253 conn(0x7fa11c0cfff0 msgr2=0x7fa11c0cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 shutdown_connections 2023-10-06T12:52:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.303+0000 7fa12228e700 1 -- 172.21.15.46:0/2024590253 wait complete. 2023-10-06T12:52:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-10-06T12:52:36.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 Processor -- start 2023-10-06T12:52:36.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 -- start start 2023-10-06T12:52:36.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea0415ab60 0x7fea0406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea040ad6c0 0x7fea040adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0406d460 con 0x7fea0415af30 2023-10-06T12:52:36.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040c7000 con 0x7fea0415ab60 2023-10-06T12:52:36.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.487+0000 7fea0c2ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c72c0 con 0x7fea040ad6c0 2023-10-06T12:52:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea09849700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fea0415af30 0x7fea0406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36256/0 (socket says 172.21.15.46:36256) 2023-10-06T12:52:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fea0415ab60 0x7fea0406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea09849700 1 -- 172.21.15.46:0/2717803747 learned_addr learned my addr 172.21.15.46:0/2717803747 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a84b700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea040ad6c0 0x7fea040adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 -- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea040ad6c0 msgr2=0x7fea040adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea040ad6c0 0x7fea040adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 -- 172.21.15.46:0/2717803747 >> v1:172.21.15.46:6789/0 conn(0x7fea0415af30 legacy=0x7fea0406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 -- 172.21.15.46:0/2717803747 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040c75a0 con 0x7fea0415ab60 2023-10-06T12:52:36.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea0415ab60 0x7fea0406e8a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fea000098d0 tx=0x7fea00009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6282ecf9791dd9ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea09048700 1 -- 172.21.15.46:0/2717803747 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00005c10 con 0x7fea0415ab60 2023-10-06T12:52:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea09048700 1 -- 172.21.15.46:0/2717803747 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00005d70 con 0x7fea0415ab60 2023-10-06T12:52:36.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.491+0000 7fea09048700 1 -- 172.21.15.46:0/2717803747 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00006280 con 0x7fea0415ab60 2023-10-06T12:52:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea0415ab60 msgr2=0x7fea0406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea0415ab60 0x7fea0406e8a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fea000098d0 tx=0x7fea00009120 comp rx=0 tx=0).stop 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2717803747 shutdown_connections 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea040ad6c0 0x7fea040adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2717803747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea0415ab60 0x7fea0406e8a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2717803747 >> 172.21.15.46:0/2717803747 conn(0x7fea040cbe30 msgr2=0x7fea040cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2717803747 shutdown_connections 2023-10-06T12:52:36.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2717803747 wait complete. 2023-10-06T12:52:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 Processor -- start 2023-10-06T12:52:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.495+0000 7fea0c2ae700 1 -- start start 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 0x7fea040c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea0415ab60 0x7fea040ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04169830 con 0x7fea0415af30 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04169b10 con 0x7fea040ad6c0 2023-10-06T12:52:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea04169df0 con 0x7fea0415ab60 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50770/0 (socket says 172.21.15.46:50770) 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 -- 172.21.15.46:0/2418331880 learned_addr learned my addr 172.21.15.46:0/2418331880 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea09849700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea0415ab60 0x7fea040ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 0x7fea040c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 -- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea0415ab60 msgr2=0x7fea040ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea0415ab60 0x7fea040ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 -- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 msgr2=0x7fea040c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 0x7fea040c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 -- 172.21.15.46:0/2418331880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000093c0 con 0x7fea0415af30 2023-10-06T12:52:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 0x7fea040c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0a84b700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009df0 tx=0x7fe9fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00caa0 con 0x7fea0415af30 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00f040 con 0x7fea0415af30 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0416a0d0 con 0x7fea0415af30 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00ce70 con 0x7fea0415af30 2023-10-06T12:52:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.499+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea04169500 con 0x7fea0415af30 2023-10-06T12:52:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.503+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea04063f90 con 0x7fea0415af30 2023-10-06T12:52:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.503+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe9fc014690 con 0x7fea0415af30 2023-10-06T12:52:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.507+0000 7fe9fa7fc700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 0x7fe9e8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.507+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fe9fc07d6e0 con 0x7fea0415af30 2023-10-06T12:52:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.507+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 0x7fe9e8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.507+0000 7fea0a04a700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 0x7fe9e8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea04151240 tx=0x7fea00016000 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:36.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.511+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe9fc051100 con 0x7fea0415af30 2023-10-06T12:52:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.715+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea0406ad00 con 0x7fea0415af30 2023-10-06T12:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.715+0000 7fe9fa7fc700 1 -- 172.21.15.46:0/2418331880 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7fe9fc050c90 con 0x7fea0415af30 2023-10-06T12:52:36.720 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:36.720 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 msgr2=0x7fe9e8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 0x7fe9e8062b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea04151240 tx=0x7fea00016000 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 msgr2=0x7fea041500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009df0 tx=0x7fe9fc00b040 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 shutdown_connections 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0415af30 0x7fea041500c0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea0415ab60 0x7fea040ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe9e8060670 0x7fe9e8062b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 --2- 172.21.15.46:0/2418331880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea040ad6c0 0x7fea040c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 >> 172.21.15.46:0/2418331880 conn(0x7fea040cbe30 msgr2=0x7fea04068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 shutdown_connections 2023-10-06T12:52:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:36.719+0000 7fea0c2ae700 1 -- 172.21.15.46:0/2418331880 wait complete. 2023-10-06T12:52:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:37.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T12:52:27.542914+0000', 'modified': '2023-10-06T12:52:31.016227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10359}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10359': {'gid': 10359, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2405550575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2405550575}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2405550575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T12:52:37.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T12:52:37.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T12:52:37.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:52:37.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:52:37.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 Processor -- start 2023-10-06T12:52:37.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 -- start start 2023-10-06T12:52:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 0x7f3ce8161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ce80c21e0 0x7f3ce80caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce80c18f0 con 0x7f3ce80c1d80 2023-10-06T12:52:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce80c2660 con 0x7f3ce80c21e0 2023-10-06T12:52:37.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cee865700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce80c0ce0 con 0x7f3ce8161470 2023-10-06T12:52:37.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cdf7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3ce80c1d80 0x7f3ce806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36260/0 (socket says 172.21.15.46:36260) 2023-10-06T12:52:37.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/1961284800 learned_addr learned my addr 172.21.15.46:0/1961284800 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:37.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3cece02700 1 --2- 172.21.15.46:0/1961284800 >> v2:172.21.15.46:3300/0 conn(0x7f3ce80c21e0 0x7f3ce80caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.307+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/1961284800 >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 0x7f3ce8161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce77fe700 1 -- 172.21.15.46:0/1961284800 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880016822 0 0) 0x7f3ce80c18f0 con 0x7f3ce80c1d80 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce77fe700 1 -- 172.21.15.46:0/1961284800 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd4003570 con 0x7f3ce80c1d80 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cece02700 1 -- 172.21.15.46:0/1961284800 >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 msgr2=0x7f3ce8161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cece02700 1 --2- 172.21.15.46:0/1961284800 >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 0x7f3ce8161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cece02700 1 -- 172.21.15.46:0/1961284800 >> v1:172.21.15.46:6789/0 conn(0x7f3ce80c1d80 legacy=0x7f3ce806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cece02700 1 -- 172.21.15.46:0/1961284800 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80c0fc0 con 0x7f3ce80c21e0 2023-10-06T12:52:37.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/1961284800 >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 0x7f3ce8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cece02700 1 --2- 172.21.15.46:0/1961284800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce80caa10 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8063c10 tx=0x7f3ce0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e24bb222263c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce77fe700 1 -- 172.21.15.46:0/1961284800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce000e070 con 0x7f3ce80c21e0 2023-10-06T12:52:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce77fe700 1 -- 172.21.15.46:0/1961284800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce000bb40 con 0x7f3ce80c21e0 2023-10-06T12:52:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3ce77fe700 1 -- 172.21.15.46:0/1961284800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0012430 con 0x7f3ce80c21e0 2023-10-06T12:52:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 -- 172.21.15.46:0/1961284800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 msgr2=0x7f3ce80caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 --2- 172.21.15.46:0/1961284800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce80caa10 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8063c10 tx=0x7f3ce0009780 comp rx=0 tx=0).stop 2023-10-06T12:52:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 -- 172.21.15.46:0/1961284800 shutdown_connections 2023-10-06T12:52:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 --2- 172.21.15.46:0/1961284800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce80caa10 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 --2- 172.21.15.46:0/1961284800 >> v2:172.21.15.132:3300/0 conn(0x7f3ce8161470 0x7f3ce8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 -- 172.21.15.46:0/1961284800 >> 172.21.15.46:0/1961284800 conn(0x7f3ce80cbe30 msgr2=0x7f3ce80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 -- 172.21.15.46:0/1961284800 shutdown_connections 2023-10-06T12:52:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.311+0000 7f3cee865700 1 -- 172.21.15.46:0/1961284800 wait complete. 2023-10-06T12:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 Processor -- start 2023-10-06T12:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 -- start start 2023-10-06T12:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 0x7f3ce80c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 0x7f3ce80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80c3360 con 0x7f3ce80c21e0 2023-10-06T12:52:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce815ff80 con 0x7f3ce80c1d80 2023-10-06T12:52:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cee865700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce8160260 con 0x7f3ce8161470 2023-10-06T12:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50792/0 (socket says 172.21.15.46:50792) 2023-10-06T12:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/4050457980 learned_addr learned my addr 172.21.15.46:0/4050457980 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cece02700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 0x7f3ce80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 0x7f3ce80c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.315+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 msgr2=0x7f3ce80cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 0x7f3ce80cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 msgr2=0x7f3ce80c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 0x7f3ce80c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/4050457980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80093c0 con 0x7f3ce80c21e0 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 0x7f3ce80c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cece02700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 0x7f3ce80cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c7c0 tx=0x7f3cd0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3ce4ff9700 1 -- 172.21.15.46:0/4050457980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0005b30 con 0x7f3ce80c21e0 2023-10-06T12:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3ce4ff9700 1 -- 172.21.15.46:0/4050457980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0005c90 con 0x7f3ce80c21e0 2023-10-06T12:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce0009a20 con 0x7f3ce80c21e0 2023-10-06T12:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3ce4ff9700 1 -- 172.21.15.46:0/4050457980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0007320 con 0x7f3ce80c21e0 2023-10-06T12:52:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.319+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce80c3ec0 con 0x7f3ce80c21e0 2023-10-06T12:52:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.323+0000 7f3ce4ff9700 1 -- 172.21.15.46:0/4050457980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f3cd001b400 con 0x7f3ce80c21e0 2023-10-06T12:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.323+0000 7f3ce4ff9700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 0x7f3cc8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.323+0000 7f3ce4ff9700 1 -- 172.21.15.46:0/4050457980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f3cd007c7f0 con 0x7f3ce80c21e0 2023-10-06T12:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.323+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 0x7f3cc8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.323+0000 7f3cf0aef740 1 Processor -- start 2023-10-06T12:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 0x7f3cc8062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009390 tx=0x7f3cd8008040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 -- start start 2023-10-06T12:52:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x15f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x1524f30 con 0x1379710 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x1520ed0 con 0x13c4790 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x14e70b0 con 0x1379ae0 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50802/0 (socket says 172.21.15.46:50802) 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3518095802 learned_addr learned my addr 172.21.15.46:0/3518095802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cece02700 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x15f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 msgr2=0x15f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 msgr2=0x15f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x15f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3518095802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13c59e0 con 0x1379710 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0007b00 tx=0x7f3cd0050430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4874e4d842e5ff5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdd7fa700 1 -- 172.21.15.46:0/3518095802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00538c0 con 0x1379710 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdd7fa700 1 -- 172.21.15.46:0/3518095802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00575b0 con 0x1379710 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.327+0000 7f3cdd7fa700 1 -- 172.21.15.46:0/3518095802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0005e00 con 0x1379710 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3518095802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 msgr2=0x15f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0007b00 tx=0x7f3cd0050430 comp rx=0 tx=0).stop 2023-10-06T12:52:37.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3518095802 shutdown_connections 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x15f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379710 0x15f6b70 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3518095802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3518095802 >> 172.21.15.46:0/3518095802 conn(0x1379eb0 msgr2=0x15b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3518095802 shutdown_connections 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3518095802 wait complete. 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- start start 2023-10-06T12:52:37.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 Processor -- start 2023-10-06T12:52:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x13c4790 0x15b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e6d50 con 0x1379ae0 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x14bf970 con 0x1379710 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.331+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x1527400 con 0x13c4790 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 10 client.0 _ll_register_callbacks cb 0x14c3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50810/0 (socket says 172.21.15.46:50810) 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3685914865 learned_addr learned my addr 172.21.15.46:0/3685914865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cece02700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x13c4790 0x15b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x13c4790 msgr2=0x15b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x13c4790 0x15b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 msgr2=0x15fac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15fac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/3685914865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce80c3ec0 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00502d0 tx=0x7f3cd0076000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6438ead518a08f9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd004e970 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f3cd0050870 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x14bfcc0 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd004fac0 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 10 client.10515 fetch_fsmap learned FSMap version 103 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x14ef370 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd0069030 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 103) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3cd0057120 con 0x1379ae0 2023-10-06T12:52:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 10 client.10515 fetch_fsmap finished waiting for FSMap version 103 2023-10-06T12:52:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 10 client.10515 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T12:52:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 20 client.10515 populate_metadata read hostname 'smithi046' 2023-10-06T12:52:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x15b22b0 0x1605de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 4 client.10515 mds_command: new command op to 10359 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T12:52:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 --> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x14c33e0 con 0x15b22b0 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.335+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x15b22b0 0x1605de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x15b22b0 0x1605de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e5fa1d2f79e91a server_cookie=7f31a0bb29bc0e in_seq=0 out_seq=0 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cc67fc700 10 client.10515 ms_handle_connect on v2:172.21.15.46:6834/2405550575 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cc67fc700 1 -- 172.21.15.46:0/3685914865 <== mds.0 v2:172.21.15.46:6834/2405550575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x14c33e0 con 0x15b22b0 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cc67fc700 10 client.10515 handle_command_reply: tid=0 2023-10-06T12:52:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 client.10515 shutdown 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 msgr2=0x15fb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00502d0 tx=0x7f3cd0076000 comp rx=0 tx=0).stop 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 shutdown_connections 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x15b22b0 0x1605de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x13c4790 0x15b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x15fb180 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3685914865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15fac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cc67fc700 0 client.10515 ms_handle_reset on v2:172.21.15.46:6834/2405550575 2023-10-06T12:52:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 >> 172.21.15.46:0/3685914865 conn(0x1379eb0 msgr2=0x15da1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 shutdown_connections 2023-10-06T12:52:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3685914865 wait complete. 2023-10-06T12:52:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.339+0000 7f3cf0aef740 20 client.10515 trim_cache size 0 max 16384 2023-10-06T12:52:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 Processor -- start 2023-10-06T12:52:37.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 -- start start 2023-10-06T12:52:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 0x15d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f4040 con 0x13c4790 2023-10-06T12:52:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x14f6010 con 0x1379710 2023-10-06T12:52:37.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.359+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x13f27c0 con 0x1379ae0 2023-10-06T12:52:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 0x15d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54376/0 (socket says 172.21.15.46:54376) 2023-10-06T12:52:37.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3ce7fff700 1 -- 172.21.15.46:0/3378663706 learned_addr learned my addr 172.21.15.46:0/3378663706 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 -- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 msgr2=0x15abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 -- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 msgr2=0x15ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 -- 172.21.15.46:0/3378663706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce806e340 con 0x13c4790 2023-10-06T12:52:37.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cece02700 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 0x15d5e90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0016000 tx=0x7f3ce0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd673e0914a14c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cddffb700 1 -- 172.21.15.46:0/3378663706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce002c070 con 0x13c4790 2023-10-06T12:52:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cddffb700 1 -- 172.21.15.46:0/3378663706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce0023040 con 0x13c4790 2023-10-06T12:52:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cddffb700 1 -- 172.21.15.46:0/3378663706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0022050 con 0x13c4790 2023-10-06T12:52:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3378663706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 msgr2=0x15d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.363+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 0x15d5e90 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0016000 tx=0x7f3ce0019000 comp rx=0 tx=0).stop 2023-10-06T12:52:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3378663706 shutdown_connections 2023-10-06T12:52:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x13c4790 0x15d5e90 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379ae0 0x15abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/3378663706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x1379710 0x15ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3378663706 >> 172.21.15.46:0/3378663706 conn(0x1379eb0 msgr2=0x15da1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3378663706 shutdown_connections 2023-10-06T12:52:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 -- 172.21.15.46:0/3378663706 wait complete. 2023-10-06T12:52:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 -- start start 2023-10-06T12:52:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.367+0000 7f3cf0aef740 1 Processor -- start 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379710 0x16503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x1604da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cfc90 con 0x1379ae0 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x14cfe00 con 0x13c4790 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x14ecc00 con 0x1379710 2023-10-06T12:52:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 10 client.0 _ll_register_callbacks cb 0x14ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50830/0 (socket says 172.21.15.46:50830) 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/1556597157 learned_addr learned my addr 172.21.15.46:0/1556597157 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379710 0x16503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cece02700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x1604da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379710 msgr2=0x16503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379710 0x16503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 msgr2=0x1604da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x1604da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 -- 172.21.15.46:0/1556597157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce80c0fc0 con 0x1379ae0 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cece02700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x1604da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cdf7fe700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000cf10 tx=0x7f3cd007a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21afdf1cbc02b997 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0018330 con 0x1379ae0 2023-10-06T12:52:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cd00575b0 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f3cd0050870 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd000e040 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 10 client.10521 fetch_fsmap learned FSMap version 103 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x14d5d80 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd00575b0 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 103) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3cd0057120 con 0x1379ae0 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.371+0000 7f3cf0aef740 10 client.10521 fetch_fsmap finished waiting for FSMap version 103 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 10 client.10521 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T12:52:37.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 20 client.10521 populate_metadata read hostname 'smithi046' 2023-10-06T12:52:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x1605730 0x1605b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 4 client.10521 mds_command: new command op to 10359 tid=0[{"prefix": "status"}] 2023-10-06T12:52:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 --> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] -- command(tid 0: {"prefix": "status"}) v1 -- 0x14f5410 con 0x1605730 2023-10-06T12:52:37.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x1605730 0x1605b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3ce7fff700 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x1605730 0x1605b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e3e74f8b4255bc server_cookie=7ba05932ca00a8cc in_seq=0 out_seq=0 2023-10-06T12:52:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cc67fc700 10 client.10521 ms_handle_connect on v2:172.21.15.46:6834/2405550575 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cc67fc700 1 -- 172.21.15.46:0/1556597157 <== mds.0 v2:172.21.15.46:6834/2405550575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x14f5410 con 0x1605730 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cc67fc700 10 client.10521 handle_command_reply: tid=0 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 client.10521 shutdown 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 msgr2=0x1650910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000cf10 tx=0x7f3cd007a000 comp rx=0 tx=0).stop 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 shutdown_connections 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:6834/2405550575,v1:172.21.15.46:6835/2405550575] conn(0x1605730 0x1605b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x13c4790 0x1604da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x1379ae0 0x1650910 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 --2- 172.21.15.46:0/1556597157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x1379710 0x16503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cc67fc700 0 client.10521 ms_handle_reset on v2:172.21.15.46:6834/2405550575 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 >> 172.21.15.46:0/1556597157 conn(0x1379eb0 msgr2=0x1601140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 shutdown_connections 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 1 -- 172.21.15.46:0/1556597157 wait complete. 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cf0aef740 20 client.10521 trim_cache size 0 max 16384 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:52:37.384 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "36b5f543-5934-4e63-898c-253575113a16", 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 10359, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 9.3712207630000002, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 103, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 197, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 197, 2023-10-06T12:52:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 20.700085845 2023-10-06T12:52:37.386 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 msgr2=0x7f3cc8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 0x7f3cc8062b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009390 tx=0x7f3cd8008040 comp rx=0 tx=0).stop 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 msgr2=0x7f3ce815b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c7c0 tx=0x7f3cd0007e00 comp rx=0 tx=0).stop 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 shutdown_connections 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ce8161470 0x7f3ce80cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce80c21e0 0x7f3ce815b220 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3cc8060670 0x7f3cc8062b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 --2- 172.21.15.46:0/4050457980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ce80c1d80 0x7f3ce80c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.375+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 >> 172.21.15.46:0/4050457980 conn(0x7f3ce80cbe30 msgr2=0x7f3ce80ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.379+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 shutdown_connections 2023-10-06T12:52:37.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.379+0000 7f3cee865700 1 -- 172.21.15.46:0/4050457980 wait complete. 2023-10-06T12:52:37.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T12:52:37.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.563+0000 7ffb6fe6e700 1 Processor -- start 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.563+0000 7ffb6fe6e700 1 -- start start 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6fe6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb680c7b50 0x7ffb680c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6fe6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680c8500 con 0x7ffb68156c80 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6fe6e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680c6470 con 0x7ffb680c0bd0 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6fe6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680c7760 con 0x7ffb680c7b50 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6dc0a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb68156c80 0x7ffb680ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36268/0 (socket says 172.21.15.46:36268) 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6dc0a700 1 -- 172.21.15.46:0/3976379416 learned_addr learned my addr 172.21.15.46:0/3976379416 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6d409700 1 --2- 172.21.15.46:0/3976379416 >> v2:172.21.15.132:3300/0 conn(0x7ffb680c7b50 0x7ffb680c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.567+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607794368 0 0) 0x7ffb680c8500 con 0x7ffb68156c80 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb54003570 con 0x7ffb68156c80 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6d409700 1 -- 172.21.15.46:0/3976379416 >> v1:172.21.15.132:6789/0 conn(0x7ffb680c0bd0 legacy=0x7ffb680c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124840067 0 0) 0x7ffb680c6470 con 0x7ffb680c0bd0 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6d409700 1 -- 172.21.15.46:0/3976379416 >> v1:172.21.15.46:6789/0 conn(0x7ffb68156c80 legacy=0x7ffb680ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6d409700 1 -- 172.21.15.46:0/3976379416 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680c66c0 con 0x7ffb680c7b50 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6d409700 1 --2- 172.21.15.46:0/3976379416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c7b50 0x7ffb680c7fc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098e0 tx=0x7ffb580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666744ad720053f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800b640 con 0x7ffb680c7b50 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800b7a0 con 0x7ffb680c7b50 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6cc08700 1 -- 172.21.15.46:0/3976379416 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800bb40 con 0x7ffb680c7b50 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3976379416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c7b50 msgr2=0x7ffb680c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3976379416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c7b50 0x7ffb680c7fc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098e0 tx=0x7ffb580091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3976379416 shutdown_connections 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3976379416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c7b50 0x7ffb680c7fc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3976379416 >> 172.21.15.46:0/3976379416 conn(0x7ffb680cbe30 msgr2=0x7ffb680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3976379416 shutdown_connections 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.571+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3976379416 wait complete. 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 Processor -- start 2023-10-06T12:52:37.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 -- start start 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c0bd0 0x7ffb68155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb680c7b50 0x7ffb6814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681573e0 con 0x7ffb68156c80 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68157660 con 0x7ffb680c0bd0 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6fe6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68155990 con 0x7ffb680c7b50 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50834/0 (socket says 172.21.15.46:50834) 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 -- 172.21.15.46:0/3140261545 learned_addr learned my addr 172.21.15.46:0/3140261545 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:37.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6dc0a700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c0bd0 0x7ffb68155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6d409700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb680c7b50 0x7ffb6814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 -- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb680c7b50 msgr2=0x7ffb6814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb680c7b50 0x7ffb6814d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 -- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c0bd0 msgr2=0x7ffb68155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.575+0000 7ffb6e40b700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c0bd0 0x7ffb68155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb6e40b700 1 -- 172.21.15.46:0/3140261545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009420 con 0x7ffb68156c80 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb6e40b700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ffb60001560 tx=0x7ffb60001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60003030 con 0x7ffb68156c80 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6000a7e0 con 0x7ffb68156c80 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb68155c70 con 0x7ffb68156c80 2023-10-06T12:52:37.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000aba0 con 0x7ffb68156c80 2023-10-06T12:52:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb68155570 con 0x7ffb68156c80 2023-10-06T12:52:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.579+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb68157880 con 0x7ffb68156c80 2023-10-06T12:52:37.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.583+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ffb60012420 con 0x7ffb68156c80 2023-10-06T12:52:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.583+0000 7ffb5dffb700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 0x7ffb4c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.583+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7ffb60004070 con 0x7ffb68156c80 2023-10-06T12:52:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.583+0000 7ffb6dc0a700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 0x7ffb4c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.583+0000 7ffb6dc0a700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 0x7ffb4c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb64009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.591+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffb68157880 con 0x7ffb68156c80 2023-10-06T12:52:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.795+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb68157820 con 0x7ffb68156c80 2023-10-06T12:52:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.795+0000 7ffb5dffb700 1 -- 172.21.15.46:0/3140261545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb6007c070 con 0x7ffb68156c80 2023-10-06T12:52:37.800 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T12:52:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 msgr2=0x7ffb4c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 0x7ffb4c062b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb64009040 comp rx=0 tx=0).stop 2023-10-06T12:52:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 msgr2=0x7ffb680c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:37.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ffb60001560 tx=0x7ffb60001770 comp rx=0 tx=0).stop 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 shutdown_connections 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb68156c80 0x7ffb680c9c10 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb680c7b50 0x7ffb6814d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ffb4c060670 0x7ffb4c062b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 --2- 172.21.15.46:0/3140261545 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb680c0bd0 0x7ffb68155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 >> 172.21.15.46:0/3140261545 conn(0x7ffb680cbe30 msgr2=0x7ffb68152670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 shutdown_connections 2023-10-06T12:52:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:37.799+0000 7ffb6fe6e700 1 -- 172.21.15.46:0/3140261545 wait complete. 2023-10-06T12:52:37.822 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T12:52:37.822 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T12:52:37.822 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T12:52:37.823 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:52:37.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:37.823 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-10-06T12:52:37.841 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T12:52:37.841 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T12:52:37.841 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T12:52:37.841 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T12:52:37.842 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 36772sec preferred_lft 36772sec 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-10-06T12:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:ec:26:5a:22 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c027:95ff:fe77:e643/64 scope link 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.844 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether be:c1:ce:a0:b2:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::bcc1:ceff:fea0:b2cb/64 scope link 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 9e:75:f0:4d:2e:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::9c75:f0ff:fe4d:2e43/64 scope link 2023-10-06T12:52:37.845 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-10-06T12:52:37.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:37.846 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T12:52:37.927 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-10-06T12:52:37.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:37.930 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-06T12:52:37.998 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T12:52:37.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:37.999 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-10-06T12:52:37.999 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-10-06T12:52:37.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T12:52:38.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T12:52:38.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T12:52:38.000 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-10-06T12:52:38.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:38.139 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:52:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T12:52:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T12:52:38.156 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:52:38.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:38.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-10-06T12:52:38.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:52:38.234 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:38.300 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T12:52:38.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T12:52:38.301 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:38.360 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T12:52:38.360 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T12:52:38.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:52:38.480 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:52:38.481 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:52:38.481 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:52:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:52:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:52:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:52:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:52:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.132:3300,172.21.15.132:3301" 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790" 2023-10-06T12:52:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2023-10-06T12:52:38.529 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=36b5f543-5934-4e63-898c-253575113a16" will pass to kernel 2023-10-06T12:52:38.533 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T12:52:38.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:38.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:38.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T12:52:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 Processor -- start 2023-10-06T12:52:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 -- start start 2023-10-06T12:52:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efcb8156f60 0x7efcb80cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efcb80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb80cd220 con 0x7efcb80af9e0 2023-10-06T12:52:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80cc610 con 0x7efcb8156f60 2023-10-06T12:52:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcc04ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80cc8f0 con 0x7efcb8157330 2023-10-06T12:52:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcbea4b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efcb80af9e0 0x7efcb80afe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36278/0 (socket says 172.21.15.46:36278) 2023-10-06T12:52:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.779+0000 7efcbea4b700 1 -- 172.21.15.46:0/4128368858 learned_addr learned my addr 172.21.15.46:0/4128368858 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 --2- 172.21.15.46:0/4128368858 >> v2:172.21.15.132:3300/0 conn(0x7efcb8156f60 0x7efcb80cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbda49700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efcb80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195322468 0 0) 0x7efcb80cd220 con 0x7efcb80af9e0 2023-10-06T12:52:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca4003570 con 0x7efcb80af9e0 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1322780678 0 0) 0x7efca4003570 con 0x7efcb80af9e0 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 msgr2=0x7efcb80cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efcb80cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 >> v2:172.21.15.132:3300/0 conn(0x7efcb8156f60 msgr2=0x7efcb80cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 --2- 172.21.15.46:0/4128368858 >> v2:172.21.15.132:3300/0 conn(0x7efcb8156f60 0x7efcb80cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80ccbd0 con 0x7efcb80af9e0 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7efcb00017f0 con 0x7efcb80af9e0 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 >> v1:172.21.15.46:6789/0 conn(0x7efcb80af9e0 legacy=0x7efcb80afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4003a60 0x7efca4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4004810 0x7efca4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efca40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80cc610 con 0x7efca4004810 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80cc8f0 con 0x7efca4003a60 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca400ba60 con 0x7efcb8157330 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4003a60 0x7efca4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbda49700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4004810 0x7efca4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbea4b700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efca40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 -- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 msgr2=0x7efca40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb8157330 0x7efca40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 -- 172.21.15.46:0/4128368858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4004810 msgr2=0x7efca4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4004810 0x7efca4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 -- 172.21.15.46:0/4128368858 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4003570 con 0x7efca4003a60 2023-10-06T12:52:38.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbe24a700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4003a60 0x7efca4003e70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009c30 tx=0x7efcb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc5767e5cba6f03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.783+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400e710 con 0x7efca4003a60 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400e870 con 0x7efca4003a60 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcbd248700 1 -- 172.21.15.46:0/4128368858 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400ec30 con 0x7efca4003a60 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 -- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4003a60 msgr2=0x7efca4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 --2- 172.21.15.46:0/4128368858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4003a60 0x7efca4003e70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009c30 tx=0x7efcb40091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcbea4b700 1 -- 172.21.15.46:0/4128368858 reap_dead start 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 -- 172.21.15.46:0/4128368858 shutdown_connections 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 -- 172.21.15.46:0/4128368858 >> 172.21.15.46:0/4128368858 conn(0x7efcb806ae60 msgr2=0x7efcb806b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 -- 172.21.15.46:0/4128368858 shutdown_connections 2023-10-06T12:52:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.787+0000 7efcc04ae700 1 -- 172.21.15.46:0/4128368858 wait complete. 2023-10-06T12:52:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 Processor -- start 2023-10-06T12:52:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 -- start start 2023-10-06T12:52:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb80c2e20 0x7efcb8158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb814fc30 0x7efcb80b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8159340 con 0x7efca4003a60 2023-10-06T12:52:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8159620 con 0x7efcb80c2e20 2023-10-06T12:52:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcc04ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb8159900 con 0x7efcb814fc30 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50862/0 (socket says 172.21.15.46:50862) 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 -- 172.21.15.46:0/2242378386 learned_addr learned my addr 172.21.15.46:0/2242378386 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbda49700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb80c2e20 0x7efcb8158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbea4b700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb814fc30 0x7efcb80b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 -- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb814fc30 msgr2=0x7efcb80b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb814fc30 0x7efcb80b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 -- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb80c2e20 msgr2=0x7efcb8158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb80c2e20 0x7efcb8158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 -- 172.21.15.46:0/2242378386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8159be0 con 0x7efca4003a60 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.791+0000 7efcbe24a700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e4b0 tx=0x7efcb401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4022070 con 0x7efca4003a60 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400e9e0 con 0x7efca4003a60 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb8159ec0 con 0x7efca4003a60 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400e710 con 0x7efca4003a60 2023-10-06T12:52:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb81500a0 con 0x7efca4003a60 2023-10-06T12:52:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb81502c0 con 0x7efca4003a60 2023-10-06T12:52:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.795+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7efcb401c660 con 0x7efca4003a60 2023-10-06T12:52:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.799+0000 7efcae7fc700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 0x7efc9c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.799+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7efcb407e320 con 0x7efca4003a60 2023-10-06T12:52:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.799+0000 7efcbda49700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 0x7efc9c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.799+0000 7efcbda49700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 0x7efc9c062b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b3580 tx=0x7efca80091a0 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:38.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.803+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efcb81502c0 con 0x7efca4003a60 2023-10-06T12:52:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.999+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efcb8159340 con 0x7efc9c0606a0 2023-10-06T12:52:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:38.999+0000 7efcae7fc700 1 -- 172.21.15.46:0/2242378386 <== mgr.10158 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7efcb8159340 con 0x7efc9c0606a0 2023-10-06T12:52:39.006 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-10-06T12:52:39.006 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-10-06T12:52:39.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephfs" 2023-10-06T12:52:39.007 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-10-06T12:52:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-10-06T12:52:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 msgr2=0x7efc9c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 0x7efc9c062b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcb80b3580 tx=0x7efca80091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 msgr2=0x7efcb815a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7efcb400e4b0 tx=0x7efcb401a000 comp rx=0 tx=0).stop 2023-10-06T12:52:39.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 shutdown_connections 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb814fc30 0x7efcb80b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7efc9c0606a0 0x7efc9c062b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb80c2e20 0x7efcb8158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 --2- 172.21.15.46:0/2242378386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca4003a60 0x7efcb815a3e0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 >> 172.21.15.46:0/2242378386 conn(0x7efcb806ae60 msgr2=0x7efcb815b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 shutdown_connections 2023-10-06T12:52:39.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.003+0000 7efcc04ae700 1 -- 172.21.15.46:0/2242378386 wait complete. 2023-10-06T12:52:39.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T12:52:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.191+0000 7f0fd8785700 1 Processor -- start 2023-10-06T12:52:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.191+0000 7f0fd8785700 1 -- start start 2023-10-06T12:52:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0fd006ed10 0x7f0fd00c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd00c1bf0 con 0x7f0fd00c6470 2023-10-06T12:52:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd00c72c0 con 0x7f0fd006ed10 2023-10-06T12:52:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd006cf80 con 0x7f0fd006e8a0 2023-10-06T12:52:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd6d22700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0fd006ed10 0x7f0fd00c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:52086/0 (socket says 172.21.15.46:52086) 2023-10-06T12:52:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 -- 172.21.15.46:0/4079574469 learned_addr learned my addr 172.21.15.46:0/4079574469 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 -- 172.21.15.46:0/4079574469 >> v2:172.21.15.132:3300/0 conn(0x7f0fd006ed10 msgr2=0x7f0fd00c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 --2- 172.21.15.46:0/4079574469 >> v2:172.21.15.132:3300/0 conn(0x7f0fd006ed10 0x7f0fd00c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 -- 172.21.15.46:0/4079574469 >> v1:172.21.15.132:6789/0 conn(0x7f0fd00c6470 legacy=0x7f0fd00ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 -- 172.21.15.46:0/4079574469 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd006d260 con 0x7f0fd006e8a0 2023-10-06T12:52:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd5d20700 1 --2- 172.21.15.46:0/4079574469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00099b0 tx=0x7f0fc0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6052432ff02a6546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd551f700 1 -- 172.21.15.46:0/4079574469 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0009ea0 con 0x7f0fd006e8a0 2023-10-06T12:52:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd551f700 1 -- 172.21.15.46:0/4079574469 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc000b860 con 0x7f0fd006e8a0 2023-10-06T12:52:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd551f700 1 -- 172.21.15.46:0/4079574469 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000bc00 con 0x7f0fd006e8a0 2023-10-06T12:52:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 -- 172.21.15.46:0/4079574469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 msgr2=0x7f0fd00c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.195+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4079574469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00099b0 tx=0x7f0fc0009250 comp rx=0 tx=0).stop 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- 172.21.15.46:0/4079574469 shutdown_connections 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4079574469 >> v2:172.21.15.132:3300/0 conn(0x7f0fd006ed10 0x7f0fd00c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4079574469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00c6840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- 172.21.15.46:0/4079574469 >> 172.21.15.46:0/4079574469 conn(0x7f0fd00cbe30 msgr2=0x7f0fd00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- 172.21.15.46:0/4079574469 shutdown_connections 2023-10-06T12:52:39.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- 172.21.15.46:0/4079574469 wait complete. 2023-10-06T12:52:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 Processor -- start 2023-10-06T12:52:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- start start 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd006ed10 0x7f0fd00aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015ffe0 con 0x7f0fd00c6470 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015f360 con 0x7f0fd006ed10 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd8785700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd015f640 con 0x7f0fd006e8a0 2023-10-06T12:52:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6d22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6d22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50870/0 (socket says 172.21.15.46:50870) 2023-10-06T12:52:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6d22700 1 -- 172.21.15.46:0/4247060030 learned_addr learned my addr 172.21.15.46:0/4247060030 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd5d20700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd006ed10 0x7f0fd00aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6521700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6d22700 1 -- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 msgr2=0x7f0fd00b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.199+0000 7f0fd6d22700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd6d22700 1 -- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd006ed10 msgr2=0x7f0fd00aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd6d22700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd006ed10 0x7f0fd00aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd6d22700 1 -- 172.21.15.46:0/4247060030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00094f0 con 0x7f0fd00c6470 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd6d22700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800ea50 tx=0x7f0fc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800c8c0 con 0x7f0fd00c6470 2023-10-06T12:52:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc800ca40 con 0x7f0fd00c6470 2023-10-06T12:52:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd015f920 con 0x7f0fd00c6470 2023-10-06T12:52:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800ce00 con 0x7f0fd00c6470 2023-10-06T12:52:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd00b18d0 con 0x7f0fd00c6470 2023-10-06T12:52:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.203+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd00660f0 con 0x7f0fd00c6470 2023-10-06T12:52:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.207+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 110) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0fc8013680 con 0x7f0fd00c6470 2023-10-06T12:52:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.207+0000 7f0fc67fc700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 0x7f0fb4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:39.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.207+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f0fc807d160 con 0x7f0fd00c6470 2023-10-06T12:52:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.207+0000 7f0fd6521700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 0x7f0fb4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:39.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.207+0000 7f0fd6521700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 0x7f0fb4062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007fd0 tx=0x7f0fcc009040 comp rx=0 tx=0).ready entity=mgr.10158 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.211+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0fc804d200 con 0x7f0fd00c6470 2023-10-06T12:52:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.419+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0fd004b5c0 con 0x7f0fd00c6470 2023-10-06T12:52:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.963+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 111) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f0fc80560d0 con 0x7f0fd00c6470 2023-10-06T12:52:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.963+0000 7f0fc67fc700 1 -- 172.21.15.46:0/4247060030 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v111) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0fc804d200 con 0x7f0fd00c6470 2023-10-06T12:52:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 msgr2=0x7f0fb4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 0x7f0fb4062b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007fd0 tx=0x7f0fcc009040 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 msgr2=0x7f0fd00af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800ea50 tx=0x7f0fc80091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 shutdown_connections 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd00c6470 0x7f0fd00af110 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0fb4060670 0x7f0fb4062b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd006ed10 0x7f0fd00aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 --2- 172.21.15.46:0/4247060030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd006e8a0 0x7f0fd00b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 >> 172.21.15.46:0/4247060030 conn(0x7f0fd00cbe30 msgr2=0x7f0fd00c1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 shutdown_connections 2023-10-06T12:52:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:39.967+0000 7f0fd8785700 1 -- 172.21.15.46:0/4247060030 wait complete. 2023-10-06T12:52:40.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T12:52:40.149 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:40.146+0000 7f8462bab000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:40.166 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:40.163+0000 7f67ddfcc000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.215+0000 7fa19cecd700 1 Processor -- start 2023-10-06T12:52:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.215+0000 7fa19cecd700 1 -- start start 2023-10-06T12:52:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa198159420 0x7fa19815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1980697e0 0x7fa19815a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980ce470 con 0x7fa198069410 2023-10-06T12:52:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa19815a890 con 0x7fa1980697e0 2023-10-06T12:52:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198069c40 con 0x7fa198159420 2023-10-06T12:52:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19659c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa198069410 0x7fa198158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36282/0 (socket says 172.21.15.46:36282) 2023-10-06T12:52:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19659c700 1 -- 172.21.15.46:0/265515805 learned_addr learned my addr 172.21.15.46:0/265515805 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 --2- 172.21.15.46:0/265515805 >> v2:172.21.15.132:3300/0 conn(0x7fa1980697e0 0x7fa19815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa195d9b700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa198159420 0x7fa19815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303051126 0 0) 0x7fa1980ce470 con 0x7fa198069410 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa180003570 con 0x7fa198069410 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 -- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa198159420 msgr2=0x7fa19815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa198159420 0x7fa19815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 -- 172.21.15.46:0/265515805 >> v1:172.21.15.46:6789/0 conn(0x7fa198069410 legacy=0x7fa198158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639892504 0 0) 0x7fa180003570 con 0x7fa198069410 2023-10-06T12:52:40.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 -- 172.21.15.46:0/265515805 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19815d470 con 0x7fa1980697e0 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa196d9d700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1980697e0 0x7fa19815a350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098e0 tx=0x7fa1880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcf66f0aba27970 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1900017f0 con 0x7fa1980697e0 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188011420 con 0x7fa1980697e0 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19559a700 1 -- 172.21.15.46:0/265515805 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880117c0 con 0x7fa1980697e0 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1980697e0 msgr2=0x7fa19815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1980697e0 0x7fa19815a350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa1880098e0 tx=0x7fa1880091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- 172.21.15.46:0/265515805 shutdown_connections 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1980697e0 0x7fa19815a350 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 --2- 172.21.15.46:0/265515805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa198159420 0x7fa19815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- 172.21.15.46:0/265515805 >> 172.21.15.46:0/265515805 conn(0x7fa1980d0210 msgr2=0x7fa1980d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- 172.21.15.46:0/265515805 shutdown_connections 2023-10-06T12:52:40.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.219+0000 7fa19cecd700 1 -- 172.21.15.46:0/265515805 wait complete. 2023-10-06T12:52:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 Processor -- start 2023-10-06T12:52:40.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 -- start start 2023-10-06T12:52:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa198069410 0x7fa198154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 0x7fa1980c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b00e0 con 0x7fa198159420 2023-10-06T12:52:40.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b03c0 con 0x7fa198069410 2023-10-06T12:52:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19cecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980b06a0 con 0x7fa1980697e0 2023-10-06T12:52:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa19659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa198069410 0x7fa198154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa195d9b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 0x7fa1980c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50886/0 (socket says 172.21.15.46:50886) 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 -- 172.21.15.46:0/1876230643 learned_addr learned my addr 172.21.15.46:0/1876230643 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 -- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 msgr2=0x7fa1980c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 0x7fa1980c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 -- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa198069410 msgr2=0x7fa198154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa198069410 0x7fa198154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa196d9d700 1 -- 172.21.15.46:0/1876230643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188009420 con 0x7fa198159420 2023-10-06T12:52:40.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.223+0000 7fa195d9b700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 0x7fa1980c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa196d9d700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fa1880093f0 tx=0x7fa188016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18801cc90 con 0x7fa198159420 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18801cdf0 con 0x7fa198159420 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980b0980 con 0x7fa198159420 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18801a680 con 0x7fa198159420 2023-10-06T12:52:40.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa198154f50 con 0x7fa198159420 2023-10-06T12:52:40.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.227+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1980b0e80 con 0x7fa198159420 2023-10-06T12:52:40.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.231+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 111) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fa18801aab0 con 0x7fa198159420 2023-10-06T12:52:40.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.231+0000 7fa1867fc700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 0x7fa178062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.231+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fa18807e820 con 0x7fa198159420 2023-10-06T12:52:40.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.231+0000 7fa19659c700 1 -- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 msgr2=0x7fa178062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:40.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.231+0000 7fa19659c700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 0x7fa178062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:40.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.239+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa18804e8c0 con 0x7fa198159420 2023-10-06T12:52:40.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.407+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fa1981551a0 con 0x7fa198159420 2023-10-06T12:52:40.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.415+0000 7fa1867fc700 1 -- 172.21.15.46:0/1876230643 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa188051ee0 con 0x7fa198159420 2023-10-06T12:52:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 msgr2=0x7fa178062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 0x7fa178062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 msgr2=0x7fa1980c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fa1880093f0 tx=0x7fa188016000 comp rx=0 tx=0).stop 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 shutdown_connections 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa198159420 0x7fa1980c5f20 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1980697e0 0x7fa1980c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa1780606c0 0x7fa178062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 --2- 172.21.15.46:0/1876230643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa198069410 0x7fa198154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 >> 172.21.15.46:0/1876230643 conn(0x7fa1980d0210 msgr2=0x7fa19815aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 shutdown_connections 2023-10-06T12:52:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.419+0000 7fa19cecd700 1 -- 172.21.15.46:0/1876230643 wait complete. 2023-10-06T12:52:40.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T12:52:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.599+0000 7fe3952f0700 1 Processor -- start 2023-10-06T12:52:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.599+0000 7fe3952f0700 1 -- start start 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 0x7fe39015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 0x7fe39015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe390159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39015a890 con 0x7fe3900697e0 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3900ce470 con 0x7fe390069410 2023-10-06T12:52:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe3952f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39015d470 con 0x7fe390158d10 2023-10-06T12:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe387fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 0x7fe39015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38effd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 0x7fe39015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe387fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 0x7fe39015a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50896/0 (socket says 172.21.15.46:50896) 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe387fff700 1 -- 172.21.15.46:0/1856546239 learned_addr learned my addr 172.21.15.46:0/1856546239 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 --2- 172.21.15.46:0/1856546239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe390159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 -- 172.21.15.46:0/1856546239 >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 msgr2=0x7fe39015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 --2- 172.21.15.46:0/1856546239 >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 0x7fe39015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 -- 172.21.15.46:0/1856546239 >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 msgr2=0x7fe39015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 --2- 172.21.15.46:0/1856546239 >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 0x7fe39015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 -- 172.21.15.46:0/1856546239 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39015d750 con 0x7fe390158d10 2023-10-06T12:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38effd700 1 --2- 172.21.15.46:0/1856546239 >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 0x7fe39015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38f7fe700 1 --2- 172.21.15.46:0/1856546239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe390159140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3900cdd90 tx=0x7fe388009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e8971a3a8f49d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.603+0000 7fe38e7fc700 1 -- 172.21.15.46:0/1856546239 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800e070 con 0x7fe390158d10 2023-10-06T12:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe38e7fc700 1 -- 172.21.15.46:0/1856546239 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38800bab0 con 0x7fe390158d10 2023-10-06T12:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe38e7fc700 1 -- 172.21.15.46:0/1856546239 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800be70 con 0x7fe390158d10 2023-10-06T12:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- 172.21.15.46:0/1856546239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 msgr2=0x7fe390159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 --2- 172.21.15.46:0/1856546239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe390159140 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3900cdd90 tx=0x7fe388009780 comp rx=0 tx=0).stop 2023-10-06T12:52:40.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- 172.21.15.46:0/1856546239 shutdown_connections 2023-10-06T12:52:40.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 --2- 172.21.15.46:0/1856546239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe390159140 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 --2- 172.21.15.46:0/1856546239 >> v2:172.21.15.46:3300/0 conn(0x7fe3900697e0 0x7fe39015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 --2- 172.21.15.46:0/1856546239 >> v2:172.21.15.132:3300/0 conn(0x7fe390069410 0x7fe39015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- 172.21.15.46:0/1856546239 >> 172.21.15.46:0/1856546239 conn(0x7fe3900d0210 msgr2=0x7fe3900d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- 172.21.15.46:0/1856546239 shutdown_connections 2023-10-06T12:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- 172.21.15.46:0/1856546239 wait complete. 2023-10-06T12:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 Processor -- start 2023-10-06T12:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.607+0000 7fe3952f0700 1 -- start start 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 0x7fe3900c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 0x7fe3900c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe3900b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900b1ce0 con 0x7fe3900697e0 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3901545c0 con 0x7fe390069410 2023-10-06T12:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900b1a00 con 0x7fe390158d10 2023-10-06T12:52:40.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe38effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 0x7fe3900c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 0x7fe3900c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe38effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 0x7fe3900c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54500/0 (socket says 172.21.15.46:54500) 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe38effd700 1 -- 172.21.15.46:0/3245057499 learned_addr learned my addr 172.21.15.46:0/3245057499 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe38f7fe700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe3900b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 -- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 msgr2=0x7fe3900b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe3900b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 -- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 msgr2=0x7fe3900c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 0x7fe3900c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 -- 172.21.15.46:0/3245057499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380009420 con 0x7fe3900697e0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe387fff700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 0x7fe3900c54f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe378006d40 tx=0x7fe378009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378013680 con 0x7fe3900697e0 2023-10-06T12:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780137e0 con 0x7fe3900697e0 2023-10-06T12:52:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe388009a20 con 0x7fe3900697e0 2023-10-06T12:52:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378013b60 con 0x7fe3900697e0 2023-10-06T12:52:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.611+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe390154250 con 0x7fe3900697e0 2023-10-06T12:52:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.615+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe390154730 con 0x7fe3900697e0 2023-10-06T12:52:40.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.619+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 111) v1 ==== 80078+0+0 (secure 0 0 0) 0x7fe37800c3e0 con 0x7fe3900697e0 2023-10-06T12:52:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.619+0000 7fe386ffd700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 0x7fe370062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.619+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7fe37807ce50 con 0x7fe3900697e0 2023-10-06T12:52:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.619+0000 7fe38effd700 1 -- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 msgr2=0x7fe370062bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:40.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.619+0000 7fe38effd700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 0x7fe370062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:40.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.623+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe37804cef0 con 0x7fe3900697e0 2023-10-06T12:52:40.699 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:40.694+0000 7f8462bab000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:40.745 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:40.739+0000 7f67ddfcc000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:40.772 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:40.770+0000 7f8462bab000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:40.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.791+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe39015a130 con 0x7fe3900697e0 2023-10-06T12:52:40.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.799+0000 7fe386ffd700 1 -- 172.21.15.46:0/3245057499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe378050510 con 0x7fe3900697e0 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 msgr2=0x7fe370062bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 0x7fe370062bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 msgr2=0x7fe3900c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 0x7fe3900c54f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe378006d40 tx=0x7fe378009250 comp rx=0 tx=0).stop 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 shutdown_connections 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe390158d10 0x7fe3900b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3900697e0 0x7fe3900c54f0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe370060740 0x7fe370062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 --2- 172.21.15.46:0/3245057499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe390069410 0x7fe3900c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 >> 172.21.15.46:0/3245057499 conn(0x7fe3900d0210 msgr2=0x7fe39015cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 shutdown_connections 2023-10-06T12:52:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:40.803+0000 7fe3952f0700 1 -- 172.21.15.46:0/3245057499 wait complete. 2023-10-06T12:52:40.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:40.827 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T12:52:40.827 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:40.819+0000 7f67ddfcc000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:40.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T12:52:40.920 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:40.918+0000 7f8462bab000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:40.977 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:40.971+0000 7f67ddfcc000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:41.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.063+0000 7f290f59e700 1 Processor -- start 2023-10-06T12:52:41.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.063+0000 7f290f59e700 1 -- start start 2023-10-06T12:52:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290f59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29101591f0 0x7f29100cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29101595c0 0x7f29100af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290f59e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100cf360 con 0x7f29100cc500 2023-10-06T12:52:41.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290f59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291015dd60 con 0x7f29101591f0 2023-10-06T12:52:41.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29100cea30 con 0x7f29101595c0 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29101591f0 0x7f29100cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29101591f0 0x7f29100cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:54514/0 (socket says 172.21.15.46:54514) 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 -- 172.21.15.46:0/1941647977 learned_addr learned my addr 172.21.15.46:0/1941647977 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290ed9d700 1 --1- 172.21.15.46:0/1941647977 >> v1:172.21.15.132:6789/0 conn(0x7f29100cc500 0x7f291015ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:45164/0 (socket says 172.21.15.46:45164) 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290dd9b700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29101595c0 0x7f29100af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 -- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29101595c0 msgr2=0x7f29100af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29101595c0 0x7f29100af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:41.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 -- 172.21.15.46:0/1941647977 >> v1:172.21.15.132:6789/0 conn(0x7f29100cc500 legacy=0x7f291015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:41.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 -- 172.21.15.46:0/1941647977 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100ced10 con 0x7f29101591f0 2023-10-06T12:52:41.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290e59c700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29101591f0 0x7f29100cf7f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f29000099b0 tx=0x7f2900009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab30ff5b8ee90af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290d59a700 1 -- 172.21.15.46:0/1941647977 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900009ea0 con 0x7f29101591f0 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290d59a700 1 -- 172.21.15.46:0/1941647977 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290000b8f0 con 0x7f29101591f0 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.067+0000 7f290d59a700 1 -- 172.21.15.46:0/1941647977 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000bc90 con 0x7f29101591f0 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29101591f0 msgr2=0x7f29100cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29101591f0 0x7f29100cf7f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f29000099b0 tx=0x7f2900009250 comp rx=0 tx=0).stop 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- 172.21.15.46:0/1941647977 shutdown_connections 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29101595c0 0x7f29100af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- 172.21.15.46:0/1941647977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29101591f0 0x7f29100cf7f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:41.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- 172.21.15.46:0/1941647977 >> 172.21.15.46:0/1941647977 conn(0x7f291006ef60 msgr2=0x7f29100d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:41.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- 172.21.15.46:0/1941647977 shutdown_connections 2023-10-06T12:52:41.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- 172.21.15.46:0/1941647977 wait complete. 2023-10-06T12:52:41.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 Processor -- start 2023-10-06T12:52:41.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- start start 2023-10-06T12:52:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29100cc500 0x7f2910146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291015c580 0x7f2910067a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101572f0 con 0x7f29101591f0 2023-10-06T12:52:41.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101575d0 con 0x7f291015c580 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.071+0000 7f290f59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29101578b0 con 0x7f29100cc500 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50906/0 (socket says 172.21.15.46:50906) 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 -- 172.21.15.46:0/289728561 learned_addr learned my addr 172.21.15.46:0/289728561 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29100cc500 0x7f2910146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:41.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290ed9d700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291015c580 0x7f2910067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:41.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29100cc500 msgr2=0x7f2910146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:41.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29100cc500 0x7f2910146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:41.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291015c580 msgr2=0x7f2910067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:41.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291015c580 0x7f2910067a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000094f0 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290dd9b700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f290400ec80 tx=0x7f29040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400c850 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290400c9b0 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400cd50 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910157b90 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2910156de0 con 0x7f29101591f0 2023-10-06T12:52:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.075+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29101570c0 con 0x7f29101591f0 2023-10-06T12:52:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.079+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 111) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f2904013680 con 0x7f29101591f0 2023-10-06T12:52:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.079+0000 7f28fe7fc700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.079+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6630+0+0 (secure 0 0 0) 0x7f290407cf60 con 0x7f29101591f0 2023-10-06T12:52:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.079+0000 7f290e59c700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.079+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:41.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.087+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f290404d0a0 con 0x7f29101591f0 2023-10-06T12:52:41.191 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:41.186+0000 7f8462bab000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:41.248 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:41.243+0000 7f67ddfcc000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:41.266 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:41.262+0000 7f8462bab000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.279+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f29101580b0 con 0x7f28f00606c0 2023-10-06T12:52:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.283+0000 7f290e59c700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.283+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:41.330 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:41.327+0000 7f67ddfcc000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:41.490 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:41.486+0000 7f8462bab000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:41.559 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:41.555+0000 7f67ddfcc000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:41.574 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:41.570+0000 7f8462bab000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:41.647 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:41.643+0000 7f67ddfcc000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:41.660 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:41.658+0000 7f8462bab000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.687+0000 7f290e59c700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:41.687+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T12:52:41.742 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:41.739+0000 7f67ddfcc000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:42.065 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.062+0000 7f8462bab000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:42.160 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:42.155+0000 7f67ddfcc000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:42.236 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.234+0000 7f8462bab000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:42.333 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:42.327+0000 7f67ddfcc000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:42.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:42.491+0000 7f290e59c700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:42.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:42.491+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T12:52:42.609 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.606+0000 7f8462bab000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:42.730 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:42.727+0000 7f67ddfcc000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:42.821 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.818+0000 7f8462bab000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:42.907 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.902+0000 7f8462bab000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:42.965 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:42.959+0000 7f67ddfcc000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:42.985 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:42.982+0000 7f8462bab000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:43.059 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.054+0000 7f8462bab000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:43.059 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.055+0000 7f67ddfcc000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:43.139 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.135+0000 7f67ddfcc000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:43.222 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.219+0000 7f67ddfcc000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:43.276 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.274+0000 7f8462bab000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:43.353 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.350+0000 7f8462bab000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:43.434 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.430+0000 7f8462bab000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:43.462 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.455+0000 7f67ddfcc000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:43.546 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.539+0000 7f67ddfcc000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:43.620 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.618+0000 7f8462bab000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:43.638 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.635+0000 7f67ddfcc000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:43.703 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:43.698+0000 7f8462bab000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:43.851 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.847+0000 7f67ddfcc000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:43.946 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:43.939+0000 7f67ddfcc000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.095+0000 7f290e59c700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.095+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T12:52:44.452 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:44.450+0000 7f8462bab000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.515+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 112) v1 ==== 80078+0+0 (secure 0 0 0) 0x7f290404d470 con 0x7f29101591f0 2023-10-06T12:52:44.738 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.735+0000 7f67ddfcc000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.755+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 113) v1 ==== 79687+0+0 (secure 0 0 0) 0x7f290407b0a0 con 0x7f29101591f0 2023-10-06T12:52:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.755+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:44.755+0000 7f28fe7fc700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:44.951 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f679d240700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.951 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f679d240700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.951 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f679d240700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.951 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f679d240700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.951 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f679d240700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.952 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f67a0246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.952 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f67a0246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.952 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f67a0246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.952 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f67a0246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:44.952 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:44.947+0000 7f67a0246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.763+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 114) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f290404dd40 con 0x7f29101591f0 2023-10-06T12:52:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.767+0000 7f28fe7fc700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.767+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f29101580b0 con 0x7f28f00606c0 2023-10-06T12:52:45.775 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.771+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:45.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.775+0000 7f290e59c700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29000099b0 tx=0x7f2900017000 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.871+0000 7f28fe7fc700 1 -- 172.21.15.46:0/289728561 <== mgr.10542 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f29101580b0 con 0x7f28f00606c0 2023-10-06T12:52:45.877 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path snap_test_dir1 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 msgr2=0x7f28f0062af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29000099b0 tx=0x7f2900017000 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 msgr2=0x7f29101568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f290400ec80 tx=0x7f29040091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 shutdown_connections 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28f00606c0 0x7f28f0062af0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291015c580 0x7f2910067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29101591f0 0x7f29101568a0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 --2- 172.21.15.46:0/289728561 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29100cc500 0x7f2910146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.875+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 >> 172.21.15.46:0/289728561 conn(0x7f291006ef60 msgr2=0x7f2910152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.879+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 shutdown_connections 2023-10-06T12:52:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:45.879+0000 7f290f59e700 1 -- 172.21.15.46:0/289728561 wait complete. 2023-10-06T12:52:45.905 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T12:52:45.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-06T12:52:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.079+0000 7fd2af705700 1 Processor -- start 2023-10-06T12:52:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.079+0000 7fd2af705700 1 -- start start 2023-10-06T12:52:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2af705700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2af705700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80c1980 con 0x7fd2a8146eb0 2023-10-06T12:52:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2af705700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80c2520 con 0x7fd2a8161fc0 2023-10-06T12:52:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2af705700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80c0d70 con 0x7fd2a8147280 2023-10-06T12:52:46.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2ad4a1700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd2a8146eb0 0x7fd2a80c1e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35542/0 (socket says 172.21.15.46:35542) 2023-10-06T12:52:46.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2ad4a1700 1 -- 172.21.15.46:0/2375050143 learned_addr learned my addr 172.21.15.46:0/2375050143 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:46.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2adca2700 1 --2- 172.21.15.46:0/2375050143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd29ffff700 1 -- 172.21.15.46:0/2375050143 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405054247 0 0) 0x7fd2a80c1980 con 0x7fd2a8146eb0 2023-10-06T12:52:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd29ffff700 1 -- 172.21.15.46:0/2375050143 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd294003570 con 0x7fd2a8146eb0 2023-10-06T12:52:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2adca2700 1 -- 172.21.15.46:0/2375050143 >> v1:172.21.15.132:6789/0 conn(0x7fd2a8161fc0 legacy=0x7fd2a80caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2adca2700 1 -- 172.21.15.46:0/2375050143 >> v1:172.21.15.46:6789/0 conn(0x7fd2a8146eb0 legacy=0x7fd2a80c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2adca2700 1 -- 172.21.15.46:0/2375050143 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80c1050 con 0x7fd2a8147280 2023-10-06T12:52:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.083+0000 7fd2adca2700 1 --2- 172.21.15.46:0/2375050143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a806cc30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009e90 tx=0x7fd2a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f1f7a75dacb78b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd29ffff700 1 -- 172.21.15.46:0/2375050143 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000e070 con 0x7fd2a8147280 2023-10-06T12:52:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd29ffff700 1 -- 172.21.15.46:0/2375050143 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000bc30 con 0x7fd2a8147280 2023-10-06T12:52:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd29ffff700 1 -- 172.21.15.46:0/2375050143 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0012430 con 0x7fd2a8147280 2023-10-06T12:52:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 -- 172.21.15.46:0/2375050143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 msgr2=0x7fd2a806cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 --2- 172.21.15.46:0/2375050143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a806cc30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009e90 tx=0x7fd2a0009780 comp rx=0 tx=0).stop 2023-10-06T12:52:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 -- 172.21.15.46:0/2375050143 shutdown_connections 2023-10-06T12:52:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 --2- 172.21.15.46:0/2375050143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a806cc30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 -- 172.21.15.46:0/2375050143 >> 172.21.15.46:0/2375050143 conn(0x7fd2a80cbe30 msgr2=0x7fd2a80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 -- 172.21.15.46:0/2375050143 shutdown_connections 2023-10-06T12:52:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.087+0000 7fd2af705700 1 -- 172.21.15.46:0/2375050143 wait complete. 2023-10-06T12:52:46.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 Processor -- start 2023-10-06T12:52:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 -- start start 2023-10-06T12:52:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8146eb0 0x7fd2a80b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b03a0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b0f20 con 0x7fd2a8146eb0 2023-10-06T12:52:46.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2af705700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a80b05f0 con 0x7fd2a8147280 2023-10-06T12:52:46.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.091+0000 7fd2adca2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34194/0 (socket says 172.21.15.46:34194) 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 -- 172.21.15.46:0/1443359829 learned_addr learned my addr 172.21.15.46:0/1443359829 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2acca0700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2ad4a1700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8146eb0 0x7fd2a80b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 -- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 msgr2=0x7fd2a815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 -- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8146eb0 msgr2=0x7fd2a80b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8146eb0 0x7fd2a80b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 -- 172.21.15.46:0/1443359829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009a20 con 0x7fd2a8161fc0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2adca2700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000b6c0 tx=0x7fd2a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0027470 con 0x7fd2a8161fc0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a00275d0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80b08d0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0027970 con 0x7fd2a8161fc0 2023-10-06T12:52:46.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.095+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a815d6e0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.099+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80660f0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.099+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 114) v1 ==== 79814+0+0 (secure 0 0 0) 0x7fd2a0024070 con 0x7fd2a8161fc0 2023-10-06T12:52:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.099+0000 7fd29d7fa700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 0x7fd28c0628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.099+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7fd2a007f4d0 con 0x7fd2a8161fc0 2023-10-06T12:52:46.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.103+0000 7fd2ad4a1700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 0x7fd28c0628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.103+0000 7fd2ad4a1700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 0x7fd28c0628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007fd0 tx=0x7fd2a4009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.107+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2a0053240 con 0x7fd2a8161fc0 2023-10-06T12:52:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.299+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2a80b1060 con 0x7fd28c0603f0 2023-10-06T12:52:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.391+0000 7fd29d7fa700 1 -- 172.21.15.46:0/1443359829 <== mgr.10542 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fd2a80b1060 con 0x7fd28c0603f0 2023-10-06T12:52:46.398 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path snap_test_dir1 2023-10-06T12:52:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 msgr2=0x7fd28c0628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 0x7fd28c0628a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007fd0 tx=0x7fd2a4009040 comp rx=0 tx=0).stop 2023-10-06T12:52:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 msgr2=0x7fd2a815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000b6c0 tx=0x7fd2a0019000 comp rx=0 tx=0).stop 2023-10-06T12:52:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 shutdown_connections 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2a8161fc0 0x7fd2a815f2d0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2a8147280 0x7fd2a815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fd28c0603f0 0x7fd28c0628a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 --2- 172.21.15.46:0/1443359829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8146eb0 0x7fd2a80b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 >> 172.21.15.46:0/1443359829 conn(0x7fd2a80cbe30 msgr2=0x7fd2a80c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 shutdown_connections 2023-10-06T12:52:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.395+0000 7fd2af705700 1 -- 172.21.15.46:0/1443359829 wait complete. 2023-10-06T12:52:46.422 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-06T12:52:46.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T12:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 Processor -- start 2023-10-06T12:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 -- start start 2023-10-06T12:52:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 0x7f65340c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f653406d340 0x7f65340caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65340c1980 con 0x7f65340c2350 2023-10-06T12:52:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340cb1e0 con 0x7f653406d340 2023-10-06T12:52:46.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.591+0000 7f653af85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c0d70 con 0x7f6534146eb0 2023-10-06T12:52:46.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f653406d340 0x7f65340caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6533fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f65340c2350 0x7f653406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52530/0 (socket says 172.21.15.46:52530) 2023-10-06T12:52:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f653406d340 0x7f65340caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:58948/0 (socket says 172.21.15.46:58948) 2023-10-06T12:52:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6533fff700 1 -- 172.21.15.46:0/2414478877 learned_addr learned my addr 172.21.15.46:0/2414478877 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6538d21700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 0x7f65340c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f65337fe700 1 -- 172.21.15.46:0/2414478877 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219245760 0 0) 0x7f65340c1980 con 0x7f65340c2350 2023-10-06T12:52:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f65337fe700 1 -- 172.21.15.46:0/2414478877 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6524003570 con 0x7f65340c2350 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 -- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 msgr2=0x7f65340c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 0x7f65340c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 -- 172.21.15.46:0/2414478877 >> v1:172.21.15.132:6789/0 conn(0x7f65340c2350 legacy=0x7f653406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 -- 172.21.15.46:0/2414478877 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65340c1050 con 0x7f653406d340 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6538d21700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 0x7f65340c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:46.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f6539522700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653406d340 0x7f65340caca0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f652c009e60 tx=0x7f652c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ba62e6407c9001b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f65337fe700 1 -- 172.21.15.46:0/2414478877 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00e070 con 0x7f653406d340 2023-10-06T12:52:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f65337fe700 1 -- 172.21.15.46:0/2414478877 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c00bab0 con 0x7f653406d340 2023-10-06T12:52:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f65337fe700 1 -- 172.21.15.46:0/2414478877 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00be70 con 0x7f653406d340 2023-10-06T12:52:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 -- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653406d340 msgr2=0x7f65340caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653406d340 0x7f65340caca0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f652c009e60 tx=0x7f652c009780 comp rx=0 tx=0).stop 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 -- 172.21.15.46:0/2414478877 shutdown_connections 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f653406d340 0x7f65340caca0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 --2- 172.21.15.46:0/2414478877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6534146eb0 0x7f65340c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.595+0000 7f653af85700 1 -- 172.21.15.46:0/2414478877 >> 172.21.15.46:0/2414478877 conn(0x7f65340cbe30 msgr2=0x7f65340cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.599+0000 7f653af85700 1 -- 172.21.15.46:0/2414478877 shutdown_connections 2023-10-06T12:52:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.599+0000 7f653af85700 1 -- 172.21.15.46:0/2414478877 wait complete. 2023-10-06T12:52:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.599+0000 7f653af85700 1 Processor -- start 2023-10-06T12:52:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.599+0000 7f653af85700 1 -- start start 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 0x7f6534142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6534146eb0 0x7f65340af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c5d40 con 0x7f653406d340 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c5370 con 0x7f6534146eb0 2023-10-06T12:52:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c4760 con 0x7f65340c2350 2023-10-06T12:52:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34204/0 (socket says 172.21.15.46:34204) 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 -- 172.21.15.46:0/3171894474 learned_addr learned my addr 172.21.15.46:0/3171894474 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6533fff700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 0x7f6534142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6539522700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6534146eb0 0x7f65340af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 -- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 msgr2=0x7f6534142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 0x7f6534142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 -- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6534146eb0 msgr2=0x7f65340af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6534146eb0 0x7f65340af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 -- 172.21.15.46:0/3171894474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528009420 con 0x7f653406d340 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6533fff700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 0x7f6534142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:46.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6538d21700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f652800ec80 tx=0x7f65280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800c850 con 0x7f653406d340 2023-10-06T12:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800c9b0 con 0x7f653406d340 2023-10-06T12:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c009a20 con 0x7f653406d340 2023-10-06T12:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652800cd30 con 0x7f653406d340 2023-10-06T12:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.603+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65340c4d20 con 0x7f653406d340 2023-10-06T12:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.607+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65340660f0 con 0x7f653406d340 2023-10-06T12:52:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.607+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 114) v1 ==== 79814+0+0 (secure 0 0 0) 0x7f6528013680 con 0x7f653406d340 2023-10-06T12:52:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.607+0000 7f6530ff9700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 0x7f65180628a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.607+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f652807d070 con 0x7f653406d340 2023-10-06T12:52:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.611+0000 7f6533fff700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 0x7f65180628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.611+0000 7f6533fff700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 0x7f65180628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65340c5ee0 tx=0x7f6520009040 comp rx=0 tx=0).ready entity=mgr.10542 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.615+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f65280acbe0 con 0x7f653406d340 2023-10-06T12:52:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.779+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 115) v1 ==== 80135+0+0 (secure 0 0 0) 0x7f652804d720 con 0x7f653406d340 2023-10-06T12:52:46.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:46.835+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f653404b5c0 con 0x7f653406d340 2023-10-06T12:52:47.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.783+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f6528050500 con 0x7f653406d340 2023-10-06T12:52:47.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.787+0000 7f6530ff9700 1 -- 172.21.15.46:0/3171894474 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v116) v1 ==== 93+0+0 (secure 0 0 0) 0x7f652804e360 con 0x7f653406d340 2023-10-06T12:52:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.787+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 msgr2=0x7f65180628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.787+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 0x7f65180628a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65340c5ee0 tx=0x7f6520009040 comp rx=0 tx=0).stop 2023-10-06T12:52:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 msgr2=0x7f65340c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f652800ec80 tx=0x7f65280091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 shutdown_connections 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f65180603f0 0x7f65180628a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6534146eb0 0x7f65340af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65340c2350 0x7f6534142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 --2- 172.21.15.46:0/3171894474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653406d340 0x7f65340c5800 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 >> 172.21.15.46:0/3171894474 conn(0x7f65340cbe30 msgr2=0x7f6534144890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 shutdown_connections 2023-10-06T12:52:47.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:47.791+0000 7f653af85700 1 -- 172.21.15.46:0/3171894474 wait complete. 2023-10-06T12:52:47.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T12:52:47.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:47.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T12:52:47.890 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T12:52:47.891 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T12:52:47.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T12:52:47.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T12:52:47.969 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:47.966+0000 7f99535c9000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:47.982 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:52:47.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:47.983 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T12:52:47.988 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:47.983+0000 7facfd586000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T12:52:48.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:48.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.203+0000 7f13964ae700 1 Processor -- start 2023-10-06T12:52:48.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.203+0000 7f13964ae700 1 -- start start 2023-10-06T12:52:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f13964ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f13901571d0 0x7f13900ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f13964ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13900c9e80 con 0x7f13900ce530 2023-10-06T12:52:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f13964ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139006ba20 con 0x7f13900ce160 2023-10-06T12:52:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f13964ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13900ca850 con 0x7f13901571d0 2023-10-06T12:52:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f13900ce530 0x7f1390156ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35554/0 (socket says 172.21.15.46:35554) 2023-10-06T12:52:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138f7fe700 1 -- 172.21.15.46:0/2011149421 learned_addr learned my addr 172.21.15.46:0/2011149421 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f1394a4b700 1 --2- 172.21.15.46:0/2011149421 >> v2:172.21.15.132:3300/0 conn(0x7f13901571d0 0x7f13900ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102899961 0 0) 0x7f13900c9e80 con 0x7f13900ce530 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1378003570 con 0x7f13900ce530 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221582762 0 0) 0x7f139006ba20 con 0x7f13900ce160 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13900c9e80 con 0x7f13900ce160 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f1394a4b700 1 -- 172.21.15.46:0/2011149421 >> v1:172.21.15.132:6789/0 conn(0x7f13900ce160 legacy=0x7f139015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f1394a4b700 1 -- 172.21.15.46:0/2011149421 >> v1:172.21.15.46:6789/0 conn(0x7f13900ce530 legacy=0x7f1390156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f1394a4b700 1 -- 172.21.15.46:0/2011149421 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139015b790 con 0x7f13901571d0 2023-10-06T12:52:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f1394a4b700 1 --2- 172.21.15.46:0/2011149421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901571d0 0x7f13900ca310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e60 tx=0x7f1388009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176e16fa9b952d98 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138800e070 con 0x7f13901571d0 2023-10-06T12:52:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138800bab0 con 0x7f13901571d0 2023-10-06T12:52:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.207+0000 7f138effd700 1 -- 172.21.15.46:0/2011149421 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138800be70 con 0x7f13901571d0 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- 172.21.15.46:0/2011149421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901571d0 msgr2=0x7f13900ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 --2- 172.21.15.46:0/2011149421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901571d0 0x7f13900ca310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1388009e60 tx=0x7f1388009780 comp rx=0 tx=0).stop 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- 172.21.15.46:0/2011149421 shutdown_connections 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 --2- 172.21.15.46:0/2011149421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13901571d0 0x7f13900ca310 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- 172.21.15.46:0/2011149421 >> 172.21.15.46:0/2011149421 conn(0x7f1390062d50 msgr2=0x7f1390063150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- 172.21.15.46:0/2011149421 shutdown_connections 2023-10-06T12:52:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- 172.21.15.46:0/2011149421 wait complete. 2023-10-06T12:52:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 Processor -- start 2023-10-06T12:52:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 -- start start 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13900ce160 0x7f1390171ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 0x7f139016d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.211+0000 7f13964ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 0x7f13900b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f13964ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390172420 con 0x7f13901571d0 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f13964ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139016e0c0 con 0x7f13900ce530 2023-10-06T12:52:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f13964ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139016e3a0 con 0x7f13900ce160 2023-10-06T12:52:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1394a4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 0x7f13900b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 0x7f139016d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138ffff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13900ce160 0x7f1390171ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1394a4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 0x7f13900b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34230/0 (socket says 172.21.15.46:34230) 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1394a4b700 1 -- 172.21.15.46:0/2083582188 learned_addr learned my addr 172.21.15.46:0/2083582188 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13900ce160 msgr2=0x7f1390171ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13900ce160 0x7f1390171ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 msgr2=0x7f13900b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 0x7f13900b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 -- 172.21.15.46:0/2083582188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388009a20 con 0x7f13900ce530 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f138f7fe700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 0x7f139016d760 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f138400c580 tx=0x7f1384007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384005b30 con 0x7f13900ce530 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384005c90 con 0x7f13900ce530 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384007320 con 0x7f13900ce530 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139016e680 con 0x7f13900ce530 2023-10-06T12:52:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.215+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139016ec40 con 0x7f13900ce530 2023-10-06T12:52:48.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.219+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1390069a20 con 0x7f13900ce530 2023-10-06T12:52:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.219+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f138401b3b0 con 0x7f13900ce530 2023-10-06T12:52:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.219+0000 7f1377fff700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 0x7f137c062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.219+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f138407c510 con 0x7f13900ce530 2023-10-06T12:52:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.219+0000 7f138ffff700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 msgr2=0x7f137c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.223+0000 7f138ffff700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 0x7f137c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:48.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.231+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1390069a20 con 0x7f13900ce530 2023-10-06T12:52:48.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.427+0000 7f138ffff700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 msgr2=0x7f137c062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:48.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.427+0000 7f138ffff700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 0x7f137c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.443+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1390069a20 con 0x7f13900ce530 2023-10-06T12:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.443+0000 7f1377fff700 1 -- 172.21.15.46:0/2083582188 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7f1390069a20 con 0x7f13900ce530 2023-10-06T12:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 msgr2=0x7f137c062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 0x7f137c062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 msgr2=0x7f139016d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 0x7f139016d760 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f138400c580 tx=0x7f1384007d70 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 shutdown_connections 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13901571d0 0x7f13900b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f137c060700 0x7f137c062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13900ce530 0x7f139016d760 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 --2- 172.21.15.46:0/2083582188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f13900ce160 0x7f1390171ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 >> 172.21.15.46:0/2083582188 conn(0x7f1390062d50 msgr2=0x7f1390152500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:48.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 shutdown_connections 2023-10-06T12:52:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.447+0000 7f13964ae700 1 -- 172.21.15.46:0/2083582188 wait complete. 2023-10-06T12:52:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:48.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:48.500 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:48.498+0000 7f99535c9000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:48.565 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:48.559+0000 7facfd586000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T12:52:48.575 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:48.570+0000 7f99535c9000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:48.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.619+0000 7f746a6cd700 1 Processor -- start 2023-10-06T12:52:48.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 -- start start 2023-10-06T12:52:48.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f746415ab60 0x7f746406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f74640adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746406d460 con 0x7f746415af30 2023-10-06T12:52:48.627 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640c7000 con 0x7f746415ab60 2023-10-06T12:52:48.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f746a6cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c72c0 con 0x7f74640ad6c0 2023-10-06T12:52:48.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f746415ab60 0x7f746406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f745b7fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f746415af30 0x7f746406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52556/0 (socket says 172.21.15.46:52556) 2023-10-06T12:52:48.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f745b7fe700 1 -- 172.21.15.46:0/3891583297 learned_addr learned my addr 172.21.15.46:0/3891583297 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:48.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f746415ab60 0x7f746406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59008/0 (socket says 172.21.15.46:59008) 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7468c6a700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f74640adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 -- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 msgr2=0x7f74640adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f74640adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 -- 172.21.15.46:0/3891583297 >> v1:172.21.15.132:6789/0 conn(0x7f746415af30 legacy=0x7f746406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 -- 172.21.15.46:0/3891583297 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640c75a0 con 0x7f746415ab60 2023-10-06T12:52:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f74637fe700 1 -- 172.21.15.46:0/3891583297 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548761368 0 0) 0x7f746406d460 con 0x7f746415af30 2023-10-06T12:52:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.623+0000 7f7463fff700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f746406e8a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f74540098d0 tx=0x7f7454009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de8f34af9d76daa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f74637fe700 1 -- 172.21.15.46:0/3891583297 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454009e70 con 0x7f746415ab60 2023-10-06T12:52:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f74637fe700 1 -- 172.21.15.46:0/3891583297 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7454005c10 con 0x7f746415ab60 2023-10-06T12:52:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f74637fe700 1 -- 172.21.15.46:0/3891583297 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400b650 con 0x7f746415ab60 2023-10-06T12:52:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 -- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 msgr2=0x7f746406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f746406e8a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f74540098d0 tx=0x7f7454009120 comp rx=0 tx=0).stop 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 -- 172.21.15.46:0/3891583297 shutdown_connections 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f74640adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 --2- 172.21.15.46:0/3891583297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f746406e8a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 -- 172.21.15.46:0/3891583297 >> 172.21.15.46:0/3891583297 conn(0x7f74640cbe30 msgr2=0x7f74640cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 -- 172.21.15.46:0/3891583297 shutdown_connections 2023-10-06T12:52:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.627+0000 7f746a6cd700 1 -- 172.21.15.46:0/3891583297 wait complete. 2023-10-06T12:52:48.634 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 Processor -- start 2023-10-06T12:52:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 -- start start 2023-10-06T12:52:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f74640ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 0x7f74640ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7464151070 con 0x7f746415af30 2023-10-06T12:52:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640af870 con 0x7f746415ab60 2023-10-06T12:52:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f746a6cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640af260 con 0x7f74640ad6c0 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f7463fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f7468c6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 0x7f74640ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f7463fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:43300/0 (socket says 172.21.15.46:43300) 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f7463fff700 1 -- 172.21.15.46:0/666842395 learned_addr learned my addr 172.21.15.46:0/666842395 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.631+0000 7f745b7fe700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f74640ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 -- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 msgr2=0x7f7464150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 -- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 msgr2=0x7f74640ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f74640ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 -- 172.21.15.46:0/666842395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540093c0 con 0x7f746415af30 2023-10-06T12:52:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7463fff700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7468c6a700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 0x7f74640ca910 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f745c007e70 tx=0x7f745c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c018070 con 0x7f746415af30 2023-10-06T12:52:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00eac0 con 0x7f746415af30 2023-10-06T12:52:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7464150a90 con 0x7f746415af30 2023-10-06T12:52:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00ee60 con 0x7f746415af30 2023-10-06T12:52:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74640af580 con 0x7f746415af30 2023-10-06T12:52:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.635+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74640afa00 con 0x7f746415af30 2023-10-06T12:52:48.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.639+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f745c003020 con 0x7f746415af30 2023-10-06T12:52:48.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.639+0000 7f7460ff9700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 0x7f7444062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:48.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.639+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f745c07d0c0 con 0x7f746415af30 2023-10-06T12:52:48.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.639+0000 7f7463fff700 1 -- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 msgr2=0x7f7444062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:48.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.639+0000 7f7463fff700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 0x7f7444062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:48.646 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:48.643+0000 7facfd586000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T12:52:48.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.647+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f745c050930 con 0x7f746415af30 2023-10-06T12:52:48.711 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:48.706+0000 7f99535c9000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:48.790 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:48.783+0000 7facfd586000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:48.826 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.823+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74640cb350 con 0x7f746415af30 2023-10-06T12:52:48.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.823+0000 7f7460ff9700 1 -- 172.21.15.46:0/666842395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+18247 (secure 0 0 0) 0x7f745c0098f0 con 0x7f746415af30 2023-10-06T12:52:48.829 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:48.830 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":198,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:44.739083+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1339853979":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1151621779":"2023-10-07T12:52:44.738959+0000","172.21.15.46:6833/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/938799999":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/580990329":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/3242963477":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1367734289":"2023-10-07T12:52:44.738959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 msgr2=0x7f7444062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 0x7f7444062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 msgr2=0x7f74640ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 0x7f74640ca910 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f745c007e70 tx=0x7f745c0097b0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 shutdown_connections 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f746415af30 0x7f74640ca910 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7444060700 0x7f7444062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f746415ab60 0x7f74640ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 --2- 172.21.15.46:0/666842395 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74640ad6c0 0x7f7464150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.827+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 >> 172.21.15.46:0/666842395 conn(0x7f74640cbe30 msgr2=0x7f74640acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.831+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 shutdown_connections 2023-10-06T12:52:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:48.831+0000 7f746a6cd700 1 -- 172.21.15.46:0/666842395 wait complete. 2023-10-06T12:52:48.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:18:10.565075+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:18:16.414619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T12:52:25.312557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T12:52:26.484497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T12:52:48.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T12:52:48.976 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:48.974+0000 7f99535c9000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:49.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.019+0000 7f014a7c5700 1 Processor -- start 2023-10-06T12:52:49.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.019+0000 7f014a7c5700 1 -- start start 2023-10-06T12:52:49.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f014a7c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0144069410 0x7f014415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f014a7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144159420 0x7f014415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f014a7c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01440ce470 con 0x7f01440697e0 2023-10-06T12:52:49.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f014a7c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014415a890 con 0x7f0144069410 2023-10-06T12:52:49.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f014a7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014415d470 con 0x7f0144159420 2023-10-06T12:52:49.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f01437fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f01440697e0 0x7f0144158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52558/0 (socket says 172.21.15.46:52558) 2023-10-06T12:52:49.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f01437fe700 1 -- 172.21.15.46:0/1713284757 learned_addr learned my addr 172.21.15.46:0/1713284757 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 --2- 172.21.15.46:0/1713284757 >> v2:172.21.15.132:3300/0 conn(0x7f0144069410 0x7f014415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0148d62700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144159420 0x7f014415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0142ffd700 1 -- 172.21.15.46:0/1713284757 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252797638 0 0) 0x7f01440ce470 con 0x7f01440697e0 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0142ffd700 1 -- 172.21.15.46:0/1713284757 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f012c003570 con 0x7f01440697e0 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 -- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144159420 msgr2=0x7f014415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144159420 0x7f014415a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 -- 172.21.15.46:0/1713284757 >> v1:172.21.15.132:6789/0 conn(0x7f01440697e0 legacy=0x7f0144158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 -- 172.21.15.46:0/1713284757 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014415d750 con 0x7f0144069410 2023-10-06T12:52:49.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.023+0000 7f0143fff700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144069410 0x7f014415ce90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f01340098e0 tx=0x7f01340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d692adf8956fdfeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f0142ffd700 1 -- 172.21.15.46:0/1713284757 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011630 con 0x7f0144069410 2023-10-06T12:52:49.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f0142ffd700 1 -- 172.21.15.46:0/1713284757 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134011790 con 0x7f0144069410 2023-10-06T12:52:49.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f0142ffd700 1 -- 172.21.15.46:0/1713284757 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011b30 con 0x7f0144069410 2023-10-06T12:52:49.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 -- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144069410 msgr2=0x7f014415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144069410 0x7f014415ce90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f01340098e0 tx=0x7f01340091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 -- 172.21.15.46:0/1713284757 shutdown_connections 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144159420 0x7f014415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 --2- 172.21.15.46:0/1713284757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144069410 0x7f014415ce90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 -- 172.21.15.46:0/1713284757 >> 172.21.15.46:0/1713284757 conn(0x7f01440d0210 msgr2=0x7f01440d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 -- 172.21.15.46:0/1713284757 shutdown_connections 2023-10-06T12:52:49.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.027+0000 7f014a7c5700 1 -- 172.21.15.46:0/1713284757 wait complete. 2023-10-06T12:52:49.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 Processor -- start 2023-10-06T12:52:49.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 -- start start 2023-10-06T12:52:49.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01440697e0 0x7f01440cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144159420 0x7f0144169230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144153c20 con 0x7f01440697e0 2023-10-06T12:52:49.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440c8a20 con 0x7f0144159420 2023-10-06T12:52:49.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f014a7c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0144155610 con 0x7f0144069410 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f01437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01440697e0 0x7f01440cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:43316/0 (socket says 172.21.15.46:43316) 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 -- 172.21.15.46:0/2676515607 learned_addr learned my addr 172.21.15.46:0/2676515607 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0148d62700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144159420 0x7f0144169230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 -- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144159420 msgr2=0x7f0144169230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144159420 0x7f0144169230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 -- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01440697e0 msgr2=0x7f01440cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01440697e0 0x7f01440cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.031+0000 7f0143fff700 1 -- 172.21.15.46:0/2676515607 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134009420 con 0x7f0144069410 2023-10-06T12:52:49.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f0143fff700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0144088000 tx=0x7f0134017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011900 con 0x7f0144069410 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134011a60 con 0x7f0144069410 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0144154850 con 0x7f0144069410 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134023620 con 0x7f0144069410 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144154e10 con 0x7f0144069410 2023-10-06T12:52:49.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.035+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01440cdce0 con 0x7f0144069410 2023-10-06T12:52:49.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.039+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f013402c3b0 con 0x7f0144069410 2023-10-06T12:52:49.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.039+0000 7f012bfff700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 0x7f0130062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.039+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f013407e3b0 con 0x7f0144069410 2023-10-06T12:52:49.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.039+0000 7f01437fe700 1 -- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 msgr2=0x7f0130062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:49.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.039+0000 7f01437fe700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 0x7f0130062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:49.046 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.042+0000 7f99535c9000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:49.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.047+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f01440cdce0 con 0x7f0144069410 2023-10-06T12:52:49.085 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.079+0000 7facfd586000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T12:52:49.160 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.155+0000 7facfd586000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:49.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.223+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01440cdce0 con 0x7f0144069410 2023-10-06T12:52:49.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.223+0000 7f012bfff700 1 -- 172.21.15.46:0/2676515607 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01440cdce0 con 0x7f0144069410 2023-10-06T12:52:49.228 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-10-06T12:52:49.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 msgr2=0x7f0130062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:49.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 0x7f0130062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 msgr2=0x7f01440c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0144088000 tx=0x7f0134017000 comp rx=0 tx=0).stop 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 shutdown_connections 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0130060700 0x7f0130062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0144159420 0x7f0144169230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01440697e0 0x7f01440cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 --2- 172.21.15.46:0/2676515607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0144069410 0x7f01440c84e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 >> 172.21.15.46:0/2676515607 conn(0x7f01440d0210 msgr2=0x7f01440ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:49.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 shutdown_connections 2023-10-06T12:52:49.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.227+0000 7f014a7c5700 1 -- 172.21.15.46:0/2676515607 wait complete. 2023-10-06T12:52:49.249 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.246+0000 7f99535c9000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:49.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T12:52:49.326 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.322+0000 7f99535c9000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:49.392 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.387+0000 7facfd586000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T12:52:49.410 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.406+0000 7f99535c9000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.415+0000 7f73a9ccd700 1 Processor -- start 2023-10-06T12:52:49.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.415+0000 7f73a9ccd700 1 -- start start 2023-10-06T12:52:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a9ccd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73a4159340 0x7f73a4067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a9ccd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a4067590 0x7f73a40ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a9ccd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a415a350 con 0x7f73a40abe50 2023-10-06T12:52:49.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a9ccd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a40679c0 con 0x7f73a4159340 2023-10-06T12:52:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a9ccd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a415a7e0 con 0x7f73a4067590 2023-10-06T12:52:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a37fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73a4159340 0x7f73a4067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a37fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73a4159340 0x7f73a4067050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34258/0 (socket says 172.21.15.46:34258) 2023-10-06T12:52:49.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.419+0000 7f73a37fe700 1 -- 172.21.15.46:0/975805019 learned_addr learned my addr 172.21.15.46:0/975805019 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a2ffd700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a4067590 0x7f73a40ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a37fe700 1 -- 172.21.15.46:0/975805019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a4067590 msgr2=0x7f73a40ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a37fe700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a4067590 0x7f73a40ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a37fe700 1 -- 172.21.15.46:0/975805019 >> v1:172.21.15.46:6789/0 conn(0x7f73a40abe50 legacy=0x7f73a415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a37fe700 1 -- 172.21.15.46:0/975805019 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a40ac670 con 0x7f73a4159340 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a37fe700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4159340 0x7f73a4067050 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f73940099b0 tx=0x7f7394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc87c814faa563a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a27fc700 1 -- 172.21.15.46:0/975805019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400b860 con 0x7f73a4159340 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a27fc700 1 -- 172.21.15.46:0/975805019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739400b9c0 con 0x7f73a4159340 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a27fc700 1 -- 172.21.15.46:0/975805019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400bd60 con 0x7f73a4159340 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 -- 172.21.15.46:0/975805019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4159340 msgr2=0x7f73a4067050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4159340 0x7f73a4067050 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f73940099b0 tx=0x7f7394009250 comp rx=0 tx=0).stop 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 -- 172.21.15.46:0/975805019 shutdown_connections 2023-10-06T12:52:49.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a4067590 0x7f73a40ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/975805019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4159340 0x7f73a4067050 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 -- 172.21.15.46:0/975805019 >> 172.21.15.46:0/975805019 conn(0x7f73a4068c30 msgr2=0x7f73a40aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 -- 172.21.15.46:0/975805019 shutdown_connections 2023-10-06T12:52:49.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.423+0000 7f73a9ccd700 1 -- 172.21.15.46:0/975805019 wait complete. 2023-10-06T12:52:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 Processor -- start 2023-10-06T12:52:49.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- start start 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a40abe50 0x7f73a4156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 0x7f73a4157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a4169fb0 con 0x7f73a4067590 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40cb0c0 con 0x7f73a4159340 2023-10-06T12:52:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a40cb3a0 con 0x7f73a40abe50 2023-10-06T12:52:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 0x7f73a4157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34260/0 (socket says 172.21.15.46:34260) 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 -- 172.21.15.46:0/2726906691 learned_addr learned my addr 172.21.15.46:0/2726906691 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a2ffd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a40abe50 0x7f73a4156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 -- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a40abe50 msgr2=0x7f73a4156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a40abe50 0x7f73a4156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 -- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 msgr2=0x7f73a4157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 0x7f73a4157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 -- 172.21.15.46:0/2726906691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73940094f0 con 0x7f73a4067590 2023-10-06T12:52:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a3fff700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 0x7f73a4157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a37fe700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f73a40cdf50 tx=0x7f7394017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394011e40 con 0x7f73a4067590 2023-10-06T12:52:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739401a420 con 0x7f73a4067590 2023-10-06T12:52:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739401a7e0 con 0x7f73a4067590 2023-10-06T12:52:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a40cb680 con 0x7f73a4067590 2023-10-06T12:52:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.427+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a40cbc40 con 0x7f73a4067590 2023-10-06T12:52:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.431+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a40cbe60 con 0x7f73a4067590 2023-10-06T12:52:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.431+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f739401ab90 con 0x7f73a4067590 2023-10-06T12:52:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.431+0000 7f738b7fe700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 0x7f7390062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.431+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f739407e8e0 con 0x7f73a4067590 2023-10-06T12:52:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.435+0000 7f73a2ffd700 1 -- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 msgr2=0x7f7390062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.435+0000 7f73a2ffd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 0x7f7390062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.439+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f739404e6c0 con 0x7f73a4067590 2023-10-06T12:52:49.477 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.471+0000 7facfd586000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T12:52:49.566 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.563+0000 7facfd586000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T12:52:49.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.607+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f73a4063820 con 0x7f73a4067590 2023-10-06T12:52:49.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.607+0000 7f738b7fe700 1 -- 172.21.15.46:0/2726906691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v198) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7394051ce0 con 0x7f73a4067590 2023-10-06T12:52:49.613 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 8 2023-10-06T12:52:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 msgr2=0x7f7390062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 0x7f7390062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 msgr2=0x7f73a4144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f73a40cdf50 tx=0x7f7394017000 comp rx=0 tx=0).stop 2023-10-06T12:52:49.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 shutdown_connections 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7390060730 0x7f7390062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73a4159340 0x7f73a4157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73a40abe50 0x7f73a4156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 --2- 172.21.15.46:0/2726906691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73a4067590 0x7f73a4144aa0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 >> 172.21.15.46:0/2726906691 conn(0x7f73a4068c30 msgr2=0x7f73a40aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 shutdown_connections 2023-10-06T12:52:49.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.611+0000 7f73a9ccd700 1 -- 172.21.15.46:0/2726906691 wait complete. 2023-10-06T12:52:49.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T12:52:49.769 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.766+0000 7f99535c9000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.795+0000 7fbc158f0700 1 Processor -- start 2023-10-06T12:52:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.795+0000 7fbc158f0700 1 -- start start 2023-10-06T12:52:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.795+0000 7fbc158f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 0x7fbc1015a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.795+0000 7fbc158f0700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1015d3c0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc158f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1015cdb0 con 0x7fbc10158da0 2023-10-06T12:52:49.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc158f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10069800 con 0x7fbc1015d850 2023-10-06T12:52:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc07fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc100cb5c0 0x7fbc100cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35580/0 (socket says 172.21.15.46:35580) 2023-10-06T12:52:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0effd700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbc10158da0 0x7fbc100caeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52568/0 (socket says 172.21.15.46:52568) 2023-10-06T12:52:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc07fff700 1 -- 172.21.15.46:0/1316198482 learned_addr learned my addr 172.21.15.46:0/1316198482 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:49.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0f7fe700 1 --2- 172.21.15.46:0/1316198482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 0x7fbc1015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0e7fc700 1 -- 172.21.15.46:0/1316198482 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739776368 0 0) 0x7fbc1015d3c0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0e7fc700 1 -- 172.21.15.46:0/1316198482 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbfc003570 con 0x7fbc100cb5c0 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0f7fe700 1 -- 172.21.15.46:0/1316198482 >> v1:172.21.15.132:6789/0 conn(0x7fbc10158da0 legacy=0x7fbc100caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0f7fe700 1 -- 172.21.15.46:0/1316198482 >> v1:172.21.15.46:6789/0 conn(0x7fbc100cb5c0 legacy=0x7fbc100cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0f7fe700 1 -- 172.21.15.46:0/1316198482 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1015d050 con 0x7fbc1015d850 2023-10-06T12:52:49.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0f7fe700 1 --2- 172.21.15.46:0/1316198482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 0x7fbc1015a180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009e90 tx=0x7fbc08009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e49c553c164dde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0e7fc700 1 -- 172.21.15.46:0/1316198482 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800e070 con 0x7fbc1015d850 2023-10-06T12:52:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0e7fc700 1 -- 172.21.15.46:0/1316198482 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0800bc30 con 0x7fbc1015d850 2023-10-06T12:52:49.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.799+0000 7fbc0e7fc700 1 -- 172.21.15.46:0/1316198482 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08012430 con 0x7fbc1015d850 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 -- 172.21.15.46:0/1316198482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 msgr2=0x7fbc1015a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1316198482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 0x7fbc1015a180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009e90 tx=0x7fbc08009780 comp rx=0 tx=0).stop 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 -- 172.21.15.46:0/1316198482 shutdown_connections 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1316198482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc1015d850 0x7fbc1015a180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 -- 172.21.15.46:0/1316198482 >> 172.21.15.46:0/1316198482 conn(0x7fbc100683f0 msgr2=0x7fbc100d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 -- 172.21.15.46:0/1316198482 shutdown_connections 2023-10-06T12:52:49.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 -- 172.21.15.46:0/1316198482 wait complete. 2023-10-06T12:52:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.803+0000 7fbc158f0700 1 Processor -- start 2023-10-06T12:52:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- start start 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc10158da0 0x7fbc100c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1015d850 0x7fbc1015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c74c0 con 0x7fbc1015d850 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c77a0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c7a80 con 0x7fbc10158da0 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1015d850 0x7fbc1015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59046/0 (socket says 172.21.15.46:59046) 2023-10-06T12:52:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 -- 172.21.15.46:0/1187586379 learned_addr learned my addr 172.21.15.46:0/1187586379 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:49.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc07fff700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc10158da0 0x7fbc100c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 -- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc10158da0 msgr2=0x7fbc100c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc10158da0 0x7fbc100c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 -- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1015d850 msgr2=0x7fbc1015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1015d850 0x7fbc1015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 -- 172.21.15.46:0/1187586379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009a20 con 0x7fbc100cb5c0 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc0effd700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000c7c0 tx=0x7fbc00007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00005a70 con 0x7fbc100cb5c0 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc00005bd0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000070d0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc100c7d60 con 0x7fbc100cb5c0 2023-10-06T12:52:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.807+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc100c7120 con 0x7fbc100cb5c0 2023-10-06T12:52:49.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc100886e0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fbc00007480 con 0x7fbc100cb5c0 2023-10-06T12:52:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc06ffd700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 0x7fbbf0062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7fbc0007c470 con 0x7fbc100cb5c0 2023-10-06T12:52:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc07fff700 1 -- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 msgr2=0x7fbbf0062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.811+0000 7fbc07fff700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 0x7fbbf0062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.823+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbc100886e0 con 0x7fbc100cb5c0 2023-10-06T12:52:49.925 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:49.922+0000 7f99535c9000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:49.985 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:49.979+0000 7facfd586000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T12:52:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.995+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc100886e0 con 0x7fbc100cb5c0 2023-10-06T12:52:50.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc06ffd700 1 -- 172.21.15.46:0/1187586379 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc100886e0 con 0x7fbc100cb5c0 2023-10-06T12:52:50.003 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-10-06T12:52:50.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 msgr2=0x7fbbf0062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 0x7fbbf0062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 msgr2=0x7fbc1016c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000c7c0 tx=0x7fbc00007f70 comp rx=0 tx=0).stop 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 shutdown_connections 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1015d850 0x7fbc1015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc10158da0 0x7fbc100c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbbf0060730 0x7fbbf0062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 --2- 172.21.15.46:0/1187586379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc100cb5c0 0x7fbc1016c2d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 >> 172.21.15.46:0/1187586379 conn(0x7fbc100683f0 msgr2=0x7fbc100b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 shutdown_connections 2023-10-06T12:52:50.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:49.999+0000 7fbc158f0700 1 -- 172.21.15.46:0/1187586379 wait complete. 2023-10-06T12:52:50.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T12:52:50.159 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:50.155+0000 7facfd586000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T12:52:50.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.179+0000 7f7718ecd700 1 Processor -- start 2023-10-06T12:52:50.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.179+0000 7f7718ecd700 1 -- start start 2023-10-06T12:52:50.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7714069410 0x7f771415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 0x7f771415a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7714158d10 0x7f7714159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.187 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7714069c50 con 0x7f7714158d10 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771415a890 con 0x7f77140697e0 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140ce470 con 0x7f7714069410 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7714158d10 0x7f7714159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7714158d10 0x7f7714159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34288/0 (socket says 172.21.15.46:34288) 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 -- 172.21.15.46:0/1280148311 learned_addr learned my addr 172.21.15.46:0/1280148311 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7711d9b700 1 --2- 172.21.15.46:0/1280148311 >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 0x7f771415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f771259c700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7714069410 0x7f771415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 -- 172.21.15.46:0/1280148311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7714069410 msgr2=0x7f771415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7714069410 0x7f771415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 -- 172.21.15.46:0/1280148311 >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 msgr2=0x7f771415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 --2- 172.21.15.46:0/1280148311 >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 0x7f771415a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.188 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 -- 172.21.15.46:0/1280148311 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771415d470 con 0x7f7714158d10 2023-10-06T12:52:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7711d9b700 1 --2- 172.21.15.46:0/1280148311 >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 0x7f771415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7712d9d700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f7714159170 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7714088200 tx=0x7f770c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab91912b159cbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f771159a700 1 -- 172.21.15.46:0/1280148311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c00e070 con 0x7f7714158d10 2023-10-06T12:52:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f771159a700 1 -- 172.21.15.46:0/1280148311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c00ba40 con 0x7f7714158d10 2023-10-06T12:52:50.189 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f771159a700 1 -- 172.21.15.46:0/1280148311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0092e0 con 0x7f7714158d10 2023-10-06T12:52:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 -- 172.21.15.46:0/1280148311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 msgr2=0x7f7714159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.183+0000 7f7718ecd700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f7714159170 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7714088200 tx=0x7f770c009840 comp rx=0 tx=0).stop 2023-10-06T12:52:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 -- 172.21.15.46:0/1280148311 shutdown_connections 2023-10-06T12:52:50.190 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 --2- 172.21.15.46:0/1280148311 >> v2:172.21.15.132:3300/0 conn(0x7f77140697e0 0x7f771415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f7714159170 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 --2- 172.21.15.46:0/1280148311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7714069410 0x7f771415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 -- 172.21.15.46:0/1280148311 >> 172.21.15.46:0/1280148311 conn(0x7f77140d0210 msgr2=0x7f77140d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 -- 172.21.15.46:0/1280148311 shutdown_connections 2023-10-06T12:52:50.191 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 -- 172.21.15.46:0/1280148311 wait complete. 2023-10-06T12:52:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 Processor -- start 2023-10-06T12:52:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.187+0000 7f7718ecd700 1 -- start start 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 0x7f77140c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 0x7f7714155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7714154b20 con 0x7f7714158d10 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77141561a0 con 0x7f7714069410 2023-10-06T12:52:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140c7ab0 con 0x7f77140697e0 2023-10-06T12:52:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34290/0 (socket says 172.21.15.46:34290) 2023-10-06T12:52:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 -- 172.21.15.46:0/3176927081 learned_addr learned my addr 172.21.15.46:0/3176927081 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7711d9b700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 0x7f7714155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f771259c700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 0x7f77140c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 -- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 msgr2=0x7f7714155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 0x7f7714155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 -- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 msgr2=0x7f77140c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 0x7f77140c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 -- 172.21.15.46:0/3176927081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c009ae0 con 0x7f7714158d10 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f771259c700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 0x7f77140c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7711d9b700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 0x7f7714155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7712d9d700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f770c009880 tx=0x7f770c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0032f0 con 0x7f7714158d10 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c0094e0 con 0x7f7714158d10 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c025690 con 0x7f7714158d10 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77140c7d90 con 0x7f7714158d10 2023-10-06T12:52:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.191+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7714154740 con 0x7f7714158d10 2023-10-06T12:52:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7714154c60 con 0x7f7714158d10 2023-10-06T12:52:50.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f770c003040 con 0x7f7714158d10 2023-10-06T12:52:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f77027fc700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 0x7f76f4062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f770c07eda0 con 0x7f7714158d10 2023-10-06T12:52:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f771259c700 1 -- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 msgr2=0x7f76f4062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.195+0000 7f771259c700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 0x7f76f4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:50.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.203+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7714154c60 con 0x7f7714158d10 2023-10-06T12:52:50.265 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.262+0000 7f99535c9000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.375+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7714154c60 con 0x7f7714158d10 2023-10-06T12:52:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.375+0000 7f77027fc700 1 -- 172.21.15.46:0/3176927081 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7714154c60 con 0x7f7714158d10 2023-10-06T12:52:50.382 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 msgr2=0x7f76f4062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 0x7f76f4062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 msgr2=0x7f77140b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f770c009880 tx=0x7f770c018000 comp rx=0 tx=0).stop 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 shutdown_connections 2023-10-06T12:52:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7714158d10 0x7f77140b5160 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f77140697e0 0x7f7714155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f76f4060700 0x7f76f4062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 --2- 172.21.15.46:0/3176927081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7714069410 0x7f77140c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 >> 172.21.15.46:0/3176927081 conn(0x7f77140d0210 msgr2=0x7f771415cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 shutdown_connections 2023-10-06T12:52:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.379+0000 7f7718ecd700 1 -- 172.21.15.46:0/3176927081 wait complete. 2023-10-06T12:52:50.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:50.476 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.474+0000 7f99535c9000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:50.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:50.547+0000 7facfd586000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T12:52:50.563 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.558+0000 7f99535c9000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.563+0000 7f7986945700 1 Processor -- start 2023-10-06T12:52:50.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.563+0000 7f7986945700 1 -- start start 2023-10-06T12:52:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79800b39b0 0x7f79800b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800b4f80 con 0x7f79800b39b0 2023-10-06T12:52:50.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800b4370 con 0x7f7980159050 2023-10-06T12:52:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7986945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800b4650 con 0x7f79800d0aa0 2023-10-06T12:52:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f797ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7984ee2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79800b39b0 0x7f79800b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f797f7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7984ee2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f79800b39b0 0x7f79800b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34300/0 (socket says 172.21.15.46:34300) 2023-10-06T12:52:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f797ffff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59062/0 (socket says 172.21.15.46:59062) 2023-10-06T12:52:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f7984ee2700 1 -- 172.21.15.46:0/2907819805 learned_addr learned my addr 172.21.15.46:0/2907819805 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.567+0000 7f797f7fe700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:43374/0 (socket says 172.21.15.46:43374) 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 -- 172.21.15.46:0/2907819805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 msgr2=0x7f79800b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 -- 172.21.15.46:0/2907819805 >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 msgr2=0x7f7980159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 --2- 172.21.15.46:0/2907819805 >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 -- 172.21.15.46:0/2907819805 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800b4930 con 0x7f79800b39b0 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f797ffff700 1 --2- 172.21.15.46:0/2907819805 >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f797f7fe700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7984ee2700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800b39b0 0x7f79800b1fc0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f79800cdc50 tx=0x7f7978009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7226feeef61c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f797effd700 1 -- 172.21.15.46:0/2907819805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800e070 con 0x7f79800b39b0 2023-10-06T12:52:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f797effd700 1 -- 172.21.15.46:0/2907819805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797800b970 con 0x7f79800b39b0 2023-10-06T12:52:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f797effd700 1 -- 172.21.15.46:0/2907819805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780092e0 con 0x7f79800b39b0 2023-10-06T12:52:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 -- 172.21.15.46:0/2907819805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800b39b0 msgr2=0x7f79800b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800b39b0 0x7f79800b1fc0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f79800cdc50 tx=0x7f7978009840 comp rx=0 tx=0).stop 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 -- 172.21.15.46:0/2907819805 shutdown_connections 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800b39b0 0x7f79800b1fc0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 --2- 172.21.15.46:0/2907819805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800d0aa0 0x7f79800b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 --2- 172.21.15.46:0/2907819805 >> v2:172.21.15.132:3300/0 conn(0x7f7980159050 0x7f7980159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 -- 172.21.15.46:0/2907819805 >> 172.21.15.46:0/2907819805 conn(0x7f798006e780 msgr2=0x7f798006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 -- 172.21.15.46:0/2907819805 shutdown_connections 2023-10-06T12:52:50.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.571+0000 7f7986945700 1 -- 172.21.15.46:0/2907819805 wait complete. 2023-10-06T12:52:50.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 Processor -- start 2023-10-06T12:52:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 -- start start 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800b39b0 0x7f79801572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7980159050 0x7f79800c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800c4030 con 0x7f79800d0aa0 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980157830 con 0x7f7980159050 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7986945700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980159e00 con 0x7f79800b39b0 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34310/0 (socket says 172.21.15.46:34310) 2023-10-06T12:52:50.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 -- 172.21.15.46:0/709668504 learned_addr learned my addr 172.21.15.46:0/709668504 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f7984ee2700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7980159050 0x7f79800c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797ffff700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800b39b0 0x7f79801572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800b39b0 msgr2=0x7f79801572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800b39b0 0x7f79801572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7980159050 msgr2=0x7f79800c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7980159050 0x7f79800c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.575+0000 7f797f7fe700 1 -- 172.21.15.46:0/709668504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974009420 con 0x7f79800d0aa0 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f797f7fe700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f797400ec80 tx=0x7f79740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400c920 con 0x7f79800d0aa0 2023-10-06T12:52:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400ca80 con 0x7f79800d0aa0 2023-10-06T12:52:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7970009420 con 0x7f79800d0aa0 2023-10-06T12:52:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400ce00 con 0x7f79800d0aa0 2023-10-06T12:52:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7978009ae0 con 0x7f79800d0aa0 2023-10-06T12:52:50.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.579+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980088820 con 0x7f79800d0aa0 2023-10-06T12:52:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.583+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f7974013680 con 0x7f79800d0aa0 2023-10-06T12:52:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.583+0000 7f7967fff700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 0x7f796c062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.583+0000 7f797ffff700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 msgr2=0x7f796c062c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.583+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f797407d310 con 0x7f79800d0aa0 2023-10-06T12:52:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.583+0000 7f797ffff700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 0x7f796c062c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.591+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f797404d170 con 0x7f79800d0aa0 2023-10-06T12:52:50.636 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.634+0000 7f99535c9000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:50.715 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.710+0000 7f99535c9000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:50.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.787+0000 7f797ffff700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 msgr2=0x7f796c062c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:50.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.787+0000 7f797ffff700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 0x7f796c062c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:50.792 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:50.787+0000 7facfd586000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T12:52:50.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.795+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79800b4c10 con 0x7f79800d0aa0 2023-10-06T12:52:50.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.795+0000 7f7967fff700 1 -- 172.21.15.46:0/709668504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7f7974050790 con 0x7f79800d0aa0 2023-10-06T12:52:50.802 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:50.802 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 msgr2=0x7f796c062c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 0x7f796c062c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 msgr2=0x7f798015aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f797400ec80 tx=0x7f79740091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 shutdown_connections 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f796c0607d0 0x7f796c062c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7980159050 0x7f79800c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0aa0 0x7f798015aea0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 --2- 172.21.15.46:0/709668504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79800b39b0 0x7f79801572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 >> 172.21.15.46:0/709668504 conn(0x7f798006e780 msgr2=0x7f79800b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 shutdown_connections 2023-10-06T12:52:50.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.799+0000 7f7986945700 1 -- 172.21.15.46:0/709668504 wait complete. 2023-10-06T12:52:50.807 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:50.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:50.889 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:50.883+0000 7facfd586000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T12:52:50.925 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.922+0000 7f99535c9000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:50.974 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:50.967+0000 7facfd586000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T12:52:50.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.971+0000 7f8251d23700 1 Processor -- start 2023-10-06T12:52:50.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.971+0000 7f8251d23700 1 -- start start 2023-10-06T12:52:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251d23700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 0x7f824c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251d23700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f824c0b1130 0x7f824c0ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251d23700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824c0ae340 con 0x7f824c159290 2023-10-06T12:52:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251d23700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824c0b15b0 con 0x7f824c0b1130 2023-10-06T12:52:50.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251d23700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824c0b09a0 con 0x7f824c0d07b0 2023-10-06T12:52:50.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f824c0b1130 0x7f824c0ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f824c0b1130 0x7f824c0ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34316/0 (socket says 172.21.15.46:34316) 2023-10-06T12:52:50.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 -- 172.21.15.46:0/603052493 learned_addr learned my addr 172.21.15.46:0/603052493 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f824bfff700 1 --2- 172.21.15.46:0/603052493 >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 0x7f824c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 -- 172.21.15.46:0/603052493 >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 msgr2=0x7f824c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 --2- 172.21.15.46:0/603052493 >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 0x7f824c0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 -- 172.21.15.46:0/603052493 >> v1:172.21.15.46:6789/0 conn(0x7f824c159290 legacy=0x7f824c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 -- 172.21.15.46:0/603052493 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c159660 con 0x7f824c0b1130 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f824bfff700 1 --2- 172.21.15.46:0/603052493 >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 0x7f824c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f8251522700 1 --2- 172.21.15.46:0/603052493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c0b1130 0x7f824c0ae7d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f824c0b60f0 tx=0x7f8244009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49655834136f6cea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f824b7fe700 1 -- 172.21.15.46:0/603052493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824400e070 con 0x7f824c0b1130 2023-10-06T12:52:50.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f824b7fe700 1 -- 172.21.15.46:0/603052493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824400bae0 con 0x7f824c0b1130 2023-10-06T12:52:50.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.975+0000 7f824b7fe700 1 -- 172.21.15.46:0/603052493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82440092e0 con 0x7f824c0b1130 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 -- 172.21.15.46:0/603052493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c0b1130 msgr2=0x7f824c0ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 --2- 172.21.15.46:0/603052493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c0b1130 0x7f824c0ae7d0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f824c0b60f0 tx=0x7f8244009840 comp rx=0 tx=0).stop 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 -- 172.21.15.46:0/603052493 shutdown_connections 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 --2- 172.21.15.46:0/603052493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c0b1130 0x7f824c0ae7d0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 --2- 172.21.15.46:0/603052493 >> v2:172.21.15.132:3300/0 conn(0x7f824c0d07b0 0x7f824c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 -- 172.21.15.46:0/603052493 >> 172.21.15.46:0/603052493 conn(0x7f824c06ef60 msgr2=0x7f824c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 -- 172.21.15.46:0/603052493 shutdown_connections 2023-10-06T12:52:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.979+0000 7f8251d23700 1 -- 172.21.15.46:0/603052493 wait complete. 2023-10-06T12:52:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 Processor -- start 2023-10-06T12:52:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 -- start start 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824c0b1130 0x7f824c146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 0x7f824c09c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c09ccd0 con 0x7f824c159290 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c09cfb0 con 0x7f824c0d07b0 2023-10-06T12:52:50.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251d23700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824c09d290 con 0x7f824c0b1130 2023-10-06T12:52:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f824bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 0x7f824c09c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f824bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 0x7f824c09c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59090/0 (socket says 172.21.15.46:59090) 2023-10-06T12:52:50.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f824bfff700 1 -- 172.21.15.46:0/1589462369 learned_addr learned my addr 172.21.15.46:0/1589462369 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34324/0 (socket says 172.21.15.46:34324) 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8250d21700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824c0b1130 0x7f824c146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 -- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824c0b1130 msgr2=0x7f824c146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824c0b1130 0x7f824c146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 -- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 msgr2=0x7f824c09c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 0x7f824c09c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:50.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 -- 172.21.15.46:0/1589462369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244009ae0 con 0x7f824c159290 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.983+0000 7f8251522700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f8244015000 tx=0x7f8244018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82440092e0 con 0x7f824c159290 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244008cc0 con 0x7f824c159290 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824c09d570 con 0x7f824c159290 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824401d650 con 0x7f824c159290 2023-10-06T12:52:50.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824c146fd0 con 0x7f824c159290 2023-10-06T12:52:50.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824c09da70 con 0x7f824c159290 2023-10-06T12:52:50.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.987+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f8244003040 con 0x7f824c159290 2023-10-06T12:52:50.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.991+0000 7f8248ff9700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 0x7f8230062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:50.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.991+0000 7f8250d21700 1 -- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 msgr2=0x7f8230062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:50.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.991+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f824407ecd0 con 0x7f824c159290 2023-10-06T12:52:50.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.991+0000 7f8250d21700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 0x7f8230062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:51.001 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:50.998+0000 7f99535c9000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:51.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:50.999+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f824404e9a0 con 0x7f824c159290 2023-10-06T12:52:51.068 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.063+0000 7facfd586000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T12:52:51.086 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:51.082+0000 7f99535c9000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:51.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.167+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f824c064590 con 0x7f824c159290 2023-10-06T12:52:51.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.167+0000 7f8248ff9700 1 -- 172.21.15.46:0/1589462369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+18247 (secure 0 0 0) 0x7f8244051fc0 con 0x7f824c159290 2023-10-06T12:52:51.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:51.174 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":198,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:44.739083+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1339853979":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1151621779":"2023-10-07T12:52:44.738959+0000","172.21.15.46:6833/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/938799999":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/580990329":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/3242963477":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1367734289":"2023-10-07T12:52:44.738959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 msgr2=0x7f8230062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 0x7f8230062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 msgr2=0x7f824c0acbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f8244015000 tx=0x7f8244018000 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 shutdown_connections 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f824c159290 0x7f824c0acbd0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8230060700 0x7f8230062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f824c0d07b0 0x7f824c09c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.171+0000 7f8251d23700 1 --2- 172.21.15.46:0/1589462369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824c0b1130 0x7f824c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.175+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 >> 172.21.15.46:0/1589462369 conn(0x7f824c06ef60 msgr2=0x7f824c154c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.175+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 shutdown_connections 2023-10-06T12:52:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.175+0000 7f8251d23700 1 -- 172.21.15.46:0/1589462369 wait complete. 2023-10-06T12:52:51.200 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:52:51.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:51.271 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:51.266+0000 7f99535c9000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:51.306 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.303+0000 7facfd586000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T12:52:51.355 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:51.350+0000 7f99535c9000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.359+0000 7fc309d85700 1 Processor -- start 2023-10-06T12:52:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.359+0000 7fc309d85700 1 -- start start 2023-10-06T12:52:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.359+0000 7fc309d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc304159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.359+0000 7fc309d85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040b4f80 con 0x7fc3040b5410 2023-10-06T12:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc309d85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040b40c0 con 0x7fc304159050 2023-10-06T12:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc309d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040b4370 con 0x7fc3040d0aa0 2023-10-06T12:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3037fe700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc304159050 0x7fc3040b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52580/0 (socket says 172.21.15.46:52580) 2023-10-06T12:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3037fe700 1 -- 172.21.15.46:0/1365442120 learned_addr learned my addr 172.21.15.46:0/1365442120 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc302ffd700 1 --2- 172.21.15.46:0/1365442120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc304159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888077079 0 0) 0x7fc3040b40c0 con 0x7fc304159050 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ec003570 con 0x7fc304159050 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc302ffd700 1 -- 172.21.15.46:0/1365442120 >> v1:172.21.15.132:6789/0 conn(0x7fc304159050 legacy=0x7fc3040b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc302ffd700 1 -- 172.21.15.46:0/1365442120 >> v1:172.21.15.46:6789/0 conn(0x7fc3040b5410 legacy=0x7fc3040b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284326122 0 0) 0x7fc3040b4f80 con 0x7fc3040b5410 2023-10-06T12:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc302ffd700 1 -- 172.21.15.46:0/1365442120 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3040b4650 con 0x7fc3040d0aa0 2023-10-06T12:52:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc302ffd700 1 --2- 172.21.15.46:0/1365442120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc304159420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009910 tx=0x7fc2f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a90e309b21946c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800b680 con 0x7fc3040d0aa0 2023-10-06T12:52:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f800b7e0 con 0x7fc3040d0aa0 2023-10-06T12:52:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc3027fc700 1 -- 172.21.15.46:0/1365442120 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800bba0 con 0x7fc3040d0aa0 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc309d85700 1 -- 172.21.15.46:0/1365442120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 msgr2=0x7fc304159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.363+0000 7fc309d85700 1 --2- 172.21.15.46:0/1365442120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc304159420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009910 tx=0x7fc2f80091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- 172.21.15.46:0/1365442120 shutdown_connections 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 --2- 172.21.15.46:0/1365442120 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc304159420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- 172.21.15.46:0/1365442120 >> 172.21.15.46:0/1365442120 conn(0x7fc30406e780 msgr2=0x7fc30406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- 172.21.15.46:0/1365442120 shutdown_connections 2023-10-06T12:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- 172.21.15.46:0/1365442120 wait complete. 2023-10-06T12:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 Processor -- start 2023-10-06T12:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- start start 2023-10-06T12:52:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc3040c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc304159050 0x7fc3040c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415a120 con 0x7fc3040b5410 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040ca210 con 0x7fc304159050 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc309d85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040c58f0 con 0x7fc3040d0aa0 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34326/0 (socket says 172.21.15.46:34326) 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.367+0000 7fc3037fe700 1 -- 172.21.15.46:0/3719510331 learned_addr learned my addr 172.21.15.46:0/3719510331 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc302ffd700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc3040c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc303fff700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc304159050 0x7fc3040c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 msgr2=0x7fc3040c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc3040c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc304159050 msgr2=0x7fc3040c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc304159050 0x7fc3040c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 -- 172.21.15.46:0/3719510331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009420 con 0x7fc3040b5410 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc3037fe700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400c5c0 tx=0x7fc2f4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4005b30 con 0x7fc3040b5410 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4005cb0 con 0x7fc3040b5410 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4007320 con 0x7fc3040b5410 2023-10-06T12:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3040c9f30 con 0x7fc3040b5410 2023-10-06T12:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304159db0 con 0x7fc3040b5410 2023-10-06T12:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.371+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304088820 con 0x7fc3040b5410 2023-10-06T12:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.375+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fc2f401b3b0 con 0x7fc3040b5410 2023-10-06T12:52:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.375+0000 7fc2eb7fe700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 0x7fc2f0062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.375+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7fc2f407c5b0 con 0x7fc3040b5410 2023-10-06T12:52:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.375+0000 7fc302ffd700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 msgr2=0x7fc2f0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.375+0000 7fc302ffd700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 0x7fc2f0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:51.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.383+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc2f404c430 con 0x7fc3040b5410 2023-10-06T12:52:51.391 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.387+0000 7facfd586000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T12:52:51.485 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.479+0000 7facfd586000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T12:52:51.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.579+0000 7fc302ffd700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 msgr2=0x7fc2f0062bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:51.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.579+0000 7fc302ffd700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 0x7fc2f0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.587+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc30404b5c0 con 0x7fc3040b5410 2023-10-06T12:52:51.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.587+0000 7fc2eb7fe700 1 -- 172.21.15.46:0/3719510331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7fc2f4017090 con 0x7fc3040b5410 2023-10-06T12:52:51.593 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:51.594 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:51.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 msgr2=0x7fc2f0062bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:51.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 0x7fc2f0062bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 msgr2=0x7fc3040c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400c5c0 tx=0x7fc2f4007e00 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 shutdown_connections 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fc2f0060700 0x7fc2f0062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc304159050 0x7fc3040c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3040d0aa0 0x7fc3040c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 --2- 172.21.15.46:0/3719510331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3040b5410 0x7fc3040c53b0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 >> 172.21.15.46:0/3719510331 conn(0x7fc30406e780 msgr2=0x7fc304158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 shutdown_connections 2023-10-06T12:52:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.591+0000 7fc309d85700 1 -- 172.21.15.46:0/3719510331 wait complete. 2023-10-06T12:52:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:51.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:52:51.693 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.687+0000 7facfd586000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T12:52:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 Processor -- start 2023-10-06T12:52:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 -- start start 2023-10-06T12:52:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d68156c80 0x7f5d680ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d6815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680c6470 con 0x7f5d680c7b50 2023-10-06T12:52:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680c7760 con 0x7f5d68156c80 2023-10-06T12:52:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6ccf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680ca710 con 0x7f5d680c7fc0 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d68156c80 0x7f5d680ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d66d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d6815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d68156c80 0x7f5d680ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59110/0 (socket says 172.21.15.46:59110) 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d66d9d700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d6815d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:43434/0 (socket says 172.21.15.46:43434) 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 -- 172.21.15.46:0/176063776 learned_addr learned my addr 172.21.15.46:0/176063776 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d5ffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f5d680c7b50 0x7f5d6815cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52582/0 (socket says 172.21.15.46:52582) 2023-10-06T12:52:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 -- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 msgr2=0x7f5d6815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 --2- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d6815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 -- 172.21.15.46:0/176063776 >> v1:172.21.15.132:6789/0 conn(0x7f5d680c7b50 legacy=0x7f5d6815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.775+0000 7f5d6659c700 1 -- 172.21.15.46:0/176063776 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680c66c0 con 0x7f5d68156c80 2023-10-06T12:52:51.783 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:51.779+0000 7facfd586000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T12:52:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6659c700 1 --2- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d68156c80 0x7f5d680ca1d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009990 tx=0x7f5d58009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86b8d8db0869160a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d65d9b700 1 -- 172.21.15.46:0/176063776 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58009e80 con 0x7f5d68156c80 2023-10-06T12:52:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d65d9b700 1 -- 172.21.15.46:0/176063776 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58005c10 con 0x7f5d68156c80 2023-10-06T12:52:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d65d9b700 1 -- 172.21.15.46:0/176063776 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5800b650 con 0x7f5d68156c80 2023-10-06T12:52:51.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d68156c80 msgr2=0x7f5d680ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d68156c80 0x7f5d680ca1d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009990 tx=0x7f5d58009220 comp rx=0 tx=0).stop 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/176063776 shutdown_connections 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d6815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/176063776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d68156c80 0x7f5d680ca1d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/176063776 >> 172.21.15.46:0/176063776 conn(0x7f5d680cbe30 msgr2=0x7f5d680cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/176063776 shutdown_connections 2023-10-06T12:52:51.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.779+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/176063776 wait complete. 2023-10-06T12:52:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 Processor -- start 2023-10-06T12:52:51.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 -- start start 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 0x7f5d68151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d680c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680af980 con 0x7f5d68156c80 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68151f40 con 0x7f5d680c7b50 2023-10-06T12:52:51.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6ccf0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680aed70 con 0x7f5d680c7fc0 2023-10-06T12:52:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34340/0 (socket says 172.21.15.46:34340) 2023-10-06T12:52:51.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d66d9d700 1 -- 172.21.15.46:0/754349931 learned_addr learned my addr 172.21.15.46:0/754349931 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d5ffff700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d680c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.783+0000 7f5d6659c700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 0x7f5d68151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 msgr2=0x7f5d680c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d680c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 msgr2=0x7f5d68151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 0x7f5d68151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 -- 172.21.15.46:0/754349931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58009480 con 0x7f5d68156c80 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d6659c700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 0x7f5d68151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d66d9d700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009e90 tx=0x7f5d6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:51.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000cc20 con 0x7f5d68156c80 2023-10-06T12:52:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6000f040 con 0x7f5d68156c80 2023-10-06T12:52:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60014480 con 0x7f5d68156c80 2023-10-06T12:52:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680af050 con 0x7f5d68156c80 2023-10-06T12:52:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d680af610 con 0x7f5d68156c80 2023-10-06T12:52:51.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.787+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d680afb30 con 0x7f5d68156c80 2023-10-06T12:52:51.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.791+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7f5d60014830 con 0x7f5d68156c80 2023-10-06T12:52:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.791+0000 7f5d5e7fc700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 0x7f5d48062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:51.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.791+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7f5d6007ddc0 con 0x7f5d68156c80 2023-10-06T12:52:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.791+0000 7f5d6659c700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 msgr2=0x7f5d48062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:51.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.791+0000 7f5d6659c700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 0x7f5d48062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.795+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d680afb30 con 0x7f5d68156c80 2023-10-06T12:52:51.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.995+0000 7f5d6659c700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 msgr2=0x7f5d48062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:51.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:51.995+0000 7f5d6659c700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 0x7f5d48062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T12:52:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.007+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d68063f90 con 0x7f5d68156c80 2023-10-06T12:52:52.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.007+0000 7f5d5e7fc700 1 -- 172.21.15.46:0/754349931 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4614 (secure 0 0 0) 0x7f5d60018090 con 0x7f5d68156c80 2023-10-06T12:52:52.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:52.012 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T12:52:27.542914+0000","modified":"2023-10-06T12:52:31.016227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10359":{"gid":10359,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2405550575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2405550575},{"type":"v1","addr":"172.21.15.46:6835","nonce":2405550575}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 msgr2=0x7f5d48062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 0x7f5d48062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 msgr2=0x7f5d680afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009e90 tx=0x7f5d6000b040 comp rx=0 tx=0).stop 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 shutdown_connections 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d68156c80 0x7f5d680afe10 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d680c7fc0 0x7f5d680c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5d48060730 0x7f5d48062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 --2- 172.21.15.46:0/754349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d680c7b50 0x7f5d68151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 >> 172.21.15.46:0/754349931 conn(0x7f5d680cbe30 msgr2=0x7f5d68068c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 shutdown_connections 2023-10-06T12:52:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.011+0000 7f5d6ccf0700 1 -- 172.21.15.46:0/754349931 wait complete. 2023-10-06T12:52:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 103 2023-10-06T12:52:52.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:52.055 INFO:tasks.ceph.mgr.x.smithi132.stderr:2023-10-06T12:52:52.050+0000 7f99535c9000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.195+0000 7fed46a2e700 1 Processor -- start 2023-10-06T12:52:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.195+0000 7fed46a2e700 1 -- start start 2023-10-06T12:52:52.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed46a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed46a2e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed400b5410 0x7fed400b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed46a2e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed400b4f80 con 0x7fed400d0aa0 2023-10-06T12:52:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed46a2e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed400b40c0 con 0x7fed400b5410 2023-10-06T12:52:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed46a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed400b4370 con 0x7fed40159050 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed37fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed400d0aa0 0x7fed400b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35608/0 (socket says 172.21.15.46:35608) 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed37fff700 1 -- 172.21.15.46:0/2713119535 learned_addr learned my addr 172.21.15.46:0/2713119535 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed44fcb700 1 --2- 172.21.15.46:0/2713119535 >> v2:172.21.15.46:3300/0 conn(0x7fed400b5410 0x7fed400b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed3ffff700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255292820 0 0) 0x7fed400b4f80 con 0x7fed400d0aa0 2023-10-06T12:52:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed20003570 con 0x7fed400d0aa0 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4288664077 0 0) 0x7fed20003570 con 0x7fed400d0aa0 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed44fcb700 1 -- 172.21.15.46:0/2713119535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 msgr2=0x7fed40159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed44fcb700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed44fcb700 1 -- 172.21.15.46:0/2713119535 >> v1:172.21.15.46:6789/0 conn(0x7fed400d0aa0 legacy=0x7fed400b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.199+0000 7fed44fcb700 1 -- 172.21.15.46:0/2713119535 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400b4650 con 0x7fed400b5410 2023-10-06T12:52:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed44fcb700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed400b1fc0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fed38009e60 tx=0x7fed38009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5de7442a9b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800e070 con 0x7fed400b5410 2023-10-06T12:52:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3800b7a0 con 0x7fed400b5410 2023-10-06T12:52:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed3f7fe700 1 -- 172.21.15.46:0/2713119535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28001470 con 0x7fed400b5410 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 -- 172.21.15.46:0/2713119535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 msgr2=0x7fed400b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed400b1fc0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fed38009e60 tx=0x7fed38009780 comp rx=0 tx=0).stop 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 -- 172.21.15.46:0/2713119535 shutdown_connections 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed400b1fc0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2713119535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 -- 172.21.15.46:0/2713119535 >> 172.21.15.46:0/2713119535 conn(0x7fed4006e780 msgr2=0x7fed4006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 -- 172.21.15.46:0/2713119535 shutdown_connections 2023-10-06T12:52:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.203+0000 7fed46a2e700 1 -- 172.21.15.46:0/2713119535 wait complete. 2023-10-06T12:52:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 Processor -- start 2023-10-06T12:52:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 -- start start 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 0x7fed400c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed400c4350 con 0x7fed400b5410 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed400ca090 con 0x7fed400d0aa0 2023-10-06T12:52:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed46a2e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed401570a0 con 0x7fed40159050 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34372/0 (socket says 172.21.15.46:34372) 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 -- 172.21.15.46:0/2379576109 learned_addr learned my addr 172.21.15.46:0/2379576109 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed44fcb700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed37fff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 0x7fed400c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 -- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 msgr2=0x7fed40157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 -- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 msgr2=0x7fed400c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 0x7fed400c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 -- 172.21.15.46:0/2379576109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38009a20 con 0x7fed400b5410 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed37fff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 0x7fed400c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed44fcb700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.207+0000 7fed3ffff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e960 tx=0x7fed300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3000c850 con 0x7fed400b5410 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3000c9b0 con 0x7fed400b5410 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed400c3dd0 con 0x7fed400b5410 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3000cd50 con 0x7fed400b5410 2023-10-06T12:52:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed40147b00 con 0x7fed400b5410 2023-10-06T12:52:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed401574b0 con 0x7fed400b5410 2023-10-06T12:52:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.211+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 116) v1 ==== 80118+0+0 (secure 0 0 0) 0x7fed30013680 con 0x7fed400b5410 2023-10-06T12:52:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.215+0000 7fed3cff9700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 0x7fed24062be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.215+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6974+0+0 (secure 0 0 0) 0x7fed3007d1d0 con 0x7fed400b5410 2023-10-06T12:52:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.215+0000 7fed37fff700 1 -- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 msgr2=0x7fed24062be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6832/59347 2023-10-06T12:52:52.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.215+0000 7fed37fff700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 0x7fed24062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T12:52:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.223+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fed3004cfb0 con 0x7fed400b5410 2023-10-06T12:52:52.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.399+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed4004b5c0 con 0x7fed400b5410 2023-10-06T12:52:52.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.403+0000 7fed3cff9700 1 -- 172.21.15.46:0/2379576109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+18247 (secure 0 0 0) 0x7fed300505d0 con 0x7fed400b5410 2023-10-06T12:52:52.408 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:52:52.408 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":198,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:44.739083+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T12:52:25.312557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T12:52:26.484497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6832/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1339853979":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1151621779":"2023-10-07T12:52:44.738959+0000","172.21.15.46:6833/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/938799999":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/580990329":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/3242963477":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1367734289":"2023-10-07T12:52:44.738959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 msgr2=0x7fed24062be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 0x7fed24062be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 msgr2=0x7fed401475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e960 tx=0x7fed300091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 shutdown_connections 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed40159050 0x7fed40157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fed24060730 0x7fed24062be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed400d0aa0 0x7fed400c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 --2- 172.21.15.46:0/2379576109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed400b5410 0x7fed401475c0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 >> 172.21.15.46:0/2379576109 conn(0x7fed4006e780 msgr2=0x7fed400cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 shutdown_connections 2023-10-06T12:52:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.407+0000 7fed46a2e700 1 -- 172.21.15.46:0/2379576109 wait complete. 2023-10-06T12:52:52.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T12:52:52.561 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.555+0000 7facfd586000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T12:52:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.583+0000 7f67af363700 1 Processor -- start 2023-10-06T12:52:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.583+0000 7f67af363700 1 -- start start 2023-10-06T12:52:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67af363700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 0x7f67a80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67af363700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 0x7f67a80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67af363700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a80ae340 con 0x7f67a8159290 2023-10-06T12:52:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67af363700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a80b15b0 con 0x7f67a80b1130 2023-10-06T12:52:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67af363700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80b09a0 con 0x7f67a80d07b0 2023-10-06T12:52:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67aeb62700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 0x7f67a80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67adb60700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 0x7f67a80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67aeb62700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 0x7f67a80ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:59162/0 (socket says 172.21.15.46:59162) 2023-10-06T12:52:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67ae361700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f67a8159290 0x7f67a80b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:52586/0 (socket says 172.21.15.46:52586) 2023-10-06T12:52:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67aeb62700 1 -- 172.21.15.46:0/1743545917 learned_addr learned my addr 172.21.15.46:0/1743545917 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67adb60700 1 -- 172.21.15.46:0/1743545917 >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 msgr2=0x7f67a80ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67adb60700 1 --2- 172.21.15.46:0/1743545917 >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 0x7f67a80ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67adb60700 1 -- 172.21.15.46:0/1743545917 >> v1:172.21.15.132:6789/0 conn(0x7f67a8159290 legacy=0x7f67a80b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.587+0000 7f67adb60700 1 -- 172.21.15.46:0/1743545917 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8159660 con 0x7f67a80d07b0 2023-10-06T12:52:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67adb60700 1 --2- 172.21.15.46:0/1743545917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 0x7f67a80b0bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67980098e0 tx=0x7f67980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b2c663e133f3d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67ad35f700 1 -- 172.21.15.46:0/1743545917 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800b6d0 con 0x7f67a80d07b0 2023-10-06T12:52:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67ad35f700 1 -- 172.21.15.46:0/1743545917 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679800b830 con 0x7f67a80d07b0 2023-10-06T12:52:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67ad35f700 1 -- 172.21.15.46:0/1743545917 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800bbd0 con 0x7f67a80d07b0 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 -- 172.21.15.46:0/1743545917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 msgr2=0x7f67a80b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 --2- 172.21.15.46:0/1743545917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 0x7f67a80b0bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67980098e0 tx=0x7f67980091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 -- 172.21.15.46:0/1743545917 shutdown_connections 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 --2- 172.21.15.46:0/1743545917 >> v2:172.21.15.132:3300/0 conn(0x7f67a80b1130 0x7f67a80ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 --2- 172.21.15.46:0/1743545917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80d07b0 0x7f67a80b0bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 -- 172.21.15.46:0/1743545917 >> 172.21.15.46:0/1743545917 conn(0x7f67a806ef60 msgr2=0x7f67a80d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 -- 172.21.15.46:0/1743545917 shutdown_connections 2023-10-06T12:52:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.591+0000 7f67af363700 1 -- 172.21.15.46:0/1743545917 wait complete. 2023-10-06T12:52:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 Processor -- start 2023-10-06T12:52:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 -- start start 2023-10-06T12:52:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 0x7f67a816a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8159290 0x7f67a8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8140300 con 0x7f67a80d07b0 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a81586c0 con 0x7f67a8159290 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67af363700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8158b50 con 0x7f67a80b1130 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67adb60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67ae361700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 0x7f67a816a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67adb60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34402/0 (socket says 172.21.15.46:34402) 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67adb60700 1 -- 172.21.15.46:0/187652292 learned_addr learned my addr 172.21.15.46:0/187652292 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.595+0000 7f67aeb62700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8159290 0x7f67a8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 -- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8159290 msgr2=0x7f67a8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8159290 0x7f67a8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 -- 172.21.15.46:0/187652292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 msgr2=0x7f67a8157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 -- 172.21.15.46:0/187652292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798009420 con 0x7f67a80b1130 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67adb60700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67ae361700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 0x7f67a816a1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cca0 tx=0x7f67a4001a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4005df0 con 0x7f67a80b1130 2023-10-06T12:52:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400a6c0 con 0x7f67a80b1130 2023-10-06T12:52:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400aa80 con 0x7f67a80b1130 2023-10-06T12:52:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a8157ab0 con 0x7f67a80b1130 2023-10-06T12:52:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a8158070 con 0x7f67a80b1130 2023-10-06T12:52:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.599+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a8064590 con 0x7f67a80b1130 2023-10-06T12:52:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.603+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 117) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f67a4035a30 con 0x7f67a80b1130 2023-10-06T12:52:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.603+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 7275+0+0 (secure 0 0 0) 0x7f67a407ca90 con 0x7f67a80b1130 2023-10-06T12:52:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.607+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f67a404cea0 con 0x7f67a80b1130 2023-10-06T12:52:52.672 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.667+0000 7faccafce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.672 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.667+0000 7faccafce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.672 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.667+0000 7faccafce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.673 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.667+0000 7faccafce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.673 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.667+0000 7faccafce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.674 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.671+0000 7facc67c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.674 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.671+0000 7facc67c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.674 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.671+0000 7facc67c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.671+0000 7facc67c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.675 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-10-06T12:52:52.671+0000 7facc67c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T12:52:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.839+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f67a80b06c0 con 0x7f67a80b1130 2023-10-06T12:52:52.847 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-10-06T12:52:52.843+0000 7f9462363700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T12:52:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.855+0000 7f679e7fc700 1 -- 172.21.15.46:0/187652292 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7f67a40504c0 con 0x7f67a80b1130 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 msgr2=0x7f67a816a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 0x7f67a816a1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67a400cca0 tx=0x7f67a4001a30 comp rx=0 tx=0).stop 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 shutdown_connections 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67a8159290 0x7f67a8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67a80d07b0 0x7f67a8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 --2- 172.21.15.46:0/187652292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67a80b1130 0x7f67a816a1a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 >> 172.21.15.46:0/187652292 conn(0x7f67a806ef60 msgr2=0x7f67a8140870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 shutdown_connections 2023-10-06T12:52:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:52.859+0000 7f67af363700 1 -- 172.21.15.46:0/187652292 wait complete. 2023-10-06T12:52:52.865 INFO:tasks.ceph.mds.a.smithi046.stderr: -2159> 2023-10-06T12:52:32.563+0000 7f6e3c95b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T12:52:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T12:52:52.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T12:52:52.907 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-10-06T12:52:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.031+0000 7f07b7545700 1 Processor -- start 2023-10-06T12:52:53.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 -- start start 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07b0146eb0 0x7f07b00c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00c2350 0x7f07b006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b0161fc0 0x7f07b006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00c1980 con 0x7f07b00c2350 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00c0d70 con 0x7f07b0146eb0 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b7545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00c1050 con 0x7f07b0161fc0 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00c2350 0x7f07b006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b00c2350 0x7f07b006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34534/0 (socket says 172.21.15.46:34534) 2023-10-06T12:52:53.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 -- 172.21.15.46:0/3247672210 learned_addr learned my addr 172.21.15.46:0/3247672210 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b52e1700 1 --2- 172.21.15.46:0/3247672210 >> v2:172.21.15.132:3300/0 conn(0x7f07b0146eb0 0x7f07b00c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b5ae2700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b0161fc0 0x7f07b006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 -- 172.21.15.46:0/3247672210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b0161fc0 msgr2=0x7f07b006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b0161fc0 0x7f07b006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 -- 172.21.15.46:0/3247672210 >> v2:172.21.15.132:3300/0 conn(0x7f07b0146eb0 msgr2=0x7f07b00c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 --2- 172.21.15.46:0/3247672210 >> v2:172.21.15.132:3300/0 conn(0x7f07b0146eb0 0x7f07b00c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.035+0000 7f07b4ae0700 1 -- 172.21.15.46:0/3247672210 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00c1330 con 0x7f07b00c2350 2023-10-06T12:52:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b4ae0700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00c2350 0x7f07b006cc30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f07b006bc20 tx=0x7f07a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da1f77a52768f24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07a7fff700 1 -- 172.21.15.46:0/3247672210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000b860 con 0x7f07b00c2350 2023-10-06T12:52:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07a7fff700 1 -- 172.21.15.46:0/3247672210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a000b9c0 con 0x7f07b00c2350 2023-10-06T12:52:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07a7fff700 1 -- 172.21.15.46:0/3247672210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000bd60 con 0x7f07b00c2350 2023-10-06T12:52:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 -- 172.21.15.46:0/3247672210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00c2350 msgr2=0x7f07b006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00c2350 0x7f07b006cc30 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f07b006bc20 tx=0x7f07a0009250 comp rx=0 tx=0).stop 2023-10-06T12:52:53.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 -- 172.21.15.46:0/3247672210 shutdown_connections 2023-10-06T12:52:53.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b0161fc0 0x7f07b006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 --2- 172.21.15.46:0/3247672210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b00c2350 0x7f07b006cc30 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 --2- 172.21.15.46:0/3247672210 >> v2:172.21.15.132:3300/0 conn(0x7f07b0146eb0 0x7f07b00c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 -- 172.21.15.46:0/3247672210 >> 172.21.15.46:0/3247672210 conn(0x7f07b00cbe30 msgr2=0x7f07b00cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:53.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 -- 172.21.15.46:0/3247672210 shutdown_connections 2023-10-06T12:52:53.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.039+0000 7f07b7545700 1 -- 172.21.15.46:0/3247672210 wait complete. 2023-10-06T12:52:53.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.043+0000 7f07b7545700 1 Processor -- start 2023-10-06T12:52:53.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.043+0000 7f07b7545700 1 -- start start 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b00c2350 0x7f07b016e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07b0146eb0 0x7f07b015ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00aebd0 con 0x7f07b0161fc0 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00af750 con 0x7f07b0146eb0 2023-10-06T12:52:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b01604b0 con 0x7f07b00c2350 2023-10-06T12:52:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34542/0 (socket says 172.21.15.46:34542) 2023-10-06T12:52:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 -- 172.21.15.46:0/2286753097 learned_addr learned my addr 172.21.15.46:0/2286753097 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b4ae0700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07b0146eb0 0x7f07b015ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b52e1700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b00c2350 0x7f07b016e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 -- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b00c2350 msgr2=0x7f07b016e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b00c2350 0x7f07b016e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 -- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07b0146eb0 msgr2=0x7f07b015ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07b0146eb0 0x7f07b015ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 -- 172.21.15.46:0/2286753097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00094f0 con 0x7f07b0161fc0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b5ae2700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009fa0 tx=0x7f07a8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800ebc0 con 0x7f07b0161fc0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a800ed20 con 0x7f07b0161fc0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b015ce10 con 0x7f07b0161fc0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8014670 con 0x7f07b0161fc0 2023-10-06T12:52:53.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.047+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b00af100 con 0x7f07b0161fc0 2023-10-06T12:52:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.051+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b00af8c0 con 0x7f07b0161fc0 2023-10-06T12:52:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.051+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 117) v1 ==== 79670+0+0 (secure 0 0 0) 0x7f07a80149e0 con 0x7f07b0161fc0 2023-10-06T12:52:53.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.055+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7361+0+0 (secure 0 0 0) 0x7f07a807cfe0 con 0x7f07b0161fc0 2023-10-06T12:52:53.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.063+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f07a80acbe0 con 0x7f07b0161fc0 2023-10-06T12:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.275+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07b00660f0 con 0x7f07b0161fc0 2023-10-06T12:52:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.587+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 118) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f07a804cd30 con 0x7f07b0161fc0 2023-10-06T12:52:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.587+0000 7f07a57fa700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 0x7f0794068f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:53.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.591+0000 7f07b52e1700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 0x7f0794068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.591+0000 7f07b52e1700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 0x7f0794068f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:53.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.875+0000 7f07a57fa700 1 -- 172.21.15.46:0/2286753097 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07a8068760 con 0x7f07b0161fc0 2023-10-06T12:52:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 msgr2=0x7f0794068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 0x7f0794068f10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2023-10-06T12:52:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 msgr2=0x7f07b015c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009fa0 tx=0x7f07a8009900 comp rx=0 tx=0).stop 2023-10-06T12:52:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 shutdown_connections 2023-10-06T12:52:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b0161fc0 0x7f07b015c8d0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0794066b30 0x7f0794068f10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07b0146eb0 0x7f07b015ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 --2- 172.21.15.46:0/2286753097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07b00c2350 0x7f07b016e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 >> 172.21.15.46:0/2286753097 conn(0x7f07b00cbe30 msgr2=0x7f07b0144080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 shutdown_connections 2023-10-06T12:52:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:53.879+0000 7f07b7545700 1 -- 172.21.15.46:0/2286753097 wait complete. 2023-10-06T12:52:53.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T12:52:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.123+0000 7f64e26ae700 1 Processor -- start 2023-10-06T12:52:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.123+0000 7f64e26ae700 1 -- start start 2023-10-06T12:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64e26ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f64dc0cf050 0x7f64dc0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64e26ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64dc0c5db0 con 0x7f64dc1596a0 2023-10-06T12:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64e26ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64dc0c5ab0 con 0x7f64dc1592d0 2023-10-06T12:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64e26ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0c6e90 con 0x7f64dc0cf050 2023-10-06T12:52:54.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64d3fff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64dc1596a0 0x7f64dc0c6240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43666/0 (socket says 172.21.15.46:43666) 2023-10-06T12:52:54.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64d3fff700 1 -- 172.21.15.46:0/1097048957 learned_addr learned my addr 172.21.15.46:0/1097048957 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:54.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64e0c4b700 1 --2- 172.21.15.46:0/1097048957 >> v2:172.21.15.132:3300/0 conn(0x7f64dc0cf050 0x7f64dc0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136695530 0 0) 0x7f64dc0c5db0 con 0x7f64dc1596a0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c8003570 con 0x7f64dc1596a0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023341583 0 0) 0x7f64dc0c5ab0 con 0x7f64dc1592d0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64dc0c5db0 con 0x7f64dc1592d0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4041537736 0 0) 0x7f64c8003570 con 0x7f64dc1596a0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 >> v2:172.21.15.132:3300/0 conn(0x7f64dc0cf050 msgr2=0x7f64dc0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 --2- 172.21.15.46:0/1097048957 >> v2:172.21.15.132:3300/0 conn(0x7f64dc0cf050 0x7f64dc0c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 >> v1:172.21.15.132:6789/0 conn(0x7f64dc1592d0 legacy=0x7f64dc0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0d0aa0 con 0x7f64dc1596a0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f64c40015f0 con 0x7f64dc1596a0 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 >> v1:172.21.15.46:6789/0 conn(0x7f64dc1596a0 legacy=0x7f64dc0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 0x7f64c8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8004810 0x7f64c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64c80051c0 0x7f64c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0c6e90 con 0x7f64c8003bf0 2023-10-06T12:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800c160 con 0x7f64c8004810 2023-10-06T12:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.127+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800c320 con 0x7f64c80051c0 2023-10-06T12:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 0x7f64c8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64d3fff700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8004810 0x7f64c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e0c4b700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64c80051c0 0x7f64c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 -- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64c80051c0 msgr2=0x7f64c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64c80051c0 0x7f64c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 -- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8004810 msgr2=0x7f64c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8004810 0x7f64c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 -- 172.21.15.46:0/1097048957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8003570 con 0x7f64c8003bf0 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e0c4b700 1 -- 172.21.15.46:0/1097048957 reap_dead start 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64dbfff700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 0x7f64c8003fe0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f64cc007fd0 tx=0x7f64cc007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0b1b1d16180686 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0013f0 con 0x7f64c8003bf0 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00c590 con 0x7f64c8003bf0 2023-10-06T12:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64db7fe700 1 -- 172.21.15.46:0/1097048957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00c950 con 0x7f64c8003bf0 2023-10-06T12:52:54.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 -- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 msgr2=0x7f64c8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 0x7f64c8003fe0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f64cc007fd0 tx=0x7f64cc007d80 comp rx=0 tx=0).stop 2023-10-06T12:52:54.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 -- 172.21.15.46:0/1097048957 shutdown_connections 2023-10-06T12:52:54.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 --2- 172.21.15.46:0/1097048957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64c8003bf0 0x7f64c8003fe0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 -- 172.21.15.46:0/1097048957 >> 172.21.15.46:0/1097048957 conn(0x7f64dc06ef60 msgr2=0x7f64dc0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 -- 172.21.15.46:0/1097048957 shutdown_connections 2023-10-06T12:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.131+0000 7f64e26ae700 1 -- 172.21.15.46:0/1097048957 wait complete. 2023-10-06T12:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.135+0000 7f64e26ae700 1 Processor -- start 2023-10-06T12:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.135+0000 7f64e26ae700 1 -- start start 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8003bf0 0x7f64dc155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc0c9bb0 0x7f64dc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc09c3c0 con 0x7f64dc09d780 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc1697f0 con 0x7f64c8003bf0 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc09dbd0 con 0x7f64dc0c9bb0 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55492/0 (socket says 172.21.15.46:55492) 2023-10-06T12:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 -- 172.21.15.46:0/180556615 learned_addr learned my addr 172.21.15.46:0/180556615 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64dbfff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8003bf0 0x7f64dc155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e0c4b700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc0c9bb0 0x7f64dc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 -- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc0c9bb0 msgr2=0x7f64dc0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc0c9bb0 0x7f64dc0ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 -- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8003bf0 msgr2=0x7f64dc155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8003bf0 0x7f64dc155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 -- 172.21.15.46:0/180556615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc09c070 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d3fff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f64c400f920 tx=0x7f64c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40072e0 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4007460 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc09b760 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40057f0 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.139+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc09ba20 con 0x7f64dc09d780 2023-10-06T12:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.143+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc09c200 con 0x7f64dc09d780 2023-10-06T12:52:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.143+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 118) v1 ==== 79797+0+0 (secure 0 0 0) 0x7f64c4005ba0 con 0x7f64dc09d780 2023-10-06T12:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.143+0000 7f64d8ff9700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 0x7f64bc062850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.143+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7361+0+0 (secure 0 0 0) 0x7f64c407cc20 con 0x7f64dc09d780 2023-10-06T12:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.143+0000 7f64dbfff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 0x7f64bc062850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:54.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.147+0000 7f64dbfff700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 0x7f64bc062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc015000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:54.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.151+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64c404cd70 con 0x7f64dc09d780 2023-10-06T12:52:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.323+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f64dc069e00 con 0x7f64dc09d780 2023-10-06T12:52:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.887+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f64c404d140 con 0x7f64dc09d780 2023-10-06T12:52:54.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.895+0000 7f64d8ff9700 1 -- 172.21.15.46:0/180556615 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v201) v1 ==== 181+0+0 (secure 0 0 0) 0x7f64c4050800 con 0x7f64dc09d780 2023-10-06T12:52:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 msgr2=0x7f64bc062850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 0x7f64bc062850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc015000 comp rx=0 tx=0).stop 2023-10-06T12:52:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 msgr2=0x7f64dc1692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:54.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f64c400f920 tx=0x7f64c4009040 comp rx=0 tx=0).stop 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 shutdown_connections 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc0c9bb0 0x7f64dc0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64dc09d780 0x7f64dc1692b0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f64bc0603a0 0x7f64bc062850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 --2- 172.21.15.46:0/180556615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64c8003bf0 0x7f64dc155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:54.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.899+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 >> 172.21.15.46:0/180556615 conn(0x7f64dc06ef60 msgr2=0x7f64dc155af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.903+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 shutdown_connections 2023-10-06T12:52:54.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:54.903+0000 7f64e26ae700 1 -- 172.21.15.46:0/180556615 wait complete. 2023-10-06T12:52:54.908 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-10-06T12:52:54.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T12:52:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.091+0000 7f8ca4c16700 1 Processor -- start 2023-10-06T12:52:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.091+0000 7f8ca4c16700 1 -- start start 2023-10-06T12:52:55.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8ca4c16700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8ca0158de0 0x7f8ca00b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:55.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8ca4c16700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00b5eb0 con 0x7f8ca0069770 2023-10-06T12:52:55.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8ca4c16700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00cef30 con 0x7f8ca01591b0 2023-10-06T12:52:55.101 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8ca4c16700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca00cf4f0 con 0x7f8ca0158de0 2023-10-06T12:52:55.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8c9ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ca0069770 0x7f8ca00b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43672/0 (socket says 172.21.15.46:43672) 2023-10-06T12:52:55.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8c9ffff700 1 -- 172.21.15.46:0/1639535649 learned_addr learned my addr 172.21.15.46:0/1639535649 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:55.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.095+0000 7f8c9f7fe700 1 --2- 172.21.15.46:0/1639535649 >> v2:172.21.15.132:3300/0 conn(0x7f8ca0158de0 0x7f8ca00b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:55.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147515437 0 0) 0x7f8c98001490 con 0x7f8ca0069770 2023-10-06T12:52:55.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c88003570 con 0x7f8ca0069770 2023-10-06T12:52:55.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789048822 0 0) 0x7f8ca00cef30 con 0x7f8ca01591b0 2023-10-06T12:52:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98001490 con 0x7f8ca01591b0 2023-10-06T12:52:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9f7fe700 1 -- 172.21.15.46:0/1639535649 >> v1:172.21.15.132:6789/0 conn(0x7f8ca01591b0 legacy=0x7f8ca0069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9f7fe700 1 -- 172.21.15.46:0/1639535649 >> v1:172.21.15.46:6789/0 conn(0x7f8ca0069770 legacy=0x7f8ca00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9f7fe700 1 -- 172.21.15.46:0/1639535649 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00ccac0 con 0x7f8ca0158de0 2023-10-06T12:52:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9f7fe700 1 --2- 172.21.15.46:0/1639535649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0158de0 0x7f8ca00b5970 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8c900098e0 tx=0x7f8c900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cd59595616b591 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90011630 con 0x7f8ca0158de0 2023-10-06T12:52:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c90011790 con 0x7f8ca0158de0 2023-10-06T12:52:55.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8c9e7fc700 1 -- 172.21.15.46:0/1639535649 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90011b30 con 0x7f8ca0158de0 2023-10-06T12:52:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8ca4c16700 1 -- 172.21.15.46:0/1639535649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0158de0 msgr2=0x7f8ca00b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.099+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/1639535649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0158de0 0x7f8ca00b5970 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8c900098e0 tx=0x7f8c900091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.103+0000 7f8ca4c16700 1 -- 172.21.15.46:0/1639535649 shutdown_connections 2023-10-06T12:52:55.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.103+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/1639535649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0158de0 0x7f8ca00b5970 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.103+0000 7f8ca4c16700 1 -- 172.21.15.46:0/1639535649 >> 172.21.15.46:0/1639535649 conn(0x7f8ca0068390 msgr2=0x7f8ca00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.103+0000 7f8ca4c16700 1 -- 172.21.15.46:0/1639535649 shutdown_connections 2023-10-06T12:52:55.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.103+0000 7f8ca4c16700 1 -- 172.21.15.46:0/1639535649 wait complete. 2023-10-06T12:52:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 Processor -- start 2023-10-06T12:52:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 -- start start 2023-10-06T12:52:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 0x7f8ca00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ca0158de0 0x7f8ca0143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00c6cc0 con 0x7f8ca01591b0 2023-10-06T12:52:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00c6990 con 0x7f8ca0069770 2023-10-06T12:52:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8ca4c16700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0144200 con 0x7f8ca0158de0 2023-10-06T12:52:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 0x7f8ca00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55498/0 (socket says 172.21.15.46:55498) 2023-10-06T12:52:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 -- 172.21.15.46:0/3820923110 learned_addr learned my addr 172.21.15.46:0/3820923110 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9effd700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ca0158de0 0x7f8ca0143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 -- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ca0158de0 msgr2=0x7f8ca0143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ca0158de0 0x7f8ca0143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 -- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 msgr2=0x7f8ca00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 0x7f8ca00c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 -- 172.21.15.46:0/3820923110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90009420 con 0x7f8ca01591b0 2023-10-06T12:52:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9f7fe700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 0x7f8ca00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.107+0000 7f8c9ffff700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8c98001460 tx=0x7f8c98001c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800aa80 con 0x7f8ca01591b0 2023-10-06T12:52:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9800ac00 con 0x7f8ca01591b0 2023-10-06T12:52:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca00cc140 con 0x7f8ca01591b0 2023-10-06T12:52:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980070f0 con 0x7f8ca01591b0 2023-10-06T12:52:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca00cc700 con 0x7f8ca01591b0 2023-10-06T12:52:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.111+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca00c6e70 con 0x7f8ca01591b0 2023-10-06T12:52:55.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.115+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8c98007460 con 0x7f8ca01591b0 2023-10-06T12:52:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.115+0000 7f8c877fe700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 0x7f8c8c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.115+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6923+0+0 (secure 0 0 0) 0x7f8c9807d150 con 0x7f8ca01591b0 2023-10-06T12:52:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.115+0000 7f8c9f7fe700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 0x7f8c8c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:55.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.115+0000 7f8c9f7fe700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 0x7f8c8c062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8c900093f0 tx=0x7f8c90017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.123+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ca00c6e70 con 0x7f8ca01591b0 2023-10-06T12:52:55.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.307+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ca00b5eb0 con 0x7f8ca01591b0 2023-10-06T12:52:55.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.899+0000 7f8c877fe700 1 -- 172.21.15.46:0/3820923110 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v202) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ca00b5eb0 con 0x7f8ca01591b0 2023-10-06T12:52:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 msgr2=0x7f8c8c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 0x7f8c8c062b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8c900093f0 tx=0x7f8c90017000 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 msgr2=0x7f8ca00cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8c98001460 tx=0x7f8c98001c50 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 shutdown_connections 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca01591b0 0x7f8ca00cb920 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ca0158de0 0x7f8ca0143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8c8c060670 0x7f8c8c062b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 --2- 172.21.15.46:0/3820923110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ca0069770 0x7f8ca00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 >> 172.21.15.46:0/3820923110 conn(0x7f8ca0068390 msgr2=0x7f8ca00b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 shutdown_connections 2023-10-06T12:52:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:55.907+0000 7f8ca4c16700 1 -- 172.21.15.46:0/3820923110 wait complete. 2023-10-06T12:52:55.914 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-10-06T12:52:55.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T12:52:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.103+0000 7f6da586b700 1 Processor -- start 2023-10-06T12:52:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.103+0000 7f6da586b700 1 -- start start 2023-10-06T12:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da586b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6da01591b0 0x7f6da0069640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da586b700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00ce6e0 con 0x7f6da0159850 2023-10-06T12:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da586b700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00cddb0 con 0x7f6da0069270 2023-10-06T12:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da586b700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da015a5c0 con 0x7f6da01591b0 2023-10-06T12:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da506a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6da0159850 0x7f6da0159cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43676/0 (socket says 172.21.15.46:43676) 2023-10-06T12:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6da506a700 1 -- 172.21.15.46:0/15369125 learned_addr learned my addr 172.21.15.46:0/15369125 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9ffff700 1 --2- 172.21.15.46:0/15369125 >> v2:172.21.15.46:3300/0 conn(0x7f6da01591b0 0x7f6da0069640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186451099 0 0) 0x7f6da00ce6e0 con 0x7f6da0159850 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d90003570 con 0x7f6da0159850 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9ffff700 1 -- 172.21.15.46:0/15369125 >> v1:172.21.15.132:6789/0 conn(0x7f6da0069270 legacy=0x7f6da0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500668614 0 0) 0x7f6da00cddb0 con 0x7f6da0069270 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9ffff700 1 -- 172.21.15.46:0/15369125 >> v1:172.21.15.46:6789/0 conn(0x7f6da0159850 legacy=0x7f6da0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9ffff700 1 -- 172.21.15.46:0/15369125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00ce090 con 0x7f6da01591b0 2023-10-06T12:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9ffff700 1 --2- 172.21.15.46:0/15369125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da0069640 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098e0 tx=0x7f6d8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5497146fc5f6c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00b640 con 0x7f6da01591b0 2023-10-06T12:52:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00b7a0 con 0x7f6da01591b0 2023-10-06T12:52:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.107+0000 7f6d9f7fe700 1 -- 172.21.15.46:0/15369125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00bb40 con 0x7f6da01591b0 2023-10-06T12:52:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- 172.21.15.46:0/15369125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 msgr2=0x7f6da0069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 --2- 172.21.15.46:0/15369125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da0069640 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098e0 tx=0x7f6d8c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- 172.21.15.46:0/15369125 shutdown_connections 2023-10-06T12:52:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 --2- 172.21.15.46:0/15369125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da0069640 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- 172.21.15.46:0/15369125 >> 172.21.15.46:0/15369125 conn(0x7f6da0068390 msgr2=0x7f6da00d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- 172.21.15.46:0/15369125 shutdown_connections 2023-10-06T12:52:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- 172.21.15.46:0/15369125 wait complete. 2023-10-06T12:52:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 Processor -- start 2023-10-06T12:52:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- start start 2023-10-06T12:52:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da0069270 0x7f6da014cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 0x7f6da015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da015a3e0 con 0x7f6da01591b0 2023-10-06T12:52:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00b5300 con 0x7f6da0069270 2023-10-06T12:52:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6da586b700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da014d0e0 con 0x7f6da0159850 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.111+0000 7f6d9ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55512/0 (socket says 172.21.15.46:55512) 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 -- 172.21.15.46:0/2440121452 learned_addr learned my addr 172.21.15.46:0/2440121452 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da4869700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da0069270 0x7f6da014cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da506a700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 0x7f6da015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 -- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 msgr2=0x7f6da015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 0x7f6da015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 -- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da0069270 msgr2=0x7f6da014cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da0069270 0x7f6da014cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 -- 172.21.15.46:0/2440121452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009420 con 0x7f6da01591b0 2023-10-06T12:52:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da506a700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 0x7f6da015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9ffff700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0093f0 tx=0x7f6d8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01f070 con 0x7f6da01591b0 2023-10-06T12:52:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c000bc0 con 0x7f6da01591b0 2023-10-06T12:52:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da00b3ce0 con 0x7f6da01591b0 2023-10-06T12:52:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c025420 con 0x7f6da01591b0 2023-10-06T12:52:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da00b49d0 con 0x7f6da01591b0 2023-10-06T12:52:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.115+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da0064590 con 0x7f6da01591b0 2023-10-06T12:52:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.119+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6d8c023070 con 0x7f6da01591b0 2023-10-06T12:52:56.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.119+0000 7f6d9cff9700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 0x7f6d84062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.119+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f6d8c07ddf0 con 0x7f6da01591b0 2023-10-06T12:52:56.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.119+0000 7f6da4869700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 0x7f6d84062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.119+0000 7f6da4869700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 0x7f6d84062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d94003820 tx=0x7f6d94009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.127+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6d8c04df10 con 0x7f6da01591b0 2023-10-06T12:52:56.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.311+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6da00ce370 con 0x7f6da01591b0 2023-10-06T12:52:56.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.319+0000 7f6d9cff9700 1 -- 172.21.15.46:0/2440121452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d8c051530 con 0x7f6da01591b0 2023-10-06T12:52:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 msgr2=0x7f6d84062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 0x7f6d84062b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d94003820 tx=0x7f6d94009040 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 msgr2=0x7f6da00b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0093f0 tx=0x7f6d8c017000 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 shutdown_connections 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da0159850 0x7f6da015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6da01591b0 0x7f6da00b37a0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f6d84060670 0x7f6d84062b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 --2- 172.21.15.46:0/2440121452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da0069270 0x7f6da014cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 >> 172.21.15.46:0/2440121452 conn(0x7f6da0068390 msgr2=0x7f6da00cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 shutdown_connections 2023-10-06T12:52:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.327+0000 7f6da586b700 1 -- 172.21.15.46:0/2440121452 wait complete. 2023-10-06T12:52:56.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T12:52:56.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.507+0000 7f895d84c700 1 Processor -- start 2023-10-06T12:52:56.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.507+0000 7f895d84c700 1 -- start start 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.511+0000 7f895d84c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895815b2b0 0x7f89580b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.511+0000 7f895d84c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895815b6d0 con 0x7f89580abc40 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d84c700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89580b04e0 con 0x7f89580af030 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d84c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580afbb0 con 0x7f895815b2b0 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89580abc40 0x7f89580ae920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43686/0 (socket says 172.21.15.46:43686) 2023-10-06T12:52:56.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 -- 172.21.15.46:0/1136918453 learned_addr learned my addr 172.21.15.46:0/1136918453 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d04b700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895815b2b0 0x7f89580b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325139914 0 0) 0x7f894c001110 con 0x7f89580abc40 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8948003570 con 0x7f89580abc40 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516969090 0 0) 0x7f89580b04e0 con 0x7f89580af030 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f894c001110 con 0x7f89580af030 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2579980706 0 0) 0x7f8948003570 con 0x7f89580abc40 2023-10-06T12:52:56.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895815b2b0 msgr2=0x7f89580b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895815b2b0 0x7f89580b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 >> v1:172.21.15.132:6789/0 conn(0x7f89580af030 legacy=0x7f895815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580afe90 con 0x7f89580abc40 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f894c001620 con 0x7f89580abc40 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 >> v1:172.21.15.46:6789/0 conn(0x7f89580abc40 legacy=0x7f89580ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f8948004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8948004810 0x7f8948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89480051c0 0x7f894800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580afbb0 con 0x7f8948003c20 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894800c020 con 0x7f8948004810 2023-10-06T12:52:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894800c1e0 con 0x7f89480051c0 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d04b700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895815b2b0 0x7f89580b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f8948004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f8957fff700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8948004810 0x7f8948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d04b700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89480051c0 0x7f894800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 -- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89480051c0 msgr2=0x7f894800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89480051c0 0x7f894800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 -- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8948004810 msgr2=0x7f8948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8948004810 0x7f8948004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895c84a700 1 -- 172.21.15.46:0/1136918453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948003570 con 0x7f8948003c20 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.515+0000 7f895d04b700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89480051c0 0x7f894800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:52:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895c84a700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f8948004010 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c5c0 tx=0x7f894c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06a55b7c89d65df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00cdc0 con 0x7f8948003c20 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c0048f0 con 0x7f8948003c20 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f89577fe700 1 -- 172.21.15.46:0/1136918453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c004cb0 con 0x7f8948003c20 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- 172.21.15.46:0/1136918453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 msgr2=0x7f8948004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 --2- 172.21.15.46:0/1136918453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f8948004010 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c5c0 tx=0x7f894c007a00 comp rx=0 tx=0).stop 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d04b700 1 -- 172.21.15.46:0/1136918453 reap_dead start 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- 172.21.15.46:0/1136918453 shutdown_connections 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- 172.21.15.46:0/1136918453 >> 172.21.15.46:0/1136918453 conn(0x7f89580ccd50 msgr2=0x7f89580cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- 172.21.15.46:0/1136918453 shutdown_connections 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- 172.21.15.46:0/1136918453 wait complete. 2023-10-06T12:52:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 Processor -- start 2023-10-06T12:52:56.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.519+0000 7f895d84c700 1 -- start start 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89580a9840 0x7f895816a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895814e970 0x7f8958157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580aaa10 con 0x7f8948003c20 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580a9e00 con 0x7f89580a9840 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580aa0e0 con 0x7f895814e970 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55536/0 (socket says 172.21.15.46:55536) 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 -- 172.21.15.46:0/3172548672 learned_addr learned my addr 172.21.15.46:0/3172548672 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f8957fff700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89580a9840 0x7f895816a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d04b700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895814e970 0x7f8958157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 -- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895814e970 msgr2=0x7f8958157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895814e970 0x7f8958157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 -- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89580a9840 msgr2=0x7f895816a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89580a9840 0x7f895816a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 -- 172.21.15.46:0/3172548672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580aa3c0 con 0x7f8948003c20 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895c84a700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f894c008fd0 tx=0x7f894c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c004d40 con 0x7f8948003c20 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c007360 con 0x7f8948003c20 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c004ea0 con 0x7f8948003c20 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89580aa6a0 con 0x7f8948003c20 2023-10-06T12:52:56.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.523+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895814ede0 con 0x7f8948003c20 2023-10-06T12:52:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.527+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8958064590 con 0x7f8948003c20 2023-10-06T12:52:56.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.527+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f894c014630 con 0x7f8948003c20 2023-10-06T12:52:56.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.527+0000 7f8954ff9700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 0x7f893c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.527+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f894c07e920 con 0x7f8948003c20 2023-10-06T12:52:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.527+0000 7f8957fff700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 0x7f893c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.531+0000 7f8957fff700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 0x7f893c062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8944000c00 tx=0x7f8944007d10 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.535+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f894c04eb00 con 0x7f8948003c20 2023-10-06T12:52:56.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.711+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f895815b6d0 con 0x7f8948003c20 2023-10-06T12:52:56.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.719+0000 7f8954ff9700 1 -- 172.21.15.46:0/3172548672 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7f895815b6d0 con 0x7f8948003c20 2023-10-06T12:52:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.723+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 msgr2=0x7f893c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 0x7f893c062b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8944000c00 tx=0x7f8944007d10 comp rx=0 tx=0).stop 2023-10-06T12:52:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 msgr2=0x7f89580aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f894c008fd0 tx=0x7f894c015000 comp rx=0 tx=0).stop 2023-10-06T12:52:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 shutdown_connections 2023-10-06T12:52:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895814e970 0x7f8958157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f893c0606a0 0x7f893c062b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89580a9840 0x7f895816a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 --2- 172.21.15.46:0/3172548672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8948003c20 0x7f89580aaea0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 >> 172.21.15.46:0/3172548672 conn(0x7f89580ccd50 msgr2=0x7f89581565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 shutdown_connections 2023-10-06T12:52:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.727+0000 7f895d84c700 1 -- 172.21.15.46:0/3172548672 wait complete. 2023-10-06T12:52:56.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T12:52:56.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.911+0000 7f88182ae700 1 Processor -- start 2023-10-06T12:52:56.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.911+0000 7f88182ae700 1 -- start start 2023-10-06T12:52:56.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f88182ae700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8810159550 0x7f88100b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f88182ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88100af6f0 con 0x7f88100b2360 2023-10-06T12:52:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f88182ae700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88100b43e0 con 0x7f88100aee00 2023-10-06T12:52:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f88182ae700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88100b04b0 con 0x7f8810159550 2023-10-06T12:52:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f881604a700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f88100b2360 0x7f8810158e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43700/0 (socket says 172.21.15.46:43700) 2023-10-06T12:52:56.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f881604a700 1 -- 172.21.15.46:0/1147936330 learned_addr learned my addr 172.21.15.46:0/1147936330 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.915+0000 7f8815849700 1 --2- 172.21.15.46:0/1147936330 >> v2:172.21.15.46:3300/0 conn(0x7f8810159550 0x7f88100b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683410387 0 0) 0x7f88100af6f0 con 0x7f88100b2360 2023-10-06T12:52:56.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87fc003570 con 0x7f88100b2360 2023-10-06T12:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382139726 0 0) 0x7f88100b43e0 con 0x7f88100aee00 2023-10-06T12:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815849700 1 -- 172.21.15.46:0/1147936330 >> v1:172.21.15.132:6789/0 conn(0x7f88100aee00 legacy=0x7f88100b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815849700 1 -- 172.21.15.46:0/1147936330 >> v1:172.21.15.46:6789/0 conn(0x7f88100b2360 legacy=0x7f8810158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815849700 1 -- 172.21.15.46:0/1147936330 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100b0790 con 0x7f8810159550 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815849700 1 --2- 172.21.15.46:0/1147936330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8810159550 0x7f88100b2730 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f88000098e0 tx=0x7f88000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef672e9b00d74a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c001470 con 0x7f8810159550 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8800011420 con 0x7f8810159550 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f8815048700 1 -- 172.21.15.46:0/1147936330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88000117c0 con 0x7f8810159550 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 -- 172.21.15.46:0/1147936330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8810159550 msgr2=0x7f88100b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 --2- 172.21.15.46:0/1147936330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8810159550 0x7f88100b2730 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f88000098e0 tx=0x7f88000091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 -- 172.21.15.46:0/1147936330 shutdown_connections 2023-10-06T12:52:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 --2- 172.21.15.46:0/1147936330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8810159550 0x7f88100b2730 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 -- 172.21.15.46:0/1147936330 >> 172.21.15.46:0/1147936330 conn(0x7f881006e780 msgr2=0x7f881006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 -- 172.21.15.46:0/1147936330 shutdown_connections 2023-10-06T12:52:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.919+0000 7f88182ae700 1 -- 172.21.15.46:0/1147936330 wait complete. 2023-10-06T12:52:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 Processor -- start 2023-10-06T12:52:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 -- start start 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88100b2360 0x7f881015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8810159550 0x7f88100c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88100c7200 con 0x7f88100aee00 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88100c74e0 con 0x7f8810159550 2023-10-06T12:52:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f88182ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88100c77c0 con 0x7f88100b2360 2023-10-06T12:52:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f881604a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f881604a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55548/0 (socket says 172.21.15.46:55548) 2023-10-06T12:52:56.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.923+0000 7f881604a700 1 -- 172.21.15.46:0/3626211015 learned_addr learned my addr 172.21.15.46:0/3626211015 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881684b700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8810159550 0x7f88100c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f8815849700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88100b2360 0x7f881015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 -- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88100b2360 msgr2=0x7f881015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88100b2360 0x7f881015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 -- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8810159550 msgr2=0x7f88100c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8810159550 0x7f88100c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 -- 172.21.15.46:0/3626211015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800009420 con 0x7f88100aee00 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f881604a700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f880c0015a0 tx=0x7f880c001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c00ce40 con 0x7f88100aee00 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880c007360 con 0x7f88100aee00 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c004830 con 0x7f88100aee00 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88100c7aa0 con 0x7f88100aee00 2023-10-06T12:52:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88100c9e00 con 0x7f88100aee00 2023-10-06T12:52:56.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.927+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8810088840 con 0x7f88100aee00 2023-10-06T12:52:56.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.931+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f880c004ba0 con 0x7f88100aee00 2023-10-06T12:52:56.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.931+0000 7f88067fc700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 0x7f87f4062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:56.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.931+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f880c07c070 con 0x7f88100aee00 2023-10-06T12:52:56.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.931+0000 7f8815849700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 0x7f87f4062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.935+0000 7f8815849700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 0x7f87f4062b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88000030c0 tx=0x7f8800017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:56.939+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f880c04c210 con 0x7f88100aee00 2023-10-06T12:52:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.111+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f88100b0a70 con 0x7f88100aee00 2023-10-06T12:52:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.123+0000 7f88067fc700 1 -- 172.21.15.46:0/3626211015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7f880c04f830 con 0x7f88100aee00 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 msgr2=0x7f87f4062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 0x7f87f4062b50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88000030c0 tx=0x7f8800017000 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 msgr2=0x7f88101585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f880c0015a0 tx=0x7f880c001bf0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 shutdown_connections 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f87f40606a0 0x7f87f4062b50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8810159550 0x7f88100c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88100b2360 0x7f881015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 --2- 172.21.15.46:0/3626211015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88100aee00 0x7f88101585c0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 >> 172.21.15.46:0/3626211015 conn(0x7f881006e780 msgr2=0x7f88100b5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 shutdown_connections 2023-10-06T12:52:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.127+0000 7f88182ae700 1 -- 172.21.15.46:0/3626211015 wait complete. 2023-10-06T12:52:57.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T12:52:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.307+0000 7f5826123700 1 Processor -- start 2023-10-06T12:52:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.311+0000 7f5826123700 1 -- start start 2023-10-06T12:52:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5826123700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58200b5af0 0x7f58200adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5826123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58200b2fd0 con 0x7f58200b4450 2023-10-06T12:52:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5826123700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58200b17b0 con 0x7f582015d480 2023-10-06T12:52:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5826123700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58200b0ba0 con 0x7f58200b5af0 2023-10-06T12:52:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5824920700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58200b5af0 0x7f58200adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5825922700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58200b4450 0x7f58200ae120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43710/0 (socket says 172.21.15.46:43710) 2023-10-06T12:52:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5825922700 1 -- 172.21.15.46:0/1247676195 learned_addr learned my addr 172.21.15.46:0/1247676195 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5817fff700 1 -- 172.21.15.46:0/1247676195 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290870722 0 0) 0x7f58200b17b0 con 0x7f582015d480 2023-10-06T12:52:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5817fff700 1 -- 172.21.15.46:0/1247676195 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f580c003570 con 0x7f582015d480 2023-10-06T12:52:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5824920700 1 -- 172.21.15.46:0/1247676195 >> v1:172.21.15.132:6789/0 conn(0x7f582015d480 legacy=0x7f58200b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5824920700 1 -- 172.21.15.46:0/1247676195 >> v1:172.21.15.46:6789/0 conn(0x7f58200b4450 legacy=0x7f58200ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5824920700 1 -- 172.21.15.46:0/1247676195 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200b0e80 con 0x7f58200b5af0 2023-10-06T12:52:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5824920700 1 --2- 172.21.15.46:0/1247676195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b5af0 0x7f58200adbe0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f58100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19e6b50a8b9e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5817fff700 1 -- 172.21.15.46:0/1247676195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000b640 con 0x7f58200b5af0 2023-10-06T12:52:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5817fff700 1 -- 172.21.15.46:0/1247676195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581000b7a0 con 0x7f58200b5af0 2023-10-06T12:52:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.315+0000 7f5817fff700 1 -- 172.21.15.46:0/1247676195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000bb40 con 0x7f58200b5af0 2023-10-06T12:52:57.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 -- 172.21.15.46:0/1247676195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b5af0 msgr2=0x7f58200adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 --2- 172.21.15.46:0/1247676195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b5af0 0x7f58200adbe0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f58100091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 -- 172.21.15.46:0/1247676195 shutdown_connections 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 --2- 172.21.15.46:0/1247676195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b5af0 0x7f58200adbe0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 -- 172.21.15.46:0/1247676195 >> 172.21.15.46:0/1247676195 conn(0x7f582006ef60 msgr2=0x7f58200d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 -- 172.21.15.46:0/1247676195 shutdown_connections 2023-10-06T12:52:57.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.319+0000 7f5826123700 1 -- 172.21.15.46:0/1247676195 wait complete. 2023-10-06T12:52:57.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 Processor -- start 2023-10-06T12:52:57.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 -- start start 2023-10-06T12:52:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 0x7f5820144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582015d480 0x7f5820160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582015e3e0 con 0x7f58200b4450 2023-10-06T12:52:57.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820158890 con 0x7f58200b5af0 2023-10-06T12:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5826123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582015e5d0 con 0x7f582015d480 2023-10-06T12:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 0x7f5820144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825922700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582015d480 0x7f5820160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5824920700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5824920700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.46:39986/0 (socket says 172.21.15.46:39986) 2023-10-06T12:52:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5824920700 1 -- 172.21.15.46:0/133809068 learned_addr learned my addr 172.21.15.46:0/133809068 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 -- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582015d480 msgr2=0x7f5820160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582015d480 0x7f5820160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 -- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 msgr2=0x7f582009b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5825121700 1 -- 172.21.15.46:0/133809068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810009420 con 0x7f58200b4450 2023-10-06T12:52:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.323+0000 7f5824920700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:52:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f5825121700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 0x7f5820144600 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c560 tx=0x7f581c007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c011590 con 0x7f58200b4450 2023-10-06T12:52:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c0116f0 con 0x7f58200b4450 2023-10-06T12:52:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c011ab0 con 0x7f58200b4450 2023-10-06T12:52:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582015e8b0 con 0x7f58200b4450 2023-10-06T12:52:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582015ee70 con 0x7f58200b4450 2023-10-06T12:52:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5820064590 con 0x7f58200b4450 2023-10-06T12:52:57.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.327+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f581c01c460 con 0x7f58200b4450 2023-10-06T12:52:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.331+0000 7f58157fa700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 0x7f5804062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.331+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f581c07c8b0 con 0x7f58200b4450 2023-10-06T12:52:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.331+0000 7f5824920700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 0x7f5804062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.331+0000 7f5824920700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 0x7f5804062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58100093a0 tx=0x7f5810017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:57.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.335+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f581c04ca70 con 0x7f58200b4450 2023-10-06T12:52:57.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.571+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f58200b1160 con 0x7f58200b4450 2023-10-06T12:52:57.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.907+0000 7f58157fa700 1 -- 172.21.15.46:0/133809068 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3066) v1 ==== 155+0+0 (secure 0 0 0) 0x7f581c050090 con 0x7f58200b4450 2023-10-06T12:52:57.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 msgr2=0x7f5804062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 0x7f5804062b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58100093a0 tx=0x7f5810017000 comp rx=0 tx=0).stop 2023-10-06T12:52:57.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 msgr2=0x7f5820144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:57.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 0x7f5820144600 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c560 tx=0x7f581c007d50 comp rx=0 tx=0).stop 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 shutdown_connections 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582015d480 0x7f5820160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5804060670 0x7f5804062b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58200b5af0 0x7f582009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 --2- 172.21.15.46:0/133809068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58200b4450 0x7f5820144600 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 >> 172.21.15.46:0/133809068 conn(0x7f582006ef60 msgr2=0x7f582015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 shutdown_connections 2023-10-06T12:52:57.919 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:57.915+0000 7f5826123700 1 -- 172.21.15.46:0/133809068 wait complete. 2023-10-06T12:52:57.941 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ERROR 2023-10-06T12:52:57.941 INFO:tasks.cephfs_test_runner: 2023-10-06T12:52:57.941 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-06T12:52:57.941 INFO:tasks.cephfs_test_runner:ERROR: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs/test_snap_schedules.py", line 552, in test_snap_dir_name 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs/test_snap_schedules.py", line 531, in get_snap_dir_name 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:ModuleNotFoundError: No module named 'tasks.cephfs.fuse_mount' 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner: 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5642.628s 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner: 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-06T12:52:57.942 INFO:tasks.cephfs_test_runner: 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:ERROR: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs/test_snap_schedules.py", line 552, in test_snap_dir_name 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs/test_snap_schedules.py", line 531, in get_snap_dir_name 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner:ModuleNotFoundError: No module named 'tasks.cephfs.fuse_mount' 2023-10-06T12:52:57.943 INFO:tasks.cephfs_test_runner: 2023-10-06T12:52:57.943 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs_test_runner.py", line 211, in task RuntimeError: Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T12:52:58.013 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=86a0ad59ced74a0287b2f3042caed979 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks/cephfs_test_runner.py", line 211, in task RuntimeError: Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T12:52:58.015 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T12:52:58.026 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T12:52:58.056 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T12:52:58.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.419 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:52:58.448 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T12:52:58.564 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T12:52:58.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T12:52:58.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T12:52:58.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-10-06T12:52:58.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.969 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-10-06T12:52:58.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:58.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-10-06T12:52:59.072 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2023-10-06T12:52:59.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T12:52:59.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.080 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T12:52:59.099 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T12:52:59.099 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T12:52:59.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.100 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T12:52:59.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T12:52:59.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-10-06T12:52:59.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.524 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2023-10-06T12:52:59.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T12:52:59.538 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2023-10-06T12:52:59.626 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2023-10-06T12:52:59.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T12:52:59.630 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T12:52:59.643 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T12:52:59.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:52:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.807+0000 7fa6b4465700 1 Processor -- start 2023-10-06T12:52:59.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.811+0000 7fa6b4465700 1 -- start start 2023-10-06T12:52:59.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b4465700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6ac161470 0x7fa6ac161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b4465700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 0x7fa6ac0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b4465700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0c18f0 con 0x7fa6ac06cc30 2023-10-06T12:52:59.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b4465700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac0c0ce0 con 0x7fa6ac161470 2023-10-06T12:52:59.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b4465700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0c0fc0 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b2a02700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa6ac06cc30 0x7fa6ac15d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43712/0 (socket says 172.21.15.46:43712) 2023-10-06T12:52:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b2a02700 1 -- 172.21.15.46:0/473730929 learned_addr learned my addr 172.21.15.46:0/473730929 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:52:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/473730929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 0x7fa6ac0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b2201700 1 --2- 172.21.15.46:0/473730929 >> v2:172.21.15.132:3300/0 conn(0x7fa6ac161470 0x7fa6ac161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b11ff700 1 -- 172.21.15.46:0/473730929 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035817456 0 0) 0x7fa6ac0c18f0 con 0x7fa6ac06cc30 2023-10-06T12:52:59.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b11ff700 1 -- 172.21.15.46:0/473730929 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa698003570 con 0x7fa6ac06cc30 2023-10-06T12:52:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 -- 172.21.15.46:0/473730929 >> v2:172.21.15.132:3300/0 conn(0x7fa6ac161470 msgr2=0x7fa6ac161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:52:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/473730929 >> v2:172.21.15.132:3300/0 conn(0x7fa6ac161470 0x7fa6ac161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 -- 172.21.15.46:0/473730929 >> v1:172.21.15.46:6789/0 conn(0x7fa6ac06cc30 legacy=0x7fa6ac15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 -- 172.21.15.46:0/473730929 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0c12a0 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/473730929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 0x7fa6ac0c21b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0098e0 tx=0x7fa69c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78520dc344c4c8ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.815+0000 7fa6b11ff700 1 -- 172.21.15.46:0/473730929 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011630 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b11ff700 1 -- 172.21.15.46:0/473730929 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011790 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b11ff700 1 -- 172.21.15.46:0/473730929 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011b30 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 -- 172.21.15.46:0/473730929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 msgr2=0x7fa6ac0c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 --2- 172.21.15.46:0/473730929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 0x7fa6ac0c21b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0098e0 tx=0x7fa69c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 -- 172.21.15.46:0/473730929 shutdown_connections 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 --2- 172.21.15.46:0/473730929 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac0c1d80 0x7fa6ac0c21b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 --2- 172.21.15.46:0/473730929 >> v2:172.21.15.132:3300/0 conn(0x7fa6ac161470 0x7fa6ac161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 -- 172.21.15.46:0/473730929 >> 172.21.15.46:0/473730929 conn(0x7fa6ac0cbe30 msgr2=0x7fa6ac0cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 -- 172.21.15.46:0/473730929 shutdown_connections 2023-10-06T12:52:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.819+0000 7fa6b4465700 1 -- 172.21.15.46:0/473730929 wait complete. 2023-10-06T12:52:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 Processor -- start 2023-10-06T12:52:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 -- start start 2023-10-06T12:52:59.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac06cc30 0x7fa6ac0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6ac161470 0x7fa6ac0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0c6b40 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac171100 con 0x7fa6ac161470 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b4465700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac1713e0 con 0x7fa6ac06cc30 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b1a00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55584/0 (socket says 172.21.15.46:55584) 2023-10-06T12:52:59.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b1a00700 1 -- 172.21.15.46:0/3093127022 learned_addr learned my addr 172.21.15.46:0/3093127022 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b2a02700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6ac161470 0x7fa6ac0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.823+0000 7fa6b2201700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac06cc30 0x7fa6ac0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 -- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac06cc30 msgr2=0x7fa6ac0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac06cc30 0x7fa6ac0c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 -- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6ac161470 msgr2=0x7fa6ac0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6ac161470 0x7fa6ac0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:52:59.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 -- 172.21.15.46:0/3093127022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009420 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b1a00700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0c1100 tx=0x7fa69c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c000d10 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011b10 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac1716c0 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c025660 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac171c80 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.827+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac063f90 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.831+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa69c025a50 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.831+0000 7fa6a27fc700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 0x7fa690062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:52:59.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.831+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fa69c07e560 con 0x7fa6ac0c1d80 2023-10-06T12:52:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.831+0000 7fa6b2201700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 0x7fa690062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:52:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.831+0000 7fa6b2201700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 0x7fa690062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009c70 tx=0x7fa6a8009120 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:52:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:52:59.839+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa69c04e680 con 0x7fa6ac0c1d80 2023-10-06T12:53:00.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.011+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6ac163600 con 0x7fa6ac0c1d80 2023-10-06T12:53:00.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.015+0000 7fa6a27fc700 1 -- 172.21.15.46:0/3093127022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+15590 (secure 0 0 0) 0x7fa69c051ca0 con 0x7fa6ac0c1d80 2023-10-06T12:53:00.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:00.020 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":202,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:55.893898+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2405550575":"2023-10-07T12:52:52.844581+0000","172.21.15.46:6834/2405550575":"2023-10-07T12:52:52.844581+0000","172.21.15.46:0/131996082":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/2304841460":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/2332664225":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1324755838":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1912199789":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1393872098":"2023-10-07T12:52:52.561150+0000","172.21.15.46:6832/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1339853979":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1151621779":"2023-10-07T12:52:44.738959+0000","172.21.15.46:6833/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/938799999":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/580990329":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1645444510":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/3242963477":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1367734289":"2023-10-07T12:52:44.738959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:53:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 msgr2=0x7fa690062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 0x7fa690062b20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009c70 tx=0x7fa6a8009120 comp rx=0 tx=0).stop 2023-10-06T12:53:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 msgr2=0x7fa6ac15c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0c1100 tx=0x7fa69c017000 comp rx=0 tx=0).stop 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 shutdown_connections 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa690060670 0x7fa690062b20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6ac161470 0x7fa6ac0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ac0c1d80 0x7fa6ac15c000 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 --2- 172.21.15.46:0/3093127022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6ac06cc30 0x7fa6ac0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 >> 172.21.15.46:0/3093127022 conn(0x7fa6ac0cbe30 msgr2=0x7fa6ac0ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 shutdown_connections 2023-10-06T12:53:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.019+0000 7fa6b4465700 1 -- 172.21.15.46:0/3093127022 wait complete. 2023-10-06T12:53:00.044 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T12:53:00.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:53:00.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.199+0000 7fc01ed63700 1 Processor -- start 2023-10-06T12:53:00.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 -- start start 2023-10-06T12:53:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc018158100 0x7fc0180c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 0x7fc0180cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0180c90b0 con 0x7fc0180b02a0 2023-10-06T12:53:00.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0180cb180 con 0x7fc018158100 2023-10-06T12:53:00.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01ed63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180cc980 con 0x7fc0180ca7c0 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc018158100 0x7fc0180c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc018158100 0x7fc0180c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55586/0 (socket says 172.21.15.46:55586) 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 -- 172.21.15.46:0/4158355442 learned_addr learned my addr 172.21.15.46:0/4158355442 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01e562700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 0x7fc0180cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 -- 172.21.15.46:0/4158355442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 msgr2=0x7fc0180cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 0x7fc0180cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 -- 172.21.15.46:0/4158355442 >> v1:172.21.15.132:6789/0 conn(0x7fc0180b02a0 legacy=0x7fc0180cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 -- 172.21.15.46:0/4158355442 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0180ccc60 con 0x7fc018158100 2023-10-06T12:53:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01e562700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 0x7fc0180cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:53:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.203+0000 7fc01d560700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc0180c2c20 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fc0080099b0 tx=0x7fc008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ea68fcf17d32f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01cd5f700 1 -- 172.21.15.46:0/4158355442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800b860 con 0x7fc018158100 2023-10-06T12:53:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01cd5f700 1 -- 172.21.15.46:0/4158355442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00800b9c0 con 0x7fc018158100 2023-10-06T12:53:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01cd5f700 1 -- 172.21.15.46:0/4158355442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800bd60 con 0x7fc018158100 2023-10-06T12:53:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- 172.21.15.46:0/4158355442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 msgr2=0x7fc0180c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc0180c2c20 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fc0080099b0 tx=0x7fc008009250 comp rx=0 tx=0).stop 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- 172.21.15.46:0/4158355442 shutdown_connections 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180ca7c0 0x7fc0180cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 --2- 172.21.15.46:0/4158355442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc0180c2c20 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- 172.21.15.46:0/4158355442 >> 172.21.15.46:0/4158355442 conn(0x7fc0180970c0 msgr2=0x7fc018067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- 172.21.15.46:0/4158355442 shutdown_connections 2023-10-06T12:53:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- 172.21.15.46:0/4158355442 wait complete. 2023-10-06T12:53:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 Processor -- start 2023-10-06T12:53:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.207+0000 7fc01ed63700 1 -- start start 2023-10-06T12:53:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180b02a0 0x7fc018154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0180ca7c0 0x7fc01816cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0180b1f60 con 0x7fc018158100 2023-10-06T12:53:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018155070 con 0x7fc0180ca7c0 2023-10-06T12:53:00.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc018155350 con 0x7fc0180b02a0 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55598/0 (socket says 172.21.15.46:55598) 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01dd61700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180b02a0 0x7fc018154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 -- 172.21.15.46:0/1141298485 learned_addr learned my addr 172.21.15.46:0/1141298485 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01d560700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0180ca7c0 0x7fc01816cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 -- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180b02a0 msgr2=0x7fc018154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180b02a0 0x7fc018154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 -- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0180ca7c0 msgr2=0x7fc01816cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0180ca7c0 0x7fc01816cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 -- 172.21.15.46:0/1141298485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080094f0 con 0x7fc018158100 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01e562700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fc01000cef0 tx=0x7fc010007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000ad20 con 0x7fc018158100 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010016420 con 0x7fc018158100 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000aea0 con 0x7fc018158100 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.211+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc018155630 con 0x7fc018158100 2023-10-06T12:53:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.215+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01816a420 con 0x7fc018158100 2023-10-06T12:53:00.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.215+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01816a610 con 0x7fc018158100 2023-10-06T12:53:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.215+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fc010016a70 con 0x7fc018158100 2023-10-06T12:53:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.219+0000 7fc00dffb700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 0x7fbffc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.219+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fc010012070 con 0x7fc018158100 2023-10-06T12:53:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.219+0000 7fc01dd61700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 0x7fbffc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.219+0000 7fc01dd61700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 0x7fbffc062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc014003a20 tx=0x7fc014007f10 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.223+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc01004cbe0 con 0x7fc018158100 2023-10-06T12:53:00.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.395+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc018064590 con 0x7fbffc060670 2023-10-06T12:53:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.403+0000 7fc00dffb700 1 -- 172.21.15.46:0/1141298485 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70134 (secure 0 0 0) 0x7fc018064590 con 0x7fbffc060670 2023-10-06T12:53:00.411 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 msgr2=0x7fbffc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 0x7fbffc062b20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc014003a20 tx=0x7fc014007f10 comp rx=0 tx=0).stop 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 msgr2=0x7fc01816d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fc01000cef0 tx=0x7fc010007860 comp rx=0 tx=0).stop 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 shutdown_connections 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc018158100 0x7fc01816d240 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fbffc060670 0x7fbffc062b20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0180ca7c0 0x7fc01816cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 --2- 172.21.15.46:0/1141298485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0180b02a0 0x7fc018154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 >> 172.21.15.46:0/1141298485 conn(0x7fc0180970c0 msgr2=0x7fc0180aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 shutdown_connections 2023-10-06T12:53:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.411+0000 7fc01ed63700 1 -- 172.21.15.46:0/1141298485 wait complete. 2023-10-06T12:53:00.418 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:53:00.436 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2023-10-06T12:52:58.589850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2457812,"kb_used_data":22044,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752516908,"statfs":{"total":773094113280,"available":770577313792,"internally_reserved":0,"allocated":22573056,"data_stored":13165550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663452,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-216,"num_read_kb":-190,"num_write":-216,"num_write_kb":-1394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.988980"},"pg_stats":[{"pgid":"1.0","version":"199'370","reported_seq":1212,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.703125+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:52.703125+0000","last_peered":"2023-10-06T12:52:52.703125+0000","last_clean":"2023-10-06T12:52:52.703125+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:52.703125+0000","last_undegraded":"2023-10-06T12:52:52.703125+0000","last_fullsized":"2023-10-06T12:52:52.703125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":184,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.577593+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:52.577593+0000","last_peered":"2023-10-06T12:52:52.577593+0000","last_clean":"2023-10-06T12:52:52.577593+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:52.577593+0000","last_undegraded":"2023-10-06T12:52:52.577593+0000","last_fullsized":"2023-10-06T12:52:52.577593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.616888+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:52.616888+0000","last_peered":"2023-10-06T12:52:52.616888+0000","last_clean":"2023-10-06T12:52:52.616888+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:52.616888+0000","last_undegraded":"2023-10-06T12:52:52.616888+0000","last_fullsized":"2023-10-06T12:52:52.616888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.655090+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:52.655090+0000","last_peered":"2023-10-06T12:52:52.655090+0000","last_clean":"2023-10-06T12:52:52.655090+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:52.655090+0000","last_undegraded":"2023-10-06T12:52:52.655090+0000","last_fullsized":"2023-10-06T12:52:52.655090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":184,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.576894+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:52.576894+0000","last_peered":"2023-10-06T12:52:52.576894+0000","last_clean":"2023-10-06T12:52:52.576894+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:52.576894+0000","last_undegraded":"2023-10-06T12:52:52.576894+0000","last_fullsized":"2023-10-06T12:52:52.576894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.573678+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:52.573678+0000","last_peered":"2023-10-06T12:52:52.573678+0000","last_clean":"2023-10-06T12:52:52.573678+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:52.573678+0000","last_undegraded":"2023-10-06T12:52:52.573678+0000","last_fullsized":"2023-10-06T12:52:52.573678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.616802+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:52.616802+0000","last_peered":"2023-10-06T12:52:52.616802+0000","last_clean":"2023-10-06T12:52:52.616802+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:52.616802+0000","last_undegraded":"2023-10-06T12:52:52.616802+0000","last_fullsized":"2023-10-06T12:52:52.616802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.573947+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:52.573947+0000","last_peered":"2023-10-06T12:52:52.573947+0000","last_clean":"2023-10-06T12:52:52.573947+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:52.573947+0000","last_undegraded":"2023-10-06T12:52:52.573947+0000","last_fullsized":"2023-10-06T12:52:52.573947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":163,"reported_epoch":199,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.062681+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:53.062681+0000","last_peered":"2023-10-06T12:52:53.062681+0000","last_clean":"2023-10-06T12:52:53.062681+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:53.062681+0000","last_undegraded":"2023-10-06T12:52:53.062681+0000","last_fullsized":"2023-10-06T12:52:53.062681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575974,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304400,"kb_used_data":3016,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067440,"statfs":{"total":96636764160,"available":96325058560,"internally_reserved":0,"allocated":3088384,"data_stored":1919769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":0.871},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.0009999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.143},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.101}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.2629999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":1.6439999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.506},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.9179999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":2.0179999999999998},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.7509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575974,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300160,"kb_used_data":2752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071680,"statfs":{"total":96636764160,"available":96329400320,"internally_reserved":0,"allocated":2818048,"data_stored":1649849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.004}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.661},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.431}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.502}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":1.583},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.73}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.8169999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312032,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059808,"statfs":{"total":96636764160,"available":96317243392,"internally_reserved":0,"allocated":2514944,"data_stored":1336193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":0.97099999999999997}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":1.204},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.365},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":1.542}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":0.877},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.236}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.702},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.272}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312032,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059808,"statfs":{"total":96636764160,"available":96317243392,"internally_reserved":0,"allocated":2514944,"data_stored":1336205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":1.1830000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.024}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.399},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.6679999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.615}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.486}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.833},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.8819999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575976,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300068,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071772,"statfs":{"total":96636764160,"available":96329494528,"internally_reserved":0,"allocated":2846720,"data_stored":1665432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.014}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":0.87},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.133}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.071},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.5469999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.4850000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.7629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575974,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312348,"kb_used_data":2772,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059492,"statfs":{"total":96636764160,"available":96316919808,"internally_reserved":0,"allocated":2838528,"data_stored":1665274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.79500000000000004}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":1.155},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":1.286}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":1.105},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":0.88}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":1.236},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":1.329}]}]},{"osd":1,"up_from":13,"seq":55834575974,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312344,"kb_used_data":2768,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059496,"statfs":{"total":96636764160,"available":96316923904,"internally_reserved":0,"allocated":2834432,"data_stored":1664720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.1950000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.167},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.3100000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.351}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.421},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.55}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.6279999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575975,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304428,"kb_used_data":3044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067412,"statfs":{"total":96636764160,"available":96325029888,"internally_reserved":0,"allocated":3117056,"data_stored":1928108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":0.98199999999999998}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":1.091},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.377}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":1.048}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.347}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.484}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:53:00.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:53:00.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.595+0000 7fcfacd85700 1 Processor -- start 2023-10-06T12:53:00.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.595+0000 7fcfacd85700 1 -- start start 2023-10-06T12:53:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfacd85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 0x7fcfa8161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfacd85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcfa80c1d80 0x7fcfa80c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfacd85700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa80c18f0 con 0x7fcfa806cc30 2023-10-06T12:53:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfacd85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa80c0ce0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfacd85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80c0fc0 con 0x7fcfa8161470 2023-10-06T12:53:00.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcfa80c1d80 0x7fcfa80c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcfa80c1d80 0x7fcfa80c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55602/0 (socket says 172.21.15.46:55602) 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/3793270352 learned_addr learned my addr 172.21.15.46:0/3793270352 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa659c700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 0x7fcfa8161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/3793270352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 msgr2=0x7fcfa8161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 0x7fcfa8161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/3793270352 >> v1:172.21.15.132:6789/0 conn(0x7fcfa806cc30 legacy=0x7fcfa815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/3793270352 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80c12a0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa659c700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 0x7fcfa8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:00.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa5d9b700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa80c21b0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0099b0 tx=0x7fcf9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952f2ad642cb01f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa559a700 1 -- 172.21.15.46:0/3793270352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00b8f0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa559a700 1 -- 172.21.15.46:0/3793270352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c00ba50 con 0x7fcfa80c1d80 2023-10-06T12:53:00.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.599+0000 7fcfa559a700 1 -- 172.21.15.46:0/3793270352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00bdf0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 -- 172.21.15.46:0/3793270352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 msgr2=0x7fcfa80c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa80c21b0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0099b0 tx=0x7fcf9c009250 comp rx=0 tx=0).stop 2023-10-06T12:53:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 -- 172.21.15.46:0/3793270352 shutdown_connections 2023-10-06T12:53:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa80c21b0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 --2- 172.21.15.46:0/3793270352 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa8161470 0x7fcfa8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 -- 172.21.15.46:0/3793270352 >> 172.21.15.46:0/3793270352 conn(0x7fcfa80cbe30 msgr2=0x7fcfa80cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 -- 172.21.15.46:0/3793270352 shutdown_connections 2023-10-06T12:53:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 -- 172.21.15.46:0/3793270352 wait complete. 2023-10-06T12:53:00.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.603+0000 7fcfacd85700 1 Processor -- start 2023-10-06T12:53:00.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 -- start start 2023-10-06T12:53:00.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa806cc30 0x7fcfa8167930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa8161470 0x7fcfa815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa81626a0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80afe00 con 0x7fcfa8161470 2023-10-06T12:53:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfacd85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80b03c0 con 0x7fcfa806cc30 2023-10-06T12:53:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55608/0 (socket says 172.21.15.46:55608) 2023-10-06T12:53:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/4261931401 learned_addr learned my addr 172.21.15.46:0/4261931401 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa659c700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa806cc30 0x7fcfa8167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa6d9d700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa8161470 0x7fcfa815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa806cc30 msgr2=0x7fcfa8167930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa806cc30 0x7fcfa8167930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa8161470 msgr2=0x7fcfa815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa8161470 0x7fcfa815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 -- 172.21.15.46:0/4261931401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0094f0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.607+0000 7fcfa5d9b700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0090b0 tx=0x7fcf9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c011e00 con 0x7fcfa80c1d80 2023-10-06T12:53:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c01f070 con 0x7fcfa80c1d80 2023-10-06T12:53:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0266d0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa815dd70 con 0x7fcfa80c1d80 2023-10-06T12:53:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8162bd0 con 0x7fcfa80c1d80 2023-10-06T12:53:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa80b0550 con 0x7fcfa80c1d80 2023-10-06T12:53:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.611+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fcf9c026a40 con 0x7fcfa80c1d80 2023-10-06T12:53:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.615+0000 7fcf967fc700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 0x7fcf88062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.615+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fcf9c021070 con 0x7fcfa80c1d80 2023-10-06T12:53:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.615+0000 7fcfa659c700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 0x7fcf88062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.615+0000 7fcfa659c700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 0x7fcf88062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfa816b4e0 tx=0x7fcf98009980 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.619+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcf9c04e600 con 0x7fcfa80c1d80 2023-10-06T12:53:00.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.787+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfa8063f90 con 0x7fcf88060670 2023-10-06T12:53:00.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.791+0000 7fcf967fc700 1 -- 172.21.15.46:0/4261931401 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+69994 (secure 0 0 0) 0x7fcfa8063f90 con 0x7fcf88060670 2023-10-06T12:53:00.797 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 msgr2=0x7fcf88062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 0x7fcf88062b20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfa816b4e0 tx=0x7fcf98009980 comp rx=0 tx=0).stop 2023-10-06T12:53:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 msgr2=0x7fcfa815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0090b0 tx=0x7fcf9c017000 comp rx=0 tx=0).stop 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 shutdown_connections 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fcf88060670 0x7fcf88062b20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa8161470 0x7fcfa815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfa80c1d80 0x7fcfa815d830 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 --2- 172.21.15.46:0/4261931401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfa806cc30 0x7fcfa8167930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 >> 172.21.15.46:0/4261931401 conn(0x7fcfa80cbe30 msgr2=0x7fcfa80ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 shutdown_connections 2023-10-06T12:53:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.795+0000 7fcfacd85700 1 -- 172.21.15.46:0/4261931401 wait complete. 2023-10-06T12:53:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:53:00.820 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-06T12:53:00.590425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460744,"kb_used_data":21000,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752513976,"statfs":{"total":773094113280,"available":770574311424,"internally_reserved":0,"allocated":21504000,"data_stored":11994528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663452,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-216,"num_read_kb":-190,"num_write":-216,"num_write_kb":-1394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.989555"},"pg_stats":[{"pgid":"1.0","version":"199'370","reported_seq":1213,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061246+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:53.061246+0000","last_peered":"2023-10-06T12:52:53.061246+0000","last_clean":"2023-10-06T12:52:53.061246+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:53.061246+0000","last_undegraded":"2023-10-06T12:52:53.061246+0000","last_fullsized":"2023-10-06T12:52:53.061246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.900458+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:54.900458+0000","last_peered":"2023-10-06T12:52:54.900458+0000","last_clean":"2023-10-06T12:52:54.900458+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:54.900458+0000","last_undegraded":"2023-10-06T12:52:54.900458+0000","last_fullsized":"2023-10-06T12:52:54.900458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061057+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:53.061057+0000","last_peered":"2023-10-06T12:52:53.061057+0000","last_clean":"2023-10-06T12:52:53.061057+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:53.061057+0000","last_undegraded":"2023-10-06T12:52:53.061057+0000","last_fullsized":"2023-10-06T12:52:53.061057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.853344+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:52.853344+0000","last_peered":"2023-10-06T12:52:52.853344+0000","last_clean":"2023-10-06T12:52:52.853344+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:52.853344+0000","last_undegraded":"2023-10-06T12:52:52.853344+0000","last_fullsized":"2023-10-06T12:52:52.853344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.899945+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:54.899945+0000","last_peered":"2023-10-06T12:52:54.899945+0000","last_clean":"2023-10-06T12:52:54.899945+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:54.899945+0000","last_undegraded":"2023-10-06T12:52:54.899945+0000","last_fullsized":"2023-10-06T12:52:54.899945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119154+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:53.119154+0000","last_peered":"2023-10-06T12:52:53.119154+0000","last_clean":"2023-10-06T12:52:53.119154+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:53.119154+0000","last_undegraded":"2023-10-06T12:52:53.119154+0000","last_fullsized":"2023-10-06T12:52:53.119154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.060861+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:53.060861+0000","last_peered":"2023-10-06T12:52:53.060861+0000","last_clean":"2023-10-06T12:52:53.060861+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:53.060861+0000","last_undegraded":"2023-10-06T12:52:53.060861+0000","last_fullsized":"2023-10-06T12:52:53.060861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119032+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:53.119032+0000","last_peered":"2023-10-06T12:52:53.119032+0000","last_clean":"2023-10-06T12:52:53.119032+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:53.119032+0000","last_undegraded":"2023-10-06T12:52:53.119032+0000","last_fullsized":"2023-10-06T12:52:53.119032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.118917+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:53.118917+0000","last_peered":"2023-10-06T12:52:53.118917+0000","last_clean":"2023-10-06T12:52:53.118917+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:53.118917+0000","last_undegraded":"2023-10-06T12:52:53.118917+0000","last_fullsized":"2023-10-06T12:52:53.118917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.006}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.147}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.365},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.327},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":1.446}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.518},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.5549999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.8700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":1.5},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.998}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.788},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.4430000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.675},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.611}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":1.583},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.546}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.647}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":1.175},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":1.456}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0269999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":1.556},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.1240000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.282},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.329}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":1.508}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.373},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.7470000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.494},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.21},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.321}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.169},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.103}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.139},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.355},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.266}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":1.236},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.034}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.365}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.3979999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.353},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.482}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.778}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.865},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.3}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316152,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055688,"statfs":{"total":96636764160,"available":96313024512,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":4.9249999999999998},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":5.3360000000000003}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":5.3860000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":5.282}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":5.4790000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":5.0369999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":6.5890000000000004},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":6.2069999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":7.5170000000000003},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":10.026999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":5.569},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":7.556}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":6.6390000000000002},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":5.4370000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.3149999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":1.258},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.1839999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.4550000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.603},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.669}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.3640000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":1.083}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.353}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.474},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":1.504}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.298},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.595}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:53:00.822 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T12:53:00.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:53:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.967+0000 7f5ad88cd700 1 Processor -- start 2023-10-06T12:53:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.967+0000 7f5ad88cd700 1 -- start start 2023-10-06T12:53:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ad415ab60 0x7f5ad406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5ad415af30 0x7f5ad406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad406d460 con 0x7f5ad415ab60 2023-10-06T12:53:00.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad40c7000 con 0x7f5ad415af30 2023-10-06T12:53:00.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad88cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40c72c0 con 0x7f5ad40ad6c0 2023-10-06T12:53:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ad415ab60 0x7f5ad406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ad415ab60 0x7f5ad406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55622/0 (socket says 172.21.15.46:55622) 2023-10-06T12:53:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 -- 172.21.15.46:0/1880657917 learned_addr learned my addr 172.21.15.46:0/1880657917 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad1d9b700 1 --2- 172.21.15.46:0/1880657917 >> v2:172.21.15.132:3300/0 conn(0x7f5ad415af30 0x7f5ad406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad2d9d700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 -- 172.21.15.46:0/1880657917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 msgr2=0x7f5ad40adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 -- 172.21.15.46:0/1880657917 >> v2:172.21.15.132:3300/0 conn(0x7f5ad415af30 msgr2=0x7f5ad406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 --2- 172.21.15.46:0/1880657917 >> v2:172.21.15.132:3300/0 conn(0x7f5ad415af30 0x7f5ad406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 -- 172.21.15.46:0/1880657917 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40c75a0 con 0x7f5ad415ab60 2023-10-06T12:53:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad2d9d700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.971+0000 7f5ad259c700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad406e8a0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406f0b0 tx=0x7f5ac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4c788af9cc55a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad159a700 1 -- 172.21.15.46:0/1880657917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4009e60 con 0x7f5ad415ab60 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad159a700 1 -- 172.21.15.46:0/1880657917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac400b860 con 0x7f5ad415ab60 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad159a700 1 -- 172.21.15.46:0/1880657917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac400bc20 con 0x7f5ad415ab60 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- 172.21.15.46:0/1880657917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 msgr2=0x7f5ad406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad406e8a0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406f0b0 tx=0x7f5ac4009250 comp rx=0 tx=0).stop 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- 172.21.15.46:0/1880657917 shutdown_connections 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/1880657917 >> v2:172.21.15.132:3300/0 conn(0x7f5ad415af30 0x7f5ad406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/1880657917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad406e8a0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- 172.21.15.46:0/1880657917 >> 172.21.15.46:0/1880657917 conn(0x7f5ad40cbe30 msgr2=0x7f5ad40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- 172.21.15.46:0/1880657917 shutdown_connections 2023-10-06T12:53:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- 172.21.15.46:0/1880657917 wait complete. 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 Processor -- start 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- start start 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad415af30 0x7f5ad406a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad413eb90 con 0x7f5ad415ab60 2023-10-06T12:53:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad413df80 con 0x7f5ad415af30 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.975+0000 7f5ad88cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad413e260 con 0x7f5ad40ad6c0 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55630/0 (socket says 172.21.15.46:55630) 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 -- 172.21.15.46:0/3870225982 learned_addr learned my addr 172.21.15.46:0/3870225982 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad2d9d700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad415af30 0x7f5ad406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad259c700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 -- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 msgr2=0x7f5ad40b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 -- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad415af30 msgr2=0x7f5ad406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad415af30 0x7f5ad406a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 -- 172.21.15.46:0/3870225982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac4009490 con 0x7f5ad415ab60 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad1d9b700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800ec80 tx=0x7f5ac80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800c9d0 con 0x7f5ad415ab60 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac800cb30 con 0x7f5ad415ab60 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad413e540 con 0x7f5ad415ab60 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8013470 con 0x7f5ad415ab60 2023-10-06T12:53:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad40b0b60 con 0x7f5ad415ab60 2023-10-06T12:53:00.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.979+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad40b0de0 con 0x7f5ad415ab60 2023-10-06T12:53:00.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.983+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f5ac8013820 con 0x7f5ad415ab60 2023-10-06T12:53:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.983+0000 7f5ac27fc700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 0x7f5ab4062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.983+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f5ac807d080 con 0x7f5ad415ab60 2023-10-06T12:53:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.983+0000 7f5ad259c700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 0x7f5ab4062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.983+0000 7f5ad259c700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 0x7f5ab4062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406f0b0 tx=0x7f5ac4017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:00.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:00.987+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ac80acbe0 con 0x7f5ad415ab60 2023-10-06T12:53:01.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.155+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ad4063f90 con 0x7f5ab4060670 2023-10-06T12:53:01.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.159+0000 7f5ac27fc700 1 -- 172.21.15.46:0/3870225982 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+69994 (secure 0 0 0) 0x7f5ad4063f90 con 0x7f5ab4060670 2023-10-06T12:53:01.165 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.163+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 msgr2=0x7f5ab4062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.163+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 0x7f5ab4062b20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406f0b0 tx=0x7f5ac4017000 comp rx=0 tx=0).stop 2023-10-06T12:53:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.163+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 msgr2=0x7f5ad4069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.163+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800ec80 tx=0x7f5ac80091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 shutdown_connections 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f5ab4060670 0x7f5ab4062b20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad415af30 0x7f5ad406a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ad415ab60 0x7f5ad4069c50 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 --2- 172.21.15.46:0/3870225982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad40ad6c0 0x7f5ad40b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 >> 172.21.15.46:0/3870225982 conn(0x7f5ad40cbe30 msgr2=0x7f5ad4068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:01.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 shutdown_connections 2023-10-06T12:53:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.167+0000 7f5ad88cd700 1 -- 172.21.15.46:0/3870225982 wait complete. 2023-10-06T12:53:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:53:01.190 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-10-06T12:53:00.590425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460744,"kb_used_data":21000,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752513976,"statfs":{"total":773094113280,"available":770574311424,"internally_reserved":0,"allocated":21504000,"data_stored":11994528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-663452,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-216,"num_read_kb":-190,"num_write":-216,"num_write_kb":-1394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.989555"},"pg_stats":[{"pgid":"1.0","version":"199'370","reported_seq":1213,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061246+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:53.061246+0000","last_peered":"2023-10-06T12:52:53.061246+0000","last_clean":"2023-10-06T12:52:53.061246+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:53.061246+0000","last_undegraded":"2023-10-06T12:52:53.061246+0000","last_fullsized":"2023-10-06T12:52:53.061246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.900458+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:54.900458+0000","last_peered":"2023-10-06T12:52:54.900458+0000","last_clean":"2023-10-06T12:52:54.900458+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:54.900458+0000","last_undegraded":"2023-10-06T12:52:54.900458+0000","last_fullsized":"2023-10-06T12:52:54.900458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061057+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:53.061057+0000","last_peered":"2023-10-06T12:52:53.061057+0000","last_clean":"2023-10-06T12:52:53.061057+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:53.061057+0000","last_undegraded":"2023-10-06T12:52:53.061057+0000","last_fullsized":"2023-10-06T12:52:53.061057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.853344+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:52.853344+0000","last_peered":"2023-10-06T12:52:52.853344+0000","last_clean":"2023-10-06T12:52:52.853344+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:52.853344+0000","last_undegraded":"2023-10-06T12:52:52.853344+0000","last_fullsized":"2023-10-06T12:52:52.853344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.899945+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:54.899945+0000","last_peered":"2023-10-06T12:52:54.899945+0000","last_clean":"2023-10-06T12:52:54.899945+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:54.899945+0000","last_undegraded":"2023-10-06T12:52:54.899945+0000","last_fullsized":"2023-10-06T12:52:54.899945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119154+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:53.119154+0000","last_peered":"2023-10-06T12:52:53.119154+0000","last_clean":"2023-10-06T12:52:53.119154+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:53.119154+0000","last_undegraded":"2023-10-06T12:52:53.119154+0000","last_fullsized":"2023-10-06T12:52:53.119154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.060861+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:53.060861+0000","last_peered":"2023-10-06T12:52:53.060861+0000","last_clean":"2023-10-06T12:52:53.060861+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:53.060861+0000","last_undegraded":"2023-10-06T12:52:53.060861+0000","last_fullsized":"2023-10-06T12:52:53.060861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119032+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:53.119032+0000","last_peered":"2023-10-06T12:52:53.119032+0000","last_clean":"2023-10-06T12:52:53.119032+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:53.119032+0000","last_undegraded":"2023-10-06T12:52:53.119032+0000","last_fullsized":"2023-10-06T12:52:53.119032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.118917+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:53.118917+0000","last_peered":"2023-10-06T12:52:53.118917+0000","last_clean":"2023-10-06T12:52:53.118917+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:53.118917+0000","last_undegraded":"2023-10-06T12:52:53.118917+0000","last_fullsized":"2023-10-06T12:52:53.118917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.006}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.147}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.365},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.327},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":1.446}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.518},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.5549999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.8700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":1.5},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.998}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.788},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.4430000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.675},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.611}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":1.583},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.546}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.647}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":1.175},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":1.456}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0269999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":1.556},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.1240000000000001}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.282},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.329}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.393},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":1.508}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.373},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.7470000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.494},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.21},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.321}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.169},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.103}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.139},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.355},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.266}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":1.236},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.034}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.365}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.3979999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.353},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.482}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.778}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.865},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.3}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316152,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055688,"statfs":{"total":96636764160,"available":96313024512,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":4.9249999999999998},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":5.3360000000000003}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":5.3860000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":5.282}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":5.4790000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":5.0369999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":6.5890000000000004},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":6.2069999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":7.5170000000000003},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":10.026999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":5.569},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":7.556}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":6.6390000000000002},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":5.4370000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.3149999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":1.258},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.1839999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.4550000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.603},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.669}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.3640000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":1.083}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.353}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.474},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":1.504}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.298},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.595}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:53:01.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T12:53:01.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 Processor -- start 2023-10-06T12:53:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 -- start start 2023-10-06T12:53:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32c4067760 0x7f32c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c415d6c0 con 0x7f32c41591a0 2023-10-06T12:53:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c415cd90 con 0x7f32c4158dd0 2023-10-06T12:53:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32cb123700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c40699c0 con 0x7f32c4067760 2023-10-06T12:53:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32c4067760 0x7f32c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32c4067760 0x7f32c4068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55646/0 (socket says 172.21.15.46:55646) 2023-10-06T12:53:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 -- 172.21.15.46:0/640536122 learned_addr learned my addr 172.21.15.46:0/640536122 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32ca922700 1 --1- 172.21.15.46:0/640536122 >> v1:172.21.15.46:6789/0 conn(0x7f32c41591a0 0x7f32c4069130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43718/0 (socket says 172.21.15.46:43718) 2023-10-06T12:53:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 -- 172.21.15.46:0/640536122 >> v1:172.21.15.132:6789/0 conn(0x7f32c4158dd0 legacy=0x7f32c4067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 -- 172.21.15.46:0/640536122 >> v1:172.21.15.46:6789/0 conn(0x7f32c41591a0 legacy=0x7f32c4069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.347+0000 7f32c9920700 1 -- 172.21.15.46:0/640536122 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c415d070 con 0x7f32c4067760 2023-10-06T12:53:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32c9920700 1 --2- 172.21.15.46:0/640536122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c4067760 0x7f32c4068bf0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f32b40099b0 tx=0x7f32b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04814a11768c65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32c911f700 1 -- 172.21.15.46:0/640536122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400b860 con 0x7f32c4067760 2023-10-06T12:53:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32c911f700 1 -- 172.21.15.46:0/640536122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b400b9c0 con 0x7f32c4067760 2023-10-06T12:53:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32c911f700 1 -- 172.21.15.46:0/640536122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400bd60 con 0x7f32c4067760 2023-10-06T12:53:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 -- 172.21.15.46:0/640536122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c4067760 msgr2=0x7f32c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 --2- 172.21.15.46:0/640536122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c4067760 0x7f32c4068bf0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f32b40099b0 tx=0x7f32b4009250 comp rx=0 tx=0).stop 2023-10-06T12:53:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 -- 172.21.15.46:0/640536122 shutdown_connections 2023-10-06T12:53:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 --2- 172.21.15.46:0/640536122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c4067760 0x7f32c4068bf0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 -- 172.21.15.46:0/640536122 >> 172.21.15.46:0/640536122 conn(0x7f32c4062ef0 msgr2=0x7f32c40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 -- 172.21.15.46:0/640536122 shutdown_connections 2023-10-06T12:53:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.351+0000 7f32cb123700 1 -- 172.21.15.46:0/640536122 wait complete. 2023-10-06T12:53:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 Processor -- start 2023-10-06T12:53:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 -- start start 2023-10-06T12:53:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c4067760 0x7f32c40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c4158dd0 0x7f32c40acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40ad9e0 con 0x7f32c41591a0 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40ad710 con 0x7f32c4067760 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32cb123700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c40c2cf0 con 0x7f32c4158dd0 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32ca922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32ca922700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55650/0 (socket says 172.21.15.46:55650) 2023-10-06T12:53:01.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32ca922700 1 -- 172.21.15.46:0/2187207635 learned_addr learned my addr 172.21.15.46:0/2187207635 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32ca121700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c4067760 0x7f32c40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.355+0000 7f32c9920700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c4158dd0 0x7f32c40acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 -- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c4158dd0 msgr2=0x7f32c40acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c4158dd0 0x7f32c40acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 -- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c4067760 msgr2=0x7f32c40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c4067760 0x7f32c40b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 -- 172.21.15.46:0/2187207635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40094f0 con 0x7f32c41591a0 2023-10-06T12:53:01.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ca922700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00cc70 tx=0x7f32bc001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc005df0 con 0x7f32c41591a0 2023-10-06T12:53:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc00a5b0 con 0x7f32c41591a0 2023-10-06T12:53:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00a930 con 0x7f32c41591a0 2023-10-06T12:53:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c40c2fd0 con 0x7f32c41591a0 2023-10-06T12:53:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c40c3590 con 0x7f32c41591a0 2023-10-06T12:53:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.359+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c4064590 con 0x7f32c41591a0 2023-10-06T12:53:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.363+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f32bc00ace0 con 0x7f32c41591a0 2023-10-06T12:53:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.363+0000 7f32ba7fc700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 0x7f32a8062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.363+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f32bc07ccd0 con 0x7f32c41591a0 2023-10-06T12:53:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.363+0000 7f32ca121700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 0x7f32a8062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:01.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.363+0000 7f32ca121700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 0x7f32a8062b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32c0003a80 tx=0x7f32c0007ec0 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:01.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.371+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f32bc04ce70 con 0x7f32c41591a0 2023-10-06T12:53:01.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.539+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32c404b5c0 con 0x7f32c41591a0 2023-10-06T12:53:01.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.543+0000 7f32ba7fc700 1 -- 172.21.15.46:0/2187207635 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+15590 (secure 0 0 0) 0x7f32bc050490 con 0x7f32c41591a0 2023-10-06T12:53:01.547 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:01.547 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":202,"fsid":"36b5f543-5934-4e63-898c-253575113a16","created":"2023-10-06T11:17:57.281942+0000","modified":"2023-10-06T12:52:55.893898+0000","last_up_change":"2023-10-06T11:18:09.492152+0000","last_in_change":"2023-10-06T11:18:00.203519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T11:18:10.565075+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T11:18:16.414619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba6dfc44-0d94-4db5-bc84-34c7777dc7e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":33078},{"type":"v1","addr":"172.21.15.46:6825","nonce":33078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":33078},{"type":"v1","addr":"172.21.15.46:6827","nonce":33078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":33078},{"type":"v1","addr":"172.21.15.46:6831","nonce":33078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":33078},{"type":"v1","addr":"172.21.15.46:6829","nonce":33078}]},"public_addr":"172.21.15.46:6825/33078","cluster_addr":"172.21.15.46:6827/33078","heartbeat_back_addr":"172.21.15.46:6831/33078","heartbeat_front_addr":"172.21.15.46:6829/33078","state":["exists","up"]},{"osd":1,"uuid":"fe46237c-5610-4359-a899-959b16e568ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":33081},{"type":"v1","addr":"172.21.15.46:6812","nonce":33081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":33081},{"type":"v1","addr":"172.21.15.46:6816","nonce":33081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":33081},{"type":"v1","addr":"172.21.15.46:6823","nonce":33081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":33081},{"type":"v1","addr":"172.21.15.46:6820","nonce":33081}]},"public_addr":"172.21.15.46:6812/33081","cluster_addr":"172.21.15.46:6816/33081","heartbeat_back_addr":"172.21.15.46:6823/33081","heartbeat_front_addr":"172.21.15.46:6820/33081","state":["exists","up"]},{"osd":2,"uuid":"9362b61f-06e8-402e-9afa-754a1cc5f1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":33077},{"type":"v1","addr":"172.21.15.46:6801","nonce":33077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":33077},{"type":"v1","addr":"172.21.15.46:6803","nonce":33077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6807","nonce":33077},{"type":"v1","addr":"172.21.15.46:6808","nonce":33077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":33077},{"type":"v1","addr":"172.21.15.46:6805","nonce":33077}]},"public_addr":"172.21.15.46:6801/33077","cluster_addr":"172.21.15.46:6803/33077","heartbeat_back_addr":"172.21.15.46:6808/33077","heartbeat_front_addr":"172.21.15.46:6805/33077","state":["exists","up"]},{"osd":3,"uuid":"7d6b6c9d-807d-4f23-be87-3e08f3536612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":33080},{"type":"v1","addr":"172.21.15.46:6809","nonce":33080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6811","nonce":33080},{"type":"v1","addr":"172.21.15.46:6813","nonce":33080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":33080},{"type":"v1","addr":"172.21.15.46:6821","nonce":33080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":33080},{"type":"v1","addr":"172.21.15.46:6817","nonce":33080}]},"public_addr":"172.21.15.46:6809/33080","cluster_addr":"172.21.15.46:6813/33080","heartbeat_back_addr":"172.21.15.46:6821/33080","heartbeat_front_addr":"172.21.15.46:6817/33080","state":["exists","up"]},{"osd":4,"uuid":"d3bb5b7f-c6c9-4975-99ad-8585b870fe34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":33423},{"type":"v1","addr":"172.21.15.132:6801","nonce":33423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":33423},{"type":"v1","addr":"172.21.15.132:6803","nonce":33423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":33423},{"type":"v1","addr":"172.21.15.132:6807","nonce":33423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":33423},{"type":"v1","addr":"172.21.15.132:6805","nonce":33423}]},"public_addr":"172.21.15.132:6801/33423","cluster_addr":"172.21.15.132:6803/33423","heartbeat_back_addr":"172.21.15.132:6807/33423","heartbeat_front_addr":"172.21.15.132:6805/33423","state":["exists","up"]},{"osd":5,"uuid":"6cb5c16b-a678-4717-a443-5bdb45be8e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6809","nonce":33426},{"type":"v1","addr":"172.21.15.132:6812","nonce":33426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":33426},{"type":"v1","addr":"172.21.15.132:6819","nonce":33426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":33426},{"type":"v1","addr":"172.21.15.132:6830","nonce":33426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":33426},{"type":"v1","addr":"172.21.15.132:6825","nonce":33426}]},"public_addr":"172.21.15.132:6812/33426","cluster_addr":"172.21.15.132:6819/33426","heartbeat_back_addr":"172.21.15.132:6830/33426","heartbeat_front_addr":"172.21.15.132:6825/33426","state":["exists","up"]},{"osd":6,"uuid":"a60ee872-6821-4158-876c-68a1353a6c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6811","nonce":33424},{"type":"v1","addr":"172.21.15.132:6813","nonce":33424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":33424},{"type":"v1","addr":"172.21.15.132:6817","nonce":33424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":33424},{"type":"v1","addr":"172.21.15.132:6831","nonce":33424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":33424},{"type":"v1","addr":"172.21.15.132:6824","nonce":33424}]},"public_addr":"172.21.15.132:6813/33424","cluster_addr":"172.21.15.132:6817/33424","heartbeat_back_addr":"172.21.15.132:6831/33424","heartbeat_front_addr":"172.21.15.132:6824/33424","state":["exists","up"]},{"osd":7,"uuid":"19b3420f-679c-4173-baa5-e6418959fd88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":33425},{"type":"v1","addr":"172.21.15.132:6810","nonce":33425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":33425},{"type":"v1","addr":"172.21.15.132:6818","nonce":33425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":33425},{"type":"v1","addr":"172.21.15.132:6829","nonce":33425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":33425},{"type":"v1","addr":"172.21.15.132:6823","nonce":33425}]},"public_addr":"172.21.15.132:6810/33425","cluster_addr":"172.21.15.132:6818/33425","heartbeat_back_addr":"172.21.15.132:6829/33425","heartbeat_front_addr":"172.21.15.132:6823/33425","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.192508+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.250507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.107640+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.213273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.876454+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:05.857642+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.174479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:18:06.219820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2405550575":"2023-10-07T12:52:52.844581+0000","172.21.15.46:6834/2405550575":"2023-10-07T12:52:52.844581+0000","172.21.15.46:0/131996082":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/2304841460":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/2332664225":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1324755838":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1912199789":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/1393872098":"2023-10-07T12:52:52.561150+0000","172.21.15.46:6832/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1339853979":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1151621779":"2023-10-07T12:52:44.738959+0000","172.21.15.46:6833/59347":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/938799999":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/580990329":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1645444510":"2023-10-07T12:52:52.561150+0000","172.21.15.46:0/3242963477":"2023-10-07T12:52:44.738959+0000","172.21.15.46:0/1367734289":"2023-10-07T12:52:44.738959+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":2},{"begin":11,"length":2},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":31,"length":1},{"begin":36,"length":2},{"begin":40,"length":1},{"begin":44,"length":1},{"begin":49,"length":2},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":58,"length":1},{"begin":66,"length":1},{"begin":68,"length":1},{"begin":71,"length":2},{"begin":77,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":104,"length":1},{"begin":109,"length":3},{"begin":188,"length":34}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T12:53:01.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.543+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 msgr2=0x7f32a8062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.543+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 0x7f32a8062b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32c0003a80 tx=0x7f32c0007ec0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.543+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 msgr2=0x7f32c40ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.543+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00cc70 tx=0x7f32bc001a70 comp rx=0 tx=0).stop 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 shutdown_connections 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32c41591a0 0x7f32c40ad1d0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c4158dd0 0x7f32c40acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f32a8060670 0x7f32a8062b20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 --2- 172.21.15.46:0/2187207635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c4067760 0x7f32c40b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 >> 172.21.15.46:0/2187207635 conn(0x7f32c4062ef0 msgr2=0x7f32c406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 shutdown_connections 2023-10-06T12:53:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:01.547+0000 7f32cb123700 1 -- 172.21.15.46:0/2187207635 wait complete. 2023-10-06T12:53:02.570 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T12:53:02.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.739+0000 7f0f58f85700 1 Processor -- start 2023-10-06T12:53:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.743+0000 7f0f58f85700 1 -- start start 2023-10-06T12:53:02.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.743+0000 7f0f58f85700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f540b1090 0x7f0f540b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f58f85700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0f540b0500 0x7f0f540aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f58f85700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f540b2600 con 0x7f0f54097c60 2023-10-06T12:53:02.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f58f85700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f540a8ab0 con 0x7f0f540b1090 2023-10-06T12:53:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f58f85700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f540b09a0 con 0x7f0f540b0500 2023-10-06T12:53:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f51d9b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f54097c60 0x7f0f540afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43730/0 (socket says 172.21.15.46:43730) 2023-10-06T12:53:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f51d9b700 1 -- 172.21.15.46:0/3934005308 learned_addr learned my addr 172.21.15.46:0/3934005308 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 --2- 172.21.15.46:0/3934005308 >> v2:172.21.15.46:3300/0 conn(0x7f0f540b1090 0x7f0f540b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f52d9d700 1 --2- 172.21.15.46:0/3934005308 >> v2:172.21.15.132:3300/0 conn(0x7f0f540b0500 0x7f0f540aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5159a700 1 -- 172.21.15.46:0/3934005308 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671992385 0 0) 0x7f0f540b2600 con 0x7f0f54097c60 2023-10-06T12:53:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5159a700 1 -- 172.21.15.46:0/3934005308 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f3c003570 con 0x7f0f54097c60 2023-10-06T12:53:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 -- 172.21.15.46:0/3934005308 >> v2:172.21.15.132:3300/0 conn(0x7f0f540b0500 msgr2=0x7f0f540aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 --2- 172.21.15.46:0/3934005308 >> v2:172.21.15.132:3300/0 conn(0x7f0f540b0500 0x7f0f540aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 -- 172.21.15.46:0/3934005308 >> v1:172.21.15.46:6789/0 conn(0x7f0f54097c60 legacy=0x7f0f540afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 -- 172.21.15.46:0/3934005308 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f540b3a10 con 0x7f0f540b1090 2023-10-06T12:53:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5259c700 1 --2- 172.21.15.46:0/3934005308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540b1460 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0f540b3510 tx=0x7f0f440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7243f8a5d01fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5159a700 1 -- 172.21.15.46:0/3934005308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44011630 con 0x7f0f540b1090 2023-10-06T12:53:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5159a700 1 -- 172.21.15.46:0/3934005308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44011790 con 0x7f0f540b1090 2023-10-06T12:53:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.747+0000 7f0f5159a700 1 -- 172.21.15.46:0/3934005308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44011b30 con 0x7f0f540b1090 2023-10-06T12:53:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- 172.21.15.46:0/3934005308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 msgr2=0x7f0f540b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 --2- 172.21.15.46:0/3934005308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540b1460 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f0f540b3510 tx=0x7f0f440091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- 172.21.15.46:0/3934005308 shutdown_connections 2023-10-06T12:53:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 --2- 172.21.15.46:0/3934005308 >> v2:172.21.15.132:3300/0 conn(0x7f0f540b0500 0x7f0f540aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 --2- 172.21.15.46:0/3934005308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540b1460 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- 172.21.15.46:0/3934005308 >> 172.21.15.46:0/3934005308 conn(0x7f0f54067830 msgr2=0x7f0f540ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- 172.21.15.46:0/3934005308 shutdown_connections 2023-10-06T12:53:02.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- 172.21.15.46:0/3934005308 wait complete. 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 Processor -- start 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.751+0000 7f0f58f85700 1 -- start start 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f54097c60 0x7f0f5409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f540b0500 0x7f0f540ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f540ac8c0 con 0x7f0f540b1090 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54167110 con 0x7f0f54097c60 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541673e0 con 0x7f0f540b0500 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55666/0 (socket says 172.21.15.46:55666) 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f5259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f54097c60 0x7f0f5409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 -- 172.21.15.46:0/2675527632 learned_addr learned my addr 172.21.15.46:0/2675527632 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f51d9b700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f540b0500 0x7f0f540ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 -- 172.21.15.46:0/2675527632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f540b0500 msgr2=0x7f0f540ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f540b0500 0x7f0f540ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 -- 172.21.15.46:0/2675527632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f54097c60 msgr2=0x7f0f5409a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f54097c60 0x7f0f5409a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440093c0 con 0x7f0f540b1090 2023-10-06T12:53:02.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f52d9d700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540a9a00 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009e90 tx=0x7f0f4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00cc00 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c00f040 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c014480 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.755+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f540acaf0 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.759+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f540ad0b0 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.759+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f540ad2d0 con 0x7f0f540b1090 2023-10-06T12:53:02.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.759+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f0f4c0147f0 con 0x7f0f540b1090 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0f34060670 0x7f0f34062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f0f4c07dae0 con 0x7f0f540b1090 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f5259c700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0f34060670 0x7f0f34062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7f0f340675a0 0x7f0f340699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f3406a020 con 0x7f0f340675a0 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f4c082220 con 0x7f0f540b1090 2023-10-06T12:53:02.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f51d9b700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7f0f340675a0 0x7f0f340699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f5259c700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0f34060670 0x7f0f34062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009390 tx=0x7f0f44017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f51d9b700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7f0f340675a0 0x7f0f340699f0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.763+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== osd.0 v2:172.21.15.46:6824/33078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0f3406a020 con 0x7f0f340675a0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 --> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f540b3c40 con 0x7f0f340675a0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f427fc700 1 -- 172.21.15.46:0/2675527632 <== osd.0 v2:172.21.15.46:6824/33078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0f540b3c40 con 0x7f0f340675a0 2023-10-06T12:53:02.798 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:02.799 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:02.799 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7f0f340675a0 msgr2=0x7f0f340699f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6824/33078,v1:172.21.15.46:6825/33078] conn(0x7f0f340675a0 0x7f0f340699f0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0f34060670 msgr2=0x7f0f34062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f0f34060670 0x7f0f34062b20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009390 tx=0x7f0f44017000 comp rx=0 tx=0).stop 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 msgr2=0x7f0f540a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 --2- 172.21.15.46:0/2675527632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f540b1090 0x7f0f540a9a00 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009e90 tx=0x7f0f4c00b040 comp rx=0 tx=0).stop 2023-10-06T12:53:02.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f52d9d700 1 -- 172.21.15.46:0/2675527632 reap_dead start 2023-10-06T12:53:02.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 shutdown_connections 2023-10-06T12:53:02.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 >> 172.21.15.46:0/2675527632 conn(0x7f0f54067830 msgr2=0x7f0f540afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:02.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 shutdown_connections 2023-10-06T12:53:02.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.783+0000 7f0f58f85700 1 -- 172.21.15.46:0/2675527632 wait complete. 2023-10-06T12:53:02.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T12:53:02.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.971+0000 7f31391c5700 1 Processor -- start 2023-10-06T12:53:02.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.971+0000 7f31391c5700 1 -- start start 2023-10-06T12:53:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3134155870 0x7f31340b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340b25a0 con 0x7f3134068290 2023-10-06T12:53:02.978 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31340ac9e0 con 0x7f3134068660 2023-10-06T12:53:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31340ae5d0 con 0x7f3134155870 2023-10-06T12:53:02.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3132d9d700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3134068290 0x7f31340af520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43732/0 (socket says 172.21.15.46:43732) 2023-10-06T12:53:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3132d9d700 1 -- 172.21.15.46:0/1225814714 learned_addr learned my addr 172.21.15.46:0/1225814714 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f313359e700 1 --2- 172.21.15.46:0/1225814714 >> v2:172.21.15.46:3300/0 conn(0x7f3134155870 0x7f31340b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f313359e700 1 -- 172.21.15.46:0/1225814714 >> v1:172.21.15.132:6789/0 conn(0x7f3134068660 legacy=0x7f3134155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3131d9b700 1 -- 172.21.15.46:0/1225814714 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343712562 0 0) 0x7f31340b25a0 con 0x7f3134068290 2023-10-06T12:53:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f313359e700 1 -- 172.21.15.46:0/1225814714 >> v1:172.21.15.46:6789/0 conn(0x7f3134068290 legacy=0x7f31340af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f313359e700 1 -- 172.21.15.46:0/1225814714 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340b2350 con 0x7f3134155870 2023-10-06T12:53:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f313359e700 1 --2- 172.21.15.46:0/1225814714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134155870 0x7f31340b1e10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f312c00aef0 tx=0x7f312c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4191e8449f9a4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3131d9b700 1 -- 172.21.15.46:0/1225814714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00f070 con 0x7f3134155870 2023-10-06T12:53:02.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3131d9b700 1 -- 172.21.15.46:0/1225814714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00a2e0 con 0x7f3134155870 2023-10-06T12:53:02.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f3131d9b700 1 -- 172.21.15.46:0/1225814714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00cc20 con 0x7f3134155870 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 -- 172.21.15.46:0/1225814714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134155870 msgr2=0x7f31340b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.975+0000 7f31391c5700 1 --2- 172.21.15.46:0/1225814714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134155870 0x7f31340b1e10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f312c00aef0 tx=0x7f312c00a890 comp rx=0 tx=0).stop 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 -- 172.21.15.46:0/1225814714 shutdown_connections 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 --2- 172.21.15.46:0/1225814714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134155870 0x7f31340b1e10 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 -- 172.21.15.46:0/1225814714 >> 172.21.15.46:0/1225814714 conn(0x7f313406d490 msgr2=0x7f313406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:02.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 -- 172.21.15.46:0/1225814714 shutdown_connections 2023-10-06T12:53:02.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 -- 172.21.15.46:0/1225814714 wait complete. 2023-10-06T12:53:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 Processor -- start 2023-10-06T12:53:02.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.979+0000 7f31391c5700 1 -- start start 2023-10-06T12:53:02.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3134068660 0x7f31340ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 0x7f31340ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313414f870 con 0x7f3134068290 2023-10-06T12:53:02.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313415b8a0 con 0x7f3134068660 2023-10-06T12:53:02.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f31391c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31340caa50 con 0x7f3134155870 2023-10-06T12:53:02.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55686/0 (socket says 172.21.15.46:55686) 2023-10-06T12:53:02.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 -- 172.21.15.46:0/2984466642 learned_addr learned my addr 172.21.15.46:0/2984466642 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:02.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f313359e700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 0x7f31340ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f313259c700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3134068660 0x7f31340ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 -- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 msgr2=0x7f31340ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 0x7f31340ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 -- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3134068660 msgr2=0x7f31340ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:02.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3134068660 0x7f31340ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:02.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 -- 172.21.15.46:0/2984466642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c00ab30 con 0x7f3134068290 2023-10-06T12:53:02.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f313359e700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 0x7f31340ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.983+0000 7f3132d9d700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f312400c810 tx=0x7f3124007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124005c00 con 0x7f3134068290 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124005d60 con 0x7f3134068290 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124007320 con 0x7f3134068290 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31340cad30 con 0x7f3134068290 2023-10-06T12:53:02.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313415bb40 con 0x7f3134068290 2023-10-06T12:53:02.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.987+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31340af100 con 0x7f3134068290 2023-10-06T12:53:02.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.991+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f312401b3b0 con 0x7f3134068290 2023-10-06T12:53:02.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.991+0000 7f311affd700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 0x7f3120062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.991+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f312407c150 con 0x7f3134068290 2023-10-06T12:53:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.991+0000 7f313259c700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 0x7f3120062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:02.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.991+0000 7f313259c700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 0x7f3120062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3134170db0 tx=0x7f3128007e20 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:02.995+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f31340af100 con 0x7f3134068290 2023-10-06T12:53:03.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.167+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f31340af100 con 0x7f3120060670 2023-10-06T12:53:03.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.167+0000 7f311affd700 1 -- 172.21.15.46:0/2984466642 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31340af100 con 0x7f3120060670 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 msgr2=0x7f3120062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 0x7f3120062b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3134170db0 tx=0x7f3128007e20 comp rx=0 tx=0).stop 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 msgr2=0x7f313414f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f312400c810 tx=0x7f3124007d90 comp rx=0 tx=0).stop 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 shutdown_connections 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3134155870 0x7f31340ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f3120060670 0x7f3120062b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3134068660 0x7f31340ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 --2- 172.21.15.46:0/2984466642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134068290 0x7f313414f330 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 >> 172.21.15.46:0/2984466642 conn(0x7f313406d490 msgr2=0x7f3134153560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 shutdown_connections 2023-10-06T12:53:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.175+0000 7f31391c5700 1 -- 172.21.15.46:0/2984466642 wait complete. 2023-10-06T12:53:03.181 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T12:53:03.198 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T12:53:03.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 Processor -- start 2023-10-06T12:53:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 -- start start 2023-10-06T12:53:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89340b04c0 0x7f8934067050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8934067590 0x7f8934162c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340b1700 con 0x7f8934159000 2023-10-06T12:53:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340679b0 con 0x7f8934067590 2023-10-06T12:53:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f893359e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89340c9720 con 0x7f89340b04c0 2023-10-06T12:53:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.359+0000 7f8932d9d700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8934067590 0x7f8934162c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8931d9b700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89340b04c0 0x7f8934067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893259c700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8934159000 0x7f89340af160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:36268/0 (socket says 172.21.15.46:36268) 2023-10-06T12:53:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893259c700 1 -- 172.21.15.46:0/751074246 learned_addr learned my addr 172.21.15.46:0/751074246 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893159a700 1 -- 172.21.15.46:0/751074246 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617430580 0 0) 0x7f89340b1700 con 0x7f8934159000 2023-10-06T12:53:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893159a700 1 -- 172.21.15.46:0/751074246 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003570 con 0x7f8934159000 2023-10-06T12:53:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8932d9d700 1 -- 172.21.15.46:0/751074246 >> v2:172.21.15.132:3300/0 conn(0x7f89340b04c0 msgr2=0x7f8934067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8932d9d700 1 --2- 172.21.15.46:0/751074246 >> v2:172.21.15.132:3300/0 conn(0x7f89340b04c0 0x7f8934067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8932d9d700 1 -- 172.21.15.46:0/751074246 >> v1:172.21.15.132:6789/0 conn(0x7f8934159000 legacy=0x7f89340af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8932d9d700 1 -- 172.21.15.46:0/751074246 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340b0af0 con 0x7f8934067590 2023-10-06T12:53:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f8932d9d700 1 --2- 172.21.15.46:0/751074246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8934067590 0x7f8934162c80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8934064fc0 tx=0x7f892c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1f4568f790817a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893159a700 1 -- 172.21.15.46:0/751074246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c00e070 con 0x7f8934067590 2023-10-06T12:53:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893159a700 1 -- 172.21.15.46:0/751074246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c00bab0 con 0x7f8934067590 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893159a700 1 -- 172.21.15.46:0/751074246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c00be70 con 0x7f8934067590 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 -- 172.21.15.46:0/751074246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8934067590 msgr2=0x7f8934162c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 --2- 172.21.15.46:0/751074246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8934067590 0x7f8934162c80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8934064fc0 tx=0x7f892c009780 comp rx=0 tx=0).stop 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 -- 172.21.15.46:0/751074246 shutdown_connections 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 --2- 172.21.15.46:0/751074246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8934067590 0x7f8934162c80 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 --2- 172.21.15.46:0/751074246 >> v2:172.21.15.132:3300/0 conn(0x7f89340b04c0 0x7f8934067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 -- 172.21.15.46:0/751074246 >> 172.21.15.46:0/751074246 conn(0x7f89340ccd40 msgr2=0x7f89340cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 -- 172.21.15.46:0/751074246 shutdown_connections 2023-10-06T12:53:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.363+0000 7f893359e700 1 -- 172.21.15.46:0/751074246 wait complete. 2023-10-06T12:53:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 Processor -- start 2023-10-06T12:53:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 -- start start 2023-10-06T12:53:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8934067590 0x7f8934152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 0x7f893415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8934159000 0x7f89341547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340cb490 con 0x7f89340b04c0 2023-10-06T12:53:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893416d100 con 0x7f8934159000 2023-10-06T12:53:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893359e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340cb1b0 con 0x7f8934067590 2023-10-06T12:53:03.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 0x7f893415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 0x7f893415bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49026/0 (socket says 172.21.15.46:49026) 2023-10-06T12:53:03.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 -- 172.21.15.46:0/2282232101 learned_addr learned my addr 172.21.15.46:0/2282232101 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:03.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f893259c700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8934067590 0x7f8934152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8932d9d700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8934159000 0x7f89341547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 -- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8934067590 msgr2=0x7f8934152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8934067590 0x7f8934152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 -- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8934159000 msgr2=0x7f89341547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8934159000 0x7f89341547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8931d9b700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c009a20 con 0x7f89340b04c0 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.367+0000 7f8932d9d700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8934159000 0x7f89341547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:03.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f8931d9b700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 0x7f893415bde0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f892800ec80 tx=0x7f89280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800c850 con 0x7f89340b04c0 2023-10-06T12:53:03.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800c9d0 con 0x7f89340b04c0 2023-10-06T12:53:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800cd90 con 0x7f89340b04c0 2023-10-06T12:53:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893416cab0 con 0x7f89340b04c0 2023-10-06T12:53:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340cadd0 con 0x7f89340b04c0 2023-10-06T12:53:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8934064590 con 0x7f89340b04c0 2023-10-06T12:53:03.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.371+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f8928013680 con 0x7f89340b04c0 2023-10-06T12:53:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f89227fc700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8914060740 0x7f8914062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f892807cf70 con 0x7f89340b04c0 2023-10-06T12:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f893259c700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8914060740 0x7f8914062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f89227fc700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f8914067670 0x7f8914069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f891406a0f0 con 0x7f8914067670 2023-10-06T12:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f892807f470 con 0x7f89340b04c0 2023-10-06T12:53:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f8932d9d700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f8914067670 0x7f8914069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f893259c700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8914060740 0x7f8914062bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8924007fd0 tx=0x7f8924009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.375+0000 7f8932d9d700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f8914067670 0x7f8914069ac0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.379+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== osd.1 v2:172.21.15.46:6810/33081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f891406a0f0 con 0x7f8914067670 2023-10-06T12:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 --> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f893416cf30 con 0x7f8914067670 2023-10-06T12:53:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f89227fc700 1 -- 172.21.15.46:0/2282232101 <== osd.1 v2:172.21.15.46:6810/33081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f893416cf30 con 0x7f8914067670 2023-10-06T12:53:03.411 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:03.411 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:03.411 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f8914067670 msgr2=0x7f8914069ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6810/33081,v1:172.21.15.46:6812/33081] conn(0x7f8914067670 0x7f8914069ac0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8914060740 msgr2=0x7f8914062bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f8914060740 0x7f8914062bf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8924007fd0 tx=0x7f8924009040 comp rx=0 tx=0).stop 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 msgr2=0x7f893415bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 --2- 172.21.15.46:0/2282232101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89340b04c0 0x7f893415bde0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f892800ec80 tx=0x7f89280091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f8932d9d700 1 -- 172.21.15.46:0/2282232101 reap_dead start 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 shutdown_connections 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 >> 172.21.15.46:0/2282232101 conn(0x7f89340ccd40 msgr2=0x7f89340afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 shutdown_connections 2023-10-06T12:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.395+0000 7f893359e700 1 -- 172.21.15.46:0/2282232101 wait complete. 2023-10-06T12:53:03.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T12:53:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.583+0000 7f62156ae700 1 Processor -- start 2023-10-06T12:53:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.583+0000 7f62156ae700 1 -- start start 2023-10-06T12:53:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f62156ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 0x7f62100cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f62156ae700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62100ccb50 0x7f62100ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f62156ae700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100cc150 con 0x7f62100cea30 2023-10-06T12:53:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f62156ae700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100cfbb0 con 0x7f62100ccb50 2023-10-06T12:53:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f62156ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100cdbf0 con 0x7f621015c770 2023-10-06T12:53:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620f7fe700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f62100cea30 0x7f62100c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43774/0 (socket says 172.21.15.46:43774) 2023-10-06T12:53:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620f7fe700 1 -- 172.21.15.46:0/4044972558 learned_addr learned my addr 172.21.15.46:0/4044972558 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 --2- 172.21.15.46:0/4044972558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 0x7f62100cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620e7fc700 1 --2- 172.21.15.46:0/4044972558 >> v2:172.21.15.132:3300/0 conn(0x7f62100ccb50 0x7f62100ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620dffb700 1 -- 172.21.15.46:0/4044972558 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352156246 0 0) 0x7f62100cc150 con 0x7f62100cea30 2023-10-06T12:53:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620dffb700 1 -- 172.21.15.46:0/4044972558 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f8003570 con 0x7f62100cea30 2023-10-06T12:53:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 -- 172.21.15.46:0/4044972558 >> v2:172.21.15.132:3300/0 conn(0x7f62100ccb50 msgr2=0x7f62100ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 --2- 172.21.15.46:0/4044972558 >> v2:172.21.15.132:3300/0 conn(0x7f62100ccb50 0x7f62100ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 -- 172.21.15.46:0/4044972558 >> v1:172.21.15.46:6789/0 conn(0x7f62100cea30 legacy=0x7f62100c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 -- 172.21.15.46:0/4044972558 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100cee70 con 0x7f621015c770 2023-10-06T12:53:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.587+0000 7f620effd700 1 --2- 172.21.15.46:0/4044972558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 0x7f62100cad40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f62000098d0 tx=0x7f6200009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70330cf48e9b785 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f620dffb700 1 -- 172.21.15.46:0/4044972558 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200005c10 con 0x7f621015c770 2023-10-06T12:53:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f620dffb700 1 -- 172.21.15.46:0/4044972558 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6200005d70 con 0x7f621015c770 2023-10-06T12:53:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f620dffb700 1 -- 172.21.15.46:0/4044972558 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200006280 con 0x7f621015c770 2023-10-06T12:53:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 -- 172.21.15.46:0/4044972558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 msgr2=0x7f62100cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 --2- 172.21.15.46:0/4044972558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 0x7f62100cad40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f62000098d0 tx=0x7f6200009120 comp rx=0 tx=0).stop 2023-10-06T12:53:03.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 -- 172.21.15.46:0/4044972558 shutdown_connections 2023-10-06T12:53:03.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 --2- 172.21.15.46:0/4044972558 >> v2:172.21.15.132:3300/0 conn(0x7f62100ccb50 0x7f62100ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 --2- 172.21.15.46:0/4044972558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f621015c770 0x7f62100cad40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 -- 172.21.15.46:0/4044972558 >> 172.21.15.46:0/4044972558 conn(0x7f6210099480 msgr2=0x7f621006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 -- 172.21.15.46:0/4044972558 shutdown_connections 2023-10-06T12:53:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.591+0000 7f62156ae700 1 -- 172.21.15.46:0/4044972558 wait complete. 2023-10-06T12:53:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.595+0000 7f62156ae700 1 Processor -- start 2023-10-06T12:53:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.595+0000 7f62156ae700 1 -- start start 2023-10-06T12:53:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62100ccb50 0x7f6210151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 0x7f62100b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c7f60 con 0x7f62100cea30 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c7390 con 0x7f621015c770 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c7630 con 0x7f62100ccb50 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49042/0 (socket says 172.21.15.46:49042) 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 -- 172.21.15.46:0/2008404960 learned_addr learned my addr 172.21.15.46:0/2008404960 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:03.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620f7fe700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 0x7f62100b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620effd700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62100ccb50 0x7f6210151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 -- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62100ccb50 msgr2=0x7f6210151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62100ccb50 0x7f6210151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 -- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 msgr2=0x7f62100b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 0x7f62100b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 -- 172.21.15.46:0/2008404960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000093c0 con 0x7f62100cea30 2023-10-06T12:53:03.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620f7fe700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 0x7f62100b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:03.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f620e7fc700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f620400ec80 tx=0x7f62040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400c9d0 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620400cb30 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100c7910 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204013470 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.599+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62100ca040 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.603+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62100660f0 con 0x7f62100cea30 2023-10-06T12:53:03.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.603+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6204013820 con 0x7f62100cea30 2023-10-06T12:53:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.603+0000 7f61f6ffd700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 0x7f61fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.603+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f620407d080 con 0x7f62100cea30 2023-10-06T12:53:03.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.607+0000 7f620effd700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 0x7f61fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.607+0000 7f620effd700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 0x7f61fc062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6200009340 tx=0x7f6200016000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.611+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62040acbe0 con 0x7f62100cea30 2023-10-06T12:53:03.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.775+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f621004b5c0 con 0x7f61fc060670 2023-10-06T12:53:03.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.775+0000 7f61f6ffd700 1 -- 172.21.15.46:0/2008404960 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f621004b5c0 con 0x7f61fc060670 2023-10-06T12:53:03.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 msgr2=0x7f61fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 0x7f61fc062b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6200009340 tx=0x7f6200016000 comp rx=0 tx=0).stop 2023-10-06T12:53:03.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 msgr2=0x7f62100c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f620400ec80 tx=0x7f62040091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 shutdown_connections 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f61fc060670 0x7f61fc062b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621015c770 0x7f62100b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62100cea30 0x7f62100c8dd0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 --2- 172.21.15.46:0/2008404960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62100ccb50 0x7f6210151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 >> 172.21.15.46:0/2008404960 conn(0x7f6210099480 msgr2=0x7f62100cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 shutdown_connections 2023-10-06T12:53:03.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.779+0000 7f62156ae700 1 -- 172.21.15.46:0/2008404960 wait complete. 2023-10-06T12:53:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T12:53:03.804 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T12:53:03.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.971+0000 7fb7a14e3700 1 Processor -- start 2023-10-06T12:53:03.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.971+0000 7fb7a14e3700 1 -- start start 2023-10-06T12:53:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb79c1591f0 0x7fb79c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c1595c0 0x7fb79c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb79c0cc500 0x7fb79c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79c0cf360 con 0x7fb79c0cc500 2023-10-06T12:53:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79c0b03c0 con 0x7fb79c1591f0 2023-10-06T12:53:03.980 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a14e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c15dd60 con 0x7fb79c1595c0 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb79c0cc500 0x7fb79c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb79b7fe700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c1595c0 0x7fb79c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb79bfff700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb79c1591f0 0x7fb79c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb79c0cc500 0x7fb79c0afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49046/0 (socket says 172.21.15.46:49046) 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/1094125475 learned_addr learned my addr 172.21.15.46:0/1094125475 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/1094125475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c1595c0 msgr2=0x7fb79c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:03.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/1094125475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c1595c0 0x7fb79c0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/1094125475 >> v2:172.21.15.132:3300/0 conn(0x7fb79c1591f0 msgr2=0x7fb79c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/1094125475 >> v2:172.21.15.132:3300/0 conn(0x7fb79c1591f0 0x7fb79c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/1094125475 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c0cea30 con 0x7fb79c0cc500 2023-10-06T12:53:03.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.975+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/1094125475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0cc500 0x7fb79c0afe80 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0892e0 tx=0x7fb794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692572f30ea74757 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb79affd700 1 -- 172.21.15.46:0/1094125475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400e070 con 0x7fb79c0cc500 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb79affd700 1 -- 172.21.15.46:0/1094125475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400ba00 con 0x7fb79c0cc500 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb79affd700 1 -- 172.21.15.46:0/1094125475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940092e0 con 0x7fb79c0cc500 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 -- 172.21.15.46:0/1094125475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0cc500 msgr2=0x7fb79c0afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/1094125475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0cc500 0x7fb79c0afe80 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb79c0892e0 tx=0x7fb794009840 comp rx=0 tx=0).stop 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 -- 172.21.15.46:0/1094125475 shutdown_connections 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/1094125475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0cc500 0x7fb79c0afe80 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/1094125475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c1595c0 0x7fb79c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/1094125475 >> v2:172.21.15.132:3300/0 conn(0x7fb79c1591f0 0x7fb79c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 -- 172.21.15.46:0/1094125475 >> 172.21.15.46:0/1094125475 conn(0x7fb79c06ef60 msgr2=0x7fb79c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 -- 172.21.15.46:0/1094125475 shutdown_connections 2023-10-06T12:53:03.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.979+0000 7fb7a14e3700 1 -- 172.21.15.46:0/1094125475 wait complete. 2023-10-06T12:53:03.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 Processor -- start 2023-10-06T12:53:03.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 -- start start 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c0cc500 0x7fb79c151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb79c1591f0 0x7fb79c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 0x7fb79c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b3190 con 0x7fb79c0b1520 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b3470 con 0x7fb79c1591f0 2023-10-06T12:53:03.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a14e3700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0b3750 con 0x7fb79c0cc500 2023-10-06T12:53:03.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a0ce2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 0x7fb79c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb79b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb79c1591f0 0x7fb79c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a0ce2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 0x7fb79c154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49054/0 (socket says 172.21.15.46:49054) 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/2799381203 learned_addr learned my addr 172.21.15.46:0/2799381203 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.983+0000 7fb79bfff700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c0cc500 0x7fb79c151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/2799381203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c0cc500 msgr2=0x7fb79c151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb79c0cc500 0x7fb79c151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/2799381203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb79c1591f0 msgr2=0x7fb79c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb79c1591f0 0x7fb79c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009ae0 con 0x7fb79c0b1520 2023-10-06T12:53:03.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a0ce2700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 0x7fb79c154730 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c010 tx=0x7fb794018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794008ad0 con 0x7fb79c0b1520 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794008c50 con 0x7fb79c0b1520 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c0b3a30 con 0x7fb79c0b1520 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794025460 con 0x7fb79c0b1520 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c155f10 con 0x7fb79c0b1520 2023-10-06T12:53:03.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.987+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb79c16cce0 con 0x7fb79c0b1520 2023-10-06T12:53:03.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb794003040 con 0x7fb79c0b1520 2023-10-06T12:53:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb788060670 0x7fb788062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fb79407ece0 con 0x7fb79c0b1520 2023-10-06T12:53:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb79bfff700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb788060670 0x7fb788062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fb7880675a0 0x7fb7880699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb78806a020 con 0x7fb7880675a0 2023-10-06T12:53:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb79407efd0 con 0x7fb79c0b1520 2023-10-06T12:53:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.991+0000 7fb79b7fe700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fb7880675a0 0x7fb7880699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.995+0000 7fb79b7fe700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fb7880675a0 0x7fb7880699f0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.995+0000 7fb79bfff700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb788060670 0x7fb788062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d10 tx=0x7fb78c0091a0 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:03.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:03.995+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== osd.2 v2:172.21.15.46:6800/33077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb78806a020 con 0x7fb7880675a0 2023-10-06T12:53:04.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.011+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 --> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb79c064590 con 0x7fb7880675a0 2023-10-06T12:53:04.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb783fff700 1 -- 172.21.15.46:0/2799381203 <== osd.2 v2:172.21.15.46:6800/33077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb79c064590 con 0x7fb7880675a0 2023-10-06T12:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fb7880675a0 msgr2=0x7fb7880699f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6800/33077,v1:172.21.15.46:6801/33077] conn(0x7fb7880675a0 0x7fb7880699f0 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb788060670 msgr2=0x7fb788062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb788060670 0x7fb788062b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d10 tx=0x7fb78c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 msgr2=0x7fb79c154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 --2- 172.21.15.46:0/2799381203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb79c0b1520 0x7fb79c154730 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c010 tx=0x7fb794018000 comp rx=0 tx=0).stop 2023-10-06T12:53:04.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a0ce2700 1 -- 172.21.15.46:0/2799381203 reap_dead start 2023-10-06T12:53:04.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 shutdown_connections 2023-10-06T12:53:04.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 >> 172.21.15.46:0/2799381203 conn(0x7fb79c06ef60 msgr2=0x7fb79c067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 shutdown_connections 2023-10-06T12:53:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.015+0000 7fb7a14e3700 1 -- 172.21.15.46:0/2799381203 wait complete. 2023-10-06T12:53:04.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T12:53:04.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.235+0000 7f290ee6e700 1 Processor -- start 2023-10-06T12:53:04.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.235+0000 7f290ee6e700 1 -- start start 2023-10-06T12:53:04.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290ee6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 0x7f29080cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290ee6e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29080ccb50 0x7f29080ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290ee6e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29080cc150 con 0x7f29080cea30 2023-10-06T12:53:04.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290ee6e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29080cfbb0 con 0x7f29080ccb50 2023-10-06T12:53:04.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290ee6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29080cdbf0 con 0x7f290815c770 2023-10-06T12:53:04.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290d40b700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29080cea30 0x7f29080c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43780/0 (socket says 172.21.15.46:43780) 2023-10-06T12:53:04.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290d40b700 1 -- 172.21.15.46:0/126007359 learned_addr learned my addr 172.21.15.46:0/126007359 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:04.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 --2- 172.21.15.46:0/126007359 >> v2:172.21.15.132:3300/0 conn(0x7f29080ccb50 0x7f29080ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290cc0a700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 0x7f29080cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f29077fe700 1 -- 172.21.15.46:0/126007359 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522683487 0 0) 0x7f29080cc150 con 0x7f29080cea30 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f29077fe700 1 -- 172.21.15.46:0/126007359 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f8003570 con 0x7f29080cea30 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 -- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 msgr2=0x7f29080cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 0x7f29080cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 -- 172.21.15.46:0/126007359 >> v1:172.21.15.46:6789/0 conn(0x7f29080cea30 legacy=0x7f29080c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 -- 172.21.15.46:0/126007359 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29080cee70 con 0x7f29080ccb50 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f290cc0a700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 0x7f29080cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:04.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f2907fff700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29080ccb50 0x7f29080ce4f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f28f40098e0 tx=0x7f28f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20111f7a080f2b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f29077fe700 1 -- 172.21.15.46:0/126007359 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011630 con 0x7f29080ccb50 2023-10-06T12:53:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f29077fe700 1 -- 172.21.15.46:0/126007359 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4011790 con 0x7f29080ccb50 2023-10-06T12:53:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.239+0000 7f29077fe700 1 -- 172.21.15.46:0/126007359 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011b30 con 0x7f29080ccb50 2023-10-06T12:53:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 -- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29080ccb50 msgr2=0x7f29080ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29080ccb50 0x7f29080ce4f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f28f40098e0 tx=0x7f28f40091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 -- 172.21.15.46:0/126007359 shutdown_connections 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29080ccb50 0x7f29080ce4f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 --2- 172.21.15.46:0/126007359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f290815c770 0x7f29080cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 -- 172.21.15.46:0/126007359 >> 172.21.15.46:0/126007359 conn(0x7f2908099480 msgr2=0x7f290806ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 -- 172.21.15.46:0/126007359 shutdown_connections 2023-10-06T12:53:04.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.243+0000 7f290ee6e700 1 -- 172.21.15.46:0/126007359 wait complete. 2023-10-06T12:53:04.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 Processor -- start 2023-10-06T12:53:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 -- start start 2023-10-06T12:53:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29080ccb50 0x7f2908158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f290815c770 0x7f29080c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290809dda0 con 0x7f29080cea30 2023-10-06T12:53:04.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290809d190 con 0x7f290815c770 2023-10-06T12:53:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290ee6e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290809d470 con 0x7f29080ccb50 2023-10-06T12:53:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49056/0 (socket says 172.21.15.46:49056) 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 -- 172.21.15.46:0/3403610360 learned_addr learned my addr 172.21.15.46:0/3403610360 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290cc0a700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29080ccb50 0x7f2908158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f290d40b700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f290815c770 0x7f29080c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 -- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29080ccb50 msgr2=0x7f2908158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29080ccb50 0x7f2908158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 -- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f290815c770 msgr2=0x7f29080c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f290815c770 0x7f29080c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.247+0000 7f2907fff700 1 -- 172.21.15.46:0/3403610360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0093c0 con 0x7f29080cea30 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f2907fff700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f2908064b20 tx=0x7f28f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4000c40 con 0x7f29080cea30 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4011900 con 0x7f29080cea30 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4026430 con 0x7f29080cea30 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f4009420 con 0x7f29080cea30 2023-10-06T12:53:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290809da30 con 0x7f29080cea30 2023-10-06T12:53:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.251+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29080660f0 con 0x7f29080cea30 2023-10-06T12:53:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.255+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f28f4023030 con 0x7f29080cea30 2023-10-06T12:53:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.255+0000 7f2904ff9700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 0x7f28ec062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.255+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f28f407e290 con 0x7f29080cea30 2023-10-06T12:53:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.255+0000 7f290cc0a700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 0x7f28ec062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.255+0000 7f290cc0a700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 0x7f28ec062b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e00 tx=0x7f28fc009120 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.263+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f28f404e3f0 con 0x7f29080cea30 2023-10-06T12:53:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.435+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2908146710 con 0x7f28ec0606a0 2023-10-06T12:53:04.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.435+0000 7f2904ff9700 1 -- 172.21.15.46:0/3403610360 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2908146710 con 0x7f28ec0606a0 2023-10-06T12:53:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 msgr2=0x7f28ec062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 0x7f28ec062b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e00 tx=0x7f28fc009120 comp rx=0 tx=0).stop 2023-10-06T12:53:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 msgr2=0x7f29080c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f2908064b20 tx=0x7f28f4017000 comp rx=0 tx=0).stop 2023-10-06T12:53:04.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 shutdown_connections 2023-10-06T12:53:04.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f28ec0606a0 0x7f28ec062b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f290815c770 0x7f29080c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29080cea30 0x7f29080c4fe0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 --2- 172.21.15.46:0/3403610360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29080ccb50 0x7f2908158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 >> 172.21.15.46:0/3403610360 conn(0x7f2908099480 msgr2=0x7f29080c67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 shutdown_connections 2023-10-06T12:53:04.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.439+0000 7f290ee6e700 1 -- 172.21.15.46:0/3403610360 wait complete. 2023-10-06T12:53:04.446 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T12:53:04.465 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T12:53:04.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.631+0000 7f9181a16700 1 Processor -- start 2023-10-06T12:53:04.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.631+0000 7f9181a16700 1 -- start start 2023-10-06T12:53:04.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181a16700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f917c1595c0 0x7f917c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181a16700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0cc500 0x7f917c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181a16700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0cf360 con 0x7f917c1591f0 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181a16700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0cff00 con 0x7f917c1595c0 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181a16700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0b03c0 con 0x7f917c0cc500 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f917bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f917c1595c0 0x7f917c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f917bfff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f917c1595c0 0x7f917c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49068/0 (socket says 172.21.15.46:49068) 2023-10-06T12:53:04.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f917bfff700 1 -- 172.21.15.46:0/2006581247 learned_addr learned my addr 172.21.15.46:0/2006581247 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:04.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.635+0000 7f9181215700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0cc500 0x7f917c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917bfff700 1 -- 172.21.15.46:0/2006581247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0cc500 msgr2=0x7f917c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917bfff700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0cc500 0x7f917c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917bfff700 1 -- 172.21.15.46:0/2006581247 >> v1:172.21.15.132:6789/0 conn(0x7f917c1591f0 legacy=0x7f917c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917bfff700 1 -- 172.21.15.46:0/2006581247 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c15dd60 con 0x7f917c1595c0 2023-10-06T12:53:04.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917bfff700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c1595c0 0x7f917c0af940 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f91680099b0 tx=0x7f9168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011b20053351ec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917b7fe700 1 -- 172.21.15.46:0/2006581247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800b8f0 con 0x7f917c1595c0 2023-10-06T12:53:04.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917b7fe700 1 -- 172.21.15.46:0/2006581247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916800ba50 con 0x7f917c1595c0 2023-10-06T12:53:04.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f917b7fe700 1 -- 172.21.15.46:0/2006581247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800bdf0 con 0x7f917c1595c0 2023-10-06T12:53:04.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 -- 172.21.15.46:0/2006581247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c1595c0 msgr2=0x7f917c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c1595c0 0x7f917c0af940 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f91680099b0 tx=0x7f9168009250 comp rx=0 tx=0).stop 2023-10-06T12:53:04.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 -- 172.21.15.46:0/2006581247 shutdown_connections 2023-10-06T12:53:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0cc500 0x7f917c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 --2- 172.21.15.46:0/2006581247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c1595c0 0x7f917c0af940 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 -- 172.21.15.46:0/2006581247 >> 172.21.15.46:0/2006581247 conn(0x7f917c06ef60 msgr2=0x7f917c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 -- 172.21.15.46:0/2006581247 shutdown_connections 2023-10-06T12:53:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.639+0000 7f9181a16700 1 -- 172.21.15.46:0/2006581247 wait complete. 2023-10-06T12:53:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 Processor -- start 2023-10-06T12:53:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 -- start start 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cc500 0x7f917c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c1591f0 0x7f917c068ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 0x7f917c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c155040 con 0x7f917c069400 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0b4190 con 0x7f917c0cc500 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181a16700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c16d590 con 0x7f917c1591f0 2023-10-06T12:53:04.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 0x7f917c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 0x7f917c1416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49070/0 (socket says 172.21.15.46:49070) 2023-10-06T12:53:04.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 -- 172.21.15.46:0/678318621 learned_addr learned my addr 172.21.15.46:0/678318621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f917bfff700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c1591f0 0x7f917c068ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9180a14700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cc500 0x7f917c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 -- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c1591f0 msgr2=0x7f917c068ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c1591f0 0x7f917c068ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 -- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cc500 msgr2=0x7f917c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cc500 0x7f917c154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.643+0000 7f9181215700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680094f0 con 0x7f917c069400 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9181215700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 0x7f917c1416a0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9174007f20 tx=0x7f9174009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400ebf0 con 0x7f917c069400 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400ed70 con 0x7f917c069400 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c16d870 con 0x7f917c069400 2023-10-06T12:53:04.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174014670 con 0x7f917c069400 2023-10-06T12:53:04.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c16de30 con 0x7f917c069400 2023-10-06T12:53:04.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f917c158e20 con 0x7f917c069400 2023-10-06T12:53:04.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.647+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f91740149e0 con 0x7f917c069400 2023-10-06T12:53:04.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9178ff9700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f91600606a0 0x7f9160062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f917407d2d0 con 0x7f917c069400 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9180a14700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f91600606a0 0x7f9160062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9178ff9700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f91600675d0 0x7f9160069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f916006a050 con 0x7f91600675d0 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917407f830 con 0x7f917c069400 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f917bfff700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f91600675d0 0x7f9160069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f9180a14700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f91600606a0 0x7f9160062b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f917c069930 tx=0x7f9170009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.651+0000 7f917bfff700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f91600675d0 0x7f9160069a20 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.655+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== osd.3 v2:172.21.15.46:6806/33080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f916006a050 con 0x7f91600675d0 2023-10-06T12:53:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.671+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 --> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f917c064590 con 0x7f91600675d0 2023-10-06T12:53:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.671+0000 7f9178ff9700 1 -- 172.21.15.46:0/678318621 <== osd.3 v2:172.21.15.46:6806/33080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f917c064590 con 0x7f91600675d0 2023-10-06T12:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:04.677 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.671+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f91600675d0 msgr2=0x7f9160069a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.678 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.671+0000 7f9181a16700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6806/33080,v1:172.21.15.46:6809/33080] conn(0x7f91600675d0 0x7f9160069a20 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f91600606a0 msgr2=0x7f9160062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f91600606a0 0x7f9160062b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f917c069930 tx=0x7f9170009040 comp rx=0 tx=0).stop 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 msgr2=0x7f917c1416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 --2- 172.21.15.46:0/678318621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f917c069400 0x7f917c1416a0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9174007f20 tx=0x7f9174009860 comp rx=0 tx=0).stop 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181215700 1 -- 172.21.15.46:0/678318621 reap_dead start 2023-10-06T12:53:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 shutdown_connections 2023-10-06T12:53:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 >> 172.21.15.46:0/678318621 conn(0x7f917c06ef60 msgr2=0x7f917c1582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 shutdown_connections 2023-10-06T12:53:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.675+0000 7f9181a16700 1 -- 172.21.15.46:0/678318621 wait complete. 2023-10-06T12:53:04.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T12:53:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 Processor -- start 2023-10-06T12:53:04.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 -- start start 2023-10-06T12:53:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b800af3f0 0x7f4b8006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8015bb70 con 0x7f4b80156ea0 2023-10-06T12:53:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b800cab20 con 0x7f4b800b2750 2023-10-06T12:53:04.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b86078700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8015aff0 con 0x7f4b800af3f0 2023-10-06T12:53:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b7effd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b800af3f0 0x7f4b8006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b7ffff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4b800b2750 0x7f4b8006a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:36280/0 (socket says 172.21.15.46:36280) 2023-10-06T12:53:04.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b7ffff700 1 -- 172.21.15.46:0/2479842665 learned_addr learned my addr 172.21.15.46:0/2479842665 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.863+0000 7f4b7f7fe700 1 --1- 172.21.15.46:0/2479842665 >> v1:172.21.15.46:6789/0 conn(0x7f4b80156ea0 0x7f4b800b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43782/0 (socket says 172.21.15.46:43782) 2023-10-06T12:53:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7e7fc700 1 -- 172.21.15.46:0/2479842665 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463462144 0 0) 0x7f4b800cab20 con 0x7f4b800b2750 2023-10-06T12:53:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7e7fc700 1 -- 172.21.15.46:0/2479842665 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b68003570 con 0x7f4b800b2750 2023-10-06T12:53:04.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7effd700 1 -- 172.21.15.46:0/2479842665 >> v1:172.21.15.132:6789/0 conn(0x7f4b800b2750 legacy=0x7f4b8006a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7effd700 1 -- 172.21.15.46:0/2479842665 >> v1:172.21.15.46:6789/0 conn(0x7f4b80156ea0 legacy=0x7f4b800b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7effd700 1 -- 172.21.15.46:0/2479842665 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8015b2d0 con 0x7f4b800af3f0 2023-10-06T12:53:04.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7effd700 1 --2- 172.21.15.46:0/2479842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b8006a0f0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4b740098e0 tx=0x7f4b740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fd46d21474ac75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7e7fc700 1 -- 172.21.15.46:0/2479842665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400b640 con 0x7f4b800af3f0 2023-10-06T12:53:04.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7e7fc700 1 -- 172.21.15.46:0/2479842665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7400b7a0 con 0x7f4b800af3f0 2023-10-06T12:53:04.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b7e7fc700 1 -- 172.21.15.46:0/2479842665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7400bb40 con 0x7f4b800af3f0 2023-10-06T12:53:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 -- 172.21.15.46:0/2479842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 msgr2=0x7f4b8006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 --2- 172.21.15.46:0/2479842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b8006a0f0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f4b740098e0 tx=0x7f4b740091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 -- 172.21.15.46:0/2479842665 shutdown_connections 2023-10-06T12:53:04.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 --2- 172.21.15.46:0/2479842665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b8006a0f0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 -- 172.21.15.46:0/2479842665 >> 172.21.15.46:0/2479842665 conn(0x7f4b80068bf0 msgr2=0x7f4b800ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 -- 172.21.15.46:0/2479842665 shutdown_connections 2023-10-06T12:53:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.867+0000 7f4b86078700 1 -- 172.21.15.46:0/2479842665 wait complete. 2023-10-06T12:53:04.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 Processor -- start 2023-10-06T12:53:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 -- start start 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b800b2750 0x7f4b80149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b80156ea0 0x7f4b800cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8016d190 con 0x7f4b800af3f0 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8016ce90 con 0x7f4b80156ea0 2023-10-06T12:53:04.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b86078700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8016a090 con 0x7f4b800b2750 2023-10-06T12:53:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49088/0 (socket says 172.21.15.46:49088) 2023-10-06T12:53:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.871+0000 7f4b7f7fe700 1 -- 172.21.15.46:0/4270146285 learned_addr learned my addr 172.21.15.46:0/4270146285 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7ffff700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b80156ea0 0x7f4b800cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7effd700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b800b2750 0x7f4b80149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 -- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b800b2750 msgr2=0x7f4b80149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b800b2750 0x7f4b80149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 -- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b80156ea0 msgr2=0x7f4b800cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b80156ea0 0x7f4b800cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 -- 172.21.15.46:0/4270146285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009420 con 0x7f4b800af3f0 2023-10-06T12:53:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b7f7fe700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c7f0 tx=0x7f4b70007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70005a70 con 0x7f4b800af3f0 2023-10-06T12:53:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70005bf0 con 0x7f4b800af3f0 2023-10-06T12:53:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700070d0 con 0x7f4b800af3f0 2023-10-06T12:53:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8016a370 con 0x7f4b800af3f0 2023-10-06T12:53:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.875+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8016a650 con 0x7f4b800af3f0 2023-10-06T12:53:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.879+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8006cc30 con 0x7f4b800af3f0 2023-10-06T12:53:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.879+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f4b70007480 con 0x7f4b800af3f0 2023-10-06T12:53:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.879+0000 7f4b677fe700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 0x7f4b6c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.883+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f4b7007c170 con 0x7f4b800af3f0 2023-10-06T12:53:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.883+0000 7f4b7effd700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 0x7f4b6c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.883+0000 7f4b7effd700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 0x7f4b6c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b740093f0 tx=0x7f4b74017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:04.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:04.891+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4b7004c290 con 0x7f4b800af3f0 2023-10-06T12:53:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b8016cfd0 con 0x7f4b6c060670 2023-10-06T12:53:05.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b677fe700 1 -- 172.21.15.46:0/4270146285 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4b8016cfd0 con 0x7f4b6c060670 2023-10-06T12:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 msgr2=0x7f4b6c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 0x7f4b6c062b20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b740093f0 tx=0x7f4b74017000 comp rx=0 tx=0).stop 2023-10-06T12:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 msgr2=0x7f4b801697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c7f0 tx=0x7f4b70007f70 comp rx=0 tx=0).stop 2023-10-06T12:53:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 shutdown_connections 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f4b6c060670 0x7f4b6c062b20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b80156ea0 0x7f4b800cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b800b2750 0x7f4b80149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 --2- 172.21.15.46:0/4270146285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800af3f0 0x7f4b801697f0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.063+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 >> 172.21.15.46:0/4270146285 conn(0x7f4b80068bf0 msgr2=0x7f4b8015a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.067+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 shutdown_connections 2023-10-06T12:53:05.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.067+0000 7f4b86078700 1 -- 172.21.15.46:0/4270146285 wait complete. 2023-10-06T12:53:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T12:53:05.088 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T12:53:05.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.255+0000 7fa04e22e700 1 Processor -- start 2023-10-06T12:53:05.259 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.255+0000 7fa04e22e700 1 -- start start 2023-10-06T12:53:05.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa048159130 0x7fa0480d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa048069190 0x7fa0480695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480ce2c0 con 0x7fa048159500 2023-10-06T12:53:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0480afb30 con 0x7fa048069190 2023-10-06T12:53:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0480cc090 con 0x7fa048159130 2023-10-06T12:53:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa048069190 0x7fa0480695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa048069190 0x7fa0480695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49104/0 (socket says 172.21.15.46:49104) 2023-10-06T12:53:05.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 -- 172.21.15.46:0/1116751972 learned_addr learned my addr 172.21.15.46:0/1116751972 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:05.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa0477fe700 1 --2- 172.21.15.46:0/1116751972 >> v2:172.21.15.132:3300/0 conn(0x7fa048159130 0x7fa0480d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 -- 172.21.15.46:0/1116751972 >> v2:172.21.15.132:3300/0 conn(0x7fa048159130 msgr2=0x7fa0480d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 --2- 172.21.15.46:0/1116751972 >> v2:172.21.15.132:3300/0 conn(0x7fa048159130 0x7fa0480d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 -- 172.21.15.46:0/1116751972 >> v1:172.21.15.132:6789/0 conn(0x7fa048159500 legacy=0x7fa0480aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 -- 172.21.15.46:0/1116751972 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048069af0 con 0x7fa048069190 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa047fff700 1 --2- 172.21.15.46:0/1116751972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048069190 0x7fa0480695b0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fa0480892d0 tx=0x7fa040009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48115495b34c85f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa0467fc700 1 -- 172.21.15.46:0/1116751972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000e070 con 0x7fa048069190 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa0467fc700 1 -- 172.21.15.46:0/1116751972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04000ba40 con 0x7fa048069190 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa0467fc700 1 -- 172.21.15.46:0/1116751972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400092e0 con 0x7fa048069190 2023-10-06T12:53:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- 172.21.15.46:0/1116751972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048069190 msgr2=0x7fa0480695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1116751972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048069190 0x7fa0480695b0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fa0480892d0 tx=0x7fa040009840 comp rx=0 tx=0).stop 2023-10-06T12:53:05.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- 172.21.15.46:0/1116751972 shutdown_connections 2023-10-06T12:53:05.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1116751972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048069190 0x7fa0480695b0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1116751972 >> v2:172.21.15.132:3300/0 conn(0x7fa048159130 0x7fa0480d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- 172.21.15.46:0/1116751972 >> 172.21.15.46:0/1116751972 conn(0x7fa048068510 msgr2=0x7fa0480d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- 172.21.15.46:0/1116751972 shutdown_connections 2023-10-06T12:53:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.259+0000 7fa04e22e700 1 -- 172.21.15.46:0/1116751972 wait complete. 2023-10-06T12:53:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 Processor -- start 2023-10-06T12:53:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 -- start start 2023-10-06T12:53:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa048069190 0x7fa048150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa048159130 0x7fa0480b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 0x7fa0480b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481513d0 con 0x7fa048159500 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480b0090 con 0x7fa048069190 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.263+0000 7fa04e22e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480b0350 con 0x7fa048159130 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 0x7fa0480b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 0x7fa0480b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49108/0 (socket says 172.21.15.46:49108) 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 -- 172.21.15.46:0/1797383939 learned_addr learned my addr 172.21.15.46:0/1797383939 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa0477fe700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa048069190 0x7fa048150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa046ffd700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa048159130 0x7fa0480b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 -- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa048159130 msgr2=0x7fa0480b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa048159130 0x7fa0480b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 -- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa048069190 msgr2=0x7fa048150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa048069190 0x7fa048150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040009ae0 con 0x7fa048159500 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa046ffd700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa048159130 0x7fa0480b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa047fff700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 0x7fa0480b1c70 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fa040009880 tx=0x7fa040018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000be50 con 0x7fa048159500 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040008d90 con 0x7fa048159500 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400266b0 con 0x7fa048159500 2023-10-06T12:53:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0480b0630 con 0x7fa048159500 2023-10-06T12:53:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0480b0bf0 con 0x7fa048159500 2023-10-06T12:53:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.267+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa048088540 con 0x7fa048159500 2023-10-06T12:53:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa040003040 con 0x7fa048159500 2023-10-06T12:53:05.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa034060670 0x7fa034062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.277 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fa04007e7d0 con 0x7fa048159500 2023-10-06T12:53:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa0477fe700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa034060670 0x7fa034062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7fa0340675a0 0x7fa0340699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa03406a020 con 0x7fa0340675a0 2023-10-06T12:53:05.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.271+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa040080cd0 con 0x7fa048159500 2023-10-06T12:53:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.275+0000 7fa046ffd700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7fa0340675a0 0x7fa0340699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.275+0000 7fa0477fe700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa034060670 0x7fa034062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.275+0000 7fa046ffd700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7fa0340675a0 0x7fa0340699f0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.275+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== osd.4 v2:172.21.15.132:6800/33423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa03406a020 con 0x7fa0340675a0 2023-10-06T12:53:05.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 --> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0480af570 con 0x7fa0340675a0 2023-10-06T12:53:05.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa02f7fe700 1 -- 172.21.15.46:0/1797383939 <== osd.4 v2:172.21.15.132:6800/33423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa0480af570 con 0x7fa0340675a0 2023-10-06T12:53:05.300 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:05.300 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:05.300 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:05.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7fa0340675a0 msgr2=0x7fa0340699f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.132:6800/33423,v1:172.21.15.132:6801/33423] conn(0x7fa0340675a0 0x7fa0340699f0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa034060670 msgr2=0x7fa034062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa034060670 0x7fa034062b20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).stop 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 msgr2=0x7fa0480b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 --2- 172.21.15.46:0/1797383939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa048159500 0x7fa0480b1c70 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fa040009880 tx=0x7fa040018000 comp rx=0 tx=0).stop 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa047fff700 1 -- 172.21.15.46:0/1797383939 reap_dead start 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 shutdown_connections 2023-10-06T12:53:05.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.295+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 >> 172.21.15.46:0/1797383939 conn(0x7fa048068510 msgr2=0x7fa048156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.299+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 shutdown_connections 2023-10-06T12:53:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.299+0000 7fa04e22e700 1 -- 172.21.15.46:0/1797383939 wait complete. 2023-10-06T12:53:05.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T12:53:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.487+0000 7fa0f78f0700 1 Processor -- start 2023-10-06T12:53:05.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.487+0000 7fa0f78f0700 1 -- start start 2023-10-06T12:53:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f78f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f78f0700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0f0157330 0x7fa0f00cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f78f0700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00cd220 con 0x7fa0f00af9e0 2023-10-06T12:53:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f78f0700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f00cc610 con 0x7fa0f0157330 2023-10-06T12:53:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f78f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00cc8f0 con 0x7fa0f0156f60 2023-10-06T12:53:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f568c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f568c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:46596/0 (socket says 172.21.15.46:46596) 2023-10-06T12:53:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f568c700 1 -- 172.21.15.46:0/2783944020 learned_addr learned my addr 172.21.15.46:0/2783944020 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/2783944020 >> v2:172.21.15.132:3300/0 conn(0x7fa0f0157330 0x7fa0f00cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f5e8d700 1 --1- 172.21.15.46:0/2783944020 >> v1:172.21.15.132:6789/0 conn(0x7fa0f00af9e0 0x7fa0f00afe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:36290/0 (socket says 172.21.15.46:36290) 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 -- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 msgr2=0x7fa0f00cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 -- 172.21.15.46:0/2783944020 >> v1:172.21.15.132:6789/0 conn(0x7fa0f00af9e0 legacy=0x7fa0f00afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0e7fff700 1 -- 172.21.15.46:0/2783944020 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009065137 0 0) 0x7fa0f00cd220 con 0x7fa0f00af9e0 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 -- 172.21.15.46:0/2783944020 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00ccbd0 con 0x7fa0f0157330 2023-10-06T12:53:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f568c700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.491+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0157330 0x7fa0f00cbf90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098e0 tx=0x7fa0e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c62b2fdb6201631 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0e7fff700 1 -- 172.21.15.46:0/2783944020 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000b640 con 0x7fa0f0157330 2023-10-06T12:53:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0e7fff700 1 -- 172.21.15.46:0/2783944020 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e000b7a0 con 0x7fa0f0157330 2023-10-06T12:53:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0e7fff700 1 -- 172.21.15.46:0/2783944020 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000bb40 con 0x7fa0f0157330 2023-10-06T12:53:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 -- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0157330 msgr2=0x7fa0f00cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0157330 0x7fa0f00cbf90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098e0 tx=0x7fa0e00091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 -- 172.21.15.46:0/2783944020 shutdown_connections 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0157330 0x7fa0f00cbf90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/2783944020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f0156f60 0x7fa0f00cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 -- 172.21.15.46:0/2783944020 >> 172.21.15.46:0/2783944020 conn(0x7fa0f006ae60 msgr2=0x7fa0f006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 -- 172.21.15.46:0/2783944020 shutdown_connections 2023-10-06T12:53:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.495+0000 7fa0f78f0700 1 -- 172.21.15.46:0/2783944020 wait complete. 2023-10-06T12:53:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 Processor -- start 2023-10-06T12:53:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 -- start start 2023-10-06T12:53:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0156f60 0x7fa0f0146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f0157330 0x7fa0f0148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0146650 con 0x7fa0f0157330 2023-10-06T12:53:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01479d0 con 0x7fa0f0156f60 2023-10-06T12:53:05.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.499+0000 7fa0f78f0700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0146dc0 con 0x7fa0f00af9e0 2023-10-06T12:53:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f5e8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f0157330 0x7fa0f0148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.46:46608/0 (socket says 172.21.15.46:46608) 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 -- 172.21.15.46:0/3541163610 learned_addr learned my addr 172.21.15.46:0/3541163610 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0156f60 0x7fa0f0146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 -- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0156f60 msgr2=0x7fa0f0146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0156f60 0x7fa0f0146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 -- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f0157330 msgr2=0x7fa0f0148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f0157330 0x7fa0f0148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 -- 172.21.15.46:0/3541163610 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0094f0 con 0x7fa0f00af9e0 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f568c700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00ec80 tx=0x7fa0ec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00c850 con 0x7fa0f00af9e0 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00c9b0 con 0x7fa0f00af9e0 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e0009420 con 0x7fa0f00af9e0 2023-10-06T12:53:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00cd50 con 0x7fa0f00af9e0 2023-10-06T12:53:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f0147380 con 0x7fa0f00af9e0 2023-10-06T12:53:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.503+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f006da40 con 0x7fa0f00af9e0 2023-10-06T12:53:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.507+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fa0ec013680 con 0x7fa0f00af9e0 2023-10-06T12:53:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.507+0000 7fa0e57fa700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 0x7fa0d4062c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.507+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fa0ec07cc90 con 0x7fa0f00af9e0 2023-10-06T12:53:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.507+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 0x7fa0d4062c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.511+0000 7fa0f4e8b700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 0x7fa0d4062c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098e0 tx=0x7fa0e0017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.515+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa0ec04ce30 con 0x7fa0f00af9e0 2023-10-06T12:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.687+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0f00cceb0 con 0x7fa0d40607c0 2023-10-06T12:53:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.687+0000 7fa0e57fa700 1 -- 172.21.15.46:0/3541163610 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa0f00cceb0 con 0x7fa0d40607c0 2023-10-06T12:53:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 msgr2=0x7fa0d4062c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 0x7fa0d4062c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00098e0 tx=0x7fa0e0017000 comp rx=0 tx=0).stop 2023-10-06T12:53:05.696 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 msgr2=0x7fa0f01467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00ec80 tx=0x7fa0ec009250 comp rx=0 tx=0).stop 2023-10-06T12:53:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 shutdown_connections 2023-10-06T12:53:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0f0157330 0x7fa0f0148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fa0d40607c0 0x7fa0d4062c70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f0156f60 0x7fa0f0146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 --2- 172.21.15.46:0/3541163610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0f00af9e0 0x7fa0f01467e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 >> 172.21.15.46:0/3541163610 conn(0x7fa0f006ae60 msgr2=0x7fa0f00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 shutdown_connections 2023-10-06T12:53:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.691+0000 7fa0f78f0700 1 -- 172.21.15.46:0/3541163610 wait complete. 2023-10-06T12:53:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T12:53:05.714 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T12:53:05.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 Processor -- start 2023-10-06T12:53:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 -- start start 2023-10-06T12:53:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83ac1595c0 0x7f83ac0af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83ac0cc500 0x7f83ac0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ac0cf360 con 0x7f83ac1591f0 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac0cff00 con 0x7f83ac1595c0 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83ab59e700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac0b03c0 con 0x7f83ac0cc500 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83a9d9b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f83ac1595c0 0x7f83ac0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83aa59c700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83ac1591f0 0x7f83ac0cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43798/0 (socket says 172.21.15.46:43798) 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83aa59c700 1 -- 172.21.15.46:0/4085378819 learned_addr learned my addr 172.21.15.46:0/4085378819 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.879+0000 7f83aad9d700 1 --2- 172.21.15.46:0/4085378819 >> v2:172.21.15.132:3300/0 conn(0x7f83ac0cc500 0x7f83ac0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a9d9b700 1 -- 172.21.15.46:0/4085378819 >> v2:172.21.15.132:3300/0 conn(0x7f83ac0cc500 msgr2=0x7f83ac0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a9d9b700 1 --2- 172.21.15.46:0/4085378819 >> v2:172.21.15.132:3300/0 conn(0x7f83ac0cc500 0x7f83ac0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a9d9b700 1 -- 172.21.15.46:0/4085378819 >> v1:172.21.15.46:6789/0 conn(0x7f83ac1591f0 legacy=0x7f83ac0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a9d9b700 1 -- 172.21.15.46:0/4085378819 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac15dd60 con 0x7f83ac1595c0 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a959a700 1 -- 172.21.15.46:0/4085378819 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097852018 0 0) 0x7f83ac0cf360 con 0x7f83ac1591f0 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a9d9b700 1 --2- 172.21.15.46:0/4085378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac1595c0 0x7f83ac0af940 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0acdd0 tx=0x7f83a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b33b352b1b66115 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a959a700 1 -- 172.21.15.46:0/4085378819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000b690 con 0x7f83ac1595c0 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a959a700 1 -- 172.21.15.46:0/4085378819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a000b7f0 con 0x7f83ac1595c0 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83a959a700 1 -- 172.21.15.46:0/4085378819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000bb90 con 0x7f83ac1595c0 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 -- 172.21.15.46:0/4085378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac1595c0 msgr2=0x7f83ac0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 --2- 172.21.15.46:0/4085378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac1595c0 0x7f83ac0af940 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0acdd0 tx=0x7f83a00091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 -- 172.21.15.46:0/4085378819 shutdown_connections 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 --2- 172.21.15.46:0/4085378819 >> v2:172.21.15.132:3300/0 conn(0x7f83ac0cc500 0x7f83ac0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 --2- 172.21.15.46:0/4085378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac1595c0 0x7f83ac0af940 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 -- 172.21.15.46:0/4085378819 >> 172.21.15.46:0/4085378819 conn(0x7f83ac06ef60 msgr2=0x7f83ac0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 -- 172.21.15.46:0/4085378819 shutdown_connections 2023-10-06T12:53:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.883+0000 7f83ab59e700 1 -- 172.21.15.46:0/4085378819 wait complete. 2023-10-06T12:53:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 Processor -- start 2023-10-06T12:53:05.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 -- start start 2023-10-06T12:53:05.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 0x7f83ac15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83ac1591f0 0x7f83ac16cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83ac16d460 0x7f83ac155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac15c740 con 0x7f83ac0cc500 2023-10-06T12:53:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac16d8e0 con 0x7f83ac1591f0 2023-10-06T12:53:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83ab59e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac16dbc0 con 0x7f83ac16d460 2023-10-06T12:53:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 0x7f83ac15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 0x7f83ac15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49136/0 (socket says 172.21.15.46:49136) 2023-10-06T12:53:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 -- 172.21.15.46:0/1186363106 learned_addr learned my addr 172.21.15.46:0/1186363106 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83a9d9b700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83ac1591f0 0x7f83ac16cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aad9d700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83ac16d460 0x7f83ac155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 -- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83ac16d460 msgr2=0x7f83ac155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83ac16d460 0x7f83ac155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 -- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83ac1591f0 msgr2=0x7f83ac16cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83ac1591f0 0x7f83ac16cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aa59c700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009420 con 0x7f83ac0cc500 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.887+0000 7f83aad9d700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83ac16d460 0x7f83ac155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f83aa59c700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 0x7f83ac15cbd0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f839c00c7e0 tx=0x7f839c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c005b30 con 0x7f83ac0cc500 2023-10-06T12:53:05.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c005c90 con 0x7f83ac0cc500 2023-10-06T12:53:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c007300 con 0x7f83ac0cc500 2023-10-06T12:53:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac15c3d0 con 0x7f83ac0cc500 2023-10-06T12:53:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac1565c0 con 0x7f83ac0cc500 2023-10-06T12:53:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83ac15c230 con 0x7f83ac0cc500 2023-10-06T12:53:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.891+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f839c01b3b0 con 0x7f83ac0cc500 2023-10-06T12:53:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f839a7fc700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f838c060670 0x7f838c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f839c07c1c0 con 0x7f83ac0cc500 2023-10-06T12:53:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f83a9d9b700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f838c060670 0x7f838c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f839a7fc700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f838c0675a0 0x7f838c0699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f838c06a020 con 0x7f838c0675a0 2023-10-06T12:53:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f839c07e720 con 0x7f83ac0cc500 2023-10-06T12:53:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f83aad9d700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f838c0675a0 0x7f838c0699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f83a9d9b700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f838c060670 0x7f838c062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83a00093f0 tx=0x7f83a0017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.895+0000 7f83aad9d700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f838c0675a0 0x7f838c0699f0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.899+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== osd.5 v2:172.21.15.132:6809/33426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f838c06a020 con 0x7f838c0675a0 2023-10-06T12:53:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.915+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 --> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f83ac0cc0a0 con 0x7f838c0675a0 2023-10-06T12:53:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.915+0000 7f839a7fc700 1 -- 172.21.15.46:0/1186363106 <== osd.5 v2:172.21.15.132:6809/33426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f83ac0cc0a0 con 0x7f838c0675a0 2023-10-06T12:53:05.921 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:05.921 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:05.922 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.915+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f838c0675a0 msgr2=0x7f838c0699f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.915+0000 7f83ab59e700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.132:6809/33426,v1:172.21.15.132:6812/33426] conn(0x7f838c0675a0 0x7f838c0699f0 crc :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f838c060670 msgr2=0x7f838c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f838c060670 0x7f838c062b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f83a00093f0 tx=0x7f83a0017000 comp rx=0 tx=0).stop 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 msgr2=0x7f83ac15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 --2- 172.21.15.46:0/1186363106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83ac0cc500 0x7f83ac15cbd0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f839c00c7e0 tx=0x7f839c007df0 comp rx=0 tx=0).stop 2023-10-06T12:53:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83aad9d700 1 -- 172.21.15.46:0/1186363106 reap_dead start 2023-10-06T12:53:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 shutdown_connections 2023-10-06T12:53:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 >> 172.21.15.46:0/1186363106 conn(0x7f83ac06ef60 msgr2=0x7f83ac0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 shutdown_connections 2023-10-06T12:53:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:05.919+0000 7f83ab59e700 1 -- 172.21.15.46:0/1186363106 wait complete. 2023-10-06T12:53:05.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T12:53:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 Processor -- start 2023-10-06T12:53:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 -- start start 2023-10-06T12:53:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f689006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68900ad6c0 0x7f68900adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689006d460 con 0x7f689015ab60 2023-10-06T12:53:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689006ed10 con 0x7f68900ad6c0 2023-10-06T12:53:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f68952a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900c7000 con 0x7f689006e8a0 2023-10-06T12:53:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f688f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68900ad6c0 0x7f68900adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f688f7fe700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68900ad6c0 0x7f68900adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49152/0 (socket says 172.21.15.46:49152) 2023-10-06T12:53:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f688f7fe700 1 -- 172.21.15.46:0/321938111 learned_addr learned my addr 172.21.15.46:0/321938111 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.103+0000 7f688effd700 1 --1- 172.21.15.46:0/321938111 >> v1:172.21.15.46:6789/0 conn(0x7f689015ab60 0x7f689015af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43800/0 (socket says 172.21.15.46:43800) 2023-10-06T12:53:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688e7fc700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f689006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688f7fe700 1 -- 172.21.15.46:0/321938111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 msgr2=0x7f689006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688f7fe700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f689006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688f7fe700 1 -- 172.21.15.46:0/321938111 >> v1:172.21.15.46:6789/0 conn(0x7f689015ab60 legacy=0x7f689015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688f7fe700 1 -- 172.21.15.46:0/321938111 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900c72c0 con 0x7f68900ad6c0 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688f7fe700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f68900adb20 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f6888009f20 tx=0x7f6888009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf4c10abd89e48a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688dffb700 1 -- 172.21.15.46:0/321938111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e070 con 0x7f68900ad6c0 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688dffb700 1 -- 172.21.15.46:0/321938111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800b9d0 con 0x7f68900ad6c0 2023-10-06T12:53:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f688dffb700 1 -- 172.21.15.46:0/321938111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880092e0 con 0x7f68900ad6c0 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 -- 172.21.15.46:0/321938111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 msgr2=0x7f68900adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f68900adb20 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f6888009f20 tx=0x7f6888009840 comp rx=0 tx=0).stop 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 -- 172.21.15.46:0/321938111 shutdown_connections 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f68900adb20 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 --2- 172.21.15.46:0/321938111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f689006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 -- 172.21.15.46:0/321938111 >> 172.21.15.46:0/321938111 conn(0x7f68900cbe30 msgr2=0x7f68900cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 -- 172.21.15.46:0/321938111 shutdown_connections 2023-10-06T12:53:06.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.107+0000 7f68952a5700 1 -- 172.21.15.46:0/321938111 wait complete. 2023-10-06T12:53:06.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 Processor -- start 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 -- start start 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f68900aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f689015ab60 0x7f6890159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900af830 con 0x7f68900ad6c0 2023-10-06T12:53:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900afb10 con 0x7f689015ab60 2023-10-06T12:53:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f68952a5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900afdf0 con 0x7f689006e8a0 2023-10-06T12:53:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49166/0 (socket says 172.21.15.46:49166) 2023-10-06T12:53:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 -- 172.21.15.46:0/3816844736 learned_addr learned my addr 172.21.15.46:0/3816844736 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688effd700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f68900aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688f7fe700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f689015ab60 0x7f6890159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 -- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 msgr2=0x7f68900aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f68900aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 -- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f689015ab60 msgr2=0x7f6890159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f689015ab60 0x7f6890159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.111+0000 7f688e7fc700 1 -- 172.21.15.46:0/3816844736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009ae0 con 0x7f68900ad6c0 2023-10-06T12:53:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f688e7fc700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f688400e930 tx=0x7f68840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400eed0 con 0x7f68900ad6c0 2023-10-06T12:53:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688400c850 con 0x7f68900ad6c0 2023-10-06T12:53:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400cbf0 con 0x7f68900ad6c0 2023-10-06T12:53:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900b00d0 con 0x7f68900ad6c0 2023-10-06T12:53:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890150ed0 con 0x7f68900ad6c0 2023-10-06T12:53:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.115+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890063f90 con 0x7f68900ad6c0 2023-10-06T12:53:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.119+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f6884013470 con 0x7f68900ad6c0 2023-10-06T12:53:06.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.119+0000 7f6876ffd700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 0x7f687c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.119+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f688407cef0 con 0x7f68900ad6c0 2023-10-06T12:53:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.119+0000 7f688effd700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 0x7f687c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.119+0000 7f688effd700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 0x7f687c062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f689006d5f0 tx=0x7f6880009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.123+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f688404d010 con 0x7f68900ad6c0 2023-10-06T12:53:06.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.291+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f68900afc80 con 0x7f687c060670 2023-10-06T12:53:06.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f6876ffd700 1 -- 172.21.15.46:0/3816844736 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f68900afc80 con 0x7f687c060670 2023-10-06T12:53:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 msgr2=0x7f687c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 0x7f687c062b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f689006d5f0 tx=0x7f6880009040 comp rx=0 tx=0).stop 2023-10-06T12:53:06.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 msgr2=0x7f6890150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f688400e930 tx=0x7f68840091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 shutdown_connections 2023-10-06T12:53:06.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f687c060670 0x7f687c062b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f689015ab60 0x7f6890159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68900ad6c0 0x7f6890150990 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 --2- 172.21.15.46:0/3816844736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f689006e8a0 0x7f68900aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 >> 172.21.15.46:0/3816844736 conn(0x7f68900cbe30 msgr2=0x7f68900ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 shutdown_connections 2023-10-06T12:53:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.295+0000 7f68952a5700 1 -- 172.21.15.46:0/3816844736 wait complete. 2023-10-06T12:53:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T12:53:06.321 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T12:53:06.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.487+0000 7f7c5520c700 1 Processor -- start 2023-10-06T12:53:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.487+0000 7f7c5520c700 1 -- start start 2023-10-06T12:53:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c5520c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c50069770 0x7f7c500b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c5520c700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c501591b0 0x7f7c500b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c5520c700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500b5eb0 con 0x7f7c50158de0 2023-10-06T12:53:06.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c5520c700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500cef30 con 0x7f7c501591b0 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c5520c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500cf4f0 con 0x7f7c50069770 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c501591b0 0x7f7c500b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c4ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c50158de0 0x7f7c50069060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43806/0 (socket says 172.21.15.46:43806) 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c4ffff700 1 -- 172.21.15.46:0/826275745 learned_addr learned my addr 172.21.15.46:0/826275745 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c47fff700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c50069770 0x7f7c500b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 -- 172.21.15.46:0/826275745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c50069770 msgr2=0x7f7c500b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c50069770 0x7f7c500b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 -- 172.21.15.46:0/826275745 >> v1:172.21.15.46:6789/0 conn(0x7f7c50158de0 legacy=0x7f7c50069060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 -- 172.21.15.46:0/826275745 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500ccac0 con 0x7f7c501591b0 2023-10-06T12:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.491+0000 7f7c54a0b700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c501591b0 0x7f7c500b2e10 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e60 tx=0x7f7c48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a17e563a5a50c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c4f7fe700 1 -- 172.21.15.46:0/826275745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800e070 con 0x7f7c501591b0 2023-10-06T12:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c4f7fe700 1 -- 172.21.15.46:0/826275745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4800bb60 con 0x7f7c501591b0 2023-10-06T12:53:06.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c4f7fe700 1 -- 172.21.15.46:0/826275745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48012430 con 0x7f7c501591b0 2023-10-06T12:53:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- 172.21.15.46:0/826275745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c501591b0 msgr2=0x7f7c500b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c501591b0 0x7f7c500b2e10 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e60 tx=0x7f7c48009780 comp rx=0 tx=0).stop 2023-10-06T12:53:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- 172.21.15.46:0/826275745 shutdown_connections 2023-10-06T12:53:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c501591b0 0x7f7c500b2e10 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 --2- 172.21.15.46:0/826275745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c50069770 0x7f7c500b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- 172.21.15.46:0/826275745 >> 172.21.15.46:0/826275745 conn(0x7f7c50068390 msgr2=0x7f7c500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- 172.21.15.46:0/826275745 shutdown_connections 2023-10-06T12:53:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- 172.21.15.46:0/826275745 wait complete. 2023-10-06T12:53:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 Processor -- start 2023-10-06T12:53:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.495+0000 7f7c5520c700 1 -- start start 2023-10-06T12:53:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c50069770 0x7f7c500c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 0x7f7c5015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c501591b0 0x7f7c50154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5015b7f0 con 0x7f7c50158de0 2023-10-06T12:53:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50168e10 con 0x7f7c50069770 2023-10-06T12:53:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50169de0 con 0x7f7c501591b0 2023-10-06T12:53:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 0x7f7c5015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 0x7f7c5015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49180/0 (socket says 172.21.15.46:49180) 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 -- 172.21.15.46:0/1220755345 learned_addr learned my addr 172.21.15.46:0/1220755345 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c4ffff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c50069770 0x7f7c500c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c54a0b700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c501591b0 0x7f7c50154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 -- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c501591b0 msgr2=0x7f7c50154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c501591b0 0x7f7c50154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 -- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c50069770 msgr2=0x7f7c500c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c50069770 0x7f7c500c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48009a20 con 0x7f7c50158de0 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c4ffff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c50069770 0x7f7c500c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 0x7f7c5015b010 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800ecb0 tx=0x7f7c380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800c9d0 con 0x7f7c50158de0 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3800cb30 con 0x7f7c50158de0 2023-10-06T12:53:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c50169950 con 0x7f7c50158de0 2023-10-06T12:53:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800ced0 con 0x7f7c50158de0 2023-10-06T12:53:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.499+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c50169020 con 0x7f7c50158de0 2023-10-06T12:53:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c50064590 con 0x7f7c50158de0 2023-10-06T12:53:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f7c38013680 con 0x7f7c50158de0 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7c30060770 0x7f7c30062c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f7c3807cf70 con 0x7f7c50158de0 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f7c30067530 0x7f7c30069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c30069fb0 con 0x7f7c30067530 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.503+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3807f4d0 con 0x7f7c50158de0 2023-10-06T12:53:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.507+0000 7f7c4ffff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7c30060770 0x7f7c30062c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.507+0000 7f7c54a0b700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f7c30067530 0x7f7c30069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.507+0000 7f7c4ffff700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7c30060770 0x7f7c30062c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b57e0 tx=0x7f7c40009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.507+0000 7f7c54a0b700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f7c30067530 0x7f7c30069980 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.507+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== osd.6 v2:172.21.15.132:6811/33424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7c30069fb0 con 0x7f7c30067530 2023-10-06T12:53:06.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 --> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7c50169500 con 0x7f7c30067530 2023-10-06T12:53:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c4cff9700 1 -- 172.21.15.46:0/1220755345 <== osd.6 v2:172.21.15.132:6811/33424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7c50169500 con 0x7f7c30067530 2023-10-06T12:53:06.532 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:06.532 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:06.532 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f7c30067530 msgr2=0x7f7c30069980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.132:6811/33424,v1:172.21.15.132:6813/33424] conn(0x7f7c30067530 0x7f7c30069980 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7c30060770 msgr2=0x7f7c30062c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f7c30060770 0x7f7c30062c20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c500b57e0 tx=0x7f7c40009040 comp rx=0 tx=0).stop 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 msgr2=0x7f7c5015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 --2- 172.21.15.46:0/1220755345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c50158de0 0x7f7c5015b010 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800ecb0 tx=0x7f7c380091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c54a0b700 1 -- 172.21.15.46:0/1220755345 reap_dead start 2023-10-06T12:53:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 shutdown_connections 2023-10-06T12:53:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 >> 172.21.15.46:0/1220755345 conn(0x7f7c50068390 msgr2=0x7f7c50157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.527+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 shutdown_connections 2023-10-06T12:53:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.531+0000 7f7c5520c700 1 -- 172.21.15.46:0/1220755345 wait complete. 2023-10-06T12:53:06.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T12:53:06.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.719+0000 7fe05abcc700 1 Processor -- start 2023-10-06T12:53:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.719+0000 7fe05abcc700 1 -- start start 2023-10-06T12:53:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.719+0000 7fe05abcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe054155c70 0x7fe05406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05abcc700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe054067050 0x7fe0540674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05abcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0540c74d0 con 0x7fe0540c6150 2023-10-06T12:53:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05abcc700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe054067a10 con 0x7fe054067050 2023-10-06T12:53:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05abcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540c68c0 con 0x7fe054155c70 2023-10-06T12:53:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe054067050 0x7fe0540674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe054067050 0x7fe0540674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49184/0 (socket says 172.21.15.46:49184) 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 -- 172.21.15.46:0/2623231659 learned_addr learned my addr 172.21.15.46:0/2623231659 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe059bca700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe054155c70 0x7fe05406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 -- 172.21.15.46:0/2623231659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe054155c70 msgr2=0x7fe05406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe054155c70 0x7fe05406da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 -- 172.21.15.46:0/2623231659 >> v1:172.21.15.132:6789/0 conn(0x7fe0540c6150 legacy=0x7fe0540c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 -- 172.21.15.46:0/2623231659 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540c6ba0 con 0x7fe054067050 2023-10-06T12:53:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe05a3cb700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540674d0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009f20 tx=0x7fe04c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20244a0f5c63b99a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe058bc8700 1 -- 172.21.15.46:0/2623231659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00e070 con 0x7fe054067050 2023-10-06T12:53:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe058bc8700 1 -- 172.21.15.46:0/2623231659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c00b9d0 con 0x7fe054067050 2023-10-06T12:53:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.723+0000 7fe058bc8700 1 -- 172.21.15.46:0/2623231659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0092e0 con 0x7fe054067050 2023-10-06T12:53:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- 172.21.15.46:0/2623231659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 msgr2=0x7fe0540674d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540674d0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009f20 tx=0x7fe04c009840 comp rx=0 tx=0).stop 2023-10-06T12:53:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- 172.21.15.46:0/2623231659 shutdown_connections 2023-10-06T12:53:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540674d0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2623231659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe054155c70 0x7fe05406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- 172.21.15.46:0/2623231659 >> 172.21.15.46:0/2623231659 conn(0x7fe054062ef0 msgr2=0x7fe0540cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- 172.21.15.46:0/2623231659 shutdown_connections 2023-10-06T12:53:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- 172.21.15.46:0/2623231659 wait complete. 2023-10-06T12:53:06.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 Processor -- start 2023-10-06T12:53:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.727+0000 7fe05abcc700 1 -- start start 2023-10-06T12:53:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 0x7fe0540b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe054155c70 0x7fe0540b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540bffb0 con 0x7fe054067050 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415c220 con 0x7fe054155c70 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540c0910 con 0x7fe0540c6150 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49198/0 (socket says 172.21.15.46:49198) 2023-10-06T12:53:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 -- 172.21.15.46:0/2962874368 learned_addr learned my addr 172.21.15.46:0/2962874368 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05a3cb700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe054155c70 0x7fe0540b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe0593c9700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 0x7fe0540b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 -- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 msgr2=0x7fe0540b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 0x7fe0540b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 -- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe054155c70 msgr2=0x7fe0540b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe054155c70 0x7fe0540b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 -- 172.21.15.46:0/2962874368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c009ae0 con 0x7fe054067050 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe0593c9700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 0x7fe0540b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe059bca700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe05000e930 tx=0x7fe0500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050013680 con 0x7fe054067050 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0500137e0 con 0x7fe054067050 2023-10-06T12:53:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050013b80 con 0x7fe054067050 2023-10-06T12:53:06.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0540ad990 con 0x7fe054067050 2023-10-06T12:53:06.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.731+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540bf8f0 con 0x7fe054067050 2023-10-06T12:53:06.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.735+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054064590 con 0x7fe054067050 2023-10-06T12:53:06.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.735+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fe05000c460 con 0x7fe054067050 2023-10-06T12:53:06.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.739+0000 7fe049ffb700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 0x7fe038062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:06.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.739+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fe05007ce30 con 0x7fe054067050 2023-10-06T12:53:06.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.739+0000 7fe0593c9700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 0x7fe038062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:06.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.739+0000 7fe0593c9700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 0x7fe038062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe044007fd0 tx=0x7fe044009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:06.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.743+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe05004cf50 con 0x7fe054067050 2023-10-06T12:53:06.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.915+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe054069860 con 0x7fe038060670 2023-10-06T12:53:06.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe049ffb700 1 -- 172.21.15.46:0/2962874368 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe054069860 con 0x7fe038060670 2023-10-06T12:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 msgr2=0x7fe038062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 0x7fe038062b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe044007fd0 tx=0x7fe044009040 comp rx=0 tx=0).stop 2023-10-06T12:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 msgr2=0x7fe0540c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe05000e930 tx=0x7fe0500091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 shutdown_connections 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fe038060670 0x7fe038062b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe054155c70 0x7fe0540b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0540c6150 0x7fe0540b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 --2- 172.21.15.46:0/2962874368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe054067050 0x7fe0540c03d0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:06.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 >> 172.21.15.46:0/2962874368 conn(0x7fe054062ef0 msgr2=0x7fe054155090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 shutdown_connections 2023-10-06T12:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:06.919+0000 7fe05abcc700 1 -- 172.21.15.46:0/2962874368 wait complete. 2023-10-06T12:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T12:53:06.946 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T12:53:06.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T12:53:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.111+0000 7f2a29dcc700 1 Processor -- start 2023-10-06T12:53:07.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.111+0000 7f2a29dcc700 1 -- start start 2023-10-06T12:53:07.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a29dcc700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 0x7f2a24159150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a29dcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 0x7f2a240af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a29dcc700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a240adc20 con 0x7f2a240d0770 2023-10-06T12:53:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a29dcc700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a240adec0 con 0x7f2a24158d80 2023-10-06T12:53:07.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a29dcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240ae1a0 con 0x7f2a240aec80 2023-10-06T12:53:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a295cb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 0x7f2a240af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a28dca700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 0x7f2a24159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a23fff700 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f2a240d0770 0x7f2a240b02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.46:36304/0 (socket says 172.21.15.46:36304) 2023-10-06T12:53:07.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a23fff700 1 -- 172.21.15.46:0/491984075 learned_addr learned my addr 172.21.15.46:0/491984075 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a237fe700 1 -- 172.21.15.46:0/491984075 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605993821 0 0) 0x7f2a240adc20 con 0x7f2a240d0770 2023-10-06T12:53:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a237fe700 1 -- 172.21.15.46:0/491984075 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a14003570 con 0x7f2a240d0770 2023-10-06T12:53:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a295cb700 1 -- 172.21.15.46:0/491984075 >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 msgr2=0x7f2a24159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:07.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a295cb700 1 --2- 172.21.15.46:0/491984075 >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 0x7f2a24159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a295cb700 1 -- 172.21.15.46:0/491984075 >> v1:172.21.15.132:6789/0 conn(0x7f2a240d0770 legacy=0x7f2a240b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a295cb700 1 -- 172.21.15.46:0/491984075 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240ae480 con 0x7f2a240aec80 2023-10-06T12:53:07.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.115+0000 7f2a28dca700 1 --2- 172.21.15.46:0/491984075 >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 0x7f2a24159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a295cb700 1 --2- 172.21.15.46:0/491984075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 0x7f2a240af100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009e60 tx=0x7f2a1c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a562fdeaf68f5dc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a237fe700 1 -- 172.21.15.46:0/491984075 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00e070 con 0x7f2a240aec80 2023-10-06T12:53:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a237fe700 1 -- 172.21.15.46:0/491984075 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00bb40 con 0x7f2a240aec80 2023-10-06T12:53:07.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a237fe700 1 -- 172.21.15.46:0/491984075 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c012430 con 0x7f2a240aec80 2023-10-06T12:53:07.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 -- 172.21.15.46:0/491984075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 msgr2=0x7f2a240af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/491984075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 0x7f2a240af100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009e60 tx=0x7f2a1c009780 comp rx=0 tx=0).stop 2023-10-06T12:53:07.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 -- 172.21.15.46:0/491984075 shutdown_connections 2023-10-06T12:53:07.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/491984075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240aec80 0x7f2a240af100 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/491984075 >> v2:172.21.15.132:3300/0 conn(0x7f2a24158d80 0x7f2a24159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 -- 172.21.15.46:0/491984075 >> 172.21.15.46:0/491984075 conn(0x7f2a2406ef60 msgr2=0x7f2a240d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 -- 172.21.15.46:0/491984075 shutdown_connections 2023-10-06T12:53:07.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.119+0000 7f2a29dcc700 1 -- 172.21.15.46:0/491984075 wait complete. 2023-10-06T12:53:07.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 Processor -- start 2023-10-06T12:53:07.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 -- start start 2023-10-06T12:53:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 0x7f2a2406ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240d0770 0x7f2a2416d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a24158d80 0x7f2a24154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2416e670 con 0x7f2a240aec80 2023-10-06T12:53:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2416dbc0 con 0x7f2a24158d80 2023-10-06T12:53:07.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a29dcc700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2416ddd0 con 0x7f2a240d0770 2023-10-06T12:53:07.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a28dca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 0x7f2a2406ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a28dca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 0x7f2a2406ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49204/0 (socket says 172.21.15.46:49204) 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.123+0000 7f2a28dca700 1 -- 172.21.15.46:0/3637034857 learned_addr learned my addr 172.21.15.46:0/3637034857 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a23fff700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240d0770 0x7f2a2416d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a295cb700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a24158d80 0x7f2a24154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 -- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240d0770 msgr2=0x7f2a2416d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a240d0770 0x7f2a2416d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 -- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a24158d80 msgr2=0x7f2a24154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a24158d80 0x7f2a24154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18009420 con 0x7f2a240aec80 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a295cb700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a24158d80 0x7f2a24154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a28dca700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 0x7f2a2406ad30 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ec80 tx=0x7f2a180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800c850 con 0x7f2a240aec80 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1800c9b0 con 0x7f2a240aec80 2023-10-06T12:53:07.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c009a20 con 0x7f2a240aec80 2023-10-06T12:53:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800cd30 con 0x7f2a240aec80 2023-10-06T12:53:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a240cd740 con 0x7f2a240aec80 2023-10-06T12:53:07.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.127+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a24064590 con 0x7f2a240aec80 2023-10-06T12:53:07.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f2a18013680 con 0x7f2a240aec80 2023-10-06T12:53:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a20ff9700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2a08060670 0x7f2a08062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f2a1807d310 con 0x7f2a240aec80 2023-10-06T12:53:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a23fff700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2a08060670 0x7f2a08062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a20ff9700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f2a080675a0 0x7f2a080699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.131+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a0806a020 con 0x7f2a080675a0 2023-10-06T12:53:07.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.135+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1807f870 con 0x7f2a240aec80 2023-10-06T12:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.135+0000 7f2a295cb700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f2a080675a0 0x7f2a080699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.135+0000 7f2a23fff700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2a08060670 0x7f2a08062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a2406e2a0 tx=0x7f2a10009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.135+0000 7f2a295cb700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f2a080675a0 0x7f2a080699f0 crc :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.135+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== osd.7 v2:172.21.15.132:6808/33425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2a0806a020 con 0x7f2a080675a0 2023-10-06T12:53:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 --> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a240ae760 con 0x7f2a080675a0 2023-10-06T12:53:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a20ff9700 1 -- 172.21.15.46:0/3637034857 <== osd.7 v2:172.21.15.132:6808/33425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2a240ae760 con 0x7f2a080675a0 2023-10-06T12:53:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-10-06T12:53:07.161 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T12:53:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-10-06T12:53:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f2a080675a0 msgr2=0x7f2a080699f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.132:6808/33425,v1:172.21.15.132:6810/33425] conn(0x7f2a080675a0 0x7f2a080699f0 crc :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2a08060670 msgr2=0x7f2a08062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f2a08060670 0x7f2a08062b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a2406e2a0 tx=0x7f2a10009040 comp rx=0 tx=0).stop 2023-10-06T12:53:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 msgr2=0x7f2a2406ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a29dcc700 1 --2- 172.21.15.46:0/3637034857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a240aec80 0x7f2a2406ad30 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ec80 tx=0x7f2a180091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.155+0000 7f2a295cb700 1 -- 172.21.15.46:0/3637034857 reap_dead start 2023-10-06T12:53:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.159+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 shutdown_connections 2023-10-06T12:53:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.159+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 >> 172.21.15.46:0/3637034857 conn(0x7f2a2406ef60 msgr2=0x7f2a24156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.159+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 shutdown_connections 2023-10-06T12:53:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.159+0000 7f2a29dcc700 1 -- 172.21.15.46:0/3637034857 wait complete. 2023-10-06T12:53:07.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T12:53:07.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:53:07.344 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:53:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.343+0000 7fb39462e700 1 Processor -- start 2023-10-06T12:53:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.343+0000 7fb39462e700 1 -- start start 2023-10-06T12:53:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb39462e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 0x7fb38c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb39462e700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0cc150 con 0x7fb38c15c770 2023-10-06T12:53:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb39462e700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0cb540 con 0x7fb38c0ccb50 2023-10-06T12:53:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb39462e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0cfbb0 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb3923ca700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb38c15c770 0x7fb38c0cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43808/0 (socket says 172.21.15.46:43808) 2023-10-06T12:53:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb3923ca700 1 -- 172.21.15.46:0/3681639994 learned_addr learned my addr 172.21.15.46:0/3681639994 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb391bc9700 1 --2- 172.21.15.46:0/3681639994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 0x7fb38c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb3913c8700 1 -- 172.21.15.46:0/3681639994 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450078508 0 0) 0x7fb38c0cc150 con 0x7fb38c15c770 2023-10-06T12:53:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.347+0000 7fb3913c8700 1 -- 172.21.15.46:0/3681639994 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb378003570 con 0x7fb38c15c770 2023-10-06T12:53:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb391bc9700 1 -- 172.21.15.46:0/3681639994 >> v1:172.21.15.132:6789/0 conn(0x7fb38c0ccb50 legacy=0x7fb38c162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb391bc9700 1 -- 172.21.15.46:0/3681639994 >> v1:172.21.15.46:6789/0 conn(0x7fb38c15c770 legacy=0x7fb38c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb391bc9700 1 -- 172.21.15.46:0/3681639994 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0cb820 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb391bc9700 1 --2- 172.21.15.46:0/3681639994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 0x7fb38c0ce4f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009910 tx=0x7fb37c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df64f94e3ec6b1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb3913c8700 1 -- 172.21.15.46:0/3681639994 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00b680 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb3913c8700 1 -- 172.21.15.46:0/3681639994 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00b7e0 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb3913c8700 1 -- 172.21.15.46:0/3681639994 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00bba0 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 -- 172.21.15.46:0/3681639994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 msgr2=0x7fb38c0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 --2- 172.21.15.46:0/3681639994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 0x7fb38c0ce4f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009910 tx=0x7fb37c0091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 -- 172.21.15.46:0/3681639994 shutdown_connections 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 --2- 172.21.15.46:0/3681639994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c0c5ff0 0x7fb38c0ce4f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 -- 172.21.15.46:0/3681639994 >> 172.21.15.46:0/3681639994 conn(0x7fb38c099480 msgr2=0x7fb38c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 -- 172.21.15.46:0/3681639994 shutdown_connections 2023-10-06T12:53:07.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.351+0000 7fb39462e700 1 -- 172.21.15.46:0/3681639994 wait complete. 2023-10-06T12:53:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 Processor -- start 2023-10-06T12:53:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 -- start start 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 0x7fb38c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ccb50 0x7fb38c14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 0x7fb38c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b3910 con 0x7fb38c0c5ff0 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b3bf0 con 0x7fb38c0ccb50 2023-10-06T12:53:07.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb39462e700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b3ed0 con 0x7fb38c15c770 2023-10-06T12:53:07.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb3923ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 0x7fb38c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb392bcb700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 0x7fb38c15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb3923ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 0x7fb38c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49210/0 (socket says 172.21.15.46:49210) 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb3923ca700 1 -- 172.21.15.46:0/3448159165 learned_addr learned my addr 172.21.15.46:0/3448159165 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.355+0000 7fb391bc9700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ccb50 0x7fb38c14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 -- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ccb50 msgr2=0x7fb38c14bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ccb50 0x7fb38c14bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 -- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 msgr2=0x7fb38c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 0x7fb38c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 -- 172.21.15.46:0/3448159165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c009420 con 0x7fb38c15c770 2023-10-06T12:53:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb392bcb700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 0x7fb38c15acc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c830 tx=0x7fb384001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384012630 con 0x7fb38c15c770 2023-10-06T12:53:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384012790 con 0x7fb38c15c770 2023-10-06T12:53:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0b41b0 con 0x7fb38c15c770 2023-10-06T12:53:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384012b50 con 0x7fb38c15c770 2023-10-06T12:53:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c1694b0 con 0x7fb38c15c770 2023-10-06T12:53:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.359+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c0660f0 con 0x7fb38c15c770 2023-10-06T12:53:07.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.363+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7fb384024420 con 0x7fb38c15c770 2023-10-06T12:53:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.363+0000 7fb3827fc700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 0x7fb370062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.367+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7fb38407d560 con 0x7fb38c15c770 2023-10-06T12:53:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.367+0000 7fb3923ca700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 0x7fb370062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.367+0000 7fb3923ca700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 0x7fb370062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3880010e0 tx=0x7fb388009040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.371+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb38404d700 con 0x7fb38c15c770 2023-10-06T12:53:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.539+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fb38c04b5c0 con 0x7fb370060670 2023-10-06T12:53:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.539+0000 7fb3827fc700 1 -- 172.21.15.46:0/3448159165 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb38c04b5c0 con 0x7fb370060670 2023-10-06T12:53:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 msgr2=0x7fb370062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 0x7fb370062b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3880010e0 tx=0x7fb388009040 comp rx=0 tx=0).stop 2023-10-06T12:53:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 msgr2=0x7fb38c15acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 0x7fb38c15acc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c830 tx=0x7fb384001690 comp rx=0 tx=0).stop 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 shutdown_connections 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb38c15c770 0x7fb38c15acc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7fb370060670 0x7fb370062b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ccb50 0x7fb38c14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 --2- 172.21.15.46:0/3448159165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb38c0c5ff0 0x7fb38c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 >> 172.21.15.46:0/3448159165 conn(0x7fb38c099480 msgr2=0x7fb38c15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 shutdown_connections 2023-10-06T12:53:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.543+0000 7fb39462e700 1 -- 172.21.15.46:0/3448159165 wait complete. 2023-10-06T12:53:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T12:53:07.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:53:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.723+0000 7ff7207c5700 1 Processor -- start 2023-10-06T12:53:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.723+0000 7ff7207c5700 1 -- start start 2023-10-06T12:53:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff7207c5700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff71806e8a0 0x7ff71806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff7207c5700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff7180ad6c0 0x7ff7180adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff7207c5700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71806d460 con 0x7ff71815ab60 2023-10-06T12:53:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff7207c5700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71806ed40 con 0x7ff7180ad6c0 2023-10-06T12:53:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff7207c5700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7180c7000 con 0x7ff71806e8a0 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71e561700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff71815ab60 0x7ff71815af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43818/0 (socket says 172.21.15.46:43818) 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71e561700 1 -- 172.21.15.46:0/3142871263 learned_addr learned my addr 172.21.15.46:0/3142871263 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 --2- 172.21.15.46:0/3142871263 >> v2:172.21.15.46:3300/0 conn(0x7ff7180ad6c0 0x7ff7180adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71dd60700 1 --2- 172.21.15.46:0/3142871263 >> v2:172.21.15.132:3300/0 conn(0x7ff71806e8a0 0x7ff71806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71d55f700 1 -- 172.21.15.46:0/3142871263 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963183997 0 0) 0x7ff71806d460 con 0x7ff71815ab60 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71d55f700 1 -- 172.21.15.46:0/3142871263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704003570 con 0x7ff71815ab60 2023-10-06T12:53:07.733 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 -- 172.21.15.46:0/3142871263 >> v2:172.21.15.132:3300/0 conn(0x7ff71806e8a0 msgr2=0x7ff71806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 --2- 172.21.15.46:0/3142871263 >> v2:172.21.15.132:3300/0 conn(0x7ff71806e8a0 0x7ff71806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 -- 172.21.15.46:0/3142871263 >> v1:172.21.15.46:6789/0 conn(0x7ff71815ab60 legacy=0x7ff71815af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 -- 172.21.15.46:0/3142871263 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180c72c0 con 0x7ff7180ad6c0 2023-10-06T12:53:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71ed62700 1 --2- 172.21.15.46:0/3142871263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7180ad6c0 0x7ff7180adaf0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7ff71806f0b0 tx=0x7ff7100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accdaebf33e58f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.727+0000 7ff71d55f700 1 -- 172.21.15.46:0/3142871263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710011630 con 0x7ff7180ad6c0 2023-10-06T12:53:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff71d55f700 1 -- 172.21.15.46:0/3142871263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100117b0 con 0x7ff7180ad6c0 2023-10-06T12:53:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff71d55f700 1 -- 172.21.15.46:0/3142871263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710011b70 con 0x7ff7180ad6c0 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- 172.21.15.46:0/3142871263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7180ad6c0 msgr2=0x7ff7180adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- 172.21.15.46:0/3142871263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7180ad6c0 0x7ff7180adaf0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7ff71806f0b0 tx=0x7ff7100091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- 172.21.15.46:0/3142871263 shutdown_connections 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- 172.21.15.46:0/3142871263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7180ad6c0 0x7ff7180adaf0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- 172.21.15.46:0/3142871263 >> v2:172.21.15.132:3300/0 conn(0x7ff71806e8a0 0x7ff71806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- 172.21.15.46:0/3142871263 >> 172.21.15.46:0/3142871263 conn(0x7ff7180cbe30 msgr2=0x7ff7180cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- 172.21.15.46:0/3142871263 shutdown_connections 2023-10-06T12:53:07.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- 172.21.15.46:0/3142871263 wait complete. 2023-10-06T12:53:07.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 Processor -- start 2023-10-06T12:53:07.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- start start 2023-10-06T12:53:07.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7180ad6c0 0x7ff7180c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 0x7ff7180b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180b07b0 con 0x7ff71806e8a0 2023-10-06T12:53:07.739 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180afba0 con 0x7ff71815ab60 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.731+0000 7ff7207c5700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180afe80 con 0x7ff7180ad6c0 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49230/0 (socket says 172.21.15.46:49230) 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 -- 172.21.15.46:0/2163350886 learned_addr learned my addr 172.21.15.46:0/2163350886 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71dd60700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7180ad6c0 0x7ff7180c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71ed62700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 0x7ff7180b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 -- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7180ad6c0 msgr2=0x7ff7180c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7180ad6c0 0x7ff7180c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 -- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 msgr2=0x7ff7180b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 0x7ff7180b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 -- 172.21.15.46:0/2163350886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009420 con 0x7ff71806e8a0 2023-10-06T12:53:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71ed62700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 0x7ff7180b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T12:53:07.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff71e561700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7ff714004ed0 tx=0x7ff7140054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400cea0 con 0x7ff71806e8a0 2023-10-06T12:53:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71400ab70 con 0x7ff71806e8a0 2023-10-06T12:53:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7180b0160 con 0x7ff71806e8a0 2023-10-06T12:53:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140070f0 con 0x7ff71806e8a0 2023-10-06T12:53:07.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71814fd20 con 0x7ff71806e8a0 2023-10-06T12:53:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.735+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff718063f90 con 0x7ff71806e8a0 2023-10-06T12:53:07.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.739+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff7140074e0 con 0x7ff71806e8a0 2023-10-06T12:53:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.739+0000 7ff70e7fc700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 0x7ff6fc062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.739+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7ff71407d200 con 0x7ff71806e8a0 2023-10-06T12:53:07.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.739+0000 7ff71dd60700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 0x7ff6fc062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:07.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.743+0000 7ff71dd60700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 0x7ff6fc062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c6cd0 tx=0x7ff708008040 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:07.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.747+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff71404d3a0 con 0x7ff71806e8a0 2023-10-06T12:53:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.915+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7180b08f0 con 0x7ff6fc060670 2023-10-06T12:53:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.919+0000 7ff70e7fc700 1 -- 172.21.15.46:0/2163350886 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+69902 (secure 0 0 0) 0x7ff7080098e0 con 0x7ff6fc060670 2023-10-06T12:53:07.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 msgr2=0x7ff6fc062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 0x7ff6fc062b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c6cd0 tx=0x7ff708008040 comp rx=0 tx=0).stop 2023-10-06T12:53:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 msgr2=0x7ff71814f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7ff714004ed0 tx=0x7ff7140054f0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 shutdown_connections 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff6fc060670 0x7ff6fc062b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff71815ab60 0x7ff7180b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7180ad6c0 0x7ff7180c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 --2- 172.21.15.46:0/2163350886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff71806e8a0 0x7ff71814f7e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 >> 172.21.15.46:0/2163350886 conn(0x7ff7180cbe30 msgr2=0x7ff7180c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 shutdown_connections 2023-10-06T12:53:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:07.927+0000 7ff7207c5700 1 -- 172.21.15.46:0/2163350886 wait complete. 2023-10-06T12:53:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:53:07.950 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-10-06T12:53:06.592408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460744,"kb_used_data":21000,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752513976,"statfs":{"total":773094113280,"available":770574311424,"internally_reserved":0,"allocated":21504000,"data_stored":11994528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.693186"},"pg_stats":[{"pgid":"1.0","version":"199'370","reported_seq":1213,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061246+0000","last_change":"2023-10-06T11:18:12.184609+0000","last_active":"2023-10-06T12:52:53.061246+0000","last_peered":"2023-10-06T12:52:53.061246+0000","last_clean":"2023-10-06T12:52:53.061246+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:52:53.061246+0000","last_undegraded":"2023-10-06T12:52:53.061246+0000","last_fullsized":"2023-10-06T12:52:53.061246+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:10.604513+0000","last_clean_scrub_stamp":"2023-10-06T11:18:10.604513+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:47:16.251552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.900458+0000","last_change":"2023-10-06T11:18:17.689258+0000","last_active":"2023-10-06T12:52:54.900458+0000","last_peered":"2023-10-06T12:52:54.900458+0000","last_clean":"2023-10-06T12:52:54.900458+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:52:54.900458+0000","last_undegraded":"2023-10-06T12:52:54.900458+0000","last_fullsized":"2023-10-06T12:52:54.900458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.061057+0000","last_change":"2023-10-06T11:18:17.692314+0000","last_active":"2023-10-06T12:52:53.061057+0000","last_peered":"2023-10-06T12:52:53.061057+0000","last_clean":"2023-10-06T12:52:53.061057+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:52:53.061057+0000","last_undegraded":"2023-10-06T12:52:53.061057+0000","last_fullsized":"2023-10-06T12:52:53.061057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:37:06.407126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:52.853344+0000","last_change":"2023-10-06T11:18:18.240325+0000","last_active":"2023-10-06T12:52:52.853344+0000","last_peered":"2023-10-06T12:52:52.853344+0000","last_clean":"2023-10-06T12:52:52.853344+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:52:52.853344+0000","last_undegraded":"2023-10-06T12:52:52.853344+0000","last_fullsized":"2023-10-06T12:52:52.853344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":186,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-06T12:52:54.899945+0000","last_change":"2023-10-06T11:18:17.691391+0000","last_active":"2023-10-06T12:52:54.899945+0000","last_peered":"2023-10-06T12:52:54.899945+0000","last_clean":"2023-10-06T12:52:54.899945+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:52:54.899945+0000","last_undegraded":"2023-10-06T12:52:54.899945+0000","last_fullsized":"2023-10-06T12:52:54.899945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119154+0000","last_change":"2023-10-06T11:18:18.118323+0000","last_active":"2023-10-06T12:52:53.119154+0000","last_peered":"2023-10-06T12:52:53.119154+0000","last_clean":"2023-10-06T12:52:53.119154+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:52:53.119154+0000","last_undegraded":"2023-10-06T12:52:53.119154+0000","last_fullsized":"2023-10-06T12:52:53.119154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:19:40.754455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.060861+0000","last_change":"2023-10-06T11:18:18.121906+0000","last_active":"2023-10-06T12:52:53.060861+0000","last_peered":"2023-10-06T12:52:53.060861+0000","last_clean":"2023-10-06T12:52:53.060861+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:52:53.060861+0000","last_undegraded":"2023-10-06T12:52:53.060861+0000","last_fullsized":"2023-10-06T12:52:53.060861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:30.137120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.119032+0000","last_change":"2023-10-06T11:18:17.693408+0000","last_active":"2023-10-06T12:52:53.119032+0000","last_peered":"2023-10-06T12:52:53.119032+0000","last_clean":"2023-10-06T12:52:53.119032+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:52:53.119032+0000","last_undegraded":"2023-10-06T12:52:53.119032+0000","last_fullsized":"2023-10-06T12:52:53.119032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:48.849027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T12:52:53.118917+0000","last_change":"2023-10-06T11:18:17.692821+0000","last_active":"2023-10-06T12:52:53.118917+0000","last_peered":"2023-10-06T12:52:53.118917+0000","last_clean":"2023-10-06T12:52:53.118917+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:52:53.118917+0000","last_undegraded":"2023-10-06T12:52:53.118917+0000","last_fullsized":"2023-10-06T12:52:53.118917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:18:16.660943+0000","last_clean_scrub_stamp":"2023-10-06T11:18:16.660943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.075470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.1990000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.756},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.361}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.486}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.407},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.623}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":1.5489999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.823},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.591}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.71},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.663}]}]},{"osd":6,"up_from":13,"seq":55834575976,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3029999999999999,"15min":1.3009999999999999},"min":{"1min":0.83899999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7370000000000001,"5min":1.857,"15min":1.857},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.393,"5min":1.304,"15min":1.3180000000000001},"min":{"1min":1.0089999999999999,"5min":0.86899999999999999,"15min":0.74399999999999999},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1850000000000001,"15min":1.1830000000000001},"min":{"1min":0.83499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.653,"5min":1.671,"15min":1.7789999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.331,"5min":1.24,"15min":1.2070000000000001},"min":{"1min":0.81799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.288}]},{"osd":2,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.2050000000000001},"min":{"1min":0.85699999999999998,"5min":0.76600000000000001,"15min":0.68500000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.2450000000000001},"min":{"1min":0.90500000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.0289999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2749999999999999,"15min":1.3080000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3100000000000001,"15min":1.331},"min":{"1min":0.94299999999999995,"5min":0.70899999999999996,"15min":0.66200000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.8560000000000001},"last":1.3100000000000001}]},{"osd":4,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.353},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.268,"15min":1.2949999999999999},"min":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1719999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.228,"15min":1.2430000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7749999999999999,"5min":1.778,"15min":1.837},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3759999999999999,"15min":1.3919999999999999},"min":{"1min":0.79500000000000004,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.222}]},{"osd":7,"last update":"Fri Oct 6 12:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.456,"15min":1.478},"min":{"1min":1.131,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9590000000000001,"5min":1.986,"15min":2.0840000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.448,"15min":1.46},"min":{"1min":1.1060000000000001,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":2.048,"5min":2.048,"15min":2.194},"last":1.254}]}]},{"osd":5,"up_from":13,"seq":55834575976,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2729999999999999,"15min":1.2749999999999999},"min":{"1min":0.85899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6890000000000001,"5min":1.6919999999999999,"15min":1.756},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3500000000000001,"15min":1.3640000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.718,"5min":1.718,"15min":2.657},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.145,"15min":1.1739999999999999},"min":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.61799999999999999},"max":{"1min":1.5449999999999999,"5min":1.7470000000000001,"15min":3.6880000000000002},"last":1.341},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1859999999999999,"15min":1.2170000000000001},"min":{"1min":0.89300000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.573,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1870000000000001,"15min":1.2170000000000001},"min":{"1min":0.86399999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.659,"5min":1.7549999999999999,"15min":2.754},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.274,"15min":1.284},"min":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6080000000000001,"5min":1.7010000000000001,"15min":3.8540000000000001},"last":1.224}]},{"osd":3,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2549999999999999,"15min":1.2829999999999999},"min":{"1min":1,"5min":0.746,"15min":0.746},"max":{"1min":1.5960000000000001,"5min":1.6519999999999999,"15min":1.8480000000000001},"last":1.395},{"interface":"front","average":{"1min":1.397,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":1.0960000000000001,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":2.9100000000000001},"last":1.2769999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2689999999999999,"15min":1.2669999999999999},"min":{"1min":0.91900000000000004,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":4.0899999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.387,"15min":1.395},"min":{"1min":1.1240000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6459999999999999,"5min":1.8400000000000001,"15min":2.2069999999999999},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.284,"15min":1.302},"min":{"1min":0.99299999999999999,"5min":0.68600000000000005,"15min":0.61699999999999999},"max":{"1min":1.6950000000000001,"5min":1.77,"15min":2.8300000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.3480000000000001,"15min":1.3600000000000001},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.76100000000000001},"max":{"1min":1.742,"5min":1.8280000000000001,"15min":4.1849999999999996},"last":1.4490000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3360000000000001,"15min":1.3520000000000001},"min":{"1min":0.99299999999999999,"5min":0.80300000000000005,"15min":0.76000000000000001},"max":{"1min":1.665,"5min":1.8740000000000001,"15min":2.5670000000000002},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":1.0600000000000001,"5min":0.97699999999999998,"15min":0.74099999999999999},"max":{"1min":1.7350000000000001,"5min":1.887,"15min":3.0049999999999999},"last":1.4890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.494},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.21},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.321}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.169},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.103}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.139},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.2849999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.355},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.266}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834575978,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.86299999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.268}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":1.409}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.621}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.597}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.653},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.679}]}]},{"osd":2,"up_from":13,"seq":55834575976,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316152,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055688,"statfs":{"total":96636764160,"available":96313024512,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.83299999999999996,"5min":0.54500000000000004,"15min":0.46200000000000002},"max":{"1min":1.373,"5min":1.4159999999999999,"15min":1.665},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.141,"15min":1.1240000000000001},"min":{"1min":0.89200000000000002,"5min":0.70199999999999996,"15min":0.68600000000000005},"max":{"1min":1.573,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.1459999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.91300000000000003,"15min":0.95999999999999996},"min":{"1min":0.73799999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.728},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0369999999999999,"15min":1.0549999999999999},"min":{"1min":0.81299999999999994,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":1.3859999999999999,"5min":2.194,"15min":2.194},"last":1.101}]},{"osd":3,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.226,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6719999999999999,"5min":1.7450000000000001,"15min":1.962},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.377,"15min":1.373},"min":{"1min":1.1970000000000001,"5min":0.79700000000000004,"15min":0.64400000000000002},"max":{"1min":1.7250000000000001,"5min":1.806,"15min":1.9510000000000001},"last":0.99199999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.282,"15min":1.266},"min":{"1min":0.94499999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.6890000000000001,"5min":1.706,"15min":1.897},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.8560000000000001},"last":1.3360000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.367,"15min":1.399},"min":{"1min":0.98399999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":1.976,"5min":1.9790000000000001,"15min":2.351},"last":1.224},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":1.022,"5min":0.97499999999999998,"15min":0.93700000000000006},"max":{"1min":1.75,"5min":1.8380000000000001,"15min":2.0619999999999998},"last":1.0740000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.401,"15min":1.4239999999999999},"min":{"1min":1.1739999999999999,"5min":0.95099999999999996,"15min":0.88900000000000001},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.0699999999999998},"last":1.397},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.474,"15min":1.4950000000000001},"min":{"1min":1.0820000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.206},"last":1.27}]},{"osd":7,"last update":"Fri Oct 6 12:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.419,"15min":1.4399999999999999},"min":{"1min":1.22,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4890000000000001,"15min":1.516},"min":{"1min":1.2769999999999999,"5min":1.056,"15min":0.94099999999999995},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.1520000000000001},"last":1.456}]}]},{"osd":1,"up_from":13,"seq":55834575976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.66300000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.244,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0189999999999999,"15min":1.028},"min":{"1min":0.71099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.94499999999999995}]},{"osd":2,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.026,"15min":0.999},"min":{"1min":0.72399999999999998,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.4419999999999999,"5min":1.573,"15min":1.6970000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.117,"15min":1.1339999999999999},"min":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.63700000000000001},"max":{"1min":1.694,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.554}]},{"osd":3,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1930000000000001,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.552,"5min":1.595,"15min":1.8580000000000001},"last":1.427},{"interface":"front","average":{"1min":1.125,"5min":1.1739999999999999,"15min":1.165},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.7,"15min":1.7609999999999999},"last":1.4019999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2829999999999999},"min":{"1min":0.91900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.6539999999999999,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.196},{"interface":"front","average":{"1min":1.454,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":1.78,"5min":2.093,"15min":2.093},"last":1.6639999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4019999999999999,"15min":1.397},"min":{"1min":0.88,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7589999999999999,"5min":1.887,"15min":2.052},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.722,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.357}]},{"osd":6,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.504,"15min":1.526},"min":{"1min":1.0469999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.903,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.615},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4670000000000001,"15min":1.4850000000000001},"min":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.474}]},{"osd":7,"last update":"Fri Oct 6 12:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.589,"15min":1.585},"min":{"1min":1.1599999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.542,"15min":1.53},"min":{"1min":1.1279999999999999,"5min":1.0269999999999999,"15min":0.99299999999999999},"max":{"1min":2.145,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.6970000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575977,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.91600000000000004,"15min":0.92300000000000004},"min":{"1min":0.56599999999999995,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.099,"5min":1.492,"15min":1.724},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0169999999999999,"15min":0.99399999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53900000000000003},"max":{"1min":1.593,"5min":1.669,"15min":1.669},"last":1.0049999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.0669999999999999,"15min":1.042},"min":{"1min":0.79600000000000004,"5min":0.70299999999999996,"15min":0.57599999999999996},"max":{"1min":1.5840000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.0920000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4259999999999999,"5min":1.528,"15min":2.028},"last":1.262}]},{"osd":3,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.127,"15min":1.129},"min":{"1min":0.89800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.829,"5min":1.829,"15min":2.004},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.091,"5min":0.98899999999999999,"15min":0.80800000000000005},"max":{"1min":1.724,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.22}]},{"osd":4,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3779999999999999,"15min":1.363},"min":{"1min":0.996,"5min":0.95499999999999996,"15min":0.61299999999999999},"max":{"1min":1.8280000000000001,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":1.357},{"interface":"front","average":{"1min":1.357,"5min":1.3700000000000001,"15min":1.371},"min":{"1min":1.179,"5min":0.98699999999999999,"15min":0.91200000000000003},"max":{"1min":1.6140000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.3069999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4430000000000001,"15min":1.4159999999999999},"min":{"1min":1.1659999999999999,"5min":0.82299999999999995,"15min":0.79600000000000004},"max":{"1min":1.899,"5min":2.012,"15min":2.7959999999999998},"last":1.748},{"interface":"front","average":{"1min":1.554,"5min":1.448,"15min":1.4339999999999999},"min":{"1min":1.258,"5min":1.0669999999999999,"15min":0.77400000000000002},"max":{"1min":2.036,"5min":2.0870000000000002,"15min":2.23},"last":1.829}]},{"osd":6,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4279999999999999,"15min":1.419},"min":{"1min":1.2010000000000001,"5min":0.749,"15min":0.749},"max":{"1min":1.6719999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.502,"15min":1.484},"min":{"1min":1.2330000000000001,"5min":1.0129999999999999,"15min":0.93400000000000005},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.802}]},{"osd":7,"last update":"Fri Oct 6 12:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.538,"15min":1.5109999999999999},"min":{"1min":1.258,"5min":0.96599999999999997,"15min":0.81999999999999995},"max":{"1min":2.194,"5min":2.194,"15min":2.194},"last":1.611},{"interface":"front","average":{"1min":1.675,"5min":1.5369999999999999,"15min":1.5069999999999999},"min":{"1min":1.274,"5min":1.127,"15min":0.89800000000000002},"max":{"1min":2.052,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.7150000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:53:07.954 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T11:18:10.604513+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=10, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.955 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.955 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.955 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.955 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.955 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.956 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.956 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.956 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T11:18:16.660943+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=18, tm_sec=16, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=12, tm_min=53, tm_sec=1, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T12:53:07.956 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T12:53:27.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T12:53:28.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.119+0000 7ff955ccd700 1 Processor -- start 2023-10-06T12:53:28.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.119+0000 7ff955ccd700 1 -- start start 2023-10-06T12:53:28.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff955ccd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff950161840 0x7ff9500caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff955ccd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500c18f0 con 0x7ff9500caf50 2023-10-06T12:53:28.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff955ccd700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9500c0ce0 con 0x7ff950161470 2023-10-06T12:53:28.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff955ccd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500c0fc0 con 0x7ff950161840 2023-10-06T12:53:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94effd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff950161840 0x7ff9500caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94ffff700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9500caf50 0x7ff95006cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44542/0 (socket says 172.21.15.46:44542) 2023-10-06T12:53:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94ffff700 1 -- 172.21.15.46:0/668049780 learned_addr learned my addr 172.21.15.46:0/668049780 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:28.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94effd700 1 -- 172.21.15.46:0/668049780 >> v1:172.21.15.132:6789/0 conn(0x7ff950161470 legacy=0x7ff9500c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94effd700 1 -- 172.21.15.46:0/668049780 >> v1:172.21.15.46:6789/0 conn(0x7ff9500caf50 legacy=0x7ff95006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94effd700 1 -- 172.21.15.46:0/668049780 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500c12a0 con 0x7ff950161840 2023-10-06T12:53:28.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94effd700 1 --2- 172.21.15.46:0/668049780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff950161840 0x7ff9500caa10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff9440098e0 tx=0x7ff9440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995f79cdb0f0086a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94e7fc700 1 -- 172.21.15.46:0/668049780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944009e80 con 0x7ff950161840 2023-10-06T12:53:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94e7fc700 1 -- 172.21.15.46:0/668049780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94400b8f0 con 0x7ff950161840 2023-10-06T12:53:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.123+0000 7ff94e7fc700 1 -- 172.21.15.46:0/668049780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94400bc70 con 0x7ff950161840 2023-10-06T12:53:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- 172.21.15.46:0/668049780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff950161840 msgr2=0x7ff9500caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 --2- 172.21.15.46:0/668049780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff950161840 0x7ff9500caa10 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff9440098e0 tx=0x7ff9440091a0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- 172.21.15.46:0/668049780 shutdown_connections 2023-10-06T12:53:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 --2- 172.21.15.46:0/668049780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff950161840 0x7ff9500caa10 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- 172.21.15.46:0/668049780 >> 172.21.15.46:0/668049780 conn(0x7ff9500cbe30 msgr2=0x7ff9500cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- 172.21.15.46:0/668049780 shutdown_connections 2023-10-06T12:53:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- 172.21.15.46:0/668049780 wait complete. 2023-10-06T12:53:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 Processor -- start 2023-10-06T12:53:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.127+0000 7ff955ccd700 1 -- start start 2023-10-06T12:53:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff950161470 0x7ff9500c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff950161840 0x7ff9500b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950171e80 con 0x7ff9500caf50 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95015ea20 con 0x7ff950161840 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff955ccd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500c3490 con 0x7ff950161470 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55708/0 (socket says 172.21.15.46:55708) 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 -- 172.21.15.46:0/1033357234 learned_addr learned my addr 172.21.15.46:0/1033357234 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94effd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff950161470 0x7ff9500c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94ffff700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff950161840 0x7ff9500b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 -- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff950161470 msgr2=0x7ff9500c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff950161470 0x7ff9500c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 -- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff950161840 msgr2=0x7ff9500b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff950161840 0x7ff9500b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 -- 172.21.15.46:0/1033357234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff944009420 con 0x7ff9500caf50 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff94f7fe700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c7c0 tx=0x7ff940007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.131+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940011590 con 0x7ff9500caf50 2023-10-06T12:53:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400116f0 con 0x7ff9500caf50 2023-10-06T12:53:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9501719f0 con 0x7ff9500caf50 2023-10-06T12:53:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940011ab0 con 0x7ff9500caf50 2023-10-06T12:53:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff950171680 con 0x7ff9500caf50 2023-10-06T12:53:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95006c7d0 con 0x7ff9500caf50 2023-10-06T12:53:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.135+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7ff94001b3b0 con 0x7ff9500caf50 2023-10-06T12:53:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.139+0000 7ff9377fe700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 0x7ff93c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.139+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7ff94007bfd0 con 0x7ff9500caf50 2023-10-06T12:53:28.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.139+0000 7ff94effd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 0x7ff93c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.139+0000 7ff94effd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 0x7ff93c062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9440091e0 tx=0x7ff944017000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.143+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff95006c7d0 con 0x7ff9500caf50 2023-10-06T12:53:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.315+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 --> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff95006c7d0 con 0x7ff93c060670 2023-10-06T12:53:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.323+0000 7ff9377fe700 1 -- 172.21.15.46:0/1033357234 <== mgr.10620 v2:172.21.15.46:6832/59347 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70098 (secure 0 0 0) 0x7ff95006c7d0 con 0x7ff93c060670 2023-10-06T12:53:28.330 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 msgr2=0x7ff93c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 0x7ff93c062b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9440091e0 tx=0x7ff944017000 comp rx=0 tx=0).stop 2023-10-06T12:53:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 msgr2=0x7ff95015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c7c0 tx=0x7ff940007f70 comp rx=0 tx=0).stop 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 shutdown_connections 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7ff93c060670 0x7ff93c062b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff950161840 0x7ff9500b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff950161470 0x7ff9500c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 --2- 172.21.15.46:0/1033357234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9500caf50 0x7ff95015e390 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 >> 172.21.15.46:0/1033357234 conn(0x7ff9500cbe30 msgr2=0x7ff95006a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.327+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 shutdown_connections 2023-10-06T12:53:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.331+0000 7ff955ccd700 1 -- 172.21.15.46:0/1033357234 wait complete. 2023-10-06T12:53:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-10-06T12:53:28.354 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T12:53:26.598329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2460744,"kb_used_data":21000,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752513976,"statfs":{"total":773094113280,"available":770574311424,"internally_reserved":0,"allocated":21504000,"data_stored":11994528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003532"},"pg_stats":[{"pgid":"1.0","version":"202'371","reported_seq":1225,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:08.867398+0000","last_change":"2023-10-06T12:53:08.867339+0000","last_active":"2023-10-06T12:53:08.867398+0000","last_peered":"2023-10-06T12:53:08.867398+0000","last_clean":"2023-10-06T12:53:08.867398+0000","last_became_active":"2023-10-06T11:18:12.182629+0000","last_became_peered":"2023-10-06T11:18:12.182629+0000","last_unstale":"2023-10-06T12:53:08.867398+0000","last_undegraded":"2023-10-06T12:53:08.867398+0000","last_fullsized":"2023-10-06T12:53:08.867398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'371","last_scrub_stamp":"2023-10-06T12:53:08.867214+0000","last_deep_scrub":"202'371","last_deep_scrub_stamp":"2023-10-06T12:53:08.867214+0000","last_clean_scrub_stamp":"2023-10-06T12:53:08.867214+0000","objects_scrubbed":2,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:08.844052+0000","scrub_duration":0.011719205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":197,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:10.634540+0000","last_change":"2023-10-06T12:53:10.634439+0000","last_active":"2023-10-06T12:53:10.634540+0000","last_peered":"2023-10-06T12:53:10.634540+0000","last_clean":"2023-10-06T12:53:10.634540+0000","last_became_active":"2023-10-06T11:18:17.688504+0000","last_became_peered":"2023-10-06T11:18:17.688504+0000","last_unstale":"2023-10-06T12:53:10.634540+0000","last_undegraded":"2023-10-06T12:53:10.634540+0000","last_fullsized":"2023-10-06T12:53:10.634540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:10.634174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:10.634174+0000","last_clean_scrub_stamp":"2023-10-06T12:53:10.634174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:49:09.308991+0000","scrub_duration":0.0068043299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:10.883353+0000","last_change":"2023-10-06T12:53:10.883291+0000","last_active":"2023-10-06T12:53:10.883353+0000","last_peered":"2023-10-06T12:53:10.883353+0000","last_clean":"2023-10-06T12:53:10.883353+0000","last_became_active":"2023-10-06T11:18:17.691978+0000","last_became_peered":"2023-10-06T11:18:17.691978+0000","last_unstale":"2023-10-06T12:53:10.883353+0000","last_undegraded":"2023-10-06T12:53:10.883353+0000","last_fullsized":"2023-10-06T12:53:10.883353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:10.883153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:10.883153+0000","last_clean_scrub_stamp":"2023-10-06T12:53:10.883153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:03:22.303369+0000","scrub_duration":0.006145406,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:11.269767+0000","last_change":"2023-10-06T12:53:11.269716+0000","last_active":"2023-10-06T12:53:11.269767+0000","last_peered":"2023-10-06T12:53:11.269767+0000","last_clean":"2023-10-06T12:53:11.269767+0000","last_became_active":"2023-10-06T11:18:18.239564+0000","last_became_peered":"2023-10-06T11:18:18.239564+0000","last_unstale":"2023-10-06T12:53:11.269767+0000","last_undegraded":"2023-10-06T12:53:11.269767+0000","last_fullsized":"2023-10-06T12:53:11.269767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:11.269574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:11.269574+0000","last_clean_scrub_stamp":"2023-10-06T12:53:11.269574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:29.543496+0000","scrub_duration":0.0058333090000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":197,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:07.557218+0000","last_change":"2023-10-06T12:53:07.557160+0000","last_active":"2023-10-06T12:53:07.557218+0000","last_peered":"2023-10-06T12:53:07.557218+0000","last_clean":"2023-10-06T12:53:07.557218+0000","last_became_active":"2023-10-06T11:18:17.691007+0000","last_became_peered":"2023-10-06T11:18:17.691007+0000","last_unstale":"2023-10-06T12:53:07.557218+0000","last_undegraded":"2023-10-06T12:53:07.557218+0000","last_fullsized":"2023-10-06T12:53:07.557218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:07.556972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:07.556972+0000","last_clean_scrub_stamp":"2023-10-06T12:53:07.556972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:52:21.843006+0000","scrub_duration":0.0054807709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":177,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:11.127058+0000","last_change":"2023-10-06T12:53:11.126986+0000","last_active":"2023-10-06T12:53:11.127058+0000","last_peered":"2023-10-06T12:53:11.127058+0000","last_clean":"2023-10-06T12:53:11.127058+0000","last_became_active":"2023-10-06T11:18:18.117972+0000","last_became_peered":"2023-10-06T11:18:18.117972+0000","last_unstale":"2023-10-06T12:53:11.127058+0000","last_undegraded":"2023-10-06T12:53:11.127058+0000","last_fullsized":"2023-10-06T12:53:11.127058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:11.126768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:11.126768+0000","last_clean_scrub_stamp":"2023-10-06T12:53:11.126768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:29.400690+0000","scrub_duration":0.0070844879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":174,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:13.873694+0000","last_change":"2023-10-06T12:53:13.873624+0000","last_active":"2023-10-06T12:53:13.873694+0000","last_peered":"2023-10-06T12:53:13.873694+0000","last_clean":"2023-10-06T12:53:13.873694+0000","last_became_active":"2023-10-06T11:18:18.121085+0000","last_became_peered":"2023-10-06T11:18:18.121085+0000","last_unstale":"2023-10-06T12:53:13.873694+0000","last_undegraded":"2023-10-06T12:53:13.873694+0000","last_fullsized":"2023-10-06T12:53:13.873694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:13.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:13.873411+0000","last_clean_scrub_stamp":"2023-10-06T12:53:13.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:35:16.018176+0000","scrub_duration":0.0071011470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":177,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:14.108232+0000","last_change":"2023-10-06T12:53:14.108130+0000","last_active":"2023-10-06T12:53:14.108232+0000","last_peered":"2023-10-06T12:53:14.108232+0000","last_clean":"2023-10-06T12:53:14.108232+0000","last_became_active":"2023-10-06T11:18:17.693003+0000","last_became_peered":"2023-10-06T11:18:17.693003+0000","last_unstale":"2023-10-06T12:53:14.108232+0000","last_undegraded":"2023-10-06T12:53:14.108232+0000","last_fullsized":"2023-10-06T12:53:14.108232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:14.107887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:14.107887+0000","last_clean_scrub_stamp":"2023-10-06T12:53:14.107887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:03:25.528103+0000","scrub_duration":0.0058490959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":175,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-06T12:53:10.942418+0000","last_change":"2023-10-06T12:53:10.942379+0000","last_active":"2023-10-06T12:53:10.942418+0000","last_peered":"2023-10-06T12:53:10.942418+0000","last_clean":"2023-10-06T12:53:10.942418+0000","last_became_active":"2023-10-06T11:18:17.692439+0000","last_became_peered":"2023-10-06T11:18:17.692439+0000","last_unstale":"2023-10-06T12:53:10.942418+0000","last_undegraded":"2023-10-06T12:53:10.942418+0000","last_fullsized":"2023-10-06T12:53:10.942418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T12:53:10.942256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T12:53:10.942256+0000","last_clean_scrub_stamp":"2023-10-06T12:53:10.942256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:14:48.260838+0000","scrub_duration":0.0034713220000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.139,"15min":1.159},"min":{"1min":0.72799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6120000000000001,"5min":1.758,"15min":1.758},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.155},"min":{"1min":0.73199999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.661,"5min":1.72,"15min":1.8660000000000001},"last":1.3109999999999999}]},{"osd":1,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.343,"15min":1.361},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6439999999999999,"5min":1.8420000000000001,"15min":1.885},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2709999999999999,"15min":1.276},"min":{"1min":0.873,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.597,"5min":1.6970000000000001,"15min":1.76},"last":1.2709999999999999}]},{"osd":2,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.2809999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.56299999999999994},"max":{"1min":1.788,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.254,"15min":1.2669999999999999},"min":{"1min":0.92900000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.05}]},{"osd":3,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.329,"15min":1.3049999999999999},"min":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":2.105},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.81999999999999995},"max":{"1min":1.7569999999999999,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.94999999999999996}]},{"osd":4,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1319999999999999,"15min":1.113},"min":{"1min":0.73099999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5329999999999999,"5min":1.667,"15min":1.681},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.29,"15min":1.2889999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.2989999999999999},"last":1.5620000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3169999999999999,"15min":1.335},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.68300000000000005},"max":{"1min":1.998,"5min":2.036,"15min":2.036},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.325,"15min":1.3220000000000001},"min":{"1min":0.80400000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.7569999999999999,"5min":1.883,"15min":1.9099999999999999},"last":1.448}]},{"osd":6,"last update":"Fri Oct 6 12:52:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":0.92500000000000004,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.873,"5min":1.873,"15min":2.0009999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.462,"15min":1.456},"min":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2839999999999998,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":1.3839999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.2629999999999999,"15min":1.2909999999999999},"min":{"1min":0.80100000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4390000000000001,"5min":1.7370000000000001,"15min":1.857},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2749999999999999,"15min":1.306},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5880000000000001,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.3700000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.177,"15min":1.177},"min":{"1min":0.78000000000000003,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.7789999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.218,"15min":1.206},"min":{"1min":0.89600000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5920000000000001,"5min":2.105,"15min":2.105},"last":1.0960000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.167,"15min":1.1950000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.68500000000000005},"max":{"1min":1.788,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.24,"15min":1.2410000000000001},"min":{"1min":1.0289999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5229999999999999,"5min":1.671,"15min":1.671},"last":1.296}]},{"osd":3,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.254,"15min":1.302},"min":{"1min":0.75900000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.661,"5min":1.76,"15min":2.0649999999999999},"last":1.419},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.99399999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.8560000000000001},"last":1.484}]},{"osd":4,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2330000000000001,"15min":1.238},"min":{"1min":0.86499999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.865,"5min":2.395,"15min":2.395},"last":1.323},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":1,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9339999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2529999999999999,"15min":1.2430000000000001},"min":{"1min":0.93100000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7729999999999999,"5min":1.7749999999999999,"15min":1.837},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.395,"15min":1.3939999999999999},"min":{"1min":1.145,"5min":0.76400000000000001,"15min":0.75900000000000001},"max":{"1min":1.833,"5min":2.286,"15min":2.286},"last":1.518}]},{"osd":7,"last update":"Fri Oct 6 12:53:14 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4670000000000001,"15min":1.4810000000000001},"min":{"1min":0.99299999999999999,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9339999999999999,"5min":1.9590000000000001,"15min":2.0840000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.462,"15min":1.4610000000000001},"min":{"1min":1.1879999999999999,"5min":0.98199999999999998,"15min":0.86699999999999999},"max":{"1min":1.883,"5min":2.048,"15min":2.194},"last":1.899}]}]},{"osd":5,"up_from":13,"seq":55834575980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.274},"min":{"1min":0.83899999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.613,"5min":1.6919999999999999,"15min":1.756},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.335,"15min":1.359},"min":{"1min":0.97099999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.599,"5min":1.718,"15min":2.657},"last":1.206}]},{"osd":1,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1679999999999999,"15min":1.1799999999999999},"min":{"1min":0.83499999999999996,"5min":0.81499999999999995,"15min":0.61799999999999999},"max":{"1min":1.4970000000000001,"5min":1.5449999999999999,"15min":3.6880000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1950000000000001,"15min":1.212},"min":{"1min":0.80200000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.73,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.4830000000000001}]},{"osd":2,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1879999999999999,"15min":1.222},"min":{"1min":0.76700000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.617,"5min":1.659,"15min":2.754},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.89400000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":3.8540000000000001},"last":1.276}]},{"osd":3,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2689999999999999,"15min":1.2809999999999999},"min":{"1min":0.82499999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.8480000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.363,"15min":1.357},"min":{"1min":0.96299999999999997,"5min":0.89600000000000002,"15min":0.86899999999999999},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":2.9100000000000001},"last":1.726}]},{"osd":4,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.292,"15min":1.2789999999999999},"min":{"1min":0.90900000000000003,"5min":0.80500000000000005,"15min":0.64800000000000002},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":4.0899999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.401,"15min":1.395},"min":{"1min":0.82099999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.966,"5min":1.966,"15min":2.2069999999999999},"last":1.609}]},{"osd":6,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2949999999999999,"15min":1.302},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.61699999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.8300000000000001},"last":1.579},{"interface":"front","average":{"1min":1.403,"5min":1.345,"15min":1.359},"min":{"1min":0.94399999999999995,"5min":0.93300000000000005,"15min":0.76100000000000001},"max":{"1min":1.819,"5min":1.819,"15min":4.1849999999999996},"last":1.5409999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:53:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3420000000000001,"15min":1.3500000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.76000000000000001},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":2.5670000000000002},"last":1.659},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4350000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.74099999999999999},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":3.0049999999999999},"last":1.6699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.173,"15min":1.1870000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.468,"5min":1.851,"15min":1.851},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.202},"min":{"1min":0.72099999999999997,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7969999999999999},"last":0.97399999999999998}]},{"osd":1,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.641,"5min":1.8120000000000001,"15min":1.8779999999999999},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.3149999999999999,"15min":1.3129999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5229999999999999,"5min":1.948,"15min":1.948},"last":1.383}]},{"osd":2,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7250000000000001,"5min":2.024,"15min":2.024},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.284,"15min":1.296},"min":{"1min":0.754,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.609,"5min":1.917,"15min":1.917},"last":1.298}]},{"osd":3,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2410000000000001,"15min":1.2609999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5700000000000001,"5min":1.7430000000000001,"15min":1.768},"last":1.145},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.339,"15min":1.3340000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.657,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.47}]},{"osd":5,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5529999999999999,"5min":2.012,"15min":2.012},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.284,"15min":1.246},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5700000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.028}]},{"osd":6,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.2849999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.694,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.419,"15min":1.405},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.758,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.415,"15min":1.3740000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.673,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.268},{"interface":"front","average":{"1min":1.458,"5min":1.4570000000000001,"15min":1.4390000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.65600000000000003},"max":{"1min":1.768,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2549999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299768,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072072,"statfs":{"total":96636764160,"available":96329801728,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.92400000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.4139999999999999,"15min":1.605},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.995,"15min":1.02},"min":{"1min":0.67000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3500000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.095}]},{"osd":1,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93700000000000006,"15min":0.97599999999999998},"min":{"1min":0.59699999999999998,"5min":0.57299999999999995,"15min":0.53100000000000003},"max":{"1min":1.2090000000000001,"5min":1.5049999999999999,"15min":1.639},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.113,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55400000000000005},"max":{"1min":1.381,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.258}]},{"osd":2,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1419999999999999,"15min":1.161},"min":{"1min":0.628,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6160000000000001,"5min":1.641,"15min":1.8779999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.177,"5min":1.208,"15min":1.228},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.63900000000000001},"max":{"1min":1.5369999999999999,"5min":1.738,"15min":1.768},"last":0.98799999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.238,"15min":1.2609999999999999},"min":{"1min":0.97299999999999998,"5min":0.94499999999999995,"15min":0.76600000000000001},"max":{"1min":1.611,"5min":1.611,"15min":2.008},"last":1.216},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.381},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.782,"5min":1.9970000000000001,"15min":2.2770000000000001},"last":1.518}]},{"osd":5,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.425,"15min":1.4430000000000001},"min":{"1min":1.0980000000000001,"5min":0.92200000000000004,"15min":0.82299999999999995},"max":{"1min":1.754,"5min":2.1309999999999998,"15min":2.1680000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4419999999999999,"15min":1.4470000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.64200000000000002},"max":{"1min":2.0099999999999998,"5min":2.0449999999999999,"15min":2.1930000000000001},"last":1.391}]},{"osd":6,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.488},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.72199999999999998},"max":{"1min":2.0899999999999999,"5min":2.1589999999999998,"15min":2.3119999999999998},"last":1.45},{"interface":"front","average":{"1min":1.458,"5min":1.419,"15min":1.415},"min":{"1min":1.0629999999999999,"5min":0.90500000000000003,"15min":0.84099999999999997},"max":{"1min":1.8160000000000001,"5min":1.9159999999999999,"15min":1.929},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Oct 6 12:52:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4530000000000001,"15min":1.4690000000000001},"min":{"1min":1.04,"5min":1.016,"15min":0.95099999999999996},"max":{"1min":1.893,"5min":2.085,"15min":2.0899999999999999},"last":1.468},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5029999999999999,"15min":1.54},"min":{"1min":1.2090000000000001,"5min":0.877,"15min":0.877},"max":{"1min":2.0289999999999999,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.4299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316152,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055688,"statfs":{"total":96636764160,"available":96313024512,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.0269999999999999,"15min":0.997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.46200000000000002},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1499999999999999,"15min":1.125},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":0.96299999999999997,"15min":0.97299999999999998},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.3860000000000001,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0780000000000001,"15min":1.0620000000000001},"min":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.45100000000000001},"max":{"1min":5.282,"5min":5.282,"15min":5.282},"last":1.369}]},{"osd":3,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.25,"15min":1.272},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":1.835},{"interface":"front","average":{"1min":1.351,"5min":1.371,"15min":1.365},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.64400000000000002},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.7749999999999999}]},{"osd":4,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.319,"15min":1.278},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":6.5890000000000004,"5min":6.5890000000000004,"15min":6.5890000000000004},"last":1.397},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.3560000000000001,"15min":1.355},"min":{"1min":0.90000000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":6.2069999999999999,"5min":6.2069999999999999,"15min":6.2069999999999999},"last":1.7250000000000001}]},{"osd":5,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.413,"15min":1.411},"min":{"1min":0.93600000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":7.5170000000000003,"5min":7.5170000000000003,"15min":7.5170000000000003},"last":1.645},{"interface":"front","average":{"1min":1.75,"5min":1.4690000000000001,"15min":1.423},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":10.026999999999999,"5min":10.026999999999999,"15min":10.026999999999999},"last":1.679}]},{"osd":6,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.4350000000000001,"15min":1.431},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":5.569,"5min":5.569,"15min":5.569},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.512,"15min":1.504},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":7.556,"5min":7.556,"15min":7.556},"last":1.986}]},{"osd":7,"last update":"Fri Oct 6 12:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.4710000000000001,"15min":1.4510000000000001},"min":{"1min":0.92800000000000005,"5min":0.86599999999999999,"15min":0.753},"max":{"1min":6.6390000000000002,"5min":6.6390000000000002,"15min":6.6390000000000002},"last":1.623},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.506,"15min":1.5169999999999999},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.9359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312056,"kb_used_data":2480,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059784,"statfs":{"total":96636764160,"available":96317218816,"internally_reserved":0,"allocated":2539520,"data_stored":1351724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.94199999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5349999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.64500000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.403,"5min":1.976,"15min":1.976},"last":0.96699999999999997}]},{"osd":2,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.70299999999999996,"5min":0.63600000000000001,"15min":0.56499999999999995},"max":{"1min":1.093,"5min":1.573,"15min":1.6970000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.03,"5min":1.1020000000000001,"15min":1.127},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.63700000000000001},"max":{"1min":1.554,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1819999999999999,"15min":1.25},"min":{"1min":0.73399999999999999,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5549999999999999,"5min":1.595,"15min":1.8580000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1619999999999999,"15min":1.163},"min":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.421,"5min":1.7,"15min":1.7609999999999999},"last":1.121}]},{"osd":4,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2909999999999999,"15min":1.2829999999999999},"min":{"1min":0.94099999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.819,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.01},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.415,"15min":1.4390000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.6819999999999999,"5min":1.869,"15min":2.093},"last":1.55}]},{"osd":5,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.391,"15min":1.3939999999999999},"min":{"1min":0.99299999999999999,"5min":0.82899999999999996,"15min":0.81599999999999995},"max":{"1min":1.7509999999999999,"5min":1.887,"15min":2.052},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.383,"15min":1.4039999999999999},"min":{"1min":0.97699999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.71,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":1.2170000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.486,"15min":1.52},"min":{"1min":1.123,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.6339999999999999,"5min":1.9630000000000001,"15min":2.1680000000000001},"last":1.46},{"interface":"front","average":{"1min":1.417,"5min":1.4570000000000001,"15min":1.4790000000000001},"min":{"1min":1.091,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.7789999999999999,"5min":2.1030000000000002,"15min":2.1339999999999999},"last":1.28}]},{"osd":7,"last update":"Fri Oct 6 12:53:15 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.577,"15min":1.5840000000000001},"min":{"1min":1.1359999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8520000000000001,"5min":2.2290000000000001,"15min":2.2480000000000002},"last":1.5},{"interface":"front","average":{"1min":1.403,"5min":1.5209999999999999,"15min":1.53},"min":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":0.99299999999999999},"max":{"1min":1.7210000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.399}]}]},{"osd":0,"up_from":13,"seq":55834575981,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304444,"kb_used_data":3060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067396,"statfs":{"total":96636764160,"available":96325013504,"internally_reserved":0,"allocated":3133440,"data_stored":1942092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.91200000000000003,"15min":0.91600000000000004},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.4590000000000001,"5min":1.492,"15min":1.724},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.006,"15min":0.98999999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2749999999999999,"5min":1.669,"15min":1.669},"last":1.022}]},{"osd":2,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.077,"15min":1.036},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.57599999999999996},"max":{"1min":1.528,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0860000000000001,"15min":1.0900000000000001},"min":{"1min":0.627,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":2.028},"last":1.1519999999999999}]},{"osd":3,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.61099999999999999},"max":{"1min":1.337,"5min":1.829,"15min":2.004},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3540000000000001,"15min":1.3640000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.80800000000000005},"max":{"1min":1.7210000000000001,"5min":1.8140000000000001,"15min":2.4620000000000002},"last":1.609}]},{"osd":4,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3620000000000001,"15min":1.359},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.61299999999999999},"max":{"1min":1.6359999999999999,"5min":1.9430000000000001,"15min":2.0880000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3600000000000001,"15min":1.365},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.8320000000000001,"5min":2.1150000000000002,"15min":2.387},"last":1.3879999999999999}]},{"osd":5,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.425,"15min":1.411},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.748,"5min":1.899,"15min":2.7959999999999998},"last":1.212},{"interface":"front","average":{"1min":1.389,"5min":1.4430000000000001,"15min":1.4319999999999999},"min":{"1min":0.875,"5min":0.875,"15min":0.77400000000000002},"max":{"1min":1.829,"5min":2.0870000000000002,"15min":2.23},"last":1.6850000000000001}]},{"osd":6,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.417},"min":{"1min":1.002,"5min":1.002,"15min":0.749},"max":{"1min":1.7569999999999999,"5min":1.9830000000000001,"15min":2.0910000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.4910000000000001,"15min":1.478},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.901,"5min":1.9930000000000001,"15min":2.7530000000000001},"last":1.5009999999999999}]},{"osd":7,"last update":"Fri Oct 6 12:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4379999999999999,"5min":1.526,"15min":1.506},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":0.81999999999999995},"max":{"1min":1.7230000000000001,"5min":2.194,"15min":2.194},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.532,"15min":1.504},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.974,"5min":2.0569999999999999,"15min":2.2530000000000001},"last":1.54}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T12:53:28.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T12:53:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.527+0000 7f92b90cd700 1 Processor -- start 2023-10-06T12:53:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.527+0000 7f92b90cd700 1 -- start start 2023-10-06T12:53:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b90cd700 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b40b23c0 0x7f92b40b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b90cd700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 0x7f92b40cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b90cd700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40cc510 con 0x7f92b40ccb40 2023-10-06T12:53:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b90cd700 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40cb900 con 0x7f92b40b23c0 2023-10-06T12:53:28.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b90cd700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40cbbe0 con 0x7f92b40cae40 2023-10-06T12:53:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b359e700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92b40ccb40 0x7f92b415cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44548/0 (socket says 172.21.15.46:44548) 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b359e700 1 -- 172.21.15.46:0/1990964302 learned_addr learned my addr 172.21.15.46:0/1990964302 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92abfff700 1 --2- 172.21.15.46:0/1990964302 >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 0x7f92b40cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/1990964302 >> v2:172.21.15.46:3300/0 conn(0x7f92b40b23c0 0x7f92b40b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083203479 0 0) 0x7f92b40cc510 con 0x7f92b40ccb40 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a0003570 con 0x7f92b40ccb40 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 -- 172.21.15.46:0/1990964302 >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 msgr2=0x7f92b40cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/1990964302 >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 0x7f92b40cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 -- 172.21.15.46:0/1990964302 >> v1:172.21.15.46:6789/0 conn(0x7f92b40ccb40 legacy=0x7f92b415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738868998 0 0) 0x7f92a0003570 con 0x7f92b40ccb40 2023-10-06T12:53:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 -- 172.21.15.46:0/1990964302 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40cbec0 con 0x7f92b40b23c0 2023-10-06T12:53:28.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92abfff700 1 --2- 172.21.15.46:0/1990964302 >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 0x7f92b40cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T12:53:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/1990964302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40b23c0 0x7f92b40b2790 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f92b4089420 tx=0x7f92a4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9c0674126a48cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0017f0 con 0x7f92b40b23c0 2023-10-06T12:53:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4011420 con 0x7f92b40b23c0 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.531+0000 7f92b259c700 1 -- 172.21.15.46:0/1990964302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40117c0 con 0x7f92b40b23c0 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- 172.21.15.46:0/1990964302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40b23c0 msgr2=0x7f92b40b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 --2- 172.21.15.46:0/1990964302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40b23c0 0x7f92b40b2790 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f92b4089420 tx=0x7f92a4009120 comp rx=0 tx=0).stop 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- 172.21.15.46:0/1990964302 shutdown_connections 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 --2- 172.21.15.46:0/1990964302 >> v2:172.21.15.132:3300/0 conn(0x7f92b40cae40 0x7f92b40cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 --2- 172.21.15.46:0/1990964302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40b23c0 0x7f92b40b2790 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- 172.21.15.46:0/1990964302 >> 172.21.15.46:0/1990964302 conn(0x7f92b4099480 msgr2=0x7f92b406ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- 172.21.15.46:0/1990964302 shutdown_connections 2023-10-06T12:53:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- 172.21.15.46:0/1990964302 wait complete. 2023-10-06T12:53:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 Processor -- start 2023-10-06T12:53:28.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.535+0000 7f92b90cd700 1 -- start start 2023-10-06T12:53:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92b40b23c0 0x7f92b409af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92b40cae40 0x7f92b40c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4159df0 con 0x7f92b40ccb40 2023-10-06T12:53:28.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40b5ed0 con 0x7f92b40cae40 2023-10-06T12:53:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4159ac0 con 0x7f92b40b23c0 2023-10-06T12:53:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55714/0 (socket says 172.21.15.46:55714) 2023-10-06T12:53:28.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 -- 172.21.15.46:0/3920677273 learned_addr learned my addr 172.21.15.46:0/3920677273 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92b40b23c0 0x7f92b409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92abfff700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92b40cae40 0x7f92b40c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 -- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92b40b23c0 msgr2=0x7f92b409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92b40b23c0 0x7f92b409af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 -- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92b40cae40 msgr2=0x7f92b40c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92b40cae40 0x7f92b40c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 -- 172.21.15.46:0/3920677273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c009420 con 0x7f92b40ccb40 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b359e700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f92ac001560 tx=0x7f92ac001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004dc0 con 0x7f92b40ccb40 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00a6a0 con 0x7f92b40ccb40 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00aa60 con 0x7f92b40ccb40 2023-10-06T12:53:28.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40093c0 con 0x7f92b40ccb40 2023-10-06T12:53:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.539+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b415c910 con 0x7f92b40ccb40 2023-10-06T12:53:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.543+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b4159f60 con 0x7f92b40ccb40 2023-10-06T12:53:28.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.543+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f92ac007150 con 0x7f92b40ccb40 2023-10-06T12:53:28.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.543+0000 7f92aaffd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 0x7f9294062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.547+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f92ac07c2e0 con 0x7f92b40ccb40 2023-10-06T12:53:28.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.547+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 0x7f9294062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.547+0000 7f92b2d9d700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 0x7f9294062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92a4003080 tx=0x7f92a4016000 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.551+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f92ac04c480 con 0x7f92b40ccb40 2023-10-06T12:53:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.731+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f92b40660f0 con 0x7f92b40ccb40 2023-10-06T12:53:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.739+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f92ac04c480 con 0x7f92b40ccb40 2023-10-06T12:53:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.743+0000 7f92aaffd700 1 -- 172.21.15.46:0/3920677273 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v76)=0 v76) v1 ==== 123+0+0 (secure 0 0 0) 0x7f92ac04faa0 con 0x7f92b40ccb40 2023-10-06T12:53:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 msgr2=0x7f9294062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 0x7f9294062b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92a4003080 tx=0x7f92a4016000 comp rx=0 tx=0).stop 2023-10-06T12:53:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 msgr2=0x7f92b415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f92ac001560 tx=0x7f92ac001640 comp rx=0 tx=0).stop 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 shutdown_connections 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b40ccb40 0x7f92b415b7e0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f9294060670 0x7f9294062b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92b40cae40 0x7f92b40c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 --2- 172.21.15.46:0/3920677273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92b40b23c0 0x7f92b409af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 >> 172.21.15.46:0/3920677273 conn(0x7f92b4099480 msgr2=0x7f92b40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 shutdown_connections 2023-10-06T12:53:28.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.747+0000 7f92b90cd700 1 -- 172.21.15.46:0/3920677273 wait complete. 2023-10-06T12:53:28.769 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T12:53:28.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T12:53:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.923+0000 7f987e923700 1 Processor -- start 2023-10-06T12:53:28.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.927+0000 7f987e923700 1 -- start start 2023-10-06T12:53:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.927+0000 7f987e923700 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98780bf1a0 0x7f98781683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e923700 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9878069660 con 0x7f9878067760 2023-10-06T12:53:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e923700 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9878068ac0 con 0x7f98780bedd0 2023-10-06T12:53:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e923700 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9878068d30 con 0x7f98780bf1a0 2023-10-06T12:53:28.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9878067760 0x7f98780c7a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44560/0 (socket says 172.21.15.46:44560) 2023-10-06T12:53:28.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 -- 172.21.15.46:0/2713610732 learned_addr learned my addr 172.21.15.46:0/2713610732 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-10-06T12:53:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e122700 1 --2- 172.21.15.46:0/2713610732 >> v2:172.21.15.132:3300/0 conn(0x7f98780bf1a0 0x7f98781683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629057058 0 0) 0x7f9878069660 con 0x7f9878067760 2023-10-06T12:53:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9864003570 con 0x7f9878067760 2023-10-06T12:53:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649999091 0 0) 0x7f9864003570 con 0x7f9878067760 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 >> v2:172.21.15.132:3300/0 conn(0x7f98780bf1a0 msgr2=0x7f98781683d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 --2- 172.21.15.46:0/2713610732 >> v2:172.21.15.132:3300/0 conn(0x7f98780bf1a0 0x7f98781683d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 >> v1:172.21.15.132:6789/0 conn(0x7f98780bedd0 legacy=0x7f9878067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878069010 con 0x7f9878067760 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2475628043 0 0) 0x7f9868001470 con 0x7f9878067760 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 >> v1:172.21.15.46:6789/0 conn(0x7f9878067760 legacy=0x7f98780c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9864003a60 0x7f9864003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f9864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98640051c0 0x7f986400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878068d30 con 0x7f9864004810 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c0d0 con 0x7f9864003a60 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986400c290 con 0x7f98640051c0 2023-10-06T12:53:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f9864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e122700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98640051c0 0x7f986400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d921700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9864003a60 0x7f9864003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 -- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98640051c0 msgr2=0x7f986400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98640051c0 0x7f986400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 -- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9864003a60 msgr2=0x7f9864003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9864003a60 0x7f9864003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 -- 172.21.15.46:0/2713610732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864003570 con 0x7f9864004810 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e122700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98640051c0 0x7f986400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987e122700 1 -- 172.21.15.46:0/2713610732 reap_dead start 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987d120700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f9864004c80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f986800c3c0 tx=0x7f9868001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf670b87b12dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800cc90 con 0x7f9864004810 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f986800cdf0 con 0x7f9864004810 2023-10-06T12:53:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.931+0000 7f987c91f700 1 -- 172.21.15.46:0/2713610732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868005b70 con 0x7f9864004810 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 msgr2=0x7f9864004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f9864004c80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f986800c3c0 tx=0x7f9868001e50 comp rx=0 tx=0).stop 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- 172.21.15.46:0/2713610732 shutdown_connections 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 --2- 172.21.15.46:0/2713610732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f9864004c80 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- 172.21.15.46:0/2713610732 >> 172.21.15.46:0/2713610732 conn(0x7f9878062ef0 msgr2=0x7f98780cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- 172.21.15.46:0/2713610732 shutdown_connections 2023-10-06T12:53:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- 172.21.15.46:0/2713610732 wait complete. 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 Processor -- start 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.935+0000 7f987e923700 1 -- start start 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 0x7f98780c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98780c7e80 0x7f987816dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780c8320 con 0x7f9864004810 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98780bdef0 con 0x7f98780be380 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780b1960 con 0x7f98780c7e80 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55736/0 (socket says 172.21.15.46:55736) 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 -- 172.21.15.46:0/1999582227 learned_addr learned my addr 172.21.15.46:0/1999582227 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e122700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98780c7e80 0x7f987816dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d120700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 0x7f98780c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 -- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98780c7e80 msgr2=0x7f987816dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98780c7e80 0x7f987816dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 -- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 msgr2=0x7f98780c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 0x7f98780c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 -- 172.21.15.46:0/1999582227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987816eaf0 con 0x7f9864004810 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d120700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 0x7f98780c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987d921700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f987400e580 tx=0x7f9874009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012590 con 0x7f9864004810 2023-10-06T12:53:28.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f98740126f0 con 0x7f9864004810 2023-10-06T12:53:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98780bd8a0 con 0x7f9864004810 2023-10-06T12:53:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.939+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874012ab0 con 0x7f9864004810 2023-10-06T12:53:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.943+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98780be7f0 con 0x7f9864004810 2023-10-06T12:53:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.943+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98780670e0 con 0x7f9864004810 2023-10-06T12:53:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.943+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 119) v1 ==== 80061+0+0 (secure 0 0 0) 0x7f987401d460 con 0x7f9864004810 2023-10-06T12:53:28.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.947+0000 7f986dffb700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 0x7f985c062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T12:53:28.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.947+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6509+0+0 (secure 0 0 0) 0x7f987407ccd0 con 0x7f9864004810 2023-10-06T12:53:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.947+0000 7f987d120700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 0x7f985c062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T12:53:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.947+0000 7f987d120700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 0x7f985c062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9868015040 tx=0x7f9868001e50 comp rx=0 tx=0).ready entity=mgr.10620 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T12:53:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:28.951+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98780670e0 con 0x7f9864004810 2023-10-06T12:53:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.167+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98780670e0 con 0x7f9864004810 2023-10-06T12:53:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.167+0000 7f986dffb700 1 -- 172.21.15.46:0/1999582227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3540 (secure 0 0 0) 0x7f98780670e0 con 0x7f9864004810 2023-10-06T12:53:29.172 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-10-06T12:53:29.172 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3248103492","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3248103492},{"type":"v1","addr":"172.21.15.132:6835","nonce":3248103492}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10087,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/4251566090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4251566090},{"type":"v1","addr":"172.21.15.132:6833","nonce":4251566090}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10090,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/2609471644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":2609471644},{"type":"v1","addr":"172.21.15.46:6837","nonce":2609471644}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":10734,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1670475004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670475004},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670475004}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":105}],"filesystems":[]} 2023-10-06T12:53:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 msgr2=0x7f985c062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 0x7f985c062b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9868015040 tx=0x7f9868001e50 comp rx=0 tx=0).stop 2023-10-06T12:53:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 msgr2=0x7f98780b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T12:53:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f987400e580 tx=0x7f9874009040 comp rx=0 tx=0).stop 2023-10-06T12:53:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 shutdown_connections 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98780c7e80 0x7f987816dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:6832/59347,v1:172.21.15.46:6833/59347] conn(0x7f985c060670 0x7f985c062b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98780be380 0x7f98780c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 --2- 172.21.15.46:0/1999582227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9864004810 0x7f98780b1420 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 >> 172.21.15.46:0/1999582227 conn(0x7f9878062ef0 msgr2=0x7f987815cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 shutdown_connections 2023-10-06T12:53:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-10-06T12:53:29.171+0000 7f987e923700 1 -- 172.21.15.46:0/1999582227 wait complete. 2023-10-06T12:53:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 105 2023-10-06T12:53:29.197 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T12:53:29.197 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T12:53:29.197 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T12:53:29.197 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:29.218 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T12:53:29.219 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T12:53:29.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:29.288 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T12:53:29.289 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T12:53:29.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:29.310 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T12:53:29.311 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T12:53:29.311 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:29.373 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T12:53:29.373 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T12:53:29.373 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T12:53:29.373 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:35.475 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T12:53:35.475 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T12:53:35.476 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:35.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-06T12:53:35.577 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T12:53:35.577 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T12:53:35.577 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:35.674 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T12:53:35.674 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T12:53:35.674 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:37.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:53:37.507 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:53:41.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-06T12:53:41.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-10-06T12:53:41.776 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T12:53:41.776 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T12:53:41.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:47.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-10-06T12:53:47.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-06T12:53:47.878 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T12:53:47.878 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T12:53:47.878 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:47.975 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T12:53:47.975 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T12:53:47.975 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:48.074 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T12:53:48.075 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T12:53:48.075 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:52.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-06T12:53:52.747 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-10-06T12:53:54.176 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T12:53:54.176 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T12:53:54.177 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T12:53:54.177 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:54.243 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T12:53:54.244 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T12:53:54.244 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:54.305 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T12:53:54.305 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T12:53:54.305 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T12:53:54.305 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:54.354 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T12:53:54.354 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T12:53:54.354 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:54.389 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T12:53:54.390 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T12:53:54.390 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T12:53:54.470 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T12:53:54.470 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T12:53:54.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T12:53:54.516 INFO:teuthology.orchestra.run.smithi046.stdout:1696591103.0490196 mon.a (mon.0) 220 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T12:53:54.517 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T12:53:54.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:53:54.518 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T12:53:55.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:53:55.047 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T12:53:55.463 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:53:55.463 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T12:53:55.867 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:53:55.867 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T12:53:56.351 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T12:53:56.351 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T12:53:56.769 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T12:53:56.770 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T12:53:57.164 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T12:53:57.166 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T12:53:57.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T12:53:57.548 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T12:53:57.748 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-06T12:53:57.977 INFO:tasks.ceph:Archiving mon data... 2023-10-06T12:53:57.979 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/data/mon.a.tgz 2023-10-06T12:53:57.979 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-10-06T12:53:57.996 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.FDNMzMXucS 2023-10-06T12:53:57.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FDNMzMXucS 2023-10-06T12:53:58.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.FDNMzMXucS 2023-10-06T12:53:58.572 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.FDNMzMXucS is 2MB 2023-10-06T12:53:58.686 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.FDNMzMXucS 2023-10-06T12:53:58.705 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/data/mon.b.tgz 2023-10-06T12:53:58.705 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2023-10-06T12:53:58.724 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.Tx7DP4qAaf 2023-10-06T12:53:58.725 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Tx7DP4qAaf 2023-10-06T12:53:59.195 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.Tx7DP4qAaf 2023-10-06T12:53:59.292 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.Tx7DP4qAaf is 2MB 2023-10-06T12:53:59.399 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.Tx7DP4qAaf 2023-10-06T12:53:59.418 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/data/mon.c.tgz 2023-10-06T12:53:59.419 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2023-10-06T12:53:59.481 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.As3YEXQ1wl 2023-10-06T12:53:59.482 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.As3YEXQ1wl 2023-10-06T12:53:59.952 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.As3YEXQ1wl 2023-10-06T12:54:00.049 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.As3YEXQ1wl is 2MB 2023-10-06T12:54:00.155 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.As3YEXQ1wl 2023-10-06T12:54:00.174 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T12:54:00.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T12:54:00.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T12:54:00.248 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T12:54:00.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T12:54:00.335 DEBUG:teuthology.orchestra.run.smithi132:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T12:54:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T12:54:00.392 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T12:54:00.404 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T12:54:00.406 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi046/crash 2023-10-06T12:54:00.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T12:54:00.444 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi132/crash 2023-10-06T12:54:00.445 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T12:54:00.481 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T12:54:00.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:54:00.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T12:54:00.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T12:54:00.661 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T12:54:00.695 INFO:tasks.ceph:Compressing logs... 2023-10-06T12:54:00.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T12:54:00.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T12:54:20.728 INFO:tasks.ceph:Archiving logs... 2023-10-06T12:54:20.729 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi046/log 2023-10-06T12:54:20.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T12:54:23.548 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi132/log 2023-10-06T12:54:23.549 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T12:54:25.210 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T12:54:25.219 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T12:54:25.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T12:54:25.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T12:54:25.317 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T12:54:25.326 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T12:54:25.327 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T12:54:25.334 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T12:54:25.371 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T12:54:25.372 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-10-06T12:54:25.372 INFO:teuthology.orchestra.run.smithi046.stdout:*hv01.front.sepi 44.4.53.2 2 u 52 128 377 0.095 0.872 0.243 2023-10-06T12:54:25.372 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 77 128 377 0.146 0.564 0.244 2023-10-06T12:54:25.372 INFO:teuthology.orchestra.run.smithi046.stdout:+hv03.front.sepi 74.6.168.72 3 u 6 128 377 0.153 -0.439 0.324 2023-10-06T12:54:25.372 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout: remote refid st t when poll reach delay offset jitter 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================== 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout:*hv01.front.sepi 44.4.53.2 2 u 85 256 377 0.215 -0.066 0.121 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout:+hv02.front.sepi 63.231.80.2 3 u 44 256 377 0.208 -0.238 0.105 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 256 377 0.207 -1.329 0.097 2023-10-06T12:54:25.377 INFO:teuthology.orchestra.run.smithi132.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2023-10-06T12:54:25.379 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T12:54:25.388 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T12:54:25.388 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T12:54:25.397 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T12:54:25.405 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T12:54:25.414 INFO:teuthology.task.internal:Duration was 6502.491860 seconds 2023-10-06T12:54:25.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T12:54:25.459 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T12:54:25.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T12:54:25.466 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T12:54:25.613 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T12:54:25.613 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2023-10-06T12:54:25.613 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T12:54:25.650 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2023-10-06T12:54:25.650 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T12:54:25.687 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T12:54:25.688 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T12:54:25.704 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T12:54:25.790 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T12:54:25.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T12:54:25.798 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T12:54:25.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T12:54:25.922 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T12:54:25.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T12:54:25.958 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T12:54:25.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T12:54:26.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T12:54:26.010 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T12:54:26.041 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2023-10-06T12:54:26.070 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2023-10-06T12:54:26.108 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T12:54:26.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T12:54:26.141 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T12:54:26.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T12:54:26.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T12:54:26.183 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T12:54:26.184 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi046 2023-10-06T12:54:26.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T12:54:26.229 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390/remote/smithi132 2023-10-06T12:54:26.229 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T12:54:26.269 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T12:54:26.270 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T12:54:26.280 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T12:54:26.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T12:54:26.339 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T12:54:26.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T12:54:26.350 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T12:54:26.350 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T12:54:26.358 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T12:54:26.374 INFO:teuthology.orchestra.run.smithi046.stdout: 1048609 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 12:54 /home/ubuntu/cephtest 2023-10-06T12:54:26.398 INFO:teuthology.orchestra.run.smithi132.stdout: 1053531 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 12:54 /home/ubuntu/cephtest 2023-10-06T12:54:26.404 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T12:54:26.415 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T12:54:26.438 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T12:54:26.466 DEBUG:teuthology.nuke:shortname: smithi046 2023-10-06T12:54:26.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T12:54:26.490 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:55:45.328367', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T12:54:26.494 DEBUG:teuthology.nuke:shortname: smithi132 2023-10-06T12:54:26.494 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T12:54:26.512 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:55:45.329843', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-06T12:54:26.547 INFO:teuthology.orchestra.console.smithi046:Power off 2023-10-06T12:54:26.547 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T12:54:26.563 INFO:teuthology.orchestra.console.smithi132:Power off 2023-10-06T12:54:26.564 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T12:54:26.574 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-10-06T12:54:26.574 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:26.586 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-10-06T12:54:26.587 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:26.598 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-06T12:54:26.609 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-06T12:54:30.600 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:30.610 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:30.710 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-10-06T12:54:30.719 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-06T12:54:34.713 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:34.720 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:34.822 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-10-06T12:54:34.829 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-06T12:54:34.924 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-10-06T12:54:38.831 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T12:54:38.941 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-06T12:54:39.042 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-10-06T12:54:39.147 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6502.491859912872 failure_reason: 'Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=86a0ad59ced74a0287b2f3042caed979 status: fail success: false 2023-10-06T12:54:39.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T12:54:39.231 INFO:teuthology.run:FAIL